From patchwork Thu Jan 11 15:24:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13517536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9555FC47258 for ; Thu, 11 Jan 2024 15:24:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D2AB6B008A; Thu, 11 Jan 2024 10:24:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8824B6B0092; Thu, 11 Jan 2024 10:24:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 688376B0095; Thu, 11 Jan 2024 10:24:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4E6AB6B0092 for ; Thu, 11 Jan 2024 10:24:42 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 25CF3A2188 for ; Thu, 11 Jan 2024 15:24:42 +0000 (UTC) X-FDA: 81667402404.07.E12F751 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id EB63E180007 for ; Thu, 11 Jan 2024 15:24:39 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JGH+1Vk7; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704986680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ICF9LS0a/Yck6c1d8aWnlAw0yJ+x3RGOq1LGLEZtkVo=; b=n7DQJ6he2rEbdC79+KBjuupat0oify/HgMqGlc9vV/s+tXM+ont/Cxgr0J8Emd+eEc7cSL nHU1q/nW0K2oW3dAdMPtkrlG/pD+iC7JBMjYqJOKGpHBJTnSK4dG83xDcDJ7VZXzjuDHfI UTHAu7yITy6o96vIiu+yP/0gRKVfmNo= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=JGH+1Vk7; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704986680; a=rsa-sha256; cv=none; b=5tEI53I0UuaE7GFHUPM2pLqM9Xi9CVKGi1uGVu7xwSkd1OyeGn2MXQA+l5L3RAdDM/lD5x d5SsoqVuKi5Nwl61AP6zFRCDMZQFhS0cebbX3TtDk9CFC7R6MGZxhAHk8IbuuEUnEYJhOo sNSVdZoE+cycvzgqLCNREssBT1aSDgw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ICF9LS0a/Yck6c1d8aWnlAw0yJ+x3RGOq1LGLEZtkVo=; b=JGH+1Vk7AGOiSVpl5+BV2HzGX0 9c1y8jIRfULhE2wJAq3YB/z0IPwlB7Xj5P9N76TNsquMxDfWMk3SFPMUduDwEOcyVlPhbG1Qcg5d5 u/nI66o2/QajMLkwCPQVj7YSJMuxzJsM5xzdK5vFEG1ClahD2F2E2pnfgGFAZZFl4Q1bOKvcDSn3R wZWxNhkkkOZCWzS/de/AQ1C9UEjkAvgflelkgFXQcdVKbXq8RlFsmOJZ/alqdHIOVslNA2hNXx7az Rd4ow0Ot8KB6YV2318qqPoXHi1CvtujhQxs4PjvSIfxdK0p3ZJ18D1nS5UOIi17Q3mXR2s5VquvTs bNeC0CVA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1rNwv2-00E9sw-21; Thu, 11 Jan 2024 15:24:32 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: Kefeng Wang , linux-mm@kvack.org, david@redhat.com, linux-s390@vger.kernel.org, Matthew Wilcox Subject: [PATCH v3 09/10] mm: convert mm_counter() to take a folio Date: Thu, 11 Jan 2024 15:24:28 +0000 Message-Id: <20240111152429.3374566-10-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20240111152429.3374566-1-willy@infradead.org> References: <20240111152429.3374566-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: wji8apfythkndqbkfc6h4dp7xqm6zcci X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EB63E180007 X-HE-Tag: 1704986679-662177 X-HE-Meta: U2FsdGVkX1961tjC55Oj8JCNJ8/DHTMHjGBatK9505bhMudAaUrTMEkVbl11Pu0VHhxpIk1YxQ01UtKDU6zyGR+nYbCliyQzMNma+Py0J8u3/x4x51D5+Y8e5Fn6CE6i/tHlMi6HuW3jiVlgL0Yw1x5HFClK16kE/vRfheQ3NrZeafJxNeHAcnvgd3ZN+soeXqxTvQ/oGRwOeq3B2sBTRXqKEEToFfxQvssAJkdgMUkJ7R3D5DQKPy4dFRBDpUIXsYvj8hmcI9ZsyEHyzm9X/MbF4WqdYp+aI9rJmNxWvBapke9C48bUUJhDf9EchyKGqon6odu8+JfiIe7YOTl2Mmo1Ubx5LvAWJzYWkeae5FSi4VifAVpynbSu7tl28+CG/cCGHQ6Ul6pTuJ4v9RDOEE7Qynhn6EEPMtTAeD4VTY3OOo4SpF1SJSS4Qv0Z3lJ7u82e+PYZJJoHFzOoUgtCs4BJ940wYWj7/u0/302qA9OxDxsJmYipl7ME/3pLtS5CBeZZJtOosrtIRh0y5bQiNdgBX55rW3u2N2YkIfAoI+tcT/NZ/JeFkUo8Rt5+wxp2kiJqV6BA98L0oxbQJAUjDZiSjfADyWvkliYg9Qe7avGABE4lYLd6q55Ecj4kcD0NlIKsYxrpkQYiKIknfFShNhaWDiONNxrZ7BDnIOCW/JngMBMfKs0SPDjdiwUiiGG3p9hfivfz0T/K43oO1cfedJA4LyIc39yCOAyIz/wQhvJ/90RJRzJ4KkZE+07KL4zhCqgMCI4U0jcdkqr3WYl3r9wsMdqxFch87e7CMCSW91LOsfqMuogLjVJ5nEPfVI0tTy4plZ3BgPiV+bSXVcGRW1bfFuO/n/vSc50Tyym7RGVodABWi9ix0F3IaWCt0YWkTKu3fdZpJISyUhqvngWG9B1+QhTEclt8/ZIHNTHvfLGPBF6dV7klhMVT44DUVLnJSBuBPz50Yw91Xbw79vc W9YYGLyC 0oLWkZVW4XC3rBa552XGzz4lqSyYMr+HYNxvrM3vJs/p2qaLCJ/uePzlNWw9oF450AlnAccG7rLS7xIahc7/n1mUpJ7+qm0+bEQlTFT/jZcd6lMdXyyFTNWJFArScY9JrU8R4fXszRGvIH7lBQ+hWybJM5MbutukO7D1DoIFsnLok5mBZKvBJcmFOD7X9o4YwYTPpHsVmBGQVT8iO6or4brV3+z7IPtKnww7yVB/ymUN0DN3WL6ZSRYjGmw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kefeng Wang Now all callers of mm_counter() have a folio, convert mm_counter() to take a folio. Saves a call to compound_head() hidden inside PageAnon(). Signed-off-by: Kefeng Wang Signed-off-by: Matthew Wilcox (Oracle) --- arch/s390/mm/pgtable.c | 2 +- include/linux/mm.h | 6 +++--- mm/memory.c | 10 +++++----- mm/rmap.c | 8 ++++---- mm/userfaultfd.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 7e5dd4b17664..b71432b15d66 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -723,7 +723,7 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) else if (is_migration_entry(entry)) { struct folio *folio = pfn_swap_entry_folio(entry); - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } free_swap_and_cache(entry); } diff --git a/include/linux/mm.h b/include/linux/mm.h index f5a97dec5169..22e597b36b38 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2603,11 +2603,11 @@ static inline int mm_counter_file(struct page *page) return MM_FILEPAGES; } -static inline int mm_counter(struct page *page) +static inline int mm_counter(struct folio *folio) { - if (PageAnon(page)) + if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(page); + return mm_counter_file(&folio->page); } static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/mm/memory.c b/mm/memory.c index b73322ab9fd6..53ef7ae96440 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -808,7 +808,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } else if (is_migration_entry(entry)) { folio = pfn_swap_entry_folio(entry); - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -840,7 +840,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * keep things as they are. */ folio_get(folio); - rss[mm_counter(page)]++; + rss[mm_counter(folio)]++; /* Cannot fail as these pages cannot get pinned. */ folio_try_dup_anon_rmap_pte(folio, page, src_vma); @@ -1476,7 +1476,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (!delay_rmap) { folio_remove_rmap_pte(folio, page, vma); if (unlikely(page_mapcount(page) < 0)) @@ -1504,7 +1504,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (is_device_private_entry(entry)) folio_remove_rmap_pte(folio, page, vma); folio_put(folio); @@ -1519,7 +1519,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, folio = pfn_swap_entry_folio(entry); if (!should_zap_folio(details, folio)) continue; - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only diff --git a/mm/rmap.c b/mm/rmap.c index f5d43edad529..4648cf1d8178 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1780,7 +1780,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -1795,7 +1795,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else if (folio_test_anon(folio)) { swp_entry_t entry = page_swap_entry(subpage); pte_t swp_pte; @@ -2181,7 +2181,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -2196,7 +2196,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else { swp_entry_t entry; pte_t swp_pte; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 216ab4c8621f..662ab304cca3 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -124,7 +124,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * Must happen after rmap, as mm_counter() checks mapping (via * PageAnon()), which is set by __page_set_anon_rmap(). */ - inc_mm_counter(dst_mm, mm_counter(page)); + inc_mm_counter(dst_mm, mm_counter(folio)); set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte);