diff mbox series

[11/75] mm/gup: Remove hpage_pincount_sub()

Message ID	20220204195852.1751729-12-willy@infradead.org (mailing list archive)
State	New
Headers	show Return-Path: <owner-linux-mm@kvack.org> From: "Matthew Wilcox (Oracle)" <willy@infradead.org> To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>, linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de>, John Hubbard <jhubbard@nvidia.com>, Jason Gunthorpe <jgg@nvidia.com>, William Kucharski <william.kucharski@oracle.com> Subject: [PATCH 11/75] mm/gup: Remove hpage_pincount_sub() Date: Fri, 4 Feb 2022 19:57:48 +0000 Message-Id: <20220204195852.1751729-12-willy@infradead.org> In-Reply-To: <20220204195852.1751729-1-willy@infradead.org> References: <20220204195852.1751729-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	MM folio patches for 5.18 \| expand [00/75] MM folio patches for 5.18 [01/75] mm/gup: Increment the page refcount before the pincount [02/75] mm/gup: Remove for_each_compound_range() [03/75] mm/gup: Remove for_each_compound_head() [04/75] mm/gup: Change the calling convention for compound_range_next() [05/75] mm/gup: Optimise compound_range_next() [06/75] mm/gup: Change the calling convention for compound_next() [07/75] mm/gup: Fix some contiguous memmap assumptions [08/75] mm/gup: Remove an assumption of a contiguous memmap [09/75] mm/gup: Handle page split race more efficiently [10/75] mm/gup: Remove hpage_pincount_add() [11/75] mm/gup: Remove hpage_pincount_sub() [12/75] mm: Make compound_pincount always available [13/75] mm: Add folio_pincount_ptr() [14/75] mm: Turn page_maybe_dma_pinned() into folio_maybe_dma_pinned() [15/75] mm/gup: Add try_get_folio() and try_grab_folio() [16/75] mm/gup: Convert try_grab_page() to use a folio [17/75] mm: Remove page_cache_add_speculative() and page_cache_get_speculative() [18/75] mm/gup: Add gup_put_folio() [19/75] mm/hugetlb: Use try_grab_folio() instead of try_grab_compound_head() [20/75] mm/gup: Convert gup_pte_range() to use a folio [21/75] mm/gup: Convert gup_hugepte() to use a folio [22/75] mm/gup: Convert gup_huge_pmd() to use a folio [23/75] mm/gup: Convert gup_huge_pud() to use a folio [24/75] mm/gup: Convert gup_huge_pgd() to use a folio [25/75] mm/gup: Turn compound_next() into gup_folio_next() [26/75] mm/gup: Turn compound_range_next() into gup_folio_range_next() [27/75] mm: Turn isolate_lru_page() into folio_isolate_lru() [28/75] mm/gup: Convert check_and_migrate_movable_pages() to use a folio [29/75] mm/workingset: Convert workingset_eviction() to take a folio [30/75] mm/memcg: Convert mem_cgroup_swapout() to take a folio [31/75] mm: Add lru_to_folio() [32/75] mm: Turn putback_lru_page() into folio_putback_lru() [33/75] mm/vmscan: Convert __remove_mapping() to take a folio [34/75] mm/vmscan: Turn page_check_dirty_writeback() into folio_check_dirty_writeback() [35/75] mm: Turn head_compound_mapcount() into folio_entire_mapcount() [36/75] mm: Add folio_mapcount() [37/75] mm: Add split_folio_to_list() [38/75] mm: Add folio_is_zone_device() and folio_is_device_private() [39/75] mm: Add folio_pgoff() [40/75] mm: Add pvmw_set_page() and pvmw_set_folio() [41/75] hexagon: Add pmd_pfn() [42/75] mm: Convert page_vma_mapped_walk to work on PFNs [43/75] mm/page_idle: Convert page_idle_clear_pte_refs() to use a folio [44/75] mm/rmap: Use a folio in page_mkclean_one() [45/75] mm/rmap: Turn page_referenced() into folio_referenced() [46/75] mm/mlock: Turn clear_page_mlock() into folio_end_mlock() [47/75] mm/mlock: Turn mlock_vma_page() into mlock_vma_folio() [48/75] mm/rmap: Turn page_mlock() into folio_mlock() [49/75] mm/mlock: Turn munlock_vma_page() into munlock_vma_folio() [50/75] mm/huge_memory: Convert __split_huge_pmd() to take a folio [51/75] mm/rmap: Convert try_to_unmap() to take a folio [52/75] mm/rmap: Convert try_to_migrate() to folios [53/75] mm/rmap: Convert make_device_exclusive_range() to use folios [54/75] mm/migrate: Convert remove_migration_ptes() to folios [55/75] mm/damon: Convert damon_pa_mkold() to use a folio [56/75] mm/damon: Convert damon_pa_young() to use a folio [57/75] mm/rmap: Turn page_lock_anon_vma_read() into folio_lock_anon_vma_read() [58/75] mm: Turn page_anon_vma() into folio_anon_vma() [59/75] mm/rmap: Convert rmap_walk() to take a folio [60/75] mm/rmap: Constify the rmap_walk_control argument [61/75] mm/vmscan: Free non-shmem folios without splitting them [62/75] mm/vmscan: Optimise shrink_page_list for non-PMD-sized folios [63/75] mm/vmscan: Account large folios correctly [64/75] mm/vmscan: Turn page_check_references() into folio_check_references() [65/75] mm/vmscan: Convert pageout() to take a folio [66/75] mm: Turn can_split_huge_page() into can_split_folio() [67/75] mm/filemap: Allow large folios to be added to the page cache [68/75] mm: Fix READ_ONLY_THP warning [69/75] mm: Make large folios depend on THP [70/75] mm: Support arbitrary THP sizes [71/75] mm/readahead: Add large folio readahead [72/75] mm/readahead: Align file mappings for non-DAX [73/75] mm/readahead: Switch to page_cache_ra_order [74/75] mm/filemap: Support VM_HUGEPAGE for file mappings [75/75] selftests/vm/transhuge-stress: Support file-backed PMD folios

Commit Message

Matthew Wilcox Feb. 4, 2022, 7:57 p.m. UTC

Move the assertion (and correct it to be a cheaper variant),
and inline the atomic_sub() operation.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: William Kucharski <william.kucharski@oracle.com>
---
 mm/gup.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)

diff mbox series

Patch

diff --git a/mm/gup.c b/mm/gup.c
index 60168a09d52a..af623a139995 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -29,14 +29,6 @@  struct follow_page_context {
 	unsigned int page_mask;
 };
 
-static void hpage_pincount_sub(struct page *page, int refs)
-{
-	VM_BUG_ON_PAGE(!hpage_pincount_available(page), page);
-	VM_BUG_ON_PAGE(page != compound_head(page), page);
-
-	atomic_sub(refs, compound_pincount_ptr(page));
-}
-
 /* Equivalent to calling put_page() @refs times. */
 static void put_page_refs(struct page *page, int refs)
 {
@@ -169,12 +161,13 @@  __maybe_unused struct page *try_grab_compound_head(struct page *page,
 
 static void put_compound_head(struct page *page, int refs, unsigned int flags)
 {
+	VM_BUG_ON_PAGE(PageTail(page), page);
+
 	if (flags & FOLL_PIN) {
 		mod_node_page_state(page_pgdat(page), NR_FOLL_PIN_RELEASED,
 				    refs);
-
 		if (hpage_pincount_available(page))
-			hpage_pincount_sub(page, refs);
+			atomic_sub(refs, compound_pincount_ptr(page));
 		else
 			refs *= GUP_PIN_COUNTING_BIAS;
 	}