| Message ID | 20220204195852.1751729-21-willy@infradead.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | MM folio patches for 5.18 | expand |
On Fri, 4 Feb 2022 at 20:21, Matthew Wilcox (Oracle) <willy@infradead.org> wrote: > > We still call try_grab_folio() once per PTE; a future patch could > optimise to just adjust the reference count for each page within > the folio. > > Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Reviewed-by: John Hubbard <jhubbard@nvidia.com> > Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> > Reviewed-by: William Kucharski <william.kucharski@oracle.com> > --- > mm/gup.c | 16 +++++++--------- > 1 file changed, 7 insertions(+), 9 deletions(-) > > diff --git a/mm/gup.c b/mm/gup.c > index 00227b2cb1cf..44281350db1a 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -2252,7 +2252,8 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > ptem = ptep = pte_offset_map(&pmd, addr); > do { > pte_t pte = ptep_get_lockless(ptep); > - struct page *head, *page; > + struct page *page; > + struct folio *folio; > > /* > * Similar to the PMD case below, NUMA hinting must take slow > @@ -2279,22 +2280,20 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > VM_BUG_ON(!pfn_valid(pte_pfn(pte))); > page = pte_page(pte); > > - head = try_grab_compound_head(page, 1, flags); > - if (!head) > + folio = try_grab_folio(page, 1, flags); > + if (!folio) > goto pte_unmap; > > if (unlikely(page_is_secretmem(page))) { > - put_compound_head(head, 1, flags); > + gup_put_folio(folio, 1, flags); > goto pte_unmap; > } > > if (unlikely(pte_val(pte) != pte_val(*ptep))) { > - put_compound_head(head, 1, flags); > + gup_put_folio(folio, 1, flags); > goto pte_unmap; > } > > - VM_BUG_ON_PAGE(compound_head(page) != head, page); > - > /* > * We need to make the page accessible if and only if we are > * going to access its content (the FOLL_PIN case). Please > @@ -2308,10 +2307,9 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > goto pte_unmap; > } V. Minor/nit: Above the "goto pte_unmap;" is the code block; if (flags & FOLL_PIN) { ret = arch_make_page_accessible(page); if (ret) { unpin_user_page(page); goto pte_unmap; } } Other conditions which goto pte_unmap, after successful try_grab_folio(), call gup_put_folio() (rather than unpin_user_page()). No change in functionality, but suggest calling gup_put_folio() here too for consistency. Cheers, Mark
On Sun, Feb 06, 2022 at 02:52:46PM +0000, Mark Hemment wrote: > On Fri, 4 Feb 2022 at 20:21, Matthew Wilcox (Oracle) > <willy@infradead.org> wrote: > V. Minor/nit: Above the "goto pte_unmap;" is the code block; > if (flags & FOLL_PIN) { > ret = arch_make_page_accessible(page); > if (ret) { > unpin_user_page(page); > goto pte_unmap; > } > } > Other conditions which goto pte_unmap, after successful > try_grab_folio(), call gup_put_folio() (rather than > unpin_user_page()). > No change in functionality, but suggest calling gup_put_folio() here > too for consistency. Thanks! Changed.
diff --git a/mm/gup.c b/mm/gup.c index 00227b2cb1cf..44281350db1a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2252,7 +2252,8 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, ptem = ptep = pte_offset_map(&pmd, addr); do { pte_t pte = ptep_get_lockless(ptep); - struct page *head, *page; + struct page *page; + struct folio *folio; /* * Similar to the PMD case below, NUMA hinting must take slow @@ -2279,22 +2280,20 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); - head = try_grab_compound_head(page, 1, flags); - if (!head) + folio = try_grab_folio(page, 1, flags); + if (!folio) goto pte_unmap; if (unlikely(page_is_secretmem(page))) { - put_compound_head(head, 1, flags); + gup_put_folio(folio, 1, flags); goto pte_unmap; } if (unlikely(pte_val(pte) != pte_val(*ptep))) { - put_compound_head(head, 1, flags); + gup_put_folio(folio, 1, flags); goto pte_unmap; } - VM_BUG_ON_PAGE(compound_head(page) != head, page); - /* * We need to make the page accessible if and only if we are * going to access its content (the FOLL_PIN case). Please @@ -2308,10 +2307,9 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, goto pte_unmap; } } - SetPageReferenced(page); + folio_set_referenced(folio); pages[*nr] = page; (*nr)++; - } while (ptep++, addr += PAGE_SIZE, addr != end); ret = 1;