[0/2] Reliable testing for hugetlb

Message

Matthew Wilcox March 14, 2024, 1:25 a.m. UTC

We need to handle pages allocated by hugetlbfs differently in a number of
places.  If we have a reference to the folio, there are many methods which
will work.  If we don't, most of our previous attempts can misidentify a
page which never belonged to hugetlb as belonging to hugetlb.  It's quite
hard to hit these kinds of races, and generally the results are relatively
benign, but I don't feel good about them existing.  Obviously since we
don't have a reference to the page, the answer can be stale ("it was part
of hugetlb when we asked, but now it isn't"), but that's very different
from "This page never belonged to hugetlb, but we said it did".

We've tried various things in the past.  My most recent approach was
to use a bit in the second page's flags, reusing the "active" flag.
This is very susceptible to this problem.  Before that, we used to
check that folio_dtor (previously compound_dtor) had a specific value.
This field was also stored in the second page, shared with the LRU.
It was less susceptible to this problem, which may be why nobody noticed
it since its introduction in 2009.

We have considered various approaches to make folio_test_hugetlb() more
reliable.  We could use a word in the second page, but that occupies
space that we probably have a better use for.  We could use a word in
the third page, but then we have to test that there is a third page so
we don't run off the end of memmap.  We considered using a special value
in nr_pages_mapped, but weren't convinced we could ensure that it would
be unique if the second page was reallocated to a different purpose.

Using PageType is completely reliable.  It will never return true for a
page if that page was never in a hugetlb allocation.  It doesn't occupy
any extra space, and we have plenty of bits left in PageType.  It's also
the most efficient definition of PageHuge() we've ever had, allowing it
to be a static inline function rather than an exported function.

Matthew Wilcox (Oracle) (2):
  mm: Turn folio_test_hugetlb into a PageType
  mm: Remove a call to compound_head() from is_page_hwpoison()

 include/linux/mm.h         |  3 ++
 include/linux/page-flags.h | 73 +++++++++++++++++++-------------------
 mm/hugetlb.c               | 22 ++----------
 3 files changed, 42 insertions(+), 56 deletions(-)