[v3,0/3] Optimize large folio interaction with deferred split

Message

Ryan Roberts July 20, 2023, 11:29 a.m. UTC

Hi All,

[Sending v3 to replace yesterday's v2 after Yu Zhou's feedback]

This is v3 of a small series in support of my work to enable the use of large
folios for anonymous memory (known as "FLEXIBLE_THP" or "LARGE_ANON_FOLIO") [1].
It first makes it possible to add large, non-pmd-mappable folios to the deferred
split queue. Then it modifies zap_pte_range() to batch-remove spans of
physically contiguous pages from the rmap, which means that in the common case,
we elide the need to ever put the folio on the deferred split queue, thus
reducing lock contention and improving performance.

This becomes more visible once we have lots of large anonymous folios in the
system, and Huang Ying has suggested solving this needs to be a prerequisit for
merging the main FLEXIBLE_THP/LARGE_ANON_FOLIO work.

The series applies on top of v6.5-rc2 and a branch is available at [2].

I don't have a full test run with the latest versions of all the patches on top
of the latest baseline, so not posting results formally. I can get these if
people feel they are neccessary though. But anecdotally, for the kernel
compilation workload, this series reduces kernel time by ~4% and reduces
real-time by ~0.4%, compared with [1].


Changes since v2 [3]
--------------------

- patch 2:
    - Reworked at Yu Zhou's request to reduce duplicated code.
    - page_remove_rmap() now forwards to folio_remove_rmap_range() for the
      !compound (PMD mapped) case.
    - Both page_remove_rmap() and folio_remove_rmap_range() share common
      epilogue via new helper function __remove_rmap_finish().
    - As a result of the changes, I've removed the previous Reviewed-bys.

- other 2 patches are unchanged.


Changes since v1 [4]
--------------------

- patch 2: Modified doc comment for folio_remove_rmap_range()
- patch 2: Hoisted _nr_pages_mapped manipulation out of page loop so its now
  modified once per folio_remove_rmap_range() call.
- patch 2: Added check that page range is fully contained by folio in
  folio_remove_rmap_range()
- patch 2: Fixed some nits raised by Huang, Ying for folio_remove_rmap_range()
- patch 3: Support batch-zap of all anon pages, not just those in anon vmas
- patch 3: Renamed various functions to make their use clear
- patch 3: Various minor refactoring/cleanups
- Added Reviewed-By tags - thanks!

[1] https://lore.kernel.org/linux-mm/[e-mail]/
[2] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/granule_perf/deferredsplit-lkml_v3
[3] https://lore.kernel.org/linux-mm/[e-mail]/
[4] https://lore.kernel.org/linux-mm/[e-mail]/

Thanks,
Ryan


Ryan Roberts (3):
  mm: Allow deferred splitting of arbitrary large anon folios
  mm: Implement folio_remove_rmap_range()
  mm: Batch-zap large anonymous folio PTE mappings

 include/linux/rmap.h |   2 +
 mm/memory.c          | 120 +++++++++++++++++++++++++++++++++++++++++
 mm/rmap.c            | 125 ++++++++++++++++++++++++++++++++-----------
 3 files changed, 217 insertions(+), 30 deletions(-)

--
2.25.1