[v4,0/6] Per-VMA lock support for swap and userfaults

Message

Suren Baghdasaryan June 28, 2023, 7:17 a.m. UTC

When per-VMA locks were introduced in [1] several types of page faults
would still fall back to mmap_lock to keep the patchset simple. Among them
are swap and userfault pages. The main reason for skipping those cases was
the fact that mmap_lock could be dropped while handling these faults and
that required additional logic to be implemented.
Implement the mechanism to allow per-VMA locks to be dropped for these
cases.
First, change handle_mm_fault to drop per-VMA locks when returning
VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way
mmap_lock is handled. Then change folio_lock_or_retry to accept vm_fault
and return vm_fault_t which simplifies later patches. Finally allow swap
and uffd page faults to be handled under per-VMA locks by dropping per-VMA
and retrying, the same way it's done under mmap_lock.
Naturally, once VMA lock is dropped that VMA should be assumed unstable
and can't be used.

Changes since v3 posted at [2]
- Renamed folio_lock_or_retry back to folio_lock_fault, per Peter Xu
- Moved per-VMA lock release to where VM_FAULT_RETRY is returned,
per Peter Xu
- Dropped FAULT_FLAG_LOCK_DROPPED usage, per Peter Xu
- Introduced release_fault_lock() helper function, per Peter Xu
- Dropped the patch releasing per-VMA lock before migration_entry_wait,
per Peter Xu
- Introduced assert_fault_locked() helper function, per Peter Xu
- Added BUG_ON to prevent FAULT_FLAG_RETRY_NOWAIT usage with per-VMA locks

Note: patch 3/8 will cause a trivial merge conflict in arch/arm64/mm/fault.c
when applied over mm-unstable branch due to a patch from ARM64 tree [3]
which is missing in mm-unstable.

[1] https://lore.kernel.org/all/[e-mail]/
[2] https://lore.kernel.org/all/[e-mail]/
[3] https://lore.kernel.org/all/[e-mail]/

Suren Baghdasaryan (6):
  swap: remove remnants of polling from read_swap_cache_async
  mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED
  mm: drop per-VMA lock when returning VM_FAULT_RETRY or
    VM_FAULT_COMPLETED
  mm: change folio_lock_or_retry to use vm_fault directly
  mm: handle swap page faults under per-VMA lock
  mm: handle userfaults under VMA lock

 arch/arm64/mm/fault.c    |  3 ++-
 arch/powerpc/mm/fault.c  |  3 ++-
 arch/s390/mm/fault.c     |  3 ++-
 arch/x86/mm/fault.c      |  3 ++-
 fs/userfaultfd.c         | 39 ++++++++++++++++++---------------------
 include/linux/mm.h       | 39 +++++++++++++++++++++++++++++++++++++++
 include/linux/mm_types.h |  3 ++-
 include/linux/pagemap.h  |  9 ++++-----
 mm/filemap.c             | 37 +++++++++++++++++++------------------
 mm/madvise.c             |  4 ++--
 mm/memory.c              | 38 ++++++++++++++++----------------------
 mm/swap.h                |  1 -
 mm/swap_state.c          | 12 +++++-------
 13 files changed, 113 insertions(+), 81 deletions(-)