[0/4] mm/swap_cgroup: remove global swap cgroup lock

Message

Kairui Song Dec. 2, 2024, 6:41 p.m. UTC

From: Kairui Song <[e-mail]>

This series removes the global swap cgroup lock. The critical section of
this lock is minimal but it's still a bottle neck for mass parallel
swap workloads.

Improvement of this series is more significant after the si lock rework:
https://lore.kernel.org/linux-mm/[e-mail]/
But this series works very well on its own.

Testing using 64G brd and build with build kernel with make -j96 in 1.5G
memory cgroup using 4k folios showed below improvement (10 test run):

Before this series:
Sys time: 10730.08 (stdev 49.030728)
Real time: 171.03 (stdev 0.850355)

After this series:
Sys time: 9612.24 (stdev 66.310789), -10.42%
Real time: 159.78 (stdev 0.577193), -6.57%

With 64k folios and 2G memcg:
Before this series:
Sys time: 7626.77 (stdev 43.545517)
Real time: 136.22 (stdev 1.265544)

After this series:
Sys time: 6936.03 (stdev 39.996280), -9.06%
Real time: 129.65 (stdev 0.880039), -4.82%

Sequential swapout of 8G 4k zero folios (24 test run):
Before this series:
5461409.12 us (stdev 183957.827084)

After this commit:
5420447.26 us (stdev 196419.240317)

Kairui Song (4):
  mm, memcontrol: avoid duplicated memcg enable check
  mm/swap_cgroup: remove swap_cgroup_cmpxchg
  mm/swap_cgroup: simplify swap cgroup definitions
  mm, swap_cgroup: remove global swap cgroup lock

 include/linux/swap_cgroup.h |   2 -
 mm/memcontrol.c             |   2 +-
 mm/swap_cgroup.c            | 110 ++++++++++++++++--------------------
 3 files changed, 51 insertions(+), 63 deletions(-)