[0/9] convert hugetlb fault functions to folios

Message

Sidhartha Kumar Jan. 19, 2023, 9:14 p.m. UTC

============== OVERVIEW ===========================
This series converts the hugetlb page faulting functions to operate on
folios. These include hugetlb_no_page(), hugetlb_wp(),
copy_hugetlb_page_range(), and hugetlb_mcopy_atomic_pte().

patches 1-3:
	- convert prerequisite helper functions to folios.
patch 4:
	- add a folio variable to the hugetlb fault functions and start
	  a partial conversion to folios.
patch 5:
	- fully convert hugetlb fault functions to folios.
patches 6-8: 
	- convert three functions to take in a folio rather than a page as
	  all callers now use folios.
patch 9:
  - update documentation that references alloc_huge_page

============== TEST COVERAGE ============================

Linux Test Project Hugetlb Test Fault Coverage

[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_no_page'
FUNC                              COUNT
hugetlb_no_page                    7796

[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_wp'
FUNC                              COUNT
hugetlb_wp                         4623

Using fallocate commands to create files on hugeltbfs

[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlbfs*'
FUNC                              COUNT
hugetlbfs_fallocate                   1


Userfaultfd selftest
./userfaultfd hugetlb 256 50 /dev/hugepages/hugefile

[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'hugetlb_mcopy_atomic_pte' 
FUNC                              COUNT
hugetlb_mcopy_atomic_pte           6240


[opc@sidhakum-devel kernel (master)]$ sudo ./funccount 'copy_hugetlb_page_range' 
FUNC                              COUNT
copy_hugetlb_page_range               3

============== PERFORMANCE ============================
using bpftrace to track time spent in fault functions over 10 rounds of
the LTP hugetlb tests

pre-patch:

@hugetlb_wp_nsecs:
[256, 512)          3675 |@@@@@@@@@@                                          |
[512, 1K)          18875 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[1K, 2K)            1366 |@@@                                                 |
[2K, 4K)              77 |                                                    |
[4K, 8K)              12 |                                                    |
[8K, 16K)             10 |                                                    |
[16K, 32K)             2 |                                                    |
[32K, 64K)             0 |                                                    |
[64K, 128K)            0 |                                                    |
[128K, 256K)           0 |                                                    |
[256K, 512K)           0 |                                                    |
[512K, 1M)             0 |                                                    |
[1M, 2M)               0 |                                                    |
[2M, 4M)               0 |                                                    |
[4M, 8M)               0 |                                                    |
[8M, 16M)              0 |                                                    |
[16M, 32M)             0 |                                                    |
[32M, 64M)             0 |                                                    |
[64M, 128M)            0 |                                                    |
[128M, 256M)           0 |                                                    |
[256M, 512M)           0 |                                                    |
[512M, 1G)             0 |                                                    |
[1G, 2G)               0 |                                                    |
[2G, 4G)               1 |                                                    |

@hugetlb_no_page_nsecs:                                                                                                                                                                                                                    
[64, 128)              1 |                                                    |                                                                                                                                                            
[128, 256)             0 |                                                    |                                                                                                                                                            
[256, 512)            67 |                                                    |                                                                                                                                                            
[512, 1K)             66 |                                                    |                                                                                                                                                            
[1K, 2K)              65 |                                                    |                                                                                                                                                            
[2K, 4K)             198 |                                                    |                                                                                                                                                            
[4K, 8K)              97 |                                                    |                                                                                                                                                            
[8K, 16K)              3 |                                                    |                                                                                                                                                            
[16K, 32K)             4 |                                                    |                                                                                                                                                            
[32K, 64K)           678 |                                                    |                                                                                                                                                            
[64K, 128K)         3401 |@                                                   |                                                                                                                                                            
[128K, 256K)       96746 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|                                                                                                                                                            
[256K, 512K)         107 |                                                    |                                                                                                                                                            
[512K, 1M)             0 |                                                    |                                                                                                                                                            
[1M, 2M)               1 |                                                    |                                                                                                                                                            
[2M, 4M)               0 |                                                    |                                                                                                                                                            
[4M, 8M)               1 |                                                    |                                                                                                                                                            
[8M, 16M)              0 |                                                    |                                                                                                                                                            
[16M, 32M)             0 |                                                    |                                                                                                                                                            
[32M, 64M)             0 |                                                    |                                                                                                                                                            
[64M, 128M)            0 |                                                    |                                                                                                                                                            
[128M, 256M)           0 |                                                    |                                                                                                                                                            
[256M, 512M)           0 |                                                    |                                                                                                                                                            
[512M, 1G)             0 |                                                    |                                                                                                                                                            
[1G, 2G)               0 |                                                    |                                                                                                                                                            
[2G, 4G)             283 |                                                    |


post patch:

@hugetlb_wp_nsecs: 
[256, 512)          7640 |@@@@@@@@@@@@@@@@@                                   |
[512, 1K)          23314 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[1K, 2K)            1171 |@@                                                  |
[2K, 4K)              60 |                                                    |
[4K, 8K)              14 |                                                    |
[8K, 16K)             29 |                                                    |
[16K, 32K)             2 |                                                    |

@hugetlb_no_page_nsecs:                                                                                                                                                                                                             [11/27]
[256, 512)            64 |                                                    |                                                                                                                                                            
[512, 1K)             75 |                                                    |                                                                                                                                                            
[1K, 2K)              70 |                                                    |                                                                                                                                                            
[2K, 4K)             209 |                                                    |                                                                                                                                                            
[4K, 8K)              73 |                                                    |                                                                                                                                                            
[8K, 16K)              5 |                                                    |                                                                                                                                                            
[16K, 32K)             1 |                                                    |                                                                                                                                                            
[32K, 64K)          2562 |@                                                   |                                                                                                                                                            
[64K, 128K)         7084 |@@@                                                 |                                                                                                                                                            
[128K, 256K)       99292 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|                                                                                                                                                            
[256K, 512K)          66 |                                                    |                                                                                                                                                            
[512K, 1M)             0 |                                                    |
[1M, 2M)               1 |                                                    |
[2M, 4M)               0 |                                                    |
[4M, 8M)               0 |                                                    |
[8M, 16M)              0 |                                                    |
[16M, 32M)             0 |                                                    |
[32M, 64M)             0 |                                                    |
[64M, 128M)            0 |                                                    |
[128M, 256M)           0 |                                                    |
[256M, 512M)           0 |                                                    |
[512M, 1G)             0 |                                                    |
[1G, 2G)               0 |                                                    |
[2G, 4G)             428 |                                                    |



rebased on 01/18/2023

Sidhartha Kumar (9):
  mm/hugetlb: convert hugetlb_install_page to folios
  mm/hugetlb: convert hugetlbfs_pagecache_present() to folios
  mm/hugetlb: convert putback_active_hugepage to take in a folio
  mm/rmap: change hugepage_add_new_anon_rmap to take in a folio
  mm/hugetlb: convert alloc_huge_page to alloc_hugetlb_folio
  mm/hugetlb: convert restore_reserve_on_error to take in a folio
  mm/hugetlb: convert hugetlb_add_to_page_cache to take in a folio
  mm/hugetlb: convert hugetlb_wp() to take in a folio
  Documentation/mm: update hugetlbfs documentation to mention
    alloc_hugetlb_folio

 Documentation/mm/hugetlbfs_reserv.rst         |  21 +-
 .../zh_CN/mm/hugetlbfs_reserv.rst             |  14 +-
 fs/hugetlbfs/inode.c                          |  38 +--
 include/linux/hugetlb.h                       |  16 +-
 include/linux/rmap.h                          |   2 +-
 mm/hugetlb.c                                  | 249 +++++++++---------
 mm/mempolicy.c                                |   6 +-
 mm/migrate.c                                  |   8 +-
 mm/rmap.c                                     |   6 +-
 9 files changed, 179 insertions(+), 181 deletions(-)