| Message ID | 20231018080332.2663192-1-wangkefeng.wang@huawei.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm: mm: try VMA lock-based page fault handling first | expand |
On Wed, Oct 18, 2023 at 04:03:32PM +0800, Kefeng Wang wrote: > Attempt VMA lock-based page fault handling first, and fall back to the > existing mmap_lock-based handling if that fails, the ebizzy benchmark > shows 25% improvement(records/s) on qemu with 2 cpus. Looks fine to me, but I wonder how much of this code gets repeated in the other architectures. What is the plan for merging this? If you would like me to apply it, please submit it to the patch system, details in my signature below. Thanks.
On 2023/10/19 18:55, Russell King (Oracle) wrote: > On Wed, Oct 18, 2023 at 04:03:32PM +0800, Kefeng Wang wrote: >> Attempt VMA lock-based page fault handling first, and fall back to the >> existing mmap_lock-based handling if that fails, the ebizzy benchmark >> shows 25% improvement(records/s) on qemu with 2 cpus. > > Looks fine to me, but I wonder how much of this code gets repeated in > the other architectures. Thanks, I post a rfc to reduce the code reduplication between archs, but not too much reply about it[1], so I give up. > > What is the plan for merging this? If you would like me to apply it, > please submit it to the patch system, details in my signature below. > sure, I will submit via arm patch system. [1]https://lore.kernel.org/linux-mm/20230821123056.2109942-1-wangkefeng.wang@huawei.com/
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 9557808e8937..ec82078ddac0 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -34,6 +34,7 @@ config ARM select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT if CPU_V7 select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE + select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_MEMTEST diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index fef62e4a9edd..e96fb40b9cc3 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -278,6 +278,35 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); + if (!(flags & FAULT_FLAG_USER)) + goto lock_mmap; + + vma = lock_vma_under_rcu(mm, addr); + if (!vma) + goto lock_mmap; + + if (!(vma->vm_flags & vm_flags)) { + vma_end_read(vma); + goto lock_mmap; + } + fault = handle_mm_fault(vma, addr, flags | FAULT_FLAG_VMA_LOCK, regs); + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); + + if (!(fault & VM_FAULT_RETRY)) { + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + goto done; + } + count_vm_vma_lock_event(VMA_LOCK_RETRY); + + /* Quick path to respond to signals */ + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + goto no_context; + return 0; + } +lock_mmap: + retry: vma = lock_mm_and_find_vma(mm, addr, regs); if (unlikely(!vma)) { @@ -316,6 +345,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) } mmap_read_unlock(mm); +done: /* * Handle the "normal" case first - VM_FAULT_MAJOR
Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails, the ebizzy benchmark shows 25% improvement(records/s) on qemu with 2 cpus. Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> --- arch/arm/Kconfig | 1 + arch/arm/mm/fault.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+)