From patchwork Fri Mar 27 22:50:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Lespinasse X-Patchwork-Id: 11463243 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D6B85913 for ; Fri, 27 Mar 2020 22:51:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B9342071B for ; Fri, 27 Mar 2020 22:51:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sYueZpHe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B9342071B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3EF66B000E; Fri, 27 Mar 2020 18:51:06 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9EE6E6B0032; Fri, 27 Mar 2020 18:51:06 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DDAE6B0036; Fri, 27 Mar 2020 18:51:06 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0109.hostedemail.com [216.40.44.109]) by kanga.kvack.org (Postfix) with ESMTP id 75AA26B000E for ; Fri, 27 Mar 2020 18:51:06 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3583E8248068 for ; Fri, 27 Mar 2020 22:51:06 +0000 (UTC) X-FDA: 76642639332.23.ray50_43ca996eeab0f X-Spam-Summary: 2,0,0,12f9cf4619ae2a8d,d41d8cd98f00b204,3win-xgykcfmhv65z819916z.x97638fi-775gvx5.9c1@flex--walken.bounces.google.com,,RULES_HIT:4:41:152:355:379:541:967:968:973:988:989:1260:1277:1313:1314:1345:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1801:2198:2199:2393:2525:2553:2559:2563:2638:2682:2685:2859:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6261:6653:6742:7875:7903:8660:8784:9025:9969:10004:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13149:13161:13229:13230:13868:14096:14097:14394:14659:21080:21212:21220:21324:21433:21444:21451:21627:21740:21773:21987:30010:30012:30054:30070:30090,0,RBL:209.85.215.202:@flex--walken.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neu tral,Cus X-HE-Tag: ray50_43ca996eeab0f X-Filterd-Recvd-Size: 15132 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Fri, 27 Mar 2020 22:51:05 +0000 (UTC) Received: by mail-pg1-f202.google.com with SMTP id f14so9131673pgj.15 for ; Fri, 27 Mar 2020 15:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=M7oO3MHpWPayR/syKuA72wISmvZSlzzeQl/7LNsmjJQ=; b=sYueZpHeMT+3s26mkxBo+sl425vKua1fgkUjzsWksm3L5PQtgM60jT3YqcTFiqYEPV 6aSGhYFHtr8CCS/5mR4QwdWtFsno5l8cRaT1WMRewgn1uY+INmcC1u/UjRRAYGEvGLai EP9rHN3TUKYE3ZNMHVtP5Oc1GYC808AOpkBbaV9sMsWDIZOj90++vIxAnCwkGg0BPKbd tPTACr8xTDvMWM1W68DJwzMJX5LMQc4vNZjrVFHcPISZldK+HYshh5nvQDxTwtC/n1Qo UhEsU6tQGjqLX/tevRx7f6CuL8HxQVHmPkacO01kNZmc/BIHaLdNDGk4m1jx+0jSEOXc Czkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=M7oO3MHpWPayR/syKuA72wISmvZSlzzeQl/7LNsmjJQ=; b=ZQJgWMNfU3d/0vSldei1uDKU7Gb+FUXGbBHEMUfA3b4pyfwg+i90kOGpoKyYZaXcr4 5yRAVWnhnYZX0D54lVkC1tfkZgQHehe6FX7UoaiXbFnENq2s57wZk7NByWEvYd9gZKML uwmq2f+3GCDUqJuMhnE4cuJHkxInHrtwezlySBiwK9MmQtJ70HBi169/InRX39iCpckR fRh0jt3ndGjfrqEQFfSD//GqxIm14X39b1DL0s5G+r/hEVBZUESZnd3iBB5bY8Glyx9L n7znJvEAeD7PQ5d3VysGY0+TfRkhSfSArdkvX4y8jMyT460lxYiTN5RTk0t6tQlGy2x3 ATmA== X-Gm-Message-State: ANhLgQ0GHHxgMOHUhMS3OkazUKTtE8DZb13fM1uYc7YLqPnjP3vIygwh SbwnfxLTpNeId53YuVpBW97kHOTReY8= X-Google-Smtp-Source: ADFU+vvRbwhgkghdGj4n50QD1tY24sJnwU6JPpLSAsfutOCQJdrc1mAq8679Lx75P8/gBjkrHuh0UWUzf58= X-Received: by 2002:a17:90b:3610:: with SMTP id ml16mr1852852pjb.106.1585349464337; Fri, 27 Mar 2020 15:51:04 -0700 (PDT) Date: Fri, 27 Mar 2020 15:50:52 -0700 Message-Id: <20200327225102.25061-1-walken@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.26.0.rc2.310.g2932bb562d-goog Subject: [PATCH v3 00/10] Add a new mmap locking API wrapping mmap_sem calls From: Michel Lespinasse To: Andrew Morton , linux-mm Cc: LKML , Peter Zijlstra , Laurent Dufour , Vlastimil Babka , Matthew Wilcox , Liam Howlett , Jerome Glisse , Davidlohr Bueso , David Rientjes , Hugh Dickins , Ying Han , Jason Gunthorpe , Markus Elfring , Michel Lespinasse X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch series adds a new mmap locking API replacing the existing mmap_sem lock and unlocks. Initially the API is just implemente in terms of inlined rwsem calls, so it doesn't provide any new functionality. There are two justifications for the new API: - At first, it provides an easy hooking point to instrument mmap_sem locking latencies independently of any other rwsems. - In the future, it may be a starting point for replacing the rwsem implementation with a different one, such as range locks. This is something that is being explored, even though there is no wide concensus about this possible direction yet. (see https://patchwork.kernel.org/cover/11401483/) Changes since v2 of the patchset: - Removed the mmap_is_locked API - v2 had removed all uses of it, but the actual function definition was still there unused. Thanks to Jason Gunthorpe for noticing the unused mmap_is_locked function. Changes since v1 of the patchset: - Manually convert drivers/dma-buf/dma-resv.c ahead of the automated coccinelle conversion as this file requires a new include statement. Thanks to Intel's kbuild test bot for finding the issue. - In coccinelle automated conversion, apply a single coccinelle rule as suggested by Markus Elfring. - In manual conversion of sites missed by coccinelle, fix an issue where I had used mm_read_unlock (from an older version of my patchset) instead of mmap_read_unlock in some arch/mips code. This was also identified by Intel's kbuild test bot. - Do not add a new mmap_is_locked API, and use lockdep_assert_held instead. Thanks to Jason Gunthorpe and Matthew Wilcox for the suggestion. The changes apply on top of v5.6-rc7. I think it would be feasible to push these changes towards mainline if given sufficient approval. The coccinelle part of the change is relatively invasive, but can be skipped over on a file by file basis if it causes any conflicts with other pending changes. The new mmap locking API can interoperate with new code that is still using direct rwsem calls, until the last patch in the series which renames mmap_sem to enforce using the new API. Maybe that last patch could be delayed for a bit, so that we'd get a chance to convert any new code that locks mmap_sem in the -rc1 release before applying that last patch. Michel Lespinasse (10): mmap locking API: initial implementation as rwsem wrappers MMU notifier: use the new mmap locking API DMA reservations: use the new mmap locking API mmap locking API: use coccinelle to convert mmap_sem rwsem call sites mmap locking API: convert mmap_sem call sites missed by coccinelle mmap locking API: convert nested write lock sites mmap locking API: add mmap_read_release() and mmap_read_unlock_non_owner() mmap locking API: add MMAP_LOCK_INITIALIZER mmap locking API: use lockdep_assert_held mmap locking API: rename mmap_sem to mmap_lock arch/alpha/kernel/traps.c | 4 +- arch/alpha/mm/fault.c | 10 +-- arch/arc/kernel/process.c | 4 +- arch/arc/kernel/troubleshoot.c | 4 +- arch/arc/mm/fault.c | 4 +- arch/arm/kernel/process.c | 4 +- arch/arm/kernel/swp_emulate.c | 4 +- arch/arm/lib/uaccess_with_memcpy.c | 16 ++-- arch/arm/mm/fault.c | 6 +- arch/arm64/kernel/traps.c | 4 +- arch/arm64/kernel/vdso.c | 8 +- arch/arm64/mm/fault.c | 8 +- arch/csky/kernel/vdso.c | 4 +- arch/csky/mm/fault.c | 8 +- arch/hexagon/kernel/vdso.c | 4 +- arch/hexagon/mm/vm_fault.c | 8 +- arch/ia64/kernel/perfmon.c | 8 +- arch/ia64/mm/fault.c | 12 +-- arch/ia64/mm/init.c | 12 +-- arch/m68k/kernel/sys_m68k.c | 14 ++-- arch/m68k/mm/fault.c | 8 +- arch/microblaze/mm/fault.c | 12 +-- arch/mips/kernel/traps.c | 4 +- arch/mips/kernel/vdso.c | 4 +- arch/mips/mm/fault.c | 10 +-- arch/nds32/kernel/vdso.c | 6 +- arch/nds32/mm/fault.c | 12 +-- arch/nios2/mm/fault.c | 12 +-- arch/nios2/mm/init.c | 4 +- arch/openrisc/mm/fault.c | 10 +-- arch/parisc/kernel/traps.c | 6 +- arch/parisc/mm/fault.c | 8 +- arch/powerpc/kernel/vdso.c | 6 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +- arch/powerpc/kvm/book3s_hv.c | 6 +- arch/powerpc/kvm/book3s_hv_uvmem.c | 12 +-- arch/powerpc/kvm/e500_mmu_host.c | 4 +- arch/powerpc/mm/book3s64/iommu_api.c | 4 +- arch/powerpc/mm/book3s64/subpage_prot.c | 12 +-- arch/powerpc/mm/copro_fault.c | 4 +- arch/powerpc/mm/fault.c | 12 +-- arch/powerpc/oprofile/cell/spu_task_sync.c | 6 +- arch/powerpc/platforms/cell/spufs/file.c | 4 +- arch/riscv/kernel/vdso.c | 4 +- arch/riscv/mm/fault.c | 10 +-- arch/s390/kernel/vdso.c | 4 +- arch/s390/kvm/gaccess.c | 4 +- arch/s390/kvm/kvm-s390.c | 24 +++--- arch/s390/kvm/priv.c | 32 ++++---- arch/s390/mm/fault.c | 6 +- arch/s390/mm/gmap.c | 40 +++++----- arch/s390/pci/pci_mmio.c | 4 +- arch/sh/kernel/sys_sh.c | 6 +- arch/sh/kernel/vsyscall/vsyscall.c | 4 +- arch/sh/mm/fault.c | 14 ++-- arch/sparc/mm/fault_32.c | 18 ++--- arch/sparc/mm/fault_64.c | 12 +-- arch/sparc/vdso/vma.c | 4 +- arch/um/include/asm/mmu_context.h | 5 +- arch/um/kernel/tlb.c | 2 +- arch/um/kernel/trap.c | 6 +- arch/unicore32/mm/fault.c | 6 +- arch/x86/entry/vdso/vma.c | 14 ++-- arch/x86/events/core.c | 4 +- arch/x86/kernel/tboot.c | 2 +- arch/x86/kernel/vm86_32.c | 4 +- arch/x86/kvm/mmu/paging_tmpl.h | 8 +- arch/x86/mm/fault.c | 10 +-- arch/x86/um/vdso/vma.c | 4 +- arch/xtensa/mm/fault.c | 10 +-- drivers/android/binder_alloc.c | 10 +-- drivers/dma-buf/dma-resv.c | 5 +- drivers/firmware/efi/efi.c | 2 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 10 +-- drivers/gpu/drm/amd/amdkfd/kfd_events.c | 4 +- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_mman.c | 4 +- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 8 +- drivers/gpu/drm/nouveau/nouveau_svm.c | 20 ++--- drivers/gpu/drm/radeon/radeon_cs.c | 4 +- drivers/gpu/drm/radeon/radeon_gem.c | 6 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 4 +- drivers/infiniband/core/umem_odp.c | 4 +- drivers/infiniband/core/uverbs_main.c | 4 +- drivers/infiniband/hw/mlx4/mr.c | 4 +- drivers/infiniband/hw/qib/qib_user_pages.c | 6 +- drivers/infiniband/hw/usnic/usnic_uiom.c | 4 +- drivers/infiniband/sw/siw/siw_mem.c | 4 +- drivers/iommu/amd_iommu_v2.c | 4 +- drivers/iommu/intel-svm.c | 4 +- drivers/media/v4l2-core/videobuf-core.c | 4 +- drivers/media/v4l2-core/videobuf-dma-contig.c | 4 +- drivers/media/v4l2-core/videobuf-dma-sg.c | 4 +- drivers/misc/cxl/cxllib.c | 4 +- drivers/misc/cxl/fault.c | 4 +- drivers/misc/sgi-gru/grufault.c | 16 ++-- drivers/misc/sgi-gru/grufile.c | 4 +- drivers/oprofile/buffer_sync.c | 10 +-- drivers/staging/kpc2000/kpc_dma/fileops.c | 4 +- drivers/tee/optee/call.c | 4 +- drivers/vfio/vfio_iommu_type1.c | 8 +- drivers/xen/gntdev.c | 4 +- drivers/xen/privcmd.c | 14 ++-- fs/aio.c | 4 +- fs/coredump.c | 4 +- fs/exec.c | 16 ++-- fs/io_uring.c | 4 +- fs/proc/base.c | 18 ++--- fs/proc/task_mmu.c | 28 +++---- fs/proc/task_nommu.c | 18 ++--- fs/userfaultfd.c | 28 +++---- include/linux/mm.h | 1 + include/linux/mm_types.h | 2 +- include/linux/mmap_lock.h | 77 +++++++++++++++++++ include/linux/mmu_notifier.h | 5 +- ipc/shm.c | 8 +- kernel/acct.c | 4 +- kernel/bpf/stackmap.c | 13 ++-- kernel/events/core.c | 4 +- kernel/events/uprobes.c | 16 ++-- kernel/exit.c | 8 +- kernel/fork.c | 14 ++-- kernel/futex.c | 4 +- kernel/sched/fair.c | 4 +- kernel/sys.c | 18 ++--- kernel/trace/trace_output.c | 4 +- mm/filemap.c | 6 +- mm/frame_vector.c | 4 +- mm/gup.c | 20 ++--- mm/hmm.c | 2 +- mm/init-mm.c | 2 +- mm/internal.h | 2 +- mm/khugepaged.c | 36 ++++----- mm/ksm.c | 34 ++++---- mm/madvise.c | 18 ++--- mm/memcontrol.c | 8 +- mm/memory.c | 14 ++-- mm/mempolicy.c | 22 +++--- mm/migrate.c | 8 +- mm/mincore.c | 4 +- mm/mlock.c | 16 ++-- mm/mmap.c | 36 ++++----- mm/mmu_notifier.c | 22 +++--- mm/mprotect.c | 12 +-- mm/mremap.c | 6 +- mm/msync.c | 8 +- mm/nommu.c | 16 ++-- mm/oom_kill.c | 4 +- mm/pagewalk.c | 15 ++-- mm/process_vm_access.c | 4 +- mm/ptdump.c | 4 +- mm/swapfile.c | 4 +- mm/userfaultfd.c | 14 ++-- mm/util.c | 12 +-- net/ipv4/tcp.c | 4 +- net/xdp/xdp_umem.c | 4 +- virt/kvm/arm/mmu.c | 14 ++-- virt/kvm/async_pf.c | 4 +- virt/kvm/kvm_main.c | 8 +- 160 files changed, 774 insertions(+), 693 deletions(-) create mode 100644 include/linux/mmap_lock.h