From patchwork Thu Mar 26 07:02:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Lespinasse X-Patchwork-Id: 11459385 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 463B7913 for ; Thu, 26 Mar 2020 07:02:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDAA320838 for ; Thu, 26 Mar 2020 07:02:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="b2lHZXWy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDAA320838 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 139CC6B000A; Thu, 26 Mar 2020 03:02:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0EAD46B000C; Thu, 26 Mar 2020 03:02:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF4E56B000D; Thu, 26 Mar 2020 03:02:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id D90866B000A for ; Thu, 26 Mar 2020 03:02:40 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CB5AB181AC9CC for ; Thu, 26 Mar 2020 07:02:40 +0000 (UTC) X-FDA: 76636620480.26.drain96_1756edbc9040d X-Spam-Summary: 2,0,0,4f76bc80a8b9b5bb,d41d8cd98f00b204,3jln8xgykccet7ihbkdlldib.9ljifkru-jjhs79h.lod@flex--walken.bounces.google.com,,RULES_HIT:1:41:152:355:379:541:967:973:988:989:1260:1277:1313:1314:1345:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1801:2198:2199:2393:2525:2553:2559:2563:2637:2682:2685:2859:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6261:6653:6742:7903:8660:9025:9969:10004:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13149:13161:13229:13230:13868:14096:14097:14394:14659:21080:21212:21220:21433:21444:21451:21627:21740:21773:21987:30010:30012:30054:30070:30090,0,RBL:209.85.210.201:@flex--walken.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0: 0:0,LFti X-HE-Tag: drain96_1756edbc9040d X-Filterd-Recvd-Size: 13933 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Mar 2020 07:02:40 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id i1so4399617pfo.19 for ; Thu, 26 Mar 2020 00:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=tNlW6L/2XFnR6aPCHierE31WIPDldccPhC+22hE3ZVE=; b=b2lHZXWyAL9IsGHA9JgXPhinVADFrfV9rDjWtTSCM92Aq8ct3DBGEgVq03EmqtkOzn bjOlYcorf20tBxWgg1X4SkKswoj6Yzbpv+0d5jtpaIRHrpN4mykiLkrQ0P5jr9Yji7+G rMHwXZJSz0LPka0yhbTprijjYSWHgjYT1rPIWA0pJiPFsRfkMT0eQIVaY3nvkZ9CXBLr cqyHCCqiTFlUWzJpD1IgAWfTJw56y/vEQUVHNfprfl+tKu3kl4jSrpHfBn0DSW+9p3wM bxROwQuw6Z6PpCO1cX8TXLgo2sRiW0YBKFmzrFbVjpOQaAk22OxsOeS0ie9oBw28j9nR cnMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=tNlW6L/2XFnR6aPCHierE31WIPDldccPhC+22hE3ZVE=; b=O29toBwL3L97GpMIOwuMfaAHa+bLnRw9AX5xO9V7uWm/KdllLqvXpyX8nkSQnsK4ry XurH4gu/pGNPouajyP+wfPHrKQtsyFDlHk18ZDoxzJRGsy7QOrc79Xuf6XJr4H+8KyLS 5pQaoAge8T9rAg9vFhyOEa8rJ3b7mu4f9c7KiBKgDV5WqWHDrA0/FgNT195Z9tPWb0se MnjHDzR9cMA+cfErB95Z3pknGkHCP8fu9I83csCr3955XcjDl7CQgSFmg2MPdazsklfg 8d3FWvLIseRKKEF75YHj8+bXBpyAlCDFDm7dlOJ8Gr2fYxqRQ5Xd4RLFsYHINkeUtSUg awCQ== X-Gm-Message-State: ANhLgQ1RctfKVGvi/NiOGJThMLRN+C9Iii5mfBtMIMuUe018BgLTTFVv 1CvbpFb29rqy5rPWBm+Oop8BMVdmR6c= X-Google-Smtp-Source: ADFU+vu6qBdrGMLpNcPs/heI9wuDJk04pDv/BVcj0tTQ3y1FTbkwJJaDGi4LQUeD1gN7ODG8b95akrU2p4M= X-Received: by 2002:a17:90b:490b:: with SMTP id kr11mr1555412pjb.21.1585206158617; Thu, 26 Mar 2020 00:02:38 -0700 (PDT) Date: Thu, 26 Mar 2020 00:02:28 -0700 Message-Id: <20200326070236.235835-1-walken@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.25.1.696.g5e7596f4ac-goog Subject: [PATCH 0/8] Add a new mmap locking API wrapping mmap_sem calls From: Michel Lespinasse To: Andrew Morton , linux-mm Cc: LKML , Peter Zijlstra , Laurent Dufour , Vlastimil Babka , Matthew Wilcox , Liam Howlett , Jerome Glisse , Davidlohr Bueso , David Rientjes , Hugh Dickins , Ying Han , Michel Lespinasse X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch series adds a new mmap locking API replacing the existing mmap_sem lock and unlocks. Initially the API is just implemente in terms of inlined rwsem calls, so it doesn't provide any new functionality. There are two justifications for the new API: - At first, it provides an easy hooking point to instrument mmap_sem locking latencies independently of any other rwsems. - In the future, it may be a starting point for replacing the rwsem implementation with a different one, such as range locks. This is something that is being explored, even though there is no wide concensus about this possible direction yet. (see https://patchwork.kernel.org/cover/11401483/) The changes apply on top of v5.6-rc7. I think it would be feasible to apply them in the next merge cycle if given sufficient approval. The coccinelle part of the change is relatively invasive, but can be skipped over on a file by file basis if it causes any conflicts with other pending changes. The new mmap locking API can interoperate with new code that is still using direct rwsem calls, until the last patch in the series which renames mmap_sem to enforce using the new API. Maybe that last patch could be delayed for a bit, so that we'd get a chance to convert any new code that locks mmap_sem in the -rc1 release before applying that last patch. Michel Lespinasse (8): mmap locking API: initial implementation as rwsem wrappers MMU notifier: use the new mmap locking API mmap locking API: use coccinelle to convert mmap_sem rwsem call sites mmap locking API: convert mmap_sem call sites missed by coccinelle mmap locking API: convert nested write lock sites mmap locking API: add mmap_read_release() and mmap_read_unlock_non_owner() mmap locking API: add MMAP_LOCK_INITIALIZER mmap locking API: rename mmap_sem to mmap_lock arch/alpha/kernel/traps.c | 4 +- arch/alpha/mm/fault.c | 10 +-- arch/arc/kernel/process.c | 4 +- arch/arc/kernel/troubleshoot.c | 4 +- arch/arc/mm/fault.c | 4 +- arch/arm/kernel/process.c | 4 +- arch/arm/kernel/swp_emulate.c | 4 +- arch/arm/lib/uaccess_with_memcpy.c | 16 ++-- arch/arm/mm/fault.c | 6 +- arch/arm64/kernel/traps.c | 4 +- arch/arm64/kernel/vdso.c | 8 +- arch/arm64/mm/fault.c | 8 +- arch/csky/kernel/vdso.c | 4 +- arch/csky/mm/fault.c | 8 +- arch/hexagon/kernel/vdso.c | 4 +- arch/hexagon/mm/vm_fault.c | 8 +- arch/ia64/kernel/perfmon.c | 8 +- arch/ia64/mm/fault.c | 12 +-- arch/ia64/mm/init.c | 12 +-- arch/m68k/kernel/sys_m68k.c | 14 ++-- arch/m68k/mm/fault.c | 8 +- arch/microblaze/mm/fault.c | 12 +-- arch/mips/kernel/traps.c | 4 +- arch/mips/kernel/vdso.c | 4 +- arch/mips/mm/fault.c | 10 +-- arch/nds32/kernel/vdso.c | 6 +- arch/nds32/mm/fault.c | 12 +-- arch/nios2/mm/fault.c | 12 +-- arch/nios2/mm/init.c | 4 +- arch/openrisc/mm/fault.c | 10 +-- arch/parisc/kernel/traps.c | 6 +- arch/parisc/mm/fault.c | 8 +- arch/powerpc/kernel/vdso.c | 6 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 4 +- arch/powerpc/kvm/book3s_hv.c | 6 +- arch/powerpc/kvm/book3s_hv_uvmem.c | 12 +-- arch/powerpc/kvm/e500_mmu_host.c | 4 +- arch/powerpc/mm/book3s64/iommu_api.c | 4 +- arch/powerpc/mm/book3s64/subpage_prot.c | 12 +-- arch/powerpc/mm/copro_fault.c | 4 +- arch/powerpc/mm/fault.c | 12 +-- arch/powerpc/oprofile/cell/spu_task_sync.c | 6 +- arch/powerpc/platforms/cell/spufs/file.c | 4 +- arch/riscv/kernel/vdso.c | 4 +- arch/riscv/mm/fault.c | 10 +-- arch/s390/kernel/vdso.c | 4 +- arch/s390/kvm/gaccess.c | 4 +- arch/s390/kvm/kvm-s390.c | 24 +++--- arch/s390/kvm/priv.c | 32 ++++---- arch/s390/mm/fault.c | 6 +- arch/s390/mm/gmap.c | 40 ++++----- arch/s390/pci/pci_mmio.c | 4 +- arch/sh/kernel/sys_sh.c | 6 +- arch/sh/kernel/vsyscall/vsyscall.c | 4 +- arch/sh/mm/fault.c | 14 ++-- arch/sparc/mm/fault_32.c | 18 ++-- arch/sparc/mm/fault_64.c | 12 +-- arch/sparc/vdso/vma.c | 4 +- arch/um/include/asm/mmu_context.h | 5 +- arch/um/kernel/tlb.c | 2 +- arch/um/kernel/trap.c | 6 +- arch/unicore32/mm/fault.c | 6 +- arch/x86/entry/vdso/vma.c | 14 ++-- arch/x86/events/core.c | 4 +- arch/x86/kernel/tboot.c | 2 +- arch/x86/kernel/vm86_32.c | 4 +- arch/x86/kvm/mmu/paging_tmpl.h | 8 +- arch/x86/mm/fault.c | 10 +-- arch/x86/um/vdso/vma.c | 4 +- arch/xtensa/mm/fault.c | 10 +-- drivers/android/binder_alloc.c | 10 +-- drivers/dma-buf/dma-resv.c | 4 +- drivers/firmware/efi/efi.c | 2 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 10 +-- drivers/gpu/drm/amd/amdkfd/kfd_events.c | 4 +- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_mman.c | 4 +- drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 8 +- drivers/gpu/drm/nouveau/nouveau_svm.c | 20 ++--- drivers/gpu/drm/radeon/radeon_cs.c | 4 +- drivers/gpu/drm/radeon/radeon_gem.c | 6 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 4 +- drivers/infiniband/core/umem_odp.c | 4 +- drivers/infiniband/core/uverbs_main.c | 4 +- drivers/infiniband/hw/mlx4/mr.c | 4 +- drivers/infiniband/hw/qib/qib_user_pages.c | 6 +- drivers/infiniband/hw/usnic/usnic_uiom.c | 4 +- drivers/infiniband/sw/siw/siw_mem.c | 4 +- drivers/iommu/amd_iommu_v2.c | 4 +- drivers/iommu/intel-svm.c | 4 +- drivers/media/v4l2-core/videobuf-core.c | 4 +- drivers/media/v4l2-core/videobuf-dma-contig.c | 4 +- drivers/media/v4l2-core/videobuf-dma-sg.c | 4 +- drivers/misc/cxl/cxllib.c | 4 +- drivers/misc/cxl/fault.c | 4 +- drivers/misc/sgi-gru/grufault.c | 16 ++-- drivers/misc/sgi-gru/grufile.c | 4 +- drivers/oprofile/buffer_sync.c | 10 +-- drivers/staging/kpc2000/kpc_dma/fileops.c | 4 +- drivers/tee/optee/call.c | 4 +- drivers/vfio/vfio_iommu_type1.c | 8 +- drivers/xen/gntdev.c | 4 +- drivers/xen/privcmd.c | 14 ++-- fs/aio.c | 4 +- fs/coredump.c | 4 +- fs/exec.c | 16 ++-- fs/io_uring.c | 4 +- fs/proc/base.c | 18 ++-- fs/proc/task_mmu.c | 28 +++---- fs/proc/task_nommu.c | 18 ++-- fs/userfaultfd.c | 28 +++---- include/linux/mm.h | 1 + include/linux/mm_types.h | 2 +- include/linux/mmap_lock.h | 82 +++++++++++++++++++ include/linux/mmu_notifier.h | 5 +- ipc/shm.c | 8 +- kernel/acct.c | 4 +- kernel/bpf/stackmap.c | 13 ++- kernel/events/core.c | 4 +- kernel/events/uprobes.c | 16 ++-- kernel/exit.c | 8 +- kernel/fork.c | 14 ++-- kernel/futex.c | 4 +- kernel/sched/fair.c | 4 +- kernel/sys.c | 18 ++-- kernel/trace/trace_output.c | 4 +- mm/filemap.c | 6 +- mm/frame_vector.c | 4 +- mm/gup.c | 20 ++--- mm/hmm.c | 2 +- mm/init-mm.c | 2 +- mm/internal.h | 2 +- mm/khugepaged.c | 36 ++++---- mm/ksm.c | 34 ++++---- mm/madvise.c | 18 ++-- mm/memcontrol.c | 8 +- mm/memory.c | 14 ++-- mm/mempolicy.c | 22 ++--- mm/migrate.c | 8 +- mm/mincore.c | 4 +- mm/mlock.c | 16 ++-- mm/mmap.c | 36 ++++---- mm/mmu_notifier.c | 22 ++--- mm/mprotect.c | 12 +-- mm/mremap.c | 6 +- mm/msync.c | 8 +- mm/nommu.c | 16 ++-- mm/oom_kill.c | 4 +- mm/pagewalk.c | 15 ++-- mm/process_vm_access.c | 4 +- mm/ptdump.c | 4 +- mm/swapfile.c | 4 +- mm/userfaultfd.c | 14 ++-- mm/util.c | 12 +-- net/ipv4/tcp.c | 4 +- net/xdp/xdp_umem.c | 4 +- virt/kvm/arm/mmu.c | 14 ++-- virt/kvm/async_pf.c | 4 +- virt/kvm/kvm_main.c | 8 +- 160 files changed, 778 insertions(+), 693 deletions(-) create mode 100644 include/linux/mmap_lock.h