From patchwork Tue Apr 18 15:48:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13215859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99022C6FD18 for ; Tue, 18 Apr 2023 15:49:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 254888E0005; Tue, 18 Apr 2023 11:49:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2047C8E0001; Tue, 18 Apr 2023 11:49:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A6308E0005; Tue, 18 Apr 2023 11:49:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EEDBA8E0001 for ; Tue, 18 Apr 2023 11:49:05 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C218FA03B2 for ; Tue, 18 Apr 2023 15:49:05 +0000 (UTC) X-FDA: 80694945450.11.0FA45C4 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by imf30.hostedemail.com (Postfix) with ESMTP id EBE1D80006 for ; Tue, 18 Apr 2023 15:49:03 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=BGjrtyIP; spf=pass (imf30.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.47 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681832944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=5XZsgr9HPh8Eljn7jVTyLk/JO3F53C57zrpzuRSuBx0=; b=ybvqFdU+Ie5wN2CiooD9Mwf8bveIRshztuDnFoB+IEiPvzum6+czHR4Xd1KBFuN9nzrn41 +TQKLIGk4VqXVc4ofsVrahbT+CJuXrMNrmCCVJEkGvNH8fOiEHxMROu3tg12I+l2gu9vdj N5QErgpnUjN9bCzkcL9AnGlIs6vh3Xo= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=BGjrtyIP; spf=pass (imf30.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.47 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681832944; a=rsa-sha256; cv=none; b=fZI5Xx+eWsfDdzbqq22R87PsVBt1n/QeGWXFBNjkX4hkRfDUXx5qHXd1rWUcRV02YVKffS 2SAvqOJzJP6si1bcUyX9ZFDWtDtDQhQtbog9HmlxnKJPerC98XHTTeTmHYH/ZQljzI7NB6 n9JfP3Z9eoXgZ7KNFkG0Ib9jYXfReDU= Received: by mail-wm1-f47.google.com with SMTP id eo4-20020a05600c82c400b003f05a99a841so80834wmb.3 for ; Tue, 18 Apr 2023 08:49:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681832942; x=1684424942; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=5XZsgr9HPh8Eljn7jVTyLk/JO3F53C57zrpzuRSuBx0=; b=BGjrtyIPdobYVVvPnyN41D+plP/T/aisRYwjujeeSF5rqDke7NOT9gOT5Jti4CpMTh NPe6pAipOD+L8Q8uTrRl7eyzCEQTz6dFBIAEkkZwvEml4FQfGE100ItCdJpNENZtnpzb mlcZlDNFN4vwM7mk4/Us9OEROS5c45tokgXFdhSroEViSDC/ecZ25rJp4NBrLf2QVJ8/ d4Aj1Nkui/2bK3DQABqPlVNtw+QWy/KHSTrnwrCJZ5quVdgRwGrtxEkc+ijTRF2QWTt+ lBJ4HmMMGCVNHVxlJ2vUFf7XqHBsRuJJ4zdUjkAJCV2xzTrO7Y4lRdDSjZlexHRK1CB1 yxsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681832942; x=1684424942; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=5XZsgr9HPh8Eljn7jVTyLk/JO3F53C57zrpzuRSuBx0=; b=dPIXCLWNQ/FRQgZJeii+GM5RKtpFDR2RGzVkGbes1q/wvELRR7sqZOJe+3qLc4AVcC bnPBnpIDRYNWjhXvn68Cr3jRc8r+ZCUGEirVnWyrCUPKwNErXfuoO+TK/AFmorrDff0p Xz8qft/COdXhu0pIZSv95/IkYePLUsh7fR4iKkY0+MwQrJRKKFulzMYKKuxii1ESSzXe ODKGKm6b3et6TQHG6W+aWT91/ggj3+/2WMZLZ8bi/3Ph8y1iOu/FwqTOin9aJz3P7Pk5 Xx9p+Psu/ZBEPj2aIbetiyUbWYhlIXekiAyfQS50AcPOz4jWBKx5VdD/ZA6goIrnJx+a vLRw== X-Gm-Message-State: AAQBX9fBbaba6SA8giBYXnP0hOHRTEoVQN9+RjkkJrcdHiWBjfRG72lZ Xf4L2Wun5T8+El/47i8q91irtG2kw0c3vg== X-Google-Smtp-Source: AKy350ZMV4eTs/AXWwPaCAkTiqRSIY+VIgxE1ouWpU7lD6qlipuQ0OE84sjtpn+eoTSyu1ic3TBy0w== X-Received: by 2002:a05:600c:253:b0:3f1:733d:f3c0 with SMTP id 19-20020a05600c025300b003f1733df3c0mr5648148wmj.14.1681832941680; Tue, 18 Apr 2023 08:49:01 -0700 (PDT) Received: from lucifer.home (host86-156-84-164.range86-156.btcentralplus.com. [86.156.84.164]) by smtp.googlemail.com with ESMTPSA id i4-20020a5d55c4000000b002f74578f494sm11011481wrw.41.2023.04.18.08.49.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 08:49:00 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton Cc: Matthew Wilcox , David Hildenbrand , Jens Axboe , Lorenzo Stoakes Subject: [PATCH v4 0/6] remove the vmas parameter from GUP APIs Date: Tue, 18 Apr 2023 16:48:58 +0100 Message-Id: X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EBE1D80006 X-Stat-Signature: d9c7a194g6up713wqyr8nddicxdcehju X-HE-Tag: 1681832943-770854 X-HE-Meta: U2FsdGVkX18vKstVsYK+nepD8XGQzxk3f09XLI+DUPerIRxRgGE62s5mTtV/HYj5pX1DRHD1pR91A7BWNB2HiEVJqbuDWA8Xde4hHAPCO/ni0rl9p4xzkFcCYdKeIuPCf2HJFLbhciBTSSQ+DQ/bdGnQOu2CFcZka3KJx3Z43IvH147p6MRjs4xe4eKdHIEaYTquXoc3ORw+Sl/TdGnhaeY1BqNPbocW2c9xt6d2qzCyrgS9lxAxGFy9yi6I3TcovVthXsTZDinQrRblZcXCHZ9PCMTwTOUwzZVcSMf/GFYCvs0rhV8KwRlNsHmF0rjpxEi1TN3hhbkuNbgT1365GlM6yN9vxFv9LNq/qLD7DomAysBNC5V/LhNXcW/79iqwrpLGE7RYi+nSXXPyh1kMi8U38l+rgqKlC0m49/+FGQiEpsbssiylU2ByG+8PnhCGk7OVh4Q8/FiF01DYUi5vaR9zWjYXW+ieUAcgvVPgR9yBmdz3nOuTRcsXkIXBxeTZTe7DGh1/0ZgH/qRwIav696LsjKl0j2fD5g7ajDphqS+iv55s6kFoJvwlU10zInnL6Dj5DUey8xQMJWFbIIUZez5WrYNWJqsQ1y/dr1c603wyOpmWj4yqd7pbJfH+dQWewkJwzdyuBD9QKjws/dvPnU2WPjrvWVmSC/e1Vm/cgyyf8IQL+0rt1VReJ9qoE86HCyMs1XJejTlM0839s0UtxCy3VpgyU67X+Il6znTLbM7uHm2RUqWheBgLN4HUuOY0J+MV2VNZDkl3hiz7q6yIQAR4pSbazmhlT+/Uc8FAnFWbAP+momkvoAQ7Grze220tMKQJVjrLgYi7ITQUlthu4O3cf8hty+79/XAIjp/w77QDedDt51uQqO/fZguFmD8D8vwmGylSYxri0ZABu0xKC8q5GVBgDqQCiXTQU8EguX0e6eJBIB6s73wQyxRQoSxjnmEmpl+iQIXUVcu8N4R j1NWR7Lh 6ONPKmB933+4bqDKc3MBeUoa4LCIia8rpxeM0KHUPY3hmjJI5W2WkowdAqntef8AIpZGXJrKeIc25AVZa2UfE06Nd0nGNp7vHUNNA+EQrsCPsDaQQAWM8eVpVBlQW2SQY38qqMDSUDjYB2nYSqeeIwQNcp6h1+LZkxisk0BCXJO8JPjJtRAZTYlX/J8K9Urc2kO5MWaEwjVg2bcrv2PmhmsgfEnIJN1q5OF+43sHh1y0ss8TMj/uOK2W9bkv0HmaKSLvgDSTrYiSfxxUGYFCooKZOr8x0wDcidNqzRkex1FX5DeWNQbfhU4yHEOdVaoBJlG8bQW8euPCpI0HbGbxUE7gRmFO2zEob5KboFSr3bxglDgStfpT1EQ7yt9N4TqY/PlCb7Lm5sD8/SnuQvoiWoo0EecXyZrOGdeclSC/AgcltNRz796J/EU5bWJLY6V/sAX/0bSC2ncztBTSgq6AkC8brF82Koz51wCop X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: (pin_/get)_user_pages[_remote]() each provide an optional output parameter for an array of VMA objects associated with each page in the input range. These provide the means for VMAs to be returned, as long as mm->mmap_lock is never released during the GUP operation (i.e. the internal flag FOLL_UNLOCKABLE is not specified). In addition, these VMAs have also to only be accessed under the mmap_lock, and become invalidated the moment it is released. The vast majority of invocations do not use this functionality and of those that do, all but one retrieve a single VMA to perform checks upon. It is not egregious in the single VMA cases to simply replace the operation with a vma_lookup(). In these cases we duplicate the (fast) lookup on a slow path already under the mmap_lock, abstracted to a new get_user_page_vma_remote() inline helper function which also performs error checking and reference count maintenance. The special case is io_uring, where io_pin_pages() specifically needs to assert that all the VMAs possess the same vm->vm_file (possibly NULL) and they are either anonymous or hugetlb pages. We adjust this to perform its own VMA lookup, which in most cases should consist of a single VMA, so the performance cost on an already slow path should be minimal. By doing so, we avoid an allocation in any case. In future it is sensible to simply restrict write pinning of file-backed folios in which case we may be able to simply avoid this check altogether, but for the time being we maintain it as-is. Eliminating the vmas parameter eliminates an entire class of possible danging pointer errors should the lock have been incorrectly released. In addition the API is simplified and now clearly expresses what it is for - applying the specified GUP flags and (if pinning) returning pinned pages. This change additionally opens the door to further potential improvements in GUP and the possible marrying of disparate code paths. I have run the gup_test and a simple io_uring program which exercises the use of FOLL_SAME_PAGE with no issues. This patch series is rebased on mm-unstable as of 17th April. Thanks to Matthew Wilcox for suggesting this refactoring! v4: - Drop FOLL_SAME_FILE as the complexity costs exceed the benefit of having it for a single case. - Update io_pin_pages() to perform VMA lookup directly. - Add get_user_page_vma_remote() to perform the single page/VMA lookup with error checks performed correctly. v3: - Always explicitly handle !vma cases, feeding back an error to the user if appropriate, indicating the operation did not completely succeed if not and always with a warning since these conditions should be impossible. https://lore.kernel.org/linux-mm/cover.1681558407.git.lstoakes@gmail.com/ v2: - Only lookup the VMA if the pin succeeded (other than __access_remote_vm() which has different semantics) - Be pedantically careful about ensuring that under no circumstances can we fail to unpin a page https://lore.kernel.org/linux-mm/cover.1681547405.git.lstoakes@gmail.com/ v1: https://lore.kernel.org/linux-mm/cover.1681508038.git.lstoakes@gmail.com/ Lorenzo Stoakes (6): mm/gup: remove unused vmas parameter from get_user_pages() mm/gup: remove unused vmas parameter from pin_user_pages_remote() mm/gup: remove vmas parameter from get_user_pages_remote() io_uring: rsrc: avoid use of vmas parameter in pin_user_pages() mm/gup: remove vmas parameter from pin_user_pages() mm/gup: remove vmas array from internal GUP functions arch/arm64/kernel/mte.c | 17 ++-- arch/powerpc/mm/book3s64/iommu_api.c | 2 +- arch/s390/kvm/interrupt.c | 2 +- arch/x86/kernel/cpu/sgx/ioctl.c | 2 +- drivers/gpu/drm/radeon/radeon_ttm.c | 2 +- drivers/infiniband/hw/qib/qib_user_pages.c | 2 +- drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +- drivers/infiniband/sw/siw/siw_mem.c | 2 +- drivers/iommu/iommufd/pages.c | 4 +- drivers/media/v4l2-core/videobuf-dma-sg.c | 2 +- drivers/misc/sgi-gru/grufault.c | 2 +- drivers/vdpa/vdpa_user/vduse_dev.c | 2 +- drivers/vfio/vfio_iommu_type1.c | 2 +- drivers/vhost/vdpa.c | 2 +- fs/exec.c | 2 +- include/linux/hugetlb.h | 10 +- include/linux/mm.h | 42 +++++++-- io_uring/rsrc.c | 53 ++++++----- kernel/events/uprobes.c | 13 +-- mm/gup.c | 105 +++++++-------------- mm/gup_test.c | 14 ++- mm/hugetlb.c | 24 ++--- mm/memory.c | 14 +-- mm/process_vm_access.c | 2 +- mm/rmap.c | 2 +- net/xdp/xdp_umem.c | 2 +- security/tomoyo/domain.c | 2 +- virt/kvm/async_pf.c | 3 +- virt/kvm/kvm_main.c | 2 +- 29 files changed, 161 insertions(+), 174 deletions(-) --- 2.40.0