From patchwork Tue Apr 18 15:49:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13215866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C9AFC77B75 for ; Tue, 18 Apr 2023 15:49:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D03338E0009; Tue, 18 Apr 2023 11:49:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB3778E0001; Tue, 18 Apr 2023 11:49:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A916D8E0009; Tue, 18 Apr 2023 11:49:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 994598E0001 for ; Tue, 18 Apr 2023 11:49:18 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 605F4C037F for ; Tue, 18 Apr 2023 15:49:18 +0000 (UTC) X-FDA: 80694945996.07.624E015 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf26.hostedemail.com (Postfix) with ESMTP id 7670214001C for ; Tue, 18 Apr 2023 15:49:16 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=RxtxdKNe; spf=pass (imf26.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681832956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HeNjOFrTuZuCuhpkRwg6ivljw7cwq/rMRsd+qVjznzY=; b=WqTLlGzj7vZQ4COaGuomIO2f7fAGRXLFXN/JXzy9LzVY1mmttfd0CiFJslMiOecw7j2+2z G65gefoTUqvq/i8wsUiBkO4SgxtWQ68g4T54e6+Zh+g3hjIbX3oKLBikHoHTWL787wEudu uD8tUDT6s9i9n0uw5BUquj+La3v++w0= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=RxtxdKNe; spf=pass (imf26.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681832956; a=rsa-sha256; cv=none; b=SMWO3AFG8Y0bcc7ba3+cznXvjP7iepa6njdoZZb/Bd39GptbB7P1xueOvLDyN51WHqhqVF lRJef1ifidtPIlHGVIXqfGu5vQyWgufGQGuaANeYDgjAyCRXHyy6yullx3ScYsWHgP7RkX 18oTiSyU0XVzT3JTJ/NhkMSqVIn0uIQ= Received: by mail-wm1-f48.google.com with SMTP id l31-20020a05600c1d1f00b003f1718d89b2so109563wms.0 for ; Tue, 18 Apr 2023 08:49:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681832955; x=1684424955; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HeNjOFrTuZuCuhpkRwg6ivljw7cwq/rMRsd+qVjznzY=; b=RxtxdKNe4EOZeD/rsX9M86UlJh03a7oh+S5LkToD8wsEvZ7jKSdBSVphhFX2sOoDAJ bGzpAmPKEejxjg71GV52q7a27iyL/xYy4M/d2kApUF7C+GZ+i6FY2vODi6xC59fYPoKX c+ffAJ2BlU+6VKn6AhApQvJ4O+X8GOQrRqOzCoMjjEYTHeon9FGJpOWOo8nVtRq5dUSY /k/khXDfWND879oTdy2gnPRzduT+fU2tXidyBOXmbllH62qMwYg4xOhgBDPojHgESj5i guCcOU/kpK4NndpXwN8LmeA2Ce3BkwTQv/1Bnqhj2uDLwF2BOWRuyAOyl5jBHW0QJa1g Sg9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681832955; x=1684424955; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HeNjOFrTuZuCuhpkRwg6ivljw7cwq/rMRsd+qVjznzY=; b=DJxDxRbwZd9ujrRJ+qsv+kEFGExKDe28JsSLUfXn/IsSDlIWnlEkZrQ/BRzVID9XS4 bi+54ch8su1V41zlqujgKqfM7lk6xbMdzb53DFqQxStcmP/vHv1sPUn794D0OEFuIyn5 5FLqIOWabZqSvo1RpGx5/VeXC4AdAjj5o4wN5KJtKyY4zba93ValmA3PzrmVzdLRM81X axapBwRDXUu4jiI7jg5vHZcPc7qNO2R0TCP2co51amkwfAKKpsn81j4rtZMyj3T0gHbV Kjn0sW2jxHCbbJ9AiX2xCwxywnS80XVr5XAEGiFTAQsu9OMbJGU3LTzl+JwbHctVW0+0 V3DA== X-Gm-Message-State: AAQBX9fe6D4zliNJhkBiIIX+7yyCfKoS59E77Mh0ryWqQKkeNIMYR61e MvLVgJlQnjAsEvxjd2muscs6CiimQ8/UhA== X-Google-Smtp-Source: AKy350Zx/7jVACHVDEd59MlmNb4yh3Qa9I4tP2Z8+vUyRAqrt3dGiF4TfRMd44P/Wpn+tmYhFgyqMQ== X-Received: by 2002:a7b:ce97:0:b0:3f1:6fea:790a with SMTP id q23-20020a7bce97000000b003f16fea790amr7103576wmj.30.1681832954636; Tue, 18 Apr 2023 08:49:14 -0700 (PDT) Received: from lucifer.home (host86-156-84-164.range86-156.btcentralplus.com. [86.156.84.164]) by smtp.googlemail.com with ESMTPSA id k10-20020a7bc40a000000b003f1736fdfedsm6708400wmi.10.2023.04.18.08.49.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 08:49:13 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton Cc: Matthew Wilcox , David Hildenbrand , Jens Axboe , Pavel Begunkov , io-uring@vger.kernel.org, Lorenzo Stoakes Subject: [PATCH v4 4/6] io_uring: rsrc: avoid use of vmas parameter in pin_user_pages() Date: Tue, 18 Apr 2023 16:49:12 +0100 Message-Id: <956f4fc2204f23e4c00e9602ded80cb4e7b5df9b.1681831798.git.lstoakes@gmail.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7670214001C X-Stat-Signature: 64n3qo64c5aicmb6z67t5di3uxk3i116 X-HE-Tag: 1681832956-1148 X-HE-Meta: U2FsdGVkX19A4x0uJSUppcrm6CsVXqtMSBqTcLDopaZqsZda/fGXygXgzxvWZeBzeqZ/cXaQCBLecMiHuJyN3UdsPxjuFr2Y8sxMoLGck+iKt2OB82pep+wUjdoVl3swrSguKpuKHdpwEElI/Ok5umOkMxugYGEFN/Unoj2vZiHyQ/9bntePAvRhCGidT/FEO40lZWr39XBylcWnYoRSKlcoZkdKaixyUUS5LTbW0DC+ZDh92oXS2pzFe4MXFqld7cX7PRcAgviDdXq+dCZ9KerTRsEhP+seSCBuEubqWvvNz9V6rGSaDJe/zwk8nfPEfJZOzNZHaPvSUu+FAp9Vyhc/cwE0NS8hhRYrICpyOOo7kTp+tsQwGKT3LGV4ZlRcPL4YhibvESYNdqfmfkvuGkQkgMR2gvVK6MJT2LoLbzspJ0HOABbQn3NqCX40gRiVh8QGJDvsjm5XXwbYFwNJ5VVv2PKHtc/PKjDS7Ii/xzKolkmeUhJwkf3kc6bidJ+4tBtEPiZHEslunTq5eOt0o2CklIXPAdm/nQs6vzrDwoXeKz9mLdlVZl3Ra/NsAnZaYsgFAsoJKj2mGtSPwpQPFH/MZgsM6pcDEq6+0IX2DEu71MAFZCfH6e4Md0HwB1+S1zTN9Z8NUGqUvcd0cS/ZS+Ndnl8LxLsyC36ATDDvyCz9dAW9QPxA4eaLwStS8nMXe6PlgqapeDTI7vxYgG2hT4p4K9YupSWkRVd9e92vZN16fa6vatY4jCkOSE6veCSMJCTgqIGGt/Np3FMFWUlCmhGlWeX7KypQpv8ypwsa/NtSXfltrF4elGSvtHgU3+9yoi9324MyHVECH6XY9m2MM8gNZY/JJHt85UIAHCxhsMWKKOZPZNqXZJuA2n4rjhNU8nxgLld1IVltReRhLQi7+HjNKzIScjlc8uNBvIHddjZ8obqWYVN5jg5Cmyduxo7vHqdHiG3LAhec53esnC3 HDaY/7mA WN4Y47WtiXf6phlxRlPUtnu8yL1S5z+STjDPh5r1I2ReqtXHSLAjz5SumWQr+q42cioZiUkOAQCBbdGpWCcVWUS7ivAKMP6dQE3voCgGMTy/HCiraAe6/sg4x1nGXjaCmm2MsIK+am8e7qnZt2eBh2HAdvq+y9zyFxXJr9D4zt+IwziyE4zCQuCWyA2Oh30WXvgsVE+aPv1H8eSqIRRPnnNTRSD84jusLOQnsLx1nvIe+MwrARcEKfs1hvn2EdeUKFRA0M3GZIRqt7D8mgxf9XNKzj/79uqYygKfG7j4bIIOULdMp4zXvfM9PZiOXAGgo06RNbZavOmFFW3v3+aH3t2aw/ihAR/zrcGcSuXH+X6+GCvj2OvhXNtgkW9YlfqEcLY7AR4HDX0k6eYqXr2J0IZPV8RTOMq3kOXC89XIIbC2fOGRVQwJX1oKpPy6gPfO9G5F2XVXWzgFICiRJ7KIZCjNEjaKs9f4OKGcwVzweUnehiH2HzQ8bQU9DF2IN5tqGiCJb+NYiqfqTP7gU6HIsN0/DXu+G62U3liCkwk0TiN/v/AV83QhTjlNulX4Abr/Q9qYpKqzO7Da3F3Dk7Eapc4n+5c2HrFIH2ON+Y09jqJ2hV4tG97eFjYLhleTcXZ8WToXG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We are shortly to remove pin_user_pages(), and instead perform the required VMA checks ourselves. In most cases there will be a single VMA so this should caues no undue impact on an already slow path. Doing this eliminates the one instance of vmas being used by pin_user_pages(). Suggested-by: Matthew Wilcox (Oracle) Signed-off-by: Lorenzo Stoakes --- io_uring/rsrc.c | 55 ++++++++++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 24 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 7a43aed8e395..3a927df9d913 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -1138,12 +1138,37 @@ static int io_buffer_account_pin(struct io_ring_ctx *ctx, struct page **pages, return ret; } +static int check_vmas_locked(unsigned long addr, unsigned long len) +{ + struct file *file; + VMA_ITERATOR(vmi, current->mm, addr); + struct vm_area_struct *vma = vma_next(&vmi); + unsigned long end = addr + len; + + if (WARN_ON_ONCE(!vma)) + return -EINVAL; + + file = vma->vm_file; + if (file && !is_file_hugepages(file)) + return -EOPNOTSUPP; + + /* don't support file backed memory */ + for_each_vma_range(vmi, vma, end) { + if (vma->vm_file != file) + return -EINVAL; + + if (file && !vma_is_shmem(vma)) + return -EOPNOTSUPP; + } + + return 0; +} + struct page **io_pin_pages(unsigned long ubuf, unsigned long len, int *npages) { unsigned long start, end, nr_pages; - struct vm_area_struct **vmas = NULL; struct page **pages = NULL; - int i, pret, ret = -ENOMEM; + int pret, ret = -ENOMEM; end = (ubuf + len + PAGE_SIZE - 1) >> PAGE_SHIFT; start = ubuf >> PAGE_SHIFT; @@ -1153,31 +1178,14 @@ struct page **io_pin_pages(unsigned long ubuf, unsigned long len, int *npages) if (!pages) goto done; - vmas = kvmalloc_array(nr_pages, sizeof(struct vm_area_struct *), - GFP_KERNEL); - if (!vmas) - goto done; - ret = 0; mmap_read_lock(current->mm); + pret = pin_user_pages(ubuf, nr_pages, FOLL_WRITE | FOLL_LONGTERM, - pages, vmas); - if (pret == nr_pages) { - struct file *file = vmas[0]->vm_file; + pages, NULL); - /* don't support file backed memory */ - for (i = 0; i < nr_pages; i++) { - if (vmas[i]->vm_file != file) { - ret = -EINVAL; - break; - } - if (!file) - continue; - if (!vma_is_shmem(vmas[i]) && !is_file_hugepages(file)) { - ret = -EOPNOTSUPP; - break; - } - } + if (pret == nr_pages) { + ret = check_vmas_locked(ubuf, len); *npages = nr_pages; } else { ret = pret < 0 ? pret : -EFAULT; @@ -1194,7 +1202,6 @@ struct page **io_pin_pages(unsigned long ubuf, unsigned long len, int *npages) } ret = 0; done: - kvfree(vmas); if (ret < 0) { kvfree(pages); pages = ERR_PTR(ret);