From patchwork Sat Jan 7 00:33:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13091910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C3A4C54EBD for ; Sat, 7 Jan 2023 00:33:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9693E8E0002; Fri, 6 Jan 2023 19:33:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 919A38E0001; Fri, 6 Jan 2023 19:33:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E1E88E0002; Fri, 6 Jan 2023 19:33:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6AAC78E0001 for ; Fri, 6 Jan 2023 19:33:40 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 466B0C0E56 for ; Sat, 7 Jan 2023 00:33:40 +0000 (UTC) X-FDA: 80326129800.29.713D029 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 9C8BD1C000E for ; Sat, 7 Jan 2023 00:33:37 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Q7BWaeD/"; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673051618; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=SUBPjX7Z5mZLCRVw93gerYdodDIv8TWR14UIlWwTlcc=; b=QSjDT02MXZduuv9s/Z85Qq9GIHHHXD8gdBO3yf36WQI7dIs8r4rGB0mcZN39iwLB6MmE8e 3AVooZgLCk2wc/0EqF2mcGH8pRAC5oOAKqteVNdXQJzu4cMKBiMtSVqtZ2b520D0Ni4jCy UGzHKfspvUFQgOlb5VIY7S/VtLnE44Q= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Q7BWaeD/"; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673051618; a=rsa-sha256; cv=none; b=X5byiTCrWEVjnT4KRbCZs6pAZh8ybs4WIZE49IYEPkqt0KlLS2GIy7S+xsbPC+o7qUXl73 M5QSm71qOR6y4AhoY4tto3bvD+zHbrWF3B5ANlZMm9WnmD69Z6+UdYT6uU0jmU2nX3uF79 LilSJWEvq4ypWRFgRasNfFCOBpfIrsk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673051616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SUBPjX7Z5mZLCRVw93gerYdodDIv8TWR14UIlWwTlcc=; b=Q7BWaeD///51lYU5oj6FAsuVsFXqgh7RslCpveKz+UpU4nZwcLHzqenw0ZIvA/C1HY3LNc dHyNRuHLco4UhgeRQREi+LQDL4uBGpCOM1g3Vbsszggh8ujNmszZbF+lwAR379at4ONxG8 6d+UVepY66BajrKNIk/PDJa4dBV31m4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-479-esqWfMWzMZ2QK6nB1gTV2A-1; Fri, 06 Jan 2023 19:33:33 -0500 X-MC-Unique: esqWfMWzMZ2QK6nB1gTV2A-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7711F802C1D; Sat, 7 Jan 2023 00:33:32 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 251CC492B06; Sat, 7 Jan 2023 00:33:29 +0000 (UTC) Subject: [PATCH v4 0/7] iov_iter: Add extraction helpers From: David Howells To: Al Viro Cc: linux-fsdevel@vger.kernel.org, linux-cifs@vger.kernel.org, Christoph Hellwig , Steve French , linux-mm@kvack.org, Jens Axboe , Logan Gunthorpe , Matthew Wilcox , John Hubbard , linux-cachefs@redhat.com, Rohith Surabattula , Shyam Prasad N , Jeff Layton , dhowells@redhat.com, Christoph Hellwig , Matthew Wilcox , Jens Axboe , Jeff Layton , Logan Gunthorpe , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Date: Sat, 07 Jan 2023 00:33:29 +0000 Message-ID: <167305160937.1521586.133299343565358971.stgit@warthog.procyon.org.uk> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9C8BD1C000E X-Rspam-User: X-Stat-Signature: 59m1zutzo4priupdfwgrw1f738geczg3 X-HE-Tag: 1673051617-641339 X-HE-Meta: U2FsdGVkX1+9GJupteRFfJkJGrgh6iHPAVs+HDOOaXWZj2+qVOkxpt8V8/AlAc7kjovhfKBDPbJZMqreArMYNzxYCpdOORrGT13xQhMgjd5zsA5m8dD3Bv4w3bENhBilTkS8Z/xTB1I2CQiF6l/FzAPGkrK5F1lXQt+kONMcojNKbpKIf7JYxPbuJ6Dm6fYTOK05gfYqO0eq6jMjyEVsdT80zSTojLqJrRrGlm4iC7N6p7cdeuJ8GFEtHLt10wMmCotX550lGxXUt9Uc47M223t7ccYHkincARcxh92OM8OGZs7h7dWi+0pwe12unNWEVkku1Hvmwu7jgnv1KUXriWSSITAb9ip9+eAd5S2y7aqr5KtupiVBJmUZFyBYP5NvSXx0MsWuHNxxYVjTqHBNm8E9FS2DJcVIe96mXt5LhwEhseRHfUqpg9WeKieLLTA2FAkuDlcs3myUqJAwPwA0XpOblmN606vZ7bO91TGPhK66V75p7nnGop3LuFmbaEDuzX6Rsr9VfZdEokgXyS3R65qX+MeFyNnQNG62tckYVuhQfF+iFfoeYDQUPx8ECTy9zPFFf9sgmJxF370JE3TP2NhV9Ki8mtNau3uDtVkv+cXisBNBQfBv9qR5cmPv6ihFI4mPTTN14BNmUbX5/eIcK+tFVN0xLbAT5wd66VFlf4RT6DiaIEF0OX5tKOK4NS/CCLDZjRDORtnfUtB5lsUKV0Nv2hTs/Dez8rCXW5kgHT7SCtM7mFh7ndhTXXWZ/hJ0GrmQ2eYuDk0kAHG8ncXetVGzmz7Ymn84het12vkj7qMNCtGXy+q7guJdU7LWLAyC8NinqA1vtX+Rlvzt1UCNt5MMJO8pNnRPQOu96e+r7xujoVnpPj0209JA9cYQ/1XiN2rojWfBX40z4PoraTnq+9g6HFeP1Fa0PGEt8MkwSm6yZJe11O3jesV4QfB9xXU68WVMKcR5apA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Al, Here are patches clean up some use of READ/WRITE and ITER_SOURCE/DEST, patches to provide support for extracting pages from an iov_iter and a patch to use the primary extraction function in the block layer bio code if you could take a look? [!] NOTE that I've switched the functions to be exported GPL-only at Christoph's request[1]. They are, however, intended as a replacement for iov_iter_get_pages*(), which is not marked _GPL - so that functionality will probably become unavailable to non-GPL 3rd party modules in future. The first three patches deal with ITER_SOURCE/DEST: (1) Switch ITER_SOURCE/DEST to an enum and add a couple of helper functions to query if an iterator represents a source or a destination buffer. Using an enum may allow extra consistency warnings from the compiler. (2) Use the ITER_SOURCE/DEST values in the iov_iter core functions rather than READ/WRITE. (3) Get rid of most of the callers of iov_iter_rw(), using the IOCB_WRITE and IOMAP_WRITE instead where available. This leaves only two places looking at the this value: a consistency check in cifs and a single place in the block layer. The next patch adds a replacement for iov_iter_get_pages*(), including Logan's new version: (4) Add a function to list-only, get or pin pages from an iterator as a future replacement for iov_iter_get_pages*(). Pointers to the pages are placed into an array (which will get allocated if not provided) and, depending on the iterator type and direction, the pages will have a ref or a pin get on them or be left untouched (on the assumption that the caller manages their lifetime). The determination is: UBUF/IOVEC + DEST -> pin UBUF/IOVEC + SOURCE -> get PIPE + DEST -> list-only BVEC/XARRAY -> list-only Anything else -> EFAULT The function also returns an indication of which of "list only, get or pin" the extraction function did to aid in cleaning up (returning 0, FOLL_GET or FOLL_PIN as appropriate). Then there are a couple of patches that add stuff to netfslib that I want to use there as well as in cifs: (5) Add a netfslib function to use (4) to extract pages from an ITER_IOBUF or ITER_UBUF iterator into an ITER_BVEC iterator. This will get or pin the pages as appropriate. (6) Add a netfslib function to extract pages from an iterator that's of type ITER_UBUF/IOVEC/BVEC/KVEC/XARRAY and add them to a scatterlist. The function in (4) is used for a UBUF and IOVEC iterators, so those need cleaning up afterwards. BVEC and XARRAY iterators are ungot and unpinned and may be rendered into elements that span multiple pages, for example if large folios are present. Finally, there's an example test patch for the extract pages: (7) Make the block layer's BIO code pin pages or leave pages unaltered rather than getting a ref on the pages when the circumstances warrant, and noting in the bio struct what cleanups should be performed so that the bio cleanup code then does the right thing. Changes: ======== ver #4) - Drop the patch to move the FOLL_* flags to linux/mm_types.h as they're no longer referenced by linux/uio.h. - Add patches 1-3 and 7. - Make patches 4-6 use ITER_SOURCE/DEST. - Allow additional gup_flags to be passed into iov_iter_extract_pages(). ver #3) - Switch to using EXPORT_SYMBOL_GPL to prevent indirect 3rd-party access to get/pin_user_pages_fast()[1]. ver #2) - Rolled the extraction cleanup mode query function into the extraction function, returning the indication through the argument list. - Fixed patch 4 (extract to scatterlist) to actually use the new extraction API. I've pushed the patches here also: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract David Link: https://lore.kernel.org/r/Y3zFzdWnWlEJ8X8/@infradead.org/ [1] Link: https://lore.kernel.org/r/166697254399.61150.1256557652599252121.stgit@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r/166722777223.2555743.162508599131141451.stgit@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r/166732024173.3186319.18204305072070871546.stgit@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r/166869687556.3723671.10061142538708346995.stgit@warthog.procyon.org.uk/ # rfc Link: https://lore.kernel.org/r/166920902005.1461876.2786264600108839814.stgit@warthog.procyon.org.uk/ # v2 Link: https://lore.kernel.org/r/166997419665.9475.15014699817597102032.stgit@warthog.procyon.org.uk/ # v3 --- David Howells (7): iov_iter: Change the direction macros into an enum iov_iter: Use the direction in the iterator functions iov_iter: Use IOCB/IOMAP_WRITE if available rather than iterator direction iov_iter: Add a function to extract a page list from an iterator netfs: Add a function to extract a UBUF or IOVEC into a BVEC iterator netfs: Add a function to extract an iterator into a scatterlist iov_iter, block: Make bio structs pin pages rather than ref'ing if appropriate block/bio.c | 47 +++-- block/fops.c | 8 +- fs/9p/vfs_addr.c | 2 +- fs/affs/file.c | 4 +- fs/ceph/file.c | 2 +- fs/dax.c | 6 +- fs/direct-io.c | 22 +-- fs/exfat/inode.c | 6 +- fs/ext2/inode.c | 2 +- fs/f2fs/file.c | 10 +- fs/fat/inode.c | 4 +- fs/fuse/dax.c | 2 +- fs/fuse/file.c | 8 +- fs/hfs/inode.c | 2 +- fs/hfsplus/inode.c | 2 +- fs/iomap/direct-io.c | 6 +- fs/jfs/inode.c | 2 +- fs/netfs/Makefile | 1 + fs/netfs/iterator.c | 367 ++++++++++++++++++++++++++++++++++ fs/nfs/direct.c | 2 +- fs/nilfs2/inode.c | 2 +- fs/ntfs3/inode.c | 2 +- fs/ocfs2/aops.c | 2 +- fs/orangefs/inode.c | 2 +- fs/reiserfs/inode.c | 2 +- fs/udf/inode.c | 2 +- include/linux/blk_types.h | 1 + include/linux/netfs.h | 7 + include/linux/uio.h | 59 ++++-- lib/iov_iter.c | 407 +++++++++++++++++++++++++++++++++++--- mm/vmalloc.c | 1 + 31 files changed, 889 insertions(+), 103 deletions(-) create mode 100644 fs/netfs/iterator.c