From patchwork Fri May 26 14:30:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FAA4C7EE23 for ; Fri, 26 May 2023 14:31:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE8F26B0075; Fri, 26 May 2023 10:31:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E99886B0078; Fri, 26 May 2023 10:31:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3AFD900002; Fri, 26 May 2023 10:31:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C0F946B0075 for ; Fri, 26 May 2023 10:31:28 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 80246140186 for ; Fri, 26 May 2023 14:31:28 +0000 (UTC) X-FDA: 80832644256.20.B4F104D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id 301C6100049 for ; Fri, 26 May 2023 14:31:21 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="MR0u42A/"; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111482; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p9op+s8ehZtzYPeOI0yMqOPJWZ6Q5oZk8H3B4oX4f10=; b=8PSdY8oGByJXnHDDEugnBCW4jXxU6c30RzRE2bzf3CKTcLw3MpTfUGtyHbZ0lwzbPVrC98 3EzbQr7FUJJwFM2l1EOW87JMGr0K6SXCCWMkEthvA1lxSqVST0B1hd+gDw4kiXWhJ2MK+D 6GxHx45ravSrvp6pMe9fWwNTwVCf9Dw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111482; a=rsa-sha256; cv=none; b=mCADY0LT6T2SNvx0XNXXUIRFeBJd3/i8Y9wpmwQv18/xK4/QRjAeGe8WYSsr+AVastEZBy uza2f8V+67KsWbEbZHZlvHj7YSGqn/ooHzm7WKSGdsv2tPjDB8slBERd/7xHyQLnBo1eBe xyvm5FYuLyNH4Eixx7NzkCRTxQgdPEY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="MR0u42A/"; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111480; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p9op+s8ehZtzYPeOI0yMqOPJWZ6Q5oZk8H3B4oX4f10=; b=MR0u42A/az8KeSZrFPtMaTtgiT879Pq9D3pF+pieuhGZrBrYTB4ugoHxOVpfa/QAp5usyh O63n5OiQQdGe/27yY/MS7V0ls8fTekrSj8JM5AyUk43FohyNIV35/9G/BboRJGxUD85dKf 3qkK4t83nATasRXxi+za32fcXfmovjY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-120-tg83vpdEPYCdFdE1NVv6fQ-1; Fri, 26 May 2023 10:31:14 -0400 X-MC-Unique: tg83vpdEPYCdFdE1NVv6fQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 67D4F280D582; Fri, 26 May 2023 14:31:13 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7BED1140E95D; Fri, 26 May 2023 14:31:10 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jeff Layton , Steve French , Shyam Prasad N , Rohith Surabattula , linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH net-next 1/8] Move netfs_extract_iter_to_sg() to lib/scatterlist.c Date: Fri, 26 May 2023 15:30:57 +0100 Message-Id: <20230526143104.882842-2-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: 301C6100049 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: gih3u739966xnm98stphsymzdd4zybyo X-HE-Tag: 1685111481-541229 X-HE-Meta: U2FsdGVkX1/SaeTKymETs+tnTnxLRRS+YDb/2ls2RxZyo3rOpT3uO1T5yEYlkkluhUSNqEY9ODjlW+uCFAhx/X6fRwKWoRor+Mss2ujPkPzRT5HKudLAspUg8E57JQ5nJUmlBl6uf5K9TV7KVn81ALM1/UeYhlF38tvhDGJZOP/AdDGfWrOThsLaXIQZuiORrh1iVlByRRNGCRZBD6DKsPYE5Dfk8XKFLtLsprPuSg+tXsjC74gbOZvQr+ah0LDGGm0DFQlpdK4IskJPMpLi+hPDJo9z5hWVjkB2KBkn0gP/SmU8geURZgtka7/K87MReHHDYEIrSl9atA59iwMbxr4v3FsZ7M1bCkEh/Rbx1zrKG1N62zr+KLoWHQG5KIlUsyE4GHUqdS4UuHfp2TWy9Lneerl0342lSbRARaveETZDPydtW1IRpOru6MSIvA6U1cP2u61VAHGZvp4iAIdsE6O6oZTCYFrJGC4UNGxTDCHlnZAynqIO0rqgG4ZDRdLSNWaqa5kG6oGKKSaavQnlIFXMvZQLixTFcMCNwBI1Gay7eYQBS5DbZDtHP3X248XA3hEot3GiAlUShnzgzULyIATzbjRGl7RCoxsFs7mFUxVC4LexcJifVJNOo+eHn8I8sjzhBHEuxssGWmI0sL2FLYn7es65ULfGf8XgnvpASx5Uv5S57RZL/cFRSHVq79Mq/5wOpJ/Ux2GaPT6gQJoPfXcxcqS+try04VQ+AvWR+DVDTw4tnGAOGAAZ7ffsSHUqKJxQMVyMOtgza2TC2RpOJQdBRPrfqhYyOSYSH5lTrEFxtwydeFSpJwVj9vApHCmVsrVd0jL4VOrXKvRc7gzmDvuYXdi2yEepzlIngQvUSaf4c4ImhkoO1xf1Hl4ZvO1iyAq/ejjkgPrLk5OhH/qkPFx8tO97A95rbfzjWK280+zyWN+ZLgG9o3r2Wc6OSTgI0DnIiX0Msv9aAIiiqna 2z1E5NCh e+nqc9jFOjn8HS4dquR8xx+9eyDQXKOv+Btf4lfVd7znFQPpb6S9PHnARiYomBJO1Sz/NuBKqFr9wlDkVeO4i65FaLlw0BAvZ9b52FlyVf38Wed6rSUeRHVbpr1CO44mkHoZd55HjcbGBgujo8luacX0hG4toXyFjyP1ZAPSySaL/IPfvpnm07OE8g3k83rK5d4oMxAhj/EBLhQKSp2W1rPa7xUmABI+TTSYoU4tYAPqIKLPziJKWwbOUqb9kEbWjftxIJG3nlks9PFH4t+5mi1PXwfuiPSRE7IGUCF7GEU34kszsql8tBRwVkWlQTAKObnOel2j4eNf99LQJOXmb+HyCW5SA+X8rFz4M9d4jDZT0gRFv0bxuEnJbVDDXzRpJFIB3x9F5+W6tXHo5d616r+8sn+mFA/c4i7amHRR2VHq+kQoyzcn+x3ULN4Ly6UMePtOV9bErSuH9rCCQz0vXFQ3W8e/a9vLoj1zLXBE800fEVrw4VxFMSJLf6zEPM/pjpuGjKZ93uZymGUsb7fr74WjsK8r9WFHl+u3v4RJN4F3rUg8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move netfs_extract_iter_to_sg() to lib/scatterlist.c as it's going to be used by more than just network filesystems (AF_ALG, for example). Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "David S. Miller" cc: linux-crypto@vger.kernel.org cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: netdev@vger.kernel.org --- fs/netfs/iterator.c | 266 ----------------------------------- include/linux/netfs.h | 4 - include/linux/scatterlist.h | 1 + include/linux/uio.h | 5 + lib/scatterlist.c | 267 ++++++++++++++++++++++++++++++++++++ 5 files changed, 273 insertions(+), 270 deletions(-) diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c index 8a4c86687429..2ff07ba655a0 100644 --- a/fs/netfs/iterator.c +++ b/fs/netfs/iterator.c @@ -101,269 +101,3 @@ ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len, return npages; } EXPORT_SYMBOL_GPL(netfs_extract_user_iter); - -/* - * Extract and pin a list of up to sg_max pages from UBUF- or IOVEC-class - * iterators, and add them to the scatterlist. - */ -static ssize_t netfs_extract_user_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) -{ - struct scatterlist *sg = sgtable->sgl + sgtable->nents; - struct page **pages; - unsigned int npages; - ssize_t ret = 0, res; - size_t len, off; - - /* We decant the page list into the tail of the scatterlist */ - pages = (void *)sgtable->sgl + array_size(sg_max, sizeof(struct scatterlist)); - pages -= sg_max; - - do { - res = iov_iter_extract_pages(iter, &pages, maxsize, sg_max, - extraction_flags, &off); - if (res < 0) - goto failed; - - len = res; - maxsize -= len; - ret += len; - npages = DIV_ROUND_UP(off + len, PAGE_SIZE); - sg_max -= npages; - - for (; npages > 0; npages--) { - struct page *page = *pages; - size_t seg = min_t(size_t, PAGE_SIZE - off, len); - - *pages++ = NULL; - sg_set_page(sg, page, seg, off); - sgtable->nents++; - sg++; - len -= seg; - off = 0; - } - } while (maxsize > 0 && sg_max > 0); - - return ret; - -failed: - while (sgtable->nents > sgtable->orig_nents) - put_page(sg_page(&sgtable->sgl[--sgtable->nents])); - return res; -} - -/* - * Extract up to sg_max pages from a BVEC-type iterator and add them to the - * scatterlist. The pages are not pinned. - */ -static ssize_t netfs_extract_bvec_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) -{ - const struct bio_vec *bv = iter->bvec; - struct scatterlist *sg = sgtable->sgl + sgtable->nents; - unsigned long start = iter->iov_offset; - unsigned int i; - ssize_t ret = 0; - - for (i = 0; i < iter->nr_segs; i++) { - size_t off, len; - - len = bv[i].bv_len; - if (start >= len) { - start -= len; - continue; - } - - len = min_t(size_t, maxsize, len - start); - off = bv[i].bv_offset + start; - - sg_set_page(sg, bv[i].bv_page, len, off); - sgtable->nents++; - sg++; - sg_max--; - - ret += len; - maxsize -= len; - if (maxsize <= 0 || sg_max == 0) - break; - start = 0; - } - - if (ret > 0) - iov_iter_advance(iter, ret); - return ret; -} - -/* - * Extract up to sg_max pages from a KVEC-type iterator and add them to the - * scatterlist. This can deal with vmalloc'd buffers as well as kmalloc'd or - * static buffers. The pages are not pinned. - */ -static ssize_t netfs_extract_kvec_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) -{ - const struct kvec *kv = iter->kvec; - struct scatterlist *sg = sgtable->sgl + sgtable->nents; - unsigned long start = iter->iov_offset; - unsigned int i; - ssize_t ret = 0; - - for (i = 0; i < iter->nr_segs; i++) { - struct page *page; - unsigned long kaddr; - size_t off, len, seg; - - len = kv[i].iov_len; - if (start >= len) { - start -= len; - continue; - } - - kaddr = (unsigned long)kv[i].iov_base + start; - off = kaddr & ~PAGE_MASK; - len = min_t(size_t, maxsize, len - start); - kaddr &= PAGE_MASK; - - maxsize -= len; - ret += len; - do { - seg = min_t(size_t, len, PAGE_SIZE - off); - if (is_vmalloc_or_module_addr((void *)kaddr)) - page = vmalloc_to_page((void *)kaddr); - else - page = virt_to_page(kaddr); - - sg_set_page(sg, page, len, off); - sgtable->nents++; - sg++; - sg_max--; - - len -= seg; - kaddr += PAGE_SIZE; - off = 0; - } while (len > 0 && sg_max > 0); - - if (maxsize <= 0 || sg_max == 0) - break; - start = 0; - } - - if (ret > 0) - iov_iter_advance(iter, ret); - return ret; -} - -/* - * Extract up to sg_max folios from an XARRAY-type iterator and add them to - * the scatterlist. The pages are not pinned. - */ -static ssize_t netfs_extract_xarray_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) -{ - struct scatterlist *sg = sgtable->sgl + sgtable->nents; - struct xarray *xa = iter->xarray; - struct folio *folio; - loff_t start = iter->xarray_start + iter->iov_offset; - pgoff_t index = start / PAGE_SIZE; - ssize_t ret = 0; - size_t offset, len; - XA_STATE(xas, xa, index); - - rcu_read_lock(); - - xas_for_each(&xas, folio, ULONG_MAX) { - if (xas_retry(&xas, folio)) - continue; - if (WARN_ON(xa_is_value(folio))) - break; - if (WARN_ON(folio_test_hugetlb(folio))) - break; - - offset = offset_in_folio(folio, start); - len = min_t(size_t, maxsize, folio_size(folio) - offset); - - sg_set_page(sg, folio_page(folio, 0), len, offset); - sgtable->nents++; - sg++; - sg_max--; - - maxsize -= len; - ret += len; - if (maxsize <= 0 || sg_max == 0) - break; - } - - rcu_read_unlock(); - if (ret > 0) - iov_iter_advance(iter, ret); - return ret; -} - -/** - * netfs_extract_iter_to_sg - Extract pages from an iterator and add ot an sglist - * @iter: The iterator to extract from - * @maxsize: The amount of iterator to copy - * @sgtable: The scatterlist table to fill in - * @sg_max: Maximum number of elements in @sgtable that may be filled - * @extraction_flags: Flags to qualify the request - * - * Extract the page fragments from the given amount of the source iterator and - * add them to a scatterlist that refers to all of those bits, to a maximum - * addition of @sg_max elements. - * - * The pages referred to by UBUF- and IOVEC-type iterators are extracted and - * pinned; BVEC-, KVEC- and XARRAY-type are extracted but aren't pinned; PIPE- - * and DISCARD-type are not supported. - * - * No end mark is placed on the scatterlist; that's left to the caller. - * - * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA - * be allowed on the pages extracted. - * - * If successul, @sgtable->nents is updated to include the number of elements - * added and the number of bytes added is returned. @sgtable->orig_nents is - * left unaltered. - * - * The iov_iter_extract_mode() function should be used to query how cleanup - * should be performed. - */ -ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t maxsize, - struct sg_table *sgtable, unsigned int sg_max, - iov_iter_extraction_t extraction_flags) -{ - if (maxsize == 0) - return 0; - - switch (iov_iter_type(iter)) { - case ITER_UBUF: - case ITER_IOVEC: - return netfs_extract_user_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); - case ITER_BVEC: - return netfs_extract_bvec_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); - case ITER_KVEC: - return netfs_extract_kvec_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); - case ITER_XARRAY: - return netfs_extract_xarray_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); - default: - pr_err("%s(%u) unsupported\n", __func__, iov_iter_type(iter)); - WARN_ON_ONCE(1); - return -EIO; - } -} -EXPORT_SYMBOL_GPL(netfs_extract_iter_to_sg); diff --git a/include/linux/netfs.h b/include/linux/netfs.h index a1f3522daa69..b11a84f6c32b 100644 --- a/include/linux/netfs.h +++ b/include/linux/netfs.h @@ -300,10 +300,6 @@ void netfs_stats_show(struct seq_file *); ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len, struct iov_iter *new, iov_iter_extraction_t extraction_flags); -struct sg_table; -ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t len, - struct sg_table *sgtable, unsigned int sg_max, - iov_iter_extraction_t extraction_flags); /** * netfs_inode - Get the netfs inode context from the inode diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..9e6a4457f4fc 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -6,6 +6,7 @@ #include #include #include +#include #include struct scatterlist { diff --git a/include/linux/uio.h b/include/linux/uio.h index 044c1d8c230c..09b8b107956e 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -433,4 +433,9 @@ static inline bool iov_iter_extract_will_pin(const struct iov_iter *iter) return user_backed_iter(iter); } +struct sg_table; +ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t len, + struct sg_table *sgtable, unsigned int sg_max, + iov_iter_extraction_t extraction_flags); + #endif diff --git a/lib/scatterlist.c b/lib/scatterlist.c index 8d7519a8f308..31ef86e6a33a 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -9,6 +9,7 @@ #include #include #include +#include /** * sg_next - return the next scatterlist entry in a list @@ -1095,3 +1096,269 @@ size_t sg_zero_buffer(struct scatterlist *sgl, unsigned int nents, return offset; } EXPORT_SYMBOL(sg_zero_buffer); + +/* + * Extract and pin a list of up to sg_max pages from UBUF- or IOVEC-class + * iterators, and add them to the scatterlist. + */ +static ssize_t netfs_extract_user_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) +{ + struct scatterlist *sg = sgtable->sgl + sgtable->nents; + struct page **pages; + unsigned int npages; + ssize_t ret = 0, res; + size_t len, off; + + /* We decant the page list into the tail of the scatterlist */ + pages = (void *)sgtable->sgl + array_size(sg_max, sizeof(struct scatterlist)); + pages -= sg_max; + + do { + res = iov_iter_extract_pages(iter, &pages, maxsize, sg_max, + extraction_flags, &off); + if (res < 0) + goto failed; + + len = res; + maxsize -= len; + ret += len; + npages = DIV_ROUND_UP(off + len, PAGE_SIZE); + sg_max -= npages; + + for (; npages > 0; npages--) { + struct page *page = *pages; + size_t seg = min_t(size_t, PAGE_SIZE - off, len); + + *pages++ = NULL; + sg_set_page(sg, page, seg, off); + sgtable->nents++; + sg++; + len -= seg; + off = 0; + } + } while (maxsize > 0 && sg_max > 0); + + return ret; + +failed: + while (sgtable->nents > sgtable->orig_nents) + put_page(sg_page(&sgtable->sgl[--sgtable->nents])); + return res; +} + +/* + * Extract up to sg_max pages from a BVEC-type iterator and add them to the + * scatterlist. The pages are not pinned. + */ +static ssize_t netfs_extract_bvec_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) +{ + const struct bio_vec *bv = iter->bvec; + struct scatterlist *sg = sgtable->sgl + sgtable->nents; + unsigned long start = iter->iov_offset; + unsigned int i; + ssize_t ret = 0; + + for (i = 0; i < iter->nr_segs; i++) { + size_t off, len; + + len = bv[i].bv_len; + if (start >= len) { + start -= len; + continue; + } + + len = min_t(size_t, maxsize, len - start); + off = bv[i].bv_offset + start; + + sg_set_page(sg, bv[i].bv_page, len, off); + sgtable->nents++; + sg++; + sg_max--; + + ret += len; + maxsize -= len; + if (maxsize <= 0 || sg_max == 0) + break; + start = 0; + } + + if (ret > 0) + iov_iter_advance(iter, ret); + return ret; +} + +/* + * Extract up to sg_max pages from a KVEC-type iterator and add them to the + * scatterlist. This can deal with vmalloc'd buffers as well as kmalloc'd or + * static buffers. The pages are not pinned. + */ +static ssize_t netfs_extract_kvec_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) +{ + const struct kvec *kv = iter->kvec; + struct scatterlist *sg = sgtable->sgl + sgtable->nents; + unsigned long start = iter->iov_offset; + unsigned int i; + ssize_t ret = 0; + + for (i = 0; i < iter->nr_segs; i++) { + struct page *page; + unsigned long kaddr; + size_t off, len, seg; + + len = kv[i].iov_len; + if (start >= len) { + start -= len; + continue; + } + + kaddr = (unsigned long)kv[i].iov_base + start; + off = kaddr & ~PAGE_MASK; + len = min_t(size_t, maxsize, len - start); + kaddr &= PAGE_MASK; + + maxsize -= len; + ret += len; + do { + seg = min_t(size_t, len, PAGE_SIZE - off); + if (is_vmalloc_or_module_addr((void *)kaddr)) + page = vmalloc_to_page((void *)kaddr); + else + page = virt_to_page(kaddr); + + sg_set_page(sg, page, len, off); + sgtable->nents++; + sg++; + sg_max--; + + len -= seg; + kaddr += PAGE_SIZE; + off = 0; + } while (len > 0 && sg_max > 0); + + if (maxsize <= 0 || sg_max == 0) + break; + start = 0; + } + + if (ret > 0) + iov_iter_advance(iter, ret); + return ret; +} + +/* + * Extract up to sg_max folios from an XARRAY-type iterator and add them to + * the scatterlist. The pages are not pinned. + */ +static ssize_t netfs_extract_xarray_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) +{ + struct scatterlist *sg = sgtable->sgl + sgtable->nents; + struct xarray *xa = iter->xarray; + struct folio *folio; + loff_t start = iter->xarray_start + iter->iov_offset; + pgoff_t index = start / PAGE_SIZE; + ssize_t ret = 0; + size_t offset, len; + XA_STATE(xas, xa, index); + + rcu_read_lock(); + + xas_for_each(&xas, folio, ULONG_MAX) { + if (xas_retry(&xas, folio)) + continue; + if (WARN_ON(xa_is_value(folio))) + break; + if (WARN_ON(folio_test_hugetlb(folio))) + break; + + offset = offset_in_folio(folio, start); + len = min_t(size_t, maxsize, folio_size(folio) - offset); + + sg_set_page(sg, folio_page(folio, 0), len, offset); + sgtable->nents++; + sg++; + sg_max--; + + maxsize -= len; + ret += len; + if (maxsize <= 0 || sg_max == 0) + break; + } + + rcu_read_unlock(); + if (ret > 0) + iov_iter_advance(iter, ret); + return ret; +} + +/** + * netfs_extract_iter_to_sg - Extract pages from an iterator and add ot an sglist + * @iter: The iterator to extract from + * @maxsize: The amount of iterator to copy + * @sgtable: The scatterlist table to fill in + * @sg_max: Maximum number of elements in @sgtable that may be filled + * @extraction_flags: Flags to qualify the request + * + * Extract the page fragments from the given amount of the source iterator and + * add them to a scatterlist that refers to all of those bits, to a maximum + * addition of @sg_max elements. + * + * The pages referred to by UBUF- and IOVEC-type iterators are extracted and + * pinned; BVEC-, KVEC- and XARRAY-type are extracted but aren't pinned; PIPE- + * and DISCARD-type are not supported. + * + * No end mark is placed on the scatterlist; that's left to the caller. + * + * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA + * be allowed on the pages extracted. + * + * If successul, @sgtable->nents is updated to include the number of elements + * added and the number of bytes added is returned. @sgtable->orig_nents is + * left unaltered. + * + * The iov_iter_extract_mode() function should be used to query how cleanup + * should be performed. + */ +ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t maxsize, + struct sg_table *sgtable, unsigned int sg_max, + iov_iter_extraction_t extraction_flags) +{ + if (maxsize == 0) + return 0; + + switch (iov_iter_type(iter)) { + case ITER_UBUF: + case ITER_IOVEC: + return netfs_extract_user_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); + case ITER_BVEC: + return netfs_extract_bvec_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); + case ITER_KVEC: + return netfs_extract_kvec_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); + case ITER_XARRAY: + return netfs_extract_xarray_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); + default: + pr_err("%s(%u) unsupported\n", __func__, iov_iter_type(iter)); + WARN_ON_ONCE(1); + return -EIO; + } +} +EXPORT_SYMBOL_GPL(netfs_extract_iter_to_sg); From patchwork Fri May 26 14:30:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF62AC77B73 for ; Fri, 26 May 2023 14:31:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF2396B0082; Fri, 26 May 2023 10:31:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA2226B0083; Fri, 26 May 2023 10:31:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 968BF900002; Fri, 26 May 2023 10:31:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 87FA86B0082 for ; Fri, 26 May 2023 10:31:49 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5A09D1A0EE6 for ; Fri, 26 May 2023 14:31:49 +0000 (UTC) X-FDA: 80832645138.18.C982A87 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id F0A1B1C0012 for ; Fri, 26 May 2023 14:31:24 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NYnAlgDi; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111485; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WtUJzNGs4hI/qozgKZw9Ev1spnarsyg42e3BEFrKZ44=; b=iRMlvM2P90ocYITGFpFql/ZbZ4+97n9eaSdJz81hO3OqQX9wZws8klGJX6XXsof6zw2Y3R q6LzJ7phqK5CCfajS3NTGF4pxFoHzFxC5fbTXGQL6bTRfSwkKvgRYwdD53jD2GLkM3rP90 JMXbWtq4HChLDw5uOv9N6uWfXAiL91E= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NYnAlgDi; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111485; a=rsa-sha256; cv=none; b=WBoRVb6z3uZmquQuwNEYymCBf5yygVGKR+Dp6HPTR5jC/2sNEv7SLD/6RPES1SJoD3BQEG hxUiCvbjou39GDQspWQ52zdxIbZrgQXJPlaAc9VAa9HSwXYILZiiV9+I6yFM66RPfDh1a4 t+Q29sY1KrVrCpfCrO2uCZ5zp0pDApU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111483; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WtUJzNGs4hI/qozgKZw9Ev1spnarsyg42e3BEFrKZ44=; b=NYnAlgDi1mMjr23lwW0sJR6k/EgqWKrPhklsxWg4D9QgMdAgPTlUCbLFEdVymJxvl/z5za ipIiNBzZbKCg8TQphJ5CP7/w879UwIGf/ui+TBYfZAebeFgH/MgMl6eiFNW25KJa7cGFUd 7QjLd+Vl3mX5hafZOGwtxGqNc3rmlgw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-625-JW7yVrnSMTmdp0eU8oppyA-1; Fri, 26 May 2023 10:31:18 -0400 X-MC-Unique: JW7yVrnSMTmdp0eU8oppyA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EDF71801182; Fri, 26 May 2023 14:31:16 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 39ACC2166B2B; Fri, 26 May 2023 14:31:14 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Jeff Layton , Steve French , Shyam Prasad N , Rohith Surabattula , linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH net-next 2/8] Drop the netfs_ prefix from netfs_extract_iter_to_sg() Date: Fri, 26 May 2023 15:30:58 +0100 Message-Id: <20230526143104.882842-3-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: F0A1B1C0012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 6tue9m5js9a5fdpbb5i7jzma8dwqmx7p X-HE-Tag: 1685111484-242088 X-HE-Meta: U2FsdGVkX18orDATg4ceRcMkAirWGLLQtWaUH0Um9Zm9yrzZ3e2wab4lbXUInbG0qKS9+4SLLbg8ArYprN5IRx+B6UhHyZahmCpFvUm8yaaM58VGAYwG5wKJQgxJVTbqdGpBNmwkfXtxuvrYWGsXFigSogAgPuytAFoSuJEvU/2f7ZL/9632tFj04Ca7AzgoYc0s2BHv9VPGPa+Hc904j4/w364HEB6WitmMb4BFj8HZbdK+4XGjI6p4gUNB2OB+bQSmw+ZO8DhTXNGM32nqHfqNR8wd02A8ieXC5L27cWLGYmZyoMau2LPtoxWp8Bgv6/7QSus5FD4tJPGOrwjH8XzWIKpjkq7jwmgoJFP4J7nvOo4XkE2rdX22RKKO3ws+/6GoL/hP9z4XyKCgftPnFDPSrdOTUSFTzKeXUynhQPn2RnHQjBJrKWFwXlmdzLbNCwEmIbhyamNtFPd/tsHlCfBgvBOXfbg/4Ps1btpl5zuDU4yVFvKl6CNS3teUXQIPYCZOhISs4yK9NaH9C1nB1lWiPWtt+ywpR6IqrZ1nt/SMltI9XKRbhKLRIFk45jc7vAUg+L+z7IVnK/3dkkTOa4RdEW1ZRxoeBPmfX6eeG8xHrY4AvN0NKtDoCnhgZfmHlGMXNwx5Ih9p60NSfnsthhTzk8uc1iw2TFPx0O2dp2PIUPk73sxk2Ag9ti5W72vNXOS8AK4oVZhb/c3Xg9XdS/pqX6jLfBUXq2MTwitkvsm5eW8OYrjzwGoVYxQD8psVtLqrfHraIENxglDh6ghGlp5svDsRT4fRfC6Vs/1atxh7y+MiyX6XbS8W90bkvrXrHJj7ECoc8P3UcIIn1oixrbImIF3Ont4n7zjLzq3pesuz1FO30khmewevJikp3ial6fuhEHfQZCQVTWvaFBKmOdbCuI10I3x8HrEexPMN+5/DuzP1J7OrJFkSa9RXOHI3iKj28lF0HsANbhTr/RM pxx+VOZZ aqObtMmJeBOfwVTOtSCklG9CQtM1tJbnvdrNFOcB8nZJwGFzRYljToIiNhQPVYd3gCiDQOsQLwfGTex65/u+n4e8n+f3REnDNsRAU/AHbmIcp/YTQ/+91VZDfNnwsJFHDXqCkYJ55vamVzxSLj5yFHPxoJ9iTEBDDce5yCQX9Y9R2udOzqkSVa/WKqf+QfFuLA5RnkbNxR9X/R+VSVanB6OITUkZB/fHklkCKvu/NVqSnDvxD4GV+fRC+v+uswhOnkNlIfejsClhIv+CJ2CSF+eMEmfkB9tZjq03efV+6p2qGRqN5py5hUhmWofH/xQkIOb/yFMMH1KXfR6WsA5hdnfGYyjuhvy4En2vLrT/06PKjeG6l8Vfu6N8j9vmeVuBx6ugMo7ePpbbloRkpr249TYByaCQZN3XOslNkHMqXp87ced6JraaQKA0TTg+Sg/D8415zuGaT3tcEm0E5xNEmpozOHP5ehIVTe+XLETQYFLQqCmM0zaM19MM9qmlJCUXlD2Xsw55kYSC0Vg2TTk71TgzBdR+i6CbWBIUcVQUU+6dsCMM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rename netfs_extract_iter_to_sg() and its auxiliary functions to drop the netfs_ prefix. Signed-off-by: David Howells cc: Jeff Layton cc: Steve French cc: Shyam Prasad N cc: Rohith Surabattula cc: Jens Axboe cc: Herbert Xu cc: "David S. Miller" cc: linux-crypto@vger.kernel.org cc: linux-cachefs@redhat.com cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: netdev@vger.kernel.org --- fs/cifs/smb2ops.c | 4 +-- fs/cifs/smbdirect.c | 2 +- include/linux/uio.h | 6 ++--- lib/scatterlist.c | 66 ++++++++++++++++++++++----------------------- 4 files changed, 39 insertions(+), 39 deletions(-) diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c index a295e4c2d54e..196bc49e73b8 100644 --- a/fs/cifs/smb2ops.c +++ b/fs/cifs/smb2ops.c @@ -4335,8 +4335,8 @@ static void *smb2_get_aead_req(struct crypto_aead *tfm, struct smb_rqst *rqst, } sgtable.orig_nents = sgtable.nents; - rc = netfs_extract_iter_to_sg(iter, count, &sgtable, - num_sgs - sgtable.nents, 0); + rc = extract_iter_to_sg(iter, count, &sgtable, + num_sgs - sgtable.nents, 0); iov_iter_revert(iter, rc); sgtable.orig_nents = sgtable.nents; } diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c index 0362ebd4fa0f..223e17c16b60 100644 --- a/fs/cifs/smbdirect.c +++ b/fs/cifs/smbdirect.c @@ -2227,7 +2227,7 @@ static int smbd_iter_to_mr(struct smbd_connection *info, memset(sgt->sgl, 0, max_sg * sizeof(struct scatterlist)); - ret = netfs_extract_iter_to_sg(iter, iov_iter_count(iter), sgt, max_sg, 0); + ret = extract_iter_to_sg(iter, iov_iter_count(iter), sgt, max_sg, 0); WARN_ON(ret < 0); if (sgt->nents > 0) sg_mark_end(&sgt->sgl[sgt->nents - 1]); diff --git a/include/linux/uio.h b/include/linux/uio.h index 09b8b107956e..0ccb983cf645 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -434,8 +434,8 @@ static inline bool iov_iter_extract_will_pin(const struct iov_iter *iter) } struct sg_table; -ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t len, - struct sg_table *sgtable, unsigned int sg_max, - iov_iter_extraction_t extraction_flags); +ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t len, + struct sg_table *sgtable, unsigned int sg_max, + iov_iter_extraction_t extraction_flags); #endif diff --git a/lib/scatterlist.c b/lib/scatterlist.c index 31ef86e6a33a..8612b9deaa7e 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -1101,11 +1101,11 @@ EXPORT_SYMBOL(sg_zero_buffer); * Extract and pin a list of up to sg_max pages from UBUF- or IOVEC-class * iterators, and add them to the scatterlist. */ -static ssize_t netfs_extract_user_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) +static ssize_t extract_user_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) { struct scatterlist *sg = sgtable->sgl + sgtable->nents; struct page **pages; @@ -1154,11 +1154,11 @@ static ssize_t netfs_extract_user_to_sg(struct iov_iter *iter, * Extract up to sg_max pages from a BVEC-type iterator and add them to the * scatterlist. The pages are not pinned. */ -static ssize_t netfs_extract_bvec_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) +static ssize_t extract_bvec_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) { const struct bio_vec *bv = iter->bvec; struct scatterlist *sg = sgtable->sgl + sgtable->nents; @@ -1200,11 +1200,11 @@ static ssize_t netfs_extract_bvec_to_sg(struct iov_iter *iter, * scatterlist. This can deal with vmalloc'd buffers as well as kmalloc'd or * static buffers. The pages are not pinned. */ -static ssize_t netfs_extract_kvec_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) +static ssize_t extract_kvec_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) { const struct kvec *kv = iter->kvec; struct scatterlist *sg = sgtable->sgl + sgtable->nents; @@ -1261,11 +1261,11 @@ static ssize_t netfs_extract_kvec_to_sg(struct iov_iter *iter, * Extract up to sg_max folios from an XARRAY-type iterator and add them to * the scatterlist. The pages are not pinned. */ -static ssize_t netfs_extract_xarray_to_sg(struct iov_iter *iter, - ssize_t maxsize, - struct sg_table *sgtable, - unsigned int sg_max, - iov_iter_extraction_t extraction_flags) +static ssize_t extract_xarray_to_sg(struct iov_iter *iter, + ssize_t maxsize, + struct sg_table *sgtable, + unsigned int sg_max, + iov_iter_extraction_t extraction_flags) { struct scatterlist *sg = sgtable->sgl + sgtable->nents; struct xarray *xa = iter->xarray; @@ -1307,7 +1307,7 @@ static ssize_t netfs_extract_xarray_to_sg(struct iov_iter *iter, } /** - * netfs_extract_iter_to_sg - Extract pages from an iterator and add ot an sglist + * extract_iter_to_sg - Extract pages from an iterator and add ot an sglist * @iter: The iterator to extract from * @maxsize: The amount of iterator to copy * @sgtable: The scatterlist table to fill in @@ -1334,9 +1334,9 @@ static ssize_t netfs_extract_xarray_to_sg(struct iov_iter *iter, * The iov_iter_extract_mode() function should be used to query how cleanup * should be performed. */ -ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t maxsize, - struct sg_table *sgtable, unsigned int sg_max, - iov_iter_extraction_t extraction_flags) +ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize, + struct sg_table *sgtable, unsigned int sg_max, + iov_iter_extraction_t extraction_flags) { if (maxsize == 0) return 0; @@ -1344,21 +1344,21 @@ ssize_t netfs_extract_iter_to_sg(struct iov_iter *iter, size_t maxsize, switch (iov_iter_type(iter)) { case ITER_UBUF: case ITER_IOVEC: - return netfs_extract_user_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); + return extract_user_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); case ITER_BVEC: - return netfs_extract_bvec_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); + return extract_bvec_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); case ITER_KVEC: - return netfs_extract_kvec_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); + return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); case ITER_XARRAY: - return netfs_extract_xarray_to_sg(iter, maxsize, sgtable, sg_max, - extraction_flags); + return extract_xarray_to_sg(iter, maxsize, sgtable, sg_max, + extraction_flags); default: pr_err("%s(%u) unsupported\n", __func__, iov_iter_type(iter)); WARN_ON_ONCE(1); return -EIO; } } -EXPORT_SYMBOL_GPL(netfs_extract_iter_to_sg); +EXPORT_SYMBOL_GPL(extract_iter_to_sg); From patchwork Fri May 26 14:30:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8090C7EE2E for ; Fri, 26 May 2023 14:31:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEA8C6B0078; Fri, 26 May 2023 10:31:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E27246B007B; Fri, 26 May 2023 10:31:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C50B5900002; Fri, 26 May 2023 10:31:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B4C836B0078 for ; Fri, 26 May 2023 10:31:29 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1772A80ED7 for ; Fri, 26 May 2023 14:31:28 +0000 (UTC) X-FDA: 80832644298.17.FF66392 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id AB6191C0018 for ; Fri, 26 May 2023 14:31:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Nlk7HVaS; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jPGukvQXP4Tai7PUqcbnIswr++Y11hB7rT1QDpn/afo=; b=MH5xcseSRT0N9WZzXiPsIlJzafJZHEMG8tIivW/OY1jr80o6UmWgwnrT/YXNSfLSIGNBwM VY7ngHJM9ycfu7PmQVm982mSVWDBkuoLEgjSaKPgs6dtS2s7w4IMbxFSdmVl2RV9Q3WBAv NjKBa9oqRcRCxl48UaUP9qK5T2tHYkQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Nlk7HVaS; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111486; a=rsa-sha256; cv=none; b=ryc6IwyHrXF6EBAEAguyfQ90YU3dOdP9n5LpOgi7iZl2P2HNFq+VrA9bQAFY07m8mph2Sc 66beyzQ37vfM6SkhKu9GYNtnoW5k69JmkWifqmfq8MFGV+XiQIjstm0Q1HvpUNL+/92wPM 4/8QD15p0H62kmiG0wopbuvTmuvp36Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111485; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPGukvQXP4Tai7PUqcbnIswr++Y11hB7rT1QDpn/afo=; b=Nlk7HVaS7Ojsw59gG+Indrde5s5LkTPXDUgDBSXV+KeuDwDEfDnrDpYbjjMETU/8Jl7znT oylTY9Kj3m8X8yKH3mGHL6jlpZkSei8ad9xBNuSzCXdDJVdGPCf78mJV7Oau2s7FWRdTrM SVzzwuSr0yKnyWEjEJa1RtS+zpBmjks= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-29-Ivy8e833NRWSYrtoVTgS-g-1; Fri, 26 May 2023 10:31:21 -0400 X-MC-Unique: Ivy8e833NRWSYrtoVTgS-g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8A6DD811E92; Fri, 26 May 2023 14:31:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4D0C140E961; Fri, 26 May 2023 14:31:17 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 3/8] crypto: af_alg: Pin pages rather than ref'ing if appropriate Date: Fri, 26 May 2023 15:30:59 +0100 Message-Id: <20230526143104.882842-4-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: AB6191C0018 X-Stat-Signature: 3ro6dduzgi16w4515wmfibt53bo3ambx X-Rspam-User: X-HE-Tag: 1685111486-654245 X-HE-Meta: U2FsdGVkX1/2R9XBUXtUX5wWLRVpmWq2Ld5/4cKJsN99puRkP3Gpj9RI+4Nf+SMGl7Zzi2yvz97QYUKyHJL9KghZNZNCpc06t0ZzR71L2RoJ1ONgKh2ETxhiCK1MPU7IvqwbQOBxliv0mFK9Ur8tyNxehXCxFNgtkD6WzkebckaN5kLoorA1tC3SofQD1mSnsAAAJbbR9Q2iqnXmXn2N5c/4QwuboCRJeTIkJ+DocT7vBOYb2+zIs7RMkdJitXIjKgn5BZuKeT4vvi06XawW4KiqGLKC8AEEEjM/ba/qznAW1XoFK9u/3XoG5sn7X/RZ7jh1O7vkZA/lRRgbgoBj5ZULFfoNa1ad9GoqUsobwDdN/hQDz5R8hKVcMWRqN5585UZmamMO5KL5Ug/apLMVMuKURaEtHoVgWH0Q1XfpfqjA93+ufzfuVaJfuLQnABxFa3MmK5m50OeTDF8DEWC46QHNJHSqFIf5oOUnJsqKKEBbkYaGfqS8Y2s3+j7GBhUGyV3B9ro1mdMUIatCdyDW80e7F0NpNfjm40zw5HV/N2oNz+MQxvzN+I+bFbhc9ruim7v2faMqj1RVd9G19lC3bkXRYFeaq1+pF3JaXubetTxpQEH3DU2YOQiP/lxzWWTh4ZJYs4HYS+avg1cdLG9ek6grtdeHgKGoXyZF8X5TukWTw09RrRzWsx5Zl/uloSc4d7BQwS3eYnny75+XEU4K4/H5y76uPcIRBEF3juib6Q53CR6MqYrmoXsgOgaCXGK+4jV440hMLlWhh63Ww9OzxXoxPNL2ivS6bLBHFXH1m9mCUH6MAkjIWlIZHxovbYE3J8to6Asvs+OTm30E7wYYNy2y14bji6XP2C4pIMa+fYPc5aJOCRjDF5Ky4uvtYCXGhGQn8Q/A3h0GD3Msgcnj4bTTFv1XDUc3h/TcbDn2IcZQT4zfiIgBecoQ4osu2wivzQqVL1wNtGMNTKDe7dJ jFUZTp/w on/OU9QAlccqFxDZsduPb9KGMipNHO6DBi/fmyfMgO+HBO+YQWZ8+6r46F8dWAFRnrZABTHezyKE3q2hi5C9ozU9yLQzLWcjjEcnTi/6AubuDut4HBIDDo54n1Nyx27UiyTwdnoCeNp8nH55Y83MjpD2WDVv3Nj5oniWCKxNbv8/LT/RJNwOlYwaCp13m3tk81C7dQylWc045QB8P0COlsFpBmbFlofyPs5JATQU1gHD8Q/IHivUpqayFax+CYA+t7YYdHP9OvWsB6eresW9YCctsNrFVQEylx+YEYq7+hcW/fxbpXI88CYjY/S7dMr1NuKmmnba5IJHJdVthZ+qA0MDd/7xGV2K1+HdVNr8dpmcYBxUj7aGl+zaGE0Gc0GxggvMMVzZHMwyw1yF8UswZupw1jpOnCgYRQGEAYR0tdkv9cM7Uc44ssIVL+TSE7j00YVmNpUpA5PbOwrCMBg7ei0EmmrBlIVojSBLk7dmDUzrALP2RIJb+W5qwf0CHwPMrh3/KOqqSNdcORrEMKvNazACKZ13bZsXKwCkqRq5iVxUoYQc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert AF_ALG to use iov_iter_extract_pages() instead of iov_iter_get_pages(). This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO-read rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O would otherwise end up only visible to the child process and not the parent). Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 10 +++++++--- include/crypto/if_alg.h | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 5f7252a5b7b4..7caff10df643 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -533,14 +533,17 @@ static const struct net_proto_family alg_family = { int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len) { + struct page **pages = sgl->pages; size_t off; ssize_t n; int npages, i; - n = iov_iter_get_pages2(iter, sgl->pages, len, ALG_MAX_PAGES, &off); + n = iov_iter_extract_pages(iter, &pages, len, ALG_MAX_PAGES, 0, &off); if (n < 0) return n; + sgl->need_unpin = iov_iter_extract_will_pin(iter); + npages = DIV_ROUND_UP(off + n, PAGE_SIZE); if (WARN_ON(npages == 0)) return -EINVAL; @@ -573,8 +576,9 @@ void af_alg_free_sg(struct af_alg_sgl *sgl) { int i; - for (i = 0; i < sgl->npages; i++) - put_page(sgl->pages[i]); + if (sgl->need_unpin) + for (i = 0; i < sgl->npages; i++) + unpin_user_page(sgl->pages[i]); } EXPORT_SYMBOL_GPL(af_alg_free_sg); diff --git a/include/crypto/if_alg.h b/include/crypto/if_alg.h index 7e76623f9ec3..46494b33f5bc 100644 --- a/include/crypto/if_alg.h +++ b/include/crypto/if_alg.h @@ -59,6 +59,7 @@ struct af_alg_sgl { struct scatterlist sg[ALG_MAX_PAGES + 1]; struct page *pages[ALG_MAX_PAGES]; unsigned int npages; + bool need_unpin; }; /* TX SGL entry */ From patchwork Fri May 26 14:31:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1131AC7EE23 for ; Fri, 26 May 2023 14:31:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A07416B007D; Fri, 26 May 2023 10:31:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9909B900002; Fri, 26 May 2023 10:31:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8583A6B0080; Fri, 26 May 2023 10:31:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 75A086B007D for ; Fri, 26 May 2023 10:31:38 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 350E1120F5E for ; Fri, 26 May 2023 14:31:38 +0000 (UTC) X-FDA: 80832644676.04.C8C84C5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 0DD1A140042 for ; Fri, 26 May 2023 14:31:27 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XNCsjkUA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r2T5n0FpgtBULW8Nd93uDmoxOfiFTT4h2g2uaR7sC0M=; b=HC6XTESRpmzXGct/IXFcu7rhENimLkF2/e+n3+4czp588Besri2F/70gqSNvxbVahC0tCi 1+yHoe5m+WEqZAZDiG922Vaiab8N5HrqBgdOTEWv5u5zpNwI4skbvMyoHJHgifhndNOZcS ULzs11pZpeFyHVJacHc2Asn1WG+FmbU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XNCsjkUA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111488; a=rsa-sha256; cv=none; b=er+6vXO60vSfxkUaQuWgIYL4WCLF5DLRfonfBuNLXRhE7MnzPD5bDR8/NBV2I30y/9aewp qsXdgWpaCNrx5ycha4uPQcAwieiCWD9ibOjKxWDnCDdmQHgfA9iYGvN/gGmt7kd2UVABrW Np2rKmTnoGXA21pOa74e8HSGKJMGtkA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r2T5n0FpgtBULW8Nd93uDmoxOfiFTT4h2g2uaR7sC0M=; b=XNCsjkUA4ubLpTapHgOpuai5oP9IZyvBPhsHvRjFu8aOKZVx75ui4l/A3IvZf3IYrWSQeV 4Z1F1yNiAFdkB1oysLPkUBBF1Oo8WvxkRXITpe7Umu2DHp+BejNJU31d9/FM9ML8WrgXjm 8aNxLVSxhl0CfVcZ4crQ/XQX7NcUDBk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-Q_IdaTbKOKuzxyYKR0sFFA-1; Fri, 26 May 2023 10:31:23 -0400 X-MC-Unique: Q_IdaTbKOKuzxyYKR0sFFA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 669123855564; Fri, 26 May 2023 14:31:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 666C948205E; Fri, 26 May 2023 14:31:20 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 4/8] crypto: af_alg: Use extract_iter_to_sg() to create scatterlists Date: Fri, 26 May 2023 15:31:00 +0100 Message-Id: <20230526143104.882842-5-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0DD1A140042 X-Stat-Signature: qshew7cxqhzqzig5c1tnxcgxcf58eb3n X-Rspam-User: X-HE-Tag: 1685111487-222701 X-HE-Meta: U2FsdGVkX18rEg0MbXbOB4c18GvnAUrDcHgi08aGN9SvV44ouGySTO/DZlWRDgGbvIXTLETCmK5b6jJyKLdYfTjVDVRsOEUeblfE00JbqlXkQIeD7L8SRszAucFysN7UYilMIT5kqIS8XVkdCZgOZKmN0XlBFs2Ya1QpHx3vtuMihqNgjmtayTHifCC+cJ28kWsffsn96D+dpOUT1Veh6z5E1nfZkeUtA36B+9p4/CL/B48Ym6wY6THkvIcyvXb0+3vxtb05cdY2y5RFRBiLAMe+pzGEr7SQrTvc2Ym8FszBSKzK2UVlsru1W3FavG9qEtSddJ81lfpoct+Uvfaw4Hv9+pgQLH1BgHSb2+hlVe9di6apYBs5/l/mgaFVIuUKp4NYZcTrLF3RNulfI7Jt5wGrLzR7APd6cLDLKMTj/iMhzrCRBqfUxF8t+i8V9AqrC6rOIcDEPdgJF3oqK4zQRC+Rvgz7vrox8faj7R0P/A0PSPlrcIGD1F1chEvC/uz2uBhI1bzoXEv9uD7Fx52z8u5D2w/AzFUD/mGtujpVl1bjfaNAKFMa72RNlrx2ntK8zI6MtcIYjFiNZWchSQ43wk/BqVBIM4c9CfEsnBg9Z13feOsXt32/D8ocgo1rx+9/IdlwgWsiEQ7t9wt0b+emnx6z/NHQ1FcpXEECuTAZZniReYXenoWHMcH4nK526Ued1G5ykljf37v1QM/3uhSrTCvJ+mWT04ZzuWrFPvpr5OqPGYkHmeNNzvYHZOAV3xRPR8KY8FgvFq3+/fT1k/6ZaNpXxzlNDbcSLT8MO5EQbef+e9LJQh5N861Vb+TJtg+779LT5//ye6zUYDJl4oCaKkBBhCRFbO7WaouYKK+BVQvqQ5FNHQD8yVzq/0cIHk3Efc5XiS/FzvSy2L+TR3y708KfU+s8OsN2F6miJQpgMXbuDF6ECWvqckWfXJ45Deb6qZP0FmzfLQW9bGxsZQs PkClD1DQ ucyCGZDtp1KamVDws5IoVjHYOvf515ZfAagBJ7ZejMXJpNNpBH7Y/lfQvtNN3i0pS6gUDJeLvO0NtvxPs9c/4gbyz0vJaF56Mj3OGAEtvwQoCLGivcTc4zemePUJdWUqfs0lva/PB3gp792mCP3RDDDF+D/z2vwO07glGLs5vQQ95dGlbBO+cgmwS2dMDnkupKl3Owzf8XD0XzMa26r93a6fX9oi35v4cDH3qvGT/mlDfBdwgUfjd37Sz9lSyvjn9eKzgGL4aUWSSGvcBZyO/UXo00I3PD44tU0TiMOTFl25bHrvAw1UfahjPtrD3yhq+YiQEM9zXjWjQCu2raYPlJI1DB+5WxgOmnYVpEf0oTYhr3EqU7tRfoP8sX9Mn2w4nww3IhzE5JBmLp950iwpz+ZSqSYjRw1DAC503BBmFoK13wJOpeGgpEghVkCUF+U/yeu01fZnPbLL/Hbvv202HodmNnyiYxSaJ7RJAQSxqcfzsfVeo1STQLWLGkmH0oiaTMqnfN268YYVoezDQrxroVjHTgqg6re/wD85rklhwAPm/DeE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use extract_iter_to_sg() to decant the destination iterator into a scatterlist in af_alg_get_rsgl(). af_alg_make_sg() can then be removed. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 56 +++++++++++------------------------------ crypto/algif_aead.c | 12 ++++----- crypto/algif_hash.c | 18 +++++++++---- crypto/algif_skcipher.c | 2 +- include/crypto/if_alg.h | 6 ++--- 5 files changed, 36 insertions(+), 58 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 7caff10df643..8a35f1364ac3 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -531,45 +531,11 @@ static const struct net_proto_family alg_family = { .owner = THIS_MODULE, }; -int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len) -{ - struct page **pages = sgl->pages; - size_t off; - ssize_t n; - int npages, i; - - n = iov_iter_extract_pages(iter, &pages, len, ALG_MAX_PAGES, 0, &off); - if (n < 0) - return n; - - sgl->need_unpin = iov_iter_extract_will_pin(iter); - - npages = DIV_ROUND_UP(off + n, PAGE_SIZE); - if (WARN_ON(npages == 0)) - return -EINVAL; - /* Add one extra for linking */ - sg_init_table(sgl->sg, npages + 1); - - for (i = 0, len = n; i < npages; i++) { - int plen = min_t(int, len, PAGE_SIZE - off); - - sg_set_page(sgl->sg + i, sgl->pages[i], plen, off); - - off = 0; - len -= plen; - } - sg_mark_end(sgl->sg + npages - 1); - sgl->npages = npages; - - return n; -} -EXPORT_SYMBOL_GPL(af_alg_make_sg); - static void af_alg_link_sg(struct af_alg_sgl *sgl_prev, struct af_alg_sgl *sgl_new) { - sg_unmark_end(sgl_prev->sg + sgl_prev->npages - 1); - sg_chain(sgl_prev->sg, sgl_prev->npages + 1, sgl_new->sg); + sg_unmark_end(sgl_prev->sgt.sgl + sgl_prev->sgt.nents - 1); + sg_chain(sgl_prev->sgt.sgl, sgl_prev->sgt.nents + 1, sgl_new->sgt.sgl); } void af_alg_free_sg(struct af_alg_sgl *sgl) @@ -577,8 +543,8 @@ void af_alg_free_sg(struct af_alg_sgl *sgl) int i; if (sgl->need_unpin) - for (i = 0; i < sgl->npages; i++) - unpin_user_page(sgl->pages[i]); + for (i = 0; i < sgl->sgt.nents; i++) + unpin_user_page(sg_page(&sgl->sgt.sgl[i])); } EXPORT_SYMBOL_GPL(af_alg_free_sg); @@ -1292,8 +1258,8 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr *msg, int flags, while (maxsize > len && msg_data_left(msg)) { struct af_alg_rsgl *rsgl; + ssize_t err; size_t seglen; - int err; /* limit the amount of readable buffers */ if (!af_alg_readable(sk)) @@ -1310,16 +1276,22 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr *msg, int flags, return -ENOMEM; } - rsgl->sgl.npages = 0; + rsgl->sgl.sgt.sgl = rsgl->sgl.sgl; + rsgl->sgl.sgt.nents = 0; + rsgl->sgl.sgt.orig_nents = 0; list_add_tail(&rsgl->list, &areq->rsgl_list); - /* make one iovec available as scatterlist */ - err = af_alg_make_sg(&rsgl->sgl, &msg->msg_iter, seglen); + sg_init_table(rsgl->sgl.sgt.sgl, ALG_MAX_PAGES); + err = extract_iter_to_sg(&msg->msg_iter, seglen, &rsgl->sgl.sgt, + ALG_MAX_PAGES, 0); if (err < 0) { rsgl->sg_num_bytes = 0; return err; } + sg_mark_end(rsgl->sgl.sgt.sgl + rsgl->sgl.sgt.nents - 1); + rsgl->sgl.need_unpin = iov_iter_extract_will_pin(&msg->msg_iter); + /* chain the new scatterlist with previous one */ if (areq->last_rsgl) af_alg_link_sg(&areq->last_rsgl->sgl, &rsgl->sgl); diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c index 42493b4d8ce4..f6aa3856d8d5 100644 --- a/crypto/algif_aead.c +++ b/crypto/algif_aead.c @@ -210,7 +210,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, */ /* Use the RX SGL as source (and destination) for crypto op. */ - rsgl_src = areq->first_rsgl.sgl.sg; + rsgl_src = areq->first_rsgl.sgl.sgt.sgl; if (ctx->enc) { /* @@ -224,7 +224,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, * RX SGL: AAD || PT || Tag */ err = crypto_aead_copy_sgl(null_tfm, tsgl_src, - areq->first_rsgl.sgl.sg, processed); + areq->first_rsgl.sgl.sgt.sgl, processed); if (err) goto free; af_alg_pull_tsgl(sk, processed, NULL, 0); @@ -242,7 +242,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* Copy AAD || CT to RX SGL buffer for in-place operation. */ err = crypto_aead_copy_sgl(null_tfm, tsgl_src, - areq->first_rsgl.sgl.sg, outlen); + areq->first_rsgl.sgl.sgt.sgl, outlen); if (err) goto free; @@ -268,8 +268,8 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* RX SGL present */ struct af_alg_sgl *sgl_prev = &areq->last_rsgl->sgl; - sg_unmark_end(sgl_prev->sg + sgl_prev->npages - 1); - sg_chain(sgl_prev->sg, sgl_prev->npages + 1, + sg_unmark_end(sgl_prev->sgt.sgl + sgl_prev->sgt.nents - 1); + sg_chain(sgl_prev->sgt.sgl, sgl_prev->sgt.nents + 1, areq->tsgl); } else /* no RX SGL present (e.g. authentication only) */ @@ -278,7 +278,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* Initialize the crypto operation */ aead_request_set_crypt(&areq->cra_u.aead_req, rsgl_src, - areq->first_rsgl.sgl.sg, used, ctx->iv); + areq->first_rsgl.sgl.sgt.sgl, used, ctx->iv); aead_request_set_ad(&areq->cra_u.aead_req, ctx->aead_assoclen); aead_request_set_tfm(&areq->cra_u.aead_req, tfm); diff --git a/crypto/algif_hash.c b/crypto/algif_hash.c index 63af72e19fa8..16c69c4b9c62 100644 --- a/crypto/algif_hash.c +++ b/crypto/algif_hash.c @@ -91,13 +91,21 @@ static int hash_sendmsg(struct socket *sock, struct msghdr *msg, if (len > limit) len = limit; - len = af_alg_make_sg(&ctx->sgl, &msg->msg_iter, len); + ctx->sgl.sgt.sgl = ctx->sgl.sgl; + ctx->sgl.sgt.nents = 0; + ctx->sgl.sgt.orig_nents = 0; + + len = extract_iter_to_sg(&msg->msg_iter, len, &ctx->sgl.sgt, + ALG_MAX_PAGES, 0); if (len < 0) { err = copied ? 0 : len; goto unlock; } + sg_mark_end(ctx->sgl.sgt.sgl + ctx->sgl.sgt.nents); + + ctx->sgl.need_unpin = iov_iter_extract_will_pin(&msg->msg_iter); - ahash_request_set_crypt(&ctx->req, ctx->sgl.sg, NULL, len); + ahash_request_set_crypt(&ctx->req, ctx->sgl.sgt.sgl, NULL, len); err = crypto_wait_req(crypto_ahash_update(&ctx->req), &ctx->wait); @@ -141,8 +149,8 @@ static ssize_t hash_sendpage(struct socket *sock, struct page *page, flags |= MSG_MORE; lock_sock(sk); - sg_init_table(ctx->sgl.sg, 1); - sg_set_page(ctx->sgl.sg, page, size, offset); + sg_init_table(ctx->sgl.sgl, 1); + sg_set_page(ctx->sgl.sgl, page, size, offset); if (!(flags & MSG_MORE)) { err = hash_alloc_result(sk, ctx); @@ -151,7 +159,7 @@ static ssize_t hash_sendpage(struct socket *sock, struct page *page, } else if (!ctx->more) hash_free_result(sk, ctx); - ahash_request_set_crypt(&ctx->req, ctx->sgl.sg, ctx->result, size); + ahash_request_set_crypt(&ctx->req, ctx->sgl.sgl, ctx->result, size); if (!(flags & MSG_MORE)) { if (ctx->more) diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c index ee8890ee8f33..a251cd6bd5b9 100644 --- a/crypto/algif_skcipher.c +++ b/crypto/algif_skcipher.c @@ -105,7 +105,7 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg, /* Initialize the crypto operation */ skcipher_request_set_tfm(&areq->cra_u.skcipher_req, tfm); skcipher_request_set_crypt(&areq->cra_u.skcipher_req, areq->tsgl, - areq->first_rsgl.sgl.sg, len, ctx->iv); + areq->first_rsgl.sgl.sgt.sgl, len, ctx->iv); if (msg->msg_iocb && !is_sync_kiocb(msg->msg_iocb)) { /* AIO operation */ diff --git a/include/crypto/if_alg.h b/include/crypto/if_alg.h index 46494b33f5bc..34224e77f5a2 100644 --- a/include/crypto/if_alg.h +++ b/include/crypto/if_alg.h @@ -56,9 +56,8 @@ struct af_alg_type { }; struct af_alg_sgl { - struct scatterlist sg[ALG_MAX_PAGES + 1]; - struct page *pages[ALG_MAX_PAGES]; - unsigned int npages; + struct sg_table sgt; + struct scatterlist sgl[ALG_MAX_PAGES + 1]; bool need_unpin; }; @@ -164,7 +163,6 @@ int af_alg_release(struct socket *sock); void af_alg_release_parent(struct sock *sk); int af_alg_accept(struct sock *sk, struct socket *newsock, bool kern); -int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len); void af_alg_free_sg(struct af_alg_sgl *sgl); static inline struct alg_sock *alg_sk(struct sock *sk) From patchwork Fri May 26 14:31:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D41BC7EE23 for ; Fri, 26 May 2023 14:31:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DCB96B0080; Fri, 26 May 2023 10:31:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28C1D6B0081; Fri, 26 May 2023 10:31:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153756B0082; Fri, 26 May 2023 10:31:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 072A86B0080 for ; Fri, 26 May 2023 10:31:47 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D0FBBA013C for ; Fri, 26 May 2023 14:31:46 +0000 (UTC) X-FDA: 80832645012.25.8E2F508 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 193332003B for ; Fri, 26 May 2023 14:31:42 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gyWfqspA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CKSB6Rx2bx1fJ9JqD5rxOprNdt4epz1O6h3wMG9pSn8=; b=ZEFOE7/2wBSpS4z0YYdBDwjikrupJgYSdwKt+0AEyA66tHNnwf8H4MErOjySQdX128xW6r x29LJenfUlc68tPSJlDvgRevBul4ABe1805HPVEtxwVrv4ODWgKJ0Bdp20EQ4Xh+9sWMOl WhMpZjRPZXOwGfrCPNt9zMbZny/F7BU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gyWfqspA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111503; a=rsa-sha256; cv=none; b=BQWrwFeNvJmVhLZAcLKpUGmKq2M1ygBpoDEVIXrm1fT5Bsm/2oe5IU9T/Bl8YyYBazTerw AHQiIozmY44VCmPvRPtMxo7YKPNDBnok3KN7V2JFtqPWxVm3SIXTep2hdYzOwM46ZVRylv LmEjA1gOcm3aOy0hfCbhW++AbCjeFXk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111502; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CKSB6Rx2bx1fJ9JqD5rxOprNdt4epz1O6h3wMG9pSn8=; b=gyWfqspAL2ukHovRqjqfYBKvjbjf4Z5NXpE7tGcOUhBhgNT12RmRqg7riY+ZYCoZB8OcPr 4YAFUHsc45EaxRWXOVLCKA7SEVVKQWYdRK7LWrnoQRcVORqwF+OEW/5it30o2P+xv7T2GD QKqvJgRNr/YVbSEYw6LSp1gDSgR6hnQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-25-jLnS6QVHOfGVhUBPQMxYFw-1; Fri, 26 May 2023 10:31:25 -0400 X-MC-Unique: jLnS6QVHOfGVhUBPQMxYFw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0AC7B3855561; Fri, 26 May 2023 14:31:25 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25EEC140E95D; Fri, 26 May 2023 14:31:23 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 5/8] crypto: af_alg: Indent the loop in af_alg_sendmsg() Date: Fri, 26 May 2023 15:31:01 +0100 Message-Id: <20230526143104.882842-6-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 193332003B X-Stat-Signature: oyxme9f8nz6g8n68sdzzrrsahc3ktwk1 X-HE-Tag: 1685111502-618092 X-HE-Meta: U2FsdGVkX1/ItfhiPDYV5pzCnSdH1ZU3AGuOxRydvayMy1k/911S+scYvRj0LqpnOoxvLkEID2NkAF4pVXBa66zpfmXfvLnOokQTkFpTNTDevRs49mYC50XDhWn8Le7N4JkeqVefP5wEdpHe2HjLFh5VJt3nYcLYVZAWCENzYHseKXpIV54TJgLhFQX8/5CnjFbTbGM71ZXSL7eu9o20Nes9YfeV/kGCLfR6tgTXwyhRMmWs4T4xr4Ny5rfc5YrZSDGnO/6ZIlaq5PTutd62uuezx9ZUfB4RuLLM/MyGgBP/w+fLwxJfAyHR/I+8uD1mWtPQMeNKMgNa5seXiSEjhvvSX3gB2hSAraDbYDcxf2EY0EqB99a1ji63kx4MdtZ+0kVNue0XfEeHbFI3fj6oZwWJS6cI3+E7JfRgR81FjXYJKcshTTBCUBYvSmot2/m02+gdM7B6rMBXzyFwUXwiHYtdZgJZH5TCAsoy9eHDo9uOWqXnVjjvKHRXSfDzoCMHvF1LlkAsoogAaP0s92uE7K2YYUa9+MpNQ0/i2hBZArSrzj2t4N6FkDV8Q0cgm9GfGOzXgiDRx/b21c928RQzTQ+SRyaJcGn0Sv42HWf/ciz5v3Qw8jPHdXaeeB/uF7yQXEOFcfid9ykNjxIxHQ66W+s3NHKY2nFg6cY5cEJrMfDt3oYasHZBBkdD600t4smAKyycd3yYO23ipTMHM6QvlCnhFrcoIhRw9V8b3F1+zzXJeDhhDx1f88P8pGtOIGSa5xZFWRGXD6vhGpkiIPkpofvgJMZlD6sGH9ZlWHcfWPttsW4QNUo1ViV0ZFNChd7U+TA/JiZ70qUFfkbu3hosyjf8ane4lSmYFr4dcPo7XwLdY0ziiCUHGEYkuucFRPBG32dtz3ofTSz2IgbJjmjRfsWH8TNz7DBW4LZ5hf3oBAVhJMo3s32MhmzYc/KVZw4xsB/uyNJEe+VwBUtmPDF 5n0HIYKW G6+fiUNHBZ/czEkZFLIiBPb5shvMG5fT7F8k1Bo9UhnLwQzohq+/ICo+bBIMkD480HBRDa0uSwM77yyqXjeRU83L88Epi+o/1FSDUwUrd8yMlakbpP+TDxE+IcTTBtBiZENy0qZ6WG7OvUvvOY9/JydbBXH6B68D4phWg6AtlAiLK7iaRL4KABK1mYBfCaTUmuxr/xmZrt/cWxS9m1OB/CMLLsoiS1iUdO5bWMtNry+zmOY3y7o523VfEA3gVu7+ZE0sZaQNvnSD1cahMpbYfpFbebaycDoJ7PQCpnNxLF/P/U8MexWNtatnnVSdMidSe7CNi8jya8pyvETe121YvoueD0KOzoabJI8jD7eE42HEAhq2qyvgV2/Y8XuI3vJ2MhWByDrwJ7gHW909UywJq8DtNNWNUetvnbU8H1b5gDkiQ2mvtI6IA/rL+uzkrIFRiwkPsKFhpVNZV1kVCe0lc01TygratiX82HRxTX59vNw1jRKhynAabIyPhcRCQkzT3DOJkNZI7x3fw3+3eLxBaub5FDJjYilQlgWjAh+GTkck0Hvw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Put the loop in af_alg_sendmsg() into an if-statement to indent it to make the next patch easier to review as that will add another branch to handle MSG_SPLICE_PAGES to the if-statement. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 50 +++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 8a35f1364ac3..17ecaae50af7 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -1030,35 +1030,37 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size, if (sgl->cur) sg_unmark_end(sg + sgl->cur - 1); - do { - struct page *pg; - unsigned int i = sgl->cur; + if (1 /* TODO check MSG_SPLICE_PAGES */) { + do { + struct page *pg; + unsigned int i = sgl->cur; - plen = min_t(size_t, len, PAGE_SIZE); + plen = min_t(size_t, len, PAGE_SIZE); - pg = alloc_page(GFP_KERNEL); - if (!pg) { - err = -ENOMEM; - goto unlock; - } + pg = alloc_page(GFP_KERNEL); + if (!pg) { + err = -ENOMEM; + goto unlock; + } - sg_assign_page(sg + i, pg); + sg_assign_page(sg + i, pg); - err = memcpy_from_msg(page_address(sg_page(sg + i)), - msg, plen); - if (err) { - __free_page(sg_page(sg + i)); - sg_assign_page(sg + i, NULL); - goto unlock; - } + err = memcpy_from_msg(page_address(sg_page(sg + i)), + msg, plen); + if (err) { + __free_page(sg_page(sg + i)); + sg_assign_page(sg + i, NULL); + goto unlock; + } - sg[i].length = plen; - len -= plen; - ctx->used += plen; - copied += plen; - size -= plen; - sgl->cur++; - } while (len && sgl->cur < MAX_SGL_ENTS); + sg[i].length = plen; + len -= plen; + ctx->used += plen; + copied += plen; + size -= plen; + sgl->cur++; + } while (len && sgl->cur < MAX_SGL_ENTS); + } if (!size) sg_mark_end(sg + sgl->cur - 1); From patchwork Fri May 26 14:31:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256962 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D61FC77B73 for ; Fri, 26 May 2023 14:31:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B9D36B007E; Fri, 26 May 2023 10:31:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 969F66B0080; Fri, 26 May 2023 10:31:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 831546B0081; Fri, 26 May 2023 10:31:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 740EC6B007E for ; Fri, 26 May 2023 10:31:46 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B771A1A08B4 for ; Fri, 26 May 2023 14:31:45 +0000 (UTC) X-FDA: 80832644970.16.5B6B108 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf08.hostedemail.com (Postfix) with ESMTP id AB24816000E for ; Fri, 26 May 2023 14:31:30 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="MJRGBD/c"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j6NjOw4LTe31mdhiQO2H52w9dxcv2LHBSUfpmtyBwDw=; b=wqu/4vUlwkWUrtJiiL1J6/365oCAuzjn6c6xjws+IBMTO8XFPeMJCdrqadH9LDEAdKKWat r6vcDE7cI/+Xe6lkugt4lxF3u4jsXUwQnLlGLKukQy6q8M+KAafeSiJluMuDme1VDXK6y0 LOyRtERtuT/Pf+U08REpiYj/jhQbRyU= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="MJRGBD/c"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111490; a=rsa-sha256; cv=none; b=wHtjQeX3SLD9w7e3qhVgv780rWuTyNcMxMhdtOYFwtcOAyXlqfMK2zdOS+qC39E8jl6tIM u7ae4ZSyIcq4R4ul1KjMa6rFm1L3tjqgR8qbLKFDclFFuTGYlt0gsJsBUlRekksM2cMDRe PTp/7tAEwoMsItmhnqA8o8eac0ppi2I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j6NjOw4LTe31mdhiQO2H52w9dxcv2LHBSUfpmtyBwDw=; b=MJRGBD/cGwJ+/NDqy7sMAmqXJ5lLIWfZPtepp9fY5JrVmgZLfNtfATtThif7eOMHvqunO9 G/ZInI/APH69c9Pq6bqWt6saO8zZc/82CSAz6MHK863lu6z2hbEIiSAMAzAp7zanYCHdEZ Os5k4n5U8DVl6XcFlxWhj2T828MOARk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-540-giAiFR8VOP2yBdrrkgccjg-1; Fri, 26 May 2023 10:31:28 -0400 X-MC-Unique: giAiFR8VOP2yBdrrkgccjg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B9D41800B2A; Fri, 26 May 2023 14:31:27 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id D3B91407DEC3; Fri, 26 May 2023 14:31:25 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 6/8] crypto: af_alg: Support MSG_SPLICE_PAGES Date: Fri, 26 May 2023 15:31:02 +0100 Message-Id: <20230526143104.882842-7-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Queue-Id: AB24816000E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: a38piufuepmgtp5nwrzox8185porgpdk X-HE-Tag: 1685111490-755504 X-HE-Meta: U2FsdGVkX1/VSSMvKs3S2Ly+0xe3jO32aU0mu8vtrJXAKwJH4w30HHWS413yev1BI5jhJSUWuLetGdFet7XED9NfRUkgFGvJ0dZxPnsybaaOFXPmFcxQzCj61fN9s0E+o/mGFcmgfHuZm+ez1AgWAKioicbOZrWNjQXcBIcxaSo1UB4OZz/7MCKpAU7aJErVISHD1C38P829h5uhdJVyfanZEt2PR59u7saVXHuRrd28e2xVQBrWjDefe/9etvrRqLsnZXAoAZUpDAAOYXbA7jLAJbqlKbhYnVx6vFVriElEOreFBeMgbgydsFZE8jC0IeVaktwkPccogKvIreJ4k5/qP0WxNQKE8JJmIveqesi2u7g2X9D8hGL9T+eON5/tUBILylJW5QeraL1ZAlyxojwhVAg/mRsYyPXGRIOiNpWEexEZTX58QUYs3uKDwewEFqPZN2lW4D82OEmumHgGPPyDlhKQz6qWoaM7gYVlE7jsxjJXQPei4zm75TZTOkiom4bogQssG5+g0steCrLriSTA5EnhV6etLzP9Qyx1TNXCb7A1dBisvslKXwqvYMclrnwft6e7u6+3QyR9oqZH8d0TwU1nGi9c9x7/VxyBNUvzeaK+GyU6eKncOKZJURJdxYVPamyEGYmNgTa0b01MzNzKaQLsDJv2bknaWnkhwbthmyWCW8M360cE8dlnZvbGEcNC7cgQYg0ROxn87zGlicYRp2zXOUAfxWMtn3uFMQL4V16cJdwRKKHVJP95AOiBSbiW/t8v/ne/R8n7Zu1EiYezXihXtFyw/5ade2RESn7mMjiCK1EdsmS8qe5Jhi6c/kbaxCd7XzVJ1++1NFife3WBBGUekRX5dcizYTfzN2jrdf84eoklQ9eBd4zWbSUOUMhRXIco35UxSNbK8MMe8viXfx1nygJGao8jO7TGwybSMrx7etHOlgw4uR0QejLHHazZgR/0leVH/DeUUE0 hEcKKvbc MnZV6mbICHbXZJu/e+/18ej8LJhC7MFonKjNphZIKNDUUpjVEc0O7llsQw4zzy4ahf+wgAxZ3+ubiCPdzJ5iSBvmaDBj1ERnvC+IAIuXxFKn9qMd0V4H3mwMky4++jA7wmH3mtcrXjbrqFll56yI0ZVlSX1J+0a1KwoCy6lqL216ZZSGSfHKOkXloWYqRrsmzTkS5u5hmgaX0AdVzW/tnJza0l/U1hpBvDR6foE8f2tGHrknt9Oq49jRt7068/wm3mqXe9m+hSsd497AxmjdGUBmQUfwr9w1ZOFiteDbI3Iqldu/27rQU4GwfmhumRSerwXLI25zz3refwNiQ0Wx2ZsfvGhDgtHXUchnec4wnvFSoqMjyUvS+FFkn/LEeDvVjSoDxewHuE/FT3s4CAvLp+jiOrVlg7Bq5cYCLQWtlsJdgbLJk43ZqbBQPLz+CDV/yG8ZY5qFIaQGyuk8Skz356omZzLypTslip05XVQ/zomPDC4FpfCQhIEQk/GttgXOqDbOLFotldmi2IxqaOwauchji5ejqQt/EB9qKCf1QPwEbhyQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make AF_ALG sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 28 ++++++++++++++++++++++++++-- crypto/algif_aead.c | 22 +++++++++++----------- crypto/algif_skcipher.c | 8 ++++---- 3 files changed, 41 insertions(+), 17 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 17ecaae50af7..979033600185 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -940,6 +940,10 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size, bool init = false; int err = 0; + if ((msg->msg_flags & MSG_SPLICE_PAGES) && + !iov_iter_is_bvec(&msg->msg_iter)) + return -EINVAL; + if (msg->msg_controllen) { err = af_alg_cmsg_send(msg, &con); if (err) @@ -985,7 +989,7 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size, while (size) { struct scatterlist *sg; size_t len = size; - size_t plen; + ssize_t plen; /* use the existing memory in an allocated page */ if (ctx->merge) { @@ -1030,7 +1034,27 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size, if (sgl->cur) sg_unmark_end(sg + sgl->cur - 1); - if (1 /* TODO check MSG_SPLICE_PAGES */) { + if (msg->msg_flags & MSG_SPLICE_PAGES) { + struct sg_table sgtable = { + .sgl = sg, + .nents = sgl->cur, + .orig_nents = sgl->cur, + }; + + plen = extract_iter_to_sg(&msg->msg_iter, len, &sgtable, + MAX_SGL_ENTS, 0); + if (plen < 0) { + err = plen; + goto unlock; + } + + for (; sgl->cur < sgtable.nents; sgl->cur++) + get_page(sg_page(&sg[sgl->cur])); + len -= plen; + ctx->used += plen; + copied += plen; + size -= plen; + } else { do { struct page *pg; unsigned int i = sgl->cur; diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c index f6aa3856d8d5..b16111a3025a 100644 --- a/crypto/algif_aead.c +++ b/crypto/algif_aead.c @@ -9,8 +9,8 @@ * The following concept of the memory management is used: * * The kernel maintains two SGLs, the TX SGL and the RX SGL. The TX SGL is - * filled by user space with the data submitted via sendpage/sendmsg. Filling - * up the TX SGL does not cause a crypto operation -- the data will only be + * filled by user space with the data submitted via sendpage. Filling up + * the TX SGL does not cause a crypto operation -- the data will only be * tracked by the kernel. Upon receipt of one recvmsg call, the caller must * provide a buffer which is tracked with the RX SGL. * @@ -113,19 +113,19 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, } /* - * Data length provided by caller via sendmsg/sendpage that has not - * yet been processed. + * Data length provided by caller via sendmsg that has not yet been + * processed. */ used = ctx->used; /* - * Make sure sufficient data is present -- note, the same check is - * also present in sendmsg/sendpage. The checks in sendpage/sendmsg - * shall provide an information to the data sender that something is - * wrong, but they are irrelevant to maintain the kernel integrity. - * We need this check here too in case user space decides to not honor - * the error message in sendmsg/sendpage and still call recvmsg. This - * check here protects the kernel integrity. + * Make sure sufficient data is present -- note, the same check is also + * present in sendmsg. The checks in sendmsg shall provide an + * information to the data sender that something is wrong, but they are + * irrelevant to maintain the kernel integrity. We need this check + * here too in case user space decides to not honor the error message + * in sendmsg and still call recvmsg. This check here protects the + * kernel integrity. */ if (!aead_sufficient_data(sk)) return -EINVAL; diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c index a251cd6bd5b9..b1f321b9f846 100644 --- a/crypto/algif_skcipher.c +++ b/crypto/algif_skcipher.c @@ -9,10 +9,10 @@ * The following concept of the memory management is used: * * The kernel maintains two SGLs, the TX SGL and the RX SGL. The TX SGL is - * filled by user space with the data submitted via sendpage/sendmsg. Filling - * up the TX SGL does not cause a crypto operation -- the data will only be - * tracked by the kernel. Upon receipt of one recvmsg call, the caller must - * provide a buffer which is tracked with the RX SGL. + * filled by user space with the data submitted via sendmsg. Filling up the TX + * SGL does not cause a crypto operation -- the data will only be tracked by + * the kernel. Upon receipt of one recvmsg call, the caller must provide a + * buffer which is tracked with the RX SGL. * * During the processing of the recvmsg operation, the cipher request is * allocated and prepared. As part of the recvmsg operation, the processed From patchwork Fri May 26 14:31:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C28E5C7EE23 for ; Fri, 26 May 2023 14:31:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43C086B0081; Fri, 26 May 2023 10:31:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 412556B0082; Fri, 26 May 2023 10:31:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DB41900002; Fri, 26 May 2023 10:31:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1F12D6B0081 for ; Fri, 26 May 2023 10:31:49 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 01F78AEC32 for ; Fri, 26 May 2023 14:31:48 +0000 (UTC) X-FDA: 80832645138.29.C30A80F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 3E88540034 for ; Fri, 26 May 2023 14:31:34 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bIbshqlD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dc/SuOpUlYqAq5ekNpSzLieNRD28aCzVtUAi8o3Unyg=; b=FSyOB5XnhAPSxh/VovgAZBZ2dxpE8KEdbUx0p2LpaQW+V8JaTd/SwqB0JxhVmcok/FTJGa k0BzNNJELCvmO2pgBjX4NO4Iz6hr2IjJoB2hqwgFy076CD2LyQy3XCeU+GkoplHh0gMD3b 1ZvSmKCoounQuUGvKGznH82ASp70vfk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bIbshqlD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111495; a=rsa-sha256; cv=none; b=vKBfYIL2oaO3OkJ6JGmqFLgTit03wSF7d+2w1acEzkzTFn31JoywbRDFETXflxOdegvvt0 omR/yOePOWKRTUUTGSZyrKXOBAJt1kXrjNGCQu1otcHeuyTuDJKfgt+T/+RZuN6mebC+TJ Uw7vMnAEScIYmhwpgTz+TbqRkha3gWk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111494; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dc/SuOpUlYqAq5ekNpSzLieNRD28aCzVtUAi8o3Unyg=; b=bIbshqlD48gMmLdHWEfV/vk04aQSk5ZvM+2I4p3lmJ317Af+eHQgXqCU9/IYIxSAhGzt4s EboLzTsuyfpVAym1OmrybRiWTirGrGp9Va/RT/nOCHLi9Etqxkl7NYakvlp8ZzswKh4/Wl zMb8IT77pDxAgZhq7yJti+9Qa2gg/L4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-570-ERCJ1r-zM0q7W6bBwBEpZg-1; Fri, 26 May 2023 10:31:31 -0400 X-MC-Unique: ERCJ1r-zM0q7W6bBwBEpZg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 576643855566; Fri, 26 May 2023 14:31:30 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 756352166B2B; Fri, 26 May 2023 14:31:28 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 7/8] crypto: af_alg: Convert af_alg_sendpage() to use MSG_SPLICE_PAGES Date: Fri, 26 May 2023 15:31:03 +0100 Message-Id: <20230526143104.882842-8-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3E88540034 X-Stat-Signature: bphumatmqpqqcf6rz797cm7akghharka X-Rspam-User: X-HE-Tag: 1685111494-567523 X-HE-Meta: U2FsdGVkX19m++RFkHpsWZ1DXzUIATOBQ/9S7sivRw9GkfB+MwnkmLPCpIuUoS3WzINDljd+SfBuXb6AxFBzJdF+O9yRNlvhlHClaOHoH+i/Lbm9rqAVjFXpQAsVLyqI1dHI59sprXNNA4OCulHOCsg+iw+lHpmunyKczwvizRHUyWBwDooiZ5+p+zvFb208jUjCVneU+OGNT6E+QhFd60jevB0h/S7H52pSq+c8SyMoloo8Pg5s39iSOaSwhYYDQm2C+uWTbTUg33Sp8Lxjek5q076IubxPHvBNj4DIz4w8wYPlmXCQMh1nLJE5OQWCxawWBQhg9ZM9M265LYAdmgDLpuqGOd//A3sicggQFW9pECOLgdG9ycb1bDsu+EL3A3H8GtaX3Zn0ZmUWggeb2mIvVrRRO00TaY/qxlzuFIY/COiOIaENhHhPsW9Ma1E5qNOq7T9414rj6zvzYdY/hmYpu9wqJ7AUK28VagzKcEVlz+vNSLJ3bWk0XFV5GzjCQ8B05094vt8LrkJLfUsya8KR7bdDRK9115avhoap/IbZhv9Xd3PrXqCi7ZeOvaNZc1cEvndQPfXWmZKbzVQXssinm5uHAwGLHfUpT2ZziwP7yNFKXSCnoWM24LJNo8+tMasN+mXYf3qPwQxSmKjA6ozs25EzfnDb4jmPjeIWzBHyB2Kt+0G9tcS3TgcSjorocEeEwtILJ+k47ga+HKeDhx8bzddCBKTrOq5IinL+LzDktXuWvYILRkopirQbECBvwDUfP5JEbGGNYtSkRhhJJ4ibh4dqOxxcXquHVw8xfKDNdT+2V5BTIpOHSVbNfQhrnSO8Ed0qY9xM2y2UlWsLDL8n/Us3axVXNjrwE4q1QbiU2lq8bNaAzHPwC9uus9GUyF0G/hS1nsa5ub4kdpLDAPAXbxjhqCKGq93tcY6W+TiMpF9hsX15jsg6Tq4CLjgIdYTEFsmlPTurkUwFAT5 yjrKo5K1 FpwTcFwsY5marwbLBLQDyz9TDfKKM/z9bx5PZ8SgQ6dOOPiYWCuhpquO7wE8XawFHrX8seocHWQ6FhHXse+ylhzPclKnyHCfP1CaO2+bzBvaPhRjT+tB5ZHe7dV+U6ur5OFccYMt2qggrt2E6oqd6v7E42/bnF6Y+6t754v0ThwUARTHglZqzupo2/43be0gfJwIj3P79vvKznh2nLCYnYEjQrS3fcXGj0ZvjWgMHWxLX4KQiqpcYEGlkrBoCqOX8uvQMFxR1L6j083JxwOQ/nasdcYGUsVv1EXapPastUsf4OBRk7sjU/smychDcjbHmWujKCILTfSL4xGdTEiJA+EYAXyB/CZQV/SK48VSw+aD+LjG9QQTZNYmkSY/BdOS6Y4AOSJt504ViCeuKigXbD5gxdhnNlNZiFun3Xi6RfJzWKQvzX6VYskwjN3RKXzPUhWq/t6cvaHzGTVIsbS8pRXV4wYo5PGNHcdMBEXshCd/A7hhEJg9GkssZ0BOQ+hW49uYfLLFMPdsPIBoFrk/PIAFNsdxQt272rLB+xZwvpxvCdX8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert af_alg_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 52 ++++++++----------------------------------------- 1 file changed, 8 insertions(+), 44 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 979033600185..105afd77a064 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -1117,53 +1117,17 @@ EXPORT_SYMBOL_GPL(af_alg_sendmsg); ssize_t af_alg_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags) { - struct sock *sk = sock->sk; - struct alg_sock *ask = alg_sk(sk); - struct af_alg_ctx *ctx = ask->private; - struct af_alg_tsgl *sgl; - int err = -EINVAL; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = flags | MSG_SPLICE_PAGES, + }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - lock_sock(sk); - if (!ctx->more && ctx->used) - goto unlock; - - if (!size) - goto done; - - if (!af_alg_writable(sk)) { - err = af_alg_wait_for_wmem(sk, flags); - if (err) - goto unlock; - } - - err = af_alg_alloc_tsgl(sk); - if (err) - goto unlock; - - ctx->merge = 0; - sgl = list_entry(ctx->tsgl_list.prev, struct af_alg_tsgl, list); - - if (sgl->cur) - sg_unmark_end(sgl->sg + sgl->cur - 1); - - sg_mark_end(sgl->sg + sgl->cur); - - get_page(page); - sg_set_page(sgl->sg + sgl->cur, page, size, offset); - sgl->cur++; - ctx->used += size; - -done: - ctx->more = flags & MSG_MORE; - -unlock: - af_alg_data_wakeup(sk); - release_sock(sk); + msg.msg_flags |= MSG_MORE; - return err ?: size; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return sock_sendmsg(sock, &msg); } EXPORT_SYMBOL_GPL(af_alg_sendpage); From patchwork Fri May 26 14:31:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 005ABC7EE23 for ; Fri, 26 May 2023 14:31:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 945596B0083; Fri, 26 May 2023 10:31:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F4686B0085; Fri, 26 May 2023 10:31:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79569900002; Fri, 26 May 2023 10:31:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6C8D66B0083 for ; Fri, 26 May 2023 10:31:55 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3C061A01A6 for ; Fri, 26 May 2023 14:31:55 +0000 (UTC) X-FDA: 80832645390.08.876819B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 0F24E14002D for ; Fri, 26 May 2023 14:31:35 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AHGIlRME; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111496; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M2+JeX97sTAuGBdDq4pkhVGY1pvhFtHUoAAG71out0Q=; b=72DGpWCO5FHyHoNOiaSJpv/L3pBQlX6NGPE10J4YUsFXDu9mIS8ugKKckwdGtUyjdrcCew UTj2/s+sAh1ZHzlfHjZQjgcpPxlTfh9hcANQs+Av6YYlh67dwqPEZnyzXQ3yCrrD7/8CbJ 0EdJrwRAGX4hlZXS8dVQuEa2FvUwDYk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AHGIlRME; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111496; a=rsa-sha256; cv=none; b=4s+o7XI39K8FUK5e0aJNF/Lxd3odYuLEC+QHuCtlaVm44gnX9YCI0iJee87QoOjSukllwV VcLqHAan8iKX0uDACEkJpz/jbJmENcw+E79yOVIJQ94WTtPyjsG1qgrCjk8JbZBMOSbRhs E+rwaYMnPSILngaU4Beuuu6oSKnt9z0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111495; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M2+JeX97sTAuGBdDq4pkhVGY1pvhFtHUoAAG71out0Q=; b=AHGIlRMEE24qgZjqwraJzmEPvVEM2STCQGyMIDJDGoxcBwcv06EQx/x63coGChY0bTfzoA M7sydpeDwhoTKBICiBaL1ZQGgg2cT2npFkhdT58zgkLIn2oyT51j8vyCehHUSiFXHpJEmq Yqq0QL7dqu1nSdJLzwN64HW6ueFjXWY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-642-xMkPZctLMmGrywq7icCpaA-1; Fri, 26 May 2023 10:31:33 -0400 X-MC-Unique: xMkPZctLMmGrywq7icCpaA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 09E968007D9; Fri, 26 May 2023 14:31:33 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 296062166B2B; Fri, 26 May 2023 14:31:31 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 8/8] crypto: af_alg/hash: Support MSG_SPLICE_PAGES Date: Fri, 26 May 2023 15:31:04 +0100 Message-Id: <20230526143104.882842-9-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0F24E14002D X-Stat-Signature: u8xjuat84rnd1g3jierkkkdgnkfz3km6 X-HE-Tag: 1685111495-111913 X-HE-Meta: U2FsdGVkX1+tpJ8oZi00/woBvG1CJTMBryAq0IIseALIW5pvPMfLRS1kjcNQ90giMyDk3v0iq+ATmpwuxfiMEHffXfpycEGTPjJHGp2ue3X5TrcDcZpPP3x3hIl1EZLzYvS9c7FIu4SsRJ4p8MrDrm0VFoqftMOLs/wKMehaaiN9neCC9SiS1R5MfWVLEb0fvJah9U8Q4zK8J1KOpnfDBB0SjtPArlH7IUi/BubUx8rFemAJ2gxZNkqSblmlUleQBHbLwSOhaU/Xim54DBA4t/Q5fCEvd3/lvCSoTf9001ld2Gw2Wtl4USxu00VkPdjNnYikRSXJgH15g9kFXK7Y1NphRmR1qe0EuiYk3dhsYoAJaOarfvFLDlgH/KK04PYFNk1BSH8SPeaod1HHL5NyHUNobkZEj4+A+WSjGe1xxuRIUcixlu2z6RtUDf5KQvl9JWm9MMkOM9jEBsGuiY0VgwIUkr1JPs/S5lfH9UoLOin1aCtQGJSW2SZlHXJKt+NCofMQy2xZ/pDo6be9XQZZ/AlYQNdcfm9NjQmue4D+2Y6818Y4w6mYn8n9fJVr9F86YTvS0IVrNdQvEGZL3aJoodkEyYv5kRSH48yNtWSwTFvyt2Zk4Q2zHZHw+evXzPrj6eYbTDRcf7qy78aAvibn1KshwfxAwxPQrHwQY+2smDlaSu4Aoyz70mtV9NwRU3cmbVYxYwFpACvCGKFTN9RfBK4wvmGXaPupMA50ghjzXJZ4j1xhp4pOE1uSRV/MUgBb0IYNgEGb/YHjPhoBvLvniKOaa8UOhN7KaHC1PLYGI7zZTiizxRnNbA6aB8/7vIgKz7gOm5t7+I3gshf2mBrldQkA0sjhO5WVow4VLsWobmC9qRPnqyhEXGKHnT4s2YjBfTL1PG921/poYo35Ik2CftWFFWqMzhPAvYnjC0RggfpVfEbns0ynuPZKiQ7f4++uJRb2I27LXbHaJ6NNIyi YDcLisam NgvfQWXSzZeppbUzB5UqS0a+fFtd1+a3GFjhALk0c9bZiBOn2eVqMGAmvEjRcXe5tV/1K+ZQCJdLl2vaJ2iC2xmtPe2XLFswvdGDfCDUO7JXU6fmOEvLKEPfu7WdHBIxf1Kv1sZaXOmpQ0xBTq5jZnZktWkoHUL/wjRZP15Je+QtyPTiyxHl++8vK/nYIRFcsPPw8Z70XcccosOQxcgidYOksvx7dt5HlsQtl8tHR9/E+i7Vun0PIdheUhIekYGG3usm5szdRzQ+TR4b39FqdSw7/Az08Cfs6slVOf9BZS5d5dzSKEHux7JrCwL5q6qMOKd2dx8TsHgqhf7RjjcA5CF09mXih60nzxKoJwn74nuIBwnmfMBmOVYJ0LPs/awwPiFTej3b6TgqtgRGQMaJSVCrBuMLEK38UzyjhcFNiwI6xcTWPIhYjfRpQwOq795zFUuUVq4idPFf8hSWPpbC5jOXb7E2LXWdw6kSe/yliA4YnjUMUX9uxQiOxGOTDnFZZZ0RAHs9pXpq9+C/z6dtJRhRIla49LOI3CgGrVYZ/rDjEHDs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make AF_ALG sendmsg() support MSG_SPLICE_PAGES in the hashing code. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 11 +++-- crypto/algif_hash.c | 100 +++++++++++++++++++++++++++----------------- 2 files changed, 70 insertions(+), 41 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 105afd77a064..1965fc4641ed 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -542,9 +542,14 @@ void af_alg_free_sg(struct af_alg_sgl *sgl) { int i; - if (sgl->need_unpin) - for (i = 0; i < sgl->sgt.nents; i++) - unpin_user_page(sg_page(&sgl->sgt.sgl[i])); + if (sgl->sgt.sgl) { + if (sgl->need_unpin) + for (i = 0; i < sgl->sgt.nents; i++) + unpin_user_page(sg_page(&sgl->sgt.sgl[i])); + if (sgl->sgt.sgl != sgl->sgl) + kvfree(sgl->sgt.sgl); + sgl->sgt.sgl = NULL; + } } EXPORT_SYMBOL_GPL(af_alg_free_sg); diff --git a/crypto/algif_hash.c b/crypto/algif_hash.c index 16c69c4b9c62..3d96c0e06ca6 100644 --- a/crypto/algif_hash.c +++ b/crypto/algif_hash.c @@ -63,78 +63,102 @@ static void hash_free_result(struct sock *sk, struct hash_ctx *ctx) static int hash_sendmsg(struct socket *sock, struct msghdr *msg, size_t ignored) { - int limit = ALG_MAX_PAGES * PAGE_SIZE; struct sock *sk = sock->sk; struct alg_sock *ask = alg_sk(sk); struct hash_ctx *ctx = ask->private; - long copied = 0; + ssize_t copied = 0; + size_t len, max_pages = ALG_MAX_PAGES, npages; + bool continuing = ctx->more, need_init = false; int err; - if (limit > sk->sk_sndbuf) - limit = sk->sk_sndbuf; + /* Don't limit to ALG_MAX_PAGES if the pages are all already pinned. */ + if (!user_backed_iter(&msg->msg_iter)) + max_pages = INT_MAX; + else + max_pages = min_t(size_t, max_pages, + DIV_ROUND_UP(sk->sk_sndbuf, PAGE_SIZE)); lock_sock(sk); - if (!ctx->more) { + if (!continuing) { if ((msg->msg_flags & MSG_MORE)) hash_free_result(sk, ctx); - - err = crypto_wait_req(crypto_ahash_init(&ctx->req), &ctx->wait); - if (err) - goto unlock; + need_init = true; } ctx->more = false; while (msg_data_left(msg)) { - int len = msg_data_left(msg); - - if (len > limit) - len = limit; - ctx->sgl.sgt.sgl = ctx->sgl.sgl; ctx->sgl.sgt.nents = 0; ctx->sgl.sgt.orig_nents = 0; - len = extract_iter_to_sg(&msg->msg_iter, len, &ctx->sgl.sgt, - ALG_MAX_PAGES, 0); - if (len < 0) { - err = copied ? 0 : len; - goto unlock; + err = -EIO; + npages = iov_iter_npages(&msg->msg_iter, max_pages); + if (npages == 0) + goto unlock_free; + + if (npages > ARRAY_SIZE(ctx->sgl.sgl)) { + err = -ENOMEM; + ctx->sgl.sgt.sgl = + kvmalloc(array_size(npages, sizeof(*ctx->sgl.sgt.sgl)), + GFP_KERNEL); + if (!ctx->sgl.sgt.sgl) + goto unlock_free; } - sg_mark_end(ctx->sgl.sgt.sgl + ctx->sgl.sgt.nents); + sg_init_table(ctx->sgl.sgl, npages); ctx->sgl.need_unpin = iov_iter_extract_will_pin(&msg->msg_iter); - ahash_request_set_crypt(&ctx->req, ctx->sgl.sgt.sgl, NULL, len); + err = extract_iter_to_sg(&msg->msg_iter, LONG_MAX, + &ctx->sgl.sgt, npages, 0); + if (err < 0) + goto unlock_free; + len = err; + sg_mark_end(ctx->sgl.sgt.sgl + ctx->sgl.sgt.nents - 1); - err = crypto_wait_req(crypto_ahash_update(&ctx->req), - &ctx->wait); - af_alg_free_sg(&ctx->sgl); - if (err) { - iov_iter_revert(&msg->msg_iter, len); - goto unlock; + if (!msg_data_left(msg)) { + err = hash_alloc_result(sk, ctx); + if (err) + goto unlock_free; } - copied += len; - } + ahash_request_set_crypt(&ctx->req, ctx->sgl.sgt.sgl, ctx->result, len); - err = 0; + if (!msg_data_left(msg) && !continuing && !(msg->msg_flags & MSG_MORE)) { + err = crypto_ahash_digest(&ctx->req); + } else { + if (need_init) { + err = crypto_wait_req(crypto_ahash_init(&ctx->req), + &ctx->wait); + if (err) + goto unlock_free; + need_init = false; + } + + if (msg_data_left(msg) || (msg->msg_flags & MSG_MORE)) + err = crypto_ahash_update(&ctx->req); + else + err = crypto_ahash_finup(&ctx->req); + continuing = true; + } - ctx->more = msg->msg_flags & MSG_MORE; - if (!ctx->more) { - err = hash_alloc_result(sk, ctx); + err = crypto_wait_req(err, &ctx->wait); if (err) - goto unlock; + goto unlock_free; - ahash_request_set_crypt(&ctx->req, NULL, ctx->result, 0); - err = crypto_wait_req(crypto_ahash_final(&ctx->req), - &ctx->wait); + copied += len; + af_alg_free_sg(&ctx->sgl); } + ctx->more = msg->msg_flags & MSG_MORE; + err = 0; unlock: release_sock(sk); + return copied ?: err; - return err ?: copied; +unlock_free: + af_alg_free_sg(&ctx->sgl); + goto unlock; } static ssize_t hash_sendpage(struct socket *sock, struct page *page,