From patchwork Fri Mar 31 16:08:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13196189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46003C76196 for ; Fri, 31 Mar 2023 16:10:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F3C86B0078; Fri, 31 Mar 2023 12:10:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A4026B009D; Fri, 31 Mar 2023 12:10:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F61B6B00A4; Fri, 31 Mar 2023 12:10:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 106726B0078 for ; Fri, 31 Mar 2023 12:10:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C88B3160487 for ; Fri, 31 Mar 2023 16:10:46 +0000 (UTC) X-FDA: 80629681692.26.A401449 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 0192A4000C for ; Fri, 31 Mar 2023 16:10:43 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fH7BfXpL; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680279044; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nNsV/6c3ho01ZHWXYC82bpiErYX0eKAApSJsSVNmIO8=; b=px1hhQoAJBEcL8zqgJpGLC/YRsqyj3Hgwg3DYTgwKj02zrAJNd41oCyOA/vmtpi5n4YOIC GRutWwc35TsUQP1FfoWjWw8O8BXn1YaoEtMW56gTwy0kXW8K7LAFkgLtvDCDwjA/YOBj2i JG3f85kyEUrMK6ZTOmGczCuiIA7YYHU= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fH7BfXpL; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680279044; a=rsa-sha256; cv=none; b=J7lJbBtXILdTHtX4SJ+JeDq6S2mAp8IVK90Z6iwgfBmSlbgj95CCa/N8XWLP8fcsl7+mQT ydyT3uDfLuBzoJjzLwksoEc/Vm31vqcCVLfoTmPzrrz3IbIfx9z9LVVJt85n8J0nmVkhqw Gn0pafNSWOlNrYmXpcqgx69fVQ/BMys= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680279043; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nNsV/6c3ho01ZHWXYC82bpiErYX0eKAApSJsSVNmIO8=; b=fH7BfXpL7FgjF6WuxfsOqJh28brPsEvbH3YQfovWT6RMA4fVU33KXXhpO4FBzu7Uj9dPnh IwIflFhhDKaZA3x3ltkfvpbI4hZDL9Mz5JK2Ln2/zycRS9+9CvYep10NWniieclolmDOSF KjDrmntJcuiNDsTCgNbG4/TXao7FyEE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-138-CQ22tLM2M2iPd2GX0NVahQ-1; Fri, 31 Mar 2023 12:10:40 -0400 X-MC-Unique: CQ22tLM2M2iPd2GX0NVahQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19982101A54F; Fri, 31 Mar 2023 16:10:39 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08B612166B33; Fri, 31 Mar 2023 16:10:36 +0000 (UTC) From: David Howells To: Matthew Wilcox , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: David Howells , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Boris Pismenny , John Fastabend Subject: [PATCH v3 28/55] tls/device: Convert tls_device_sendpage() to use MSG_SPLICE_PAGES Date: Fri, 31 Mar 2023 17:08:47 +0100 Message-Id: <20230331160914.1608208-29-dhowells@redhat.com> In-Reply-To: <20230331160914.1608208-1-dhowells@redhat.com> References: <20230331160914.1608208-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0192A4000C X-Stat-Signature: t5d3m1kfyejgb15zzzo4ntwgcu58esma X-HE-Tag: 1680279043-629575 X-HE-Meta: U2FsdGVkX1+F+AQkKG+8gyZuDS1oPH2Nvsdv3bgLWBwMipkTwAWwvN35/KcxIl6nF2DQc/7YVWU9okQiW06b9hv93S+JOJM85EPIOMGs201qsIzw62ExR/TwDxFPqUp5JNUi66Z2tjOU6uvyIQBjalIUDQjxK9v+95NyXzpHm7EyaAL6UvLHMe1xa413sQYCp04d4fKqsBLjGmIjXfjI3wKjzjRMjWALRAIR7g7s7OYp5DTbwFdtJLZuUHtQBtJs/TqGZNi8RNneqo0BbpwANjDbNd0ooSpb4t/igWo5RkPA6rt/QsmGX+GlU0ftSISWH/KTd+ElLTXJJyVH+rz+330jWDx3qWqsFIHxRpFYr1fNHmnPIo5lb2co8yrywHDsgU4IP/8gaOPx0jMTmeW6a335Sb2nQ/o6BtjTygX9jrwEhqgfvA8No3zs3/HZ6ACu4MncQV9Uplc/dnYQHLHxoezIxBzP9LwMyr8+SXMvGG3QpNVwIGg3t8GY4zVT0KeGwaM8jcfy6ERAA1aypdSMlCYJDlv4KGJkIroPITHzlod+IPHMW+rpF5H66jaekJyQdc8gaQBAQMkbKoqqRHuBWNLbANX8D4qnUf1MU/7fetrIOPfwr8Fg7Gs01XVICt1UJvR6MXMGgIjyi4W+FEvctGcowCnF5OabfQ6zu70W3tbdPxm4E3dG9Ie3jSl9dH776vJiVPrH9x8FD1N03J6tZWiUf4RnFTuHmQ5t6dHI6aVT8txGcouwC5rJZJPc1C8+Sv8p+bqZZWcUY0RfM01fhkQVD5W2YQeNW4hy9lVGdZLcEDv7wNBYDpejo91LbdZrKLrDqR+/OEZnDH+tGePfHvUPqfzflwfEp0IpYmBPJHlaQ6wZZatvFsSU/B0Zc/hDBbL7X4fyH+c/7vOIPvjCZVcV9VZqA4PwgpqViPxhjWnbErplwRQTOp5QVh56roXALC1jLp3yjvFZyje7wex ROrQe4WX RFcZgJdxKPIS8aonAzxO1TgBkMO8cTYhWqZQzOiizm2xbqik38C4ZgAP+ouZe5Iq2FzwyPdkMjD8ZTEkMbJLyIJ5vbS4OzJw4aJJLMqhHRYe7KBuorAYLhJUMGeYhyqWk/5Tdo1NosRWW8YLbhy39KGuoutpXFssvsbOSymPvp1/1Zz7AreR2ZbtDgAZuAD2ixD8qgqv+J21tbLm7IjeqT37DxXjGjxwkFkwEAllR2OippyioQBgD8bkJ0Vl8c7n4pJu//q6L+m5nAL+cpUpnCf7mfvPnpOQtQOJZiX3JJAVUy8vU8u/YyRykjjf4lphyVxpsvGd8ZQPt/8200tEQN/sVR+W3rDNmCsyRlxgM2UrrDrJDY6ZedDFrVEkfhq196RRiKjyon+lRMP+gBDVNF8HEwh+agAa8FYrnPeR9oYuHscqfGHu7W/L/wm/dDjhv8qnpc4wAu1ttFCB24mYQcPbtLepqE1At2LDVHNvu4LWt8XFFVAl7ypGy6JoLkXLBQ8fvRyKMsWjPZ0DVuo8/kPYQx4TL7omHx2c0xv6Iqtcz1AryOZBEiFkkxxlZP3zCjwygngkSnrTV4MI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_device_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. With that, the tls_iter_offset union is no longer necessary and can be replaced with an iov_iter pointer and the zc_page argument to tls_push_data() can also be removed. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 79 ++++++++++---------------------------------- 1 file changed, 18 insertions(+), 61 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index f5c3b56ac1ce..6cfd1577a212 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -424,16 +424,10 @@ static int tls_device_copy_data(void *addr, size_t bytes, struct iov_iter *i) return 0; } -union tls_iter_offset { - struct iov_iter *msg_iter; - int offset; -}; - static int tls_push_data(struct sock *sk, - union tls_iter_offset iter_offset, + struct iov_iter *iter, size_t size, int flags, - unsigned char record_type, - struct page *zc_page) + unsigned char record_type) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_prot_info *prot = &tls_ctx->prot_info; @@ -501,19 +495,12 @@ static int tls_push_data(struct sock *sk, record = ctx->open_record; copy = min_t(size_t, size, max_open_record_len - record->len); - if (copy && zc_page) { - struct page_frag zc_pfrag; - - zc_pfrag.page = zc_page; - zc_pfrag.offset = iter_offset.offset; - zc_pfrag.size = copy; - tls_append_frag(record, &zc_pfrag, copy); - } else if (copy && (flags & MSG_SPLICE_PAGES)) { + if (copy && (flags & MSG_SPLICE_PAGES)) { struct page_frag zc_pfrag; struct page **pages = &zc_pfrag.page; size_t off; - rc = iov_iter_extract_pages(iter_offset.msg_iter, &pages, + rc = iov_iter_extract_pages(iter, &pages, copy, 1, 0, &off); if (rc <= 0) { if (rc == 0) @@ -523,7 +510,7 @@ static int tls_push_data(struct sock *sk, copy = rc; if (!sendpage_ok(zc_pfrag.page)) { - iov_iter_revert(iter_offset.msg_iter, copy); + iov_iter_revert(iter, copy); goto no_zcopy_this_page; } @@ -536,7 +523,7 @@ static int tls_push_data(struct sock *sk, rc = tls_device_copy_data(page_address(pfrag->page) + pfrag->offset, copy, - iter_offset.msg_iter); + iter); if (rc) goto handle_error; tls_append_frag(record, pfrag, copy); @@ -591,7 +578,6 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { unsigned char record_type = TLS_RECORD_TYPE_DATA; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; int rc; if (!tls_ctx->zerocopy_sendfile) @@ -606,8 +592,7 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto out; } - iter.msg_iter = &msg->msg_iter; - rc = tls_push_data(sk, iter, size, msg->msg_flags, record_type, NULL); + rc = tls_push_data(sk, &msg->msg_iter, size, msg->msg_flags, record_type); out: release_sock(sk); @@ -618,44 +603,18 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter_offset; - struct iov_iter msg_iter; - char *kaddr; - struct kvec iov; - int rc; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - mutex_lock(&tls_ctx->tx_lock); - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (flags & MSG_OOB) { - rc = -EOPNOTSUPP; - goto out; - } - - if (tls_ctx->zerocopy_sendfile) { - iter_offset.offset = offset; - rc = tls_push_data(sk, iter_offset, size, - flags, TLS_RECORD_TYPE_DATA, page); - goto out; - } - - kaddr = kmap(page); - iov.iov_base = kaddr + offset; - iov.iov_len = size; - iov_iter_kvec(&msg_iter, ITER_SOURCE, &iov, 1, size); - iter_offset.msg_iter = &msg_iter; - rc = tls_push_data(sk, iter_offset, size, flags, TLS_RECORD_TYPE_DATA, - NULL); - kunmap(page); + if (flags & MSG_OOB) + return -EOPNOTSUPP; -out: - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return rc; + bvec_set_page(&bvec, page, offset, size); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_device_sendmsg(sk, &msg, size); } struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context, @@ -720,12 +679,10 @@ EXPORT_SYMBOL(tls_get_record); static int tls_device_push_pending_record(struct sock *sk, int flags) { - union tls_iter_offset iter; - struct iov_iter msg_iter; + struct iov_iter iter; - iov_iter_kvec(&msg_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &msg_iter; - return tls_push_data(sk, iter, 0, flags, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_kvec(&iter, ITER_SOURCE, NULL, 0, 0); + return tls_push_data(sk, &iter, 0, flags, TLS_RECORD_TYPE_DATA); } void tls_device_write_space(struct sock *sk, struct tls_context *ctx)