From patchwork Wed May 31 12:45:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 467D6C77B7A for ; Wed, 31 May 2023 12:45:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB3528E0007; Wed, 31 May 2023 08:45:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B63C28E0002; Wed, 31 May 2023 08:45:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A041A8E0007; Wed, 31 May 2023 08:45:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8D2E68E0002 for ; Wed, 31 May 2023 08:45:46 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 52F4540142 for ; Wed, 31 May 2023 12:45:46 +0000 (UTC) X-FDA: 80850521892.24.37FC9DF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 0A12F40013 for ; Wed, 31 May 2023 12:45:42 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H5yRwNeV; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537143; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VUrQef8DmIGGqxD5N5RNkU1BHyTMou9oFGzjwHHlMuE=; b=40ZF2ni4mcGhols6TA6uzK/Xgbcp/rTaSWq0W8egmDeyq1NjuCPI+CVBifcnNv91vDnWO+ 0JUjHcEbkHckUzFzpeknSiR7SfD/1dOvHVmc0CmsyKUQ7PvdFkaUEYsDZZaQeaNQwD8AJV WxR7qkmH4EhdZuhzg/z4lZA0MnzUeMA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537143; a=rsa-sha256; cv=none; b=6tDBWjv/1fnl+BvHyzC13WdV0tFVvfgmoeQwp5UQ3ghrtkOiXLGfEr9LE+Y8O5ildMNCiF hmdJOh57v5Gx60gs8lpUeIbYZxTAGXyt/BpYqgCs19CtmP42ohmddrgDGG8ty9CC9XVxHV jxGe68Un/spJbhurD7Sgr1eiAmHjA0k= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H5yRwNeV; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VUrQef8DmIGGqxD5N5RNkU1BHyTMou9oFGzjwHHlMuE=; b=H5yRwNeVIjOnlS5NbLxWGa6hsALP2DgsXpmWB5byNmH/bj/eNw71uz3bZdOwWFtI/nvJhU 5rWTUaAGkSL09UJIWE0YSaiDFR5GScNUu1OziMoAHBDYKBhVEbvvAVgAwEk2q4no7YnUE/ IN+QKUgcOkPNSI68si7CnfvicpooxVk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-479-MLbnjYjsMtCMKcbnjqfptw-1; Wed, 31 May 2023 08:45:38 -0400 X-MC-Unique: MLbnjYjsMtCMKcbnjqfptw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DE0311C0150B; Wed, 31 May 2023 12:45:37 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1CF7F20296C8; Wed, 31 May 2023 12:45:34 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Linus Torvalds , Al Viro , Jan Kara , Jeff Layton , David Hildenbrand , Christian Brauner , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH net-next v2 1/6] splice, net: Fix MSG_MORE signalling in splice_direct_to_actor() Date: Wed, 31 May 2023 13:45:23 +0100 Message-ID: <20230531124528.699123-2-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Stat-Signature: eqif73fwybh5nw6331k7amxjn3b751nk X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0A12F40013 X-Rspam-User: X-HE-Tag: 1685537142-252207 X-HE-Meta: U2FsdGVkX1+e0YOS24H0En+yZ4E2Lql4NspP6gOnspmH9q/S2+T0I2pEsVfXxJy46gI5GJwgQClrfBIvxYxQuyqeAeL6nW1WMXAzoBvjjBv6X8d4EMOoV52fKTeCgIqNFJ58Ai/hUNmTUnH3bXQ4b/p3jb3VYgsFuJOGN1vmkavTNq7Rz9XfeBbY00i29m1yymu64QaY/OJO1yydTNozVqRhCkdyHnRafKRivXGIlFlr4v6muQGrfzMIK/r1xBY7I80kHv/+a4U0mZlT3znOLRdBrnXIcrBsZVGU81SxKpi8ZgRKG7XNK/8udQ6RMiYNmCor9aoxHWviMeE8CZOdOd9V7/bM7vLy4424czd9dHmikj9fWoOBs+SPZxS9I0dNl1GZNpOrruquXnCEEAXr3t8kSVtUpvmNh2ChPCNz2NB54i//RTwiW1Hko5re/HVxEuI1Ahk2ZGINJpw5vaEz7THSm19S9KKu7u5oW+MHYbAjq/kNc4vYRqWHb/ybfFizCXNsUaDemr40hZ90vBNvflblLBHWJGuIuBRb04jhwwT6ZENK36yC9ohrepF7/70dDGdsqRDv0xVTyxysXP/CKyupwshB1VZo5fkO+MIpjxbU+ZKf6d/szKJ6xt7uHIPTCyjRDbSBGgkSScuaZ8KNa8vjpkPYEzsfgunqVcyZpj2PI7bF4QcIW7E+kyo0Bi6XdMzwJZrcLYwF6cDw6kZlykVefp0jM2PvMnhfJybneKyhsebILZ3sLxWDPJfdV03Dq34waB7/+doExmrCg762elrbwNYAnyrmkN0lAy+Ri19liYVO8myde48BK7o+paZCpqbxMLakMdtljvZj7jEYuuYm+yoRF69RNVbH24VOKK8Lz/wOwGkjWbvmmJXDUg/qqBRlO/qJzgZ4YL0kh2xwDyiAvabODE7XZJ4BKqnCsSfIqcwtSkKRoy8eeSEn4j76jOnRqd3t1oeKRljeeYt ERFneYnn cOFT81xYZhviEtRsDzh4m3OMVdvJC4V5A+8StpsP6JTdm7LwYBYAz9l87m9MjQwDnNwRcuMBptq9Mm+xZo6Kb8nN3bn873yK3hFYAQ5wVfRki0xawGnC6GFDYUmaRp1GuQ+Rp5RXVAiQPS4RJfrP+pwVe1bDjfejH16jpp2Ns/tB7FvxU9mecdF9r9ijUQ55ps+rlqGgC5fRYfvvYgbGxB0Bmf0sbGlki207RXPb+/nEVfk4doF4vuXvi/70s6tkunlVZpb40HJFCj/xM/6mbMF18vBS5pSL3QDQjyHJXm6l9bANc548bamPzFoOkPa1p5NPAVU1TSMxkJUCee9u9aMKm9kGEGemd4Zo78SkmQ7SVrHxmlqPpJDHghUZ/DQBbtpdMYZvgnFHeMS1b6oGO7UX/v7OSv9giYlIxyfDe41Oo5BeAc4wQtlFrDukUHJaCMEpGnmkA3t7ae39pGK93cBUnE6Owh62725J/FvOVak5O0vFDHFVMS0pM7h1fmqJnT7Nw6rDZZ21Q4LHLseyLNp9V4DFazUdEL+QNDu7/SDq31U9nP+Yw+FE41y8X9fIDkZU5ufpdXPxUFCVUD2yLkjDdWa7lAr8T3ylK64qUORM3oxgJwWtRRSX0LjuvLeCG2XIuCuhULTL+6gxzLcl0HUN9o+hg1CpFloqYkqNKVbk6m/0fLzkqzlRmrIVLh2paI9ee0Nnr/tHEBxoK6VuXay2LNTAT9Jt0QMcXcFjWGTKg7POXBSvd79gs1yv/JT7rqIUhOaGppVSWrLwaKVZKbREPDvNePLdFAbyydD5e35w59YU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: splice_direct_to_actor() doesn't manage SPLICE_F_MORE correctly - and, as a result, incorrectly signals MSG_MORE when splicing to a socket. The problem happens when a short splice occurs because we got a short read due to hitting the EOF on a file. Because the length read (read_len) is less than the remaining size to be spliced (len), SPLICE_F_MORE is set. This causes MSG_MORE to be set by pipe_to_sendpage(), indicating to the network protocol that more data is to be expected. With the changes I want to make to switch from using sendpage to using sendmsg(MSG_SPLICE_PAGES), MSG_MORE needs to work properly. This was observed with the multi_chunk_sendfile tests in the tls kselftest program. Some of those tests would hang and time out when the last chunk of file was less than the sendfile request size. This has been observed before[1] and worked around in AF_TLS[2]. Fix this by checking to see if the source file is seekable if we get a short read and, if it is, checking to see if we hit the file size. This should also work for block devices. This won't help procfiles and suchlike as they're zero length files that can be read from[3]. To handle that, should splice make a zero-length call with SPLICE_F_MORE cleared (assuming it wasn't set by userspace via splice()) if it gets a zero-length read? Signed-off-by: David Howells cc: Jakub Kicinski cc: Jens Axboe cc: Christoph Hellwig cc: Linus Torvalds cc: Al Viro cc: Matthew Wilcox cc: Jan Kara cc: Jeff Layton cc: David Hildenbrand cc: Christian Brauner cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org Link: https://lore.kernel.org/netdev/1591392508-14592-1-git-send-email-pooja.trivedi@stackpath.com/ [1] Link: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=d452d48b9f8b1a7f8152d33ef52cfd7fe1735b0a [2] Link: https://lore.kernel.org/r/CAHk-=wjDq5_wLWrapzFiJ3ZNn6aGFWeMJpAj5q+4z-Ok8DD9dA@mail.gmail.com/ [3] --- fs/splice.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 3e06611d19ae..a7cf216c02a7 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -982,10 +982,21 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, * If this is the last data and SPLICE_F_MORE was not set * initially, clears it. */ - if (read_len < len) - sd->flags |= SPLICE_F_MORE; - else if (!more) + if (read_len < len) { + struct inode *ii = in->f_mapping->host; + + if (ii->i_fop->llseek != noop_llseek && + pos >= i_size_read(ii)) { + if (!more) + sd->flags &= ~SPLICE_F_MORE; + } else { + sd->flags |= SPLICE_F_MORE; + } + + } else if (!more) { sd->flags &= ~SPLICE_F_MORE; + } + /* * NOTE: nonblocking mode only applies to the input. We * must not do the output in nonblocking mode as then we From patchwork Wed May 31 12:45:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38F71C77B73 for ; Wed, 31 May 2023 12:45:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEBC38E0008; Wed, 31 May 2023 08:45:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9AD28E0002; Wed, 31 May 2023 08:45:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B63938E0008; Wed, 31 May 2023 08:45:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A78F38E0002 for ; Wed, 31 May 2023 08:45:48 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6E9DF14013D for ; Wed, 31 May 2023 12:45:48 +0000 (UTC) X-FDA: 80850521976.03.9075B3A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf15.hostedemail.com (Postfix) with ESMTP id A6E8CA000A for ; Wed, 31 May 2023 12:45:45 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="WEbs/vgq"; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537145; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=ypUhZx3ZOKsMxHReu1At3H2UUrPnMCb1wKevx4KA8x3pTecN+Q8nDtsdAqkndSYJvjdRJ6 H3X349vQrQGL3dmsUr4TD8PnFQ4MLT+QILpd0WQ2LmJWff6DxQffgsUvg8+jxr28yIaLkl JzKJlTtHTHGVp2aNKHxCiIwBJN8DLf8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537145; a=rsa-sha256; cv=none; b=zNieVKQPYPnxuQ+G08MdoDdBETP2o5UyX5MmBf/oDKwi3RUTzezJAXi5UE4GItkE8Z4TLv kEWX6+v2sEEyF3M0QljH9fEBuI/ZgzqGFngBrXvLo1FvdsHsO+oD9Icm/+PRaUMtCOBD6Z BM5qFaCrqfImBHs3YSCgOTBZTB+z5BA= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="WEbs/vgq"; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=WEbs/vgqtdc7UsAEh8O4M0ghNHBj1PiB9QnqdNiMATXe6KuZ8WDckAVxnrIYpEeGtRZpVz Dgg/KbMZeVf2+T/0e7BHrIPzac6BBLQbKaYG70TI6YRnEFVgKwlbEQiY+ZZqfCwLCr/t0y 9hwdKHHc0uU/gAIZpQIyaU3KoQxhhLU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-625-u-3ocbAEPfKvWCDqcrmGEg-1; Wed, 31 May 2023 08:45:41 -0400 X-MC-Unique: u-3ocbAEPfKvWCDqcrmGEg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A721F3C0C897; Wed, 31 May 2023 12:45:40 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id B74EE112132E; Wed, 31 May 2023 12:45:38 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v2 2/6] net: Block MSG_SENDPAGE_* from being passed to sendmsg() by userspace Date: Wed, 31 May 2023 13:45:24 +0100 Message-ID: <20230531124528.699123-3-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Stat-Signature: y9k39bp6mchbfeqgc1s8sswx4siftu6s X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A6E8CA000A X-Rspam-User: X-HE-Tag: 1685537145-147968 X-HE-Meta: U2FsdGVkX18DDrrAKz6zmlHpfGTb47FUtT1vhgaHeN/GPk+EX/SS15Jxq5N9vgb/AE5WpaGXVVrsMRGmZEveqfYi+NLUJxghmK5UCIfP9cRF0qwnsKFk+qyN4JoqaHAo3KzzHyPHc1fHjpjJ0AhAZcrxlyL2cstPw2qQwlo3cA1nSxyMnerHfEpPWiMxPc3r+MZM/ITVf5bK+H94hgGKtDUSPeYpwspdWEj5AIu4vMYpF7MxHD9QHCDYjgTPe9+U8hX0WzKYEjYmc8p+PmUIhB4W14HCg2UL2/ich02xuSuhBbND2c8d0Rz/6ay6y5caRiFUotnbu0d2iI+eHlZRm20a2+hkGEapSqQE6aWDod69YOhy/qdFqBjiYwJgcpxyFDWqCbmt3WOeZbXtBnD8eNxmyQ2XEJxpys33gtLjsfetRv/kmoV2nlXQ4WyuPTBnHoUGxdmMOODymHB7Ph5kmWLJwqH1DVwE0sIfJMjlG2aUXC+fulsaSv0pCvMexr056W9p1miO7gDNZGZptSkIKdUg7hIvjwshnXdyK9rDNr+f5gvXLVezZparCV7og/hsuVLtbJIOTbaJssCsmq+fD4rnSNeqHbN2ZjMzDLcY25ZCWib48+aT/M/rUZE9saOXFh8CnFme/+2JwIGCwKfoJI5eeelKIVEEzW+IgQDV050ZtT575gUQ62nW2Ef1buFa9t7eB33Y9eTeeZkk9GIhJS4X2WkExM5ZAvcvA8ZTKLe33gV+bQvZwp9Lx84cBBtR2XqR4o86cbCePk/9GBAZ7dchIkfLd64Vd1gZfJm9K9FTq49X3YreyNGHltwSlfSXurhaRq/OLxITnL0289Vdb3euKoF1u8S/KmqbhKEiiI5T51VDESJ4MGpwnLqjJqSFUv3xe7Lq1Mx8uuNfIG9D+GLzZIDk0DGngJ8fE64FUnOt2SRejc+jsimXaForiY8UdSBE3h4IHEWAJI0eBN9 KcDSpLFO FMQ6wQ7RNkhCc6fVK/PIcX63g1AsBxgySmFHxu5uEnDcqZjYUCOtE9E5nv3LMTtxUHuM9isXWG9WuOnmiZLGnypHmzC2QAP2AR2dvIC/qm+RXId54xJWldKr4JOZyxROTjp1+7WcM7MY+FKakjbKbCO8rO3eIFnOq2Nv4qo+Yp5CPyzIooMdk+43cEoPqAr9yVzesAvZ+rYSrTMwnC17DDqOm0Im33r4Hlp+h5Amp97/A1v4NbTjrn440y7/AUqQou0y8DMbw4oM6S63q/BprpoMLymoft0MjVv8UqkYGFG3qCiVuFSredrSVWk8Xxmtu2F1msMLBYFdFaqh3RhE9gFW8IHOYqoG8G0Z9G6+05NLa4cSxjIcP1FoLXNQTg9kGdBFfHTwJKUp6ROgp/exnX3aGboxF23D6aHYByoQYAX3vTgrIQJYYMkxh4BsUCHt+5YIU5AulnNCUUQaT543Nw33t/SDTC7/xo7+/CkxBf6cDUv3wFBEE3Eg5/8uBlGC1Csqoj2nORODdhnYtXDb3d9LRTwK1Nrbf0BCH4xypox+WdADQ9gujE0VpJA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is necessary to allow MSG_SENDPAGE_* to be passed into ->sendmsg() to allow sendmsg(MSG_SPLICE_PAGES) to replace ->sendpage(). Unblocking them in the network protocol, however, allows these flags to be passed in by userspace too[1]. Fix this by marking MSG_SENDPAGE_NOPOLICY, MSG_SENDPAGE_NOTLAST and MSG_SENDPAGE_DECRYPTED as internal flags, which causes sendmsg() to object if they are passed to sendmsg() by userspace. Network protocol ->sendmsg() implementations can then allow them through. Note that it should be possible to remove MSG_SENDPAGE_NOTLAST once sendpage is removed as a whole slew of pages will be passed in in one go by splice through sendmsg, with MSG_MORE being set if it has more data waiting in the pipe. Signed-off-by: David Howells cc: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Link: https://lore.kernel.org/r/20230526181338.03a99016@kernel.org/ [1] --- include/linux/socket.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index bd1cc3238851..3fd3436bc09f 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -339,7 +339,9 @@ struct ucred { #endif /* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ -#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) +#define MSG_INTERNAL_SENDMSG_FLAGS \ + (MSG_SPLICE_PAGES | MSG_SENDPAGE_NOPOLICY | MSG_SENDPAGE_NOTLAST | \ + MSG_SENDPAGE_DECRYPTED) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 From patchwork Wed May 31 12:45:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B8C5C77B7A for ; Wed, 31 May 2023 12:45:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9CD38E0009; Wed, 31 May 2023 08:45:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4CB78E0002; Wed, 31 May 2023 08:45:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B15798E0009; Wed, 31 May 2023 08:45:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9FCDC8E0002 for ; Wed, 31 May 2023 08:45:53 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 71B0E1C6E34 for ; Wed, 31 May 2023 12:45:53 +0000 (UTC) X-FDA: 80850522186.07.6A4A33E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 88B9140010 for ; Wed, 31 May 2023 12:45:51 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZmL1paaQ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537151; a=rsa-sha256; cv=none; b=sVRMoZ1IhRx+V9h3pow2WkImYPorFshqC2NeXdkPjkpqnkHpaCmDNQWTch7kEeTtN+GOun Yf4UUaJLtFqx4ITSaeVzREEFsDIopT4nfKSMQTOm8pHWwDb0YnE2B7PVJ/r9BctYIrTpYw dc9rDtbQN7ITluhfSizBiWmxTc1EMk0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZmL1paaQ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537151; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xFpJiBMaV6haKLFUiEEfF2pWt3wFFqyfTnw9cLbfIuI=; b=MWKGnXXKzs+DxGcMbok8cWisX6a2TLGZlv36T69AmIiUcEk/GXvhUysWjvn9ilGN0U+nj0 9uYYZSqEh4gpoo5sKaq3h8FUvfXMNFySNBZIK2NQS/EVn6hYA8F8DimPbJmwBpgO5Qj1Kg Iawzv286exG1GmJ0XqBOV9TtzzgrUFo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537150; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xFpJiBMaV6haKLFUiEEfF2pWt3wFFqyfTnw9cLbfIuI=; b=ZmL1paaQyFbbEeYw7tKJNUlewte+Te/q/hK7PT9wtgkJUKMdzim9mKMVjFm317cYq/wzv2 Bm47gd09Cq21Kj3sQf9bZCRAs1RyzlC+EFaghTU4pPuf5ldl0MBBppzcDdvnY7TvqRQcEx HyAjCON40gh4IFclVQnXrta7VusiR4U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-116-ZBHr42RPMCeBYoDi-5JaaA-1; Wed, 31 May 2023 08:45:44 -0400 X-MC-Unique: ZBHr42RPMCeBYoDi-5JaaA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C5504811E7F; Wed, 31 May 2023 12:45:43 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99826112132C; Wed, 31 May 2023 12:45:41 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v2 3/6] tls/sw: Support MSG_SPLICE_PAGES Date: Wed, 31 May 2023 13:45:25 +0100 Message-ID: <20230531124528.699123-4-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 88B9140010 X-Stat-Signature: 8cz5ykoi9krbjh9nd6yy8thhay68rjga X-HE-Tag: 1685537151-319108 X-HE-Meta: U2FsdGVkX199+5Jw1m56R6XvS1NokEOnR0RSDvOCPQr7NzRU1EsTOG9tWzZVKMqxBvr8btgZX4KvWDgFrF6/ZBdLtbh8PcVHhA3AofpxuOaN29ZgUI9rRRjnw2r6PR0LWur0mFTa8hsCzMMhwJzyPc1CkVAJydz0aiXArMiVUl84fdYKu30hrdbFkYImbJBxiTFRqfuQlO3xW9yh7tqNWlPVBe31YpbWlHNh+1Bet9EsTf0OfxEnz0CJbE39OtAdLZ4NlCrW8gztc3YkYzzaU4SSIQMQj7pSRbFqpnicbnv0Sz8XxCbktozimmQmlNgUHF0k2ck3aR0kKbjMfBltniBCQya+qtAXwevKeePu6XLexARbgINjG9Wns0pTZIe5Z9vvzua946cfmvlNW0vBWvK3uCBaQ/to1mfP+5e1gb36BNDL3d2aDWx/Mo7lX/DmWKiJOZJU1jpnyJmSxLd2Ago2yzRB3TRPaBY1JML0/DghidZtaJMLvoeXoOddqXTZGE5I+5iJk3EKH33XLApG/7a5iwlsiTeMMOrwkpQkfc1aGVINuutGOYQFsG4HTwsHEsC2dYE2XFerxdL4GTcEgYiAo5da86kzDgTzNG9OsElJM410V8LM56hAMsff4i1TVow5rgFTzcpi8Qhmg0AeJwG+78GmyhVTYtmIoxIzXyKF/aWTjxdBY+PtImyNV5zqX7AmD5S8g6eyhJK83Gm8WiNlzToxqzj8TUGhok9+B3TvOcm7Tob1dPeVj3xa4GZWOXaSpVcsFHBg3nyvaXDWmlk9bAHa0JR7XtgEDL3A2+w1dfoPmSO5iU9lumNMKGGh7XwGLH7Gb302D0MCabEz5Q0x6Kb/fISJ3BGShBz8vJMCL2nt3gVZ9WQfQl9ZH5wnriSp/DoBC4yBF8IhhaVdsvTQK6S05QF3Ei6MIdawAXZO5W+L+kET6Ggh1f0GnFPZWJldA6BFHUj1lyy5uD0 ycXVHJ+P y+No/yHLQBPlY7ytDYL2S7eb5FU2ttlwciFkpymZIo8JCUpnk0bicZhegJh/nNmWib+yEG9hMPXczn0/5fQWjDJv9Aw/btxoXD08Mjz382rVpBTqclYTeFCA+3rW0fsdM7z24vF+yvxxBQzsUNcQYFaC/wG/hFflOc7XiUaIAVg9605KZeC7QlG/5iuo2Y7j9r/QQfwcehjykq6nZqMYB+kEPnAdRYJIeUY0xaANdd3SM86Dv+QRFgGb6teKoPiH1OQsQgfilC9ULAMEYeGybd0MjgBz9n3W/NUpZId7jn5XXEwneI1rbuEhyCY49pfZGLAcxh3r0TlMILsQ8p/rIO+JwjNCoCm6gvxmbIN4ApB2ZD4jmVVm2NfV75cii22pKIBSQTRfsn3bo2VdO+Eo6BmO9wm5vqW7xEGfeRwCRikt8Z8VCJy8ZV03IcNLKZhzLuA1cSNfeAS9BmhLC4dIps9LTSfIORHGaq0Y279spO3IdSZKV5B57um1PQW+ZDG3xBzLagJRxxTvEOF1gpKUFUB9sECSOI7l05UNd5TwVYb1inEuxjuByDuo8xw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #2) - "rls_" should be "tls_". net/tls/tls_sw.c | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 6e6a7c37d685..60d807b63f75 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -929,6 +929,38 @@ static int tls_sw_push_pending_record(struct sock *sk, int flags) &copied, flags); } +static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, + struct sk_msg *msg_pl, size_t try_to_copy, + ssize_t *copied) +{ + struct page *page = NULL, **pages = &page; + + do { + ssize_t part; + size_t off; + bool put = false; + + part = iov_iter_extract_pages(&msg->msg_iter, &pages, + try_to_copy, 1, 0, &off); + if (part <= 0) + return part ?: -EIO; + + if (WARN_ON_ONCE(!sendpage_ok(page))) { + iov_iter_revert(&msg->msg_iter, part); + return -EIO; + } + + sk_msg_page_add(msg_pl, page, part, off); + sk_mem_charge(sk, part); + if (put) + put_page(page); + *copied += part; + try_to_copy -= part; + } while (try_to_copy && !sk_msg_full(msg_pl)); + + return 0; +} + int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); @@ -1018,6 +1050,17 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) full_record = true; } + if (try_to_copy && (msg->msg_flags & MSG_SPLICE_PAGES)) { + ret = tls_sw_sendmsg_splice(sk, msg, msg_pl, + try_to_copy, &copied); + if (ret < 0) + goto send_end; + tls_ctx->pending_open_record_frags = true; + if (full_record || eor || sk_msg_full(msg_pl)) + goto copied; + continue; + } + if (!is_kvec && (full_record || eor) && !async_capable) { u32 first = msg_pl->sg.end; @@ -1080,8 +1123,9 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) /* Open records defined only if successfully copied, otherwise * we would trim the sg but not reset the open record frags. */ - tls_ctx->pending_open_record_frags = true; copied += try_to_copy; +copied: + tls_ctx->pending_open_record_frags = true; if (full_record || eor) { ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, record_type, &copied, From patchwork Wed May 31 12:45:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A67A8C77B73 for ; Wed, 31 May 2023 12:45:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 439918E000A; Wed, 31 May 2023 08:45:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E93F8E0002; Wed, 31 May 2023 08:45:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23D538E000A; Wed, 31 May 2023 08:45:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 149C18E0002 for ; Wed, 31 May 2023 08:45:56 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9BDE01A018D for ; Wed, 31 May 2023 12:45:55 +0000 (UTC) X-FDA: 80850522270.11.D5895C4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf10.hostedemail.com (Postfix) with ESMTP id A336CC0023 for ; Wed, 31 May 2023 12:45:53 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eaTXNOEV; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W8aEdMrYnuprZg7eZCUzLy+kKHthZDT31VQbMyJsAn8=; b=I1w6O3TyaMFRM0AvXTdPH87WGDOSn2mNkHAjm3WO1PXiiS6FkAhad4KteKOkBsxXy5RGXE OqtWXFQRnS0NeQO6NjNpjI4g2ZRf1tEUtLLMnIzpYEZL5VAF5CD5j3+r4Py9b+yJslplt4 qiGYkAv0alWZbXsNaD9AtFMcKcQDfCk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537153; a=rsa-sha256; cv=none; b=oK3vxWULLs7/pVDBe/6Wcivd9Hie7vwt1PE2xcsF/nzhY5Pw025vzlcCvkeEem59lJE73O 0VTR9wjzEpdmJ05H0ed8Dt9zDGJuJExwApg2azU2okEpm5xr1rtM7p6fNNUWPE9hJQ9M3Y 7rqNar6r84u0anKkFhzQqAuc7Z9fJXI= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eaTXNOEV; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W8aEdMrYnuprZg7eZCUzLy+kKHthZDT31VQbMyJsAn8=; b=eaTXNOEVZzqJSbHKasDeuNYAsd6mxcp/ot3sEY/pE/7KSM6zXRUhOm/Woz3c0sVZPu1Zi6 jXdyZ6zW0ROaqKK27vZAr+7MLw1SIrTc91q/YtLU6MpwFjG1IDAH4RqHA3xZ/0tvwMZfyv PHXiTIzzMcwkJqRrqOl/HkOF+TV1bi8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-370-LbK9yftePiayyMeYHhVc5g-1; Wed, 31 May 2023 08:45:48 -0400 X-MC-Unique: LbK9yftePiayyMeYHhVc5g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D0DF53825BA9; Wed, 31 May 2023 12:45:47 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3555140E963; Wed, 31 May 2023 12:45:44 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH net-next v2 4/6] tls/sw: Convert tls_sw_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 31 May 2023 13:45:26 +0100 Message-ID: <20230531124528.699123-5-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: A336CC0023 X-Rspam-User: X-Stat-Signature: qqz3ydugir8q4fj4pbksomxj3gayubxf X-Rspamd-Server: rspam03 X-HE-Tag: 1685537153-879555 X-HE-Meta: U2FsdGVkX19ssiptl7BaqJARUokmyB5VFHuoyUu26KhbG2/UWNyvl4VPg7YIZWHPlRuH9VPMqICPxtLOM2SrCJkensIOyfTPUwe2afa+s7p5m8K2sYYlBMzREB+O3AVHIAMK21AC5ZDl00a4pgT3kYHLER98SeHcDcif7VV/NGAFyxA39xWw0/WVG0fmProdt9UfCJvzk8ee5mej+JJ2Yis8PBTDK9ikthrlVddUsswrl2xzfLZnl5562DVmACu9TXv+IJgs/uP+g063c58OZjVYZUJXuDcvHvX9u8toQ1hgJuE0FlJDb6tGAJ4fvCCNLPkih9U0b6/h0EOna4LjCIAlFLJVjSRnxcyw/q5BMVdf1J7Ubnbqb6a4VAdeURKdUEkBHtu+cIkahVV/nHP5Vfkr/UVus2sPZxDcSktF5kawT69Z4oD4C83pqcw6ohDL0xwD44CN9TZOYNTiuZtF0A3G8xVsxgQtV8EScRJRgM5E2mMBmmbxWioQ9/P2tfaet9SaLT3MYTUjUk5xxXOG7YdJYfqRDDb8FFHIAyKTchOcWRwIexZcFskzrBqyUYrVIWnmdYwWOIuSmgLwDauceEUDw6GL+T3M1IGKXeL1FJJrNEb8nok6ebfPRUTzWSX46eGoCMHeFwoxIUhfzDgQhPpuTo9c2jASvSfSpHdoiQFSmU/FiuuyFu4LJF7H11GllFqoCsowpcLjjSQAcUF82xV1AgifVF83o6Duk3RsKc8zhPF9G8fiiCV+dsupdbHDvvpmrvM1xXn2Wbfpyn5713DjRgYyv13yU/qWzdq3sA5ua4JnoLnn6UcL8osT5x43ADVz8uEPHWeGGoqpnM1NLddRo+o+ZXvLALMU9XFj7qkLOR3VM1j9uwxjy1Y+Y/uLEsRGTT5bOq0VqMIijXlhEujGm5037xPYmmDGhTXy1VNAFGgDacPmQXCrRRq4by/Dieh4QI3Umg+o7QAICcz 91+3LrZy +ByPeOixWmRwx4yuD+TZLj+ElTfbVxUL98zyqtLSpNbJuV6HA50aljJgLh4Mrps2RHndv3rVbKJC2pcChEtdSeot8qv0PA5PT5vtCr+lbJ8V2j9IcfxQboG+RuR9AGif0ClD3NW5l37kwYdECGJ8Lk1t+4FuVto6fprPYD23IAHM81E4xY1ijmOoJ9EAbZqwkH2V94k2Lx6uCjgMqqixJnrPLR/r4ya/2+p0Uvu/a7VBk5962cQwPaYRkvF2BVgwXqMkDwC1yZ4mQFJnoIpSVnx+coC4gEPbk3lNJ86yQSu59h9cNiO6G0Qu4HHSalJsNuOJKlFICos9HCVOyVrD7MMhQAOhg/NM85yBbZF031NpUMKG4Uuw0wwH2LIA/04d3x82+lUUhwgUS/F6HdE3L49bDHt7SuMI0puy82/dPUCo+PQ43GKcLK8zl9AeoSB3hgafQGp2W6KMVxawLupZUiVARW91z+h4LcweiiWDqvO3mJu0v2QefvJOQMNn1s+i28zAcCcfjfFFSvXLu5CcP9RYEKPF+/3qwXj0jSVWFgs8OUbAii6BFanwPwA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_sw_sendpage() and tls_sw_sendpage_locked() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. [!] Note that tls_sw_sendpage_locked() appears to have the wrong locking upstream. I think the caller will only hold the socket lock, but it should hold tls_ctx->tx_lock too. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org --- net/tls/tls_sw.c | 165 +++++++++-------------------------------------- 1 file changed, 31 insertions(+), 134 deletions(-) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 60d807b63f75..f63e4405cf34 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -961,7 +961,8 @@ static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, return 0; } -int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) +static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, + size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); struct tls_context *tls_ctx = tls_get_ctx(sk); @@ -984,15 +985,6 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int ret = 0; int pending; - if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | - MSG_CMSG_COMPAT)) - return -EOPNOTSUPP; - - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - if (unlikely(msg->msg_controllen)) { ret = tls_process_cmsg(sk, msg, &record_type); if (ret) { @@ -1193,157 +1185,62 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) send_end: ret = sk_stream_error(sk, msg->msg_flags, ret); - - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); return copied > 0 ? copied : ret; } -static int tls_sw_do_sendpage(struct sock *sk, struct page *page, - int offset, size_t size, int flags) +int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); struct tls_context *tls_ctx = tls_get_ctx(sk); - struct tls_sw_context_tx *ctx = tls_sw_ctx_tx(tls_ctx); - struct tls_prot_info *prot = &tls_ctx->prot_info; - unsigned char record_type = TLS_RECORD_TYPE_DATA; - struct sk_msg *msg_pl; - struct tls_rec *rec; - int num_async = 0; - ssize_t copied = 0; - bool full_record; - int record_room; - int ret = 0; - bool eor; - - eor = !(flags & MSG_SENDPAGE_NOTLAST); - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); - - /* Call the sk_stream functions to manage the sndbuf mem. */ - while (size > 0) { - size_t copy, required_size; - - if (sk->sk_err) { - ret = -sk->sk_err; - goto sendpage_end; - } - - if (ctx->open_rec) - rec = ctx->open_rec; - else - rec = ctx->open_rec = tls_get_rec(sk); - if (!rec) { - ret = -ENOMEM; - goto sendpage_end; - } - - msg_pl = &rec->msg_plaintext; - - full_record = false; - record_room = TLS_MAX_PAYLOAD_SIZE - msg_pl->sg.size; - copy = size; - if (copy >= record_room) { - copy = record_room; - full_record = true; - } - - required_size = msg_pl->sg.size + copy + prot->overhead_size; - - if (!sk_stream_memory_free(sk)) - goto wait_for_sndbuf; -alloc_payload: - ret = tls_alloc_encrypted_msg(sk, required_size); - if (ret) { - if (ret != -ENOSPC) - goto wait_for_memory; - - /* Adjust copy according to the amount that was - * actually allocated. The difference is due - * to max sg elements limit - */ - copy -= required_size - msg_pl->sg.size; - full_record = true; - } - - sk_msg_page_add(msg_pl, page, copy, offset); - sk_mem_charge(sk, copy); - - offset += copy; - size -= copy; - copied += copy; - - tls_ctx->pending_open_record_frags = true; - if (full_record || eor || sk_msg_full(msg_pl)) { - ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, - record_type, &copied, flags); - if (ret) { - if (ret == -EINPROGRESS) - num_async++; - else if (ret == -ENOMEM) - goto wait_for_memory; - else if (ret != -EAGAIN) { - if (ret == -ENOSPC) - ret = 0; - goto sendpage_end; - } - } - } - continue; -wait_for_sndbuf: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); -wait_for_memory: - ret = sk_stream_wait_memory(sk, &timeo); - if (ret) { - if (ctx->open_rec) - tls_trim_both_msgs(sk, msg_pl->sg.size); - goto sendpage_end; - } + int ret; - if (ctx->open_rec) - goto alloc_payload; - } + if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | + MSG_CMSG_COMPAT | MSG_SPLICE_PAGES | + MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) + return -EOPNOTSUPP; - if (num_async) { - /* Transmit if any encryptions have completed */ - if (test_and_clear_bit(BIT_TX_SCHEDULED, &ctx->tx_bitmask)) { - cancel_delayed_work(&ctx->tx_work.work); - tls_tx_records(sk, flags); - } - } -sendpage_end: - ret = sk_stream_error(sk, flags, ret); - return copied > 0 ? copied : ret; + ret = mutex_lock_interruptible(&tls_ctx->tx_lock); + if (ret) + return ret; + lock_sock(sk); + ret = tls_sw_sendmsg_locked(sk, msg, size); + release_sock(sk); + mutex_unlock(&tls_ctx->tx_lock); + return ret; } int tls_sw_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags) { + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY | MSG_NO_SHARED_FRAGS)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - return tls_sw_do_sendpage(sk, page, offset, size, flags); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg_locked(sk, &msg, size); } int tls_sw_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - ret = tls_sw_do_sendpage(sk, page, offset, size, flags); - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg(sk, &msg, size); } static int From patchwork Wed May 31 12:45:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA989C77B7A for ; Wed, 31 May 2023 12:45:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 830E48E000B; Wed, 31 May 2023 08:45:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 793E28E0002; Wed, 31 May 2023 08:45:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65A548E000B; Wed, 31 May 2023 08:45:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 54F1C8E0002 for ; Wed, 31 May 2023 08:45:58 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 30D3C1A0193 for ; Wed, 31 May 2023 12:45:58 +0000 (UTC) X-FDA: 80850522396.16.87D9083 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 1F215A001E for ; Wed, 31 May 2023 12:45:55 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UAnF1UGX; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uuYRuRqPrqYrWBy2myOfoGGw2Qio81ibrygQkKyKemI=; b=acdyFgnjpT6qXnDXQ0IT9WVjVXu9S2aFBz3aQPAy0uc2Z0VUOL8mGsxAmVK+cSWea9rGhv VQ6OAcchnK4BVXPHBo78Nx35MyY+JelQXzDrEa5HJ5CcGFrCmLtqNyVS3WNmFgW6P4BWxw NtPJ1l8wI4IkMmBuY6QQ9MoCpN4RiLY= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UAnF1UGX; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537156; a=rsa-sha256; cv=none; b=kdiDkrQc8RQcTx90nm39mWybFedR0x0db6hYL8vuN0/7GnX6oitWy6bZjoZZRrbnfYXwA/ LKQWeTTfWYsMGCxZdV5yWPAsvl8yx5hhssAs7fY1P7rSIrPkiJTEOC0/C12U7uTcNb6flf f4+ApjZ5Hgb4f2Ju/K6vO6fDNe7xVQI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uuYRuRqPrqYrWBy2myOfoGGw2Qio81ibrygQkKyKemI=; b=UAnF1UGXBA952Sq37/4ApDvy66IgcelMaaRx+Ecw28Be76ZHh5nSI0Ga00pDgQxBR45jcY MDPmnVn5H4lN2rHhuPtN9m1+Ft7Oe68o/HJwk0vHLksorhHcxVaQPhrJtJZMN1SfxCauJQ Hvk/yB9MG5o/ggEDpj9048Ln/6X9BeI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-54-R6gsZvE5P7aGqYDd_4hZXw-1; Wed, 31 May 2023 08:45:51 -0400 X-MC-Unique: R6gsZvE5P7aGqYDd_4hZXw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 650F3811E7F; Wed, 31 May 2023 12:45:50 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 994162166B25; Wed, 31 May 2023 12:45:48 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v2 5/6] tls/device: Support MSG_SPLICE_PAGES Date: Wed, 31 May 2023 13:45:27 +0100 Message-ID: <20230531124528.699123-6-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Stat-Signature: h6ajr7zsdenggeegokeugdpnz88kseen X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1F215A001E X-HE-Tag: 1685537155-247204 X-HE-Meta: U2FsdGVkX181lf2JRylp9JiseW4WPG1QFmKwb5V87LbABFap37YPBiPckwon0i2Veu4zeRiS3v4cQRqD91k40K/9lDmzhKCbkiBfo3oMimC98bHthkkCsXc4dT2cdRjETzLv34xUzZLDei5G2NQ+iw/HKhEH8+0IdqkxGbZbkPKLsvhZpd2OapDLGStQVN695mfNj6zradKITwYe6F+tdVKexx2WYu92NGV8UbX4PyEANkOKuH3Rx3OrmV88sPmo2pz48oaX+XhDjC8ZX53oJcyNIafYjzZmieef8W6Pw14SbewCQz/IWC7pN3S3lzzsyA/+HyCSRTTgQuiGuApOYB1eZY1XkzK5miAAwXlwt1zxQ2yC3fdVbDy/n6mzX3dBNr5fLwPKsmkuded2ZH7pZguKXbRMKwjQXlMAM1w+Cs5o6uBGc/+QICHYanqX1fHMtb7PNFXhTJ/gbxwXGE5TC1J0jftwSnZoF/WjO75fq8boDeKPiwxwRFzAKfsuGf49kflAMCCkQklfBZ3VH0kJr/c0YmpkOkyN0IIDVW3jUgcDu5Jm/F7BJ/0NTJPDc+79HyvXTnb9ViGfbbCBiGlJqFdaZZlQBdcwXEhl1opshbDDE/rO3woZRbrm79n966+OVpHaRbLiHZdAbDDqy/kwezUbhTSswBtcvgzX3/CS5yDyp7IqKsyd+nKYfxV2Nm6cYT1B9afTu4ybeVEHMdfI2LD1U0rGk4Ntb9q1uQM2s4gRHQVXc9Q9ZnGPpMLkq5v9JCnZdM/I/kr7W9wbSL+wnn01yhMqtUOerC8xeo2qmHfFgBq6rMFHgK3yrf1tm2RDDLNVdFhkeWve2QkY+/Ib7e1SNOXNWsWfalGb9+1PlhfK0jtAQtpr9vmlL6CZyVuMbJVaNt5D4NP/Ry8aLXlHUgqOyRnDtSA6ifN9QdqYr2MWjoTLDaS5AiybPAmgJ+InN6pwmjECz9OixG8hT4f jcmGe50h dklAeg6ZrzmJaVnA+rI5/+u7ydnaXtYcQMjWZXdqUxGnMY8UxjkZfmSZUwulHuVHg7CHeddzZzdCaU+0KEC8RCxZwzL/5+4EY/qfui5wW4kFOWZ7Ighu2L73+3wYquhZoYb1BItFLuy2HfdNuIGG161bABBEqp52jVKl0Yspg6uKGuqAY2XFGrJnoV2pIqfqjgbNqdMLhKvHw0ay+dfBgxP+Mhl07ByM1vorjKu6Jk0X00w04dPgCcwm4Fmns6fY/YJF/BX5bhkCUdXWLqN2FE+/zolLDAKybCcZKZBLnVOkqvJYmxjnGmYftkDZXj771Eep/U6wCsnPMaAlB9kl7dSqZf6JI4xxyvKjFtY1KelCb91mn1w4CLnLM2ExVZhcI5M+Y797rt669WCv5ckveEJQopy2kYUJ3h/KSjSImdWcXMsmgWQg6HIekzP0sfiL7BzI0LT8+bEOOaZYV8d8Y37ACsZN+rRG5T6cx9CRl4bS7NjUMO22dfOUJvXUoutV8Dlf6UTDOIeKKz7vfylxLCgLjs7mjo5ZTPFBWHGI05sb3N/0pct6gXu4BGg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's device sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index a959572a816f..221fb30cba51 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -508,6 +508,29 @@ static int tls_push_data(struct sock *sk, tls_append_frag(record, &zc_pfrag, copy); iter_offset.offset += copy; + } else if (copy && (flags & MSG_SPLICE_PAGES)) { + struct page_frag zc_pfrag; + struct page **pages = &zc_pfrag.page; + size_t off; + + rc = iov_iter_extract_pages(iter_offset.msg_iter, + &pages, copy, 1, 0, &off); + if (rc <= 0) { + if (rc == 0) + rc = -EIO; + goto handle_error; + } + copy = rc; + + if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { + iov_iter_revert(iter_offset.msg_iter, copy); + rc = -EIO; + goto handle_error; + } + + zc_pfrag.offset = off; + zc_pfrag.size = copy; + tls_append_frag(record, &zc_pfrag, copy); } else if (copy) { copy = min_t(size_t, copy, pfrag->size - pfrag->offset); @@ -571,6 +594,9 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) union tls_iter_offset iter; int rc; + if (!tls_ctx->zerocopy_sendfile) + msg->msg_flags &= ~MSG_SPLICE_PAGES; + mutex_lock(&tls_ctx->tx_lock); lock_sock(sk); From patchwork Wed May 31 12:45:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13262150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0935C77B7A for ; Wed, 31 May 2023 12:46:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 682338E000C; Wed, 31 May 2023 08:46:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 632A28E0002; Wed, 31 May 2023 08:46:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FA358E000C; Wed, 31 May 2023 08:46:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3F7378E0002 for ; Wed, 31 May 2023 08:46:01 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0207D1C6D79 for ; Wed, 31 May 2023 12:46:00 +0000 (UTC) X-FDA: 80850522522.18.84423FB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 80F831A0021 for ; Wed, 31 May 2023 12:45:58 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NdFnmZiS; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685537158; a=rsa-sha256; cv=none; b=ubpzkq4Z1MKpx/2RKJBBFLmUqYQbqDUVSMOCAxje/aiNYkBbgM8mb0EPfJW64oN3dU55iS EnyJNQ93NVAsSy74nZZ+0ciVa5lnZ+wA3WZ9mkxs9AGkvz57Gyx+ewZwG+MmVVmtjUSmeB 8dELrP/VJFqJztZHnEkk7j/r34Ndm4U= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NdFnmZiS; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685537158; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jiNOYIAVcjsuvSMtZGlGjCJlFKnRRxqCgJAQxTqAqH0=; b=R/3cFBG2HAh9DxFScqteNLDmTQ+ecdQYvrCWkWckPs8R07oMvN7+lrKvRktpFiDJBxXVCd pnfiZazvmBmP4Okcvx6IhOjoZ1bFXR8c1Ac8kN81og2Uv8C1jnOg+ZIHHEIt1zmd/aQk3J B5xtTd+VYX8f8VyI8g/FhfPGbzmJHOs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685537157; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiNOYIAVcjsuvSMtZGlGjCJlFKnRRxqCgJAQxTqAqH0=; b=NdFnmZiS/B7SLStdFjIV2J8FT+XhjLMjTiAkgTUvymqhlZE+6GbnC+oDNqwQcciq6krQ69 4AgUm0gyzXstfc25LJPQ6smeyZPB8IsbDSBhnmetmJ6R/KaWX1gPgC0OYs/tHwEz4EBK6A MMJUX1I8xk04rwNV2b7YcHymJ98CY3o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-198-cKdTb43SN-mmwECm3okGRg-1; Wed, 31 May 2023 08:45:54 -0400 X-MC-Unique: cKdTb43SN-mmwECm3okGRg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3AE7B8032E4; Wed, 31 May 2023 12:45:53 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2F12040C6EC4; Wed, 31 May 2023 12:45:51 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v2 6/6] tls/device: Convert tls_device_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 31 May 2023 13:45:28 +0100 Message-ID: <20230531124528.699123-7-dhowells@redhat.com> In-Reply-To: <20230531124528.699123-1-dhowells@redhat.com> References: <20230531124528.699123-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 80F831A0021 X-Stat-Signature: kkadskafdrds598dt48fi5szaz3aw4wx X-HE-Tag: 1685537158-87085 X-HE-Meta: U2FsdGVkX1+qgeOykpHXEbqr8vrAXWPWjpNzlivBdix5cU+q0wbb8+Vwm0DmZF0bLNLwcJ9Y8i78o+QYk/35XYDe40t3R7FxCCKgwFNsLJne3QtJHQWgGEbSMQHzZ6f3r2ap79dse3JhiY1Js5kdhEhrczAFuYlg0eXB69qBQGulinc/rQkkfSwyO3/6gv0AGQXaheo9wYJrtcSBH406c4pV4+Zby3qz8QenD90cKac3TX0DtM9eap0Xk/Ibkx+QxsgVC/2/z59kQlJpsf+4KLA7VB8ZWQp+trS/A3A0GQsDoB5yWUbgFe/IYrKGFCqBEt6dyI65fF7ur6ZGF3BT9dcNzLoeDSjKDvPL2PL6trJzctgnAW49oTc1ljiX0xEBN+GpAvQgdjQASguuFqWKg0IR4kSeUyFh/cia3yZns2UK4iS710UDFt4gzBWfN6Z+0crwvMt8Zw1bOaQe1Tc1lld4WFxAlwMgGCgbxX9qEASlfsd5MkSBeboQCSM3tNC9GFqFGiGiQcL5o6OQM0C5gOwpDIzxDm9SQUBSvNnOpDrEHjfJW3sMnBLjDZq8wN7E8bTVWixD7UHc9x4mzsbghPR+xvf0EeVoU7FKhJNzhA4TOwgtHQ3B4yoaq5P0e6yM2BIYc4AvKNJc0gVFdnX0pOT1ZUsEfdrw4k6m6czJjWaUl2TuHOv7f6k1nEuJTNW2n89t2iqgzYigaRWEoVo8XgZJzuiNbMZ/ePbq+XwEU4n45Kld91dQGKbDQkMFTGX0RoRjqlMkMZ8dkSyQLXiijBg1LDiyHHClBkC9nbCsxtfcsNRGi+mIboGiqTSFMrJiQerSIsbQS/jfslTci61S8EZkrXLFSa+Q7I+uVorJHxXOMgpDlQi1k5o+iebGHVYpTxb32XmjiG10Oh6P3OrbLMLyh9ogQhhhkRL/K5rYMtSlsUPZWIusAx3DMbhvqrPQvvpnfpytKPASvdchs6j CVwEAUFi Fq/ap8b+R97j0Ko96N7IXtJfpJSujvHSRyLdOes15nOCso/ujN6Ezws030YLgOUc8nFpaHZ1dKSiP3RQfkzKDxs6JrEGlqHX8+2Tuj8Ta6xuPz3CpkIuatkN6mxKLXfWw/B0TEIBF+v5EGaQ0LahfWPyKblvkPo1NEb/ehmRFGxz0KpvN7/jn7cAouoM2T8tjIRov591G48WBPiSKZ5SiONp4uoUQKu1ikm3bMpMVii5iamgjGlUd6mdGdjuBYNIocuSzmllqGSr0rikE5yzBjNhMtJla3KqCasFdQEj6boMOFnqvG7+jmuFUgJqijCbnKScDQsXi1bJ0S4DvHZQlcMU2WWxADgJPgHQOZ04TD/RAm0RhMo/D2xprhajLENUECTEdwti/cq9qe7apOED5tqmDYRYopruEkMNK0PBAy5BIA3AaHZh5KtgBtWInCDBa4r4s+/si1bCOyUnaon2pI+4lFiCiVuhFJQDV69jXyHm4z3Vf9PQFUKxcyOaGZ6MD6a4GZ1Lvkzu9AtRs2L5/yI80HVQP8UPMZizyscR7QPjV0M+c0u9jtr+XfQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_device_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. With that, the tls_iter_offset union is no longer necessary and can be replaced with an iov_iter pointer and the zc_page argument to tls_push_data() can also be removed. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 84 +++++++++++--------------------------------- 1 file changed, 20 insertions(+), 64 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 221fb30cba51..684171363dd9 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -422,16 +422,10 @@ static int tls_device_copy_data(void *addr, size_t bytes, struct iov_iter *i) return 0; } -union tls_iter_offset { - struct iov_iter *msg_iter; - int offset; -}; - static int tls_push_data(struct sock *sk, - union tls_iter_offset iter_offset, + struct iov_iter *iter, size_t size, int flags, - unsigned char record_type, - struct page *zc_page) + unsigned char record_type) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_prot_info *prot = &tls_ctx->prot_info; @@ -499,22 +493,13 @@ static int tls_push_data(struct sock *sk, record = ctx->open_record; copy = min_t(size_t, size, max_open_record_len - record->len); - if (copy && zc_page) { - struct page_frag zc_pfrag; - - zc_pfrag.page = zc_page; - zc_pfrag.offset = iter_offset.offset; - zc_pfrag.size = copy; - tls_append_frag(record, &zc_pfrag, copy); - - iter_offset.offset += copy; - } else if (copy && (flags & MSG_SPLICE_PAGES)) { + if (copy && (flags & MSG_SPLICE_PAGES)) { struct page_frag zc_pfrag; struct page **pages = &zc_pfrag.page; size_t off; - rc = iov_iter_extract_pages(iter_offset.msg_iter, - &pages, copy, 1, 0, &off); + rc = iov_iter_extract_pages(iter, &pages, + copy, 1, 0, &off); if (rc <= 0) { if (rc == 0) rc = -EIO; @@ -523,7 +508,7 @@ static int tls_push_data(struct sock *sk, copy = rc; if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { - iov_iter_revert(iter_offset.msg_iter, copy); + iov_iter_revert(iter, copy); rc = -EIO; goto handle_error; } @@ -536,7 +521,7 @@ static int tls_push_data(struct sock *sk, rc = tls_device_copy_data(page_address(pfrag->page) + pfrag->offset, copy, - iter_offset.msg_iter); + iter); if (rc) goto handle_error; tls_append_frag(record, pfrag, copy); @@ -591,7 +576,6 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { unsigned char record_type = TLS_RECORD_TYPE_DATA; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; int rc; if (!tls_ctx->zerocopy_sendfile) @@ -606,8 +590,8 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto out; } - iter.msg_iter = &msg->msg_iter; - rc = tls_push_data(sk, iter, size, msg->msg_flags, record_type, NULL); + rc = tls_push_data(sk, &msg->msg_iter, size, msg->msg_flags, + record_type); out: release_sock(sk); @@ -618,44 +602,18 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter_offset; - struct iov_iter msg_iter; - char *kaddr; - struct kvec iov; - int rc; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - mutex_lock(&tls_ctx->tx_lock); - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (flags & MSG_OOB) { - rc = -EOPNOTSUPP; - goto out; - } - - if (tls_ctx->zerocopy_sendfile) { - iter_offset.offset = offset; - rc = tls_push_data(sk, iter_offset, size, - flags, TLS_RECORD_TYPE_DATA, page); - goto out; - } - - kaddr = kmap(page); - iov.iov_base = kaddr + offset; - iov.iov_len = size; - iov_iter_kvec(&msg_iter, ITER_SOURCE, &iov, 1, size); - iter_offset.msg_iter = &msg_iter; - rc = tls_push_data(sk, iter_offset, size, flags, TLS_RECORD_TYPE_DATA, - NULL); - kunmap(page); + if (flags & MSG_OOB) + return -EOPNOTSUPP; -out: - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return rc; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_device_sendmsg(sk, &msg, size); } struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context, @@ -720,12 +678,10 @@ EXPORT_SYMBOL(tls_get_record); static int tls_device_push_pending_record(struct sock *sk, int flags) { - union tls_iter_offset iter; - struct iov_iter msg_iter; + struct iov_iter iter; - iov_iter_kvec(&msg_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &msg_iter; - return tls_push_data(sk, iter, 0, flags, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_kvec(&iter, ITER_SOURCE, NULL, 0, 0); + return tls_push_data(sk, &iter, 0, flags, TLS_RECORD_TYPE_DATA); } void tls_device_write_space(struct sock *sk, struct tls_context *ctx)