From patchwork Wed Jun 7 14:05:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87F49C7EE29 for ; Wed, 7 Jun 2023 14:30:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26AFB8E0002; Wed, 7 Jun 2023 10:30:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 21B328E0001; Wed, 7 Jun 2023 10:30:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10A3F8E0002; Wed, 7 Jun 2023 10:30:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0116F8E0001 for ; Wed, 7 Jun 2023 10:30:22 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A8C598027F for ; Wed, 7 Jun 2023 14:30:22 +0000 (UTC) X-FDA: 80876187084.01.3BE4B25 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 44ACCA0005 for ; Wed, 7 Jun 2023 14:30:18 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="bn/qOzkA"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148219; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=ZWDYDwvuxpsJcpT8CYojB9f+V7RuYt/qz0gEJv/92ubJ0kvDCArmJXRVSnSw8L/eJY4W7s c9a/vEh7VSDZEGkDXwl4+pMXW2a2bj7q7tG3fjg3gTxbOH5ZmP2hCAWwOCYDMiUV5vf1H2 7FEyQqTnoCsuLZeQVlgEPg9RaaPb8jE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="bn/qOzkA"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148219; a=rsa-sha256; cv=none; b=ehIRYPJqIESTjLrwBGaH7yTFY4sH0t+O5VcTJqP2blTacJPzBnxz58Sh2INhc4QFaCNaAK gSEZyGYyFpwPgLzlhR2hs8Rynn2LfHCisf1AYv+7QMefelGnVh+hksNd3xOcjFfH5FOfdL jb5SganoZRn5bz4GCI1NqUyZSVvZmds= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=bn/qOzkApdQM/LWC9c1oeeDg0dgq6ElOFl4Di0rErVEsdObvHdvKaa9iA9TRtMLb7pDzpV StYMQY/GpcoOA9MGv1T1gMibxQUZS7Ktd4AUnJQDnZhrKPzsLXsc5sbBfY+QraGjf++hkD U6s6P6QqgSGbj9Z9jGZHrK+D9IgXwTc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-635-TJH48CwcMEypUtBg4mC4JQ-1; Wed, 07 Jun 2023 10:06:10 -0400 X-MC-Unique: TJH48CwcMEypUtBg4mC4JQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E0D86800969; Wed, 7 Jun 2023 14:06:09 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id EE97F2166B25; Wed, 7 Jun 2023 14:06:05 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 01/14] net: Block MSG_SENDPAGE_* from being passed to sendmsg() by userspace Date: Wed, 7 Jun 2023 15:05:46 +0100 Message-ID: <20230607140559.2263470-2-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 44ACCA0005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: dw54ahzodgqfsoxp411aiscm83wu4ioq X-HE-Tag: 1686148218-43425 X-HE-Meta: U2FsdGVkX1/Lsq3wvFw6ydqvZmdbbP4yObHzRk6mORzvtSwe6tXAqYEZFeF0lOG8p0m8VStACZlg9LxG1VtW957A5LJIV8eJde0XZvLhRRpOd+2E+/2XaV80I7JMhzGSxwPKNr4AwvGk2t0/JeMOJiJDHvsNQ8Bx6geyH41014eTCySDbHRz1PWI1DIBqwF29VctP5lRG5DKTROTw9yNsJN+VhWDHePgEjXhxs+COcVQHdbYN7w2YjYZRNhdCY03CbOorWdxlxr2fu/8a5Egg1YKq28Yx01H5wgE6Z2Fv4K7LCPP7gpmRM7Q2vv1gO4eg3nhz4WAv6wEbDmzpkGJ+0bi/mPfroCKsVN4/lHjzrxsZsht30YHaxtBgqsIvWqJwCjeB8PR9/6KEget6pgg8IdbvPUyRbZLzi7apxpFKFu8kt0tqZeerdkGt67+CirIB9Rn1zd6c0mL4v59N3FswO5QFeaXUWSxVIMFIR75R6VJn/4MHuDmxLkvRE/SGQc5I6/8NoqfGYkyuLTkvUvHF7q0qTw0lZfeDKz70ky68NwOxHF4xlx7v3i5sKpaIJqb0MZaoAWOH6qKUlywmokIbkwVB4KPvA7fYJHAvTrGHXjiZyY1I5S+17Dv+zVu0AZLChdHw6YU719Cdr+foBkWlasTOc81XfzACZ0dDKDeMFe0YBg465buEApdELqA5hwQq9oybfBepAqmImAu2mrUecEvffM/Nxmy9fQLbCQPzaEIwJOiuKe6tsJYqZ7hwWAOrSK1DBYjfqlZBA9vwBy04SAhPacEowUwR4HqHXbn+3RIer96B18NZQ8ZXJmDga8rNAnR9w3HIdIrL1Mc72iJpypdP/kpnCjGNDWTYT4irzcYUDu7V98xoqI6NejDM0E+Q53Xr1TKq2xsxvhMkhMpgIXctBACGVUzrtPQhRGYklBpXPUQ1Mt0SWW2hREsByrUH15l8I4SM8sCidg06Dh A/K4SXQ/ Qxf9Y+ebXTZfDfnwDxtFOE4NSsBUpWfdKR85dOy/b09xxrJSmwRfQclta07JVejtFbXG225ipaIti53fB0N/0Xt7Rpqpy56KB2yD0TF56P3X1EJdJB8VWjPP+1DywzLF6rtpVvk1U7iChXtTsss37d3B+O7ZdrzaAg4wHm83UJaiAuBHfv5r4wctV7g3OpTfbhKLbX01KWdUdMOG8jEGhoMKiujvlQgrEW2yeFhy7zznpvINP33z+5ASp01+lZF9jClxKnVqOm4erwcj1U24+SchIjjtxorjJ8U/w5MR6+eu9bUXjicCNrdCXEqI7Fkk8lxptUbri+X1ryuzk4Rq8iWaz7OJ/3OEv+Uz/aPeC0Tc5X6ATVQjEdDw9kKM/W02YSUFxml2//ZDc5bwzMES2G0QvKfK+8wdqYdTDlnRlfx3R+76W/C4WZn5gC9A3lNi/qsrtpdix2Lt9VVKi/hWg3y/pknOrxrq8WacwS8Q0vjqlvZqSWWWGI1MF9aTr4qGnIR0K0HK1efkwUxP6V9+A63uK/7dXfPu5i/3NO4nAc9tEUjvk778xCuBiWg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is necessary to allow MSG_SENDPAGE_* to be passed into ->sendmsg() to allow sendmsg(MSG_SPLICE_PAGES) to replace ->sendpage(). Unblocking them in the network protocol, however, allows these flags to be passed in by userspace too[1]. Fix this by marking MSG_SENDPAGE_NOPOLICY, MSG_SENDPAGE_NOTLAST and MSG_SENDPAGE_DECRYPTED as internal flags, which causes sendmsg() to object if they are passed to sendmsg() by userspace. Network protocol ->sendmsg() implementations can then allow them through. Note that it should be possible to remove MSG_SENDPAGE_NOTLAST once sendpage is removed as a whole slew of pages will be passed in in one go by splice through sendmsg, with MSG_MORE being set if it has more data waiting in the pipe. Signed-off-by: David Howells cc: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Link: https://lore.kernel.org/r/20230526181338.03a99016@kernel.org/ [1] --- include/linux/socket.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index bd1cc3238851..3fd3436bc09f 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -339,7 +339,9 @@ struct ucred { #endif /* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ -#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) +#define MSG_INTERNAL_SENDMSG_FLAGS \ + (MSG_SPLICE_PAGES | MSG_SENDPAGE_NOPOLICY | MSG_SENDPAGE_NOTLAST | \ + MSG_SENDPAGE_DECRYPTED) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 From patchwork Wed Jun 7 14:05:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF404C7EE25 for ; Wed, 7 Jun 2023 14:43:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F52D8E0005; Wed, 7 Jun 2023 10:43:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A51D8E0001; Wed, 7 Jun 2023 10:43:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 794C88E0005; Wed, 7 Jun 2023 10:43:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 64EED8E0001 for ; Wed, 7 Jun 2023 10:43:56 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E1783AEBC1 for ; Wed, 7 Jun 2023 14:43:55 +0000 (UTC) X-FDA: 80876221230.26.7690677 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id CDC41120027 for ; Wed, 7 Jun 2023 14:43:53 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cYa92MoQ; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686149034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oJUkpOQ7e9hBKCJRUR3yZoWIoUWj9MtOId6LGPzA6h0=; b=PjcBJ7IIvVppvocwsW08gMJ291QIO8ZvgtdLmKMcAMi3hYV9tExjhSczsGHWK4mJaKufRe 2ku5zxD+R3F7lmTSzvOJgaTA5S+G95mZWIIGWQoEjZt4CpO26tVB8qlgJ707R2Z/tHO9Sh tWd0sSK1pDyPYJwOa3qbxsN8I3mbz7o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686149034; a=rsa-sha256; cv=none; b=zpOlQRECSANCGFmPdS+wcSttPXNBXUZnzwMbB6X+gM9uHB4MHJEaxq3gY6pSthBi/tyxA7 08CbSGjVjO74VWSbXy5DBeUvXNGd8OXViOrBQp30OTezFWW73QQ3w/5yg178E9Gk68y1IZ EUx2EL7GrLpZA7OYyNJsyT240A37Vbc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cYa92MoQ; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686149032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oJUkpOQ7e9hBKCJRUR3yZoWIoUWj9MtOId6LGPzA6h0=; b=cYa92MoQ8h4gMYAqT1/7KhoW4QxyMnREu3fdX+5s9bWjLI3IlY7B1OZSNzT8/LtrvAwoIg fi+CJvs/RrrvqeZmM7JqDx9V7VKwITTdOseklWAyslpRsYaoJqsWGN5R4luO4HsWhX0TNK cF5W9s+XdHvqxyEn/YsVJ/CHTIrOry0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-529-okqwxf2jM56vi69Z88OO0A-1; Wed, 07 Jun 2023 10:06:14 -0400 X-MC-Unique: okqwxf2jM56vi69Z88OO0A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCD9080027F; Wed, 7 Jun 2023 14:06:13 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id EBE75140E955; Wed, 7 Jun 2023 14:06:10 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 02/14] tls: Allow MSG_SPLICE_PAGES but treat it as normal sendmsg Date: Wed, 7 Jun 2023 15:05:47 +0100 Message-ID: <20230607140559.2263470-3-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Stat-Signature: 33xn4g85o1w87yxcgstnpqfqqhyy1k34 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CDC41120027 X-Rspam-User: X-HE-Tag: 1686149033-11239 X-HE-Meta: U2FsdGVkX1/YnOhga2RtzABfgo1uk19cAd8uV6/20FAqC5E0PtLrQJ1xvz36n3nvgWM6YYHrI4o8junjQW6Lu+oYE3/qnKd0L4v9CZj6FCXg0Zc6wATd2FHorT/aZaU1TkO/xUroD49+b44ArKJVsx+SzhvlwIvr/IOylKmy8kChvI9oCN+AKsnqyZff7Cx5EZjDTCArXNNBoKToiQ/x8U2ZvxRTiUnz3v/WW5VNM47VV+ojRVGMXaoBQfdBHEYQj2dG+C16DvG4M36MuTHHwx/7r/VrlrsSNTgI3AU6KP8M7f/Zo9t7gC9l6tNtBTdLEFfg824P8u3pEOc28WZrGuj4n6Hk6YwROQaVpeCORjQyEPQJilcLmzmTbd9wt2u/pA5U8/JTTsMHlF2DRow1YIDnWNFNhe6izUa6Xr6BkMT3uMss37gJGtGJcEdVQB4vxrOzbYPmECgrJKNLin55Dma8Y02koKyyN99oJyJ8+IPhtqWPZ3NtDVLtHmJSbcnzMhNdEIISJNGYnyy8tNjoI5s4nci9Nye8zKa2XUXfzYqqUzfq3Wb0+LwocDTClYoK/hNg1XMuL9mxQyE7d17bB2NHwokz1CPNxBPvbkVC08kH4pu2tw+zcvBE580nPGuVUs+yWtSFbY5Zu7sYykaoGWo/BeewEqKpAcCiMZwKCYjwVqjm+JszCVK4fTd4XTU4tjVx5ZX3WqQEziD0yf/XcZCn94O6Q1WuS8CCIU+P2I//wjjSbPAUwKl9NjHRi7G1AGzyJ6iH5xJcurpRe93F3TQ+COofOWubcFx+jiGrsKEg1a1UO4uILBlIlY3CkyqPpvHryp6Ev/cEWec+3zE7kNzWjLj02+DdwvGvkv51ZxoyQBchNGCoROQmzOEG6dtQcIm9nln0p27Kp4tXQPnVcLZBJZGQPsyYgAsQOel4EaHytD98MiaTLsuEBfoxv7nOO04MzYCjlG2TwpEusv+ f5QBa3P5 W0wmKLJW5ejtq1VgJAshp59xzP/h3MzA5nKUNnl06yIXJIcVyY/sm8qha42HayBa474BFkTuRWair4Zmte7EVtsBYxymgMW4mKi2BC6oFJSB+8liQ4kA8npW61xlv6c1hEW+ANpGLNjP+PdtR7ah5DVCVzYPvMN4mlxFJCI90C2uQlCan/UgtGYofj9bE1Sb/CGcRWreGv8e9uZsil9c5sFD93qklSUG3ULBqVPPnF0Dv+63G1B1vBfLadqCah0DoPaCSgKxsXRfCIQvsF/Z94QlTSvF39JX8P+/0sV5UyuAQU8rHg2lX37T7eCOoedQOYZgWlNC97dzlXLq+hZYZoISmHrJ/PWk5FRlmyGjezIC2F5HYUBqcovIxtLCWea1Tv0YsNoiMIdyp+Rm62sij1P6dvK2ZwR8lEgw0VW8ozxPZ9wV4v06Mm5NYkw0osVPW2rUYHF/zAp8DnpO0EsgOWLd88MeYimjAb9V+ZGf5cMiPBXNxyX6DMydvj0tB4MidwXO9ZhCX8rgZi2ptDrAadad1/EhNIb+qwF/i6dYLgUtW42yhIzbY1f82oA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow MSG_SPLICE_PAGES to be specified to sendmsg() but treat it as normal sendmsg for now. This means the data will just be copied until MSG_SPLICE_PAGES is handled. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 3 ++- net/tls/tls_sw.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index a959572a816f..9ef766e41c7a 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -447,7 +447,8 @@ static int tls_push_data(struct sock *sk, long timeo; if (flags & - ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST)) + ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | + MSG_SPLICE_PAGES)) return -EOPNOTSUPP; if (unlikely(sk->sk_err)) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 1a53c8f481e9..38acc27a0dd0 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -955,7 +955,7 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int pending; if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | - MSG_CMSG_COMPAT)) + MSG_CMSG_COMPAT | MSG_SPLICE_PAGES)) return -EOPNOTSUPP; ret = mutex_lock_interruptible(&tls_ctx->tx_lock); From patchwork Wed Jun 7 14:05:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6161FC7EE23 for ; Wed, 7 Jun 2023 14:34:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01F778E0003; Wed, 7 Jun 2023 10:34:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F11888E0001; Wed, 7 Jun 2023 10:34:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB4938E0003; Wed, 7 Jun 2023 10:34:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C50328E0001 for ; Wed, 7 Jun 2023 10:34:05 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9640F1C741B for ; Wed, 7 Jun 2023 14:34:05 +0000 (UTC) X-FDA: 80876196450.12.0BCFCFB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 441F218001D for ; Wed, 7 Jun 2023 14:34:03 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Eqnm3iFT; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148443; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=giSsyvw65+wGBvy08mokv6RazuqXgtGNOU7L/nVYPJE=; b=H8B22me7sxFamxVLDodaro8/dOqnmFD+WU2EBUPvp103plaC0954+queOmQg9MZa8YNZHU ugPiKJZzZjQU6mZUSFFFwHXGWE9yIh83cgcEeASN5NVTUkQFj9YLhS5+1OvuL2vLUaDqcl zBSj7B+DMu9os8gyyF8sthPzpQ0pqWw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Eqnm3iFT; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148443; a=rsa-sha256; cv=none; b=4ouSPsL+XZlXWkNnTW5izojE0cQ0hvgGyVcXrhsa285qzb/tgqHGUmnLwzGyXrS/LH9FPc PBFJr/+KwHnA5pWb1XSj6HKFX2nDSbpZD2gL2Ihqi0dmA8pL8GVB0BPZ5wdwaVN/abYl3j Od7muVnozFBRJsspL1mCtWUJMSrxwNA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148338; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=giSsyvw65+wGBvy08mokv6RazuqXgtGNOU7L/nVYPJE=; b=Eqnm3iFTf1wmxuIKahIk+5oRZmeiUPgmKrmCeudW8eissRJC6Vnqucx0GA7F+1QZqbT+iv asPJZUPOMasQZDwExCSN5hSaMx/kBkgZl1pR/yX7mluniCWwxblov6dWEAjCO5e37rdioY gly3RJgfnJL6BsXnwZqKFRt6PmzvaWc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-224-TqvFM-tAOYeZSJyt06-g_w-1; Wed, 07 Jun 2023 10:06:18 -0400 X-MC-Unique: TqvFM-tAOYeZSJyt06-g_w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B2C19185A791; Wed, 7 Jun 2023 14:06:17 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id C013D40D1B66; Wed, 7 Jun 2023 14:06:14 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 03/14] splice, net: Use sendmsg(MSG_SPLICE_PAGES) rather than ->sendpage() Date: Wed, 7 Jun 2023 15:05:48 +0100 Message-ID: <20230607140559.2263470-4-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 441F218001D X-Stat-Signature: cf33e1o8w17y9g8iqyhsmjxrnrtmooko X-HE-Tag: 1686148443-471137 X-HE-Meta: U2FsdGVkX197t5/WzY7Vmvcasw5hjrbnKiNco786TvrNrltLu0eooa2m34xAbgko0NfkVmvc/KJJktE0eUHDzYBTjeWAeoRzIyQZwFaJUfcXICaK74jAOU8XUveA41NMTzSFKcbaJQjmcH9652f+v80PhHQZFviaEs5lKwyMynZDzrd+K3sx5o/6Uvzn3Rmwgnln5kvJFLrXv9WpqReJJp2l6leHBNI2taRx6Swxacvs/xmdQ2vk6XQcRkoUwcCqHDr7vTf03VSg4UMTjdroOTjBqTG0xnzecCeegID0k6NDzuWaUOXG56+XH3A4V7Zd+r8jUmTUz2ReFNEHWJ6z3Tc+BsnJf0P3THljv8bMM4nS/qS47kDVKeblKxfZa+doBpqr52NN+krsKWm19u20SeVil9ZM8HH7DHFLf/QXDw4Xrhj0FDU/sXgx36O6eh0pX4cKW3jdO7FVsFSxLChcMGGj68fpXV9wDUNJHJKEX/FDavdKIDU0qFF0Bx915E/3aShoBQSjB2P1f7UIrFMOhiJ7LC6kzVb6Wm/uY2sc6p2S/pfvGikuSmR/IaHcRz3x0AMYZVdfrmy3xLIEll0E+5qlyI6dGRBwxw4FGhLkOQ7TFhhf7Mi6IDLiaNKgf14OPms0vGPPaz/8zKdYOn4pCxmy+gmqw/cuWtWyuSGR+FU3vm/LVFoecEyFDN/05Y1FJIxEvv3vZt+J6iuURCEE1V60CAu6Y8RHiO44DlszGTbjJRNIjz1wQskMrNO4SdLzi5VdxjnJL/OHZa9RRPCMDAGJ3wirqVBycJKMQE0bPaa41YaDf2SAEkze1gDiGzqGwQ8H9LTT5MgjLNxb+Ot7ILSKtuD0qLxxhXwkuPhJK8TpE63rQODJkzrAxEf85tBTinXz5oNts+c8nYxM66tflniY53PqVTJBa4Ac77/XOPKxzxt+sVsram94iwAn7rIDcyD1e0aGwKvWQFcpCrf nwC1/Q6m LQH2E5WQb1y+ss4QjP0dTgiv31nef1fdOI2UrLLW2aG6aHMtJRPlYlntxdYZHLEjNVijgujsBX/ihdAff6yZzgMCGYgMOA/KbB7Ghp/L42mdms/WB3oVf0gmUB30jcqBpu18qqMBy157JnXHGK/pEJOz4zmIagCAGlOF5Vayu35VYS42/OlDZWQx0ALLEqgf8yCVn/VvdhqGnvxjZxBVeeWyR0BAlxoTkWATnPT0NAFFEqoyGyp9sukQDjAcobE7F9pqeoR8vkTCzJ12WY28AYHHKZ+71W2yvQOetJKOtR3wel0mmsoFr9I2tSSRE3az+Tm+yYXmG2A7fQ4MmXrrACkEDHwvp1Pe8eUWTlOtr3FPubuOLoT9tffcn8d5yFrfVSuCOlUON3Bm06D1Ker195azXB5Z3+wT5QH2hJG8zpWxkeiI4rFPIw9cCXB8zdilxa/e6ZlSwma0ZDctV+zpG5XCWAK04qzF5te8/BGIYIVtu86Vu2HCu2/6nBA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace generic_splice_sendpage() + splice_from_pipe + pipe_to_sendpage() with a net-specific handler, splice_to_socket(), that calls sendmsg() with MSG_SPLICE_PAGES set instead of calling ->sendpage(). MSG_MORE is used to indicate if the sendmsg() is expected to be followed with more data. This allows multiple pipe-buffer pages to be passed in a single call in a BVEC iterator, allowing the processing to be pushed down to a loop in the protocol driver. This helps pave the way for passing multipage folios down too. Protocols that haven't been converted to handle MSG_SPLICE_PAGES yet should just ignore it and do a normal sendmsg() for now - although that may be a bit slower as it may copy everything. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Reviewed-by: Jakub Kicinski --- Notes: ver #5) - Preclear ret just in case len is 0. fs/splice.c | 158 +++++++++++++++++++++++++++++++++-------- include/linux/fs.h | 2 - include/linux/splice.h | 2 + net/socket.c | 26 +------ 4 files changed, 131 insertions(+), 57 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 3e06611d19ae..e337630aed64 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -448,30 +449,6 @@ const struct pipe_buf_operations nosteal_pipe_buf_ops = { }; EXPORT_SYMBOL(nosteal_pipe_buf_ops); -/* - * Send 'sd->len' bytes to socket from 'sd->file' at position 'sd->pos' - * using sendpage(). Return the number of bytes sent. - */ -static int pipe_to_sendpage(struct pipe_inode_info *pipe, - struct pipe_buffer *buf, struct splice_desc *sd) -{ - struct file *file = sd->u.file; - loff_t pos = sd->pos; - int more; - - if (!likely(file->f_op->sendpage)) - return -EINVAL; - - more = (sd->flags & SPLICE_F_MORE) ? MSG_MORE : 0; - - if (sd->len < sd->total_len && - pipe_occupancy(pipe->head, pipe->tail) > 1) - more |= MSG_SENDPAGE_NOTLAST; - - return file->f_op->sendpage(file, buf->page, buf->offset, - sd->len, &pos, more); -} - static void wakeup_pipe_writers(struct pipe_inode_info *pipe) { smp_mb(); @@ -652,7 +629,7 @@ static void splice_from_pipe_end(struct pipe_inode_info *pipe, struct splice_des * Description: * This function does little more than loop over the pipe and call * @actor to do the actual moving of a single struct pipe_buffer to - * the desired destination. See pipe_to_file, pipe_to_sendpage, or + * the desired destination. See pipe_to_file, pipe_to_sendmsg, or * pipe_to_user. * */ @@ -833,8 +810,9 @@ iter_file_splice_write(struct pipe_inode_info *pipe, struct file *out, EXPORT_SYMBOL(iter_file_splice_write); +#ifdef CONFIG_NET /** - * generic_splice_sendpage - splice data from a pipe to a socket + * splice_to_socket - splice data from a pipe to a socket * @pipe: pipe to splice from * @out: socket to write to * @ppos: position in @out @@ -846,13 +824,131 @@ EXPORT_SYMBOL(iter_file_splice_write); * is involved. * */ -ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe, struct file *out, - loff_t *ppos, size_t len, unsigned int flags) +ssize_t splice_to_socket(struct pipe_inode_info *pipe, struct file *out, + loff_t *ppos, size_t len, unsigned int flags) { - return splice_from_pipe(pipe, out, ppos, len, flags, pipe_to_sendpage); -} + struct socket *sock = sock_from_file(out); + struct bio_vec bvec[16]; + struct msghdr msg = {}; + ssize_t ret = 0; + size_t spliced = 0; + bool need_wakeup = false; + + pipe_lock(pipe); + + while (len > 0) { + unsigned int head, tail, mask, bc = 0; + size_t remain = len; + + /* + * Check for signal early to make process killable when there + * are always buffers available + */ + ret = -ERESTARTSYS; + if (signal_pending(current)) + break; -EXPORT_SYMBOL(generic_splice_sendpage); + while (pipe_empty(pipe->head, pipe->tail)) { + ret = 0; + if (!pipe->writers) + goto out; + + if (spliced) + goto out; + + ret = -EAGAIN; + if (flags & SPLICE_F_NONBLOCK) + goto out; + + ret = -ERESTARTSYS; + if (signal_pending(current)) + goto out; + + if (need_wakeup) { + wakeup_pipe_writers(pipe); + need_wakeup = false; + } + + pipe_wait_readable(pipe); + } + + head = pipe->head; + tail = pipe->tail; + mask = pipe->ring_size - 1; + + while (!pipe_empty(head, tail)) { + struct pipe_buffer *buf = &pipe->bufs[tail & mask]; + size_t seg; + + if (!buf->len) { + tail++; + continue; + } + + seg = min_t(size_t, remain, buf->len); + seg = min_t(size_t, seg, PAGE_SIZE); + + ret = pipe_buf_confirm(pipe, buf); + if (unlikely(ret)) { + if (ret == -ENODATA) + ret = 0; + break; + } + + bvec_set_page(&bvec[bc++], buf->page, seg, buf->offset); + remain -= seg; + if (seg >= buf->len) + tail++; + if (bc >= ARRAY_SIZE(bvec)) + break; + } + + if (!bc) + break; + + msg.msg_flags = MSG_SPLICE_PAGES; + if (flags & SPLICE_F_MORE) + msg.msg_flags |= MSG_MORE; + if (remain && pipe_occupancy(pipe->head, tail) > 0) + msg.msg_flags |= MSG_MORE; + + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, bvec, bc, + len - remain); + ret = sock_sendmsg(sock, &msg); + if (ret <= 0) + break; + + spliced += ret; + len -= ret; + tail = pipe->tail; + while (ret > 0) { + struct pipe_buffer *buf = &pipe->bufs[tail & mask]; + size_t seg = min_t(size_t, ret, buf->len); + + buf->offset += seg; + buf->len -= seg; + ret -= seg; + + if (!buf->len) { + pipe_buf_release(pipe, buf); + tail++; + } + } + + if (tail != pipe->tail) { + pipe->tail = tail; + if (pipe->files) + need_wakeup = true; + } + } + +out: + pipe_unlock(pipe); + if (need_wakeup) + wakeup_pipe_writers(pipe); + return spliced ?: ret; +} +#endif static int warn_unsupported(struct file *file, const char *op) { diff --git a/include/linux/fs.h b/include/linux/fs.h index 133f0640fb24..df92f4b3d122 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2759,8 +2759,6 @@ extern ssize_t generic_file_splice_read(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int); extern ssize_t iter_file_splice_write(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int); -extern ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe, - struct file *out, loff_t *, size_t len, unsigned int flags); extern long do_splice_direct(struct file *in, loff_t *ppos, struct file *out, loff_t *opos, size_t len, unsigned int flags); diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..991ae318b6eb 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -84,6 +84,8 @@ extern long do_splice(struct file *in, loff_t *off_in, extern long do_tee(struct file *in, struct file *out, size_t len, unsigned int flags); +extern ssize_t splice_to_socket(struct pipe_inode_info *pipe, struct file *out, + loff_t *ppos, size_t len, unsigned int flags); /* * for dynamic pipe sizing diff --git a/net/socket.c b/net/socket.c index 3df96e9ba4e2..c4d9104418c8 100644 --- a/net/socket.c +++ b/net/socket.c @@ -57,6 +57,7 @@ #include #include #include +#include #include #include #include @@ -126,8 +127,6 @@ static long compat_sock_ioctl(struct file *file, unsigned int cmd, unsigned long arg); #endif static int sock_fasync(int fd, struct file *filp, int on); -static ssize_t sock_sendpage(struct file *file, struct page *page, - int offset, size_t size, loff_t *ppos, int more); static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); @@ -162,8 +161,7 @@ static const struct file_operations socket_file_ops = { .mmap = sock_mmap, .release = sock_close, .fasync = sock_fasync, - .sendpage = sock_sendpage, - .splice_write = generic_splice_sendpage, + .splice_write = splice_to_socket, .splice_read = sock_splice_read, .show_fdinfo = sock_show_fdinfo, }; @@ -1066,26 +1064,6 @@ int kernel_recvmsg(struct socket *sock, struct msghdr *msg, } EXPORT_SYMBOL(kernel_recvmsg); -static ssize_t sock_sendpage(struct file *file, struct page *page, - int offset, size_t size, loff_t *ppos, int more) -{ - struct socket *sock; - int flags; - int ret; - - sock = file->private_data; - - flags = (file->f_flags & O_NONBLOCK) ? MSG_DONTWAIT : 0; - /* more is a combination of MSG_MORE and MSG_SENDPAGE_NOTLAST */ - flags |= more; - - ret = kernel_sendpage(sock, page, offset, size, flags); - - if (trace_sock_send_length_enabled()) - call_trace_sock_send_length(sock->sk, ret, 0); - return ret; -} - static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) From patchwork Wed Jun 7 14:05:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270808 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C609C7EE25 for ; Wed, 7 Jun 2023 14:44:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 990F28E0005; Wed, 7 Jun 2023 10:44:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 942078E0001; Wed, 7 Jun 2023 10:44:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 809658E0005; Wed, 7 Jun 2023 10:44:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 718998E0001 for ; Wed, 7 Jun 2023 10:44:49 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 43C6FAECE7 for ; Wed, 7 Jun 2023 14:44:49 +0000 (UTC) X-FDA: 80876223498.08.D3EC383 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 668D9A001B for ; Wed, 7 Jun 2023 14:44:47 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Z/ctT0Rg"; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686149087; a=rsa-sha256; cv=none; b=ofNUBg1IpiiAw9QrWUMZXTW6UCfRhJM31NvWsOuGAux9A9i0rO6KGbWg7lIWJWHWc8nxz4 s3lfJ6uq4rlZvqJuZYGhtyyX9DmLjVuTVMTucYkIEmACsQn4waw/fnsiDPFIT/n87Knu6c LMwXPyTvHdRh8vFAK0AULV1dIT5FwTQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Z/ctT0Rg"; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686149087; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wDx0kX7sOlYmDcZ3N4Vvwkr41hWgtuybWy4Cs3UPj0U=; b=WfQZ2hJ8Q3qyJ/AfzYeHJFKVMDo9DSlClKbF70ViGET7Kl+rDhlURRE+jXI0UX/PaqrLKX U4nvl4XZ3F9HuU2oR9sfmcVgAp/p9Ya1HHwXHegVkONUm2U8qFBBEYGvvleVLIMLetL8B7 bGSnrYxf4ocaUhvayt3DV0klXp2C4Pg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686149086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wDx0kX7sOlYmDcZ3N4Vvwkr41hWgtuybWy4Cs3UPj0U=; b=Z/ctT0RgmKe2tkud9+m1Z9Qxb2mowzx7VEDajhW/qs34IaFclKVteL9PElbUBEauTh0X1x g4xU+XnLIK150J7YDdoIAVSHY/YtR8lcMLMRwgEhn7jGGuJpyvoJHvyZgORsRnMAqY7iSP acC69h+GilkAvb4la20u1KJfUsgGbro= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-524-qtl0EMkPNmyhAFISY1XUnw-1; Wed, 07 Jun 2023 10:06:24 -0400 X-MC-Unique: qtl0EMkPNmyhAFISY1XUnw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EA032101A53A; Wed, 7 Jun 2023 14:06:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0095140E955; Wed, 7 Jun 2023 14:06:18 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Al Viro , Jan Kara , Jeff Layton , David Hildenbrand , Christian Brauner , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH net-next v5 04/14] splice, net: Add a splice_eof op to file-ops and socket-ops Date: Wed, 7 Jun 2023 15:05:49 +0100 Message-ID: <20230607140559.2263470-5-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 668D9A001B X-Stat-Signature: s6nnbgwhajanky7hwai18wikdkjyajjw X-Rspam-User: X-HE-Tag: 1686149087-394945 X-HE-Meta: U2FsdGVkX19xGc+DqHWjyaxYQSqaXs9KWdh8R5YWknivRjjZw+Z1YJ2FUAlJHuzelvcMFhc0POhE7DG4mzjnA2nTJvYAcJTocXvOPIE2N/19dUrGv0+sW0JxZ0Bp9dn/a8t08HCfXxwxSO94gzwctDztptnT92fAauJ0iBgjy9LX9VF4DsQXkTbvA+1BSWzMys6CTlVBKz/crvITaVt+epO17BmRPGPTbNfYJpGkUU59CXxwy45cSMgSQg4i4ao3/JgOE4E4FRNINX3DhSPRMebVmRYn7W+f12fQ7UpFHar9gtIRCVBUgqA8U04uDnMGsGbJUo+AXyV3cC6gtgb1WbXJOVmtcTOFYppV5A2P0u/70QCVKNElPM6GuIz6zEfQv/3m7yPIHvCi9ZGbANnKOMHq/8sRa2tICvw7Ii8RAUoryeOE4+bmeeYqUvkfgJ/Z0NytsDVdqFFwrr4ACQUw2/j1QBj2NW7fDS7Kce396NB9p6YmsA1B6lifWUII2iFnRbPqoMs/cKJjA7Sw/j7iMSsxfNzc70UerN3tcByen5hW+HI4Lm0iZKgqJdK2YxhTU9tXTXPFv0KDJXXaSiUd4MOpCkvtIs9HmtK63LbMCmBjg+IXw2k9Nvnowq4w9JQslQIQHa7PDuM7guPh0TenBixycIy8XvbQScYUz/D/2k6sfZW6KcKyW+2gjoFrdTDXvzj9Ji9OKy68fdnWn9+92lfsjYWC/Aku3zkS9ugQbU4DozMSn98BvRnJISFVy+e5f/Ju3vv1fXlZMKDA3y6OfmsZvoSHRKGla02dEck8VnyKsC+tJejXrXxY7hkPzC+Np83fh0JguE9J4HreDIR5qzy5vy47E6yfP/b454+yrFyFrMU23WfxmOuQvE6XskyxBb28QlIZQPqVbDu+6RNNjyo7QhAz14Hqn7V7zkOinznDfKS/vHlqQ3+d4h1s10d9ADPiQcyl46DoGFRHW/p Jl4Ufzlo HbM5ahq4MEM9iTjrIMPRn0RTNYmvmn6qQrPgiYy5AA1hrJ3iZjoF9A3Y0aimDanghQ7I80R6biOoj3mmoLDurP6Q6PYBvEEd8K62ywKQUiGo/igC4ivDQXA2ynRyAoVeFrtgTGHjuyAWPl20gnuscoykVjxlrFo13V9Iu3gWTHB6bKpJyC89i0PhcvaCJs547qIFY7lf/3Sww5mDKzol5teXvMUuZwIYN1SltXA/5bV4B7O7N8oUmkEJITGpb9mL65ys/uuoBOqzKUCMJhp45o3BERTVHV0Mv1MmJikN9ZRZLiYr/Lk9/g6A5HZtLXMsRlRRYLDLY1CgLswmDcsxsb93/E5palMPWM24+GTYdlofUuuYK82/zyipu/PHJeXj2tJ7MEo76VmQjfWeHVS/VYvTdTqQwy7tOihnCVE40Uk6UGpv+Jo4vQBMqGWAiJy75uu5GSy5uPHrz96ZBVOAyxk1b8y6InpOyoeTInMChLAoEv9LJc83df47Lbi7eFebl/9oFnaNOnkxrClCah8ayLqpaEbvRz0mZQ1/F5oJm48N5W2+FwMN4Y2RUNm7Czj8pvXBZaShkGE206pcatrHOmxwI6bgU1NAOmEq9U+lNqeUt4iGobcPzuAUGCl9HM2A10mPrV8+6X7l4G805vc3bJAep82XvgFtVHGqyxPb/9PM8jfFxYzcWb+zPIboI8v6TnDycP66DZcNbxU9Y65GZ0PnQ01QyautdVuwHTqbD4f2iSQg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add an optional method, ->splice_eof(), to allow splice to indicate the premature termination of a splice to struct file_operations and struct proto_ops. This is called if sendfile() or splice() encounters all of the following conditions inside splice_direct_to_actor(): (1) the user did not set SPLICE_F_MORE (splice only), and (2) an EOF condition occurred (->splice_read() returned 0), and (3) we haven't read enough to fulfill the request (ie. len > 0 still), and (4) we have already spliced at least one byte. A further patch will modify the behaviour of SPLICE_F_MORE to always be passed to the actor if either the user set it or we haven't yet read sufficient data to fulfill the request. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Jakub Kicinski cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: Matthew Wilcox cc: Jan Kara cc: Jeff Layton cc: David Hildenbrand cc: Christian Brauner cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org Reviewed-by: Jakub Kicinski --- fs/splice.c | 31 ++++++++++++++++++++++++++++++- include/linux/fs.h | 1 + include/linux/net.h | 1 + include/linux/splice.h | 1 + include/net/sock.h | 1 + net/socket.c | 10 ++++++++++ 6 files changed, 44 insertions(+), 1 deletion(-) diff --git a/fs/splice.c b/fs/splice.c index e337630aed64..67dbd85db207 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -969,6 +969,17 @@ static long do_splice_from(struct pipe_inode_info *pipe, struct file *out, return out->f_op->splice_write(pipe, out, ppos, len, flags); } +/* + * Indicate to the caller that there was a premature EOF when reading from the + * source and the caller didn't indicate they would be sending more data after + * this. + */ +static void do_splice_eof(struct splice_desc *sd) +{ + if (sd->splice_eof) + sd->splice_eof(sd); +} + /* * Attempt to initiate a splice from a file to a pipe. */ @@ -1068,7 +1079,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, ret = do_splice_to(in, &pos, pipe, len, flags); if (unlikely(ret <= 0)) - goto out_release; + goto read_failure; read_len = ret; sd->total_len = read_len; @@ -1108,6 +1119,15 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, file_accessed(in); return bytes; +read_failure: + /* + * If the user did *not* set SPLICE_F_MORE *and* we didn't hit that + * "use all of len" case that cleared SPLICE_F_MORE, *and* we did a + * "->splice_in()" that returned EOF (ie zero) *and* we have sent at + * least 1 byte *then* we will also do the ->splice_eof() call. + */ + if (ret == 0 && !more && len > 0 && bytes) + do_splice_eof(sd); out_release: /* * If we did an incomplete transfer we must release @@ -1136,6 +1156,14 @@ static int direct_splice_actor(struct pipe_inode_info *pipe, sd->flags); } +static void direct_file_splice_eof(struct splice_desc *sd) +{ + struct file *file = sd->u.file; + + if (file->f_op->splice_eof) + file->f_op->splice_eof(file); +} + /** * do_splice_direct - splices data directly between two files * @in: file to splice from @@ -1161,6 +1189,7 @@ long do_splice_direct(struct file *in, loff_t *ppos, struct file *out, .flags = flags, .pos = *ppos, .u.file = out, + .splice_eof = direct_file_splice_eof, .opos = opos, }; long ret; diff --git a/include/linux/fs.h b/include/linux/fs.h index df92f4b3d122..de2cb1132f07 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1796,6 +1796,7 @@ struct file_operations { int (*flock) (struct file *, int, struct file_lock *); ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int); ssize_t (*splice_read)(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int); + void (*splice_eof)(struct file *file); int (*setlease)(struct file *, long, struct file_lock **, void **); long (*fallocate)(struct file *file, int mode, loff_t offset, loff_t len); diff --git a/include/linux/net.h b/include/linux/net.h index b73ad8e3c212..8defc8f1d82e 100644 --- a/include/linux/net.h +++ b/include/linux/net.h @@ -210,6 +210,7 @@ struct proto_ops { int offset, size_t size, int flags); ssize_t (*splice_read)(struct socket *sock, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); + void (*splice_eof)(struct socket *sock); int (*set_peek_off)(struct sock *sk, int val); int (*peek_len)(struct socket *sock); diff --git a/include/linux/splice.h b/include/linux/splice.h index 991ae318b6eb..4fab18a6e371 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -38,6 +38,7 @@ struct splice_desc { struct file *file; /* file to read/write */ void *data; /* cookie */ } u; + void (*splice_eof)(struct splice_desc *sd); /* Unexpected EOF handler */ loff_t pos; /* file position */ loff_t *opos; /* sendfile: output position */ size_t num_spliced; /* number of bytes already spliced */ diff --git a/include/net/sock.h b/include/net/sock.h index b418425d7230..ae2d74a0bc4c 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1271,6 +1271,7 @@ struct proto { size_t len, int flags, int *addr_len); int (*sendpage)(struct sock *sk, struct page *page, int offset, size_t size, int flags); + void (*splice_eof)(struct socket *sock); int (*bind)(struct sock *sk, struct sockaddr *addr, int addr_len); int (*bind_add)(struct sock *sk, diff --git a/net/socket.c b/net/socket.c index c4d9104418c8..b778fc03c6e0 100644 --- a/net/socket.c +++ b/net/socket.c @@ -130,6 +130,7 @@ static int sock_fasync(int fd, struct file *filp, int on); static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); +static void sock_splice_eof(struct file *file); #ifdef CONFIG_PROC_FS static void sock_show_fdinfo(struct seq_file *m, struct file *f) @@ -163,6 +164,7 @@ static const struct file_operations socket_file_ops = { .fasync = sock_fasync, .splice_write = splice_to_socket, .splice_read = sock_splice_read, + .splice_eof = sock_splice_eof, .show_fdinfo = sock_show_fdinfo, }; @@ -1076,6 +1078,14 @@ static ssize_t sock_splice_read(struct file *file, loff_t *ppos, return sock->ops->splice_read(sock, ppos, pipe, len, flags); } +static void sock_splice_eof(struct file *file) +{ + struct socket *sock = file->private_data; + + if (sock->ops->splice_eof) + sock->ops->splice_eof(sock); +} + static ssize_t sock_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct file *file = iocb->ki_filp; From patchwork Wed Jun 7 14:05:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C104FC7EE23 for ; Wed, 7 Jun 2023 14:32:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6136A8E0002; Wed, 7 Jun 2023 10:32:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59C0E8E0001; Wed, 7 Jun 2023 10:32:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 416A98E0002; Wed, 7 Jun 2023 10:32:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2F4E78E0001 for ; Wed, 7 Jun 2023 10:32:33 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EC8E81C7501 for ; Wed, 7 Jun 2023 14:32:32 +0000 (UTC) X-FDA: 80876192544.26.C3CC55F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id D2251180007 for ; Wed, 7 Jun 2023 14:32:30 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AD1xV2Hp; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148350; a=rsa-sha256; cv=none; b=IUSvJpCN54lqK+7RWdEmxRKNK30hUeMD6VQ2VKLS3XFhlfcQU44gyrRJ0juC6+9wD1OuSA 9aSPnQygcqHa9ttFx3uIvMPOX6QXkGzuuMqRzwKS1yrj9RhGUXwrV54i/bLqaVbppjOQwu jPmVa9fu5rRD33pxU+1t9r0PBbr+Mgs= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AD1xV2Hp; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148350; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Snl4OmEGYdB1uQLVvSThTI2hbb8EzVIFjutC5pzihDs=; b=b/ljLGBni2B0F+S+M4b+W5DZNd+SGAh729xltNyZeNebhFXq061ToL6xNc0cPgmV0OLXd3 6X12x2fesjvf9No2Ozynqhk/fx/wfVZUPC8VXwmZ80PJ8ohxGRrFAVEZpvt4WbJZc+AWiE YcW3hicvevUZq+fu+yyj7tr036bLXzc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Snl4OmEGYdB1uQLVvSThTI2hbb8EzVIFjutC5pzihDs=; b=AD1xV2Hpeu6lEDRWJQSNeHBIl6KbCrKvvPLddj6Bu56vXU+nI2IbSeFoACBFFRF+97eaUX DjWEAdOjTC3s//a6rGbxarjB3O6r3RN72kj4QOEUjeWo0JTyMxGJPxymiR+szx5gVS8rPc TlRZKtRY38KBjtxq0d7cKitwK63/2zY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-433-JI8qk0mUOkqYgX5x8LPA6Q-1; Wed, 07 Jun 2023 10:06:29 -0400 X-MC-Unique: JI8qk0mUOkqYgX5x8LPA6Q-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E6FFB811E8F; Wed, 7 Jun 2023 14:06:26 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7C62A477F61; Wed, 7 Jun 2023 14:06:24 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 05/14] tls/sw: Use splice_eof() to flush Date: Wed, 7 Jun 2023 15:05:50 +0100 Message-ID: <20230607140559.2263470-6-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D2251180007 X-Stat-Signature: ypmar9zjj9564fmebdfe8a186gdptpk6 X-Rspam-User: X-HE-Tag: 1686148350-191855 X-HE-Meta: U2FsdGVkX19Ei1s+30odS/Z1lAXgaxAcUHMvqnfQOtn5UiAe2Jr+emNL+Ja5PdmLIGKg1DMyUQBYahwZoyVFl/HoLzDoWlzT95KxXGRpz1Ll7PwoscS4cbmKXcRlgLXRjfaXA2cmMRAg1u/zN0dOcb8qAlpP8zSqve4+//RGqOQtRND7+/O1Mx4xLKNHO+SNDTewi/MWW/XMcF7J3C7wPlOl9crlPJCN34RfnqyALybXK6daBcMihG1fB6YLFmTuj3/g0kF6U3T9NLT9/6baCDxgW1725i7ri1i760z55ZJgo+N6K+uix9/OGzDH3U8gp3+gsujswBfWCQFveh/oDYu+qWExN4Z2RZ2UigTE5B4p92S3qNPU1SH3kcqVL9zynmwK6272LzEklmKUB0PPWHe4qiF22zl5HAaCNHDRgAEOovYM6Ro6FYUbncg7+0G+zPE9IZ10O/ViHBzQe9a08CXOI6yNx8K0BY8Y4xWEVpgK5tArjWybo+9K/QtE6ndd55ArOBylD3CRlqLOqiurE5j91G0zkCzptAwXAWSgCJ6tZQfPjg58Ta2RbB8tLrc70RN98/8UN7eziAdyLFzqA/XBhH7CC7o6mNaKUMSQkOte6JwzR8Iukd4LvnGOQfnCDfDefS7zX67UTRO6KB544GPYlz+eHWYuKjAoFeAPIPYwqy/dVjahAQogEdkj7SXd5FWMX7xhURyvBcg3zIFf/0rGnHgjrA9sy1wntFwchQ6A2J8ptke0e/y213rHyElQEe0B1Dg5qv0pawnHN4v3IbePVhUrVB3our/JIw6Ft2uFu5fsvAaNK02UMfAmujodQWc6hz5M3Nsj8WEJzUmz0+K/GwiCOXreRsBEoMbH/53c+7vMunQ6FX2CFIQGYSvIm574EnziN1ONuwdoNHqfY7BRGmlFLyk7+jVjwci44qTsMTdi/jmFWjKFZ+fNAmmpJ1G2x8qs52CrPsTNkv8 gOacFOkf 4RpJJyc0bg8bhDAS2vQNQaWMi0NLUw3GXkjce059zAxjBTFUDCNlUARJlKBEhEcaTjIS1yHalBjaZaiYTpmN3dDrmTBeEfNVqlgARgPTNUmB/Q5wbi0BLUQ8P43ckVY3ddPdldhdmbF9Ij4olO0BZ+6GyZN6aQqXbzDbMGRbQJkb8GirkOhCSSWPydg4gSTeItxO1FVTbRTQV/Vqh7oZza++xOGVW11bDVvVcnmdKvScPaVmssbsPbRElyC0fK6gkwfrCPRCcNkvK9zPypBEiLp/ptXJI5MGuEMmR8jfKRayyeEj7C/pjz1DUILL+7i2Luz+h0vnoQhjyprybF0RTxzuxnQYxldaXpZys/GxrkbXFxvGRZsIgcT8ah3gnRoZooR9UDglDmGmoe74SLvDyY31zjUoDIt0DHEsrTmZCwWR/q97jC5ga+V0bH58DA6m+Y2xIJ5pmeb7lwePKklYmAhMr84EG3bjYG2TSzAxAvZ9S2VEBLhnowkCCOmU4RMmoqD+1hyfGMQ3y/CZAdYifwdczpgjBOGFwJclUJWZ0/X61DOkUxd9oz3MelKrw5tHWXiVxJw5w3p2MHWfCf7p6PG7mMRYkeGbrLot5nGYNOV4UKXVWskFMJrcgCQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called TLS with a sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Reviewed-by: Jakub Kicinski --- net/tls/tls.h | 1 + net/tls/tls_main.c | 2 ++ net/tls/tls_sw.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+) diff --git a/net/tls/tls.h b/net/tls/tls.h index 0672acab2773..4922668fefaa 100644 --- a/net/tls/tls.h +++ b/net/tls/tls.h @@ -97,6 +97,7 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx); void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx); void tls_sw_strparser_done(struct tls_context *tls_ctx); int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); +void tls_sw_splice_eof(struct socket *sock); int tls_sw_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tls_sw_sendpage(struct sock *sk, struct page *page, diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index e02a0d882ed3..82ec5c654f32 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -957,6 +957,7 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG] ops[TLS_BASE][TLS_BASE] = *base; ops[TLS_SW ][TLS_BASE] = ops[TLS_BASE][TLS_BASE]; + ops[TLS_SW ][TLS_BASE].splice_eof = tls_sw_splice_eof; ops[TLS_SW ][TLS_BASE].sendpage_locked = tls_sw_sendpage_locked; ops[TLS_BASE][TLS_SW ] = ops[TLS_BASE][TLS_BASE]; @@ -1027,6 +1028,7 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG], prot[TLS_SW][TLS_BASE] = prot[TLS_BASE][TLS_BASE]; prot[TLS_SW][TLS_BASE].sendmsg = tls_sw_sendmsg; + prot[TLS_SW][TLS_BASE].splice_eof = tls_sw_splice_eof; prot[TLS_SW][TLS_BASE].sendpage = tls_sw_sendpage; prot[TLS_BASE][TLS_SW] = prot[TLS_BASE][TLS_BASE]; diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 38acc27a0dd0..a2fb0256ff1c 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -1157,6 +1157,80 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) return copied > 0 ? copied : ret; } +/* + * Handle unexpected EOF during splice without SPLICE_F_MORE set. + */ +void tls_sw_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tls_context *tls_ctx = tls_get_ctx(sk); + struct tls_sw_context_tx *ctx = tls_sw_ctx_tx(tls_ctx); + struct tls_rec *rec; + struct sk_msg *msg_pl; + ssize_t copied = 0; + bool retrying = false; + int ret = 0; + int pending; + + if (!ctx->open_rec) + return; + + mutex_lock(&tls_ctx->tx_lock); + lock_sock(sk); + +retry: + rec = ctx->open_rec; + if (!rec) + goto unlock; + + msg_pl = &rec->msg_plaintext; + + /* Check the BPF advisor and perform transmission. */ + ret = bpf_exec_tx_verdict(msg_pl, sk, false, TLS_RECORD_TYPE_DATA, + &copied, 0); + switch (ret) { + case 0: + case -EAGAIN: + if (retrying) + goto unlock; + retrying = true; + goto retry; + case -EINPROGRESS: + break; + default: + goto unlock; + } + + /* Wait for pending encryptions to get completed */ + spin_lock_bh(&ctx->encrypt_compl_lock); + ctx->async_notify = true; + + pending = atomic_read(&ctx->encrypt_pending); + spin_unlock_bh(&ctx->encrypt_compl_lock); + if (pending) + crypto_wait_req(-EINPROGRESS, &ctx->async_wait); + else + reinit_completion(&ctx->async_wait.completion); + + /* There can be no concurrent accesses, since we have no pending + * encrypt operations + */ + WRITE_ONCE(ctx->async_notify, false); + + if (ctx->async_wait.err) + goto unlock; + + /* Transmit if any encryptions have completed */ + if (test_and_clear_bit(BIT_TX_SCHEDULED, &ctx->tx_bitmask)) { + cancel_delayed_work(&ctx->tx_work.work); + tls_tx_records(sk, 0); + } + +unlock: + release_sock(sk); + mutex_unlock(&tls_ctx->tx_lock); +} + static int tls_sw_do_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { From patchwork Wed Jun 7 14:05:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16532C7EE23 for ; Wed, 7 Jun 2023 14:31:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5A2A8E0002; Wed, 7 Jun 2023 10:31:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E2C28E0001; Wed, 7 Jun 2023 10:31:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 835A58E0002; Wed, 7 Jun 2023 10:31:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6C8C08E0001 for ; Wed, 7 Jun 2023 10:31:30 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 40DCA1201D0 for ; Wed, 7 Jun 2023 14:31:30 +0000 (UTC) X-FDA: 80876189940.18.8D7CD1C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id E761E140036 for ; Wed, 7 Jun 2023 14:31:14 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YJsEne3M; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148275; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fFpvHzgaRY/5j/9ELb8QxJWBpaTq1Vb8GSt760xxeFM=; b=ubjIooI7oVHuzLYOFghjHh3TwHvybmH4CIWF7fXdBVWNPpb46qT/fUyfdfZFcpBqndCIHF 3GbbWyQlHd7B5VIzTEg3aya7XYXcgTrZPZbKDn2qTSNCHqoQJ92Lg5/mdg+/gGVyk7gC7+ NKFwACUEqIL2IEvzVPUkUexAiKybukA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YJsEne3M; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148275; a=rsa-sha256; cv=none; b=xkjh5P7ZN0V2a1xe+xCFrwoyHfsRF2KsY7Z34ngU+ScNAG3XurFLe4HZHDIG6gF7gy1hXA Wn95N0FAEixavnSfPaUI1QcEWsBOqTY00hRCOU9s4w2q0+cIi4wHKbqtvqixGss8+9EXgT JGLsffwQqSM0/DjzQqnW6Lp/Kp7Kxsc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fFpvHzgaRY/5j/9ELb8QxJWBpaTq1Vb8GSt760xxeFM=; b=YJsEne3MlLK2tezWQAr5WW4Hl8iGYofxwPfV3lySr54Rwr0bQ/IrHJsZ+GyVXrzPZ4LMMa kk288qQMrE/787/S1QqY6Toql7pxZXZQgQLieavBrVaasmPXVqm7eBAQQeTZLV+lvSyA1z lFLsuRWOszi/FsCnoK2OToxZ8yL6z8w= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-442-0ndtglCKMsqgcKb-RlBYPQ-1; Wed, 07 Jun 2023 10:06:31 -0400 X-MC-Unique: 0ndtglCKMsqgcKb-RlBYPQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 382973C02195; Wed, 7 Jun 2023 14:06:30 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1BF922166B25; Wed, 7 Jun 2023 14:06:27 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 06/14] tls/device: Use splice_eof() to flush Date: Wed, 7 Jun 2023 15:05:51 +0100 Message-ID: <20230607140559.2263470-7-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: E761E140036 X-Rspam-User: X-Stat-Signature: 4oderhxajdekne17wt98gqtrxxgsizqq X-Rspamd-Server: rspam01 X-HE-Tag: 1686148274-524130 X-HE-Meta: U2FsdGVkX1/OPRWSzqRGM/xvN3ycySb3CPtiHy9cU3Y1+ykYZBALhedaFN24tdweiHKkmzq2/IaR1CptNRtD+7pXSybjeQU/ZlHV4Yd4rnmDapiGfQNCijsYcFBKn5+YX3TPEqMUl8/VNLGiz4VYthkLx5iscO1IY42BLgHRR/ZtiVyAIShOEptKNFe2KjmDmtvPJeadPdjwKczAo/mD7pGIiucGF7Af44QtGXDhEqorJOD9cwuoa//wisYR+pT/GKQbh13jOyhWcj7CIfwhEiPa9EMEW9vA9Jk78UAIXxG8Dn5X6SVdYRFa4xFM6Y+EWxQC/0t8tXeHKS22XgYsBe073tWIMFm5HwahRT79M20B0s3Guebkw/okESx0R+A+yTIv4hGvgykUxXNYVuNcLb9iz8IBGPBVZOlYBevpXpbkaogjGU0bBUeFtGz6HD8Re7wNKDe3y5KzEcERol4b5bIuxfpqJVooasVT8LbgT45ngkWpXmFTpafZuWSCtNJwEaLNCNqC3BGXxkihKN5Pyid7FZ0EIUpgI0gRVIFm/PzDwYkonNIZjUCU6XtYkSrLuO4aC4u+WfBog2MuCmZfQbVeS7RWirM9n8Z1ThJpJTNIIi9Tg99ngpztBmU78/abjhMKs8bQBbOtQID6p5j4dEFy6BrmWEAEJ13YqrOmW0FYhyJb5hl/72vuEdBvevIt3ZG2UqcueF0aWmQwE/iEqsTsQ7sBUQ7wYfDNHmVff/vmtZpGJsQhrLtLOcuc2OfPO3iPFFW0oWLZEHUH/6iD+eEEPmgceGPWYPEkmDfbDYfke/cs0rOqoA7FIu//KbWpkVo+NvrFG1Z4kgkfoX1B2rEwv4jWYWWraRDoxsyRhSG/f6lewdghRhBEOKgGI9I6drlf3sq5K9LAq4pngEZ3DFdjxl9i7AGLnUXLGism5ivrcMRYW2xwD6Lu5SiFpGXdBSSYAXPmHQfkPPxQK6S Rw4KTlTD 7LcJemvxwNkoGq2EvYzp9EFsvAF5PbH/nh99aQL/i30qeGVk2jfbhrGS8D9aQeTCZUdXB4GyOXW5EZlcZqrmAOYuN1bpJ+5jW+O5b8c4oNdXxmbWaXWTikC/5D7km1eMEmjFWKShpKPrK8Af7L374ylKLXO8xSMx9PEt/Ok7nz0ZXE19G1oPbojuaj5+fojGtnSG0s7pW7q56ddzGfKNIlNhDrmkp67WZCNx28wEunOTBAFYXDyAC6sSz32y7e7+/2ylUu9Kfx0d/QRf9wKg0+sPaSZFmeIFdDgYslbxp/f+pRjm6c7gHEQBRthCHeOE3WXMKHb77tB/TIYz988eQ70dkfYITjlF0DmkwBnhHbSK+ftDcIXZYdbXOkB2vuAn+vfhC3WaCzdE3Z0bcVPi8vBCmEPx9l3lRR4iX7NR03xuIulHcEclDDs7M331sSJuJ5gVKmYP5NVsKLSsvw8brcceTJkMLfOeCC2yOGoko55jgeEiIN1voQkTlq1g7itATo/5VT12ufqVv4vX/mL6HPygWlo3VU1JWcdcDBYKMBSlCYxdNdhELrlbons/kkwJXhHk89IYlzhRMalmvksViC44nQebsPfizljQRUyZUm/pgM/OHvR23DFTDzw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called TLS with a sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Reviewed-by: Jakub Kicinski --- net/tls/tls.h | 1 + net/tls/tls_device.c | 23 +++++++++++++++++++++++ net/tls/tls_main.c | 2 ++ 3 files changed, 26 insertions(+) diff --git a/net/tls/tls.h b/net/tls/tls.h index 4922668fefaa..d002c3af1966 100644 --- a/net/tls/tls.h +++ b/net/tls/tls.h @@ -116,6 +116,7 @@ ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos, size_t len, unsigned int flags); int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); +void tls_device_splice_eof(struct socket *sock); int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tls_tx_records(struct sock *sk, int flags); diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 9ef766e41c7a..439be833dcf9 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -590,6 +590,29 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) return rc; } +void tls_device_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tls_context *tls_ctx = tls_get_ctx(sk); + union tls_iter_offset iter; + struct iov_iter iov_iter = {}; + + if (!tls_is_partially_sent_record(tls_ctx)) + return; + + mutex_lock(&tls_ctx->tx_lock); + lock_sock(sk); + + if (tls_is_partially_sent_record(tls_ctx)) { + iov_iter_bvec(&iov_iter, ITER_SOURCE, NULL, 0, 0); + iter.msg_iter = &iov_iter; + tls_push_data(sk, iter, 0, 0, TLS_RECORD_TYPE_DATA, NULL); + } + + release_sock(sk); + mutex_unlock(&tls_ctx->tx_lock); +} + int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 82ec5c654f32..7b9c83dd7de2 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -1044,10 +1044,12 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG], #ifdef CONFIG_TLS_DEVICE prot[TLS_HW][TLS_BASE] = prot[TLS_BASE][TLS_BASE]; prot[TLS_HW][TLS_BASE].sendmsg = tls_device_sendmsg; + prot[TLS_HW][TLS_BASE].splice_eof = tls_device_splice_eof; prot[TLS_HW][TLS_BASE].sendpage = tls_device_sendpage; prot[TLS_HW][TLS_SW] = prot[TLS_BASE][TLS_SW]; prot[TLS_HW][TLS_SW].sendmsg = tls_device_sendmsg; + prot[TLS_HW][TLS_SW].splice_eof = tls_device_splice_eof; prot[TLS_HW][TLS_SW].sendpage = tls_device_sendpage; prot[TLS_BASE][TLS_HW] = prot[TLS_BASE][TLS_SW]; From patchwork Wed Jun 7 14:05:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6160C7EE25 for ; Wed, 7 Jun 2023 14:32:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E0488E0003; Wed, 7 Jun 2023 10:32:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 268258E0001; Wed, 7 Jun 2023 10:32:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BB398E0003; Wed, 7 Jun 2023 10:32:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EAEBD8E0001 for ; Wed, 7 Jun 2023 10:32:39 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 76A551C7501 for ; Wed, 7 Jun 2023 14:32:39 +0000 (UTC) X-FDA: 80876192838.17.7D48DEE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 818EEC001C for ; Wed, 7 Jun 2023 14:32:37 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gQhVHgMB; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148357; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vvYX0FjZprfEU41+87A4pQCc64jz60fQuum5imcGYY8=; b=k7gWNGfpfHpfVfHbHSb7OmwKXPCWkKjEV6SQySprKXbVFT6SQZE6nV7+T02kqG5Diy979t Po+GXUOZBBRlEs5HMcliStPFSls/+LvvFmhdafZlfh4wFYFAio1jxmcuU/7C3kboodYE3H mC44AHubV+u/ctnlu3EWq2xpEByZ55E= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gQhVHgMB; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148357; a=rsa-sha256; cv=none; b=ngPJmeE/3yxqdBiPQgfCnG88Ge3RXYHUOXpeg7QrrAYnSKoY18aBpkZiq9Fm4UD+rV6PTt cUnqUeyF1GIwSl8wf8rx7NHLde/RM4TZo45J5CJylBMuFs1SBVV4lHQj+8x33pKNdzRHrB 9lCH/4JyjnthrDlCm3cc6qEuUvSF0zo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vvYX0FjZprfEU41+87A4pQCc64jz60fQuum5imcGYY8=; b=gQhVHgMB5XD3lCqNrfL7SBl+f/qe+HAt2rv2L8VfARm0/Nphcb8RZK/od6SENujUmDZbAs nkgeWKET9Mm9MX+V3A2RfGzPPW6gUVyzQb29WVlRpmRTc2pmoDB4SztYQ4Au/UH5HUKoQ9 j3j+DSb0w3g9dVtVBB96eB75/gBSNYc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-327-1Z_Rr_ttOkOs9_GQq91k1g-1; Wed, 07 Jun 2023 10:06:35 -0400 X-MC-Unique: 1Z_Rr_ttOkOs9_GQq91k1g-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3EBD185A794; Wed, 7 Jun 2023 14:06:33 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7B901492B00; Wed, 7 Jun 2023 14:06:31 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 07/14] ipv4, ipv6: Use splice_eof() to flush Date: Wed, 7 Jun 2023 15:05:52 +0100 Message-ID: <20230607140559.2263470-8-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: 818EEC001C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: p7rwxc1ohjcz46gg73fejajis47ngwwq X-HE-Tag: 1686148357-888785 X-HE-Meta: U2FsdGVkX1/F2F4PJOhKPGEaLFPocCLRnV0JsXuUfBvgzXkFd44qOG5JLrlVqfascg5XQZnZqERMfTWNHpYhIfUKTETCPo9uiRNVQBVzBKQRz+9DrL15nXA7Yh+P9B2+WYMsiqCmhniYfFRqDxV3j8/XSaASOAld+spsJ5ZjZAuMHbvMdoFGDm0b0Qq7TrLS8vKgnbnOi7Tja05g0LBrL46cWrfvGMUGTfxSELRsbKI1F+Mv/4RqaOtjx0V3da6c749Y+qU9F06RdFsPsFlBjSU+qgZtMK1nABJ6QasdKpwW0TKiOg72j/xBFi78I7YKWoLQ7SlWUZvPHm9cTYrbvFI4teHcvYB7izY2YTF3KmhCv+E3jD60IfRM9QSn3/i1sgvyVzuskajwOGrOY/fbaV5eawut/Oypm5Pmwbee+Xt8BJT4g9DqBYXZoVfA9uvp+t1PbcfqLgOfNvMote1JQ7XzOE3bJ00eoOclaBrQwPEI8+CRbX8IQCObEtCoPYFIZq9QnvWeEjPb2EzMj9Lxe1VIQ6bNJvIifmx/NiQMqsjKNrTc59Q7eGnJ/usonN3cYcYeO/GqdD5P0VSglL93puNPghMFrN42mFeFgBxIThyVvyldsd3pHjtoxjPBhUp/1k7mXo9lpM/xhCHPuU1deVxnnMoDiOEUwcmlK8+/rn0SXi94IlDJxdqiFS3X1CzEoS+Dq/sDwim2YCLhIDqUdEhNNgJCgFUKgUPc7Uz2IvDQcH9JcUWny0zsSTZLDVw8t8Q9OfqK4VcjGyBK9m9yzZtii8RGA1GKLrhbxg2Laa+wiTuFhLV11naVv1gwNqXJ4ALtlpnTtfYCiXcCR6fps+WG+tOZ+oRcumUPIo4eS2xvFIemqAWEEn+OeA7wqcQVGtyV3gPyxzQ7Nql8k+MY2J/Z0ry71rg5TKeYg4Wn09wYuJwoNBq2SP5UOhf3icgiz7eJ997yMD0aVPZKa8u uGcknSAr JPREyvKlAiJLuKvnoIM5zN8QQ4vRbb0H6f1NzEV0TvQVdGw5c7I+COL5SfeFE0YXaywmW7Dd9gdebXDcLzWYTvfIdzpB74xUdRoqFeYxqsLaBJKrzalBBt9vS5oAYkXtRzJfx/hhwF+qbhEfZGqbsrxSzncjf6mB05c06nrNFyUEKQfbAcab20PTLsMQ9GFrGqXAPW83ak43f1v2K8octvIPPqhxfpRobfLB6Q8ijrq9xcoz0hBaKYKDo7NOUQ0Vqh+8p4H/ybBPflMuGLANRjPSvEBZIo3iixm/uMsZNHRKQ+4lpvXHz7HtxZsw5cBVJclws2b0YvWYO4XynvIREBHykHwWgA321MLhES1DNTQbFSvDMW8Fz3F8031LXABOLi6xQyuI68JagaG6+pQzaYfzwkIeCymnsZ5Qlh9on1EsYlcrCPvG1n83RpU78H0Y/iYpbgwEsoqH3dmqnS9evSPTLahYH7qe1RkNedKcQoW4kMdDGZdpK2ArnaXqMIQAPBBaj+48ZvFpI5FVaD6aFGCSfB8rDjDNGBHtHwXIlGBjQRBdkwp3ncmMyQlIvX7MJ03nPF8jOvWy/8FE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to undo the effects of MSG_MORE after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. For UDP, a pending packet will not be emitted if the socket is closed before it is flushed; with this change, it be flushed by ->splice_eof(). For TCP, it's not clear that MSG_MORE is actually effective. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Eric Dumazet cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/inet_common.h | 1 + include/net/tcp.h | 1 + include/net/udp.h | 1 + net/ipv4/af_inet.c | 18 ++++++++++++++++++ net/ipv4/tcp.c | 16 ++++++++++++++++ net/ipv4/tcp_ipv4.c | 1 + net/ipv4/udp.c | 16 ++++++++++++++++ net/ipv6/af_inet6.c | 1 + net/ipv6/tcp_ipv6.c | 1 + net/ipv6/udp.c | 18 ++++++++++++++++++ 10 files changed, 74 insertions(+) diff --git a/include/net/inet_common.h b/include/net/inet_common.h index 77f4b0ef5b92..a75333342c4e 100644 --- a/include/net/inet_common.h +++ b/include/net/inet_common.h @@ -35,6 +35,7 @@ void __inet_accept(struct socket *sock, struct socket *newsock, struct sock *newsk); int inet_send_prepare(struct sock *sk); int inet_sendmsg(struct socket *sock, struct msghdr *msg, size_t size); +void inet_splice_eof(struct socket *sock); ssize_t inet_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags); int inet_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, diff --git a/include/net/tcp.h b/include/net/tcp.h index 68990a8f556a..49611af31bb7 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -327,6 +327,7 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size); int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied, size_t size, struct ubuf_info *uarg); +void tcp_splice_eof(struct socket *sock); int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, diff --git a/include/net/udp.h b/include/net/udp.h index 5cad44318d71..4ed0b47c5582 100644 --- a/include/net/udp.h +++ b/include/net/udp.h @@ -278,6 +278,7 @@ int udp_get_port(struct sock *sk, unsigned short snum, int udp_err(struct sk_buff *, u32); int udp_abort(struct sock *sk, int err); int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len); +void udp_splice_eof(struct socket *sock); int udp_push_pending_frames(struct sock *sk); void udp_flush_pending_frames(struct sock *sk); int udp_cmsg_send(struct sock *sk, struct msghdr *msg, u16 *gso_size); diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index b5735b3551cf..6cfb78592836 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -831,6 +831,21 @@ int inet_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) } EXPORT_SYMBOL(inet_sendmsg); +void inet_splice_eof(struct socket *sock) +{ + const struct proto *prot; + struct sock *sk = sock->sk; + + if (unlikely(inet_send_prepare(sk))) + return; + + /* IPV6_ADDRFORM can change sk->sk_prot under us. */ + prot = READ_ONCE(sk->sk_prot); + if (prot->splice_eof) + sk->sk_prot->splice_eof(sock); +} +EXPORT_SYMBOL_GPL(inet_splice_eof); + ssize_t inet_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags) { @@ -1050,6 +1065,7 @@ const struct proto_ops inet_stream_ops = { #ifdef CONFIG_MMU .mmap = tcp_mmap, #endif + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .splice_read = tcp_splice_read, .read_sock = tcp_read_sock, @@ -1084,6 +1100,7 @@ const struct proto_ops inet_dgram_ops = { .read_skb = udp_read_skb, .recvmsg = inet_recvmsg, .mmap = sock_no_mmap, + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .set_peek_off = sk_set_peek_off, #ifdef CONFIG_COMPAT @@ -1115,6 +1132,7 @@ static const struct proto_ops inet_sockraw_ops = { .sendmsg = inet_sendmsg, .recvmsg = inet_recvmsg, .mmap = sock_no_mmap, + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, #ifdef CONFIG_COMPAT .compat_ioctl = inet_compat_ioctl, diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 53b7751b68e1..09f03221a6f1 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1371,6 +1371,22 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) } EXPORT_SYMBOL(tcp_sendmsg); +void tcp_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tcp_sock *tp = tcp_sk(sk); + int mss_now, size_goal; + + if (!tcp_write_queue_tail(sk)) + return; + + lock_sock(sk); + mss_now = tcp_send_mss(sk, &size_goal, 0); + tcp_push(sk, 0, mss_now, tp->nonagle, size_goal); + release_sock(sk); +} +EXPORT_SYMBOL_GPL(tcp_splice_eof); + /* * Handle reading urgent data. BSD has very simple semantics for * this, no blocking and very strange errors 8) diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 53e9ce2f05bb..84a5d557dc1a 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -3116,6 +3116,7 @@ struct proto tcp_prot = { .keepalive = tcp_set_keepalive, .recvmsg = tcp_recvmsg, .sendmsg = tcp_sendmsg, + .splice_eof = tcp_splice_eof, .sendpage = tcp_sendpage, .backlog_rcv = tcp_v4_do_rcv, .release_cb = tcp_release_cb, diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index fd3dae081f3a..df5e407286d7 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1324,6 +1324,21 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } EXPORT_SYMBOL(udp_sendmsg); +void udp_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct udp_sock *up = udp_sk(sk); + + if (!up->pending || READ_ONCE(up->corkflag)) + return; + + lock_sock(sk); + if (up->pending && !READ_ONCE(up->corkflag)) + udp_push_pending_frames(sk); + release_sock(sk); +} +EXPORT_SYMBOL_GPL(udp_splice_eof); + int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { @@ -2918,6 +2933,7 @@ struct proto udp_prot = { .getsockopt = udp_getsockopt, .sendmsg = udp_sendmsg, .recvmsg = udp_recvmsg, + .splice_eof = udp_splice_eof, .sendpage = udp_sendpage, .release_cb = ip4_datagram_release_cb, .hash = udp_lib_hash, diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c index 2bbf13216a3d..564942bee067 100644 --- a/net/ipv6/af_inet6.c +++ b/net/ipv6/af_inet6.c @@ -695,6 +695,7 @@ const struct proto_ops inet6_stream_ops = { #ifdef CONFIG_MMU .mmap = tcp_mmap, #endif + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .sendmsg_locked = tcp_sendmsg_locked, .sendpage_locked = tcp_sendpage_locked, diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index d657713d1c71..c17c8ff94b79 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -2150,6 +2150,7 @@ struct proto tcpv6_prot = { .keepalive = tcp_set_keepalive, .recvmsg = tcp_recvmsg, .sendmsg = tcp_sendmsg, + .splice_eof = tcp_splice_eof, .sendpage = tcp_sendpage, .backlog_rcv = tcp_v6_do_rcv, .release_cb = tcp_release_cb, diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index e5a337e6b970..6c5975b13ae3 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1653,6 +1653,23 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } EXPORT_SYMBOL(udpv6_sendmsg); +static void udpv6_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct udp_sock *up = udp_sk(sk); + + if (!up->pending || READ_ONCE(up->corkflag)) + return; + + if (up->pending == AF_INET) + udp_splice_eof(sock); + + lock_sock(sk); + if (up->pending && !READ_ONCE(up->corkflag)) + udp_push_pending_frames(sk); + release_sock(sk); +} + void udpv6_destroy_sock(struct sock *sk) { struct udp_sock *up = udp_sk(sk); @@ -1764,6 +1781,7 @@ struct proto udpv6_prot = { .getsockopt = udpv6_getsockopt, .sendmsg = udpv6_sendmsg, .recvmsg = udpv6_recvmsg, + .splice_eof = udpv6_splice_eof, .release_cb = ip6_datagram_release_cb, .hash = udp_lib_hash, .unhash = udp_lib_unhash, From patchwork Wed Jun 7 14:05:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A04AC7EE25 for ; Wed, 7 Jun 2023 14:41:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 155608E0003; Wed, 7 Jun 2023 10:41:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DF5B8E0001; Wed, 7 Jun 2023 10:41:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9B7B8E0003; Wed, 7 Jun 2023 10:41:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D59D48E0001 for ; Wed, 7 Jun 2023 10:41:40 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 53B26AED2D for ; Wed, 7 Jun 2023 14:41:40 +0000 (UTC) X-FDA: 80876215560.06.1763C4D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 506F418000D for ; Wed, 7 Jun 2023 14:41:38 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ggvvRXV0; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148898; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=axbQ2fmWT7oadEpv+RKPywBYD27/saJPi8A/5AZqeFc=; b=62QzR9aZi8EDo5tKD3foI30uBFIl44m7xGtXRELSql0gIKeMQFeNIV2tJCNKTlaL2AGAZz c1xmSx4ajmToTOBCi0Ljd7NDn9viPmD3yOu6WlA0IjWwIYwjLS37f4yscVsdnBffQ3AFIQ yeFem1LguLu6/q9X7XQgkUlYsYYQyTc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148898; a=rsa-sha256; cv=none; b=Mgbedo5h7WtclKOUk0B/zum3jeSP28WFr2iQaN4hEHeduX72HtiNopX5azG2yGFL9vsNHo 4EA9SaqjpFW5e/t+X25dzHUWc+TTzEqSWtiG/Evg06bLdLPpU3U6uljwG8BZ9WGiaKnMli CSjTXQ05+FtXX1ZmCkEEYKUfoAIJceQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ggvvRXV0; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=axbQ2fmWT7oadEpv+RKPywBYD27/saJPi8A/5AZqeFc=; b=ggvvRXV0g5zfkv2ScHvyLqQcYBEBE+IO7FM3x+z1PIXueWOjG9M4zkW15z3FP2keY4Y5wZ x7Tw/tAOz/cwsupe864qtNCaxOYdmS9pGSTefa8YNrxdthPBoSn5u31fIgKlqY39terj+/ +jhbaFUnnoQxOTfd00eTOrS4yEywOrk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-O41qDC5EPpCAaVWMZOdERQ-1; Wed, 07 Jun 2023 10:06:42 -0400 X-MC-Unique: O41qDC5EPpCAaVWMZOdERQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6EF1E1C02D26; Wed, 7 Jun 2023 14:06:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5062403367; Wed, 7 Jun 2023 14:06:34 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ayush Sawal Subject: [PATCH net-next v5 08/14] chelsio/chtls: Use splice_eof() to flush Date: Wed, 7 Jun 2023 15:05:53 +0100 Message-ID: <20230607140559.2263470-9-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Queue-Id: 506F418000D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: a6dbmp7fup5o3537e4h534ska1auerwu X-HE-Tag: 1686148898-714306 X-HE-Meta: U2FsdGVkX19r/2pY0va6oOA0gwLNN6yaxuObMLFNL7MlD7zWqgqLS8yU6UtdzX8laNWhgQDOXtAebEQi0IQXFXiS3mmS87i44lZwB60+eHfOaV/LrpjjIFkfoahaQGD06ws7udUtUdPMoO5Z3TBnGV6ePjKPaNPGd8sRuscl/0xAehILNOTjch6lWOG+kB8e9lpFiC02KRX6suCPRsA3Ari1oQO2hwe6Pb6zPpsamLeCCOZVBA8CSrHWMlx4WQd2ytdJTEYyAl7MLPlpC48uK5sRYzOflKuhxOgc/K7CDLQ45cVEIilHPZ//KgOm1zqC7xgVesuFUpUwWE+M0cMZBUF0e/GHqsZBXuPLVbeKJ6gHcU3+bZGHfHJRVuA6avqS2n7ifUIFAYezWbJTATtpBj5YRvBdk9pWEPMBlZWjdOJiabVSratrRP+xczvNsFmqJzhnFTxigqav9kDFixmZB7Vk+AvmymI0C/CqLKXiPuRGPiSaLnO4Vf4Dza7Tb8EJxUXagO+xDt0U/V/t0yieVa+hK51Av8uUHdfK4ciQavFRLoSa82p2YIt47jvDCT+vbExM4wIAIhbm3TABFO9Ag9ypRJ1nqVGsqz3BKcm6DuUXWRJJ7iMUYNmSgBEssRiZmQ7n28zOUZcaGXmnniwTtZXjQR9Fe7LitMaKxWIQGCoIKjcLcETb46e3dAewPz/b2rDkTwW75MAvkq/EwVIBwb27GR2vom+51fmRxbnonCLd5C8m4lf03bF724LEAqQLZT3zXZdAK4x7iom9CJ7SW7WiWcIFjM8m9od30qqJaUZhe5sjDSRVh8sQ6mSY702MlFHK3t1M9PHCDdqYM4sA3q0bxkWDgvnwodT6Gh2ilbROYs9zAMEfpCUJC8oa4KRn8v8lZQHZz7QoeDSSPTA5j9v6ShDRnUur/wvYSKjF/Pc8JfYg1Y+Lcamul+1kM5is5fzgBLhJfOGvqsSs3cA OmqB2ptp NcXhNIG1Y+NwMxXXlGCsTPXJbCdGlrKPEaUXZCzcI3Qq3i4v1D9g8A6WiP4x0H2PAdAAY6E5LqHbPoMUrtW8IPkqG1nY2ciVG4frj50qeZhQfK+UKMvZqCnO/sVf0TlHdHFFzgDSprxCE7UEp+vOfZ9OlKiMmJQoINAMzcoSoY3nkainPH7xQud5G94u+4NOIlLv2Hg+4UKT78g6PzuqNV+UuPAUg3ZFH3yEykrw5gt4gDIjwptcKPHmlnBjOVE/WdctYaNj+7J/DeWkVM00qB2KE7E+UJ+qi2xomWW7tImJe4jZ8nnODK4siatTxjkug46eKF7bPwjQrSYnP7jrffAzlu+4ldMDIzlaYf85O9HJybiGBhHuxw/ABtTUm7Uc/ivL6tNlmPHCcXD03jLcGCQPZiB2OAYBN1REEazGNv9R61slLMNrl9upqNwWW8bGpXBcQoYWBC/rVEzwFYesimSmxk6CbH6Ilo2Ekq5C9d6Ij5Qi6gtipft7nIPo/u+6/em33Zjr/VAl+WitCcs1Yd0iPK040jy0SOZgsnDNf+QRiAYlVtdGUSx9gR4EAPpJ8eu5gU3pIGqU2TnZbO7obXl9cMlGVVSr9mbypeTKra9oBH7uODyh3tiOWBQQ4LTJHNqGE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a Chelsio TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Ayush Sawal cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h | 1 + .../net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c | 9 +++++++++ .../ethernet/chelsio/inline_crypto/chtls/chtls_main.c | 1 + 3 files changed, 11 insertions(+) diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h index 41714203ace8..da4818d2c856 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h @@ -568,6 +568,7 @@ void chtls_destroy_sock(struct sock *sk); int chtls_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); int chtls_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int flags, int *addr_len); +void chtls_splice_eof(struct socket *sock); int chtls_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int send_tx_flowc_wr(struct sock *sk, int compl, diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c index 5724bbbb6ee0..e08ac960c967 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c @@ -1237,6 +1237,15 @@ int chtls_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto done; } +void chtls_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + + lock_sock(sk); + chtls_tcp_push(sk, 0); + release_sock(sk); +} + int chtls_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c index 1e55b12fee51..6b6787eafd2f 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c @@ -606,6 +606,7 @@ static void __init chtls_init_ulp_ops(void) chtls_cpl_prot.destroy = chtls_destroy_sock; chtls_cpl_prot.shutdown = chtls_shutdown; chtls_cpl_prot.sendmsg = chtls_sendmsg; + chtls_cpl_prot.splice_eof = chtls_splice_eof; chtls_cpl_prot.sendpage = chtls_sendpage; chtls_cpl_prot.recvmsg = chtls_recvmsg; chtls_cpl_prot.setsockopt = chtls_setsockopt; From patchwork Wed Jun 7 14:05:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DCC0C77B7A for ; Wed, 7 Jun 2023 14:53:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12A6A900003; Wed, 7 Jun 2023 10:53:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DA4D8E0001; Wed, 7 Jun 2023 10:53:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC8BB900003; Wed, 7 Jun 2023 10:53:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DA78A8E0001 for ; Wed, 7 Jun 2023 10:53:13 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9D9251602C1 for ; Wed, 7 Jun 2023 14:53:13 +0000 (UTC) X-FDA: 80876244666.03.440D47F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 603A91A000B for ; Wed, 7 Jun 2023 14:53:10 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MYHBhwG4; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686149590; a=rsa-sha256; cv=none; b=SE45imA1mwbKsTLlGLjQtO7eT5UFIY7lW+mggEOhcFKyJIDwXXdg95dLgA2MW0PCrTOVFN W77ffNeuQQvwG03LmZ24CxnqTAJmBKcTDmm2mmHT5JddIyBupBK/jCOUPUjp+P7vnLnd9l nzNFugX21TYLF+TxjUPIfVMLXuRPodM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MYHBhwG4; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686149590; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YYh07r9nLp9Gj3Ft6ZNJdfT/+HiAyQEDDAL/yG5YQDg=; b=xrKmh6kSX+1JRFM+oEJ5LwPhQvrY8Bzpooz2Zr6YGUC2PltdbXMcGOOKxTlZ4F/RWHgERx sejVv8b6KyxCV2wNUqFWo9kAOTGQTzlqCouYTttChAi5sJWnWk3zKdbM3UBkBr6yVlXW+6 96QTu3I09bRsL47nvVXHxuW6p+/qAhw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686149589; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YYh07r9nLp9Gj3Ft6ZNJdfT/+HiAyQEDDAL/yG5YQDg=; b=MYHBhwG4k7TLciovXSdWPVX8JSNXAWL6z45gWnPJRAignTATPxNXhMWEgfRAYCKk9DfEKV ZoT9x2RrwMW5uo7EAJW9jY36qBgGy9M+01iHuNc8F02IBKb1Rg0uKA07Pxnqo1+pFZi0PD QaemOT++0HZg8w9ZVqfEeCFkp6qq7uw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-134-blPYcdEnN0m5yZ91Tl7iqQ-1; Wed, 07 Jun 2023 10:06:44 -0400 X-MC-Unique: blPYcdEnN0m5yZ91Tl7iqQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E47528028B1; Wed, 7 Jun 2023 14:06:42 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id C031E4021AA; Wed, 7 Jun 2023 14:06:39 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Tom Herbert , Tom Herbert , Cong Wang Subject: [PATCH net-next v5 09/14] kcm: Use splice_eof() to flush Date: Wed, 7 Jun 2023 15:05:54 +0100 Message-ID: <20230607140559.2263470-10-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 603A91A000B X-Stat-Signature: s65ftddn1i6673uio1dack85bau5jjq4 X-Rspam-User: X-HE-Tag: 1686149590-554026 X-HE-Meta: U2FsdGVkX19fABN3IMrCyzZ9zmgA/mV2J/TZr2uhAZ7yidc5OcwKmejducsflJnUohiSLbEvWUUiJbDJPi+iUoZmCKz2siAKKsI74/KZbUEl1vNRKCs6ll5AjtLmx3JCWtEj9Tt8JQ9zMuQmephKLP92Sw5itu5O51W6KUlQwgtQEBFwho4kORwtyvBNyzwijEStjagTnzSkv6as1xBCtT+qbueQnEKj62mwffC2fyJscobPbIyM5W/QXqNJypdVAuvTR0xExU7c2TgnKWEb9c8Vqmhpm3Z+vCc69AyCto/EAXIko+mFRcrCwtOfTPWU6CwoBQULBLu2/Y8th/EJ5KegXp7cuBgTC9DZcFuc0hWwDUFYq1rm8I4EedEestEfHoLEa00boEAydD+FbGSBvoGdpb6JrRm/slZtnvEjDvOnu0H4TnwaXM4H9fcO+qQ1bZfKc4njRmDzx00zigjU5Q0b97IMCSnVWXeUW9Y9ucKKUmtkaVfHK5Vn8HawyQHrY7aUKhR4ailzfXFyJuIzBFizC3+yqVVIhKHOJp6O5G8eIZIUeyDsLTaBSVwLD1GVtTCQqOrfZfKrAefk6P5pfK3iO7WC0zRCP/30wB9VQoBsqK0Us55QaNMJicthbFykdAqtSvzwvqOIKZTwFsLf6qRoxqxiByinkp4jcGoUu/1Og4QCa83Y227rolzEns8y6ysV5XR8uYEnYGPlKjVLfg+m2fGMt/MMRUD8RZJTfdBv/Rg6R0iBxyQNQiaHUhqcMFAlKUMnmqK3Jy02+L5pdKYLVpSdNENRCvi0ipq3uiDX9S1kRh4jmoJhlLNMkfCJw80a+MCQGJ+AAmeal1+Z0CLUrn35f0neW8goY5ppPQFO5RTsSREd+PtMmjJ/C0DhXIrVbNyZl1n74YG+Kqyw8FHH6EOn4iCmdzHHAPgzAPbdbkbvBBaqVc6gdt7qwACRaAK5XndbMjgygi/sGmt A4aSC6CW cLgTS7cgIFXN7Kf3V5fg4fAONpMDLchTsibCPTSB0HSMSg9TfAqlyK15q9CQYZ0NsUhZ3q7D6Yn5ztRC6+sYZXtNrQQgiY5C9Po3B3TkXX2FkPV19hgJ4S/cMnzD9RnIGXi5OQ0RGsAAT/r2z/L7xQEfjdSz0iUbiaxvogSyNqokLe++Vf4gtLPV95XvxIFQT6qCzSJCDPGaIuzi5ocGR7RLIkcFWXuOVmnS2oF/ZaG9/Vq+fzbjguwmbAIJrFsQfUTZVNsa+uwPWx8MwV/SDYEsmK8uyv7Jnugn3MQeEGJgr6qL0mFgmoMzKHHdv5HFu92X5DGUgmVQNEZxA72K8rYLvgfEUPMhS2eDrSkmV1q2/2udh4ZxXEi0EUP/gnRzukzYPpTpEuHYE+Pac9qKE0urlvHLJ87lwizFQX0Uy6kAEEj5nIUXr0Q9JyjKy3rnJMmWPVBhjCjkgfQIxxQ/mKRujIYmVcgKgiJOM+nbJnkRxMwCDhlG3y075hRaxPBIZiqICClk9UDETrOl7sNa/m2Rh9Dnfe1M/4bg+oFvrO+Ydqy1lQShP/brDdt55rqpe1kbsFeh+8h8fbcE9MAKA59yZvfEuG8Uq1Es19L44PiRvbyrG7Bbbx4+RPEEdGHVBh+a2LJuoGhk4oFz92h+Jj9zgFIv11aBDt16wlA1wulNvpRJiwp9yCkw5P31nYxo/bBDXMgy3TJcfG1p2JP7o2WRj4TKSc0jwzRvavoBcTg1Rs7+sd/68WFPw8A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to undo the effects of MSG_MORE after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Tom Herbert cc: Tom Herbert cc: Cong Wang cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/kcm/kcmsock.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index ba22af16b96d..d0d8c54562d6 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -968,6 +968,19 @@ static int kcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) return err; } +static void kcm_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct kcm_sock *kcm = kcm_sk(sk); + + if (skb_queue_empty(&sk->sk_write_queue)) + return; + + lock_sock(sk); + kcm_write_msgs(kcm); + release_sock(sk); +} + static ssize_t kcm_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags) @@ -1773,6 +1786,7 @@ static const struct proto_ops kcm_dgram_ops = { .sendmsg = kcm_sendmsg, .recvmsg = kcm_recvmsg, .mmap = sock_no_mmap, + .splice_eof = kcm_splice_eof, .sendpage = kcm_sendpage, }; @@ -1794,6 +1808,7 @@ static const struct proto_ops kcm_seqpacket_ops = { .sendmsg = kcm_sendmsg, .recvmsg = kcm_recvmsg, .mmap = sock_no_mmap, + .splice_eof = kcm_splice_eof, .sendpage = kcm_sendpage, .splice_read = kcm_splice_read, }; From patchwork Wed Jun 7 14:05:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC669C77B7A for ; Wed, 7 Jun 2023 14:42:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E5648E0003; Wed, 7 Jun 2023 10:42:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 76D818E0001; Wed, 7 Jun 2023 10:42:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E8128E0003; Wed, 7 Jun 2023 10:42:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4771D8E0001 for ; Wed, 7 Jun 2023 10:42:08 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 008BF1602F6 for ; Wed, 7 Jun 2023 14:42:07 +0000 (UTC) X-FDA: 80876216694.16.B2622BF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf08.hostedemail.com (Postfix) with ESMTP id 23446160014 for ; Wed, 7 Jun 2023 14:42:03 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="M/t9JxpG"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148924; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VfP0KSvbwDpdDK3r99Lo/bGf0PzU9/ETfdNk/1wpqNM=; b=gP0ByWR0Rd7sP557iVrb+4r2WK9sDgokHEHx2x9rWhQhrIXpmnu4Lw845GPRZkAAMOU8ff llFd7ybcN9K83WFu/XMz8qVWue62n1ddRsh9WBUPTooXsc0sH2zm82DqnnjLW9opwR8up+ n79llniYEkpCj+WmVkrR+AJKVZFRF10= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="M/t9JxpG"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148924; a=rsa-sha256; cv=none; b=VdfDK2dISnjEfA66nYHFQ06gKLGocAd7UjQ1FEG9riOotYhh/V2onYJBEnZ9HjySCRlRbZ x0etKg8x3gYGX1VXeZKBQBC+SLxrEa0fmyd5pPa9EEODerT2oI1f2KP+4tQgMujfFB5jqi dX3w5djt+IzWXsEj8OBszBFrOyAtdBc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VfP0KSvbwDpdDK3r99Lo/bGf0PzU9/ETfdNk/1wpqNM=; b=M/t9JxpGG1lbOnwlNGraRSDI9k3eoqUMm34VSpojC1SuXDkyN1Q6V7yXoxUuV2scLYcIb5 KVaangsk4sJou22PCdyWtVM+WeIKAxTmW9KtQvq7DDjttR44yCnUcfjAgi49Jzc9eFwvxu 3AN1CdREjEuFg0MA6MSIAvMdFvKTfXA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-81-yTkbrzWfOVqVewkXPojfRg-1; Wed, 07 Jun 2023 10:06:48 -0400 X-MC-Unique: yTkbrzWfOVqVewkXPojfRg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C33823806705; Wed, 7 Jun 2023 14:06:47 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id F23512026D6A; Wed, 7 Jun 2023 14:06:43 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Al Viro , Jan Kara , Jeff Layton , David Hildenbrand , Christian Brauner , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH net-next v5 10/14] splice, net: Fix SPLICE_F_MORE signalling in splice_direct_to_actor() Date: Wed, 7 Jun 2023 15:05:55 +0100 Message-ID: <20230607140559.2263470-11-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 23446160014 X-Stat-Signature: fpx8nffmayt565pgezu7fifjdzro3y45 X-Rspam-User: X-HE-Tag: 1686148923-660481 X-HE-Meta: U2FsdGVkX182XdGwyBRcGSCpukLjEXlaeTbZsqt5+/RPi2X/s7skeZvkL8Rc7PVQWJueR1qtclFV3/fuCvSV+ySpR31nh4uYQhydh4VdqzB8TTd3+JdIcD63JteCDLJXFan0Oq0k5kRP+0ErBvreTPnjn3FdYvF6ApHcEavwE2ybMQPBLOCe3EKtgi/Ijuy5UxnlP4mZIE2eeI46Pue9ok5OiaPZgeeDjCXC4a4iStrkY4OUKejK1+wPtv7XYoxHuakgbOSCsuMQ0fMAkzLNhLWwTa2DROMG4tZGdPIoC7ufgBMazLC4PJoGX3df3jiVppdyj4RvmugThuJABH+BGkigF/u0/QZod4E6AxBmGVgw+QlTRS7n9CGLYmL55k7oFjRpNCDjCD7zSDmShMDdEqJTB7XbQscHGVqjBJ3BE20/lT0k+Z09ytvvrxVTF67cnngwwuC+S78zQyqDD4J3qWyIyQy0rkvIkU6Z19Ixb7d6UCyfqcQfCh4UKdihGJ0o9MojyCZWh97JkAmybEYPCmy8nrgQLMFVki7qXBhH4vLr9vxky2lLzBTaXyDNW1BujDm6M0naRdm9jfsIi1M/cblbib4MOK/f8fc/CJpiQirTRoeWzJeP59AL8USSxzlWB6iH45gds4gtARnfB06ZOyxD/bRP2DP3CxqJkx+i2BIR6iLM68L2Tkfw2fnv/UC+IgB0uTMQrDpsPJN78LDBix6AoNQEPbwIfCUDVIow1mcj3MZYlWzVzNcKhJGh7ep6kZcDFzSwYk1h+E7oU1Cm8Q0aeRdMw02NVgVQG1bXJfp8Gt4SRYa3Rs7/74ALczOu0Sg9QGZ2fJ8Wq5Q4KxNMuKx2W4B8ItaNgZWwCGH0BHk9waH1VWPduvEU/RQVIR6Yg8QUOW2IBNmEUQjjvPcYA8FenoJhyKZYAmoF9isztOtTSCALK8FOENz1K1J4dxlKWQQfgeBlaJbJjt/GrOW vU+IOkMX F3O+1mmryEL3BETo1naRiMGo3ob4FsMw5KKQlYyHDGXmFvAVujhMl6cACGaWGUrkkB2jc4d2NC2u0oLYdZmx+YkrELz+lkCXE2vJACzmB+wZBVfWXwbCsjcQHGFNTLI8vQDHcacVaPNk876ZYnJiYZ4TwG89JwstaTM97+bI5y62NF0hkQlSSlUPeEmTX7d8FYlxrgN4/g7vTENPSDpSRrSZTQXgrSLO045sFFpZCpziqsFCC79zAtMRm7x2AEz31/5tcDWSax65vpWOcnEH1UG5gBu2i/bT/NLja8YkuVvhmYDwSqIENpPTD+pGi0q5sn0qzQqGzkO5fdabFWtbUJS9rqpC2bjAI3BtjG/BpcwZJzMpUXQ30ROi5xqgtd02E0wXBDc4nQVwyCDixqI+CVxyjTbba9OrSGUqiQDsOATyiAuE4c82/eHrRICpYIS2UO42q8mQt9EMgKc2V/zvsboGsNjSe1x8VahlhAuAOhhnAYRveLW8QlSTovRI648jxJnB8htMPG4DethuWi6TVsOJium6IMWHH432TR+6MkST9vQx+XfRYU1/tu9ouTntywOXKc/VuRQl8M0Fj9LG3D7HgOfxK8/oqLvuyBpm//Ephcp2wt2ndHVIljxky4k69mpNR0ujfnkUVnGCEb2XLX3O9lWvCQgGpiDdw+KwZ2MQid65e5bBys4iHCpj+inxwIHi4B8qBWDXnkEhdSHGNdW1Ik1CIK8/jLw6UF/v5jwU8FVuXu62vPK3EczuADBMGbsGV4RWdXaFfET3FZo6arxAiQvtoPWuIOu3L/vgaKrBVAytN/dsOAoJ7Ccd7S8hkjq2k3fLqwRrlHfvvLVmepoea9VSdAavzR7KSn2AUITqWnVU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: splice_direct_to_actor() doesn't manage SPLICE_F_MORE correctly[1] - and, as a result, it incorrectly signals/fails to signal MSG_MORE when splicing to a socket. The problem I'm seeing happens when a short splice occurs because we got a short read due to hitting the EOF on a file: as the length read (read_len) is less than the remaining size to be spliced (len), SPLICE_F_MORE (and thus MSG_MORE) is set. The issue is that, for the moment, we have no way to know *why* the short read occurred and so can't make a good decision on whether we *should* keep MSG_MORE set. MSG_SENDPAGE_NOTLAST was added to work around this, but that is also set incorrectly under some circumstances - for example if a short read fills a single pipe_buffer, but the next read would return more (seqfile can do this). This was observed with the multi_chunk_sendfile tests in the tls kselftest program. Some of those tests would hang and time out when the last chunk of file was less than the sendfile request size: build/kselftest/net/tls -r tls.12_aes_gcm.multi_chunk_sendfile This has been observed before[2] and worked around in AF_TLS[3]. Fix this by making splice_direct_to_actor() always signal SPLICE_F_MORE if we haven't yet hit the requested operation size. SPLICE_F_MORE remains signalled if the user passed it in to splice() but otherwise gets cleared when we've read sufficient data to fulfill the request. If, however, we get a premature EOF from ->splice_read(), have sent at least one byte and SPLICE_F_MORE was not set by the caller, ->splice_eof() will be invoked. Signed-off-by: David Howells cc: Linus Torvalds cc: Jakub Kicinski cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: Matthew Wilcox cc: Jan Kara cc: Jeff Layton cc: David Hildenbrand cc: Christian Brauner cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org Link: https://lore.kernel.org/r/499791.1685485603@warthog.procyon.org.uk/ [1] Link: https://lore.kernel.org/r/1591392508-14592-1-git-send-email-pooja.trivedi@stackpath.com/ [2] Link: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=d452d48b9f8b1a7f8152d33ef52cfd7fe1735b0a [3] --- Notes: ver #4) - Use ->splice_eof() to signal a premature EOF to the splice output. fs/splice.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 67dbd85db207..67ddaac1f5c5 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -1063,13 +1063,17 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, */ bytes = 0; len = sd->total_len; + + /* Don't block on output, we have to drain the direct pipe. */ flags = sd->flags; + sd->flags &= ~SPLICE_F_NONBLOCK; /* - * Don't block on output, we have to drain the direct pipe. + * We signal MORE until we've read sufficient data to fulfill the + * request and we keep signalling it if the caller set it. */ - sd->flags &= ~SPLICE_F_NONBLOCK; more = sd->flags & SPLICE_F_MORE; + sd->flags |= SPLICE_F_MORE; WARN_ON_ONCE(!pipe_empty(pipe->head, pipe->tail)); @@ -1085,14 +1089,12 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, sd->total_len = read_len; /* - * If more data is pending, set SPLICE_F_MORE - * If this is the last data and SPLICE_F_MORE was not set - * initially, clears it. + * If we now have sufficient data to fulfill the request then + * we clear SPLICE_F_MORE if it was not set initially. */ - if (read_len < len) - sd->flags |= SPLICE_F_MORE; - else if (!more) + if (read_len >= len && !more) sd->flags &= ~SPLICE_F_MORE; + /* * NOTE: nonblocking mode only applies to the input. We * must not do the output in nonblocking mode as then we From patchwork Wed Jun 7 14:05:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F5BC7EE23 for ; Wed, 7 Jun 2023 14:32:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA3858E0005; Wed, 7 Jun 2023 10:32:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2C538E0001; Wed, 7 Jun 2023 10:32:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACD028E0005; Wed, 7 Jun 2023 10:32:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 98F5D8E0001 for ; Wed, 7 Jun 2023 10:32:58 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6B8961A026E for ; Wed, 7 Jun 2023 14:32:58 +0000 (UTC) X-FDA: 80876193636.22.E3FE79E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 2C84D40026 for ; Wed, 7 Jun 2023 14:32:55 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=J07mOMpy; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148376; a=rsa-sha256; cv=none; b=DMUfVi/UzT3wL6PGKdRBJ0cI+jYWHotMa670hAOvJ+8DFHNsobRNPotIjaRGqgEqO2Qg9q cj7VGT0E6g2uiuxIJZTKukqP/TcOEdfOhmHY20H4yMLN8i+gqgkmMDnBME6HhGZyjyfGMZ wbcqkDn34QORxaMzrDNousqSDfsfvLk= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=J07mOMpy; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148376; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mOKXaftBBhMTIlFKfwLIH8NhxIpiz+rLzmWYl6SoU0k=; b=YCPQ4vmv6yX0f6RkVBlq1lyPy3JFjB3Sa2CBI2fMu1xyNffqqplMbd4Wo74vgUL7XtbG7K aYfkTyeIwC5uaa/JKsveQbVJfo+W+mYfLh0InBClWMxclVLuTFj+FdYjDBRxsMwtNCO+YY hOfmbUJumUHL/qy1ztzkCkqa2QlBMkg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mOKXaftBBhMTIlFKfwLIH8NhxIpiz+rLzmWYl6SoU0k=; b=J07mOMpyq8B20fZrjMWuPZiMnNzRVsyeLEBl2HnYKVri9doRqd6TPf73pH+WUmKvkeHpce QB1Kx4lQoFBYQgADIhlThtpb3n9wgGsGMX40uC9T3WA8v8N1L+oxGD63FL6OuiR5pXflkj niXQtbRp3Y0STjngD4mZwsok0YZOjRk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-581-691rPYokOqqZ4ufsxb1e3w-1; Wed, 07 Jun 2023 10:06:51 -0400 X-MC-Unique: 691rPYokOqqZ4ufsxb1e3w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E32302999B2E; Wed, 7 Jun 2023 14:06:50 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91F90140E954; Wed, 7 Jun 2023 14:06:48 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 11/14] tls/sw: Support MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 15:05:56 +0100 Message-ID: <20230607140559.2263470-12-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2C84D40026 X-Stat-Signature: cm7jfqaefescfkdk1wtf174tmhi77oux X-Rspam-User: X-HE-Tag: 1686148375-244976 X-HE-Meta: U2FsdGVkX19a3WnPlviukEV/zSUVzQHaYhUQCSS2go/p7+Nf3kqCIdH7Jqd4Q+1NJ1i72rT6N95DwmWvG1L2Bp/bqOEKCuh3J6MTr3aRudrTS4z2p6rpdAIdF27sk1isVduzOTJ8Blg4iAF13ucCv9sNnkw4WWaSd2PPl1ZoYn70gzgMgpOVWqe2d9N/ZUYgvuWL55ELFX4oaWMt5YYJz4XCTy7e4NX8QIhZEUGOZQMnxA5087SjMnhhUpn+VF/ggRjwVD8ujy0Q3fXKI5yyMw1Zjp3MKBJk50d+Qpq42TCMgUEho8G0NQGDQW4L6qu8brlP5RhNVQK3qd9Iqm2E/9opYSus1rV3oBgs/aVE4iYjZcRhixBbeYRgY9w2avDd0eyR5XwVf6kAvJMmUCIaxgyG0lL3KW0+AZfAUxKbAuqby1PIfhe8iPkIPnH3iokC6tgUlfYEUvWvjrh0AXAq9NZYBaEbtoOhtj8VO72QBw3Q7V0uduzTROXoTkbcrAraSGHzyVuO9ODWJOHk339Xe4SAu8eCDTITbqG5gFURQg3rFz7JpCQz2wsgMPt+MTNAG7EW9Bu/PCx4QNmASCbqCs5kZIhDXS6lCdEa4Y29ONrOqVct8kdug8fSSq9bi0hd/MTLHohXpkvmCmH7jZzGSvAvTXgeY2xnY5wWw2OWvRxi7T5V749uBwX8ulHB8GPnqsoJR5431bznY7+B0wZDQZadr/412Qe/lQ3s1qbWinVYcQ2OknAKYofDePic+CEClzJ33HMTo6oHHMGj4khGmPyHCnz9xlLZDleoxgNoNTf9kjWAf9eOfugftRqWjT/Pp3VyR/VETxgZ0xsXJVJVha4NtyvEsVdjei3fr1ewYJPzz//6niYeTGgpNl10R/jxzxa7gPgUobdRgqtBGVolR6pzyQztauMcJvc7zTDm4D3xwvF15JtyYgJg3RAcS6PxrVLcPMpffG+DB+m1UQE Vdr/zIUi q3z3H0RIUMv7ePvNwCDbsmczrJjOC8jbYThP1I2pIHK590IpDtJ6hJKXDciBZeP0EL28hPh+zfkMArj3YjlY0PhqsE6FWLF7MISMIUX57Pu22pM0TBVyD05C5pbMPCPYSsWUeM9i8BeNnEKC8+Me8otoBfWl5mwAHXk2uc6B4a0li7keOLPWJSF0SqnbGU4/azAf1wMJVTr/Jukv6D8nm3aTXwjhI4IV/fTuOdfvA6rZsouyX9MUMSOD7YUz6pMDNCcRzLx3ISJEJ0BNVVqaW8gGMMDWhhbKCH9Pet9UFYV5gNCv9KYz2COxZTxPoDeT9QbQFvKgp+koSR2znGEvDTSSYB+vy9OrKGLJg9PYJoInIBKCvlQW9bprgRn6GSAPSj3Z6USoV9OktOamecanol/vqhSZIecMstKiyXbNLRMxKFCL+KC15kwvJBIrJSsuvqoMm8dEJWcEJjOCvyKXXx0J0Vg2MafaR5miuTJ3E29MRANQsboZrzasPuDt7qPoTlDGTyZeNH73aPEl/CpsXBB1Xvd0vw/lLkey9W1VsYxTPRkXOI0jjxBfXXg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #2) - "rls_" should be "tls_". net/tls/tls_sw.c | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index a2fb0256ff1c..fcbaf594c300 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -931,6 +931,38 @@ static int tls_sw_push_pending_record(struct sock *sk, int flags) &copied, flags); } +static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, + struct sk_msg *msg_pl, size_t try_to_copy, + ssize_t *copied) +{ + struct page *page = NULL, **pages = &page; + + do { + ssize_t part; + size_t off; + bool put = false; + + part = iov_iter_extract_pages(&msg->msg_iter, &pages, + try_to_copy, 1, 0, &off); + if (part <= 0) + return part ?: -EIO; + + if (WARN_ON_ONCE(!sendpage_ok(page))) { + iov_iter_revert(&msg->msg_iter, part); + return -EIO; + } + + sk_msg_page_add(msg_pl, page, part, off); + sk_mem_charge(sk, part); + if (put) + put_page(page); + *copied += part; + try_to_copy -= part; + } while (try_to_copy && !sk_msg_full(msg_pl)); + + return 0; +} + int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); @@ -1020,6 +1052,17 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) full_record = true; } + if (try_to_copy && (msg->msg_flags & MSG_SPLICE_PAGES)) { + ret = tls_sw_sendmsg_splice(sk, msg, msg_pl, + try_to_copy, &copied); + if (ret < 0) + goto send_end; + tls_ctx->pending_open_record_frags = true; + if (full_record || eor || sk_msg_full(msg_pl)) + goto copied; + continue; + } + if (!is_kvec && (full_record || eor) && !async_capable) { u32 first = msg_pl->sg.end; @@ -1082,8 +1125,9 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) /* Open records defined only if successfully copied, otherwise * we would trim the sg but not reset the open record frags. */ - tls_ctx->pending_open_record_frags = true; copied += try_to_copy; +copied: + tls_ctx->pending_open_record_frags = true; if (full_record || eor) { ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, record_type, &copied, From patchwork Wed Jun 7 14:05:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76558C7EE25 for ; Wed, 7 Jun 2023 14:38:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 092988E0005; Wed, 7 Jun 2023 10:38:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 042D88E0001; Wed, 7 Jun 2023 10:38:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E260C8E0005; Wed, 7 Jun 2023 10:38:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D28198E0001 for ; Wed, 7 Jun 2023 10:38:11 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A8034160262 for ; Wed, 7 Jun 2023 14:38:11 +0000 (UTC) X-FDA: 80876206782.17.BCF1ED2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 9FC504002E for ; Wed, 7 Jun 2023 14:38:09 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KmtRkVTg; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686148689; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LDTGeEfLo2GvEbWeYwCYunQ9JPRlxJinWTpGXOSqvLY=; b=I0ZOecNxmZhefAc0vp8Rvf3W+zKhlnoQOsusmnz7KqSdp/SruUs9zJrvb43KPcZ3iQ5K1B let8HRxrn4O7GMlw5BV3GfqGhzfndBVduCZ2XReH/1lGcaYVqpV5z6aEerObdyJ/DPnOLe PGHun+aHAid5sRve7wpGcYmKL5rM0h0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KmtRkVTg; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686148689; a=rsa-sha256; cv=none; b=fKSbkubuSwS4T0l/YXcAQgPJLVUPJZHAG+YK9Bvq/gHP4OX/oKXv+Of8pr7ZCLEEZhsC1d bfbZIyNOa5cqZp682BjnbiLXUpueI58utFyLI6Kn3+ps8irbiQwHN88rWvZhhd8xbUmqAC WipyD1R5E/xamq27+7aeW5TH08Sak34= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686148688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LDTGeEfLo2GvEbWeYwCYunQ9JPRlxJinWTpGXOSqvLY=; b=KmtRkVTgxunsBi7PVJu8HJVILYyI8E7eWwsCjPKtsq5zdkgm+oXZHlbbxnncvRXRA5JsV8 5sy0n+LVOFcpkHGC7fZdnjmy7XK9MCrPmoKC8osxOAkrNAOUHgJLkOap2lN1Pxb+zw3HnW xA3uwZ0JajyMeuHOt10bJwUqKo1SFXE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-331-X5b66WfYOJqYx_MNR-dG8A-1; Wed, 07 Jun 2023 10:06:55 -0400 X-MC-Unique: X5b66WfYOJqYx_MNR-dG8A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C08E2185A78B; Wed, 7 Jun 2023 14:06:54 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA26E2166B25; Wed, 7 Jun 2023 14:06:51 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH net-next v5 12/14] tls/sw: Convert tls_sw_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 15:05:57 +0100 Message-ID: <20230607140559.2263470-13-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 9FC504002E X-Stat-Signature: k8o6imw1u6ghf73cd4ih5zitwngb6ozd X-Rspam-User: X-HE-Tag: 1686148689-112244 X-HE-Meta: U2FsdGVkX1/TPte5vnfrw/PkJZbFDPFhFWj8M+2D193354dbsp5loUrb5CsBB7fyJ3nyjc5AeCRQ6h8IWxLPGMnRX0pxIn7kjl/VliaHdXnASiw/xUd6vkp7MnYJHOBW/sQUDYaWoBM1N3aXI6XLW1GDQUjAVASEo3Ow0EYSNrEe3J7k/tfQHbAwdm38NvO4zjBQW3ABC+bP4x4Fzcxln6VIEtf1XDcGJZHz7JdEucFXlcVSMy1mqEadpH3rVmz46f+Zj5OpJ5maLBJnPROa0TzHuVOoWW2v7fDqBQLn03VtYIqBk+MoOHaWQHhYI4TA+TGTYd6T4427KVf828r8ix0EXmcTDaLPhUsEY1wMOAKeTScankzZSUbLTYmqKwl7uaLcghCdyQ1qE90LOb7QJ2mmwov0EpsrP7R0g3T6vYX1rtQnR6lRs6whbcKxMnl64sesR20dKstT8ZTxye47kjwAE7OVvOwacVQ3GTK+ymHojv0x45K9pCZIm3xTxJnMIOW9FEQcKP5eR++KjesVf8+rT28vqUfgm2qUhMzeoUguhDrV+FkUkNE0DtWDnnrwVviF95MUmvkMHzz9U3alZS6JvXwDMIwZ7jt4XtPp8udGCGlnJV0ElJv6YfvIjelcJqlw4G6OeC3fQeaT1LDvaTqYYkbBrZT2SsRSD+ggfwuMmvSdLMfyA7qnhyPUl4iOL3BtsMMGjtKUWcJy2wAUniPK2yD0hEu/FqyeOIyB0914i0wxhdssYpVnsoGCwB8BwmDGiLN5fl1FqD4NwW2TKEdalgOrYRx9pwyCgiXpuDHT7vlxMfxmC0R9BAKzL3T0npOz5XRn1uSEyyBJj3ehsKLiwiSqIY9ZZGPXK5m65wpZU1G9a5pa2LWp5Q4YPDIKKZdKoqujoILwHckH65lglWw0myMDlArpTJNtX04EPCB3DcQm+kknql6V4t7kbYOL+U5KPOLoM9HXFmpmPyw 1ePG0PjS xqQe3lFgv32jWjVM2qtAGMLwd2CmO91bUut8UoRPUjwRDPk2R34lng4JN35394XEYpWcp+XGRw34pW39kRwCB00BgnxfbY4DRGj1P3+5HuNHJH3oo8dP5NWkOMz8pvh3nGSsZoq0tIkd0arToYXnJlRdSf0OwEHrHLbJBLMmVXDAcBqkE+UY25eEcjF+nxBeRPpxAkWSUOUyaoEFaWFIyuwU8ESGoUWkEzlNxyfAs3yP+G9nBNNCU1WlvIWPsa25XBmnX9yesveXAfhya3fNrjHQFq9EICTIjkRX5ag+K30/er+QglP3ONtzw2ByN79Yux268wn6h6nxK3800ro8brwGT4AFrK73f2SFyUFwf74JCG7onBDqmyV/L9lUuC2C43pm6i1nIIjJLQZCK37TGtz46v4WZQ+LWboTDL23iUTx3YZyTNnhKf9ZOR0SJMTjH/frt85I/rsYmJmMQ1lnNw+6X+j6cY7dDaNc4BiquJg59/fTRPYv2pzIAwW0FT/dLP6oJ/tzta6IxGduck8XP6Yr7kHb3xCLLaKWcfJYJnL+KnDOK+BjRI8zYQw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_sw_sendpage() and tls_sw_sendpage_locked() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. [!] Note that tls_sw_sendpage_locked() appears to have the wrong locking upstream. I think the caller will only hold the socket lock, but it should hold tls_ctx->tx_lock too. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org Reviewed-by: Jakub Kicinski --- net/tls/tls_sw.c | 173 ++++++++++------------------------------------- 1 file changed, 35 insertions(+), 138 deletions(-) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index fcbaf594c300..aafa02bb5723 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -963,7 +963,8 @@ static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, return 0; } -int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) +static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, + size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); struct tls_context *tls_ctx = tls_get_ctx(sk); @@ -986,15 +987,6 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int ret = 0; int pending; - if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | - MSG_CMSG_COMPAT | MSG_SPLICE_PAGES)) - return -EOPNOTSUPP; - - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - if (unlikely(msg->msg_controllen)) { ret = tls_process_cmsg(sk, msg, &record_type); if (ret) { @@ -1195,10 +1187,27 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) send_end: ret = sk_stream_error(sk, msg->msg_flags, ret); + return copied > 0 ? copied : ret; +} +int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) +{ + struct tls_context *tls_ctx = tls_get_ctx(sk); + int ret; + + if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | + MSG_CMSG_COMPAT | MSG_SPLICE_PAGES | + MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) + return -EOPNOTSUPP; + + ret = mutex_lock_interruptible(&tls_ctx->tx_lock); + if (ret) + return ret; + lock_sock(sk); + ret = tls_sw_sendmsg_locked(sk, msg, size); release_sock(sk); mutex_unlock(&tls_ctx->tx_lock); - return copied > 0 ? copied : ret; + return ret; } /* @@ -1275,151 +1284,39 @@ void tls_sw_splice_eof(struct socket *sock) mutex_unlock(&tls_ctx->tx_lock); } -static int tls_sw_do_sendpage(struct sock *sk, struct page *page, - int offset, size_t size, int flags) -{ - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - struct tls_context *tls_ctx = tls_get_ctx(sk); - struct tls_sw_context_tx *ctx = tls_sw_ctx_tx(tls_ctx); - struct tls_prot_info *prot = &tls_ctx->prot_info; - unsigned char record_type = TLS_RECORD_TYPE_DATA; - struct sk_msg *msg_pl; - struct tls_rec *rec; - int num_async = 0; - ssize_t copied = 0; - bool full_record; - int record_room; - int ret = 0; - bool eor; - - eor = !(flags & MSG_SENDPAGE_NOTLAST); - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); - - /* Call the sk_stream functions to manage the sndbuf mem. */ - while (size > 0) { - size_t copy, required_size; - - if (sk->sk_err) { - ret = -sk->sk_err; - goto sendpage_end; - } - - if (ctx->open_rec) - rec = ctx->open_rec; - else - rec = ctx->open_rec = tls_get_rec(sk); - if (!rec) { - ret = -ENOMEM; - goto sendpage_end; - } - - msg_pl = &rec->msg_plaintext; - - full_record = false; - record_room = TLS_MAX_PAYLOAD_SIZE - msg_pl->sg.size; - copy = size; - if (copy >= record_room) { - copy = record_room; - full_record = true; - } - - required_size = msg_pl->sg.size + copy + prot->overhead_size; - - if (!sk_stream_memory_free(sk)) - goto wait_for_sndbuf; -alloc_payload: - ret = tls_alloc_encrypted_msg(sk, required_size); - if (ret) { - if (ret != -ENOSPC) - goto wait_for_memory; - - /* Adjust copy according to the amount that was - * actually allocated. The difference is due - * to max sg elements limit - */ - copy -= required_size - msg_pl->sg.size; - full_record = true; - } - - sk_msg_page_add(msg_pl, page, copy, offset); - sk_mem_charge(sk, copy); - - offset += copy; - size -= copy; - copied += copy; - - tls_ctx->pending_open_record_frags = true; - if (full_record || eor || sk_msg_full(msg_pl)) { - ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, - record_type, &copied, flags); - if (ret) { - if (ret == -EINPROGRESS) - num_async++; - else if (ret == -ENOMEM) - goto wait_for_memory; - else if (ret != -EAGAIN) { - if (ret == -ENOSPC) - ret = 0; - goto sendpage_end; - } - } - } - continue; -wait_for_sndbuf: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); -wait_for_memory: - ret = sk_stream_wait_memory(sk, &timeo); - if (ret) { - if (ctx->open_rec) - tls_trim_both_msgs(sk, msg_pl->sg.size); - goto sendpage_end; - } - - if (ctx->open_rec) - goto alloc_payload; - } - - if (num_async) { - /* Transmit if any encryptions have completed */ - if (test_and_clear_bit(BIT_TX_SCHEDULED, &ctx->tx_bitmask)) { - cancel_delayed_work(&ctx->tx_work.work); - tls_tx_records(sk, flags); - } - } -sendpage_end: - ret = sk_stream_error(sk, flags, ret); - return copied > 0 ? copied : ret; -} - int tls_sw_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags) { + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY | MSG_NO_SHARED_FRAGS)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - return tls_sw_do_sendpage(sk, page, offset, size, flags); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg_locked(sk, &msg, size); } int tls_sw_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - ret = tls_sw_do_sendpage(sk, page, offset, size, flags); - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg(sk, &msg, size); } static int From patchwork Wed Jun 7 14:05:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 313FCC7EE23 for ; Wed, 7 Jun 2023 16:25:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A08506B0072; Wed, 7 Jun 2023 12:25:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 96A148E0002; Wed, 7 Jun 2023 12:25:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80A948E0001; Wed, 7 Jun 2023 12:25:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6DEBE6B0072 for ; Wed, 7 Jun 2023 12:25:47 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3B16F1C7802 for ; Wed, 7 Jun 2023 16:25:47 +0000 (UTC) X-FDA: 80876477934.25.52A3243 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 376D7C0027 for ; Wed, 7 Jun 2023 16:25:44 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=atsy9MJ+; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686155144; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/RO1yxPz5vv1zCyaq/Wi9xLZBTYHqYC8FIFy+4PrNrg=; b=r+PTAiB/4e/pvWSLaXRJhaGer3n0cjz+d76iy06XKG45eImEyU96tC0O8v3UcgQv5kRmUT MIo+5vnO/EPVSwfpsL9hl/XUNgu44/7ELzj+FKoWL+mfC7fOGxzGoFKLql5BODTMbVn2bt CFsBMnP7G0vA/L9/wZgoAs4PTe1pqaI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=atsy9MJ+; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686155144; a=rsa-sha256; cv=none; b=VUEDC3swgvRMXnxgLZ0QNV+yKwxWyHoOG08DGlrFVILC2AKW/8KBiaUKrRNzFK9h3oqfWe b7/w/1UC+n8hSan4ZteTidI7u9d3qtz3uCiDMEU7qZlFiYYUcLdzUOstOMcnnB2aixJBqu 6srjJ8qqy7AipAPVJBTjMxTf4Gy7Nkw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686155143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/RO1yxPz5vv1zCyaq/Wi9xLZBTYHqYC8FIFy+4PrNrg=; b=atsy9MJ+JprhZDs5eXZTLGhfM7c+OPKEuWCcYmIKx37kUkY+IfQV4MdDFa5Q984CwUnavL SaDwsKOuR+X0CQloXHT7CzqFVW3qfx3VIE9oweyDtBPHy1UZvOeJnBXfzXm6ksbg19E2dp RPR9aHamhw+KXfdX3fdy8d6I6lPtBmU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-2UqzVZS9PQ2uhUjL3YSU2g-1; Wed, 07 Jun 2023 10:06:59 -0400 X-MC-Unique: 2UqzVZS9PQ2uhUjL3YSU2g-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C3ABF811E7C; Wed, 7 Jun 2023 14:06:58 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C5992166B25; Wed, 7 Jun 2023 14:06:55 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 13/14] tls/device: Support MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 15:05:58 +0100 Message-ID: <20230607140559.2263470-14-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 376D7C0027 X-Stat-Signature: hk1nod1o5rjpq7zobd49zidwnuoy61gu X-HE-Tag: 1686155144-891968 X-HE-Meta: U2FsdGVkX18TcDbTXo1NHF5232xxby+gudO+8MXp/gzjS8Pnggj172Eo7CegWkBjke1ChEjzKHoTfEb5o8mkpnwbwQY6v8GTsidYjG1H860/RKfywbm3xz0HsQ36SwQ8ceYVS4cBJkjpI/zDLIMzWqvQ+i7yJAVoIgoaHh9C6ZfoAGsVAjGGxdHpsd3L4LDNf30Sd+HMCQKrME/FYOzqGz5ap3v4PMgjsZ+ZnfaqKYH51h/UuHFF9e0AUWXE4GUwi2cbg5Znb/Vf/HDYaDm43gmBYN8aGrheCG5JOlKU2R1Kr5zQIqiqj95Ui8zOfaEHEF50s2LuvAofA94MV7qEOxfEa3kIDFlnrX1gypcERFgjQryhbXgIVvMukLzReJfIKflw2c3KMSK2Ty+97F6KA5OAiJUqwwlBBhCycsFn85Eg3nM5sLBUdBC00FWrE8dqCfTUNooz0pWxXIkHXh5Cmn/GxdDX/2fj8LWOYIhq4ArERoKB/1s+4NdLptQ0Ogl5oyUyn+Qj756lx3ojc6s2FDNXIYW+BWgfWJUwm7FLOPUwhPKynhSyyRIc7pLdPMnpveO2tRjVtBl2lS/T+1nrCvuf6uKM49/QAIfxc056zd8y3MWgEeZ7n/DpfE78PjBcEgQdesjI6lvlcFYYIW8U0+tQHt7qfTmzqQGf38eJKYv5IUaSZcu8oS1WgFajBRWEMecvhhv2J8QA3vM2leX5P2YEl4GOUtYdvv/VIfsnBrqUYRUw316QR8/rrYDmAMQdMgT4IGeMDe96rITSxQ3bscuEkLM3fb2uhYRtPcnBKxzaw8A6cqtGtRMHs/4tlHXpGWuciYsgsenF2PLhr8YSuI+cTFV8ZRpYCnNodYh74EH8TcGjeErmkTu9nc3Wqqi62uFzZgv/buSjJex6W888qB5V5lQ+Mum4QD9Sp6yiZEQrcphJWWqhRlvWAECkcWNXZQDvf2m1GapdQnF6SxZ V17006lo 1rE1Wl1omm2amn5fDHe5J8oz9Jym36gj4EX8k4MRGSunOQaMzHmd4pZA6Eo3Nui1yMT+MthkYBg7c7RLBQXH5Xf+DybBkdbpnQHoKCkcnP4D5Xs18WLUM/yG65NZiedDdlPZNxB202OeOryTRitdcp1cGbMxNQaHiFhdkx+r59Pb7ofSEIuVd78qkb67O50YlJ22eyFu7uuobg6r/PZ9R8nIlmMVZxnbZOYonrdbEgOUAWIJ2YOdSjxrhot9umIDsiEEkBAfYUxGFTVASxFzElScXWOnzhu9WSnP+S4660gLJffS8qTaRKx8IwXDN0J2ID7fqyHRWJZ2MHSaf8/2gtULdfRAEkgG8Xru9dyvFKomfmSakSoxMQgbQWPguycfiiw4PaaDk0OBO2HL9eoXb4KoI+lgbevOLquf9UtlKmjJove75gb+b1Vm+iDurV5axmN2RIgIZqt1FNOjrS52g30cCGYYMIAZKkv+x7aDOb+/ZfvjDwG44dgSUuVCnPeL4+6kTlCCSP5JHrnBwYoSmrEKpALMI7nrAEdnojGpI7xyynuWEDavhPtB/2Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's device sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Reviewed-by: Jakub Kicinski --- net/tls/tls_device.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 439be833dcf9..bb3bb523544e 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -509,6 +509,29 @@ static int tls_push_data(struct sock *sk, tls_append_frag(record, &zc_pfrag, copy); iter_offset.offset += copy; + } else if (copy && (flags & MSG_SPLICE_PAGES)) { + struct page_frag zc_pfrag; + struct page **pages = &zc_pfrag.page; + size_t off; + + rc = iov_iter_extract_pages(iter_offset.msg_iter, + &pages, copy, 1, 0, &off); + if (rc <= 0) { + if (rc == 0) + rc = -EIO; + goto handle_error; + } + copy = rc; + + if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { + iov_iter_revert(iter_offset.msg_iter, copy); + rc = -EIO; + goto handle_error; + } + + zc_pfrag.offset = off; + zc_pfrag.size = copy; + tls_append_frag(record, &zc_pfrag, copy); } else if (copy) { copy = min_t(size_t, copy, pfrag->size - pfrag->offset); @@ -572,6 +595,9 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) union tls_iter_offset iter; int rc; + if (!tls_ctx->zerocopy_sendfile) + msg->msg_flags &= ~MSG_SPLICE_PAGES; + mutex_lock(&tls_ctx->tx_lock); lock_sock(sk); From patchwork Wed Jun 7 14:05:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13270815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA0E1C77B7A for ; Wed, 7 Jun 2023 14:46:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6AB0B8E0003; Wed, 7 Jun 2023 10:46:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65A218E0001; Wed, 7 Jun 2023 10:46:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D63F8E0003; Wed, 7 Jun 2023 10:46:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3DA118E0001 for ; Wed, 7 Jun 2023 10:46:25 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F2234140206 for ; Wed, 7 Jun 2023 14:46:24 +0000 (UTC) X-FDA: 80876227488.25.B0B0E30 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf01.hostedemail.com (Postfix) with ESMTP id CDCD34000C for ; Wed, 7 Jun 2023 14:46:20 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W+0m1DLc; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686149180; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fV2I/xZhOL8TFCqO/iJxKQbBwABt3P8CReEOZMBw48A=; b=eRMpRXs8gfLn47ZQ71xY2ClB3Fs9qsxidkJ8avxbx1SusJXkZN9uVDg8YNEfLsgv4kkRzc wdYIE+/gjbwsGttqWPHHDw+WiiqWiM/NRdFmNr9pRAKi69tOcwI+oL388yfje0UkSSIVG9 g33pgNdNwZ17E4fatLX1JXid3CW3du4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686149180; a=rsa-sha256; cv=none; b=PO8bfQGaUSj1jYfzEzr2b9ramDSk1kkpx7/A/MOPTUXiCARUM2WGEvqHGzG5cEy9uAQIan 2qvciPGSdhYQvchC/4Ky9ju74UCakjSsWvKawN/3Tiv0vE85wQsBkte/2HhCpT1XGbCMrT wEI7kdtDkUG41s25D6opRSYq1TK+8UI= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W+0m1DLc; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686149179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fV2I/xZhOL8TFCqO/iJxKQbBwABt3P8CReEOZMBw48A=; b=W+0m1DLcSh0s9uZcvuJrAaINGRJ4MvZHyTJZ8cjZwXGOpTv3atwCDPLaDjt695g9rw2wI3 8EUEELNgmMWs5RaTlealwZQwtjjdIaKt+141HZNxKg3ER2iI5Au9+rroqtZwI5/e1piYxo Ch/9fsE5xBLlB2lcPstRESqQJKUe484= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-161-2n5XUvA3MTCdrcbHpThPWA-1; Wed, 07 Jun 2023 10:07:06 -0400 X-MC-Unique: 2n5XUvA3MTCdrcbHpThPWA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 210662999B2F; Wed, 7 Jun 2023 14:07:03 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1F8B7140E954; Wed, 7 Jun 2023 14:07:00 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 14/14] tls/device: Convert tls_device_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 15:05:59 +0100 Message-ID: <20230607140559.2263470-15-dhowells@redhat.com> In-Reply-To: <20230607140559.2263470-1-dhowells@redhat.com> References: <20230607140559.2263470-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: CDCD34000C X-Rspam-User: X-Stat-Signature: ykqzkc6w379xd4hp1ypifu87o5omtgh1 X-Rspamd-Server: rspam03 X-HE-Tag: 1686149180-432982 X-HE-Meta: U2FsdGVkX19jbTb58Gz53oPu7OgSlPN/WBSkJuRlibOlqFRPoB2UhgRPLlptHRzD3IExvlMigdLyMg5iap+w7GJfmQmtEd9u1xyiQPz1MbcxiiP7gHLbObAnybyL7tYG9CVjD1nwyJ+Qzfnj/OAhl18gXCnHZMo0qrzS65kRBApg1o9Euj2QmKbzvl7F7VjsWtdwKhPSansbV4/EKbXsJiEX5ADddw3M/DP4UgXVyVojMgG5TcCSiNmjV85jvotZXp5siooNgG1EZ2JhkStZYxwS5+yKJON80y0EyA5LSjPMEa8YGP92mrmTdS/ufyDiIMTKocUuFxb+mcMQfEKnocGUkK2sYavHKKLrTqBCxaUszdJQtA4ujs3kHbSrJXTk8+J5ZmisPnlNzdn0C8ZPAwzq8VG2aPIPJAJNKoOWFeeU4CKAVTgHdsJEDljZ20dOmPUmrqBWqlMB5K77QOE6cWvBQwa/HtLsRSUiQINt4YMV4drChwnf6YXROehWV5zcN7l8ycBMqfAqVKn5odOGivQ3yFZzUcVG9pu/xSHN0a8O7hDd+RZAt+y3emVMTdb+jrZCocEse4ZWa+DBF9CCGnE9vGD13K4oHJtbA/my6L7hcNUrLh5nrU23ey6oLemjUlEr9OT4U5o/WsqawgjjIRqoCVzBEF4SAw3IaYLLJhoFp0SMZGVY6QDTeOB6HwwKOMwCAzDZ5A/wBJRH8Ltc9jBsgO39lowyLOaRCAPs2sKCkByBJtrtlDb8PxJFtQohlMuJNSs7FxPQJsr2GR0HD3KnQc+kBVEZO6dHO5oNBY7OY+IxhtMpQaI/JDIQlAk58iphfSX9kkM1I4ydndoDkZk+MttKG7FlDP4HHnLVSem8i6+E3YKLYNQ2Lh5oK2ihsIxtZpu6mSflIplYQNXdQhrJ6kbl07IEPOTv4iVWMBJtigAv1W5sJqueomk9GTGnn+M7hwb0YtX6AK3izBe WwkqR7sw ttVRl6AZwKmU5ePgwbyfLmj/f36zIjxpP77EsISqSBgbWr/lXRJzPqnevH/6XKWnVEfNomRoc0oiDAFkzw/vZUU9jnH5Mg3A915zatYZcDZqKeINE9foGKxuZ0zhWtvj6rCbFkDiAI5AVf06yY1//bYJd6U05PKnlx2mL99LQodH6qgG1FJyqoOnJ53xIcdhk/wTDDFesf4WUuFJYp33dACXvFMBvazPqelERHwYb2QhCjcwpDiF1U4oapANZcNIqLS9soDdI1N+tsiORa4071hHvRqVWUO+cKW4jnig3kasSY+ELCbaJx/Gua3IvEGmiYHZgLytaTtdPmAPyf+zJSn1rlXqo5buXmF91C8TRCRuxLjChwUphA3UNsM0i2QsDRgh6sibQhsN996qw/qK/9UX+HIuTKDg5/+OSVCvzKiiQUyHtKORn/oBgYiti+teSy9B2F81aqjytWQfCR8yH8EWvGWVaJeAJLVmaBLBNU+KqxCG8r9JnyHBri7D0XdAovEgSq4sf0AiYnfeBD+fR9PqaSd7AZ9oyHEkC7nrDl5uhItVdSqOj6bB95A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_device_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. With that, the tls_iter_offset union is no longer necessary and can be replaced with an iov_iter pointer and the zc_page argument to tls_push_data() can also be removed. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Acked-by: Jakub Kicinski --- net/tls/tls_device.c | 92 +++++++++++--------------------------------- 1 file changed, 23 insertions(+), 69 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index bb3bb523544e..b4864d55900f 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -422,16 +422,10 @@ static int tls_device_copy_data(void *addr, size_t bytes, struct iov_iter *i) return 0; } -union tls_iter_offset { - struct iov_iter *msg_iter; - int offset; -}; - static int tls_push_data(struct sock *sk, - union tls_iter_offset iter_offset, + struct iov_iter *iter, size_t size, int flags, - unsigned char record_type, - struct page *zc_page) + unsigned char record_type) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_prot_info *prot = &tls_ctx->prot_info; @@ -500,22 +494,13 @@ static int tls_push_data(struct sock *sk, record = ctx->open_record; copy = min_t(size_t, size, max_open_record_len - record->len); - if (copy && zc_page) { - struct page_frag zc_pfrag; - - zc_pfrag.page = zc_page; - zc_pfrag.offset = iter_offset.offset; - zc_pfrag.size = copy; - tls_append_frag(record, &zc_pfrag, copy); - - iter_offset.offset += copy; - } else if (copy && (flags & MSG_SPLICE_PAGES)) { + if (copy && (flags & MSG_SPLICE_PAGES)) { struct page_frag zc_pfrag; struct page **pages = &zc_pfrag.page; size_t off; - rc = iov_iter_extract_pages(iter_offset.msg_iter, - &pages, copy, 1, 0, &off); + rc = iov_iter_extract_pages(iter, &pages, + copy, 1, 0, &off); if (rc <= 0) { if (rc == 0) rc = -EIO; @@ -524,7 +509,7 @@ static int tls_push_data(struct sock *sk, copy = rc; if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { - iov_iter_revert(iter_offset.msg_iter, copy); + iov_iter_revert(iter, copy); rc = -EIO; goto handle_error; } @@ -537,7 +522,7 @@ static int tls_push_data(struct sock *sk, rc = tls_device_copy_data(page_address(pfrag->page) + pfrag->offset, copy, - iter_offset.msg_iter); + iter); if (rc) goto handle_error; tls_append_frag(record, pfrag, copy); @@ -592,7 +577,6 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { unsigned char record_type = TLS_RECORD_TYPE_DATA; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; int rc; if (!tls_ctx->zerocopy_sendfile) @@ -607,8 +591,8 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto out; } - iter.msg_iter = &msg->msg_iter; - rc = tls_push_data(sk, iter, size, msg->msg_flags, record_type, NULL); + rc = tls_push_data(sk, &msg->msg_iter, size, msg->msg_flags, + record_type); out: release_sock(sk); @@ -620,8 +604,7 @@ void tls_device_splice_eof(struct socket *sock) { struct sock *sk = sock->sk; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; - struct iov_iter iov_iter = {}; + struct iov_iter iter = {}; if (!tls_is_partially_sent_record(tls_ctx)) return; @@ -630,9 +613,8 @@ void tls_device_splice_eof(struct socket *sock) lock_sock(sk); if (tls_is_partially_sent_record(tls_ctx)) { - iov_iter_bvec(&iov_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &iov_iter; - tls_push_data(sk, iter, 0, 0, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_bvec(&iter, ITER_SOURCE, NULL, 0, 0); + tls_push_data(sk, &iter, 0, 0, TLS_RECORD_TYPE_DATA); } release_sock(sk); @@ -642,44 +624,18 @@ void tls_device_splice_eof(struct socket *sock) int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter_offset; - struct iov_iter msg_iter; - char *kaddr; - struct kvec iov; - int rc; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - mutex_lock(&tls_ctx->tx_lock); - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (flags & MSG_OOB) { - rc = -EOPNOTSUPP; - goto out; - } - - if (tls_ctx->zerocopy_sendfile) { - iter_offset.offset = offset; - rc = tls_push_data(sk, iter_offset, size, - flags, TLS_RECORD_TYPE_DATA, page); - goto out; - } - - kaddr = kmap(page); - iov.iov_base = kaddr + offset; - iov.iov_len = size; - iov_iter_kvec(&msg_iter, ITER_SOURCE, &iov, 1, size); - iter_offset.msg_iter = &msg_iter; - rc = tls_push_data(sk, iter_offset, size, flags, TLS_RECORD_TYPE_DATA, - NULL); - kunmap(page); + if (flags & MSG_OOB) + return -EOPNOTSUPP; -out: - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return rc; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_device_sendmsg(sk, &msg, size); } struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context, @@ -744,12 +700,10 @@ EXPORT_SYMBOL(tls_get_record); static int tls_device_push_pending_record(struct sock *sk, int flags) { - union tls_iter_offset iter; - struct iov_iter msg_iter; + struct iov_iter iter; - iov_iter_kvec(&msg_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &msg_iter; - return tls_push_data(sk, iter, 0, flags, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_kvec(&iter, ITER_SOURCE, NULL, 0, 0); + return tls_push_data(sk, &iter, 0, flags, TLS_RECORD_TYPE_DATA); } void tls_device_write_space(struct sock *sk, struct tls_context *ctx)