From patchwork Mon May 15 09:33:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2209BC7EE22 for ; Mon, 15 May 2023 09:34:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF058900004; Mon, 15 May 2023 05:34:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A78BB900002; Mon, 15 May 2023 05:34:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F29D900004; Mon, 15 May 2023 05:34:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 81411900002 for ; Mon, 15 May 2023 05:34:05 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 23E181413A3 for ; Mon, 15 May 2023 09:34:05 +0000 (UTC) X-FDA: 80791978050.24.D678BF7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf01.hostedemail.com (Postfix) with ESMTP id 4614B40010 for ; Mon, 15 May 2023 09:34:02 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LRrJDIAx; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143242; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b0te0jKR5ygwGnEzoKAl4ELcEmhdRmxdQq46C0lqTV4=; b=7Jmnyhm9lGMLancWfvcd6d9YpwzgQuOQmnfO9ZQGLH3VYqypV4DqSrZIMDRv98DazqveYW u9PRgxdF9CVumJ2babo8/Uq61/jrYGFSFVgLA3VNTIUx6q9b533o2rX191LztD1pL/nSkC oqW2KX4zDChfiwXe7zjC/m3jtTjXy2k= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LRrJDIAx; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143242; a=rsa-sha256; cv=none; b=2gmeZV8deYvplMsLWmX1oWwy6Wybfd8sb+Mkg+HXF7M8BJ1pPZAwNC92omWnVuyYtTFWuw P9NiSNrTc70C17gUBWts/T4tZwcImQyVYESdgUf0TXHW7DZOBD7H39HC/FK/H1GVUfgv6k 78RzYO4/nrrZt9XE2LOVwRwQzuaEZnk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b0te0jKR5ygwGnEzoKAl4ELcEmhdRmxdQq46C0lqTV4=; b=LRrJDIAxIwvFfQSeW4Qdc06WmdYaB3JpN4+iUyunBK5IqckxIHsu6K7DY447oOzOPTgE3d 2zgrybQXrPBHEt9tepma695crYHUmBaRvhI7KQuV/3n5KCpNyO6gVECiwMCziJI2v+koOW fZvki1ML6gScb14RxFVAT1+ZVw1q4hE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-0XilkyABPe2SO0dMlnANnw-1; Mon, 15 May 2023 05:33:57 -0400 X-MC-Unique: 0XilkyABPe2SO0dMlnANnw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 571E487082A; Mon, 15 May 2023 09:33:56 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4DF7E2026D16; Mon, 15 May 2023 09:33:53 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Willem de Bruijn Subject: [PATCH net-next v7 01/16] net: Declare MSG_SPLICE_PAGES internal sendmsg() flag Date: Mon, 15 May 2023 10:33:30 +0100 Message-Id: <20230515093345.396978-2-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4614B40010 X-Stat-Signature: 3j9bda18hg3n6afudjmcoq5euwfiptzd X-HE-Tag: 1684143242-856561 X-HE-Meta: U2FsdGVkX19DYXOEJSD8OR0uMH81U0DrYJlgFY7/7pOsHDuYgaIOoTDAUVwwkCtoy1KYFJBEmxo+xkpzxIRYN5Buapxl0Raw4imCzj16oHB2gEy6PcrJbwRPALVHda6e9raE0bzoqwBee6x8kv2J3iEk+A3v+BYRjDcjFHeyOaQazP4E460vI4qtoGpD35TQ1L6efdMHmftVW4vud94WYCj+rIPLiEqcrRDZ9eTLOF9/T8Ekq3iqV1a9M9y6V3ZQZiLO+WQzWrOAAc+TJupAseR8fTQ/mMAHsPtCmOA9/qirfgfybYxsT2K9Z4Sdw4CNmrvTUZY8dQTN9r3tBB8f8hr/EIllLc6wPFS8V7/zEaGAbFc/KJnstlQ/V5uWJ6ySOEdoCufW7GMcLL4JMpjpEzSC92cInE/ayqfRsVCBRaoPXoBlJf9Zjujztrg6t3L4qZuY4joOBz9UK/+qiPr+NnrUUHnUo6JNPXU0WBlQxdE50PxqHQfKRT0Dg7vV085Nl52HEVOpv12lUpNalMSzk/QLip6ACR1bHzhHzifx67cotS+fg5QIAoJiVXhaUvoIje6CqtzIG9Yic574sd1uUzI4Dyvc+9aLLkR4f/+VAJqu6qaeicL1VhCpeSXSqm/ExfHyg9F3IySQKUqYjc+rHHbTJL0pdh9B4Wwr3h4KYz6vP3icG3B+kyc1j/ewCs0Dzr47sn5vFb3nQAohKYmMtzIVySupb1OlySuiLWtYvOsi+CRopnA6oX98vF1ue3GnV6wWYWBmFDTnV7NFPQKU59RXHUpuX109XSNw+sTTEKWRDvAg2OZhFv5pzDespcWaAWDRsT0cTUJtY2Yog/31tnrKZFNvO3Je5LyXUF/KQ4nF9YCNditjf1v5zK973d/R5W2uEMtxot8yyI+VWJKwAqm9WeDG33oRSfqnyG7cQSlx14n5YvCFM58JBVgxNRNE6HJnfxUUuqAKkd1ZYrI rYpgrLPp zjxBkgg2Pv/kcir/mjzN9UuBnViJaMUXd5+Idz0UCNlXh4+BEhtfHuMp09liJcqbESkykVALEFntXN6EnAIWBknElIZrryMICC+N6xoelyRGZuLKUoQD1cdV4njkkSLUupNdgQ6cBSfgOydVtJmc1cd4eZIr5EC4ZDfdUR34e8mKYtfLBYHusbSW5++UPcGDOn16U2PvwCKWfwEVNltS8D0TAif0qeaW/ipZ16WYBl2TxiKH5bqPA0aoozk6TLD+6itoldbW/h2BwX5aHlLzhgw4/MV5MXbEdndtykYXMsH+z1tfGCCHva4/TWERNWNjxij5K7HSRT/RzsaUK4MBo+SNCPV6pQ5WnjzRU5s1l04dKlrXd+tqyetnPYfAwSmbXK+xOaY6Kt+vpO/5lrZgsSQlzHE7++puuiAsO3b2jtIs9sa0G1A9rcxffwTZJuoa0MUUG5pd3as5zMslElILyNXWN8rabDgVdD8hkPEJC0sDNcclBDSCUTW1q2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Declare MSG_SPLICE_PAGES, an internal sendmsg() flag, that hints to a network protocol that it should splice pages from the source iterator rather than copying the data if it can. This flag is added to a list that is cleared by sendmsg syscalls on entry. This is intended as a replacement for the ->sendpage() op, allowing a way to splice in several multipage folios in one go. Signed-off-by: David Howells Reviewed-by: Willem de Bruijn cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - In ____sys_sendmsg(), clear internal flags before setting msg_flags. - Clear internal flags in uring io_send{,_zc}(). include/linux/socket.h | 3 +++ io_uring/net.c | 2 ++ net/socket.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/include/linux/socket.h b/include/linux/socket.h index 13c3a237b9c9..bd1cc3238851 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -327,6 +327,7 @@ struct ucred { */ #define MSG_ZEROCOPY 0x4000000 /* Use user data in kernel path */ +#define MSG_SPLICE_PAGES 0x8000000 /* Splice the pages from the iterator in sendmsg() */ #define MSG_FASTOPEN 0x20000000 /* Send data in TCP SYN */ #define MSG_CMSG_CLOEXEC 0x40000000 /* Set close_on_exec for file descriptor received through @@ -337,6 +338,8 @@ struct ucred { #define MSG_CMSG_COMPAT 0 /* We never have 32 bit fixups */ #endif +/* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ +#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 diff --git a/io_uring/net.c b/io_uring/net.c index 89e839013837..f7cbb3c7a575 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -389,6 +389,7 @@ int io_send(struct io_kiocb *req, unsigned int issue_flags) if (flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = flags; ret = sock_sendmsg(sock, &msg); if (ret < min_ret) { @@ -1136,6 +1137,7 @@ int io_send_zc(struct io_kiocb *req, unsigned int issue_flags) msg_flags |= MSG_DONTWAIT; if (msg_flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + msg_flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = msg_flags; msg.msg_ubuf = &io_notif_to_data(zc->notif)->uarg; diff --git a/net/socket.c b/net/socket.c index b7e01d0fe082..3df96e9ba4e2 100644 --- a/net/socket.c +++ b/net/socket.c @@ -2138,6 +2138,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags, msg.msg_name = (struct sockaddr *)&address; msg.msg_namelen = addr_len; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; if (sock->file->f_flags & O_NONBLOCK) flags |= MSG_DONTWAIT; msg.msg_flags = flags; @@ -2483,6 +2484,7 @@ static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys, msg_sys->msg_control = ctl_buf; msg_sys->msg_control_is_user = false; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg_sys->msg_flags = flags; if (sock->file->f_flags & O_NONBLOCK) From patchwork Mon May 15 09:33:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04464C77B75 for ; Mon, 15 May 2023 09:34:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96924900005; Mon, 15 May 2023 05:34:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 91903900002; Mon, 15 May 2023 05:34:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BA2E900005; Mon, 15 May 2023 05:34:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6D6E8900002 for ; Mon, 15 May 2023 05:34:09 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4278C121407 for ; Mon, 15 May 2023 09:34:09 +0000 (UTC) X-FDA: 80791978218.22.7AF96FE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 81AC040013 for ; Mon, 15 May 2023 09:34:07 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MBZrvF71; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143247; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Oms7mbQShTpY01zvjwYgDwHWlSZDYU/BDFTf+CWwguM=; b=zmdwA4PgsU0Z+4y/W5ovpBxWFXHbyC1d5PG7KgW872AYEqMrIWyHi7rl04+aHWVXC84eiM AN7pizPDLWp8FqqbamVU4tupxwORGtQ22xOZkjD/whLGFHmXbpV7h3/jGWULBx+6h5XEER l4yu28fHTBNLTdn6i2GbWNhwn6orMvc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143247; a=rsa-sha256; cv=none; b=nwU5N/aCASPnlKIK+Rj8c17LaCWsXzOaHjOxN/dQErzkV4zrb6vo2Fpifvfu51zXrYTThl J1fPAEy7cxtXp69+n/fjAXK8CncO0dwNQ1s7r+l8tYhcLEDlLEul2j6izsZI12v3C93Njn 35F+CAYBNZ2NSfr1UiPwJdwVnKkfGLo= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MBZrvF71; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Oms7mbQShTpY01zvjwYgDwHWlSZDYU/BDFTf+CWwguM=; b=MBZrvF71aEDWbry85MpSUSI8afO85N357wnAuFKz4GAmDmh1gRLN9l5HGWg2iyDSpWdJ9h ZQ0NnPtPakMck4KFCobd0tpV08eyF4oUw3//pL313hBcI8r4IzBs3Ygo3rmsWw+rosu2/n oyJalDh0f4Y+GAgOzpLgNdEhs9tqYkY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-445-AkM32TIKNTGBQP3iQjlccw-1; Mon, 15 May 2023 05:34:00 -0400 X-MC-Unique: AkM32TIKNTGBQP3iQjlccw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B58403C0CF00; Mon, 15 May 2023 09:33:59 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08F5040C206F; Mon, 15 May 2023 09:33:56 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 02/16] net: Pass max frags into skb_append_pagefrags() Date: Mon, 15 May 2023 10:33:31 +0100 Message-Id: <20230515093345.396978-3-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Stat-Signature: 331xbksdcqjyf7zfpqxzpsgs5hjsr94e X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 81AC040013 X-HE-Tag: 1684143247-162241 X-HE-Meta: U2FsdGVkX18Z9FBlBXf66b7JMvALSZphOuneICU9+EyljlkbirRBWlcjTvYxEss2CBW8E9CPeJyXE/Lws0z7C4RoUFjhhOYRj7strYR2cSPxqCNN9pHSs1oGJYevw22+B0GapfajMPHkjMag7M2J83vfQJnbTDmgdeSW5CNjNp0YOaHz/OXjVdisgLaFaKvqM/vuRLMvk51RhFZvY8AfsOv8KU3zn5+qY7GznRZBo3brVLYwHOH1XK9FYa442Ho6SZZz/kuvV9i0U33s9XYC0jySOgZ6lSM4c31l3+7Ce2hkntriq4OTJOD3pVFeVumQ84IudO2s4lIpfVpNte6LpQDEyquKIiOGEzGB59J/lBW49JJ8pSoZEQ+eQ4lWvV42VNAs/VolXXF0bj6xbuwNjvficmwBdB6qEBoYjLGuPCvirMtbHWACqNHO0bHGHSgLqBZEVpnB29IeCl9WbYJp1EoerfW89ED/lkkaRT9UV8n/HlQ/ekW2e8EuafTuz1qqTqnBEQwDKn3dLI51keh8CpxfWvTjWtYm6QRm+ueZxDOzpiJMjKYFShnLhojG9pUVC/2uNTbz1mmYykyGcCmfSVFs4ySgbVqbqKF/tGzjVDpH4ifSN5VA7iwQwZaXcbptBa7f+dPX44bAcX5/AJL+2TokeqL9mRdmCXOb1QQKW7JXcfS0H8NmIHA4pwAa2+97aMS2y+XmBRIYjfZdrTxKmC3DuKHPOVJtzUlsTZ1YzKk6Er9hVMrAEb9l0GQ6LHPXK3qfR7dANa5mhZMVHahwmuTYJcPehLhEujeVUhFyziQ22XMLu6AGWfQkh1FWZpd5+6oGUgVngz1tAAvKrU8MS5Tc5AMelpRm8tXjejlNMfELaef7judE02ssnn9UJEFocrDAj9sbtI8CQE3lL1WVykTVmdek9gAhMccdyXMaL2XMz6NW7AfttRaEXP0QFGH0Xk20sdvZiTr3VGUmJO4 2cQpoQmi gjoV+V1uDHIpbyslo0rEgdAnRnr6Fmui3PF017uNc8+6jfpXRq9wxDyOY6bN2/KzJcwURmkDMonaT8V/g8xbxu/++77kYVob66IAgDXf8xd2wyie9Hgmw7FcU1eOsgMNwamRO5CD/oUWREkzaTsDIm4zgvzLBg+zWQ9//8HGtnS2isljLlYY2oybM+dV99fxhMNVWGLHOLIvbLJg2XAWVssRuU3+0dGwqpnDp6/+oCjOTuq0mg4HXu+oqMYPNgM1JiK6DUBiGCAHty/yH2jdyhX1f94IZz2FJdbulQD26q38XFrqlLoV3oiylmN4qFDI1ALQsxXFqmcdOFrd4aGQ1GiWkAzGyVDICFesxEul5zYFbB5Ah1NcSwO3t8QsXQBz/8RfcZPRa/SOTW23+3G1Zc6gCkqD+wa09aAwfKOT/xX7VyUdtPm05gPzdM34ZkEGclxWno1+YIpkwiE3cDOCQ+FzSEUyOc/s0gdps0USWr8cHQx0TTXm2bzBDcDg+fpivHftMovPecNpdbcKSfTsRtumdEAqbucLk6P8p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass the maximum number of fragments into skb_append_pagefrags() rather than using MAX_SKB_FRAGS so that it can be used from code that wants to specify sysctl_max_skb_frags. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 2 +- net/core/skbuff.c | 4 ++-- net/ipv4/ip_output.c | 3 ++- net/unix/af_unix.c | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 00e8c435fa1a..4c0ad48e38ca 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1383,7 +1383,7 @@ static inline int skb_pad(struct sk_buff *skb, int pad) #define dev_kfree_skb(a) consume_skb(a) int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size); + int offset, size_t size, size_t max_frags); struct skb_seq_state { __u32 lower_offset; diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6724a84ebb09..7f53dcb26ad3 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -4188,13 +4188,13 @@ unsigned int skb_find_text(struct sk_buff *skb, unsigned int from, EXPORT_SYMBOL(skb_find_text); int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size) + int offset, size_t size, size_t max_frags) { int i = skb_shinfo(skb)->nr_frags; if (skb_can_coalesce(skb, i, page, offset)) { skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], size); - } else if (i < MAX_SKB_FRAGS) { + } else if (i < max_frags) { skb_zcopy_downgrade_managed(skb); get_page(page); skb_fill_page_desc_noacc(skb, i, page, offset, size); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 61892268e8a6..52fc840898d8 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1450,7 +1450,8 @@ ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, if (len > size) len = size; - if (skb_append_pagefrags(skb, page, offset, len)) { + if (skb_append_pagefrags(skb, page, offset, len, + MAX_SKB_FRAGS)) { err = -EMSGSIZE; goto error; } diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index cc695c9f09ec..dd55506b4632 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2349,7 +2349,7 @@ static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, newskb = NULL; } - if (skb_append_pagefrags(skb, page, offset, size)) { + if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { tail = skb; goto alloc_skb; } From patchwork Mon May 15 09:33:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F3C6C7EE24 for ; Mon, 15 May 2023 09:34:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC96A900006; Mon, 15 May 2023 05:34:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4FE6900002; Mon, 15 May 2023 05:34:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA4A3900006; Mon, 15 May 2023 05:34:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9A300900002 for ; Mon, 15 May 2023 05:34:11 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6E4E7141390 for ; Mon, 15 May 2023 09:34:11 +0000 (UTC) X-FDA: 80791978302.07.9C37B52 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf18.hostedemail.com (Postfix) with ESMTP id BEDAE1C0002 for ; Mon, 15 May 2023 09:34:09 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eTfvkF9c; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143249; a=rsa-sha256; cv=none; b=P/U3j+Dr/PlZj5Mp9+t1ZOgZcZEHF509khvYegcD5ur3VOKbXcWAbGjfmNDgn/IHwRj8vN D3aGCJWtkFDst+95/k4gNL8QJ8JnhGjhFzxXYIbY73KguNX2K+PmXZGS8tn6vGu0s3aGyn +zkVp6ZH8iXR1AG6hgrNz+45asHesg0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eTfvkF9c; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rQETt65RGaUyqHGF2SRFOY3ON9kc871DZcLjRArLuXs=; b=0v+8CC5w0cYce7SqfxYBZfWpzsOpmKp4xWHaAmEtr2BohAQGGgqAp25Opxmu7bx+kd1Gx/ R1uOF9N6CZZS8CKAUrkaUTF7WTOR6c8GEv3lMLcXDvLM+H1qpD4g/sAFapmyWaRRZ6Hllu oMRhikkJeyBlERYjD3EwmLG1789TiP0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rQETt65RGaUyqHGF2SRFOY3ON9kc871DZcLjRArLuXs=; b=eTfvkF9cEDJ9lcmE1hxeOwN6Dgf/Rex+/TTXbT9Yovjnp/VqyE/OdHpT/nL5D1YNJec3KY tfmGU4PNzXIAfNMZ1txv4T+cSr7Qg7fS4yPiTtfOg5S20w7AU4TW22r+KxjdlFNtdf0tlD K0BYzVu7Zpwj2jdPt07X3K/9xuShQ/g= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-516-ea5lFs8TOwOYnV30iV30Iw-1; Mon, 15 May 2023 05:34:05 -0400 X-MC-Unique: ea5lFs8TOwOYnV30iV30Iw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B2060185A79C; Mon, 15 May 2023 09:34:04 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1B4DF63F84; Mon, 15 May 2023 09:34:00 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 03/16] net: Add a function to splice pages into an skbuff for MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:32 +0100 Message-Id: <20230515093345.396978-4-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspam-User: X-Stat-Signature: b3z59pkwuh15tpqbinsu8yzgbwt4idcs X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BEDAE1C0002 X-HE-Tag: 1684143249-619382 X-HE-Meta: U2FsdGVkX1+vCwu0Kms1BX0Sl2HaZGeuoe8FgWARgcKsngrtO0oqJcfEZ6CwUH489cD+RdFGv/l0FkkoBVmlasjkH76ezos99VAqWuirB5jg2sTFSCmAUy+nlDfmW9COGgDn81XnxCrfh4hMX6TJP5pLylEAzvGPtmjfNL34II2iXpF3MG06sTNTK+Levxsvm/JWQX5WeRoeW+4huq6ndP4EM+j2cIVPhuQoJN7b60l6258nGgnMdcyuX0wUoVf9U9xht3sGpZPa4qicH7dzivV5AaEJyfkQdlmFWy2qcANchHCppNS4Ql7ypTlCXnJOwuVydIgLtVnZ9HoX/mgJ0SAAA+Slvul8XvLLO0QmpTXQHd93QHB9AnbtlM4WPnZQTVqChwAs87rYr+EgzYtKSBbi7k71S4/Oui32iDmwGWduLinXoNDCzSdt5S0wwJCnmZL9Y/WJjfEYk/13kDAUKUSEz9YEo+DzmF+sECpiihJ3nj0es2erG8BvDEzHPdrA010cdxeL4LP4HpCK9llLDejCelDzvNsvRSNXx043yhPyC/cWZTI3nt9qV6BYq0x9IurimMxQTwGzlUOIs7Tmbzh6PrgT5+IBTECO8p7CrLlMx8XKRPys6u7FQ8IHkrYGUhXdLsJd3x7PBdEXs339jjVZ0KNtqBy8j7tH+19g45cuVob5BZtW6DSN/YfTL/lSzYP5xj1ROP81Nk4ScqFYoYK6SdXdEjHl9eh30awMkEIQGIzoBP6vORcexUMiOY6WRhwmjg6tprfe07975yhWyXofSjEmm9bTHRl8jKPPY+eEG02e99b3n573VTlL9pSI+5vgBNE4Bwv95SUt+8yW9+BbqrD88YOQgy1zR1L9vwSwqryGpEF67rHN3Z/XltOqdgx78bB3hmkrAt3O3kce/9W7EVttCZPFmQbVdTtM6W/PEfnf3Xzse62TFfa00xLfCSsLUwpPdmhMV7TXkL4 yOmjXNH/ XOGrGw67HsbngpjyU/kfn6+18rBmMCtuBa/v4vbDM5MLDGNHdHFALTq1dHqfnk/xdVEVqppMblnhfy2kMYSSWwxANyROMshOK+uaY/7V72zrWdDhr0nZwr1gLk4Bba/0rMlp61Qo+sRos+IJWB5aYlE77D5H6v4DuorWp8MMURJzGcRT17ucws/s41rDVrWQUNQZl1pnstfVuo00CzyGGJ7+3cPgkqrpNJqLZfxk2J+6gsasAKOh+hBGQiZM9uD8dMoaGSixBSwyVJ8FBxxMiR4CfG2qL6oIEoJtG/7Jmb5Aq11vBDwd+Vm++K0+OmrexJcRsDZTlOVdzduDLUujlJePhzyMFm5B+deEWso6NtTgVA4aAkprbWic3a0nZjK41LQcyfPibBcsGNvuwPyOTyA3kv4MBjyDNq63wZUk3alNUXfyEFTBUR8NaUUbCafvWx1R+l7uzqJBrdi9xT8ntRfmi7OIaSghnpQIc2zYj85gPyriVEQEHdSCDCkvcNYyZEX2cgnCzCi9Gtp4Y/QbScGXnPkbOPVubNe5FHSxAV3fOCUg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a function to handle MSG_SPLICE_PAGES being passed internally to sendmsg(). Pages are spliced into the given socket buffer if possible and copied in if not (e.g. they're slab pages or have a zero refcount). Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Al Viro cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Export function. - Never copy data, return -EIO if sendpage_ok() returns false. include/linux/skbuff.h | 3 ++ net/core/skbuff.c | 95 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 4c0ad48e38ca..1c5f0ac6f8c3 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5097,5 +5097,8 @@ static inline void skb_mark_for_recycle(struct sk_buff *skb) #endif } +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp); + #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7f53dcb26ad3..56d629ea2f3d 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6892,3 +6892,98 @@ nodefer: __kfree_skb(skb); if (unlikely(kick) && !cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) smp_call_function_single_async(cpu, &sd->defer_csd); } + +static void skb_splice_csum_page(struct sk_buff *skb, struct page *page, + size_t offset, size_t len) +{ + const char *kaddr; + __wsum csum; + + kaddr = kmap_local_page(page); + csum = csum_partial(kaddr + offset, len, 0); + kunmap_local(kaddr); + skb->csum = csum_block_add(skb->csum, csum, skb->len); +} + +/** + * skb_splice_from_iter - Splice (or copy) pages to skbuff + * @skb: The buffer to add pages to + * @iter: Iterator representing the pages to be added + * @maxsize: Maximum amount of pages to be added + * @gfp: Allocation flags + * + * This is a common helper function for supporting MSG_SPLICE_PAGES. It + * extracts pages from an iterator and adds them to the socket buffer if + * possible, copying them to fragments if not possible (such as if they're slab + * pages). + * + * Returns the amount of data spliced/copied or -EMSGSIZE if there's + * insufficient space in the buffer to transfer anything. + */ +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp) +{ + struct page *pages[8], **ppages = pages; + unsigned int i; + ssize_t spliced = 0, ret = 0; + size_t frag_limit = READ_ONCE(sysctl_max_skb_frags); + + while (iter->count > 0) { + ssize_t space, nr; + size_t off, len; + + ret = -EMSGSIZE; + space = frag_limit - skb_shinfo(skb)->nr_frags; + if (space < 0) + break; + + /* We might be able to coalesce without increasing nr_frags */ + nr = clamp_t(size_t, space, 1, ARRAY_SIZE(pages)); + + len = iov_iter_extract_pages(iter, &ppages, maxsize, nr, 0, &off); + if (len <= 0) { + ret = len ?: -EIO; + break; + } + + if (space == 0 && + !skb_can_coalesce(skb, skb_shinfo(skb)->nr_frags, + pages[0], off)) { + iov_iter_revert(iter, len); + break; + } + + i = 0; + do { + struct page *page = pages[i++]; + size_t part = min_t(size_t, PAGE_SIZE - off, len); + + ret = -EIO; + if (!sendpage_ok(page)) + goto out; + + ret = skb_append_pagefrags(skb, page, off, part, + frag_limit); + if (ret < 0) { + iov_iter_revert(iter, len); + goto out; + } + + if (skb->ip_summed == CHECKSUM_NONE) + skb_splice_csum_page(skb, page, off, part); + + off = 0; + spliced += part; + maxsize -= part; + len -= part; + } while (len > 0); + + if (maxsize <= 0) + break; + } + +out: + skb_len_add(skb, spliced); + return spliced ?: ret; +} +EXPORT_SYMBOL(skb_splice_from_iter); From patchwork Mon May 15 09:33:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 920BEC77B75 for ; Mon, 15 May 2023 09:34:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 263D7900007; Mon, 15 May 2023 05:34:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2141B900002; Mon, 15 May 2023 05:34:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08E48900007; Mon, 15 May 2023 05:34:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E697D900002 for ; Mon, 15 May 2023 05:34:14 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ADDA0A13CF for ; Mon, 15 May 2023 09:34:14 +0000 (UTC) X-FDA: 80791978428.13.F0E1F82 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 09D1FA0010 for ; Mon, 15 May 2023 09:34:12 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CQ6BNhs3; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DZXfiar5CKohUwsPymjzwnCAT+wFqXG5Kam/6y6M8pg=; b=0iwd7AS9I3pC5dV1Pj1DW98YhkbqODJ7/BSqqrvFcMvn80eOVKJzUB5pMzvCCJMyYEvS7T hCqGL4cKMoT+l6+Nwyl/FcXapNzfmoAw6PawhrH5SbuCtxKQ16Dagn8R03rqo3e1yftewY XriSfaq8Sl+Y75Vi9RxtrgnZ1LPmlNk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CQ6BNhs3; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143253; a=rsa-sha256; cv=none; b=ZhPjTp4wGRoLTMhCD/ccmEWuOVIXlgtRz9Jhry5/+sfA2lgz2GDh/b1jopsEb6G4dPbGVA 29pbGpLiipiZEkiw2pdLYfxFmuKXvX/ITkMxEiuti0h9EtXUwf64xyQyf8cEc4ClcqMi4y 6De6AYw39oQH0N6rBaRMF8V7vloKTgk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DZXfiar5CKohUwsPymjzwnCAT+wFqXG5Kam/6y6M8pg=; b=CQ6BNhs3fUU9eJ2xniiys9NVsyn7wyrgk+6p8F7aCsBG8JaKRinrgfqMXEM1e9BJM1TShG q6v+qeGyEApVCxqlYaPxUuomc9hPd0eeIg5Wl+UgC3LmnWXR4IL3gsuDczurMvrH5t4yf+ 20E1DG0jGzwRa6dO3W81he2BKEAkS+U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-365-QGYT3avTOnGhGIbiGpk5qg-1; Mon, 15 May 2023 05:34:09 -0400 X-MC-Unique: QGYT3avTOnGhGIbiGpk5qg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 360B3811E7C; Mon, 15 May 2023 09:34:08 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8F615C15BA0; Mon, 15 May 2023 09:34:05 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 04/16] tcp: Support MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:33 +0100 Message-Id: <20230515093345.396978-5-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Stat-Signature: e5ynx9zurcqbmuqkenbti4kwau5nf3os X-Rspam-User: X-Rspamd-Queue-Id: 09D1FA0010 X-Rspamd-Server: rspam07 X-HE-Tag: 1684143252-199513 X-HE-Meta: U2FsdGVkX1/znknPSNIixSlkTmJqnMXW5VrSoXZYFft/UTFs9K7bnPCbZw2LXuzMUWrfFULMmVVEUYS05MPjOYe9FU2yyac11Ojv6SBR7CVB4wLTi0mpBgBCJJ8uKrBIHUUL9P2ZqgRPpSYHw92iKy4q9w+RQDoUD22qFC01Cpm56uSiMTXgSCQLV/XkbwCjPMr3+ChW2kgKVizqqxDdH81L685iT6D9zV4Sz6ufLwdw3jbqQIboRMUdVWJL6z1NI9lBNrPdGRFswDm+DQc/TOMDoWTFVuSn79tLxm10R8lB+gN5AHf5U1I2mI2vuYpghpbN234ZTymWbxxnCMJu/RiagblQ1+mp0879q9kY4SWcxHXjov46Ldjhzq1XncXC9+YPF43IHhcHWrnQo6H+qCGbmbn8r8bUaKGr7O0T34PAQ7/P09MPFs85kOe0o519g19QJyS+O8mklnEO+h8Ns86iNOkdMA2uQPL+bBhwIs523zXhGVLVCYf7NKiqn6B3qKDcfasDQMwdv1NgxCU0dAXBilCXKQvDZLpyqKxqQ4S7rfijQOoeEGPWgWoNUdhb/QSB97eM65pkU+lSH5amD51ctzTrpxteV8lTmGS8801RT094xDMzWf0LPTMLq0mKH6HiIsCpmj/hKzC8is7FNWyrdQRCY8Ccu5Kmo0pt7/hE3Hp23u1tt58TkyPo0uMkEolg/K+eR0EOOzh06p11ehf1wQMBbv3TXdQybmsRoY0CGq3t7FRaLTKtUo5rTbut9jJjfuaIHYEiFKWwA2+CsTrYnngaGYZmnZvyDQagXw46CEBD1Nfao+NBPM2XIwd5yJczU48b0ehCek5TlskJpXvhoAz7Kf68o6YjB8CTF63M9FC2EhDOjmrUKr//02GLxU3yfvMc3kw4WTj5T9pCC03Co1hzq7guh36nPku80TaFaBHNF/JIEtXzHkxQgo5ZSIoX0kAF3Xu+Qxne//+ Um4d10XJ hRLl4nqXbgHWxk3GvuCejp4VEIIl07yFr2XyrSGvZwIvbq+zlSMcebp1widDwdPxgXpaRfTDyDTWOr2RY93cMFQC0FO+XY6TWwx+bjDmmWbpwkvwBSEqc4iripwmvm5hwyryzIzNLwWHC3//JMGVKltHTxI0utKsg589X1eRXiGBbXFofuuJuZ+4xq3udok19A82bsHKUkuPErGyLSSejvuSnhaqy7p6LOo+ygqhY7sK/lCnXUhU1yeQm720EPEtaQ39/2wGNAjAklVDxNSay0CQQyaVtAgyDistapWf8j7e3Em7DIyJNL1zVEXGHJ4pZv4cZeqpwmhG6DAVFq1sjAl8yakstOEvWy2hGprOt4JXerdXgFVDWf9Gfh99E6qGO/a6a3A0ORJW6fFSShmtMGXMf3fGH1mamVhStbD/82AuKyYCHtjlAUtRpWPk+M2LuI7tKjp/VJyidxHEa6580f5pOLmIxbJD2vshKBXDIm74BlSxxe6pfx4P2RA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TCP's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced or copied (if it cannot be spliced) from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Missed a "zc = 1" in tcp_sendmsg_locked(). ver #6) - Set zc to 0/MSG_ZEROCOPY/MSG_SPLICE_PAGES rather than 0/1/2. - Use common helper. net/ipv4/tcp.c | 43 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 4d6392c16b7a..026f483f42e3 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1223,7 +1223,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) int flags, err, copied = 0; int mss_now = 0, size_goal, copied_syn = 0; int process_backlog = 0; - bool zc = false; + int zc = 0; long timeo; flags = msg->msg_flags; @@ -1234,17 +1234,22 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (msg->msg_ubuf) { uarg = msg->msg_ubuf; net_zcopy_get(uarg); - zc = sk->sk_route_caps & NETIF_F_SG; + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; } else if (sock_flag(sk, SOCK_ZEROCOPY)) { uarg = msg_zerocopy_realloc(sk, size, skb_zcopy(skb)); if (!uarg) { err = -ENOBUFS; goto out_err; } - zc = sk->sk_route_caps & NETIF_F_SG; - if (!zc) + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; + else uarg_to_msgzc(uarg)->zerocopy = 0; } + } else if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES) && size) { + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_SPLICE_PAGES; } if (unlikely(flags & MSG_FASTOPEN || inet_sk(sk)->defer_connect) && @@ -1307,7 +1312,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) goto do_error; while (msg_data_left(msg)) { - int copy = 0; + ssize_t copy = 0; skb = tcp_write_queue_tail(sk); if (skb) @@ -1348,7 +1353,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (copy > msg_data_left(msg)) copy = msg_data_left(msg); - if (!zc) { + if (zc == 0) { bool merge = true; int i = skb_shinfo(skb)->nr_frags; struct page_frag *pfrag = sk_page_frag(sk); @@ -1393,7 +1398,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) page_ref_inc(pfrag->page); } pfrag->offset += copy; - } else { + } else if (zc == MSG_ZEROCOPY) { /* First append to a fragless skb builds initial * pure zerocopy skb */ @@ -1414,6 +1419,30 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (err < 0) goto do_error; copy = err; + } else if (zc == MSG_SPLICE_PAGES) { + /* Splice in data if we can; copy if we can't. */ + if (tcp_downgrade_zcopy_pure(sk, skb)) + goto wait_for_space; + copy = tcp_wmem_schedule(sk, copy); + if (!copy) + goto wait_for_space; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) { + if (err == -EMSGSIZE) { + tcp_mark_push(tp, skb); + goto new_segment; + } + goto do_error; + } + copy = err; + + if (!(flags & MSG_NO_SHARED_FRAGS)) + skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; + + sk_wmem_queued_add(sk, copy); + sk_mem_charge(sk, copy); } if (!copied) From patchwork Mon May 15 09:33:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0011C77B75 for ; Mon, 15 May 2023 09:34:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F816900008; Mon, 15 May 2023 05:34:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A7E0900002; Mon, 15 May 2023 05:34:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52240900008; Mon, 15 May 2023 05:34:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3EAD8900002 for ; Mon, 15 May 2023 05:34:18 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1A40F12140B for ; Mon, 15 May 2023 09:34:18 +0000 (UTC) X-FDA: 80791978596.19.43F1906 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 60643140007 for ; Mon, 15 May 2023 09:34:16 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JCL1HuSA; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143256; a=rsa-sha256; cv=none; b=eemmyAqH1g35435HWsv/Ii++uW2JBBIMWe+BUB9WvREZE/zKhok7EWj44kvLOjNcw8v614 kMXZk9xx0twCr3ViSkd9hxa3HhBjSOzUDnZEjbtex6o5BYpi4oy8tMn3ONMlrL5Mr7hjpa W8d93JJo1YGVul7ro8CXX2/ATKIMj7I= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JCL1HuSA; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143256; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9s33rFBaxABfh3fue+YSREMQMkxGRapWweS5v7j7klU=; b=CR6a9yjHmKpm0qCyeRebGoupjGkl6Q2DymMVpZ+yEPmycpLaHp1XNtlcw71//7m8XFoget 8TV0BuKzNEw+/5dvic4ai9b4V+a3Whq42jv6AB+BcIi//tolGK9FM9K8593M7M4MLaZPoX e3F54y9wTaI22wkB1Aj0WrZg8cXfQQQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9s33rFBaxABfh3fue+YSREMQMkxGRapWweS5v7j7klU=; b=JCL1HuSAyvpgl/H3k+tiCxzmuyRmiw+LbZgRDLC3yUm5ydyARIvNJGAS8UrOOIc+GhXUkC lTdc2Yahk2ta637cK2j/ofwhP/Ktf9IB29gmEnFZjr5uQPIn1omuVWx4U5jnKiRD7omYkB moJJlJo7nFsrDVqiw50ptg0d8TpQxPs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-ctaAd9Y3OV6hQ5fXV43Iug-1; Mon, 15 May 2023 05:34:12 -0400 X-MC-Unique: ctaAd9Y3OV6hQ5fXV43Iug-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 60F4338149AE; Mon, 15 May 2023 09:34:11 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0534C15BA0; Mon, 15 May 2023 09:34:08 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 05/16] tcp: Convert do_tcp_sendpages() to use MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:34 +0100 Message-Id: <20230515093345.396978-6-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 60643140007 X-Stat-Signature: dze59ywrwozhhigkezc9r8xrmxuoywq4 X-HE-Tag: 1684143256-671586 X-HE-Meta: U2FsdGVkX1/jfxfQVh+MN6eddmJKrZiJnUJYSMcUR2sGhzbHtIhcPt7YqLC4cAPCwAYiRL2aeAjJWg8PUeuUlaPzxq+09NOG9DW2skefsMjukk0l8icC22KEkIMTzxI3AhsOtLJn//qc/cYUjqF6wTJPoEeKQE6heLQZe1PPFMxbbi3NP5UX/RwhqtMk8JmIvEfAnkwcj+E7kzDG1C0Zj8HPzIsiWTRgKd/ecZT43yZoaJ1zUXwp345gNX0zOZRDi2mUZBMPN9CgUVWBoPBDkJwKb1m7g7FHYMYqHP0K/zG+dCilt+vlEkYLixA38LQxHA65ri9QkVoUIp8/5dFlK83M4m4gDaLkrOWkyke9WutjgkDjv/ZguCFjV1mRF7fVmL7N/qGK9YX3cXkE1IDVisXsE9givc+CQTXV1WqU++GMiYSCR1eYtXzB0g+NSgp7F5nfftbtBNaL7pEBFKZ325dkuJUftlKr8JpYuYMW8JkWuDVwiKw45APUw5UG5WY14NrdqojpVKwvOGv5CViUec11spz9yEzJkSteOW5cOCG29PyBJjOZ7X9x7IhAsx85mpXrcpcAGKA056FUhTVbBKVlI+lcjbvnX7sw1UAq62SdPUq4whRZDfnqi4lGhSnD7bl0RsqYIbVQqeqYLzFVhthcMSDR/uTkEwvWd6UcpL9tk1/DseREXZapKXl7VCyearYD/REL647Emh16TX9MJWeq29g5cH1xGDcmw+hGi62sUAPfsOvM4uwnecN6cgIL/vpnPYLScFdrrjPlO4S9qXz13oPsfWAaXrDu2rgkPcQ8gaEsG5nEgtWKRZ/3j4O4SHlXRhVKNFS8tfc4n/x3NJNar35Dyt1av54ccNWV5gfd9vJf3G0kXs8iLXHx9Opc5+kTznJSLGErk5XWrpEVQoWD0FDwwtOy0CsejiZ/+/eSTAiYrg55itNAVUzWbj5r+a9pWgMCrf1gnY0IonO qU+I/YBu 8c1UWn/yKaGx+xFT5S6Iij2X7c65HHo/NuTR/BSenjyBkVG9NwgAWgw2d/EVw7Nt7ytO3iGaVkB9xNa3QCGuKXmUiTDy7vS2Zw5kSh+8W3O09gCoTHOjucBc1ya6b4HydgNS4RyV0TIRZyFjYoYxDJyzLCxdYUVlro41U2y+ljFxQzsK43rpd35mXepIlmtk/2Fe3e5DdvbdjPhQ9DaAllRlC1IIhJoxi1qb/MjsHfhtKQw6uUr4mrX+5Fg7cC/IU7dhPfLyPpeFNPVLgAPhM4SOqDwZu93V1ctEC0pYy0HvpzKSRXOVJRdcQ8dUqSUWFt6PsWZywbmU6rDbU3eaVohsUNG39n0R8hlV4DF8F8tMDOtV5d+K5APfBABBn5Ff1AowZM3jK7VNHZEMcOASZwfe+MWkvQ23lhoEh47eEnDsrVlliB9XwE1lXXSqkXAFAlLpk9d1XHTKpt2F53uCDyaksvREXSQMZamPChoXZrC3yoA2zNelgYDGeyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert do_tcp_sendpages() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. do_tcp_sendpages() can then be inlined in subsequent patches into its callers. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/ipv4/tcp.c | 158 +++---------------------------------------------- 1 file changed, 7 insertions(+), 151 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 026f483f42e3..28231e503af9 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,163 +974,19 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags, - struct page *page, int offset, size_t *size) -{ - struct sk_buff *skb = tcp_write_queue_tail(sk); - struct tcp_sock *tp = tcp_sk(sk); - bool can_coalesce; - int copy, i; - - if (!skb || (copy = size_goal - skb->len) <= 0 || - !tcp_skb_can_collapse_to(skb)) { -new_segment: - if (!sk_stream_memory_free(sk)) - return NULL; - - skb = tcp_stream_alloc_skb(sk, 0, sk->sk_allocation, - tcp_rtx_and_write_queues_empty(sk)); - if (!skb) - return NULL; - -#ifdef CONFIG_TLS_DEVICE - skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED); -#endif - tcp_skb_entail(sk, skb); - copy = size_goal; - } - - if (copy > *size) - copy = *size; - - i = skb_shinfo(skb)->nr_frags; - can_coalesce = skb_can_coalesce(skb, i, page, offset); - if (!can_coalesce && i >= READ_ONCE(sysctl_max_skb_frags)) { - tcp_mark_push(tp, skb); - goto new_segment; - } - if (tcp_downgrade_zcopy_pure(sk, skb)) - return NULL; - - copy = tcp_wmem_schedule(sk, copy); - if (!copy) - return NULL; - - if (can_coalesce) { - skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); - } else { - get_page(page); - skb_fill_page_desc_noacc(skb, i, page, offset, copy); - } - - if (!(flags & MSG_NO_SHARED_FRAGS)) - skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; - - skb->len += copy; - skb->data_len += copy; - skb->truesize += copy; - sk_wmem_queued_add(sk, copy); - sk_mem_charge(sk, copy); - WRITE_ONCE(tp->write_seq, tp->write_seq + copy); - TCP_SKB_CB(skb)->end_seq += copy; - tcp_skb_pcount_set(skb, 0); - - *size = copy; - return skb; -} - ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tcp_sock *tp = tcp_sk(sk); - int mss_now, size_goal; - int err; - ssize_t copied; - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - - if (IS_ENABLED(CONFIG_DEBUG_VM) && - WARN_ONCE(!sendpage_ok(page), - "page must not be a Slab one and have page_count > 0")) - return -EINVAL; - - /* Wait for a connection to finish. One exception is TCP Fast Open - * (passive side) where data is allowed to be sent before a connection - * is fully established. - */ - if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) && - !tcp_passive_fastopen(sk)) { - err = sk_stream_wait_connect(sk, &timeo); - if (err != 0) - goto out_err; - } + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); - mss_now = tcp_send_mss(sk, &size_goal, flags); - copied = 0; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - err = -EPIPE; - if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) - goto out_err; - - while (size > 0) { - struct sk_buff *skb; - size_t copy = size; - - skb = tcp_build_frag(sk, size_goal, flags, page, offset, ©); - if (!skb) - goto wait_for_space; - - if (!copied) - TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; - - copied += copy; - offset += copy; - size -= copy; - if (!size) - goto out; - - if (skb->len < size_goal || (flags & MSG_OOB)) - continue; - - if (forced_push(tp)) { - tcp_mark_push(tp, skb); - __tcp_push_pending_frames(sk, mss_now, TCP_NAGLE_PUSH); - } else if (skb == tcp_send_head(sk)) - tcp_push_one(sk, mss_now); - continue; - -wait_for_space: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); - tcp_push(sk, flags & ~MSG_MORE, mss_now, - TCP_NAGLE_PUSH, size_goal); - - err = sk_stream_wait_memory(sk, &timeo); - if (err != 0) - goto do_error; - - mss_now = tcp_send_mss(sk, &size_goal, flags); - } - -out: - if (copied) { - tcp_tx_timestamp(sk, sk->sk_tsflags); - if (!(flags & MSG_SENDPAGE_NOTLAST)) - tcp_push(sk, flags, mss_now, tp->nonagle, size_goal); - } - return copied; - -do_error: - tcp_remove_empty_skb(sk); - if (copied) - goto out; -out_err: - /* make sure we wake any epoll edge trigger waiter */ - if (unlikely(tcp_rtx_and_write_queues_empty(sk) && err == -EAGAIN)) { - sk->sk_write_space(sk); - tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); - } - return sk_stream_error(sk, flags, err); + return tcp_sendmsg_locked(sk, &msg, size); } EXPORT_SYMBOL_GPL(do_tcp_sendpages); From patchwork Mon May 15 09:33:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42CA2C7EE22 for ; Mon, 15 May 2023 09:34:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEB49900009; Mon, 15 May 2023 05:34:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9BB2900002; Mon, 15 May 2023 05:34:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1510900009; Mon, 15 May 2023 05:34:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9A77A900002 for ; Mon, 15 May 2023 05:34:20 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6ABEF16139F for ; Mon, 15 May 2023 09:34:20 +0000 (UTC) X-FDA: 80791978680.08.25EE34A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf14.hostedemail.com (Postfix) with ESMTP id A6ED4100009 for ; Mon, 15 May 2023 09:34:18 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZUvheB0h; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=TLukAbXMZ9dNxMuDgmXiCENpOnXWL2IDGRzj88E/rCyMTMZTmV3NmoJzQPMCSHjREvm+L+ 6qcXhRYMMg+p9qeIWMUBwK2zgahAa3UUr9vLw6+8j9JMAINzLPrXXFEFKwOvYT6tDgq+ME 3SwRCHJOJcpYqQ0TjkEt3UrlYRZm+Pw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143258; a=rsa-sha256; cv=none; b=jiu98ch4oj/b+usDF58qUh2fjb735lGaNafJXF4y2a7oF15QLFimlnSshOVEIYQ+v1odnX Em2dp7L4yRx2Go0KtKdUmzSKcbBwHc2Nwq3wAb6ODbvV1hPytSd2CtB3lKlym3qOV4hVQY S62hw4kmNd/tf52+3FiSfy3sfQTeXSo= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZUvheB0h; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=ZUvheB0hn5C4B/pbO7XCsqaNupZ5eLA69FS2Y8QaR7gzmcPKsOUe30eP53P8C8IzB+Bh1A HC4R+DjEBwIDFhJ9xYM2XGmS6+ioBBVnFPe2oxORIh1+BkdoGNrZwz6VZ+6MzXtg8e76bW q4EIy4RAPOU7UnO4fzL2bJ02xp/4+3o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-515-ekVa9ADbMTarb4EWYMOq1Q-1; Mon, 15 May 2023 05:34:16 -0400 X-MC-Unique: ekVa9ADbMTarb4EWYMOq1Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BF541185A790; Mon, 15 May 2023 09:34:15 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E68B2026D16; Mon, 15 May 2023 09:34:12 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Fastabend , Jakub Sitnicki , bpf@vger.kernel.org Subject: [PATCH net-next v7 06/16] tcp_bpf: Inline do_tcp_sendpages as it's now a wrapper around tcp_sendmsg Date: Mon, 15 May 2023 10:33:35 +0100 Message-Id: <20230515093345.396978-7-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Stat-Signature: iir88d89o3qzpxfzp497xs5956jwy4c3 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A6ED4100009 X-Rspam-User: X-HE-Tag: 1684143258-957119 X-HE-Meta: U2FsdGVkX18HLhqRnjL3/dL1thZU+PB1S9HA/PHR4RZZVvEtsV2rYNT6DZ73HjfWUEFa9HjtAEPIax/gbW0/TZ/su1H6lm1IcnqR9k2DYlRHQ3NM757hIF319c6jccWftk0Ua/eeIu2m7TS4rKKvHRBqnoG+X/x3tLCFN6/1SExI8kVaS3zob4vPjMcTzeQUFI35Jun5frwO2R/8HPsWL7eUBjDjRZ8rd7zpFMcs3yS8Ikny7IgkdJcmU63hFJ5vak8+HleXOIyFD5ZvcplmysIk38ASo4HwuvRcZA+cOlV7opcjKjF8YxAy87YZiJBiS9racDS8didIO6GPfl01X17Mc58Zacixp5/6ZRlASHKUnuZv18Z0unsUlLxvQfHa8tbRpL2mlseLZldXBEl0pv4E0cQMuzfxWMXEVgp7mMxyDn2X6mkZKOti+F7RM0i1GcU3wlpeDnijom2DVC2B+cM3cD4kbVr/r5DbsgVTPTgpbhQ2cJTaxPEaHMJoV4o4Suo7r+9SdioFVuIYNoqLmFwoZgVG9DlP0WrQ6GcP7vXVQXS093XheETqe5QEyjrV5SU9030sZJO73e+UGeRbf0/d0z/Suv+EY8jJUOZloBKaXHDLJv9zdll6OpbPtgbaKcCfOIwDW3T6iTiwCdkK1Fp/KA6xslVA7BR4iGBp20bwmrIEhVBWfk/BmqZumpJJnSshOXBA/42SRv1ug27WYBh5RV00nQ5e9ixSOz7oUzc4y/JBHqvrI3che+wG1DYjWJfQBBNg+tyc/zibkbm5Jb7OkoqkFRkr1ToVI8PKHc274OYSqOoxpP+BQWdaY02bbKQHCLrOkJZ/wnFT1KMGBc82dnf2342z2w60W6UhdmhcDq7sstcL06oAbhck1VKwArJYr1uOs62FA6N0WQtDWmhySE2Q0RHhjmNW2Fp1UHHlOz/HIm/Mdn5XAwJvegBtUCG+/KD9UQxeMhpeciO NAtum1al PfA4IENqInylThO8aHS4du0Rt44PjoTNfmS7ftIX6U839B1go6KDlpgAQ1x3WhsG/kZWCUEy236X1Ml9BZfEEapnCsOgO1zZuuBXUtYKNISUeKG1sfR6yPVoSByJM1FbrktjDePBJmk/lSRY/an3/uOn4CGuCdRLQWfOu2RirSHMjCceZKwIywpcmAsGdR1UUXwyEw4QJPHvR2lEuYngI3dcJgW71lGMRsO5ynT5ITfTXAfetmDalavKq6hlNoXxZEzHls2bHt65RUoL9n+iG94W4U9q87OVsAVwGs49Y71JIUkDnD3MsCQBg5ZT4jwOjjJ2IeiaUR8bQsMuVx/xMNms8mjs15geBan85DsmsuiNDl+jXJJqxx62lxUcyKWZE1OkOSdVWyiLonFt8H4bWDlSi5mSY8zKwOesPRe9CvS8OHXdMman329hMFOEEje6giHi9R9RizC7Rq+JPyqFHJSOBLApkMC7A/FkH7PJJVbdX9JYaUdzbKz1RwwKjpNDhPnyNap2RSOq+8/g6hGMm+paQzxkrrTju4PAy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: John Fastabend cc: Jakub Sitnicki cc: "David S. Miller" cc: Eric Dumazet cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org --- net/ipv4/tcp_bpf.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 2e9547467edb..0291d15acd19 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -72,11 +72,13 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, { bool apply = apply_bytes; struct scatterlist *sge; + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct page *page; int size, ret = 0; u32 off; while (1) { + struct bio_vec bvec; bool has_tx_ulp; sge = sk_msg_elem(msg, msg->sg.start); @@ -88,16 +90,18 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, tcp_rate_check_app_limited(sk); retry: has_tx_ulp = tls_sw_has_ctx_tx(sk); - if (has_tx_ulp) { - flags |= MSG_SENDPAGE_NOPOLICY; - ret = kernel_sendpage_locked(sk, - page, off, size, flags); - } else { - ret = do_tcp_sendpages(sk, page, off, size, flags); - } + if (has_tx_ulp) + msghdr.msg_flags |= MSG_SENDPAGE_NOPOLICY; + if (flags & MSG_SENDPAGE_NOTLAST) + msghdr.msg_flags |= MSG_MORE; + + bvec_set_page(&bvec, page, size, off); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret <= 0) return ret; + if (apply) apply_bytes -= ret; msg->sg.size -= ret; @@ -404,7 +408,7 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) long timeo; int flags; - /* Don't let internal do_tcp_sendpages() flags through */ + /* Don't let internal sendpage flags through */ flags = (msg->msg_flags & ~MSG_SENDPAGE_DECRYPTED); flags |= MSG_NO_SHARED_FRAGS; From patchwork Mon May 15 09:33:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 406DEC77B75 for ; Mon, 15 May 2023 09:34:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D39ED90000A; Mon, 15 May 2023 05:34:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE980900002; Mon, 15 May 2023 05:34:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B640E90000A; Mon, 15 May 2023 05:34:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A4054900002 for ; Mon, 15 May 2023 05:34:29 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 72EB78134C for ; Mon, 15 May 2023 09:34:29 +0000 (UTC) X-FDA: 80791979058.06.884CCC5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id A0E744000A for ; Mon, 15 May 2023 09:34:26 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ebtoFHUE; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143266; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=01BWh9E5+5H6ybEWtq/RRMmSP9a3tSQtRXkjJXipYDdc2NS8Wh/ynktY1gzJF9efPDTrvL DUJR7/YzsK0U8IeDdtrbqzfKy0Syp425cZNthWzXTCTg0B+IsEvJ/NRQZTTKJjuxZ8ngbd tpjhUv1pz2XeES8cLE9JOfN+lmJBL6g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143266; a=rsa-sha256; cv=none; b=8YPFUYzch/lrWp0AUPurTMcBu+JwRPR4DeXdJyPA+obUcDCJ3BCMh0/0N4HlUe7p8KCsYn cV3Va47VZlibHwUa3TBZuGRtQRtN3q7pIusC/9LR46h+PHTztq4F8rGbeliSH13IdZE1Qx glydl4LOKkn6/YFbQXlujFV1Q2JgVsE= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ebtoFHUE; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=ebtoFHUEUwWYqmdixBBOfQNOz1rh//p0kJZPIQVAGTYIGh62n2y6w1n8opm24/fe8zUyAT JZLmS7j9n9Au+qWzzjdbhtIkn43lGWzPDw1UJlmdEnEFEpqwI6Vb65un9Cn3ubxpiMz0Bd a+FfVJU/bjLoSqQibgrpW+h6jYhPn6Q= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-640-1nXxZzOvNO2kG1IIBR_aRA-1; Mon, 15 May 2023 05:34:20 -0400 X-MC-Unique: 1nXxZzOvNO2kG1IIBR_aRA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BFE803C0D858; Mon, 15 May 2023 09:34:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9CE063F3D; Mon, 15 May 2023 09:34:16 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steffen Klassert , Herbert Xu Subject: [PATCH net-next v7 07/16] espintcp: Inline do_tcp_sendpages() Date: Mon, 15 May 2023 10:33:36 +0100 Message-Id: <20230515093345.396978-8-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Stat-Signature: gzi7rkhwn1fz5iryoy9km4b15xjkdtty X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A0E744000A X-HE-Tag: 1684143266-330310 X-HE-Meta: U2FsdGVkX19gMLy7t6BUeBKWBuXagg9xBgrGXGLl/bm223oPNWOl9dk7x9kkGVYdHIRXYe9HkGPzFiZpukTZSjYAz0zazAC/dvznlJY4qEO2i7XSAO9GdgK0hdk/Ve7kRAuTCB8XlpMN00p/i2lMIk5+DYcn+vWhyQmyyujWFoPc+zw8NEthJYkI54EMggSE09hF6Y6nteEhmvquZsop5bYGDCAl7IUF1N8S1OtRhO6muIut0iovZOxHpv157PdKKPbmvmTa0XjtIJPuzLtLhtkqiR34u1wbJQxFHJy4CM86att1yvWlI+0kUSCsrSCOE04ORivTd4pXwQxOOs5UG6nZi6Fysf60mCi1EuYLaSZ9CQeCiZg8VMOExvFWXNdjG0RC8zxRwArMpuJmU8USix89ROVP/Hd3/5vU3BMVfWHsP82hbZ3cipvHLrGQIHALg5UWOi3kYAY51JXLPagWtpx5QPEFR9b/hccrDroDyCaK/D6gz3uPQgdM6brsUOzTwEjpJAc0dp084SCLXXyfdSwAGJNqEbHsx0Nep4YqJ2CqGRhWmjQ969guHPljkabvW/s/RnxXJ0KUvaMPMUCPKNZX5yt6X8XuRRMdiPcOy+wupk7uc47KfGXDZ9RFMKC2E9ezvV6aSisHhobsW+o29Psx2HGGGsLPt0kFf/LmJOlMLoRwTGs2D5wLtdzqd7qmuGBXyrocCiJMmUlsaUqhTomlbt5yGWPJEFJ0LhC3C2vH+UisZWS1QCtGpqKCYdW2HaGTOYfimxSL2W+2H34mBdaqXvK/HLtKyN6bKzFMsFf4RmXqTsscM4WpFR7o8chKg21jC8IRcnZ9tsdkPgconrd8v9epEtNoQa9vO2IT8jcqsoARB/cWCnfSejQaFRnQnQp0sUxd0JbMi9HC/tNFZNb6rySCmsJtQ2mh0WreMFYZMRwz5MxCZM8sFA3gjw+2M7A+OlK2BkheRFkH4J3 rumPjgjq gO+t7bqQNvkhfw1eIL4/JRCcMNwkAaTaiS0WXdsMYQF8RcT8b980HaJmjSm8VsjYhcyftFcZBPTW8jeKl1aX2BsoXGWLkp2b9s116x98ZXX0lSKlg3sFfgw0SzValfvcti0/AIs4fRjXJWmblnN7ZdaxiZg5LMaDIpXVRXxx4+dVNUtubP0FLPqxZOEI1JOz7tz/2jby5daIzmz0euWnLcVvCX91C+oO4Cd0UjFTsOtKBR9YVRqS8Bn89IQkvM0TDSkMn8fOaU66j1YlP5AZ7MFwXo1pUPswtkJgyMswP6TchmdrrQUHLpNS9iThRce0BYIQddIclbeoD1FxI9ru4GeJ06kIu8VHgoqTkVS+SfL6paeLjedqs0eRx5lSrYoFP2vTXEA0itg6sai/HBD4w3oRxJxWYrJFWbyfs/Z/P3/gStmu9e0kXcfcaSK6XJAsVNhzZ+meR/S1EL/qsn8lZv/JYehMAGkf6z6Ubd8fsQiw1O0Ra+WtaSbsynIEDD45+RY6ZaXexfnS344slUL6PqZTkYwx1P+FZMdvc0u9rO1a0vfP/q+8OT73IEHZP8hL5RJm9Qkrz+fpL+tPiPcCT76Q3ty716oUWVfqd1AxeXUkfoAg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Steffen Klassert cc: Herbert Xu cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/xfrm/espintcp.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index 872b80188e83..3504925babdb 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -205,14 +205,16 @@ static int espintcp_sendskb_locked(struct sock *sk, struct espintcp_msg *emsg, static int espintcp_sendskmsg_locked(struct sock *sk, struct espintcp_msg *emsg, int flags) { + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct sk_msg *skmsg = &emsg->skmsg; struct scatterlist *sg; int done = 0; int ret; - flags |= MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags |= MSG_SENDPAGE_NOTLAST; sg = &skmsg->sg.data[skmsg->sg.start]; do { + struct bio_vec bvec; size_t size = sg->length - emsg->offset; int offset = sg->offset + emsg->offset; struct page *p; @@ -220,11 +222,13 @@ static int espintcp_sendskmsg_locked(struct sock *sk, emsg->offset = 0; if (sg_is_last(sg)) - flags &= ~MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags &= ~MSG_SENDPAGE_NOTLAST; p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret < 0) { emsg->offset = offset - sg->offset; skmsg->sg.start += done; From patchwork Mon May 15 09:33:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB192C77B75 for ; Mon, 15 May 2023 09:34:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 488EF90000B; Mon, 15 May 2023 05:34:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45E39900002; Mon, 15 May 2023 05:34:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FF5A90000B; Mon, 15 May 2023 05:34:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1CDEF900002 for ; Mon, 15 May 2023 05:34:35 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E9A6BA13DF for ; Mon, 15 May 2023 09:34:34 +0000 (UTC) X-FDA: 80791979268.06.723DFC5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 4AAC1140006 for ; Mon, 15 May 2023 09:34:33 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JoV9Foex; spf=pass (imf26.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143273; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=peDVltt0T6YxdDzohhlTuV82evEnJxRzMMWq0qGEGxd9yaZM/na6zbCOS50tbvFu2ntvuY eTNc4nYtAs3FD8FZmNJzbWdD3SLv009hz0Cvekg5IRgMHGK8GxpOE3TD/lIooOeckdC5rb +v/nvHywLNxbYJUPA2pQvn7YnnL0aaw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143273; a=rsa-sha256; cv=none; b=3pIgOxB5e6yXSZcR4Ari9+Beq6mFLskxNzF9EMCoAdBoWxW+mHfV3lft/S4Qz/cfP2tBwQ TS9G+DAaXd13dMY8/+BvmyVUMOpcXekiTm3ETzWHGZqLwUXGJHAbGKoW+2SfxsL/l4gypw YWkxdd4cxMDsD7IbUARAKyVq1E1FP64= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JoV9Foex; spf=pass (imf26.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=JoV9FoexsZz6pUK7tuSYzHWj9E0T/iq0dZqEESvTWfA59SYNimuDwy3FqRY/qgk5T2y8V1 ZOFXFz78DR7RSRSSmseBrfNk3lAnwZywVsenr+M3yTzs7F9/Ynw7sl3DgpEvUzcyNIL678 X9AFwi9rcj3SW1OaQ4YWQ6IzOjt1mMU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-_VdNKKjjMc2ew_BgJPdgKA-1; Mon, 15 May 2023 05:34:25 -0400 X-MC-Unique: _VdNKKjjMc2ew_BgJPdgKA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 781491C0950C; Mon, 15 May 2023 09:34:24 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5861C15BA0; Mon, 15 May 2023 09:34:20 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Boris Pismenny , John Fastabend Subject: [PATCH net-next v7 08/16] tls: Inline do_tcp_sendpages() Date: Mon, 15 May 2023 10:33:37 +0100 Message-Id: <20230515093345.396978-9-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Stat-Signature: ycugqp9jz6hieddjxbs4j4ptti5czte6 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 4AAC1140006 X-HE-Tag: 1684143273-264810 X-HE-Meta: U2FsdGVkX19bBlnriziRR5dNDPlJoTTzA3uGQOf9ugy49CUs73Jzd8G2DOLl97yriUR6wskLlMNRLB6GYJ33wmoK/wa+CRzlp7cv9qR+uBe1t/Ab6DPmTwOxbL1k8vKlubdmj2DKIRVjS5yOPA8tkqfg7ab5PLgZxEBWmhwGC0d2zNW88m7dnYIRRecioO43Pqn3mAUy730IJby6q6+eY352gMHl1va9GqXBMEyJWFn5uMHi4h9KsflJ+A1wKXYpbjOhUXJK0CVw8/x4TmoqfHaLXgq2w+faY9pltnj2Zg8n3hMYCdy+o3Oj3n63h1KdZTseNAnyGyVDQtt23k6sBHBt72AClBkr+KlZlAXp6a1PCINwwEZbRAOUr51mt8QVU/LbfoJzSutfElXa5opQKvhcEoW04PiwDc2n4V5Z/ClHq7YcVIcNakr2698eM6hjh/Oz5qxTm/7Z/QE+e7a6tXJi7YqmqVqpefLJvnERRTRBB3Lv8mUes8YkxhUSm6SXutWx4KCGwjaj9pI2Po7bFyzP0OeAyCI05HTOOBM1tMrM8oJWjKC9wQ5eKStaRMMyqbwWdOZmUU3f/8nub8a27AUvTcVKx3id+VY1Ftu6F4xdAcFzC2FH4C2w+ACnUMv4adetf3YE3JOqpcLxj4SEo/ohu+3tuFsb9QiMdyn73vx1FaTaLInVPQHwTM+eBvJri39tH/bslEB5hPYEhA7OBTeu2rG8s9Gu8GY3SGDZLI7HGhXvDobLEzsXvi63j+Nwn4sGExZLp5LN6KLKMzdeEG532Zmob41NeIyB+PiC30bnrpcdvHzrk0WWfKwVKiPQI4xCTc7WGXJSrNZ6EubTRrA8ViRUOjem7BGi/92BGLqq4lYC1nKuy7hHYZt6KBbp87vnDpMZ6YK6sqENiDAB3gsgef7EJiZDf4vamptJVKqTV0T0Liag8qplFesmt3jjefSEbpQbvGKZp12mjpF MwLzG+14 9VBu/7RQO005A4T/9vXV79Jz/l+WVn8/GfARbthugP0WxN8JetqXXTl032DZ9KiEZ32kZJx8hhXycT0R9PoOYi7SL/EbSxlMpGcnmbSiAYC/e42PB6qx8J8fEOJZTtxb7tR25AfVQOxK4oL4M3pcrUGv3j5+dk1S/GRn4VRZIfHcyeaZSDsTAMGqAqRFjtls5vBofrn+MokopanwMKSElxuscjdfX0EpnWgHpCjJJUoi3x4CmsjEyskS/YJsMy4sNmZlO0NBBxOFwS39AggmGofyXropCBTtdJ02t6kMO6qfxN2OVmDfK9fY4mLcgEUzLQttqcIBDsUliF3B/ZUSrZqYQOW7Iy0QjKvzJHnAIs2t2MKJLWD5udl19EwsSNdiODkmqjbbyPzR41sJUnC7rKcm4iUNGlNe+fz32YkqIs3fPoIyuOfHIT44Oz/W7ViAuAaw2O5sWrEMqrxb0HY0jCbyd+wfz657lKGieMcw6HJmBe/SuKxtgsQGOhgXn8svl8vYHX9Nm26bBzqramg2vfIlNsiUwQfa/I4ym X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: "David S. Miller" cc: Eric Dumazet cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/tls.h | 2 +- net/tls/tls_main.c | 24 +++++++++++++++--------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/net/tls.h b/include/net/tls.h index 6056ce5a2aa5..5791ca7a189c 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -258,7 +258,7 @@ struct tls_context { struct scatterlist *partially_sent_record; u16 partially_sent_offset; - bool in_tcp_sendpages; + bool splicing_pages; bool pending_open_record_frags; struct mutex tx_lock; /* protects partially_sent_* fields and diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index f2e7302a4d96..3d45fdb5c4e9 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -125,7 +125,10 @@ int tls_push_sg(struct sock *sk, u16 first_offset, int flags) { - int sendpage_flags = flags | MSG_SENDPAGE_NOTLAST; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SENDPAGE_NOTLAST | MSG_SPLICE_PAGES | flags, + }; int ret = 0; struct page *p; size_t size; @@ -134,16 +137,19 @@ int tls_push_sg(struct sock *sk, size = sg->length - offset; offset += sg->offset; - ctx->in_tcp_sendpages = true; + ctx->splicing_pages = true; while (1) { if (sg_is_last(sg)) - sendpage_flags = flags; + msg.msg_flags = flags; /* is sending application-limited? */ tcp_rate_check_app_limited(sk); p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, sendpage_flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + + ret = tcp_sendmsg_locked(sk, &msg, size); if (ret != size) { if (ret > 0) { @@ -155,7 +161,7 @@ int tls_push_sg(struct sock *sk, offset -= sg->offset; ctx->partially_sent_offset = offset; ctx->partially_sent_record = (void *)sg; - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return ret; } @@ -169,7 +175,7 @@ int tls_push_sg(struct sock *sk, size = sg->length; } - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return 0; } @@ -247,11 +253,11 @@ static void tls_write_space(struct sock *sk) { struct tls_context *ctx = tls_get_ctx(sk); - /* If in_tcp_sendpages call lower protocol write space handler + /* If splicing_pages call lower protocol write space handler * to ensure we wake up any waiting operations there. For example - * if do_tcp_sendpages where to call sk_wait_event. + * if splicing pages where to call sk_wait_event. */ - if (ctx->in_tcp_sendpages) { + if (ctx->splicing_pages) { ctx->sk_write_space(sk); return; } From patchwork Mon May 15 09:33:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB390C7EE24 for ; Mon, 15 May 2023 09:34:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 524D790000C; Mon, 15 May 2023 05:34:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AC76900002; Mon, 15 May 2023 05:34:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34D0790000C; Mon, 15 May 2023 05:34:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1D7AE900002 for ; Mon, 15 May 2023 05:34:37 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DAF1281381 for ; Mon, 15 May 2023 09:34:36 +0000 (UTC) X-FDA: 80791979352.21.CCD95AE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 25C7F100012 for ; Mon, 15 May 2023 09:34:34 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OdPxdO8D; spf=pass (imf05.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143275; a=rsa-sha256; cv=none; b=bQ1purf9QgzHj5YzHhSTAB1sI7KFKO9wU2pwyDn4EBQ58Tj6KeMt0rcWR4ad1AjJrwY0kW DsIzZDzBhR/6z16TZXACPn2NcEa1Emqqxz/5KFDGSMNI8PwlAcqZ8eaSrotjUoEMlnXUwx dASpQRaY/suEKZsGMibc4K+f9INpoJA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OdPxdO8D; spf=pass (imf05.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143275; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=Zt5BwGD1X6vtlqkgQBdguh8pwbJrz7VpVFMxJwlVpGWtMIHA+RXXiefl7El1b25nzehJbn 6w1dAiQF4T05pZgGJ1DMlVjGJ/3eztfCtYBAxkaryUAVYnnrT1oxdF7FnQgTi/FNG1pd7C 40ONy8zjWEfPWCQSAnk8kyVgB5X1DFc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=OdPxdO8DFhPRe+KQNXGqPz0goIuh6Cz24y23ZnczbWxojkORWecMZaWZjrXSgqAl2Akw0m AuoA1yKaNgU3IwilzM5kExlNAObSlvd2FMJyLdcUimszQRSlAOOqHRvdW7UWtg715DoP/B BpGpSdjXIwBEOS8pLlVPYJX2g1f4LR0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-428-lT4Ao4tgPxOf3GaRnh_sfA-1; Mon, 15 May 2023 05:34:31 -0400 X-MC-Unique: lT4Ao4tgPxOf3GaRnh_sfA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 02F81185A790; Mon, 15 May 2023 09:34:30 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 270BD492B00; Mon, 15 May 2023 09:34:25 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Bernard Metzler , Tom Talpey , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH net-next v7 09/16] siw: Inline do_tcp_sendpages() Date: Mon, 15 May 2023 10:33:38 +0100 Message-Id: <20230515093345.396978-10-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Stat-Signature: kmyhwpftdfodxcrrymwk7pdedfyoe1n8 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 25C7F100012 X-HE-Tag: 1684143274-889265 X-HE-Meta: U2FsdGVkX19Jef7yJvVagC7llQisLKxWUNbd7+hdH9YfegBxNwavz3cSRsEyqdi+9EBNNBTJ9SiIbqvsVkSFKDVxaQ46uNnaxskj2sD1wsWRf2gfQ812aWIWKC1fn+CDPI1DBzbavPz+g59Kjg62PGQiLMMdbaK3y7DS8T2KReBKKHM0PjpbU8EDG57f5JgqC9T/IhI6BU5kbLm1mem+xgBTumigL74t2nyWwg0G/j48FIHVguUmpvY2Zh5b/F5oRwycmPHlOyBWnJZhVqiB6UuDX/mxMouczHUbx1aPlSimDo9br1bpM3J49Wz+8e+RkRGAeGkk8xOsWWc9Bbgdyt0+HkEDAUYrDoNXQ+RqTgSvNQq++JXQhsCoql5lyjEbF06RZvBK1AKsE69olrT1A3v1+j4rotTs82PjswIVyX4SnFdMk+mNFUx2jjKwF5vtE7+zKFJ9DgIlE0ffl2VLUd6Z526lUxFgHfEK4IjnfVijErS4TjQCRENb2C3mwkJL8UI7wzogPuMCSUG6EIe2TX9qHL8O+vHP/ZcqVTLJgZrZKiMu7WjOmnyBboIQxlq4n3RX2NU3eYdYah1zaPYQUmJ8Dm9YJcs1EXH90WfturUGDVkiKADun7tcYWISzUi01cXFs6OGSI5bgxGZroSbYx4k9iN9fEecX+R6ZcmwkO5NspK3wQbHKIOvFlF97Ip+3K7r12Ll4kAXggItVyajWVfH2+zeX6LgnNJfIc508yRO2cvZWvuSztNOoJcyUxi9ZiEONOIZ92mT2SKP2EUPnEBtIsodW1nRgH9yThJT6C8B//vj5ECfG26NeXB5F/cPmTRu1phuXyFRwCPhECSV0J4KXJMn5xGJQZmZ+GMtzd7JHdDEiixrGbGRh1Caij1OJI8DYwLKi9oQsJQvUN8lJnLZVjLhwWWDSGS/ddwzvRjezF0gq60KQ7bI5dIgSwxmY4PUH8sZffh7tnNuFzZ p6WR3uw/ R/vyhEj597gL2XotUk7lWeYnNEmZM6kLtbtrSRyXGrKFZCHzQWrAOq80YN61mhkdlPMjuNrRIJAP5Vw65//DuHPDO7iBlQ8AqEDQ3nHUx+dy3+5ZC5mcjMHGjzKzljwIFhTRC6S8KFQVRXoOXRSiVEFTAxznidw+QzplKtUlsbK1Q4tNlRJkgWx1bjA8Uqs87zm872733U8KywREsWkyFImBFZAal0/5RLUuBWRh2CRZ4youb210xPGD7XSIpJUPlFwOtSK5pJvgIxfif1wa8LweeXZJ0IoC9MsqfXt7sl1NtjH/+PaNwfjIpS3RFdgCEVFnzK577JW/wnK+35bHL1eBSyhlDXDa1wojmMdoVtfJkaQK2JaV0wkgFZuCkjkq8aMvoNnE8apAzlHBbQdNQdpQuNG5Ct9dQevd5Z1wkABHJwTtgV+HRS0NodZ9z2dWZkl0wrZn/p5EWg645G2kQe3ZBsWx1FRqLRJn/rULsEkPnys1bMxJ+8d44CG2tkp17x/6C9j4DzLQ0cUPCNWxYgGO4JBluZxegFezoT+zpuXplD6x9o0QVd89S2L/0512s3rUv+tgwwWO+lSXTywrR7NXVLC79bUtM02E+nK2hMZpE9yVN5FwFSfxhsStZzjDqzYut X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells Reviewed-by: Bernard Metzler Reviewed-by: Tom Talpey cc: Jason Gunthorpe cc: Leon Romanovsky cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-rdma@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #6) - Don't clear MSG_SPLICE_PAGES on the last page. drivers/infiniband/sw/siw/siw_qp_tx.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index 4b292e0504f1..ffb16beb6c30 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -312,7 +312,7 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, } /* - * 0copy TCP transmit interface: Use do_tcp_sendpages. + * 0copy TCP transmit interface: Use MSG_SPLICE_PAGES. * * Using sendpage to push page by page appears to be less efficient * than using sendmsg, even if data are copied. @@ -323,20 +323,27 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset, size_t size) { + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = (MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST | + MSG_SPLICE_PAGES), + }; struct sock *sk = s->sk; - int i = 0, rv = 0, sent = 0, - flags = MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST; + int i = 0, rv = 0, sent = 0; while (size) { size_t bytes = min_t(size_t, PAGE_SIZE - offset, size); if (size + offset <= PAGE_SIZE) - flags = MSG_MORE | MSG_DONTWAIT; + msg.msg_flags &= ~MSG_SENDPAGE_NOTLAST; tcp_rate_check_app_limited(sk); + bvec_set_page(&bvec, page[i], bytes, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + try_page_again: lock_sock(sk); - rv = do_tcp_sendpages(sk, page[i], offset, bytes, flags); + rv = tcp_sendmsg_locked(sk, &msg, size); release_sock(sk); if (rv > 0) { From patchwork Mon May 15 09:33:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1688C77B75 for ; Mon, 15 May 2023 09:34:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A56E90000D; Mon, 15 May 2023 05:34:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8559E900002; Mon, 15 May 2023 05:34:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71EC890000D; Mon, 15 May 2023 05:34:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5C3CF900002 for ; Mon, 15 May 2023 05:34:41 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3516DC141F for ; Mon, 15 May 2023 09:34:41 +0000 (UTC) X-FDA: 80791979562.15.36DB304 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 76FBBC0006 for ; Mon, 15 May 2023 09:34:39 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YK2eHaml; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143279; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=enBN5jT1muKlfAJKNsWEtZoec3VPKG1S27Z+IZtkOMs=; b=rlQy53URHQdNMjw6CqxcCkmNsy3hSAEy/OQemY9f+jVHcmLGsjfm/9fQyl2qPWAkK7E8BC 1vzglCvbjpdyJcREtb2mQtKSLyYeHokZm3WVyzhUcM55mO0GI+z3/u6Ee3dfSp+Kv58Tns kTLRHOow3SPAkm3i3mwi0cgq/XIhQ0c= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=YK2eHaml; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143279; a=rsa-sha256; cv=none; b=zD9kqjAmlR8cxOiWm7ejtsxYGpe66IqBIcCMHyOoFH3mSj/qMQgGbff+GWCrig/SaLXqQS fAxu8Vq2J1UGckW3ASVHo6W7XtiBLog1xetRBUf6yKzw2aQpihF+2176+35lg7dylTE5D+ pqMj7PXr7AWmVdmo8ja3/VABcHQVB+w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=enBN5jT1muKlfAJKNsWEtZoec3VPKG1S27Z+IZtkOMs=; b=YK2eHamlabAMIFIq1IzusvcLPAaiyBcNMI3ZxYwbSXotMlXFqEJxJtfiHMtySYcfXOoJ8B nWXQidCi2GJwLr0U/KFpPgt6lufvee/MCtm7InHhoQbUVOI6nwTnRU9u2GkfD9IzXtpshp avmSEUpX38HR9dCZmPksielHN4cX42U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-U5XToGkmMZCD_W7I6Y32Ng-1; Mon, 15 May 2023 05:34:35 -0400 X-MC-Unique: U5XToGkmMZCD_W7I6Y32Ng-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B8275891750; Mon, 15 May 2023 09:34:34 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27EA32026D25; Mon, 15 May 2023 09:34:31 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 10/16] tcp: Fold do_tcp_sendpages() into tcp_sendpage_locked() Date: Mon, 15 May 2023 10:33:39 +0100 Message-Id: <20230515093345.396978-11-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 76FBBC0006 X-Stat-Signature: ptpbkthz9ig5x34kkk1h4tksc3177bhs X-HE-Tag: 1684143279-982272 X-HE-Meta: U2FsdGVkX1/wFg2nE9GWQyqYQVC4lw/ruA6RtMItt9FtAdQtJumpv4d3eHwb/F6Nt0wkCd0RrXUC+4jSe/9P3+XotzcA6de74+qKqR3bZ1XHilgGK/gdA4YheeOdKWo5cN/T0L0HBSvbhQUWtLOquMXAfOMDtUitHOqRY0W505WLfdzFssw11A43t1VNWnndiOBqjqiKoESfuNNoiImiD4yx7Vc/XYylB9NIxqU8ji+lV+MX+RtXGmuxFrXNNBm1FzwJNYCuLC6s3Gi8A4ilgVqSYTh7ESigSPQ+RGD1u0LmY694HW1fAzXFJEhvC6xgDjneHoOB7s1l7tfJ/syZtNYoM6UzJBOZxDhAZNPkIBlNq3W3sCe5KbKKvIPxb1Jt2/HRH8SJTfFpHR+g9BTglS2hppooFyh0fjq8Lc23hMY+ZFrkixD3Pg0FSubvncOp0wWGCWei1HiIVN1KzKRX7DofOO/xX+RR88DAqnRdSSxCgIfk6VozcJdrGZBNtDz/avWqNaWEuaxUZTyQFC/sX/Rd16GCLAtWdOI21bK+OKq5TE7qYQ3gJkv3vND7HTCEx8PlKimxjhRciDIM4NHVK1X93QPHZNWnM6/dTKRlkb7yYICfeW5MlxdDtBhM9FwlhngYf2kERNMrXIZ5/adsDVWU3gG6oeJhlIWAznj2a8ptXUqrrW2dhd5JvWIl7KQoGFrKWMM8psfEgutK3kimTNZLfQnkLI6HwuTI7Xpa9KW0t+GEuX9HeX4/bSjl8/ThHR1chV0+LLkWzoUc2QkNlCugTGwzA7yl58u01BGhtg6l5WWiEFLXqyI1Tef21Yyv4e1VOypCLS2PJAl2++rmSY8SnR94vwiumPMYzlcUzMhrdFCxDStZAO1JQ9U9766NO2wzrI4raIdGPo7f+2WVrU6MtvHjQ6gTQZ42Lw/G3dA5xm8VtnsLovf2jNJHLCZpwXoikRrwgzlcoMaAQ9Y nhdpL+Y5 oYCFGU2oAc+9PUdCrj1uGBKveVSpORJcPEqYQHwY+iHSj6O3t20jRIU4mgbcyqQGk8xE59Gt44lxNImiwS5lU5H3N8g5hcFuvq7npCHj8TckZyEoib28I7Yjkth+mCpQLEeUa49BFGXjN+RsZDxgnrlrua3DEIK12fayV+szIy4mtPZNHI8CwA8UFDZMrrOhQjZGwJS3FU2zZFxqXt3hRIN+g/RuS3pAYchPP/xBwg2kpad0Dam8LqgTQze8bWKK7/UDW+zj4H0PoGYNn0Wo11wtmGnFMBxuAKt99lP0+fXUz18v6B5xrCBtbpNxL3Dt0zUuyja7qj1Ci3QUj/Reemgq4kFBc1ZmE2YnGFfRIVkHgaSjz7ssV8qxwtdV9fdRxr7Cwp25qJxNr1Rc47ifP7DKGvFkY5pcQmPelLaGexVPNqj4upeJX1M1iU+l38Ih+52qSr6NUbYV11v9OoAQK2We8Ub6ctC7L4sVU3HHNlqrTWchRPjEIX9zoEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold do_tcp_sendpages() into its last remaining caller, tcp_sendpage_locked(). Signed-off-by: David Howells cc: Eric Dumazet cc: David Ahern cc: "David S. Miller" cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/tcp.h | 2 -- net/ipv4/tcp.c | 21 +++++++-------------- 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 04a31643cda3..02a6cff1827e 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -333,8 +333,6 @@ int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags); -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags); int tcp_send_mss(struct sock *sk, int *size_goal, int flags); void tcp_push(struct sock *sk, int flags, int mss_now, int nonagle, int size_goal); diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 28231e503af9..5e4752131583 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,12 +974,17 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags) +int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, + size_t size, int flags) { struct bio_vec bvec; struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (!(sk->sk_route_caps & NETIF_F_SG)) + return sock_no_sendpage_locked(sk, page, offset, size, flags); + + tcp_rate_check_app_limited(sk); /* is sending application-limited? */ + bvec_set_page(&bvec, page, size, offset); iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); @@ -988,18 +993,6 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, return tcp_sendmsg_locked(sk, &msg, size); } -EXPORT_SYMBOL_GPL(do_tcp_sendpages); - -int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, - size_t size, int flags) -{ - if (!(sk->sk_route_caps & NETIF_F_SG)) - return sock_no_sendpage_locked(sk, page, offset, size, flags); - - tcp_rate_check_app_limited(sk); /* is sending application-limited? */ - - return do_tcp_sendpages(sk, page, offset, size, flags); -} EXPORT_SYMBOL_GPL(tcp_sendpage_locked); int tcp_sendpage(struct sock *sk, struct page *page, int offset, From patchwork Mon May 15 09:33:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE469C77B75 for ; Mon, 15 May 2023 09:34:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 522D590000E; Mon, 15 May 2023 05:34:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D354900002; Mon, 15 May 2023 05:34:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 39B3890000E; Mon, 15 May 2023 05:34:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 238FB900002 for ; Mon, 15 May 2023 05:34:46 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 01DEAC1425 for ; Mon, 15 May 2023 09:34:46 +0000 (UTC) X-FDA: 80791979772.10.CC8C2BF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 5349618000F for ; Mon, 15 May 2023 09:34:44 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E6ih37pJ; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143284; a=rsa-sha256; cv=none; b=ieWJFL634vgUBIOlHw+fChMgfggDN12QV04isWsMfKGTgLveYN1Nabr7vxni0jyuH6p7iR Hh7fpsD4W8aZwQL8dsT5orjH3Yfsf6tFhko2cVyoJVDwnaXVEaE88va/cuHqU7HROo8MSX jQFLP3dm3IX9qQfZtsN9WdbphlnLbA0= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E6ih37pJ; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143284; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=6Qs1GXWA+U5lu1ois2QDEiYCZyXswn5aL8pGBgMZRlgAmkFUjU2lUMo2N7DQrSO+RCc6hU A5LzeQgDZPMQ/S75l1BMeTaF/gaQ2iKfCPAGXPNjo/1e7o0EwoGLom6m5n9kQewlGUFl+T N0z4sWe2SkmcC1ZdXH4Zs9rrytB2pWE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=E6ih37pJek0uvGPIfN1OdSG5M0C0KPTutTqd8Uhu9oZSO4WBQgRRIFY6M2z0H1vfcfjwPo wwQ+t0PM8YsI0UsUXlCyzVLCZbnmMXFyQKUFbw+syafuHdcjL4/05j1fTNFW0USEnn3W3X JEh+Bl2Wes8cM1ZlSW+qaWqra+yJbyw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-675-DqP9KNUYOT-s0mTetspL_g-1; Mon, 15 May 2023 05:34:39 -0400 X-MC-Unique: DqP9KNUYOT-s0mTetspL_g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6ABAF891750; Mon, 15 May 2023 09:34:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8BC60C15BA0; Mon, 15 May 2023 09:34:35 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 11/16] ip, udp: Support MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:40 +0100 Message-Id: <20230515093345.396978-12-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Stat-Signature: tfni9kn9gs37hm353ex53c1pdnirk33r X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 5349618000F X-HE-Tag: 1684143284-520450 X-HE-Meta: U2FsdGVkX1+MNOfdLLvHQzdKeCEoXwVgE+zSmJP6ZV6Nx3z+rcyS/fOI2tynrHQzO/z2wj+DgAxhz123aqYr4luEDPIfedA+4cW8SL3eHaMmi2L4RqZ9jJNV5mE4rrVMHOFJt1be+oPA6orQk/TW34N89LbhTysH1AIdHZRSP9BarHs/NWy5XanJ5GqFFTm7OboSmCXBDEa9tQV5AB7sYiBYFeWwEo4iiu6AdJLYYIEHPC911HOCJ4EdIfhersCw7IcRgRnLPempJIlHTlvABLQvmxcZhoerE68Kf3mxZHmPHJ2JlvKdV9HMcCQgYlpQsVYGH8C9VNLrxsuJ4lvlgkbxvE1uIP5c79UAIjOuagOcIvzdorVceR860X/flCzYexFdY0HWjxBAgHh5+uF55fe4D0fcz9bjljUBbN90g+BNTQ846CSlpvkQaLx3d6y1as61x7ZufranrnKNor+4n2KdX1pfaS3USUWftQ5cUXkTV1UNxE6iSVIVm5YljVECmAuJaac2OsW0iZ+G/VvEm5K3fJyo07gK2jX7R1N7E5wBpUhCptFcfr0KWJaMMotSZFbK6xUs4LiM8w0LxKG5Qb9Anc38jt7Xj80RkhasLQuzz/KgL3i8n/qk8DF8S1txnEqYsj8cnaLeYO/mbi9tDHKPzO7H90IAwwZL0zfBIVfCzTKtn+t9xyrruDjURpR+vwEzQD8S0cfZnMdyMPxDoh4C8Af2LvfNbOAO4FL1Rs2XTscUTztTJSLZutWDtipkul93iyYFGAH0vV46q3vLUav8+chBe2tW/uCQJGQTgHgNTUh4yqPVJUfUZb0hQRulUhjbaQIJoA5RQVv4fhEeM4/HUEFhfgbv4y5XlZPeUorxyQEbI6IzGqqQZG5fYiZRadL5QSTTwJRLKoSQKkeMzIEkEhfDzK8DL0iwnqt5AZuc9jc2608YiaiaODM4DY+G4uLE63bobDmF4yowmpY 36LvlL7B DbFkGOpBgVCHf5dzG5fDPPtT2XVg8PHi+ECakYhVCWZlhNs9jG89/AAXiPltXROiX7jbTn60QNKFTXtTPmHByv0ntPrU6PGENEiyK2inqH4Z68rbH0ekTiQdZBT1chgJNjgRAXzGEyG6a5v617/DRGiYaMD2GtrfqM6irSAlddyXcQ4NIZZUeAxBLYRvpA/ehGIcWmMbDuFlnE+BwWZ6ds3aexAyFaODwc6EKsV552GmFot/2cOXAiqlESsWIwjEWd5X4nNFnZsS7b2Emo2OQSYSjc6ui1hG2Z5RtyxVWVuv4ANCFJpJz90yUR7Zxj0+/hezMKRxWY1DSPTsFMu/G+Q1kyUXbJ/97bpCHCT8YPCVaGlGLvrDKYn0tMrXG01frc2G4EiPKZ1vd/wZd64bIyaM+sGi/FaCJyUvVlltRkYmH6cD8um7TKmDVCzManVeHntD+vvyAGVNmmvynPhvlkXR77oCm2Pv/8xLPpzpis7e8tyYNgx0SDOV6nAiOjqF9q67N1ZPYBlwA3c0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP/UDP sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv4/ip_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 52fc840898d8..c7db973b5d29 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1048,6 +1048,14 @@ static int __ip_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } cork->length += length; @@ -1207,6 +1215,15 @@ static int __ip_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Mon May 15 09:33:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 910D5C7EE22 for ; Mon, 15 May 2023 09:34:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B22D90000F; Mon, 15 May 2023 05:34:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 262F8900002; Mon, 15 May 2023 05:34:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1032390000F; Mon, 15 May 2023 05:34:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F062D900002 for ; Mon, 15 May 2023 05:34:49 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C4678C1423 for ; Mon, 15 May 2023 09:34:49 +0000 (UTC) X-FDA: 80791979898.13.7E41033 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 0900D120003 for ; Mon, 15 May 2023 09:34:47 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gqV2PDOd; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143288; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=pVPqXvIjCrEjYGVudaDtQ2U4ucwmIQE6dwmksiJvjcSqvrc3QGztIh/wXKwQ4Ltlepx4Ku OyByCBmnZhpk858EKXjV0nfPC6+mm6+scZyFmnbhVLs28UGfSHuh+JnivDhDeabVBRlLCn idMnTkwws7UmHE5z0SyDT9UnrO1evf8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gqV2PDOd; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143288; a=rsa-sha256; cv=none; b=LiqCGtwy/DEDrl+XqWS38ZDBt8K3GPbnnQu0V/lKYW0J9XLPBP4V1tiQtsn9R33Qvaot9d D/e/QLCDBb396hqHkpU2AIJCprb+Nm/xVE36NxYxNtqnmsUBw1j8viDlLN7PLwAdOrzcQS 3PJ7nbGs/jJZg2Xxrptnrjh7QxERhtk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143287; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=gqV2PDOdSOtvhYwZyarkp9am8JU9GeNJMEcOlijk/7lq1oLv4KBqTjL8RmMHNL56hLbehO rJD+gz9VnGietzea0BN0UYvCrBmx7zzbILA5G1Whshr0EugBkfDtN/UBUK+x6DXGBm1p4Q dIg6OI2HFeZu/eOsaoxiw4pmP71REfs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-196-7yv7CbjeNdu0aC-6W8LDyw-1; Mon, 15 May 2023 05:34:43 -0400 X-MC-Unique: 7yv7CbjeNdu0aC-6W8LDyw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7E350870820; Mon, 15 May 2023 09:34:42 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id AED6840C6EC4; Mon, 15 May 2023 09:34:39 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 12/16] ip6, udp6: Support MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:41 +0100 Message-Id: <20230515093345.396978-13-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Stat-Signature: c4s1rmigjmj5fnaqnq741drqw67ffqy1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0900D120003 X-Rspam-User: X-HE-Tag: 1684143287-40807 X-HE-Meta: U2FsdGVkX1+QItQa0hjOwwIShkUyVmBH7Elme8J74klcHuHqy1MGD2yTkWdE6hxJ9N6C3pU1dUze6yoFiih2U46u/uffB/DX8Jc47O61EfsLB1MNJ98y5dSAfYebqi3HU08ueah4BgfzEGyOnsVMRnokGTpHgWFhU7mWYOlU5I1ploA4HmREc833cZaRT+h9DOztqEbanSHMPd6mnVUpmpGMXpgwShSpt30p8LD3Y9YF6f/VODM7Fs7AVFKD+Z5DYTJxNOHT+d0dUAF8xD0PCHYP0SuZIBkM1GBCRSHX5hlBo7l3G77iOHUao4kGsk5DbPOsVSnp8SIe97rBTwt3i0ysQlRMg5FDrEtxxuYzUOxkYMjhcW2839YOsyZGgZFzZ9PXoNhrT3e05wbtrW/HZs5UfoKnv3aFwd45dAyRy0ONOFvvm/nioU3IP0DpHNBq8NbOMjH3CxpU1qTXPtcDxy09fwio9X4RNfq6YvaefxMz5wbpUOGN9RVz/02YisN2I0YaVKndwMEnx1l5TGP2I/VwmKHEgOXxiPWeIu5JiA4uEQCwXlc2f8iLG6Av93woPOILX9DR3sKxg92Ixr6M3o3wjWsDKbH/ZtBee2G8lgzxboWuiY3GznfV/vkWM9JuXp0gRl2Vj+xF+SYEpxsB+12BnV6DcAQk62QiS4lqkFlkZoNIuLpJUajyRrDRwrnTmulgRdwXMA88o6Hr0VrrTlrj4JJKhFwkm+7tiHFDLrU3enUFz8J5BAU9vgDuXr+d3TkUVQaZnoAiqD33904RA1EmACM5sGc00pqAc1rgDcR0q2v9IfVEFNCTznkf0hK4SG3EyxHA8hH2Di+4n1inQ17UBrkc3eePdb3WM1u/Bn44IUY5uvPvd4zsuCdgBOH7zJ9OyeNMoy/+o71RJ7ikvSQoMp7oq34kflXhRSkHSZsh1s3s7/24825aXurThTZuZEHsNefChZqWJr8kaSq ZFpv9hCG YvT/4GHYwhfbzuEXmWBXD6nd9yynrOfAvzkR5P4D1lbIXa0jqKd1D9pghW2LIabLyecZHjjQb/M/PSd76/evk5HYRFz39VBW+uKRK5DiYKVA50hjNY1DqIUe5fSLBVzWgJ/sXtiUEHCye5+PgM/94dGiEL+AG+ZO75Jdd7EWwTdy3lEQbiaQLbgNFFDUU0b964MA7aC3DAh3W3YgvoEaSW27CfTDNadECE9bn2maU0klMHmaxzigdWuUAcO2+6Of6R3AQUXjId7W2JMNYlHmxl1T9ck4imuFi2mUlyVDOAXGTcAzXo5uvECKpYae5jAxRP0FylnPfXQRf6e9mdBV30FICPvBNImMnV9ObPk5pyGo7PxA2T/hv3rfWIKKB3U/mthl2Jxfjch82fwY4p59D0I8bmM+6O2sFdRS0776jPBsvptD2dbdR48rQqXKiiPJrpVbR5vuazh3mRY7CTfMbywFSXrvoTUN2bgJ3dQK3EE41XrdVXJZj8xLMYBTcIIkuTPWSeXbh7y9eEeg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP6/UDP6 sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible, copying the data if not. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv6/ip6_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 9554cf46ed88..c722cb881b2d 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1589,6 +1589,14 @@ static int __ip6_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet_sk(sk)->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } /* @@ -1778,6 +1786,15 @@ static int __ip6_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Mon May 15 09:33:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9146CC77B75 for ; Mon, 15 May 2023 09:34:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28BEC900010; Mon, 15 May 2023 05:34:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23C51900002; Mon, 15 May 2023 05:34:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B666900010; Mon, 15 May 2023 05:34:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E7163900002 for ; Mon, 15 May 2023 05:34:51 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B944D1A13C4 for ; Mon, 15 May 2023 09:34:51 +0000 (UTC) X-FDA: 80791979982.05.F3E7B95 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 0FD7540008 for ; Mon, 15 May 2023 09:34:49 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QILSxKlE; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=tIkAOtkPH6M/THbnTsk9oArHvLPjh18aSTPyNuwtCweCkFsi++3QjR357NbmCPt60DawRu 2gM/YBsiZXUShxVlHN0Ypu5AT58NJ79yBFClhGXkD0n4X7ySGRvLyBPXklxMp8zBpGGk+q w5xeVlAgVUWpP3YrdEINHNcqc0VlFwk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143290; a=rsa-sha256; cv=none; b=r3qxTSHz9ba4m2Pw/UqZbgWxzYSlO9I3tSKZ1mKpYeDNrVIbcKuod6BxBEwNvSrg3Oj66O KgSywfVlSVTvCEADMMdu4DnS4dejMOMRrGjwnA1xI4oxu//5/BGHPuxo9SoFzn/j6IfwMi m9cGdMa4zh+AfHqMAgiXn/GDtbXZCsk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QILSxKlE; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143289; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=QILSxKlEeSwWm0hmyyvoF2yXz/+HBoJCLpssAx2yDeeQOw5sHxcKtj4sMhxxlK/2ivDnKw FprraD2KNpMruDWiY8FbA0EruqXd52dAqJa1K2I1cID5vyNk+MOw5rDJ3w4URiqvzIXXII 2ua8YGFV0f+zWrIYXqT6vxx0A2CnWKY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-66-s4PfbxzJO6eXzsD79oKorw-1; Mon, 15 May 2023 05:34:47 -0400 X-MC-Unique: s4PfbxzJO6eXzsD79oKorw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 23A9387082F; Mon, 15 May 2023 09:34:46 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40B9963F5F; Mon, 15 May 2023 09:34:43 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 13/16] udp: Convert udp_sendpage() to use MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:42 +0100 Message-Id: <20230515093345.396978-14-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 0FD7540008 X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: efhh6wnbmg6p6k3j966f58arudjfkko8 X-HE-Tag: 1684143289-612761 X-HE-Meta: U2FsdGVkX18lCFdIlRQGJmHttefgvTDqrmPGqiM45R4Mh4MhP5KmJEyW9undHP+FswvkzTWvxtnWHiTtfQ6X0Gxjz57faUhz1BHzrO32RQmu1Lr3AS2g8HJE/8SNaOHqqwRt8zhRBoHn+jBmTdkVA5dwUsH86fFIb6Q0lsaeznqN7LyRP6B23BZAH7EmtWSg8azF/TYhgbPDxbxuQcKcAcPKq0rDLwwQzgif9FxACen3rZUR5sED2i1YYB2gRYrP+O2mT9HZ7ZKZtGuIuym6wCY3oEL/PydsYB3KK/hOaGP+cAWKkO8CCUkF3/okv9Nq+1IspoKf7jiO07VKcyrTjXLizP3A6g6GJ75NeOWRjiAtsxNZFi+gDA/OtSTx2CiB6z3t98pcUD7poNxxYpbvl00Nqfwogrk1jqXvvfiJzlZFBhmVwvllQOpeA1RsRvWsL80hmk/dGikfK6o+LKkK7ikmkFg1lHexpSm4cG+Qz9vMH9LY23Fo0NA1pxw+LpXeky6O0nSbTgu2RlegWaBMB4oK1F3cu/iVVmIKFS5McgXphgWcrpcjKeK5nagWR5i6bV5LrPbxowUWpHfo9o3h4frgsN0hahMlIqs6mXszc2ANudQVpWQT2CNVxypFczr8lBbdEmbYn67jmLq7w03gLllof8rIzx8VhsFpd79xFlz3pO2DUNOLh+QeuIv+p+LZL7Sgyzx13XvfOsuOCdM04qNlDfspaMwwy+VshQG6p/CVZ+W05qI6ys8jGl76uIaZrdNcAHw6UF4gUP3aCeD1f/4HH1g33rycLWUyHezih6O6RxErF1srOUYtfIJgNiy9moK1CvMzaXuqHVYhKCw0TiRDWHkNPGpNwoGP6sI4Bnf/3Ic2VLpY9PLWdEVXOfkPcQP01b56+7ehg1yR+95OT7c/JdZ6wcORH4oh9RROR7oS1Wrn3gDSrhe+/C9fMeZ1R2wd7dcuvLf0BnzBnWI h0sNUjdH DdBKNvki17OuhZlLvVv4V2IeAU6iTgsbFv5pIXk//bRa4BZB1p2v1U1GDDG6AlUpWT8Z9FwE5Ei8udcM6fibQ+SrdRS+oOKMKKnh0TT6DntKTZnM/cNVZ6a9XJrEWbDf9KVbJkoUxa5v2U1M+nSkpUIjlmpnHEaTM+jMQBhxQeJfp1WZ6nSDY//DIh7rhCAbqeeTTQ0n7L/gcBSGSHnfUTYQf9d77a1vOlnRBpOvRq7jySR3dwB1ci+4sHzLwnByhXvM/yebMdX7GyUBJSkQbxY+o3q1Hee7Z9FQ3b3gVRU57OvFiLq6uF9qX8tq8kAWfaDSZdg5mYwsWyQrVf9aqtYpvnx8qXyyzx0I1yVNvEyDEr0/UJ0S5HVGNkmVcKC2W3fJMFWCCeXmL450qLZrlKV/QSYXdNMI2yLGdWuFZJQS8gKAHTPOYbpigjUaSDKZVwPTTNcPtko9lH2nNRhVtUPVguo48WUK2IQ9nho0iprvSOX8VETPvfn12Ki3s1HIvZcTQNIuCJAqYqqo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert udp_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - udp_sendpage() shouldn't lock the socket around udp_sendpage(). - udp_sendpage() should only set MSG_MORE if MSG_SENDPAGE_NOTLAST is set. net/ipv4/udp.c | 51 ++++++-------------------------------------------- 1 file changed, 6 insertions(+), 45 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index aa32afd871ee..2879dc6d66ea 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1332,54 +1332,15 @@ EXPORT_SYMBOL(udp_sendmsg); int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct inet_sock *inet = inet_sk(sk); - struct udp_sock *up = udp_sk(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - if (!up->pending) { - struct msghdr msg = { .msg_flags = flags|MSG_MORE }; - - /* Call udp_sendmsg to specify destination address which - * sendpage interface can't pass. - * This will succeed only when the socket is connected. - */ - ret = udp_sendmsg(sk, &msg, 0); - if (ret < 0) - return ret; - } - - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (unlikely(!up->pending)) { - release_sock(sk); - - net_dbg_ratelimited("cork failed\n"); - return -EINVAL; - } - - ret = ip_append_page(sk, &inet->cork.fl.u.ip4, - page, offset, size, flags); - if (ret == -EOPNOTSUPP) { - release_sock(sk); - return sock_no_sendpage(sk->sk_socket, page, offset, - size, flags); - } - if (ret < 0) { - udp_flush_pending_frames(sk); - goto out; - } - - up->len += size; - if (!(READ_ONCE(up->corkflag) || (flags&MSG_MORE))) - ret = udp_push_pending_frames(sk); - if (!ret) - ret = size; -out: - release_sock(sk); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return udp_sendmsg(sk, &msg, size); } #define UDP_SKB_IS_STATELESS 0x80000000 From patchwork Mon May 15 09:33:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF1D1C7EE22 for ; Mon, 15 May 2023 09:34:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D1D1900011; Mon, 15 May 2023 05:34:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 680F2900002; Mon, 15 May 2023 05:34:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 521C1900011; Mon, 15 May 2023 05:34:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4289C900002 for ; Mon, 15 May 2023 05:34:57 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1D9F91C8953 for ; Mon, 15 May 2023 09:34:57 +0000 (UTC) X-FDA: 80791980234.02.FA028CD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 5CADF180006 for ; Mon, 15 May 2023 09:34:55 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B8EtneKn; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=E38vqhq85Z1Ni9xIOzFYfSP18HbbEd6JRy0LxSp6RFlR6xXHuC1EOQJ8nlvcsD8xRoFT49 4jFkFZpAkr0pPWcAUeyF9L9wCIHXvYcwFWJUtiVhoS3bGrKeP4ILcLLC30NiTbOxlo05FJ l5ROafLo+VGQJxkQ9M3+SPlAUT/OkYs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143295; a=rsa-sha256; cv=none; b=eUr/B8YBKcuewB9PQgztjRiHjHTTvsHX3ITVbkqQQvYEp6dDbaiDZS91CNyyniQW0OjER4 S9Oa4Y/hCrzf+77d2sleuC+jMZxPnF376slgtVXLNz541xymfTehtWO/SOkYqUbeh9tAev Av9NiXPV9e9vQOTtEMoM5VbSfpl+K9s= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B8EtneKn; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143294; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=B8EtneKnfazipX5zzGu2VJXt3pLn33aPFx8p9MM5c7e9gOB/ldFCf/PZt4LgVPwK29juqX pImO40SZxRFB5Zg+6VN3rwbveY5+6T7nhX5s+/OtL/7OCMm6NZYJaIzwKw6j6ZCHlbeko1 khp8GGBn5tCJ7hD9lejqMDNt38q2J/w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-447-17qNAQVlPLmO5920f3R9FQ-1; Mon, 15 May 2023 05:34:51 -0400 X-MC-Unique: 17qNAQVlPLmO5920f3R9FQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 83AFB80080E; Mon, 15 May 2023 09:34:50 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1799A40C206F; Mon, 15 May 2023 09:34:46 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v7 14/16] ip: Remove ip_append_page() Date: Mon, 15 May 2023 10:33:43 +0100 Message-Id: <20230515093345.396978-15-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Stat-Signature: hmx67upmxgyo4ikqr3yx17r47dexoj5d X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5CADF180006 X-Rspam-User: X-HE-Tag: 1684143295-268609 X-HE-Meta: U2FsdGVkX1+gVKcvMRBr/gSN0M/0Zn9c75bA37pjFG7Bxwtx656ivVeUd2fIVLOw23OOqWooI0Aeynq1nbHuNARbg0mo53trcPGBwINBJ8WeY3ZSNUzokts6QGUHqU9V6wvZFdfcmzqr0ZqO9VyZid0g2SJA7LA6yVoxUxTT8jlogltYdO6b5DmtieCnt6CTN1sel5FeRjKMR2zjtOF5vLLlAJmfwdKJ9IoKghUpqZKJZGsYwY2LHERX7dx6yan1jeH6hjWB6w1pxhn1nY+Ua6mVyk0H3d3s0c0fh+oFRHnccHY8xbA/grFrL7KK+s0e5nXxU8sePfyDy20FUg1me7wDFmpTeGjoS3leU5ltZt1HqpF1td02MMy1sM2tb4V7YIUJPSfjoSFRTfOdlVup5ZovSlE0NCP62m+cdJdJEF3or08A22AEuWaiAkt4wjk4w6TYAVXJ4aZG/KGoJmpfZo7Mh4+tQnYaRv+XCLtZRXc737+dvrXmE3sUxjBgtZsaHmF3S1S4TQA8vbAj28Vx+hiIbug1EX6l20hQl141xQ+J25s2eDjNuzBk+XHoDHvE8hZmm9yQWGnIqVgN2cpQubn70O7sbDvxFScfDN2nQBQclzlCfYNp/e1K0VqFK71tetStPyygp+cG/jvOdo7XdXQWu6tag6V4uVWOkPB78hzqkjrSWSj+3Mkl4xJvPPoL4V5/boBWvxpwZ53KJPTbfWzme52IeItcUGBFbfPDnp6XNToyFyOoj56XQcRZ1BwhbviVpEdKf1Q4mDihcT/VwGur+wjsV7dXug9FLFdSSQilLOp4pZoxP1W7z4ozohMznBPXAaercEMjugtPnYIYpYctjtcKf0PwpbPTE2q5jJ24E9YwcQZg2awzG4cC8TXPeF6W9frJlKq+A/RS09p4DbCnbNruQ/vrdPYiK3MSdLHV3ICNE+VyrsZwHu50Z+wvbNo+ShxvwlhHH457Um2 0s0J+kZZ fFUJz8HFHbi36J+Dmb0mSmzxt7jt8PKBec3IqQy30l4Z9ovv5xuOB8H268e/2w3CyQGfk6ZNUXRsEi09vXYDk9Hxt4l+T7i/QPPBhbx/ey0CYJOORYmYcs4C+fShpLr7L6KXyvdTQzH3wUI3zHRS3pts6GBqgITx5b0i2GbMmkeAwZIagyMXY5iPEyaslWsSv9Cump81h6lVAD3sN1+Wth2eIbuSzYjaIDAJ7MOI4RcMDEJCfq2HseAek84sOWAm0KVJNIk3UGBxB7CLKAEdpIl36QBuZN5YBzKbYzazX1kSAq6wJnPQMc9HH53AtabGoZ6qZ4326Jg917epdOnV4PTuej8b06Bhg4NBXb3hEiTklxQyZRpEfwmDprbkod3dsB48o2p/QLqrv1+hIgKdVbipQqCT2DNL4PhR2GIBUuv+mSWtFa1thIVI6K7UaUBCKz/gz2XeF6QlDFxXmJ3cEBPLcoNeskEfYOB6mVbt2RPwc06BKKZrFv1R/hatwuy9K67wHN6HrkjLbD6c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ip_append_page() is no longer used with the removal of udp_sendpage(), so remove it. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Remove now-unused csum_page(). include/net/ip.h | 2 - net/ipv4/ip_output.c | 148 ++----------------------------------------- 2 files changed, 4 insertions(+), 146 deletions(-) diff --git a/include/net/ip.h b/include/net/ip.h index c3fffaa92d6e..7627a4df893b 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -220,8 +220,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, unsigned int flags); int ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk_buff *skb); -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags); struct sk_buff *__ip_make_skb(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, struct inet_cork *cork); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index c7db973b5d29..553c740a6bfb 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -946,17 +946,6 @@ ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk } EXPORT_SYMBOL(ip_generic_getfrag); -static inline __wsum -csum_page(struct page *page, int offset, int copy) -{ - char *kaddr; - __wsum csum; - kaddr = kmap(page); - csum = csum_partial(kaddr + offset, copy, 0); - kunmap(page); - return csum; -} - static int __ip_append_data(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, @@ -1327,10 +1316,10 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, } /* - * ip_append_data() and ip_append_page() can make one large IP datagram - * from many pieces of data. Each pieces will be holded on the socket - * until ip_push_pending_frames() is called. Each piece can be a page - * or non-page data. + * ip_append_data() can make one large IP datagram from many pieces of + * data. Each piece will be held on the socket until + * ip_push_pending_frames() is called. Each piece can be a page or + * non-page data. * * Not only UDP, other transport protocols - e.g. raw sockets - can use * this interface potentially. @@ -1363,135 +1352,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, from, length, transhdrlen, flags); } -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags) -{ - struct inet_sock *inet = inet_sk(sk); - struct sk_buff *skb; - struct rtable *rt; - struct ip_options *opt = NULL; - struct inet_cork *cork; - int hh_len; - int mtu; - int len; - int err; - unsigned int maxfraglen, fragheaderlen, fraggap, maxnonfragsize; - - if (inet->hdrincl) - return -EPERM; - - if (flags&MSG_PROBE) - return 0; - - if (skb_queue_empty(&sk->sk_write_queue)) - return -EINVAL; - - cork = &inet->cork.base; - rt = (struct rtable *)cork->dst; - if (cork->flags & IPCORK_OPT) - opt = cork->opt; - - if (!(rt->dst.dev->features & NETIF_F_SG)) - return -EOPNOTSUPP; - - hh_len = LL_RESERVED_SPACE(rt->dst.dev); - mtu = cork->gso_size ? IP_MAX_MTU : cork->fragsize; - - fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0); - maxfraglen = ((mtu - fragheaderlen) & ~7) + fragheaderlen; - maxnonfragsize = ip_sk_ignore_df(sk) ? 0xFFFF : mtu; - - if (cork->length + size > maxnonfragsize - fragheaderlen) { - ip_local_error(sk, EMSGSIZE, fl4->daddr, inet->inet_dport, - mtu - (opt ? opt->optlen : 0)); - return -EMSGSIZE; - } - - skb = skb_peek_tail(&sk->sk_write_queue); - if (!skb) - return -EINVAL; - - cork->length += size; - - while (size > 0) { - /* Check if the remaining data fits into current packet. */ - len = mtu - skb->len; - if (len < size) - len = maxfraglen - skb->len; - - if (len <= 0) { - struct sk_buff *skb_prev; - int alloclen; - - skb_prev = skb; - fraggap = skb_prev->len - maxfraglen; - - alloclen = fragheaderlen + hh_len + fraggap + 15; - skb = sock_wmalloc(sk, alloclen, 1, sk->sk_allocation); - if (unlikely(!skb)) { - err = -ENOBUFS; - goto error; - } - - /* - * Fill in the control structures - */ - skb->ip_summed = CHECKSUM_NONE; - skb->csum = 0; - skb_reserve(skb, hh_len); - - /* - * Find where to start putting bytes. - */ - skb_put(skb, fragheaderlen + fraggap); - skb_reset_network_header(skb); - skb->transport_header = (skb->network_header + - fragheaderlen); - if (fraggap) { - skb->csum = skb_copy_and_csum_bits(skb_prev, - maxfraglen, - skb_transport_header(skb), - fraggap); - skb_prev->csum = csum_sub(skb_prev->csum, - skb->csum); - pskb_trim_unique(skb_prev, maxfraglen); - } - - /* - * Put the packet on the pending queue. - */ - __skb_queue_tail(&sk->sk_write_queue, skb); - continue; - } - - if (len > size) - len = size; - - if (skb_append_pagefrags(skb, page, offset, len, - MAX_SKB_FRAGS)) { - err = -EMSGSIZE; - goto error; - } - - if (skb->ip_summed == CHECKSUM_NONE) { - __wsum csum; - csum = csum_page(page, offset, len); - skb->csum = csum_block_add(skb->csum, csum, skb->len); - } - - skb_len_add(skb, len); - refcount_add(len, &sk->sk_wmem_alloc); - offset += len; - size -= len; - } - return 0; - -error: - cork->length -= size; - IP_INC_STATS(sock_net(sk), IPSTATS_MIB_OUTDISCARDS); - return err; -} - static void ip_cork_release(struct inet_cork *cork) { cork->flags &= ~IPCORK_OPT; From patchwork Mon May 15 09:33:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B1AAC7EE2A for ; Mon, 15 May 2023 09:35:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01990900012; Mon, 15 May 2023 05:35:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0D0A900002; Mon, 15 May 2023 05:34:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5F1E900012; Mon, 15 May 2023 05:34:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C40DA900002 for ; Mon, 15 May 2023 05:34:59 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 978AC41430 for ; Mon, 15 May 2023 09:34:59 +0000 (UTC) X-FDA: 80791980318.12.0B1C652 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf29.hostedemail.com (Postfix) with ESMTP id D64C3120010 for ; Mon, 15 May 2023 09:34:57 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hbE2U9BC; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143297; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=Noer+xEcweIaxqI3Wp40eBeOHcad5oDERWfCtnu/+pPBIg07niPoAI9wrussf5DSXoJuXf rP4cTcDHoWdhBYd5xQSyBJuHgf94E8efqp6Wq9Tnoj7pEv+j3Htz5Vxu7N7kf7z8VtT8/e elL9PT+wkgPHKyp4ZwygrffphiHQQLc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143297; a=rsa-sha256; cv=none; b=edK4+1UwWUmMLw2NDgXJwldi2jz0ngZVnFuCD2m9Qpvf8XbC/GlbAE+KGoaXPTqT1hsJ5O NdqqcPeGGsZZtRtyaJoftvl7rpm3xKNfdEJk3XffKk/zO9oquNHJQ+cJnn/D68p18mTTzR jahaAy8e+wa1U11v+4xbGS6zDJLILdM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hbE2U9BC; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=hbE2U9BC+0EyCMwSxQnEHXFrkYWeB58WJTvkgpeRQ7f8saAPNcHptIjT+xqctx5/Ptatf4 VbLH+9K9qpd2eQFlYVgnsyzpLklizarbH6m3W2H41DWz/FGgxDiFhxNYSZ7Vr5IL2hhm1t Tg8J4YF8TU8rFKhqOUfA4ioEEfmVuaY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-351-oLogL-wgMESbZ_pbZl0Txg-1; Mon, 15 May 2023 05:34:55 -0400 X-MC-Unique: oLogL-wgMESbZ_pbZl0Txg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DE904185A79C; Mon, 15 May 2023 09:34:54 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id E86781121314; Mon, 15 May 2023 09:34:51 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v7 15/16] af_unix: Support MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:44 +0100 Message-Id: <20230515093345.396978-16-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Stat-Signature: 99amf7gyoct411om5j5toeidhddetu48 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D64C3120010 X-Rspam-User: X-HE-Tag: 1684143297-666522 X-HE-Meta: U2FsdGVkX19gHMsQ4s4zeiNX8KjKyhVmm/yleCgaYAXFQfoE2t/PjUhpoJELNkx0TbG/PI17GxW5wXGCLm3mg1NvJyCzmfF3gm328mAjETkeF58By9esydCKEnqqa2PBKDnZIHn4m+Qg0hi/l247N1OrDIYisbiI+BXASMogzy0eSNn68asxnRc/LG3bbVuxZ3WdvaWq0lNvYOEQ6AsWAlNwO7Mm8WQpS6YQx9vrfJLJZROUCog9Z3PqFWqb4+f/E+gQP6PcEWxdQ4uwfQTfkBnRlAgAGxKut9vf3HqNrHTss6Lt4tJPoZlzi/m9OS/jtkt0ywyowS7twlBx6Ac2kKf6FCN74h3NdcgGl7o4efeav6Er8TrcOM+uejhN/Dht12G6AsRxZ32sTLRTv6PX6dkhgkUxMobJaqOo9wSzcr8tdqIHkuqCpKLB/DJrfmR/hRfjeky4fAtYt1C4jzyQroLoK3aGfKw6LkKRxqdThAO9jHHaGbNTqP5o+gXPMdJnc4X2zW7IKynzgZNVve6PG+G7TgN7VhU8O85PFXpQewpgh/4iBt70Xfi23rRyTFoywdyLiwSWRGO3uX0iQuUhOLkPo+P4xloMIn1QgFGxjNoskUxG0M7jx1dSLXeQirrDSn9mLmwt/dUhHzYWSzZVGc8OjDKaDEX7vzT7aubqB/UrCafYwyzea/AuFZNlTBjRB7EsFsdcCicyEgrTDdxPYulAaNOF3Q6I7/z6miCpkdL+MaapgJCExFzYXpuHTZgXbzCr4m4fHkEKUQu72GjAmb0Ky9er4Kosq4NET/66dn8WNvQ722ywBBqlNCnrvBc/5Z32rY3vJYhOLe6kJVkzxL3deOZqixpDcV752XrpL0x02IECTKx21xSlfrHcNry8Q/Je11LK1C7OcxwiJjG/cPVuvc/mppqVvzwP1EcRgecU/SsKfeUOGZSrU3BzdG+rwvJA7aeyXLdu0h6b50t L2sQnISE hlSIcbZ8Jutks08lsYBDmYvN63GLlIYwdCkHyi3IixkvIGxja73XfnoXY5UFelXFvzgTZGV6XxMRQCciRpEYOqrKE5xjKznxjTzAIXmeVuUKNCVF6L4EqPbuMfrJwqKoMvevQ8/oH0FG+NO070iGSvRF+YjmlGXbwxjHgbqF+JjIhxYCaIuiZpbe/rNyH9gVDI0640ZvZRJJ6i9nEUDRHsS6hEmjfEkx46VXJIwCOgG9AaoaSPIuqq2+fmrs/Ng0jhPdLWVPf50turlNDhVdcrYJj3KHVJiCgDcWw0MsMK8VK3iTE5rZNP/u6pw1ZRSnpTrQYqoYFF/ZVPxwmv1/OpRFXRUmSPjrnwsXS/QRV/8FHUarOxKVqPnaRlCVoOWBrJyy1Ju71+/mDs2CzkIyrzVkz4zpx2KTOox59gVkErr7Owt/6k3m4q28+rURVmOhqZ5h2UVkMiDmZymcqH5QTi6xbf2nwlNI7yKD+kx35YfZExTzt8ikIi4+tkAnS84WISDfQe3zdXxh3esQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make AF_UNIX sendmsg() support MSG_SPLICE_PAGES, splicing in pages from the source iterator if possible and copying the data in otherwise. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/unix/af_unix.c | 49 +++++++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index dd55506b4632..976bc1c5e11b 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2200,19 +2200,25 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, while (sent < len) { size = len - sent; - /* Keep two messages in the pipe so it schedules better */ - size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + skb = sock_alloc_send_pskb(sk, 0, 0, + msg->msg_flags & MSG_DONTWAIT, + &err, 0); + } else { + /* Keep two messages in the pipe so it schedules better */ + size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); - /* allow fallback to order-0 allocations */ - size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); + /* allow fallback to order-0 allocations */ + size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); - data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); + data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); - data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); + data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); - skb = sock_alloc_send_pskb(sk, size - data_len, data_len, - msg->msg_flags & MSG_DONTWAIT, &err, - get_order(UNIX_SKB_FRAGS_SZ)); + skb = sock_alloc_send_pskb(sk, size - data_len, data_len, + msg->msg_flags & MSG_DONTWAIT, &err, + get_order(UNIX_SKB_FRAGS_SZ)); + } if (!skb) goto out_err; @@ -2224,13 +2230,24 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, } fds_sent = true; - skb_put(skb, size - data_len); - skb->data_len = data_len; - skb->len = size; - err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); - if (err) { - kfree_skb(skb); - goto out_err; + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + err = skb_splice_from_iter(skb, &msg->msg_iter, size, + sk->sk_allocation); + if (err < 0) { + kfree_skb(skb); + goto out_err; + } + size = err; + refcount_add(size, &sk->sk_wmem_alloc); + } else { + skb_put(skb, size - data_len); + skb->data_len = data_len; + skb->len = size; + err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); + if (err) { + kfree_skb(skb); + goto out_err; + } } unix_state_lock(other); From patchwork Mon May 15 09:33:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13241096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A93BFC7EE22 for ; Mon, 15 May 2023 09:35:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4611D900015; Mon, 15 May 2023 05:35:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 40F57900002; Mon, 15 May 2023 05:35:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B099900015; Mon, 15 May 2023 05:35:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A079900002 for ; Mon, 15 May 2023 05:35:06 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AE6BF121411 for ; Mon, 15 May 2023 09:35:05 +0000 (UTC) X-FDA: 80791980570.07.D203BB7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id B2FEE1C000A for ; Mon, 15 May 2023 09:35:03 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JycCFDJH; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684143304; a=rsa-sha256; cv=none; b=tKqclS9dPSex4TGhrne39KYDGBKr3dG/VP2ddcVaaoTXa1Ib29D5FFFW62J5gmA9R4kyOk HWmbcAwSvNRVT3qXWO5ZUj4iuPQjaRsG8u6BacaEDQzFSEmWcnXmW82FDgHE3sWhyAiVgE PQydf06LGQ3C5nxWbM2QodgIn31VBRo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JycCFDJH; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684143304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wF+2b4fBEfy00AN3VDBr6eYpnbX7vBgnsz0rKmJdTGo=; b=XyXR9TFlbCSUK6EixR6GcR/BdbFrTLME4Hzt9A453OcDUPr8qZ3NV1qihthtkknd9zJDWI CaWZsoJIV7KmbhshtGjERyTZHhkUrO76rfmd0F7bJ81Gp4ies2egCe4xoQzqnQOeTQRTo0 ddMBR5sFsVOzIEcQCAL3Y9kyV9S8N6M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684143303; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wF+2b4fBEfy00AN3VDBr6eYpnbX7vBgnsz0rKmJdTGo=; b=JycCFDJHG01b2UP8fyduzTAF3ITz3l/zGnBfg7LOJ8fqRWbQB4qC5JQGGAV6jtEKOItqqO w7G4i7i4j6ayRv9jM9LSQ+zSWcQ/5etHNobAl/HTeL1VF1lAVkQ8iCIj/+nEqxRcbbugKs 9WZLlRWjB+I/xeu4tcggVC8V6QB4J7U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-616-mv3QcuzZMBiItf3Uupm36w-1; Mon, 15 May 2023 05:34:59 -0400 X-MC-Unique: mv3QcuzZMBiItf3Uupm36w-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6FEA6101A551; Mon, 15 May 2023 09:34:58 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 880FB4021C6; Mon, 15 May 2023 09:34:55 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v7 16/16] unix: Convert udp_sendpage() to use MSG_SPLICE_PAGES Date: Mon, 15 May 2023 10:33:45 +0100 Message-Id: <20230515093345.396978-17-dhowells@redhat.com> In-Reply-To: <20230515093345.396978-1-dhowells@redhat.com> References: <20230515093345.396978-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspam-User: X-Stat-Signature: js8y7i77apsx5qk43shqzwpkc7578s6a X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B2FEE1C000A X-HE-Tag: 1684143303-232625 X-HE-Meta: U2FsdGVkX192NuZUnVr/TfdwrRQ3Oyo+9DR0gaAVtk4AYGr2GJt/wfNtSnVbjpKxugqSUFvkFf8HakkY/uGTBv1p4a8WmMW7bC61qG2zjErcJy9fiiEPjLn/uTncmx2HZOeleyb7SNyGBPUqMvmRUJDrpDMSt1WZxM7UCfgKvyRPSds0AtkGKk2j1ha98+grXSO5y0LOPg3JFSl4D7hb1PIgew18Hlj3JcVHEC6RLlhyNwBzUzuNeB6iOLrFmp2p6qq9SmtX0dUauqNeFudw+f+OoBAC8qVCSdZ33NMJP6U+McjgH6TYQ80bBzTSz5+E1GmynvgkYgKaqgKJmNgTzDa027LawEig/WEEqVZ2ab0J+VSBRGv7xPw2VGXvt9nAHYRqJaPip2FNVxmRCnUO7AWO8yOpRlHNif+Cs7Bj4+wm9notOUit8X6bl4By1c0Yzi3tdmWawLlYgPYwU4jkZEtDAM7plYKNP62NNLCJOliN+LIeUeyO5bddORoBa1jMY7vx1fhKuViFfaZrYK3tmklXTRHuC5QRfg48dFcC1HKk+5s4DMCl1GjPXg0Ho+WXLw3rulh53zoCIUytOHqnsQiMkzhFQP9bhOcFMZZRcTG5XiBlyMF9QJQTkQNSlR1ibpkGbEkHWEg41j0UqztFYS7tOcgI+5AwvML6T2qG2Ubw9lRsDN9Zrc4ixmgmJ+ryO/9WvX2hwzbdhvJoIvMmpgq6sBCbh4kf6j+rps8TkYM9uAIdxK6DoQAmTdPgYE+h+EJwPGit+rE5k1x8uQoT+gAITFrQOAHWyPZDl2Y3shcL8Jx12LEpkqQC8jIDkKgo5p1M1V/tOZbRi1C3NettuPwihptT11naqP9/aCz6JwDl/r97IB0pFhkpqauxbSAYvcJvaREoPXEYOXLvELgR1a6QIxvUdug5EgLBFtKflFasP0FxGaf6uXsloC3MxYwZYx2dIXXKf16XBAicyzO wkipDNb/ PQAQroWHXh7TLLzgNucI52NbVLkl9wHB5PwF5/rKKObzoXPMwAW98oISJ90Iqo5+v023KR+Un8K27N3te9kczYEyCO7JOfA8PjbyPhNnW313n//GQJHsGDXO8m8cnvLiXQ7ceZhyIJaKILPZXDLNEuKpXIms9aNGHOrDqq5LaCR9y16OJ/NNsJZWvuOT8Gj2hAm7Drf5S0tVvqMnxW2a5N4DikWa3x5rDWtjY462wz1yDU4g9tpKygR6Iv9UHC6zohADnKVPPUTwrOwI482ZaNsjAlpM8YqH0NoyMu7TUJ5P0DbRE5XypYJyhy8rnAFfCYOVSVPedeQ/NfdjSBAZBQbg92AzrWJC2MUQjIDsNmCC7coEP29JhSldd5BFl5lmrUqVUKtTUY8M/HFR+qyN3x0LifEWlK9YdhPzhFXayzLhMDdcmkobS3tYNfSYOE2xjASyzJLwK69hiA8I3QQXn2LchsOY4UQnXNEMbSPx6L8UQ3X2QR1JGq77CzQ2ffSX29ZgwR/jfIMPBigs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert unix_stream_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/unix/af_unix.c | 134 +++------------------------------------------ 1 file changed, 7 insertions(+), 127 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 976bc1c5e11b..115436ce1f8a 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1839,24 +1839,6 @@ static void maybe_add_creds(struct sk_buff *skb, const struct socket *sock, } } -static int maybe_init_creds(struct scm_cookie *scm, - struct socket *socket, - const struct sock *other) -{ - int err; - struct msghdr msg = { .msg_controllen = 0 }; - - err = scm_send(socket, &msg, scm, false); - if (err) - return err; - - if (unix_passcred_enabled(socket, other)) { - scm->pid = get_pid(task_tgid(current)); - current_uid_gid(&scm->creds.uid, &scm->creds.gid); - } - return err; -} - static bool unix_skb_scm_eq(struct sk_buff *skb, struct scm_cookie *scm) { @@ -2292,117 +2274,15 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, int offset, size_t size, int flags) { - int err; - bool send_sigpipe = false; - bool init_scm = true; - struct scm_cookie scm; - struct sock *other, *sk = socket->sk; - struct sk_buff *skb, *newskb = NULL, *tail = NULL; - - if (flags & MSG_OOB) - return -EOPNOTSUPP; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; - other = unix_peer(sk); - if (!other || sk->sk_state != TCP_ESTABLISHED) - return -ENOTCONN; - - if (false) { -alloc_skb: - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - newskb = sock_alloc_send_pskb(sk, 0, 0, flags & MSG_DONTWAIT, - &err, 0); - if (!newskb) - goto err; - } - - /* we must acquire iolock as we modify already present - * skbs in the sk_receive_queue and mess with skb->len - */ - err = mutex_lock_interruptible(&unix_sk(other)->iolock); - if (err) { - err = flags & MSG_DONTWAIT ? -EAGAIN : -ERESTARTSYS; - goto err; - } - - if (sk->sk_shutdown & SEND_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_unlock; - } - - unix_state_lock(other); + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - if (sock_flag(other, SOCK_DEAD) || - other->sk_shutdown & RCV_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_state_unlock; - } - - if (init_scm) { - err = maybe_init_creds(&scm, socket, other); - if (err) - goto err_state_unlock; - init_scm = false; - } - - skb = skb_peek_tail(&other->sk_receive_queue); - if (tail && tail == skb) { - skb = newskb; - } else if (!skb || !unix_skb_scm_eq(skb, &scm)) { - if (newskb) { - skb = newskb; - } else { - tail = skb; - goto alloc_skb; - } - } else if (newskb) { - /* this is fast path, we don't necessarily need to - * call to kfree_skb even though with newskb == NULL - * this - does no harm - */ - consume_skb(newskb); - newskb = NULL; - } - - if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { - tail = skb; - goto alloc_skb; - } - - skb->len += size; - skb->data_len += size; - skb->truesize += size; - refcount_add(size, &sk->sk_wmem_alloc); - - if (newskb) { - err = unix_scm_to_skb(&scm, skb, false); - if (err) - goto err_state_unlock; - spin_lock(&other->sk_receive_queue.lock); - __skb_queue_tail(&other->sk_receive_queue, newskb); - spin_unlock(&other->sk_receive_queue.lock); - } - - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - - other->sk_data_ready(other); - scm_destroy(&scm); - return size; - -err_state_unlock: - unix_state_unlock(other); -err_unlock: - mutex_unlock(&unix_sk(other)->iolock); -err: - kfree_skb(newskb); - if (send_sigpipe && !(flags & MSG_NOSIGNAL)) - send_sig(SIGPIPE, current, 0); - if (!init_scm) - scm_destroy(&scm); - return err; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return unix_stream_sendmsg(socket, &msg, size); } static int unix_seqpacket_sendmsg(struct socket *sock, struct msghdr *msg,