From patchwork Mon May 22 12:11:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26329C77B75 for ; Mon, 22 May 2023 12:11:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7052900003; Mon, 22 May 2023 08:11:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F95F900002; Mon, 22 May 2023 08:11:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84C08900003; Mon, 22 May 2023 08:11:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6EB5D900002 for ; Mon, 22 May 2023 08:11:42 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 321F440290 for ; Mon, 22 May 2023 12:11:42 +0000 (UTC) X-FDA: 80817776844.03.F274640 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 6B66B1A0018 for ; Mon, 22 May 2023 12:11:39 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cWm5s77X; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757499; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zo53+MxtcExCaAvP1E3d2cbKIX8PzAGUnvJuN4UlUmE=; b=1Y2ms7jUk+GlBSBUxcCzGRlCJ37m7paB9k28iHJSqVJxveosuf73/6+qr2glMM71yAmL+k J3mQTzs8qzaP4My20+t6itBIXlUlbUlcXgGLg2HnbnogOkEm/htgjokOBTaogZp578bKDo dpxKt/otKhMekFfPZW2YVadoMEJtF3g= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cWm5s77X; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757499; a=rsa-sha256; cv=none; b=ssahoStKPR5FjShvNUqFxbTEg4PMn+VnDLFgedUTt/AD3tKdxDqF+TZ0LTQLELZuqko0o3 IL1kKFwaJ37Og0lls7p2Ko29JsL5YXcimnOisQrhdTBBBQUoJ5kTTO+FMVkGAWe3M5uG/K LI924eF/+sU2Xn8SbF8pvRyQvj66Ub8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zo53+MxtcExCaAvP1E3d2cbKIX8PzAGUnvJuN4UlUmE=; b=cWm5s77XtGqo27sxpLyTWLrmOe4XoB/2V4fNOHcaK88hwCs3P045plsHGayumDdi6rMFMr cZznylM5QtQzsF2moq6qoH3cuJsjobEvUpwGvNaKTNuIibK4fxFIcXk6YeKr2GAMYAOYaM u4f35MpHR/CH7A17OQ4q8R24JsBdgV0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-LAZUC1NYOiyGQR7ae9mZ2Q-1; Mon, 22 May 2023 08:11:34 -0400 X-MC-Unique: LAZUC1NYOiyGQR7ae9mZ2Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6FA7E185A78F; Mon, 22 May 2023 12:11:33 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id A54FE140E95D; Mon, 22 May 2023 12:11:30 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Willem de Bruijn , io-uring@vger.kernel.org Subject: [PATCH net-next v10 01/16] net: Declare MSG_SPLICE_PAGES internal sendmsg() flag Date: Mon, 22 May 2023 13:11:10 +0100 Message-Id: <20230522121125.2595254-2-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6B66B1A0018 X-Stat-Signature: 3yhwshiqstmd6zdzttticormkihuba9b X-HE-Tag: 1684757499-678842 X-HE-Meta: U2FsdGVkX1/GYrmQe8O/C0xmKkWbliJMAwovUUpUBQESn69zDpbpc4tH6Gdq8p9GSzv0Y7bKC7Xh9TQLy8QIdd/S3Trat0mAm8INu17HkmpXB8hDxEXsUKkVgOLu5BQRNEZ52KZzwP4yVlSqBk3u7QXCzVjWM2SEmlJ5cEYkpuo0Ry8bNepQz/8BnCkK8Gpy16HMK65Jx8y9AbN1zX1edGjtNhdzipkohHz+xDfd0txgxwF2jmHB+XwwNthynnJAPaIXCyWKPQ6hJ3d7Q+1XEwfCzp669sEAl3jX8rqPPzdZPTwCd/cd+QzVA+PnLc1s0R89DztqljRzUC/UJvtHPurD+5XwZGsfRnEja2SVlEThEnV8OiFYb8UbhR9N0NYzKH+ulpYTdNINOZAg3dEznZ3wLWm39iXsgFtaJMEX4MMD8HQVqYbbd0naWH/6LuGMWTqjfijNfrYqSR9BQxykb+OrWuRWuc+g9OQ/4biloQdnlw052vm6c0SWujKn9S+l0ZuC0UUMeoLzIwBiq/ymLw+Su7kkCjsWY2digcKb5eChagKssB/wdIaFmNbTClwsTCD7DBmGy2PkD7IgHmdrX9xqEQMSPp51vAdIT0J8tZ8/bZpXkXK/8p3/NPlEysvDuIZMbaaYvi8ldI3CTMxt4yNhr/e8ZnaCpANLEc1Vm4jiQXEo6+sKVsE9wb8bCUCANspZb6LC4nLpz1CFxhJEhrtwE5chL7bQnN1fwBe090L81ommC9ghYYk+Xt39ql0MrAhavcUoNNb+lHE34lN61+O7SSufzHXEAzKw26rdoeXbsvivReZzaM5EG6uje0ReRzo0sDVC4bXraXQT1S3aZB42PFnMbkHiEe0BLnAW6L/MiTQjVz+Lb0145AGfDTiJfe+OD3fPX62JEDS+a3Ka2Fjl8fX9JY6t45GIFMMVucQmJdKt1T1L9vJMZnqb8Q1d7dWt8u+EA+yTnzvEYQo jliZsKqf OR1P/jFj3W/A3WnBJvJ1w18dSeW10x2kxq1viHjadRd6fHKnUZ27D/QSjGsayRSmh2EKUIYmuicyjz+Va14pmMa7V8nzKFyGF0b02JvNtsDejtbF7qmZco6HuCATHT3Gm2NxxXqtWvlrBzKooY0IAqOdA4/XVGjJd/zp4nki0HW3b4eHsZyVMysogW60gFA6OvaKSFFOF9JK4AtItRKMr5ZtV4fbJeYjc+NI8QzgidmncH1jPCOJ/+kn6ccBC1zsXxj035xoZBQS0ohHJ0/pmTcyCPMFMQlhAx4pypj6YRjLtssbAQ4dr+xdGrMZwrY3c9cgcZFCoC8p1TxuvYrp7gBFr0L/D1WAjZHnARep7wvNQWyLBwucVj13k2cFxoBc/pvtUQVoWpMOPLRHYKrfftf2+GRYE0mSS/Rk2HrLHQYDTmpA9tzKYBQUKv34L3O/0nP78DHfXtR7eb3YELnUtKwoej5UBkuJ5m4wYYvSxbDRhpFjGiuxwiW4qZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Declare MSG_SPLICE_PAGES, an internal sendmsg() flag, that hints to a network protocol that it should splice pages from the source iterator rather than copying the data if it can. This flag is added to a list that is cleared by sendmsg syscalls on entry. This is intended as a replacement for the ->sendpage() op, allowing a way to splice in several multipage folios in one go. Signed-off-by: David Howells Reviewed-by: Willem de Bruijn cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: io-uring@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #7) - In ____sys_sendmsg(), clear internal flags before setting msg_flags. - Clear internal flags in uring io_send{,_zc}(). include/linux/socket.h | 3 +++ io_uring/net.c | 2 ++ net/socket.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/include/linux/socket.h b/include/linux/socket.h index 13c3a237b9c9..bd1cc3238851 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -327,6 +327,7 @@ struct ucred { */ #define MSG_ZEROCOPY 0x4000000 /* Use user data in kernel path */ +#define MSG_SPLICE_PAGES 0x8000000 /* Splice the pages from the iterator in sendmsg() */ #define MSG_FASTOPEN 0x20000000 /* Send data in TCP SYN */ #define MSG_CMSG_CLOEXEC 0x40000000 /* Set close_on_exec for file descriptor received through @@ -337,6 +338,8 @@ struct ucred { #define MSG_CMSG_COMPAT 0 /* We never have 32 bit fixups */ #endif +/* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ +#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 diff --git a/io_uring/net.c b/io_uring/net.c index 89e839013837..f7cbb3c7a575 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -389,6 +389,7 @@ int io_send(struct io_kiocb *req, unsigned int issue_flags) if (flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = flags; ret = sock_sendmsg(sock, &msg); if (ret < min_ret) { @@ -1136,6 +1137,7 @@ int io_send_zc(struct io_kiocb *req, unsigned int issue_flags) msg_flags |= MSG_DONTWAIT; if (msg_flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + msg_flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = msg_flags; msg.msg_ubuf = &io_notif_to_data(zc->notif)->uarg; diff --git a/net/socket.c b/net/socket.c index b7e01d0fe082..3df96e9ba4e2 100644 --- a/net/socket.c +++ b/net/socket.c @@ -2138,6 +2138,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags, msg.msg_name = (struct sockaddr *)&address; msg.msg_namelen = addr_len; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; if (sock->file->f_flags & O_NONBLOCK) flags |= MSG_DONTWAIT; msg.msg_flags = flags; @@ -2483,6 +2484,7 @@ static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys, msg_sys->msg_control = ctl_buf; msg_sys->msg_control_is_user = false; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg_sys->msg_flags = flags; if (sock->file->f_flags & O_NONBLOCK) From patchwork Mon May 22 12:11:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A89B2C77B73 for ; Mon, 22 May 2023 12:11:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4324D900004; Mon, 22 May 2023 08:11:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B9E8900002; Mon, 22 May 2023 08:11:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16ED9900004; Mon, 22 May 2023 08:11:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EFB1B900002 for ; Mon, 22 May 2023 08:11:43 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B86FDADB30 for ; Mon, 22 May 2023 12:11:43 +0000 (UTC) X-FDA: 80817776886.15.4826DD2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id D478C1C000B for ; Mon, 22 May 2023 12:11:41 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="fBY/C8Hp"; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757501; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dMBJPtbzh4IjFjJ1HeY2h7yLKLc/0VkfcRg1OdFJq1c=; b=wKDEj9/Mra4hHhncihWdW6d8P3DmA4m8zytqdNIYoDIVZsJd2ISo2eOrbvCLVKAZlb8QrJ a9HBCWl9z4StRWouFDcUtnS+TSWfMwB2mToIaF/YtqEclr17KYR3yZU/GMEKrwgJ7a1v+x BVoyE83++ldlXhGQGhJvcbi4ZcAeN4U= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="fBY/C8Hp"; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757501; a=rsa-sha256; cv=none; b=VOLgjHrjtyIxn+BdZgG3WExBxEuHXWyOuyu4iVKRrn8lYpjNvQvliyMyLvGt8DOQUl4giU L8m3OVgibpktfXOeBfHHE1g/4N90marmlmc/gBvIoE65p+kMju5d64PMsyTNvf2HwE7Aws 3JADryXPtdUkZ50B8ZFg0SGWTLSJo20= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757501; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dMBJPtbzh4IjFjJ1HeY2h7yLKLc/0VkfcRg1OdFJq1c=; b=fBY/C8HpOJeKnd5SWjguBlniacaIEyMUrRRiYZ5FMqGcVN0vwSOPPrTNt4clFp8K0BE9if HOx4KSGpybXeKylY4zzpUXJS2wv22VlC1A2za25IJVrB9cvQyg9EoNRYNmwB+qzdfzqew2 2AdlM3m/ewe86QV1pWRlg3Hs7BeXCKI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-652-UlqM_PkRPIidB5yyeKr35w-1; Mon, 22 May 2023 08:11:37 -0400 X-MC-Unique: UlqM_PkRPIidB5yyeKr35w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D8CBB3800E87; Mon, 22 May 2023 12:11:36 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5FD694F2DC5; Mon, 22 May 2023 12:11:34 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 02/16] net: Pass max frags into skb_append_pagefrags() Date: Mon, 22 May 2023 13:11:11 +0100 Message-Id: <20230522121125.2595254-3-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: D478C1C000B X-Rspam-User: X-Stat-Signature: 55pj6pyb1oan8g1mpdt9bay1k4pnouud X-Rspamd-Server: rspam01 X-HE-Tag: 1684757501-757049 X-HE-Meta: U2FsdGVkX1/u2dNObyUJOQKAAosLeZci8X9C+qRZJbZwcmjT5Ts0UfE7mdBO55UJRgYd+PV3eARTSJLvHYKQaG87ncXCIjFRO1YHY38omDBUxUu6yLIcOwWXD1zF8r2Q+GEIP39U0X19KofIfpq9NENpbXDJOyFbCCDAPPQGC8wckHsjhewvWZ+ZCPU6L4Zu1+5UnfwEfYQcNr1ZArPhG9hjNuIO4SrfdRzxSFyLYuMn+EH6CxRqV+aR9Ukb1asUN0INMkDq9+D/WFJQlwXSl3nwBIl1djn77RD9Oxe479x5P6uZ/jEYJPy+cFHaE+Diukv5+BNbA3kJMTFn58sbOREa/cPmFZErAbhNd7wzNCD1b4WBTM2jrO7wUcXdD5Rrq5OchMS9GzN25XmqADqQrx1dQ8zB5Qs/JaCj/bfbZonZv+9u4YcCYmKi9lxyb10BGSFhUnnNYiBb2jfXTY9uqZPmjxwzgbOZC7VFefwvJBzdDILHHEJc2A0ZZq3ow8GfVS8qJp951W8KQSNihvmHCMw+hXiGOfBt00jjUmxlvKHnFiNOjMcpndwXw/HLe/foZeBrfISKP6H0WABBYKSo+VLFxgBOH7eFOWMj7KsqGtLIcDje01CQtKKBz01ras9l+CYazo/Ib9XXJCR+QUZCPi3/dYa3ZB0X1J9frhQgCaVZBRJiMDm5A/FzSI+ZL4QWD1IFRDe5IdcmYtpn/DoyY5vXexYzAWq3vxpfcIIelET4/vG0HCKHB9NG+wKO2oOQfvO7TH4Shnndx1pJFleps17oa6nxynwc1YF5kO1o+172/CWX5+E6XLtUOBKxhg9nJGkl8puOOwM2aCW9fcnPYUwvhHkUEZ6FUNnHFuszhkrsbxv998XPzg5hp7LlfKYvgWndE5cC5ie44WigPeo1Y1pSIzxF+5GSS3fs2Evq/EZxAQe7ngOhyJYzHPn3apH38a2Cf+B8fmQnczBsOQa 7RdZdfgj QL3/BATeGxZ4X3i+ESZoly4DG5qcLpk9ONLztK/nDL5PDm39jchmC0178DFGTKYXlFOMsxEzkleLQTF9Pxm2MlbJU18j6cT0sjWiQoxlGSMUL6lbxc46hTreWcpW3cFtGja9aT4bwGQ5lTKGyz4VHAzOCRQPpjT+GORGpgk2yU2uNEzahHKreCgz7jBFnb9D0G/nUp7hyIcsqHsynpjJitWu0Wpyx9qj3UcL1Jk7ZW0b031A5q8+4zUymxdVh5zIK+wUV/m4mHa/NM2Sdytuw9XqjzBAUCmsChcalEYDZBnR9oX/5NS7T5igLJOJhmwJtmMsV19Fc0bbVXIVJBOsqT42s4bXy6GiRaCRQpTUG/na7x+7TeWBcKd4l/pzQgCPQn9NoYBr6TUJgQPz+6RE90kjO52XudZQ0thvJXKCADJnG2E5wvbvLhpjFNJJCdzBtnRuCmL7TWBvYO+sZwS0g0HTMmRbbuslU2nn8YqKbGwwFOJA8oEqU05l5VNAHrPSihncaWlt0kGcDl36TD/iJIyyZR6VsJAFECfhY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass the maximum number of fragments into skb_append_pagefrags() rather than using MAX_SKB_FRAGS so that it can be used from code that wants to specify sysctl_max_skb_frags. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 2 +- net/core/skbuff.c | 4 ++-- net/ipv4/ip_output.c | 3 ++- net/unix/af_unix.c | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 8cff3d817131..15011408c47c 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1383,7 +1383,7 @@ static inline int skb_pad(struct sk_buff *skb, int pad) #define dev_kfree_skb(a) consume_skb(a) int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size); + int offset, size_t size, size_t max_frags); struct skb_seq_state { __u32 lower_offset; diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6724a84ebb09..7f53dcb26ad3 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -4188,13 +4188,13 @@ unsigned int skb_find_text(struct sk_buff *skb, unsigned int from, EXPORT_SYMBOL(skb_find_text); int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size) + int offset, size_t size, size_t max_frags) { int i = skb_shinfo(skb)->nr_frags; if (skb_can_coalesce(skb, i, page, offset)) { skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], size); - } else if (i < MAX_SKB_FRAGS) { + } else if (i < max_frags) { skb_zcopy_downgrade_managed(skb); get_page(page); skb_fill_page_desc_noacc(skb, i, page, offset, size); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 61892268e8a6..52fc840898d8 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1450,7 +1450,8 @@ ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, if (len > size) len = size; - if (skb_append_pagefrags(skb, page, offset, len)) { + if (skb_append_pagefrags(skb, page, offset, len, + MAX_SKB_FRAGS)) { err = -EMSGSIZE; goto error; } diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index cc695c9f09ec..dd55506b4632 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2349,7 +2349,7 @@ static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, newskb = NULL; } - if (skb_append_pagefrags(skb, page, offset, size)) { + if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { tail = skb; goto alloc_skb; } From patchwork Mon May 22 12:11:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB695C77B73 for ; Mon, 22 May 2023 12:11:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 568BD900005; Mon, 22 May 2023 08:11:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F07F900002; Mon, 22 May 2023 08:11:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36AD6900005; Mon, 22 May 2023 08:11:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 272C4900002 for ; Mon, 22 May 2023 08:11:47 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EBE331A02A5 for ; Mon, 22 May 2023 12:11:46 +0000 (UTC) X-FDA: 80817777012.04.22D5445 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 22A24C0009 for ; Mon, 22 May 2023 12:11:44 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OLD3rVgG; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757505; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fl+2zC2cOWTcM7FjMckfIDo5NeTUJ6H1d1XKuMzp1Mk=; b=ISJs0HuThfe03Rpmb45BTT+hY1xD+TmkRPYGk22QPmzmWdafqxPOf5sR2DyYfbu1+DM+hZ ia/0VlXxewZXDwdntpUFLto5kqj5eTZD14UUVhXdqfvvqVPYol0LHFCw7DvMF41XH3e6+x JINLzzQ1qHZtl/pQKfRsWlIjRy0ysgs= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OLD3rVgG; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757505; a=rsa-sha256; cv=none; b=qSCqJmAVOFNL/M40xqsUfmtX3OWXNU7Cyjh972NEBUOxrMiE2XmHJPNZr7QwaezhSJHuKf KTV9v96UMwjJn36i8Vq7hXjqA2yBvAXrUorAdG5efiGuQy/f9skurteh0zsvvLR9GYSCUy t4EQQf55zvXCLDWA5cxjZ101kjqQl2w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fl+2zC2cOWTcM7FjMckfIDo5NeTUJ6H1d1XKuMzp1Mk=; b=OLD3rVgGbPPElNzSBQR4vEKCR2APkAP90YdwE5m9wWPqT1i7UJR1i9txorm91IuOQYETaj VBiqDeHH0XFyzOtbtHwdFBkph2tZnlkOx8+9LKmRNa1/M4FVpykCIAxe7x+v77r4/V+Yzc 1UHI7YP77Zpir1T80qfeeAAMrwuQT2w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-569-AAbKytGoOYy6qpGKlO0Rjg-1; Mon, 22 May 2023 08:11:41 -0400 X-MC-Unique: AAbKytGoOYy6qpGKlO0Rjg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 12CA4185A78F; Mon, 22 May 2023 12:11:40 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 90F96140E95D; Mon, 22 May 2023 12:11:37 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 03/16] net: Add a function to splice pages into an skbuff for MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:12 +0100 Message-Id: <20230522121125.2595254-4-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: 22A24C0009 X-Rspam-User: X-Stat-Signature: ki4bte96kpmb17x63x7zef854sfy7npd X-Rspamd-Server: rspam01 X-HE-Tag: 1684757504-156492 X-HE-Meta: U2FsdGVkX18GbJiu9P7vVaZAQI90n9bAMAEBo7shAbqxSriTVA9/gAQHf0/IPF4spMr9vnonpLVTCxm39Coq4p9yL7VG07altbthUwBI1e3ggi7ld7KFu8LxvJsK4Ia1oL+haW9+EHnndDj+d2bVzSWvoxdx1Nbs0if3ppVBvSlyjiriRilL5eoAaxch3Kxr/9QYx5/dcEI4kkPV2B6hCCIHaYKjlPdfcHci5OlsLB2csB5qzovily07wip65klbjSyPRzK+VNdKbxRIwICVRuzrdrRKo0w2GinJLda/gbClsITHl6iUM8DyJjaSf9t+fwL/9yCBqXtw0brGMZT3It/0xn7TT16glK2/3w+j738rzkFFFzxHRUnxZ+r7tTjBGlIqHThlLbOn2Vsr3GnATnz5+BJm9xWsFtwvp2mkBACH0Kod8fV6Dom6TRUsflBQxRd1i3yVV5w+hV7czxoNV57obnjrf+E7+9s28Nj4fQIlm8GOWMJirvAMUc0Xqz9wfwUg2haNLWBnSTnn9tZ3BEmNkWT81k49EZ7pHYeLbj51y5Vulf7dfojYyj8Mx0+00ZfJ6nFjP6zK+9uvKloJ6k2FRi4S4aMZmBHDJ8KPCwiEEBTRxwyq81G0Loze4FE9TJt4vir7ypeTruXqkUOZEXUkG8gWNrJjmKs4NCLaqqbtpV/Ch5ztdZjFQrW7/f3VRZF4rTfR5lx7VMAvkZ1IWOqLlg+XmtDmV1jsdvw9o6Xr+KPlllJ++MvZPJIqs0uGW8cWaosQu2x+HYAU8Mvzv3ti6OpKK6BbMOhVOOUltMZH3a4PlpI1ueyB5TEDa+2ddh4oEpJyXeuiNkLsQUyVdK84Y7f1UotSlTCz1KkP8o32710F17gNkr2JR0qQfnQJLV05Slt1/MZafpj+fwKSBip8LXLBQvYDSO4q4xyOsAz4uKIb7MQZogg2Z6s3QsKmyd7Q0rI7Ycu6bKlWkk+ 3fl80oO1 5St0XWLPkx/lUnpOI1cAdSz+3Aa9GsDB6W0qazJm2OJdZf6bWckNyjyIqPhtbdVBJTTXYQ1wFdBihiZWKZsfAccuEJjiHPPJPSHiO0VwbbsrRQHp4VV9zcR2TdSnHc4IXFVFvaYCFLA7EHK6nfvRzQbHzX1bVBH4vbiXLcXwyytE4D5wfImGYKp4wilrEeGGCMn/hLgPvnaiExWfm8cnzhGqHTR/kAKB3eDy2bRz7K7xPZHZlbEeNzMQQH2ZMWJ/mZkdYeYiq/bll7UKLr80Uek8nIMDnLuh+rtl00TjGbJsv7hodMz9tUQIWTmQPdEM5qy1vvvHq03n/Ka0/IwYGLf4z1jSqlAc2Dy+JkugYFA0l3425KqS+8p9G1ldAGr+iq1kF+rFhoUUI88Wgd0WKP/1mrwLg4OxYs6RyVvY9Djf8C0jREIvLK2DEcrPhbh7dJJmbRLbftIDCDkq7eGTP1Psb4v0IA3zkDLg/d5CHgd7hzdj+OYVt0m5KgbS00PbolqEMe6g0mic/4o9Op5ymBmnyvt5Pm5ndeY3BEAW3+lsgNrM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a function to handle MSG_SPLICE_PAGES being passed internally to sendmsg(). Pages are spliced into the given socket buffer if possible and copied in if not (e.g. they're slab pages or have a zero refcount). Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Al Viro cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #8) - Order local variables in reverse xmas tree order. - Remove duplicate coalescence check. - Warn if sendpage_ok() fails. ver #7) - Export function. - Never copy data, return -EIO if sendpage_ok() returns false. include/linux/skbuff.h | 3 ++ net/core/skbuff.c | 88 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 15011408c47c..1b2ebf6113e0 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5097,5 +5097,8 @@ static inline void skb_mark_for_recycle(struct sk_buff *skb) #endif } +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp); + #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7f53dcb26ad3..f4a5b51aed22 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6892,3 +6892,91 @@ nodefer: __kfree_skb(skb); if (unlikely(kick) && !cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) smp_call_function_single_async(cpu, &sd->defer_csd); } + +static void skb_splice_csum_page(struct sk_buff *skb, struct page *page, + size_t offset, size_t len) +{ + const char *kaddr; + __wsum csum; + + kaddr = kmap_local_page(page); + csum = csum_partial(kaddr + offset, len, 0); + kunmap_local(kaddr); + skb->csum = csum_block_add(skb->csum, csum, skb->len); +} + +/** + * skb_splice_from_iter - Splice (or copy) pages to skbuff + * @skb: The buffer to add pages to + * @iter: Iterator representing the pages to be added + * @maxsize: Maximum amount of pages to be added + * @gfp: Allocation flags + * + * This is a common helper function for supporting MSG_SPLICE_PAGES. It + * extracts pages from an iterator and adds them to the socket buffer if + * possible, copying them to fragments if not possible (such as if they're slab + * pages). + * + * Returns the amount of data spliced/copied or -EMSGSIZE if there's + * insufficient space in the buffer to transfer anything. + */ +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp) +{ + size_t frag_limit = READ_ONCE(sysctl_max_skb_frags); + struct page *pages[8], **ppages = pages; + ssize_t spliced = 0, ret = 0; + unsigned int i; + + while (iter->count > 0) { + ssize_t space, nr; + size_t off, len; + + ret = -EMSGSIZE; + space = frag_limit - skb_shinfo(skb)->nr_frags; + if (space < 0) + break; + + /* We might be able to coalesce without increasing nr_frags */ + nr = clamp_t(size_t, space, 1, ARRAY_SIZE(pages)); + + len = iov_iter_extract_pages(iter, &ppages, maxsize, nr, 0, &off); + if (len <= 0) { + ret = len ?: -EIO; + break; + } + + i = 0; + do { + struct page *page = pages[i++]; + size_t part = min_t(size_t, PAGE_SIZE - off, len); + + ret = -EIO; + if (WARN_ON_ONCE(!sendpage_ok(page))) + goto out; + + ret = skb_append_pagefrags(skb, page, off, part, + frag_limit); + if (ret < 0) { + iov_iter_revert(iter, len); + goto out; + } + + if (skb->ip_summed == CHECKSUM_NONE) + skb_splice_csum_page(skb, page, off, part); + + off = 0; + spliced += part; + maxsize -= part; + len -= part; + } while (len > 0); + + if (maxsize <= 0) + break; + } + +out: + skb_len_add(skb, spliced); + return spliced ?: ret; +} +EXPORT_SYMBOL(skb_splice_from_iter); From patchwork Mon May 22 12:11:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC9A5C77B73 for ; Mon, 22 May 2023 12:11:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73C7A900006; Mon, 22 May 2023 08:11:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C602900002; Mon, 22 May 2023 08:11:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53EFB900006; Mon, 22 May 2023 08:11:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 45B08900002 for ; Mon, 22 May 2023 08:11:50 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DD8B280332 for ; Mon, 22 May 2023 12:11:49 +0000 (UTC) X-FDA: 80817777138.03.CEFCD5B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 2378640010 for ; Mon, 22 May 2023 12:11:47 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UYi0EFZE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757508; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CbEUQCnChP/PzTidzzQWL6iO0yyE2e0IQp2l4Z4U9vY=; b=I7qJe/iGn08EED/edATENt4K/WpdGfWgb/V74qDBelNXf/eE6wCg8Mi/Jz3H0T2bT+rDd3 zQMzqgSbOffBUNSb5MSUnsSGhppzxW2NggdGWBWjbR4CYg3lk357dEfGPYK1Mg1xl4LVRn 1IEoAY5sKHZ4Q1OLa3wNI8/rFjsxtWk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UYi0EFZE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757508; a=rsa-sha256; cv=none; b=RoScS+D4GIFZpc1Bg8YjLKcwVBKVtBBFHL5sh2wpEnneL/31mFBTlpzsjjzQiWn9Q3Jfio 20Z4pyZz+RVFawccIjL73eahbuQ7WgnB9kEALIPBTGqciM4S5QQMNE15HaJhImEBBZEQoC FZVbbVv1/VuvL9R+0y7nDpyzGGrTzn4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CbEUQCnChP/PzTidzzQWL6iO0yyE2e0IQp2l4Z4U9vY=; b=UYi0EFZE4QxwXjQDBqAuzaGiN+HWO0+we82QVPIXmVmRdrxCx1Lz9LGT+UIdUTqMr26iRQ xUXIOIEA8GhLAlwmCAH0YxV8NgCAajPtMQo8Y6dcMWhhsf7jnF7FubOqUV2p163WAUyYJI Ispigw0oSPdP0DXhIcSXFPOaruDkkoc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-316-9JSwXOCGNNaJscCJXa4Eqg-1; Mon, 22 May 2023 08:11:44 -0400 X-MC-Unique: 9JSwXOCGNNaJscCJXa4Eqg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5FC812A5956E; Mon, 22 May 2023 12:11:43 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id C96347B7C; Mon, 22 May 2023 12:11:40 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 04/16] tcp: Support MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:13 +0100 Message-Id: <20230522121125.2595254-5-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 2378640010 X-Stat-Signature: 8zzb7rturyye5feuzuuewihwm7s5n17b X-Rspam-User: X-HE-Tag: 1684757507-837718 X-HE-Meta: U2FsdGVkX1+Gqb8ukAH5WixBNYtSATmsybSELD8kqXFZNTKBT43X/tbETKc0yvqELQifudvSyvCieLLP+I0QEQSlB8oDGO1nvGlPonFTXOUJGI4zgcGd3Jo7ULc6/0GeaKCQ/e25UvIwXq2iL6XwsG8JBJy7JNfaQrrHqftCgAyjY4eMZsAPn4M7Xehc9bwYZcr3MDXQsYPU+E6MI23pOT0GkjkLeKE5acgxkD5F+ebagViUgdw9Gf2q5JzY8csUVGPqlN7VC5jkj3pvM+tTlfvBX3Foo8cY4VPTyJQxkmf9h//0v2qqpzgjmcw8H8Hx5ZxLHddFznYoaAxhru+BnsxexOK3WYzRNp2YWsLqFvao+UAKHwPT2wQbqzRIAibp1X3R30kBcQ7zYurTirHHN26KdlVJ5IvSWRPq3pS1U6/Y+368srW0PXeLErrrdQlEJV5+BPfAxEr0hz2Gnwop7WUN4orxL/al9yWvRjLzOLP7Eo8hp4h5tmuw9wLdLmuENXgczH0czsqrv5zrYzwo/wH0wFd8gflGV7b1dPqAyyqCZ4pAwoNH0ZhJ+m7lPKyqpRA65Zn03HLjvLMVNDUEB6s1U16exjLkR+Tgv9YB1kUQuaK7DQi5cnQkROz7OMKMq869X4iR34HZRZtSNwo7zAQ2GLw6ubOhX/WC4lD5C+YI6edwguv2qEF4k8oa6EQKHggSIIqhjzhUmSZBWrfDn44elkbmTk+l5sg0YTqKe+TyG+vpADlvSPKnBuYABI3ijWeKFO6De+Ngvw7y8zmFKyoXJ2nnlsuKAKJ5K/xYWkOZA5rdEYRNvciYIcgdtty/5vzasAlcXIzE052rCn457dq1HSzcj+h/C+Gz8SfcjirmXUFX/4PHQ/j/2h3pa9DcIRmL+RPXQ4WZ7rvcmFzLbneBj6t4c4PeTjkojnioDtLogZeNqTQo1eSOJ7Mw0KK6k5dDYPnHi5b74tuQxnB V88K2sGE /P1wzX9gCKEBw318ncUOKPWM6sJQeDD7mX5mEH20np8tkLrnStYEHcpgxZTCrfgNijPVG2f0WJ4Ts88kwR0mQXmpOsqFzeFWYUn09DMEot6NUk7TpeR6VV0ChUZZK6GY1MQk4d0Ful8n/rmC6cB4QcE4vWkf8Vy5JedUIPzgh7OI+CWk7n2+M5El0B3YCDQAAJDg3uDejZ8CpbsbruRJE2Bl/YMBHPO2Z0E68JsMnXVQbowaraf4zQXzkvCZGXi7budpvhsbfLejL1+fs/LOI+UAHgk2FYQDNhrURDx0r3ETUYSGpNV1oQvc5j+btf39/RuOQUvosku78JMluiXPop+GAdXYcq6Qtpjlf8vmkgRuf+L+keFqjmKr9nYdLOOxLPApvezVM7bv3gLXdhPUEHBTcbHNA5pt5ZqDAuxlX2F96TnLncZ+ggjzB61M9mzlzXJ/ppcjMYuxhBBiLePzMnaUYhTVG4hRnvO3Z6HQFmFtQCckGOjoM0ZBsVA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TCP's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced or copied (if it cannot be spliced) from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #9) - Fix a merge conflict with commit eea96a3e2c909. ver #7) - Missed a "zc = 1" in tcp_sendmsg_locked(). ver #6) - Set zc to 0/MSG_ZEROCOPY/MSG_SPLICE_PAGES rather than 0/1/2. - Use common helper. net/ipv4/tcp.c | 43 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 3d18e295bb2f..2d61150d01f1 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1223,7 +1223,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) int flags, err, copied = 0; int mss_now = 0, size_goal, copied_syn = 0; int process_backlog = 0; - bool zc = false; + int zc = 0; long timeo; flags = msg->msg_flags; @@ -1231,7 +1231,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if ((flags & MSG_ZEROCOPY) && size) { if (msg->msg_ubuf) { uarg = msg->msg_ubuf; - zc = sk->sk_route_caps & NETIF_F_SG; + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; } else if (sock_flag(sk, SOCK_ZEROCOPY)) { skb = tcp_write_queue_tail(sk); uarg = msg_zerocopy_realloc(sk, size, skb_zcopy(skb)); @@ -1239,10 +1240,14 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) err = -ENOBUFS; goto out_err; } - zc = sk->sk_route_caps & NETIF_F_SG; - if (!zc) + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; + else uarg_to_msgzc(uarg)->zerocopy = 0; } + } else if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES) && size) { + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_SPLICE_PAGES; } if (unlikely(flags & MSG_FASTOPEN || inet_sk(sk)->defer_connect) && @@ -1305,7 +1310,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) goto do_error; while (msg_data_left(msg)) { - int copy = 0; + ssize_t copy = 0; skb = tcp_write_queue_tail(sk); if (skb) @@ -1346,7 +1351,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (copy > msg_data_left(msg)) copy = msg_data_left(msg); - if (!zc) { + if (zc == 0) { bool merge = true; int i = skb_shinfo(skb)->nr_frags; struct page_frag *pfrag = sk_page_frag(sk); @@ -1391,7 +1396,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) page_ref_inc(pfrag->page); } pfrag->offset += copy; - } else { + } else if (zc == MSG_ZEROCOPY) { /* First append to a fragless skb builds initial * pure zerocopy skb */ @@ -1412,6 +1417,30 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (err < 0) goto do_error; copy = err; + } else if (zc == MSG_SPLICE_PAGES) { + /* Splice in data if we can; copy if we can't. */ + if (tcp_downgrade_zcopy_pure(sk, skb)) + goto wait_for_space; + copy = tcp_wmem_schedule(sk, copy); + if (!copy) + goto wait_for_space; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) { + if (err == -EMSGSIZE) { + tcp_mark_push(tp, skb); + goto new_segment; + } + goto do_error; + } + copy = err; + + if (!(flags & MSG_NO_SHARED_FRAGS)) + skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; + + sk_wmem_queued_add(sk, copy); + sk_mem_charge(sk, copy); } if (!copied) From patchwork Mon May 22 12:11:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05606C7EE2A for ; Mon, 22 May 2023 12:11:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93869900008; Mon, 22 May 2023 08:11:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89A18900002; Mon, 22 May 2023 08:11:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7152E900008; Mon, 22 May 2023 08:11:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5C27E900002 for ; Mon, 22 May 2023 08:11:54 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 383A3C022C for ; Mon, 22 May 2023 12:11:54 +0000 (UTC) X-FDA: 80817777348.12.6C15BB7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 61FE9180002 for ; Mon, 22 May 2023 12:11:52 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BdfMOBfH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757512; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=79TjdqGEZWzMR9284kh0SsbKnhgq/mMmLDGgzzPhuR4=; b=SAr93o/PyWa37GoupUnfPhz9zut+DYBSCsvNopomluv3rgihrqY5mQIOdM3wUlCgt4sN/C MQPZD6S+1k9iytHGchJqd77Urfcs2ag5/F/C/E1s0G3+uLhC2AUx/hyk0K6ax8FbwOIBYl mvM8Gu8ppQYoeUm7oaENoK1MIpZxHq4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BdfMOBfH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757512; a=rsa-sha256; cv=none; b=ke9EOezildbRwGm1IgiGY9ovJXDDPAVsTEgjRhqs3UJzqGD9/N0wXKIuM0jx8wgzVdAnBC JKhnJDLkw5RaomVFHwkyFpa8HHQY+s0H7HLPugIftr1a20sWRuUBqFh6q3CdHbw6ygDhI1 ZOusoEfCEPRq/zQNVe0fChp3X6ypl/g= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=79TjdqGEZWzMR9284kh0SsbKnhgq/mMmLDGgzzPhuR4=; b=BdfMOBfH12UbPi9wnkVGopOGB+VUo2KZ4BBQ+kcPZc6pWRFvU2aGAMnP5sV20wNirSqowE F8o2oZmF7hlEn+u24/jPhqPu1PYhDiYW90JJyPMae+ofjQVTS4RjN6XjyNNpa272YwCaG5 j4Dv5GnLhk6EJ2uIkVtDlLlV53ZSf4Y= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-s8VTVl0_ND6DLvjr5Pd7CA-1; Mon, 22 May 2023 08:11:48 -0400 X-MC-Unique: s8VTVl0_ND6DLvjr5Pd7CA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 87AEA3800E87; Mon, 22 May 2023 12:11:46 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 14AA7492B0A; Mon, 22 May 2023 12:11:43 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 05/16] tcp: Convert do_tcp_sendpages() to use MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:14 +0100 Message-Id: <20230522121125.2595254-6-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Queue-Id: 61FE9180002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bgqpq1o1btcx7jmda4szy3a93czdh3bt X-HE-Tag: 1684757512-712837 X-HE-Meta: U2FsdGVkX18lDU94rrcJ5tYAb557OEtgmGwoeilFwfHmfekHXOfcAbfzgHh5p1alTOfepgjqsG0mj8B4QVj9FlDoSocFultVD2lCnoMXXJne1N64dAuFp9IRkEUbWheZGzU2HDtAtjU2m2We1SbS3astsc4nMIuRX2/qzWfs6chTfXMNF8rEZGSLfFaq2bFn/sN4wHSE4X+ENi1k4IWHCPoCQ9rRLuv5MgpLPzzEWZKSOk+1bBs2WeGFZb3cNGIkDrm772R+HS/43FPZyDz3x5B0m4cbPSrCz1AmdMq/2qKvgSxpO3xLv5TKKYUt9f1STbCCaOhWItz6EtNSTv+OtNUM7oukTMdYAyQ4l9L3TFkhYMDRCanUOf46LqEQfVpGbvV/IvDnYRMIvh9sAcoX5UT8HrHI+rEG04dK4t4FnCEyVpPe9rLYFbuxWvhIvAaRFWqRO3n/VAlPXoSyYm+Pzvm72CeDsMaQGGhMOa6rBiAMuJDoO5Vijk9uisXLSM0B9V8Ma9LTyOnlBcFvZ4K3K1IYTFb98KhnjdH5hZ/N3G6d104480tpaOsv4ZTz4YvQQ2RyH0BI234dJSrXZPHGxf3bJFVyDCE31kn+JWKQjxgwMbbRh1uHy0PBFZ/cko0NBoYH1VKJW+8JKhEKx2fuiwD31N19Ae8LrZCM5GGveA7gFJkBIZ7H80PSR/fcXIs4rlJmcYbt1pu/Df/zqJVBlfX7np+MsL2AGhXUk9e9MZzVgPAbAh+bFrpYWk/gRDB/ecK1eiMfGASJxW+IfCgKkt10C3BrD+gE6hzWqrc4UL1NJ86Ep8oeIXp3yPrO0gurfdo5ATfKFYYxAR/zpaMWSCzQvCz9J6lhxoGjciB9z6YffeTyxI1xKZrm4HlAjlsWJmDFzWNn03Bf1NjUk2bF9rQDYQADftZ3UiTtfHPYE38vVzo5tDI1RSNjRJgz1s4IElkE2w50FATtvUTjcG6 J076IHZa slL+wzYkG+LZ9j71PlW82RQ/K1asmMfGsxroiI4mXFdxl/3V9B89oKUevZDG4Cx4fxMJPsl09v4afaY/1YpIxC0GOjABUhKixk2sWDMZ6B8Kwa0RIMZf0KAHL1btMRMxlgI3TgPPvDfKAmgYqHFAgu9lg4a3aeX5BI7fQIhqJXkDvv8z7yXgizVA/nnFeohE6b6eO7VsW5fpUX+KTLHBRt3ffrvhcWmwURPCbEsXh+jw0RKzUsLpXTREsP0nVwqOJsQtwvNrmljc9YHUKf2WAodtBkKZo1TmfZI5xIZWbbeuxSyOkNjU1N3LwtZVx0+B+xXPf8FrGtR1XO5x8uNiFNo9zoQ1uT/5tCHA5JczwasFD+d7hHFT/81qmVFCl6Wm9Nhr2JY4uFNIHKO9ubAZzmb6xifQrrpOiAv5aR524YmEX7LpcBfmMRPA7bzJ9B300WwrVioTfakFdDWiIeRMDE+LT2L75yxoc6zs9kzmaB1sYMKn2Ow16RzWdYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert do_tcp_sendpages() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. do_tcp_sendpages() can then be inlined in subsequent patches into its callers. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/ipv4/tcp.c | 158 +++---------------------------------------------- 1 file changed, 7 insertions(+), 151 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 2d61150d01f1..f3a0c02678e0 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,163 +974,19 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags, - struct page *page, int offset, size_t *size) -{ - struct sk_buff *skb = tcp_write_queue_tail(sk); - struct tcp_sock *tp = tcp_sk(sk); - bool can_coalesce; - int copy, i; - - if (!skb || (copy = size_goal - skb->len) <= 0 || - !tcp_skb_can_collapse_to(skb)) { -new_segment: - if (!sk_stream_memory_free(sk)) - return NULL; - - skb = tcp_stream_alloc_skb(sk, 0, sk->sk_allocation, - tcp_rtx_and_write_queues_empty(sk)); - if (!skb) - return NULL; - -#ifdef CONFIG_TLS_DEVICE - skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED); -#endif - tcp_skb_entail(sk, skb); - copy = size_goal; - } - - if (copy > *size) - copy = *size; - - i = skb_shinfo(skb)->nr_frags; - can_coalesce = skb_can_coalesce(skb, i, page, offset); - if (!can_coalesce && i >= READ_ONCE(sysctl_max_skb_frags)) { - tcp_mark_push(tp, skb); - goto new_segment; - } - if (tcp_downgrade_zcopy_pure(sk, skb)) - return NULL; - - copy = tcp_wmem_schedule(sk, copy); - if (!copy) - return NULL; - - if (can_coalesce) { - skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); - } else { - get_page(page); - skb_fill_page_desc_noacc(skb, i, page, offset, copy); - } - - if (!(flags & MSG_NO_SHARED_FRAGS)) - skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; - - skb->len += copy; - skb->data_len += copy; - skb->truesize += copy; - sk_wmem_queued_add(sk, copy); - sk_mem_charge(sk, copy); - WRITE_ONCE(tp->write_seq, tp->write_seq + copy); - TCP_SKB_CB(skb)->end_seq += copy; - tcp_skb_pcount_set(skb, 0); - - *size = copy; - return skb; -} - ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tcp_sock *tp = tcp_sk(sk); - int mss_now, size_goal; - int err; - ssize_t copied; - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - - if (IS_ENABLED(CONFIG_DEBUG_VM) && - WARN_ONCE(!sendpage_ok(page), - "page must not be a Slab one and have page_count > 0")) - return -EINVAL; - - /* Wait for a connection to finish. One exception is TCP Fast Open - * (passive side) where data is allowed to be sent before a connection - * is fully established. - */ - if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) && - !tcp_passive_fastopen(sk)) { - err = sk_stream_wait_connect(sk, &timeo); - if (err != 0) - goto out_err; - } + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); - mss_now = tcp_send_mss(sk, &size_goal, flags); - copied = 0; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - err = -EPIPE; - if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) - goto out_err; - - while (size > 0) { - struct sk_buff *skb; - size_t copy = size; - - skb = tcp_build_frag(sk, size_goal, flags, page, offset, ©); - if (!skb) - goto wait_for_space; - - if (!copied) - TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; - - copied += copy; - offset += copy; - size -= copy; - if (!size) - goto out; - - if (skb->len < size_goal || (flags & MSG_OOB)) - continue; - - if (forced_push(tp)) { - tcp_mark_push(tp, skb); - __tcp_push_pending_frames(sk, mss_now, TCP_NAGLE_PUSH); - } else if (skb == tcp_send_head(sk)) - tcp_push_one(sk, mss_now); - continue; - -wait_for_space: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); - tcp_push(sk, flags & ~MSG_MORE, mss_now, - TCP_NAGLE_PUSH, size_goal); - - err = sk_stream_wait_memory(sk, &timeo); - if (err != 0) - goto do_error; - - mss_now = tcp_send_mss(sk, &size_goal, flags); - } - -out: - if (copied) { - tcp_tx_timestamp(sk, sk->sk_tsflags); - if (!(flags & MSG_SENDPAGE_NOTLAST)) - tcp_push(sk, flags, mss_now, tp->nonagle, size_goal); - } - return copied; - -do_error: - tcp_remove_empty_skb(sk); - if (copied) - goto out; -out_err: - /* make sure we wake any epoll edge trigger waiter */ - if (unlikely(tcp_rtx_and_write_queues_empty(sk) && err == -EAGAIN)) { - sk->sk_write_space(sk); - tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); - } - return sk_stream_error(sk, flags, err); + return tcp_sendmsg_locked(sk, &msg, size); } EXPORT_SYMBOL_GPL(do_tcp_sendpages); From patchwork Mon May 22 12:11:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE5C2C77B75 for ; Mon, 22 May 2023 12:11:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9101A900003; Mon, 22 May 2023 08:11:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87265900002; Mon, 22 May 2023 08:11:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ED9B900003; Mon, 22 May 2023 08:11:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 60B71900002 for ; Mon, 22 May 2023 08:11:59 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3B6DDA0378 for ; Mon, 22 May 2023 12:11:59 +0000 (UTC) X-FDA: 80817777558.17.58FFDD7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 6CE421A0029 for ; Mon, 22 May 2023 12:11:57 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=e8SDArFG; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=pIA3VEvAX4gkF3XMJax0YGFASnw+f/Pvm9Xm+B2OgrY9qslheTeFLRToFyYMbucfU+bzHO 0aROwNVcWI3FQQd7xgS5PpbbmqnGORuwGQwGwrSZVznjprtolmPZg4KTs/JwypMfbNMHml KvVxXgfH2RHPRdJsZruWZBpHu5asiV0= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=e8SDArFG; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757517; a=rsa-sha256; cv=none; b=rC7Ggvxc10HVCT0ccQ0FGtgLuYjA7paRstefYJrk/bl3ePlGDuh2qSFOzmTwXj8YhDg2yk jfMr+Phh47mx218tUC8jcyVUN6835v3ebqpNKzeX7pazDvheDyFE65CLySfndtc9D8OZd7 Vnoc0T+Y3wELPFRTsQo1FiasHOLS3aI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=e8SDArFGbawiESCCnOF3nppfyh+Ho2U5yL0I1J1mHKCnuslG4fsujcmxTOA+whwCeu6Wrp DzzYiq5w00N5dOzqaVuaZAiG5frNOcU06XR4AxeOuin7WR3PBvzn2sD2t+3BUKB70STk0c ChZUU4R3UbENqzL40KuGZeyYEphSOmE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-447-fpogOg9rPfSKKoQ1z-hkZA-1; Mon, 22 May 2023 08:11:51 -0400 X-MC-Unique: fpogOg9rPfSKKoQ1z-hkZA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 298F3802355; Mon, 22 May 2023 12:11:50 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5AD417C52; Mon, 22 May 2023 12:11:47 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Fastabend , Jakub Sitnicki , bpf@vger.kernel.org Subject: [PATCH net-next v10 06/16] tcp_bpf: Inline do_tcp_sendpages as it's now a wrapper around tcp_sendmsg Date: Mon, 22 May 2023 13:11:15 +0100 Message-Id: <20230522121125.2595254-7-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 6CE421A0029 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: sp31zo16pc3p7makh7gpr6qg3kjui9ss X-HE-Tag: 1684757517-469296 X-HE-Meta: U2FsdGVkX1//LU9c4n1oa5NOHFqY95+BAmI2m91+Iq5EjTprHrl2UNu/rXeXAM9lorCN5QFQUbxlB95KMoCM77oVZejZxvHtypJyxr3yGbtL+RP5b+O1pstzne/j0aa54L0bRZhegfAWkJCv8rmTkGin8e7fJrH5ZqIPpvszkwWkQ/TLPZ9LAZNuZwpyCK3vawFu0oFgdvYZ1NI/WE6NVKS3bOmiO647w+z0ILBniTMxB+ZFVt/TS6cuBbXjUO1X6sMdrQo8NnjOBGw9XTmvTf3rlf5JzCz/cN0vNthkQNwEKGw230GWhaHWn9dKEuGfplBwlaa8XblzTe+2nzyckpsRvLEGR8g1uXf3Pt31ZxSGetbmJcmbFadoDmNNjuEBYqRoAcB2FnBQacaHvmc0/18bi0xvJ6c7ftCvDQMo3aZi28tyDl//Om+8+eM8eW8RxE0xFdwtu/9LtMnHmdVnvSJhPxsu1GtxqCeKYHMAxf8yNprsO0gK5FtMMjfKEDdzcAcRYnme/eUeR2KnFQc+AHExAReSTL3D1UzOIVQsRltffVdjHvwvBz8nQ9dUfsva5hy9VWN0nB6VnafM5EoQa2soxVnu5EnTKit3ozDpSEUdxOevdV8xJNF5NRPp6z5oVBrQnVYQHU6TaWyYNejM7rp4KnwLlItjE83qdCdALzLFIQHaBDtexKEMIiMs4a+2kHgbICxu2RbZfQQrmyZbwRIhQk9bgK+6dco6rtTGDM/oAJ66X0WRTC2ZzxVIfYL/7x2Guy3HiZa0msgtDYAW9RpQE4IcM4bi6voMqo6FkYQjh7NbWsASNc0tGbmFmR7Iy0NtNX8DGGzde2lB9POaurSMaFIxw6Fj06LcUgWJ2yCZrlMxzFa6t65vseQfTFfrEoxELGtGN7oGq/Um1hkjY59A6pTlj1duM9ofXAMNeuHxCNfUS1irRFjAG/5/ecdvbXskN2duRISg/uyvoKh eF1CcAsd OvCJ7KOkq6J5wOJVSyPneEWS26tR3Td7JWyvcXCfIMJmYIBA1oIeJ6er/GAhmLme/WiuhUkuXPyO4oHVx4diMur3M9tU7n6emKJ3Q2p5fX4vccWxuDn1lQVUpR/66ab28Us0hby4L8bJxAaIjBeD6viXb7m7jPhND6X875udvfYTWfLeeT4O2KIq8ziwl0wqbAlGGocNW1R/TS7VX0/BAu9+mHXqvgQ9xdGrBm0otmbu8LHNK2G5wA4olH1FDggBnn3xH3565SH76LHWNm6P/blC7+fSvHP858lPC+bUNhg5R406Zr9qMLcH7xy60SjXGlNyC5B5LOpfofn/f/sRTavtKl0v+jjSMXFRq/WC8LYo7Z+vvoiE7v2Bd+LI905ZndnGpRB1Px06efcpjv8CXAQHmeZNOQhoW5XSU2HQJs2FVHC3kgWIIggOe33IFHGnrQ0J9SayLbXwzOVyTfJrmsNz7/gWK4w/PsmG8/rCRIV+5LMS6y2ybvvKpMrb4soyRo/LKJhRnF9TFdzKmNvN0Du05VEq2wZIWgH16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: John Fastabend cc: Jakub Sitnicki cc: "David S. Miller" cc: Eric Dumazet cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org --- net/ipv4/tcp_bpf.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 2e9547467edb..0291d15acd19 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -72,11 +72,13 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, { bool apply = apply_bytes; struct scatterlist *sge; + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct page *page; int size, ret = 0; u32 off; while (1) { + struct bio_vec bvec; bool has_tx_ulp; sge = sk_msg_elem(msg, msg->sg.start); @@ -88,16 +90,18 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, tcp_rate_check_app_limited(sk); retry: has_tx_ulp = tls_sw_has_ctx_tx(sk); - if (has_tx_ulp) { - flags |= MSG_SENDPAGE_NOPOLICY; - ret = kernel_sendpage_locked(sk, - page, off, size, flags); - } else { - ret = do_tcp_sendpages(sk, page, off, size, flags); - } + if (has_tx_ulp) + msghdr.msg_flags |= MSG_SENDPAGE_NOPOLICY; + if (flags & MSG_SENDPAGE_NOTLAST) + msghdr.msg_flags |= MSG_MORE; + + bvec_set_page(&bvec, page, size, off); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret <= 0) return ret; + if (apply) apply_bytes -= ret; msg->sg.size -= ret; @@ -404,7 +408,7 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) long timeo; int flags; - /* Don't let internal do_tcp_sendpages() flags through */ + /* Don't let internal sendpage flags through */ flags = (msg->msg_flags & ~MSG_SENDPAGE_DECRYPTED); flags |= MSG_NO_SHARED_FRAGS; From patchwork Mon May 22 12:11:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0099C77B73 for ; Mon, 22 May 2023 12:12:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF5E2900002; Mon, 22 May 2023 08:11:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7B86900009; Mon, 22 May 2023 08:11:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA78F900002; Mon, 22 May 2023 08:11:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9E559900009 for ; Mon, 22 May 2023 08:11:59 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6B60A1C6D33 for ; Mon, 22 May 2023 12:11:59 +0000 (UTC) X-FDA: 80817777558.14.EF96A78 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id A2A3F18000E for ; Mon, 22 May 2023 12:11:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fMqi8Psc; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=SEoCcRKS0d29iF/Cuci+KAtgwvSPD78m0DLC8QFAkA3Il8bJ9Ql0s9sBmpsxZYV7Ym7UF4 uBcip/yGXlPud/thnkBsKGpN1iD/GoRQaIiI336xLhjy31G1klUin9k+/R88Md/eHzfjKw aWyQiYwkEhsIkhZm5Q7ScgVrJzY9U7k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757517; a=rsa-sha256; cv=none; b=HICpr38SygISIi5DUtGlP/Z96cM4aaTWx0LU+9eJJJNNr1/BlQmdS0R4L4kcexsQRWax/M IbfTTn620b0KfWVGzQ1T6AUSm/xBCvSVLKKVNE9myRFuvv+MhdTbtgqoB5+Kc0y3UpUquq L4RdtSv2yMp9fIM1uFMG0Ed4IveANms= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fMqi8Psc; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=fMqi8PscSIe9Sgg1KlqW2tWlB/n3zSZOqpWYU+aRGY94roLVCIFwPxp+MGXQ7JVPDqrwrX kBNJIgSST+EapzYj0iMpBom7pDIf51kZUKGSvaiPKSb0psBcx3/EHWePPme8q68yO5LFfw bIiqRJd7eTjv1e1OWT1ci0u+r+yHKww= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-329-z_k6b1dFNWyCPfMJlHKjGw-1; Mon, 22 May 2023 08:11:55 -0400 X-MC-Unique: z_k6b1dFNWyCPfMJlHKjGw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 096DF811E8F; Mon, 22 May 2023 12:11:54 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id D9261C54F80; Mon, 22 May 2023 12:11:50 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steffen Klassert , Herbert Xu Subject: [PATCH net-next v10 07/16] espintcp: Inline do_tcp_sendpages() Date: Mon, 22 May 2023 13:11:16 +0100 Message-Id: <20230522121125.2595254-8-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Stat-Signature: am9ffdxok3zh8cw7dr4tfeiem6jmcrwx X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A2A3F18000E X-Rspam-User: X-HE-Tag: 1684757517-936317 X-HE-Meta: U2FsdGVkX19VNuQdXeF1cfvIktj3yRNiRkMJyUPqcznN9pNW0WQ52VPK/vBHWhGxYgvsL2UfTdE8YYjySNmv/xDq2d4OuQE4uvfjZoh+qYCg+APIxXFPdjn9Op7h7LYcpsvLUpyIaiE1bk0T+skh04cxB3PFXVW6NKVjN78u7SmRz3mCRyKcco4G8nioakzhciOzz0XkMYqXN6iAJQZ9zzDr0SWUnRVMjQkdax+QSdx8rzasspypW12OOFOtFWmaQWDUpUEg7yr2MiN4afHQbdGwLbe+wwc5cJcU2EH1fqXYedKWdZejrKyjaxGC/1ETgI/LULJ7mpzMpAwjijpFFa4VnxfKHvxXxrPeLUMh6xfDC+Wu02bmXL5HzeFL7SDCYq+nONQtM+/3L6sgFeV9nbV0raaXkYLPw2HkndpWMxX0sWlQ5MYHdTbUt8sJpNRRShJYj94lqQwzuIj5ui/PLChK+JRH3gJXgaVEtN38VjoXdOck6vKROeWLETLjRIAkqQfyuxGEXxmGUWX2+dHra1O6FGKUtzq5cX93QFNEu9SGtF8yKJR3nX9s8ORiJC4N58GK/eilGcndECq37wFj7WJzw+4w/6IdWxWhYPEm4MHpW4nzTvxREtjmWn38MLxqh4rbVHtThRGeLb5KDgccNP6d5ewV07N+h+4dbvkGL5wvjbbu1FtJNfrfMlhuUsJh6vpF7OXf8CrG5PG4LBYz/MoYgXw6JuSvjYDwmLpx5CuxHY3kqWvgpZtWEEjWYqlznXM0mKYyO4/v8LUZVJnJZixyyqeeuCV3fxI6J/stE81RNnreoTdUP9NaOdWSdvEG5IMmFWREVZB6gZByf+X9njMSxOL/N3PdTAEwVLytu1RH3Ui7p8+5Cx5HZrP+KWRmMhwJVOoDlYtb+BdoxYDSjqw9IDFGVDN/xpQJZovs35AdcXtiennIxV3xP6UzjPBBAeGds2Sw5Cep/6pyLHj B6VH2w+2 CyaQvD6ylw3ENxOwfcNkDOLRiu4eSbB2lIg9TG6Bi6Kr3Xh0T045it6RzrHZp64wVde+ANaYPjzKAVZ6j3PrrexST15I7L52pixE5UeGLG3VdakdgxmStsTq7Rx9RNr09iThQFXs9k7xtj6gvVnm5HED2o1JJa28Jwj1xVtU+fbGPspewIOsM9e7jM710KwGL5v3Tiax0Co024sjdR2j3a03ZCA+3pMnCxAia2z9EDIcjsx09baL7O5Xw0naVVkyURuI19XcB3r18FrkROinYsPL3LQByvh0gbPz7wamLQWGmbdYULfvOiPU2pcReGfhWIj3eEjJyAEcIdRwHpNew6vjLEPp2dzJOuQEYMdxabw7W37J8wtTXexQGJr8ttRi51Xrczq4GUDXSjcqSwKif8Gl2TowjCeHEtCw0e/KVGxY3tBM/g70XS2cKh7AnE34qOcQMi3tL49+iQwXQp/poCGCoGaRQEUwv2myxCOHASScX78FysNugdEesvmV+fhpDDlix79TAGJhEwL+A4MH7IbekWptT+J4SoQmi2EUK7CkO5iAX6MlZeBURoel+0dJ9999Vhw1r2Tz8/8VJeMPvtZEfFyE6L4u3FsrwmSOTubawDTI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Steffen Klassert cc: Herbert Xu cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/xfrm/espintcp.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index 872b80188e83..3504925babdb 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -205,14 +205,16 @@ static int espintcp_sendskb_locked(struct sock *sk, struct espintcp_msg *emsg, static int espintcp_sendskmsg_locked(struct sock *sk, struct espintcp_msg *emsg, int flags) { + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct sk_msg *skmsg = &emsg->skmsg; struct scatterlist *sg; int done = 0; int ret; - flags |= MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags |= MSG_SENDPAGE_NOTLAST; sg = &skmsg->sg.data[skmsg->sg.start]; do { + struct bio_vec bvec; size_t size = sg->length - emsg->offset; int offset = sg->offset + emsg->offset; struct page *p; @@ -220,11 +222,13 @@ static int espintcp_sendskmsg_locked(struct sock *sk, emsg->offset = 0; if (sg_is_last(sg)) - flags &= ~MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags &= ~MSG_SENDPAGE_NOTLAST; p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret < 0) { emsg->offset = offset - sg->offset; skmsg->sg.start += done; From patchwork Mon May 22 12:11:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D215CC77B73 for ; Mon, 22 May 2023 12:12:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7146D90000B; Mon, 22 May 2023 08:12:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6768F90000A; Mon, 22 May 2023 08:12:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F1DC90000B; Mon, 22 May 2023 08:12:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 375E2900009 for ; Mon, 22 May 2023 08:12:09 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 0EE6B1A02C8 for ; Mon, 22 May 2023 12:12:09 +0000 (UTC) X-FDA: 80817777978.02.59CC9F1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 5AFF61A001D for ; Mon, 22 May 2023 12:12:07 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fYwCoS5S; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757527; a=rsa-sha256; cv=none; b=5Vxuh/1huIeKIQK+tzXCyGHYxXuAZWF1obE7m1NT/RTR+Phc4afZ+vdKx79/FRWBJGtK6k ZpHIGSwRoaiehoGVDyCNilq+FEBmPrxNsE5jU8/bNGRk/jgtJLGchkII6XA1lvsPW8BQ93 eYnml/P9BfPrZ1Jg0fxwoTDunBsPIls= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=fYwCoS5S; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757527; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=LJILyNHUQ3teznR7XRGiVmIDClbmtRSwOz+YrEdh4yJCP2/0k/LWRvQm9xwnWyr/npwL92 naIT2NUOEQHO1LYata2JuFjImcupm0EP0ouxDtiGzZI/G8sEjfoiT/KnkRjMuy5CAuRhjJ jNuxmaTpIOatxkctFIkudRaNxFyjtmY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=fYwCoS5S/B1GcHZPQNYllbSWdtnk4npnFEHOyObZY7Ei96r/6KMrsIj7j1EyPJrSeApGH/ JreNsLFkXtMZdD7jsSDX1tsYfBaBmHSxOUYsk1+uRS/ora1LaQCruILoOdXXLzFVI7e238 AuC+PpDbH50jWR2gpwUeqZjp4NqExjE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-587-GZXCZZECNuaQheY_clprmg-1; Mon, 22 May 2023 08:12:00 -0400 X-MC-Unique: GZXCZZECNuaQheY_clprmg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 73454802A55; Mon, 22 May 2023 12:11:59 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id B74D740E6A4E; Mon, 22 May 2023 12:11:56 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Boris Pismenny , John Fastabend Subject: [PATCH net-next v10 08/16] tls: Inline do_tcp_sendpages() Date: Mon, 22 May 2023 13:11:17 +0100 Message-Id: <20230522121125.2595254-9-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 5AFF61A001D X-Stat-Signature: emafwripx9tkqpeczymjwkwyicp3m4bs X-Rspam-User: X-HE-Tag: 1684757527-183082 X-HE-Meta: U2FsdGVkX1/y+INfyErXcN004ud19tWFpyamno/EwYITc5xWGOVGthdke5Ive8Nwn05gmZYTTRAaMRSVGynYRgEVDalegnqpBo+9R4nlN7vP5MTOlaJDmIyYTGmAHnZWlxaYJjf7ZSpJZED2wUw37KI6p5XjrmddP91xTkQ5mPlwGF+PiBEq8yB4oaDOE5whDvfkEblpFPt1YaTIVGDZw6gLd0qw8joxkFp6AdpUJ/7TDC6/MwJFQ5PeR567HKQzLxDFO9dQ7WyiO2vjVMZFH+m+84ETqNzd4LPObdpBnlUt9/gtrfGko2Blrpu/k3v+qKdlwq5VksqNL1CU6oAclOAKIg/qb+Lgpx9REyEY7wpOirV4ANPgWm9E2jadZrvXkYPDBESHkmLKs5f67HxT6Dh2Bt2AMMacn5OhzFgNGBUnSBKqef/YrpNm3xp7WEtyBfmVmE6rB2QQZKQVnEDAKzVFTYq1tOV9n11FRwaakYhoUj0rsNeqtcwA6Xg7MRQi3iQ01uCI8PQItrfrjpbiHUA/i7UuUctN80OYoFfE5RkqFBZk+NAS8Ql8CsOMnsef0ja/zn/8reTp3FJQbkw4T4p533M4owvVoVhx3WTMw1gH4wjuY2l+w0zv/DTLcYlsTiFZcaLh889KrliI4wm+p4iF6RoLTyYI9TSv+FfzWqNfmu68ZmWS3a71e/ZEgxLo/tbx61entaUA6XhHJ8Mg2pZbeMuiVixNgQpZ25oWGANBTazaEvKxMNo+cu7qVjuFOHylW5IMkhI5uOTW3QxIrwResPIcqELovnbi41jo3mpPY6ZO9y83cFk95Yo7sxNRu61A8xla4P6m1dAh195hT/XAjA30I3GBE00TLgYqvNGujoSQYskAY5ikEjyM6P3j2+AKY7yrnP42OhyHlR51wMI+YJuZj3iHNf73PKBHQl/vE0KbHGOuw0SC76q0m6pVldpwRgYDYYyDFQl+3sB 2VN3hz1T xtc3uIYYjd2eaecHhJd/bzJ5+Xz8i4L/BgL8x2rS0QtTOU/fhOEhHaQHvyCgApGcA6nBNRZXOr72s1XFPE+bqgKBBFG65xD0C/8jBIMjClw8pftLLzf78dzXFzofONozBGkRTIk3gVRoctH4fzfgOg9y2qa2xf4GHGSP7ebbPrWRvnpjGpjlL7K5IdJv70qLzj/egJn5l9F2gt/PZTQwQU60bFY9naUxQ/5lUTu7Kuok58PZIN8bRdvSRYT7QtV2bzzU40asw2tvUihZzJizkHXO30cEBZEyIXkfAgj9Fvl79p7y7WjrT8L+WxXmcUW/68+kDuQyhGLjktv9UFC/24OfI2yjByx88fK388NYADaG0TE4eKrOrModAYBm3C4A4cMS8c1afaCECPcr3E6cXPTXWHMUoCg/7/O2qEh8ZC/p12BwnEmRzSqTp0sCF9FYBsePsCWNMaFxJn01ZP3jz7NsyepbT50+iBfY2PJArCXDucbs60LfIlEiRsnuuBczdD7lsAucqIx0YROBTliFPY3Wzt+frTzF8ot+C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: "David S. Miller" cc: Eric Dumazet cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Signed-off-by: Jakub Kicinski Tested-by: Tariq Toukan --- include/net/tls.h | 2 +- net/tls/tls_main.c | 24 +++++++++++++++--------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/net/tls.h b/include/net/tls.h index 6056ce5a2aa5..5791ca7a189c 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -258,7 +258,7 @@ struct tls_context { struct scatterlist *partially_sent_record; u16 partially_sent_offset; - bool in_tcp_sendpages; + bool splicing_pages; bool pending_open_record_frags; struct mutex tx_lock; /* protects partially_sent_* fields and diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index f2e7302a4d96..3d45fdb5c4e9 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -125,7 +125,10 @@ int tls_push_sg(struct sock *sk, u16 first_offset, int flags) { - int sendpage_flags = flags | MSG_SENDPAGE_NOTLAST; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SENDPAGE_NOTLAST | MSG_SPLICE_PAGES | flags, + }; int ret = 0; struct page *p; size_t size; @@ -134,16 +137,19 @@ int tls_push_sg(struct sock *sk, size = sg->length - offset; offset += sg->offset; - ctx->in_tcp_sendpages = true; + ctx->splicing_pages = true; while (1) { if (sg_is_last(sg)) - sendpage_flags = flags; + msg.msg_flags = flags; /* is sending application-limited? */ tcp_rate_check_app_limited(sk); p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, sendpage_flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + + ret = tcp_sendmsg_locked(sk, &msg, size); if (ret != size) { if (ret > 0) { @@ -155,7 +161,7 @@ int tls_push_sg(struct sock *sk, offset -= sg->offset; ctx->partially_sent_offset = offset; ctx->partially_sent_record = (void *)sg; - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return ret; } @@ -169,7 +175,7 @@ int tls_push_sg(struct sock *sk, size = sg->length; } - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return 0; } @@ -247,11 +253,11 @@ static void tls_write_space(struct sock *sk) { struct tls_context *ctx = tls_get_ctx(sk); - /* If in_tcp_sendpages call lower protocol write space handler + /* If splicing_pages call lower protocol write space handler * to ensure we wake up any waiting operations there. For example - * if do_tcp_sendpages where to call sk_wait_event. + * if splicing pages where to call sk_wait_event. */ - if (ctx->in_tcp_sendpages) { + if (ctx->splicing_pages) { ctx->sk_write_space(sk); return; } From patchwork Mon May 22 12:11:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9A99C7EE2A for ; Mon, 22 May 2023 12:12:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 960F1900009; Mon, 22 May 2023 08:12:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8737790000C; Mon, 22 May 2023 08:12:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DA1D900009; Mon, 22 May 2023 08:12:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4309990000A for ; Mon, 22 May 2023 08:12:09 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B2D61C6F1F for ; Mon, 22 May 2023 12:12:09 +0000 (UTC) X-FDA: 80817777978.09.1D5DC30 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 478E7A0007 for ; Mon, 22 May 2023 12:12:07 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XIdQ5fH5; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757527; a=rsa-sha256; cv=none; b=Yi3h+PNhW6APp/vZI+KGICb85DEL90swtvRJkL/wz6O5PjbwXugjM8wHLt3D+pkXglw3lD OYTRr2hqRet8Hk/G5hC2SSEVRcjVud8zIYCwty0CoRpesk1tnimnfxqVGz5X6/BY10Hcit vUOAvPzW/BPkGnyK4SggSZtSrLqVE9g= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XIdQ5fH5; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757527; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=lZd5zsjSz6+cqMIrBiUaNBHdTaE4GerFR+elBp15KiBK42/rQpJIlcc4Q9Lv4H2sER4ueI N5HMzZ+uI1nvBa+UyE4Xjqv7u2d7WL3c0/Q0eVnWNhjlI3BNCAZuY3XzWDuwHFVWZv3dWo EBdTOfX9VZMMqQHx+nZG886audDhb/w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=XIdQ5fH50MoeriXh9fss1Wtc3cR9HXNxgHbTfFuA5NS5gb70CXcaF8CCKptIKySpCoxrrR Nnm97s6uN/lyR2zRJyFKmPkTZFhumwzsekSLiNzWwHnfSdez/Dkm2S78FIKrsBSzkqVb4x ARwrWMaOwyQdyLT8IeQI6JukjtcV+cg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-474-MWYfU8vlNOi-xXlJkHtgaw-1; Mon, 22 May 2023 08:12:04 -0400 X-MC-Unique: MWYfU8vlNOi-xXlJkHtgaw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6F58E858F14; Mon, 22 May 2023 12:12:03 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BC277B7C; Mon, 22 May 2023 12:12:00 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Bernard Metzler , Tom Talpey , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH net-next v10 09/16] siw: Inline do_tcp_sendpages() Date: Mon, 22 May 2023 13:11:18 +0100 Message-Id: <20230522121125.2595254-10-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 478E7A0007 X-Stat-Signature: g3i36mfcb7ttjo3cchhmphheh5s9b5kn X-HE-Tag: 1684757527-677696 X-HE-Meta: U2FsdGVkX1+FfKeK6Og9LYqJ1m4ne6iCh39cMCFuhdbez2u62bqhWHmC8UQRyNb9enDS6bax/2qHHorWKjJGkOJNSg44iLh69XydbOU8gwuCi9zYW57WTy5qMAJK3X5QW3l8gPCPNypQ6iVdr+Ge4ZxdRRLrycYzTF7X+ogwVnnSQksh1HiHm/Tc8QroVQ31N2SknODM6L0Ihu3kOnY0pQ45wWRik9X5PIrq7+NuJtOPDTPWfPBLmYVJjjK9U1xxKrnEWH2yW2jOr3Yp0u4qL79PmAxLpIYL0ZYwN/vu5zzrMb6vDnWX/gODu+eWWHT3SKHGX1SQG1TPyJW6A8PfqINUv1g1+kIgkp7G/Vboi5GcosH8Bl6oXd0Czy9Jiswk5nGB8iHURxM2gDkQfgnCW2ciPZvEg3hyQWKPoDoeICGFwV4WEqyi2le4s4bN3ukVnP4KiUwpixC2PtikWw90E3cOnPEn+UYhOei003t/KJ/8VOiXLD17ofVibKAjwZ51vb9afM1OOSCgTzUOX5trbw6nVbPFPrHVt/t+mxV0Je5Yt75u9E187u6crmd5h4UT/7Dj4AfLb/Yb8hY7JLYX/qQ6LLEUwwN8u0W34mmpCAIgnRVO73u2l7t5F0WsdUYhpmASJwg5N98ZEGjICp6AVdQaYl2VdrFKvJiXZmI+lr/igtjbHKD3NdFFB7RL/b1cxbj7aGoykTAFJ8HHDixJpfR5HLlyJmNwqbnHaLpUroamy/GwQeIoSC95sAQ3zGUL9AWYA1fO6TT/D23nY0mU6XtErTr+Wj5aL8LeQBPYvZR9vcs7LBB8XMNjLiXgFpjkb23TGaZuD0krzgJA26nQGbag9jhAJ07R8jHEAcoTrQXc+vkEJqzX/Lf2dlb+xW0+vET6RnDEdMKez6Z6oV8YwGDC9vdbuwfPLvTKKYcMqIgT1xJzvkJj5rI8RqTf5ksV4RNAT8wWCjEmMrMIjgE g6L8e1KP HSy7GGywA3XffAKVVKLlSNItpYDsX35fSs6JXpJbtbPkpJlDTCYw8xUUoSlAwR35R1AoaDZw/DlCVJg0pVOz/EFxDxbWAWZ7zdXGPpgPnRgMd1lc6SjXZrp8WoHup77yUhheohCE1r62jSrCtTCSd4vLbVttdu9qpy/90VMGdbDhjOHWGIGfApVTQcfW2J7noPLfLBM0MtpVaGPRUZdJpUaizPLcOKGOHjPn580pPOHeLdke1GU4kpDBgutmQZS3Cq0h8En3OmOxV4p/kRRmpfAwdFpDK5ZQud6ZeNicabD7+NZeAiuG6FlhXkyLtkM4ytqkgt746w6u6Xi/8A4ucgjopC5uQOWS6tjjbNeUq/Pz1KZW7DkSZDxrhp7Tr25P7HPC0huRwoaLdgVwJsoJpaGSYUyzXlAT3WEifCIdAy29TZc9TTkv1tef09kclk51SxD2hai0lVvYz4ooYTawlFJIOMRDl1uYMnI8fdkL6MnqJ9xbEAxojmAiqmWdsqlmbzM+GfggBHWt1GSbsv8WReiyfv8WkYjzCclotF2i8wUjRjud2Pd+e+I/FskYi9LFKacpXHpUgWg45t8DrkGddbggwaUDlWDB/ihYVvEmxre92/tG1G7hBVg43hW53xsqizx5f X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells Reviewed-by: Bernard Metzler Reviewed-by: Tom Talpey cc: Jason Gunthorpe cc: Leon Romanovsky cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-rdma@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #6) - Don't clear MSG_SPLICE_PAGES on the last page. drivers/infiniband/sw/siw/siw_qp_tx.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index 4b292e0504f1..ffb16beb6c30 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -312,7 +312,7 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, } /* - * 0copy TCP transmit interface: Use do_tcp_sendpages. + * 0copy TCP transmit interface: Use MSG_SPLICE_PAGES. * * Using sendpage to push page by page appears to be less efficient * than using sendmsg, even if data are copied. @@ -323,20 +323,27 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset, size_t size) { + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = (MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST | + MSG_SPLICE_PAGES), + }; struct sock *sk = s->sk; - int i = 0, rv = 0, sent = 0, - flags = MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST; + int i = 0, rv = 0, sent = 0; while (size) { size_t bytes = min_t(size_t, PAGE_SIZE - offset, size); if (size + offset <= PAGE_SIZE) - flags = MSG_MORE | MSG_DONTWAIT; + msg.msg_flags &= ~MSG_SENDPAGE_NOTLAST; tcp_rate_check_app_limited(sk); + bvec_set_page(&bvec, page[i], bytes, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + try_page_again: lock_sock(sk); - rv = do_tcp_sendpages(sk, page[i], offset, bytes, flags); + rv = tcp_sendmsg_locked(sk, &msg, size); release_sock(sk); if (rv > 0) { From patchwork Mon May 22 12:11:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAAA6C7EE2A for ; Mon, 22 May 2023 12:12:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 603BA90000A; Mon, 22 May 2023 08:12:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B37A900003; Mon, 22 May 2023 08:12:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42D6F90000A; Mon, 22 May 2023 08:12:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2D34C900003 for ; Mon, 22 May 2023 08:12:16 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D33F6120418 for ; Mon, 22 May 2023 12:12:13 +0000 (UTC) X-FDA: 80817778146.20.EA2E2AE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf01.hostedemail.com (Postfix) with ESMTP id D31A340015 for ; Mon, 22 May 2023 12:12:11 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E9McnsqJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757531; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FkLIsuWcxLdm5m/RLKw/6bFEvudw3Dbf8/jbJKlIa0s=; b=ORT2sEcDvXQ6QnGtuvFIkGiUjB8pyFOFXENlQBluqKExBNuL8PJoXNP2egWBTNY8MP/BGA cmSa4gwEFTiBCpL1SRrzTbh62hpewxwhM/Q+GZovg6iocz7ynT0L53Ekv70qdbzthu6gLx M+892PL3ydmHDNvds2d+Msa//D25Llo= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E9McnsqJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757531; a=rsa-sha256; cv=none; b=jAtVt3RPNPvOakVGs4G8ThR1KxyTx+CQEng/rNYjqJuduDNAKMhisMU1a0utkIxuVGC6zq Vzb8aabXObIjfCyRzfm7HVhzeDJd+tJruBvDZOnpfBKBB78VenJonIfcWAndxGY11tY2Sm oiFkNPyRtAd2mtgFwQV4QzehNcUc/Vw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757531; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FkLIsuWcxLdm5m/RLKw/6bFEvudw3Dbf8/jbJKlIa0s=; b=E9McnsqJDqMq3GFSeMOZZ5MImB1i409AA4xnGszNzlEkXAYad5XP6nU09j1bM6LjVqxKQI htLdxNODiRjr8tG5o7PR1Gokx5CySpy07M3dn4O18uSgQ5QKWgYF6qza9WCc33SLFXLA0+ 9i6JtPbwl6LgkmdetT48WFFQ1Pi5GQM= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-54-cIwQpXAwOju38rvcwCBp5g-1; Mon, 22 May 2023 08:12:07 -0400 X-MC-Unique: cIwQpXAwOju38rvcwCBp5g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CCA9C3800E8C; Mon, 22 May 2023 12:12:06 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 57ECF7B7C; Mon, 22 May 2023 12:12:04 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 10/16] tcp: Fold do_tcp_sendpages() into tcp_sendpage_locked() Date: Mon, 22 May 2023 13:11:19 +0100 Message-Id: <20230522121125.2595254-11-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D31A340015 X-Stat-Signature: ju7i1rxpxwyy17k61siwoj96uhy6q18p X-Rspam-User: X-HE-Tag: 1684757531-152476 X-HE-Meta: U2FsdGVkX19N3pABv0Upc+WJBOKTq38grxsacM6wZOVxDJf0U9xVIUerp1jcR++Ar7j/e9yaWLNOvrpzeREnoPwDkr6BEQ5yDQAD5llFUcAnTUgs4l3i222BpnieGVWYo7JtUOUMyLX88tD0knMMbiTEc1l/vAAAKjcqsjEda3BAAbcMlHISpwN9TnZ+AHOvmehNjQLT+LohnzvTdz81Z7E5wTNm7HFFrJC6ICdpR4BZqibkrOVm6UezAE/sxto6MRa58sGTNxu1v5xteN4qwNwlfWkgKVWO/Rai1OQsL02rHoKfSzPf0lsjsJ+PIIAp0TVfrs5pjK0UU/kS7S4WD17nXRbLPXO4QP4ROmQbtHjAFC+JIq0HHXCKyayE8IrbToT0QcEvUHS5KpnitYLHZLLum09IU04V8/ZV00MuI+YqBeyR9SjakkXdL350s3GvZSkR7s2FC9pMpAq7aiMg17Fk17uasFF2IYaJUoqz6My2oM6YHnb7Ks1PPglZzfybUo7ex9M68DFHRHD78ylMg8RBcedYcQDhfHC8TLungJ/CRrGYMpWttXiEgOZJFaDL+7R5vH8e36pmdJRdQ1jSshl0EDdV2DOH1/01lD5a9XEscNEUhZ+5yCGZ5HKiADAKoYfWquUZd7fPhFy2CCmQfPbSGFzx7YxYEYVNj9vFVDbncunaz5pM/0f+ImSRas0G1LC038iQqi2sxzemX4g5M50l89zSR9aQ9uvKrOydFeExvDaGNG/mR8qSIFF03rqjGaJbpWjR9iwMKqtezQmgJ/orYDSMJkAaMO9PpiDTtVrBvzjVhG3nbJcPbI3kgofEdaGmqoHDAyvpmk+ylo6QHyOgdoDeh/exj9dDaAn3R7FF/Sd6sQMBPJZ52KseDu1kPh5nYHMk50yU8GcqC7+axn4VDJ/6XljOYUMm0xrWBl1M0LwEDbjV7SHeRSlRs3wPuX1X+R4rqtdHVOMrQU2 QMezg75O 8pxVbkV/KVU18wij58d/+kHWBMvxMm+64FnMRQCq60CICK6ze1F6tlnm4QqYN1GjzLH7WkKunEfX5eVHlgKnKdWNzBX+bKROWjIbV1VzWOBvcnRle6WCn4jP1arEO0aDda/FE15SMLqpDbUe/sHa6Kic/2RdSOk2p5tAn10lPki5jyayOrm5WdImjc3DxWo9GY6+UJILOWev3g43rBvxSCzB+q3TZnTYA6uYDw4n3cgOII4ojjlb9PK++Uwzs/Im2EwcqMK8gr36DopJmXJ6Oz5lOk+Wyks3MI32w0eMsUDYJafPYPCXHsIJxYVOv2oKGnbt92MVLcJzUR2JEwckgX4FJXkcis7EqkB8wisDSiC5sJCNYhHFDmRm40MUtoVc2JPg/uftQX/+8RE3xQNvQGLp82r9M4N5ohXfFhZOoo5kYxUmiksnhwKlI7sc5/W68RQf7zKhGXqy2Qe8B9LKRDz0FxscUdkDSea7vQ0ranUCG1lsxlbD1q7YBBw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold do_tcp_sendpages() into its last remaining caller, tcp_sendpage_locked(). Signed-off-by: David Howells cc: Eric Dumazet cc: David Ahern cc: "David S. Miller" cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/tcp.h | 2 -- net/ipv4/tcp.c | 21 +++++++-------------- 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 04a31643cda3..02a6cff1827e 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -333,8 +333,6 @@ int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags); -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags); int tcp_send_mss(struct sock *sk, int *size_goal, int flags); void tcp_push(struct sock *sk, int flags, int mss_now, int nonagle, int size_goal); diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index f3a0c02678e0..e9506cebecce 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,12 +974,17 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags) +int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, + size_t size, int flags) { struct bio_vec bvec; struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (!(sk->sk_route_caps & NETIF_F_SG)) + return sock_no_sendpage_locked(sk, page, offset, size, flags); + + tcp_rate_check_app_limited(sk); /* is sending application-limited? */ + bvec_set_page(&bvec, page, size, offset); iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); @@ -988,18 +993,6 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, return tcp_sendmsg_locked(sk, &msg, size); } -EXPORT_SYMBOL_GPL(do_tcp_sendpages); - -int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, - size_t size, int flags) -{ - if (!(sk->sk_route_caps & NETIF_F_SG)) - return sock_no_sendpage_locked(sk, page, offset, size, flags); - - tcp_rate_check_app_limited(sk); /* is sending application-limited? */ - - return do_tcp_sendpages(sk, page, offset, size, flags); -} EXPORT_SYMBOL_GPL(tcp_sendpage_locked); int tcp_sendpage(struct sock *sk, struct page *page, int offset, From patchwork Mon May 22 12:11:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90920C77B73 for ; Mon, 22 May 2023 12:12:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CF1190000C; Mon, 22 May 2023 08:12:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 27F73900003; Mon, 22 May 2023 08:12:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11FE590000C; Mon, 22 May 2023 08:12:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F091B900003 for ; Mon, 22 May 2023 08:12:20 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D23FC1609F4 for ; Mon, 22 May 2023 12:12:20 +0000 (UTC) X-FDA: 80817778440.02.83AB299 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 1F1BE40015 for ; Mon, 22 May 2023 12:12:18 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cul1hH2Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757539; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=ql5QlqLjgsH2Uu7n8C5u7JwzxbpdqPGSxH+IUnFMDl2nPEzvI1sXMcgaplBLNHylF+kT9e xKDJkhY/PdOPKTwFAj09+kKI3CpQYR56o8KGImI4TCRHvM1OY/OL+BhbhWttk7KRJxMOWx 2jBqPqsp9NYTr0uIYH6rjphgCKm2T5I= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cul1hH2Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757539; a=rsa-sha256; cv=none; b=NXclWgAv0AmNEPht6jq9q/Ma20XK7qTJO7HWjH8z6zGATyNnIgAto/tmExq2cOwpFV7gkl f2sw3me8bUZydsaJJsGHfeDaNZ/8Xcx+PWu1ne38HRngxzfI04U4DaB7rvDP2f/DndA5eF hK+JwZF/eM8FW5PxffziLH3Du7J4eSE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757538; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=cul1hH2ZjcACPvgRvbC7yUyba812B7cG0nlVmVZxlpIWux6lQdyw1drqQXvTRCrNx8qBKI VWtv1Q5ELX1Oo5k6013eHW/2PwmW9oIlK21QVZHjdHNgF6zRrQ39P7VPMsSaLcpch2xaPr 0Vv+/6Q6EGDHvvameWjEzSZ/XJ5mUhU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-389-i0ekLMi1P3uITGz1Ox_XFQ-1; Mon, 22 May 2023 08:12:11 -0400 X-MC-Unique: i0ekLMi1P3uITGz1Ox_XFQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2602780349B; Mon, 22 May 2023 12:12:10 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 856907B7C; Mon, 22 May 2023 12:12:07 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 11/16] ip, udp: Support MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:20 +0100 Message-Id: <20230522121125.2595254-12-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 1F1BE40015 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 8spymjftuuw3sou39kfu4c4h58x5gt36 X-HE-Tag: 1684757538-741142 X-HE-Meta: U2FsdGVkX19Br/l4ZSjjv5AqhX0IGZZ8rFOs+5D892nS2gy8AfSDQ7WyPZdXiGpmSpS0yMDyMVojMCDOcMKhqwP27urko2dw+fGOtlK+OQ2bro1k377qHCqYIi2anHv7PqbAGpoBlBIh+D7Lml9NvgjFuj8HmrIcSP1PCqtuLt3U69HG8DvfYNGKKeolgPbJnOC+TIEJ6mYXd/MNSF3B2x+VfTeczhwnOtxqmSzcJCwNtdBCYFbz6kRrXXMnCRKK3CAXe1qAzGDrLQr2mBryyDUWvujYDQUUdrD4JkdvNmXq94ilLz1CxltQLK7xXm91p4r/FVwaI51RA+xkvo5Y7J7QpIx5TxdQTaLh4ZvonDObA4CU71SyN9qiPsdnjvGuDZhq5BlrP1SSm8NdYVCmACAd4qGQy3GySI1UgpE5yeR9JEdQxsPa7SP+5h45Pzw39O2UAvMYZ+IcovO7Bt1LyXlpGQBOT5thAPnvN+j4eUHnmx/Und1SfgSF4MqXjFDskEAP4L3WX2Aq1Gox50pFRcG66X50XSzcyR3UkpqVZ3n5MTB9caxO91BO2Nel0oYB+bt0jhdXRg+vvnQtw4DTGvxS+nyYRk5XivcjLw0RWJgOAHEt7vHYjlnclpiyHj+rnXKAcdSXnMIex2p2k92dP27wfBx6TZqoJhiuA2Z0I9XAhlb+oTHSaTxjkjLvfSSXJ1LrikokrLONQL/9qiyYbir6a+VyomhLoUGJ9w7nApuDrkPWLegxGxmvMLq1vYaBtKY35evMaliQexDKlcLJjPwzyZM30nsOKQ0oOJQvgkjnNi+AnU+OetMivBCCXMfqaXHlZSHuIo6J9bnK71jMEyxlpVpxovjuITyWUfNFPmI3ZrVY45JocyZ3kf8NdTSr6DBuY6/Cq6XHEmySLsR+4sRyHvxzxE1YwePJGImTweYOMq9Ur82YBrDRU/xzRGKoX0aA6sapxQ8CFf027CE a34yPayH MhEOLCbDm4v/XauPpc7s8/lvDfxixVZ8EvB47CJQ/B6Z7cOnZTyJ/fSOrIEcEmYdKXD02b79lYyWzgK1voXeZugX26lLHmXwpC6eDFu9IWF9/bhCjQNCh/AYIh4kPxZP//5NjCeh0ZPT+6fhpPH024qs3n9zH0K9lIj6ayJBaVWMmyVv02D91onRE6MxuSyboKcZJTkUH/7QMF0btvXX66mJ/yFRw9XApg9pWawCSeWD3/g3DcxqHF3KfZaZNCTHVR/RMw5MOyYHSkd7VyZ3dezmdw0J+zVPCV6Uwt2ZrJD/I/EabKzdQAn+c91b4qrR3Neyoxrle0vd0O550lLq28quRNVGyKH6g+cTL04siMLPFdULwTYylEig9zukKPHmHbIFXK0TKw/GunDXdnCwirXZ+PX25pmM/J5z2cnKQorkvEShZuDQwSvNGbQEPgP4kT+3pEzOs7+tc5POtyoQQ4zC0K1OFIkjTYVTQ0xqHxF9ohxJl0uAzSP129wBn6f3l9QjdpPWNA1XQHq0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP/UDP sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv4/ip_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 52fc840898d8..c7db973b5d29 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1048,6 +1048,14 @@ static int __ip_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } cork->length += length; @@ -1207,6 +1215,15 @@ static int __ip_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Mon May 22 12:11:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05C2FC7EE2D for ; Mon, 22 May 2023 12:12:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9392B90000D; Mon, 22 May 2023 08:12:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C0B0900003; Mon, 22 May 2023 08:12:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7618990000D; Mon, 22 May 2023 08:12:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 631AB900003 for ; Mon, 22 May 2023 08:12:22 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 20C8C1404EF for ; Mon, 22 May 2023 12:12:22 +0000 (UTC) X-FDA: 80817778524.16.6A99AB3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 6AB09180014 for ; Mon, 22 May 2023 12:12:20 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H8WSKRZD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757540; a=rsa-sha256; cv=none; b=8OfOhVip02rr0rftm02nDtAAnP3475jXpnxrBP0IZ1/MHOlIwhkKeQLes7TR566zLxPzEc D9j+YOscf9LS6PplIQXEXipn+Jc9GPZHkF3EFhNZwBB1jswDvpffcYzb24XkHLRlWK8yIK YmIGzMbNPiKwP4IKmQPcC1zKqEryjnk= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H8WSKRZD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757540; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=ipSWE+ERowiG11oXxhJWZAvMuWtNg0pj52DOoIQBz3MA4om+CRBAnn5w7fI+9UPpKqZQYW 1Q4UKtZis8mEy/trHAeIv8br6BEvmV3ss2y4qqFNIqK8lKgFtexFGgWRdeYQWXUXO0obtd lwMKWwtX6HVKOGA8OVIL7qSv1xDXLzE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=H8WSKRZDo82RCXfVpZOrezmdo6ULq+BPkyKRJr3f6E+EJGWjlFpTpMlDIJyr4lK53+TIrr 322XEIJNl0x2xRhQm8K062wUUeqMCUoJWwmGpt55WpBeoUieGL1FLssDXQz7X9WkvZM23u wwxJklM0ldYvrMn1bO7kjn/heUrm3ak= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-569-bsmo5ELVM-2TXNBTHt6zHg-1; Mon, 22 May 2023 08:12:14 -0400 X-MC-Unique: bsmo5ELVM-2TXNBTHt6zHg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8B87B85A5A8; Mon, 22 May 2023 12:12:13 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EFBB1121314; Mon, 22 May 2023 12:12:10 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 12/16] ip6, udp6: Support MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:21 +0100 Message-Id: <20230522121125.2595254-13-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6AB09180014 X-Stat-Signature: q4o6qfydpxgdyxbcdyrtwzudt7gq7pi9 X-HE-Tag: 1684757540-10925 X-HE-Meta: U2FsdGVkX1/oSHhmTiFyGOadWnfwr98SyfyOfoqRpxZrDfkV6MF3A80YZIFBB3pP0pDrcP1Mh6qjxfjgYxi/T1iva93CjqqImeejXg1zv5mJEH7J2F1TkzQlQwEwFoMsHJV+ACy/8f+evJvmIYIuvROPVMK7hVkci+Wd1ZeVRk2yqKbly9cL5XFWaCzdddde2gEUoGmIU+oniD9hUY/j8nXjWNLYwF+AkTKb19L/B/gqTmjfjZbo6Ga7s4t5iAx0j3qtTsiV5V9wK3lVrMaXYsTqImRayuIDymAmtIvH0dkQrFnqZx8HnP9Lyu56RL3h1CJIiptV/3wLpu5zIqUGXF4/WkGG9mc1kLmY0fFnvP2KrXoLjpM9tg6dtzeAju0MUzdvf1GJf7U2cAn90Zwupvt/hk3uCNamLZusLUYVKN0SACDjlaTJ4EjmjxU+j/9Snx54DXsCVmX5vlGthOH4ZrUR5SzkCNd0d2fnUCoKz839PWuz0Ug0j2c8+jbE/gJWVZiBqX88GQr+k/r8f5x7lvfs4gV/iylRlqljgwwf1p8MrGNeUOSHz2IKZjIPF9rtjyWrVK7wxSsikKRzDgCgSKeXvk7DctV2SMnLaj0QmwwyQdZ2mld36x2wrlseI36kIA1PBssq5U4spqF5L2FGJlkcKYCFGTKjc+gy1sPGM/RWOVfxOEgQ05tn3wm96vy+/7fnWzEUE1MwdXN5kEfR806z3NVXxkcz2uJnjI+8vpiKlG8DFNp6RYFnglulrO/5G7PTr7D9DSYR/7DDrJphO9YR7AB7NlMEKM5eHWyHYvayQPKtp5f1k4iMU7NBuqu16fUAKsnG/gb/voE3yEy/bO0ZyPPeeR810NJx9lIfZvjnKN4lrRp8E9Xh81WEhsslyDqwQgxo2SBZPLgfVF/UzrEFqTbCk+B9woMuRCGlczdsyLorFzASvKgm5XX5xEatNMyEaVvR+xo46fKSrsy 0HaCxIKo WzCner8B8nMOcbyfAU+kBlcbn3nhYn1vVyX7Befk3ENOT6YUAHxnXHrYPJ6EEksoeR04oAF/0KzwCZhfAcHEqEfMgE4LVTsOdVwu+FoYbd/TGmiiCgPusySgTGSRggoxlMo/jAzBHaF8hGzce+tam8vJcoudChSURAQUqXfTzQKt5VZeYcaAjeRYeMyVDuqk+yJNEQ2kDOSBvm9DqMAJR2oCgtXcoL7ECUiKwhreqPwyCh+NsRHCWxOrYbqJma5Flj5pzacY00ryaafdgLniPkyicgFbqI7HPeBicmk/UiHLBT80DF3SWWCF/3/OIUAR3i0RqzlKUX23aLNWz0noGFby/gwX/w/hzvKGQ0j4q4qpl0bjdwKoXVGqmRThR4u3ZDd60T4oOmNAJo8viYs2FMbOJ4k88ROXPjN0UyYzQGFzb8TobdTdD0CbAt2DWGEmIAOCiDL5SpMbiTfuloUoP8k7WozNJbuuUbk887fWd53HGBL3CymPndP3FJV/SXfFnv0vbTBDvHgNNMJk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP6/UDP6 sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible, copying the data if not. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv6/ip6_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 9554cf46ed88..c722cb881b2d 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1589,6 +1589,14 @@ static int __ip6_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet_sk(sk)->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } /* @@ -1778,6 +1786,15 @@ static int __ip6_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Mon May 22 12:11:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EA3FC77B75 for ; Mon, 22 May 2023 12:12:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 197EC900009; Mon, 22 May 2023 08:12:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 122EF900003; Mon, 22 May 2023 08:12:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6639900009; Mon, 22 May 2023 08:12:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CFA11900003 for ; Mon, 22 May 2023 08:12:24 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AC6C1AD938 for ; Mon, 22 May 2023 12:12:24 +0000 (UTC) X-FDA: 80817778608.16.6A71B7C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id CC90B40015 for ; Mon, 22 May 2023 12:12:22 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ByDDlfAB; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757542; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=sEsFuA0Flh6wB8AJoRuXEYMwwsvokO6cUY6xjIiTBH6hq68uCHvGbFtqcELTMIZldNoHRE zLVdtRQE3IiwDf3t6xj6bnH2woR/BVWf3Y7UsroUxC857nLNU9jJahk5mHODCNmP+BpApV yuyRYmCebaa0gFDNW6q5PyG3uP83qks= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ByDDlfAB; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757542; a=rsa-sha256; cv=none; b=C/GQtm/ljQvqhIKMwgkFGNlHDYl3nexyst4uTWXHtzVS+iXR8BeqBnquljHQ79HalQGN6H 2BNL0ArEpDWCCBdiFIUHQKs+sNy7HtfddxN0FZD0ir5qw5KS92NHitqyegEifqQX3/PxmU wBjpYqw0CQil+vUbdo2LOddDcu+EfvU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=ByDDlfABJHZo/PMM6aKZxnVJhxV8yK9Q5WCD7rCAXd6Ipp7oomEGXBfJ6kcd0dqxmaeqZ2 pj4DLGLEPkjM4SeoFJl57nKME7Lv6t2yjN7ofYr+ovkGLi1jL32D5jlEHcGH2WGU/m3Gen F5fB8G///N4+tX0a31jow0n78u4pBi4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-407-eMUMHyWdN1qeVHDjTVvlzg-1; Mon, 22 May 2023 08:12:17 -0400 X-MC-Unique: eMUMHyWdN1qeVHDjTVvlzg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AF505185A78B; Mon, 22 May 2023 12:12:16 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4439120296C6; Mon, 22 May 2023 12:12:14 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 13/16] udp: Convert udp_sendpage() to use MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:22 +0100 Message-Id: <20230522121125.2595254-14-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspam-User: X-Stat-Signature: q4nzz8c6am6ydgfsnukuxh33dsjny1cg X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CC90B40015 X-HE-Tag: 1684757542-820099 X-HE-Meta: U2FsdGVkX19FJs2DHbCRZOdcOw1w2+wBM9T/u1XapUxGdl44dZ6nIKOdSV+D3WYs0kcy9/co2XoQ++R70pCqokYjguHLQrMtd+8j2Kd0tJRLAX3E5t75yJl1KlO874s5yAb0NIGGeAu0GotuawrWj80SOeORlMcSc+2q3J2xfSyjn9dHCXPUFyb2itBF5ysbsTH7dM8Kpzu+HltZzVszJDhP4lqphitQ+sB4IQoiSkhHnuvnwaTmSdQK3p0DSxDZ4Fvzm44VPYlYDBbNRAOj/aZCxZvoP0sVPgwhSpADThiHmJoKErdANp2lFgGM67O/mLEsQznp8HU82I2hv6aQizcKR3Rxc82ezzfAkBUvh87pmJJUrBVV4d3N4Ib3awKkr4RpKubTcERMjyIIH2g6pBt/GHhcEj9ZhmOCtIBv02GtRxzrNMyWs0JVuWgfw21KDThWbuQw8fpsHknvnF7+tUjQExxnvuBDOVutcMaYA8Mg4vWIawjsNQGpxa7BN464Lm9/jzshpQ6z2jiMXzLy3WFhOVWNsjpn7cpCH7wcxcsPLMuL6toqVfyFS0pQWiZawCo3FCZAeOEz/PStiSNBJ08/qqr6JRhiEXOkXh8Eo+5bRpY0tzjgS/40ojnYfcWSB7Zz91mzlvJsCS9Zp4q06mO1s0bZF2apA9+wOwh81JJH2gC5ey/NGw3mQPV02pd51evWuq7rR2MF7vq+0vv682H7ZNqJzgdIcrAuFR9eFnrYglAsfiZv217vb4f7cTAij7ExsjBxyV4D9eVGJpXw9XgoUI/dXfgUg9lbpQGrVH/ghkSNXxM0BIQPQ/+OINMgzj36BvKVPpBRyWbMr5rzU7X2AOytpmD4+n18lUtVEgyqmhjxrqIN2VTCaZYk3SpZ+bV+9J+jbpnvh05BNtn/qJBKXbR2iz6deKhx8ZfZ8xEQ8BGZUn0tlw/Ykqimf45VKPkSussMqjbikBV+7af yFZSTkrq mU7EAFajVyoPJ3jRgg5AfTP9JBTsscm5MjNDllckOW4zn5NNe/Fsx+edyuQB74MdeOYIIXK0/6HRVjWb62UGAOEj5vO8O8Sa64VmQIGuMGRKh17uKwZpNDPOLHUU6H1LM6Lc2r/ZGVcLVm7Hl59Nv7fCbXpAE9qebrBoL5X2W0jDBa/38NMeBS2NxPP2U2V7LBaprr8tsVbcTCH9sTXTBOVPX4eXlnEK0XxZsOaSgyywDMN3LWGRyeeXG5ClHItOljkZ56iAh4Cb+o6R3BSKMXMlIpOlCKgccrn+aGYoRwQu6pp2iuxenDP4N5EYYh+ia/e4gL/wNaM39y0QRrFqe7fkQ1zyK1mzCpLiUhSMuKUNFDwXlx7LtC+RaB42fTarFoQRFmvsALLLuJlC5ICS1jvVIR4xVRSFd81jW7j2m4vx6CYRTC+c9W1GLM9NAPCPE0gACeOhYRVS/4OfLSIDQUdVxrrNWyemeERswsp1VR7/Rr7G34u92Sc2wOY3247OieeOKji8D75y14UY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert udp_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - udp_sendpage() shouldn't lock the socket around udp_sendpage(). - udp_sendpage() should only set MSG_MORE if MSG_SENDPAGE_NOTLAST is set. net/ipv4/udp.c | 51 ++++++-------------------------------------------- 1 file changed, 6 insertions(+), 45 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index aa32afd871ee..2879dc6d66ea 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1332,54 +1332,15 @@ EXPORT_SYMBOL(udp_sendmsg); int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct inet_sock *inet = inet_sk(sk); - struct udp_sock *up = udp_sk(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - if (!up->pending) { - struct msghdr msg = { .msg_flags = flags|MSG_MORE }; - - /* Call udp_sendmsg to specify destination address which - * sendpage interface can't pass. - * This will succeed only when the socket is connected. - */ - ret = udp_sendmsg(sk, &msg, 0); - if (ret < 0) - return ret; - } - - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (unlikely(!up->pending)) { - release_sock(sk); - - net_dbg_ratelimited("cork failed\n"); - return -EINVAL; - } - - ret = ip_append_page(sk, &inet->cork.fl.u.ip4, - page, offset, size, flags); - if (ret == -EOPNOTSUPP) { - release_sock(sk); - return sock_no_sendpage(sk->sk_socket, page, offset, - size, flags); - } - if (ret < 0) { - udp_flush_pending_frames(sk); - goto out; - } - - up->len += size; - if (!(READ_ONCE(up->corkflag) || (flags&MSG_MORE))) - ret = udp_push_pending_frames(sk); - if (!ret) - ret = size; -out: - release_sock(sk); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return udp_sendmsg(sk, &msg, size); } #define UDP_SKB_IS_STATELESS 0x80000000 From patchwork Mon May 22 12:11:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1BA0C77B73 for ; Mon, 22 May 2023 12:12:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EE39900003; Mon, 22 May 2023 08:12:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 676CA90000E; Mon, 22 May 2023 08:12:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C8DA900003; Mon, 22 May 2023 08:12:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 33ECE90000E for ; Mon, 22 May 2023 08:12:25 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F1C9A40290 for ; Mon, 22 May 2023 12:12:24 +0000 (UTC) X-FDA: 80817778608.12.DE1721C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 36042A000C for ; Mon, 22 May 2023 12:12:22 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NJ9IISfF; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757543; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=6pwO6gYH/LRFZrT4pcn0AGz7j3uCIag1raThcXEhjXuL+qB8IxNaCk5Q7rHZC8/GkHSwIk oZbJp+BvWwpE8HLUVcqVo7xt7+7YVJwywDoW/84UsEzfa3I6iaZx+N4yMFXwzXJw/xrKqv Ob1ZG2T5ehgLWXX6FExK97EFj5KXZ4k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757543; a=rsa-sha256; cv=none; b=7D7vVYi/WCR9SpiZLOprzncu3dHepqdJB35K/FzLmDPknO3EiiR3pKfu47Mb17pIO2yWXa dwuM9A3ldo5mjVpdyLNHogZqXyojf9WYkQRu51DWW9cTtOe5+OVl+ELu3Cr2bfztwJvFRl nb/uesiCxJAhQMqtMNBeaeUUc+4HATI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NJ9IISfF; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=NJ9IISfFnnxcFk6TxHX3V7b0aqynKa7nQhSWxQgmGi6N6qv5ZHd32c2T5a6B75DaiQWG+6 nMvEWFnaAl4gmZPWyOSjZwNGLMJNjFRp8wTaew3O/25363ShY0DQFyb1h94POwdhFLRX/V n79Pp4VmMqBoVhen6dHGoUxIp2SNhU4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-296-LPYcjbL_PmKPilJ00U_9eQ-1; Mon, 22 May 2023 08:12:21 -0400 X-MC-Unique: LPYcjbL_PmKPilJ00U_9eQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 544FB3C0BE2F; Mon, 22 May 2023 12:12:20 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id ABF042029F6D; Mon, 22 May 2023 12:12:17 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v10 14/16] ip: Remove ip_append_page() Date: Mon, 22 May 2023 13:11:23 +0100 Message-Id: <20230522121125.2595254-15-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Queue-Id: 36042A000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jsduxieb9m5js8kc4o39hj4pz66rgw88 X-HE-Tag: 1684757542-931763 X-HE-Meta: U2FsdGVkX1/UZxdKmk0p7nxRM5pOfyk3iQcKUDfDN8oY72yHmJQ4cNYoL14RZ90jBT76kBbZ+Ui1gYR2m53g5+1yaziw+DjA10L9Q6QPg5x80Q03VSuS+uIfPu4FV8hC9krdxAUK47HjBD18Xmla8lL0ePOfFlIsl0cnWRTyUfbNrI6I6fWQhskhsZEzVlltkVahQxY75msgR/yQ1PwlxP1js9OazAq5vX/69H4AM90maiMejWdufKV8c5ruakf7A8o7txHSmcavkt1QIZh5ico+J0AKlV6vEVl0EvVEjPoeCRWFIym0FK8q46DDe9R8DSCGsEF6nOLMSIOJBqrGyrJn2QWlWE0V9kbSCLwDgnV3SjqLaiQMg2OIG8KWa/ijnaIqiAAbec3Wys212JWO4LjfPD0wyQiANIq+xngH4mTBqNxEB0UIv+MrYPa7FIb5Pr9Auz8AZxyBD7reLZeVeTW6ZT9+x8XcriwY34/MZjJRxTgKTCKTmtP6D/9Zw/rfHTg9X18iqGxI1jNQU3zeE/hXOCTXPqM1N+4b6DGNA/+HJ2Xi/3rdgrlWwy9udzaY1LdizRUfgb5t3S2hI+u5qSieu6vgsWaU1tLlywNGZTUEkjemgxPlNHU7HsmncK/GrDMRaHpflN7OHM+2lnQN6qibhcoXIGdxQ0Fz9etm7zDBOLGnHs6v9VMc0mP93mFRmtkIcCfD9D6RZv17RC2KqlN1IDS9he1dzG3d6RZfwvwsCvGdwqAR/BGnUWkYvIriAicQzAV6HrJxeBcChnfxF9zRkS13reaUtxogNL51MSNPDNNqEMI0f/oLzhhT7VRZbsOW0WByGpx8YPUugyC5II5ZplA+jmBOmnaUr2bT9cJnAkzD17a2sMVAK9Zl8liuYvJ4PqjyEbLdy9byYTemXqK3aMETRPusmO5yRmmgj14g0sAvUbZ0lIRvGBZoPyU3C01GsgH744LxdVi+ifx cmkoZxw0 cHSU3Zkpw+wj3TNeQYwSlZm7MZxTiUChPeB827TYWKhgr3LBh8Hqs6LkqcmxBVjOsQVus16RbDrXg2cTeT4MKzAbCqDh5Yc3AVUEW883XTjukA1XjVnTPeEubZqC4AARymwMAzoR+HNk+F8CoswJQytmcU64DKGZ1RG8y+kPVILYhp2edmW31EopVQ57a6YihlOVMCfRdR7V3GiTxhvt3YzVPSYdEcozzbg+ZXJ14l3GQnTrKGSmo74+7BUEMoCVBT9lMZuyZ5cKhI/iI4gevvcKDqZsYVj/1AVanGY2PpcFfRLtRflgQEKqbrOEVFktAOWzC6eOA86FhyqaDd0jXmmSiiKn4Yr6NRjR0HJJqwMT8h6wfHXVgtK4gvhlGSSDJIKdTcEck3qGWEMw+HfvNKye+MyQP/oBb3W8HAnT4HMzzeVwkTAc5cemsAk0W+R5BVO5oLjVUMDmMAq7vZhDo0eoJkbOIZsnhOLE4hAWVJipDHOCUIogLHdIkIN84oCwyT59zZhAk8O7yZdk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ip_append_page() is no longer used with the removal of udp_sendpage(), so remove it. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Remove now-unused csum_page(). include/net/ip.h | 2 - net/ipv4/ip_output.c | 148 ++----------------------------------------- 2 files changed, 4 insertions(+), 146 deletions(-) diff --git a/include/net/ip.h b/include/net/ip.h index c3fffaa92d6e..7627a4df893b 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -220,8 +220,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, unsigned int flags); int ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk_buff *skb); -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags); struct sk_buff *__ip_make_skb(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, struct inet_cork *cork); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index c7db973b5d29..553c740a6bfb 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -946,17 +946,6 @@ ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk } EXPORT_SYMBOL(ip_generic_getfrag); -static inline __wsum -csum_page(struct page *page, int offset, int copy) -{ - char *kaddr; - __wsum csum; - kaddr = kmap(page); - csum = csum_partial(kaddr + offset, copy, 0); - kunmap(page); - return csum; -} - static int __ip_append_data(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, @@ -1327,10 +1316,10 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, } /* - * ip_append_data() and ip_append_page() can make one large IP datagram - * from many pieces of data. Each pieces will be holded on the socket - * until ip_push_pending_frames() is called. Each piece can be a page - * or non-page data. + * ip_append_data() can make one large IP datagram from many pieces of + * data. Each piece will be held on the socket until + * ip_push_pending_frames() is called. Each piece can be a page or + * non-page data. * * Not only UDP, other transport protocols - e.g. raw sockets - can use * this interface potentially. @@ -1363,135 +1352,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, from, length, transhdrlen, flags); } -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags) -{ - struct inet_sock *inet = inet_sk(sk); - struct sk_buff *skb; - struct rtable *rt; - struct ip_options *opt = NULL; - struct inet_cork *cork; - int hh_len; - int mtu; - int len; - int err; - unsigned int maxfraglen, fragheaderlen, fraggap, maxnonfragsize; - - if (inet->hdrincl) - return -EPERM; - - if (flags&MSG_PROBE) - return 0; - - if (skb_queue_empty(&sk->sk_write_queue)) - return -EINVAL; - - cork = &inet->cork.base; - rt = (struct rtable *)cork->dst; - if (cork->flags & IPCORK_OPT) - opt = cork->opt; - - if (!(rt->dst.dev->features & NETIF_F_SG)) - return -EOPNOTSUPP; - - hh_len = LL_RESERVED_SPACE(rt->dst.dev); - mtu = cork->gso_size ? IP_MAX_MTU : cork->fragsize; - - fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0); - maxfraglen = ((mtu - fragheaderlen) & ~7) + fragheaderlen; - maxnonfragsize = ip_sk_ignore_df(sk) ? 0xFFFF : mtu; - - if (cork->length + size > maxnonfragsize - fragheaderlen) { - ip_local_error(sk, EMSGSIZE, fl4->daddr, inet->inet_dport, - mtu - (opt ? opt->optlen : 0)); - return -EMSGSIZE; - } - - skb = skb_peek_tail(&sk->sk_write_queue); - if (!skb) - return -EINVAL; - - cork->length += size; - - while (size > 0) { - /* Check if the remaining data fits into current packet. */ - len = mtu - skb->len; - if (len < size) - len = maxfraglen - skb->len; - - if (len <= 0) { - struct sk_buff *skb_prev; - int alloclen; - - skb_prev = skb; - fraggap = skb_prev->len - maxfraglen; - - alloclen = fragheaderlen + hh_len + fraggap + 15; - skb = sock_wmalloc(sk, alloclen, 1, sk->sk_allocation); - if (unlikely(!skb)) { - err = -ENOBUFS; - goto error; - } - - /* - * Fill in the control structures - */ - skb->ip_summed = CHECKSUM_NONE; - skb->csum = 0; - skb_reserve(skb, hh_len); - - /* - * Find where to start putting bytes. - */ - skb_put(skb, fragheaderlen + fraggap); - skb_reset_network_header(skb); - skb->transport_header = (skb->network_header + - fragheaderlen); - if (fraggap) { - skb->csum = skb_copy_and_csum_bits(skb_prev, - maxfraglen, - skb_transport_header(skb), - fraggap); - skb_prev->csum = csum_sub(skb_prev->csum, - skb->csum); - pskb_trim_unique(skb_prev, maxfraglen); - } - - /* - * Put the packet on the pending queue. - */ - __skb_queue_tail(&sk->sk_write_queue, skb); - continue; - } - - if (len > size) - len = size; - - if (skb_append_pagefrags(skb, page, offset, len, - MAX_SKB_FRAGS)) { - err = -EMSGSIZE; - goto error; - } - - if (skb->ip_summed == CHECKSUM_NONE) { - __wsum csum; - csum = csum_page(page, offset, len); - skb->csum = csum_block_add(skb->csum, csum, skb->len); - } - - skb_len_add(skb, len); - refcount_add(len, &sk->sk_wmem_alloc); - offset += len; - size -= len; - } - return 0; - -error: - cork->length -= size; - IP_INC_STATS(sock_net(sk), IPSTATS_MIB_OUTDISCARDS); - return err; -} - static void ip_cork_release(struct inet_cork *cork) { cork->flags &= ~IPCORK_OPT; From patchwork Mon May 22 12:11:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 764A1C7EE2D for ; Mon, 22 May 2023 12:12:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F9D690000F; Mon, 22 May 2023 08:12:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0830E90000E; Mon, 22 May 2023 08:12:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E3F9390000F; Mon, 22 May 2023 08:12:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D0E0590000E for ; Mon, 22 May 2023 08:12:32 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A459A1402F8 for ; Mon, 22 May 2023 12:12:32 +0000 (UTC) X-FDA: 80817778944.25.83973D2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id B01CB100011 for ; Mon, 22 May 2023 12:12:30 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="G/CKFMnq"; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757550; a=rsa-sha256; cv=none; b=4yMlhiVyS2EyYJs8X08FEDpgZ5Un3wuYlG7iiGM8jschuM1jvdCDl8WZJ3dbHrIbl/9swh 0d6+hwdToYZ4UAGVVWG4s7Frd1vxMjD1rVZuXimARQmo6fx+oUO08bB8bfT5Xa15UbfP3y M1+4bAYkJQy2TC50fQRT9t3cJBEkkTg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="G/CKFMnq"; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757550; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=dzAiRkV1Ub+LJIeP4VMoR8oJaZJG0OIofAnzAFpztqp2bFWvyAh4U4lYmeMkesfro/Ho6Q J4Rw8esCReE09kJPKaP1XKfEkmqMYG2ZNDHZfFm4yg6JPvFHiZ1ohFB5P9nWZsuPlVv0ZU WsAtGtDTHd/tUKABXrKvUMSTrP8sz+M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=G/CKFMnqzvP1STpQK6HprlhUmBsCn2PIwUrB8a8+Bc1OuV2NxV61Fw4/huDzDB442cyE/X 3Vbl6/jCWQ5cXGFQL0OuNW580yKQmh5GjaWqR9k8LXTv4+RVzC7TpoRBvRPKjLlUpqatlN ++d/mkPAcOZn6gnR2jfaV4zfLZxHUbE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-160-SjSudd0MOWalkYvBaLq9kg-1; Mon, 22 May 2023 08:12:24 -0400 X-MC-Unique: SjSudd0MOWalkYvBaLq9kg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B4B991C04602; Mon, 22 May 2023 12:12:23 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 112762166B26; Mon, 22 May 2023 12:12:20 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v10 15/16] af_unix: Support MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:24 +0100 Message-Id: <20230522121125.2595254-16-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B01CB100011 X-Stat-Signature: 364yebhyu17j3t6nz868sc49ofahs3aq X-Rspam-User: X-HE-Tag: 1684757550-196227 X-HE-Meta: U2FsdGVkX1+3mWq3DgHQBS2gxbtVHcMJphvPZBtruv9x0sqmVRdcwqC6iExWAg/4vU3iAHMCOCz6ZiFj66X9jkLqF03YLcReXG8VuYwNQcy9W9KKYY82KxkLUkZbUOXJRRgE0e9XAroore7TfAfz5sNKbOTMiS/AWJEZn7Z5Mfw4wenhtnzaZM5JiB/30OukrZBIns/XquQdxsv5IODo95rSRKpYyiL8GWset2va5hpC7B0YRZDc76VOaAql91h74J4Ig1FCreJF6cDDnLNzRWaRg1GNCuTPrbNqZxF2ceDmEcJhI9p9v6egAMPO9osgTp+M8NIj+6f9Vl2TaEV2QYb2zlVqlPQ1LAzd95elmtQUkzgwzUbuOfXZTAqFcxXcZ5UJFTkBtL+wzIFGJRWnJhM14wvnhVvo4CTV1P0RtRkxY+0WHwcU9OVH8eYaNsHHBG0jMA9xasb12HvKwN5YsqS6b6G6BJyp1DdXCQbCZre6HVyuGvSid8TXF9oyU27MV00oDdUt5Q4UDzzEXts9IKS+UBkVOMVCYcry8tqx+BM2MH0hVNZw6DphXMEbdgX/SfP45JwYPf+rjz/lROUeG/zs99pziEksoOQYOmdpWPKS9lDWQ296ap7InyRMvRVjQvErOnZ71Eh0hmW+wWlL7cEueNIlhPHnc+PhP61SzHqtEM1g021O1oSSJj69FiiFp+wT5dbqx3T51wtebXNwQvVOAKcu7nxHJEuN+9eBUevBFLiXXQaDhvoB6wDIWZ1+nhewCEMcE0EHToIPHhuQohgb9IvGYc6ol7wUosxBfHwad/8CTS68Db5VbwBCW5ows2SZtb4ymcHeTi6QvcmOaAVY8wa5LJyJq1SGyst3hbKiDhHr7gU/NiMj7Q13u0IFJoIbKJ3XAaXciDEMdV4WKkijD804HhU/vs084XxWrIjy+SlALBFJ2RmvfPJTzS10qyX6F6cUq9SDqLF21fI xyfAO5Pe 9UAldNImJJgZnqgA8l/qHmF3ONKF92LMRrKG/+JRawTltNVQMu9xCsjGgxXo/PK7dtA+OFX6jzjJPnnwT8dPh4W+4YhsZvAh9Yt3Jt5LJcvv2fGLZ8onvreWA8dBhoK2qIBTcQiO/w/mj+19z8rLJ5C/6AJVhUpQbdslX6CVjMzgwwWjTfQvdYQhYaAw3bJYjIN5v5qAj9dpCrhQRkA2rBnInbYQsto/HXYGZ9QGNVeGdaUdKGaVOAfDWhNHC2eZcP4jWD3WvYaC5WbQjfsq6muw558xK98jfC4hkCBTLx05sV0gqN7d1UyA3L5RsWbF9IY97GP0JHvn3qvdLfdeQYBuWdBexlrtitgz342Q/jVCSa8B1Yfw/rUgyAr0l3CwkkrCtEJ8kTBhM8p1taTFXDEnDaqAkp9GvdSI+ydJksiSvC2ZjaEudjzJ9d7uicRIUV5gmyZZPvhrSs5By3QcWCEhqeXKq+O7+wZWVWp+6pGy8HJAfiGn/Z0+D3uvYdReUV6vPS3x959+2FkU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make AF_UNIX sendmsg() support MSG_SPLICE_PAGES, splicing in pages from the source iterator if possible and copying the data in otherwise. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/unix/af_unix.c | 49 +++++++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index dd55506b4632..976bc1c5e11b 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2200,19 +2200,25 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, while (sent < len) { size = len - sent; - /* Keep two messages in the pipe so it schedules better */ - size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + skb = sock_alloc_send_pskb(sk, 0, 0, + msg->msg_flags & MSG_DONTWAIT, + &err, 0); + } else { + /* Keep two messages in the pipe so it schedules better */ + size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); - /* allow fallback to order-0 allocations */ - size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); + /* allow fallback to order-0 allocations */ + size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); - data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); + data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); - data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); + data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); - skb = sock_alloc_send_pskb(sk, size - data_len, data_len, - msg->msg_flags & MSG_DONTWAIT, &err, - get_order(UNIX_SKB_FRAGS_SZ)); + skb = sock_alloc_send_pskb(sk, size - data_len, data_len, + msg->msg_flags & MSG_DONTWAIT, &err, + get_order(UNIX_SKB_FRAGS_SZ)); + } if (!skb) goto out_err; @@ -2224,13 +2230,24 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, } fds_sent = true; - skb_put(skb, size - data_len); - skb->data_len = data_len; - skb->len = size; - err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); - if (err) { - kfree_skb(skb); - goto out_err; + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + err = skb_splice_from_iter(skb, &msg->msg_iter, size, + sk->sk_allocation); + if (err < 0) { + kfree_skb(skb); + goto out_err; + } + size = err; + refcount_add(size, &sk->sk_wmem_alloc); + } else { + skb_put(skb, size - data_len); + skb->data_len = data_len; + skb->len = size; + err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); + if (err) { + kfree_skb(skb); + goto out_err; + } } unix_state_lock(other); From patchwork Mon May 22 12:11:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13250309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C05C77B75 for ; Mon, 22 May 2023 12:12:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E05C900010; Mon, 22 May 2023 08:12:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3698E90000E; Mon, 22 May 2023 08:12:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23107900010; Mon, 22 May 2023 08:12:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 115F090000E for ; Mon, 22 May 2023 08:12:36 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C4BA71C6F40 for ; Mon, 22 May 2023 12:12:35 +0000 (UTC) X-FDA: 80817779070.22.E42615A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 17FF7C0019 for ; Mon, 22 May 2023 12:12:33 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=S02zCYjB; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684757554; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4nmIdUwXs8xUZa8LHr6SjLu9qXp25hoa6tjiHzVF7uA=; b=J9F+FwMPb95YiSIOnhsEfx8gN353sL1XjPaItpbYhDeg4tOKPU8lIfgvEomq7rssfqk4LE LwnPcp0L+dPlM/sVHmaENJpYgepJn2K+XQjozdslPYxB/YdpmYZDNxXOhUM6CXIEnJsTqz wCmmqZGaMKmgKBashSZKuxsWvuhibck= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=S02zCYjB; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684757554; a=rsa-sha256; cv=none; b=lmE1nCpTgQ5BUePwEz7rIQdb121iB4hvgKUz/XiWSEJyLM6yIK6C9wv0XBjx5LXwlm2j27 8Byeojli0JgKqW+nQIeSpr/76nsRpO19JvcfKsliT3hFdWPeTV79yAg8+o+RnPDFUsn0dg KLpxSdBw7lj2pA437HuqEdq7npR+55Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684757553; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4nmIdUwXs8xUZa8LHr6SjLu9qXp25hoa6tjiHzVF7uA=; b=S02zCYjBj2eSxU3JR+nCF+w5NpsRZd6XDz1Okh7EibaO0D7FRW1zNSYLViPAC1V0ybegBb Mo5q/84+kaIS8pN4qZ5RzjYj2W85x072SKjbmT6E6qCkBmFPnfCivnR2HnO02x3/UmJPRj 6KgLAFv4LSqfy3yV3wluqP+//8k1nIs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-576-SCXcFoZWMeW_c6Kl9rP7Pg-1; Mon, 22 May 2023 08:12:28 -0400 X-MC-Unique: SCXcFoZWMeW_c6Kl9rP7Pg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4357E185A792; Mon, 22 May 2023 12:12:27 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id A11807B7C; Mon, 22 May 2023 12:12:24 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v10 16/16] unix: Convert unix_stream_sendpage() to use MSG_SPLICE_PAGES Date: Mon, 22 May 2023 13:11:25 +0100 Message-Id: <20230522121125.2595254-17-dhowells@redhat.com> In-Reply-To: <20230522121125.2595254-1-dhowells@redhat.com> References: <20230522121125.2595254-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 17FF7C0019 X-Rspam-User: X-Stat-Signature: sj4zpjmwsu6nwrisfphcmotrff4mrha8 X-Rspamd-Server: rspam01 X-HE-Tag: 1684757553-160252 X-HE-Meta: U2FsdGVkX1+Y/oPKT64o1mw71FVtmo0cpp5NiNod+tANOUoVOLWD7gmcuxDRhh5kqsVZq741+t5FXrYji2F1dDCX2BNmIyJPWYmS5ULqpXGtem6BnxI9bv7HXmUBdoKykR/kmqWYFvz/k6AmtNdOGgTMoV4dalhR0Vd3fbigbOYn+r41IMbC8xBwO01sBosfo9xAEl1+IwGGQkZFQa4/FxtJ03THp2YaURZpfpUl56up8vt/TYvdTZW4Gx/v6Kn436cLryXiilHt6vwFErkMTAJozT9P/nqkuNmJ3f2NOIcHeI+UD4RQtQr9bWUHRrpjAhbz3iYL5KsDOd9g/FdtahVDnU7J4JLvaTN0lb6CxIm1WpM2KjV4xD3qUYX7ZrUOtotEz6ZJre1/NtCiHd8Q4UaMjiksN00uQbPtDHBWSEz66kkJj0Ra+vprIvAAwIPq8vUkQAbbibsrOVcFOczKqooHbE4kj1dgBR8tZRzrnqfCXW/RDQxJdhnZNLf4Gfnq/5kT2X6d8BadRfuzAzYodM4nV2UuOKfotYdgb0VrCs1XMtq5Mm8OTxURWBwLEi21RruWAXm7KyOOHR8FdI8ZWzXPSpypP2UZIrV7NEij3a9kzxrpAoBoJ/8ZW05a0EUtkvTB2x41rm+wisFNT6nfzAEYO9ctk2wmm+YnMNZjOApJd23ICW+OPnlBncZ1XVYXxFYWEW+Hw2pba525J/TH7O324SYRBzsLaTWSL/nWgNKokj2197hkS50L+FWJW/RtoW/srx8flw/P09xl7RI4tjAOEWivg5ECmnFDpn12YLEMYph0m6pYpuDB3TlHOzplQlnxtP5n1tr/d5igsXRqwtpmuEq4aTaqqFeDSNTi4WXQFjVsewrQ8yhfhluXeUgRRkpWS/72zFoDh7mFhuD8sKlIQcnn0RmAyKKipIyW2WFohGeiNx7yNi/kP3GMlqeRN7+JReMGtNqytDMUW6W 9S3I+SRM AFUhySM+P4cAr3pEcYffCw5bQMoVCMyMlapW4RZSq9jlNg1MxaagfjSMpQVovS/iBjiK8ZLARIK2ROUGU93pHRmfLgiFMZ70GEPttVWtEnaC/Gf5qE7svsfZmOY/J5+zdeATtPZlgG1b6MQtJ74otnBB8+xelXcx0yP+kqvGuHJ5f7B6x4xzhpyhnBamyV3O/TF3sVz9feWWZJ2RIOQWjfREaL92jNqjNv/eo9wMZiCa9ou3fWvNensd+3c8ikFYGn25ZLbK4asrDl7yFQtUB8qXLjeMDgz50D6Tyoz3xB6sHeTP8bIFjNUNznYnl+9l7e5YNHW0vB9VuOkL8er/pRE9YD/rv3vy8XYYf8/nRiTuQRxDpMBP6JfZ52wo3y18qZomt9cOykqWRS41Y3wqwUDZn0HAqB+TYJjus0JPPyirJsxI4e5tnKKcF+GJC5wsKcQMVUdFOuZne63Xsl4We32qPzZzkKpl3oxthVoPp2OSFbsciVIHDEM912GeCg5FSqxrIMqzGpXDriyo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert unix_stream_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #10) - Fix subject to refer to unix_stream_sendpage() not udp_sendpage(). net/unix/af_unix.c | 134 +++------------------------------------------ 1 file changed, 7 insertions(+), 127 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 976bc1c5e11b..115436ce1f8a 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1839,24 +1839,6 @@ static void maybe_add_creds(struct sk_buff *skb, const struct socket *sock, } } -static int maybe_init_creds(struct scm_cookie *scm, - struct socket *socket, - const struct sock *other) -{ - int err; - struct msghdr msg = { .msg_controllen = 0 }; - - err = scm_send(socket, &msg, scm, false); - if (err) - return err; - - if (unix_passcred_enabled(socket, other)) { - scm->pid = get_pid(task_tgid(current)); - current_uid_gid(&scm->creds.uid, &scm->creds.gid); - } - return err; -} - static bool unix_skb_scm_eq(struct sk_buff *skb, struct scm_cookie *scm) { @@ -2292,117 +2274,15 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, int offset, size_t size, int flags) { - int err; - bool send_sigpipe = false; - bool init_scm = true; - struct scm_cookie scm; - struct sock *other, *sk = socket->sk; - struct sk_buff *skb, *newskb = NULL, *tail = NULL; - - if (flags & MSG_OOB) - return -EOPNOTSUPP; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; - other = unix_peer(sk); - if (!other || sk->sk_state != TCP_ESTABLISHED) - return -ENOTCONN; - - if (false) { -alloc_skb: - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - newskb = sock_alloc_send_pskb(sk, 0, 0, flags & MSG_DONTWAIT, - &err, 0); - if (!newskb) - goto err; - } - - /* we must acquire iolock as we modify already present - * skbs in the sk_receive_queue and mess with skb->len - */ - err = mutex_lock_interruptible(&unix_sk(other)->iolock); - if (err) { - err = flags & MSG_DONTWAIT ? -EAGAIN : -ERESTARTSYS; - goto err; - } - - if (sk->sk_shutdown & SEND_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_unlock; - } - - unix_state_lock(other); + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - if (sock_flag(other, SOCK_DEAD) || - other->sk_shutdown & RCV_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_state_unlock; - } - - if (init_scm) { - err = maybe_init_creds(&scm, socket, other); - if (err) - goto err_state_unlock; - init_scm = false; - } - - skb = skb_peek_tail(&other->sk_receive_queue); - if (tail && tail == skb) { - skb = newskb; - } else if (!skb || !unix_skb_scm_eq(skb, &scm)) { - if (newskb) { - skb = newskb; - } else { - tail = skb; - goto alloc_skb; - } - } else if (newskb) { - /* this is fast path, we don't necessarily need to - * call to kfree_skb even though with newskb == NULL - * this - does no harm - */ - consume_skb(newskb); - newskb = NULL; - } - - if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { - tail = skb; - goto alloc_skb; - } - - skb->len += size; - skb->data_len += size; - skb->truesize += size; - refcount_add(size, &sk->sk_wmem_alloc); - - if (newskb) { - err = unix_scm_to_skb(&scm, skb, false); - if (err) - goto err_state_unlock; - spin_lock(&other->sk_receive_queue.lock); - __skb_queue_tail(&other->sk_receive_queue, newskb); - spin_unlock(&other->sk_receive_queue.lock); - } - - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - - other->sk_data_ready(other); - scm_destroy(&scm); - return size; - -err_state_unlock: - unix_state_unlock(other); -err_unlock: - mutex_unlock(&unix_sk(other)->iolock); -err: - kfree_skb(newskb); - if (send_sigpipe && !(flags & MSG_NOSIGNAL)) - send_sig(SIGPIPE, current, 0); - if (!init_scm) - scm_destroy(&scm); - return err; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return unix_stream_sendmsg(socket, &msg, size); } static int unix_seqpacket_sendmsg(struct socket *sock, struct msghdr *msg,