From patchwork Thu May 18 11:34:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C817DC77B7A for ; Thu, 18 May 2023 11:35:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BA2E90000A; Thu, 18 May 2023 07:35:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66A8F900003; Thu, 18 May 2023 07:35:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50BF290000A; Thu, 18 May 2023 07:35:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 412AF900003 for ; Thu, 18 May 2023 07:35:10 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1400580548 for ; Thu, 18 May 2023 11:35:10 +0000 (UTC) X-FDA: 80803169580.27.93E84DD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf26.hostedemail.com (Postfix) with ESMTP id 3ED7A14000F for ; Thu, 18 May 2023 11:35:08 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bJxBtKtF; spf=pass (imf26.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409708; a=rsa-sha256; cv=none; b=QJjuGTjTk4vLeDNtXI/mfj1Mmyr/z/iK6Rdc0XN3+Ui22si3FmL9HwvJhRrafmDMecwkx4 xpnITuPSIrXsf6YTmsYJElT0jDW/l3PThvq1XSgmOLoPU+mpSKVw4HCio2fpKzbGf5ep5M q1j4el4KN7u4E4lWSu/8q3TpCpSyS7c= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bJxBtKtF; spf=pass (imf26.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409708; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zo53+MxtcExCaAvP1E3d2cbKIX8PzAGUnvJuN4UlUmE=; b=bC2IxLyYRcSztwU0d3YQ/mvVsStIriqP74HzzM6BiTjRbUrseijROKXPPDLYOtaWKAz7Uo icJLuOM15mbxU6HQP4hoSpmUw2hm3RUWaG19KoTCAimt1lEdOyZmXc2IZ9Bb2rGH70eiCV Y1O4Tn6w9lOJCFA0MjNEzzuqb4fNrwQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zo53+MxtcExCaAvP1E3d2cbKIX8PzAGUnvJuN4UlUmE=; b=bJxBtKtF+FyWlVLIQaYo5SnmIp0knfvSmXfAPeUPAUQ/cIeiyMqerVAzF/6HJ8bWyToJRq 2wcq5dMoK746qyoDavsj5VG0LMhCuSVMKHKKWpyfbWNzkLqvuPcswxTcVrL1wIy4xrpEVH ae8RK+zh0DemSd2ZRxEiv1a221rBy20= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-122-Q1oqsoVdOXek48f82BcBpA-1; Thu, 18 May 2023 07:35:03 -0400 X-MC-Unique: Q1oqsoVdOXek48f82BcBpA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0B82786C60A; Thu, 18 May 2023 11:35:02 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id A3A93492B01; Thu, 18 May 2023 11:34:58 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Willem de Bruijn , io-uring@vger.kernel.org Subject: [PATCH net-next v8 01/16] net: Declare MSG_SPLICE_PAGES internal sendmsg() flag Date: Thu, 18 May 2023 12:34:38 +0100 Message-Id: <20230518113453.1350757-2-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3ED7A14000F X-Stat-Signature: azbt6tfxrrtutka6498xyi1o1cxg667m X-HE-Tag: 1684409708-201422 X-HE-Meta: U2FsdGVkX1/0tjCzHZm77BwCHOzj3jiauuh4gJT4VttQNkAxym3wBtkl98uq1C5ObPAnNeygpJrstgF9pUrnMA7UaSM81NX9gsPvawG77eW1MRJeORYJsHRDv0Dwm97NHyiP/rsI8Ej5hcIZ0aAQyxaWlvnNhACPyhNrLXZcqHkj8j3XXB9Uq/XKGdrsgB6EPB+F+KkH3fNT/16U8gP2rJxJqKP+IO6VhsryaQDXJTJq7I4M8vWci9B3gN1qcbSo8i5EaqaoI4ScJAbNAMZciBdIpJtGRKd9MutFnMRQUFaCffVjDUPc8p68YMo5Lui44WGlt/ti+AChUIG6wLiJY8MVwM8/a572m+yeAbj45AAVg7XK/VbaNpjlot0XcjvaYOt9agdTNbf3qsDc7uKpVvVFu0n5xbCTvDA/l77yo5ohmaD3Y+wOs9rAWuxvbFYbsn0iVwwiTBKzTz3E3olpsfuDekAM/10UTx/hhn3BQArm725jM6mu3SGIIRgTc28/nYFJcfGS2iqEHDoGY3TJ2gH4M9xGsP04ChMEtyc2W5L0XCFXL7Tl9YqkGVFTZXKLUee+sTf6dEaBzhtcvIo+YBXfJT+G7ScaIyW6D8f6IWd+l/DDqVsvegRwQhyvk/0TwZMzGRh7VX+Qh8dfOa/hX7ojVZzeEF9zojKTmd1FRF5KpG5BajWFtRERGC2mLflRpBjIctZ7ySb3gujzAIgMwaGZuzjofJnDCw4Cuk7q0SfhH5dDIkpjFh+Fjs2ytpjWBe7Ygq73mqO/MutZB+k3mxEzrbAJSn8WvUP3a+e18r0CSXW/YaiHkpyV/AlFZsKbMKuahCSzOPmDhDC7SonzJLa83RF44ACimkbt7A9BaBX+93DZEbuvk1VhH77/TIFOc68BDH7ag+W4xKIcF2ahHA23bdN0x3NV3zBhY2uczUJGxwSescPmt+vi0pjNmjRBbDKGvQE9J7UZxYh+HXn 4sPX5urw x1fXF0im3rV1JFC1dTe7ybNASZLnaipn2AYtSczwYag1OmqVEVCOF+qRehshP6AWZeXS9opvwImix1pd1K8qKjN2JVtq58wjwzoooG/bD1cB5/LQT0sYlw5M91Hen36ETEdpnUPWZA1k1JhgOxn8X2A6soSGj+PmW8kwn2OlAFlSfDZ2Zw7onRXyuxxqeL3cbNExvpPtkSF8ckMWm5c3V8xXoJWFayYcSMZfCfrb6pqkoeHe4qh2sGwvWCrrnhkB42Y+nYXqsZoTP+h408e1CWl9Y6nbjsrHun1kT/B+KfmD1Sxhe4arBw75azZbhhOLmQSoQFCj1IVOcuDe1kDvmTp9KTGnbObTRM6HNsVkCoECcZCxDniNzrAFC2EXlGN7CURaXuno7BJiTnKLrWljEUNJS2FwnfyXWR94qc0lv6tF2PwkVPyYIs5Qy756Fx57urCiuQLhzaYIt+UYLiRN6IWLYvXTBNh5A1zYDYxIaMijEMocb18hUqVnIhQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Declare MSG_SPLICE_PAGES, an internal sendmsg() flag, that hints to a network protocol that it should splice pages from the source iterator rather than copying the data if it can. This flag is added to a list that is cleared by sendmsg syscalls on entry. This is intended as a replacement for the ->sendpage() op, allowing a way to splice in several multipage folios in one go. Signed-off-by: David Howells Reviewed-by: Willem de Bruijn cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: io-uring@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #7) - In ____sys_sendmsg(), clear internal flags before setting msg_flags. - Clear internal flags in uring io_send{,_zc}(). include/linux/socket.h | 3 +++ io_uring/net.c | 2 ++ net/socket.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/include/linux/socket.h b/include/linux/socket.h index 13c3a237b9c9..bd1cc3238851 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -327,6 +327,7 @@ struct ucred { */ #define MSG_ZEROCOPY 0x4000000 /* Use user data in kernel path */ +#define MSG_SPLICE_PAGES 0x8000000 /* Splice the pages from the iterator in sendmsg() */ #define MSG_FASTOPEN 0x20000000 /* Send data in TCP SYN */ #define MSG_CMSG_CLOEXEC 0x40000000 /* Set close_on_exec for file descriptor received through @@ -337,6 +338,8 @@ struct ucred { #define MSG_CMSG_COMPAT 0 /* We never have 32 bit fixups */ #endif +/* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ +#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 diff --git a/io_uring/net.c b/io_uring/net.c index 89e839013837..f7cbb3c7a575 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -389,6 +389,7 @@ int io_send(struct io_kiocb *req, unsigned int issue_flags) if (flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = flags; ret = sock_sendmsg(sock, &msg); if (ret < min_ret) { @@ -1136,6 +1137,7 @@ int io_send_zc(struct io_kiocb *req, unsigned int issue_flags) msg_flags |= MSG_DONTWAIT; if (msg_flags & MSG_WAITALL) min_ret = iov_iter_count(&msg.msg_iter); + msg_flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg.msg_flags = msg_flags; msg.msg_ubuf = &io_notif_to_data(zc->notif)->uarg; diff --git a/net/socket.c b/net/socket.c index b7e01d0fe082..3df96e9ba4e2 100644 --- a/net/socket.c +++ b/net/socket.c @@ -2138,6 +2138,7 @@ int __sys_sendto(int fd, void __user *buff, size_t len, unsigned int flags, msg.msg_name = (struct sockaddr *)&address; msg.msg_namelen = addr_len; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; if (sock->file->f_flags & O_NONBLOCK) flags |= MSG_DONTWAIT; msg.msg_flags = flags; @@ -2483,6 +2484,7 @@ static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys, msg_sys->msg_control = ctl_buf; msg_sys->msg_control_is_user = false; } + flags &= ~MSG_INTERNAL_SENDMSG_FLAGS; msg_sys->msg_flags = flags; if (sock->file->f_flags & O_NONBLOCK) From patchwork Thu May 18 11:34:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68B3CC77B7D for ; Thu, 18 May 2023 11:35:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 079D990000B; Thu, 18 May 2023 07:35:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 028C7900003; Thu, 18 May 2023 07:35:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E32EF90000B; Thu, 18 May 2023 07:35:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D620C900003 for ; Thu, 18 May 2023 07:35:21 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B6293C0792 for ; Thu, 18 May 2023 11:35:21 +0000 (UTC) X-FDA: 80803170042.20.C578EB7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id DBBF740012 for ; Thu, 18 May 2023 11:35:19 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="OaTwg/8N"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409719; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Oms7mbQShTpY01zvjwYgDwHWlSZDYU/BDFTf+CWwguM=; b=f9jt4yuz5flfBa1vNuFvF0MHkP+MDAZrAsUY/P10cdqTkZjPAVCP4cJxsLw8acSaj/E1Pg t7LvBe5N6/4jo+IP9gmXekxmnxHtjgFchfevN0p5/jSFtRK59Pc3TNNb1zxSieFyIeoBph /1lYMMwx2e7avLUOYkmsNkAh35yphCY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="OaTwg/8N"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409719; a=rsa-sha256; cv=none; b=o0QQTgknSqegr0vJtdpyHasOskal0fYdalNdeS8Y/Ta7NhKhK0TKjs6prQvHsD4Z+FVaP8 Ag2gtnBAjSBZ9VjfRG33rk7gQ2B9sEXAkWl6bwtnJ962FB/UpsJgaKZly9nWTywFegYcfh Xcf0gHkgl6OyczkqloqXvhevWErMshU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Oms7mbQShTpY01zvjwYgDwHWlSZDYU/BDFTf+CWwguM=; b=OaTwg/8N3CjytwKTtML90kMQBqjhUGI6T3UVofqrezkkBQkR6NqqGNEIoOk1hw2OLqiGoe W19iBVyHsaqB2wxqpzx1ibykZttzhai8p0AqRirwqlAvGZE41HMy8rhBcVxeQgJRX9EYan 0xKuCdhC4sq4hi+QkaDgC0VFMjAKsTk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-61-ELmJWgkyMLiJ2SkyRnHWsQ-1; Thu, 18 May 2023 07:35:13 -0400 X-MC-Unique: ELmJWgkyMLiJ2SkyRnHWsQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9157910146FC; Thu, 18 May 2023 11:35:12 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB5E214171C0; Thu, 18 May 2023 11:35:06 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 02/16] net: Pass max frags into skb_append_pagefrags() Date: Thu, 18 May 2023 12:34:39 +0100 Message-Id: <20230518113453.1350757-3-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Stat-Signature: nshtwogaxjsgopkfsemqoqysuwtce33a X-Rspam-User: X-Rspamd-Queue-Id: DBBF740012 X-Rspamd-Server: rspam07 X-HE-Tag: 1684409719-906254 X-HE-Meta: U2FsdGVkX1+EiAjjPpFXm1uRSAJWeZeStiseMRDKYCmnf7swpmsoS/4EVmLolouPaob5ts/XrNxodz/n749owkWW3+o8H5czoctHp37i8IDXZMYU8wbjDnEMJkAzJKvAEWfzfmLNcIjGDoGE5LD2RST4CeZQ2N/m4xGVdcqNo5af+rsKpwrBB35Qn+0I27RTUs1dChQYIyXi8zw6mKpgeB/QwpyyKsHj80QRnyF79pVQ1avUepCTkhLZzxhYkJo4gRkhmAc/Cv6bdmymyCY6NHa/vpEP8FBH1YL5rTk3DIRkj5npjgnHp1/pfkxfZCcrNn0iIs4GfuRVtBmtcrbVNzTjeJcheqPMZ06Dt0x+Ko4LoC2MM5AA0OY+7UwOuiimwztGnbL05CJEoZa32j5KcrzRi5Fq0kOL4nO0s5D6t2OhUpU+/h2qEwXc+7M46s1Xa1zaywqnPcHcw8IxH9q1L6XPfCxaZ2DYIUkjwyrMPFKfHiDeqIDsYMEpy+30lHvL6mKWiXyEIlfsxoqHapiM0mo2K8rbFlMtef0716bHnxwvJCRaQR91qQ2NH5ja0BilYqjzQR3BXoZvZ5JigjFSYk97JDP+7OguZLaL+JkAnNMtyLVOCkniXQIz8Nle6AHogunqldNMF/Ubpj16PsrYNNCtvUPyDuLsxaZ1iW+XWhwL+G/QGz3RflW1QfaxIKxHrfSdK0a/kSfCgLXRzqBE/oo2/eg898jv5418jbAX6g6Tw8KQmo405PDiMHFlJfb2MZ3+8V1cPgRtjdfZsR7cvFWQhLpoDN8hs3jrInGOKIcsE33AsrSARWCLRQG7UXTzjy3DIVDVh0OPYsOBa0posDDOY13/5yzIrj3ZVISi7MGfXgYh1pslA6aRivEC6VBJrQzqO8C2coL0rztvOrOW1vxkVoQjSgX3Zw6N37H4PJKa5EM7eWl24CRGzf/IVFQM/R3COfGFqxOuKK1lGja TXF8r64E CNq3mYVNTPwRbA6Gr0wFI1hmfi9jVGez4NrHP+OHan75lYf5a+rhKVGGj7X8hHsoeFwXr2cYrA6NJhexMOpvgXeicDX5Luwy8Z5kZxkB9Li40itTucIIku86l/43CKktkqxj3CEl7BGTAiGdwhhWoh9QP8dz8shSL5AXi/665hJzXJutiiounuvCKNufmJgUIizLpQNcOhKpkQVOPfXQvvc/u5tfJhSjxKl93Tht02TIbKEX+amMyxACHG65Bi5TmPSD9jAw1agSAA+Ufxnoita29d50r9AcWO9q8ibHGKjgFTHHcWocg1p0CWtPgPpjIBhNCf/S5kU7oV1MkPreVkc42+nPluqrq9pOu/CUE7SBdS0J6wKEVt+374Cpffl9i19UWy2W3rf845ddyUSdYbYOQPxBt/xrKuIIdehJNMhfxKgSUMHOr7e7dCmf2pDsUGOlmytKbyF8Lbf7AUQ1H+wrUYRc9XkVCgh7ubRuJXoXeLpzLyDb12ufWiRL9TxPvFpWmq+c764tHvbqxLUV6kqv0xncVb6pQ0bbj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pass the maximum number of fragments into skb_append_pagefrags() rather than using MAX_SKB_FRAGS so that it can be used from code that wants to specify sysctl_max_skb_frags. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 2 +- net/core/skbuff.c | 4 ++-- net/ipv4/ip_output.c | 3 ++- net/unix/af_unix.c | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 00e8c435fa1a..4c0ad48e38ca 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1383,7 +1383,7 @@ static inline int skb_pad(struct sk_buff *skb, int pad) #define dev_kfree_skb(a) consume_skb(a) int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size); + int offset, size_t size, size_t max_frags); struct skb_seq_state { __u32 lower_offset; diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6724a84ebb09..7f53dcb26ad3 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -4188,13 +4188,13 @@ unsigned int skb_find_text(struct sk_buff *skb, unsigned int from, EXPORT_SYMBOL(skb_find_text); int skb_append_pagefrags(struct sk_buff *skb, struct page *page, - int offset, size_t size) + int offset, size_t size, size_t max_frags) { int i = skb_shinfo(skb)->nr_frags; if (skb_can_coalesce(skb, i, page, offset)) { skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], size); - } else if (i < MAX_SKB_FRAGS) { + } else if (i < max_frags) { skb_zcopy_downgrade_managed(skb); get_page(page); skb_fill_page_desc_noacc(skb, i, page, offset, size); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 61892268e8a6..52fc840898d8 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1450,7 +1450,8 @@ ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, if (len > size) len = size; - if (skb_append_pagefrags(skb, page, offset, len)) { + if (skb_append_pagefrags(skb, page, offset, len, + MAX_SKB_FRAGS)) { err = -EMSGSIZE; goto error; } diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index cc695c9f09ec..dd55506b4632 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2349,7 +2349,7 @@ static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, newskb = NULL; } - if (skb_append_pagefrags(skb, page, offset, size)) { + if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { tail = skb; goto alloc_skb; } From patchwork Thu May 18 11:34:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2744C77B7A for ; Thu, 18 May 2023 11:35:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9056D90000C; Thu, 18 May 2023 07:35:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B485900003; Thu, 18 May 2023 07:35:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7559290000C; Thu, 18 May 2023 07:35:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 667A9900003 for ; Thu, 18 May 2023 07:35:25 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 435E640240 for ; Thu, 18 May 2023 11:35:25 +0000 (UTC) X-FDA: 80803170210.21.9DF2D59 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 82FFA1C000F for ; Thu, 18 May 2023 11:35:23 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WQq4Va8D; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jACcDubg9VuPG9obSLLRy2+X1qFqLp435QyVtyfFY1I=; b=l6l1B8AfvLtWwTztsZyBATR9Ln9+SfnTkxWtALQcgXHYhF44au9URE2ow4dYw0cBhoRlnv eOXUpeffznnj8l7PP258Jj6K05xqyOaSmVsMqYFB5PJeSpPEEiPn7MeEDlMrMKm7Q5iBy4 4jkfDxuc97T372cLpGVQyR5DDKaE8Xg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WQq4Va8D; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409723; a=rsa-sha256; cv=none; b=NQCvCrCi5idANzPZNdJj20RqHwwbGk74OaiLB6p5nYUaej+XNGjbpnu3o3Y2lFJNrQPvUh 0BLJV2Xw5/RChTKEZ77Sdu+1tD+ME4uchqf35cn8MzTm458l3ukC/51TFGqqKsSRoWmaBh BnvyGtcW0q3sjmQx5Wt/FRESUL4IM7I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409722; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jACcDubg9VuPG9obSLLRy2+X1qFqLp435QyVtyfFY1I=; b=WQq4Va8DhIAOlHOuLNgCg+xiRLa7lUYGg48EsLxCbfBlov7ZwbzuUnxGgtkAsZRccRa/A9 B/qjnj6EqmD31+usONJFL3I8vo/eBn6/yi1Lvu/yR+skVCADHC1CJaRN/c1MxwTeJw7RZE 1Ojy7qsKJZGs1/BUFk9zM1IXOqCtorw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-70-TOUdyjn5N_CVtCdOeDCHHg-1; Thu, 18 May 2023 07:35:16 -0400 X-MC-Unique: TOUdyjn5N_CVtCdOeDCHHg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A16C71805221; Thu, 18 May 2023 11:35:15 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7B00414171C0; Thu, 18 May 2023 11:35:13 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 03/16] net: Add a function to splice pages into an skbuff for MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:40 +0100 Message-Id: <20230518113453.1350757-4-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Stat-Signature: eho85ramhnwe5361gtu75mg88rdfs8dx X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 82FFA1C000F X-Rspam-User: X-HE-Tag: 1684409723-761374 X-HE-Meta: U2FsdGVkX1+xtq/UnWdxpRn99ROAYAIy6nH1HNth7hqhi/DG4T0iRWAbZz+wqRse+eCFB+icxDyfhBKa6pRQx/AVlsn3gaNeDECUeDxQaxYTy++dfwQqaNxrjLNWgAPytF+leIIcEx7VMGH/x53zKizD3tqU0fgGm7M6+eiVwK0lPVZxenDGOOOtAho1Nlp0JpExiieABWaiS3WBO7SzZ0dLwkDneRjsYvb+i4lpxVzz6n21kLCp4zKgXv0TizfF/Wb9FIn1o4EVDO4kffMnTCiw0IG3NmJUpKlWlN87XicaUDHYkwamx52wuaCO1Mm/IQiNU/ikauJNPdLeHx3C0HLOi0vewJQaeYNbpNM9v8Er/PYg+oXVx/ZFbv76alermUiy0HY05SmQIDpMbRJKk80ATayE1eXKMufvcCDO2bvdh8eRgmgRgUVah+M0QXiFUjhDFB29EtaSiNHZMdySgMA6bjVVriiLIGN8PFkFhIAUWGEGbWT0Lfyf3nmZYIk5jtR4bUl+KcUC5L7bfKmDKksz4jXsIa6xUXUM3pLXWs0FrSXXNaqme/vvnsa5O1R5CvzHSkpuaQKFFHLiC23Z3HgzXTlAb74KsMhXsoDQx4zzH4I5BZKA+OOOGtQAgfgC2WnMYROWF2K2jxouGHOLla4U/MWsAojvh3uDSwPrOLr5PE7kJ+7P8HMSco+zsUWIEd9iQ6uDqqFyKCEnRmAGf0nravqGXXBkWAEUeXeGKU0PnSchSj5dq9x6id41G2JpkUPQVAuMorFFUR5xaasROZzhFBj6+qk+ZKBlPX6Kg1jArxzNuAIjDGaM/ysOzEuOvBxgGwjgNtdCeQ4NaUxrWN7GVxJFC7M3GCDZ4P9WnN/qZzIcp1FJmU/zM8+Xc5ugyQ0Wygw/zP2ePDLPkVoFShcVL0i2NERU7xZ+p9BivRo1T9a/O4/BC6HAx8rca2wcHI0BXjUT+QNGZfoc700 aGMhcbJZ akkcBo0ttWJ9s30jfqpYRa5kEDA0pa4oqyGrAuPqDkTA+R6yFLZjvklW/70C6rzIniufCwXnFsc4vb4bqpSPkN3iaPXh0vAHPaZEaRPtj7finFXxoxoos37pQXZ4cB9GeWCRYNcVM3mv+rTPmwDfpwVWCLhOES+sW2tNFO61nYY9uCogRCKmIHhY8VZPTqSQ21KkcyTbU/IPGECUzamGlw/4iByQhS0QUCADbztCpMU7MEU+prbruSP/vZIA/FVImpADho7S8RveMDnTA6N0fH4ZdfWyoZaUcIqdmotYSfPSY+DkFs1/IWiCl0qr2m2qxvEg2KKhqH0BV2gT5YXWoDcv2LpzMdKPKC51ESd8NsVGn2wQefZ1uik040l9jOfGO0fH6FoPV22DxcKdvQtsfYhFwBtvo0Ffm77fcRoIAErTDctyq66AiqB4tUsC7QWNw8R0HqBBPp0SF7JRxLYs2Bl/jDW5X0oQ2QJX+DUArGZSyES1zJXYyOGEcRpoUiuxFfk5u9+0zvZRYHu4chBZPJHMRJcUvddblZHG5RFckwnqpX4I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a function to handle MSG_SPLICE_PAGES being passed internally to sendmsg(). Pages are spliced into the given socket buffer if possible and copied in if not (e.g. they're slab pages or have a zero refcount). Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Al Viro cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #8) - Order local variables in reverse xmas tree order. - Remove duplicate coalescence check. - Warn if sendpage_ok() fails. ver #7) - Export function. - Never copy data, return -EIO if sendpage_ok() returns false. include/linux/skbuff.h | 3 ++ net/core/skbuff.c | 88 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 4c0ad48e38ca..1c5f0ac6f8c3 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5097,5 +5097,8 @@ static inline void skb_mark_for_recycle(struct sk_buff *skb) #endif } +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp); + #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7f53dcb26ad3..f4a5b51aed22 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6892,3 +6892,91 @@ nodefer: __kfree_skb(skb); if (unlikely(kick) && !cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) smp_call_function_single_async(cpu, &sd->defer_csd); } + +static void skb_splice_csum_page(struct sk_buff *skb, struct page *page, + size_t offset, size_t len) +{ + const char *kaddr; + __wsum csum; + + kaddr = kmap_local_page(page); + csum = csum_partial(kaddr + offset, len, 0); + kunmap_local(kaddr); + skb->csum = csum_block_add(skb->csum, csum, skb->len); +} + +/** + * skb_splice_from_iter - Splice (or copy) pages to skbuff + * @skb: The buffer to add pages to + * @iter: Iterator representing the pages to be added + * @maxsize: Maximum amount of pages to be added + * @gfp: Allocation flags + * + * This is a common helper function for supporting MSG_SPLICE_PAGES. It + * extracts pages from an iterator and adds them to the socket buffer if + * possible, copying them to fragments if not possible (such as if they're slab + * pages). + * + * Returns the amount of data spliced/copied or -EMSGSIZE if there's + * insufficient space in the buffer to transfer anything. + */ +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp) +{ + size_t frag_limit = READ_ONCE(sysctl_max_skb_frags); + struct page *pages[8], **ppages = pages; + ssize_t spliced = 0, ret = 0; + unsigned int i; + + while (iter->count > 0) { + ssize_t space, nr; + size_t off, len; + + ret = -EMSGSIZE; + space = frag_limit - skb_shinfo(skb)->nr_frags; + if (space < 0) + break; + + /* We might be able to coalesce without increasing nr_frags */ + nr = clamp_t(size_t, space, 1, ARRAY_SIZE(pages)); + + len = iov_iter_extract_pages(iter, &ppages, maxsize, nr, 0, &off); + if (len <= 0) { + ret = len ?: -EIO; + break; + } + + i = 0; + do { + struct page *page = pages[i++]; + size_t part = min_t(size_t, PAGE_SIZE - off, len); + + ret = -EIO; + if (WARN_ON_ONCE(!sendpage_ok(page))) + goto out; + + ret = skb_append_pagefrags(skb, page, off, part, + frag_limit); + if (ret < 0) { + iov_iter_revert(iter, len); + goto out; + } + + if (skb->ip_summed == CHECKSUM_NONE) + skb_splice_csum_page(skb, page, off, part); + + off = 0; + spliced += part; + maxsize -= part; + len -= part; + } while (len > 0); + + if (maxsize <= 0) + break; + } + +out: + skb_len_add(skb, spliced); + return spliced ?: ret; +} +EXPORT_SYMBOL(skb_splice_from_iter); From patchwork Thu May 18 11:34:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52736C77B7A for ; Thu, 18 May 2023 11:35:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E34B2280001; Thu, 18 May 2023 07:35:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE4B4900003; Thu, 18 May 2023 07:35:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD3B0280001; Thu, 18 May 2023 07:35:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BDEFB900003 for ; Thu, 18 May 2023 07:35:30 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 91ABA1C75FE for ; Thu, 18 May 2023 11:35:30 +0000 (UTC) X-FDA: 80803170420.10.1A34B75 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id C9AD1180015 for ; Thu, 18 May 2023 11:35:28 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ghqrAqnD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DZXfiar5CKohUwsPymjzwnCAT+wFqXG5Kam/6y6M8pg=; b=NcNymczFHzV76SlMoIy2fPCG0LlMzlq3LlFweESijpmQUw03xOyQqURK2ShufI7nYpaKoW 9zE6HrKM1q33mG/15m0TG0aI2+yUT5bvmpucnYErHqQyTDx2Fcu/vM5cOPMIVvB8wDpmiE HBNQ6O0GA4nvlqJTZxnXpjaHv8UCdRg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ghqrAqnD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409728; a=rsa-sha256; cv=none; b=PmbJly1TVCuaV0z4NtyHKQry8tV1xsOUYKKGIyfY3zqBfUPdUnamRvMDZ5/WL+tixvfzgo 9NZIwayx4O6wBOu8wv+N5YxZSjAqdRprmzes/lXX1rsj50gKk/ETHWozmPDvJ8nmUJIGnP VppDEFoE71zGCeeNZgGEhkHnwgxlgJA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409728; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DZXfiar5CKohUwsPymjzwnCAT+wFqXG5Kam/6y6M8pg=; b=ghqrAqnDoODeY4g0YN49NXkq5gcbMXsRe3pSmd2HqixBOfzKK8Y1SXIDC6O1ZukimiUKlB 97/A4XkbXQMb2U15wv66KOjLhFUTiOxJZ0u+R0e+4pmZLLnGRm4GaPd6ZGiH3BObZ86UPz FdUOsDtRQAp8ie1Hjd2Mo/aDNUKdhWk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-206-lw-qVOlBOruNJ3l5NNmWVQ-1; Thu, 18 May 2023 07:35:21 -0400 X-MC-Unique: lw-qVOlBOruNJ3l5NNmWVQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D7B0C10146FE; Thu, 18 May 2023 11:35:20 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 78A2B492B01; Thu, 18 May 2023 11:35:16 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 04/16] tcp: Support MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:41 +0100 Message-Id: <20230518113453.1350757-5-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: b7nkz485mifw78zfxze8iatuuuzx31kc X-Rspam-User: X-Rspamd-Queue-Id: C9AD1180015 X-Rspamd-Server: rspam07 X-HE-Tag: 1684409728-290804 X-HE-Meta: U2FsdGVkX19jXBD1YULwxuivsDXbWJht/PVwr16YiodjGk+X7476cCkjqk3xvQccTwdxpUH/Lq3yft2ONdnkjqH30X0lrdihPAg3c7CHIJ8/4rwZfMVqjr6poQNEwt2Ds4PGX5QO2ME/m0w0yPY2BNdhxE5hwMC/5jDYuQVUOmHwbhrN9tV5GDSaqlMsWWDDw352gGL8ztdkiKe7wUG0oLpeqA3azxl2WytHd/QA+8TEzkm9sludzXg5RCqf+JpxSz0zdsYVBcjAFRb3CTNZdAmcWFLcGI6ekz97kVbok9wTKhRmA+3EcQ7Q1hjAnARpCL2MnK+ojZgpBF3Ftx8EuxwsAa59E5gX0tIMn9ejQiE7WrgJns0xmK0hY/57qYgHS52RP5FrFbtWWAxM1QQo1aPmzEv+gIpCTJoNnl7CXnyNNnyQ6fOYYCiHw/sF/2HCoAAgUIbUxAxvm/TAmku0P7LkondWOe46Jg22jnR2Tj9BZtGwB3LGpDsJj5gCBWtEhiHmCNfF/uTVRThzzMxTD3Ofg+UikCoG2JqviA77Xg1mLbV5TWw6GLuzlahyRzcj0PtnQkGylX40nV69qzlcY57vR2olARTJYENJispyVn7NUjW+Rd33mHKrQNxUIs0GsOj9I2nRNApeedWcrgj4DrobVASGsYdUY/pt2PW8p1ZMTA9fxkpnMU+YWAGgiZZ6qgbcKv6ctmyJUwhQ9oFAdk9/tgl6zDctgXaAF+/AnQcQ+cfURXsOse+ob0iUyyGIvz57OLj5B1E+TFprGpBjvGc6Jx3WLFS28dEavv+ZZrkqJXesNHkH6X6al9XheQskiW/Lr5/JuGavxmGZvlUIxe4MMTHSstSN24L60tSYq/QlcwLMOTPhxUuStYMRbMUgJE9nLHIUk8KrUsEUC90ZpTeyKmZzgEJUNOWqOFw/JKF66nyVR7uw5m7Qz8HVSa/r+86lhkAUe8Z9Dpr344t Thoy+Anl NSkq0+18/xfSvamgNw8bI0yqOZ3sGHVO9jSl7fa9DZiBFgsN1BH/E6mTq4F/Bc1kuCJDbY+BX4tPVfFTJknk7Jvw9trwtMJ7mv3cDxWX/xVXiBS/4XPmYeiqxXbbHxFkleVqkXoj06jiedbex4I9fB6MnwLLDb9wkVWAOF6Qrc0HmjyF1YmO7pqwky+nUi9WpGCS296G70KoC/PjdN9awfUPu2Nsio3xRqo2QcjuNxR/nlrtLUzxLv2GvbvkVcnMUzx4rDrXKet/JqhVJaEmsWAuo3w07VRw3PMfhVD1Cmm/SYMDngBt89w4RZJ+0dheYUotsBXRQGrbCX1oo6CQPIi+Si90F9b5TO9knkT1ypqYfLmo3OLFyckLGcnUdm18KZZphXXQ93E3S6xb232HsvWG5e68YMs1EANdvFmd/v+hRs19Wl2DzerUSEf2p1sraAri0LLrdZZEyQENBTfYxUs1LPsYOMb0eiBUtgsIped6sXQZw+zl3JnfYkw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TCP's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced or copied (if it cannot be spliced) from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Missed a "zc = 1" in tcp_sendmsg_locked(). ver #6) - Set zc to 0/MSG_ZEROCOPY/MSG_SPLICE_PAGES rather than 0/1/2. - Use common helper. net/ipv4/tcp.c | 43 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 4d6392c16b7a..026f483f42e3 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1223,7 +1223,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) int flags, err, copied = 0; int mss_now = 0, size_goal, copied_syn = 0; int process_backlog = 0; - bool zc = false; + int zc = 0; long timeo; flags = msg->msg_flags; @@ -1234,17 +1234,22 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (msg->msg_ubuf) { uarg = msg->msg_ubuf; net_zcopy_get(uarg); - zc = sk->sk_route_caps & NETIF_F_SG; + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; } else if (sock_flag(sk, SOCK_ZEROCOPY)) { uarg = msg_zerocopy_realloc(sk, size, skb_zcopy(skb)); if (!uarg) { err = -ENOBUFS; goto out_err; } - zc = sk->sk_route_caps & NETIF_F_SG; - if (!zc) + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_ZEROCOPY; + else uarg_to_msgzc(uarg)->zerocopy = 0; } + } else if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES) && size) { + if (sk->sk_route_caps & NETIF_F_SG) + zc = MSG_SPLICE_PAGES; } if (unlikely(flags & MSG_FASTOPEN || inet_sk(sk)->defer_connect) && @@ -1307,7 +1312,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) goto do_error; while (msg_data_left(msg)) { - int copy = 0; + ssize_t copy = 0; skb = tcp_write_queue_tail(sk); if (skb) @@ -1348,7 +1353,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (copy > msg_data_left(msg)) copy = msg_data_left(msg); - if (!zc) { + if (zc == 0) { bool merge = true; int i = skb_shinfo(skb)->nr_frags; struct page_frag *pfrag = sk_page_frag(sk); @@ -1393,7 +1398,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) page_ref_inc(pfrag->page); } pfrag->offset += copy; - } else { + } else if (zc == MSG_ZEROCOPY) { /* First append to a fragless skb builds initial * pure zerocopy skb */ @@ -1414,6 +1419,30 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) if (err < 0) goto do_error; copy = err; + } else if (zc == MSG_SPLICE_PAGES) { + /* Splice in data if we can; copy if we can't. */ + if (tcp_downgrade_zcopy_pure(sk, skb)) + goto wait_for_space; + copy = tcp_wmem_schedule(sk, copy); + if (!copy) + goto wait_for_space; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) { + if (err == -EMSGSIZE) { + tcp_mark_push(tp, skb); + goto new_segment; + } + goto do_error; + } + copy = err; + + if (!(flags & MSG_NO_SHARED_FRAGS)) + skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; + + sk_wmem_queued_add(sk, copy); + sk_mem_charge(sk, copy); } if (!copied) From patchwork Thu May 18 11:34:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE7ECC77B7A for ; Thu, 18 May 2023 11:35:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BAC1280002; Thu, 18 May 2023 07:35:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66AAE900003; Thu, 18 May 2023 07:35:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53238280002; Thu, 18 May 2023 07:35:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 42C3F900003 for ; Thu, 18 May 2023 07:35:36 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0E95A140794 for ; Thu, 18 May 2023 11:35:36 +0000 (UTC) X-FDA: 80803170672.05.AD257D7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 3F57540013 for ; Thu, 18 May 2023 11:35:34 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JTCf32Hh; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409734; a=rsa-sha256; cv=none; b=b+9hSwa6AwU5kYEFkj4zTu3VF1fZTU9P1CjRzHU3GtdyK14irCwG4QBW/S2q2BS7v+gjgs /ZxHv25MaB4Hkafkzup0adAmPHWnizSGofjKR7vw12llwBsrru3+YjgC6kGMQZwAW1ilte 4MS2yXbrNRUhjT3qZ9nH7DweGL6ot9c= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JTCf32Hh; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409734; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9s33rFBaxABfh3fue+YSREMQMkxGRapWweS5v7j7klU=; b=iPel1Kb9CX8Qj29p1MTlfKrkV6EIET5ufKWpLGjPtP8pTV1QGoTwpBw9kqagdld8nusamx GN9h9b3fGvluRYgMDpEVVlS+g42WSlLYTzZcMZl1CswY89+REAp1YPbIuFJn8D0dDfwL35 VlLPX4OXy5FL8hD+7PtKk/tTzM+X4J4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9s33rFBaxABfh3fue+YSREMQMkxGRapWweS5v7j7klU=; b=JTCf32HhcCcHVGVyElnSyKNUXxqRjZd1pOCWgMr7PHGKjVIMcOYqfW1hTSSm/jhL34LgtX SoymDxy6f/BmzPBwyydI+Cgq+xSAV7olMb/k0CsIZh4Zl/lUAt31HNPPpo+4V48XDqBjBR piL0yqTxkqwHJmDwXstXIwxTK1rWxAw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-474-w1z0J2L2NUqr5n53vZ04mA-1; Thu, 18 May 2023 07:35:26 -0400 X-MC-Unique: w1z0J2L2NUqr5n53vZ04mA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 75BFF86C60A; Thu, 18 May 2023 11:35:25 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44FBAC15BA0; Thu, 18 May 2023 11:35:22 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 05/16] tcp: Convert do_tcp_sendpages() to use MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:42 +0100 Message-Id: <20230518113453.1350757-6-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Stat-Signature: 3tp8f11jxjr38mbztnoc48z8qosuoptq X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3F57540013 X-HE-Tag: 1684409734-934298 X-HE-Meta: U2FsdGVkX1+/i5/ABr5PZDuWEaTN1i0+nEgzzS8JRX7BeB4qmWOvKcFbgg8MVGn6t10C7jBt2iqdWWjTzfpcGCtVRf2McMwYdtoj4qRXBXWtCLf6VWwLovrYtYpqPXFqDXdcB8XMWv4OvKc6PlFIn0d1uJo9H5BWLdnT2MYGovfPBlQHMcp1BSBu8er5aU/vWMyZQRgRYjnDR2b2fhk1SfWvYPOKrELL9RxzfJnMmlNRO75kexPZ+h1B3aBQ5fDgnRve8GJWE+1zMTtnOndP/m/dEihcHGLm4sjR5eGzz2D/tjGjOK5nZHf8irmaJwfi50Q3jM9tAcri1KVvvwtAA+X5ZLV020U56kgZ1as14RZ3d0vzTDrSj29OypVlh0yPabAmdgLSWwR+hO8jrkIpFG9USva7MrdusOB/K6CWpz3ZGxYjrYgw1nwujtRv3OdV8/7/FHsACRHSYdldhAjM7sBA+9ljyh4DbQ2X+LBAjn8FCadcgIl5XDmOWIxZWZUZUyeRa7iIQwRiVGuunBIoMdcyG/eSUUiYCxizgIXME4UXbVhzXQgR+I8prJ7ZdkZJ9B/ImeH2CwMwdIS3MnVpNur9lA4Li3De97Ebuoi8VIPjImYIPOYS5J1D7kUTEI6WaThMctRv+wRXIO47IOejYTlgjP7C/lyif3Jmh7iqwTvXXXmEfmMtOga9B87XPUPM1QAD/Zm4EUA0tVdF5Rz6XqZj1lIBPyh/GeKIIJaPoJ8cOHWCd4Y76OPCUdZf669gpgjPvTD5Y2iUln8OXXPQzbYhrdK/BlFiJZ9lBpCip8w9grGnPstWGKPceq8hm3nTYFQMBpFpXZ7s1bsk19vBeU9CKX0x4P1IE3Olr3Qf8XzVToJh5RrsnNlP/NagQ9uWpZo9Fi5OykkYJ0ff8pudHkPSWSmCZshI9bO8uvkDFLLSrcmiGRkgsEHJ9GfLise9DIxzmLCUmIM6Lb80++j u3Utp5sE y5BP7IB2mhyhAAA8N7PS4umm4HfT3XAb7y/FJKsnuaLm6ti66vpKuJsb1qOp4nigtIfNaInchWe+Brgk9G4Kkg47rOg1xh7VvDE2W43B3uB8v+Dy1p0QZ4M++xkqBX6P35gm+52Pd+yjoxaZtTxpnfizLRF9Zh0irQ0hZS9VA86+rGOpfR6kedT/n5UwUsKHHeCtjDMyAKUXpIXvrd9ClRGjjs94LMVJ4HvZKKji+qIGnNAMQn3RddyyyqZGiTCPR7jgjD7IuR9TpbGf4SXY1K+yL+JUkkXASRo9i0TwPD+iefx7WKxcUdEj/ffJn0jACeimxSu4qFJ13IkPLQc/KJmCAzujX8iGjayIR6GBjWXGPJj9zvi4bWp2Aw4Xe/vA1bLT3rVYrCp2/NjXpBBlKztluxn+XUQRZ3YWLwHxJd/eDG6/6XU15JYa1ManzgN7dpRokZCbBHd89Oc4l3jdyLihdZ8XuP2BXW+imvgyuz9Jkp3bvsTvIdwA6vA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert do_tcp_sendpages() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. do_tcp_sendpages() can then be inlined in subsequent patches into its callers. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/ipv4/tcp.c | 158 +++---------------------------------------------- 1 file changed, 7 insertions(+), 151 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 026f483f42e3..28231e503af9 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,163 +974,19 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags, - struct page *page, int offset, size_t *size) -{ - struct sk_buff *skb = tcp_write_queue_tail(sk); - struct tcp_sock *tp = tcp_sk(sk); - bool can_coalesce; - int copy, i; - - if (!skb || (copy = size_goal - skb->len) <= 0 || - !tcp_skb_can_collapse_to(skb)) { -new_segment: - if (!sk_stream_memory_free(sk)) - return NULL; - - skb = tcp_stream_alloc_skb(sk, 0, sk->sk_allocation, - tcp_rtx_and_write_queues_empty(sk)); - if (!skb) - return NULL; - -#ifdef CONFIG_TLS_DEVICE - skb->decrypted = !!(flags & MSG_SENDPAGE_DECRYPTED); -#endif - tcp_skb_entail(sk, skb); - copy = size_goal; - } - - if (copy > *size) - copy = *size; - - i = skb_shinfo(skb)->nr_frags; - can_coalesce = skb_can_coalesce(skb, i, page, offset); - if (!can_coalesce && i >= READ_ONCE(sysctl_max_skb_frags)) { - tcp_mark_push(tp, skb); - goto new_segment; - } - if (tcp_downgrade_zcopy_pure(sk, skb)) - return NULL; - - copy = tcp_wmem_schedule(sk, copy); - if (!copy) - return NULL; - - if (can_coalesce) { - skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); - } else { - get_page(page); - skb_fill_page_desc_noacc(skb, i, page, offset, copy); - } - - if (!(flags & MSG_NO_SHARED_FRAGS)) - skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; - - skb->len += copy; - skb->data_len += copy; - skb->truesize += copy; - sk_wmem_queued_add(sk, copy); - sk_mem_charge(sk, copy); - WRITE_ONCE(tp->write_seq, tp->write_seq + copy); - TCP_SKB_CB(skb)->end_seq += copy; - tcp_skb_pcount_set(skb, 0); - - *size = copy; - return skb; -} - ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tcp_sock *tp = tcp_sk(sk); - int mss_now, size_goal; - int err; - ssize_t copied; - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - - if (IS_ENABLED(CONFIG_DEBUG_VM) && - WARN_ONCE(!sendpage_ok(page), - "page must not be a Slab one and have page_count > 0")) - return -EINVAL; - - /* Wait for a connection to finish. One exception is TCP Fast Open - * (passive side) where data is allowed to be sent before a connection - * is fully established. - */ - if (((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) && - !tcp_passive_fastopen(sk)) { - err = sk_stream_wait_connect(sk, &timeo); - if (err != 0) - goto out_err; - } + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); - mss_now = tcp_send_mss(sk, &size_goal, flags); - copied = 0; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - err = -EPIPE; - if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)) - goto out_err; - - while (size > 0) { - struct sk_buff *skb; - size_t copy = size; - - skb = tcp_build_frag(sk, size_goal, flags, page, offset, ©); - if (!skb) - goto wait_for_space; - - if (!copied) - TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH; - - copied += copy; - offset += copy; - size -= copy; - if (!size) - goto out; - - if (skb->len < size_goal || (flags & MSG_OOB)) - continue; - - if (forced_push(tp)) { - tcp_mark_push(tp, skb); - __tcp_push_pending_frames(sk, mss_now, TCP_NAGLE_PUSH); - } else if (skb == tcp_send_head(sk)) - tcp_push_one(sk, mss_now); - continue; - -wait_for_space: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); - tcp_push(sk, flags & ~MSG_MORE, mss_now, - TCP_NAGLE_PUSH, size_goal); - - err = sk_stream_wait_memory(sk, &timeo); - if (err != 0) - goto do_error; - - mss_now = tcp_send_mss(sk, &size_goal, flags); - } - -out: - if (copied) { - tcp_tx_timestamp(sk, sk->sk_tsflags); - if (!(flags & MSG_SENDPAGE_NOTLAST)) - tcp_push(sk, flags, mss_now, tp->nonagle, size_goal); - } - return copied; - -do_error: - tcp_remove_empty_skb(sk); - if (copied) - goto out; -out_err: - /* make sure we wake any epoll edge trigger waiter */ - if (unlikely(tcp_rtx_and_write_queues_empty(sk) && err == -EAGAIN)) { - sk->sk_write_space(sk); - tcp_chrono_stop(sk, TCP_CHRONO_SNDBUF_LIMITED); - } - return sk_stream_error(sk, flags, err); + return tcp_sendmsg_locked(sk, &msg, size); } EXPORT_SYMBOL_GPL(do_tcp_sendpages); From patchwork Thu May 18 11:34:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75963C7EE24 for ; Thu, 18 May 2023 11:35:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0EDEA280003; Thu, 18 May 2023 07:35:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 09EC1900003; Thu, 18 May 2023 07:35:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E812690000D; Thu, 18 May 2023 07:35:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D4441900003 for ; Thu, 18 May 2023 07:35:38 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A5B3B1A0814 for ; Thu, 18 May 2023 11:35:38 +0000 (UTC) X-FDA: 80803170756.13.B622549 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id AC7DC40010 for ; Thu, 18 May 2023 11:35:36 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=a5A25vU4; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409736; a=rsa-sha256; cv=none; b=dxHd+Zd5F96QC7Scp836gTRtvFVEZSBRwdTTwxoe1h6/x/Mp4/BhIx9sREMb0V6ZxkvIIC f1YY1CMMC0nGgVciZqzBlXQKMwHw4CJ1x5RK1t8CnTakfKt7Iy1ySoy0G9QySOHFfEdzRR MGQc59m1vihXmmUuMl+S7OkAnpDLXjk= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=a5A25vU4; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=mfTr0IlBSv00lNflbQlJlesZUusNZR131JQGSJ2NQ/SfAm0tchHzUU5uAyPh8EoPlU89OK NIEC/jXnMVz9SqOI0vn2x5zyA9cY/G+5T8GEcWKvBXB89Jx57D1JjAOSk8xqMNhywNn0SH S1Cqf0GKsl09xJtCdRRZjFzKfZDCRpI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2EXLlBUI8HMrlq1LdzuePmnE9vo0jChMg019VHUJEgs=; b=a5A25vU4ox72y1WgqajmPDPLXxBJLanQ8l+VgC9NiqMvEWkJDaOOPujomDYyKT1wDUDbhL rZEie6Lwcr3QSXpKsx/4q7Evf3FytP73UVox1p+LmLqJR0onHXGtpq3zsmXTBFv5Tm+sku 5DhXcp4Poo3QhayFs98IEAeI7JnGlho= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-84-Mvf0PKrvOkyxjyMevBqNLQ-1; Thu, 18 May 2023 07:35:29 -0400 X-MC-Unique: Mvf0PKrvOkyxjyMevBqNLQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C1BF83C10221; Thu, 18 May 2023 11:35:28 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27987492B01; Thu, 18 May 2023 11:35:26 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Fastabend , Jakub Sitnicki , bpf@vger.kernel.org Subject: [PATCH net-next v8 06/16] tcp_bpf: Inline do_tcp_sendpages as it's now a wrapper around tcp_sendmsg Date: Thu, 18 May 2023 12:34:43 +0100 Message-Id: <20230518113453.1350757-7-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Stat-Signature: b99okafsig18f43pfb9nc4kr78ow7iqq X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: AC7DC40010 X-HE-Tag: 1684409736-759316 X-HE-Meta: U2FsdGVkX19OwgeZwQ3ViAlYun1pdBWxEGsuH6U7FV/UID4GVp2tkop0ntPw2f/Rg6Y753aMkbwMHOO5HVfR7Ijh1Hab4hBZqyX2hYTknLr4y7xcYent3W4slHZkjz9b4r6GvMeH8KlJ/oE2m+dRl5i+UM9jdHDGJnkAaFYchHcUCmQm4/BVi8ANVF9uVDfG4uSvCGpmQ+OM5HSwpXVF+t6dCIm8AC+3ZdYoCq7d88nkx9yINvmDDY7zqJiD9QPq4vkqROXHCKz5jFVcW0mRZpMjmbB92Ho6JI5d2nbsR0hLHLEy0V4G6V63n2EpPkT1waXC47O0VtqafKB7t2RaGsNLxejqStElBZ6Aae56Iynpze5ln9lFYBnUprOFig5HT4ZzQ1DuHdHv8nrSRZbjNHwrss70CQCtdLzoJvaoqwFHRyJhgXk5XQdiXNiOiFBcAj+Zd5HkIxVuHpbFmjTxRQSj89+Uev/ByILvUK54a6jxQm+yQWy/U1CbISI5WB2PnvzgX5do8nJgFQc+IiolwFggTYZ6Y67yD3+z80SxD4/wI1t9NUreLf0CXT6tASs7X9rXFj3DP82fhIJZGvnQIUssF626WzfUcSzHMoabbpIleFxWTJO8g8/8ruozOr+VTjgTO4JuQVS0F3XjXz3iSxizncBk4zYYrrlgeHBXj5BNqnUeYd94U95kZS8pcM6ms+yM2IVmKDJRhwR45+iY/HxTBI1nH4a8K7coRyXZCjr74HP4PPQoYM4EXw3oG94V9UxKzo+3bPvHiwlv2zYDstoRsLLxID4nP7g3NniE2dorts/Um+aQrHW5rHgq7K3QdJ1oVYgSX69+QnjmIiY+joX1/nya9QM60FXIDleLRu9s/kKD6FKprUVJUrk2gFVbJEv8BLZisL1kQqgE32dNMXNoaDbwZFx8wGoGStjQgWKQraxjzGHTCIIVygOrpZveSizwQSriD5bVLAWw9cu xcDxXkwK WN/XfBf8XPUL4VOn6btCXjW+3pKrR0HnrI5OAJUCr+KOKGVRfbSUf9Z9y4TduDvVwUQdPbV9DYKXwkG7jkRSUOqsvpqjRtUknUWS0LD12epjSqwRlJfelrdWUNxwkI3cjl7ZKSMRWxy7+var52D/zjoeeqfkzZmk7M6x57geuI0Ub/PGF0OUCFowlscsnzo6pFK6lZ8nc+Mq0XCp5aZN3bnKZJsJ36ZoPJijCz0J4LF0kEuuigVVa/8q/+Z8kl0ygZS/HaItvCn3pBG21Z4xz2+fteHgubRZFqJo5C03JK8ftxB9uYtuk+jnRAPghwLMu3Is17zhzW9l73Zq7JWsdlzJYtFvZioIVxyx4/HlzZMhfRuInd2SMEmL8x/dwtfJCjd5+dT6JF4XjrAp39kFIf0sHaDG6auuCdZTDTFPI1XczY7IvCT7Z/sqit+KrR6xB95hun4BYzKm3zFNJ/89MJv79FzV5+yIsfpvbxXbAxImq0rZDPuJmHtWgdblNnuQXkZCnmsaRCrRCdw3bSIoVnr+Rbg16u/fHnUy6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: John Fastabend cc: Jakub Sitnicki cc: "David S. Miller" cc: Eric Dumazet cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org --- net/ipv4/tcp_bpf.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 2e9547467edb..0291d15acd19 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -72,11 +72,13 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, { bool apply = apply_bytes; struct scatterlist *sge; + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct page *page; int size, ret = 0; u32 off; while (1) { + struct bio_vec bvec; bool has_tx_ulp; sge = sk_msg_elem(msg, msg->sg.start); @@ -88,16 +90,18 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, tcp_rate_check_app_limited(sk); retry: has_tx_ulp = tls_sw_has_ctx_tx(sk); - if (has_tx_ulp) { - flags |= MSG_SENDPAGE_NOPOLICY; - ret = kernel_sendpage_locked(sk, - page, off, size, flags); - } else { - ret = do_tcp_sendpages(sk, page, off, size, flags); - } + if (has_tx_ulp) + msghdr.msg_flags |= MSG_SENDPAGE_NOPOLICY; + if (flags & MSG_SENDPAGE_NOTLAST) + msghdr.msg_flags |= MSG_MORE; + + bvec_set_page(&bvec, page, size, off); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret <= 0) return ret; + if (apply) apply_bytes -= ret; msg->sg.size -= ret; @@ -404,7 +408,7 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) long timeo; int flags; - /* Don't let internal do_tcp_sendpages() flags through */ + /* Don't let internal sendpage flags through */ flags = (msg->msg_flags & ~MSG_SENDPAGE_DECRYPTED); flags |= MSG_NO_SHARED_FRAGS; From patchwork Thu May 18 11:34:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4685EC77B7D for ; Thu, 18 May 2023 11:35:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4A3D280004; Thu, 18 May 2023 07:35:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFAD3900003; Thu, 18 May 2023 07:35:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE961280004; Thu, 18 May 2023 07:35:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ACC03900003 for ; Thu, 18 May 2023 07:35:41 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1EDBC1A07D1 for ; Thu, 18 May 2023 11:35:41 +0000 (UTC) X-FDA: 80803170882.27.2D786E0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 6266A18000E for ; Thu, 18 May 2023 11:35:39 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IkpdP4dY; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409739; a=rsa-sha256; cv=none; b=h91Qx5/Dpb5yynwdNOQfr6yKFNQQqplnbVVcMLs8Dhjq0zNbEnw4OC5553V0qLVAP7deVJ tJZtannMZbBT1NIZZglqtShx3ImrfROUiXCqy8Z5PoadBjo/OHktkZwmSwS8+LGfaA3rYx NO3kLqnaSmViZx2B9hGPDmCRGqFCoXw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IkpdP4dY; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409739; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=HD8bmpJwZr1DGRbPQQkCErbdqwXyUpKSYstJlCEaHGQtSg+rWQKmLIF0lwtYSGgwbjAXwc cqnNqref+qB3XuJTE3c0ihqCZ69tho8O2cN5f5oxO4F+awRzxC244MI0WpiApKCltjAwCt 3wxncC8mg3JHYxcjkNl73F972HT/rwQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tYeoTX9B8dad49qWPHePVLYzX6oyR1jBlVymL6Q5Zsc=; b=IkpdP4dYySEBDFP3NmdBkEPmbop7YFSufs50b6Qepgd+hJPxvCq+tGwbuNQE4iOxh9EVCa bXY1c9ycz85dxWDOBmiM6095FIj+gSbXevzZ0ve70uCFDgO6NRnekG/zDtxwu0FPVEmMkD JLe/EpN4CqtEYN7a8QT9VIE+C0yhU4A= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-214-HshzOJ6CNxe9-YY1obDoFw-1; Thu, 18 May 2023 07:35:35 -0400 X-MC-Unique: HshzOJ6CNxe9-YY1obDoFw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 724DA1C09512; Thu, 18 May 2023 11:35:34 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id E91B3492B01; Thu, 18 May 2023 11:35:29 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steffen Klassert , Herbert Xu Subject: [PATCH net-next v8 07/16] espintcp: Inline do_tcp_sendpages() Date: Thu, 18 May 2023 12:34:44 +0100 Message-Id: <20230518113453.1350757-8-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6266A18000E X-Stat-Signature: 8i4w1hzajjgffbwoa3iq565t4ccwtinh X-HE-Tag: 1684409739-715196 X-HE-Meta: U2FsdGVkX18QXCAQBoSPJEx79p5FIbeZqQZDXZnFbnXSm82cSKznYe5V727bMH+4QiLlk/eHvPIudGl8HbNCNNi3vMtfxlIqmevl1Nnn11yghwK2QrOYdTLDzfZyJ02wVWrj1ZdhsOxjWwfpgBHjsVrd1cDJU/CvSZEVGKBJsghGuTON5JbCzeuFX0g/IO9TIF1XMNthaeBkY3jh4HFCyC2oEPvkjTZEja5Ma0Rlct0nt8JZ0SJYO0g1Iz1KiwbL+3GmA4VG/e9wogXzy/l3vgWM5sqJ2flPH4ysK/gi8vvRRNCNC7Z6qxmdrjkJQpqmCVR36TQA8YscM7pPrnsH0Dvj4OiW8datcRQ7JQgGHEgY1zOgeHS0YYbgnsCIYMjiS4BdowT/i67kudfk4odX/JzpU7t7ohidWDgZNUlp21Ik8yDrwfsr5AE4wnVPVIKZY1U4BLKFteRlrM1MF3HLCfBfC2bre9QVENrvCRGOFZoBRmsWIu6hCMPOtwdqwKd0RLJ5652RhGeiyNV9P3+9Y/G809vFJsC/SDLqsHgLFqe5ngqsvfmGtigDWf+/ExrWA+ZqeoJ5j0f6TSnAeHL0kFgRSppF89B9P5CDer1VueQkATWtkZLXk20O0lRyK3tl9QB5o/kyq8vCCL0e1LDcpo/MNjYYV9SlRkkNOkBvCQGztnf1qjjAIgUF1fj2Y5ngWOPCceCTnln4CBNjStyxcMYGvjLlh/RatL2Ufk4YU4WyztjYRLz1fkitF0P9kBcMQHKEvN2RfOk4K5C3X42kvwQwIle0plJTfTxuwI4SwS0Nzo6ZQeKFgnNulBLhMqCWGT+gi4aRrj98C/4Ly6npN1DTB0pzbQ/fVjm1D8TArSBNXq+0vbs2DWGeV9oAbOSqz9NgM1mi3TeoX+XHBnxb4t5YJZBmVAc2CjdTQ8/vFcxQ10m6gJsXl1GyiEvjOXY+3sGwmEg2Hc5bmtMYZvb ClLHGUuv Htfwmw3zDXWYOlP7pvkGUll+A90a9QgU1rqtUM8xhc+13r1HGggnWxxNCECGJxTyd6L16JAEGniXK6+jSxdxh7DfxGQl9x0YNsKkHSIpVmXy4Yyx7o3t8SDGm+CCgc+C9+nVg4UNs2OFyjYDkSngl4NeZ2K5DpRsloTHeia32QM5YQvvU1hwo19/+84t4KBqyELySY1xn56h2j3i7sDSrmzOyTuVqxzIGulI9mpkKpnweYuY8KzMZ/Kwn1iNOoWgor7gzP7vuDDYFlvDu71K/8DqiIvUNWcbu5nrjJxi4BPSwmHeOjNSOtXIB65IES+jIYToULw2yX9fiL/rIPVBKB4vY8SWhOSa3eVNBwSf8I/lxksKmafeIeYG8uvlfyG1NU+JHDyWnIf1iOfdTKvRtoNDWrcFZThDkLZV/lFGcXABTZc6JNGwEunCQhJrazT4URdkTkjBnbtloAptqOcfho/nx9AbOxza1o2ZrUAdPYwN3cLE0N6v/6iPwZJPSGLHylvChDma3IELcBNdajKtmX2mKh5RImAYks3jQPQ/buvB9rjtEgYAZ/l+rmfOIbdqcLbEcp3fMdteKxxto5wzr3LQGf7InIxHCgz5q6ye1CgNckIg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Steffen Klassert cc: Herbert Xu cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/xfrm/espintcp.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index 872b80188e83..3504925babdb 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -205,14 +205,16 @@ static int espintcp_sendskb_locked(struct sock *sk, struct espintcp_msg *emsg, static int espintcp_sendskmsg_locked(struct sock *sk, struct espintcp_msg *emsg, int flags) { + struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct sk_msg *skmsg = &emsg->skmsg; struct scatterlist *sg; int done = 0; int ret; - flags |= MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags |= MSG_SENDPAGE_NOTLAST; sg = &skmsg->sg.data[skmsg->sg.start]; do { + struct bio_vec bvec; size_t size = sg->length - emsg->offset; int offset = sg->offset + emsg->offset; struct page *p; @@ -220,11 +222,13 @@ static int espintcp_sendskmsg_locked(struct sock *sk, emsg->offset = 0; if (sg_is_last(sg)) - flags &= ~MSG_SENDPAGE_NOTLAST; + msghdr.msg_flags &= ~MSG_SENDPAGE_NOTLAST; p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + ret = tcp_sendmsg_locked(sk, &msghdr, size); if (ret < 0) { emsg->offset = offset - sg->offset; skmsg->sg.start += done; From patchwork Thu May 18 11:34:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FDD9C7EE23 for ; Thu, 18 May 2023 11:35:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0463280005; Thu, 18 May 2023 07:35:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB18E900003; Thu, 18 May 2023 07:35:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92A63280005; Thu, 18 May 2023 07:35:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 843C2900003 for ; Thu, 18 May 2023 07:35:47 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5A8E114025B for ; Thu, 18 May 2023 11:35:47 +0000 (UTC) X-FDA: 80803171134.12.4BAEC3E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 7E8C7140009 for ; Thu, 18 May 2023 11:35:45 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d1ZQgZ5e; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409745; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=thP0XJD2q9xHSVtGqf3E5rqOdlI8XpP1DJztrWscRwfQbMzI7EUEjfGfwEw0Z1gqwxZNh8 eCkcytKfhBNRac6d/3U24uZVSutkB1sv6fW3KUEVPY+FxQ7XwInzDaSC8H7fglWeJltXw9 HLLgyP3Qbq6dZpIYniebNwFBIzES0Z0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409745; a=rsa-sha256; cv=none; b=oRL44CS8eAzpoBlkPTlz20+KmduZuEWSURdbiu9TQ3zanicXxoTPWrkmRgZH0eIU4ALhDS 5frjaW/ZivbmNk1YwFkqIBKJMBSfy4xUqn3Iv+wUTvkutx4rJjlJ93Z9mzgL2YIc/bN5GN 3DeoAixlKEfoG/md7J2g8ZQoq/lHWO8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d1ZQgZ5e; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JUrQ0rsOJEKe2VauAy4RSjWfQDMPveXienxXCaL8A+w=; b=d1ZQgZ5ePQPrrfY8jedIgC/v5ofo2qI9kXjBNTPZklkTSs+FadSWII13LBI4p9R8d/sKOA 7Iqslyo3zmjhBsn95XSAkq0KSx+sgb7pNLBZINMOCP7bQa/ztUtLGLdexVhLE4EmXsWCO0 G3XxEeL7HHkQlGQBeVP16pJAvX2KeUU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-575-Of9dg0BOOn66lre7Opvf8A-1; Thu, 18 May 2023 07:35:39 -0400 X-MC-Unique: Of9dg0BOOn66lre7Opvf8A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D25829DD980; Thu, 18 May 2023 11:35:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id D496C2166B31; Thu, 18 May 2023 11:35:35 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Boris Pismenny , John Fastabend Subject: [PATCH net-next v8 08/16] tls: Inline do_tcp_sendpages() Date: Thu, 18 May 2023 12:34:45 +0100 Message-Id: <20230518113453.1350757-9-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 7E8C7140009 X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: fkkqr6b38mygyi6fxgb4x84no7q17uow X-HE-Tag: 1684409745-210711 X-HE-Meta: U2FsdGVkX19FL2BVa9mF+Xxw0Xk3gChp3z+9Z7thUmdarCERIAUt4rSjUs7ekJotQCXk5tallMSWg2O/3SiszfZvsQ/fCfeAlzLxl09WwkhMkSWhLvjF8T//lN8NBV6cFgfKn4SdBrVlXrJ9X7+gFRjMl5b2hJ1URUB/U/4/cXa78JiH0r98TL3fBPg+sZWW7vPdOVz48T/go+AsCG+J6t1y9rljcZMdcvPC1nNUUtItjQQqG6qU2KvwNjtc3dR4394TWDIbTUP7T6AC9tJdgNOuolEeASC3Ro5s/v8MB5m8HJeUUOoXUXi2KdSQzreggdi05qCs909lAqMYm6LknXw6JozKZrl/CTY0FRwS3HFl6EsspEdDGwPCvmPUHuKT208V5dai3hNGgxIrzZEiLGGFI+f3/htq+zDE/i4WPq8+n5DuBApDjmT3bb1ue9lDsW5yz7KlOS8UTHjW8iNP7EnzEs40FMwdE1mVtGvrCjLfF9Slvc/uURku93PUoquQUrqL7h2hO6EDeQcf4LCO+NrRO3mKuxW9pYBr5Jf5zv448SdBQYWi4gdkU1jprlrTkn5Y7j0JsYm7m2mkCISvPnEWfSkiym/IT2lZzsV5j2ubYrb+1VEqqlRmyg9oGsdSzGBoUkXEOBS1cjaUXOsX0KAk+Dt/770dxUFn0tHBnUhWnEYxfUcJtF5BOXCNyAYO3HZ2T7r5+Ug+3pKPIYaCLd4YHEwQyCaNozVz6q3EUxuhI+/BQ6fZtCV11LqnxydJl14RhEvKWOUb1TZii9EebbzakTgWsl2nBPaR1MYLQJSS22TaI5DvK2SKdMHKJ67jDWkDdzkjZub88ItU2fYE9HY+8f1sWfTeCAZokDWCV6x9Q0TsuNLYbJWyimgjASSeVvN360ZFkMm3zQWyO4eZ1Q/mYZ0OTVVCDWGftcAQoaWHGgahflGbS77yalH/Q2GK/mUFugRchirqagn6CiD KJZXsvh/ QYv7Ceidg+m/iBu+F0IqBOLoYr5uCx4efQRCf1mrAJymE85lc1/S0pnoDnNGeWPPWZKG9zMN17yhSpQkXsBieUOOibkLN+IFJDRt7U2UJgnDQ0qsDuhmX1PZNhDnHI9XNsGeCshKyGN99Ohsexyxlk2okl3r5t0Yfjc+eY+oTmrhT7JR4hwy/x3eiS59drvTzqxbT2priN+Mn36ixJDkCWinjDsK5XfI0ij9x4pKpEJFzkAA/zRCLDSuZJA0RsvuiJFyJfAAtWG+TzibnYuuYRPgxUdM3TtPIlxKvRWWPY6DgnHI1zrHZ8k0313qBcmjPerJY3pOzHX+LXuRgkX/BbzyHLYoWeaFSU1ne5Hwuehnzorqeo1bstEUh1jEC3Hy8ObWkAsETsEPnsMxk17BEoow27yqFWoEaBhF7sVtud2ewKGWFbsQqTBDY0lZDd00o2NTJU/tDRIqXI8tb2S6UMOEkYk/NxeoK8w3y95LMLD3R8gqVEkDARRlq18lgCKqIQXdHMNwL2fW0lqkOXtAr/myT5eUn2Hf+ONyg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: "David S. Miller" cc: Eric Dumazet cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/tls.h | 2 +- net/tls/tls_main.c | 24 +++++++++++++++--------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/net/tls.h b/include/net/tls.h index 6056ce5a2aa5..5791ca7a189c 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -258,7 +258,7 @@ struct tls_context { struct scatterlist *partially_sent_record; u16 partially_sent_offset; - bool in_tcp_sendpages; + bool splicing_pages; bool pending_open_record_frags; struct mutex tx_lock; /* protects partially_sent_* fields and diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index f2e7302a4d96..3d45fdb5c4e9 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -125,7 +125,10 @@ int tls_push_sg(struct sock *sk, u16 first_offset, int flags) { - int sendpage_flags = flags | MSG_SENDPAGE_NOTLAST; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SENDPAGE_NOTLAST | MSG_SPLICE_PAGES | flags, + }; int ret = 0; struct page *p; size_t size; @@ -134,16 +137,19 @@ int tls_push_sg(struct sock *sk, size = sg->length - offset; offset += sg->offset; - ctx->in_tcp_sendpages = true; + ctx->splicing_pages = true; while (1) { if (sg_is_last(sg)) - sendpage_flags = flags; + msg.msg_flags = flags; /* is sending application-limited? */ tcp_rate_check_app_limited(sk); p = sg_page(sg); retry: - ret = do_tcp_sendpages(sk, p, offset, size, sendpage_flags); + bvec_set_page(&bvec, p, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + + ret = tcp_sendmsg_locked(sk, &msg, size); if (ret != size) { if (ret > 0) { @@ -155,7 +161,7 @@ int tls_push_sg(struct sock *sk, offset -= sg->offset; ctx->partially_sent_offset = offset; ctx->partially_sent_record = (void *)sg; - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return ret; } @@ -169,7 +175,7 @@ int tls_push_sg(struct sock *sk, size = sg->length; } - ctx->in_tcp_sendpages = false; + ctx->splicing_pages = false; return 0; } @@ -247,11 +253,11 @@ static void tls_write_space(struct sock *sk) { struct tls_context *ctx = tls_get_ctx(sk); - /* If in_tcp_sendpages call lower protocol write space handler + /* If splicing_pages call lower protocol write space handler * to ensure we wake up any waiting operations there. For example - * if do_tcp_sendpages where to call sk_wait_event. + * if splicing pages where to call sk_wait_event. */ - if (ctx->in_tcp_sendpages) { + if (ctx->splicing_pages) { ctx->sk_write_space(sk); return; } From patchwork Thu May 18 11:34:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D06DC77B7A for ; Thu, 18 May 2023 11:35:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31166280006; Thu, 18 May 2023 07:35:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C134900003; Thu, 18 May 2023 07:35:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 188F290000D; Thu, 18 May 2023 07:35:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 08D5E900003 for ; Thu, 18 May 2023 07:35:51 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 90AC3A081E for ; Thu, 18 May 2023 11:35:50 +0000 (UTC) X-FDA: 80803171260.20.03CB947 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf30.hostedemail.com (Postfix) with ESMTP id A475380011 for ; Thu, 18 May 2023 11:35:48 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SiJDKULR; spf=pass (imf30.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409748; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=Cj+Fpv0LQNhPPzvttl67HM01GX1KmlTGfn3vyCHI8mM9cyfn+VayUe2ULLcRXqcgNQyzk/ vxd8OcirG9J2kWQhS/iELy7ahnOZBhk250NkEmtDhC2zR9FdxvDM+V4QdaLduPG2+/CYKf PZvAzlYei3DexkcS1X3Y5EhgTm4a6Mc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409748; a=rsa-sha256; cv=none; b=DaW0tin5hzmggMXvgXe0P5+rYDTn3F+/VUuRYeedPyeBDlFmpdWXKo0uOMoEm5rP2priOl DoBE3SGKJU7Na/fwbgY99Z4MfYunBJmQCGS9WuXpqjc/vQX0OaFNl30XgzLMEF3qvNoydP ZXhF0H7CIAcZw4/UuQQgr5gGwQiGsmI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SiJDKULR; spf=pass (imf30.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409748; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEe6KsGIUg5Ug9eymZUK4cq5L1lxhWlwpIXxPsGo3ck=; b=SiJDKULRY4pVNSIqdnDURgtwkZNATMrf7S57oVVXI4TXBjr9Rs2Mq1RMjsM5ADvmqqcx9b 2Se0/DHKh/91Vp59XrTSU6RlfbKKIZPd7GbXZacV1R4hDRETU+FjdGuNnF3dogc/+n0JPk tZpQvleFXCOVmFLU0pCIevxPGTHxO2Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-146-D-Rd9ENnNDKPveHNDxPUbw-1; Thu, 18 May 2023 07:35:43 -0400 X-MC-Unique: D-Rd9ENnNDKPveHNDxPUbw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B38E010146F8; Thu, 18 May 2023 11:35:42 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1878240C2063; Thu, 18 May 2023 11:35:39 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Bernard Metzler , Tom Talpey , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH net-next v8 09/16] siw: Inline do_tcp_sendpages() Date: Thu, 18 May 2023 12:34:46 +0100 Message-Id: <20230518113453.1350757-10-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Stat-Signature: ps1bq8hk5wog5miz1sp8oqdkzzfxtmyg X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A475380011 X-HE-Tag: 1684409748-880063 X-HE-Meta: U2FsdGVkX1+bQNINQMPGj9+pDWPDbEo4mvrRTBSSlgojJWdRMNSnKhT1GsPRz8yKgHKRom6jZyZpl3S/s8lmQhza1XzY5XpQ009r1yBdiSUNU9Mt3zT2M7VDBaPvlaUFxbvSEtOqJDCqhDDQEKKSi56FJGTBD6wGNxcX87/3AXVWxRBj6hJm7+GI4Xn0mXq7TY89UPZ3aikTRFUoNNLYmzBMd7e96WrqX5aQCLUKtNzxx92y24uOZQLpLHHEGFyT8wnqGYf7P3RzOxpTuqvjTQo3vTBY5aL2K7144wWPpzzH760FWEFpXwIwAr+v1oYumK11e9A2UmyN7Et2hgcOZU1Au/XXGhphDWAbDRR/TYnyd6oX9d7STzZyULN+qUFUJ7WF+YFF8kbHvP+0k3wM/iRYv/KXI2C0sj1X/M1jZ2gwvBpspRsueexzUWJ28Xnslox7sbsivoJIggD8RzjJQlnCeBbZO5DH8fAfMRRdZe/9NdLggiXr9y6W8siDuF0/hk6sH3P/nJYQhSNmY2d/SrxnbyPFA+N2neot095nw7kPMtJFKDB11BWn8e4X4+pgzSMfpZD/3EEGx9z6z2Di6NbYLzqOPhkHaJFsE8KGbYWP+2W1Bj8AZER9t1n+II4N0dSHVq5V3UKiYxUe0ZnPt64YX3dVYxahHc5+blFIPD37AwTrlfRC7kcichcVuUpIF9mMoXQL0MwG4QXDicfMN/BPGIrHCg47lRwTKb3GCn14zjAetXTMfkh8IkIyEfCiS/9h/XrXYVo5RiLPJm9PYJRYwss7qK0vk3WAS3hz+mzP/6taT5uERZAGe0dja5pQS3ILIR+qDaWFuzyPA9+TU+FMRTtXQyB8iw4jN8VRMMNJM9SKJsfbyMa1Buey2AWfgsUz2KkllDNhJEYZJkphd7apEoofpJDi+nIwlBvdMaW3DEOkvGV7xhF0xcbOLv6N4iyl5hQjPB+gXMwSmIN 2VVrPFl7 p9Wg+dv9tYyvY5jWNXIN5VmrDXOwMzhC5QLepZ7exUpZIKcF4wy4FEVEv74K2SX2IfBt+uxpozXbxyX3OpAhnsNcx9FIchNq/XoUxC7cz4XykYgi9IQEszt4uayGMODaJg5tMBWJxvEXepmv3AjbFMhr6I0GpRHU9Mr/a1CnRvuwMOntUrnbEXLZ7gIYm0Uxt58RhW1KevqPXZfsQvv1VpB/oToBkIW1C9/abe5+OQcRnDtYeBcaLwP0r+Mm/SShErBtFVhkKF36n7xp+K1sXAJKgYzDZRgGm+IBdTfYqbAUuMoSsMB9rYi2Sx4dpwwKIG1s0vDuKF/dIUfiqO1L7d7LOHr1p0c9Tq2YVjje/mcxGXTZqn2uUdXnnpXtMhdLcG4FRqhajeX0TShqv4JGwKWYXo0NuohIhIIZCv0t9rKplNMAV68K619oBZwmjhP0rKpPPjk6CehXVh+kmfVpq9df8XaZreiCain/OxlIPLYntwAK4d2Z4sKX/zLOV77SHFrJfpAJQ3iHHg2L5SpL+a9SzcZ/T7q/KRczjFPQfLd8yGqsS/W2yZ5KzDzoCFLJqMzsJDC2wi/01W+wyOOUYupsm/kHXmwjGzNG7mI4D0tc/DCIjSAWWE2niERODD1Ro1VLs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: do_tcp_sendpages() is now just a small wrapper around tcp_sendmsg_locked(), so inline it, allowing do_tcp_sendpages() to be removed. This is part of replacing ->sendpage() with a call to sendmsg() with MSG_SPLICE_PAGES set. Signed-off-by: David Howells Reviewed-by: Bernard Metzler Reviewed-by: Tom Talpey cc: Jason Gunthorpe cc: Leon Romanovsky cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-rdma@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #6) - Don't clear MSG_SPLICE_PAGES on the last page. drivers/infiniband/sw/siw/siw_qp_tx.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index 4b292e0504f1..ffb16beb6c30 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -312,7 +312,7 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, } /* - * 0copy TCP transmit interface: Use do_tcp_sendpages. + * 0copy TCP transmit interface: Use MSG_SPLICE_PAGES. * * Using sendpage to push page by page appears to be less efficient * than using sendmsg, even if data are copied. @@ -323,20 +323,27 @@ static int siw_tx_ctrl(struct siw_iwarp_tx *c_tx, struct socket *s, static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset, size_t size) { + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = (MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST | + MSG_SPLICE_PAGES), + }; struct sock *sk = s->sk; - int i = 0, rv = 0, sent = 0, - flags = MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST; + int i = 0, rv = 0, sent = 0; while (size) { size_t bytes = min_t(size_t, PAGE_SIZE - offset, size); if (size + offset <= PAGE_SIZE) - flags = MSG_MORE | MSG_DONTWAIT; + msg.msg_flags &= ~MSG_SENDPAGE_NOTLAST; tcp_rate_check_app_limited(sk); + bvec_set_page(&bvec, page[i], bytes, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + try_page_again: lock_sock(sk); - rv = do_tcp_sendpages(sk, page[i], offset, bytes, flags); + rv = tcp_sendmsg_locked(sk, &msg, size); release_sock(sk); if (rv > 0) { From patchwork Thu May 18 11:34:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29F1BC7EE24 for ; Thu, 18 May 2023 11:35:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11659280008; Thu, 18 May 2023 07:35:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 00266900003; Thu, 18 May 2023 07:35:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBD2490000D; Thu, 18 May 2023 07:35:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CD238900003 for ; Thu, 18 May 2023 07:35:57 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 805A51607E8 for ; Thu, 18 May 2023 11:35:57 +0000 (UTC) X-FDA: 80803171554.14.8643709 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf14.hostedemail.com (Postfix) with ESMTP id BB8F4100011 for ; Thu, 18 May 2023 11:35:55 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=C+fVtVmO; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409755; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=enBN5jT1muKlfAJKNsWEtZoec3VPKG1S27Z+IZtkOMs=; b=o6tGiY32LxfIADFAhBXF+BtitrpV6kaKItFlIvhWZ94ONy6kXVNokUnpm+sZNJH8ZpSfos ZGXEYo+HcoxMjvUqQef0D/oB8AlhxwR+jF+yCr2d+63eZhwikW5xERl6oKTvTGxLpi73WO RIQPF6Ur9wQxpzK4N17Op3gRd3nNQGc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409755; a=rsa-sha256; cv=none; b=p3Z+pq4YtZ8bdCDUSsS5OB1KZ+1v+9fCmFIsir/uRGkanFCdnI8T3K+nyb/6mAPUWNsHGf EZ4wJRa3OrI1ohHSMp/C0jhW3rlXihbJ8wcITZMRreKa3iySc2n9ifb3h/tR3ph1zY7hLT owFWvqNUZA3IBpso35AXRr+n41YhGU8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=C+fVtVmO; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=enBN5jT1muKlfAJKNsWEtZoec3VPKG1S27Z+IZtkOMs=; b=C+fVtVmOFxo1nHGvgoXlMjwJvKN9Lyb8vDqBijHVjGjGMCdyC6pdG4AjW3hoy0hj0Twayn ySk9De/CjxZEPhZ1wC2xTCBYC1q2DOoPxlt6Y8RHpI4Qw9BeBTwlQOUMa2MCOhOyDeJdOD /8enmPdUjgFMxuzm4t48fzRp+cB5N3I= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-669-zHr-wG-8MFKIw9HgrsEr2A-1; Thu, 18 May 2023 07:35:47 -0400 X-MC-Unique: zHr-wG-8MFKIw9HgrsEr2A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EAFF61014702; Thu, 18 May 2023 11:35:46 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id DADA640C2063; Thu, 18 May 2023 11:35:43 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 10/16] tcp: Fold do_tcp_sendpages() into tcp_sendpage_locked() Date: Thu, 18 May 2023 12:34:47 +0100 Message-Id: <20230518113453.1350757-11-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Stat-Signature: wwe93ab6tn6x5toaa7rzi3ptgtz3juz8 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BB8F4100011 X-Rspam-User: X-HE-Tag: 1684409755-579084 X-HE-Meta: U2FsdGVkX18BVbSSchEc4+ch1PGn+XTIdYfKvByLgxfAQmO+lC8RsAmEwkGcDXZe4nSxCi6wIsIOhI1M028Uhs1faJ+F69RTi4JzCG32y7mBU4jQq8MzJxa5k2L+q+OISDcLHPgYEpmqFtoOVvGiUTbxW0l0IWtdeT/2FlKPZb6VasYp8q+PbqkpFZudpCbfk8H3t39Q/Vs/GmbTzIgLrSjkQX62VgXZxuhM8JVCY3nOkXEcVFd6C/+4lKqv/OsZhxqgrZRj+qdbw/zWAHJxZv2OVkK+ucZKeNl0et2w0uHXgcR+WFCRDxoJ8yiWamKSCEgy4BcH7A5lywCzDIPVFbCGgXg60i6HS6JpgHJqSgskUOrOFVMcouVeVUnvvtWeKiI26V/+Qe7PhMmHk2kF1MtM96280cx2RxN7s+BDx6UPEb8a4ma9VWHEufsS9goRmaXKwXILCf7aW40naQdyW64TfL/tk9K+ObncMV2wEfN9lCzR/LL1+/mBEW+IPZPzvRcA1WWiiGoe9ySAVVoJ1SkRQUDwW1jEmMhYzBrK10YobnDUr1nP8zPz+bGaBKN6W7/rUEA5tcse9UsevYda8kDwRAeZDzseIiojX/Fn+AU2/D9XL6JrfATD7+2PAV9A8qFGWI75UWWHCq8cfSitiC8MEK35AFp7/R6OZvoVEdNUuGp1yjQutIR9f9lus25A4mqHdXqUXVeQI629R18Emnlbrvtnxe+NkHIkoKcwaTqjbeQKaHwgQU2z8DUfdnQi1C168zs5iopbbhEb6ThTqI4OxunDOIlERkXlK7jmcxYMK05tnUehn2SZrXpPNMSQcY8wjECSTH60cxrJzEQj2K9nNhm6mPe5mjsl55GbmVFjVTvAKCxbTqcbIviUnNwj/iH6+4zCiyGcCLbtpxvj9wmaARB5OhCe6noDNdmwqWllSP7EovCgZBJtYVtiRLbyOZjXeq4faAkAQXXzBn2 38pojBMs HthNuoYw34aOK4HQ+9cYAEdRLiFrIaB9wtm31rM44N2ejx6e6hG5DuwcMTbwVsj2uFJ82l7uoukVWJKmwNA2KjwnkwKoXLWP2nrZUtZ+NFRmE7p+B78fWgx/Omy3sULQLfN80yspDvecYLq06Z4cHC38eX51IoGzxnF+AF/4SwP4oAY0+MYR1X/+abYLa2ad3ybdseaz63w4HRmYjJw5QG8EGwCziT5a9DmvIbueiXJq1cNFZq7lHX0k7lAl+QXdBKwlMBGYKZRrlTYqBNd/7uml4DYJpCa3/cF1uTAJ4fKjvNZ/QEC+mWxJK0hpAK/AE8Cbh8D9TyREcJLVDfw+kVE5VMOXBvz6sa3ZCBKMeeYXp28uDke3VlTRGwBzq70GQTT2Jlsh+HuWzb+yevM7OzP1R8Svl0uIuS/CQ/l6W7ut3t20lB6gYpGlOjC7ejMTWODQEBqyi0Q2lX5mhI51cRFoKetbdbMTZ2W3nKsmTxgfJbZfj5kRIGX3+6w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold do_tcp_sendpages() into its last remaining caller, tcp_sendpage_locked(). Signed-off-by: David Howells cc: Eric Dumazet cc: David Ahern cc: "David S. Miller" cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- include/net/tcp.h | 2 -- net/ipv4/tcp.c | 21 +++++++-------------- 2 files changed, 7 insertions(+), 16 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 04a31643cda3..02a6cff1827e 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -333,8 +333,6 @@ int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags); -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags); int tcp_send_mss(struct sock *sk, int *size_goal, int flags); void tcp_push(struct sock *sk, int flags, int mss_now, int nonagle, int size_goal); diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 28231e503af9..5e4752131583 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -974,12 +974,17 @@ static int tcp_wmem_schedule(struct sock *sk, int copy) return min(copy, sk->sk_forward_alloc); } -ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, - size_t size, int flags) +int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, + size_t size, int flags) { struct bio_vec bvec; struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (!(sk->sk_route_caps & NETIF_F_SG)) + return sock_no_sendpage_locked(sk, page, offset, size, flags); + + tcp_rate_check_app_limited(sk); /* is sending application-limited? */ + bvec_set_page(&bvec, page, size, offset); iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); @@ -988,18 +993,6 @@ ssize_t do_tcp_sendpages(struct sock *sk, struct page *page, int offset, return tcp_sendmsg_locked(sk, &msg, size); } -EXPORT_SYMBOL_GPL(do_tcp_sendpages); - -int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, - size_t size, int flags) -{ - if (!(sk->sk_route_caps & NETIF_F_SG)) - return sock_no_sendpage_locked(sk, page, offset, size, flags); - - tcp_rate_check_app_limited(sk); /* is sending application-limited? */ - - return do_tcp_sendpages(sk, page, offset, size, flags); -} EXPORT_SYMBOL_GPL(tcp_sendpage_locked); int tcp_sendpage(struct sock *sk, struct page *page, int offset, From patchwork Thu May 18 11:34:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B81C2C77B7D for ; Thu, 18 May 2023 11:35:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55BC2280007; Thu, 18 May 2023 07:35:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50B2F900003; Thu, 18 May 2023 07:35:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D2F5280007; Thu, 18 May 2023 07:35:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2D1F8900003 for ; Thu, 18 May 2023 07:35:57 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EFC384076A for ; Thu, 18 May 2023 11:35:56 +0000 (UTC) X-FDA: 80803171512.18.3F612D6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 3A1B2A0008 for ; Thu, 18 May 2023 11:35:54 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=aotO7zU9; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409755; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=snpvK7qt1AKxqyFhHvWw/29TAmbjh/QtFXxYAwCQmsC+4ezSwYxdb2Bw0o/ESIpFlFfPpN Xw3MNeKIi6b2YFDnoL8sNM9Qr3Z1DAxwVLkrrTCXZ0V2skmp1mx8nMyead5yHXr/X7vLcK Q3YiKCalauvHN/KK9o/+uOStNpXjVoA= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=aotO7zU9; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409755; a=rsa-sha256; cv=none; b=24vmFPN3J6tlmx/K4tqnuqccL4vpEswHeEgtTLjO3t9/HfPAwXykEEzFlfh4dEQ7VxuL+m YhX93UzoCxCjLcYvo46k9Eo2phTAEPwCCEzcYmeawJ6LmAStWPpi/ERqhv3jTKv7mg0Qvt DNq3v6XLFcgsJmQDc0msX2yLsAh9oOs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Di14zlocOEDECmKOzgXOEyS4sjRLATonTy1WJdIXAUU=; b=aotO7zU9dz3HsZpxVBcs+RFC0QhdjXZc5SBOyYx3Td18y93Pg4VYGpk+Ngn/6LqGBQnLV5 HOHRlgTHiAiFI6M4gIx3GOArAFoZHeooVC01z7JI/8DjfyriftC2RnttVPDmBDDPffxOAI CgfLZTCMv/F8TuOeSay6qM64uFT70SQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-175-D_mli70IPguE5JHBJeMmUg-1; Thu, 18 May 2023 07:35:51 -0400 X-MC-Unique: D_mli70IPguE5JHBJeMmUg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 260A93814597; Thu, 18 May 2023 11:35:50 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB20B492B02; Thu, 18 May 2023 11:35:47 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 11/16] ip, udp: Support MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:48 +0100 Message-Id: <20230518113453.1350757-12-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: cfadoskcc9i8pr78uquj5ew4f7wcbwyo X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3A1B2A0008 X-Rspam-User: X-HE-Tag: 1684409754-273765 X-HE-Meta: U2FsdGVkX1+2oqu3trfjanJXxdUmbeVhDZaC656WFUGdQr1tysh7eBEN7dfB7v2PwFWoNZOlBn1T1pFpBNhDEkSgfQvFqQHCYh506h5Tta8rCyX0XFF0Eh0Heij4q9KPZJP2DLKrbysZ65pFxFye1s1ECJRebNlzz8F5ypV3f65saMzclKpu4xno3bD+Zusxs3zfYerERcglh9qcYQkpPzT1yByf5RejHXlJ45ovKGEo3vJJd3Jc6ECZW8/O9Mww6Uw48W2HCQCEIn+IgA5K4DhPm9GWNyUQkoOSyd4JWJT99vOYp2hRvrM/IazL9ewUosMhAmFkV9UCdQN3xb6N1uPdRMOCunP/t8KLHMJvtGm5POtikNSh8pVpBCPQ/dIE+Kz+Gj+E07Sf+O21FvrfOA9SU+r71uoC/sqi7oH1WmQV5E+IpzCrYXFf5iqWE3eipGUNpscdhFkRSEIvTs1Os8lFO3AC/p5A0DAVWVUjDf2fAeLTA2wCaip0jnZXWp47g2FKThhqzLKpn4RJV9nR2lBctqYAdUSMkske8yoyGrw/hPNKFb1Q3OAxIJdUapwQMa7Y4k+b3tl1BQ/z6IZXqEUvxx4PAUrs8jwrMR7lilnI1g1eqScS4NCHInie3fM6BuXGDwYVy+ZMA9adfRm3RnDFKAIve+uI2rcvb4fre6TLFuFfnbhIeP2j9Cv2uHvOLrQS2KKIm0yhlRwmhNjGlwgUmXynGjZd3/RCnS1ikwxACViqwJL93QxrxVIv1dRipE2VWz7UvMJo8hEGoQWm1U/mgakJsj8hepJ7pGVN589tYI+/UsQapyfS9S+QuKVOdHl9Js54ieS6NmvcD7NbfN+kb+9i02nCt1NFXvjg0vDb8Qa+omBiRv4zGwFHm2UvugxaaEPX25OJaYstpdZfuNKNZqvVVNvAkpnYjYmzfzkLQ1/S9KrQBNAZB/dciIdnqf0O82LGb3oXwakXNrK XLOaLASS 0880Zke4ODc5hz2gXfH1fMbiCCg8eesF5VWubKAVTQcVEz9d4ZY8IVzuoQFeWLvnXk4y2kPgppFL9WDgGx9/1a16mtoEiTw2Nu2lorzTJAUbAm3P160C3erqGZvXyekAV6uPGgvsdymz/1ni4bQEmgkejjp7o3nmhpeE1oQvN5Y8x0np3gw9+jiSsCtrGmbQkKRolY7bd9G0AoAPL/juoLujIDSHvKuDMI+TbjyUu0iMwcbiv6SYJcj7lSdllRrvToBXYhcWjHfvXEN1sSJVjyGihOb3j9jzsu9McwrKbRHHWyNvj87aJ+v5LNkNVLXRi3ZvwNrzddzOjCLvF36N0RY3159JZznUvrwDxtHm0ovV2XjGETqbhoQYIdRHsWN3R7vl01AaO+E6+MwRyLH51fzsXZxfIwjJ9dQUyV4gc5BmlrMhugZVhBCsonGtw4vniIUnoLgkzQLzkS5yHgL6VzVunRtnnAI+Arcgc6emXYaFpvj0nDpT6vurovYIno7n4QfF5RqMhKP4+0wU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP/UDP sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv4/ip_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index 52fc840898d8..c7db973b5d29 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -1048,6 +1048,14 @@ static int __ip_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } cork->length += length; @@ -1207,6 +1215,15 @@ static int __ip_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Thu May 18 11:34:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACCF0C77B7D for ; Thu, 18 May 2023 11:36:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C060280009; Thu, 18 May 2023 07:36:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FB05900003; Thu, 18 May 2023 07:36:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24CAD90000D; Thu, 18 May 2023 07:36:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0C6A8900003 for ; Thu, 18 May 2023 07:36:01 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D396C1407AA for ; Thu, 18 May 2023 11:36:00 +0000 (UTC) X-FDA: 80803171680.22.2B914EC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 11107180006 for ; Thu, 18 May 2023 11:35:58 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CGnwbFrC; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409759; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=xtuqtcO7szTVDFOyl+oBuG6ttBAXG34tGHSnm7PhP+V3kaQVSbXMOm/Q6La1+ParVs4QaA 9bYx0dVXP8EM/zDajZ9q49Nmg6Qlha/GY/A4IvA8YHs7DoWHxfXHYdSV6mEQDF/i/ctLcN Ii7fj4y6qkCgqZ7zOckAAgAMqtXtLMg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409759; a=rsa-sha256; cv=none; b=mSSg659OjxAHdD6OtLMS8fTUCFFJ/nXv8ThJuEc3QUcxuDUWzRFLs0BOteTKoPTu2S/ph4 6PjphHNxkSmFr3uq/SPgR/8Y17kXXdaVQ/QL+12l/aI79ltJn6e2ElTI+W3fIonCG0oUWa EQ2HhNlsym/8tGWS71Ghd9++0vhdSf8= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CGnwbFrC; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5dEuCyEYbAeTLbptDWG+lpG3TePaGyDY6iKA6gIr8Gk=; b=CGnwbFrCDxOc/8OkIFX/JbeKvZ65dWhs13s6LrviYH/rIYiWotwStagBjADwnA/Tg3XbMg g2CIIavjbCiAoxymvB9u+TC9SLKv14AWuCpZ4cntkYJy7Df2fKhRld3BD3TeXADJX1C9Sf GH8xaIy2i3q9kavLPTbzlVsj3F7xSRI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-668-jwZ3dLuJNq-_aRCVShqxqQ-1; Thu, 18 May 2023 07:35:54 -0400 X-MC-Unique: jwZ3dLuJNq-_aRCVShqxqQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7959E86C60C; Thu, 18 May 2023 11:35:53 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D35614171C0; Thu, 18 May 2023 11:35:51 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 12/16] ip6, udp6: Support MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:49 +0100 Message-Id: <20230518113453.1350757-13-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: 11107180006 X-Stat-Signature: qtbj1smq94a9ekygidtmrdguihfxp8mx X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1684409758-42015 X-HE-Meta: U2FsdGVkX18yY0ZGo9qR6rK5sqw6LgqizjNY9fn1s+a3e+ODjosvvxDRjMgmPB7h51U5Mf7f2ypaccGB1aycp5bVyrwqz1dmBgVqIEbg1QWF2AuBa1la1Nmww6arZ3agNw0JlKGOFTFTiE+B4j5Nho8JpXoA16GbjloUQplr1Hqrq0KxA+tdVKM4WqRfdCi6oIopnZ1F+2C/YtVg4SyyHPSnSiZPbK4OBCGSZ1TbsjcYZUOiV76Sq0pxty9bDyntJft5i3UNOLutDtIc9d9GOOz/Wzh3R2MB+3ODoDBgSx6HYSLanMSoFcXBWsjqQs0JmVjXrU19AYr5RLeJJ9XVRxI2n+F/4OMP00P51RmMIg1Hvag8GPrAXu03wSVcgdVu979AWwD189GP2HVZXhFlUlWqDblE5i5630SS1qDEONIzm5AgL6zfk9xH4/6JmHV97DJTgqWaj+CMYss7al8Ts8/fzQXp3Q7OVvIQ/2D3mi/qOsiTQED65zvuZXRpq225xIeP3ZX9I7Ogck0yAzKlLQJfMU1Y6Fd6HmXBQDqmKXGYZwbezdUQwplzbyAbDt0xRlI5fJxrJZmR5UeAdcVPiKYmkOVOZxEb/LeBrJlQDWUBpXjf6FSbs/dqZlWyUOgD7ci2z2oOzZvQDqgZVnjtyizUOgyqv8CRW3SIw3r3iFQyy8H6AFSNVh1ZJEWUWZ46XrPRbQ+G2envqWy+yLy/4KPKQSRjtjXPXrUzNtP3Fq8uLutu0y5r57DWcyBbIl6tR3H+Qlu5L0ra5rycVoxPOIQJx0szJeEyn8wIm6lsrJASF1BPxth0s4uqg3sMj9SwmNExd3E0C8N4iaMpaMZkFUupr/U33NdMIvFgwbru6y6Ter4yPzpNU+ajT6lHBXq2+I3HGYA7Dz/x0Z0FlNky/Lzb9zrLDeiwduDYeL/0hxtsVbGqLWiNR6bKFF7K3wU9syDn5zT3Ur43Da3YbPn Wq9NZgXs ZMKWrUZLo2P4onDTtnxyQ+7w+MTifTh7kWg37qnC4pGKga8t5SbL0eL1yp0/KVY7Cm0Euivnu4x+tt0eLMRWxN52oT4CLLj1ilSaS/zp3Lc5OOgCXCU7F0ZUcxFRJ1J3mEHKYfQdLkh2TLRi+nkpJZbMHj6rpXLZuudQo9j89lZqm1v/qH+CnsBBepm+uNWoqEqc/OV2WvPZg0AYZgN1OZcDYs2T7i6chR+JBgPVG7WLU95I/K+mDajxDOweGtBgaBHV9sB0gOXyhmZcd7RS2+xvCovy4LzqhiNXABaqMfLgCa1NdGAz+EWS87EnT4Q0983Tmf0UQPTiOn7hnSyZHw2YqcrA4sezA2zUxMTe7LnKgAQ57ySNzF7G6zUX/HgSeUKWwCITmv35Nenf8hcd7Ml5Vi1lTo6GInrUxiSV9/OCyoSIsbPSh57SzAX/0cIlxnp2FCm8Xs0g3GhdRgnBYP9Pgza8dL0oQyllOCkNkRpsJ7X6Ol+R53gUy9Zak2tnMbj6D7fmP0QIMh+E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make IP6/UDP6 sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible, copying the data if not. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/ipv6/ip6_output.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 9554cf46ed88..c722cb881b2d 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -1589,6 +1589,14 @@ static int __ip6_append_data(struct sock *sk, skb_zcopy_set(skb, uarg, &extra_uref); } } + } else if ((flags & MSG_SPLICE_PAGES) && length) { + if (inet_sk(sk)->hdrincl) + return -EPERM; + if (rt->dst.dev->features & NETIF_F_SG) + /* We need an empty buffer to attach stuff to */ + paged = true; + else + flags &= ~MSG_SPLICE_PAGES; } /* @@ -1778,6 +1786,15 @@ static int __ip6_append_data(struct sock *sk, err = -EFAULT; goto error; } + } else if (flags & MSG_SPLICE_PAGES) { + struct msghdr *msg = from; + + err = skb_splice_from_iter(skb, &msg->msg_iter, copy, + sk->sk_allocation); + if (err < 0) + goto error; + copy = err; + wmem_alloc_delta += copy; } else if (!zc) { int i = skb_shinfo(skb)->nr_frags; From patchwork Thu May 18 11:34:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DFEAC77B7A for ; Thu, 18 May 2023 11:36:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35F2228000A; Thu, 18 May 2023 07:36:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30F22900003; Thu, 18 May 2023 07:36:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1890490000D; Thu, 18 May 2023 07:36:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 09990900003 for ; Thu, 18 May 2023 07:36:04 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B2C481C75F6 for ; Thu, 18 May 2023 11:36:03 +0000 (UTC) X-FDA: 80803171806.27.7DA3CD1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 0C622180015 for ; Thu, 18 May 2023 11:36:01 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=I7FHcbAM; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409762; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=1sVW2QTlhw29Tkhrh9I6N8TqZ/F2m3qaPjkFmZ13QGxHQfMGIO/oedMveF/rmcugHgO/gm Ya2WbdJLmmB/yUfS4AL/dxaPYI+SR4/OVJdF2SFR0GtnPKWX/R8blBdmJ/R4ktIkxgsotI tyeV/8vlDI/a6u8xqP5y7luNu5AHkq4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409762; a=rsa-sha256; cv=none; b=phBMVg0stddF2983daQKR5q/J1kWBi0b5qkc9eE2og9wCluZQovOc8Tnrqu1+KS6e3OXkQ QogXc2hKMSKIYIgUHOPMfs1ekSGO9AZeIbOUYJEiDCMq97rHRfhsDKXYMkL6JP4iRFTkAz viwbmQpVigZl1rnJJhRA0gpp9ccQ6jQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=I7FHcbAM; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409761; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=arPLIYOMzcdUPWLjuF22tHo0vP4jVzFFThikgCzP+CE=; b=I7FHcbAMnATRGpSiON5mihkHy/Tk1MmKq+RITgCMrKym+/ncZOxuCoqleD/qKtadiaizro MJ+hvsa5DbTxNicRiwmBBKilXaH5rGoPwUOzEijDAaQ9/CCCp9PSMFnKARCTTbV8bk02Kc zSRGj5+AQ1GVYxAe8iWfM6otUsuQwPY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-253-zr9gaF3vOCeYO5wuwbnLTQ-1; Thu, 18 May 2023 07:35:58 -0400 X-MC-Unique: zr9gaF3vOCeYO5wuwbnLTQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 19F6086C60D; Thu, 18 May 2023 11:35:57 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6844440C206F; Thu, 18 May 2023 11:35:54 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 13/16] udp: Convert udp_sendpage() to use MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:50 +0100 Message-Id: <20230518113453.1350757-14-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Queue-Id: 0C622180015 X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: 7hehchejspa6jg7njfoxnz7e4oc7rfnh X-HE-Tag: 1684409761-966786 X-HE-Meta: U2FsdGVkX1/3n9rZuviM7NKR57nMMEvsdvrLQbCHcES+aPcNDS9I05g8u2zunpJuF/GY1hv2sB6LmfKwbrkFUx15N9Fon8J+KcO4s8cV2STVl07csOr69n4LSwsItkNEP9icrlYBi8kJBSalZmsQGGL5Bn4ZXT76R65ytIh8VOvkwktl9yNNj1IjC7uMxZrMKASAAN0+fxTr2L0fJ/IBougtUAPU2Ujj3KR9QTJTSuJ5bNjSbNef61rQLDElKgWwc5LBtqd4l4ybWkufgDCNfn4BXJCdOoL/G5lffHNvyEX3mAYw99Ll0cmyXUGhnyKxH1yyo5vLjK4q2FCr6U/ea50wbpBmRXdp+VkfvFSEWyccUmo9Jz5Uw08s9sqerFYd4bQOwbnPGuoOUc3DIdhNMFi3+8M6kcfp3AuI5/uE6EieSJXwu70BidRgOaJEeZGsh6uEIYiuFojG3sSPGPsCYfPXLySWO18zGtRB3gFR+Re24KCcdb7ADh9nqRSnMBNozIuxHt/NGlBiwNX1heIVt+vXdvOUBkZyr4RnFKEFtfa56U4kJ32KY7RtzXoTAay8B9LmuIfJ0+G8EDIXbXtdQVRF1E0fG/q89BOUHfUZzfK3N3NQbOQwyiI3+Xhftx7a5TadAVv/c8h2cYibNmm705yPMI0NWdZs66pXL/B4OLEd0hcKxto+xASGZ3GS4GY3W0CpjlivEuWgnt7eRE8/I8u/owgjVXfBrWdemQMpio8y2tLEZFCnVcizBKfRZ1+ksERp08ofsfUYn5RBmyRDVDNTaxV5PpoMcu70O+CR6tb6XRndTgP8v9nobzRPh7Yd1aAk4cJdOzLKMkDvVGTSsSItiH3Xiokf8Eaz/3PKVFqMgOM7oMHZ22UfB8SbzkdHiZtX5OEHxB9ERubGPkV0pv93uv9r5NXJhltc+gad9/+gxgLDnlZJqroB4JYkc/4LR4NLsqoPC5dQKKJnLQe tp21FALG d9s8mWHHaQC+2Roin+c2rFTTW8UkjHRLcHR+ReWPet8t1RUKqCFj3V8BY0F9ZWjnH2b+ZsK9TGHFJ06RbcdVpaLZ6Hxpu0o64FLUeleTS2M8OOB9kc80V6Q+mmI2Xozw+EeDB15CFzBnpvvv9Gi0RTOtZ0CkDN3EZ4zQfHabtE9PI8jBmHPojk1C7Fu/YA7f5WOXEn8lpUpYS6bv2DuAMJn96F5uAdFVExeoo37sbOeBai3wUOtq1Hviz1r1kf4Ys6QV4kRgd5EeRHZScowd4p6IrmQdc9CNVp9Fe8uPRw1+WxM/uG3UFDWYkDxDjzcGMpBQe/HAp6VFzvhHn3DMjZnenVNVa6lLfhmxXCYUhcj/ctMesReaXez6ofwmEt1X81DHzVAoa70TNI/ItAleOA3qzM28JyFE/4v9aNp/yaI7wZz4nCVI6tzEYirX0mpSq1YCD8xdsZJShPfE6jB1VaBbGAwt5+DoO+hCb6nCsMyOa33Q/lqTRF8V8L2RIH15ezzy5yP0Pxzf6d4k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert udp_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - udp_sendpage() shouldn't lock the socket around udp_sendpage(). - udp_sendpage() should only set MSG_MORE if MSG_SENDPAGE_NOTLAST is set. net/ipv4/udp.c | 51 ++++++-------------------------------------------- 1 file changed, 6 insertions(+), 45 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index aa32afd871ee..2879dc6d66ea 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1332,54 +1332,15 @@ EXPORT_SYMBOL(udp_sendmsg); int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct inet_sock *inet = inet_sk(sk); - struct udp_sock *up = udp_sk(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - if (!up->pending) { - struct msghdr msg = { .msg_flags = flags|MSG_MORE }; - - /* Call udp_sendmsg to specify destination address which - * sendpage interface can't pass. - * This will succeed only when the socket is connected. - */ - ret = udp_sendmsg(sk, &msg, 0); - if (ret < 0) - return ret; - } - - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (unlikely(!up->pending)) { - release_sock(sk); - - net_dbg_ratelimited("cork failed\n"); - return -EINVAL; - } - - ret = ip_append_page(sk, &inet->cork.fl.u.ip4, - page, offset, size, flags); - if (ret == -EOPNOTSUPP) { - release_sock(sk); - return sock_no_sendpage(sk->sk_socket, page, offset, - size, flags); - } - if (ret < 0) { - udp_flush_pending_frames(sk); - goto out; - } - - up->len += size; - if (!(READ_ONCE(up->corkflag) || (flags&MSG_MORE))) - ret = udp_push_pending_frames(sk); - if (!ret) - ret = size; -out: - release_sock(sk); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return udp_sendmsg(sk, &msg, size); } #define UDP_SKB_IS_STATELESS 0x80000000 From patchwork Thu May 18 11:34:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3F49C77B7D for ; Thu, 18 May 2023 11:36:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72F8C28000B; Thu, 18 May 2023 07:36:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DF5D900003; Thu, 18 May 2023 07:36:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A7E290000D; Thu, 18 May 2023 07:36:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 49677900003 for ; Thu, 18 May 2023 07:36:08 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1F61480527 for ; Thu, 18 May 2023 11:36:08 +0000 (UTC) X-FDA: 80803172016.22.690FF7B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 070B4C0012 for ; Thu, 18 May 2023 11:36:05 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IMrHobwu; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409766; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=0Bxbzb+DbKoXoMmsL7yYWekL4YvwVQ6cT0Z/R69f+KpkP6BY5/W9JQOwUzfRrEE8TE6YfF 09LTNH1ifHf3ksxvZpNwZejhiMpzQ4kYQ1diYb8AQ4lANfrwj1d0u0hlor6r2LpujTrD2C IBk2JFy58475l40HwTPMQLdbCbzsNC8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IMrHobwu; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409766; a=rsa-sha256; cv=none; b=htBy71gAAZAIt6M2zk+9m0P4U1puW5i+NwR+OUZxCdhKCKjiYiiYOXHIaQBqGfvJYe4Lzk 0VEgwdh3uSyD+uqyqQtAvYOWk+tAAoElFfrIF6gHhFIVjb4Rpk6J+y8J/5Itv1+mDDyvUq zGy3sw0TX7wcxctKCy6G6msrzK3lX20= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vVs0CsGrBKFATDm79TCVboT9gSnZlC7XDeXTGZfAJzM=; b=IMrHobwuwL0OdVOO8/wULcxQnoc3GDU6fTkdLJ0WR4D/jJ3cYeuaoNhr63cA5EMrX8fOqp QKeLfzYAP5qn0CoR9iP0sej4kzZOutzIKw2ZOqQHBfgvww65YmnKvVwIaB7Ezai5VS8TbF 7+ih65QqcJEkA5weUc0mJHoNsZ7Ie0A= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-266-miPRkfAKPhyxWJ8vCer5_A-1; Thu, 18 May 2023 07:36:01 -0400 X-MC-Unique: miPRkfAKPhyxWJ8vCer5_A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C56AD10146F8; Thu, 18 May 2023 11:36:00 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A1FB2166B31; Thu, 18 May 2023 11:35:58 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 14/16] ip: Remove ip_append_page() Date: Thu, 18 May 2023 12:34:51 +0100 Message-Id: <20230518113453.1350757-15-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Stat-Signature: f8eku1t4ih6uq4m76sss1mgn7861dte1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 070B4C0012 X-Rspam-User: X-HE-Tag: 1684409765-692473 X-HE-Meta: U2FsdGVkX19lUvxjSMLh2MXyT4u8czobv/CXc0hwfHUZZdiwrGdKsAWYn+JseGWfAfBzsIRjobQ+/CyJAADSFT7BIVvAUVLHykh78W8UJ1X15C2vqMJuIrVdPGgHn9l/gDWl1EEox6NFT9ZxmWm5VXltMkpTOfTrdkck5MTciTsKvBlfAlnXcBqqu7i2plytb8ooYfzCKMT88u8YIcfoHrWtkX2TySPqRlXtn1gMFZ84dt36T1V4bSYFfiEn6DauKTbeeEsEmyQ/NPpPZG4+LvI2zKwMJAEkSVO2nHResKoPFJvfZGF+Lzt9EmpO6UMHf6N5CEfrhQpLf9ZbeHNe7fSM+oO4n89IvkrrCzFu2KhOSVg2WOa1L2S07MlhfieW7gMIEXaDjvQE30dhSPnkYIWNHaJLJClihtKQuhWN+kZz/32NvdiNlLAdZpIzGsgNSibo3xCF0lT7RmqsAlfnfxm8AJ5iQbtO3rdYHGsdPIqpnjCkEWSAs/XgTcHB/UZ3MR5q32QXe1oJZyIFUNKZwXrUwozfIy8ErUiXcGu6brVsloj+rGaaQaa3ZipXHieDy9Ova/1yeV2kfbJ8DVuv4kR5VZZWaWLHE/7XdMwDtERAtDsl50ahA+wMewQIKuGZs7bqJS6MblhnU6fzoGO110y+Lv4RqfO0PjxVVYdMFVPFohXITOGxRMN1tvMVcNNT6GxunrfIHcL1qzgOBqXghiPsIc1IlK//NItX4tSCVYGKWd6EXcSnapRIVLxN8ku+aFU/e58gac2NH2xM2McL6UmeFoi2oBDlr/qLz4o36me1OUeviFKWmfGNTTnSdG75+pAGTpkqMxGFqgjW28ee5vwDEj8S46BwW6j76VBtdE0+q1XmA4QC0RziwciVVUD5tMVRLPi4h0stXZCjd5XNmC3YuY3SPvXfSmww0/ypSWNUv2CnP9BvJHBeEzxPA9RBQ6nCW84hBUkFwF0QW8v ZQne/oqT nUnuz+WjfITOSj06X4N7DLOfI/cUCkRpAFFne77r6WbvLm9Kn4ZvXArMkd64umE/VmPImuqjmmiaZ1KvsUcrqOWUNHBFMNjcexyiDGu6SJS0XPg5zh/HbnMkloO8vDpIM+Pyrkxpy775BQqjeqs1a6JzZJyuSQkV8Y06DBRAki5/xG193vzXxpcRPH9YsBpWhU4TD0t31no1+jyLx35ka0gFVxe+Ffxwcgxt+x0/oLVAUPCpbSDu1j/zJFgBQ3bIl9QWtE8UnaoeUZVaBSFyf+sgI5zUop118TjuZwNiWNsugroXopsasdcO8g0p5CASjfxlrjAN0QCz1XINPXiJSzIAjAdalexcgl/rJj2MPHyXe8flM4HLm0L4nqm0vbV3rNvvcA+Jd6hXrzdk450RuK+rlCBWKPdwSyLcogEwysGxHr73tMLH27bKdeNfG7Y0z6u4jBH6YMg4DElKp504vNQHS0TYJipTVY2/jrzYvTT3PcVG5ohhb8KM/aDgzo33qr511PbPhtlaIBcY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ip_append_page() is no longer used with the removal of udp_sendpage(), so remove it. Signed-off-by: David Howells cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #7) - Remove now-unused csum_page(). include/net/ip.h | 2 - net/ipv4/ip_output.c | 148 ++----------------------------------------- 2 files changed, 4 insertions(+), 146 deletions(-) diff --git a/include/net/ip.h b/include/net/ip.h index c3fffaa92d6e..7627a4df893b 100644 --- a/include/net/ip.h +++ b/include/net/ip.h @@ -220,8 +220,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, unsigned int flags); int ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk_buff *skb); -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags); struct sk_buff *__ip_make_skb(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, struct inet_cork *cork); diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c index c7db973b5d29..553c740a6bfb 100644 --- a/net/ipv4/ip_output.c +++ b/net/ipv4/ip_output.c @@ -946,17 +946,6 @@ ip_generic_getfrag(void *from, char *to, int offset, int len, int odd, struct sk } EXPORT_SYMBOL(ip_generic_getfrag); -static inline __wsum -csum_page(struct page *page, int offset, int copy) -{ - char *kaddr; - __wsum csum; - kaddr = kmap(page); - csum = csum_partial(kaddr + offset, copy, 0); - kunmap(page); - return csum; -} - static int __ip_append_data(struct sock *sk, struct flowi4 *fl4, struct sk_buff_head *queue, @@ -1327,10 +1316,10 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork, } /* - * ip_append_data() and ip_append_page() can make one large IP datagram - * from many pieces of data. Each pieces will be holded on the socket - * until ip_push_pending_frames() is called. Each piece can be a page - * or non-page data. + * ip_append_data() can make one large IP datagram from many pieces of + * data. Each piece will be held on the socket until + * ip_push_pending_frames() is called. Each piece can be a page or + * non-page data. * * Not only UDP, other transport protocols - e.g. raw sockets - can use * this interface potentially. @@ -1363,135 +1352,6 @@ int ip_append_data(struct sock *sk, struct flowi4 *fl4, from, length, transhdrlen, flags); } -ssize_t ip_append_page(struct sock *sk, struct flowi4 *fl4, struct page *page, - int offset, size_t size, int flags) -{ - struct inet_sock *inet = inet_sk(sk); - struct sk_buff *skb; - struct rtable *rt; - struct ip_options *opt = NULL; - struct inet_cork *cork; - int hh_len; - int mtu; - int len; - int err; - unsigned int maxfraglen, fragheaderlen, fraggap, maxnonfragsize; - - if (inet->hdrincl) - return -EPERM; - - if (flags&MSG_PROBE) - return 0; - - if (skb_queue_empty(&sk->sk_write_queue)) - return -EINVAL; - - cork = &inet->cork.base; - rt = (struct rtable *)cork->dst; - if (cork->flags & IPCORK_OPT) - opt = cork->opt; - - if (!(rt->dst.dev->features & NETIF_F_SG)) - return -EOPNOTSUPP; - - hh_len = LL_RESERVED_SPACE(rt->dst.dev); - mtu = cork->gso_size ? IP_MAX_MTU : cork->fragsize; - - fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0); - maxfraglen = ((mtu - fragheaderlen) & ~7) + fragheaderlen; - maxnonfragsize = ip_sk_ignore_df(sk) ? 0xFFFF : mtu; - - if (cork->length + size > maxnonfragsize - fragheaderlen) { - ip_local_error(sk, EMSGSIZE, fl4->daddr, inet->inet_dport, - mtu - (opt ? opt->optlen : 0)); - return -EMSGSIZE; - } - - skb = skb_peek_tail(&sk->sk_write_queue); - if (!skb) - return -EINVAL; - - cork->length += size; - - while (size > 0) { - /* Check if the remaining data fits into current packet. */ - len = mtu - skb->len; - if (len < size) - len = maxfraglen - skb->len; - - if (len <= 0) { - struct sk_buff *skb_prev; - int alloclen; - - skb_prev = skb; - fraggap = skb_prev->len - maxfraglen; - - alloclen = fragheaderlen + hh_len + fraggap + 15; - skb = sock_wmalloc(sk, alloclen, 1, sk->sk_allocation); - if (unlikely(!skb)) { - err = -ENOBUFS; - goto error; - } - - /* - * Fill in the control structures - */ - skb->ip_summed = CHECKSUM_NONE; - skb->csum = 0; - skb_reserve(skb, hh_len); - - /* - * Find where to start putting bytes. - */ - skb_put(skb, fragheaderlen + fraggap); - skb_reset_network_header(skb); - skb->transport_header = (skb->network_header + - fragheaderlen); - if (fraggap) { - skb->csum = skb_copy_and_csum_bits(skb_prev, - maxfraglen, - skb_transport_header(skb), - fraggap); - skb_prev->csum = csum_sub(skb_prev->csum, - skb->csum); - pskb_trim_unique(skb_prev, maxfraglen); - } - - /* - * Put the packet on the pending queue. - */ - __skb_queue_tail(&sk->sk_write_queue, skb); - continue; - } - - if (len > size) - len = size; - - if (skb_append_pagefrags(skb, page, offset, len, - MAX_SKB_FRAGS)) { - err = -EMSGSIZE; - goto error; - } - - if (skb->ip_summed == CHECKSUM_NONE) { - __wsum csum; - csum = csum_page(page, offset, len); - skb->csum = csum_block_add(skb->csum, csum, skb->len); - } - - skb_len_add(skb, len); - refcount_add(len, &sk->sk_wmem_alloc); - offset += len; - size -= len; - } - return 0; - -error: - cork->length -= size; - IP_INC_STATS(sock_net(sk), IPSTATS_MIB_OUTDISCARDS); - return err; -} - static void ip_cork_release(struct inet_cork *cork) { cork->flags &= ~IPCORK_OPT; From patchwork Thu May 18 11:34:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64A41C7EE23 for ; Thu, 18 May 2023 11:36:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02A6628000C; Thu, 18 May 2023 07:36:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F1D25900003; Thu, 18 May 2023 07:36:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0B6128000C; Thu, 18 May 2023 07:36:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CE609900003 for ; Thu, 18 May 2023 07:36:13 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A38ABAE1CB for ; Thu, 18 May 2023 11:36:13 +0000 (UTC) X-FDA: 80803172226.15.FAB029B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id EC8D31A0017 for ; Thu, 18 May 2023 11:36:11 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=b3uGA1jf; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409772; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=d5pH3Rf9Az1rT/o+dk5h7S4O3OTsZbNI52mz6tb7Sp55ymo8YuW78GPgWOgCO8MuMFrBwy sLHE/lANB0b1YNAYhP8MoEyM83VvSvaZlYFGDrKrEg+NO2Xcvl0CO9kT9jGMhHfuO3XT2t /OiLF3vvPp+cTxlnHA16PK4aF7QS/Aw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409772; a=rsa-sha256; cv=none; b=J7NYzEdZc6jBzqhjUyzP4DioJ3gCo7MA4rmVIRLc+FBL/vNR2QIDtf5VmiC6jXCII8mAxN J00QIvLEbUEb07hqG4orDgWKvGPiD1PUGoHiHIrofqUNRmqKQSWGwSluZQCCRdjspqb7DZ YDRIgW8Llr9Dmt9GtYc7QZROBINNXjo= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=b3uGA1jf; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4sbDm+4sgaynAjmiQFVbDl+OyfXvG2vWGJtRH+IAST4=; b=b3uGA1jft/K4VwXzui8owpcLpNXIYj5OLn1RjgLewPgyxwdO6ed+o8DewMxb4SQ2eFZp3J BqpghkjUqwt3z96PYKtRxwO8PyxALYB4ane+CiUHS9oiceczit/X5TUkHpWquVtLjQXz+V 3oUQX6qlSxFKvJGVmjQaAadgsZ4jpSs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-sU7nZ7tANKyeN_LLALrjqw-1; Thu, 18 May 2023 07:36:05 -0400 X-MC-Unique: sU7nZ7tANKyeN_LLALrjqw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 121C787DC27; Thu, 18 May 2023 11:36:04 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8346BC0297C; Thu, 18 May 2023 11:36:01 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v8 15/16] af_unix: Support MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:52 +0100 Message-Id: <20230518113453.1350757-16-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspamd-Queue-Id: EC8D31A0017 X-Stat-Signature: oz165u5n4qnzjp8ej7duq9xf1aphs7di X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1684409771-531404 X-HE-Meta: U2FsdGVkX1/sfUYmhWmOJOdUkVRaM3Z/J3YI4f1/neDEDLZaCWLR9qoXZ26A1pmISl+bGtU0nmxnYfTkgwX1odYU5Nihe45ggN5HMTJOMtQvJ5AzIX7jfZI0kiD191GBnuT17fT3DHdaxxtQqPwllp5YXwZns4fa+EBz5WSew6i+x0kKnr+ZL7r+Fubu/VQ5QvYmiWRVPcZVkLL3yhW1XZ2lLpiJAeGtyieCP7E0wm9mhRzniW85fb0S0xCXNunu9JYambLXC/HAdnR1F3xkAsXIsNoSmNLGMfmjj+33XqXTGa1ptd7hA+KkFDxcIZOgpXyqh2hy3vtKojFoP/8QSPzrozPIEX8xdNg2dJM2aviIk56+fFwcRNa0QhKwkKUw3xtBmWIvuOnUnxa1FGzholVhjiVAl9oLiYr4VSKBsSOFWwBvLrTFY18vLtO0RU29oRFgF2ZQeJMqyp9s0gjc8UVbRsKMeTQ1SNv4xUama2kQzWKEYv3zgZMYCsv2C0NNhWqajGe4nH/oej+fVr/XmVuutPTOpQ1mtOWqqT7k+tFOYtlbP3t4QA46XEN6VwTsW1ok7QmoG5tWV06PW/YCrvYkkUgZPpQsjoKMwjw7iD/GjAIrd4K2KVFabpK7hsG/SRZkGO2Ro3fm0lMvn5TDkSqLQY2zQBVKMddoIjWal4WZN79lia+Npn3bo9TGxvdXFQhce9VlAFDLPQyN+xiRuGCIv7yH8yWqzd0nmjYrK0nHy3Yk1CqOnyBbmllxluYRZPQx9+eBgOxoO8EacbVtD4GihwUtutMU9stT7hR3Ek78C9uqJ1Mac0DhhMRmBO6iAvdAqzrL3KDtt12Sc4NuA74/ko5SfIOkksyw4MnQMq7ylBih3KBMKBO0A/Rs69BXKfXKgXYnGO8129/w7LhwFaKjxP09EhJ29Aww8x1QdJriKKYFetzYp2zOl67txtztYiGJg9BBL5uLxL7X/f3 elHGW/ru iMRV38TipuDLZAF+TWn16o84XlQHE8Psu/Mti7TroCjg2dBBwE0EeAskAP5uwK+W94CL1uPBoYr+CIJvcJGYZBrWD3reNysk5gNeAokUH+Z1j8omplczfOBKCVZXMVx9vNTbABWVP+khpXlin8pYDS+U7QtAs6fI6aFqT+7i4PrHi6Lfwj50V4rL2QbaWBhKdqfORRPcEVHKE7nlwViskHqBss9t0Rq4LtFpuYuoFOLFq2CNEerm5P3fn+J1oeT7QYkXzyMMBIX5Q57Vh4unYyzZxpHvi3wuUBYwdsP8o4AmGj/C/x7n0iFn9UB6iUlHSayxG76e6zs6nOyq6HKqlrese0hhDJE6R5K+0vchFC87Kz/U8E4PMXODxSqOnnwMR8DZRavdriJLDvFXN1v6C1jZSDc1qVJ6bSWik0tZbKKXxig01AewCsKIHVfgfYMGrUFZVlG5B0IcWeyHFE9EteFN06AgHOO1HmwTqzRPr1JF+44P32Ls6jjFwBJlEtWST1hLMHLyAWzn4OBM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make AF_UNIX sendmsg() support MSG_SPLICE_PAGES, splicing in pages from the source iterator if possible and copying the data in otherwise. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - Use common helper. net/unix/af_unix.c | 49 +++++++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index dd55506b4632..976bc1c5e11b 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2200,19 +2200,25 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, while (sent < len) { size = len - sent; - /* Keep two messages in the pipe so it schedules better */ - size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + skb = sock_alloc_send_pskb(sk, 0, 0, + msg->msg_flags & MSG_DONTWAIT, + &err, 0); + } else { + /* Keep two messages in the pipe so it schedules better */ + size = min_t(int, size, (sk->sk_sndbuf >> 1) - 64); - /* allow fallback to order-0 allocations */ - size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); + /* allow fallback to order-0 allocations */ + size = min_t(int, size, SKB_MAX_HEAD(0) + UNIX_SKB_FRAGS_SZ); - data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); + data_len = max_t(int, 0, size - SKB_MAX_HEAD(0)); - data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); + data_len = min_t(size_t, size, PAGE_ALIGN(data_len)); - skb = sock_alloc_send_pskb(sk, size - data_len, data_len, - msg->msg_flags & MSG_DONTWAIT, &err, - get_order(UNIX_SKB_FRAGS_SZ)); + skb = sock_alloc_send_pskb(sk, size - data_len, data_len, + msg->msg_flags & MSG_DONTWAIT, &err, + get_order(UNIX_SKB_FRAGS_SZ)); + } if (!skb) goto out_err; @@ -2224,13 +2230,24 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, } fds_sent = true; - skb_put(skb, size - data_len); - skb->data_len = data_len; - skb->len = size; - err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); - if (err) { - kfree_skb(skb); - goto out_err; + if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) { + err = skb_splice_from_iter(skb, &msg->msg_iter, size, + sk->sk_allocation); + if (err < 0) { + kfree_skb(skb); + goto out_err; + } + size = err; + refcount_add(size, &sk->sk_wmem_alloc); + } else { + skb_put(skb, size - data_len); + skb->data_len = data_len; + skb->len = size; + err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, size); + if (err) { + kfree_skb(skb); + goto out_err; + } } unix_state_lock(other); From patchwork Thu May 18 11:34:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C385FC77B7D for ; Thu, 18 May 2023 11:36:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D05228000D; Thu, 18 May 2023 07:36:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58084900003; Thu, 18 May 2023 07:36:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 420C890000D; Thu, 18 May 2023 07:36:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3189C900003 for ; Thu, 18 May 2023 07:36:17 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 049FF807F0 for ; Thu, 18 May 2023 11:36:16 +0000 (UTC) X-FDA: 80803172394.22.6F9D85E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 3592314000A for ; Thu, 18 May 2023 11:36:14 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Ubtdy+D/"; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684409775; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wF+2b4fBEfy00AN3VDBr6eYpnbX7vBgnsz0rKmJdTGo=; b=x+mpQI/80zF7P1mfK2kcPNV4t7JquEpuH09qhx+2yvwMMsyDegr7wa4wzFpvGjZc3T0/w4 GXS+9mGKpLktyDXoQjexEi6vHNvWPyMn3yChPBw//LVcSDLbRInEch71noKdCw3ZYOswub 3By53m3lnHE67yz3SGEpYYPxnSFDE14= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684409775; a=rsa-sha256; cv=none; b=uJrGPLciMabDKNm8/dGDjWMby7GL3ASyJqJWTCx3nie8TSu15tFqcjj9w44UeIOpAoAyuX ifq3QHhw2qcS/A97qULFTSESaFSh4YTD9sylDxnNEdIc8yM6Ha1OikbVuGDb9Ej1dOQcOv kkVWK1tfnajU2k/yjoqZHpQ2dfj5skA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Ubtdy+D/"; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684409774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wF+2b4fBEfy00AN3VDBr6eYpnbX7vBgnsz0rKmJdTGo=; b=Ubtdy+D/v4UaSjv/o4BnFlPhdTteNlBNQRzyzAm3YLzL/1uzmeNuX42ynXQfHaRvcJ+juv SrUy8j+ukj1EQvHJEEf7//92/Ka1zof5PpuoahR8sAbKT5hnpfEieQDokeZifvGe9LksQl Lim8YnId/QTtq99C0Jp5wQL10PrMRbQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-321-vqjt2jLQMMKuS89fOjFPaQ-1; Thu, 18 May 2023 07:36:09 -0400 X-MC-Unique: vqjt2jLQMMKuS89fOjFPaQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F189729DD98B; Thu, 18 May 2023 11:36:07 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 371ACC15BA0; Thu, 18 May 2023 11:36:05 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Kuniyuki Iwashima Subject: [PATCH net-next v8 16/16] unix: Convert udp_sendpage() to use MSG_SPLICE_PAGES Date: Thu, 18 May 2023 12:34:53 +0100 Message-Id: <20230518113453.1350757-17-dhowells@redhat.com> In-Reply-To: <20230518113453.1350757-1-dhowells@redhat.com> References: <20230518113453.1350757-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspamd-Queue-Id: 3592314000A X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: pd7c8mcetwp1cwdp4fhpuxwa98o3qjze X-HE-Tag: 1684409774-133343 X-HE-Meta: U2FsdGVkX18vyHWdf902gqG9gQ0K7RZ3wCzqkvNVI5ijHlq2xZ95WUWd/ASxoWO6IqwRDiL0B2IvrmpIM0CnVd8nQU0hpT8zULibc5U8HyBvb8S0grUMmkG0TZahr986884PaxO/wA+Ffh97J6yla0fF2Vz/wN7VxpXxFwjG96yKc0WUFtX7vf3h4YopDI3TVAxdkiDhw0xfpKe3KdVfqiyJOuKj3RGK5TQt01kn/bi4lTI4gcdAeHpJBTvGbkfOutL2EvzJDojY986Aowf7wpMGgLTUjlKd1lFtnzEPq3Ua+CW5R1rgf9f6hzNv/sn+qsZMEIdAmk0bQ3OrB33StzsPEHoO1RgtBjDilNS5ZcDadAx/3uP74FbKgG/E4uxofV1hHinIbdY4P2HUDAzRMibSnhcqelKYYyyKou3q54xgnavKbxuLUp3uoVbP3k5+eoL9M5z9TsZxbIrXktG9m27EL56gK9SsfWar6gR0GemIdTCjkODnmTmxxhfD8U3VKKJN8x3Q+xHFxchGS89weQTwi6G3w8Br5Zz7VbyOllKH/K0b7gF2oL8q3CyX4iUxPHT+iG2nFt1QQ7lhuiWvTxNVAKsEi2k/5nJQHIJUTxUJclTiSGsq0f8fAMgYuBSYbrTwbxGHpDu0ZfJD2bZnCWP8ipZymI9NxL25x/VZ0VPlfapHCKPV1sjbK81B3ZDfBcKtHvLBY4mtpMfoP4or6DoVB/Wgjz3ssKQwNr0mzv4A62m9zMUwunBJWy4bMQuYzhTQDDl9/lsCH3SGHrsj1kVCZz0vfLo8lJTppJGtQTalL+Fw2COFxvyOzla5H1SqGvvtpdjt2vmRQs9aLmptHUuLGW4j+nNBO6+aNuw7k4PVCy3Gqvllbe7i3c7+OtQn89tDGW6KcyMDqKTFDcto6x0CLtfIjfocIjmhUAO1sdApV3WUHbQQiIRVf4pwsXfhQV0u7XfX3CRoZsXhVtX HMPLVtZ9 wHKTFnZ9xDhrdv6WSc9VabeIufPwjtT9ZJ2I03vOdnhYTx3jHbPbNlnNDqH5rWgvHzehvN9qKvpP7UelT8OCShgZrzkd0tRij1oyZLqpf4n4UgjPV3xdDfifwE9jba+tKDdqj+vR7wq62lAJlk+CcP0C50Sz7VZ/pKhMbZI6VhspGrNOU8kGLuLqymmB3xhlkJ+S4MFw8GNQzjUcAQnH2tKCd0BrAymJ/CIMPDHKO8h39Uha6Xp8edU0M6PAbh1wJp1qqUsujnCbN6KejVz6M/ryZsl8JV0803dChDeeA9EXcgDvIURSFzDgMbR/BxVPYZaaZlDOg2BTrazaPD+T2d/jkguPdDCARV2J+NmH7CjkWgn6gH0q4VbYdrNgdMRLlLfpNwYPit8A6xbtYWFa2yhzckQXJYQVkO0yZLjbVatyDVL2IwJs01Wyc8X1wPbrTPhUPJG+SnsAYke6LA/DpAZRmhcUgt5wm2eDDf+j/gmBugxNV5yGQqGPD426lpD4SYb9ywPtASvEh/rI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert unix_stream_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Kuniyuki Iwashima cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/unix/af_unix.c | 134 +++------------------------------------------ 1 file changed, 7 insertions(+), 127 deletions(-) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 976bc1c5e11b..115436ce1f8a 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1839,24 +1839,6 @@ static void maybe_add_creds(struct sk_buff *skb, const struct socket *sock, } } -static int maybe_init_creds(struct scm_cookie *scm, - struct socket *socket, - const struct sock *other) -{ - int err; - struct msghdr msg = { .msg_controllen = 0 }; - - err = scm_send(socket, &msg, scm, false); - if (err) - return err; - - if (unix_passcred_enabled(socket, other)) { - scm->pid = get_pid(task_tgid(current)); - current_uid_gid(&scm->creds.uid, &scm->creds.gid); - } - return err; -} - static bool unix_skb_scm_eq(struct sk_buff *skb, struct scm_cookie *scm) { @@ -2292,117 +2274,15 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg, static ssize_t unix_stream_sendpage(struct socket *socket, struct page *page, int offset, size_t size, int flags) { - int err; - bool send_sigpipe = false; - bool init_scm = true; - struct scm_cookie scm; - struct sock *other, *sk = socket->sk; - struct sk_buff *skb, *newskb = NULL, *tail = NULL; - - if (flags & MSG_OOB) - return -EOPNOTSUPP; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES }; - other = unix_peer(sk); - if (!other || sk->sk_state != TCP_ESTABLISHED) - return -ENOTCONN; - - if (false) { -alloc_skb: - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - newskb = sock_alloc_send_pskb(sk, 0, 0, flags & MSG_DONTWAIT, - &err, 0); - if (!newskb) - goto err; - } - - /* we must acquire iolock as we modify already present - * skbs in the sk_receive_queue and mess with skb->len - */ - err = mutex_lock_interruptible(&unix_sk(other)->iolock); - if (err) { - err = flags & MSG_DONTWAIT ? -EAGAIN : -ERESTARTSYS; - goto err; - } - - if (sk->sk_shutdown & SEND_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_unlock; - } - - unix_state_lock(other); + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - if (sock_flag(other, SOCK_DEAD) || - other->sk_shutdown & RCV_SHUTDOWN) { - err = -EPIPE; - send_sigpipe = true; - goto err_state_unlock; - } - - if (init_scm) { - err = maybe_init_creds(&scm, socket, other); - if (err) - goto err_state_unlock; - init_scm = false; - } - - skb = skb_peek_tail(&other->sk_receive_queue); - if (tail && tail == skb) { - skb = newskb; - } else if (!skb || !unix_skb_scm_eq(skb, &scm)) { - if (newskb) { - skb = newskb; - } else { - tail = skb; - goto alloc_skb; - } - } else if (newskb) { - /* this is fast path, we don't necessarily need to - * call to kfree_skb even though with newskb == NULL - * this - does no harm - */ - consume_skb(newskb); - newskb = NULL; - } - - if (skb_append_pagefrags(skb, page, offset, size, MAX_SKB_FRAGS)) { - tail = skb; - goto alloc_skb; - } - - skb->len += size; - skb->data_len += size; - skb->truesize += size; - refcount_add(size, &sk->sk_wmem_alloc); - - if (newskb) { - err = unix_scm_to_skb(&scm, skb, false); - if (err) - goto err_state_unlock; - spin_lock(&other->sk_receive_queue.lock); - __skb_queue_tail(&other->sk_receive_queue, newskb); - spin_unlock(&other->sk_receive_queue.lock); - } - - unix_state_unlock(other); - mutex_unlock(&unix_sk(other)->iolock); - - other->sk_data_ready(other); - scm_destroy(&scm); - return size; - -err_state_unlock: - unix_state_unlock(other); -err_unlock: - mutex_unlock(&unix_sk(other)->iolock); -err: - kfree_skb(newskb); - if (send_sigpipe && !(flags & MSG_NOSIGNAL)) - send_sig(SIGPIPE, current, 0); - if (!init_scm) - scm_destroy(&scm); - return err; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return unix_stream_sendmsg(socket, &msg, size); } static int unix_seqpacket_sendmsg(struct socket *sock, struct msghdr *msg,