From patchwork Fri Jun 23 11:44:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F095C0015E for ; Fri, 23 Jun 2023 11:44:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6748C8D0003; Fri, 23 Jun 2023 07:44:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64A168D0001; Fri, 23 Jun 2023 07:44:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5118A8D0003; Fri, 23 Jun 2023 07:44:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 42BA58D0001 for ; Fri, 23 Jun 2023 07:44:44 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 112AA80F08 for ; Fri, 23 Jun 2023 11:44:44 +0000 (UTC) X-FDA: 80933830488.19.4033E72 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf07.hostedemail.com (Postfix) with ESMTP id C1B2540005 for ; Fri, 23 Jun 2023 11:44:40 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H4hJL5Rj; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ivhWzudgyaDY8ijkLDZpWPEWsQie4rsiJhQ0Ls+XdqI=; b=6nqEEJ6W05t2xL/4K1moktI/QEQla+3xMy/evwgKxrJbABkC4+nn0LnOIgpraiEgUu0+rE SYUyJQzRSV4nfYqB2Lihs727B3KDhVaghQdpHrPS0Hvpt5s5yOF/BoslXz07/KcMSYebhn wNMmBx4S/Eql3pFZOyAVeX8V2RgEIDI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520680; a=rsa-sha256; cv=none; b=M9CucJs4Hy9sUaKW2mKWaBBxcMJoCxXXowLgJYu4lEcebiZ068IqQPjcS2Z24CA7E4Rpla VuejqYUIWUVvLP8iGfW+1VpEM//Is+hHba+vn0ZajI9jfUJryW/eVPa5ny6oYwIH/Opmbg HoojVRNd2G+xYmOj3zK7whMkv7QYIiA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H4hJL5Rj; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ivhWzudgyaDY8ijkLDZpWPEWsQie4rsiJhQ0Ls+XdqI=; b=H4hJL5RjAdlViedPr81FLTq/UWdebsfe/jwFc4GVnkNgcJ2icgOCoMEqk2Mw13bGLX6rYc PEFAXSzCmhei4zVcVS8UhIUVHPkUXpppmUqV9hzMOqvuaFCfOa/SLkrpfqwVexmh/yOE2Y K99u3a9oNTDiBxpfIzK08+0SWmvY6xo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-644-PFLeHdiDMpWe9Zuq8IfeUA-1; Fri, 23 Jun 2023 07:44:34 -0400 X-MC-Unique: PFLeHdiDMpWe9Zuq8IfeUA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 419AB38149A6; Fri, 23 Jun 2023 11:44:33 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C9611400C35; Fri, 23 Jun 2023 11:44:30 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, John Fastabend , Jakub Sitnicki , Karsten Graul , Wenjia Zhang , Jan Karcher , "D. Wythe" , Tony Lu , Wen Gu , Boris Pismenny , Steffen Klassert , Herbert Xu , bpf@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH net-next v4 01/15] tcp_bpf, smc, tls, espintcp, siw: Reduce MSG_SENDPAGE_NOTLAST usage Date: Fri, 23 Jun 2023 12:44:11 +0100 Message-ID: <20230623114425.2150536-2-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Stat-Signature: ks7ufh747jufoyszg9ajnqoo8puisoo5 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C1B2540005 X-Rspam-User: X-HE-Tag: 1687520680-952919 X-HE-Meta: U2FsdGVkX19Em0XphTtBjlR9828Y29WIjKiVNsCHJnnWnLAZXFsYOrLmMRHCzAG+qiJ0/EBLw8hZI1ZlPW9UPw37XbPB1eTEhdyzArxcCuIsRm6bCMIivi1TLp1Y5a8sS+AeU7n8/LGvqdXq7EkicPIdvji0JCAY3hwqOmffgezpuZ8kvru2XF0zoi80GsTWHfedjgayPePPYNkeBOK/EkkLXOdlJf3Bil81IiGKhLsAeArgBDYW72vsBjqyhj+A+DcNbBVYs0yrxFk/wkkyJjw226vtRgiclWvhrmaQz1rsWjZ8wlCLu5Hs7MHntSAgBbTbeHKMl96/Zp2uru+5hYipSvTh3LehrqW1B6NVMLBxiHdM6LtNidz2lwxJXXP89OFj8SzCxnD6CJfDwaYOetHz9AIFB8wetMY4wB6g5hLD8GMYfsCJPWUpq4zqPQUy76Ki9ity0NIGgC1GN/mUKdWiPeswW+r+uJCXkOjHjyrancy9/ipULgbcWwJPG3grCw58Hkv+m53qBcwqDDV9tIvuLvm2JqEn+1dkCW65gTZI4G+13DOLTh8b8LGq+H60L48AhNyzeadbp9E2xZ0RUGezGrdlfrG2rsPkR3TkVGA6o0PCGSW0/FSPOd2A66/mLR5geLWpwMh98NKni753azqxWKoTCFckHZZJxueXlh56JgyTNMm9I1PtHQt3zrDZCu/7hsrOiviJdna9lMOx4fuA/dkly4YxazVE5crh0K8DWmhYEuX0SixYP+sfEtqev7KwFRi+55USbbMrDf5sLm2SM0SGxYXwJWuwsnoH/2zYU9QlZVsjGP9wvjETbqtHlM5L9OzESqBxGACT881IUfDZ9XMEZaDI8YYk/gUqUeaiRQulvqs8yeohS0YdKAWdFqYEPxLS2qAkL8ylGolGVC9NL8Vp0BBNRNV8hc6TpL+h17KQKK1AvKz0XLSMUHn/VhCZmWkBG1Xriy+0fje xYqinzEy FNQX3FXMKgZeiPVdbCVB0HFHeMITPHF/Ujy4ZW5p+k9kWqLDt3nhH4aBkNz0RMpI0AmfQQSXeOpgt+w1mKkysEnwJAhaKfpAtYRO3TH8sq4d4Jos6TQyKI/qZtV367EQZwdaGkeui7b0teTVyv7I/vHGVhaSeWAAF6/eeFLl8RHThAeOPE5Tt5DRJtgWVHr+fKCTeLQw6SBcyse18H46by0FYdIiZrD+3U3YAqoVUio/ZR5amvrMV4Gp+bEuokP27nynCYEzDJqCeU24aOe10G47X1PD2aMtIzKJwFvahz/zUd0OkBwaUQpZfUUiWCJbk/F/I2uXi/Bzpx7ucVgayeHybppkvrj9lkzlekoqibJdseZUv1StkMh5E5CkC3ko9OvzGPJ7yyU8/LNrihIBwqcAdWvubgX8LJmUw9saz9hBeLBBCpX9aWBkmNSH4Src5BMrWN2KtWqWsmXHvppb8/nrz+uuvOiCuX7xrtP/JxKO5HPEHvC7mZr+p0DMvcR7C6+Dzw3xPV7nEO0gjUBYai3f3HA7dQ1pITWQd/y7KgNT+A+g9RKmGD7utPXGbWWinzOKMYA2MaGed2/EokIFIkNrl4gg5HOLvHxUH3nHSuRvr5HneK7jYXDwwXlzfgaOk+fTciqDXGgVCW0vnnwW31Ped7YUmHCOyWfNc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As MSG_SENDPAGE_NOTLAST is being phased out along with sendpage(), don't use it further in than the sendpage methods, but rather translate it to MSG_MORE and use that instead. Signed-off-by: David Howells cc: Willem de Bruijn cc: John Fastabend cc: Jakub Sitnicki cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Karsten Graul cc: Wenjia Zhang cc: Jan Karcher cc: "D. Wythe" cc: Tony Lu cc: Wen Gu cc: Boris Pismenny cc: Steffen Klassert cc: Herbert Xu cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org cc: linux-s390@vger.kernel.org --- Notes: ver #3) - In tcp_bpf, reset msg_flags on each iteration to clear MSG_MORE. - In tcp_bpf, set MSG_MORE if there's more data in the sk_msg. drivers/infiniband/sw/siw/siw_qp_tx.c | 5 ++--- net/ipv4/tcp_bpf.c | 5 +++-- net/smc/smc_tx.c | 6 ++++-- net/tls/tls_device.c | 4 ++-- net/xfrm/espintcp.c | 10 ++++++---- 5 files changed, 17 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c index ffb16beb6c30..7c7a51d36d0c 100644 --- a/drivers/infiniband/sw/siw/siw_qp_tx.c +++ b/drivers/infiniband/sw/siw/siw_qp_tx.c @@ -325,8 +325,7 @@ static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset, { struct bio_vec bvec; struct msghdr msg = { - .msg_flags = (MSG_MORE | MSG_DONTWAIT | MSG_SENDPAGE_NOTLAST | - MSG_SPLICE_PAGES), + .msg_flags = (MSG_MORE | MSG_DONTWAIT | MSG_SPLICE_PAGES), }; struct sock *sk = s->sk; int i = 0, rv = 0, sent = 0; @@ -335,7 +334,7 @@ static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset, size_t bytes = min_t(size_t, PAGE_SIZE - offset, size); if (size + offset <= PAGE_SIZE) - msg.msg_flags &= ~MSG_SENDPAGE_NOTLAST; + msg.msg_flags &= ~MSG_MORE; tcp_rate_check_app_limited(sk); bvec_set_page(&bvec, page[i], bytes, offset); diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 5a84053ac62b..31d6005cea9b 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -88,9 +88,9 @@ static int bpf_tcp_ingress(struct sock *sk, struct sk_psock *psock, static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, int flags, bool uncharge) { + struct msghdr msghdr = {}; bool apply = apply_bytes; struct scatterlist *sge; - struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; struct page *page; int size, ret = 0; u32 off; @@ -107,11 +107,12 @@ static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes, tcp_rate_check_app_limited(sk); retry: + msghdr.msg_flags = flags | MSG_SPLICE_PAGES; has_tx_ulp = tls_sw_has_ctx_tx(sk); if (has_tx_ulp) msghdr.msg_flags |= MSG_SENDPAGE_NOPOLICY; - if (flags & MSG_SENDPAGE_NOTLAST) + if (size < sge->length && msg->sg.start != msg->sg.end) msghdr.msg_flags |= MSG_MORE; bvec_set_page(&bvec, page, size, off); diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c index 45128443f1f1..9b9e0a190734 100644 --- a/net/smc/smc_tx.c +++ b/net/smc/smc_tx.c @@ -168,8 +168,7 @@ static bool smc_tx_should_cork(struct smc_sock *smc, struct msghdr *msg) * should known how/when to uncork it. */ if ((msg->msg_flags & MSG_MORE || - smc_tx_is_corked(smc) || - msg->msg_flags & MSG_SENDPAGE_NOTLAST) && + smc_tx_is_corked(smc)) && atomic_read(&conn->sndbuf_space)) return true; @@ -306,6 +305,9 @@ int smc_tx_sendpage(struct smc_sock *smc, struct page *page, int offset, struct kvec iov; int rc; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; + iov.iov_base = kaddr + offset; iov.iov_len = size; iov_iter_kvec(&msg.msg_iter, ITER_SOURCE, &iov, 1, size); diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index b82770f68807..975299d7213b 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -449,7 +449,7 @@ static int tls_push_data(struct sock *sk, return -sk->sk_err; flags |= MSG_SENDPAGE_DECRYPTED; - tls_push_record_flags = flags | MSG_SENDPAGE_NOTLAST; + tls_push_record_flags = flags | MSG_MORE; timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); if (tls_is_partially_sent_record(tls_ctx)) { @@ -532,7 +532,7 @@ static int tls_push_data(struct sock *sk, if (!size) { last_record: tls_push_record_flags = flags; - if (flags & (MSG_SENDPAGE_NOTLAST | MSG_MORE)) { + if (flags & MSG_MORE) { more = true; break; } diff --git a/net/xfrm/espintcp.c b/net/xfrm/espintcp.c index 3504925babdb..d3b3f9e720b3 100644 --- a/net/xfrm/espintcp.c +++ b/net/xfrm/espintcp.c @@ -205,13 +205,15 @@ static int espintcp_sendskb_locked(struct sock *sk, struct espintcp_msg *emsg, static int espintcp_sendskmsg_locked(struct sock *sk, struct espintcp_msg *emsg, int flags) { - struct msghdr msghdr = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + struct msghdr msghdr = { + .msg_flags = flags | MSG_SPLICE_PAGES | MSG_MORE, + }; struct sk_msg *skmsg = &emsg->skmsg; + bool more = flags & MSG_MORE; struct scatterlist *sg; int done = 0; int ret; - msghdr.msg_flags |= MSG_SENDPAGE_NOTLAST; sg = &skmsg->sg.data[skmsg->sg.start]; do { struct bio_vec bvec; @@ -221,8 +223,8 @@ static int espintcp_sendskmsg_locked(struct sock *sk, emsg->offset = 0; - if (sg_is_last(sg)) - msghdr.msg_flags &= ~MSG_SENDPAGE_NOTLAST; + if (sg_is_last(sg) && !more) + msghdr.msg_flags &= ~MSG_MORE; p = sg_page(sg); retry: From patchwork Fri Jun 23 11:44:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90ECCEB64D7 for ; Fri, 23 Jun 2023 11:44:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 080508D0002; Fri, 23 Jun 2023 07:44:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 02F658D0001; Fri, 23 Jun 2023 07:44:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E39EF8D0002; Fri, 23 Jun 2023 07:44:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D47AD8D0001 for ; Fri, 23 Jun 2023 07:44:43 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 94A6F120EFB for ; Fri, 23 Jun 2023 11:44:43 +0000 (UTC) X-FDA: 80933830446.20.B56C226 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf29.hostedemail.com (Postfix) with ESMTP id C16AD120002 for ; Fri, 23 Jun 2023 11:44:40 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Y6uvUhXE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=96le6Nv7f7Jm2em1nCUehde2Ld5Ng9GbgAM0NyDY2H8=; b=xCkXn5Ztuy3f1sKalcEyG3oNp40p5LElUEerZOIvcMZNE7VPZ06BI0CBwptisqgEwbqvl6 9Iac/YMBS8CrXn5b9DpFAk3v5s9+JHlllrpZi2s3PEZSEzHpvp0Eh82RcQv5Ko3ishBmTy yQysHWZsunStyXx1JlGapAa1+RNQ53s= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Y6uvUhXE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520680; a=rsa-sha256; cv=none; b=MW6TvXcnmHQEJyOvF7hbAKebM9L4eQKkANz1/m04bhWUG/wN2rDsDXKESvspevNMKORx5t YCl7g28spn++gRAuFNA/qNSYv+4FO0gRC7Rswy6bRsRUBk7PYe2JvgwKtJIDE0/met4LLR uBvHSmCv+uVdoTsthq7oDUjQh6i6RQ8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=96le6Nv7f7Jm2em1nCUehde2Ld5Ng9GbgAM0NyDY2H8=; b=Y6uvUhXEda8+Keyk/fxi6Go0Lfi4MNduTvr2blv+qN9SZxFtMNgaXnQfnyhroy7aZsTXzX k3YmCua69rfK8vnWYGS6dVCMLsHEiGXebtkMLKreeSVt/GaN8CLPFC5zfrS/4wBUKrU3Tf nkTEgz8CqiZPnmimffQPkPba9Wx3FE8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-bx3c_Z3YOMW40JuokbnynQ-1; Fri, 23 Jun 2023 07:44:35 -0400 X-MC-Unique: bx3c_Z3YOMW40JuokbnynQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 520C68E44E6; Fri, 23 Jun 2023 11:44:35 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id E13781121314; Fri, 23 Jun 2023 11:44:33 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 02/15] net: Use sendmsg(MSG_SPLICE_PAGES) not sendpage in skb_send_sock() Date: Fri, 23 Jun 2023 12:44:12 +0100 Message-ID: <20230623114425.2150536-3-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspam-User: X-Stat-Signature: cmnmawrssbf173rpag7uwcqnudonmdaj X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C16AD120002 X-HE-Tag: 1687520680-277111 X-HE-Meta: U2FsdGVkX1+GEghyRPSLWYB76PMRg0egJeqE4qVpGfWJ3C/EAOMmhNVujRiU98w9QCpCjhrQ+h3CwF3qhISxDasUEYh4L4waFfTlHgbxmNlb+pqz0hrs+Ck8NL5lVdWC9wXTKHKdOaNH8tRSLqYmkMTkysuuVAtr5RwnG1gYP3JQrHK/1FaUo0l4ILrK7/sbQqJNf7LbRvGdKvHrMSzBnBXmvvRG2tq9MDp7W3hJNwVGkBf/h165DnTp4eVe6aMnVIoLCkjeLQ2zVEbG8PAqt+HDbZ9a2zlE6h96KQm/sLSQK3xla78p2k6ad/NTBERHM0jQ76JTpwLK5NRGdP8ndegTT78Zen0FvdWXf9SZUEUVETcxGrSO0mDqOLyCsc2fVv7QKvhD7RQg3mb5YG46it4gTY45E8X2GgLim0ZVyFlq5F+5MHEkd681HsDyyEq+UVbO197n8RnKyUoklfG2Fu0aELiI0a2sga8TEgfeLZ/LeJeEJuZQXKCLyigyCX4r2bMvZtfhgHrwJJGbBKDdlSCm8e7LPH61C9fcP1d03x/RSs2U2eX3MZ2nnIPOBS1pkPpWSTZHufRHKWn53K0yR5Uv6KIO+hc0WfGWrUmdO6SfTE61yYK64ZZ+LE5UO56b4pogC3jl+SZNI3PuFrh28xzSKQNcTaIoQ8JNXBXHNJ2j+6Ra+AEQqrfEFapLelQ+1hbqr0oNzOSZtl7rPx9L6yblgbCzqFx0eTcCwVQyjG4wQk8TD5HNFhSGSKmzQkhYVyL0FOt8WqF1ZLtID6ydBt8oeoUxW43UuT+hAGrgoe/4bpW6vFDgYPsuqR2Q+O2r9K3tOjzo4ZGt6PPKv1pTOywaUd7n8cUCDAo495ATfSEhv02di3zUc1iBCY6A1JlHP/IIJ8cdE9QawVFrHlMPd11Yxi02aGrsqCvqK/V+swRmIFmEy0n9vePzdWyLx8CSa1xn6K4QBaczyVxVt6G QyPzRWQy yrUtRXyaN4aubOqHjMnCDZp8f/yFnTYBq66SMMjzDRTqTr0MzT/z1pMhUgBBLHfwkriwIaSOTWSoCvRRmUgLvPE0iKmPqviQCuKXauHBmhcVZbWRXtgFqeSrYKyn/WMYZRDwsX7pKUugPwVlU/hwEUl3vco7fZTFboBVcDFQi2MUIuZQVmTr4Msy+6F96T9jBmvbUSyaLpFXQHJDG7qhXAoolMcND/KGyL79XblbvA8XRIfKR5+NG0f4WaIWdfytWO6eT4w35UEIdLy0RpsoCNgMU7cO/VNWJjgw/p1Jd46dJkgS4b1GkjRa9g+t4JGD0g3ljekP2JYuvVrI5KzUXUdsLcIxkZNTjBbmeIhmuc4LW9JhhW7gzvaXze+K0waRMjCTcwRw5COI+twh3Ueq86MI3c7mU5GirJY9XhAgMmod0101PUvE2MIcBD3VsgJz7qTqP4SfWwKdTr8dpxYjN9vpEzFYgGGkzMzIaPxVU5EolXyGLknDpC+HfLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use sendmsg() with MSG_SPLICE_PAGES rather than sendpage in skb_send_sock(). This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Note that this could perhaps be improved to fill out a bvec array with all the frags and then make a single sendmsg call, possibly sticking the header on the front also. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #2) - Wrap lines at 80. net/core/skbuff.c | 50 ++++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index fee2b1c105fe..6c5915efbc17 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -2989,32 +2989,32 @@ int skb_splice_bits(struct sk_buff *skb, struct sock *sk, unsigned int offset, } EXPORT_SYMBOL_GPL(skb_splice_bits); -static int sendmsg_unlocked(struct sock *sk, struct msghdr *msg, - struct kvec *vec, size_t num, size_t size) +static int sendmsg_locked(struct sock *sk, struct msghdr *msg) { struct socket *sock = sk->sk_socket; + size_t size = msg_data_left(msg); if (!sock) return -EINVAL; - return kernel_sendmsg(sock, msg, vec, num, size); + + if (!sock->ops->sendmsg_locked) + return sock_no_sendmsg_locked(sk, msg, size); + + return sock->ops->sendmsg_locked(sk, msg, size); } -static int sendpage_unlocked(struct sock *sk, struct page *page, int offset, - size_t size, int flags) +static int sendmsg_unlocked(struct sock *sk, struct msghdr *msg) { struct socket *sock = sk->sk_socket; if (!sock) return -EINVAL; - return kernel_sendpage(sock, page, offset, size, flags); + return sock_sendmsg(sock, msg); } -typedef int (*sendmsg_func)(struct sock *sk, struct msghdr *msg, - struct kvec *vec, size_t num, size_t size); -typedef int (*sendpage_func)(struct sock *sk, struct page *page, int offset, - size_t size, int flags); +typedef int (*sendmsg_func)(struct sock *sk, struct msghdr *msg); static int __skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset, - int len, sendmsg_func sendmsg, sendpage_func sendpage) + int len, sendmsg_func sendmsg) { unsigned int orig_len = len; struct sk_buff *head = skb; @@ -3034,8 +3034,9 @@ static int __skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset, memset(&msg, 0, sizeof(msg)); msg.msg_flags = MSG_DONTWAIT; - ret = INDIRECT_CALL_2(sendmsg, kernel_sendmsg_locked, - sendmsg_unlocked, sk, &msg, &kv, 1, slen); + iov_iter_kvec(&msg.msg_iter, ITER_SOURCE, &kv, 1, slen); + ret = INDIRECT_CALL_2(sendmsg, sendmsg_locked, + sendmsg_unlocked, sk, &msg); if (ret <= 0) goto error; @@ -3066,11 +3067,18 @@ static int __skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset, slen = min_t(size_t, len, skb_frag_size(frag) - offset); while (slen) { - ret = INDIRECT_CALL_2(sendpage, kernel_sendpage_locked, - sendpage_unlocked, sk, - skb_frag_page(frag), - skb_frag_off(frag) + offset, - slen, MSG_DONTWAIT); + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SPLICE_PAGES | MSG_DONTWAIT, + }; + + bvec_set_page(&bvec, skb_frag_page(frag), slen, + skb_frag_off(frag) + offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, + slen); + + ret = INDIRECT_CALL_2(sendmsg, sendmsg_locked, + sendmsg_unlocked, sk, &msg); if (ret <= 0) goto error; @@ -3107,16 +3115,14 @@ static int __skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset, int skb_send_sock_locked(struct sock *sk, struct sk_buff *skb, int offset, int len) { - return __skb_send_sock(sk, skb, offset, len, kernel_sendmsg_locked, - kernel_sendpage_locked); + return __skb_send_sock(sk, skb, offset, len, sendmsg_locked); } EXPORT_SYMBOL_GPL(skb_send_sock_locked); /* Send skb data on a socket. Socket must be unlocked. */ int skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset, int len) { - return __skb_send_sock(sk, skb, offset, len, sendmsg_unlocked, - sendpage_unlocked); + return __skb_send_sock(sk, skb, offset, len, sendmsg_unlocked); } /** From patchwork Fri Jun 23 11:44:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 915E1EB64D7 for ; Fri, 23 Jun 2023 11:44:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB9C48D0005; Fri, 23 Jun 2023 07:44:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A68B88D0001; Fri, 23 Jun 2023 07:44:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BC668D0005; Fri, 23 Jun 2023 07:44:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7B8EC8D0001 for ; Fri, 23 Jun 2023 07:44:45 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2AE2E40F42 for ; Fri, 23 Jun 2023 11:44:45 +0000 (UTC) X-FDA: 80933830530.06.D855F7E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 5E7674001C for ; Fri, 23 Jun 2023 11:44:43 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=J94SljvU; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520683; a=rsa-sha256; cv=none; b=fMTyXRmCr/bbJrygJNwMYfckryMNRd985ujFbzrx3GaX5Ecix3/n3pj57strjULMU8m++K rtiKdCmFXKndR17QD8TSxA5DbY9vNka9LW3BzI7yJe7TOBIcYxUMFLirJE6piHfEfeo/Q7 aq18LZTMZIfEm4u1ZNw81Wvh5RA2Orc= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=J94SljvU; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520683; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1DmsLXNf6UEopicQGVGHzvB3R+Io/k4xNBk1zidfJPQ=; b=iW7anLRG8fLy/v4KHr5Wfc98CqPvgC6kxeraP/S5iajhkwSXO/SSvGy61rxh1RbhaL83jm G4eRVApDRSo0WG1Y47b+fpHcVha1n6heiEGOssvDD6ar6Ss1BKEKhdi+TpFt59jVQ/btr/ 2EMGLSCKUNr3XrGFthQwknMCA59KYNI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1DmsLXNf6UEopicQGVGHzvB3R+Io/k4xNBk1zidfJPQ=; b=J94SljvUS0QQbDIAyfuodj6r6csmOLISkOlGpZWFVhUWINpCXuRNAMjmb6EZBwssAafNtW z8EF0WfzVErG1YmhqG1wfxHBTMx/E2kLc6lM3QSwRVpfHxwLws8t2YQGIQbBXiZ0q5mVyw dBgcp8tG08OqbV5Z4/GOdO0T9nJUzjs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-217-Kak7x7OWO8uDfkuCBYwaPg-1; Fri, 23 Jun 2023 07:44:38 -0400 X-MC-Unique: Kak7x7OWO8uDfkuCBYwaPg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A8F823C0F696; Fri, 23 Jun 2023 11:44:37 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id DA55E492B01; Fri, 23 Jun 2023 11:44:35 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ilya Dryomov , Xiubo Li , Jeff Layton , ceph-devel@vger.kernel.org Subject: [PATCH net-next v4 03/15] ceph: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage Date: Fri, 23 Jun 2023 12:44:13 +0100 Message-ID: <20230623114425.2150536-4-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5E7674001C X-Stat-Signature: 5wndt5enjcrefaeogenoewfckmb6uwoe X-HE-Tag: 1687520683-852416 X-HE-Meta: U2FsdGVkX18Hn3twE6kHPl7HAngAtLaHlL2Gzh6+BB3Ypg7eKGQYq4A+BbyPvIb+1oXZhp++8Joig1hvV4Q6LvdGZfUkPPG0BQ2/tR9t/noZbN4NarB27u1UhKTx+GqNnsIYe5r1F84e8znwS+TmVZvblMMCU0fKPPyjQRGlFWL1TdPBi2Bzgu5U4laqVGfaiW1n9in1Lifj6vLl96zHbvJU6+jrPso+gEnDVDLTRLHyaILnJQeUBH2yWWZYnxfk37J4wOes/LfZuES8A1mUwMJz1i+3SRnfjLtK8V6tLwhHIRJvZAlPoMMziqL85IFvNCXyI5vn8oxzAfcFmGrUbM7fptPYT27ForS43AY50hdAaN9e8Fzo733WzkRLGHqTCzXjzhWAs86tF6mmAcb2l4gfdquEmjvMGdZxCT0rEOI6hwAjHyG+BIJ7H3nTR9w9+2PVznX8CiMFV+8mnlZC18VFHIU4nChPtyYVHquzPqvuEvcQDXnCIkBl6bg0XZYY99MADZkxphSYm83ITdB4lIKehGvJX+1rJ6s9ogr6bByB+q/xMTotD7jhNscgIyNHSJbVsWRQjYLgVS9jBROWEx8FaLs8Ua6ZS+5DudoLocwF1EBK3lSVSWK2AlWCYo/jHE7tIDE0+9LCSpOHEn7PEMqnCIA5J+bPJwyDxfKmrAj5I0t6bpI7Po82zx7rsS4TPDowneXES3EL5oOTmP/Otoy1oDLGqpHdN3ocuH7+Zrs99aQxKgYp7Yb+PX+isQPXNExRe6wsHNLMg2nRlUwPqrgYw2qPINi9ctebRPxauhttwUVVIJc0KxTcmJWkuoKaQ8fTuAR/eguHKIqzsNhu266hviCvvPl1esE2vJUsEn7KgiFtA6k68B2i9alicV4L537kuZ5f7BRHkU7BME2rTh6emKxvBCyPurn7+BSWyhzbF1lWDvTwB4JdJz0wfxgYpF5OeoK2Z6GdL6CYqL4 dd7cLkbX JgxO+hvbxtP2Xuso6o1iDsxO8jsIzqaIkNW1jmsJCzaPghGjJivlCgZCT3PG6XdL2UpHCrzIguaUffc/C0C2kWdVcMRbV7VLY5wNG5Iym1PePfwrIj6xWmPyoO0ol9IGUheY8Tx4UWhcdxIAluR4bjywBj1N+Z5A/atyYY4fHCpYfuT+zcGQSo21rtb8UuPiL6m144hLmLDSwySTwhkBM0QqPmHuvp+dwfBvAI1ocOVgiipYze5of/n/C+CUUg8J59EGpR3vbE34yKSP9kph0BgF8zaX/t+qv9UVxN/YyfUY9QH84lKZZ8k1t8txaZABzzSFDILRTB0XEBYslxawOtiay0BmdnrSlgsVmXl4ebwRJP3O+WtOmO8DXbmTZHpX5RX26AskaAcgRYb8xXkkK+ziiBUm9uYBWbnsZFb4IA0PnvT406dKwui7jAOJD0fJ02CEWqbmmFv0lIFwUhZEm86SL4D4Os/kJTX2hr408Tdgay+qUW+mJ//PL2WlQjqgt5xVPqAWCces6lEE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use sendmsg() and MSG_SPLICE_PAGES rather than sendpage in ceph when transmitting data. For the moment, this can only transmit one page at a time because of the architecture of net/ceph/, but if write_partial_message_data() can be given a bvec[] at a time by the iteration code, this would allow pages to be sent in a batch. Signed-off-by: David Howells cc: Ilya Dryomov cc: Xiubo Li cc: Jeff Layton cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: ceph-devel@vger.kernel.org cc: netdev@vger.kernel.org --- net/ceph/messenger_v1.c | 58 ++++++++++++++--------------------------- 1 file changed, 19 insertions(+), 39 deletions(-) diff --git a/net/ceph/messenger_v1.c b/net/ceph/messenger_v1.c index d664cb1593a7..f082e5c780a3 100644 --- a/net/ceph/messenger_v1.c +++ b/net/ceph/messenger_v1.c @@ -74,37 +74,6 @@ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov, return r; } -/* - * @more: either or both of MSG_MORE and MSG_SENDPAGE_NOTLAST - */ -static int ceph_tcp_sendpage(struct socket *sock, struct page *page, - int offset, size_t size, int more) -{ - ssize_t (*sendpage)(struct socket *sock, struct page *page, - int offset, size_t size, int flags); - int flags = MSG_DONTWAIT | MSG_NOSIGNAL | more; - int ret; - - /* - * sendpage cannot properly handle pages with page_count == 0, - * we need to fall back to sendmsg if that's the case. - * - * Same goes for slab pages: skb_can_coalesce() allows - * coalescing neighboring slab objects into a single frag which - * triggers one of hardened usercopy checks. - */ - if (sendpage_ok(page)) - sendpage = sock->ops->sendpage; - else - sendpage = sock_no_sendpage; - - ret = sendpage(sock, page, offset, size, flags); - if (ret == -EAGAIN) - ret = 0; - - return ret; -} - static void con_out_kvec_reset(struct ceph_connection *con) { BUG_ON(con->v1.out_skip); @@ -464,7 +433,6 @@ static int write_partial_message_data(struct ceph_connection *con) struct ceph_msg *msg = con->out_msg; struct ceph_msg_data_cursor *cursor = &msg->cursor; bool do_datacrc = !ceph_test_opt(from_msgr(con->msgr), NOCRC); - int more = MSG_MORE | MSG_SENDPAGE_NOTLAST; u32 crc; dout("%s %p msg %p\n", __func__, con, msg); @@ -482,6 +450,10 @@ static int write_partial_message_data(struct ceph_connection *con) */ crc = do_datacrc ? le32_to_cpu(msg->footer.data_crc) : 0; while (cursor->total_resid) { + struct bio_vec bvec; + struct msghdr msghdr = { + .msg_flags = MSG_SPLICE_PAGES, + }; struct page *page; size_t page_offset; size_t length; @@ -494,9 +466,12 @@ static int write_partial_message_data(struct ceph_connection *con) page = ceph_msg_data_next(cursor, &page_offset, &length); if (length == cursor->total_resid) - more = MSG_MORE; - ret = ceph_tcp_sendpage(con->sock, page, page_offset, length, - more); + msghdr.msg_flags |= MSG_MORE; + + bvec_set_page(&bvec, page, length, page_offset); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, length); + + ret = sock_sendmsg(con->sock, &msghdr); if (ret <= 0) { if (do_datacrc) msg->footer.data_crc = cpu_to_le32(crc); @@ -526,7 +501,10 @@ static int write_partial_message_data(struct ceph_connection *con) */ static int write_partial_skip(struct ceph_connection *con) { - int more = MSG_MORE | MSG_SENDPAGE_NOTLAST; + struct bio_vec bvec; + struct msghdr msghdr = { + .msg_flags = MSG_SPLICE_PAGES | MSG_MORE, + }; int ret; dout("%s %p %d left\n", __func__, con, con->v1.out_skip); @@ -534,9 +512,11 @@ static int write_partial_skip(struct ceph_connection *con) size_t size = min(con->v1.out_skip, (int)PAGE_SIZE); if (size == con->v1.out_skip) - more = MSG_MORE; - ret = ceph_tcp_sendpage(con->sock, ceph_zero_page, 0, size, - more); + msghdr.msg_flags &= ~MSG_MORE; + bvec_set_page(&bvec, ZERO_PAGE(0), size, 0); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, size); + + ret = sock_sendmsg(con->sock, &msghdr); if (ret <= 0) goto out; con->v1.out_skip -= ret; From patchwork Fri Jun 23 11:44:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4F55C0015E for ; Fri, 23 Jun 2023 11:44:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42A5C8D0006; Fri, 23 Jun 2023 07:44:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D92A8D0001; Fri, 23 Jun 2023 07:44:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 252D58D0006; Fri, 23 Jun 2023 07:44:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 172CA8D0001 for ; Fri, 23 Jun 2023 07:44:49 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C7808120EF9 for ; Fri, 23 Jun 2023 11:44:48 +0000 (UTC) X-FDA: 80933830656.03.C860D8A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id F131518000C for ; Fri, 23 Jun 2023 11:44:45 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="GwI//IE1"; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520686; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fSQZQ6YboxRaXsiohOL5R9kf0uRRv2lhrE9A+pyCn4U=; b=fA1o71coQXQ8If3Yc4lrUFAQEIUI0BOPMvf7SzyrGYWRToXiJY49efAh5ISV9CagqqK8h4 YXQKFv0So3LrW+Cq3akNylTWIrfwrGxHc/O2pPShRLge6GhpQXDMGIL2wV9Y0ypePJUEr+ EqfOiH+ZqLf1IdP2w2c5NMISzV+1WqA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520686; a=rsa-sha256; cv=none; b=drL0DBlHJHqJ1CoZrXfq7dKSHnp06YmVOzFVlak9jxLLz/GxCBSMP5V1ZYfDB04niTF+gP rzmiDektj9wecDGVu2P2RpmVs2vvLWAGF+y8Tf/SFRPsAhdeToXrZyxrl9UU6NWYr0upTO 3hHXJz9sj4IbYaGeGwlde7QLJCsUpYI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="GwI//IE1"; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520685; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fSQZQ6YboxRaXsiohOL5R9kf0uRRv2lhrE9A+pyCn4U=; b=GwI//IE15xq2AWv3QmrtNM4n3JxWoafMnGQtH2ewwG6v397CRr2aam3P7K0sHQoYBo6Pba /V7dXwkbpltqArLfscW2Kolqy+scY8E7GohDO07zZlkN0kHzi2nu/o6HsC0aAiRi18Eoui YoRzMb3cC5Lw7XInGIpu3IsN9Or7USk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-642-5G0_5PVBNSa9A5rtzK9tkA-1; Fri, 23 Jun 2023 07:44:41 -0400 X-MC-Unique: 5G0_5PVBNSa9A5rtzK9tkA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 090B828EC10F; Fri, 23 Jun 2023 11:44:40 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D0A7492B01; Fri, 23 Jun 2023 11:44:38 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ilya Dryomov , Xiubo Li , Jeff Layton , ceph-devel@vger.kernel.org Subject: [PATCH net-next v4 04/15] ceph: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage() Date: Fri, 23 Jun 2023 12:44:14 +0100 Message-ID: <20230623114425.2150536-5-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: F131518000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5psq5qib493a1n8ncge4c96k8pc57gwd X-HE-Tag: 1687520685-557191 X-HE-Meta: U2FsdGVkX19rW9+txeYaCYTQ1ImZNdsiUEqlsoj2eZPK90OD8Y4X0Ckqhj1+zzdCGLYezqj6cgpHC5/9/GT081NC8Ne9ytiotGouBH9z7U0mSeiUw9G3l1/hjIijUZ6u82itBxRS65d3Wsj+h5W9pCZihPHEGOuVQssEtlGlMNyl9mlAkHJTcDVmYFjfzAd5aDDeyFAAju+g5RMlf0F946Y6LCJtmgNZkXigOV5lqP2/N1claxLSN20gN3IRg9feE6CTP9g401IIsA2RmrNcgOf1fKPFlQtgLX+ZR28F2sfSntKQvCAFSdJoK4NGo2S/SC2wNv6vSRDlmKTD/X4KFzugWlvI/4Al297a6C1fKpZ14K9FXZ/Hw5buXphLGwpRTXyrjF8LKsGN2QxkOCXh/6JxyRUDcu3xpYYGPKqH4h0swCk1Cqxph44ZVMETx5UmA0QB2oGMeT/XCYmWk9EIvKpuMei2wcxdzTg+Nbm7NY7WO6ik9ZDZuLeQq2qzVQld6JYeLiEphroG8JwsIUabxKrU/HGK/+bbO/h/IU3aleylFSUNaiauoPvml5h44pkqyEIeSlmaefynWPziw3iKoylv5WwtOxpUb27gtby3KYTLrn0X+TwQP8AzY3dx03mqcYDFqDLIVE4e507gJFcoFJ/XVQBS8D91jUepAvUZvB3Wipvp/fPR0rZaMKUnoLYz/CatczWlzdfC26+ZThFJw8YJXU+r32R3q5ylKid4R1ShGn6LI6cBVLvkNxGeUGDdiyfWv4D+/JgN4pnT5puKPT5uaeL1AOjxy2WKUAuMogCB4af/DDlLQCoqZe2z0r1mHmZPdtBhlYOsHgPk+Y15CNx8uab4Z+1RHzWFfjlKWyLKJDIrmBtUdGYnptJ2K1y3xEn9hbvXhyi4ylYeCT/k7qNdfKWxvKtGYvnXhDobrjomYq2Y9EwGkOY1PDC2PuVB4rUbGbc/HcLtiYF2++e LEU68XBw P3Sh7gVyTQjA/PiXcvnS8TF7eOAUl/jAl2Xt8QnMI6HOkBauwNf/w85qbnYKRwRZZDJGzRuX0yi8IcaDBYjHtpUEfWmJtyyEzmrgcpNgTlxGiGLijt2p/5dFN+xp++b1CSv72kJA17rwALfW0g3o3sMPKuomzLMcH9YsyjZMQ/O9tuc+8HZVZguPlFj7G2xQ/ODbQyVCNLeQroA8NT/SsdI5UjH1r4DOSyco6Thq0E0A1gIZicHbGbBcUyhX71mNqTEkpXrqplpfapeXjs7wV9Pw4TMgobW779acVqLl5ZUwe+PvrpIbECs9Lqh0ZAeLQBuhkS5MiEQhzL9jV16r0WRaLcgm3UTuAZu1YwJ/WWCLjnEvhJGT1D9ZstUNr57KjD0DV0cXazoPYZ3N8kMPByWW3QMPSH+ajYyep0SxpirmsBpvS5blPpAqE5YMvQll1yfyDB9f3ZClsAeLuy2LqYQG3i+wnD10sHmSKq8mc1Ek5+0g84kvZf7lt0+r7CgaSlL23AEK+tes58vE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use sendmsg() and MSG_SPLICE_PAGES rather than sendpage in ceph when transmitting data. For the moment, this can only transmit one page at a time because of the architecture of net/ceph/, but if write_partial_message_data() can be given a bvec[] at a time by the iteration code, this would allow pages to be sent in a batch. Signed-off-by: David Howells cc: Ilya Dryomov cc: Xiubo Li cc: Jeff Layton cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: ceph-devel@vger.kernel.org cc: netdev@vger.kernel.org --- net/ceph/messenger_v2.c | 91 +++++++++-------------------------------- 1 file changed, 19 insertions(+), 72 deletions(-) diff --git a/net/ceph/messenger_v2.c b/net/ceph/messenger_v2.c index 301a991dc6a6..87ac97073e75 100644 --- a/net/ceph/messenger_v2.c +++ b/net/ceph/messenger_v2.c @@ -117,91 +117,38 @@ static int ceph_tcp_recv(struct ceph_connection *con) return ret; } -static int do_sendmsg(struct socket *sock, struct iov_iter *it) -{ - struct msghdr msg = { .msg_flags = CEPH_MSG_FLAGS }; - int ret; - - msg.msg_iter = *it; - while (iov_iter_count(it)) { - ret = sock_sendmsg(sock, &msg); - if (ret <= 0) { - if (ret == -EAGAIN) - ret = 0; - return ret; - } - - iov_iter_advance(it, ret); - } - - WARN_ON(msg_data_left(&msg)); - return 1; -} - -static int do_try_sendpage(struct socket *sock, struct iov_iter *it) -{ - struct msghdr msg = { .msg_flags = CEPH_MSG_FLAGS }; - struct bio_vec bv; - int ret; - - if (WARN_ON(!iov_iter_is_bvec(it))) - return -EINVAL; - - while (iov_iter_count(it)) { - /* iov_iter_iovec() for ITER_BVEC */ - bvec_set_page(&bv, it->bvec->bv_page, - min(iov_iter_count(it), - it->bvec->bv_len - it->iov_offset), - it->bvec->bv_offset + it->iov_offset); - - /* - * sendpage cannot properly handle pages with - * page_count == 0, we need to fall back to sendmsg if - * that's the case. - * - * Same goes for slab pages: skb_can_coalesce() allows - * coalescing neighboring slab objects into a single frag - * which triggers one of hardened usercopy checks. - */ - if (sendpage_ok(bv.bv_page)) { - ret = sock->ops->sendpage(sock, bv.bv_page, - bv.bv_offset, bv.bv_len, - CEPH_MSG_FLAGS); - } else { - iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bv, 1, bv.bv_len); - ret = sock_sendmsg(sock, &msg); - } - if (ret <= 0) { - if (ret == -EAGAIN) - ret = 0; - return ret; - } - - iov_iter_advance(it, ret); - } - - return 1; -} - /* * Write as much as possible. The socket is expected to be corked, - * so we don't bother with MSG_MORE/MSG_SENDPAGE_NOTLAST here. + * so we don't bother with MSG_MORE here. * * Return: - * 1 - done, nothing (else) to write + * >0 - done, nothing (else) to write * 0 - socket is full, need to wait * <0 - error */ static int ceph_tcp_send(struct ceph_connection *con) { + struct msghdr msg = { + .msg_iter = con->v2.out_iter, + .msg_flags = CEPH_MSG_FLAGS, + }; int ret; + if (WARN_ON(!iov_iter_is_bvec(&con->v2.out_iter))) + return -EINVAL; + + if (con->v2.out_iter_sendpage) + msg.msg_flags |= MSG_SPLICE_PAGES; + dout("%s con %p have %zu try_sendpage %d\n", __func__, con, iov_iter_count(&con->v2.out_iter), con->v2.out_iter_sendpage); - if (con->v2.out_iter_sendpage) - ret = do_try_sendpage(con->sock, &con->v2.out_iter); - else - ret = do_sendmsg(con->sock, &con->v2.out_iter); + + ret = sock_sendmsg(con->sock, &msg); + if (ret > 0) + iov_iter_advance(&con->v2.out_iter, ret); + else if (ret == -EAGAIN) + ret = 0; + dout("%s con %p ret %d left %zu\n", __func__, con, ret, iov_iter_count(&con->v2.out_iter)); return ret; From patchwork Fri Jun 23 11:44:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23116EB64D7 for ; Fri, 23 Jun 2023 11:44:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2EEC8D0007; Fri, 23 Jun 2023 07:44:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB92B8D0001; Fri, 23 Jun 2023 07:44:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 896668D0007; Fri, 23 Jun 2023 07:44:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 775098D0001 for ; Fri, 23 Jun 2023 07:44:51 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 46AE11C933F for ; Fri, 23 Jun 2023 11:44:51 +0000 (UTC) X-FDA: 80933830782.12.7F3D4BE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 6E006C0010 for ; Fri, 23 Jun 2023 11:44:49 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VitCU7XJ; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520689; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B4yhQOVBi6GuvWYD5vpVZMpw478eQsHtbdlO705ACPw=; b=VhkbgROXqa50gK6+7hQDu5YCt4hMby0jFJBmBknXUSwYah9logswwvxCnweRBjCG28Z67J Mx5wDWwZHMuLL0JaKDBIJ/Q8VlRamOsHZqS1KLlcmTaZ5p/NcMTU0n0KWtUH9IY+H/rYZx Jr1vv9u/fqx/p9Csw/pHGCYQuw7MHPE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520689; a=rsa-sha256; cv=none; b=jEeyKUOj1hlZHyYr+0pyb5CYpHfze4GPa36nAnM7nQotlnynyZ52I4lc05HTq6ovCqGP+Q nawT46yzHDyPDn/8tPn2U4hDF4slnaaQaSC0D//72C9GcwyepJ8AJHAm9NIsyElImvXXLh 7tXEygVx4xr4OslDBHEVMpta/2/4b9c= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VitCU7XJ; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B4yhQOVBi6GuvWYD5vpVZMpw478eQsHtbdlO705ACPw=; b=VitCU7XJSLPZSbXw2irk5NN7J5TBtQrbKDAKLQsk/XMGdQTQGAmPKgxUHyM9k8QcAnEZRG QVFa10tkkt1oULYhAKcYo2h67X+C9ynxCkKgj4VFcySyibnQ/y1MXqXrFxTcmv+XNMpAf4 6FU8AhbxOGpVKvBfkJJAA7hWZ/erTAE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-126-mIML7h7GNDCVVBg4TvKovw-1; Fri, 23 Jun 2023 07:44:43 -0400 X-MC-Unique: mIML7h7GNDCVVBg4TvKovw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6989F1C0754D; Fri, 23 Jun 2023 11:44:42 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id B08CF40462BF; Fri, 23 Jun 2023 11:44:40 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Santosh Shilimkar , linux-rdma@vger.kernel.org, rds-devel@oss.oracle.com Subject: [PATCH net-next v4 05/15] rds: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage Date: Fri, 23 Jun 2023 12:44:15 +0100 Message-ID: <20230623114425.2150536-6-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Stat-Signature: qxjttahtcfz8d5t5kx3q1aa4sau6ufbb X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6E006C0010 X-Rspam-User: X-HE-Tag: 1687520689-684738 X-HE-Meta: U2FsdGVkX18ONLjjDoF9ZANhcrMpG8Ax1n9NkX4Uy1DyDkxzsTcr+yWSHBCH4VNydGfpNy419un9AU9lEf51al0i0/4oc/9oucvecNBaH3YVoE7pl4FBn1zspEDNrwCtygSxFtR26a0b5B1wKv2g8d8hxZCWVnPqG+iQArKoFcDRxgdGLRcWYrdQs5dIgFE4MX3Uy4JxkNuIoz5GdCWQ+wYXSi/fGoIdPuZgUR4YbPHoG8S4XaxPHvhg9Lx2+OOVLhISpozdT7fs3cOzUG2QvnOk0eSsoamsKOKd99+j3PEK2TGVUNV/LeXe1a2RIKvSnjfa7cEdBAX0NAN5GncryIaiK/GngDtqT9Pdp0QMMzc32PQqCl0XpCSAnRIdi3QbQIRoxENiEc5wDd/EOIsxSJ36jVCnVTgpNqLT5/qUWomTPOvJDFhESQ5+fpbAt17OJ95tHI7is4ZpKD+LlBrqBsnfEJFj2ACYku+mn7Ev5JomtqBqgpXxNBt6bdJAPr77NnvOoFCakSI8VijxuwclX0SVzoRz4bCacJxnCHlsj06pviAlXGlw905eOp0o5wkH0y2AAFQe0dI9HZ76bR7CTR/XX1Rjbmcsg/P+ZPVSY51Rz73gDQUS9VzIRjBaEnkmXb91SroESnU86GFCwLUzTfoueuvM5vUqlAJd7qi5TJEPm0h4R00ZwkkRlMLbdUHq2twT7y9kUQklzWSVJLJlJMm25W+gOLWxRHivsHrqHs50xn++SF8bXFyPuRnl3YCjMRNc8hzFBRmwD3y634Ocbop04ihwqdIMiF4ob6V76ZE3JEy3fQGooIlY3RtWlLbSZXpsoMmKkVxzAgAe5MLlK648N/O+jgtnmI/lu7IxYWG24WWfGBFo+CwHS+sVvQMKDE0HlGPZ92ulezBxRYGISoxq9MvXU3Wti6ajDoEq9PwN+IVmLEbcWlH6k58uaLN2t/f24Tbi8T28BTSg68D Jz7fpiQ6 HXCn+c/MoSbS6Ijp9M4Ue5kFPH6djBB1+s89x5s4cgPJhAxOZiW4I7SSy9FHwO0MHiuI3DUy53en8exwBCaySGvOuvnMJs3PEnTyl6OgpE9tsUdeYOpApcJCQaGpjBv10kPsSbhecCfrH+BPBaoupZtTuEcosD0dlzIg68zWqv/Llt2S1UbPIDdxCI3ZfCJYHq2QF8rurSd8qQeVTuUCdBKpA2S5sNlaqUl/cIPR/DseQymoCHWmzh2PUJTwjMJ5o7zXXAAufWZyNGMFPJkqROUERsvYtCNJGLAwwzLxpi8I1wUmjposjM1IJdyUMNEzHJ737rgKWiED8ubUW/5SkgcI+WY6vwgmeeV7CY8dVTFBzGWIZ4WdtavhAnvwp/rBxyApmSu1xilorNnTQ1RqynDPmdziYzudfrQ3sB1Eo0/gUgQ/E73Xb7xzt8FpvWfeCsabwXqPfVtW48CzUY3R3944niyvYgLIxUd7dzZbm5TRW4FsZPYdDPhBWZMeULpCDcDw4R/0Jl7BTgVM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When transmitting data, call down into TCP using a single sendmsg with MSG_SPLICE_PAGES to indicate that content should be spliced. To make this work, the data is assembled in a bio_vec array and attached to a BVEC-type iterator. Signed-off-by: David Howells cc: Santosh Shilimkar cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-rdma@vger.kernel.org cc: rds-devel@oss.oracle.com cc: netdev@vger.kernel.org --- Notes: ver #4) - Reduce change to only call sendmsg on a page at a time. net/rds/tcp_send.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/net/rds/tcp_send.c b/net/rds/tcp_send.c index 8c4d1d6e9249..7d284ac7e81a 100644 --- a/net/rds/tcp_send.c +++ b/net/rds/tcp_send.c @@ -72,9 +72,10 @@ int rds_tcp_xmit(struct rds_connection *conn, struct rds_message *rm, { struct rds_conn_path *cp = rm->m_inc.i_conn_path; struct rds_tcp_connection *tc = cp->cp_transport_data; + struct msghdr msg = {}; + struct bio_vec bvec; int done = 0; int ret = 0; - int more; if (hdr_off == 0) { /* @@ -111,15 +112,17 @@ int rds_tcp_xmit(struct rds_connection *conn, struct rds_message *rm, goto out; } - more = rm->data.op_nents > 1 ? (MSG_MORE | MSG_SENDPAGE_NOTLAST) : 0; while (sg < rm->data.op_nents) { - int flags = MSG_DONTWAIT | MSG_NOSIGNAL | more; - - ret = tc->t_sock->ops->sendpage(tc->t_sock, - sg_page(&rm->data.op_sg[sg]), - rm->data.op_sg[sg].offset + off, - rm->data.op_sg[sg].length - off, - flags); + msg.msg_flags = MSG_SPLICE_PAGES | MSG_DONTWAIT | MSG_NOSIGNAL; + if (sg + 1 < rm->data.op_nents) + msg.msg_flags |= MSG_MORE; + + bvec_set_page(&bvec, sg_page(&rm->data.op_sg[sg]), + rm->data.op_sg[sg].length - off, + rm->data.op_sg[sg].offset + off); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, + rm->data.op_sg[sg].length - off); + ret = sock_sendmsg(tc->t_sock, &msg); rdsdebug("tcp sendpage %p:%u:%u ret %d\n", (void *)sg_page(&rm->data.op_sg[sg]), rm->data.op_sg[sg].offset + off, rm->data.op_sg[sg].length - off, ret); @@ -132,8 +135,6 @@ int rds_tcp_xmit(struct rds_connection *conn, struct rds_message *rm, off = 0; sg++; } - if (sg == rm->data.op_nents - 1) - more = 0; } out: From patchwork Fri Jun 23 11:44:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A31F0EB64D7 for ; Fri, 23 Jun 2023 11:44:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A3248D0008; Fri, 23 Jun 2023 07:44:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 252F78D0001; Fri, 23 Jun 2023 07:44:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11B908D0008; Fri, 23 Jun 2023 07:44:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EFDCE8D0001 for ; Fri, 23 Jun 2023 07:44:53 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id ABC68160E83 for ; Fri, 23 Jun 2023 11:44:53 +0000 (UTC) X-FDA: 80933830866.24.5FC07FE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id E61A2100013 for ; Fri, 23 Jun 2023 11:44:51 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MGlqHsmd; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520692; a=rsa-sha256; cv=none; b=BFqgtMsYMQCyJVNbajGhpEE6dVM9kgIaIVGLyV2PtFU5inRle9iXBWlQXgjDhYSPFzoX+Q xH3/T2klP1/iSTHC8JQ73LJcN65oT8m8jiBvC9dl1mWwQHqF1gKnK4X0SIQI757iYhkcWz B/XVSjUVfO53N80NJ7Giqn1DQTYSuOs= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MGlqHsmd; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520692; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CX0szcw738ZS8htZ0RxEmjBFgUMhbKSUPMuc1XxmplE=; b=lP2ytQPqlmthdGMuFdu6yzzjuIVDtU5RCogh+NZ/aVbf4UHgefBNuFwx9kgZUq60d180i7 WJalU8zBG04AMjJLzyh+T4UMsQ3KfDX57uGI/V2j1l4W63MmMkO2L9ekg3og7tdkmeaWRk HryDbX285R33T181PIYrMIHkqIdus/w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520691; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CX0szcw738ZS8htZ0RxEmjBFgUMhbKSUPMuc1XxmplE=; b=MGlqHsmdkpPxZ/l0FYZFAqoXs4lsOJ44dJ+K9NWh6CQJLUi/ToelewQ3Vqd/lGQMjnNcKJ 33sqJWbvfgqw03pY91XBdQoqjWzOaiTISFM9CRGxnkPNUFPu7blA0nfLFSUPlHu1D6OTgs ggxsibD9DvFfEhcb50t8S15dhSMBynA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-610-pOqU4RdaNZuHTBXvo8Lefw-1; Fri, 23 Jun 2023 07:44:45 -0400 X-MC-Unique: pOqU4RdaNZuHTBXvo8Lefw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5ADC28EC10F; Fri, 23 Jun 2023 11:44:44 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04E2C1121314; Fri, 23 Jun 2023 11:44:42 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christine Caulfield , David Teigland , cluster-devel@redhat.com Subject: [PATCH net-next v4 06/15] dlm: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage Date: Fri, 23 Jun 2023 12:44:16 +0100 Message-ID: <20230623114425.2150536-7-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E61A2100013 X-Stat-Signature: rge7pkgja9n7q144nnh1wsix5s7gh3fc X-Rspam-User: X-HE-Tag: 1687520691-732872 X-HE-Meta: U2FsdGVkX18KULcX8CeFUeW47Wp/PANC304+na0S5dbQCHYBCj23CnwwmYYLnF+nIJSxkpouCheFDapAzJm9r9tF+YGH3E3f7yP1OEarDOXMDmjZCzTs2o16FlS1J5RkWNF4u+CTTsMsoc4A6ho9qLTccPkDRYgepzj6Xjt0t3Q9MyhaPmAOT1TGn8PpCR9BRjDeaEK4Xae9cO1hZCD4HW24ALjI5fvoDV38m7rwrig2QRZmi13ok7H1e7w4Nz52iPoVndQlaEVjKxenka+0coWSyM2b6myEk1H6K/WmxNkNSRYUWp9dZfiVe1Aw05tmfKkEeYvwuxVyYPr9a7hB5o5n/0srDT9Vh/WxL3mtnHotHcBXO8Hazim+iW+bCTm0y9DoAVEazPAKuPERsDCR3FnqE0U5V/vV8trJKR5RjNuL9GE4+c2eubw5SSN8m1YDxhkMPbp3h/ROTcN+LbbVJUn2t3zbGcfgxxJ9YkopC4/cnPMvSP9ZzXGvK4dFY0k3vs4sG9uQDZTdZDGCahLbcI36aczaar7KVdVAtHK4GrLzNeE0FJkbCwb5IgJ6FHhAipXM03pBx/3lkUTkHOWrGn8gqE30sbkf9cENcMVYqLlWzBar8rSfjJvMnyt/zBIVQvg6HuNEjTPd4j44aLxEuqkAyw9Nrgbv0yTzJ3o0AaIS1hfSQ2qh1uD/OFT/JPLfavuMs2LicZ/2Ssswby9n5SfHb/cgFHnwJTrXpY4Vzu9G7qS90Yl0cw+dg46qnr8FpohoGgA74kvV95eXcLEZ5QZm3DD7dt2Uu2y3H9aA6gijXSuK24gLWslUo6XKqrS715qZ/bffHDMgxZK77AaMH4p0D1BrVsY4NZ3Svb40VI22wP55HP4Ef5IuOOTOwuD6l8T81kCCFi7CmHCYCJ7dVL+bcVVDysBlk6pbnxRn7xQXHRMkHQx2lWXsjLf+rJnpZ4ZDfpmfmFJb5n+Z5iv awFkUUoJ TJiXsHJr+mCf4s4Kh/PRNvUToR8V4RlcQG1Cn9yPnlY7t6df2+eNDc0N9ifWYneYEHBBPFdtxJwdjl6hHYNzAyZ5N3TQkOJJmRY5M7W4u/0zxCqCyX88tCGVsK8ktNxAVzljzI+lyrfQ4JKg+esmcAny/YGVO3tcXIcP2TyPEt7KHSioeMh4Wvy7qd+F/aOCMhylDDEWcDTQI7Yt1Y4yiQxo7P9kQ0ljr8jfE/XpJrIkTbFEVvtQigaz3/M34nXGDelo0TxTrcAe7LR273NOy+OQdO3c6pvEmqpLhOoowXkzqJ7dVKXY52+uWCAcRXJ3fosnPTmTAg7HQQJVbJHOY2h5EcTSMxDVi8AneGRe4iMMXl6aLNZNVRL/UmHUebST5j9gIKWb6acNSFNk3fC1NyfQtLfZsCao/DjwLEPr8amyhXrSNTCHMpgjih8jHIR8sNj453JIDzVhWA4HnGVjP0gAiVLFMyubMLnUYS+v4MhnNJnYgqt3L2RUpfQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When transmitting data, call down a layer using a single sendmsg with MSG_SPLICE_PAGES to indicate that content should be spliced rather using sendpage. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Christine Caulfield cc: David Teigland cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: cluster-devel@redhat.com cc: netdev@vger.kernel.org --- fs/dlm/lowcomms.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 3d3802c47b8b..5c12d8cdfc16 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -1395,8 +1395,11 @@ int dlm_lowcomms_resend_msg(struct dlm_msg *msg) /* Send a message */ static int send_to_sock(struct connection *con) { - const int msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL; struct writequeue_entry *e; + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_SPLICE_PAGES | MSG_DONTWAIT | MSG_NOSIGNAL, + }; int len, offset, ret; spin_lock_bh(&con->writequeue_lock); @@ -1412,8 +1415,9 @@ static int send_to_sock(struct connection *con) WARN_ON_ONCE(len == 0 && e->users == 0); spin_unlock_bh(&con->writequeue_lock); - ret = kernel_sendpage(con->sock, e->page, offset, len, - msg_flags); + bvec_set_page(&bvec, e->page, len, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); + ret = sock_sendmsg(con->sock, &msg); trace_dlm_send(con->nodeid, ret); if (ret == -EAGAIN || ret == 0) { lock_sock(con->sock->sk); From patchwork Fri Jun 23 11:44:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88FB6C0015E for ; Fri, 23 Jun 2023 11:44:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D79D18D0009; Fri, 23 Jun 2023 07:44:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D022F8D0001; Fri, 23 Jun 2023 07:44:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B7AE08D0009; Fri, 23 Jun 2023 07:44:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A9FD98D0001 for ; Fri, 23 Jun 2023 07:44:55 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2646C80F08 for ; Fri, 23 Jun 2023 11:44:55 +0000 (UTC) X-FDA: 80933830950.04.CF295F4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 6297440009 for ; Fri, 23 Jun 2023 11:44:53 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dy96EYlB; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520693; a=rsa-sha256; cv=none; b=p2ulDPewEWZny7sKrgax8ZDdIcsAt4OVlKTDuDEgUtU4xcLUNd6PDHzoaurt7/pbPjGmps NZ1GgRH/5M04XcM7zCaPQ5UzEQYlobNnaW86RkoIbo95t8gpDv4S+n04b+UJrhSkWbkW7u oqpMQNLiO0XXrlhBaehaS0cwkaXIEEE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dy96EYlB; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520693; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W+/sbivnn4SSL91vpgGdKmYtGhKmLadJeuepui2ZFTU=; b=BDdB2LYSem5nlvSphMjFFwWqVMT4HMEgslPRtx6tTkVyTQqkfU2QbcstWVf126lw25bn+s KFaW+sAWec2vERPGGzC8VFsiH0rVaGJV39M/KTRqKMOftqx+h6kB8eWh46KgHjswqMccOU qY3qOXhGI3km4T72lG2syqLsCiXru88= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520692; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W+/sbivnn4SSL91vpgGdKmYtGhKmLadJeuepui2ZFTU=; b=dy96EYlBj3JFeXvB7wqQYeyUzVWvOtGsIdJk7b1ovwXZ6byitbKuDahDCqXLPKM/wkyLFZ ON+sFxtnpwJ0ytfTAP2Lktbw6gKtmBcA4m0EG6iWQ0q1661G7RX73eO0iGmAUV3RzpWO/a QZTNbqwOcuG3icFrC47MiyBbIND8WsI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-547-oy5Qkzx6N9eqWFkhdXf48g-1; Fri, 23 Jun 2023 07:44:48 -0400 X-MC-Unique: oy5Qkzx6N9eqWFkhdXf48g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6CCDC3C0F696; Fri, 23 Jun 2023 11:44:47 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52893C1ED97; Fri, 23 Jun 2023 11:44:45 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sagi Grimberg , Willem de Bruijn , Keith Busch , Jens Axboe , Christoph Hellwig , Chaitanya Kulkarni , linux-nvme@lists.infradead.org Subject: [PATCH net-next v4 07/15] nvme-tcp: Use sendmsg(MSG_SPLICE_PAGES) rather then sendpage Date: Fri, 23 Jun 2023 12:44:17 +0100 Message-ID: <20230623114425.2150536-8-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 6297440009 X-Stat-Signature: 6nmh35cf75mg47cj4rdf44e9ssh58zrk X-Rspam-User: X-HE-Tag: 1687520693-764810 X-HE-Meta: U2FsdGVkX1+pSnrmKFz4YJsn3hjgTlOE/5SYrvjEpNzSACUvzCEZjnHWDVb3gSZNaOES0d9RL5jpQWGM8yx29S1DLVVx2vErdUDOEQz4NDx++cUD4A8wSGXC+brsNisDt89sJ4B5vdsZNwnNCdobnf+wb4mTZCmWh7GrI4gP93w2YPCPveNxtrHQTuvgWZgamW9wwzaIIAPbZUWi0RtJUEXH2Tno19euwN9DMU53bJXDlK9iVDYXM8LnbWTI1KgTjb7DTbJE8JT1d+J4MCvZQqk9buDoOHFg8YaNl/D3+QE9+4mlpWbr9NyT80oUAuRAceTJZ5gnGP4cD6LlYlRi6qX6A58QT/bMzWzwDFI92r3BoLJ/zo2hxzDLBrg8oqRvPo9KLJ+6Cfd3jnlGbZESyGZsk6AWkHQ1sHAgwY2iKk+DC73Q2Ru+3M+PGrmpCUc34vnNA2hXI8DnsRiEFJGXjhD98LOshrasz6mzQibUcJIZpi9c5u+6bZ+XMq7PjmZcyx19VwOkqnEtg6MYZ1YBT9bezxYdCETluvlUmFbeX84ebbaWLmE9JVUn+WxBfwy4m/3Igwb3kbXVLc4m2lwG519cj1Pah1qoPeCJ1T7orgnyLiqYlD9PfDb1c4NtXw3jz828GA/Y8Rg6gfwhjhSZ4G1xo8DhYsJ2h6NLhveX7aBYzF7FUVemhSAWvOj9dWcqcAm79W+sIywScu0mqfz+0hv0AlBhmwE7tzPRWKWRjEKm1YiDTsX1/NbNzq/esrLxD/TRQuE3TmTHtWG5DJJ6wtGg41nyKhYQmpnvP4xcQmGsfKLoTG+lRHmQvLip5ETrqMFAxrAuQ/VnqYBg5aVaB+cDSK23QU8Nl9qJDFvxoC069ADLiH3DDAsod+aFZt5ilWMNASrcsPzDtIbxetlUDSr+UCr9xJCPJAKS62IOwr5jlgjmQFBtLLH33KbgMzvgYYmor81X7ZU4tC5BjDZ EOexu76i xZhKeIeF3PFg9C9w85yg4svCKusGjZWRSZlbiz2gafHDP/cIad7FYrnWWDMrA6dXLR5XoX2+FkpWvBkUv3voLrRznHdVhMgHlZJpTQ750oF72TQI+EBMDOZ+dNbm8Ry1/i06Lbcn1YOdqjrUrOpBpqQMlxgd0OrDu2jkSL40iiU0YVlIZKIRget+E9SKCCLFmaPT9hYPP7paoEYKYHylBlgAEmQz82QeSGAVQqd0yItbNII4Leok+nKiIMPXxi/wYsvlB+vzPvV083zOyQIUggJGJI4qqWBgAaSOqosrU2g6uLXdxDi4WwC8yzmGgU59YdQId+0a7IjTLS7aCVRFtRhWsdLq7/Bw2DKk6aE26fh5znG9zPmlLh8WAXuRGTUKNxkAu0gjoNM6Y+yK/zTpzAfKftdIFuG6UPKTTajPP30UBbndRzBbm/k7TRsQnkyZ0LwBiurJv+b7/YJZF/JPHb4EG5ycyTbB6dogXC/xUHQ5a8QQ/NE5eVyfslRnH9muugQvcVKWUNno+qyhJnyOtwmvMQ3INaoFivbrAsNme7V2+H8bEr6VLMikmzozcLKOWH5GgUafUoAJY8pc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When transmitting data, call down into TCP using a sendmsg with MSG_SPLICE_PAGES instead of sendpage. Signed-off-by: David Howells Tested-by: Sagi Grimberg Acked-by: Willem de Bruijn cc: Keith Busch cc: Jens Axboe cc: Christoph Hellwig cc: Chaitanya Kulkarni cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-nvme@lists.infradead.org cc: netdev@vger.kernel.org --- Notes: ver #4) - Cancel MSG_SPLICE_PAGES if the page being sent fails sendpage_ok(). ver #3) - Split nvme/host from nvme/target changes. ver #2) - Wrap lines at 80. drivers/nvme/host/tcp.c | 49 +++++++++++++++++++++++------------------ 1 file changed, 27 insertions(+), 22 deletions(-) diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index bf0230442d57..47ae17f16c05 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -997,25 +997,28 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req) u32 h2cdata_left = req->h2cdata_left; while (true) { + struct bio_vec bvec; + struct msghdr msg = { + .msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES, + }; struct page *page = nvme_tcp_req_cur_page(req); size_t offset = nvme_tcp_req_cur_offset(req); size_t len = nvme_tcp_req_cur_length(req); bool last = nvme_tcp_pdu_last_send(req, len); int req_data_sent = req->data_sent; - int ret, flags = MSG_DONTWAIT; + int ret; if (last && !queue->data_digest && !nvme_tcp_queue_more(queue)) - flags |= MSG_EOR; + msg.msg_flags |= MSG_EOR; else - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; - if (sendpage_ok(page)) { - ret = kernel_sendpage(queue->sock, page, offset, len, - flags); - } else { - ret = sock_no_sendpage(queue->sock, page, offset, len, - flags); - } + if (!sendpage_ok(page)) + msg.msg_flags &= ~MSG_SPLICE_PAGES, + + bvec_set_page(&bvec, page, len, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); + ret = sock_sendmsg(queue->sock, &msg); if (ret <= 0) return ret; @@ -1054,22 +1057,24 @@ static int nvme_tcp_try_send_cmd_pdu(struct nvme_tcp_request *req) { struct nvme_tcp_queue *queue = req->queue; struct nvme_tcp_cmd_pdu *pdu = nvme_tcp_req_cmd_pdu(req); + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES, }; bool inline_data = nvme_tcp_has_inline_data(req); u8 hdgst = nvme_tcp_hdgst_len(queue); int len = sizeof(*pdu) + hdgst - req->offset; - int flags = MSG_DONTWAIT; int ret; if (inline_data || nvme_tcp_queue_more(queue)) - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; else - flags |= MSG_EOR; + msg.msg_flags |= MSG_EOR; if (queue->hdr_digest && !req->offset) nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu)); - ret = kernel_sendpage(queue->sock, virt_to_page(pdu), - offset_in_page(pdu) + req->offset, len, flags); + bvec_set_virt(&bvec, (void *)pdu + req->offset, len); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); + ret = sock_sendmsg(queue->sock, &msg); if (unlikely(ret <= 0)) return ret; @@ -1093,6 +1098,8 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_tcp_request *req) { struct nvme_tcp_queue *queue = req->queue; struct nvme_tcp_data_pdu *pdu = nvme_tcp_req_data_pdu(req); + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_MORE, }; u8 hdgst = nvme_tcp_hdgst_len(queue); int len = sizeof(*pdu) - req->offset + hdgst; int ret; @@ -1101,13 +1108,11 @@ static int nvme_tcp_try_send_data_pdu(struct nvme_tcp_request *req) nvme_tcp_hdgst(queue->snd_hash, pdu, sizeof(*pdu)); if (!req->h2cdata_left) - ret = kernel_sendpage(queue->sock, virt_to_page(pdu), - offset_in_page(pdu) + req->offset, len, - MSG_DONTWAIT | MSG_MORE | MSG_SENDPAGE_NOTLAST); - else - ret = sock_no_sendpage(queue->sock, virt_to_page(pdu), - offset_in_page(pdu) + req->offset, len, - MSG_DONTWAIT | MSG_MORE); + msg.msg_flags |= MSG_SPLICE_PAGES; + + bvec_set_virt(&bvec, (void *)pdu + req->offset, len); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); + ret = sock_sendmsg(queue->sock, &msg); if (unlikely(ret <= 0)) return ret; From patchwork Fri Jun 23 11:44:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5523CEB64DD for ; Fri, 23 Jun 2023 11:44:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42A028D000A; Fri, 23 Jun 2023 07:44:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DAF08D0001; Fri, 23 Jun 2023 07:44:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 204E98D000A; Fri, 23 Jun 2023 07:44:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1304B8D0001 for ; Fri, 23 Jun 2023 07:44:57 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E09341C9325 for ; Fri, 23 Jun 2023 11:44:56 +0000 (UTC) X-FDA: 80933830992.21.C48116A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf08.hostedemail.com (Postfix) with ESMTP id 36E54160021 for ; Fri, 23 Jun 2023 11:44:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=G4vrS1Uk; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520695; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wPelxPY/D2Xo1HQrs/bARLsTqDBy+xx6UHdo64Xl3wM=; b=zNpZveNqkoej5pbLSa3r6A7uO++PSFOTNRHv+zFyJqYj7mg7vhk9ClcB3EHZR3ztWFm7CM //zc0TS9nufPBafEQc8QNuASL6iwmsrZnsI6zBSg8M0MYMH4q77czTmz4EOSPyZYaiEEyO cZvvELGSZa2GcOQPPHz9S1l5T4oaGsU= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=G4vrS1Uk; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520695; a=rsa-sha256; cv=none; b=4/MsZeEy7rHeIeOSwO+6/yNLgMndHAHB6GrUJ4MAB6GeuOaQWfcgxmPt3c/pUEShxghUdr 5vODYo7VsxmWLdpzYOe+nr7FnjEszT0rcg3s16YimCAX+PCoBL5rblQ6ACq3CSEhGgjWC3 qlMuRGgwtbS0+KJ5CYlRHIPA0GUcGls= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520694; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wPelxPY/D2Xo1HQrs/bARLsTqDBy+xx6UHdo64Xl3wM=; b=G4vrS1UkaUUPFR3Qvnz9AQ9Gm7p2ItIzI3osv++uuj4RmMU7/9WqtmDP4mz0kDZCJTJxG/ miwvyTXpFU56so0oGTsuWg0yjy4HRPTbDstLUppjZnxi4x71tAY/Nki2lpJjk1ydInJWIN SkBE5xRhdqhs0J/3REJs8/eEq7229kI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-48-yfqi9PmCPaK-TAUzgrk0lA-1; Fri, 23 Jun 2023 07:44:51 -0400 X-MC-Unique: yfqi9PmCPaK-TAUzgrk0lA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 38DD590ED25; Fri, 23 Jun 2023 11:44:50 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 229D6200B402; Fri, 23 Jun 2023 11:44:48 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sagi Grimberg , Willem de Bruijn , Keith Busch , Jens Axboe , Christoph Hellwig , Chaitanya Kulkarni , linux-nvme@lists.infradead.org Subject: [PATCH net-next v4 08/15] nvmet-tcp: Use sendmsg(MSG_SPLICE_PAGES) rather then sendpage Date: Fri, 23 Jun 2023 12:44:18 +0100 Message-ID: <20230623114425.2150536-9-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Queue-Id: 36E54160021 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ygsxdcxa395ifoz7zb88w6nxbbontny9 X-HE-Tag: 1687520694-221672 X-HE-Meta: U2FsdGVkX19BJY7vkqAs8VgGOLByHauVcxEHwvT26GkNHJnRLFcA0rUmZsH7wycXCrgCwGiDlPXwVxBYkTasl+CgU5YM+1Ae6iwlO5gBASf83ADyjJ4RbBEFbVvRUXv8eb3A09UZc/ii/ncrcYh3p5Kh3DXXRicGfsY9kKVIDNFODKa/jKQ9S25RHz1oyLzu8P6TsLrCz+QQp4f2269t/aqhVxdcQNiE4gEIdO1Pjsk62GQHwhDpB4sA/lwxM6YWZmgPORg3qsKdMTjn+L0oZv29a2xavdIi4nNPHZ/r1fv29sfcwaaxpDAGCHyqTj9L4I9QeeOWPbDNSTNwBKXkYgAm34Fnuh3vltMGGLIQ0D8FOdtRqMhI4GqDtpzSUU4B9265GkzZ1cSXvOl8ke9hCrlfMQ70u6N0bIvGFUDXgnMTQUJbXEEMqGicN1suxIeGYzo5KPXyL/HOfdDLvKph8kvBvaGgGXHUGFi/aTzms5uwPQbnBjSujgkTS+YFR651U/gp3hsTPNMNxt58/NSaitjqkCCE0qZxilB803ajlBSscY1GiWRSlJNlbOaHmCx5Z9PKPCBbne5aOABoHm/jw92tpmq+aVA5XaVHuZb2B1+O+tWg+YDH6vVFjgwzQTUnrwnMHZSeTCNi0b2oUJNhY5ua+KaWMUgaMQfeBYc7tcJQseoquMNCKyMKpv1btMzNM76VnstBAuDst/5IaSoPsowOSzGBxwPGmD5LL5ASK1krfYvOWWph+5wdsvAYub7T3kXy2FY9lCiYeI5EPyab3goAk4G83qmitpCyyeGgxl+dETFY7le/7LQohq0V2JlZo9n+uebGEqLM6xlppVON6nPtIkMcULP0VJ1L5TJwrY5lwj2mP+Pqs64+3mX0/Clwszt1SWfXnpKnmjjVMC9fGIWVF3u3tU7oQfvxvMZFFEApXYaEWe9MVs+3n/iYZEtCoPGGeCNDVniHeBo4WI1 mig+3Yp6 Ss6w3BLftvmhjVApc+A5o3EJNGTuctkzy9zWBDhsunRjE5duvDKHetGZdABjfV4UcvWWyqVKgeRjiQxAjNHN/zoEmmuu9h+MaBhqZU/gM5IOe0+G4a8iyS8IHOspNXkYsXvnEVSOXCZZpx5tlgAlwX48aacvksDRbsKterD1MSNaJB1qdQYqXIkL/UQMY+T5GN8WBceRGRIZk13Plr8GraS4mTITGaAtf4Lr/ke9Ap150HoEgbRdQS4cPX6/zBGHZTKW90Dbooza/+fG4owfXZDhFB/nz1OQbjpadEPFRWSVyTdjQ1FQV/2FgZkTiM1Qku7E+9OT9MRrEXQfbtnJFwNEUmpBMul6ac/1ZONUnpRV+6IkLlPr0CrOMrng27rNArEy0Hz446pCjEhdq92HU2y9DlxCP498m5uNdgnyzCpzyCpFMlfSuIGGUx/+83oCKPt4N5s+EcoxPRc6Fc8B9YLSUJDxFamvGweJqZ9ftbMnNGV+PCL3wPBTRL/l22roX+Y10obwFst1wFldKiORsHvvn1lCjTtuwKZJOoOIT7SqVCLzVlScbfH9BzfC9qV6WvN/1q95f0dEUOx8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When transmitting data, call down into TCP using a single sendmsg with MSG_SPLICE_PAGES to indicate that content should be spliced rather than copied instead of calling sendpage. Signed-off-by: David Howells Tested-by: Sagi Grimberg Acked-by: Willem de Bruijn cc: Keith Busch cc: Jens Axboe cc: Christoph Hellwig cc: Chaitanya Kulkarni cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-nvme@lists.infradead.org cc: netdev@vger.kernel.org --- Notes: ver #3) - Split nvme/host from nvme/target changes. ver #2) - Wrap lines at 80. drivers/nvme/target/tcp.c | 46 ++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index ed98df72c76b..868aa4de2e4c 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -576,13 +576,17 @@ static void nvmet_tcp_execute_request(struct nvmet_tcp_cmd *cmd) static int nvmet_try_send_data_pdu(struct nvmet_tcp_cmd *cmd) { + struct msghdr msg = { + .msg_flags = MSG_DONTWAIT | MSG_MORE | MSG_SPLICE_PAGES, + }; + struct bio_vec bvec; u8 hdgst = nvmet_tcp_hdgst_len(cmd->queue); int left = sizeof(*cmd->data_pdu) - cmd->offset + hdgst; int ret; - ret = kernel_sendpage(cmd->queue->sock, virt_to_page(cmd->data_pdu), - offset_in_page(cmd->data_pdu) + cmd->offset, - left, MSG_DONTWAIT | MSG_MORE | MSG_SENDPAGE_NOTLAST); + bvec_set_virt(&bvec, (void *)cmd->data_pdu + cmd->offset, left); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, left); + ret = sock_sendmsg(cmd->queue->sock, &msg); if (ret <= 0) return ret; @@ -603,17 +607,21 @@ static int nvmet_try_send_data(struct nvmet_tcp_cmd *cmd, bool last_in_batch) int ret; while (cmd->cur_sg) { + struct msghdr msg = { + .msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES, + }; struct page *page = sg_page(cmd->cur_sg); + struct bio_vec bvec; u32 left = cmd->cur_sg->length - cmd->offset; - int flags = MSG_DONTWAIT; if ((!last_in_batch && cmd->queue->send_list_len) || cmd->wbytes_done + left < cmd->req.transfer_len || queue->data_digest || !queue->nvme_sq.sqhd_disabled) - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; - ret = kernel_sendpage(cmd->queue->sock, page, cmd->offset, - left, flags); + bvec_set_page(&bvec, page, left, cmd->offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, left); + ret = sock_sendmsg(cmd->queue->sock, &msg); if (ret <= 0) return ret; @@ -649,18 +657,20 @@ static int nvmet_try_send_data(struct nvmet_tcp_cmd *cmd, bool last_in_batch) static int nvmet_try_send_response(struct nvmet_tcp_cmd *cmd, bool last_in_batch) { + struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES, }; + struct bio_vec bvec; u8 hdgst = nvmet_tcp_hdgst_len(cmd->queue); int left = sizeof(*cmd->rsp_pdu) - cmd->offset + hdgst; - int flags = MSG_DONTWAIT; int ret; if (!last_in_batch && cmd->queue->send_list_len) - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; else - flags |= MSG_EOR; + msg.msg_flags |= MSG_EOR; - ret = kernel_sendpage(cmd->queue->sock, virt_to_page(cmd->rsp_pdu), - offset_in_page(cmd->rsp_pdu) + cmd->offset, left, flags); + bvec_set_virt(&bvec, (void *)cmd->rsp_pdu + cmd->offset, left); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, left); + ret = sock_sendmsg(cmd->queue->sock, &msg); if (ret <= 0) return ret; cmd->offset += ret; @@ -677,18 +687,20 @@ static int nvmet_try_send_response(struct nvmet_tcp_cmd *cmd, static int nvmet_try_send_r2t(struct nvmet_tcp_cmd *cmd, bool last_in_batch) { + struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES, }; + struct bio_vec bvec; u8 hdgst = nvmet_tcp_hdgst_len(cmd->queue); int left = sizeof(*cmd->r2t_pdu) - cmd->offset + hdgst; - int flags = MSG_DONTWAIT; int ret; if (!last_in_batch && cmd->queue->send_list_len) - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; else - flags |= MSG_EOR; + msg.msg_flags |= MSG_EOR; - ret = kernel_sendpage(cmd->queue->sock, virt_to_page(cmd->r2t_pdu), - offset_in_page(cmd->r2t_pdu) + cmd->offset, left, flags); + bvec_set_virt(&bvec, (void *)cmd->r2t_pdu + cmd->offset, left); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, left); + ret = sock_sendmsg(cmd->queue->sock, &msg); if (ret <= 0) return ret; cmd->offset += ret; From patchwork Fri Jun 23 11:44:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290408 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29FEAEB64DD for ; Fri, 23 Jun 2023 11:45:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C65DC8D000B; Fri, 23 Jun 2023 07:45:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3DA78D0001; Fri, 23 Jun 2023 07:45:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B05EB8D000B; Fri, 23 Jun 2023 07:45:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A022F8D0001 for ; Fri, 23 Jun 2023 07:45:02 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 60D34160E84 for ; Fri, 23 Jun 2023 11:45:02 +0000 (UTC) X-FDA: 80933831244.06.766FE91 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 3E35314001E for ; Fri, 23 Jun 2023 11:44:59 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="YzV/pTMg"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520700; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PSe/ILwvcvKNAe9EBhS3HTJsxWKySCoAZiMEUxcxN8I=; b=H65hDQJboxtBKQDuDHF0Nep8wiyi9D6LHP8cNi01qsLMscahvx+zDhS/bRxIiI9YZ4Gnfl V3lleE972alsNDdszBZJ2CMsHWdOVlYalzKfr9rX3IqX4Z+ez2JWKH/1mOl+5/N0oUzXPA UNjbPOsClY+mjY2YbZV1zJt19zoxfc0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="YzV/pTMg"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520700; a=rsa-sha256; cv=none; b=UOE/yaUgc0Wem2c/Xh0MrNVcUuMZ669CiqP9s0ola4XIKh9Owsky5LZYfFzvM5cHJMs5P/ 3BPAw/QFhXbPmrd85MhKN5PXprelgChYO5KIZ6tCKMrHgru4VBLeOLnlwMWSzL1D8WVBdl re1jcXPCI6w+hPNBh4BjKnws5RiZIH4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSe/ILwvcvKNAe9EBhS3HTJsxWKySCoAZiMEUxcxN8I=; b=YzV/pTMgMCe5MzbtgVoY0HZlqPRFzLDBDpeG14gYkNANcpkQfKJvdN8ZqOW/rDMk/fy2m+ 9LIy+TSGtiSsFMtqZZdTny6Xa0VglY9f/9jMQYhTyXnj+lzuHSXg2RuO7m2QnwNYMZcTJ0 kopuy2LLnHnFUf+TgpUoTHd3aIeBQr0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-134-x-hqviP7O_yozn6WZokJng-1; Fri, 23 Jun 2023 07:44:54 -0400 X-MC-Unique: x-hqviP7O_yozn6WZokJng-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D7DEB8E44EE; Fri, 23 Jun 2023 11:44:52 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1CB740C6CD1; Fri, 23 Jun 2023 11:44:50 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Karsten Graul , Wenjia Zhang , Jan Karcher , "D. Wythe" , Tony Lu , Wen Gu , linux-s390@vger.kernel.org Subject: [PATCH net-next v4 09/15] smc: Drop smc_sendpage() in favour of smc_sendmsg() + MSG_SPLICE_PAGES Date: Fri, 23 Jun 2023 12:44:19 +0100 Message-ID: <20230623114425.2150536-10-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Queue-Id: 3E35314001E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: q87ax6jr6bt8ss5e44shbsmzruu5zjm8 X-HE-Tag: 1687520699-91246 X-HE-Meta: U2FsdGVkX1+dCVrdDPRKfzrDX+2ydIYQ5TzREGW9rr0l6PJJwTMadwGe9suDqRtd3iLmyq5w96NkcGtu/a1LmpgKbvMBG6xiy3bZeVKt2ee6wadQ3zImLdwNKgOUJxtCle9ACMzArCiJgP4OUF7ac0lIuoZzRnjdhGEY+fbguwUGT8fhXi8wsYOuXsCHTKr2hDxOWKUiXAi6auO//LJNKSqCe5ubU73GsGnMFbjU7I1IVBiS7tfBJTBMW8NPIwSQ+iMX4nwrugOfVpHSNn441SAnIgFNsspdlgAaZpmTLkN9o1GfASxJXEHQyyVL36SddbJlevFKHrOkv6mXO6nHCVFUEqYMnsvN00T9EzMq3gcZBCTj1pE6GscUEEDKaXdF7KWKWTr1zz25aAdOTz/EJvBFS3FPveBwyKkv8BZeE51gkO/r6QL4bIaJjJ29ygU25wzgmhYGibRLr67r6AR6Fu2MsUpa2/wEPhImAmFgMjKmK0/F9T+GZ+CQmGmG+jxd2F5rSBI6gTpQR2PhuWgdKbQStbgG6c97SkMygvxOHsbFxsXJxR0UPOfzBpddnjBJRP1dPPNwUQ5YVb/rU8rVCt/BPGaliGz3XEAric/W4zDR9Lwj3u3XR7SCBsYvN4VazQq9gqAdJEBRQwv9HsmgBari9o5AiozcJ6COf//e0iNdhPMjJTImSM0DwdBI0W6boiiil4vTBgBTVnTsrHjKj/qlC89sHUF2ZNzc6DsJtt4w/mCW4afTlnka1zYbCQGuomOPFxivfE31DBg7fBitMYA1ekj1HIfjwm9ZNJluOy8383gDpmR68xoAqfYL9u+ma5E4yaEsc6fZ3ZyhvdOqyqhsd/dY/Mkr7XC4VjHhBuaEJbYw9ClmMVKwGCGwFB2gwj/N6iszgvOnAYRDTGKCDbodAwAt6GkgVK80NUpINuNnDro8VweJ//l0Aqa+tFlZQy6K7MQxqtqoU71Y6oP 6X8txgOW pTZiC8cAzgZ6ZFuBtVaIQ6A8mLwYL39S8O80F1+y4G9PfYEp0X119ZCgIrmVlkWSiOTIaL8VxVNZeF5avOfeFdGVc6/rE1CkNd3mZJW4rZchlx9maSoxSkYg/+bvLeMTfimThDYaWGMXm7S/Je44LYc2UIjBM96aFdO7FDDqqlkIR+HZRdSJqxiM0Lt69z8d9pw4rpiVhzjzTVzzcu2g431duvlcB5/pr0pt+DLWUWaEskqUrIiol26bDQwYv6vo3cqZJTa39gyinBTplfK6mXozFzVX+UXqhrsmptAtwH2uwtxceCIMUIQA2mU/83m4RXAF1mSyG+lYRQ0tSvxZGLgfNCRq+14ti6EggjdfBsPip6gfSuSku2qgyYrJx/pR7zqK7moEODbX5lFMdpFKnKOydfHGoKueNy26DgEH09MwF74Yq9PL7fGQtsN2670SSaQLHMNStUCfmxk4DKGSpXBTahKeUrI4vbBi4ECKSXJVVkKrYHV1pEFf75cfL3ivyXFLJRH913nVt70ck3F9RaskfLyisWvznwQEW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Drop the smc_sendpage() code as smc_sendmsg() just passes the call down to the underlying TCP socket and smc_tx_sendpage() is just a wrapper around its sendmsg implementation. Signed-off-by: David Howells cc: Karsten Graul cc: Wenjia Zhang cc: Jan Karcher cc: "D. Wythe" cc: Tony Lu cc: Wen Gu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-s390@vger.kernel.org cc: netdev@vger.kernel.org --- net/smc/af_smc.c | 29 ----------------------------- net/smc/smc_stats.c | 2 +- net/smc/smc_stats.h | 1 - net/smc/smc_tx.c | 19 ------------------- net/smc/smc_tx.h | 2 -- 5 files changed, 1 insertion(+), 52 deletions(-) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 538e9c6ec8c9..a7f887d91d89 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -3133,34 +3133,6 @@ static int smc_ioctl(struct socket *sock, unsigned int cmd, return put_user(answ, (int __user *)arg); } -static ssize_t smc_sendpage(struct socket *sock, struct page *page, - int offset, size_t size, int flags) -{ - struct sock *sk = sock->sk; - struct smc_sock *smc; - int rc = -EPIPE; - - smc = smc_sk(sk); - lock_sock(sk); - if (sk->sk_state != SMC_ACTIVE) { - release_sock(sk); - goto out; - } - release_sock(sk); - if (smc->use_fallback) { - rc = kernel_sendpage(smc->clcsock, page, offset, - size, flags); - } else { - lock_sock(sk); - rc = smc_tx_sendpage(smc, page, offset, size, flags); - release_sock(sk); - SMC_STAT_INC(smc, sendpage_cnt); - } - -out: - return rc; -} - /* Map the affected portions of the rmbe into an spd, note the number of bytes * to splice in conn->splice_pending, and press 'go'. Delays consumer cursor * updates till whenever a respective page has been fully processed. @@ -3232,7 +3204,6 @@ static const struct proto_ops smc_sock_ops = { .sendmsg = smc_sendmsg, .recvmsg = smc_recvmsg, .mmap = sock_no_mmap, - .sendpage = smc_sendpage, .splice_read = smc_splice_read, }; diff --git a/net/smc/smc_stats.c b/net/smc/smc_stats.c index e80e34f7ac15..ca14c0f3a07d 100644 --- a/net/smc/smc_stats.c +++ b/net/smc/smc_stats.c @@ -227,7 +227,7 @@ static int smc_nl_fill_stats_tech_data(struct sk_buff *skb, SMC_NLA_STATS_PAD)) goto errattr; if (nla_put_u64_64bit(skb, SMC_NLA_STATS_T_SENDPAGE_CNT, - smc_tech->sendpage_cnt, + 0, SMC_NLA_STATS_PAD)) goto errattr; if (nla_put_u64_64bit(skb, SMC_NLA_STATS_T_CORK_CNT, diff --git a/net/smc/smc_stats.h b/net/smc/smc_stats.h index 84b7ecd8c05c..b60fe1eb37ab 100644 --- a/net/smc/smc_stats.h +++ b/net/smc/smc_stats.h @@ -71,7 +71,6 @@ struct smc_stats_tech { u64 clnt_v2_succ_cnt; u64 srv_v1_succ_cnt; u64 srv_v2_succ_cnt; - u64 sendpage_cnt; u64 urg_data_cnt; u64 splice_cnt; u64 cork_cnt; diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c index 9b9e0a190734..3b0ff3b589c7 100644 --- a/net/smc/smc_tx.c +++ b/net/smc/smc_tx.c @@ -297,25 +297,6 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len) return rc; } -int smc_tx_sendpage(struct smc_sock *smc, struct page *page, int offset, - size_t size, int flags) -{ - struct msghdr msg = {.msg_flags = flags}; - char *kaddr = kmap(page); - struct kvec iov; - int rc; - - if (flags & MSG_SENDPAGE_NOTLAST) - msg.msg_flags |= MSG_MORE; - - iov.iov_base = kaddr + offset; - iov.iov_len = size; - iov_iter_kvec(&msg.msg_iter, ITER_SOURCE, &iov, 1, size); - rc = smc_tx_sendmsg(smc, &msg, size); - kunmap(page); - return rc; -} - /***************************** sndbuf consumer *******************************/ /* sndbuf consumer: actual data transfer of one target chunk with ISM write */ diff --git a/net/smc/smc_tx.h b/net/smc/smc_tx.h index 34b578498b1f..a59f370b8b43 100644 --- a/net/smc/smc_tx.h +++ b/net/smc/smc_tx.h @@ -31,8 +31,6 @@ void smc_tx_pending(struct smc_connection *conn); void smc_tx_work(struct work_struct *work); void smc_tx_init(struct smc_sock *smc); int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len); -int smc_tx_sendpage(struct smc_sock *smc, struct page *page, int offset, - size_t size, int flags); int smc_tx_sndbuf_nonempty(struct smc_connection *conn); void smc_tx_sndbuf_nonfull(struct smc_sock *smc); void smc_tx_consumer_update(struct smc_connection *conn, bool force); From patchwork Fri Jun 23 11:44:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5E05EB64D7 for ; Fri, 23 Jun 2023 11:45:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44FC08D000C; Fri, 23 Jun 2023 07:45:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4001C8D0001; Fri, 23 Jun 2023 07:45:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C8E78D000C; Fri, 23 Jun 2023 07:45:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1E8898D0001 for ; Fri, 23 Jun 2023 07:45:19 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E8EEAB0BF2 for ; Fri, 23 Jun 2023 11:45:18 +0000 (UTC) X-FDA: 80933831916.27.99B9653 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf17.hostedemail.com (Postfix) with ESMTP id EEE9C40014 for ; Fri, 23 Jun 2023 11:45:16 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=U1KpF3eI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520717; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WiURs0vKXpYUCqXgawYZgqdOEVHMSUaWINzZV5vrdeY=; b=q+cASt33kWZUufsvLJgS/fy0vLRTmGchfVl8mFYseECD4SBpaHKR2weihuZEyxuDIGRHG2 4xqtiEFHhcVdEg9T6sJr/G2G4K5zUMFX8lMUavdUlVbyDFJeRbpgVKaUS9CwSxS1ycCiWr ZJveq5mV08bzpXAEIl4gKZtT/5XDjnU= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=U1KpF3eI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520717; a=rsa-sha256; cv=none; b=K4tZe1eBT0a84WWMF6wsAAgJe7FYAwrXNz9xj0/lBrTRdt1BpSQug2OtkF2VxL13hvrnRd 6ujVZnSqccVWGJERzk4uvEt4aczltcMNO+Kppo2NH/W3wvk+QBfIpnt9CAw4AwDq1Ueb38 t/Y7fN2mFa6t1GgBRfO1an5c3D1/cjM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WiURs0vKXpYUCqXgawYZgqdOEVHMSUaWINzZV5vrdeY=; b=U1KpF3eIYPgO2VEJNO5QriSaf4Kd89WZoctq9Gq/toVA6Fo3ZqRRYtfPByXHeWrka7iIrM XuMx85eel/a5B9+xMS+8eeeyzoyYgVib/YADhH5bAl+0Inv9mowNDUWA6J0PGaBiStvZvl kbrR/DpoPgc8isj6jjHvh+6BjVnppMw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-556-X_gMScNrP0Wfu8HM0-rlVQ-1; Fri, 23 Jun 2023 07:45:12 -0400 X-MC-Unique: X_gMScNrP0Wfu8HM0-rlVQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3D9733C15FC2; Fri, 23 Jun 2023 11:45:08 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8B4EB200B402; Fri, 23 Jun 2023 11:44:53 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Philipp Reisner , Lars Ellenberg , =?utf-8?q?Christoph_B=C3=B6hmwa?= =?utf-8?q?lder?= , drbd-dev@lists.linbit.com, linux-block@vger.kernel.org Subject: [PATCH net-next v4 10/15] drbd: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage() Date: Fri, 23 Jun 2023 12:44:20 +0100 Message-ID: <20230623114425.2150536-11-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Queue-Id: EEE9C40014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: rnhpbb1mdn7o9nyjt9tugd5gicg47ixz X-HE-Tag: 1687520716-353593 X-HE-Meta: U2FsdGVkX18giyaZD+xeknhfkMp/RWqE9tKIQTpXbJSjqIcFUHO+HxjUkOFHDAYuVu09SzigvuPbJE0W+LlX3Mbgw78xYW3bJSFPG+R7EXLiiTc4hHmWMgZKm0Mfzf6ywhjpGLe9PIKOnZnmdXjEpUhijRrq2GEBI6+ebpYt3MyNYrFT2IXMQv07uGkyc006+qPlu6DollBSCWYzvnUOnn/BMIxmsP1NxaMjrzo9v3+2jlYot1kAx/QMhsGkCt0dMQu3va5UPYxoig3Xmf5Q/MDPZKBbxC1yAzklLZbRqi0GS7PqJvdUFsN3maYZdEM+eUnZWs2aD6NlJPeEukdBUcxtuJQogH7uDvw/CW6fzpI3J6kmSqtVSAlytRcEX6Ne4WINy4quAiR5iq8ghXQR8Igdtpv6jpNTNT3DFnlkBbRowhvTJBX9xHvZmwHsVaHceO4Fl0eLebA5tlZFUg3RePrNypqxBD1yuZlmru+taq7p0Bf9uXEcr47dg6UTe3GE4c92RtTJ5y5P4zyNtvtqT5fCgpjTCEt5ZOBGMUEpmxbEAqVbQ15zO64bNHar0CbrtpJe/yazHZ5zqZ5KGPvDBp8/ndB4jFmkF3RvxXzoQJ8YR7nSse8VYgHkard+XlzrzLzTbmjnmWz2JK/ppPpxM7t3QknbfI5QNJHO2Vp9R+JXJxVtACB0h8NP9ViyoSgQwodCcoEvFsoMmH7hNZxy3vnSwlUFprqU1kE20BihvWpUNsB4cuzERIrfSebz5xTXwrwlFEMrnNj7/E/GWB7xN+B9Hhi1HCEAgEXEDnSWqwZyp5LcMbJayf82DXUAZmHNe4KglkbtDdB29cZDrmfqY1plDFGx3rAAyhZAke712NVa++Bi6dOPh56wDhg4UyFpu9MSDMO/zv6+dw4hMZSMlzDGV5v729akbQwXhcjwYgcz1AoWfN6OdrWpbL20W1rCnMJnC8/L52Iar7w4HKz vTwxDY+W lMOxzHK879Fm2/xysfh0pOygpW0uHz3mlPZDxGKYaQ6n7nVo5jiUGvEGAA1Noiz05mowGffoD331j2tGq4TSN5fj2//ObobmHaB8rWPcuswoKlso/Xr2WxVRDWZPVE9T/AlvY+ExufmTNmVv9bCxBCfr9I0C7YoVPewEo9Cm3+/4oEA1950MhJjSWkOIX6hnuN6nzPAbxuprpAc8KFL2VYhxhPihRvBx4v7W+ynVWhqRgdafZPhhgeCCddTKuQyGic3A4Zzwh6ut+1kQSQu/DEd+Y4D2V01pKwNt5LTB1R0f9/nwxBii5X2SV2DESiXuXipPCjeF9c6ISnppdgOF5AM0mtbvxuNLzGG4Lv2YyWWOY4MUknfn3lzEmg2MMeU2oedD5GeS4EeOZXrwpvSdNkYCY3vtfQ5hQu9nGM19mxN5PA9oTBagnvZQOdZA0ROMHnoiLRQ7RokAR937ynGkovisNbPuI/sgGwVBtya5TAdoLi2JzOSlAPatMiw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use sendmsg() conditionally with MSG_SPLICE_PAGES in _drbd_send_page() rather than calling sendpage() or _drbd_no_send_page(). Signed-off-by: David Howells cc: Philipp Reisner cc: Lars Ellenberg cc: "Christoph Böhmwalder" cc: Jens Axboe cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: drbd-dev@lists.linbit.com cc: linux-block@vger.kernel.org cc: netdev@vger.kernel.org --- Notes: ver #4) - Don't look at msg.msg_iter after calling sendmsg. There's no guarantee it has changed. ver #2) - Wrap lines at 80. drivers/block/drbd/drbd_main.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c index 83987e7a5ef2..ea82d6733313 100644 --- a/drivers/block/drbd/drbd_main.c +++ b/drivers/block/drbd/drbd_main.c @@ -1540,6 +1540,8 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa int offset, size_t size, unsigned msg_flags) { struct socket *socket = peer_device->connection->data.socket; + struct msghdr msg = { .msg_flags = msg_flags, }; + struct bio_vec bvec; int len = size; int err = -EIO; @@ -1549,15 +1551,17 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa * put_page(); and would cause either a VM_BUG directly, or * __page_cache_release a page that would actually still be referenced * by someone, leading to some obscure delayed Oops somewhere else. */ - if (drbd_disable_sendpage || !sendpage_ok(page)) - return _drbd_no_send_page(peer_device, page, offset, size, msg_flags); + if (!drbd_disable_sendpage && sendpage_ok(page)) + msg.msg_flags |= MSG_NOSIGNAL | MSG_SPLICE_PAGES; - msg_flags |= MSG_NOSIGNAL; drbd_update_congested(peer_device->connection); do { int sent; - sent = socket->ops->sendpage(socket, page, offset, len, msg_flags); + bvec_set_page(&bvec, page, offset, len); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, len); + + sent = sock_sendmsg(socket, &msg); if (sent <= 0) { if (sent == -EAGAIN) { if (we_should_drop_the_connection(peer_device->connection, socket)) From patchwork Fri Jun 23 11:44:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5303DC0015E for ; Fri, 23 Jun 2023 11:45:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC5878D0008; Fri, 23 Jun 2023 07:45:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D75F48D0001; Fri, 23 Jun 2023 07:45:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C64EB8D0008; Fri, 23 Jun 2023 07:45:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B81DE8D0001 for ; Fri, 23 Jun 2023 07:45:23 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8E91BC0ED6 for ; Fri, 23 Jun 2023 11:45:23 +0000 (UTC) X-FDA: 80933832126.13.79A0C4E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 1D8EE1A0005 for ; Fri, 23 Jun 2023 11:45:19 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TGafjdlH; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520720; a=rsa-sha256; cv=none; b=aOyQ2bWbry1efKPVnHKG07e5VIwLQPqgRxnwWxczrS66U5LnjYTifpoNEs9sAmdFowvoGr P9rg+NVVhVZk0yRp3vYnzvjPRt4cPCIeDt6Aj0VG72rmPh9rFrYdpJc+8vPqL7CLRqvL3m jEWLBP36/7xJ4GvSe7XswXdR8lV+zr8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=TGafjdlH; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520720; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fDJq6YgohECWMLRvYSaMvRBz5Nf3qARzDo2+NfjppDk=; b=XTgag5gHejCYKS2WswOl0eicHepEs+Ytir+D4Q7XVgAfijcGyBC7wuqgr2phyy1B12e83T H0u8+rDtgR3TwEA+m14UDZXeabflri0I96BBPyRjGbhjZPj0m7iw2tuonxEOWnK6ENt4jj ht4j4Sg7BF/UXVIKQMl+yESlc7B44I0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fDJq6YgohECWMLRvYSaMvRBz5Nf3qARzDo2+NfjppDk=; b=TGafjdlHr7wcASnAhF0mLuPE7xam6T8akcbeN+s1BW+2pIdMwjEKtc38VeQvyf3AXLEAeK SX1smMzCkzoAFWzbeug81Qr8tjTUFYXSUOskLWP2bTbMn/ELFvjJqYXSGwUz0WRUwaoiEX MbOSIisoeXHgIeW/UXO4SfpEpwCIiwI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-22-hRm2Fq0zPDmOXsotSEqhiA-1; Fri, 23 Jun 2023 07:45:14 -0400 X-MC-Unique: hRm2Fq0zPDmOXsotSEqhiA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6034588D7F7; Fri, 23 Jun 2023 11:45:11 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id E750E492C13; Fri, 23 Jun 2023 11:45:08 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lee Duncan , Chris Leech , Mike Christie , Maurizio Lombardi , "James E.J. Bottomley" , "Martin K. Petersen" , Al Viro , open-iscsi@googlegroups.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Subject: [PATCH net-next v4 11/15] iscsi: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage Date: Fri, 23 Jun 2023 12:44:21 +0100 Message-ID: <20230623114425.2150536-12-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 1D8EE1A0005 X-Stat-Signature: 4reriz19j5wqym1deiajfmu18xwzyg6k X-Rspam-User: X-HE-Tag: 1687520719-224135 X-HE-Meta: U2FsdGVkX1/kUTbrb92SMb3wCHXCm+mmuOA0u3/n0YJeTBBnG8jwGqao2U5NqAElPbsYfAL303avV5YK26LB+5Ne/rnOHJhfpC8CV0qv43z7KEg6HrMMEUh3ctKmlt7bjuAZ2u7zrfFxZFWgP6Mw6e2EvZc1CJNtCQCehMFk+V9NEoSBzZse2IU/+ijCOY/h3EyUYI3S16rsgV/qzl008Y7BnfhpU2n6LhZ1b1YfnmwRfSq4FVH5VtCmylBZRyyjnr0uoPMIMJHoY/F5ct7GJ9+AW8B1EJB7kDdS4j33gVK1HQGGea0eHqWFYNMZGqbXupVoCtRuIDjGf0x3mwn7wK2OUx7OjRvtCBcb975e1tPP7w/bzdBV+f6lPhZRPe+OLXBfUJco4aLngM0r5PvPpGF5P54FTaOHY2mEBJtHKknAOzvlqnl2WYxFVlXTmGOkAqeVq4IIbHISlWeqNARRMMj5CasvQzx0nGsT6Ej5mC/rcssJ/+Hx9n16f7snXvyvvpN6XXmL543EonX4ublfTcMd6sb9VRiAc9qB0sbWyYwZ0PHPFP5TrScdmP+6wEqPQkevpllFXkAw1QSyVNrqP7Wb/nUftrEjv2iquEN8i/KUuL8DSGqkGiI7yDMQGFbg8HDRfa+ZKyMKugPlaMZh5Y0TPAoWY0eSirUlyg2fsbS4407apCN1xWaAvziUzav1G4VTJZArMdnGO2d4YxVvPG/XwHXOUsCL9+ae5QyvapRZg5hCy6ayBloNR507dJx4jms6jdvhmaGTVIHc25rPleMmIQSAcjPgXFLJTLbtpLswXv9s7L5mDLWmleMVnFYkNdm5pMwQiynbR23g54G0iEZDfn3QSdSGLqryJcqi8ggT8iZGptWwKz8pCqaslb/e5aCt4o1q4xg1cOllZoCIrGjqeZuaa5MvVLMWUEJu/nWxYOnrU/bXLFHeONNisvJihhO1IpKEluDbDyEMDqY 0fDIdiD5 rwGT/7Bdl6nqtjjcaVl7mKwT3tCNpmne3C+SNhRB9gOM8vU3A0duKw1HfzVH0V1g6Oh7VZSmqTioezNqHDhgZsNtg5E2ZjELt6nsk3omLShBMffIznxUn3ECzTsNqoLQ+ENFg9DhvoKiU2uCHRudzNnwo45rNJEfEjPUVHPA/mAooI/fumaYJ9iPIL+tyIEgW3OOnIf3KPsvKBfT7RlFQy/ouiOteyDm8mI76I5vpi0seIacrAWNyCkOnIin5mPzCA8mWUg6hdJg40+FyNGSNKCuSBzLmz1IHZVd1FKcZypqvcZHoA6q7IGSz1W424OhI04nROjhMtc5eC3hFKoYkvGeMxxJaX7wLOCzRWi8MYcTxj/3ImqG5z2PuqQM4yihGzPhVdEjz1HRujpnCoCOjHTUHeDW39+39hRTmDQrP1DSpqhLx2gon6NuL10ahAohPyE+QxbpZW3QNyNnUVLURjQRqctIJGrIRpp9gdDxINP37D+JbE+wao0259bV739aU8yAbC6w1aMUg5o9cZy7GZRkSGd/fOyuh3e0Bl67m1RP+9ci0A0J2LQrx4Frq88DUBnQ8waSHuYIYRHKLB7gQHlLgnoIzgf5/U44gU1xrbGZ7GHrr+hltLdjpNimxRgtimlrVjdkb0+FbuL3Vk21YHq81Yb4KNfUTOUvoEHwxhv2cCflaQRg9wATdKqP6ysi7YlabwGWKzSNF+v5vZElNCRAiMA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use sendmsg() with MSG_SPLICE_PAGES rather than sendpage. This allows multiple pages and multipage folios to be passed through. TODO: iscsit_fe_sendpage_sg() should perhaps set up a bio_vec array for the entire set of pages it's going to transfer plus two for the header and trailer and page fragments to hold the header and trailer - and then call sendmsg once for the entire message. Signed-off-by: David Howells cc: Lee Duncan cc: Chris Leech cc: Mike Christie cc: Maurizio Lombardi cc: "James E.J. Bottomley" cc: "Martin K. Petersen" cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: Al Viro cc: open-iscsi@googlegroups.com cc: linux-scsi@vger.kernel.org cc: target-devel@vger.kernel.org cc: netdev@vger.kernel.org Reviewed-by: Mike Christie --- Notes: ver #2) - Wrap lines at 80. drivers/scsi/iscsi_tcp.c | 26 +++++++++--------------- drivers/scsi/iscsi_tcp.h | 2 -- drivers/target/iscsi/iscsi_target_util.c | 15 ++++++++------ 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c index 9637d4bc2bc9..9ab8555180a3 100644 --- a/drivers/scsi/iscsi_tcp.c +++ b/drivers/scsi/iscsi_tcp.c @@ -301,35 +301,32 @@ static int iscsi_sw_tcp_xmit_segment(struct iscsi_tcp_conn *tcp_conn, while (!iscsi_tcp_segment_done(tcp_conn, segment, 0, r)) { struct scatterlist *sg; + struct msghdr msg = {}; + struct bio_vec bv; unsigned int offset, copy; - int flags = 0; r = 0; offset = segment->copied; copy = segment->size - offset; if (segment->total_copied + segment->size < segment->total_size) - flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST; + msg.msg_flags |= MSG_MORE; if (tcp_sw_conn->queue_recv) - flags |= MSG_DONTWAIT; + msg.msg_flags |= MSG_DONTWAIT; - /* Use sendpage if we can; else fall back to sendmsg */ if (!segment->data) { + if (!tcp_conn->iscsi_conn->datadgst_en) + msg.msg_flags |= MSG_SPLICE_PAGES; sg = segment->sg; offset += segment->sg_offset + sg->offset; - r = tcp_sw_conn->sendpage(sk, sg_page(sg), offset, - copy, flags); + bvec_set_page(&bv, sg_page(sg), copy, offset); } else { - struct msghdr msg = { .msg_flags = flags }; - struct kvec iov = { - .iov_base = segment->data + offset, - .iov_len = copy - }; - - r = kernel_sendmsg(sk, &msg, &iov, 1, copy); + bvec_set_virt(&bv, segment->data + offset, copy); } + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bv, 1, copy); + r = sock_sendmsg(sk, &msg); if (r < 0) { iscsi_tcp_segment_unmap(segment); return r; @@ -746,7 +743,6 @@ iscsi_sw_tcp_conn_bind(struct iscsi_cls_session *cls_session, sock_no_linger(sk); iscsi_sw_tcp_conn_set_callbacks(conn); - tcp_sw_conn->sendpage = tcp_sw_conn->sock->ops->sendpage; /* * set receive state machine into initial state */ @@ -777,8 +773,6 @@ static int iscsi_sw_tcp_conn_set_param(struct iscsi_cls_conn *cls_conn, return -ENOTCONN; } iscsi_set_param(cls_conn, param, buf, buflen); - tcp_sw_conn->sendpage = conn->datadgst_en ? - sock_no_sendpage : tcp_sw_conn->sock->ops->sendpage; mutex_unlock(&tcp_sw_conn->sock_lock); break; case ISCSI_PARAM_MAX_R2T: diff --git a/drivers/scsi/iscsi_tcp.h b/drivers/scsi/iscsi_tcp.h index 68e14a344904..89a6fc552f0b 100644 --- a/drivers/scsi/iscsi_tcp.h +++ b/drivers/scsi/iscsi_tcp.h @@ -47,8 +47,6 @@ struct iscsi_sw_tcp_conn { /* MIB custom statistics */ uint32_t sendpage_failures_cnt; uint32_t discontiguous_hdr_cnt; - - ssize_t (*sendpage)(struct socket *, struct page *, int, size_t, int); }; struct iscsi_sw_tcp_host { diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c index b14835fcb033..6231fa4ef5c6 100644 --- a/drivers/target/iscsi/iscsi_target_util.c +++ b/drivers/target/iscsi/iscsi_target_util.c @@ -1129,6 +1129,8 @@ int iscsit_fe_sendpage_sg( struct iscsit_conn *conn) { struct scatterlist *sg = cmd->first_data_sg; + struct bio_vec bvec; + struct msghdr msghdr = { .msg_flags = MSG_SPLICE_PAGES, }; struct kvec iov; u32 tx_hdr_size, data_len; u32 offset = cmd->first_data_sg_off; @@ -1172,17 +1174,18 @@ int iscsit_fe_sendpage_sg( u32 space = (sg->length - offset); u32 sub_len = min_t(u32, data_len, space); send_pg: - tx_sent = conn->sock->ops->sendpage(conn->sock, - sg_page(sg), sg->offset + offset, sub_len, 0); + bvec_set_page(&bvec, sg_page(sg), sub_len, sg->offset + offset); + iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, sub_len); + + tx_sent = conn->sock->ops->sendmsg(conn->sock, &msghdr, + sub_len); if (tx_sent != sub_len) { if (tx_sent == -EAGAIN) { - pr_err("tcp_sendpage() returned" - " -EAGAIN\n"); + pr_err("sendmsg/splice returned -EAGAIN\n"); goto send_pg; } - pr_err("tcp_sendpage() failure: %d\n", - tx_sent); + pr_err("sendmsg/splice failure: %d\n", tx_sent); return -1; } From patchwork Fri Jun 23 11:44:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290410 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4090C0015E for ; Fri, 23 Jun 2023 11:45:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 366248D000D; Fri, 23 Jun 2023 07:45:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EFF18D0001; Fri, 23 Jun 2023 07:45:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18F828D000D; Fri, 23 Jun 2023 07:45:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0A8D08D0001 for ; Fri, 23 Jun 2023 07:45:20 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CBB441A0F2F for ; Fri, 23 Jun 2023 11:45:19 +0000 (UTC) X-FDA: 80933831958.16.24AF86B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 00FB14000B for ; Fri, 23 Jun 2023 11:45:17 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bbfwwU0y; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520718; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZDy9qTqAXsZiyNyw/4xfrjcFbGXml691uHuF37I4PZg=; b=2JbAi3zP1v4A9CDkNusIsz3IiHyVLGKK1ar6Y0rOICM0PRRvhstfumOg6iPxcfmTDG+Xij zmyOlxYnnmb/Kp94W6nVsWBWnfX/IBSlHcHUfxCbvq2LATWMpIeueA6gLPfsQP2dDPOvXt aA+btm4Xbat5Jb4N3HGTccFRZKqZzdw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bbfwwU0y; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520718; a=rsa-sha256; cv=none; b=vGZgeogDsINDI785t/MAXzC9u/8iyPN4c7YeYiD0L5DmDbDSzmFjAIW0PUbDbNBMy083yP opIADi+twxaVK1vGmbmHs7slMbNdadlQmwiUOTS5CUixn761jPxEFB1JffzajlXKziPSKt RGRWkbr14RK7gOPuzvyZ0X3eLcnIsO8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520717; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZDy9qTqAXsZiyNyw/4xfrjcFbGXml691uHuF37I4PZg=; b=bbfwwU0yoqBvuxMzsg7B3w1OSfShifjoAHo6wgKoeQVggdSbQjIW5zwSogLtucZvJR88K6 iReUaYByM4O1d2i4sZHF5YLxfUVn9fpU46phLzP/+5T3BK2NtBAYWC1nWsJCyhzzwL9w8n oN7mPZEf7ZS8qZZHzCNQKHUoKGds6Qo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-107-y_gjQWLAObqQcPZDkV39dQ-1; Fri, 23 Jun 2023 07:45:15 -0400 X-MC-Unique: y_gjQWLAObqQcPZDkV39dQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D14C63804A43; Fri, 23 Jun 2023 11:45:13 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED36015230A0; Fri, 23 Jun 2023 11:45:11 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mark Fasheh , Joel Becker , Joseph Qi , ocfs2-devel@oss.oracle.com Subject: [PATCH net-next v4 12/15] ocfs2: Fix use of slab data with sendpage Date: Fri, 23 Jun 2023 12:44:22 +0100 Message-ID: <20230623114425.2150536-13-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspam-User: X-Stat-Signature: f9omt7616sxcnknwang3wihtak7y1kh9 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 00FB14000B X-HE-Tag: 1687520717-670988 X-HE-Meta: U2FsdGVkX19jTaDLoD3ihVKqtyeR7XEujN16uWkZj41zroqJlQqRyUJZfbsFik9+R0dEuM2iKfQPXwvBaQXWWo2NPt6hSjmEjrRuDJC1d3lWroJ0XvrhU1haLZMsx+JEyQjqhCkdwzxnpmXM/CJlRcWLyJ/aTEIn5gGwnWZN2rzEaeV5zZySB6mIBMXRdkfUs12pYKXRs130BNNHq05c6e8aYCOjNydWeNEHVxDziZs/kHYtB6N9N3OcKKgjzbW3Hq76DFEpeydtuPA+VZ6eLV3pg+DzgQPE/b2Kz8h8yJVuohlUcUjBU3Bfm8A+O07YOjnRlwboqBdwXDFH1l4KEt/jGHSANs4r8jiMp772mb3jDejKk5/VGs2demjxfBKKVB8rlpIfoqAluoESn+lk4sHIUloQ9+sutEabDSxtOjfFn7y/XSFyFb557qbNOLzU608lQbwNOG02xKLa1j88Mynapc6dw/+NRITzMBmXD4VRX6Eiv15JJ9FsV/KZgO/WKO+OG2/9Qn7SVOmwEUNZy0m4oRnBVNFLoFbMSsuilJiXufAfg7JAxvx5DS++ze1V1OA1m4QADYQmfJqRm3Eg/2eYuDQzN+h+o0hzlD3WWlP7OMbc7ToOdtmJes932rrcruBiHg2QWCjy96w/0a69BefPZu7hDtDyEzMZFUicdD2dwb68DiV0xqHphQUUV0vsXJxALDX/F1kxktLnefOxsOLEFztfuStGk68nG73ijnmUBN4tdzxtz3wchBV5HHNlR1MAW2NyD2cL8V0OV0cL4qI8zHBPIlmznldI46L7ma9MVKefSFiNXwwU0Oull7Q8Nw1mi5i1x5gZrruI3meelq7KwIxFOzpFL5P84Sjbmcbzvr22RT9F90WnB72Ke3XmYJgx+3zSSbjXI0Z9G2UWEQtm7HOz6RaWw6khsKo8kTHNakr9l0hWkJU2cxbrB5G9jqH3Gnq+JSGUUZheyUm zFVbJALW 1bSsHauNkUdvSAt6w8Tr3xofQznALKScJsYiTcvrsH/BVyvbdlS0Qo9ukuJARj+beuTXLmBaqmyCA4Et2eIJavBksD1k/Y4+t8KmYG3XKuKlsNuj25u8xF40VvNpiUw0coWWb4NkQxSXAwXYJlF0Zix1ktiB5p5RWH/ITnGKIs9cOX/v4W48UMLVCpVwenHxYmy5jJHUNVjSn05TjZ3MSr3km17g6zQpGUnlVKxMamq0EHu3PdUhb+ACVSeLTnETSpgXWBb1vOQLd08XdtRNW6BQ4eqE6XherZz/vMu4ygYgU8kQFECx3z0i4gVTTwefIMiP7C6q6KqY8f3RgClwYAeUFaFFrhf5B3QWmFdnJsTbGGne6LQAN2I6lKQ3V7IHo31cf8e1VpvIqp7HLK7NFGM5eR0fYyMcS/p+ueAt482eoa8nSnXAHkGnMZTsEmEw4aBYTPCbd6UI6uCkY5eDc+lB8tJ51ZugUTvgy3Pq3K0fgMwi46TeW9Qsg7SlD6kCD1YqJZ9V3yCivkjvC1jl+XN+eyKn2USBgUgx2/zKw+04KGIM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ocfs2 uses kzalloc() to allocate buffers for for o2net_hand, o2net_keep_req and o2net_keep_resp and then passes these to sendpage. This isn't really allowed as the lifetime of slab objects is not controlled by page ref - though in this case it will probably work. sendmsg() with MSG_SPLICE_PAGES will, however, print a warning and give an error. Fix it to use folio_alloc() instead to allocate a buffer for the handshake message, keepalive request and reply messages. Fixes: 98211489d414 ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem") Signed-off-by: David Howells cc: Mark Fasheh cc: Joel Becker cc: Joseph Qi cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: ocfs2-devel@oss.oracle.com cc: netdev@vger.kernel.org --- fs/ocfs2/cluster/tcp.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c index aecbd712a00c..929a1133bc18 100644 --- a/fs/ocfs2/cluster/tcp.c +++ b/fs/ocfs2/cluster/tcp.c @@ -2087,18 +2087,24 @@ void o2net_stop_listening(struct o2nm_node *node) int o2net_init(void) { + struct folio *folio; + void *p; unsigned long i; o2quo_init(); - o2net_debugfs_init(); - o2net_hand = kzalloc(sizeof(struct o2net_handshake), GFP_KERNEL); - o2net_keep_req = kzalloc(sizeof(struct o2net_msg), GFP_KERNEL); - o2net_keep_resp = kzalloc(sizeof(struct o2net_msg), GFP_KERNEL); - if (!o2net_hand || !o2net_keep_req || !o2net_keep_resp) + folio = folio_alloc(GFP_KERNEL | __GFP_ZERO, 0); + if (!folio) goto out; + p = folio_address(folio); + o2net_hand = p; + p += sizeof(struct o2net_handshake); + o2net_keep_req = p; + p += sizeof(struct o2net_msg); + o2net_keep_resp = p; + o2net_hand->protocol_version = cpu_to_be64(O2NET_PROTOCOL_VERSION); o2net_hand->connector_id = cpu_to_be64(1); @@ -2124,9 +2130,6 @@ int o2net_init(void) return 0; out: - kfree(o2net_hand); - kfree(o2net_keep_req); - kfree(o2net_keep_resp); o2net_debugfs_exit(); o2quo_exit(); return -ENOMEM; @@ -2135,8 +2138,6 @@ int o2net_init(void) void o2net_exit(void) { o2quo_exit(); - kfree(o2net_hand); - kfree(o2net_keep_req); - kfree(o2net_keep_resp); o2net_debugfs_exit(); + folio_put(virt_to_folio(o2net_hand)); } From patchwork Fri Jun 23 11:44:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290412 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD48EC001B1 for ; Fri, 23 Jun 2023 11:45:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E9748D000E; Fri, 23 Jun 2023 07:45:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 673F58D0001; Fri, 23 Jun 2023 07:45:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49D288D000E; Fri, 23 Jun 2023 07:45:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3CB528D0001 for ; Fri, 23 Jun 2023 07:45:25 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E8AA3140EF5 for ; Fri, 23 Jun 2023 11:45:24 +0000 (UTC) X-FDA: 80933832168.02.3860676 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 3722EC0012 for ; Fri, 23 Jun 2023 11:45:22 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UzHNYP6Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6qW7QONKdXKQCe18ckz4PBDuqjad2A9AyVw+035R3zM=; b=glhkLT7gtlYemXxdbjuQcwEQJ0vm+GHQVqp130VY88sD4eLiZ1fiQmvROusDu3UqVTC5+M wrzSwEQL2MFTTPIHE5INlH8MmJOlerBFUJJpwgQgb/KrnyWQkKr1iLhJywB71c9q7t6NzL /gaqKeTOGO+yAxTVjJpxbnTFNu15LHo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UzHNYP6Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520723; a=rsa-sha256; cv=none; b=LtjL1cIzmjqlR/kLaprYgtoCTWXe8Ba8jVjdmUhlSO0e9sWSGCQsECYLDeJ4zZidOii//E tCHKq5j42cma6Da/EiS6i3QwJwkS0dkfojHRKpgMMkuK/2RWeqDigBYxQ7jQkMkwHQXeOt QFOQXjTa2ky8/N2DpfL3QCZi4LKQzeU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520722; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6qW7QONKdXKQCe18ckz4PBDuqjad2A9AyVw+035R3zM=; b=UzHNYP6Za19pgiNVjw4MDxLM5mYdIaSrmyRvVqGq+LNFwti4eWaCifuJpTdc0+vHLeTK9A 5drBTm5c3J21vk8M3r5/x6aqigN/hGz82PfgFZoGqJi7dqnse07wjOVe5j9yTTuHRNzw11 WWpCCAN08kLZ+loNyrEGNfTeqzlMdqY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-155-woVC1FXnMaeEutQKCYvqpQ-1; Fri, 23 Jun 2023 07:45:19 -0400 X-MC-Unique: woVC1FXnMaeEutQKCYvqpQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57BA529AB40B; Fri, 23 Jun 2023 11:45:16 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E6FC4087C6F; Fri, 23 Jun 2023 11:45:14 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mark Fasheh , Joel Becker , Joseph Qi , ocfs2-devel@oss.oracle.com Subject: [PATCH net-next v4 13/15] ocfs2: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage() Date: Fri, 23 Jun 2023 12:44:23 +0100 Message-ID: <20230623114425.2150536-14-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3722EC0012 X-Stat-Signature: 7hyfayze864khaxkxx4tupdazynanrfx X-HE-Tag: 1687520722-603431 X-HE-Meta: U2FsdGVkX19ub1EyDkB53Cxgro5m8PtXGAb7lsYfjlRibsj8NlvU/CBGVD07NeOHWct57AsE2l326Ci+ZGoYvZerIxaFp09BM+tOIySAXtECloYwyVpIf9TVyfFTmk8dJlZLtF9x2dm7dbp2AnqIL5HO6+3yz+oEjCeq7y74h2lGrQOqFSPcxWBWR8941CxDdmUO80XXMnVy2fHbO9arQ581rpr8Kelfo2C/OLHaJdSZb0lM4TtTXndgsFFO4KS1qfQ3Bcvo8zgZh23rw2y6tBdlMl+RRyBIIhz5rZktb5X8IMj3cFldUy4dGTjQU5J7H3n+1/ga0ZI56lwvmAFOYnSY0KJlaIKV218hZ9dQ1S2d93k2pLQ4F+W0qCAFZxdwM44KlAnRV+a34li2ioFmvJ4FWySXqXCMt9aZSla+Mjp8rHIL3o6huwru+sW4ALjSc9RdqzVPEbvq3L/LI7vIpEg0SdxFfuPd+8cfyQOcVozARvkpZO/S438LBTsbBPLELrWXaqNJ5cwMqDAum1NqqaKHomrmBPWOqrATQnuC4hRJk4CdyTfXG0e2LG+373mFTo5O2G4wjzkBVOovp3WS7ZYtkHK1+RVyWDHqD7Z0sYmfeQtvQvxz5neJKMSait3JoTn0NIuXVTPIQtP9HZ4GbW65NsYq5ECSR/ZuPpm7C0MYhrl6pXFBMZSctHHh48H6eF13vSn3m+K/FnLGHjVYXkbkNZV6nC4lqoLUCXfSKUA6NjCd5Ugu5OkwtGWvdas6j5DlIx0uF3H0dDn41TIJL6P7/vlkOmIbC1VIYWMStLd2siaOoUhRQvEV8aLwr+qvPlqX6r/qBAZxcohH2BRGoA9sUhc0XVn8ePPbsAZ07pMvDpNiVUUKalQTIbFeOPdmpAaCRLO5cssOvoSvLfP7yNOpPFQ+cs+YS4bjP22s/PULwF6B62xVtwFFWXIC95IEMtb2OTIs+rH+wApf7qh xHfmJzzP nWP8/FVw5Z9Uij2KuOewQnLl03R/PrdVcJ+JKGSy0XuECFcG1NbEGn3YdPKhIJz95hrQ/EP/txxYMn1k247/zDteuSoJfw1edT0RZK9WzSKTSVkqPjB4Pyb5XUalOzVa9Z0JZR0unRhB+qkfR6gmD7RS4ebRaTH62aN/adVhSM7SYzTo/zEWyNppJs4BWLCWtY8KIB+jHVe1ZeJu9LY+sBvaRMna1W3A5CsFQ2Ow3a7lawV+T0fD/xdzGfSNoly97BdTU6ulE1f1vZiIXcO6xJzOx+oD6VocVuQtgp8WlTy+mbxiubS+nRZ0/Bm9GIC6pzOWr3i0Q3J4NRQWDhEVbWxZ1hgOPQIRKGvsntwugH8Y83QQnMAfLZoDE4ZS2DcLqr/Asqjvc7yO//E0LEn/ZlGFRcPfg4kaaQ1QayjpHb+ey13ZKkhq2/XlKfOHW5iMxi7D3Ct1kHngPA/pFBB/rWUpbD1eB7+hA522rQGXAQ4SmVym+bxO1uOwa8o0JBU6lte1SYEu3RY9xO216s9kaeCP2iAKp6nLjZk2oi8+hXPLOY/71otVIiJQe1QuJJLF7e7PitlOoceGYRhVkwgJv/ncwl/sKQz1RL3pL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Switch ocfs2 from using sendpage() to using sendmsg() + MSG_SPLICE_PAGES so that sendpage can be phased out. Signed-off-by: David Howells cc: Mark Fasheh cc: Joel Becker cc: Joseph Qi cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: ocfs2-devel@oss.oracle.com cc: netdev@vger.kernel.org --- Notes: ver #4) - Use folio_alloc() for o2net_hand, o2net_keep_req and o2net_keep_resp. ver #2) - Wrap lines at 80. fs/ocfs2/cluster/tcp.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/fs/ocfs2/cluster/tcp.c b/fs/ocfs2/cluster/tcp.c index 929a1133bc18..960080753d3b 100644 --- a/fs/ocfs2/cluster/tcp.c +++ b/fs/ocfs2/cluster/tcp.c @@ -930,19 +930,22 @@ static int o2net_send_tcp_msg(struct socket *sock, struct kvec *vec, } static void o2net_sendpage(struct o2net_sock_container *sc, - void *kmalloced_virt, - size_t size) + void *virt, size_t size) { struct o2net_node *nn = o2net_nn_from_num(sc->sc_node->nd_num); + struct msghdr msg = {}; + struct bio_vec bv; ssize_t ret; + bvec_set_virt(&bv, virt, size); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bv, 1, size); + while (1) { + msg.msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES; mutex_lock(&sc->sc_send_lock); - ret = sc->sc_sock->ops->sendpage(sc->sc_sock, - virt_to_page(kmalloced_virt), - offset_in_page(kmalloced_virt), - size, MSG_DONTWAIT); + ret = sock_sendmsg(sc->sc_sock, &msg); mutex_unlock(&sc->sc_send_lock); + if (ret == size) break; if (ret == (ssize_t)-EAGAIN) { From patchwork Fri Jun 23 11:44:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13290413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA2FEC0015E for ; Fri, 23 Jun 2023 11:45:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86F108D000F; Fri, 23 Jun 2023 07:45:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81E298D0001; Fri, 23 Jun 2023 07:45:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BF468D000F; Fri, 23 Jun 2023 07:45:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5CA1A8D0001 for ; Fri, 23 Jun 2023 07:45:32 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1C99C1A072E for ; Fri, 23 Jun 2023 11:45:32 +0000 (UTC) X-FDA: 80933832504.20.C9687B1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 5DC56180014 for ; Fri, 23 Jun 2023 11:45:30 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VdG8ckHv; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687520730; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xQVg0pnXVfTxPUpA8yd29SmocczyZN6S+xS0BIA8ljI=; b=uTnLcBgaw0s533KuaoabjqtO+uxvm5k+b5J86e0XeoByn2QTOAh439WE0dUyl8pT8KiY+Y DRtACjHab3gXiTUMc50gQdu8qF6YoqbWM+SeRzZswCIgLZq/apekPxQ7zUROfYvetQkFWi 679yUefyyNCvO9fqd7DnI19+BsoTYAs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687520730; a=rsa-sha256; cv=none; b=g7mnISPZFTcPXKYKKYEVd5uXrXJ2eEQEBrn3YMg5q/TB/QcbNnvYyniVcVH/4pIcNDpzv4 vtW4dcpNK44zbFPFk2GSnhANqmlwqJQr/aTI9XV2+arSQzqoRS043kTgj1xJu7L0HRxtMz NheRjy7yufxdVl86y7w0ZXKR8B9ZbFo= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VdG8ckHv; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1687520729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xQVg0pnXVfTxPUpA8yd29SmocczyZN6S+xS0BIA8ljI=; b=VdG8ckHvYVwC2JEh+I5lL61ZpOyYQ8bfsuZouP5H5UQViHN/l/5PZaCzXBpAvtHyeRarU8 H83PjEcKOGlU/MZTmDncsR7xb5/s1O8Xsg3Ob7+A+CWztyd/Woqd298rNqZF5Y/Y/xzjWH pa2+tu3tp5VYxh7rKMTp3xaVpu2tfG0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-Fm_QQgf9PJ2p0L6RUxDWNA-1; Fri, 23 Jun 2023 07:45:26 -0400 X-MC-Unique: Fm_QQgf9PJ2p0L6RUxDWNA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C0D988E44EF; Fri, 23 Jun 2023 11:45:24 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 48F094087C6D; Fri, 23 Jun 2023 11:45:21 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Alexander Duyck , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, dccp@vger.kernel.org, linux-afs@lists.infradead.org, linux-arm-msm@vger.kernel.org, linux-can@vger.kernel.org, linux-crypto@vger.kernel.org, linux-doc@vger.kernel.org, linux-hams@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-rdma@vger.kernel.org, linux-sctp@vger.kernel.org, linux-wpan@vger.kernel.org, linux-x25@vger.kernel.org, mptcp@lists.linux.dev, rds-devel@oss.oracle.com, tipc-discussion@lists.sourceforge.net, virtualization@lists.linux-foundation.org Subject: [PATCH net-next v4 15/15] net: Kill MSG_SENDPAGE_NOTLAST Date: Fri, 23 Jun 2023 12:44:25 +0100 Message-ID: <20230623114425.2150536-16-dhowells@redhat.com> In-Reply-To: <20230623114425.2150536-1-dhowells@redhat.com> References: <20230623114425.2150536-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Queue-Id: 5DC56180014 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: x3fewagncj49rqeg38qmq16nhsjpju9b X-HE-Tag: 1687520730-972637 X-HE-Meta: U2FsdGVkX1+jlVuc06/lJat/qgsL6RiRxTSezXzXVk0PdQ4Ce+tuY0u7zFTGwVTZF3yLUfo6YCB3ABNuYe+mOgreydIIMyTHNdXFzerglLfv+lynu4d708jE5hiDX8BgcJvILaTFDivNSJxSMC/KyxOhzn8eRhqrafxjoYA+vdEiBY89SrSzdjKGrLOZiUh1/7vqq+NqwxTsj9U65JCOUpTyPIlzBhhLiVBL6DYaDjLsOIqdDnYQbRUP84yoMv8vZQS/h9nAVNqjnOelPZaJUnK663Vrl54q7WId0/SverRnYfhKmAJRUKmEFWNM1jqBOyQR9UHE1XiriCXWkU0k3goGpdHTC9Vqua3qGl9GHW+91krFaYh6xLvK/7vq179jDPnAKlz9XJWEtk5Whp6sOsf3ObhZf7U5K+5UKbNl/XJwRnZDaFSVlipEenblb1nRq3kkrpFq/kYJJ++ntKzhiYoCeMgUy7s2ww1krt6lIUnyYyLpVdOZKqm5rorcJHqNd4V7kBMBG0MV/QY5DZ9R2UouqklGV5IKmdvRlYq49zZXaeSin6BM0Dv1rYrq0Qfohh0qc0M6gNtSA7iEcNDEU8EsHBt4Rufmrz0EnYCr7XcdL/kgxIhMj7XcdIjQ3STxbUa8s9riEIr8dVwGPUxzcC+tyoXZzIzc7yYoPMfkb1S7Qi1p5yke8Ep75Q85aUpr74d3LLIQNzpjNwCN3fIrge/u/UTJsq6k9MzF1zYxNN4zjSDLWFN7K97yIweAiG6WjcaIHxmDmUGlBhBdCDFS0C8qrkfuS1iChcvuFWx4RifeidsmA019CMd6PrmDdU2S7E/cx7lGVn+lk8YakMHgRPmc4UeCEyBlEhq20CM0u+8b5V6x0TmNdRKClbAC5zd/r0MsuFdtifv7XicRclUuQCYENLHLv6Att4dF4zZGWijw7Z9P/MTss+uzndEk4bMComKTjlIIpen9i2cTwKI qdHfPn4r cuyHKy4IBtpS42xaN4ezvsBfYqR8lWus0uDDpc3KtxnjyLqBEtoUVNksETWWG0ZZJ91IrUCLFucXIKS4PbqHQm589aelA8FxAHKvvpTr0cfTrUVpkVNitUG2emvq4mD6xSr2XnBAr0AeSZXrWoiQmq2YU84S3915oFBlN20Z5J3Nd+vrK/bX17C/fnRQ2oeLjcHMwI1dn2AotQg5i0WnPM+EAMXihw+XeaUWchcSwvssHaMovLWhSX7aUQmnsAcoHHAlgSoPntj2Eqa2PZpKYOVOMKryQJ5OB6fDUSIjlyJPepNVSO4nLBzq77flnURu49IGYpgsCmH7FLHt18xJRXGsi+ECdPoqTh7PbdofpLpv3rfPyVxm8GddjIQsHH43mB0mcH5+OrntmSYXcCBj8MM6w6YMYNU9C3an/tyNf4qz0k76x/5N0QpYW1UbHyT/3Q+y9QRdKYqkrpdv2gEcAQptbfn5PZV3ILjJK+Rz32LC1r5s65Hy0ZD08exNBslPCajyfnfcTb4JJ1FduzmmTTLa+r80+ZVlV0dIp+s0HIIiJ01QxjQ4GqU9PJuQEw1YMc3ZX1nVZgowLltqdafsTKKxeKNTrmFECM9IS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that ->sendpage() has been removed, MSG_SENDPAGE_NOTLAST can be cleaned up. Things were converted to use MSG_MORE instead, but the protocol sendpage stubs still convert MSG_SENDPAGE_NOTLAST to MSG_MORE, which is now unnecessary. Signed-off-by: David Howells cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: bpf@vger.kernel.org cc: dccp@vger.kernel.org cc: linux-afs@lists.infradead.org cc: linux-arm-msm@vger.kernel.org cc: linux-can@vger.kernel.org cc: linux-crypto@vger.kernel.org cc: linux-doc@vger.kernel.org cc: linux-hams@vger.kernel.org cc: linux-perf-users@vger.kernel.org cc: linux-rdma@vger.kernel.org cc: linux-sctp@vger.kernel.org cc: linux-wpan@vger.kernel.org cc: linux-x25@vger.kernel.org cc: mptcp@lists.linux.dev cc: netdev@vger.kernel.org cc: rds-devel@oss.oracle.com cc: tipc-discussion@lists.sourceforge.net cc: virtualization@lists.linux-foundation.org --- Notes: ver #3) - tcp_bpf is now handled by an earlier patch. include/linux/socket.h | 4 +--- net/tls/tls_device.c | 3 +-- net/tls/tls_main.c | 2 +- net/tls/tls_sw.c | 2 +- tools/perf/trace/beauty/include/linux/socket.h | 1 - tools/perf/trace/beauty/msg_flags.c | 5 +---- 6 files changed, 5 insertions(+), 12 deletions(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index 58204700018a..39b74d83c7c4 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -319,7 +319,6 @@ struct ucred { #define MSG_MORE 0x8000 /* Sender will send more */ #define MSG_WAITFORONE 0x10000 /* recvmmsg(): block until 1+ packets avail */ #define MSG_SENDPAGE_NOPOLICY 0x10000 /* sendpage() internal : do no apply policy */ -#define MSG_SENDPAGE_NOTLAST 0x20000 /* sendpage() internal : not the last page */ #define MSG_BATCH 0x40000 /* sendmmsg(): more messages coming */ #define MSG_EOF MSG_FIN #define MSG_NO_SHARED_FRAGS 0x80000 /* sendpage() internal : page frags are not shared */ @@ -341,8 +340,7 @@ struct ucred { /* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ #define MSG_INTERNAL_SENDMSG_FLAGS \ - (MSG_SPLICE_PAGES | MSG_SENDPAGE_NOPOLICY | MSG_SENDPAGE_NOTLAST | \ - MSG_SENDPAGE_DECRYPTED) + (MSG_SPLICE_PAGES | MSG_SENDPAGE_NOPOLICY | MSG_SENDPAGE_DECRYPTED) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 840ee06f1708..2021fe557e50 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -441,8 +441,7 @@ static int tls_push_data(struct sock *sk, long timeo; if (flags & - ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | - MSG_SPLICE_PAGES)) + ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SPLICE_PAGES)) return -EOPNOTSUPP; if (unlikely(sk->sk_err)) diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index d5ed4d47b16e..b6896126bb92 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -127,7 +127,7 @@ int tls_push_sg(struct sock *sk, { struct bio_vec bvec; struct msghdr msg = { - .msg_flags = MSG_SENDPAGE_NOTLAST | MSG_SPLICE_PAGES | flags, + .msg_flags = MSG_SPLICE_PAGES | flags, }; int ret = 0; struct page *p; diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 9b3aa89a4292..53f944e6d8ef 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -1194,7 +1194,7 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_CMSG_COMPAT | MSG_SPLICE_PAGES | - MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) + MSG_SENDPAGE_NOPOLICY)) return -EOPNOTSUPP; ret = mutex_lock_interruptible(&tls_ctx->tx_lock); diff --git a/tools/perf/trace/beauty/include/linux/socket.h b/tools/perf/trace/beauty/include/linux/socket.h index 13c3a237b9c9..3bef212a24d7 100644 --- a/tools/perf/trace/beauty/include/linux/socket.h +++ b/tools/perf/trace/beauty/include/linux/socket.h @@ -318,7 +318,6 @@ struct ucred { #define MSG_MORE 0x8000 /* Sender will send more */ #define MSG_WAITFORONE 0x10000 /* recvmmsg(): block until 1+ packets avail */ #define MSG_SENDPAGE_NOPOLICY 0x10000 /* sendpage() internal : do no apply policy */ -#define MSG_SENDPAGE_NOTLAST 0x20000 /* sendpage() internal : not the last page */ #define MSG_BATCH 0x40000 /* sendmmsg(): more messages coming */ #define MSG_EOF MSG_FIN #define MSG_NO_SHARED_FRAGS 0x80000 /* sendpage() internal : page frags are not shared */ diff --git a/tools/perf/trace/beauty/msg_flags.c b/tools/perf/trace/beauty/msg_flags.c index ea68db08b8e7..5cdebd7ece7e 100644 --- a/tools/perf/trace/beauty/msg_flags.c +++ b/tools/perf/trace/beauty/msg_flags.c @@ -8,9 +8,6 @@ #ifndef MSG_WAITFORONE #define MSG_WAITFORONE 0x10000 #endif -#ifndef MSG_SENDPAGE_NOTLAST -#define MSG_SENDPAGE_NOTLAST 0x20000 -#endif #ifndef MSG_FASTOPEN #define MSG_FASTOPEN 0x20000000 #endif @@ -50,7 +47,7 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size, P_MSG_FLAG(NOSIGNAL); P_MSG_FLAG(MORE); P_MSG_FLAG(WAITFORONE); - P_MSG_FLAG(SENDPAGE_NOTLAST); + P_MSG_FLAG(SPLICE_PAGES); P_MSG_FLAG(FASTOPEN); P_MSG_FLAG(CMSG_CLOEXEC); #undef P_MSG_FLAG