From patchwork Thu May 18 09:56:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13246351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF10DC77B7D for ; Thu, 18 May 2023 09:56:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 374DD900004; Thu, 18 May 2023 05:56:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3250C900003; Thu, 18 May 2023 05:56:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C484900004; Thu, 18 May 2023 05:56:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 052E8900003 for ; Thu, 18 May 2023 05:56:42 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A8C681C71EE for ; Thu, 18 May 2023 09:56:41 +0000 (UTC) X-FDA: 80802921402.28.A87BBDE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id D05C0C0005 for ; Thu, 18 May 2023 09:56:39 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X2gFKhXv; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684403799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dazR50SKXuYIFVWThl0nrv2fyFSamN4MkmMLPefBMF0=; b=WoV9tIfZorsq9sjD6XpoLpDKNUwd2V2nJhw3pNAVhf0V/JZKVFFCZ8D/rZRpQhwyZqr2aY iot2BJOWmhXK6MqwK5C/q3KMaSGP34IKytzjuEWYelHafGUl3DQkkwEe1pbyvwSxgK41ry 54TjuOjnkR8KPw6Fcucv1yefeJC1nTc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X2gFKhXv; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684403799; a=rsa-sha256; cv=none; b=nxCdD4goizj/13hnNmBiLn5HqoLvnFD4++UrNGOHcYZRuoG4hP3xAIuk9D5+bgxDd7An/h D1L97mxTM0XFd8u/mk7eq0GeixE6o6KmsovohxdbuCAz6/bqGawCyxOSP3oC8fDApgjnYP w0W5lCaRiJRAsMAN0CiyW9P/236kdzQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684403798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dazR50SKXuYIFVWThl0nrv2fyFSamN4MkmMLPefBMF0=; b=X2gFKhXvosWcp8YL+9X5pR3n1H15RazWzM2IdeiWEReqij5QJEn/CHPRxXXz5jcz+DFLIs 1gtofvMVBUW7Uj8vmNlf1x8LcqOZNSwjaVOEOWpi7FYArIGNxV70iQ2kNp8t+1++xNgWRT eNwWy59ICcGTc0A+ABNiT0L1o5wsIW8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-352-sNI4QCQWN4qCpwaF8aO8pg-1; Thu, 18 May 2023 05:56:35 -0400 X-MC-Unique: sNI4QCQWN4qCpwaF8aO8pg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 07051800047; Thu, 18 May 2023 09:56:34 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.221]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66B3640C2063; Thu, 18 May 2023 09:56:31 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells In-Reply-To: <20230515093345.396978-4-dhowells@redhat.com> References: <20230515093345.396978-4-dhowells@redhat.com> <20230515093345.396978-1-dhowells@redhat.com> To: Paolo Abeni Cc: dhowells@redhat.com, netdev@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Willem de Bruijn , David Ahern , Matthew Wilcox , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH net-next v8 03/16] net: Add a function to splice pages into an skbuff for MSG_SPLICE_PAGES MIME-Version: 1.0 Content-ID: <1347366.1684403790.1@warthog.procyon.org.uk> Date: Thu, 18 May 2023 10:56:30 +0100 Message-ID: <1347367.1684403790@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Stat-Signature: 53r94nzpk1n7qqeaj57mkpfnt9bghcw7 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D05C0C0005 X-Rspam-User: X-HE-Tag: 1684403799-101168 X-HE-Meta: U2FsdGVkX1/eSXMkYjtVRucVUdc8WZFJbxI+IFfcrUCrKyVFOpyvxkWoDD9nrAqynRSOaTprfsk5Mv01ioosMXw2f/SBt+CWVonDxo+D8NK06i+gcYLWdHMWDIxvVCezdeo88m8+PMzXINdayAuZP/yyuznc5jsu9/5zniDmwICWe6bO5IRGWn6EzMsu5jvMBAOFvDFF5Fkhf9Ye1/GiB8XidYNnuy4WMX+JRRPB8RU6Clk+kumabaiNPH4NwHV0Vp6HMzXxdWPM2Hlo3JZmp8m6wLKsppKeAQ7dm4Tzg3sqvoSvfG8Bv3nYrnlLHLF+McHjRhJoQto/penMxnzAKqWGXlhC20jRY9wewLPNPcMIoiVvLlnylV+aSjHKyunNsF1nexItG5BOk23afGa8NggiRAajBKcu8DiOEcpfeR1rOp6JERMV9nr27CwyDE0EKqV3c6RsvlEWVfNu+shbBlxo4Y+I9ek73HFLmD/0WkgExRtv7jSLJno9g22TFIVtdwS3z00IASSP9YqxZZjik0MuC2gdKEAbFVN0g/y6bHJdKIelW+qLJWOGSa8+8wemBjpUkTI0mVmguyTl8+VhZ3pknW+n2kL2VEkX8TkRuVVyf0z1+PN6BU4hhKVhq8jfCUqbYrWI37sArrQzpjNQgr9Z/RRMnscMTiJUEZuZ72m5g/YLP/vDJh0R8F7WPsPdQZxy71ASC/te5dfjGdYUI37/LwrjbloyKeDAH8hPsGuVpWKeBp8c1PV9ueOXdP5qsu5/Z+sr9BFUg6kKseWfc3t2tTyJKqCuJkDVR5HWrnsx9wa9e8QQoavXO78D/yLJeLpBWO7tl+vEJLc5z6qQVHFmP+P1W5+DSutw+mfoA6y3Qq0PMn9PV7Kiaeqlr1z/r6lWdzOXZfFoi/mdDfWsTsK+Klz+o/8seELwNohMlOggyUINdbX7fw4A0+TxAtk3pJbeTKFcUUZyGGN/txi d/TAuByd bo6kvVWM2bCpaQwRJhd7oUWzLyclK/Y2QRrG5JzNpItdeheV5CiG5JX+lQ60dl4k+SI8EIFFWPaJYapXmOyHgv3yLMPlJZoDfM9FmcN2qEPqGVaeIQU+DKj8ElHR6hD6CVVLbzGwDCJGdRTDiGoD+iVp62qEn69FLlnHSJO88/kbmqrqb+fX0LhojUEomTJ1UjJT4F2d2Kyu/sgtW+UIyFDR8qanuiiY16YFKdqDQrNZE3qWn2qayhAr6W+mHCa22SgXp2NT06IONrjieKmiNuA1zRKjLLDcAevMJkE2gm2Q2B/P4kGcSxvvGzicKI8YYXsAIPUhsZ9d7Wct+HxtU7YRaxVkubNYeO3Utj0e1m3Y6YniPdlvvVBCJfocew55ZtKmbtQ7g9+6nm6MKVnzYZ03rUvggZHvpLHhuv6vh6QI+PktWUrqded7wt7ib8Gw6Uro3xqMa0L0N3G6k/KX/gMudYqUh/Z+50PUrwsH8BUNOd4T805ujmWMjPUWTKXljjPkI5/GmLhFDqXUpAnlA5u01Fcv5X6RvbMfIevbTvZGWGWGjlc7lMUIkuQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a function to handle MSG_SPLICE_PAGES being passed internally to sendmsg(). Pages are spliced into the given socket buffer if possible and copied in if not (e.g. they're slab pages or have a zero refcount). Signed-off-by: David Howells cc: Eric Dumazet cc: "David S. Miller" cc: David Ahern cc: Jakub Kicinski cc: Paolo Abeni cc: Al Viro cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #8) - Order local variables in reverse xmas tree order. - Remove duplicate coalescence check. - Warn if sendpage_ok() fails. ver #7) - Export function. - Never copy data, return -EIO if sendpage_ok() returns false. include/linux/skbuff.h | 3 + net/core/skbuff.c | 88 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 4c0ad48e38ca..1c5f0ac6f8c3 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5097,5 +5097,8 @@ static inline void skb_mark_for_recycle(struct sk_buff *skb) #endif } +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp); + #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 7f53dcb26ad3..f4a5b51aed22 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6892,3 +6892,91 @@ nodefer: __kfree_skb(skb); if (unlikely(kick) && !cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) smp_call_function_single_async(cpu, &sd->defer_csd); } + +static void skb_splice_csum_page(struct sk_buff *skb, struct page *page, + size_t offset, size_t len) +{ + const char *kaddr; + __wsum csum; + + kaddr = kmap_local_page(page); + csum = csum_partial(kaddr + offset, len, 0); + kunmap_local(kaddr); + skb->csum = csum_block_add(skb->csum, csum, skb->len); +} + +/** + * skb_splice_from_iter - Splice (or copy) pages to skbuff + * @skb: The buffer to add pages to + * @iter: Iterator representing the pages to be added + * @maxsize: Maximum amount of pages to be added + * @gfp: Allocation flags + * + * This is a common helper function for supporting MSG_SPLICE_PAGES. It + * extracts pages from an iterator and adds them to the socket buffer if + * possible, copying them to fragments if not possible (such as if they're slab + * pages). + * + * Returns the amount of data spliced/copied or -EMSGSIZE if there's + * insufficient space in the buffer to transfer anything. + */ +ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, + ssize_t maxsize, gfp_t gfp) +{ + size_t frag_limit = READ_ONCE(sysctl_max_skb_frags); + struct page *pages[8], **ppages = pages; + ssize_t spliced = 0, ret = 0; + unsigned int i; + + while (iter->count > 0) { + ssize_t space, nr; + size_t off, len; + + ret = -EMSGSIZE; + space = frag_limit - skb_shinfo(skb)->nr_frags; + if (space < 0) + break; + + /* We might be able to coalesce without increasing nr_frags */ + nr = clamp_t(size_t, space, 1, ARRAY_SIZE(pages)); + + len = iov_iter_extract_pages(iter, &ppages, maxsize, nr, 0, &off); + if (len <= 0) { + ret = len ?: -EIO; + break; + } + + i = 0; + do { + struct page *page = pages[i++]; + size_t part = min_t(size_t, PAGE_SIZE - off, len); + + ret = -EIO; + if (WARN_ON_ONCE(!sendpage_ok(page))) + goto out; + + ret = skb_append_pagefrags(skb, page, off, part, + frag_limit); + if (ret < 0) { + iov_iter_revert(iter, len); + goto out; + } + + if (skb->ip_summed == CHECKSUM_NONE) + skb_splice_csum_page(skb, page, off, part); + + off = 0; + spliced += part; + maxsize -= part; + len -= part; + } while (len > 0); + + if (maxsize <= 0) + break; + } + +out: + skb_len_add(skb, spliced); + return spliced ?: ret; +} +EXPORT_SYMBOL(skb_splice_from_iter);