From patchwork Fri Mar 31 16:08:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13196195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E154CC76196 for ; Fri, 31 Mar 2023 16:11:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7285A6B00A3; Fri, 31 Mar 2023 12:11:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D77C6B00A5; Fri, 31 Mar 2023 12:11:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 552A36B00A6; Fri, 31 Mar 2023 12:11:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4260E6B00A3 for ; Fri, 31 Mar 2023 12:11:06 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 10273C107C for ; Fri, 31 Mar 2023 16:11:06 +0000 (UTC) X-FDA: 80629682532.21.3DD4119 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 04359120027 for ; Fri, 31 Mar 2023 16:11:02 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Zi/ZhrXP"; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680279063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jPEZCXpX5VIblKmRhP3z5vIQLhv4QpeLvADltfz6mNU=; b=J+AHjJBmPRp30JSw8U5Z5nCDG3H5KvuxY/gWB271D1Q1FAUJIm8nNwnUeLpdzhbWEf1Xcc t+h9giiwKsdCH/dZxIBuzwByjvxxsM+e9zql136VuDgLv980Psv89F3bNoAlfN+hejCcdY B9z2XOOJBJB8tdCMJ+rs2gIAzmLyBJA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Zi/ZhrXP"; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680279063; a=rsa-sha256; cv=none; b=KarfhjqM0XoQWonq9OMDMIYx9Hz+xnjFYQySyb8pvvGfg2gD//k6AslL7ugfBx7z9855fr GoEJZoBKffbKhO3OP7zmgZVHD+1Zrf4tgpoqEpFNYNG2oUWiv8oWe1OeKPCB1iiQ47s1wU puQsD/s7RSi6bU1TGT95vU7w+YQ5srs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680279062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jPEZCXpX5VIblKmRhP3z5vIQLhv4QpeLvADltfz6mNU=; b=Zi/ZhrXP4LU5gQln5jrLIxrBir4FDptykfPL0BitacG7C5ACbwBT6DsVzj2EkIPccwqYIp xntMlW6aPfmW5odwIUFv3CQNt6EX7+hCIKIqWEAHtk1h5RoK4axpvRj2AUUYaFIvCgrH8C LY7Gdo0fJQk/2IN5sjLm67NUZdhPnCU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-650-1RKJZrZKPJSsBYeQeItUoQ-1; Fri, 31 Mar 2023 12:10:56 -0400 X-MC-Unique: 1RKJZrZKPJSsBYeQeItUoQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 504CB801206; Fri, 31 Mar 2023 16:10:55 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D9F71121314; Fri, 31 Mar 2023 16:10:53 +0000 (UTC) From: David Howells To: Matthew Wilcox , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: David Howells , Al Viro , Christoph Hellwig , Jens Axboe , Jeff Layton , Christian Brauner , Chuck Lever III , Linus Torvalds , netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Tom Herbert , Tom Herbert Subject: [PATCH v3 34/55] kcm: Convert kcm_sendpage() to use MSG_SPLICE_PAGES Date: Fri, 31 Mar 2023 17:08:53 +0100 Message-Id: <20230331160914.1608208-35-dhowells@redhat.com> In-Reply-To: <20230331160914.1608208-1-dhowells@redhat.com> References: <20230331160914.1608208-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 04359120027 X-Stat-Signature: 8yqywojb4r38xyfw557txxqdiui4gxw6 X-HE-Tag: 1680279062-983595 X-HE-Meta: U2FsdGVkX1+gjH4Upc1gwky6HCyepyPU41LithasUaVN2SQzW3LMwGQFzKHSxuYZriALHvODNFBTJzoTUAUS1VAv9IIi1SmN3KXf+Zyn1fSRXYtjmK0ZtsBQQVtrE25Zs00TZxW/QaP3II7oETCLHP/QzgrMdr/UWjKkKdMIioccGpqbJzu/j767n2nEFt5R9fnkqR75B8YhqXHTn1UjYpaspLWa9gDFCLFbcKRMR0HjXVcBGk+pew3fI5SiR7cfpsAgLnK8EJZvYAtQAJPwo5AlnFC0uMg25WRW9l5o9b+VNt6/+Zk+SdFsTqZmwXtMDxfmEDuKPqdDtqbQd9FHHUDfVEnUNoj+J5qpF3A6bysFI6MeR3/4mILt0a1pKCuBcXcFBdEC2G2cyZ9AAjyfw1RPbfdGVXRxG4PT113YdAeTjcratPCUNg9rfgQgoAHl5oSfnOEf5LPyPdNdPV5hjmVCdUpq13qiRjakh3Xflz/4Iir7RPf7PceZ3OuK684zDGoZ6hvRunepLr3oAGQVp9W1xIa2wtOMhZS+qrdm/2qJfkd9yW/tYECVEO3CUJ57HyTKgd3TBKs3FJx1csFggg9gmfFBUOvFytl9yr3XAz2P45yqi873dnZDIXlo5izuz/Mba3b/s+rMzBv+uynpseb2nqf1lduKY+lm54vurmHNjgb/hMVzYexKXeeMY3eT3DdR/xMWN9aPg/sF6PtGssYqcO2GZ0idhKDHpJtfl+BhGI6JDGsTJWNe+giRh9avC6XrgLJeh6gZHvhDO0RwA9kjxDAMcF86DEen3JlKE1n8mgSeIFieeOgHF0/naFyjv3WjwBRV2CAe9brJH9wh3Zez/c5u9/QYIVNxiQy84cejQdQU9B7oXLOC43iyKkjSYWoyZVpugmzCrxTe0OTZEYrx9ydAxJzknrQ3u95jASZ8HQMvAKGW0wYLk6elpTvTXatBYiHvT21cFfbEi8Z qz8agYZx 7tF4MFm9PRQ6/z9bHybT9Sjwqx8Pqayk4Lz8y22Y/7z/ub+6RFVBYmz5QjERnlNsqxVJyB7XJzxC5ie9dl2qvr2GXvHFJbGgPUBfUbMhqq4oeW4/o/kgd0Y5o3P7RRtkLfJksyl7tchwXM8xW0C4SyZ57YKceX+QFxAX4vG3LMHy7cvUvDZTIyeshTkHhEV5KfBdlByZC3oaKdOILWTnqlzD7az4mfz2SF5F1tQMyk+PiDINBI/+N6I+IUzhi6pnQhzACqZKYelFY2CP0ACm2N8Zn5vWPDPTzWeQ4qgaGU0yOhI/0nydygW7K0Q+YFBLIl9cZZ1/D3qBHeqlamq22Yx61PxW7UI8ZK2CBqSWSOC6ViP0n6Bo3BNyTXlTz/hF0W7JE1Z8qfD8sbjz9dc1TyRN+uxUopARuvSEFoW3KAzLnebPASeHRpEAw+hz+PhSGA4psDZVXCG6AJ1gTPfCxzH9SfaGrlAHIn/PmtVH7zZ6n3vKCPjCltRlsOJw9o2HRluZLf4/gTWYqXXKjlxahn+hKfARUhSecIMNOBsFOVeBs8uqPtr8CFexwdg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert kcm_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Tom Herbert cc: Tom Herbert cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/kcm/kcmsock.c | 161 ++++++---------------------------------------- 1 file changed, 18 insertions(+), 143 deletions(-) diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index 0a3f79d81595..d77d28fbf389 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -761,149 +761,6 @@ static void kcm_push(struct kcm_sock *kcm) kcm_write_msgs(kcm); } -static ssize_t kcm_sendpage(struct socket *sock, struct page *page, - int offset, size_t size, int flags) - -{ - struct sock *sk = sock->sk; - struct kcm_sock *kcm = kcm_sk(sk); - struct sk_buff *skb = NULL, *head = NULL; - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - bool eor; - int err = 0; - int i; - - if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - /* No MSG_EOR from splice, only look at MSG_MORE */ - eor = !(flags & MSG_MORE); - - lock_sock(sk); - - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); - - err = -EPIPE; - if (sk->sk_err) - goto out_error; - - if (kcm->seq_skb) { - /* Previously opened message */ - head = kcm->seq_skb; - skb = kcm_tx_msg(head)->last_skb; - i = skb_shinfo(skb)->nr_frags; - - if (skb_can_coalesce(skb, i, page, offset)) { - skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], size); - skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; - goto coalesced; - } - - if (i >= MAX_SKB_FRAGS) { - struct sk_buff *tskb; - - tskb = alloc_skb(0, sk->sk_allocation); - while (!tskb) { - kcm_push(kcm); - err = sk_stream_wait_memory(sk, &timeo); - if (err) - goto out_error; - } - - if (head == skb) - skb_shinfo(head)->frag_list = tskb; - else - skb->next = tskb; - - skb = tskb; - skb->ip_summed = CHECKSUM_UNNECESSARY; - i = 0; - } - } else { - /* Call the sk_stream functions to manage the sndbuf mem. */ - if (!sk_stream_memory_free(sk)) { - kcm_push(kcm); - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); - err = sk_stream_wait_memory(sk, &timeo); - if (err) - goto out_error; - } - - head = alloc_skb(0, sk->sk_allocation); - while (!head) { - kcm_push(kcm); - err = sk_stream_wait_memory(sk, &timeo); - if (err) - goto out_error; - } - - skb = head; - i = 0; - } - - get_page(page); - skb_fill_page_desc_noacc(skb, i, page, offset, size); - skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; - -coalesced: - skb->len += size; - skb->data_len += size; - skb->truesize += size; - sk->sk_wmem_queued += size; - sk_mem_charge(sk, size); - - if (head != skb) { - head->len += size; - head->data_len += size; - head->truesize += size; - } - - if (eor) { - bool not_busy = skb_queue_empty(&sk->sk_write_queue); - - /* Message complete, queue it on send buffer */ - __skb_queue_tail(&sk->sk_write_queue, head); - kcm->seq_skb = NULL; - KCM_STATS_INCR(kcm->stats.tx_msgs); - - if (flags & MSG_BATCH) { - kcm->tx_wait_more = true; - } else if (kcm->tx_wait_more || not_busy) { - err = kcm_write_msgs(kcm); - if (err < 0) { - /* We got a hard error in write_msgs but have - * already queued this message. Report an error - * in the socket, but don't affect return value - * from sendmsg - */ - pr_warn("KCM: Hard failure on kcm_write_msgs\n"); - report_csk_error(&kcm->sk, -err); - } - } - } else { - /* Message not complete, save state */ - kcm->seq_skb = head; - kcm_tx_msg(head)->last_skb = skb; - } - - KCM_STATS_ADD(kcm->stats.tx_bytes, size); - - release_sock(sk); - return size; - -out_error: - kcm_push(kcm); - - err = sk_stream_error(sk, flags, err); - - /* make sure we wake any epoll edge trigger waiter */ - if (unlikely(skb_queue_len(&sk->sk_write_queue) == 0 && err == -EAGAIN)) - sk->sk_write_space(sk); - - release_sock(sk); - return err; -} - static int kcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) { struct sock *sk = sock->sk; @@ -1143,6 +1000,24 @@ static int kcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) return err; } +static ssize_t kcm_sendpage(struct socket *sock, struct page *page, + int offset, size_t size, int flags) + +{ + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; + + if (flags & MSG_OOB) + return -EOPNOTSUPP; + + bvec_set_page(&bvec, page, offset, size); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return kcm_sendmsg(sock, &msg, size); +} + static int kcm_recvmsg(struct socket *sock, struct msghdr *msg, size_t len, int flags) {