From patchwork Thu Jul 8 15:26:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12365721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C62BC07E9C for ; Thu, 8 Jul 2021 16:45:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C292E61450 for ; Thu, 8 Jul 2021 16:45:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C292E61450 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AC7826B0075; Thu, 8 Jul 2021 12:45:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A91836B0078; Thu, 8 Jul 2021 12:45:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95A1B6B007B; Thu, 8 Jul 2021 12:45:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 70C056B0075 for ; Thu, 8 Jul 2021 12:45:05 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C074A819B730 for ; Thu, 8 Jul 2021 16:45:04 +0000 (UTC) X-FDA: 78339995328.12.E975AA0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id E2194D0037EF for ; Thu, 8 Jul 2021 16:44:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BC0876142C; Thu, 8 Jul 2021 15:26:23 +0000 (UTC) Subject: [PATCH v3 1/3] NFSD: Clean up splice actor From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-mm@kvack.org Cc: neilb@suse.de Date: Thu, 08 Jul 2021 11:26:23 -0400 Message-ID: <162575798303.2532.14679105103126974177.stgit@klimt.1015granger.net> In-Reply-To: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> References: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=oracle.com (policy=none); spf=pass (imf21.hostedemail.com: domain of "SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" designates 198.145.29.99 as permitted sender) smtp.mailfrom="SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" X-Stat-Signature: tyk9kqww6sbg3p4gma8wkuzykr3d4jom X-Rspamd-Queue-Id: E2194D0037EF X-Rspamd-Server: rspam01 X-HE-Tag: 1625762665-233306 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000175, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A few useful observations: - The value in @size is never modified. - splice_desc.len is an unsigned int, and so is xdr_buf.page_len. An implicit cast to size_t is unnecessary. - The computation of .page_len is the same in all three arms of the "if" statement, so hoist it out to make it clear that the operation is an unconditional invariant. The resulting function is 18 bytes shorter on my system (-Os). Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 15adf1f6ab21..da5340dc0203 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -847,26 +847,21 @@ nfsd_splice_actor(struct pipe_inode_info *pipe, struct pipe_buffer *buf, struct svc_rqst *rqstp = sd->u.data; struct page **pp = rqstp->rq_next_page; struct page *page = buf->page; - size_t size; - - size = sd->len; if (rqstp->rq_res.page_len == 0) { get_page(page); put_page(*rqstp->rq_next_page); *(rqstp->rq_next_page++) = page; rqstp->rq_res.page_base = buf->offset; - rqstp->rq_res.page_len = size; } else if (page != pp[-1]) { get_page(page); if (*rqstp->rq_next_page) put_page(*rqstp->rq_next_page); *(rqstp->rq_next_page++) = page; - rqstp->rq_res.page_len += size; - } else - rqstp->rq_res.page_len += size; + } + rqstp->rq_res.page_len += sd->len; - return size; + return sd->len; } static int nfsd_direct_splice_actor(struct pipe_inode_info *pipe, From patchwork Thu Jul 8 15:26:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12365723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C8DC07E96 for ; Thu, 8 Jul 2021 16:45:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C044061450 for ; Thu, 8 Jul 2021 16:45:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C044061450 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B2F1F6B007D; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADED86B007E; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9590E6B0080; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 66B216B007D for ; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AEB261D59E for ; Thu, 8 Jul 2021 16:45:40 +0000 (UTC) X-FDA: 78339996840.17.0807963 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id AD54BB000455 for ; Thu, 8 Jul 2021 16:44:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D6AF361450; Thu, 8 Jul 2021 15:26:29 +0000 (UTC) Subject: [PATCH v3 2/3] SUNRPC: Add svc_rqst_replace_page() API From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-mm@kvack.org Cc: neilb@suse.de Date: Thu, 08 Jul 2021 11:26:29 -0400 Message-ID: <162575798916.2532.6898942885852856593.stgit@klimt.1015granger.net> In-Reply-To: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> References: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AD54BB000455 Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of "SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" designates 198.145.29.99 as permitted sender) smtp.mailfrom="SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=oracle.com (policy=none) X-Stat-Signature: a915h9t8tmz5fjkkye33zsia9jcry1kp X-HE-Tag: 1625762666-129444 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replacing a page in rq_pages[] requires a get_page(), which is a bus-locked operation, and a put_page(), which can be even more costly. To reduce the cost of replacing a page, batch the put_page() operations by collecting "free" pages in an array, and then passing them to release_pages() when the array becomes full. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc.h | 5 +++++ net/sunrpc/svc.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index e91d51ea028b..68a9f51e2624 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -256,6 +256,9 @@ struct svc_rqst { struct page * *rq_next_page; /* next reply page to use */ struct page * *rq_page_end; /* one past the last page */ + struct page *rq_relpages[16]; + int rq_numrelpages; + struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; @@ -502,6 +505,8 @@ struct svc_rqst *svc_rqst_alloc(struct svc_serv *serv, struct svc_pool *pool, int node); struct svc_rqst *svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node); +void svc_rqst_replace_page(struct svc_rqst *rqstp, + struct page *page); void svc_rqst_free(struct svc_rqst *); void svc_exit_thread(struct svc_rqst *); unsigned int svc_pool_map_get(void); diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 0de918cb3d90..3679830cf123 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -838,6 +839,33 @@ svc_set_num_threads_sync(struct svc_serv *serv, struct svc_pool *pool, int nrser } EXPORT_SYMBOL_GPL(svc_set_num_threads_sync); +static void svc_rqst_release_replaced_pages(struct svc_rqst *rqstp) +{ + release_pages(rqstp->rq_relpages, rqstp->rq_numrelpages); + rqstp->rq_numrelpages = 0; +} + +/** + * svc_rqst_replace_page - Replace one page in rq_pages[] + * @rqstp: svc_rqst with pages to replace + * @page: replacement page + * + * When replacing a page in rq_pages, batch the release of the + * replaced pages to avoid hammering put_page(). + */ +void svc_rqst_replace_page(struct svc_rqst *rqstp, struct page *page) +{ + if (*rqstp->rq_next_page) { + if (rqstp->rq_numrelpages >= ARRAY_SIZE(rqstp->rq_relpages)) + svc_rqst_release_replaced_pages(rqstp); + rqstp->rq_relpages[rqstp->rq_numrelpages++] = *rqstp->rq_next_page; + } + + get_page(page); + *(rqstp->rq_next_page++) = page; +} +EXPORT_SYMBOL_GPL(svc_rqst_replace_page); + /* * Called from a server thread as it's exiting. Caller must hold the "service * mutex" for the service. @@ -846,6 +874,7 @@ void svc_rqst_free(struct svc_rqst *rqstp) { svc_release_buffer(rqstp); + svc_rqst_release_replaced_pages(rqstp); if (rqstp->rq_scratch_page) put_page(rqstp->rq_scratch_page); kfree(rqstp->rq_resp); From patchwork Thu Jul 8 15:26:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12365719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC388C07E9E for ; Thu, 8 Jul 2021 16:44:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8283B61464 for ; Thu, 8 Jul 2021 16:44:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8283B61464 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 516226B0073; Thu, 8 Jul 2021 12:44:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C5B46B0074; Thu, 8 Jul 2021 12:44:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38E136B0075; Thu, 8 Jul 2021 12:44:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 1467E6B0073 for ; Thu, 8 Jul 2021 12:44:33 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5292029DFC for ; Thu, 8 Jul 2021 16:44:32 +0000 (UTC) X-FDA: 78339993984.17.B9F2323 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id E3CFDB000143 for ; Thu, 8 Jul 2021 16:44:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F3F1161459; Thu, 8 Jul 2021 15:26:35 +0000 (UTC) Subject: [PATCH v3 3/3] NFSD: Batch release pages during splice read From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-mm@kvack.org Cc: neilb@suse.de Date: Thu, 08 Jul 2021 11:26:35 -0400 Message-ID: <162575799527.2532.9850430992854215546.stgit@klimt.1015granger.net> In-Reply-To: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> References: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=oracle.com (policy=none); spf=pass (imf24.hostedemail.com: domain of "SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" designates 198.145.29.99 as permitted sender) smtp.mailfrom="SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" X-Stat-Signature: b9hbewuhi9mb3xdcqjoohqarqnmoqp3q X-Rspamd-Queue-Id: E3CFDB000143 X-Rspamd-Server: rspam01 X-HE-Tag: 1625762665-964292 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Large splice reads call put_page() repeatedly. put_page() is relatively expensive to call, so replace it with the new svc_rqst_replace_page() helper to help amortize that cost. Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index da5340dc0203..43ac23b200d2 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -849,15 +849,10 @@ nfsd_splice_actor(struct pipe_inode_info *pipe, struct pipe_buffer *buf, struct page *page = buf->page; if (rqstp->rq_res.page_len == 0) { - get_page(page); - put_page(*rqstp->rq_next_page); - *(rqstp->rq_next_page++) = page; + svc_rqst_replace_page(rqstp, page); rqstp->rq_res.page_base = buf->offset; } else if (page != pp[-1]) { - get_page(page); - if (*rqstp->rq_next_page) - put_page(*rqstp->rq_next_page); - *(rqstp->rq_next_page++) = page; + svc_rqst_replace_page(rqstp, page); } rqstp->rq_res.page_len += sd->len;