From patchwork Thu Jul 8 15:26:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12365723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C8DC07E96 for ; Thu, 8 Jul 2021 16:45:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C044061450 for ; Thu, 8 Jul 2021 16:45:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C044061450 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B2F1F6B007D; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADED86B007E; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9590E6B0080; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 66B216B007D for ; Thu, 8 Jul 2021 12:45:41 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AEB261D59E for ; Thu, 8 Jul 2021 16:45:40 +0000 (UTC) X-FDA: 78339996840.17.0807963 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id AD54BB000455 for ; Thu, 8 Jul 2021 16:44:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D6AF361450; Thu, 8 Jul 2021 15:26:29 +0000 (UTC) Subject: [PATCH v3 2/3] SUNRPC: Add svc_rqst_replace_page() API From: Chuck Lever To: linux-nfs@vger.kernel.org, linux-mm@kvack.org Cc: neilb@suse.de Date: Thu, 08 Jul 2021 11:26:29 -0400 Message-ID: <162575798916.2532.6898942885852856593.stgit@klimt.1015granger.net> In-Reply-To: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> References: <162575623717.2532.8517369487503961860.stgit@klimt.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AD54BB000455 Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of "SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org" designates 198.145.29.99 as permitted sender) smtp.mailfrom="SRS0=3XBy=MA=oracle.com=chuck.lever@kernel.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=oracle.com (policy=none) X-Stat-Signature: a915h9t8tmz5fjkkye33zsia9jcry1kp X-HE-Tag: 1625762666-129444 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replacing a page in rq_pages[] requires a get_page(), which is a bus-locked operation, and a put_page(), which can be even more costly. To reduce the cost of replacing a page, batch the put_page() operations by collecting "free" pages in an array, and then passing them to release_pages() when the array becomes full. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc.h | 5 +++++ net/sunrpc/svc.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index e91d51ea028b..68a9f51e2624 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -256,6 +256,9 @@ struct svc_rqst { struct page * *rq_next_page; /* next reply page to use */ struct page * *rq_page_end; /* one past the last page */ + struct page *rq_relpages[16]; + int rq_numrelpages; + struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; @@ -502,6 +505,8 @@ struct svc_rqst *svc_rqst_alloc(struct svc_serv *serv, struct svc_pool *pool, int node); struct svc_rqst *svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node); +void svc_rqst_replace_page(struct svc_rqst *rqstp, + struct page *page); void svc_rqst_free(struct svc_rqst *); void svc_exit_thread(struct svc_rqst *); unsigned int svc_pool_map_get(void); diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 0de918cb3d90..3679830cf123 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -838,6 +839,33 @@ svc_set_num_threads_sync(struct svc_serv *serv, struct svc_pool *pool, int nrser } EXPORT_SYMBOL_GPL(svc_set_num_threads_sync); +static void svc_rqst_release_replaced_pages(struct svc_rqst *rqstp) +{ + release_pages(rqstp->rq_relpages, rqstp->rq_numrelpages); + rqstp->rq_numrelpages = 0; +} + +/** + * svc_rqst_replace_page - Replace one page in rq_pages[] + * @rqstp: svc_rqst with pages to replace + * @page: replacement page + * + * When replacing a page in rq_pages, batch the release of the + * replaced pages to avoid hammering put_page(). + */ +void svc_rqst_replace_page(struct svc_rqst *rqstp, struct page *page) +{ + if (*rqstp->rq_next_page) { + if (rqstp->rq_numrelpages >= ARRAY_SIZE(rqstp->rq_relpages)) + svc_rqst_release_replaced_pages(rqstp); + rqstp->rq_relpages[rqstp->rq_numrelpages++] = *rqstp->rq_next_page; + } + + get_page(page); + *(rqstp->rq_next_page++) = page; +} +EXPORT_SYMBOL_GPL(svc_rqst_replace_page); + /* * Called from a server thread as it's exiting. Caller must hold the "service * mutex" for the service. @@ -846,6 +874,7 @@ void svc_rqst_free(struct svc_rqst *rqstp) { svc_release_buffer(rqstp); + svc_rqst_release_replaced_pages(rqstp); if (rqstp->rq_scratch_page) put_page(rqstp->rq_scratch_page); kfree(rqstp->rq_resp);