From patchwork Thu Mar 31 13:54:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12797120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4E68C433F5 for ; Thu, 31 Mar 2022 14:00:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235107AbiCaOCi (ORCPT ); Thu, 31 Mar 2022 10:02:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234183AbiCaOCh (ORCPT ); Thu, 31 Mar 2022 10:02:37 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6D431706C for ; Thu, 31 Mar 2022 07:00:50 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 19F6461972 for ; Thu, 31 Mar 2022 14:00:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C254C340EE; Thu, 31 Mar 2022 14:00:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648735249; bh=dlXH7egcJq8kRwW0gwMkgwIuzQoc62E7Ck/glxNP4SI=; h=From:To:Cc:Subject:Date:From; b=qXvjYtWe+mUvIMkCqy2IUU4DAYNM3ofTR+A6txMWGeznVomvcPV8mdDbw9JL+8DMi UySOC0UlGsur7j+UTajczwkgqKOPjaVz8V/tw+YV30a6EIR/nM68+vin1CEmyd/NcE /ilYOD671Q7YXQfWB2ymw98hLTDd1cu7Xda5GaTbQGlZqh6oiSlKeslMwCEp0eykya OMgrWwk24i6G1sc5+rsy0iotlvmPGfTvkvrmpwgevES79fKhKX85lQ+hEfKp3YYSQp qbPsmBkwM5/sDa3tOh2qyACqSBtW7pBYrW8YQEOq6qrrOe6IrpL2haxD2wLADE6qFm asP/meA5S1W6A== From: trondmy@kernel.org To: Chuck Lever , Jan Kara Cc: linux-nfs@vger.kernel.org Subject: [PATCH 1/2] nfsd: Fix a write performance regression Date: Thu, 31 Mar 2022 09:54:01 -0400 Message-Id: <20220331135402.7187-1-trondmy@kernel.org> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust The call to filemap_flush() in nfsd_file_put() is there to ensure that we clear out any writes belonging to a NFSv3 client relatively quickly and avoid situations where the file can't be evicted by the garbage collector. It also ensures that we detect write errors quickly. The problem is this causes a regression in performance for some workloads. So try to improve matters by deferring writeback until we're ready to close the file, and need to detect errors so that we can force the client to resend. Tested-by: Jan Kara Fixes: b6669305d35a ("nfsd: Reduce the number of calls to nfsd_file_gc()") Signed-off-by: Trond Myklebust --- fs/nfsd/filecache.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index 8bc807c5fea4..9578a6317709 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -235,6 +235,13 @@ nfsd_file_check_write_error(struct nfsd_file *nf) return filemap_check_wb_err(file->f_mapping, READ_ONCE(file->f_wb_err)); } +static void +nfsd_file_flush(struct nfsd_file *nf) +{ + if (nf->nf_file && vfs_fsync(nf->nf_file, 1) != 0) + nfsd_reset_write_verifier(net_generic(nf->nf_net, nfsd_net_id)); +} + static void nfsd_file_do_unhash(struct nfsd_file *nf) { @@ -302,11 +309,14 @@ nfsd_file_put(struct nfsd_file *nf) return; } - filemap_flush(nf->nf_file->f_mapping); is_hashed = test_bit(NFSD_FILE_HASHED, &nf->nf_flags) != 0; - nfsd_file_put_noref(nf); - if (is_hashed) + if (!is_hashed) { + nfsd_file_flush(nf); + nfsd_file_put_noref(nf); + } else { + nfsd_file_put_noref(nf); nfsd_file_schedule_laundrette(); + } if (atomic_long_read(&nfsd_filecache_count) >= NFSD_FILE_LRU_LIMIT) nfsd_file_gc(); } @@ -327,6 +337,7 @@ nfsd_file_dispose_list(struct list_head *dispose) while(!list_empty(dispose)) { nf = list_first_entry(dispose, struct nfsd_file, nf_lru); list_del(&nf->nf_lru); + nfsd_file_flush(nf); nfsd_file_put_noref(nf); } } @@ -340,6 +351,7 @@ nfsd_file_dispose_list_sync(struct list_head *dispose) while(!list_empty(dispose)) { nf = list_first_entry(dispose, struct nfsd_file, nf_lru); list_del(&nf->nf_lru); + nfsd_file_flush(nf); if (!refcount_dec_and_test(&nf->nf_ref)) continue; if (nfsd_file_free(nf)) From patchwork Thu Mar 31 13:54:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12797121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BF97C433EF for ; Thu, 31 Mar 2022 14:00:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237103AbiCaOCk (ORCPT ); Thu, 31 Mar 2022 10:02:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234183AbiCaOCk (ORCPT ); Thu, 31 Mar 2022 10:02:40 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B056D1706C for ; Thu, 31 Mar 2022 07:00:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 604BEB81FB8 for ; Thu, 31 Mar 2022 14:00:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABEE3C34110; Thu, 31 Mar 2022 14:00:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648735250; bh=XNbcq8qP3AaGocr3u2a6rJ1M5USL02vQjB/QqftKCyk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VVumlVUYHaBWGgPF4VjF5THpGWGzfvVt40VgY9KsIRKEH3PVXtBbbZuRRXz/pHCGF fdDMni1rGncGyazmSHWsOaEekmRyojW6eRMVZ0F5RJeYPivZ2PoiSSw9obSL2IIITW WtU/xQVF17l6uqhcwauaTaxk/v1Md0XsNNUNOr03dRnb+sWq2+gy827PaHxSIakoOU UB5WnNFMdI3t7cZcRs33vjHGW/r8SzASGS2Ac2g5FGcIF8pRCab+vw+11yf+P3SHSZ j+NYjaDKDpjrmBSgAdmuIxeKYdPDP338djXgEzjrjbxRGnipUO34cXzSfrrIoT0IXL P+mDixKuV2s0w== From: trondmy@kernel.org To: Chuck Lever , Jan Kara Cc: linux-nfs@vger.kernel.org Subject: [PATCH 2/2] nfsd: Clean up nfsd_file_put() Date: Thu, 31 Mar 2022 09:54:02 -0400 Message-Id: <20220331135402.7187-2-trondmy@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220331135402.7187-1-trondmy@kernel.org> References: <20220331135402.7187-1-trondmy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Make it a little less racy, by removing the refcount_read() test. Then remove the redundant 'is_hashed' variable. Signed-off-by: Trond Myklebust --- fs/nfsd/filecache.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index 9578a6317709..46714a71ec87 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -301,21 +301,14 @@ nfsd_file_put_noref(struct nfsd_file *nf) void nfsd_file_put(struct nfsd_file *nf) { - bool is_hashed; - set_bit(NFSD_FILE_REFERENCED, &nf->nf_flags); - if (refcount_read(&nf->nf_ref) > 2 || !nf->nf_file) { - nfsd_file_put_noref(nf); - return; - } - - is_hashed = test_bit(NFSD_FILE_HASHED, &nf->nf_flags) != 0; - if (!is_hashed) { + if (test_bit(NFSD_FILE_HASHED, &nf->nf_flags) == 0) { nfsd_file_flush(nf); nfsd_file_put_noref(nf); } else { nfsd_file_put_noref(nf); - nfsd_file_schedule_laundrette(); + if (nf->nf_file) + nfsd_file_schedule_laundrette(); } if (atomic_long_read(&nfsd_filecache_count) >= NFSD_FILE_LRU_LIMIT) nfsd_file_gc();