From patchwork Sun Jan 2 17:35:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC7E0C433F5 for ; Sun, 2 Jan 2022 17:35:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229960AbiABRfS (ORCPT ); Sun, 2 Jan 2022 12:35:18 -0500 Received: from dfw.source.kernel.org ([139.178.84.217]:55270 "EHLO dfw.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRfR (ORCPT ); Sun, 2 Jan 2022 12:35:17 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9AF4060F4A for ; Sun, 2 Jan 2022 17:35:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5009C36AEB; Sun, 2 Jan 2022 17:35:16 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 01/10] NFSD: Fix verifier returned in stable WRITEs Date: Sun, 2 Jan 2022 12:35:15 -0500 Message-Id: <164114491573.7344.10290554606821879947.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2498; h=from:subject:message-id; bh=zto3mTzBTargRt592GzXFrBkR2B5JL8M0166wefXnxw=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJTQm0Cd+4BUlsv7QLbMBy5AZ/LP+Hmzn3QO3Xz XXlnZBGJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiUwAKCRAzarMzb2Z/lwQ6D/ 9BXdTjHIIgrTU3v7t3JHynrJADqmNni0LvSBTCvQBQ1UySZnqYjxuS1pEbgnLxXavywdPiybAIm2q6 5JMRfU5p0xyOfogpg+TBNYE60x36g5H6xSepWHb8NhHuA4lpl6/eFQq3ZaDpADE/I45gd+FUqpm8RW M4pRWKGN1C5Mh8yBIoPL61PY+tqReyLS0x1TeQw9FHqElmH8nFjAsd1rEzfp27xNFHO9f5bAdvYYHg PJQgcnUmcRvxGSihJmWQYx129A44+3QZI5UEcoNhdon9NVE9XnWuWJ7b5r0a/r3lkIvb5aoCvfB3K2 t6k92OSFyahSv685QKzuMPrThCwyRFfQJdsDGpNGTyTqnLrQtKipUe4aMdaFkOKDZKspE+FkKnESYM pTTxL73OA4mzGyBEBBg5Ev26mfiuSZRfOFiOPqJh9t2abAALXEy+yLXXKSgT4zHxaep6CKUMwnS/mY vIOGCJMy90A6Hb91yoMYxQu3aAKBjyUBWNYBs3IdY2IClhIsSKKec515+6EXm7NRyGCiwppju2HSDA XYK8IUb7xTdzr/vEGIg8IRjDRwWYyXj/TjH+SthkAJI+IINIMX/yjSFAEAbRruLH4iiF2LdbZlSMmx g79pA2HYoipYk7/bjZgy+6UWz+k0G8G8FiiC3E2lK3mpmJls+owqGiJWvj6A== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org RFC 8881 explains the purpose of the write verifier this way: > The final portion of the result is the field writeverf. This field > is the write verifier and is a cookie that the client can use to > determine whether a server has changed instance state (e.g., server > restart) between a call to WRITE and a subsequent call to either > WRITE or COMMIT. But then it says: > This cookie MUST be unchanged during a single instance of the > NFSv4.1 server and MUST be unique between instances of the NFSv4.1 > server. If the cookie changes, then the client MUST assume that > any data written with an UNSTABLE4 value for committed and an old > writeverf in the reply has been lost and will need to be > recovered. RFC 1813 has similar language for NFSv3. NFSv2 does not have a write verifier since it doesn't implement the COMMIT procedure. Since commit 19e0663ff9bc ("nfsd: Ensure sampling of the write verifier is atomic with the write"), the Linux NFS server has returned a boot-time-based verifier for UNSTABLE WRITEs, but a zero verifier for FILE_SYNC and DATA_SYNC WRITEs. FILE_SYNC and DATA_SYNC WRITEs are not followed up with a COMMIT, so there's no need for clients to compare verifiers for stable writes. However, by returning a different verifier for stable and unstable writes, the above commit puts the Linux NFS server a step farther out of compliance with the first MUST above. At least one NFS client (FreeBSD) noticed the difference, making this a potential regression. Reported-by: Rick Macklem Link: https://lore.kernel.org/linux-nfs/YQXPR0101MB096857EEACF04A6DF1FC6D9BDD749@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM/T/ Fixes: 19e0663ff9bc ("nfsd: Ensure sampling of the write verifier is atomic with the write") Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 0faa3839ea6c..74c3451c2089 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -995,6 +995,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, iov_iter_kvec(&iter, WRITE, vec, vlen, *cnt); if (flags & RWF_SYNC) { down_write(&nf->nf_rwsem); + if (verf) + nfsd_copy_boot_verifier(verf, + net_generic(SVC_NET(rqstp), + nfsd_net_id)); host_err = vfs_iter_write(file, &iter, &pos, flags); if (host_err < 0) nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), From patchwork Sun Jan 2 17:35:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702342 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD4D8C433F5 for ; Sun, 2 Jan 2022 17:35:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229743AbiABRf0 (ORCPT ); Sun, 2 Jan 2022 12:35:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRf0 (ORCPT ); Sun, 2 Jan 2022 12:35:26 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA21FC061761 for ; Sun, 2 Jan 2022 09:35:25 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 528F060F02 for ; Sun, 2 Jan 2022 17:35:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8125BC36AEB; Sun, 2 Jan 2022 17:35:23 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 02/10] nfsd: Replace use of rwsem with errseq_t Date: Sun, 2 Jan 2022 12:35:22 -0500 Message-Id: <164114492227.7344.9180022567264852304.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7369; i=chuck.lever@oracle.com; h=from:subject:message-id; bh=OXfAhAT0HTN08HfKlN681NllvsiaFOoI2rc67pLjotw=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJarbwx+WJDhGj+M8Lm61t4DviZss8cEl/Baib1 g2OLeZOJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiWgAKCRAzarMzb2Z/l57BEA CRMcmJCudEHgdd4UpAyKSModmyarZ3q5+gqvdK/WT8y/34wh4+XmPeveGc6upIoeLkyEm9VskiIcxu axKSoyb3+qjA+6IsysgYbdwxRCTrSvN+7kg15ltlMwnzXyA2kWkUevJnKhvEEgAhW4J2Gw0aLgEOdu I4iwz6zrkZ9XAzQmOyd7oqmWqF+F7akYhSFLZH+ZEQGhmBOfkHVpMefkONrod+qX2gVIY9Ms1sJ7wV K3wmyQaT2/4C4iEV5CCDo1q7p4oOqFWYKUPwycHgsFM1k17xglpPwE8WCm24lfJuVHgl1156GY1b6J nM1S3hXRBaH+O7jMYUg8qy7b6jEt5NNnJJbjjdvK9WxctExLPnjyja6RRJFKfsgGPw1Su+hQ/+qe/p fYXKfN30YqRfJMcn+hRgHE/2ujT0VjOtpuGNF0S4YtFntiOrgzePQGvHCeSMvR2Vbx5q+vnHKS7+h9 G7TrNPk9jP9AQIxxtWBmPVR4WLM7sh3LK5FZPCVDyY2ZDEbkcy9KFkvtiQ0w7B1Or1E3PRvnMrQVxi RMHVGpEiHHUdr1CZdxnl4zWbsJF5Bt6O+VtXbAURyvUs7Gp4t+G3sgnAnkPCOCLzYYMxaP40+1j6Ym 65iOqlocxAjbLNGFo5j6TxaL/lha13jPHHjUAIWpS9Mzg3odYIl5 NtNbWHaQ== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust The nfsd_file nf_rwsem is currently being used to separate file write and commit instances to ensure that we catch errors and apply them to the correct write/commit. We can improve scalability at the expense of a little accuracy (some extra false positives) by replacing the nf_rwsem with more careful use of the errseq_t mechanism to track errors across the different operations. Signed-off-by: Trond Myklebust Signed-off-by: Chuck Lever [ cel: rebased on zero-verifier fix ] --- fs/nfsd/filecache.c | 1 - fs/nfsd/filecache.h | 1 - fs/nfsd/nfs4proc.c | 16 +++++++++------- fs/nfsd/vfs.c | 40 +++++++++++++++------------------------- 4 files changed, 24 insertions(+), 34 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index aa5dca498b27..e2904540e463 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -189,7 +189,6 @@ nfsd_file_alloc(struct inode *inode, unsigned int may, unsigned int hashval, __set_bit(NFSD_FILE_BREAK_READ, &nf->nf_flags); } nf->nf_mark = NULL; - init_rwsem(&nf->nf_rwsem); trace_nfsd_file_alloc(nf); } return nf; diff --git a/fs/nfsd/filecache.h b/fs/nfsd/filecache.h index 7872df5a0fe3..435ceab27897 100644 --- a/fs/nfsd/filecache.h +++ b/fs/nfsd/filecache.h @@ -46,7 +46,6 @@ struct nfsd_file { refcount_t nf_ref; unsigned char nf_may; struct nfsd_file_mark *nf_mark; - struct rw_semaphore nf_rwsem; }; int nfsd_file_cache_init(void); diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index a6dc5e18c498..56405fc58bfc 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1510,6 +1510,9 @@ static void nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync) static ssize_t _nfsd_copy_file_range(struct nfsd4_copy *copy) { + struct file *dst = copy->nf_dst->nf_file; + struct file *src = copy->nf_src->nf_file; + errseq_t since; ssize_t bytes_copied = 0; u64 bytes_total = copy->cp_count; u64 src_pos = copy->cp_src_pos; @@ -1522,9 +1525,8 @@ static ssize_t _nfsd_copy_file_range(struct nfsd4_copy *copy) do { if (kthread_should_stop()) break; - bytes_copied = nfsd_copy_file_range(copy->nf_src->nf_file, - src_pos, copy->nf_dst->nf_file, dst_pos, - bytes_total); + bytes_copied = nfsd_copy_file_range(src, src_pos, dst, dst_pos, + bytes_total); if (bytes_copied <= 0) break; bytes_total -= bytes_copied; @@ -1534,11 +1536,11 @@ static ssize_t _nfsd_copy_file_range(struct nfsd4_copy *copy) } while (bytes_total > 0 && !copy->cp_synchronous); /* for a non-zero asynchronous copy do a commit of data */ if (!copy->cp_synchronous && copy->cp_res.wr_bytes_written > 0) { - down_write(©->nf_dst->nf_rwsem); - status = vfs_fsync_range(copy->nf_dst->nf_file, - copy->cp_dst_pos, + since = READ_ONCE(dst->f_wb_err); + status = vfs_fsync_range(dst, copy->cp_dst_pos, copy->cp_res.wr_bytes_written, 0); - up_write(©->nf_dst->nf_rwsem); + if (!status) + status = filemap_check_wb_err(dst->f_mapping, since); if (!status) copy->committed = true; } diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 74c3451c2089..316ed702d518 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -522,10 +522,11 @@ __be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, { struct file *src = nf_src->nf_file; struct file *dst = nf_dst->nf_file; + errseq_t since; loff_t cloned; __be32 ret = 0; - down_write(&nf_dst->nf_rwsem); + since = READ_ONCE(dst->f_wb_err); cloned = vfs_clone_file_range(src, src_pos, dst, dst_pos, count, 0); if (cloned < 0) { ret = nfserrno(cloned); @@ -539,6 +540,8 @@ __be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, loff_t dst_end = count ? dst_pos + count - 1 : LLONG_MAX; int status = vfs_fsync_range(dst, dst_pos, dst_end, 0); + if (!status) + status = filemap_check_wb_err(dst->f_mapping, since); if (!status) status = commit_inode_metadata(file_inode(src)); if (status < 0) { @@ -548,7 +551,6 @@ __be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, } } out_err: - up_write(&nf_dst->nf_rwsem); return ret; } @@ -956,6 +958,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, struct super_block *sb = file_inode(file)->i_sb; struct svc_export *exp; struct iov_iter iter; + errseq_t since; __be32 nfserr; int host_err; int use_wgather; @@ -993,8 +996,8 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, flags |= RWF_SYNC; iov_iter_kvec(&iter, WRITE, vec, vlen, *cnt); + since = READ_ONCE(file->f_wb_err); if (flags & RWF_SYNC) { - down_write(&nf->nf_rwsem); if (verf) nfsd_copy_boot_verifier(verf, net_generic(SVC_NET(rqstp), @@ -1003,15 +1006,12 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, if (host_err < 0) nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), nfsd_net_id)); - up_write(&nf->nf_rwsem); } else { - down_read(&nf->nf_rwsem); if (verf) nfsd_copy_boot_verifier(verf, net_generic(SVC_NET(rqstp), nfsd_net_id)); host_err = vfs_iter_write(file, &iter, &pos, flags); - up_read(&nf->nf_rwsem); } if (host_err < 0) { nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), @@ -1021,6 +1021,9 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, *cnt = host_err; nfsd_stats_io_write_add(exp, *cnt); fsnotify_modify(file); + host_err = filemap_check_wb_err(file->f_mapping, since); + if (host_err < 0) + goto out_nfserr; if (stable && use_wgather) { host_err = wait_for_concurrent_writes(file); @@ -1101,19 +1104,6 @@ nfsd_write(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset, } #ifdef CONFIG_NFSD_V3 -static int -nfsd_filemap_write_and_wait_range(struct nfsd_file *nf, loff_t offset, - loff_t end) -{ - struct address_space *mapping = nf->nf_file->f_mapping; - int ret = filemap_fdatawrite_range(mapping, offset, end); - - if (ret) - return ret; - filemap_fdatawait_range_keep_errors(mapping, offset, end); - return 0; -} - /* * Commit all pending writes to stable storage. * @@ -1144,25 +1134,25 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, if (err) goto out; if (EX_ISSYNC(fhp->fh_export)) { - int err2 = nfsd_filemap_write_and_wait_range(nf, offset, end); + errseq_t since = READ_ONCE(nf->nf_file->f_wb_err); + int err2; - down_write(&nf->nf_rwsem); - if (!err2) - err2 = vfs_fsync_range(nf->nf_file, offset, end, 0); + err2 = vfs_fsync_range(nf->nf_file, offset, end, 0); switch (err2) { case 0: nfsd_copy_boot_verifier(verf, net_generic(nf->nf_net, nfsd_net_id)); + err2 = filemap_check_wb_err(nf->nf_file->f_mapping, + since); break; case -EINVAL: err = nfserr_notsupp; break; default: - err = nfserrno(err2); nfsd_reset_boot_verifier(net_generic(nf->nf_net, nfsd_net_id)); } - up_write(&nf->nf_rwsem); + err = nfserrno(err2); } else nfsd_copy_boot_verifier(verf, net_generic(nf->nf_net, nfsd_net_id)); From patchwork Sun Jan 2 17:35:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01D6DC433EF for ; Sun, 2 Jan 2022 17:35:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230072AbiABRfc (ORCPT ); Sun, 2 Jan 2022 12:35:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRfb (ORCPT ); Sun, 2 Jan 2022 12:35:31 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E2A5C061761 for ; Sun, 2 Jan 2022 09:35:31 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D435660F4A for ; Sun, 2 Jan 2022 17:35:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BDC9C36AE7; Sun, 2 Jan 2022 17:35:30 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 03/10] NFSD: Clean up nfsd_vfs_write() Date: Sun, 2 Jan 2022 12:35:29 -0500 Message-Id: <164114492897.7344.10040354999721400314.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1388; h=from:subject:message-id; bh=NBVC2eO5FqQKDoca52ydPEQ0qo+FwjzdPT3gSFMuH88=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJhmsj+NxAECk41JR9W/GSMi8gMp/b5E0jjSP5p oa0MngqJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiYQAKCRAzarMzb2Z/lyd+EA C7Eet4WWhnylYSDF5PwZIBvqQt0q5sL55FY8MqVLXw3QGSMcrhcw+YS6J+NboQi1zeyvApb7cM5xDq DdOkkFykzK3/w4xPwdHXOel8yjJP3mFshykOrIt1txfYfJ4NO9Dkm9I0OVuh+uWeUEvAkTzERtrNPm ZvQ9JFAhPHSoZ8T9kmj9MUjSXhzAx2uVdd1In0179o/dbo61qjN5wzusYwuxSojgH8W7ZP/gdPnlvN Mrtt1IxOriObPrIlAT265HE22t5bOIlYpqBzcS84srAHBalWir9LHuw/Jqwv7Vtyel6xuu16aB3aB1 m+jfyOF8a2rKF4B9qHMWXuhebZ2svLkeDNYUaX3Zv8uaP9QzhcTb3+wg8F/7+0SbXiVd3NPfL2SnUK XQ+uf/EBScyf+Z2e5GMFTIVxZzV9lh8IMvplMlFt9QahVYCl85a+KOgUpbkzmrktwSp3R8GCsGraie EBthD2heQFxlRZJIMbnNkgvAp7drtHB1/v5AoQxUHT9qRb0W+7iOjjbFFjslH/8Tn7LlVevMpjDOpv TzUcc10WhbND7I5fLYJnuRKurtxv6xDPHRS5mF6kGuqED92yp/Xh+NkYCIqYXTVgyy34iKhfJIOX28 C39Zo2RVlu1m/njG3AP/NGM315qigeAJGmjR6y5yPfHhMxsscC3TNPK4mY5A== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The RWF_SYNC and !RWF_SYNC arms are now exactly alike except that the RWF_SYNC arm resets the boot verifier twice in a row. Fix that redundancy and de-duplicate the code. Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 316ed702d518..8f0ac710fd1a 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -997,22 +997,11 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, iov_iter_kvec(&iter, WRITE, vec, vlen, *cnt); since = READ_ONCE(file->f_wb_err); - if (flags & RWF_SYNC) { - if (verf) - nfsd_copy_boot_verifier(verf, - net_generic(SVC_NET(rqstp), - nfsd_net_id)); - host_err = vfs_iter_write(file, &iter, &pos, flags); - if (host_err < 0) - nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), - nfsd_net_id)); - } else { - if (verf) - nfsd_copy_boot_verifier(verf, - net_generic(SVC_NET(rqstp), - nfsd_net_id)); - host_err = vfs_iter_write(file, &iter, &pos, flags); - } + if (verf) + nfsd_copy_boot_verifier(verf, + net_generic(SVC_NET(rqstp), + nfsd_net_id)); + host_err = vfs_iter_write(file, &iter, &pos, flags); if (host_err < 0) { nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), nfsd_net_id)); From patchwork Sun Jan 2 17:35:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702344 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E928AC433F5 for ; Sun, 2 Jan 2022 17:35:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230038AbiABRfj (ORCPT ); Sun, 2 Jan 2022 12:35:39 -0500 Received: from dfw.source.kernel.org ([139.178.84.217]:55418 "EHLO dfw.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRfh (ORCPT ); Sun, 2 Jan 2022 12:35:37 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 60A8160DFA for ; Sun, 2 Jan 2022 17:35:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C01AC36AE7; Sun, 2 Jan 2022 17:35:36 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 04/10] NFSD: De-duplicate net_generic(SVC_NET(rqstp), nfsd_net_id) Date: Sun, 2 Jan 2022 12:35:35 -0500 Message-Id: <164114493550.7344.5237073787095407875.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1608; h=from:subject:message-id; bh=lWDSDxoEGHYRkWq/xAEWykttlyoIVyK7Eorrovr3zxo=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJnzI4d8AcJ9pPhAldd37s4wTDCwgx+JwHtU/f+ Xj3tPzGJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiZwAKCRAzarMzb2Z/lxmUD/ 9YMSyXiAwQO7gAhS9fk3QP31Z8p4L7l7Q2vhi0/oMF/+NsdO5/hNr9oMYpTmu4UxIE+GQuBjYorgGn FByWzQn6Cocm/qrxcUIhLCFd3HA4kCXzSo/D7ZGcB3iGKOw+LcLE09uzSx5iFNMd+6I47e63LBFg6y ZzkWv2fvW20C+n6N4c2Xg2iYS+gJKuTMsO2H3SnuD4luOCFoaZ/sa1yHIw9fEruufZob7Jby1yauni Y/EidCRof/wzdgov5FA2+HXioIlZahv6T/DksuNTSmQJxLAJGsJGVXo/OgacobrNaWFk1ee15Y9tJV zRuf4rRZ91RNUIv59Q32WBrn9aiprQ2kj7QBkLcGNt+7npKxnN5AY8gtp9UxeLQfZ6ambhnrKoCLG4 sMfzRGpCNmA2v3bxuRvKEjV44HqN90CYTHv2dUmLq35lfVghxyiHW3A4U+ssS/qtsYdKXmpA1wxbw4 zAAXBqp2OyxnrgFyFzJaqR3iiNgy6EbNmzEFGQtfpMy+ReC0a//FEe0FyGVPEbjJ8S4EmtfbzXtc92 kfDlVQtvRpG77wCQ38bfuOZkwptl1rVLY1cWvnZ39cMcjSGTacCJWtFAoZGNbc8PnXssG5cTxlfXq5 wKiwJZEz5IryppTgEmwIwxsakyMHABzX3fHU2MVloOiIZeaE1KCZwBV7KCoA== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Since this pointer is used repeatedly, move it to a stack variable. Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 8f0ac710fd1a..2e473d2f47e5 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -954,6 +954,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, unsigned long *cnt, int stable, __be32 *verf) { + struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); struct file *file = nf->nf_file; struct super_block *sb = file_inode(file)->i_sb; struct svc_export *exp; @@ -998,13 +999,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, iov_iter_kvec(&iter, WRITE, vec, vlen, *cnt); since = READ_ONCE(file->f_wb_err); if (verf) - nfsd_copy_boot_verifier(verf, - net_generic(SVC_NET(rqstp), - nfsd_net_id)); + nfsd_copy_boot_verifier(verf, nn); host_err = vfs_iter_write(file, &iter, &pos, flags); if (host_err < 0) { - nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), - nfsd_net_id)); + nfsd_reset_boot_verifier(nn); goto out_nfserr; } *cnt = host_err; @@ -1017,8 +1015,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, if (stable && use_wgather) { host_err = wait_for_concurrent_writes(file); if (host_err < 0) - nfsd_reset_boot_verifier(net_generic(SVC_NET(rqstp), - nfsd_net_id)); + nfsd_reset_boot_verifier(nn); } out_nfserr: From patchwork Sun Jan 2 17:35:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FB52C433EF for ; Sun, 2 Jan 2022 17:35:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230083AbiABRfo (ORCPT ); Sun, 2 Jan 2022 12:35:44 -0500 Received: from dfw.source.kernel.org ([139.178.84.217]:55460 "EHLO dfw.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRfo (ORCPT ); Sun, 2 Jan 2022 12:35:44 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E06C260DFA for ; Sun, 2 Jan 2022 17:35:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26E18C36AEB; Sun, 2 Jan 2022 17:35:43 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 05/10] NFSD: De-duplicate net_generic(nf->nf_net, nfsd_net_id) Date: Sun, 2 Jan 2022 12:35:42 -0500 Message-Id: <164114494203.7344.12014047636933166371.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1739; h=from:subject:message-id; bh=ShtcDRBKl8FeXeqtZoEO54xcdk1v/9DgeJ6iwUdqEBA=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJu+RS0HqnSqFq9Ff3KKT3F9QoVRUQv5F2T9hhl Hqo50TmJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHibgAKCRAzarMzb2Z/l3AtD/ 0ZzeB4k3EUl2ozNgxrYhOuWO7WBxkNAKThanBD6MEecRPLIuAXR/8nx96JX9/uH2UnF9Pw7mtoXMIl U1yFGqnobTAy08qYtKsmW59BqtlNiSFnI9BRkUiBJj7HdSs5AbEtuHJ5P2a+lshdCruV0j/7fkAvRq L9REDH8nTfNkx12UUFh1sgKwdlZpg+tdcPa6pernQvxg3OAa4hUi4QWdS7XsymFU98LFx20HoBjm/9 zeW16tvAo0tkj68WiCTP7g9WKQe+Q8qTiyyxc48j7x1swbZb+AKCRgM60adcfbr4LWZuvLr2uKGPFk 24dz204KDL3fKvYzTa1B0TDiushcFbeVRAqa0M3c4u5kx3UJYiW5pwW1Nc5yS5WSi26QQv3BIeYy+Y pg8lpZamCBlAyTXrd+W9NFWnZnGEwXW4l6fEr+cFPQQoP6+Z26YFVeE0kiDhW3VXJFAyUy3fpbNGCa 055u3wsJqSFR9xrwuw6lP9N5v2zmDJdBd1YfgzWjZObXjbsOpl2NjaIcXuY4+AD8x8v/eTFO8lO4+S Qj8Xi3OE/MB8t3fSb+RNAEbPZ+ayqj9HCg8ZzG7GUSxRqNkbOZEMBJLCAG/7U/2p17HutqzYP/6X5A AUucFLCYFfAJ7vhOfQ2k81xJS0/GGokOBkWrn9MfGSTll3S1WJydaCUtgJFQ== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Since this pointer is used repeatedly, move it to a stack variable. Signed-off-by: Chuck Lever --- fs/nfsd/vfs.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 2e473d2f47e5..c22511decc4c 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -1103,6 +1103,7 @@ __be32 nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, loff_t offset, unsigned long count, __be32 *verf) { + struct nfsd_net *nn; struct nfsd_file *nf; loff_t end = LLONG_MAX; __be32 err = nfserr_inval; @@ -1119,6 +1120,7 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, NFSD_MAY_WRITE|NFSD_MAY_NOT_BREAK_LEASE, &nf); if (err) goto out; + nn = net_generic(nf->nf_net, nfsd_net_id); if (EX_ISSYNC(fhp->fh_export)) { errseq_t since = READ_ONCE(nf->nf_file->f_wb_err); int err2; @@ -1126,8 +1128,7 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, err2 = vfs_fsync_range(nf->nf_file, offset, end, 0); switch (err2) { case 0: - nfsd_copy_boot_verifier(verf, net_generic(nf->nf_net, - nfsd_net_id)); + nfsd_copy_boot_verifier(verf, nn); err2 = filemap_check_wb_err(nf->nf_file->f_mapping, since); break; @@ -1135,13 +1136,11 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, err = nfserr_notsupp; break; default: - nfsd_reset_boot_verifier(net_generic(nf->nf_net, - nfsd_net_id)); + nfsd_reset_boot_verifier(nn); } err = nfserrno(err2); } else - nfsd_copy_boot_verifier(verf, net_generic(nf->nf_net, - nfsd_net_id)); + nfsd_copy_boot_verifier(verf, nn); nfsd_file_put(nf); out: From patchwork Sun Jan 2 17:35:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AD69C433F5 for ; Sun, 2 Jan 2022 17:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230092AbiABRfx (ORCPT ); Sun, 2 Jan 2022 12:35:53 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:53704 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRfw (ORCPT ); Sun, 2 Jan 2022 12:35:52 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 75F60B80B4A for ; Sun, 2 Jan 2022 17:35:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A72D8C36AEB; Sun, 2 Jan 2022 17:35:49 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 06/10] nfsd: Add a tracepoint for errors in nfsd4_clone_file_range() Date: Sun, 2 Jan 2022 12:35:48 -0500 Message-Id: <164114494855.7344.8338658376412437331.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4971; i=chuck.lever@oracle.com; h=from:subject:message-id; bh=8sEHf0F4tAt6no3KTKBkEWm+Iji4A5ODUbCUFwV/Yq8=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJ0ehk4fP2OSx5iV5CX3OKfFnryYWGSnxqHk12t lIM1wgWJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHidAAKCRAzarMzb2Z/l3EhD/ wMvC/C26RXdqy4hnisIQLa3g+/CniWAFzZoN10PdefWaCcX7CViQqstYlu5W/SaR9ZN2nBcospoQlj HRgNPAUn8xjXSnIBCoSuIF5CXiscLHLyZIVKbQVgfAbBFMrwQ8xUUblCiT0kvjgLdKCeapsa2AQnZv 0gBrmrOySHvm34v2QcLH8igK4lAg+18AwIn/bCZoNa4s+auxsJjCeW7CCi4ykKUQKQuMehOAneC+dS F+NYTwRwBxO77CLBN4p1FbT+Z/OVenMhUmi809ARQaxPpyWiMEGoy0ftZGLDaj3NMhP4xrlTMyqNpx xpekpX5gjYSD8uwAhoN+yuaLJAf0f1GpYla37GmS670UVwjCryzqJ7Ooy0Yw8z6xb5XVD36G93ek8G 1dsIzcbtW13ntEYJI2M+Nw41LL1i1mrwgeakk16iZqlWmtuzOlFCQEaykLekNZVuKx2/z5t7rt9QZ+ 6M+3fDMpWfZbPAUjjZnMc8x7GXl9rvmXxfQzSDXbOswSsTISNUb1gZ87Hd0X+6XBpnbxeRk8FOeJLC gVkpNT+PJijObvGOzoqJMGUSAEjuxUI0mwU4AjdJaf8XXJ5QVRTBRwmwklJ9NLLUPRLotJ51NqjFDZ drqjk3uMYnZPazGkzsJoJCX1T3XKz2NoXOeBmnfMLgMtQHGfXA/+ MbJ7HxkQ== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Since a clone error commit can cause the boot verifier to change, we should trace those errors. Signed-off-by: Trond Myklebust Signed-off-by: Chuck Lever [ cel: Addressed a checkpatch.pl splat in fs/nfsd/vfs.h ] --- fs/nfsd/nfs4proc.c | 2 +- fs/nfsd/trace.h | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/vfs.c | 18 ++++++++++++++++-- fs/nfsd/vfs.h | 3 ++- 4 files changed, 69 insertions(+), 4 deletions(-) diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 56405fc58bfc..43057080d2aa 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -1101,7 +1101,7 @@ nfsd4_clone(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, if (status) goto out; - status = nfsd4_clone_file_range(src, clone->cl_src_pos, + status = nfsd4_clone_file_range(rqstp, src, clone->cl_src_pos, dst, clone->cl_dst_pos, clone->cl_count, EX_ISSYNC(cstate->current_fh.fh_export)); diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index ebfe2b878574..54a84b5f7e35 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -389,6 +389,56 @@ TRACE_EVENT(nfsd_dirent, ) ) +DECLARE_EVENT_CLASS(nfsd_copy_err_class, + TP_PROTO(struct svc_rqst *rqstp, + struct svc_fh *src_fhp, + loff_t src_offset, + struct svc_fh *dst_fhp, + loff_t dst_offset, + u64 count, + int status), + TP_ARGS(rqstp, src_fhp, src_offset, dst_fhp, dst_offset, count, status), + TP_STRUCT__entry( + __field(u32, xid) + __field(u32, src_fh_hash) + __field(loff_t, src_offset) + __field(u32, dst_fh_hash) + __field(loff_t, dst_offset) + __field(u64, count) + __field(int, status) + ), + TP_fast_assign( + __entry->xid = be32_to_cpu(rqstp->rq_xid); + __entry->src_fh_hash = knfsd_fh_hash(&src_fhp->fh_handle); + __entry->src_offset = src_offset; + __entry->dst_fh_hash = knfsd_fh_hash(&dst_fhp->fh_handle); + __entry->dst_offset = dst_offset; + __entry->count = count; + __entry->status = status; + ), + TP_printk("xid=0x%08x src_fh_hash=0x%08x src_offset=%lld " + "dst_fh_hash=0x%08x dst_offset=%lld " + "count=%llu status=%d", + __entry->xid, __entry->src_fh_hash, __entry->src_offset, + __entry->dst_fh_hash, __entry->dst_offset, + (unsigned long long)__entry->count, + __entry->status) +) + +#define DEFINE_NFSD_COPY_ERR_EVENT(name) \ +DEFINE_EVENT(nfsd_copy_err_class, nfsd_##name, \ + TP_PROTO(struct svc_rqst *rqstp, \ + struct svc_fh *src_fhp, \ + loff_t src_offset, \ + struct svc_fh *dst_fhp, \ + loff_t dst_offset, \ + u64 count, \ + int status), \ + TP_ARGS(rqstp, src_fhp, src_offset, dst_fhp, dst_offset, \ + count, status)) + +DEFINE_NFSD_COPY_ERR_EVENT(clone_file_range_err); + #include "state.h" #include "filecache.h" #include "vfs.h" diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index c22511decc4c..70ea7e0aae07 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -40,6 +40,7 @@ #include "../internal.h" #include "acl.h" #include "idmap.h" +#include "xdr4.h" #endif /* CONFIG_NFSD_V4 */ #include "nfsd.h" @@ -517,8 +518,15 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, } #endif -__be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, - struct nfsd_file *nf_dst, u64 dst_pos, u64 count, bool sync) +static struct nfsd4_compound_state *nfsd4_get_cstate(struct svc_rqst *rqstp) +{ + return &((struct nfsd4_compoundres *)rqstp->rq_resp)->cstate; +} + +__be32 nfsd4_clone_file_range(struct svc_rqst *rqstp, + struct nfsd_file *nf_src, u64 src_pos, + struct nfsd_file *nf_dst, u64 dst_pos, + u64 count, bool sync) { struct file *src = nf_src->nf_file; struct file *dst = nf_dst->nf_file; @@ -545,6 +553,12 @@ __be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, if (!status) status = commit_inode_metadata(file_inode(src)); if (status < 0) { + trace_nfsd_clone_file_range_err(rqstp, + &nfsd4_get_cstate(rqstp)->save_fh, + src_pos, + &nfsd4_get_cstate(rqstp)->current_fh, + dst_pos, + count, status); nfsd_reset_boot_verifier(net_generic(nf_dst->nf_net, nfsd_net_id)); ret = nfserrno(status); diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h index b21b76e6b9a8..9f56dcb22ff7 100644 --- a/fs/nfsd/vfs.h +++ b/fs/nfsd/vfs.h @@ -57,7 +57,8 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *, struct svc_fh *, struct xdr_netobj *); __be32 nfsd4_vfs_fallocate(struct svc_rqst *, struct svc_fh *, struct file *, loff_t, loff_t, int); -__be32 nfsd4_clone_file_range(struct nfsd_file *nf_src, u64 src_pos, +__be32 nfsd4_clone_file_range(struct svc_rqst *rqstp, + struct nfsd_file *nf_src, u64 src_pos, struct nfsd_file *nf_dst, u64 dst_pos, u64 count, bool sync); #endif /* CONFIG_NFSD_V4 */ From patchwork Sun Jan 2 17:35:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E6BFC433F5 for ; Sun, 2 Jan 2022 17:36:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230090AbiABRf7 (ORCPT ); Sun, 2 Jan 2022 12:35:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRf7 (ORCPT ); Sun, 2 Jan 2022 12:35:59 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BBFAC061761 for ; Sun, 2 Jan 2022 09:35:59 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id D9C58B80B4A for ; Sun, 2 Jan 2022 17:35:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 320A1C36AE7; Sun, 2 Jan 2022 17:35:56 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 07/10] NFSD: Write verifier might go backwards Date: Sun, 2 Jan 2022 12:35:55 -0500 Message-Id: <164114495506.7344.17920348319488852614.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=887; h=from:subject:message-id; bh=6TXqYgW49mmc1pVTQPpmQTt0CSdUjl86JQjHw3VFstU=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eJ7PCPH67lWMCOLiMng/26TaNr0J0pEoVnVouFW pufmttCJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiewAKCRAzarMzb2Z/lwRFD/ wKWvHiCxNg+IEZFtbq9UmuqCwaGnhQEY7Hs+nb3bLHLf8/qCYLvpALeZbhc0IY+9SOgfl1RuzoHPD5 jrhqg8uH+1dn1Jt6NJhkc+/3m02M2c5e40wf+e8nN0/B/Eic4/P1Ohn79ANAF3vPjHFKU6j1t4sRdx Qj0Ecz75+QRH9YqWqrmBV+lfxMmjM40DrqnswseJBu2O7ZMhhDeAIpecUrNLAmdPQisLNtc8dJ+Pr1 m/KjJ9kM3tw3eVc7tjge8vL1RHB4uXCfnUrW9dWKrvCcVqZAnXy5VAey2lO8OjRMmaEzkNJXGvmZCk eEa5frUHbnTntX9LKI9bktmDvqWOFWfLa40ruxdUp0uHYrjircIMIPP6IsLOOPCWAevZgm6DDJ8F2z igwp7tY/M8PVhAqgcEAJ6KE7cFRP580so+xQFUl874rG9tEEZymf7Z8uav5UpV7FGxzJ7e1QO8MvrR acKQhPqgt+sk4deH2bZNcMsEHU5Wz2lXysRBnHUc8abQfcEc/Z/IRkxF5gfuRRGoXsLefB4JV1nYyR NtDjhzEy1gPGdwWvcBxUfcobhev7TDhjW48EC++M4OjzALF0nQFza/QC0izLCFkQz6z2zRQQpvOeuG mkemTaEXhnBfFfAZqO1ktW012z4vH64vR/IE5hhE4J1A6Bt//QP0Yvkoum+g== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org When vfs_iter_write() starts to fail because a file system is full, a bunch of writes can fail at once with ENOSPC. These writes repeatedly invoke nfsd_reset_boot_verifier() in quick succession. Ensure that the time it grabs doesn't go backwards due to an ntp adjustment going on at the same time. Signed-off-by: Chuck Lever --- fs/nfsd/nfssvc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index 14c1ef6f8cc7..6eccf6700250 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -363,7 +363,7 @@ void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn) static void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) { - ktime_get_real_ts64(&nn->nfssvc_boot); + ktime_get_raw_ts64(&nn->nfssvc_boot); } void nfsd_reset_boot_verifier(struct nfsd_net *nn) From patchwork Sun Jan 2 17:36:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E94D9C433F5 for ; Sun, 2 Jan 2022 17:36:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230104AbiABRgG (ORCPT ); Sun, 2 Jan 2022 12:36:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRgG (ORCPT ); Sun, 2 Jan 2022 12:36:06 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 764F3C061761 for ; Sun, 2 Jan 2022 09:36:05 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 32F24B80D95 for ; Sun, 2 Jan 2022 17:36:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1719C36AEB; Sun, 2 Jan 2022 17:36:02 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 08/10] NFSD: Clean up the nfsd_net::nfssvc_boot field Date: Sun, 2 Jan 2022 12:36:01 -0500 Message-Id: <164114496159.7344.7632968111470947824.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5145; h=from:subject:message-id; bh=UFDhw2jXgiVuxRpsd3yYxrnpLx4irk+DXhhUAzm/eBQ=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eKBCBqbUXcQRWMTrOzrnWxiUF0tNx7cri3osHAs rQ7NTwiJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHigQAKCRAzarMzb2Z/lwq6D/ 4iN9M4XDHzjsdw/jrETTkHmAU0wTjy3gTjO/K3xRp6+f1y0ObZB1BhkuT3U1gbYD+K9t88DB5gMZ+/ Vq2sAwmSyb+3yNla6EWwc41wKPTRRtqjlbDKiMRwVWoKzi0jBwTD83RRShHO7zvnufoBtUhI7XMj+7 Ws3ap9FAE4kdVtVM+q08E398i/UEn0bbwK1CtGnMJPSuV++gBudtRp2wYsn5YTHpoWBCl0y7fwrwUk zWyx9cvRSzY5+5DoexX0+TNiYpE5Naoyd/lgy+1roFhhgYC7id78R4K0/lohapJtNDeneDxtpqmPDb apaZMWrfDDY9QZ4vKoxGRcbClc49kqZBDoL3tOoaxe+HJkSBgAjn9Gw/KuEgKVCVuy1rEhh1VI9Q4/ 8I/hiX6q7FankO4VH7XnqFDN911pt/RcS2tETv8zypLLurTg9nuSmR7wWIG+mp3uEAOIdanXO7XXN6 EkLYmbPjM+2jvnq0hqtu/ih+bWMgNMqR/lgeqwoxwYEyoIhxhV9UoeEMt2sYk/6gR0jvsU2Q+ai+Sj PuK03GKc+pMdXPfi4uBqR81o/Rh8ZSPK1GP7khu5X+A4/BF1DRJ3WA9lFve1UdSTHO1lKuNm6ZJMTA yyHQKpXF26TPjL3ZOGr6sGi/pouxRlgIgYB3fvCg+pJ1DZ4DPHBWKUQkTbtA== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org There are two boot-time fields in struct nfsd_net: one called boot_time and one called nfssvc_boot. The latter is used only to form write verifiers, but its documenting comment declares: /* Time of server startup */ Since commit 27c438f53e79 ("nfsd: Support the server resetting the boot verifier"), this field can be reset at any time; it's no longer tied to server restart. So that comment is stale. Also, according to pahole, struct timespec64 is 16 bytes long on x86_64. The nfssvc_boot field is used only to form a write verifier, which is 8 bytes long. Let's clarify this situation by manufacturing an 8-byte verifier in nfs_reset_boot_verifier() and storing only that in struct nfsd_net. We're grabbing 128 bits of time, so compress all of those into a 64-bit verifier instead of throwing out the high-order bits. Signed-off-by: Chuck Lever --- fs/nfsd/netns.h | 7 ++++--- fs/nfsd/nfsctl.c | 3 ++- fs/nfsd/nfssvc.c | 51 ++++++++++++++++++++++++++++++++++++++------------- 3 files changed, 44 insertions(+), 17 deletions(-) diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h index 9e8b77d2a3a4..c879d80d5449 100644 --- a/fs/nfsd/netns.h +++ b/fs/nfsd/netns.h @@ -11,6 +11,7 @@ #include #include #include +#include /* Hash tables for nfs4_clientid state */ #define CLIENT_HASH_BITS 4 @@ -108,9 +109,9 @@ struct nfsd_net { bool nfsd_net_up; bool lockd_up; - /* Time of server startup */ - struct timespec64 nfssvc_boot; - seqlock_t boot_lock; + seqlock_t writeverf_lock; + unsigned char writeverf[8]; + siphash_key_t siphash_key; /* * Max number of connections this nfsd container will allow. Defaults diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index a8ad71567fc7..b9f27fbcd768 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -1483,7 +1483,8 @@ static __net_init int nfsd_init_net(struct net *net) nn->clientid_counter = nn->clientid_base + 1; nn->s2s_cp_cl_id = nn->clientid_counter++; - seqlock_init(&nn->boot_lock); + get_random_bytes(&nn->siphash_key, sizeof(nn->siphash_key)); + seqlock_init(&nn->writeverf_lock); return 0; diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index 6eccf6700250..81d47049588f 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -344,33 +345,57 @@ static bool nfsd_needs_lockd(struct nfsd_net *nn) return nfsd_vers(nn, 2, NFSD_TEST) || nfsd_vers(nn, 3, NFSD_TEST); } +/** + * nfsd_copy_boot_verifier - Atomically copy a write verifier + * @verf: buffer in which to receive the verifier cookie + * @nn: NFS net namespace + * + * This function provides a wait-free mechanism for copying the + * namespace's boot verifier without tearing it. + */ void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn) { int seq = 0; do { - read_seqbegin_or_lock(&nn->boot_lock, &seq); - /* - * This is opaque to client, so no need to byte-swap. Use - * __force to keep sparse happy. y2038 time_t overflow is - * irrelevant in this usage - */ - verf[0] = (__force __be32)nn->nfssvc_boot.tv_sec; - verf[1] = (__force __be32)nn->nfssvc_boot.tv_nsec; - } while (need_seqretry(&nn->boot_lock, seq)); - done_seqretry(&nn->boot_lock, seq); + read_seqbegin_or_lock(&nn->writeverf_lock, &seq); + memcpy(verf, nn->writeverf, sizeof(*verf)); + } while (need_seqretry(&nn->writeverf_lock, seq)); + done_seqretry(&nn->writeverf_lock, seq); } static void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) { - ktime_get_raw_ts64(&nn->nfssvc_boot); + struct timespec64 now; + u64 verf; + + /* + * Because the time value is hashed, y2038 time_t overflow + * is irrelevant in this usage. + */ + ktime_get_raw_ts64(&now); + verf = siphash_2u64(now.tv_sec, now.tv_nsec, &nn->siphash_key); + memcpy(nn->writeverf, &verf, sizeof(nn->writeverf)); } +/** + * nfsd_reset_boot_verifier - Generate a new boot verifier + * @nn: NFS net namespace + * + * This function updates the ->writeverf field of @nn. This field + * contains an opaque cookie that, according to Section 18.32.3 of + * RFC 8881, "the client can use to determine whether a server has + * changed instance state (e.g., server restart) between a call to + * WRITE and a subsequent call to either WRITE or COMMIT. This + * cookie MUST be unchanged during a single instance of the NFSv4.1 + * server and MUST be unique between instances of the NFSv4.1 + * server." + */ void nfsd_reset_boot_verifier(struct nfsd_net *nn) { - write_seqlock(&nn->boot_lock); + write_seqlock(&nn->writeverf_lock); nfsd_reset_boot_verifier_locked(nn); - write_sequnlock(&nn->boot_lock); + write_sequnlock(&nn->writeverf_lock); } static int nfsd_startup_net(struct net *net, const struct cred *cred) From patchwork Sun Jan 2 17:36:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36475C433EF for ; Sun, 2 Jan 2022 17:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230106AbiABRgM (ORCPT ); Sun, 2 Jan 2022 12:36:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRgM (ORCPT ); Sun, 2 Jan 2022 12:36:12 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06284C061761 for ; Sun, 2 Jan 2022 09:36:12 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B7012B80B4A for ; Sun, 2 Jan 2022 17:36:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4447DC36AE7; Sun, 2 Jan 2022 17:36:09 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 09/10] NFSD: Rename boot verifier functions Date: Sun, 2 Jan 2022 12:36:08 -0500 Message-Id: <164114496812.7344.5315813381496609958.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6075; h=from:subject:message-id; bh=leUUIzQY15Ms1Cr/pwOg7QhGLs0LfOlgojXcY4V/lp0=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eKIMul01DGRQGHS2HkqgnP1yEguWKesp6CdCBOD w5myQuqJAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHiiAAKCRAzarMzb2Z/l4IKD/ 9J1lWHdEYjml/guS/mK11nIDarmEwvoptFCt4X8iZjU/eYvmoE32HHmUAtfHn6Ai3d3JMfUVhIEXkN r/o67mnGLRRiKSp2Z3W0qqGbFdYCkX1urbBsdltoE6RAxkfiDlVqqSYBSNzeJ7WQZ5ov7rIubNo+KN B1veRPkMkgkZxEtfvW9aOpW9VqFYeMDD0ZNe6of1hLc0twgXyGcjq4NE22NOnXlOPg1V+75d584D+2 nGdiVEqAA9gabQ0fBlTPKWLRmdJi0T4A5F3RhNy8ZdFi2rLGnon6G7IaNPEtH86GHQ+tXbL26824FN nyBM7cz/zgq7rdHlQpO+8iccBmv0Ih5fwlQZJwWQcPAKQgeoE+PaUcU+UTQiElpDfqo/r77iax+xlu bDSc6jZVoxMbiBeq8JMZjLtwN9JLQ0dMQ60RISDqSeqJXxWobAZ8xLA1x02gLCbK8bY/LrhEtLJLaH AENNerRqTqZkKOXhKzj3jzeBZ+pQ8EceYANgclXbPiNvT2Yvz4IQx/f7jcmVk3/praeNEiJ0ZLbWkd AC1Q9OspRk5ECC7eMa6h1ivg2Qcg6m+5knXjD4e5rXc1lCjqNJcAfjjfO1QDJh4QPtT6HGlRfgU1FF GejngBeSzrDpSK0MPSOVAXAwp41J36w77znlbfP5zbiLgFFXAAQnsVO+sHhw== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: These functions handle what the specs call a write verifier, which in the Linux NFS server implementation is now divorced from the server's boot instance Signed-off-by: Chuck Lever --- fs/nfsd/filecache.c | 2 +- fs/nfsd/netns.h | 4 ++-- fs/nfsd/nfs4proc.c | 2 +- fs/nfsd/nfssvc.c | 16 ++++++++-------- fs/nfsd/vfs.c | 16 ++++++++-------- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index e2904540e463..8bc807c5fea4 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -243,7 +243,7 @@ nfsd_file_do_unhash(struct nfsd_file *nf) trace_nfsd_file_unhash(nf); if (nfsd_file_check_write_error(nf)) - nfsd_reset_boot_verifier(net_generic(nf->nf_net, nfsd_net_id)); + nfsd_reset_write_verifier(net_generic(nf->nf_net, nfsd_net_id)); --nfsd_file_hashtbl[nf->nf_hashval].nfb_count; hlist_del_rcu(&nf->nf_node); atomic_long_dec(&nfsd_filecache_count); diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h index c879d80d5449..82ef4d77c592 100644 --- a/fs/nfsd/netns.h +++ b/fs/nfsd/netns.h @@ -197,6 +197,6 @@ extern void nfsd_netns_free_versions(struct nfsd_net *nn); extern unsigned int nfsd_net_id; -void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn); -void nfsd_reset_boot_verifier(struct nfsd_net *nn); +void nfsd_copy_write_verifier(__be32 verf[2], struct nfsd_net *nn); +void nfsd_reset_write_verifier(struct nfsd_net *nn); #endif /* __NFSD_NETNS_H__ */ diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 43057080d2aa..6f53eb90c6b4 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -598,7 +598,7 @@ static void gen_boot_verifier(nfs4_verifier *verifier, struct net *net) BUILD_BUG_ON(2*sizeof(*verf) != sizeof(verifier->data)); - nfsd_copy_boot_verifier(verf, net_generic(net, nfsd_net_id)); + nfsd_copy_write_verifier(verf, net_generic(net, nfsd_net_id)); } static __be32 diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index 81d47049588f..07193595b8e0 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -346,14 +346,14 @@ static bool nfsd_needs_lockd(struct nfsd_net *nn) } /** - * nfsd_copy_boot_verifier - Atomically copy a write verifier + * nfsd_copy_write_verifier - Atomically copy a write verifier * @verf: buffer in which to receive the verifier cookie * @nn: NFS net namespace * * This function provides a wait-free mechanism for copying the - * namespace's boot verifier without tearing it. + * namespace's write verifier without tearing it. */ -void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn) +void nfsd_copy_write_verifier(__be32 verf[2], struct nfsd_net *nn) { int seq = 0; @@ -364,7 +364,7 @@ void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn) done_seqretry(&nn->writeverf_lock, seq); } -static void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) +static void nfsd_reset_write_verifier_locked(struct nfsd_net *nn) { struct timespec64 now; u64 verf; @@ -379,7 +379,7 @@ static void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) } /** - * nfsd_reset_boot_verifier - Generate a new boot verifier + * nfsd_reset_write_verifier - Generate a new write verifier * @nn: NFS net namespace * * This function updates the ->writeverf field of @nn. This field @@ -391,10 +391,10 @@ static void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) * server and MUST be unique between instances of the NFSv4.1 * server." */ -void nfsd_reset_boot_verifier(struct nfsd_net *nn) +void nfsd_reset_write_verifier(struct nfsd_net *nn) { write_seqlock(&nn->writeverf_lock); - nfsd_reset_boot_verifier_locked(nn); + nfsd_reset_write_verifier_locked(nn); write_sequnlock(&nn->writeverf_lock); } @@ -683,7 +683,7 @@ int nfsd_create_serv(struct net *net) register_inet6addr_notifier(&nfsd_inet6addr_notifier); #endif } - nfsd_reset_boot_verifier(nn); + nfsd_reset_write_verifier(nn); return 0; } diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 70ea7e0aae07..49564457bd3d 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -559,8 +559,8 @@ __be32 nfsd4_clone_file_range(struct svc_rqst *rqstp, &nfsd4_get_cstate(rqstp)->current_fh, dst_pos, count, status); - nfsd_reset_boot_verifier(net_generic(nf_dst->nf_net, - nfsd_net_id)); + nfsd_reset_write_verifier(net_generic(nf_dst->nf_net, + nfsd_net_id)); ret = nfserrno(status); } } @@ -1013,10 +1013,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, iov_iter_kvec(&iter, WRITE, vec, vlen, *cnt); since = READ_ONCE(file->f_wb_err); if (verf) - nfsd_copy_boot_verifier(verf, nn); + nfsd_copy_write_verifier(verf, nn); host_err = vfs_iter_write(file, &iter, &pos, flags); if (host_err < 0) { - nfsd_reset_boot_verifier(nn); + nfsd_reset_write_verifier(nn); goto out_nfserr; } *cnt = host_err; @@ -1029,7 +1029,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, if (stable && use_wgather) { host_err = wait_for_concurrent_writes(file); if (host_err < 0) - nfsd_reset_boot_verifier(nn); + nfsd_reset_write_verifier(nn); } out_nfserr: @@ -1142,7 +1142,7 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, err2 = vfs_fsync_range(nf->nf_file, offset, end, 0); switch (err2) { case 0: - nfsd_copy_boot_verifier(verf, nn); + nfsd_copy_write_verifier(verf, nn); err2 = filemap_check_wb_err(nf->nf_file->f_mapping, since); break; @@ -1150,11 +1150,11 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, err = nfserr_notsupp; break; default: - nfsd_reset_boot_verifier(nn); + nfsd_reset_write_verifier(nn); } err = nfserrno(err2); } else - nfsd_copy_boot_verifier(verf, nn); + nfsd_copy_write_verifier(verf, nn); nfsd_file_put(nf); out: From patchwork Sun Jan 2 17:36:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12702350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AA61C433F5 for ; Sun, 2 Jan 2022 17:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230109AbiABRgS (ORCPT ); Sun, 2 Jan 2022 12:36:18 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:53872 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbiABRgS (ORCPT ); Sun, 2 Jan 2022 12:36:18 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2CEA6B80B4D for ; Sun, 2 Jan 2022 17:36:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C938BC36AE7; Sun, 2 Jan 2022 17:36:15 +0000 (UTC) From: Chuck Lever To: linux-nfs@vger.kernel.org Cc: trondmy@kernel.org Subject: [PATCH 10/10] NFSD: Trace boot verifier resets Date: Sun, 2 Jan 2022 12:36:14 -0500 Message-Id: <164114497467.7344.3800554443290630816.stgit@bazille.1015granger.net> X-Mailer: git-send-email 2.34.0 In-Reply-To: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> References: <164114486506.7344.16096063573748374893.stgit@bazille.1015granger.net> User-Agent: StGit/1.4 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3587; h=from:subject:message-id; bh=MVjHR9VEG4Cvc8K9ffiVt3k3iLGrlD/2ba+jotdwohA=; b=owEBbQKS/ZANAwAIATNqszNvZn+XAcsmYgBh0eKOcOHTdoTh6Dn75wuMpfzLoVSL6QoFmJ2RoeuI kxonmf+JAjMEAAEIAB0WIQQosuWwEobfJDzyPv4zarMzb2Z/lwUCYdHijgAKCRAzarMzb2Z/l6ezD/ 9ts1kURe+ZsU90jwU8gUDDdmsSs4SJ3S5/mtoJITH1TZvbt4Rqx5uA4f/x6vATnmZfDFhQZBeb3CRH tvZAOmGO0BWW7d8eNZ9sGh9hsG7jSe/DzfskKh55fogWg/b18f1uboW9vvjv+8pH6WqC6BPdlYdhb/ 3hV5AkLWBcONl1UJKLs8hqyqsTP4lBd5TQdBOev2h820W1l27d3AL4pl0LEWOCA2kLfvwZ/vC5NKJy z+CHwuymwV1D9lJxt7pfyJvtMiIoRJ8oRo0kiqPRA4Av/1xKRsQqjwVjVxo8Ymr9yKTDjGKaa84ijL FyZS4F0UaMs6XcTvYxG47VNAbxSKd5l878mkWpsOur9lbBDRjUezM6R5voENz4P6tgFZWfa3Lqb2ef jZebxtBvDFyFAoLmY3blDRLPItxfx/SimtYc7wnmjbejq1bp3lOdncxoJrlgoU+44ewOrfLkCbTskJ l/Jj1xCAme9wEpBSs4tm3ZZgi7opmxzBcyoLE2JKFwuZL8MdRo5XatfdMW6ktlQA0I+WP84czrZFRw wz4Ye3OG8hk3jGJd3+4RkNEs99EgCKr+q5NpDo0zJ8TzqEiupg+qczxgrB8lvxGyhbkqx17Hs5Ojy9 MowbvC3R7H0/sqTsh27wymI7H54im2Ma4voSXtI1/fwbyCb7hgMtHtJMF+5A== X-Developer-Key: i=chuck.lever@oracle.com; a=openpgp; fpr=28B2E5B01286DF243CF23EFE336AB3336F667F97 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org According to commit bbf2f098838a ("nfsd: Reset the boot verifier on all write I/O errors"), the Linux NFS server forces all clients to resend pending unstable writes if any server-side write or commit operation encounters an error (say, ENOSPC). This is a rare and quite exceptional event that could require administrative recovery action, so it should be made trace-able. Example trace event: nfsd-938 [002] 7174.945558: nfsd_writeverf_reset: boot_time= 61cc920d xid=0xdcd62036 error=-28 new verifier=0x08aecc6142515904 Signed-off-by: Chuck Lever --- fs/nfsd/trace.h | 28 ++++++++++++++++++++++++++++ fs/nfsd/vfs.c | 13 ++++++++++--- 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index 54a84b5f7e35..b6b25f0f67a5 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -564,6 +564,34 @@ DEFINE_EVENT(nfsd_net_class, nfsd_##name, \ DEFINE_NET_EVENT(grace_start); DEFINE_NET_EVENT(grace_complete); +TRACE_EVENT(nfsd_writeverf_reset, + TP_PROTO( + const struct nfsd_net *nn, + const struct svc_rqst *rqstp, + int error + ), + TP_ARGS(nn, rqstp, error), + TP_STRUCT__entry( + __field(unsigned long long, boot_time) + __field(u32, xid) + __field(int, error) + __array(unsigned char, verifier, NFS4_VERIFIER_SIZE) + ), + TP_fast_assign( + __entry->boot_time = nn->boot_time; + __entry->xid = be32_to_cpu(rqstp->rq_xid); + __entry->error = error; + + /* avoid seqlock inside TP_fast_assign */ + memcpy(__entry->verifier, nn->writeverf, + NFS4_VERIFIER_SIZE); + ), + TP_printk("boot_time=%16llx xid=0x%08x error=%d new verifier=0x%s", + __entry->boot_time, __entry->xid, __entry->error, + __print_hex_str(__entry->verifier, NFS4_VERIFIER_SIZE) + ) +); + TRACE_EVENT(nfsd_clid_cred_mismatch, TP_PROTO( const struct nfs4_client *clp, diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c index 49564457bd3d..e4e59e1660e1 100644 --- a/fs/nfsd/vfs.c +++ b/fs/nfsd/vfs.c @@ -553,14 +553,17 @@ __be32 nfsd4_clone_file_range(struct svc_rqst *rqstp, if (!status) status = commit_inode_metadata(file_inode(src)); if (status < 0) { + struct nfsd_net *nn = net_generic(nf_dst->nf_net, + nfsd_net_id); + trace_nfsd_clone_file_range_err(rqstp, &nfsd4_get_cstate(rqstp)->save_fh, src_pos, &nfsd4_get_cstate(rqstp)->current_fh, dst_pos, count, status); - nfsd_reset_write_verifier(net_generic(nf_dst->nf_net, - nfsd_net_id)); + nfsd_reset_write_verifier(nn); + trace_nfsd_writeverf_reset(nn, rqstp, status); ret = nfserrno(status); } } @@ -1017,6 +1020,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, host_err = vfs_iter_write(file, &iter, &pos, flags); if (host_err < 0) { nfsd_reset_write_verifier(nn); + trace_nfsd_writeverf_reset(nn, rqstp, host_err); goto out_nfserr; } *cnt = host_err; @@ -1028,8 +1032,10 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct nfsd_file *nf, if (stable && use_wgather) { host_err = wait_for_concurrent_writes(file); - if (host_err < 0) + if (host_err < 0) { nfsd_reset_write_verifier(nn); + trace_nfsd_writeverf_reset(nn, rqstp, host_err); + } } out_nfserr: @@ -1151,6 +1157,7 @@ nfsd_commit(struct svc_rqst *rqstp, struct svc_fh *fhp, break; default: nfsd_reset_write_verifier(nn); + trace_nfsd_writeverf_reset(nn, rqstp, err2); } err = nfserrno(err2); } else