From patchwork Mon Aug 26 16:50:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11115113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 30A0614E5 for ; Mon, 26 Aug 2019 16:53:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0486D20850 for ; Mon, 26 Aug 2019 16:53:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sQXUoMWR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731786AbfHZQxu (ORCPT ); Mon, 26 Aug 2019 12:53:50 -0400 Received: from mail-io1-f68.google.com ([209.85.166.68]:38795 "EHLO mail-io1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728560AbfHZQxu (ORCPT ); Mon, 26 Aug 2019 12:53:50 -0400 Received: by mail-io1-f68.google.com with SMTP id p12so38996815iog.5 for ; Mon, 26 Aug 2019 09:53:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GA75ufOJonbpqC5MlA/RQIRk67dy/HSW1IeiLuiWlEU=; b=sQXUoMWRro6muf1MsbOMAs+rCGu9aPocoQUZtfzyxA4DD4E+/gRzFCVXMRK+QW/rg6 IrOyVBSJkgwuZqFu6hCQ7TLSCwXrSkeqvUDmyRAoQvncFyeqynvIS2fSmyai9ruZiuCY DYtVrYE4FZJ7Q5Q4Ocfab1r0cy9C10xkYoEEdSJYlaL+3mH4Vk7/6P6lqt6QnMws8Gbf 1TpVY3bGVKCM/oJ796xs4YG2sw4FpMaAwPaehd6AHuleFrLCeHKPoGMs1Pdor0duCMS4 V1O5m9fZSP7vvTPM9imoh88zcjkmbUjAdPdYoYThSjFi71BpX226is5/unR3h9vgHH5V ZhRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GA75ufOJonbpqC5MlA/RQIRk67dy/HSW1IeiLuiWlEU=; b=iSaIV3qUzmAh7+61Y59Azse8zkDgig+6uVbQ1kG74h9eXwtTFAm5bRLE9EKT1SiAbz FxpzwevxRkHEfpn61efWw4qKnmtupjScpnD+/ypurh+jPBX4NrmtYqkDdBmb+vHr2oFP DooaHN7fhPSdUmmIRaNo6PocS0aj4PgMhHxOvTwPaMnXxTMST6SMYZUToatUj4C9eJGx /x1Qpc4xE8guVgtN+wBbJE8fNj/XogR+K/YPDeo8iNu3yXPo5a/EC4AzfOreYhTKN031 NrmbNpBLddIXl77OUoKrYKOmHNs2ENGZW/B0PXYkNGLUoqfHY9zVwU2JljQoTIn+HnBZ vEyA== X-Gm-Message-State: APjAAAWTKiMYWOmEttebn040vi39CGCUP7KJVH5WpPV2iACGjkhNBDr+ OvjsZSnCd3w6/MkmyJmqfmOulGjq2g== X-Google-Smtp-Source: APXvYqzQ39k74lvC2owoG28WQIh/ye5oRmPZUTQEqWa7zAIQGC1nETypzYSruYsdfh5FI+nDvRkaWw== X-Received: by 2002:a5d:9d43:: with SMTP id k3mr889970iok.111.1566838428863; Mon, 26 Aug 2019 09:53:48 -0700 (PDT) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id u24sm10613490iot.38.2019.08.26.09.53.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Aug 2019 09:53:48 -0700 (PDT) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org Subject: [PATCH 1/3] nfsd: nfsd_file cache entries should be per net namespace Date: Mon, 26 Aug 2019 12:50:19 -0400 Message-Id: <20190826165021.81075-2-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190826165021.81075-1-trond.myklebust@hammerspace.com> References: <20190826165021.81075-1-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Ensure that we can safely clear out the file cache entries when the nfs server is shut down on a container. Otherwise, the file cache may end up pinning the mounts. Signed-off-by: Trond Myklebust --- fs/nfsd/export.c | 2 +- fs/nfsd/filecache.c | 33 +++++++++++++++++++++------------ fs/nfsd/filecache.h | 3 ++- fs/nfsd/nfssvc.c | 1 + 4 files changed, 25 insertions(+), 14 deletions(-) diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c index 052fac64b578..15422c951fd1 100644 --- a/fs/nfsd/export.c +++ b/fs/nfsd/export.c @@ -240,7 +240,7 @@ static void expkey_flush(void) * destroyed while we're in the middle of flushing. */ mutex_lock(&nfsd_mutex); - nfsd_file_cache_purge(); + nfsd_file_cache_purge(current->nsproxy->net_ns); mutex_unlock(&nfsd_mutex); } diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index a2fcb251d2f6..d229fd3c825d 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -17,6 +17,7 @@ #include "vfs.h" #include "nfsd.h" #include "nfsfh.h" +#include "netns.h" #include "filecache.h" #include "trace.h" @@ -168,7 +169,8 @@ nfsd_file_mark_find_or_create(struct nfsd_file *nf) } static struct nfsd_file * -nfsd_file_alloc(struct inode *inode, unsigned int may, unsigned int hashval) +nfsd_file_alloc(struct inode *inode, unsigned int may, unsigned int hashval, + struct net *net) { struct nfsd_file *nf; @@ -178,6 +180,7 @@ nfsd_file_alloc(struct inode *inode, unsigned int may, unsigned int hashval) INIT_LIST_HEAD(&nf->nf_lru); nf->nf_file = NULL; nf->nf_cred = get_current_cred(); + nf->nf_net = net; nf->nf_flags = 0; nf->nf_inode = inode; nf->nf_hashval = hashval; @@ -608,10 +611,11 @@ nfsd_file_cache_init(void) * Note this can deadlock with nfsd_file_lru_cb. */ void -nfsd_file_cache_purge(void) +nfsd_file_cache_purge(struct net *net) { unsigned int i; struct nfsd_file *nf; + struct hlist_node *next; LIST_HEAD(dispose); bool del; @@ -619,10 +623,12 @@ nfsd_file_cache_purge(void) return; for (i = 0; i < NFSD_FILE_HASH_SIZE; i++) { - spin_lock(&nfsd_file_hashtbl[i].nfb_lock); - while(!hlist_empty(&nfsd_file_hashtbl[i].nfb_head)) { - nf = hlist_entry(nfsd_file_hashtbl[i].nfb_head.first, - struct nfsd_file, nf_node); + struct nfsd_fcache_bucket *nfb = &nfsd_file_hashtbl[i]; + + spin_lock(&nfb->nfb_lock); + hlist_for_each_entry_safe(nf, next, &nfb->nfb_head, nf_node) { + if (net && nf->nf_net != net) + continue; del = nfsd_file_unhash_and_release_locked(nf, &dispose); /* @@ -631,7 +637,7 @@ nfsd_file_cache_purge(void) */ WARN_ON_ONCE(!del); } - spin_unlock(&nfsd_file_hashtbl[i].nfb_lock); + spin_unlock(&nfb->nfb_lock); nfsd_file_dispose_list(&dispose); } } @@ -650,7 +656,7 @@ nfsd_file_cache_shutdown(void) * calling nfsd_file_cache_purge */ cancel_delayed_work_sync(&nfsd_filecache_laundrette); - nfsd_file_cache_purge(); + nfsd_file_cache_purge(NULL); list_lru_destroy(&nfsd_file_lru); rcu_barrier(); fsnotify_put_group(nfsd_file_fsnotify_group); @@ -686,7 +692,7 @@ nfsd_match_cred(const struct cred *c1, const struct cred *c2) static struct nfsd_file * nfsd_file_find_locked(struct inode *inode, unsigned int may_flags, - unsigned int hashval) + unsigned int hashval, struct net *net) { struct nfsd_file *nf; unsigned char need = may_flags & NFSD_FILE_MAY_MASK; @@ -697,6 +703,8 @@ nfsd_file_find_locked(struct inode *inode, unsigned int may_flags, continue; if (nf->nf_inode != inode) continue; + if (nf->nf_net != net) + continue; if (!nfsd_match_cred(nf->nf_cred, current_cred())) continue; if (nfsd_file_get(nf) != NULL) @@ -739,6 +747,7 @@ nfsd_file_acquire(struct svc_rqst *rqstp, struct svc_fh *fhp, unsigned int may_flags, struct nfsd_file **pnf) { __be32 status; + struct net *net = SVC_NET(rqstp); struct nfsd_file *nf, *new; struct inode *inode; unsigned int hashval; @@ -753,12 +762,12 @@ nfsd_file_acquire(struct svc_rqst *rqstp, struct svc_fh *fhp, hashval = (unsigned int)hash_long(inode->i_ino, NFSD_FILE_HASH_BITS); retry: rcu_read_lock(); - nf = nfsd_file_find_locked(inode, may_flags, hashval); + nf = nfsd_file_find_locked(inode, may_flags, hashval, net); rcu_read_unlock(); if (nf) goto wait_for_construction; - new = nfsd_file_alloc(inode, may_flags, hashval); + new = nfsd_file_alloc(inode, may_flags, hashval, net); if (!new) { trace_nfsd_file_acquire(rqstp, hashval, inode, may_flags, NULL, nfserr_jukebox); @@ -766,7 +775,7 @@ nfsd_file_acquire(struct svc_rqst *rqstp, struct svc_fh *fhp, } spin_lock(&nfsd_file_hashtbl[hashval].nfb_lock); - nf = nfsd_file_find_locked(inode, may_flags, hashval); + nf = nfsd_file_find_locked(inode, may_flags, hashval, net); if (nf == NULL) goto open_file; spin_unlock(&nfsd_file_hashtbl[hashval].nfb_lock); diff --git a/fs/nfsd/filecache.h b/fs/nfsd/filecache.h index 0c0c67166b87..851d9abf54c2 100644 --- a/fs/nfsd/filecache.h +++ b/fs/nfsd/filecache.h @@ -34,6 +34,7 @@ struct nfsd_file { struct rcu_head nf_rcu; struct file *nf_file; const struct cred *nf_cred; + struct net *nf_net; #define NFSD_FILE_HASHED (0) #define NFSD_FILE_PENDING (1) #define NFSD_FILE_BREAK_READ (2) @@ -48,7 +49,7 @@ struct nfsd_file { }; int nfsd_file_cache_init(void); -void nfsd_file_cache_purge(void); +void nfsd_file_cache_purge(struct net *); void nfsd_file_cache_shutdown(void); void nfsd_file_put(struct nfsd_file *nf); struct nfsd_file *nfsd_file_get(struct nfsd_file *nf); diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index d02712ca2685..b944553c6927 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -387,6 +387,7 @@ static void nfsd_shutdown_net(struct net *net) { struct nfsd_net *nn = net_generic(net, nfsd_net_id); + nfsd_file_cache_purge(net); nfs4_state_shutdown_net(net); if (nn->lockd_up) { lockd_down(net); From patchwork Mon Aug 26 16:50:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11115115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 689AA174A for ; Mon, 26 Aug 2019 16:53:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 47E2D2173E for ; Mon, 26 Aug 2019 16:53:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Wgb2JMul" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728560AbfHZQxu (ORCPT ); Mon, 26 Aug 2019 12:53:50 -0400 Received: from mail-io1-f67.google.com ([209.85.166.67]:40960 "EHLO mail-io1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731603AbfHZQxu (ORCPT ); Mon, 26 Aug 2019 12:53:50 -0400 Received: by mail-io1-f67.google.com with SMTP id j5so38911879ioj.8 for ; Mon, 26 Aug 2019 09:53:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=74foLEL2Ws3Y+skSR9G/Et8RwoK4+ZlfQg9flyPMOkI=; b=Wgb2JMulDnlHP15m+tWAiEpNnAo3gFxTz2rCGepdQbtU6NrHCVgpBXNNi0CWySLEXB ig59LkjuUpaJZASX8JhGdbXHQwrtBTIutubxtawII4MY+tIQYUtJ6r14M+ky/ZsCbo7E zPmzVw6wvsL7+qD/GSvJMz4X+eHXG3RLmu2gFHHePaX8LnX2xlgBwMEgpC+yXjfuEz8n ExnFSK7a/8bG5rRhHkao0RtvRKwnFa5UqKuCUP1JQCvQdTa+cWdxj6bN8TyJgQRl3ON0 Ye+PxwOWGE43EA8fenoBGUGYanEGmcEejoZ+YpiqpLcboWztIKlXf0PoxRTa6gQ3zq4R MIVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=74foLEL2Ws3Y+skSR9G/Et8RwoK4+ZlfQg9flyPMOkI=; b=YzMHnyf/zC2fzpEQkUe6opY/xoymg8N2btCwAe4EOAO2kGPD69dlLivTyyw52Wj58Y F3SCbCYDh07srLdelE4M1o1Ab0HsKWXSl5ioZI3W+7o/6nD/OBEVXjUYmq2ShVd2Szp2 Nd2N729NNWdYBxuOPywx44MwSWRb2oCJX95arI7u5yelzV+aj82vEHxYLHINUo2tAEnU ea2cy+MUADq+7YGNDLkrp6OPXg+KqTB5ZO16ELroBQ4KhOBMFJXQAukaCM1kB4tR4TNj qC4xv+Zsxlimtp16lENUXB1cJV5HGF8I7/nFngbG9R3nPFrzn0mBmcNcmoaihNVnvrHa XdhQ== X-Gm-Message-State: APjAAAWlfj+jJOyQ/id6YB+vGQe04WH5cbSiOC5zx7+tbtdRwujtqqkP rDPKaqTFQxdtIwdSPPkh/QERYyAN/w== X-Google-Smtp-Source: APXvYqw1SvZd71Szl099J3Xvt5plV1SRLu8/Xvv5QAug9VI1T04uKPH14R28gtFPZ699CM/6WtlZeg== X-Received: by 2002:a05:6638:348:: with SMTP id x8mr18908688jap.31.1566838429522; Mon, 26 Aug 2019 09:53:49 -0700 (PDT) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id u24sm10613490iot.38.2019.08.26.09.53.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Aug 2019 09:53:49 -0700 (PDT) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org Subject: [PATCH 2/3] nfsd: Support the server resetting the boot verifier Date: Mon, 26 Aug 2019 12:50:20 -0400 Message-Id: <20190826165021.81075-3-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190826165021.81075-2-trond.myklebust@hammerspace.com> References: <20190826165021.81075-1-trond.myklebust@hammerspace.com> <20190826165021.81075-2-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add support to allow the server to reset the boot verifier in order to force clients to resend I/O after a timeout failure. Signed-off-by: Trond Myklebust Signed-off-by: Lance Shelton --- fs/nfsd/netns.h | 4 ++++ fs/nfsd/nfs3xdr.c | 13 +++++++++---- fs/nfsd/nfs4proc.c | 14 ++++---------- fs/nfsd/nfsctl.c | 1 + fs/nfsd/nfssvc.c | 31 ++++++++++++++++++++++++++++++- 5 files changed, 48 insertions(+), 15 deletions(-) diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h index bdfe5bcb3dcd..9a4ef815fb8c 100644 --- a/fs/nfsd/netns.h +++ b/fs/nfsd/netns.h @@ -104,6 +104,7 @@ struct nfsd_net { /* Time of server startup */ struct timespec64 nfssvc_boot; + seqlock_t boot_lock; /* * Max number of connections this nfsd container will allow. Defaults @@ -179,4 +180,7 @@ struct nfsd_net { extern void nfsd_netns_free_versions(struct nfsd_net *nn); extern unsigned int nfsd_net_id; + +void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn); +void nfsd_reset_boot_verifier(struct nfsd_net *nn); #endif /* __NFSD_NETNS_H__ */ diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c index fcf31822c74c..86e5658651f1 100644 --- a/fs/nfsd/nfs3xdr.c +++ b/fs/nfsd/nfs3xdr.c @@ -27,6 +27,7 @@ static u32 nfs3_ftypes[] = { NF3SOCK, NF3BAD, NF3LNK, NF3BAD, }; + /* * XDR functions for basic NFS types */ @@ -751,14 +752,16 @@ nfs3svc_encode_writeres(struct svc_rqst *rqstp, __be32 *p) { struct nfsd3_writeres *resp = rqstp->rq_resp; struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); + __be32 verf[2]; p = encode_wcc_data(rqstp, p, &resp->fh); if (resp->status == 0) { *p++ = htonl(resp->count); *p++ = htonl(resp->committed); /* unique identifier, y2038 overflow can be ignored */ - *p++ = htonl((u32)nn->nfssvc_boot.tv_sec); - *p++ = htonl(nn->nfssvc_boot.tv_nsec); + nfsd_copy_boot_verifier(verf, nn); + *p++ = verf[0]; + *p++ = verf[1]; } return xdr_ressize_check(rqstp, p); } @@ -1125,13 +1128,15 @@ nfs3svc_encode_commitres(struct svc_rqst *rqstp, __be32 *p) { struct nfsd3_commitres *resp = rqstp->rq_resp; struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); + __be32 verf[2]; p = encode_wcc_data(rqstp, p, &resp->fh); /* Write verifier */ if (resp->status == 0) { /* unique identifier, y2038 overflow can be ignored */ - *p++ = htonl((u32)nn->nfssvc_boot.tv_sec); - *p++ = htonl(nn->nfssvc_boot.tv_nsec); + nfsd_copy_boot_verifier(verf, nn); + *p++ = verf[0]; + *p++ = verf[1]; } return xdr_ressize_check(rqstp, p); } diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index cb51893ec1cd..4e3e77b76411 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -568,17 +568,11 @@ nfsd4_access(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, static void gen_boot_verifier(nfs4_verifier *verifier, struct net *net) { - __be32 verf[2]; - struct nfsd_net *nn = net_generic(net, nfsd_net_id); + __be32 *verf = (__be32 *)verifier->data; - /* - * This is opaque to client, so no need to byte-swap. Use - * __force to keep sparse happy. y2038 time_t overflow is - * irrelevant in this usage. - */ - verf[0] = (__force __be32)nn->nfssvc_boot.tv_sec; - verf[1] = (__force __be32)nn->nfssvc_boot.tv_nsec; - memcpy(verifier->data, verf, sizeof(verifier->data)); + BUILD_BUG_ON(2*sizeof(*verf) != sizeof(verifier->data)); + + nfsd_copy_boot_verifier(verf, net_generic(net, nfsd_net_id)); } static __be32 diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 13c548733860..c3ac1e000c4a 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -1478,6 +1478,7 @@ static __net_init int nfsd_init_net(struct net *net) atomic_set(&nn->ntf_refcnt, 0); init_waitqueue_head(&nn->ntf_wq); + seqlock_init(&nn->boot_lock); mnt = vfs_kern_mount(&nfsd_fs_type, SB_KERNMOUNT, "nfsd", NULL); if (IS_ERR(mnt)) { diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c index b944553c6927..a597fc34bc40 100644 --- a/fs/nfsd/nfssvc.c +++ b/fs/nfsd/nfssvc.c @@ -344,6 +344,35 @@ static bool nfsd_needs_lockd(struct nfsd_net *nn) return nfsd_vers(nn, 2, NFSD_TEST) || nfsd_vers(nn, 3, NFSD_TEST); } +void nfsd_copy_boot_verifier(__be32 verf[2], struct nfsd_net *nn) +{ + int seq; + + do { + read_seqbegin_or_lock(&nn->boot_lock, &seq); + /* + * This is opaque to client, so no need to byte-swap. Use + * __force to keep sparse happy. y2038 time_t overflow is + * irrelevant in this usage + */ + verf[0] = (__force __be32)nn->nfssvc_boot.tv_sec; + verf[1] = (__force __be32)nn->nfssvc_boot.tv_nsec; + } while (need_seqretry(&nn->boot_lock, seq)); + done_seqretry(&nn->boot_lock, seq); +} + +void nfsd_reset_boot_verifier_locked(struct nfsd_net *nn) +{ + ktime_get_real_ts64(&nn->nfssvc_boot); +} + +void nfsd_reset_boot_verifier(struct nfsd_net *nn) +{ + write_seqlock(&nn->boot_lock); + nfsd_reset_boot_verifier_locked(nn); + write_sequnlock(&nn->boot_lock); +} + static int nfsd_startup_net(int nrservs, struct net *net, const struct cred *cred) { struct nfsd_net *nn = net_generic(net, nfsd_net_id); @@ -596,7 +625,7 @@ int nfsd_create_serv(struct net *net) #endif } atomic_inc(&nn->ntf_refcnt); - ktime_get_real_ts64(&nn->nfssvc_boot); /* record boot time */ + nfsd_reset_boot_verifier(nn); return 0; } From patchwork Mon Aug 26 16:50:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11115117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C1FB14E5 for ; Mon, 26 Aug 2019 16:53:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5A9152173E for ; Mon, 26 Aug 2019 16:53:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qF68pKjO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732146AbfHZQxv (ORCPT ); Mon, 26 Aug 2019 12:53:51 -0400 Received: from mail-io1-f66.google.com ([209.85.166.66]:43831 "EHLO mail-io1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731603AbfHZQxv (ORCPT ); Mon, 26 Aug 2019 12:53:51 -0400 Received: by mail-io1-f66.google.com with SMTP id 18so38881425ioe.10 for ; Mon, 26 Aug 2019 09:53:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3XDL4KoWkmx9PlF4CXcLYUPo7i1ojatgLjc2RjXIZH4=; b=qF68pKjOUdYCSXpdU0kQ+I/3ayneBPAU5pxLNWbIezoPAuOg/2UuHKtbVh1LQdy5O3 4M2qJcn4VIfTAlOsQVwg7a6SQsDOyJTl4g1K06qPsKZgrO94tVDnhnbP/2rA9inUzTOg EycMJk2xK5XSta8LR6dEVxQdrQfO9m2xvjTlp5ZzAw0U+/Yu0/yUtnoenF0IV1v7gh+q hVWsQI2sQG6zbFLYL8asuaNnmYX7VhUkU6OTo3C6vvlzMHBQFN40TwCr8/VeQ+dj0q+S 59kXNlKMHsDy6I83xJrf4yQ49F6FZpfPMNLKdNxc4knCbS9z+SDnODbOF6epqVE5gOyS JRow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3XDL4KoWkmx9PlF4CXcLYUPo7i1ojatgLjc2RjXIZH4=; b=ckRm6TNlIr+Jp5ThmTAVDsSvQBOsqD+TuJKAvGDBNgxpQIpUOyGMfK2yO7c+rM8MuP AhEhJsOLkR+91WkbfhjHCB7tQWlkqssgyMtbFduF7bCj0Mui3k4tmV5JkqwECg56FBMC k5zujbMZfCc1LRifZ1ZYjnea4wXHdDxsKnjYB2lwBdI7FNinsIbC5Fw11ErXtSL6YF/P sCYUWO+sRtnagKm/vT078e9IUihFRaLJayI0DZpuuXiFyu7wtZTIm0Tza5kI+3d7r5C9 /pVHEaWrjdhlnlKExMoneF8ufMtxAQRnOZR9DeLkdTV/34zA7EwNdlvS/MTJMSiPI/1h R1ng== X-Gm-Message-State: APjAAAWK5LWVRWkmpvhpBzJRMW1ib49zVQeL139n4s/LVTeAbVL1Eh4B C7GvbHtU1lhtuhzOu1PtUEQKfYgHdA== X-Google-Smtp-Source: APXvYqzI0+0M/evVsNk+W+l8jLrcKOowsCauOhql3qqAWOhC5cB5bbijk++Rb1yCXcvNfaQa16/71g== X-Received: by 2002:a5e:c00e:: with SMTP id u14mr1114993iol.196.1566838430240; Mon, 26 Aug 2019 09:53:50 -0700 (PDT) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id u24sm10613490iot.38.2019.08.26.09.53.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Aug 2019 09:53:49 -0700 (PDT) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: "J. Bruce Fields" Cc: linux-nfs@vger.kernel.org Subject: [PATCH 3/3] nfsd: Don't garbage collect files that might contain write errors Date: Mon, 26 Aug 2019 12:50:21 -0400 Message-Id: <20190826165021.81075-4-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190826165021.81075-3-trond.myklebust@hammerspace.com> References: <20190826165021.81075-1-trond.myklebust@hammerspace.com> <20190826165021.81075-2-trond.myklebust@hammerspace.com> <20190826165021.81075-3-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org If a file may contain unstable writes that can error out, then we want to avoid garbage collecting the struct nfsd_file that may be tracking those errors. So in the garbage collector, we try to avoid collecting files that aren't clean. Furthermore, we avoid immediately kicking off the garbage collector in the case where the reference drops to zero for the case where there is a write error that is being tracked. If the file is unhashed while an error is pending, then declare a reboot, to ensure the client resends any unstable writes. Signed-off-by: Trond Myklebust --- fs/nfsd/filecache.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index d229fd3c825d..92d0998824a0 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -216,6 +216,36 @@ nfsd_file_free(struct nfsd_file *nf) return flush; } +static bool +nfsd_file_check_writeback(struct nfsd_file *nf) +{ + struct file *file = nf->nf_file; + struct address_space *mapping; + + if (!file || !(file->f_mode & FMODE_WRITE)) + return false; + mapping = file->f_mapping; + return mapping_tagged(mapping, PAGECACHE_TAG_DIRTY) || + mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK); +} + +static int +nfsd_file_check_write_error(struct nfsd_file *nf) +{ + struct file *file = nf->nf_file; + + if (!file || !(file->f_mode & FMODE_WRITE)) + return 0; + return filemap_check_wb_err(file->f_mapping, READ_ONCE(file->f_wb_err)); +} + +static bool +nfsd_file_in_use(struct nfsd_file *nf) +{ + return nfsd_file_check_writeback(nf) || + nfsd_file_check_write_error(nf); +} + static void nfsd_file_do_unhash(struct nfsd_file *nf) { @@ -223,6 +253,8 @@ nfsd_file_do_unhash(struct nfsd_file *nf) trace_nfsd_file_unhash(nf); + if (nfsd_file_check_write_error(nf)) + nfsd_reset_boot_verifier(net_generic(nf->nf_net, nfsd_net_id)); --nfsd_file_hashtbl[nf->nf_hashval].nfb_count; hlist_del_rcu(&nf->nf_node); if (!list_empty(&nf->nf_lru)) @@ -277,9 +309,10 @@ void nfsd_file_put(struct nfsd_file *nf) { bool is_hashed = test_bit(NFSD_FILE_HASHED, &nf->nf_flags) != 0; + bool unused = !nfsd_file_in_use(nf); set_bit(NFSD_FILE_REFERENCED, &nf->nf_flags); - if (nfsd_file_put_noref(nf) == 1 && is_hashed) + if (nfsd_file_put_noref(nf) == 1 && is_hashed && unused) nfsd_file_schedule_laundrette(NFSD_FILE_LAUNDRETTE_MAY_FLUSH); } @@ -345,6 +378,14 @@ nfsd_file_lru_cb(struct list_head *item, struct list_lru_one *lru, */ if (atomic_read(&nf->nf_ref) > 1) goto out_skip; + + /* + * Don't throw out files that are still undergoing I/O or + * that have uncleared errors pending. + */ + if (nfsd_file_check_writeback(nf)) + goto out_skip; + if (test_and_clear_bit(NFSD_FILE_REFERENCED, &nf->nf_flags)) goto out_rescan;