From patchwork Fri Feb 28 13:07:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412223 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD50D138D for ; Fri, 28 Feb 2020 13:09:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AFDEA2067C for ; Fri, 28 Feb 2020 13:09:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HEbENzLY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725827AbgB1NJg (ORCPT ); Fri, 28 Feb 2020 08:09:36 -0500 Received: from mail-yw1-f68.google.com ([209.85.161.68]:35858 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725861AbgB1NJf (ORCPT ); Fri, 28 Feb 2020 08:09:35 -0500 Received: by mail-yw1-f68.google.com with SMTP id y72so3220857ywg.3 for ; Fri, 28 Feb 2020 05:09:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=yjpn2eZ+qdVL730VhJKSpkgisG4rut0xZaWVnztngvs=; b=HEbENzLY8ume3OGgOBi00c20+FCIBhbRSYu+VuMaKNP18E9Vcxbyy4H9PfHHE7thdT Cr7vwJMULquNdaN11vi62EH8KkN0ULD9G9+I5PE7jiDtLGPDD5myVUw/dK1tFJj4qUwP oa9RQLSu9UP06OPhNo1hXOlJUxdwrOYx5OuFGykMIQXunWCCI5+aBUiZgRfn1fqtkULj UZ468p1AXmm1N9Uc8eMtBKf1hJGDVD5PCbEhAKcmznniZO0dzRu3UNMwOC/5TBdyBYtD g5tOFRvdngGQBZhOaIBf1NKRI90J0RPQOA1YhfoafZsFq84w9w72rdzFXQQh+f/Z1Bn0 lKJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=yjpn2eZ+qdVL730VhJKSpkgisG4rut0xZaWVnztngvs=; b=M6d/3myl4rJd4IuTSZ/Tfc0mZRF2Ldiv+dNUXnnMYrWKSLPExaAimvxbfafTWQ4T1B JHe49jdbkh9RBQSiWSuqWbgLpjqcnXqw/a6i5014DfjTlX3hO5Gk6Gz4yQeCOKqd4fRq hszWe+O28PHm240IXupMJ53F1IaEv/0Pz/l+y3oDvwMedskowkN2+LQufXr8s7EVEo4w 14cvs3J6EdJY55nbqdgh4PSvCh8atXLKGYpDrwb+WJlcxR5J5L2FgMXpOyJiwZQMjfDr omBvM+KW/S+/NNrinBqNnkciQfAY1OBmshXqlryryy5bKbibC3powY1FZUqc2ZDTOTTf IiDw== X-Gm-Message-State: APjAAAV7DIa2j9HeYop4R6IxBhFLauZLfYygEC/T8YB4lMrQ+2+gmSOb w0zxZfqPPXDtzItgu1MvUBaepkAMPQ== X-Google-Smtp-Source: APXvYqyrnz9f7Vj7XIi9rSXGm6+kbHcyP/mgot62QgYoAmbWrZtbBKwwvwtDE0IsGsLCUrgQb8jYuQ== X-Received: by 2002:a25:cf95:: with SMTP id f143mr3480602ybg.350.1582895374063; Fri, 28 Feb 2020 05:09:34 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:33 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 1/7] NFSv4: Ensure layout headers are RCU safe Date: Fri, 28 Feb 2020 08:07:19 -0500 Message-Id: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Signed-off-by: Trond Myklebust --- fs/nfs/blocklayout/blocklayout.c | 2 +- fs/nfs/filelayout/filelayout.c | 2 +- fs/nfs/flexfilelayout/flexfilelayout.c | 6 +++--- fs/nfs/pnfs.c | 12 ++++++------ fs/nfs/pnfs.h | 2 ++ 5 files changed, 13 insertions(+), 11 deletions(-) diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c index 690221747b47..d1a0e2c8b1b4 100644 --- a/fs/nfs/blocklayout/blocklayout.c +++ b/fs/nfs/blocklayout/blocklayout.c @@ -476,7 +476,7 @@ static void bl_free_layout_hdr(struct pnfs_layout_hdr *lo) err = ext_tree_remove(bl, true, 0, LLONG_MAX); WARN_ON(err); - kfree(bl); + kfree_rcu(bl, bl_layout.plh_rcu); } static struct pnfs_layout_hdr *__bl_alloc_layout_hdr(struct inode *inode, diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index c9b605f6c9cb..bd234394a87c 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1146,7 +1146,7 @@ filelayout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) static void filelayout_free_layout_hdr(struct pnfs_layout_hdr *lo) { - kfree(FILELAYOUT_FROM_HDR(lo)); + kfree_rcu(FILELAYOUT_FROM_HDR(lo), generic_hdr.plh_rcu); } static struct pnfs_ds_commit_info * diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 8b8171b48893..e7d8ae4d0cc5 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -59,14 +59,14 @@ ff_layout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) static void ff_layout_free_layout_hdr(struct pnfs_layout_hdr *lo) { + struct nfs4_flexfile_layout *ffl = FF_LAYOUT_FROM_HDR(lo); struct nfs4_ff_layout_ds_err *err, *n; - list_for_each_entry_safe(err, n, &FF_LAYOUT_FROM_HDR(lo)->error_list, - list) { + list_for_each_entry_safe(err, n, &ffl->error_list, list) { list_del(&err->list); kfree(err); } - kfree(FF_LAYOUT_FROM_HDR(lo)); + kfree_rcu(ffl, generic_hdr.plh_rcu); } static int decode_pnfs_stateid(struct xdr_stream *xdr, nfs4_stateid *stateid) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index cb99ac954688..268e7b9ff54e 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -268,11 +268,11 @@ pnfs_free_layout_hdr(struct pnfs_layout_hdr *lo) struct nfs_server *server = NFS_SERVER(lo->plh_inode); struct pnfs_layoutdriver_type *ld = server->pnfs_curr_ld; - if (!list_empty(&lo->plh_layouts)) { + if (test_and_clear_bit(NFS_LAYOUT_HASHED, &lo->plh_flags)) { struct nfs_client *clp = server->nfs_client; spin_lock(&clp->cl_lock); - list_del_init(&lo->plh_layouts); + list_del_rcu(&lo->plh_layouts); spin_unlock(&clp->cl_lock); } put_cred(lo->plh_lc_cred); @@ -784,7 +784,8 @@ pnfs_layout_bulk_destroy_byserver_locked(struct nfs_client *clp, break; inode = igrab(lo->plh_inode); if (inode != NULL) { - list_del_init(&lo->plh_layouts); + if (test_and_clear_bit(NFS_LAYOUT_HASHED, &lo->plh_flags)) + list_del_rcu(&lo->plh_layouts); if (pnfs_layout_add_bulk_destroy_list(inode, layout_list)) continue; @@ -1870,15 +1871,14 @@ static void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo) static void _add_to_server_list(struct pnfs_layout_hdr *lo, struct nfs_server *server) { - if (list_empty(&lo->plh_layouts)) { + if (!test_and_set_bit(NFS_LAYOUT_HASHED, &lo->plh_flags)) { struct nfs_client *clp = server->nfs_client; /* The lo must be on the clp list if there is any * chance of a CB_LAYOUTRECALL(FILE) coming in. */ spin_lock(&clp->cl_lock); - if (list_empty(&lo->plh_layouts)) - list_add_tail(&lo->plh_layouts, &server->layouts); + list_add_tail_rcu(&lo->plh_layouts, &server->layouts); spin_unlock(&clp->cl_lock); } } diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index cfb89d47c79d..8df9aa02d336 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -105,6 +105,7 @@ enum { NFS_LAYOUT_INVALID_STID, /* layout stateid id is invalid */ NFS_LAYOUT_FIRST_LAYOUTGET, /* Serialize first layoutget */ NFS_LAYOUT_INODE_FREEING, /* The inode is being freed */ + NFS_LAYOUT_HASHED, /* The layout visible */ }; enum layoutdriver_policy_flags { @@ -203,6 +204,7 @@ struct pnfs_layout_hdr { loff_t plh_lwb; /* last write byte for layoutcommit */ const struct cred *plh_lc_cred; /* layoutcommit cred */ struct inode *plh_inode; + struct rcu_head plh_rcu; }; struct pnfs_device { From patchwork Fri Feb 28 13:07:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412225 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 722DE1580 for ; Fri, 28 Feb 2020 13:09:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 51B27246A3 for ; Fri, 28 Feb 2020 13:09:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FHqNQPgu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725876AbgB1NJg (ORCPT ); Fri, 28 Feb 2020 08:09:36 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:37979 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725861AbgB1NJg (ORCPT ); Fri, 28 Feb 2020 08:09:36 -0500 Received: by mail-yw1-f67.google.com with SMTP id 10so3209184ywv.5 for ; Fri, 28 Feb 2020 05:09:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=aSdAmGvQ4X6dHlEIeVlLY1A3hf1FnC9He4UCG6KmM/k=; b=FHqNQPgulQZFScABd3xLMd9U5E/YcEgPdp+gKrFeG14hMBMOGZZAi9qoKq0IcNH0bV YS9Iyu3IMzjFnO89vkhL6HqrMkfAzR/i8dsMz7J9EBSGF8Jith7d15lDkALUMXfA+tpg tzIrIjUM7oLb7eemh5/kxBfutNWhhTkvfVXJXYsPSnS2R5sROyDkIz+DeoatxU8EelEP lhzMlAENqHXFL9/NSrGO/fbQ8P79CvRs3JcHHCJFEOh15+AcirTFdDpgxarmhwz7+DZR GO47tUZ0OwOFMh5D+oGv77Wht19XH7Zff8qe+XIXmH7tqOJmoJpwlgYovWyq7wGca9bB iWvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aSdAmGvQ4X6dHlEIeVlLY1A3hf1FnC9He4UCG6KmM/k=; b=kOJqILiX/GqTIJI+PZDA+AS1kyJEVqDbEWs+nvE2aBr4cs06lT2kfZPNf+U1iucqgl pKiWMTYKyqEQQcTdUeQCS2BDKOzfL2STIW36JPhS/u9aa3zG9u9FkMSkT3vZ33YLzTVZ v4vLyQGDBd7hkiYUW4qZqwYetBVempCTQLVJO0jzWS1S3oLh4mIQ97j/4Svq1Nb1ffmA n6wcVW0xhFfkuGOoB07HtNh13Qeg00TxAEM3yGJ91BR+tZE0B/RC0fJfFck9iQNpHOXo Ajepd1mk+hnclVt4CpVh81ML4pMSX8J7p6jrabjKq0A7NHWxZyIo9cyGl/oBUbTWCfhe SJyg== X-Gm-Message-State: APjAAAXA6f+hRp1vRxVZvHsVizuN6vq6D66MryaP5pNrnI5hvNdfuDgh 9xsfDZt7Q2SrMFH9T/HaahV+g9iiPg== X-Google-Smtp-Source: APXvYqwBfHDnRdVsQfgBp7ZsFYesIM9/xnXSnFfpdwBwmlBUKhOwnVJiZ0kVswMOjKbOtQHA6h/Mvg== X-Received: by 2002:a25:abef:: with SMTP id v102mr3661062ybi.269.1582895375290; Fri, 28 Feb 2020 05:09:35 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:34 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 2/7] NFSv4/pnfs: Clean up nfs_layout_find_inode() Date: Fri, 28 Feb 2020 08:07:20 -0500 Message-Id: <20200228130725.1330705-2-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Now that we can rely on just the rcu_read_lock(), remove the clp->cl_lock and clean up. Signed-off-by: Trond Myklebust --- fs/nfs/callback_proc.c | 52 +++++++++++++++++------------------------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index eb9d035451a2..97084804a953 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -121,33 +121,31 @@ __be32 nfs4_callback_recall(void *argp, void *resp, */ static struct inode *nfs_layout_find_inode_by_stateid(struct nfs_client *clp, const nfs4_stateid *stateid) + __must_hold(RCU) { struct nfs_server *server; struct inode *inode; struct pnfs_layout_hdr *lo; + rcu_read_lock(); list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) { - list_for_each_entry(lo, &server->layouts, plh_layouts) { + list_for_each_entry_rcu(lo, &server->layouts, plh_layouts) { if (!pnfs_layout_is_valid(lo)) continue; if (stateid != NULL && !nfs4_stateid_match_other(stateid, &lo->plh_stateid)) continue; + if (!nfs_sb_active(server->super)) + continue; inode = igrab(lo->plh_inode); - if (!inode) - return ERR_PTR(-EAGAIN); - if (!nfs_sb_active(inode->i_sb)) { - rcu_read_unlock(); - spin_unlock(&clp->cl_lock); - iput(inode); - spin_lock(&clp->cl_lock); - rcu_read_lock(); - return ERR_PTR(-EAGAIN); - } - return inode; + rcu_read_unlock(); + if (inode) + return inode; + nfs_sb_deactive(server->super); + return ERR_PTR(-EAGAIN); } } - + rcu_read_unlock(); return ERR_PTR(-ENOENT); } @@ -165,28 +163,25 @@ static struct inode *nfs_layout_find_inode_by_fh(struct nfs_client *clp, struct inode *inode; struct pnfs_layout_hdr *lo; + rcu_read_lock(); list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) { - list_for_each_entry(lo, &server->layouts, plh_layouts) { + list_for_each_entry_rcu(lo, &server->layouts, plh_layouts) { nfsi = NFS_I(lo->plh_inode); if (nfs_compare_fh(fh, &nfsi->fh)) continue; if (nfsi->layout != lo) continue; + if (!nfs_sb_active(server->super)) + continue; inode = igrab(lo->plh_inode); - if (!inode) - return ERR_PTR(-EAGAIN); - if (!nfs_sb_active(inode->i_sb)) { - rcu_read_unlock(); - spin_unlock(&clp->cl_lock); - iput(inode); - spin_lock(&clp->cl_lock); - rcu_read_lock(); - return ERR_PTR(-EAGAIN); - } - return inode; + rcu_read_unlock(); + if (inode) + return inode; + nfs_sb_deactive(server->super); + return ERR_PTR(-EAGAIN); } } - + rcu_read_unlock(); return ERR_PTR(-ENOENT); } @@ -196,14 +191,9 @@ static struct inode *nfs_layout_find_inode(struct nfs_client *clp, { struct inode *inode; - spin_lock(&clp->cl_lock); - rcu_read_lock(); inode = nfs_layout_find_inode_by_stateid(clp, stateid); if (inode == ERR_PTR(-ENOENT)) inode = nfs_layout_find_inode_by_fh(clp, fh); - rcu_read_unlock(); - spin_unlock(&clp->cl_lock); - return inode; } From patchwork Fri Feb 28 13:07:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 980F3138D for ; Fri, 28 Feb 2020 13:09:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 78013246A3 for ; Fri, 28 Feb 2020 13:09:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ejGqG8qu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725900AbgB1NJi (ORCPT ); Fri, 28 Feb 2020 08:09:38 -0500 Received: from mail-yw1-f65.google.com ([209.85.161.65]:34597 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725861AbgB1NJh (ORCPT ); Fri, 28 Feb 2020 08:09:37 -0500 Received: by mail-yw1-f65.google.com with SMTP id b186so3236743ywc.1 for ; Fri, 28 Feb 2020 05:09:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=R6+NgKOvGU7WnxAOSrWHkptT277VM35k/++AgcgpjME=; b=ejGqG8qu+KFADf9DvCM5VGMqrq18yE3cNed8IlmPuXO6iu3xbs615NM+K3aqb1Nga5 dR2YJPOjfRlL2vi3IOTl0FqV3Cjh2Pb3TdBnU9nAOajsJLaRqo9XONlcFyCqp53d3uXK RtsyrlVoFLBLfuvZtUc8IvR4iZK90FEHNQ7khYHOnI9m+yrO30Mt6ZSNVbGeZfJsrcSD aXnIS+wTWLo8sxr9bUML3jNkEiFtXiHWHLGDWqUigr5pk+2xpM0QkxFhC9ZnM6z1OgIz zyE5W+js+BD7aYHEsdqwVo0ZbvTmH3ZcEdlYwFxs0R7l7SP1dL3f5h3/24//c6Ei5KM5 o5Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R6+NgKOvGU7WnxAOSrWHkptT277VM35k/++AgcgpjME=; b=maC2sqsA7d/OpyECUnK6+gGi+t19zEq4GAtFs9st5w2V0M8aMHwJmtYm3kVpCV4W4k lN2EgZm1W+Tl40vsYF4KHV1deTNXLQ48JsnwQhZx3vap3e0an76nbrAytF7cbDc0TUCW jbYnTFM3u0RhOfeke8Jle+ekFWAOWPo0xxSYUWNw6yhHwm2HSHTQIQcrT3aHdS6UsnV4 6lH0CLEF/Dm6VaaUDm646yUglkxzTdc+BJRXfL+8IJFYe55LGTA0+F97sxQujCnysvCr SZ+D8ked1IYKogHTqjeBqOJic9NkLY0i9G0JkGivJueaSgImvT6sKat7QE2bNi5KPEoQ eEUw== X-Gm-Message-State: APjAAAUuB6GWVsr34TZReP+iHpDvEV+p2Vnu4RHrCn665pqgmGAJccJI NNZAZ7OnOtfD3LkJNFqap9D2Gt6aeA== X-Google-Smtp-Source: APXvYqz/CfVZZcrv8h1cNqp9JMyxDiY547TLziXNNGjiC314CuGuJLfCuy2UwGABE58sUSQQ+0pZIw== X-Received: by 2002:a25:694c:: with SMTP id e73mr268710ybc.198.1582895376267; Fri, 28 Feb 2020 05:09:36 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:35 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 3/7] NFS: Add a helper nfs_client_for_each_server() Date: Fri, 28 Feb 2020 08:07:21 -0500 Message-Id: <20200228130725.1330705-3-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-2-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> <20200228130725.1330705-2-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add a helper nfs_client_for_each_server() to iterate through all the filesystems that are attached to a struct nfs_client, and apply a function to all the active ones. Signed-off-by: Trond Myklebust --- fs/nfs/internal.h | 4 +++- fs/nfs/super.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index f80c47d5ff27..3b6fa9edc9b5 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -417,7 +417,9 @@ extern int __init register_nfs_fs(void); extern void __exit unregister_nfs_fs(void); extern bool nfs_sb_active(struct super_block *sb); extern void nfs_sb_deactive(struct super_block *sb); - +extern int nfs_client_for_each_server(struct nfs_client *clp, + int (*fn)(struct nfs_server *, void *), + void *data); /* io.c */ extern void nfs_start_io_read(struct inode *inode); extern void nfs_end_io_read(struct inode *inode); diff --git a/fs/nfs/super.c b/fs/nfs/super.c index dada09b391c6..eb3a85492396 100644 --- a/fs/nfs/super.c +++ b/fs/nfs/super.c @@ -176,6 +176,41 @@ void nfs_sb_deactive(struct super_block *sb) } EXPORT_SYMBOL_GPL(nfs_sb_deactive); +static int __nfs_list_for_each_server(struct list_head *head, + int (*fn)(struct nfs_server *, void *), + void *data) +{ + struct nfs_server *server, *last = NULL; + int ret = 0; + + rcu_read_lock(); + list_for_each_entry_rcu(server, head, client_link) { + if (!nfs_sb_active(server->super)) + continue; + rcu_read_unlock(); + if (last) + nfs_sb_deactive(last->super); + last = server; + ret = fn(server, data); + if (ret) + goto out; + rcu_read_lock(); + } + rcu_read_unlock(); +out: + if (last) + nfs_sb_deactive(last->super); + return ret; +} + +int nfs_client_for_each_server(struct nfs_client *clp, + int (*fn)(struct nfs_server *, void *), + void *data) +{ + return __nfs_list_for_each_server(&clp->cl_superblocks, fn, data); +} +EXPORT_SYMBOL_GPL(nfs_client_for_each_server); + /* * Deliver file system statistics to userspace */ From patchwork Fri Feb 28 13:07:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412229 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D612A138D for ; Fri, 28 Feb 2020 13:09:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AD699246A3 for ; Fri, 28 Feb 2020 13:09:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="uJFSHV+I" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725911AbgB1NJj (ORCPT ); Fri, 28 Feb 2020 08:09:39 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:37986 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725805AbgB1NJj (ORCPT ); Fri, 28 Feb 2020 08:09:39 -0500 Received: by mail-yw1-f67.google.com with SMTP id 10so3209325ywv.5 for ; Fri, 28 Feb 2020 05:09:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HQkTkYY4F8CDCMmaswKDG75K6tbTxn9bHXmgJuOl0wQ=; b=uJFSHV+IxCpTrZaoaQpeJRDy4o/0r7AZPr+9PgVT7ud1cvpGTgMnxBvXlHIGvjaAKD aNvuxgv6N8HvIXDOjyfeYdL6FjGCVfMGeY/rguWUQuowmpva7oTpwu2jn6UJV+uw4p7O fDHYpdoCM+UrJQi2e9pAxMStUIShO2lTNONsiWpZrNT9WafIwroXINZ6pB741Kiizkni LGJtztkan8qfF5s+AHc4utYrP0zwfLKPsQ6Fs3FndrTGKi5wi+gC27hXblm2VFuvpDIw 2vpkorQDYmBIUIgdZqgyc7nKzwi+xHM6WLxdnEPUdAPiXRdgq2M2QaA8kKUjf42tc1VT MuYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HQkTkYY4F8CDCMmaswKDG75K6tbTxn9bHXmgJuOl0wQ=; b=bHXeFjcw+RdP8AGGZHpEHqSAzQHc4+XZuP6fLRS62lH7VjKvjBAJeRXDjw035GEPca suUlOCASVqFihCD0qpT7TC5s2m1mgNLXUI2ejCCUr9ivmEHM4cYhIkHfgad+n/IWDRlu GnvCPiUgJdViLUJf6tcHbVi2pp3n5VeSdxXH/ixMlIqOAx/hDd16Au3OJuv6wDiDIzci qWW6rDXAv28V8gGKUhNAhFLqBooQCsU2fdmXyPfMcxpv5gcraNt8ER0cwP+QUHf59tCx neQRrOTxfq0NyAcgae0nWaBGYiqRJ7DJUhvmZqjG+p698E1FhlRnY/jpQ5DblKnd2vlf 0KdA== X-Gm-Message-State: APjAAAVw6/e0WCdwMlR1cBDEfgJ3fShdbelee83ZxZ3fl2QHheT8l3Kf YIG/6ALQjZjD6ITDPsYacUG+bTFriw== X-Google-Smtp-Source: APXvYqwKPYHVFUUiDmmhVkAzpD0zKdZEDl7ZYVktC2rhuebuJENdc1lYpgdYZ6peMtFDuPcAstwikg== X-Received: by 2002:a25:3d44:: with SMTP id k65mr3350589yba.390.1582895377368; Fri, 28 Feb 2020 05:09:37 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:36 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 4/7] NFSv4: Clean up nfs_client_return_marked_delegations() Date: Fri, 28 Feb 2020 08:07:22 -0500 Message-Id: <20200228130725.1330705-4-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-3-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> <20200228130725.1330705-2-trond.myklebust@hammerspace.com> <20200228130725.1330705-3-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Convert it to use the nfs_client_for_each_server() helper, and make it more efficient by skipping delegations for inodes we know are in the process of being freed. Also improve the efficiency of the cursor by skipping delegations that are being freed. Signed-off-by: Trond Myklebust --- fs/nfs/delegation.c | 129 +++++++++++++++++++++----------------------- 1 file changed, 60 insertions(+), 69 deletions(-) diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c index 509b7235b132..19f66d3e58e8 100644 --- a/fs/nfs/delegation.c +++ b/fs/nfs/delegation.c @@ -563,21 +563,11 @@ static bool nfs_delegation_need_return(struct nfs_delegation *delegation) return ret; } -/** - * nfs_client_return_marked_delegations - return previously marked delegations - * @clp: nfs_client to process - * - * Note that this function is designed to be called by the state - * manager thread. For this reason, it cannot flush the dirty data, - * since that could deadlock in case of a state recovery error. - * - * Returns zero on success, or a negative errno value. - */ -int nfs_client_return_marked_delegations(struct nfs_client *clp) +static int nfs_server_return_marked_delegations(struct nfs_server *server, + void __always_unused *data) { struct nfs_delegation *delegation; struct nfs_delegation *prev; - struct nfs_server *server; struct inode *inode; struct inode *place_holder = NULL; struct nfs_delegation *place_holder_deleg = NULL; @@ -587,78 +577,79 @@ int nfs_client_return_marked_delegations(struct nfs_client *clp) /* * To avoid quadratic looping we hold a reference * to an inode place_holder. Each time we restart, we - * list nfs_servers from the server of that inode, and - * delegation in the server from the delegations of that - * inode. + * list delegation in the server from the delegations + * of that inode. * prev is an RCU-protected pointer to a delegation which * wasn't marked for return and might be a good choice for * the next place_holder. */ - rcu_read_lock(); prev = NULL; + delegation = NULL; + rcu_read_lock(); if (place_holder) - server = NFS_SERVER(place_holder); - else - server = list_entry_rcu(clp->cl_superblocks.next, - struct nfs_server, client_link); - list_for_each_entry_from_rcu(server, &clp->cl_superblocks, client_link) { - delegation = NULL; - if (place_holder && server == NFS_SERVER(place_holder)) - delegation = rcu_dereference(NFS_I(place_holder)->delegation); - if (!delegation || delegation != place_holder_deleg) - delegation = list_entry_rcu(server->delegations.next, - struct nfs_delegation, super_list); - list_for_each_entry_from_rcu(delegation, &server->delegations, super_list) { - struct inode *to_put = NULL; - - if (!nfs_delegation_need_return(delegation)) { + delegation = rcu_dereference(NFS_I(place_holder)->delegation); + if (!delegation || delegation != place_holder_deleg) + delegation = list_entry_rcu(server->delegations.next, + struct nfs_delegation, super_list); + list_for_each_entry_from_rcu(delegation, &server->delegations, super_list) { + struct inode *to_put = NULL; + + if (test_bit(NFS_DELEGATION_INODE_FREEING, &delegation->flags)) + continue; + if (!nfs_delegation_need_return(delegation)) { + if (nfs4_is_valid_delegation(delegation, 0)) prev = delegation; - continue; - } - if (!nfs_sb_active(server->super)) - break; /* continue in outer loop */ - - if (prev) { - struct inode *tmp; + continue; + } - tmp = nfs_delegation_grab_inode(prev); - if (tmp) { - to_put = place_holder; - place_holder = tmp; - place_holder_deleg = prev; - } + if (prev) { + struct inode *tmp = nfs_delegation_grab_inode(prev); + if (tmp) { + to_put = place_holder; + place_holder = tmp; + place_holder_deleg = prev; } + } - inode = nfs_delegation_grab_inode(delegation); - if (inode == NULL) { - rcu_read_unlock(); - if (to_put) - iput(to_put); - nfs_sb_deactive(server->super); - goto restart; - } - delegation = nfs_start_delegation_return_locked(NFS_I(inode)); + inode = nfs_delegation_grab_inode(delegation); + if (inode == NULL) { rcu_read_unlock(); + iput(to_put); + goto restart; + } + delegation = nfs_start_delegation_return_locked(NFS_I(inode)); + rcu_read_unlock(); - if (to_put) - iput(to_put); + iput(to_put); - err = nfs_end_delegation_return(inode, delegation, 0); - iput(inode); - nfs_sb_deactive(server->super); - cond_resched(); - if (!err) - goto restart; - set_bit(NFS4CLNT_DELEGRETURN, &clp->cl_state); - if (place_holder) - iput(place_holder); - return err; - } + err = nfs_end_delegation_return(inode, delegation, 0); + iput(inode); + cond_resched(); + if (!err) + goto restart; + set_bit(NFS4CLNT_DELEGRETURN, &server->nfs_client->cl_state); + goto out; } rcu_read_unlock(); - if (place_holder) - iput(place_holder); - return 0; +out: + iput(place_holder); + return err; +} + +/** + * nfs_client_return_marked_delegations - return previously marked delegations + * @clp: nfs_client to process + * + * Note that this function is designed to be called by the state + * manager thread. For this reason, it cannot flush the dirty data, + * since that could deadlock in case of a state recovery error. + * + * Returns zero on success, or a negative errno value. + */ +int nfs_client_return_marked_delegations(struct nfs_client *clp) +{ + return nfs_client_for_each_server(clp, + nfs_server_return_marked_delegations, NULL); } /** From patchwork Fri Feb 28 13:07:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412231 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 818F1138D for ; Fri, 28 Feb 2020 13:09:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 61D23246A3 for ; Fri, 28 Feb 2020 13:09:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FhsxDHbq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725805AbgB1NJj (ORCPT ); Fri, 28 Feb 2020 08:09:39 -0500 Received: from mail-yw1-f68.google.com ([209.85.161.68]:35867 "EHLO mail-yw1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbgB1NJj (ORCPT ); Fri, 28 Feb 2020 08:09:39 -0500 Received: by mail-yw1-f68.google.com with SMTP id y72so3221063ywg.3 for ; Fri, 28 Feb 2020 05:09:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mQDvmIQpdISPBTLKH7Zwt20QFU0Xr0bFsaEwty4sRlU=; b=FhsxDHbqA5CkSuHr1h3OgwK/xCZ2pp7KZDONzT5CT+0T1gulhWPBSgnn4rfCFhja0p mFlD92lbE88bwqhMUdkigK0EALEsuGLBVYjYCTvNECIMLuvyLkoT1QHetVgkSKXN3EzE sToCm2sx/41WRr6PTPh+Qq/waiETgSlHe+jIcWkGOsjeSiHFy9WKQtpXcJ815SLomE4J xiZqsYA7UDu1z/vUPyz3xZCy/QL6bOmLTKOZ4ZNZ1oVwqukYYqXAqEbmcXMJZdxGyFPZ QTdVtSs8oFlzrSC3hJuEGauBc7ffhAVNHw6zSDct4QFG7lvLjSp2Xmra6SnwG9g24ME+ H8YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQDvmIQpdISPBTLKH7Zwt20QFU0Xr0bFsaEwty4sRlU=; b=dtNCks40DaHhwuaP+V/un1VEajEES01/NS7vi1GEXCM3C8vrs/DdeA6VYDzaLS9NFL 7aPSqHIWbgVXlVJxovSQxBSPEUmU+6DnPo3OW9VOw7j0CMk6FoWgzgnEyGtLB25riEhu DX3RjoqELHGejtWZLxNhHOLvZrv+g9TaSWJuE27m6GndiZKHyz/RNRpMJO28O2wWhtrQ UQOgU4e599H0wSsOajZdZ7I0ubuTvepDtC9yzKmZn03IDZFFrlHvB1aTAoHx9TOGmHNl YGdS+v86RQ8y92qbwBnqwVKtM6ics73cyZqQDJVt0tEsYiYzClOoYmdWopkguTuLxqdg HeQg== X-Gm-Message-State: APjAAAUFosc8u0JxOPOttxq0/l2/1DOrOZv0uQyTBLJ978hf7TQpVfqL y2P7eFUlVl/mkxArcHAwoG6bt18hCg== X-Google-Smtp-Source: APXvYqzNh6YC1beQR9NR7qZdmvQ8IjNC0Rc6N5I7s9OU1NeQ8G9KKyU8YArdpDx2IsH85R4bWSDuHQ== X-Received: by 2002:a25:6ac1:: with SMTP id f184mr3408679ybc.469.1582895378330; Fri, 28 Feb 2020 05:09:38 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:37 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 5/7] NFSv4: Clean up nfs_delegation_reap_unclaimed() Date: Fri, 28 Feb 2020 08:07:23 -0500 Message-Id: <20200228130725.1330705-5-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-4-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> <20200228130725.1330705-2-trond.myklebust@hammerspace.com> <20200228130725.1330705-3-trond.myklebust@hammerspace.com> <20200228130725.1330705-4-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Convert nfs_delegation_reap_unclaimed() to use nfs_client_for_each_server() for efficiency. Signed-off-by: Trond Myklebust --- fs/nfs/delegation.c | 76 ++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 39 deletions(-) diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c index 19f66d3e58e8..cb03ba99ae51 100644 --- a/fs/nfs/delegation.c +++ b/fs/nfs/delegation.c @@ -1092,53 +1092,51 @@ void nfs_delegation_mark_reclaim(struct nfs_client *clp) rcu_read_unlock(); } -/** - * nfs_delegation_reap_unclaimed - reap unclaimed delegations after reboot recovery is done - * @clp: nfs_client to process - * - */ -void nfs_delegation_reap_unclaimed(struct nfs_client *clp) +static int nfs_server_reap_unclaimed_delegations(struct nfs_server *server, + void __always_unused *data) { struct nfs_delegation *delegation; - struct nfs_server *server; struct inode *inode; - restart: rcu_read_lock(); - list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) { - list_for_each_entry_rcu(delegation, &server->delegations, - super_list) { - if (test_bit(NFS_DELEGATION_INODE_FREEING, - &delegation->flags) || - test_bit(NFS_DELEGATION_RETURNING, - &delegation->flags) || - test_bit(NFS_DELEGATION_NEED_RECLAIM, - &delegation->flags) == 0) - continue; - if (!nfs_sb_active(server->super)) - break; /* continue in outer loop */ - inode = nfs_delegation_grab_inode(delegation); - if (inode == NULL) { - rcu_read_unlock(); - nfs_sb_deactive(server->super); - goto restart; - } - delegation = nfs_start_delegation_return_locked(NFS_I(inode)); - rcu_read_unlock(); - if (delegation != NULL) { - if (nfs_detach_delegation(NFS_I(inode), delegation, - server) != NULL) - nfs_free_delegation(delegation); - /* Match nfs_start_delegation_return_locked */ - nfs_put_delegation(delegation); - } - iput(inode); - nfs_sb_deactive(server->super); - cond_resched(); - goto restart; +restart_locked: + list_for_each_entry_rcu(delegation, &server->delegations, super_list) { + if (test_bit(NFS_DELEGATION_INODE_FREEING, + &delegation->flags) || + test_bit(NFS_DELEGATION_RETURNING, + &delegation->flags) || + test_bit(NFS_DELEGATION_NEED_RECLAIM, + &delegation->flags) == 0) + continue; + inode = nfs_delegation_grab_inode(delegation); + if (inode == NULL) + goto restart_locked; + delegation = nfs_start_delegation_return_locked(NFS_I(inode)); + rcu_read_unlock(); + if (delegation != NULL) { + if (nfs_detach_delegation(NFS_I(inode), delegation, + server) != NULL) + nfs_free_delegation(delegation); + /* Match nfs_start_delegation_return_locked */ + nfs_put_delegation(delegation); } + iput(inode); + cond_resched(); + goto restart; } rcu_read_unlock(); + return 0; +} + +/** + * nfs_delegation_reap_unclaimed - reap unclaimed delegations after reboot recovery is done + * @clp: nfs_client to process + * + */ +void nfs_delegation_reap_unclaimed(struct nfs_client *clp) +{ + nfs_client_for_each_server(clp, nfs_server_reap_unclaimed_delegations, + NULL); } static inline bool nfs4_server_rebooted(const struct nfs_client *clp) From patchwork Fri Feb 28 13:07:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6EAB14B4 for ; Fri, 28 Feb 2020 13:09:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 95F382067C for ; Fri, 28 Feb 2020 13:09:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="tt+/yDMf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725906AbgB1NJl (ORCPT ); Fri, 28 Feb 2020 08:09:41 -0500 Received: from mail-yw1-f67.google.com ([209.85.161.67]:33481 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725861AbgB1NJl (ORCPT ); Fri, 28 Feb 2020 08:09:41 -0500 Received: by mail-yw1-f67.google.com with SMTP id j186so3243436ywe.0 for ; Fri, 28 Feb 2020 05:09:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cqZyln7uV8TFcW9bdiJi2KV/o+kRCqQ/8cTYkZ0Z8tU=; b=tt+/yDMfHaR7GjaB9VV58+ni6ImtTpzDYd4cSKKJXgGJ6e3r9M2jUVfVx3Ug4F05UH /rltE5Ejuy4l0t+1NOT0cRQg5ysZavgxuMGEaeaBtqdS2AuRKPe/atQ5y60f/SPL31Uq SX+XJ5+UejqcbSOblQakNAGZyVWcjrU7AaYT/AhKJD03HyhZ4MIevu7H/MpfUdlSZPEK J7PjyNs4SucZZLH2SIYD23qqHXhltEqyIcNmB7VNo3R+QFc500MTv12HbXcwt7UTO7Co xW7a5zduC04EnSUxuXSHInr9sE1H5ruL/eSnqZFh8iUki7vxMeIajip3fJFJDzIUEKTl YjYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cqZyln7uV8TFcW9bdiJi2KV/o+kRCqQ/8cTYkZ0Z8tU=; b=swOJ+9gZe/c5a/FwHaPFFsUCd/6UZGdF9LtEn89N39MkZ5KbFfI/umf69mk4CBPH+I o8Bcx87I8LdYY/J9t69khF0fYG4SAD3SM04UYIEswzhfqWqTYnYhIQ7B7iYJLJQ4SgUO 91YHDUyJE74+jjj6ezf6uOFjuEvGIIlzspOe28dILi2uT2u99qPja839fy+oQS8/grr0 9GYO9l5JjHMT8KWaf43UqCaUWFduQJOjJ1apC3URYNDDIazziSbyWZ3Oi2KS5CpuxJtJ sa1WubVIRfqJqm30CSZTLaledHLY72JflART6Jxy/5rxtiGPlcEWmBOM0cKMqrQTwLWF 18CA== X-Gm-Message-State: APjAAAXoTNjQ2UxWPViK+PaQuwTqYQll90u33ZRP/LX+oUYvq+DzGdXz 49x9ZvHVWdGFBLZgVh7hdac664JmSw== X-Google-Smtp-Source: APXvYqykT7C1rDTkz+frNo5VwyFvEzNnXc0EJFKnpVt/d8E7HXX7oZk0qKagRnm2vQ9RWMZDBB9mHA== X-Received: by 2002:a25:d20f:: with SMTP id j15mr3678135ybg.74.1582895379332; Fri, 28 Feb 2020 05:09:39 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:38 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 6/7] NFSv4: Clean up nfs_delegation_reap_expired() Date: Fri, 28 Feb 2020 08:07:24 -0500 Message-Id: <20200228130725.1330705-6-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-5-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> <20200228130725.1330705-2-trond.myklebust@hammerspace.com> <20200228130725.1330705-3-trond.myklebust@hammerspace.com> <20200228130725.1330705-4-trond.myklebust@hammerspace.com> <20200228130725.1330705-5-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Convert to use nfs_client_for_each_server() for efficiency. Signed-off-by: Trond Myklebust --- fs/nfs/delegation.c | 83 ++++++++++++++++++++++----------------------- 1 file changed, 40 insertions(+), 43 deletions(-) diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c index cb03ba99ae51..01974f17afc9 100644 --- a/fs/nfs/delegation.c +++ b/fs/nfs/delegation.c @@ -1222,62 +1222,59 @@ nfs_delegation_test_free_expired(struct inode *inode, nfs_remove_bad_delegation(inode, stateid); } -/** - * nfs_reap_expired_delegations - reap expired delegations - * @clp: nfs_client to process - * - * Iterates through all the delegations associated with this server and - * checks if they have may have been revoked. This function is usually - * expected to be called in cases where the server may have lost its - * lease. - */ -void nfs_reap_expired_delegations(struct nfs_client *clp) +static int nfs_server_reap_expired_delegations(struct nfs_server *server, + void __always_unused *data) { struct nfs_delegation *delegation; - struct nfs_server *server; struct inode *inode; const struct cred *cred; nfs4_stateid stateid; - restart: rcu_read_lock(); - list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) { - list_for_each_entry_rcu(delegation, &server->delegations, - super_list) { - if (test_bit(NFS_DELEGATION_INODE_FREEING, - &delegation->flags) || - test_bit(NFS_DELEGATION_RETURNING, - &delegation->flags) || - test_bit(NFS_DELEGATION_TEST_EXPIRED, - &delegation->flags) == 0) - continue; - if (!nfs_sb_active(server->super)) - break; /* continue in outer loop */ - inode = nfs_delegation_grab_inode(delegation); - if (inode == NULL) { - rcu_read_unlock(); - nfs_sb_deactive(server->super); - goto restart; - } - cred = get_cred_rcu(delegation->cred); - nfs4_stateid_copy(&stateid, &delegation->stateid); - clear_bit(NFS_DELEGATION_TEST_EXPIRED, &delegation->flags); - rcu_read_unlock(); - nfs_delegation_test_free_expired(inode, &stateid, cred); - put_cred(cred); - if (nfs4_server_rebooted(clp)) { - nfs_inode_mark_test_expired_delegation(server,inode); - iput(inode); - nfs_sb_deactive(server->super); - return; - } +restart_locked: + list_for_each_entry_rcu(delegation, &server->delegations, super_list) { + if (test_bit(NFS_DELEGATION_INODE_FREEING, + &delegation->flags) || + test_bit(NFS_DELEGATION_RETURNING, + &delegation->flags) || + test_bit(NFS_DELEGATION_TEST_EXPIRED, + &delegation->flags) == 0) + continue; + inode = nfs_delegation_grab_inode(delegation); + if (inode == NULL) + goto restart_locked; + cred = get_cred_rcu(delegation->cred); + nfs4_stateid_copy(&stateid, &delegation->stateid); + clear_bit(NFS_DELEGATION_TEST_EXPIRED, &delegation->flags); + rcu_read_unlock(); + nfs_delegation_test_free_expired(inode, &stateid, cred); + put_cred(cred); + if (!nfs4_server_rebooted(server->nfs_client)) { iput(inode); - nfs_sb_deactive(server->super); cond_resched(); goto restart; } + nfs_inode_mark_test_expired_delegation(server,inode); + iput(inode); + return -EAGAIN; } rcu_read_unlock(); + return 0; +} + +/** + * nfs_reap_expired_delegations - reap expired delegations + * @clp: nfs_client to process + * + * Iterates through all the delegations associated with this server and + * checks if they have may have been revoked. This function is usually + * expected to be called in cases where the server may have lost its + * lease. + */ +void nfs_reap_expired_delegations(struct nfs_client *clp) +{ + nfs_client_for_each_server(clp, nfs_server_reap_expired_delegations, + NULL); } void nfs_inode_find_delegation_state_and_recover(struct inode *inode, From patchwork Fri Feb 28 13:07:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11412235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98E4614B4 for ; Fri, 28 Feb 2020 13:09:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6F081246A3 for ; Fri, 28 Feb 2020 13:09:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jNLS1dWw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726016AbgB1NJm (ORCPT ); Fri, 28 Feb 2020 08:09:42 -0500 Received: from mail-yw1-f65.google.com ([209.85.161.65]:34610 "EHLO mail-yw1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725861AbgB1NJm (ORCPT ); Fri, 28 Feb 2020 08:09:42 -0500 Received: by mail-yw1-f65.google.com with SMTP id b186so3236965ywc.1 for ; Fri, 28 Feb 2020 05:09:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=sGnPE/JUP7OhzF7gIyYz62C2odtn3cxa6PMyuttqqhI=; b=jNLS1dWwz/ulkL/qiRcJHrx/EjpqII6ZNt+/NHTs7Dih4MyWdwCFN8fYDMX71B8C4c ccrVP4CT6nUZD7IyYBshOJhhfDCStevWlCZAhzJJcLitPnzYP8OiYYUMwEr1fp4vU1F7 mRccXYTcPv8hLgTkSm++wgpy84MFB03Ugml36cujbcngKWcZs3Xm5syX58tbB7oBYnG7 qljtbmbAK0/Np3HZ0QBpeomZk+Dl7Ra0aj0yXHYSCnbl2GnP3fqV6GrIlK4PdK3v6gHG P6d2R9Ev/yYDFdgTj/OqUfauedUq8u+b05nCy9iPPxRI/vuO+vbuwNH1nY+Vx7BsyiN3 4dgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sGnPE/JUP7OhzF7gIyYz62C2odtn3cxa6PMyuttqqhI=; b=n6OtVY6dZDkOc3DxW+BV2p76GDu9nl1iiCQun9Pum4MuiePjTXRsXIpTd8mPgTVqD2 1Lgy4rDrClVc+Ti85JpScHyNBkcCPOOs4JEuO8uGE55tO/2Fnna1D8a48FTGbUPJE508 vVUnD6ehYPGcnwFq+rOK6CjNiceutzcrlVJOTjWW/5vDfv3KNKFer2UXPMfknqOmQiEK ivLRjDeit3eJX/ZDPw+CM7CnA6GhGq5l99D8hD3qeT8L1w7NEsUVd2KOX5kWpkZ2Nr77 RYAb/Kup9EEzjkB88uLo3NROrCr2e7UlJX/+ByWaHMIcuYA2arHr1/Ym3q0YzuBuiRlH GeWw== X-Gm-Message-State: APjAAAVzVcjximGuyv+crrx4fFNa/gfCuzDUdgC25b0/2+xXpLE3sDar vPgFlRjzN7M3Fupfyhb5wIcPNwbf2Q== X-Google-Smtp-Source: APXvYqxPhAGMa2pok2XxOaVZLcvgTUAvFo2NVwG8h1TIfF9j1vBZwWGJvyCbqdCGVO3FemjyKxnztA== X-Received: by 2002:a81:5e09:: with SMTP id s9mr4514215ywb.348.1582895380689; Fri, 28 Feb 2020 05:09:40 -0800 (PST) Received: from localhost.localdomain (c-68-40-189-247.hsd1.mi.comcast.net. [68.40.189.247]) by smtp.gmail.com with ESMTPSA id j23sm3925150ywb.93.2020.02.28.05.09.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Feb 2020 05:09:39 -0800 (PST) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH 7/7] NFSv4: Add support for CB_RECALL_ANY for flexfiles layouts Date: Fri, 28 Feb 2020 08:07:25 -0500 Message-Id: <20200228130725.1330705-7-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200228130725.1330705-6-trond.myklebust@hammerspace.com> References: <20200228130725.1330705-1-trond.myklebust@hammerspace.com> <20200228130725.1330705-2-trond.myklebust@hammerspace.com> <20200228130725.1330705-3-trond.myklebust@hammerspace.com> <20200228130725.1330705-4-trond.myklebust@hammerspace.com> <20200228130725.1330705-5-trond.myklebust@hammerspace.com> <20200228130725.1330705-6-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org When we receive a CB_RECALL_ANY that asks us to return flexfiles layouts, we iterate through all the layouts and look at whether or not there are active open file descriptors that might need them for I/O. If there are no such descriptors, we return the layouts. Signed-off-by: Trond Myklebust --- fs/nfs/callback.h | 4 +- fs/nfs/callback_proc.c | 13 ++++ fs/nfs/nfs4_fs.h | 4 +- fs/nfs/nfs4state.c | 24 ++++++- fs/nfs/nfs4trace.h | 8 ++- fs/nfs/pnfs.c | 148 +++++++++++++++++++++++++++++++++++++---- fs/nfs/pnfs.h | 3 + 7 files changed, 186 insertions(+), 18 deletions(-) diff --git a/fs/nfs/callback.h b/fs/nfs/callback.h index 549350259840..6a2033131c06 100644 --- a/fs/nfs/callback.h +++ b/fs/nfs/callback.h @@ -127,7 +127,9 @@ extern __be32 nfs4_callback_sequence(void *argp, void *resp, #define RCA4_TYPE_MASK_OBJ_LAYOUT_MAX 9 #define RCA4_TYPE_MASK_OTHER_LAYOUT_MIN 12 #define RCA4_TYPE_MASK_OTHER_LAYOUT_MAX 15 -#define RCA4_TYPE_MASK_ALL 0xf31f +#define PNFS_FF_RCA4_TYPE_MASK_READ 16 +#define PNFS_FF_RCA4_TYPE_MASK_RW 17 +#define RCA4_TYPE_MASK_ALL 0x3f31f struct cb_recallanyargs { uint32_t craa_objs_to_keep; diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index 97084804a953..e61dbc9b86ae 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -597,6 +597,7 @@ __be32 nfs4_callback_recallany(void *argp, void *resp, struct cb_recallanyargs *args = argp; __be32 status; fmode_t flags = 0; + bool schedule_manager = false; status = cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION); if (!cps->clp) /* set in cb_sequence */ @@ -619,6 +620,18 @@ __be32 nfs4_callback_recallany(void *argp, void *resp, if (args->craa_type_mask & BIT(RCA4_TYPE_MASK_FILE_LAYOUT)) pnfs_recall_all_layouts(cps->clp); + + if (args->craa_type_mask & BIT(PNFS_FF_RCA4_TYPE_MASK_READ)) { + set_bit(NFS4CLNT_RECALL_ANY_LAYOUT_READ, &cps->clp->cl_state); + schedule_manager = true; + } + if (args->craa_type_mask & BIT(PNFS_FF_RCA4_TYPE_MASK_RW)) { + set_bit(NFS4CLNT_RECALL_ANY_LAYOUT_RW, &cps->clp->cl_state); + schedule_manager = true; + } + if (schedule_manager) + nfs4_schedule_state_manager(cps->clp); + out: dprintk("%s: exit with status = %d\n", __func__, ntohl(status)); return status; diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h index 8be1ba7c62bb..2b7f6dcd2eb8 100644 --- a/fs/nfs/nfs4_fs.h +++ b/fs/nfs/nfs4_fs.h @@ -42,7 +42,9 @@ enum nfs4_client_state { NFS4CLNT_LEASE_MOVED, NFS4CLNT_DELEGATION_EXPIRED, NFS4CLNT_RUN_MANAGER, - NFS4CLNT_DELEGRETURN_RUNNING, + NFS4CLNT_RECALL_RUNNING, + NFS4CLNT_RECALL_ANY_LAYOUT_READ, + NFS4CLNT_RECALL_ANY_LAYOUT_RW, }; #define NFS4_RENEW_TIMEOUT 0x01 diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c index f7723d221945..ac93715c05a4 100644 --- a/fs/nfs/nfs4state.c +++ b/fs/nfs/nfs4state.c @@ -2524,6 +2524,21 @@ static int nfs4_bind_conn_to_session(struct nfs_client *clp) } return 0; } + +static void nfs4_layoutreturn_any_run(struct nfs_client *clp) +{ + int iomode = 0; + + if (test_and_clear_bit(NFS4CLNT_RECALL_ANY_LAYOUT_READ, &clp->cl_state)) + iomode += IOMODE_READ; + if (test_and_clear_bit(NFS4CLNT_RECALL_ANY_LAYOUT_RW, &clp->cl_state)) + iomode += IOMODE_RW; + /* Note: IOMODE_READ + IOMODE_RW == IOMODE_ANY */ + if (iomode) { + pnfs_layout_return_unused_byclid(clp, iomode); + set_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state); + } +} #else /* CONFIG_NFS_V4_1 */ static int nfs4_reset_session(struct nfs_client *clp) { return 0; } @@ -2531,6 +2546,10 @@ static int nfs4_bind_conn_to_session(struct nfs_client *clp) { return 0; } + +static void nfs4_layoutreturn_any_run(struct nfs_client *clp) +{ +} #endif /* CONFIG_NFS_V4_1 */ static void nfs4_state_manager(struct nfs_client *clp) @@ -2635,12 +2654,13 @@ static void nfs4_state_manager(struct nfs_client *clp) nfs4_end_drain_session(clp); nfs4_clear_state_manager_bit(clp); - if (!test_and_set_bit(NFS4CLNT_DELEGRETURN_RUNNING, &clp->cl_state)) { + if (!test_and_set_bit(NFS4CLNT_RECALL_RUNNING, &clp->cl_state)) { if (test_and_clear_bit(NFS4CLNT_DELEGRETURN, &clp->cl_state)) { nfs_client_return_marked_delegations(clp); set_bit(NFS4CLNT_RUN_MANAGER, &clp->cl_state); } - clear_bit(NFS4CLNT_DELEGRETURN_RUNNING, &clp->cl_state); + nfs4_layoutreturn_any_run(clp); + clear_bit(NFS4CLNT_RECALL_RUNNING, &clp->cl_state); } /* Did we race with an attempt to give us more work? */ diff --git a/fs/nfs/nfs4trace.h b/fs/nfs/nfs4trace.h index 1e97e5e04cb4..543541173a3d 100644 --- a/fs/nfs/nfs4trace.h +++ b/fs/nfs/nfs4trace.h @@ -584,7 +584,9 @@ TRACE_DEFINE_ENUM(NFS4CLNT_MOVED); TRACE_DEFINE_ENUM(NFS4CLNT_LEASE_MOVED); TRACE_DEFINE_ENUM(NFS4CLNT_DELEGATION_EXPIRED); TRACE_DEFINE_ENUM(NFS4CLNT_RUN_MANAGER); -TRACE_DEFINE_ENUM(NFS4CLNT_DELEGRETURN_RUNNING); +TRACE_DEFINE_ENUM(NFS4CLNT_RECALL_RUNNING); +TRACE_DEFINE_ENUM(NFS4CLNT_RECALL_ANY_LAYOUT_READ); +TRACE_DEFINE_ENUM(NFS4CLNT_RECALL_ANY_LAYOUT_RW); #define show_nfs4_clp_state(state) \ __print_flags(state, "|", \ @@ -605,7 +607,9 @@ TRACE_DEFINE_ENUM(NFS4CLNT_DELEGRETURN_RUNNING); { NFS4CLNT_LEASE_MOVED, "LEASE_MOVED" }, \ { NFS4CLNT_DELEGATION_EXPIRED, "DELEGATION_EXPIRED" }, \ { NFS4CLNT_RUN_MANAGER, "RUN_MANAGER" }, \ - { NFS4CLNT_DELEGRETURN_RUNNING, "DELEGRETURN_RUNNING" }) + { NFS4CLNT_RECALL_RUNNING, "RECALL_RUNNING" }, \ + { NFS4CLNT_RECALL_ANY_LAYOUT_READ, "RECALL_ANY_LAYOUT_READ" }, \ + { NFS4CLNT_RECALL_ANY_LAYOUT_RW, "RECALL_ANY_LAYOUT_RW" }) TRACE_EVENT(nfs4_state_mgr, TP_PROTO( diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 268e7b9ff54e..6b25117fca5f 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -309,6 +309,16 @@ pnfs_put_layout_hdr(struct pnfs_layout_hdr *lo) } } +static struct inode * +pnfs_grab_inode_layout_hdr(struct pnfs_layout_hdr *lo) +{ + struct inode *inode = igrab(lo->plh_inode); + if (inode) + return inode; + set_bit(NFS_LAYOUT_INODE_FREEING, &lo->plh_flags); + return NULL; +} + static void pnfs_set_plh_return_info(struct pnfs_layout_hdr *lo, enum pnfs_iomode iomode, u32 seq) @@ -782,7 +792,7 @@ pnfs_layout_bulk_destroy_byserver_locked(struct nfs_client *clp, /* If the sb is being destroyed, just bail */ if (!nfs_sb_active(server->super)) break; - inode = igrab(lo->plh_inode); + inode = pnfs_grab_inode_layout_hdr(lo); if (inode != NULL) { if (test_and_clear_bit(NFS_LAYOUT_HASHED, &lo->plh_flags)) list_del_rcu(&lo->plh_layouts); @@ -795,7 +805,6 @@ pnfs_layout_bulk_destroy_byserver_locked(struct nfs_client *clp, } else { rcu_read_unlock(); spin_unlock(&clp->cl_lock); - set_bit(NFS_LAYOUT_INODE_FREEING, &lo->plh_flags); } nfs_sb_deactive(server->super); spin_lock(&clp->cl_lock); @@ -2434,29 +2443,26 @@ pnfs_mark_matching_lsegs_return(struct pnfs_layout_hdr *lo, return -ENOENT; } -void pnfs_error_mark_layout_for_return(struct inode *inode, - struct pnfs_layout_segment *lseg) +static void +pnfs_mark_layout_for_return(struct inode *inode, + const struct pnfs_layout_range *range) { - struct pnfs_layout_hdr *lo = NFS_I(inode)->layout; - struct pnfs_layout_range range = { - .iomode = lseg->pls_range.iomode, - .offset = 0, - .length = NFS4_MAX_UINT64, - }; + struct pnfs_layout_hdr *lo; bool return_now = false; spin_lock(&inode->i_lock); + lo = NFS_I(inode)->layout; if (!pnfs_layout_is_valid(lo)) { spin_unlock(&inode->i_lock); return; } - pnfs_set_plh_return_info(lo, range.iomode, 0); + pnfs_set_plh_return_info(lo, range->iomode, 0); /* * mark all matching lsegs so that we are sure to have no live * segments at hand when sending layoutreturn. See pnfs_put_lseg() * for how it works. */ - if (pnfs_mark_matching_lsegs_return(lo, &lo->plh_return_segs, &range, 0) != -EBUSY) { + if (pnfs_mark_matching_lsegs_return(lo, &lo->plh_return_segs, range, 0) != -EBUSY) { nfs4_stateid stateid; enum pnfs_iomode iomode; @@ -2469,8 +2475,126 @@ void pnfs_error_mark_layout_for_return(struct inode *inode, nfs_commit_inode(inode, 0); } } + +void pnfs_error_mark_layout_for_return(struct inode *inode, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_layout_range range = { + .iomode = lseg->pls_range.iomode, + .offset = 0, + .length = NFS4_MAX_UINT64, + }; + + pnfs_mark_layout_for_return(inode, &range); +} EXPORT_SYMBOL_GPL(pnfs_error_mark_layout_for_return); +static bool +pnfs_layout_can_be_returned(struct pnfs_layout_hdr *lo) +{ + return pnfs_layout_is_valid(lo) && + !test_bit(NFS_LAYOUT_INODE_FREEING, &lo->plh_flags) && + !test_bit(NFS_LAYOUT_RETURN, &lo->plh_flags); +} + +static struct pnfs_layout_segment * +pnfs_find_first_lseg(struct pnfs_layout_hdr *lo, + const struct pnfs_layout_range *range, + enum pnfs_iomode iomode) +{ + struct pnfs_layout_segment *lseg; + + list_for_each_entry(lseg, &lo->plh_segs, pls_list) { + if (!test_bit(NFS_LSEG_VALID, &lseg->pls_flags)) + continue; + if (test_bit(NFS_LSEG_LAYOUTRETURN, &lseg->pls_flags)) + continue; + if (lseg->pls_range.iomode != iomode && iomode != IOMODE_ANY) + continue; + if (pnfs_lseg_range_intersecting(&lseg->pls_range, range)) + return lseg; + } + return NULL; +} + +/* Find open file states whose mode matches that of the range */ +static bool +pnfs_should_return_unused_layout(struct pnfs_layout_hdr *lo, + const struct pnfs_layout_range *range) +{ + struct list_head *head; + struct nfs_open_context *ctx; + fmode_t mode = 0; + + if (!pnfs_layout_can_be_returned(lo) || + !pnfs_find_first_lseg(lo, range, range->iomode)) + return false; + + head = &NFS_I(lo->plh_inode)->open_files; + list_for_each_entry_rcu(ctx, head, list) { + if (ctx->state) + mode |= ctx->state->state & (FMODE_READ|FMODE_WRITE); + } + + switch (range->iomode) { + default: + break; + case IOMODE_READ: + mode &= ~FMODE_WRITE; + break; + case IOMODE_RW: + if (pnfs_find_first_lseg(lo, range, IOMODE_READ)) + mode &= ~FMODE_READ; + } + return mode == 0; +} + +static int +pnfs_layout_return_unused_byserver(struct nfs_server *server, void *data) +{ + const struct pnfs_layout_range *range = data; + struct pnfs_layout_hdr *lo; + struct inode *inode; +restart: + rcu_read_lock(); + list_for_each_entry_rcu(lo, &server->layouts, plh_layouts) { + if (!pnfs_layout_can_be_returned(lo) || + test_bit(NFS_LAYOUT_RETURN_REQUESTED, &lo->plh_flags)) + continue; + inode = lo->plh_inode; + spin_lock(&inode->i_lock); + if (!pnfs_should_return_unused_layout(lo, range)) { + spin_unlock(&inode->i_lock); + continue; + } + spin_unlock(&inode->i_lock); + inode = pnfs_grab_inode_layout_hdr(lo); + if (!inode) + continue; + rcu_read_unlock(); + pnfs_mark_layout_for_return(inode, range); + iput(inode); + cond_resched(); + goto restart; + } + rcu_read_unlock(); + return 0; +} + +void +pnfs_layout_return_unused_byclid(struct nfs_client *clp, + enum pnfs_iomode iomode) +{ + struct pnfs_layout_range range = { + .iomode = iomode, + .offset = 0, + .length = NFS4_MAX_UINT64, + }; + + nfs_client_for_each_server(clp, pnfs_layout_return_unused_byserver, + &range); +} + void pnfs_generic_pg_check_layout(struct nfs_pageio_descriptor *pgio) { diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 8df9aa02d336..7bfb6970134a 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -329,6 +329,9 @@ int pnfs_write_done_resend_to_mds(struct nfs_pgio_header *); struct nfs4_threshold *pnfs_mdsthreshold_alloc(void); void pnfs_error_mark_layout_for_return(struct inode *inode, struct pnfs_layout_segment *lseg); +void pnfs_layout_return_unused_byclid(struct nfs_client *clp, + enum pnfs_iomode iomode); + /* nfs4_deviceid_flags */ enum { NFS_DEVICEID_INVALID = 0, /* set when MDS clientid recalled */