From patchwork Thu Nov 9 06:20:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450642 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72741D2EE for ; Thu, 9 Nov 2023 06:20:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="Pl4QJn/j" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A435C26A9 for ; Wed, 8 Nov 2023 22:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=RgKgcB8S6q4z3Pp7qf5GZxR2llbqiOZZmoonjHjc9sA=; b=Pl4QJn/jAg2qPqN5o7dgMUkGB/ eT8UWSefsHp799UcKZGDL0tyZEKcthIIlYNTRlxcLdPBeLyRmo86g/tubQhLA3Db4CHVaUBDIEeao 5WbpiYy+9MB30By6Gw4R/yR6obUD4YXOfIyHaIml72YXUTTyrRvJdE0KEKX7O/ilaqTiadZmPRyD1 ADYrItL8+ZTxwUnP4SAJhz9K/fErwYKbeCL8bFU1qr4chVL1vo3BNZ0KMmr7wgrWRz0gp/gQNz4iF XTYvl8hbKt3J8OAHNPg5clCPNhatnnHl7BDEMkUVXidgct6vr6IU3a6MNh4vAf+2BxQ+tsGzK/XAr d02gBa1w==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPQ-00DLj4-0E; Thu, 09 Nov 2023 06:20:56 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 01/22] struct dentry: get rid of randomize_layout idiocy Date: Thu, 9 Nov 2023 06:20:35 +0000 Message-Id: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109061932.GA3181489@ZenIV> References: <20231109061932.GA3181489@ZenIV> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro This is beyond ridiculous. There is a reason why that thing is cacheline-aligned... Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- include/linux/dcache.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/dcache.h b/include/linux/dcache.h index 6b351e009f59..8b4ad3c3bba0 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -111,7 +111,7 @@ struct dentry { struct hlist_bl_node d_in_lookup_hash; /* only for in-lookup ones */ struct rcu_head d_rcu; } d_u; -} __randomize_layout; +}; /* * dentry->d_lock spinlock nesting subclasses: From patchwork Thu Nov 9 06:20:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450644 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B80C6D304 for ; Thu, 9 Nov 2023 06:20:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="JeY1xkge" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD8C826AB for ; Wed, 8 Nov 2023 22:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=OrmJa85C5elcGZzOrChSJYM4gm189u5IK4tOJQ7Bupg=; b=JeY1xkge7QFAGBdCOwOiC5OQz7 vuP1/tsDxwprQ+e/zI7Mxpj6XqGz5ucIImbULUQzQvHVs7RdLJkjyc6YEdPILUAPN/5mhc+aKb5hG LNn4m3IeYJcs091o5hMrcbs8bEFRQfDz4p1WxiMbzDn2ILWI4MHVD+k/OA0BIl9FLJ5K7N30MaaiF 06TwIdKVeWnx1gsOvPILYt6US/isN+RNzLaY7YBHU7zirxvtk9LxvevBKposPrJY/dDLU+w7o5QQr iCJ0Z4cix2EdT1m+09g/7f+c+G83KCc6jfdqCshfLoire/JitYxWxVBGet1kSwM9U+kB/MBoFNylO xnEtEJdw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPQ-00DLj6-1M; Thu, 09 Nov 2023 06:20:56 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 02/22] switch nfsd_client_rmdir() to use of simple_recursive_removal() Date: Thu, 9 Nov 2023 06:20:36 +0000 Message-Id: <20231109062056.3181775-2-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Reviewed-by: Jeff Layton Tested-by: Jeff Layton Signed-off-by: Al Viro Acked-by: Christian Brauner Acked-by: Chuck Lever > --- fs/nfsd/nfsctl.c | 70 ++++++++++-------------------------------------- 1 file changed, 14 insertions(+), 56 deletions(-) diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c index 7ed02fb88a36..035b42c1a181 100644 --- a/fs/nfsd/nfsctl.c +++ b/fs/nfsd/nfsctl.c @@ -1235,63 +1235,34 @@ static inline void _nfsd_symlink(struct dentry *parent, const char *name, #endif -static void clear_ncl(struct inode *inode) +static void clear_ncl(struct dentry *dentry) { + struct inode *inode = d_inode(dentry); struct nfsdfs_client *ncl = inode->i_private; + spin_lock(&inode->i_lock); inode->i_private = NULL; + spin_unlock(&inode->i_lock); kref_put(&ncl->cl_ref, ncl->cl_release); } -static struct nfsdfs_client *__get_nfsdfs_client(struct inode *inode) -{ - struct nfsdfs_client *nc = inode->i_private; - - if (nc) - kref_get(&nc->cl_ref); - return nc; -} - struct nfsdfs_client *get_nfsdfs_client(struct inode *inode) { struct nfsdfs_client *nc; - inode_lock_shared(inode); - nc = __get_nfsdfs_client(inode); - inode_unlock_shared(inode); + spin_lock(&inode->i_lock); + nc = inode->i_private; + if (nc) + kref_get(&nc->cl_ref); + spin_unlock(&inode->i_lock); return nc; } -/* from __rpc_unlink */ -static void nfsdfs_remove_file(struct inode *dir, struct dentry *dentry) -{ - int ret; - - clear_ncl(d_inode(dentry)); - dget(dentry); - ret = simple_unlink(dir, dentry); - d_drop(dentry); - fsnotify_unlink(dir, dentry); - dput(dentry); - WARN_ON_ONCE(ret); -} - -static void nfsdfs_remove_files(struct dentry *root) -{ - struct dentry *dentry, *tmp; - - list_for_each_entry_safe(dentry, tmp, &root->d_subdirs, d_child) { - if (!simple_positive(dentry)) { - WARN_ON_ONCE(1); /* I think this can't happen? */ - continue; - } - nfsdfs_remove_file(d_inode(root), dentry); - } -} /* XXX: cut'n'paste from simple_fill_super; figure out if we could share * code instead. */ static int nfsdfs_create_files(struct dentry *root, const struct tree_descr *files, + struct nfsdfs_client *ncl, struct dentry **fdentries) { struct inode *dir = d_inode(root); @@ -1310,8 +1281,9 @@ static int nfsdfs_create_files(struct dentry *root, dput(dentry); goto out; } + kref_get(&ncl->cl_ref); inode->i_fop = files->ops; - inode->i_private = __get_nfsdfs_client(dir); + inode->i_private = ncl; d_add(dentry, inode); fsnotify_create(dir, dentry); if (fdentries) @@ -1320,7 +1292,6 @@ static int nfsdfs_create_files(struct dentry *root, inode_unlock(dir); return 0; out: - nfsdfs_remove_files(root); inode_unlock(dir); return -ENOMEM; } @@ -1340,7 +1311,7 @@ struct dentry *nfsd_client_mkdir(struct nfsd_net *nn, dentry = nfsd_mkdir(nn->nfsd_client_dir, ncl, name); if (IS_ERR(dentry)) /* XXX: tossing errors? */ return NULL; - ret = nfsdfs_create_files(dentry, files, fdentries); + ret = nfsdfs_create_files(dentry, files, ncl, fdentries); if (ret) { nfsd_client_rmdir(dentry); return NULL; @@ -1351,20 +1322,7 @@ struct dentry *nfsd_client_mkdir(struct nfsd_net *nn, /* Taken from __rpc_rmdir: */ void nfsd_client_rmdir(struct dentry *dentry) { - struct inode *dir = d_inode(dentry->d_parent); - struct inode *inode = d_inode(dentry); - int ret; - - inode_lock(dir); - nfsdfs_remove_files(dentry); - clear_ncl(inode); - dget(dentry); - ret = simple_rmdir(dir, dentry); - WARN_ON_ONCE(ret); - d_drop(dentry); - fsnotify_rmdir(dir, dentry); - dput(dentry); - inode_unlock(dir); + simple_recursive_removal(dentry, clear_ncl); } static int nfsd_fill_super(struct super_block *sb, struct fs_context *fc) From patchwork Thu Nov 9 06:20:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450643 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8062D2E5 for ; Thu, 9 Nov 2023 06:20:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="QuSktSDN" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0E3326AF for ; Wed, 8 Nov 2023 22:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=Rm89adE/IWmDu4t0PfcRUSn6EmnV0P6FUJX93O3jV1U=; b=QuSktSDNnGBJ4VwdhG07PxRezu GPJaCGYa4TLTOR+k97K2dKDC/bFPrB0jtiYebWWXpu/vIxcSmMW+fUjPnEcVH+Eo+JpGQQhZTuWAJ yyB8iUVAk2lIqpgSGAbwoD6ewiIP+iiK4cjZEn0VIwneHK/b7oCR27EqjvkxeHVxutfJQxc3LoCJl 6VztF3Kcw+9sXfBPVCQlkYrvJxRnvAOb6I/GdSVAdHtO1YiryKKVaU+N0S72H8KYiod/Rzwmo0G96 JkXD9o5kfRys+n4PIq4FXLSuNpWbRsZ3PYK2lgh9WPiU5mc3HgPNa0wqlgziTtu1gVAHEBOCI4Md9 5PqORTHQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPQ-00DLjA-1w; Thu, 09 Nov 2023 06:20:56 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 03/22] coda_flag_children(): cope with dentries turning negative Date: Thu, 9 Nov 2023 06:20:37 +0000 Message-Id: <20231109062056.3181775-3-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ->d_lock on parent does not stabilize ->d_inode of child. We don't do much with that inode in there, but we need at least to avoid struct inode getting freed under us... Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/coda/cache.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/coda/cache.c b/fs/coda/cache.c index 3b8c4513118f..bfbc03c6b632 100644 --- a/fs/coda/cache.c +++ b/fs/coda/cache.c @@ -92,13 +92,16 @@ static void coda_flag_children(struct dentry *parent, int flag) { struct dentry *de; + rcu_read_lock(); spin_lock(&parent->d_lock); list_for_each_entry(de, &parent->d_subdirs, d_child) { + struct inode *inode = d_inode_rcu(de); /* don't know what to do with negative dentries */ - if (d_inode(de) ) - coda_flag_inode(d_inode(de), flag); + if (inode) + coda_flag_inode(inode, flag); } spin_unlock(&parent->d_lock); + rcu_read_unlock(); return; } From patchwork Thu Nov 9 06:20:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450649 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 450FFD2ED for ; Thu, 9 Nov 2023 06:20:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="ecWAmG0t" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F6D826B0 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=oWnC8SgO1jNa3YI/lcnM3tx8XM5EskzL0x6vyY7Vwn0=; b=ecWAmG0tIbc18e8uJ4hYI2iOCK nDeudbMrTvXzAMcca93FIxOGeqYPRRh63YVckrqL0q7wXHwnFoqDDAbgplUjeWaGh4JIf4Ej2f/Up jOM+4/Q5OCgQIi/Z5cm5taL/vCQ8jFTl+Wu473umxf6o/jn9NgfuRGUtLkNQUxAWZrPN07hO6Un+S 0LFzMYZjBV86fpKPUnN+1oo9QJMbYOK6injT5yKyyQH6WEPxYrmCu4LSCMYMRVohDnxPLl3RY+i9S W/9pP2Sy86daJhmkMJDkp2o2f4eZgcnPR0lzMrPa6JTS1HAv3tq1bvEyKjQ86qlzr1VZkoJp8qely lnY4ni8w==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPQ-00DLjE-2Q; Thu, 09 Nov 2023 06:20:56 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 04/22] dentry: switch the lists of children to hlist Date: Thu, 9 Nov 2023 06:20:38 +0000 Message-Id: <20231109062056.3181775-4-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Saves a pointer per struct dentry and actually makes the things less clumsy. Cleaned the d_walk() and dcache_readdir() a bit by use of hlist_for_... iterators. A couple of new helpers - d_first_child() and d_next_sibling(), to make the expressions less awful. X-fuck-kABI: gladly Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- Documentation/filesystems/porting.rst | 9 +++ arch/powerpc/platforms/cell/spufs/inode.c | 5 +- fs/afs/dynroot.c | 5 +- fs/autofs/expire.c | 7 +-- fs/ceph/dir.c | 2 +- fs/ceph/mds_client.c | 2 +- fs/coda/cache.c | 2 +- fs/dcache.c | 76 +++++++++++------------ fs/libfs.c | 45 +++++++------- fs/notify/fsnotify.c | 2 +- fs/tracefs/inode.c | 34 +++++----- include/linux/dcache.h | 20 ++++-- 12 files changed, 108 insertions(+), 101 deletions(-) diff --git a/Documentation/filesystems/porting.rst b/Documentation/filesystems/porting.rst index 4d05b9862451..58627f0baf3e 100644 --- a/Documentation/filesystems/porting.rst +++ b/Documentation/filesystems/porting.rst @@ -1045,3 +1045,12 @@ filesystem type is now moved to a later point when the devices are closed: As this is a VFS level change it has no practical consequences for filesystems other than that all of them must use one of the provided kill_litter_super(), kill_anon_super(), or kill_block_super() helpers. + +--- + +**mandatory** + +The list of children anchored in parent dentry got turned into hlist now. +Field names got changed (->d_children/->d_sib instead of ->d_subdirs/->d_child +for anchor/entries resp.), so any affected places will be immediately caught +by compiler. diff --git a/arch/powerpc/platforms/cell/spufs/inode.c b/arch/powerpc/platforms/cell/spufs/inode.c index 38c5be34c895..71019bfa0eb7 100644 --- a/arch/powerpc/platforms/cell/spufs/inode.c +++ b/arch/powerpc/platforms/cell/spufs/inode.c @@ -145,10 +145,11 @@ spufs_evict_inode(struct inode *inode) static void spufs_prune_dir(struct dentry *dir) { - struct dentry *dentry, *tmp; + struct dentry *dentry; + struct hlist_node *n; inode_lock(d_inode(dir)); - list_for_each_entry_safe(dentry, tmp, &dir->d_subdirs, d_child) { + hlist_for_each_entry_safe(dentry, n, &dir->d_children, d_sib) { spin_lock(&dentry->d_lock); if (simple_positive(dentry)) { dget_dlock(dentry); diff --git a/fs/afs/dynroot.c b/fs/afs/dynroot.c index 95bcbd7654d1..fb510c3197e4 100644 --- a/fs/afs/dynroot.c +++ b/fs/afs/dynroot.c @@ -370,7 +370,7 @@ int afs_dynroot_populate(struct super_block *sb) void afs_dynroot_depopulate(struct super_block *sb) { struct afs_net *net = afs_sb2net(sb); - struct dentry *root = sb->s_root, *subdir, *tmp; + struct dentry *root = sb->s_root, *subdir; /* Prevent more subdirs from being created */ mutex_lock(&net->proc_cells_lock); @@ -379,10 +379,11 @@ void afs_dynroot_depopulate(struct super_block *sb) mutex_unlock(&net->proc_cells_lock); if (root) { + struct hlist_node *n; inode_lock(root->d_inode); /* Remove all the pins for dirs created for manually added cells */ - list_for_each_entry_safe(subdir, tmp, &root->d_subdirs, d_child) { + hlist_for_each_entry_safe(subdir, n, &root->d_children, d_sib) { if (subdir->d_fsdata) { subdir->d_fsdata = NULL; dput(subdir); diff --git a/fs/autofs/expire.c b/fs/autofs/expire.c index 038b3d2d9f57..39d8c84c16f4 100644 --- a/fs/autofs/expire.c +++ b/fs/autofs/expire.c @@ -73,12 +73,9 @@ static int autofs_mount_busy(struct vfsmount *mnt, /* p->d_lock held */ static struct dentry *positive_after(struct dentry *p, struct dentry *child) { - if (child) - child = list_next_entry(child, d_child); - else - child = list_first_entry(&p->d_subdirs, struct dentry, d_child); + child = child ? d_next_sibling(child) : d_first_child(p); - list_for_each_entry_from(child, &p->d_subdirs, d_child) { + hlist_for_each_entry_from(child, d_sib) { spin_lock_nested(&child->d_lock, DENTRY_D_LOCK_NESTED); if (simple_positive(child)) { dget_dlock(child); diff --git a/fs/ceph/dir.c b/fs/ceph/dir.c index 854cbdd66661..87884a578ec9 100644 --- a/fs/ceph/dir.c +++ b/fs/ceph/dir.c @@ -171,7 +171,7 @@ __dcache_find_get_entry(struct dentry *parent, u64 idx, /* * When possible, we try to satisfy a readdir by peeking at the * dcache. We make this work by carefully ordering dentries on - * d_child when we initially get results back from the MDS, and + * d_children when we initially get results back from the MDS, and * falling back to a "normal" sync readdir if any dentries in the dir * are dropped. * diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index 293b93182955..a566b4b029b9 100644 --- a/fs/ceph/mds_client.c +++ b/fs/ceph/mds_client.c @@ -2100,7 +2100,7 @@ static bool drop_negative_children(struct dentry *dentry) goto out; spin_lock(&dentry->d_lock); - list_for_each_entry(child, &dentry->d_subdirs, d_child) { + hlist_for_each_entry(child, &dentry->d_children, d_sib) { if (d_really_is_positive(child)) { all_negative = false; break; diff --git a/fs/coda/cache.c b/fs/coda/cache.c index bfbc03c6b632..f5b71a35f9db 100644 --- a/fs/coda/cache.c +++ b/fs/coda/cache.c @@ -94,7 +94,7 @@ static void coda_flag_children(struct dentry *parent, int flag) rcu_read_lock(); spin_lock(&parent->d_lock); - list_for_each_entry(de, &parent->d_subdirs, d_child) { + hlist_for_each_entry(de, &parent->d_children, d_sib) { struct inode *inode = d_inode_rcu(de); /* don't know what to do with negative dentries */ if (inode) diff --git a/fs/dcache.c b/fs/dcache.c index 25ac74d30bff..1b8ec1a9bf1c 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -51,8 +51,8 @@ * - d_lru * - d_count * - d_unhashed() - * - d_parent and d_subdirs - * - childrens' d_child and d_parent + * - d_parent and d_chilren + * - childrens' d_sib and d_parent * - d_u.d_alias, d_inode * * Ordering: @@ -537,7 +537,7 @@ void d_drop(struct dentry *dentry) } EXPORT_SYMBOL(d_drop); -static inline void dentry_unlist(struct dentry *dentry, struct dentry *parent) +static inline void dentry_unlist(struct dentry *dentry) { struct dentry *next; /* @@ -545,12 +545,12 @@ static inline void dentry_unlist(struct dentry *dentry, struct dentry *parent) * attached to the dentry tree */ dentry->d_flags |= DCACHE_DENTRY_KILLED; - if (unlikely(list_empty(&dentry->d_child))) + if (unlikely(hlist_unhashed(&dentry->d_sib))) return; - __list_del_entry(&dentry->d_child); + __hlist_del(&dentry->d_sib); /* * Cursors can move around the list of children. While we'd been - * a normal list member, it didn't matter - ->d_child.next would've + * a normal list member, it didn't matter - ->d_sib.next would've * been updated. However, from now on it won't be and for the * things like d_walk() it might end up with a nasty surprise. * Normally d_walk() doesn't care about cursors moving around - @@ -558,20 +558,20 @@ static inline void dentry_unlist(struct dentry *dentry, struct dentry *parent) * of its own, we get through it without ever unlocking the parent. * There is one exception, though - if we ascend from a child that * gets killed as soon as we unlock it, the next sibling is found - * using the value left in its ->d_child.next. And if _that_ + * using the value left in its ->d_sib.next. And if _that_ * pointed to a cursor, and cursor got moved (e.g. by lseek()) * before d_walk() regains parent->d_lock, we'll end up skipping * everything the cursor had been moved past. * - * Solution: make sure that the pointer left behind in ->d_child.next + * Solution: make sure that the pointer left behind in ->d_sib.next * points to something that won't be moving around. I.e. skip the * cursors. */ - while (dentry->d_child.next != &parent->d_subdirs) { - next = list_entry(dentry->d_child.next, struct dentry, d_child); + while (dentry->d_sib.next) { + next = hlist_entry(dentry->d_sib.next, struct dentry, d_sib); if (likely(!(next->d_flags & DCACHE_DENTRY_CURSOR))) break; - dentry->d_child.next = next->d_child.next; + dentry->d_sib.next = next->d_sib.next; } } @@ -600,7 +600,7 @@ static void __dentry_kill(struct dentry *dentry) } /* if it was on the hash then remove it */ __d_drop(dentry); - dentry_unlist(dentry, parent); + dentry_unlist(dentry); if (parent) spin_unlock(&parent->d_lock); if (dentry->d_inode) @@ -1348,8 +1348,7 @@ enum d_walk_ret { static void d_walk(struct dentry *parent, void *data, enum d_walk_ret (*enter)(void *, struct dentry *)) { - struct dentry *this_parent; - struct list_head *next; + struct dentry *this_parent, *dentry; unsigned seq = 0; enum d_walk_ret ret; bool retry = true; @@ -1371,13 +1370,9 @@ static void d_walk(struct dentry *parent, void *data, break; } repeat: - next = this_parent->d_subdirs.next; + dentry = d_first_child(this_parent); resume: - while (next != &this_parent->d_subdirs) { - struct list_head *tmp = next; - struct dentry *dentry = list_entry(tmp, struct dentry, d_child); - next = tmp->next; - + hlist_for_each_entry_from(dentry, d_sib) { if (unlikely(dentry->d_flags & DCACHE_DENTRY_CURSOR)) continue; @@ -1398,7 +1393,7 @@ static void d_walk(struct dentry *parent, void *data, continue; } - if (!list_empty(&dentry->d_subdirs)) { + if (!hlist_empty(&dentry->d_children)) { spin_unlock(&this_parent->d_lock); spin_release(&dentry->d_lock.dep_map, _RET_IP_); this_parent = dentry; @@ -1413,24 +1408,23 @@ static void d_walk(struct dentry *parent, void *data, rcu_read_lock(); ascend: if (this_parent != parent) { - struct dentry *child = this_parent; - this_parent = child->d_parent; + dentry = this_parent; + this_parent = dentry->d_parent; - spin_unlock(&child->d_lock); + spin_unlock(&dentry->d_lock); spin_lock(&this_parent->d_lock); /* might go back up the wrong parent if we have had a rename. */ if (need_seqretry(&rename_lock, seq)) goto rename_retry; /* go into the first sibling still alive */ - do { - next = child->d_child.next; - if (next == &this_parent->d_subdirs) - goto ascend; - child = list_entry(next, struct dentry, d_child); - } while (unlikely(child->d_flags & DCACHE_DENTRY_KILLED)); - rcu_read_unlock(); - goto resume; + hlist_for_each_entry_continue(dentry, d_sib) { + if (likely(!(dentry->d_flags & DCACHE_DENTRY_KILLED))) { + rcu_read_unlock(); + goto resume; + } + } + goto ascend; } if (need_seqretry(&rename_lock, seq)) goto rename_retry; @@ -1530,7 +1524,7 @@ int d_set_mounted(struct dentry *dentry) * Search the dentry child list of the specified parent, * and move any unused dentries to the end of the unused * list for prune_dcache(). We descend to the next level - * whenever the d_subdirs list is non-empty and continue + * whenever the d_children list is non-empty and continue * searching. * * It returns zero iff there are no unused children, @@ -1657,7 +1651,7 @@ EXPORT_SYMBOL(shrink_dcache_parent); static enum d_walk_ret umount_check(void *_data, struct dentry *dentry) { /* it has busy descendents; complain about those instead */ - if (!list_empty(&dentry->d_subdirs)) + if (!hlist_empty(&dentry->d_children)) return D_WALK_CONTINUE; /* root with refcount 1 is fine */ @@ -1814,9 +1808,9 @@ static struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name) dentry->d_fsdata = NULL; INIT_HLIST_BL_NODE(&dentry->d_hash); INIT_LIST_HEAD(&dentry->d_lru); - INIT_LIST_HEAD(&dentry->d_subdirs); + INIT_HLIST_HEAD(&dentry->d_children); INIT_HLIST_NODE(&dentry->d_u.d_alias); - INIT_LIST_HEAD(&dentry->d_child); + INIT_HLIST_NODE(&dentry->d_sib); d_set_d_op(dentry, dentry->d_sb->s_d_op); if (dentry->d_op && dentry->d_op->d_init) { @@ -1855,7 +1849,7 @@ struct dentry *d_alloc(struct dentry * parent, const struct qstr *name) */ __dget_dlock(parent); dentry->d_parent = parent; - list_add(&dentry->d_child, &parent->d_subdirs); + hlist_add_head(&dentry->d_sib, &parent->d_children); spin_unlock(&parent->d_lock); return dentry; @@ -2993,11 +2987,15 @@ static void __d_move(struct dentry *dentry, struct dentry *target, } else { target->d_parent = old_parent; swap_names(dentry, target); - list_move(&target->d_child, &target->d_parent->d_subdirs); + if (!hlist_unhashed(&target->d_sib)) + __hlist_del(&target->d_sib); + hlist_add_head(&target->d_sib, &target->d_parent->d_children); __d_rehash(target); fsnotify_update_flags(target); } - list_move(&dentry->d_child, &dentry->d_parent->d_subdirs); + if (!hlist_unhashed(&dentry->d_sib)) + __hlist_del(&dentry->d_sib); + hlist_add_head(&dentry->d_sib, &dentry->d_parent->d_children); __d_rehash(dentry); fsnotify_update_flags(dentry); fscrypt_handle_d_move(dentry); diff --git a/fs/libfs.c b/fs/libfs.c index 37f2d34ee090..1a7d30f867f3 100644 --- a/fs/libfs.c +++ b/fs/libfs.c @@ -101,15 +101,16 @@ EXPORT_SYMBOL(dcache_dir_close); * If no such element exists, NULL is returned. */ static struct dentry *scan_positives(struct dentry *cursor, - struct list_head *p, + struct hlist_node **p, loff_t count, struct dentry *last) { struct dentry *dentry = cursor->d_parent, *found = NULL; spin_lock(&dentry->d_lock); - while ((p = p->next) != &dentry->d_subdirs) { - struct dentry *d = list_entry(p, struct dentry, d_child); + while (*p) { + struct dentry *d = hlist_entry(*p, struct dentry, d_sib); + p = &d->d_sib.next; // we must at least skip cursors, to avoid livelocks if (d->d_flags & DCACHE_DENTRY_CURSOR) continue; @@ -123,8 +124,10 @@ static struct dentry *scan_positives(struct dentry *cursor, count = 1; } if (need_resched()) { - list_move(&cursor->d_child, p); - p = &cursor->d_child; + if (!hlist_unhashed(&cursor->d_sib)) + __hlist_del(&cursor->d_sib); + hlist_add_behind(&cursor->d_sib, &d->d_sib); + p = &cursor->d_sib.next; spin_unlock(&dentry->d_lock); cond_resched(); spin_lock(&dentry->d_lock); @@ -156,13 +159,12 @@ loff_t dcache_dir_lseek(struct file *file, loff_t offset, int whence) inode_lock_shared(dentry->d_inode); if (offset > 2) - to = scan_positives(cursor, &dentry->d_subdirs, + to = scan_positives(cursor, &dentry->d_children.first, offset - 2, NULL); spin_lock(&dentry->d_lock); + hlist_del_init(&cursor->d_sib); if (to) - list_move(&cursor->d_child, &to->d_child); - else - list_del_init(&cursor->d_child); + hlist_add_behind(&cursor->d_sib, &to->d_sib); spin_unlock(&dentry->d_lock); dput(to); @@ -184,19 +186,16 @@ int dcache_readdir(struct file *file, struct dir_context *ctx) { struct dentry *dentry = file->f_path.dentry; struct dentry *cursor = file->private_data; - struct list_head *anchor = &dentry->d_subdirs; struct dentry *next = NULL; - struct list_head *p; + struct hlist_node **p; if (!dir_emit_dots(file, ctx)) return 0; if (ctx->pos == 2) - p = anchor; - else if (!list_empty(&cursor->d_child)) - p = &cursor->d_child; + p = &dentry->d_children.first; else - return 0; + p = &cursor->d_sib.next; while ((next = scan_positives(cursor, p, 1, next)) != NULL) { if (!dir_emit(ctx, next->d_name.name, next->d_name.len, @@ -204,13 +203,12 @@ int dcache_readdir(struct file *file, struct dir_context *ctx) fs_umode_to_dtype(d_inode(next)->i_mode))) break; ctx->pos++; - p = &next->d_child; + p = &next->d_sib.next; } spin_lock(&dentry->d_lock); + hlist_del_init(&cursor->d_sib); if (next) - list_move_tail(&cursor->d_child, &next->d_child); - else - list_del_init(&cursor->d_child); + hlist_add_before(&cursor->d_sib, &next->d_sib); spin_unlock(&dentry->d_lock); dput(next); @@ -489,12 +487,11 @@ const struct file_operations simple_offset_dir_operations = { static struct dentry *find_next_child(struct dentry *parent, struct dentry *prev) { - struct dentry *child = NULL; - struct list_head *p = prev ? &prev->d_child : &parent->d_subdirs; + struct dentry *child = NULL, *d; spin_lock(&parent->d_lock); - while ((p = p->next) != &parent->d_subdirs) { - struct dentry *d = container_of(p, struct dentry, d_child); + d = prev ? d_next_sibling(prev) : d_first_child(parent); + hlist_for_each_entry_from(d, d_sib) { if (simple_positive(d)) { spin_lock_nested(&d->d_lock, DENTRY_D_LOCK_NESTED); if (simple_positive(d)) @@ -654,7 +651,7 @@ int simple_empty(struct dentry *dentry) int ret = 0; spin_lock(&dentry->d_lock); - list_for_each_entry(child, &dentry->d_subdirs, d_child) { + hlist_for_each_entry(child, &dentry->d_children, d_sib) { spin_lock_nested(&child->d_lock, DENTRY_D_LOCK_NESTED); if (simple_positive(child)) { spin_unlock(&child->d_lock); diff --git a/fs/notify/fsnotify.c b/fs/notify/fsnotify.c index 7974e91ffe13..8bfd690e9f10 100644 --- a/fs/notify/fsnotify.c +++ b/fs/notify/fsnotify.c @@ -124,7 +124,7 @@ void __fsnotify_update_child_dentry_flags(struct inode *inode) * d_flags to indicate parental interest (their parent is the * original inode) */ spin_lock(&alias->d_lock); - list_for_each_entry(child, &alias->d_subdirs, d_child) { + hlist_for_each_entry(child, &alias->d_children, d_sib) { if (!child->d_inode) continue; diff --git a/fs/tracefs/inode.c b/fs/tracefs/inode.c index 891653ba9cf3..2aaa4c421938 100644 --- a/fs/tracefs/inode.c +++ b/fs/tracefs/inode.c @@ -199,26 +199,21 @@ static void change_gid(struct dentry *dentry, kgid_t gid) */ static void set_gid(struct dentry *parent, kgid_t gid) { - struct dentry *this_parent; - struct list_head *next; + struct dentry *this_parent, *dentry; this_parent = parent; spin_lock(&this_parent->d_lock); change_gid(this_parent, gid); repeat: - next = this_parent->d_subdirs.next; + dentry = d_first_child(this_parent); resume: - while (next != &this_parent->d_subdirs) { - struct list_head *tmp = next; - struct dentry *dentry = list_entry(tmp, struct dentry, d_child); - next = tmp->next; - + hlist_for_each_entry_from(dentry, d_sib) { spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED); change_gid(dentry, gid); - if (!list_empty(&dentry->d_subdirs)) { + if (!hlist_empty(&dentry->d_children)) { spin_unlock(&this_parent->d_lock); spin_release(&dentry->d_lock.dep_map, _RET_IP_); this_parent = dentry; @@ -233,21 +228,20 @@ static void set_gid(struct dentry *parent, kgid_t gid) rcu_read_lock(); ascend: if (this_parent != parent) { - struct dentry *child = this_parent; - this_parent = child->d_parent; + dentry = this_parent; + this_parent = dentry->d_parent; - spin_unlock(&child->d_lock); + spin_unlock(&dentry->d_lock); spin_lock(&this_parent->d_lock); /* go into the first sibling still alive */ - do { - next = child->d_child.next; - if (next == &this_parent->d_subdirs) - goto ascend; - child = list_entry(next, struct dentry, d_child); - } while (unlikely(child->d_flags & DCACHE_DENTRY_KILLED)); - rcu_read_unlock(); - goto resume; + hlist_for_each_entry_continue(dentry, d_sib) { + if (likely(!(dentry->d_flags & DCACHE_DENTRY_KILLED))) { + rcu_read_unlock(); + goto resume; + } + } + goto ascend; } rcu_read_unlock(); spin_unlock(&this_parent->d_lock); diff --git a/include/linux/dcache.h b/include/linux/dcache.h index 8b4ad3c3bba0..140662c3156d 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -68,12 +68,12 @@ extern const struct qstr dotdot_name; * large memory footprint increase). */ #ifdef CONFIG_64BIT -# define DNAME_INLINE_LEN 32 /* 192 bytes */ +# define DNAME_INLINE_LEN 40 /* 192 bytes */ #else # ifdef CONFIG_SMP -# define DNAME_INLINE_LEN 36 /* 128 bytes */ -# else # define DNAME_INLINE_LEN 40 /* 128 bytes */ +# else +# define DNAME_INLINE_LEN 44 /* 128 bytes */ # endif #endif @@ -101,8 +101,8 @@ struct dentry { struct list_head d_lru; /* LRU list */ wait_queue_head_t *d_wait; /* in-lookup ones only */ }; - struct list_head d_child; /* child of parent list */ - struct list_head d_subdirs; /* our children */ + struct hlist_node d_sib; /* child of parent list */ + struct hlist_head d_children; /* our children */ /* * d_alias and d_rcu can share memory */ @@ -599,4 +599,14 @@ struct name_snapshot { void take_dentry_name_snapshot(struct name_snapshot *, struct dentry *); void release_dentry_name_snapshot(struct name_snapshot *); +static inline struct dentry *d_first_child(const struct dentry *dentry) +{ + return hlist_entry_safe(dentry->d_children.first, struct dentry, d_sib); +} + +static inline struct dentry *d_next_sibling(const struct dentry *dentry) +{ + return hlist_entry_safe(dentry->d_sib.next, struct dentry, d_sib); +} + #endif /* __LINUX_DCACHE_H */ From patchwork Thu Nov 9 06:20:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450645 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 477DFD307 for ; Thu, 9 Nov 2023 06:20:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="AHpWUzRt" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EFA626B1 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=DTPyMFGMxqBUgkHNvTYyVNefwv3L9prayceKiT3qoA4=; b=AHpWUzRtKADJjdHAlLJP7IkAto klZcLKZsRZ2J4BWCZ0wgMwCvbWAVlo0rkBFF3YWIrPSFr4cRGVmXaltvhkjfSOm/onK5tcgBqnrxw cr/6XMMTQ9Va59p/nMKHL28iKARkxtYSVql2HciNCAhwDDnd0HT2DQ/4PBOZKxZZWm/sw5IK8dn1M UeFINxU579KzQTXCbe3STAyH2cNer6OR+5DSQO3LoiP96np1LL+W7M4eZ04yVeQiNi463CNSyfDj7 0meWMnKC3TVF9vTaQoTVjeiAvv7UKyUpuBbRCdl2GKMB/pMafPzw40rgv2UkeMyP5v+EoSkAD5xRz uoeF/ZUg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPQ-00DLjJ-32; Thu, 09 Nov 2023 06:20:56 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 05/22] centralize killing dentry from shrink list Date: Thu, 9 Nov 2023 06:20:39 +0000 Message-Id: <20231109062056.3181775-5-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro new helper unifying identical bits of shrink_dentry_list() and shring_dcache_for_umount() Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 1b8ec1a9bf1c..56af55f2b7d9 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1174,10 +1174,18 @@ static bool shrink_lock_dentry(struct dentry *dentry) return false; } +static inline void shrink_kill(struct dentry *victim, struct list_head *list) +{ + struct dentry *parent = victim->d_parent; + if (parent != victim) + __dput_to_list(parent, list); + __dentry_kill(victim); +} + void shrink_dentry_list(struct list_head *list) { while (!list_empty(list)) { - struct dentry *dentry, *parent; + struct dentry *dentry; dentry = list_entry(list->prev, struct dentry, d_lru); spin_lock(&dentry->d_lock); @@ -1195,10 +1203,7 @@ void shrink_dentry_list(struct list_head *list) } rcu_read_unlock(); d_shrink_del(dentry); - parent = dentry->d_parent; - if (parent != dentry) - __dput_to_list(parent, list); - __dentry_kill(dentry); + shrink_kill(dentry, list); } } @@ -1629,17 +1634,13 @@ void shrink_dcache_parent(struct dentry *parent) data.victim = NULL; d_walk(parent, &data, select_collect2); if (data.victim) { - struct dentry *parent; spin_lock(&data.victim->d_lock); if (!shrink_lock_dentry(data.victim)) { spin_unlock(&data.victim->d_lock); rcu_read_unlock(); } else { rcu_read_unlock(); - parent = data.victim->d_parent; - if (parent != data.victim) - __dput_to_list(parent, &data.dispose); - __dentry_kill(data.victim); + shrink_kill(data.victim, &data.dispose); } } if (!list_empty(&data.dispose)) From patchwork Thu Nov 9 06:20:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450648 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53430D308 for ; Thu, 9 Nov 2023 06:20:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="fmdzLxwX" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74365A4 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=K8pn4j7PhfDk0ZZ9suKzepKbvr/qzVgdalV7y/0pxco=; b=fmdzLxwXuGIsPkMMXcJ8qc7YxC BVVH9HoyGE9vkvy5tk8fs3X8JnU1NwpVKrn0JMRhrumevsbNIeF/2wI4JmqLErjglz1PtsiUnqLuN peIx+MuyVLHhXViJxArMqZ6S/vz9yqDvg2MTOiMKonu6aYHIfHgbeYtBPY0BgbmdaTe0Yq40SmZqn 3jtoRVP18QhRXIeHizoGMkh43ECm0ZiihyV53Muu+G4d4LFSMsaqWoLOHRKbmo4uHE1HCzvRVGoZn 0FU/4c8JtOGJwpYgZ1t5eeuqAC3DWJl9qCh6MiYKpe1lRNhCAoZu4lCtMqBHAQncZSfL3A/BZgtjK WIoDpqSg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLjP-0P; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 06/22] get rid of __dget() Date: Thu, 9 Nov 2023 06:20:40 +0000 Message-Id: <20231109062056.3181775-6-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro fold into the sole remaining caller Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 56af55f2b7d9..1476f2d6e9ea 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -948,11 +948,6 @@ static inline void __dget_dlock(struct dentry *dentry) dentry->d_lockref.count++; } -static inline void __dget(struct dentry *dentry) -{ - lockref_get(&dentry->d_lockref); -} - struct dentry *dget_parent(struct dentry *dentry) { int gotref; @@ -1002,7 +997,7 @@ static struct dentry * __d_find_any_alias(struct inode *inode) if (hlist_empty(&inode->i_dentry)) return NULL; alias = hlist_entry(inode->i_dentry.first, struct dentry, d_u.d_alias); - __dget(alias); + lockref_get(&alias->d_lockref); return alias; } From patchwork Thu Nov 9 06:20:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450646 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8FF1D511 for ; Thu, 9 Nov 2023 06:21:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="k4COfvS+" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A532426B3 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=af420oe9NRQhAzfLHkPI5lOdq9yCGuAuQ/ZWtCMhJ3I=; b=k4COfvS+Ilyo7paai088t9Ieq9 ETFO5fuEPgmu02k8AWx+u/lQrTEB/VoF7IlD/hjgLp7j/8tKRZTIOPHIG2G0rNTBLA227s0axdhez 7OahiRPJuiF0YOQzW1ifpLFR+Wp2qygD8wC84lBrtbO00IVrYbRa58/JeVFPBLF26k5ECosW5p3ao HbQfeP2Z54QgwnPWHPxqWFjtBRoBWICO43lIzFfCfoLAuExiTBRn+TD/O8YoeJ0h+i6hp+zKS/X7D YA7VOfjuD0QebtxISisWE5EQb6iI7pYX51NNl+9Yk7Iww4u3/sZ04j/m1jnCCCZt4R2vNqKVuegcI Il7K0lTA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLjT-0g; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 07/22] shrink_dentry_list(): no need to check that dentry refcount is marked dead Date: Thu, 9 Nov 2023 06:20:41 +0000 Message-Id: <20231109062056.3181775-7-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ... we won't see DCACHE_MAY_FREE on anything that is *not* dead and checking d_flags is just as cheap as checking refcount. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 1476f2d6e9ea..5371f32eb4bb 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1186,11 +1186,10 @@ void shrink_dentry_list(struct list_head *list) spin_lock(&dentry->d_lock); rcu_read_lock(); if (!shrink_lock_dentry(dentry)) { - bool can_free = false; + bool can_free; rcu_read_unlock(); d_shrink_del(dentry); - if (dentry->d_lockref.count < 0) - can_free = dentry->d_flags & DCACHE_MAY_FREE; + can_free = dentry->d_flags & DCACHE_MAY_FREE; spin_unlock(&dentry->d_lock); if (can_free) dentry_free(dentry); From patchwork Thu Nov 9 06:20:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450647 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBC46D51B for ; Thu, 9 Nov 2023 06:21:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="YlaEhQIc" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7D5426B5 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=dLo5pAAV7Npze4hue4nwU/De/4gQdpB3djuF9lnNgJE=; b=YlaEhQIc2ScoxzT6Cj5igZW8Qo c7SmAWUI3/+4RBnRFY4DaUzi1tkQ3GCHbm7oDtezkEi8boSbxXRUGCCd6mtWHozM4Snml4XI+nuQa G9967cgSkJJuq3jsmdT1dti3o8P2JLl7XkEz8LtPuEKN0D8DWiTWIngaYwjYB+7ejQp3+Xh1dV3sg NdiCXBwqjcHdeiST/Fetaa5orZBpp43kfBQsCusdWKs3IFu2C/QGSdIx/A++TR5+WlucZudyflJHP PXSQV2g31k7RuXdw1oI0Qj6QwcuKbGJtYvT1FlU7GYr4TTJ42fiXIF7kdZM0wvCLT7zMcfgfjozGL 0BVvTxzQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLjb-1H; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 08/22] fast_dput(): having ->d_delete() is not reason to delay refcount decrement Date: Thu, 9 Nov 2023 06:20:42 +0000 Message-Id: <20231109062056.3181775-8-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ->d_delete() is a way for filesystem to tell that dentry is not worth keeping cached. It is not guaranteed to be called every time a dentry has refcount drop down to zero; it is not guaranteed to be called before dentry gets evicted. In other words, it is not suitable for any kind of keeping track of dentry state. None of the in-tree filesystems attempt to use it that way, fortunately. So the contortions done by fast_dput() (as well as dentry_kill()) are not warranted. fast_dput() certainly should treat having ->d_delete() instance as "can't assume we'll be keeping it", but that's not different from the way we treat e.g. DCACHE_DONTCACHE (which is rather similar to making ->d_delete() returns true when called). Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 5371f32eb4bb..0d15e8852ac1 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -768,15 +768,7 @@ static inline bool fast_dput(struct dentry *dentry) unsigned int d_flags; /* - * If we have a d_op->d_delete() operation, we sould not - * let the dentry count go to zero, so use "put_or_lock". - */ - if (unlikely(dentry->d_flags & DCACHE_OP_DELETE)) - return lockref_put_or_lock(&dentry->d_lockref); - - /* - * .. otherwise, we can try to just decrement the - * lockref optimistically. + * try to decrement the lockref optimistically. */ ret = lockref_put_return(&dentry->d_lockref); @@ -830,7 +822,7 @@ static inline bool fast_dput(struct dentry *dentry) */ smp_rmb(); d_flags = READ_ONCE(dentry->d_flags); - d_flags &= DCACHE_REFERENCED | DCACHE_LRU_LIST | + d_flags &= DCACHE_REFERENCED | DCACHE_LRU_LIST | DCACHE_OP_DELETE | DCACHE_DISCONNECTED | DCACHE_DONTCACHE; /* Nothing to do? Dropping the reference was all we needed? */ From patchwork Thu Nov 9 06:20:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450650 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2A07DDB9 for ; Thu, 9 Nov 2023 06:21:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="puhZBA6X" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BDE226B7 for ; Wed, 8 Nov 2023 22:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=B1UXRuyvLvW2BRDHXDf/Cs7UY/KBOqPFDkr7R050HCQ=; b=puhZBA6X6BOQG4go1dmnNq7ssu knraSXSYOOdpwCuHuMcThuoXqeJlWDXRUppRNr5eYiJzlBO/GpDdbm9bkEv6BjZE9ROHpZAvbPFK7 1Cdx7IIcYYbe7XXuMUWVKvEtSWaOE9ZcrD2h240XS51RtDUP00U9jeOG0R8APUKGZfnJAlaXd9brm x53gP4lCUGHDnA3UZQQxNG7Brtf7sa37No7erPpimjBSWBqEClEsqyrlsleKp4laLvABFmBGmJHK5 vlhla7tx5FvBKpyywdjK+z1RtbJW5EO7jNw40sLKdaX8uMBheHo8KaX7vazn8ymWWqc4Ve+qvflOv 0uaaBDPA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLjf-1b; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 09/22] fast_dput(): handle underflows gracefully Date: Thu, 9 Nov 2023 06:20:43 +0000 Message-Id: <20231109062056.3181775-9-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro If refcount is less than 1, we should just warn, unlock dentry and return true, so that the caller doesn't try to do anything else. Taking care of that leaves the rest of "lockref_put_return() has failed" case equivalent to "decrement refcount and rejoin the normal slow path after the point where we grab ->d_lock". NOTE: lockref_put_return() is strictly a fastpath thing - unlike the rest of lockref primitives, it does not contain a fallback. Caller (and it looks like fast_dput() is the only legitimate one in the entire kernel) has to do that itself. Reasons for lockref_put_return() failures: * ->d_lock held by somebody * refcount <= 0 * ... or an architecture not supporting lockref use of cmpxchg - sparc, anything non-SMP, config with spinlock debugging... We could add a fallback, but it would be a clumsy API - we'd have to distinguish between: (1) refcount > 1 - decremented, lock not held on return (2) refcount < 1 - left alone, probably no sense to hold the lock (3) refcount is 1, no cmphxcg - decremented, lock held on return (4) refcount is 1, cmphxcg supported - decremented, lock *NOT* held on return. We want to return with no lock held in case (4); that's the whole point of that thing. We very much do not want to have the fallback in case (3) return without a lock, since the caller might have to retake it in that case. So it wouldn't be more convenient than doing the fallback in the caller and it would be very easy to screw up, especially since the test coverage would suck - no way to test (3) and (4) on the same kernel build. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 0d15e8852ac1..e02b3c81bc02 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -779,12 +779,12 @@ static inline bool fast_dput(struct dentry *dentry) */ if (unlikely(ret < 0)) { spin_lock(&dentry->d_lock); - if (dentry->d_lockref.count > 1) { - dentry->d_lockref.count--; + if (WARN_ON_ONCE(dentry->d_lockref.count <= 0)) { spin_unlock(&dentry->d_lock); return true; } - return false; + dentry->d_lockref.count--; + goto locked; } /* @@ -842,6 +842,7 @@ static inline bool fast_dput(struct dentry *dentry) * else could have killed it and marked it dead. Either way, we * don't need to do anything else. */ +locked: if (dentry->d_lockref.count) { spin_unlock(&dentry->d_lock); return true; From patchwork Thu Nov 9 06:20:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450651 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC6CFDDDE for ; Thu, 9 Nov 2023 06:21:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="cNjrZxV/" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 278A326B8 for ; Wed, 8 Nov 2023 22:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=3cl5MKfhULG7RS/Qf2JbkQO/cYoFke0EzEUpfuhtBiA=; b=cNjrZxV/eogdD/5GTYIlgm0P5N hpbsw/GtF7hXrkfN9GGEFanFEoF9+5UbATnN54K+J0jCFBU5b9W56eK5pnzoQzkGx+6CsKOPMJ/ya jleOTu9KrI/RIl6kZPjX+yPj9JnMRjrKS++jcz5KE+KmehsntGCxky7RpZ2WLPGa2wXehXZRmbMY2 1jSseMePwhEizDsfAPGOFzBFdfv7wGXAYu4dVYP7ZoXoT+QyCi6YeRnRJQ6iWC9M1NymxF4P9lnAo IehEHlOYTUP3uSt8t/5HvMqQKrzyiWzaTzeWifip0Ugs1M5dsxzY4CGDttdlohyuUHZsheyWS4e+n rtzfJF3A==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLjo-2O; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 10/22] fast_dput(): new rules for refcount Date: Thu, 9 Nov 2023 06:20:44 +0000 Message-Id: <20231109062056.3181775-10-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Currently the "need caller to do more work" path in fast_dput() has refcount decremented, then, with ->d_lock held and refcount verified to have reached 0 fast_dput() forcibly resets the refcount to 1. Move that resetting refcount to 1 into the callers; later in the series it will be massaged out of existence. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index e02b3c81bc02..9a3eeee02500 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -847,13 +847,6 @@ static inline bool fast_dput(struct dentry *dentry) spin_unlock(&dentry->d_lock); return true; } - - /* - * Re-get the reference we optimistically dropped. We hold the - * lock, and we just tested that it was zero, so we can just - * set it to 1. - */ - dentry->d_lockref.count = 1; return false; } @@ -896,6 +889,7 @@ void dput(struct dentry *dentry) } /* Slow case: now with the dentry lock held */ + dentry->d_lockref.count = 1; rcu_read_unlock(); if (likely(retain_dentry(dentry))) { @@ -930,6 +924,7 @@ void dput_to_list(struct dentry *dentry, struct list_head *list) return; } rcu_read_unlock(); + dentry->d_lockref.count = 1; if (!retain_dentry(dentry)) __dput_to_list(dentry, list); spin_unlock(&dentry->d_lock); From patchwork Thu Nov 9 06:20:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450652 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFEE9DF4C for ; Thu, 9 Nov 2023 06:21:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="dkA6p3XF" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 425A626B9 for ; Wed, 8 Nov 2023 22:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=Hu+eUF8RnXwLM8LdMLlLxA8gtrHEF5yGM0R5rTsJysg=; b=dkA6p3XF0T2Bj5I0umdl6QtXAC 1Sth58bFEi7uAO5rw1at1xmPUDYKXre+m1TK8peBc3v293WIVjVaTXHggeMiMMQDnjB6Vneug4qTE kwj3mezI+zqWgteSJV580xG0qs1W8spkD7dyPp1MOorSR4GOtO2nBX0dDneTaJvgwcmL2Uyijqg8u pRz8zoCYUt34y96XFzUT5JzTG0WICjrVWjwV/uTHNtU3dwX8qoqBAa/WeDxIIB/K2hR3tlPzJPp/g Jwu05AKUeFhXReSQlBUO6MRpJuq2sMySDTsQP7JrxuVJY+VbsptqqBL8r9PdvUz6fwucA2BHOlcNG ZkGZAXBw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPR-00DLju-2t; Thu, 09 Nov 2023 06:20:57 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 11/22] __dput_to_list(): do decrement of refcount in the callers Date: Thu, 9 Nov 2023 06:20:45 +0000 Message-Id: <20231109062056.3181775-11-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ... and rename it to to_shrink_list(), seeing that it no longer does dropping any references Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 9a3eeee02500..1899376d0189 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -902,16 +902,13 @@ void dput(struct dentry *dentry) } EXPORT_SYMBOL(dput); -static void __dput_to_list(struct dentry *dentry, struct list_head *list) +static void to_shrink_list(struct dentry *dentry, struct list_head *list) __must_hold(&dentry->d_lock) { - if (dentry->d_flags & DCACHE_SHRINK_LIST) { - /* let the owner of the list it's on deal with it */ - --dentry->d_lockref.count; - } else { + if (!(dentry->d_flags & DCACHE_SHRINK_LIST)) { if (dentry->d_flags & DCACHE_LRU_LIST) d_lru_del(dentry); - if (!--dentry->d_lockref.count) + if (!dentry->d_lockref.count) d_shrink_add(dentry, list); } } @@ -925,8 +922,10 @@ void dput_to_list(struct dentry *dentry, struct list_head *list) } rcu_read_unlock(); dentry->d_lockref.count = 1; - if (!retain_dentry(dentry)) - __dput_to_list(dentry, list); + if (!retain_dentry(dentry)) { + --dentry->d_lockref.count; + to_shrink_list(dentry, list); + } spin_unlock(&dentry->d_lock); } @@ -1160,8 +1159,10 @@ static bool shrink_lock_dentry(struct dentry *dentry) static inline void shrink_kill(struct dentry *victim, struct list_head *list) { struct dentry *parent = victim->d_parent; - if (parent != victim) - __dput_to_list(parent, list); + if (parent != victim) { + --parent->d_lockref.count; + to_shrink_list(parent, list); + } __dentry_kill(victim); } From patchwork Thu Nov 9 06:20:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450653 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BC64DF65 for ; Thu, 9 Nov 2023 06:21:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="FZZimF2D" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 739BF26BA for ; Wed, 8 Nov 2023 22:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=Dl1U3hsH+ZZfRd8OQK17UqFasenI4TzgwGl6eAgyezU=; b=FZZimF2D3bQU5OupCF+kBO0iZL oRZJX9FBbyB7SNba4FSF3SECK5Qw3xz/9eRl1kDCK2NzHIlnIcZOwO3TgNYGFOZtvOkYOd7TWbFte cqeAFW56Fs5tGmvHQPV5xFRiQHqXpWq0nFY7QpWcSm2fuN1cOxizhDKoyT5BXxIQhZBjSxxkFSPK1 inipJGVgfYhcN016X902Euxf0x1bPZBa3yMkYUm+Y1GnLAqg5W6Qh10lriSQQoD0DJBiIb66fh96V KoJFRatwC0Me6rHdiBV7qHF9kM8YeZpag2n11eylTVP7n6wzDp78UsXTRc13p3a0MJdx17OK8Y5F8 rerpXDFA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLjz-02; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 12/22] Make retain_dentry() neutral with respect to refcounting Date: Thu, 9 Nov 2023 06:20:46 +0000 Message-Id: <20231109062056.3181775-12-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro retain_dentry() used to decrement refcount if and only if it returned true. Lift those decrements into the callers. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/dcache.c b/fs/dcache.c index 1899376d0189..1f61a5d03d5b 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -680,7 +680,6 @@ static inline bool retain_dentry(struct dentry *dentry) return false; /* retain; LRU fodder */ - dentry->d_lockref.count--; if (unlikely(!(dentry->d_flags & DCACHE_LRU_LIST))) d_lru_add(dentry); else if (unlikely(!(dentry->d_flags & DCACHE_REFERENCED))) @@ -744,6 +743,8 @@ static struct dentry *dentry_kill(struct dentry *dentry) } else if (likely(!retain_dentry(dentry))) { __dentry_kill(dentry); return parent; + } else { + dentry->d_lockref.count--; } /* we are keeping it, after all */ if (inode) @@ -893,6 +894,7 @@ void dput(struct dentry *dentry) rcu_read_unlock(); if (likely(retain_dentry(dentry))) { + dentry->d_lockref.count--; spin_unlock(&dentry->d_lock); return; } @@ -925,6 +927,8 @@ void dput_to_list(struct dentry *dentry, struct list_head *list) if (!retain_dentry(dentry)) { --dentry->d_lockref.count; to_shrink_list(dentry, list); + } else { + --dentry->d_lockref.count; } spin_unlock(&dentry->d_lock); } From patchwork Thu Nov 9 06:20:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450656 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DE31E574 for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="Xr6xK9r5" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4E7D26BC for ; Wed, 8 Nov 2023 22:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=fqgEuHsebdDpFMPtEzF/nzVbQVktYGU1hkEJxaEkZjE=; b=Xr6xK9r5kYOj73ojc91FN+5yZG mirO/VPTyTgtXSwxxlDhaOKi8ekNzq3J9akpgNCa0TBF3V3v+TTXWHUzMRsrsOxZXjJQImoML8Ut6 VByJJ6e2y6/PkUJg3DjKUjoRFrarkV+BrjLmySK3b2+EmWQsRST9Ifx1sAg09uXIttzkK8MBBNc+c 6X+tdvq38iEedN0B3OO+cAe07z+nakrGw7jwv/KEyIzYUqwcztLTkJlm5p0Pa+L5H5u7yvV/52GBD GhhX2cou0tEeHnRZ0TBZZIaQASZyIxL1pzA2OtLhrbufTEJ6SD975JGsu6p0pPRwy6s0NQwncSCF4 5gMkOtqg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLk8-0h; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 13/22] __dentry_kill(): get consistent rules for victim's refcount Date: Thu, 9 Nov 2023 06:20:47 +0000 Message-Id: <20231109062056.3181775-13-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Currently we call it with refcount equal to 1 when called from dentry_kill(); all other callers have it equal to 0. Make it always be called with zero refcount; on this step we just decrement it before the calls in dentry_kill(). That is safe, since all places that care about the value of refcount either do that under ->d_lock or hold a reference to dentry in question. Either is sufficient to prevent observing a dentry immediately prior to __dentry_kill() getting called from dentry_kill(). Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/dcache.c b/fs/dcache.c index 1f61a5d03d5b..d9466cab4884 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -729,6 +729,7 @@ static struct dentry *dentry_kill(struct dentry *dentry) goto slow_positive; } } + dentry->d_lockref.count--; __dentry_kill(dentry); return parent; @@ -741,6 +742,7 @@ static struct dentry *dentry_kill(struct dentry *dentry) if (unlikely(dentry->d_lockref.count != 1)) { dentry->d_lockref.count--; } else if (likely(!retain_dentry(dentry))) { + dentry->d_lockref.count--; __dentry_kill(dentry); return parent; } else { From patchwork Thu Nov 9 06:20:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450654 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A62CCF9CA for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="KohcKxnM" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E012F26BB for ; Wed, 8 Nov 2023 22:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=PInzk+mI0vspPJDyiH3pEq66fnzbTjLEhHK7jQVOhfM=; b=KohcKxnMdW2h0GwnluNiLCtUb+ 0anTr4wcBDc4TXoZJ4By+9fXLxDEukQKSaQue6yIiqCWeYlum9Solw3T3Nlwu0LGJHvNvt8ufigFH 6UVLR12AvIlm7Zd8RwdfJoYG62PGunnovWQL+SsBByXq3GHRiroxUa1GolL4mtWnwWwMgP90jA+BR ouYSMr1Az9MWFSd8AaKsZHPwUfp5KubixxXAGR+7igs2frR92coBkF8stqF7QIvoZZOqkMNyrLziF +XwTaaaJ0zUowB09W+fLZFLiEzLQJJZ8jZ7LF3D3eQ+pe/SnsBf0apuv/ZEi7PwmuiIhl4yhIKfWl zkDZxXSg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLkE-1P; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 14/22] dentry_kill(): don't bother with retain_dentry() on slow path Date: Thu, 9 Nov 2023 06:20:48 +0000 Message-Id: <20231109062056.3181775-14-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro We have already checked it and dentry used to look not worthy of keeping. The only hard obstacle to evicting dentry is non-zero refcount; everything else is advisory - e.g. memory pressure could evict any dentry found with refcount zero. On the slow path in dentry_kill() we had dropped and regained ->d_lock; we must recheck the refcount, but everything else is not worth bothering with. Note that filesystem can not count upon ->d_delete() being called for dentry - not even once. Again, memory pressure (as well as d_prune_aliases(), or attempted rmdir() of ancestor, or...) will not call ->d_delete() at all. So from the correctness point of view we are fine doing the check only once. And it makes things simpler down the road. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index d9466cab4884..916b978bfd98 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -739,14 +739,10 @@ static struct dentry *dentry_kill(struct dentry *dentry) spin_lock(&dentry->d_lock); parent = lock_parent(dentry); got_locks: - if (unlikely(dentry->d_lockref.count != 1)) { - dentry->d_lockref.count--; - } else if (likely(!retain_dentry(dentry))) { - dentry->d_lockref.count--; + dentry->d_lockref.count--; + if (likely(dentry->d_lockref.count == 0)) { __dentry_kill(dentry); return parent; - } else { - dentry->d_lockref.count--; } /* we are keeping it, after all */ if (inode) From patchwork Thu Nov 9 06:20:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450658 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAE95F9E6 for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="rTL3odkZ" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BB1826BD for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=LRStAQRG1xPuZNqdA9lfkj9lOCP32UT35l5hG/N2K/Q=; b=rTL3odkZa+UVc+nferHu3Z+go1 KWH4cj5uvumPjMCkJNOU8f2+Cl7fGVZw3VF0JIcdeMkxTQv2Qz0KR9jP4qiPhC96DE6mzvSt3E2Lg KclYUyEl9CfVWM2VmYd8LaR12IrM2xIVuteApL91DdxhpQ7i+KAyyHpI3vAv1TEIOMwCm8K5vciIa jEd73SuB3L1MQ8TAiyONfiwl4VJN27l853JjE3CyD40D3hIu2yqfNN+PugRaRDc80E4nljJGFVxlB wxno7Ic7/kBr88bhTTPy6Xi6gLGq5vOIuVxLN6l1xDcb+SInbJJqoicumlxOwYEWHeIqBbHn0XcLb wmOERhyg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLkM-24; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 15/22] Call retain_dentry() with refcount 0 Date: Thu, 9 Nov 2023 06:20:49 +0000 Message-Id: <20231109062056.3181775-15-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Instead of bumping it from 0 to 1, calling retain_dentry(), then decrementing it back to 0 (with ->d_lock held all the way through), just leave refcount at 0 through all of that. It will have a visible effect for ->d_delete() - now it can be called with refcount 0 instead of 1 and it can no longer play silly buggers with dropping/regaining ->d_lock. Not that any in-tree instances tried to (it's pretty hard to get right). Any out-of-tree ones will have to adjust (assuming they need any changes). Note that we do not need to extend rcu-critical area here - we have verified that refcount is non-negative after having grabbed ->d_lock, so nobody will be able to free dentry until they get into __dentry_kill(), which won't happen until they manage to grab ->d_lock. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- Documentation/filesystems/porting.rst | 8 ++++++++ fs/dcache.c | 10 ++-------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/Documentation/filesystems/porting.rst b/Documentation/filesystems/porting.rst index 58627f0baf3e..6b058362938c 100644 --- a/Documentation/filesystems/porting.rst +++ b/Documentation/filesystems/porting.rst @@ -1054,3 +1054,11 @@ The list of children anchored in parent dentry got turned into hlist now. Field names got changed (->d_children/->d_sib instead of ->d_subdirs/->d_child for anchor/entries resp.), so any affected places will be immediately caught by compiler. + +--- + +**mandatory** + + ->d_delete() instances are now called for dentries with ->d_lock held +and refcount equal to 0. They are not permitted to drop/regain ->d_lock. +None of in-tree instances did anything of that sort. Make sure yours do not... diff --git a/fs/dcache.c b/fs/dcache.c index 916b978bfd98..3179156e0ad9 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -888,15 +888,14 @@ void dput(struct dentry *dentry) } /* Slow case: now with the dentry lock held */ - dentry->d_lockref.count = 1; rcu_read_unlock(); if (likely(retain_dentry(dentry))) { - dentry->d_lockref.count--; spin_unlock(&dentry->d_lock); return; } + dentry->d_lockref.count = 1; dentry = dentry_kill(dentry); } } @@ -921,13 +920,8 @@ void dput_to_list(struct dentry *dentry, struct list_head *list) return; } rcu_read_unlock(); - dentry->d_lockref.count = 1; - if (!retain_dentry(dentry)) { - --dentry->d_lockref.count; + if (!retain_dentry(dentry)) to_shrink_list(dentry, list); - } else { - --dentry->d_lockref.count; - } spin_unlock(&dentry->d_lock); } From patchwork Thu Nov 9 06:20:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450655 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC0D9F9EE for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="SYyU9Jw9" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F14B26BE for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=wC5MrqC19y5tPs0/NQr3BfdHwd4jtwlwzmqo3EzkzDU=; b=SYyU9Jw9tKUj6N3p1RUdVLp0S3 Vy895DrsQ+kDJ2wYrki76kCxiCZmDXitdt07ANUslfx6YxHwMfmA6y8W87UeP5PmtuJ/N+qHnKBdw qhQ1oTlo4nX1suIPQTol2fXt1YFRcEMYOYTep6N+Fwn4rDJFKk1XP+e13CfBNjfPP+/rBAbESc/Z0 VY29JM9Y8TA2hmnsZfimuS0SFwLnSGIpgEH37W8A3Kwd6Z1su4sKWjfmBHPAMH7BhbtLu/KJj3bHX PaX2e9AFRSen6qLvQqcf0ZVXPg6TgXCLqyefMZtuOP0lIshAss+nH8Un+WK0sxYxYzfJWT/zqpv0j S8FfD04g==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLkQ-2U; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 16/22] fold the call of retain_dentry() into fast_dput() Date: Thu, 9 Nov 2023 06:20:50 +0000 Message-Id: <20231109062056.3181775-16-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Calls of retain_dentry() happen immediately after getting false from fast_dput() and getting true from retain_dentry() is treated the same way as non-zero refcount would be treated by fast_dput() - unlock dentry and bugger off. Doing that in fast_dput() itself is simpler. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 3179156e0ad9..23afcd48c1a9 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -757,6 +757,8 @@ static struct dentry *dentry_kill(struct dentry *dentry) * Try to do a lockless dput(), and return whether that was successful. * * If unsuccessful, we return false, having already taken the dentry lock. + * In that case refcount is guaranteed to be zero and we have already + * decided that it's not worth keeping around. * * The caller needs to hold the RCU read lock, so that the dentry is * guaranteed to stay around even if the refcount goes down to zero! @@ -842,7 +844,7 @@ static inline bool fast_dput(struct dentry *dentry) * don't need to do anything else. */ locked: - if (dentry->d_lockref.count) { + if (dentry->d_lockref.count || retain_dentry(dentry)) { spin_unlock(&dentry->d_lock); return true; } @@ -889,12 +891,6 @@ void dput(struct dentry *dentry) /* Slow case: now with the dentry lock held */ rcu_read_unlock(); - - if (likely(retain_dentry(dentry))) { - spin_unlock(&dentry->d_lock); - return; - } - dentry->d_lockref.count = 1; dentry = dentry_kill(dentry); } @@ -920,8 +916,7 @@ void dput_to_list(struct dentry *dentry, struct list_head *list) return; } rcu_read_unlock(); - if (!retain_dentry(dentry)) - to_shrink_list(dentry, list); + to_shrink_list(dentry, list); spin_unlock(&dentry->d_lock); } From patchwork Thu Nov 9 06:20:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450663 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D496BFBF1 for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="deT9fqpK" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C77626BF for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=4F2IBv00iVC4mCIfZ58yB5AhswlElgWmvmEObB5ouMk=; b=deT9fqpKYmkigAK5Jhmt+NdZQ2 eJ1pe9esA9a/r8fnv5hhxBORVcehsvtJLPRSAtSmo0Hro8mmsX2wNeKBJ5eBkfY/4fpfnvCEdc7kY 0uaoJGhD+SvmIlOmxrAclTmF+ghJ4tja+CXhnTi/KrfoTbm2jlFRsjDic7Hd3vtVyqup5oXpncHI1 dt/qS4kdvLdcVGePDYt4rZBhM3aJu1libgwQoDrYAEygB9goEtLJvVaYkh4tJU+gnXjm5OWWS+4JS MBl52qdJ0BRb2s/ThOVhHhEiw4DMgpkloP4fNG5jPW+K0wsIrFdS65XEyYYk+WWrU164UozuhiHfL /M1/gFlg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPS-00DLkW-34; Thu, 09 Nov 2023 06:20:58 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 17/22] don't try to cut corners in shrink_lock_dentry() Date: Thu, 9 Nov 2023 06:20:51 +0000 Message-Id: <20231109062056.3181775-17-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro That is to say, do *not* treat the ->d_inode or ->d_parent changes as "it's hard, return false; somebody must have grabbed it, so even if has zero refcount, we don't need to bother killing it - final dput() from whoever grabbed it would've done everything". First of all, that is not guaranteed. It might have been dropped by shrink_kill() handling of victim's parent, which would've found it already on a shrink list (ours) and decided that they don't need to put it on their shrink list. What's more, dentry_kill() is doing pretty much the same thing, cutting its own set of corners (it assumes that dentry can't go from positive to negative, so its inode can change but only once and only in one direction). Doing that right allows to get rid of that not-quite-duplication and removes the only reason for re-incrementing refcount before the call of dentry_kill(). Replacement is called lock_for_kill(); called under rcu_read_lock and with ->d_lock held. If it returns false, dentry has non-zero refcount and the same locks are held. If it returns true, dentry has zero refcount and all locks required by __dentry_kill() are taken. Part of __lock_parent() had been lifted into lock_parent() to allow its reuse. Now it's called with rcu_read_lock already held and dentry already unlocked. Note that this is not the final change - locking requirements for __dentry_kill() are going to change later in the series and the set of locks taken by lock_for_kill() will be adjusted. Signed-off-by: Al Viro --- fs/dcache.c | 159 ++++++++++++++++++++++------------------------------ 1 file changed, 66 insertions(+), 93 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 23afcd48c1a9..801502871671 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -625,8 +625,6 @@ static void __dentry_kill(struct dentry *dentry) static struct dentry *__lock_parent(struct dentry *dentry) { struct dentry *parent; - rcu_read_lock(); - spin_unlock(&dentry->d_lock); again: parent = READ_ONCE(dentry->d_parent); spin_lock(&parent->d_lock); @@ -642,7 +640,6 @@ static struct dentry *__lock_parent(struct dentry *dentry) spin_unlock(&parent->d_lock); goto again; } - rcu_read_unlock(); if (parent != dentry) spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED); else @@ -657,7 +654,64 @@ static inline struct dentry *lock_parent(struct dentry *dentry) return NULL; if (likely(spin_trylock(&parent->d_lock))) return parent; - return __lock_parent(dentry); + rcu_read_lock(); + spin_unlock(&dentry->d_lock); + parent = __lock_parent(dentry); + rcu_read_unlock(); + return parent; +} + +/* + * Lock a dentry for feeding it to __dentry_kill(). + * Called under rcu_read_lock() and dentry->d_lock; the former + * guarantees that nothing we access will be freed under us. + * Note that dentry is *not* protected from concurrent dentry_kill(), + * d_delete(), etc. + * + * Return false if dentry is busy. Otherwise, return true and have + * that dentry's inode and parent both locked. + */ + +static bool lock_for_kill(struct dentry *dentry) +{ + struct inode *inode = dentry->d_inode; + struct dentry *parent = dentry->d_parent; + + if (unlikely(dentry->d_lockref.count)) + return false; + + if (inode && unlikely(!spin_trylock(&inode->i_lock))) + goto slow; + if (dentry == parent) + return true; + if (likely(spin_trylock(&parent->d_lock))) + return true; + + if (inode) + spin_unlock(&inode->i_lock); +slow: + spin_unlock(&dentry->d_lock); + + for (;;) { + if (inode) + spin_lock(&inode->i_lock); + parent = __lock_parent(dentry); + if (likely(inode == dentry->d_inode)) + break; + if (inode) + spin_unlock(&inode->i_lock); + inode = dentry->d_inode; + spin_unlock(&dentry->d_lock); + if (parent) + spin_unlock(&parent->d_lock); + } + if (likely(!dentry->d_lockref.count)) + return true; + if (inode) + spin_unlock(&inode->i_lock); + if (parent) + spin_unlock(&parent->d_lock); + return false; } static inline bool retain_dentry(struct dentry *dentry) @@ -710,45 +764,16 @@ EXPORT_SYMBOL(d_mark_dontcache); static struct dentry *dentry_kill(struct dentry *dentry) __releases(dentry->d_lock) { - struct inode *inode = dentry->d_inode; - struct dentry *parent = NULL; - if (inode && unlikely(!spin_trylock(&inode->i_lock))) - goto slow_positive; - - if (!IS_ROOT(dentry)) { - parent = dentry->d_parent; - if (unlikely(!spin_trylock(&parent->d_lock))) { - parent = __lock_parent(dentry); - if (likely(inode || !dentry->d_inode)) - goto got_locks; - /* negative that became positive */ - if (parent) - spin_unlock(&parent->d_lock); - inode = dentry->d_inode; - goto slow_positive; - } - } dentry->d_lockref.count--; - __dentry_kill(dentry); - return parent; - -slow_positive: - spin_unlock(&dentry->d_lock); - spin_lock(&inode->i_lock); - spin_lock(&dentry->d_lock); - parent = lock_parent(dentry); -got_locks: - dentry->d_lockref.count--; - if (likely(dentry->d_lockref.count == 0)) { + rcu_read_lock(); + if (likely(lock_for_kill(dentry))) { + struct dentry *parent = dentry->d_parent; + rcu_read_unlock(); __dentry_kill(dentry); - return parent; + return parent != dentry ? parent : NULL; } - /* we are keeping it, after all */ - if (inode) - spin_unlock(&inode->i_lock); - if (parent) - spin_unlock(&parent->d_lock); + rcu_read_unlock(); spin_unlock(&dentry->d_lock); return NULL; } @@ -1095,58 +1120,6 @@ void d_prune_aliases(struct inode *inode) } EXPORT_SYMBOL(d_prune_aliases); -/* - * Lock a dentry from shrink list. - * Called under rcu_read_lock() and dentry->d_lock; the former - * guarantees that nothing we access will be freed under us. - * Note that dentry is *not* protected from concurrent dentry_kill(), - * d_delete(), etc. - * - * Return false if dentry has been disrupted or grabbed, leaving - * the caller to kick it off-list. Otherwise, return true and have - * that dentry's inode and parent both locked. - */ -static bool shrink_lock_dentry(struct dentry *dentry) -{ - struct inode *inode; - struct dentry *parent; - - if (dentry->d_lockref.count) - return false; - - inode = dentry->d_inode; - if (inode && unlikely(!spin_trylock(&inode->i_lock))) { - spin_unlock(&dentry->d_lock); - spin_lock(&inode->i_lock); - spin_lock(&dentry->d_lock); - if (unlikely(dentry->d_lockref.count)) - goto out; - /* changed inode means that somebody had grabbed it */ - if (unlikely(inode != dentry->d_inode)) - goto out; - } - - parent = dentry->d_parent; - if (IS_ROOT(dentry) || likely(spin_trylock(&parent->d_lock))) - return true; - - spin_unlock(&dentry->d_lock); - spin_lock(&parent->d_lock); - if (unlikely(parent != dentry->d_parent)) { - spin_unlock(&parent->d_lock); - spin_lock(&dentry->d_lock); - goto out; - } - spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED); - if (likely(!dentry->d_lockref.count)) - return true; - spin_unlock(&parent->d_lock); -out: - if (inode) - spin_unlock(&inode->i_lock); - return false; -} - static inline void shrink_kill(struct dentry *victim, struct list_head *list) { struct dentry *parent = victim->d_parent; @@ -1165,7 +1138,7 @@ void shrink_dentry_list(struct list_head *list) dentry = list_entry(list->prev, struct dentry, d_lru); spin_lock(&dentry->d_lock); rcu_read_lock(); - if (!shrink_lock_dentry(dentry)) { + if (!lock_for_kill(dentry)) { bool can_free; rcu_read_unlock(); d_shrink_del(dentry); @@ -1609,7 +1582,7 @@ void shrink_dcache_parent(struct dentry *parent) d_walk(parent, &data, select_collect2); if (data.victim) { spin_lock(&data.victim->d_lock); - if (!shrink_lock_dentry(data.victim)) { + if (!lock_for_kill(data.victim)) { spin_unlock(&data.victim->d_lock); rcu_read_unlock(); } else { From patchwork Thu Nov 9 06:20:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450661 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 170C4FC10 for ; Thu, 9 Nov 2023 06:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="hC2w0bY7" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82B7826A9 for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=cvSkCD6Y5Lu7XfPcNtGN7XQGq4nsRS8djzBQjlp4Gtc=; b=hC2w0bY7JAaQTOdY/yWMdGx9Ji l3kSH+yW3i8djo1xUTECKVs1R8RD1FEp9Ex820mkdjFdChyILEclwlVT7uYlTXErW7f1eN6c3MMD2 g4ir8TANnDr+j+nC3CNWo+25kC3zz0Ot5Ll52+3CUiPqFik2ZZi5wcmPwky0W6GA4vVTWYnlpSnx1 HIKa7K4Ro+WMYG+Gik+Dcyau5eBgtviUSP1I1q3z3SGU2IxABGYEaZ0yIAR6GPuXH/wJ8XV4N9fwA ssnHAqEHSEEMl0/yCTfMo14FVY1wBfjGq0TrGfOy7WRY+XsUCqPAq7ZXnYWKWIjYvFzKX8mKme0h9 N+nTfDiQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPT-00DLka-0D; Thu, 09 Nov 2023 06:20:59 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 18/22] fold dentry_kill() into dput() Date: Thu, 9 Nov 2023 06:20:52 +0000 Message-Id: <20231109062056.3181775-18-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 37 ++++++++++++------------------------- 1 file changed, 12 insertions(+), 25 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 801502871671..aa9f7ee7a603 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -756,28 +756,6 @@ void d_mark_dontcache(struct inode *inode) } EXPORT_SYMBOL(d_mark_dontcache); -/* - * Finish off a dentry we've decided to kill. - * dentry->d_lock must be held, returns with it unlocked. - * Returns dentry requiring refcount drop, or NULL if we're done. - */ -static struct dentry *dentry_kill(struct dentry *dentry) - __releases(dentry->d_lock) -{ - - dentry->d_lockref.count--; - rcu_read_lock(); - if (likely(lock_for_kill(dentry))) { - struct dentry *parent = dentry->d_parent; - rcu_read_unlock(); - __dentry_kill(dentry); - return parent != dentry ? parent : NULL; - } - rcu_read_unlock(); - spin_unlock(&dentry->d_lock); - return NULL; -} - /* * Try to do a lockless dput(), and return whether that was successful. * @@ -915,9 +893,18 @@ void dput(struct dentry *dentry) } /* Slow case: now with the dentry lock held */ - rcu_read_unlock(); - dentry->d_lockref.count = 1; - dentry = dentry_kill(dentry); + if (likely(lock_for_kill(dentry))) { + struct dentry *parent = dentry->d_parent; + rcu_read_unlock(); + __dentry_kill(dentry); + if (dentry == parent) + return; + dentry = parent; + } else { + rcu_read_unlock(); + spin_unlock(&dentry->d_lock); + return; + } } } EXPORT_SYMBOL(dput); From patchwork Thu Nov 9 06:20:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450659 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58E5E101C3 for ; Thu, 9 Nov 2023 06:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="em8166ug" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A882F2700 for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=CoNG7RM/Qk8hwAjsykczrRFcU4ltOzOlM2mSsueSOJ8=; b=em8166ugkSKtKb7VFrx9mtrlHI 7zgt4msQZaTcWLLmG0ZnP1PgUqv9VSJUdIL3/IXD0LDo1GlEwMDIb3qPVj2aQdKdwOKdgRT5qG8oH zEXISTeaqbNssgGEb7eoAjBlLiLDuVoc0PVgE/Pc/PRViLCHiAwv3CYEWkg1zjjxzLkPqVYCzCOz0 ++iqM/kzF7+BVmWv0in6y37xcsqbz83jRVuEtGrq34wVYB4HcjIRdYaePe2sdBiw8/J3RrK2RFBeD SWYRlCqry4QleUEOkCciwhuz32qgQCb07l/qc1P1m8N5GeG1UyBJOfSfZkcuLiN2zY3U/TPFqLmDk Zq8MJmCw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPT-00DLki-0s; Thu, 09 Nov 2023 06:20:59 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 19/22] to_shrink_list(): call only if refcount is 0 Date: Thu, 9 Nov 2023 06:20:53 +0000 Message-Id: <20231109062056.3181775-19-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro The only thing it does if refcount is not zero is d_lru_del(); no point, IMO, seeing that plain dput() does nothing of that sort... Note that 2 of 3 current callers are guaranteed that refcount is 0. Signed-off-by: Al Viro Acked-by: Christian Brauner --- fs/dcache.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index aa9f7ee7a603..49585f2ad896 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -915,8 +915,7 @@ __must_hold(&dentry->d_lock) if (!(dentry->d_flags & DCACHE_SHRINK_LIST)) { if (dentry->d_flags & DCACHE_LRU_LIST) d_lru_del(dentry); - if (!dentry->d_lockref.count) - d_shrink_add(dentry, list); + d_shrink_add(dentry, list); } } @@ -1110,10 +1109,8 @@ EXPORT_SYMBOL(d_prune_aliases); static inline void shrink_kill(struct dentry *victim, struct list_head *list) { struct dentry *parent = victim->d_parent; - if (parent != victim) { - --parent->d_lockref.count; + if (parent != victim && !--parent->d_lockref.count) to_shrink_list(parent, list); - } __dentry_kill(victim); } From patchwork Thu Nov 9 06:20:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450660 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41B67FC1C for ; Thu, 9 Nov 2023 06:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="BHhlV0hk" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1E642701 for ; Wed, 8 Nov 2023 22:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=k7SKMdAIr8mlWzKY3NVr+eITorVToK/c6RS5RJxeim0=; b=BHhlV0hkXmWSPynTj+ejbSWvzD o6vWTOxLZ5eccV4yHfoJTHex8E9Qa/omhhfMGsx/jPy3kmiuR8E+U+Bb59DbEbVBObPJNNj7uoV8x OdSkG/7c4VQCnbwba7gQR/tr3GKVa2vrh3y0QE+nu50UzGg/t2XhAjlkLWLWfhsLRTJxnSmTT/Jtz 5VMGugEqDAR8IcxGfbv7ywWgwP8kDEAz2Rdbi6IZ14CQBVt9XQkJdcASbdQlbXAmT5i2TRbXhBMOO q10JYt/WIWfftQn39XtAmxnzhVZL09GZexYX4MMv7mkQRdqZESdb8bMCNEz+7sVsZCJHJ3Nl86VMy KldAVfkw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPT-00DLko-1I; Thu, 09 Nov 2023 06:20:59 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 20/22] switch select_collect{,2}() to use of to_shrink_list() Date: Thu, 9 Nov 2023 06:20:54 +0000 Message-Id: <20231109062056.3181775-20-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 49585f2ad896..5fdb6342f659 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1490,13 +1490,9 @@ static enum d_walk_ret select_collect(void *_data, struct dentry *dentry) if (dentry->d_flags & DCACHE_SHRINK_LIST) { data->found++; - } else { - if (dentry->d_flags & DCACHE_LRU_LIST) - d_lru_del(dentry); - if (!dentry->d_lockref.count) { - d_shrink_add(dentry, &data->dispose); - data->found++; - } + } else if (!dentry->d_lockref.count) { + to_shrink_list(dentry, &data->dispose); + data->found++; } /* * We can return to the caller if we have found some (this @@ -1517,17 +1513,13 @@ static enum d_walk_ret select_collect2(void *_data, struct dentry *dentry) if (data->start == dentry) goto out; - if (dentry->d_flags & DCACHE_SHRINK_LIST) { - if (!dentry->d_lockref.count) { + if (!dentry->d_lockref.count) { + if (dentry->d_flags & DCACHE_SHRINK_LIST) { rcu_read_lock(); data->victim = dentry; return D_WALK_QUIT; } - } else { - if (dentry->d_flags & DCACHE_LRU_LIST) - d_lru_del(dentry); - if (!dentry->d_lockref.count) - d_shrink_add(dentry, &data->dispose); + to_shrink_list(dentry, &data->dispose); } /* * We can return to the caller if we have found some (this From patchwork Thu Nov 9 06:20:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450657 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20CA8101FA for ; Thu, 9 Nov 2023 06:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="cOHpkUe5" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09D462702 for ; Wed, 8 Nov 2023 22:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=8JrfYu8MGPKD0zyqknMnZOhQ00UwCqeYF/nP0UHd/bQ=; b=cOHpkUe5eoF7PyczzdDMPXWYiT kLymMwHO8bGvBs3j6mpTObqFoGv1L2vQOPujlD8VYTp6rg9/AAJg+Ot9P9MgcJRALLJG8gl6n6RQS +RadjgKifpNBIfTINJS4QVlTN4CFzEDRE19RmDH2Z5kkdZJbl7KufNezYeTZ0SYpk+TweCdMdy34I dGeqUjRAqCCFyIoaBKH5SB4UB/+QycIlVcLuAU0V0HHBE9Mb1cIvuHA571pwkrfUtR7vRyHlbPSy4 Q9daB92SCHPGoG06pGYkUfHc8PqLzKOXlG7nrkd8tz+mpeHCorNJu5tCVKevBOyIOzj2R8OkUqjWp iTAzU0oA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPT-00DLkw-26; Thu, 09 Nov 2023 06:20:59 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 21/22] d_prune_aliases(): use a shrink list Date: Thu, 9 Nov 2023 06:20:55 +0000 Message-Id: <20231109062056.3181775-21-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Instead of dropping aliases one by one, restarting, etc., just collect them into a shrink list and kill them off in one pass. We don't really need the restarts - one alias can't pin another (directory has only one alias, and couldn't be its own ancestor anyway), so collecting everything that is not busy and taking it out would take care of everything evictable that had been there as we entered the function. And new aliases added while we'd been dropping old ones could just as easily have appeared right as we return to caller... Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- fs/dcache.c | 30 +++++------------------------- 1 file changed, 5 insertions(+), 25 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 5fdb6342f659..cea707a77e28 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -647,20 +647,6 @@ static struct dentry *__lock_parent(struct dentry *dentry) return parent; } -static inline struct dentry *lock_parent(struct dentry *dentry) -{ - struct dentry *parent = dentry->d_parent; - if (IS_ROOT(dentry)) - return NULL; - if (likely(spin_trylock(&parent->d_lock))) - return parent; - rcu_read_lock(); - spin_unlock(&dentry->d_lock); - parent = __lock_parent(dentry); - rcu_read_unlock(); - return parent; -} - /* * Lock a dentry for feeding it to __dentry_kill(). * Called under rcu_read_lock() and dentry->d_lock; the former @@ -1085,24 +1071,18 @@ struct dentry *d_find_alias_rcu(struct inode *inode) */ void d_prune_aliases(struct inode *inode) { + LIST_HEAD(dispose); struct dentry *dentry; -restart: + spin_lock(&inode->i_lock); hlist_for_each_entry(dentry, &inode->i_dentry, d_u.d_alias) { spin_lock(&dentry->d_lock); - if (!dentry->d_lockref.count) { - struct dentry *parent = lock_parent(dentry); - if (likely(!dentry->d_lockref.count)) { - __dentry_kill(dentry); - dput(parent); - goto restart; - } - if (parent) - spin_unlock(&parent->d_lock); - } + if (!dentry->d_lockref.count) + to_shrink_list(dentry, &dispose); spin_unlock(&dentry->d_lock); } spin_unlock(&inode->i_lock); + shrink_dentry_list(&dispose); } EXPORT_SYMBOL(d_prune_aliases); From patchwork Thu Nov 9 06:20:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13450662 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EA9C101F8 for ; Thu, 9 Nov 2023 06:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="U/Be74dI" Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [IPv6:2a03:a000:7:0:5054:ff:fe1c:15ff]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 322FB2703 for ; Wed, 8 Nov 2023 22:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=EbcACU2j0WN5jFUGaD4xtuX91jcX1843otiVMn3JvxU=; b=U/Be74dI9fM5buqYG6WG/HvKCD 4R+njpNZjh4jlVHEXzan9U5BQOxOl/OguxhEZ4QXHL6WNPD+uG/kNfqW+ZM7iNS3rtaRlwiPUQSqb uM7H3CnL9eT/uq7Sa500pjmltPxmHU7ij2Zpks3mU94V+uJW8rx6gRJXIAn6VewORpD/DKMEwOZCH cj1z15IoYJixSSAp4MGZaxkl+ZFROAXKYVls44Ikuz3NttTKZtjIT/LBwchBYflmU0mWXLvVZjZXV 20AsSpnYFZoWqLR8XEQCn6CxDfyNhmlCRy4y6jkHnwWR2cHB0Xysf9ik5KOGe80a3KMYpEj+zcV9C mrrAwpYA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.96 #2 (Red Hat Linux)) id 1r0yPT-00DLl0-2V; Thu, 09 Nov 2023 06:20:59 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, Christian Brauner Subject: [PATCH 22/22] __dentry_kill(): new locking scheme Date: Thu, 9 Nov 2023 06:20:56 +0000 Message-Id: <20231109062056.3181775-22-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231109062056.3181775-1-viro@zeniv.linux.org.uk> References: <20231109061932.GA3181489@ZenIV> <20231109062056.3181775-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Currently we enter __dentry_kill() with parent (along with the victim dentry and victim's inode) held locked. Then we mark dentry refcount as dead call ->d_prune() remove dentry from hash remove it from the parent's list of children unlock the parent, don't need it from that point on detach dentry from inode, unlock dentry and drop the inode (via ->d_iput()) call ->d_release() regain the lock on dentry check if it's on a shrink list (in which case freeing its empty husk has to be left to shrink_dentry_list()) or not (in which case we can free it ourselves). In the former case, mark it as an empty husk, so that shrink_dentry_list() would know it can free the sucker. drop the lock on dentry ... and usually the caller proceeds to drop a reference on the parent, possibly retaking the lock on it. That is painful for a bunch of reasons, starting with the need to take locks out of order, but not limited to that - the parent of positive dentry can change if we drop its ->d_lock, so getting these locks has to be done with care. Moreover, as soon as dentry is out of the parent's list of children, shrink_dcache_for_umount() won't see it anymore, making it appear as if the parent is inexplicably busy. We do work around that by having shrink_dentry_list() decrement the parent's refcount first and put it on shrink list to be evicted once we are done with __dentry_kill() of child, but that may in some cases lead to ->d_iput() on child called after the parent got killed. That doesn't happen in cases where in-tree ->d_iput() instances might want to look at the parent, but that's brittle as hell. Solution: do removal from the parent's list of children in the very end of __dentry_kill(). As the result, the callers do not need to lock the parent and by the time we really need the parent locked, dentry is negative and is guaranteed not to be moved around. It does mean that ->d_prune() will be called with parent not locked. It also means that we might see dentries in process of being torn down while going through the parent's list of children; those dentries will be unhashed, negative and with refcount marked dead. In practice, that's enough for in-tree code that looks through the list of children to do the right thing as-is. Out-of-tree code might need to be adjusted. Calling conventions: __dentry_kill(dentry) is called with dentry->d_lock held, along with ->i_lock of its inode (if any). It either returns the parent (locked, with refcount decremented to 0) or NULL (if there'd been no parent or if refcount decrement for parent hadn't reached 0). lock_for_kill() is adjusted for new requirements - it doesn't touch the parent's ->d_lock at all. Callers adjusted. Note that for dput() we don't need to bother with fast_dput() for the parent - we just need to check retain_dentry() for it, since its ->d_lock is still held since the moment when __dentry_kill() had taken it to remove the victim from the list of children. The kludge with early decrement of parent's refcount in shrink_dentry_list() is no longer needed - shrink_dcache_for_umount() sees the half-killed dentries in the list of children for as long as they are pinning the parent. They are easily recognized and accounted for by select_collect(), so we know we are not done yet. As the result, we always have the expected ordering for ->d_iput()/->d_release() vs. __dentry_kill() of the parent, no exceptions. Moreover, the current rules for shrink lists (one must make sure that shrink_dcache_for_umount() won't happen while any dentries from the superblock in question are on any shrink lists) are gone - shrink_dcache_for_umount() will do the right thing in all cases, taking such dentries out. Their empty husks (memory occupied by struct dentry itself + its external name, if any) will remain on the shrink lists, but they are no obstacles to filesystem shutdown. And such husks will get freed as soon as shrink_dentry_list() of the list they are on gets to them. Signed-off-by: Al Viro Reviewed-by: Christian Brauner --- Documentation/filesystems/porting.rst | 17 ++++ fs/dcache.c | 127 ++++++++++---------------- 2 files changed, 64 insertions(+), 80 deletions(-) diff --git a/Documentation/filesystems/porting.rst b/Documentation/filesystems/porting.rst index 6b058362938c..8e3e31b18374 100644 --- a/Documentation/filesystems/porting.rst +++ b/Documentation/filesystems/porting.rst @@ -1062,3 +1062,20 @@ by compiler. ->d_delete() instances are now called for dentries with ->d_lock held and refcount equal to 0. They are not permitted to drop/regain ->d_lock. None of in-tree instances did anything of that sort. Make sure yours do not... + +-- + +**mandatory** + + ->d_prune() instances are now called without ->d_lock held on the parent. +->d_lock on dentry itself is still held; if you need per-parent exclusions (none +of the in-tree instances did), use your own spinlock. + + ->d_iput() and ->d_release() are called with victim dentry still in the +list of parent's children. It is still unhashed, marked killed, etc., just not +removed from parent's ->d_children yet. + + Anyone iterating through the list of children needs to be aware of the +half-killed dentries that might be seen there; taking ->d_lock on those will +see them negative, unhashed and with negative refcount, which means that most +of the in-kernel users would've done the right thing anyway without any adjustment. diff --git a/fs/dcache.c b/fs/dcache.c index cea707a77e28..bd57b9a08894 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -575,12 +575,10 @@ static inline void dentry_unlist(struct dentry *dentry) } } -static void __dentry_kill(struct dentry *dentry) +static struct dentry *__dentry_kill(struct dentry *dentry) { struct dentry *parent = NULL; bool can_free = true; - if (!IS_ROOT(dentry)) - parent = dentry->d_parent; /* * The dentry is now unrecoverably dead to the world. @@ -600,9 +598,6 @@ static void __dentry_kill(struct dentry *dentry) } /* if it was on the hash then remove it */ __d_drop(dentry); - dentry_unlist(dentry); - if (parent) - spin_unlock(&parent->d_lock); if (dentry->d_inode) dentry_unlink_inode(dentry); else @@ -611,7 +606,14 @@ static void __dentry_kill(struct dentry *dentry) if (dentry->d_op && dentry->d_op->d_release) dentry->d_op->d_release(dentry); + cond_resched(); + /* now that it's negative, ->d_parent is stable */ + if (!IS_ROOT(dentry)) { + parent = dentry->d_parent; + spin_lock(&parent->d_lock); + } spin_lock(&dentry->d_lock); + dentry_unlist(dentry); if (dentry->d_flags & DCACHE_SHRINK_LIST) { dentry->d_flags |= DCACHE_MAY_FREE; can_free = false; @@ -619,31 +621,10 @@ static void __dentry_kill(struct dentry *dentry) spin_unlock(&dentry->d_lock); if (likely(can_free)) dentry_free(dentry); - cond_resched(); -} - -static struct dentry *__lock_parent(struct dentry *dentry) -{ - struct dentry *parent; -again: - parent = READ_ONCE(dentry->d_parent); - spin_lock(&parent->d_lock); - /* - * We can't blindly lock dentry until we are sure - * that we won't violate the locking order. - * Any changes of dentry->d_parent must have - * been done with parent->d_lock held, so - * spin_lock() above is enough of a barrier - * for checking if it's still our child. - */ - if (unlikely(parent != dentry->d_parent)) { + if (parent && --parent->d_lockref.count) { spin_unlock(&parent->d_lock); - goto again; + return NULL; } - if (parent != dentry) - spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED); - else - parent = NULL; return parent; } @@ -655,48 +636,32 @@ static struct dentry *__lock_parent(struct dentry *dentry) * d_delete(), etc. * * Return false if dentry is busy. Otherwise, return true and have - * that dentry's inode and parent both locked. + * that dentry's inode locked. */ static bool lock_for_kill(struct dentry *dentry) { struct inode *inode = dentry->d_inode; - struct dentry *parent = dentry->d_parent; if (unlikely(dentry->d_lockref.count)) return false; - if (inode && unlikely(!spin_trylock(&inode->i_lock))) - goto slow; - if (dentry == parent) - return true; - if (likely(spin_trylock(&parent->d_lock))) + if (!inode || likely(spin_trylock(&inode->i_lock))) return true; - if (inode) - spin_unlock(&inode->i_lock); -slow: - spin_unlock(&dentry->d_lock); - - for (;;) { - if (inode) - spin_lock(&inode->i_lock); - parent = __lock_parent(dentry); + do { + spin_unlock(&dentry->d_lock); + spin_lock(&inode->i_lock); + spin_lock(&dentry->d_lock); if (likely(inode == dentry->d_inode)) break; - if (inode) - spin_unlock(&inode->i_lock); + spin_unlock(&inode->i_lock); inode = dentry->d_inode; - spin_unlock(&dentry->d_lock); - if (parent) - spin_unlock(&parent->d_lock); - } + } while (inode); if (likely(!dentry->d_lockref.count)) return true; if (inode) spin_unlock(&inode->i_lock); - if (parent) - spin_unlock(&parent->d_lock); return false; } @@ -869,29 +834,27 @@ static inline bool fast_dput(struct dentry *dentry) */ void dput(struct dentry *dentry) { - while (dentry) { - might_sleep(); - - rcu_read_lock(); - if (likely(fast_dput(dentry))) { - rcu_read_unlock(); + if (!dentry) + return; + might_sleep(); + rcu_read_lock(); + if (likely(fast_dput(dentry))) { + rcu_read_unlock(); + return; + } + while (lock_for_kill(dentry)) { + rcu_read_unlock(); + dentry = __dentry_kill(dentry); + if (!dentry) return; - } - - /* Slow case: now with the dentry lock held */ - if (likely(lock_for_kill(dentry))) { - struct dentry *parent = dentry->d_parent; - rcu_read_unlock(); - __dentry_kill(dentry); - if (dentry == parent) - return; - dentry = parent; - } else { - rcu_read_unlock(); + if (retain_dentry(dentry)) { spin_unlock(&dentry->d_lock); return; } + rcu_read_lock(); } + rcu_read_unlock(); + spin_unlock(&dentry->d_lock); } EXPORT_SYMBOL(dput); @@ -1086,12 +1049,16 @@ void d_prune_aliases(struct inode *inode) } EXPORT_SYMBOL(d_prune_aliases); -static inline void shrink_kill(struct dentry *victim, struct list_head *list) +static inline void shrink_kill(struct dentry *victim) { - struct dentry *parent = victim->d_parent; - if (parent != victim && !--parent->d_lockref.count) - to_shrink_list(parent, list); - __dentry_kill(victim); + do { + rcu_read_unlock(); + victim = __dentry_kill(victim); + rcu_read_lock(); + } while (victim && lock_for_kill(victim)); + rcu_read_unlock(); + if (victim) + spin_unlock(&victim->d_lock); } void shrink_dentry_list(struct list_head *list) @@ -1112,9 +1079,8 @@ void shrink_dentry_list(struct list_head *list) dentry_free(dentry); continue; } - rcu_read_unlock(); d_shrink_del(dentry); - shrink_kill(dentry, list); + shrink_kill(dentry); } } @@ -1473,6 +1439,8 @@ static enum d_walk_ret select_collect(void *_data, struct dentry *dentry) } else if (!dentry->d_lockref.count) { to_shrink_list(dentry, &data->dispose); data->found++; + } else if (dentry->d_lockref.count < 0) { + data->found++; } /* * We can return to the caller if we have found some (this @@ -1542,8 +1510,7 @@ void shrink_dcache_parent(struct dentry *parent) spin_unlock(&data.victim->d_lock); rcu_read_unlock(); } else { - rcu_read_unlock(); - shrink_kill(data.victim, &data.dispose); + shrink_kill(data.victim); } } if (!list_empty(&data.dispose))