From patchwork Wed Jun 17 07:37:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17EAE13B1 for ; Wed, 17 Jun 2020 07:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E4E5120853 for ; Wed, 17 Jun 2020 07:38:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="j1PlPr9X"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="lmxhk20O" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726510AbgFQHh4 (ORCPT ); Wed, 17 Jun 2020 03:37:56 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:39781 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725901AbgFQHhz (ORCPT ); Wed, 17 Jun 2020 03:37:55 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id 91DB85A1; Wed, 17 Jun 2020 03:37:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:37:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= KjcVrueijtwR2vVGeRp/lq0vvqKnNOZUqYEQUpc1N74=; b=j1PlPr9XwGVu4Iit iOHKgMwy/4t+dbQP3f23Y6whcwve4gYDzYj+4QDEXdNUihgV3IF4wBWKDHgM055B rePTza2gO3cWlZmU2484u0tgi3Fpku8L5MqLPjUOelCCzkhRIPyOxdbTc2tF9mGW hJZbLbfkwr2+Z1BlCNwPFfjEehHI1JqUQojd9HhdtxAzjdZa/auUYZ9BGx7TQKPt jtI62YnP074tW3X6aH8vJeiFsLqJ46zMAWwpD7fO8IraDUWImyJDSSAiyXvs2efR IqhwdbU2mpsCx5yNYoqtFOdb5+5ef7Z0j3rAmHsujta8F2XRnNIi/xEIiVQNOuAz PDWPZg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=KjcVrueijtwR2vVGeRp/lq0vvqKnNOZUqYEQUpc1N 74=; b=lmxhk20O3iRsZtb6QdoMI6SVXIaJL3V5mbH22q7mKO4Znyxd5XNfhnba/ 6UKH43nXtYpH91IAj2jCtQ0s4fpY+7OYTjCzK2XDCyYN4ECFEn3immKw5wRUGeDQ 8MRP+taY2A0FS81FjWYSoFK/tVSKpdPpJUzIK159ewldfb15DDGWfCKPUBBWt4NH ry9025GQkRGClTqD8C7KkU72ntAxy+EktEsmAd+CcWZt15Q5HzfKtMw1DDQkfpSk pLVpNooA0o8GwPizMrfSVDwqW0iCWncsZONIchXBgDRW1z52rOxYjAK8VfwC7uwx rPJthVUXdlEL3k2zyKWooY9A77CFA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id 6719B30618C1; Wed, 17 Jun 2020 03:37:52 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 515DEA0314; Wed, 17 Jun 2020 15:37:48 +0800 (AWST) Subject: [PATCH v2 1/6] kernfs: switch kernfs to use an rwsem From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:37:48 +0800 Message-ID: <159237946827.89469.3338654922163121567.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The kernfs global lock restricts the ability to perform kernfs node lookup operations in parallel. Change the kernfs mutex to an rwsem so that, when oppertunity arises, node searches can be done in parallel. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 119 +++++++++++++++++++++++-------------------- fs/kernfs/file.c | 4 + fs/kernfs/inode.c | 16 +++--- fs/kernfs/kernfs-internal.h | 5 +- fs/kernfs/mount.c | 12 ++-- fs/kernfs/symlink.c | 4 + 6 files changed, 86 insertions(+), 74 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index 9aec80b9d7c6..d8213fc65eba 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -17,7 +17,7 @@ #include "kernfs-internal.h" -DEFINE_MUTEX(kernfs_mutex); +DECLARE_RWSEM(kernfs_rwsem); static DEFINE_SPINLOCK(kernfs_rename_lock); /* kn->parent and ->name */ static char kernfs_pr_cont_buf[PATH_MAX]; /* protected by rename_lock */ static DEFINE_SPINLOCK(kernfs_idr_lock); /* root->ino_idr */ @@ -26,10 +26,21 @@ static DEFINE_SPINLOCK(kernfs_idr_lock); /* root->ino_idr */ static bool kernfs_active(struct kernfs_node *kn) { - lockdep_assert_held(&kernfs_mutex); return atomic_read(&kn->active) >= 0; } +static bool kernfs_active_write(struct kernfs_node *kn) +{ + lockdep_assert_held_write(&kernfs_rwsem); + return kernfs_active(kn); +} + +static bool kernfs_active_read(struct kernfs_node *kn) +{ + lockdep_assert_held_read(&kernfs_rwsem); + return kernfs_active(kn); +} + static bool kernfs_lockdep(struct kernfs_node *kn) { #ifdef CONFIG_DEBUG_LOCK_ALLOC @@ -340,7 +351,7 @@ static int kernfs_sd_compare(const struct kernfs_node *left, * @kn->parent->dir.children. * * Locking: - * mutex_lock(kernfs_mutex) + * kernfs_rwsem write lock * * RETURNS: * 0 on susccess -EEXIST on failure. @@ -385,7 +396,7 @@ static int kernfs_link_sibling(struct kernfs_node *kn) * removed, %false if @kn wasn't on the rbtree. * * Locking: - * mutex_lock(kernfs_mutex) + * kernfs_rwsem write lock */ static bool kernfs_unlink_sibling(struct kernfs_node *kn) { @@ -455,14 +466,14 @@ void kernfs_put_active(struct kernfs_node *kn) * return after draining is complete. */ static void kernfs_drain(struct kernfs_node *kn) - __releases(&kernfs_mutex) __acquires(&kernfs_mutex) + __releases(&kernfs_rwsem) __acquires(&kernfs_rwsem) { struct kernfs_root *root = kernfs_root(kn); - lockdep_assert_held(&kernfs_mutex); + lockdep_assert_held_write(&kernfs_rwsem); WARN_ON_ONCE(kernfs_active(kn)); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); if (kernfs_lockdep(kn)) { rwsem_acquire(&kn->dep_map, 0, 0, _RET_IP_); @@ -481,7 +492,7 @@ static void kernfs_drain(struct kernfs_node *kn) kernfs_drain_open_files(kn); - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); } /** @@ -560,10 +571,10 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) goto out_bad_unlocked; kn = kernfs_dentry_node(dentry); - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); /* The kernfs node has been deactivated */ - if (!kernfs_active(kn)) + if (!kernfs_active_read(kn)) goto out_bad; /* The kernfs node has been moved? */ @@ -579,10 +590,10 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) kernfs_info(dentry->d_sb)->ns != kn->ns) goto out_bad; - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); return 1; out_bad: - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); out_bad_unlocked: return 0; } @@ -764,7 +775,7 @@ int kernfs_add_one(struct kernfs_node *kn) bool has_ns; int ret; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); ret = -EINVAL; has_ns = kernfs_ns_enabled(parent); @@ -779,7 +790,7 @@ int kernfs_add_one(struct kernfs_node *kn) if (parent->flags & KERNFS_EMPTY_DIR) goto out_unlock; - if ((parent->flags & KERNFS_ACTIVATED) && !kernfs_active(parent)) + if ((parent->flags & KERNFS_ACTIVATED) && !kernfs_active_write(parent)) goto out_unlock; kn->hash = kernfs_name_hash(kn->name, kn->ns); @@ -795,7 +806,7 @@ int kernfs_add_one(struct kernfs_node *kn) ps_iattr->ia_mtime = ps_iattr->ia_ctime; } - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); /* * Activate the new node unless CREATE_DEACTIVATED is requested. @@ -809,7 +820,7 @@ int kernfs_add_one(struct kernfs_node *kn) return 0; out_unlock: - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return ret; } @@ -830,7 +841,7 @@ static struct kernfs_node *kernfs_find_ns(struct kernfs_node *parent, bool has_ns = kernfs_ns_enabled(parent); unsigned int hash; - lockdep_assert_held(&kernfs_mutex); + lockdep_assert_held(&kernfs_rwsem); if (has_ns != (bool)ns) { WARN(1, KERN_WARNING "kernfs: ns %s in '%s' for '%s'\n", @@ -862,7 +873,7 @@ static struct kernfs_node *kernfs_walk_ns(struct kernfs_node *parent, size_t len; char *p, *name; - lockdep_assert_held(&kernfs_mutex); + lockdep_assert_held_read(&kernfs_rwsem); /* grab kernfs_rename_lock to piggy back on kernfs_pr_cont_buf */ spin_lock_irq(&kernfs_rename_lock); @@ -902,10 +913,10 @@ struct kernfs_node *kernfs_find_and_get_ns(struct kernfs_node *parent, { struct kernfs_node *kn; - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); kn = kernfs_find_ns(parent, name, ns); kernfs_get(kn); - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); return kn; } @@ -926,10 +937,10 @@ struct kernfs_node *kernfs_walk_and_get_ns(struct kernfs_node *parent, { struct kernfs_node *kn; - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); kn = kernfs_walk_ns(parent, path, ns); kernfs_get(kn); - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); return kn; } @@ -1084,7 +1095,7 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, struct inode *inode; const void *ns = NULL; - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); if (kernfs_ns_enabled(parent)) ns = kernfs_info(dir->i_sb)->ns; @@ -1107,7 +1118,7 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, /* instantiate and hash dentry */ ret = d_splice_alias(inode, dentry); out_unlock: - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); return ret; } @@ -1226,7 +1237,7 @@ static struct kernfs_node *kernfs_next_descendant_post(struct kernfs_node *pos, { struct rb_node *rbn; - lockdep_assert_held(&kernfs_mutex); + lockdep_assert_held_write(&kernfs_rwsem); /* if first iteration, visit leftmost descendant which may be root */ if (!pos) @@ -1262,7 +1273,7 @@ void kernfs_activate(struct kernfs_node *kn) { struct kernfs_node *pos; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); pos = NULL; while ((pos = kernfs_next_descendant_post(pos, kn))) { @@ -1276,14 +1287,14 @@ void kernfs_activate(struct kernfs_node *kn) pos->flags |= KERNFS_ACTIVATED; } - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); } static void __kernfs_remove(struct kernfs_node *kn) { struct kernfs_node *pos; - lockdep_assert_held(&kernfs_mutex); + lockdep_assert_held_write(&kernfs_rwsem); /* * Short-circuit if non-root @kn has already finished removal. @@ -1298,7 +1309,7 @@ static void __kernfs_remove(struct kernfs_node *kn) /* prevent any new usage under @kn by deactivating all nodes */ pos = NULL; while ((pos = kernfs_next_descendant_post(pos, kn))) - if (kernfs_active(pos)) + if (kernfs_active_write(pos)) atomic_add(KN_DEACTIVATED_BIAS, &pos->active); /* deactivate and unlink the subtree node-by-node */ @@ -1306,7 +1317,7 @@ static void __kernfs_remove(struct kernfs_node *kn) pos = kernfs_leftmost_descendant(kn); /* - * kernfs_drain() drops kernfs_mutex temporarily and @pos's + * kernfs_drain() drops kernfs_rwsem temporarily and @pos's * base ref could have been put by someone else by the time * the function returns. Make sure it doesn't go away * underneath us. @@ -1353,9 +1364,9 @@ static void __kernfs_remove(struct kernfs_node *kn) */ void kernfs_remove(struct kernfs_node *kn) { - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); __kernfs_remove(kn); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); } /** @@ -1442,17 +1453,17 @@ bool kernfs_remove_self(struct kernfs_node *kn) { bool ret; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); kernfs_break_active_protection(kn); /* * SUICIDAL is used to arbitrate among competing invocations. Only * the first one will actually perform removal. When the removal * is complete, SUICIDED is set and the active ref is restored - * while holding kernfs_mutex. The ones which lost arbitration - * waits for SUICDED && drained which can happen only after the - * enclosing kernfs operation which executed the winning instance - * of kernfs_remove_self() finished. + * while holding kernfs_rwsem for write. The ones which lost + * arbitration waits for SUICIDED && drained which can happen only + * after the enclosing kernfs operation which executed the winning + * instance of kernfs_remove_self() finished. */ if (!(kn->flags & KERNFS_SUICIDAL)) { kn->flags |= KERNFS_SUICIDAL; @@ -1470,9 +1481,9 @@ bool kernfs_remove_self(struct kernfs_node *kn) atomic_read(&kn->active) == KN_DEACTIVATED_BIAS) break; - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); schedule(); - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); } finish_wait(waitq, &wait); WARN_ON_ONCE(!RB_EMPTY_NODE(&kn->rb)); @@ -1480,12 +1491,12 @@ bool kernfs_remove_self(struct kernfs_node *kn) } /* - * This must be done while holding kernfs_mutex; otherwise, waiting - * for SUICIDED && deactivated could finish prematurely. + * This must be done while holding kernfs_rwsem for write; otherwise, + * waiting for SUICIDED && deactivated could finish prematurely. */ kernfs_unbreak_active_protection(kn); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return ret; } @@ -1509,13 +1520,13 @@ int kernfs_remove_by_name_ns(struct kernfs_node *parent, const char *name, return -ENOENT; } - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); kn = kernfs_find_ns(parent, name, ns); if (kn) __kernfs_remove(kn); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); if (kn) return 0; @@ -1541,10 +1552,10 @@ int kernfs_rename_ns(struct kernfs_node *kn, struct kernfs_node *new_parent, if (!kn->parent) return -EINVAL; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); error = -ENOENT; - if (!kernfs_active(kn) || !kernfs_active(new_parent) || + if (!kernfs_active_write(kn) || !kernfs_active_write(new_parent) || (new_parent->flags & KERNFS_EMPTY_DIR)) goto out; @@ -1595,7 +1606,7 @@ int kernfs_rename_ns(struct kernfs_node *kn, struct kernfs_node *new_parent, error = 0; out: - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return error; } @@ -1615,7 +1626,7 @@ static struct kernfs_node *kernfs_dir_pos(const void *ns, struct kernfs_node *parent, loff_t hash, struct kernfs_node *pos) { if (pos) { - int valid = kernfs_active(pos) && + int valid = kernfs_active_read(pos) && pos->parent == parent && hash == pos->hash; kernfs_put(pos); if (!valid) @@ -1635,7 +1646,7 @@ static struct kernfs_node *kernfs_dir_pos(const void *ns, } } /* Skip over entries which are dying/dead or in the wrong namespace */ - while (pos && (!kernfs_active(pos) || pos->ns != ns)) { + while (pos && (!kernfs_active_read(pos) || pos->ns != ns)) { struct rb_node *node = rb_next(&pos->rb); if (!node) pos = NULL; @@ -1656,7 +1667,7 @@ static struct kernfs_node *kernfs_dir_next_pos(const void *ns, pos = NULL; else pos = rb_to_kn(node); - } while (pos && (!kernfs_active(pos) || pos->ns != ns)); + } while (pos && (!kernfs_active_read(pos) || pos->ns != ns)); } return pos; } @@ -1670,7 +1681,7 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx) if (!dir_emit_dots(file, ctx)) return 0; - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); if (kernfs_ns_enabled(parent)) ns = kernfs_info(dentry->d_sb)->ns; @@ -1687,12 +1698,12 @@ static int kernfs_fop_readdir(struct file *file, struct dir_context *ctx) file->private_data = pos; kernfs_get(pos); - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); if (!dir_emit(ctx, name, len, ino, type)) return 0; - mutex_lock(&kernfs_mutex); + down_read(&kernfs_rwsem); } - mutex_unlock(&kernfs_mutex); + up_read(&kernfs_rwsem); file->private_data = NULL; ctx->pos = INT_MAX; return 0; diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index 34366db3620d..455caea6ab0b 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -879,7 +879,7 @@ static void kernfs_notify_workfn(struct work_struct *work) spin_unlock_irq(&kernfs_notify_lock); /* kick fsnotify */ - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); list_for_each_entry(info, &kernfs_root(kn)->supers, node) { struct kernfs_node *parent; @@ -916,7 +916,7 @@ static void kernfs_notify_workfn(struct work_struct *work) iput(inode); } - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); kernfs_put(kn); goto repeat; } diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c index fc2469a20fed..23a7996d06a9 100644 --- a/fs/kernfs/inode.c +++ b/fs/kernfs/inode.c @@ -106,9 +106,9 @@ int kernfs_setattr(struct kernfs_node *kn, const struct iattr *iattr) { int ret; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); ret = __kernfs_setattr(kn, iattr); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return ret; } @@ -121,7 +121,7 @@ int kernfs_iop_setattr(struct dentry *dentry, struct iattr *iattr) if (!kn) return -EINVAL; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); error = setattr_prepare(dentry, iattr); if (error) goto out; @@ -134,7 +134,7 @@ int kernfs_iop_setattr(struct dentry *dentry, struct iattr *iattr) setattr_copy(inode, iattr); out: - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return error; } @@ -189,9 +189,9 @@ int kernfs_iop_getattr(const struct path *path, struct kstat *stat, struct inode *inode = d_inode(path->dentry); struct kernfs_node *kn = inode->i_private; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); kernfs_refresh_inode(kn, inode); - mutex_unlock(&kernfs_mutex); + up_writeread(&kernfs_rwsem); generic_fillattr(inode, stat); return 0; @@ -281,9 +281,9 @@ int kernfs_iop_permission(struct inode *inode, int mask) kn = inode->i_private; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); kernfs_refresh_inode(kn, inode); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return generic_permission(inode, mask); } diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index 7ee97ef59184..097c1a989aa4 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -69,7 +70,7 @@ struct kernfs_super_info { */ const void *ns; - /* anchored at kernfs_root->supers, protected by kernfs_mutex */ + /* anchored at kernfs_root->supers, protected by kernfs_rwsem */ struct list_head node; }; #define kernfs_info(SB) ((struct kernfs_super_info *)(SB->s_fs_info)) @@ -99,7 +100,7 @@ int __kernfs_setattr(struct kernfs_node *kn, const struct iattr *iattr); /* * dir.c */ -extern struct mutex kernfs_mutex; +extern struct rw_semaphore kernfs_rwsem; extern const struct dentry_operations kernfs_dops; extern const struct file_operations kernfs_dir_fops; extern const struct inode_operations kernfs_dir_iops; diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index 9dc7e7a64e10..baa4155ba2ed 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -255,9 +255,9 @@ static int kernfs_fill_super(struct super_block *sb, struct kernfs_fs_context *k sb->s_shrink.seeks = 0; /* get root inode, initialize and unlock it */ - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); inode = kernfs_get_inode(sb, info->root->kn); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); if (!inode) { pr_debug("kernfs: could not get root inode\n"); return -ENOMEM; @@ -344,9 +344,9 @@ int kernfs_get_tree(struct fs_context *fc) } sb->s_flags |= SB_ACTIVE; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); list_add(&info->node, &info->root->supers); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); } fc->root = dget(sb->s_root); @@ -372,9 +372,9 @@ void kernfs_kill_sb(struct super_block *sb) { struct kernfs_super_info *info = kernfs_info(sb); - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); list_del(&info->node); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); /* * Remove the superblock from fs_supers/s_instances diff --git a/fs/kernfs/symlink.c b/fs/kernfs/symlink.c index 5432883d819f..7246b470de3c 100644 --- a/fs/kernfs/symlink.c +++ b/fs/kernfs/symlink.c @@ -116,9 +116,9 @@ static int kernfs_getlink(struct inode *inode, char *path) struct kernfs_node *target = kn->symlink.target_kn; int error; - mutex_lock(&kernfs_mutex); + down_write(&kernfs_rwsem); error = kernfs_get_target_path(parent, target, path); - mutex_unlock(&kernfs_mutex); + up_write(&kernfs_rwsem); return error; } From patchwork Wed Jun 17 07:37:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF0D760D for ; Wed, 17 Jun 2020 07:38:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E2CF2151B for ; Wed, 17 Jun 2020 07:38:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="rMMRedoE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Qkg0mF0L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726583AbgFQHiA (ORCPT ); Wed, 17 Jun 2020 03:38:00 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:49597 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725901AbgFQHh6 (ORCPT ); Wed, 17 Jun 2020 03:37:58 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id C652C5B1; Wed, 17 Jun 2020 03:37:56 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:37:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= FtR+UgOOZLBiQrxLIVMV0q/xz8NJCgPfEh2OWV3jrgA=; b=rMMRedoET73KVhcm v5wamPEc6LIuFh66Ri5S1XuHBwvm7ZuAdQH2nnX9TcSLKr19InvPfMHoq8DWEmqc dvS/4fF0E+F/c1bK/Ftbt0XqxKNig8D3clbR94gs7HPsSeNIM/BRHZn9t1aMeFsd LglU7Y+Cbp6xsT37amk3OccSRIm7h4PLz8mlmBOjCYthRKgghMekhICZROUNsvS0 iaCxqCdFWGy/OAbNgxKCy5eIen4XhfwPZLkhMTrVxQ1GWwGH9PdXXgAfY2aNG771 jV/wc2YDJ/FBWaEblYB0kEMkdzdoxJ5VddWRmuyDWo4WHcs1F2m2/Qmi0NNhgmJo CX5SRw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=FtR+UgOOZLBiQrxLIVMV0q/xz8NJCgPfEh2OWV3jr gA=; b=Qkg0mF0LYwEWjenY/Ia5QGA0r6oLp5YZXj6Hz2TaraRoENC2nxYLZNBfg uBbhIQUA4dKPRBVQd/OboEDQgTnNJlfvEaYEXdLBqJ2nKGXxtkoT9I8CK6dl9Xak NcjKpd9D1XCvY1ZMs4XLKeBikLMvyS0kImERwTBY1/gtj03dEwuB8W8tNmHIvPfZ Gskl+PdCJiiBYYm9xyF8+2epbMGpwWT4cBEfdju+O+bLSXazgbsre0LfgehX/RKB TDVQcnsPx9QAGFPREvgR1gTyf4aDGBiKqFLBh9LIlqKgYX3spEaUg06vkpwTZvFy E8NbT9fRrVC8jtfOdkETRYRalhFYA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id E37A93280060; Wed, 17 Jun 2020 03:37:55 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 5D080A0314; Wed, 17 Jun 2020 15:37:53 +0800 (AWST) Subject: [PATCH v2 2/6] kernfs: move revalidate to be near lookup From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:37:53 +0800 Message-ID: <159237947334.89469.11409230524327917644.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org While the dentry operation kernfs_dop_revalidate() is grouped with dentry'ish functions it also has a strong afinity to the inode operation ->lookup(). And when path walk improvements are applied it will need to call kernfs_find_ns() so move it to be near kernfs_iop_lookup() to avoid the need for a forward declaration. There's no functional change from this patch. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 86 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index d8213fc65eba..9b315f3b20ee 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -559,49 +559,6 @@ void kernfs_put(struct kernfs_node *kn) } EXPORT_SYMBOL_GPL(kernfs_put); -static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) -{ - struct kernfs_node *kn; - - if (flags & LOOKUP_RCU) - return -ECHILD; - - /* Always perform fresh lookup for negatives */ - if (d_really_is_negative(dentry)) - goto out_bad_unlocked; - - kn = kernfs_dentry_node(dentry); - down_read(&kernfs_rwsem); - - /* The kernfs node has been deactivated */ - if (!kernfs_active_read(kn)) - goto out_bad; - - /* The kernfs node has been moved? */ - if (kernfs_dentry_node(dentry->d_parent) != kn->parent) - goto out_bad; - - /* The kernfs node has been renamed */ - if (strcmp(dentry->d_name.name, kn->name) != 0) - goto out_bad; - - /* The kernfs node has been moved to a different namespace */ - if (kn->parent && kernfs_ns_enabled(kn->parent) && - kernfs_info(dentry->d_sb)->ns != kn->ns) - goto out_bad; - - up_read(&kernfs_rwsem); - return 1; -out_bad: - up_read(&kernfs_rwsem); -out_bad_unlocked: - return 0; -} - -const struct dentry_operations kernfs_dops = { - .d_revalidate = kernfs_dop_revalidate, -}; - /** * kernfs_node_from_dentry - determine kernfs_node associated with a dentry * @dentry: the dentry in question @@ -1085,6 +1042,49 @@ struct kernfs_node *kernfs_create_empty_dir(struct kernfs_node *parent, return ERR_PTR(rc); } +static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) +{ + struct kernfs_node *kn; + + if (flags & LOOKUP_RCU) + return -ECHILD; + + /* Always perform fresh lookup for negatives */ + if (d_really_is_negative(dentry)) + goto out_bad_unlocked; + + kn = kernfs_dentry_node(dentry); + down_read(&kernfs_rwsem); + + /* The kernfs node has been deactivated */ + if (!kernfs_active_read(kn)) + goto out_bad; + + /* The kernfs node has been moved? */ + if (kernfs_dentry_node(dentry->d_parent) != kn->parent) + goto out_bad; + + /* The kernfs node has been renamed */ + if (strcmp(dentry->d_name.name, kn->name) != 0) + goto out_bad; + + /* The kernfs node has been moved to a different namespace */ + if (kn->parent && kernfs_ns_enabled(kn->parent) && + kernfs_info(dentry->d_sb)->ns != kn->ns) + goto out_bad; + + up_read(&kernfs_rwsem); + return 1; +out_bad: + up_read(&kernfs_rwsem); +out_bad_unlocked: + return 0; +} + +const struct dentry_operations kernfs_dops = { + .d_revalidate = kernfs_dop_revalidate, +}; + static struct dentry *kernfs_iop_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags) From patchwork Wed Jun 17 07:37:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2805E60D for ; Wed, 17 Jun 2020 07:38:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BC9820DD4 for ; Wed, 17 Jun 2020 07:38:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="EZ+hu2XS"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="rKZrjB2o" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726698AbgFQHiD (ORCPT ); Wed, 17 Jun 2020 03:38:03 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:49493 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726629AbgFQHiD (ORCPT ); Wed, 17 Jun 2020 03:38:03 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id D23F4574; Wed, 17 Jun 2020 03:38:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:38:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= /rAbDMt6FC3fG09UnQPczhNktJ9DSAXatCwomfNKw7A=; b=EZ+hu2XSckUTh2ww 9ChVrX43WWyu1hOF1y2dRHfKAYYvNcZrlWcuhfyle3e0DOj1puR+RCeEAkryYp/m jORajl2/0TKGS8GTBloS1v8lJ6TuYDKVPsiWgFqRlAy+rtG82p/7/Ae6K7V1d72F Kqeilin2R6vSR2WKNTnLkCLxKfo6i3NtEm1L53X5LL/vEXJMQtGFgYVYpVZVtvcQ keCDqXhx+RI4SdAVXFq0R610EE/AEMCfck6odQphJz4UxcgRCjA0s1oJd0U383wE TpCEtBnUEAwXsPDH/yFUOzCgh/XHm6Zb7Vfkm9cSb0WrXWaEP8rAK5COgCdwQRjP HqqMFA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=/rAbDMt6FC3fG09UnQPczhNktJ9DSAXatCwomfNKw 7A=; b=rKZrjB2o86mRXyffoXCH8nD6a1yeD2qKkhpJdp6+qxv+0lvcO1e/nYybC nnDKyg/1iMDISUrdFHx2ibqAdC4wTZPFNe4gg/egISiHgWdOd82KhQpscPTijUpS KoCzfNrbAAYHVEFFHMqNpBxivz4+tJCwHSrnHYU96W6/fAkNhGsgl7knCA2ckCg5 lVkoVDXVKvI9KSH49+DBkGhnouVfcCsi+pIH+La1xY0o98Ko+2XNBbaj7rt8wptm MZKHEoHCq9o8PQJh5BG0Q+IwazlGwSKZ19UrLFvWRblh4Cb6/DRyTI6Haxp3SDMN aygKiVtsNkSzoNPZGJARed4p5+Pug== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgepvdenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id 0112430618B7; Wed, 17 Jun 2020 03:38:01 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 69E00A0314; Wed, 17 Jun 2020 15:37:58 +0800 (AWST) Subject: [PATCH v2 3/6] kernfs: improve kernfs path resolution From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:37:58 +0800 Message-ID: <159237947839.89469.7331804336434093565.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Now that an rwsem is used by kernfs, take advantage of it to reduce lookup overhead. If there are many lookups (possibly many negative ones) there can be a lot of overhead during path walks. To reduce lookup overhead avoid allocating a new dentry where possible. To do this stay in rcu-walk mode where possible and use the dentry cache handling of negative hashed dentries to avoid allocating (and freeing shortly after) new dentries on every negative lookup. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 72 insertions(+), 15 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index 9b315f3b20ee..f4943329e578 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -1046,15 +1046,75 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) { struct kernfs_node *kn; - if (flags & LOOKUP_RCU) + if (flags & LOOKUP_RCU) { + kn = kernfs_dentry_node(dentry); + if (!kn) { + /* Negative hashed dentry, tell the VFS to switch to + * ref-walk mode and call us again so that node + * existence can be checked. + */ + if (!d_unhashed(dentry)) + return -ECHILD; + + /* Negative unhashed dentry, this shouldn't happen + * because this case occurs in rcu-walk mode after + * dentry allocation which is followed by a call + * to ->loopup(). But if it does happen the dentry + * is surely invalid. + */ + return 0; + } + + /* Since the dentry is positive (we got the kernfs node) a + * kernfs node reference was held at the time. Now if the + * dentry reference count is still greater than 0 it's still + * positive so take a reference to the node to perform an + * active check. + */ + if (d_count(dentry) <= 0 || !atomic_inc_not_zero(&kn->count)) + return -ECHILD; + + /* The kernfs node reference count was greater than 0, if + * it's active continue in rcu-walk mode. + */ + if (kernfs_active_read(kn)) { + kernfs_put(kn); + return 1; + } + + /* Otherwise, just tell the VFS to switch to ref-walk mode + * and call us again so the kernfs node can be validated. + */ + kernfs_put(kn); return -ECHILD; + } - /* Always perform fresh lookup for negatives */ - if (d_really_is_negative(dentry)) - goto out_bad_unlocked; + down_read(&kernfs_rwsem); kn = kernfs_dentry_node(dentry); - down_read(&kernfs_rwsem); + if (!kn) { + struct kernfs_node *parent; + + /* If the kernfs node can be found this is a stale negative + * hashed dentry so it must be discarded and the lookup redone. + */ + parent = kernfs_dentry_node(dentry->d_parent); + if (parent) { + const void *ns = NULL; + + if (kernfs_ns_enabled(parent)) + ns = kernfs_info(dentry->d_parent->d_sb)->ns; + kn = kernfs_find_ns(parent, dentry->d_name.name, ns); + if (kn) + goto out_bad; + } + + /* The kernfs node doesn't exist, leave the dentry negative + * and return success. + */ + goto out; + } + /* The kernfs node has been deactivated */ if (!kernfs_active_read(kn)) @@ -1072,12 +1132,11 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) if (kn->parent && kernfs_ns_enabled(kn->parent) && kernfs_info(dentry->d_sb)->ns != kn->ns) goto out_bad; - +out: up_read(&kernfs_rwsem); return 1; out_bad: up_read(&kernfs_rwsem); -out_bad_unlocked: return 0; } @@ -1092,7 +1151,7 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, struct dentry *ret; struct kernfs_node *parent = dir->i_private; struct kernfs_node *kn; - struct inode *inode; + struct inode *inode = NULL; const void *ns = NULL; down_read(&kernfs_rwsem); @@ -1102,11 +1161,9 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, kn = kernfs_find_ns(parent, dentry->d_name.name, ns); - /* no such entry */ - if (!kn || !kernfs_active(kn)) { - ret = NULL; - goto out_unlock; - } + /* no such entry, retain as negative hashed dentry */ + if (!kn || !kernfs_active(kn)) + goto out_negative; /* attach dentry and inode */ inode = kernfs_get_inode(dir->i_sb, kn); @@ -1114,10 +1171,10 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, ret = ERR_PTR(-ENOMEM); goto out_unlock; } - +out_negative: /* instantiate and hash dentry */ ret = d_splice_alias(inode, dentry); - out_unlock: +out_unlock: up_read(&kernfs_rwsem); return ret; } From patchwork Wed Jun 17 07:38:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E76860D for ; Wed, 17 Jun 2020 07:38:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 728E720DD4 for ; Wed, 17 Jun 2020 07:38:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="VARnqolE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kA62n4dM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726879AbgFQHi3 (ORCPT ); Wed, 17 Jun 2020 03:38:29 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:43099 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726769AbgFQHiI (ORCPT ); Wed, 17 Jun 2020 03:38:08 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id D04F1561; Wed, 17 Jun 2020 03:38:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:38:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= YP81i3I+Z4aS6pPC7VYSp+SNREBPRhMa7fFT8ZR5kf8=; b=VARnqolEz9kMpyOG rueMBTTc+omAW2E/8JihoNv6+SbQLONF4FMTPJhJzGUjCqPFVvgTtZklKCqXuKQH sao7DxZ/XzN70CRTPA4E4bnihrSsTnOOIekMaob8Kp/iVRStsBGraheOmaT7Shij E2oMFstbgp00D+43D+GzXGoFCCP4ZW+v9Ejjg0RjLeg2HecT1MBulnu37pbCEi7Q aveHGFplNn/At35OKw02tWbRBE1KLjGR2XACHGKDsn+FJjU71RjR3x7pIkVa5Fis OPsVsMPv+PWwWLkrQAkF4XEl0yK87QYQ65mNLiGKQ6HfHfhOyBZfwq+pNTdpYmk6 Ncy7RA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=YP81i3I+Z4aS6pPC7VYSp+SNREBPRhMa7fFT8ZR5k f8=; b=kA62n4dMK9q8KoCq0Z/fLzeUfL/w+UQpRJXyfpc9mpEclDNnJO5790ci1 sle1qA3nJDbJmmv6h/EyIeivRAfB2Ojg4FjP3UhE3Ug9g5LNKRhB7SaOEs9iYc5g 9BS3u0SQ8s5wQzb5snW//h3SN9xqTQ0h0CnNZcih2lX2k51leP8zb5jQXast5Slk 5HOBBME9yEJXmHyaai5/Kk324Nxye1TngWDKxqLztPnUMYwQv7/mQukwpiMY/XfP MyJ5Aqj6o1C/NLvDN+5UUIk0eGdJbbKApq1euafVZD80efG3+2ma2Jj3IHGXqaA1 MD0I5gGbGC1OWOQpG17pQz8z0yqXA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgepfeenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id EC8ED328005D; Wed, 17 Jun 2020 03:38:05 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 79164A0314; Wed, 17 Jun 2020 15:38:03 +0800 (AWST) Subject: [PATCH v2 4/6] kernfs: use revision to identify directory node changes From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:38:03 +0800 Message-ID: <159237948345.89469.9839924207092477321.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org If a kernfs directory node hasn't changed there's no need to search for an added (or removed) child dentry. Add a revision counter to kernfs directory nodes so it can be used to detect if a directory node has changed. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 17 +++++++++++++++-- fs/kernfs/kernfs-internal.h | 24 ++++++++++++++++++++++++ include/linux/kernfs.h | 5 +++++ 3 files changed, 44 insertions(+), 2 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index f4943329e578..03f4f179bbc4 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -383,6 +383,7 @@ static int kernfs_link_sibling(struct kernfs_node *kn) /* successfully added, account subdir number */ if (kernfs_type(kn) == KERNFS_DIR) kn->parent->dir.subdirs++; + kernfs_inc_rev(kn->parent); return 0; } @@ -405,6 +406,7 @@ static bool kernfs_unlink_sibling(struct kernfs_node *kn) if (kernfs_type(kn) == KERNFS_DIR) kn->parent->dir.subdirs--; + kernfs_inc_rev(kn->parent); rb_erase(&kn->rb, &kn->parent->dir.children); RB_CLEAR_NODE(&kn->rb); @@ -1044,9 +1046,16 @@ struct kernfs_node *kernfs_create_empty_dir(struct kernfs_node *parent, static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) { + struct kernfs_node *parent; struct kernfs_node *kn; if (flags & LOOKUP_RCU) { + /* Directory node changed? */ + parent = kernfs_dentry_node(dentry->d_parent); + + if (!kernfs_dir_changed(parent, dentry)) + return 1; + kn = kernfs_dentry_node(dentry); if (!kn) { /* Negative hashed dentry, tell the VFS to switch to @@ -1093,8 +1102,6 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) kn = kernfs_dentry_node(dentry); if (!kn) { - struct kernfs_node *parent; - /* If the kernfs node can be found this is a stale negative * hashed dentry so it must be discarded and the lookup redone. */ @@ -1102,6 +1109,10 @@ static int kernfs_dop_revalidate(struct dentry *dentry, unsigned int flags) if (parent) { const void *ns = NULL; + /* Directory node changed? */ + if (kernfs_dir_changed(parent, dentry)) + goto out_bad; + if (kernfs_ns_enabled(parent)) ns = kernfs_info(dentry->d_parent->d_sb)->ns; kn = kernfs_find_ns(parent, dentry->d_name.name, ns); @@ -1156,6 +1167,8 @@ static struct dentry *kernfs_iop_lookup(struct inode *dir, down_read(&kernfs_rwsem); + kernfs_set_rev(dentry, parent); + if (kernfs_ns_enabled(parent)) ns = kernfs_info(dir->i_sb)->ns; diff --git a/fs/kernfs/kernfs-internal.h b/fs/kernfs/kernfs-internal.h index 097c1a989aa4..a7b0e2074260 100644 --- a/fs/kernfs/kernfs-internal.h +++ b/fs/kernfs/kernfs-internal.h @@ -82,6 +82,30 @@ static inline struct kernfs_node *kernfs_dentry_node(struct dentry *dentry) return d_inode(dentry)->i_private; } +static inline void kernfs_set_rev(struct dentry *dentry, + struct kernfs_node *kn) +{ + dentry->d_time = kn->dir.rev; +} + +static inline void kernfs_inc_rev(struct kernfs_node *kn) +{ + if (kernfs_type(kn) == KERNFS_DIR) { + if (!++kn->dir.rev) + kn->dir.rev++; + } +} + +static inline bool kernfs_dir_changed(struct kernfs_node *kn, + struct dentry *dentry) +{ + if (kernfs_type(kn) == KERNFS_DIR) { + if (kn->dir.rev != dentry->d_time) + return true; + } + return false; +} + extern const struct super_operations kernfs_sops; extern struct kmem_cache *kernfs_node_cache, *kernfs_iattrs_cache; diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 89f6a4214a70..74727d98e380 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -98,6 +98,11 @@ struct kernfs_elem_dir { * better directly in kernfs_node but is here to save space. */ struct kernfs_root *root; + /* + * Monotonic revision counter, used to identify if a directory + * node has changed during revalidation. + */ + unsigned long rev; }; struct kernfs_elem_symlink { From patchwork Wed Jun 17 07:38:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C12960D for ; Wed, 17 Jun 2020 07:38:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F1A320DD4 for ; Wed, 17 Jun 2020 07:38:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="AHZvD4oM"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Pj5ug67u" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726833AbgFQHiO (ORCPT ); Wed, 17 Jun 2020 03:38:14 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:33941 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726804AbgFQHiN (ORCPT ); Wed, 17 Jun 2020 03:38:13 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id E83FC478; Wed, 17 Jun 2020 03:38:11 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:38:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= U0hpE2t4Jb5LiAicyzV44p0oKfUxQp3342ldxD2V7j8=; b=AHZvD4oMx3a+1tZg NoNMRNSKlQrDTRN65fMY52gYUV/GR435OUwKa89btZespVl4A2nd/Z/FfXHdpX6Z T+OuxZoxYTWiCnM+wVfDAX/nvRFX61vCrIfIFH6GePQmqg5myKzCXeaxX8E9LxPY nDXWPQT0Wmqm9Q0nnNXgvilD2ZJKY/TLnROeHl3cr0k3f4OQ3nHyfCsK1Y0DxLAa 2quwgI2bTX7jA0jcqcBmA4PDLroehgGR14Gcj5SVlyV2xygbun/Q5IJDXCjnzmyS 1vUeuX4lSVjgtHMkC25ylT4P1acmC/ZXGAoEtHe3U26SC5ADfYV1VniMEG6f/quN entGtA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=U0hpE2t4Jb5LiAicyzV44p0oKfUxQp3342ldxD2V7 j8=; b=Pj5ug67ulDJK+utTDVWd5yo7ai80EN2g7Sk3p9h6ejclxbTg+PsPWrw8d v30Mc22VJ3preasooPo0jZqQlMYbdHMF/m9Ma1SEGQaFphYoOyqcZDAdky3Oin5U DKnQBNyGLCzI5DZ06LjOlg4IYq3PdGyxwO4o3mI2u81/ZcfusO2ObOMAF2abjCTe TVmsLG8E51gsSyAokSt2R5KiNysgj4dA4+NiylJp8JtW0FxcaIfzBiiZwDBsBV4f wqVtbblxoUdEvhn10aTrIagcIrrTJ40/gdROrMe2YJ2hNQiFULyG3aNgGkBtzrCv vhuAy6foC+XboRdd21ZqZ+MGrk8BQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgepfeenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id 1736B306215A; Wed, 17 Jun 2020 03:38:11 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 86338A0314; Wed, 17 Jun 2020 15:38:08 +0800 (AWST) Subject: [PATCH v2 5/6] kernfs: refactor attr locking From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:38:08 +0800 Message-ID: <159237948850.89469.14590162329652845934.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The inode operations .permission() and .getattr() use the kernfs node write lock but all that's needed is to keep the rb tree stable while copying the node attributes. And .permission() is called frequently during path walks so it can cause quite a bit of contention between kernfs node opertations and path walks when the number of concurrant walks is high. Ideally the inode mutex would protect the inode update but .permission() may be called both with and without holding the inode mutex so there's no way for kernfs .permission() to know if it is the holder of the mutex which means it could be released during the update. So refactor __kernfs_iattrs() by moving the static mutex declaration out of the function and changing the function itself a little. And also use the mutex to protect the inode attribute fields updated by .permission() and .getattr() calls to kernfs_refresh_inode(). Using the attr mutex to protect two different things, the node attributes as well as the copy of them to the inode is not ideal. But the only other choice is to use two locks which seems like excessive ovherhead when the attr mutex is so closely related to the inode fields it's protecting. Signed-off-by: Ian Kent --- fs/kernfs/inode.c | 50 ++++++++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c index 23a7996d06a9..5c3fac356ce0 100644 --- a/fs/kernfs/inode.c +++ b/fs/kernfs/inode.c @@ -17,6 +17,8 @@ #include "kernfs-internal.h" +static DEFINE_MUTEX(attr_mutex); + static const struct address_space_operations kernfs_aops = { .readpage = simple_readpage, .write_begin = simple_write_begin, @@ -32,33 +34,33 @@ static const struct inode_operations kernfs_iops = { static struct kernfs_iattrs *__kernfs_iattrs(struct kernfs_node *kn, int alloc) { - static DEFINE_MUTEX(iattr_mutex); - struct kernfs_iattrs *ret; - - mutex_lock(&iattr_mutex); + struct kernfs_iattrs *iattr = NULL; - if (kn->iattr || !alloc) + mutex_lock(&attr_mutex); + if (kn->iattr || !alloc) { + iattr = kn->iattr; goto out_unlock; + } - kn->iattr = kmem_cache_zalloc(kernfs_iattrs_cache, GFP_KERNEL); - if (!kn->iattr) + iattr = kmem_cache_zalloc(kernfs_iattrs_cache, GFP_KERNEL); + if (!iattr) goto out_unlock; /* assign default attributes */ - kn->iattr->ia_uid = GLOBAL_ROOT_UID; - kn->iattr->ia_gid = GLOBAL_ROOT_GID; + iattr->ia_uid = GLOBAL_ROOT_UID; + iattr->ia_gid = GLOBAL_ROOT_GID; - ktime_get_real_ts64(&kn->iattr->ia_atime); - kn->iattr->ia_mtime = kn->iattr->ia_atime; - kn->iattr->ia_ctime = kn->iattr->ia_atime; + ktime_get_real_ts64(&iattr->ia_atime); + iattr->ia_mtime = iattr->ia_atime; + iattr->ia_ctime = iattr->ia_atime; - simple_xattrs_init(&kn->iattr->xattrs); - atomic_set(&kn->iattr->nr_user_xattrs, 0); - atomic_set(&kn->iattr->user_xattr_size, 0); + simple_xattrs_init(&iattr->xattrs); + atomic_set(&iattr->nr_user_xattrs, 0); + atomic_set(&iattr->user_xattr_size, 0); + kn->iattr = iattr; out_unlock: - ret = kn->iattr; - mutex_unlock(&iattr_mutex); - return ret; + mutex_unlock(&attr_mutex); + return iattr; } static struct kernfs_iattrs *kernfs_iattrs(struct kernfs_node *kn) @@ -189,9 +191,11 @@ int kernfs_iop_getattr(const struct path *path, struct kstat *stat, struct inode *inode = d_inode(path->dentry); struct kernfs_node *kn = inode->i_private; - down_write(&kernfs_rwsem); + down_read(&kernfs_rwsem); + mutex_lock(&attr_mutex); kernfs_refresh_inode(kn, inode); - up_writeread(&kernfs_rwsem); + mutex_unlock(&attr_mutex); + up_read(&kernfs_rwsem); generic_fillattr(inode, stat); return 0; @@ -281,9 +285,11 @@ int kernfs_iop_permission(struct inode *inode, int mask) kn = inode->i_private; - down_write(&kernfs_rwsem); + down_read(&kernfs_rwsem); + mutex_lock(&attr_mutex); kernfs_refresh_inode(kn, inode); - up_write(&kernfs_rwsem); + mutex_unlock(&attr_mutex); + up_read(&kernfs_rwsem); return generic_permission(inode, mask); } From patchwork Wed Jun 17 07:38:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Kent X-Patchwork-Id: 11609293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9C5860D for ; Wed, 17 Jun 2020 07:38:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B19B920DD4 for ; Wed, 17 Jun 2020 07:38:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="O9uu4vFr"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Tei9xWny" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726854AbgFQHiV (ORCPT ); Wed, 17 Jun 2020 03:38:21 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:42595 "EHLO wout5-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726542AbgFQHiS (ORCPT ); Wed, 17 Jun 2020 03:38:18 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id E49C94C5; Wed, 17 Jun 2020 03:38:16 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 17 Jun 2020 03:38:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= subject:from:to:cc:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm3; bh= trUDM6J4xTL+EvlxC8X7RzYKusxzZbaupwPRTQgum+k=; b=O9uu4vFr57zyFqdT ku9gOfS4bETKmnlks7bHBs6kQyaFSZNTUmTxsUlid8DIQIqFw+NPeEj9qaPxUfiT y1Bn9ofqrChhfnC2pFLCnTbIkMYRC4rAYQ3+ATVzr9h2hFwGNitY0kiq1iVJ1Igs 6nOF0TnoiYeZXaRV8VZSb4C0BoXyWMBBaNc2dr/+4gIBQcwskII5gmCxXEFcZvWI oYGzjYJbIXn0gWU7CT6zemyOOaEW9Vznj+ISUdfDrtDCA4EZ2qOgE+OjiikRAtQA DnR2g9l1Cf/0X6Y/wL/9kUOJW1M/yU/4vylGRTN08BwYaGJBFl/PrgKzlykXq1Oa wZIaDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=trUDM6J4xTL+EvlxC8X7RzYKusxzZbaupwPRTQgum +k=; b=Tei9xWnyMb36/BJ9CIKJSta419ZA02dxi9Hff7wXNiSj3/VyLGvDxMfFk F0Wrluwr+YyGkZXeDlsKoLGAj0w6kFHH9m0gVJLkwo1+N8WaUHsY51a3L31EzsMv 4zHSfL+GvqbN5TOzDyxDCuh84bJnEAS9eqkM6Gr/Z2tnULvOXTuQ9jA8ylyKYHQG FO+lW9tLWwZupxzJTPPUoOs/HO9mW+uczUiUahTxQs+z5QeYmnLPTuz9SbHbO+DU FE6/xYYOlAFPQ/JX9Dxbok+gE7JCKmtZ2H6MziroLpdcFCGmLaZFHs95VQzJ9gFs UBbmblffbHgC30BU+M7x0xTGh143w== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudejuddguddvhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfffkjghffgggtgfgsehtjedttddtreejnecuhfhrohhmpefkrghn ucfmvghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecuggftrfgrthhtvghrnh epvddvvdfgleefhfelgfekvdejjefhvdetfeevueeggeefhfdujeegveejveejgfdunecu kfhppeehkedrjedrudelgedrkeejnecuvehluhhsthgvrhfuihiivgepheenucfrrghrrg hmpehmrghilhhfrhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (58-7-194-87.dyn.iinet.net.au [58.7.194.87]) by mail.messagingengine.com (Postfix) with ESMTPA id 1837F328005E; Wed, 17 Jun 2020 03:38:16 -0400 (EDT) Received: from mickey.themaw.net (localhost [127.0.0.1]) by mickey.themaw.net (Postfix) with ESMTP id 9413FA0314; Wed, 17 Jun 2020 15:38:13 +0800 (AWST) Subject: [PATCH v2 6/6] kernfs: make attr_mutex a local kernfs node lock From: Ian Kent To: Greg Kroah-Hartman Cc: Tejun Heo , Stephen Rothwell , Andrew Morton , Al Viro , Rick Lindsley , David Howells , Miklos Szeredi , linux-fsdevel , Kernel Mailing List Date: Wed, 17 Jun 2020 15:38:13 +0800 Message-ID: <159237949356.89469.1012120560805135591.stgit@mickey.themaw.net> In-Reply-To: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> References: <159237905950.89469.6559073274338175600.stgit@mickey.themaw.net> User-Agent: StGit/0.19 MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The global mutex attr_mutex is used to protect the update of inode attributes in kernfs_refresh_inode() (as well as kernfs node attribute structure creation) and this function is called by the inode operation .permission(). Since .permission() is called quite frequently during path walks it can lead to contention when the number of concurrent path walks is high. This mutex is used for kernfs node objects only so make it local to the kernfs node to reduce the impact of this type of contention. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 1 + fs/kernfs/inode.c | 12 ++++++------ include/linux/kernfs.h | 2 ++ 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index 03f4f179bbc4..3233e01651e4 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs/dir.c @@ -597,6 +597,7 @@ static struct kernfs_node *__kernfs_new_node(struct kernfs_root *root, kn = kmem_cache_zalloc(kernfs_node_cache, GFP_KERNEL); if (!kn) goto err_out1; + mutex_init(&kn->attr_mutex); idr_preload(GFP_KERNEL); spin_lock(&kernfs_idr_lock); diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c index 5c3fac356ce0..5eb11094bb2e 100644 --- a/fs/kernfs/inode.c +++ b/fs/kernfs/inode.c @@ -36,7 +36,7 @@ static struct kernfs_iattrs *__kernfs_iattrs(struct kernfs_node *kn, int alloc) { struct kernfs_iattrs *iattr = NULL; - mutex_lock(&attr_mutex); + mutex_lock(&kn->attr_mutex); if (kn->iattr || !alloc) { iattr = kn->iattr; goto out_unlock; @@ -59,7 +59,7 @@ static struct kernfs_iattrs *__kernfs_iattrs(struct kernfs_node *kn, int alloc) atomic_set(&iattr->user_xattr_size, 0); kn->iattr = iattr; out_unlock: - mutex_unlock(&attr_mutex); + mutex_unlock(&kn->attr_mutex); return iattr; } @@ -192,9 +192,9 @@ int kernfs_iop_getattr(const struct path *path, struct kstat *stat, struct kernfs_node *kn = inode->i_private; down_read(&kernfs_rwsem); - mutex_lock(&attr_mutex); + mutex_lock(&kn->attr_mutex); kernfs_refresh_inode(kn, inode); - mutex_unlock(&attr_mutex); + mutex_unlock(&kn->attr_mutex); up_read(&kernfs_rwsem); generic_fillattr(inode, stat); @@ -286,9 +286,9 @@ int kernfs_iop_permission(struct inode *inode, int mask) kn = inode->i_private; down_read(&kernfs_rwsem); - mutex_lock(&attr_mutex); + mutex_lock(&kn->attr_mutex); kernfs_refresh_inode(kn, inode); - mutex_unlock(&attr_mutex); + mutex_unlock(&kn->attr_mutex); up_read(&kernfs_rwsem); return generic_permission(inode, mask); diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h index 74727d98e380..8669f65d5a39 100644 --- a/include/linux/kernfs.h +++ b/include/linux/kernfs.h @@ -142,6 +142,8 @@ struct kernfs_node { struct rb_node rb; + struct mutex attr_mutex; /* protect attr updates */ + const void *ns; /* namespace tag */ unsigned int hash; /* ns + name hash */ union {