From patchwork Thu Jun 22 08:53:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13288653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 407D9EB64D8 for ; Thu, 22 Jun 2023 08:56:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D618E8D0006; Thu, 22 Jun 2023 04:56:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D118E8D0001; Thu, 22 Jun 2023 04:56:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C00E38D0006; Thu, 22 Jun 2023 04:56:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B28BA8D0001 for ; Thu, 22 Jun 2023 04:56:54 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8B209120727 for ; Thu, 22 Jun 2023 08:56:54 +0000 (UTC) X-FDA: 80929778748.08.CD8C431 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf27.hostedemail.com (Postfix) with ESMTP id B1C0140009 for ; Thu, 22 Jun 2023 08:56:52 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H+1EjwPI; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687424212; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=02DUTqACM/aSl9866C36A3I60VPYUs4OjMZ7Yfrc1/o=; b=rgfK5KPt/csdCXNbA9NLrjz+9NyaPJXpU6gQpb8/eIc+hsCokOC0i+C3nua8Yml6exAgZm hICBkacxOZ29qzhhJCZlxCYsKSt2YNKTt1HmkVf7/A3h4Z6ulEORm4SZz4SPsX6bYZqHFl 6EJI62IE1G7INSlSjLCySxA0gtEOL2w= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=H+1EjwPI; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf27.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687424212; a=rsa-sha256; cv=none; b=ewhgXBqcsy3s1Cp4iBt2xyVCbUDAiyxLimNZnshf4Rq9cvs9NLVg/VDK2ctTuVIA9A7bzI rLQv85L4s3PQa9uqsVUa0e4iOnGP4OGN/WgkjZZqirr6g8sN7HRCD5btv49a7sETQTzzDM EF4MNbaKPzNpRrqxmC5e3FPpHEw3GtM= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1b5079b8cb3so12591725ad.1 for ; Thu, 22 Jun 2023 01:56:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1687424211; x=1690016211; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=02DUTqACM/aSl9866C36A3I60VPYUs4OjMZ7Yfrc1/o=; b=H+1EjwPIJiQm5S0OenD/xh0iPse4xpKj+qmkmBU/srKV3WPi3ULtA4pLp++dC5Bqh9 7BlRLloRVIKfQ42/hs1T7Du0cHLmrcEPoZEBW7JCQkTGLYx1UQ3fbHqq8+O3YFu52yg3 0+mCV5UtaM+1WMi4gyiYw8XnFCdNZ/oJcDwzQAYJvzkblrrUfBpPXXo27BHoXCIIli5o hjfG1EZpMvdsdRL66NPCU/nz333dm9KHMwYWw99s85mydOWvyhkPwMLzowDcLeXpMtBs G9dmlhU3PRkH82ZAs6EjGsULbQMB1pyMMS03ZQNG2aou+qooblLYB2n9++fqbuAHUsk3 kqMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687424211; x=1690016211; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=02DUTqACM/aSl9866C36A3I60VPYUs4OjMZ7Yfrc1/o=; b=h6v5A8QpmkrA1nIpqD/P4/eZXrJ9jo6qtcg7zWvedPGb4NyB8alEJbN7QJnI6+uyk0 gwx78wsKlBcbv9sefn5bPKX6cwcVM7uVbpGVzihFucjS2Rjy1Tm6INyG00KdMpr+EAdp IYiwf8Xm/hrKIJMhKfw1J2OTzDZuyNfgaqEkgkDn+tpzPn96yhb4GVkbAw5sB1kCOiU6 chkJhoa/y/amlv6fNkh3CTNWcSlkQZui+yRlYhWWpU3D2DRAXGGm/RSgjReEzFB/79Ho lKl/wz3VUpThTkeyL1NvZCHSLqxwaUAOnd5qUlRkh0sL2Ad+X5EDiykHcJFuydqMWYvN 4mhQ== X-Gm-Message-State: AC+VfDyZhuDCh4c2mRitU6D2jXBFSK/Qcqm+h37fRmskkBGN0+snMm3A p+7x3T5DYATYeJK1yyC9XU7buGBJphLrQQTN9+Y= X-Google-Smtp-Source: ACHHUZ4bv27VnKnfH9aSIwqm+pi4rG014ro45egLBBsO6iOayIjodJqPa/Ct3mSpE0n2zy/iA6cg+A== X-Received: by 2002:a17:902:ecc6:b0:1b1:9272:55e2 with SMTP id a6-20020a170902ecc600b001b1927255e2mr21755968plh.3.1687424211650; Thu, 22 Jun 2023 01:56:51 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id h2-20020a170902f7c200b001b549fce345sm4806971plw.230.2023.06.22.01.56.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 01:56:51 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, dm-devel@redhat.com, linux-raid@vger.kernel.org, linux-bcache@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-nfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-btrfs@vger.kernel.org, Qi Zheng Subject: [PATCH 21/29] fs: super: dynamically allocate the s_shrink Date: Thu, 22 Jun 2023 16:53:27 +0800 Message-Id: <20230622085335.77010-22-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230622085335.77010-1-zhengqi.arch@bytedance.com> References: <20230622085335.77010-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: B1C0140009 X-Stat-Signature: feph1f1uy9fskkdcw4qk55ufei5m7or4 X-Rspam-User: X-HE-Tag: 1687424212-168595 X-HE-Meta: U2FsdGVkX19nEA9RrBktYoDl+uvaKjyIdWmKd9IVzx1k6Pi7pwu0WsEmUjs8/jBgjONEbLil98oVf8SAXQeReu/Bz2T2EDaGHWXRNRLmuibflmR/4CnL8GbOtUhoNZCSSBtpm3Yo4UYdCjWXJi1htiPDQaIQD+entl7KyRUY8+HFTicr/3cAqTS6keenxLBNH6XV16UmwMQYFc5QAW/sBIWqcuSe4VAbbIJf+WlpCsES7XH+9KSrfjndb6lzFH1NnW1aSkdUaYRUgnopCm9qbjFgP6B1rTPltp9TlAxxiP5+vPu04BkilKqV3O0QeS76uWLhD1+/yFWfkBlhWxnb5zIAFlZx5zcErcMKHsKAm0/kBQoHhkdFv/MTUK58sNEEJz+b8Y/H/7yf1929En5j6yBBAoKIjTUvpXSK5vHfoVVjLP89joDGgISxHGxsofiygdS7qv8zhjQvGXbeeQJcRllPav3WdzB/8A6xrWTkUst2hnwZQQ3811b3FDjgiV2pdTjZEHD+mQ9mET+yHFj+7Ie5Q8akKQgx1FD0HwF/28Rjm+5/hEcSxO9RpwKnzDcZnykncwqdn9ZYDg4K6J9ScBo0wqidQWa5nNlU9BV/JHAbd4vcZdNKiCH8Tq7goyv9tetlRwsIIwXVrRqvlf25cX2jNknv9+x38P12s23Qg53nCA9Md3HgMi/QEf5qaNBflITglC0HByoQnO6LoSOsByaqW6xWPSntbHWB16dZ78u1N0jGEO5yP4+ftLGvTWDebccjjeaBxuZXtfNyb+J+HG72EOaMq0rU0M6R53Pp5XxCiRkSbRms4JHHQbfv5Y3R73L4SD412hhwCgRoADlCqroy71QY/Z3pAGXvS4sI8ymCDR/JFCla5PN4SPOyYIFhlbgGd08o/DDAqQgNRVU5nvwghKft4w/pbadwv5kNGJGypiUEDIb/pTd8y2kGJ0kvg1Cd6AvNMwYe7CBMbfH AFrTkbn3 emYy1j+yGEJz7IeBdx7aATb3aUg0SXkLsflab5lIdGxJjjkJBllb/1X4eZKmXiCi9mc2g2AEmLvENW7qLEuzwPrpi5vAYBYqIaWlPjNX+bL9xw8Md7gmPdjKVr5Ka2xzQI3DEyGF4ArRSZcS9LF52xPtWg/1+vMiGJSkk+V1b6p9jBILi7u67/CVkqdaPW6fF70DwctJBX9Ca4G3caQIbmxl9D58cgzCPGGF9yYFIa/s+N6htL4vYpXwRwjk5SXXtpOagfA5xHl9TcR/j/adZYFMW9y252vDWNxTyTbhmOVjr6cdTRvSKjPOwnlVOBe69XHtAMVP/DeGVwh0lqjEa+Y1b+f8KwHmjl6KGc4QpfRAX7I5wWOSv7kgTu9MBaBmILGERxl5V0NR2QwOJyWefW+9z/MFvEjH+u3RIUtUbGgzPCwCoKCcjcjxsWTVw+KmzqUlbcyFqqbRhuTtm9fcIZnXqOQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for implementing lockless slab shrink, we need to dynamically allocate the s_shrink, so that it can be freed asynchronously using kfree_rcu(). Then it doesn't need to wait for RCU read-side critical section when releasing the struct super_block. Signed-off-by: Qi Zheng --- fs/btrfs/super.c | 2 +- fs/kernfs/mount.c | 2 +- fs/proc/root.c | 2 +- fs/super.c | 38 ++++++++++++++++++++++---------------- include/linux/fs.h | 2 +- 5 files changed, 26 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index f1dd172d8d5b..fad4ded26c80 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1513,7 +1513,7 @@ static struct dentry *btrfs_mount_root(struct file_system_type *fs_type, error = -EBUSY; } else { snprintf(s->s_id, sizeof(s->s_id), "%pg", bdev); - shrinker_debugfs_rename(&s->s_shrink, "sb-%s:%s", fs_type->name, + shrinker_debugfs_rename(s->s_shrink, "sb-%s:%s", fs_type->name, s->s_id); btrfs_sb(s)->bdev_holder = fs_type; error = btrfs_fill_super(s, fs_devices, data); diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index d49606accb07..2657ff1181f1 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -256,7 +256,7 @@ static int kernfs_fill_super(struct super_block *sb, struct kernfs_fs_context *k sb->s_time_gran = 1; /* sysfs dentries and inodes don't require IO to create */ - sb->s_shrink.seeks = 0; + sb->s_shrink->seeks = 0; /* get root inode, initialize and unlock it */ down_read(&kf_root->kernfs_rwsem); diff --git a/fs/proc/root.c b/fs/proc/root.c index a86e65a608da..22b78b28b477 100644 --- a/fs/proc/root.c +++ b/fs/proc/root.c @@ -188,7 +188,7 @@ static int proc_fill_super(struct super_block *s, struct fs_context *fc) s->s_stack_depth = FILESYSTEM_MAX_STACK_DEPTH; /* procfs dentries and inodes don't require IO to create */ - s->s_shrink.seeks = 0; + s->s_shrink->seeks = 0; pde_get(&proc_root); root_inode = proc_get_inode(s, &proc_root); diff --git a/fs/super.c b/fs/super.c index 2e83c8cd435b..791342bb8ac9 100644 --- a/fs/super.c +++ b/fs/super.c @@ -67,7 +67,7 @@ static unsigned long super_cache_scan(struct shrinker *shrink, long dentries; long inodes; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * Deadlock avoidance. We may hold various FS locks, and we don't want @@ -120,7 +120,7 @@ static unsigned long super_cache_count(struct shrinker *shrink, struct super_block *sb; long total_objects = 0; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * We don't call trylock_super() here as it is a scalability bottleneck, @@ -182,7 +182,10 @@ static void destroy_unused_super(struct super_block *s) security_sb_free(s); put_user_ns(s->s_user_ns); kfree(s->s_subtype); - free_prealloced_shrinker(&s->s_shrink); + if (s->s_shrink) { + free_prealloced_shrinker(s->s_shrink); + shrinker_free(s->s_shrink); + } /* no delays needed */ destroy_super_work(&s->destroy_work); } @@ -259,16 +262,19 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, s->s_time_min = TIME64_MIN; s->s_time_max = TIME64_MAX; - s->s_shrink.seeks = DEFAULT_SEEKS; - s->s_shrink.scan_objects = super_cache_scan; - s->s_shrink.count_objects = super_cache_count; - s->s_shrink.batch = 1024; - s->s_shrink.flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE; - if (prealloc_shrinker(&s->s_shrink, "sb-%s", type->name)) + s->s_shrink = shrinker_alloc_and_init(super_cache_count, + super_cache_scan, 1024, + DEFAULT_SEEKS, + SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE, + s); + if (!s->s_shrink) + goto fail; + + if (prealloc_shrinker(s->s_shrink, "sb-%s", type->name)) goto fail; - if (list_lru_init_memcg(&s->s_dentry_lru, &s->s_shrink)) + if (list_lru_init_memcg(&s->s_dentry_lru, s->s_shrink)) goto fail; - if (list_lru_init_memcg(&s->s_inode_lru, &s->s_shrink)) + if (list_lru_init_memcg(&s->s_inode_lru, s->s_shrink)) goto fail; return s; @@ -326,7 +332,7 @@ void deactivate_locked_super(struct super_block *s) { struct file_system_type *fs = s->s_type; if (atomic_dec_and_test(&s->s_active)) { - unregister_shrinker(&s->s_shrink); + unregister_and_free_shrinker(s->s_shrink); fs->kill_sb(s); /* @@ -599,7 +605,7 @@ struct super_block *sget_fc(struct fs_context *fc, hlist_add_head(&s->s_instances, &s->s_type->fs_supers); spin_unlock(&sb_lock); get_filesystem(s->s_type); - register_shrinker_prepared(&s->s_shrink); + register_shrinker_prepared(s->s_shrink); return s; share_extant_sb: @@ -678,7 +684,7 @@ struct super_block *sget(struct file_system_type *type, hlist_add_head(&s->s_instances, &type->fs_supers); spin_unlock(&sb_lock); get_filesystem(type); - register_shrinker_prepared(&s->s_shrink); + register_shrinker_prepared(s->s_shrink); return s; } EXPORT_SYMBOL(sget); @@ -1308,7 +1314,7 @@ int get_tree_bdev(struct fs_context *fc, down_write(&s->s_umount); } else { snprintf(s->s_id, sizeof(s->s_id), "%pg", bdev); - shrinker_debugfs_rename(&s->s_shrink, "sb-%s:%s", + shrinker_debugfs_rename(s->s_shrink, "sb-%s:%s", fc->fs_type->name, s->s_id); sb_set_blocksize(s, block_size(bdev)); error = fill_super(s, fc); @@ -1381,7 +1387,7 @@ struct dentry *mount_bdev(struct file_system_type *fs_type, down_write(&s->s_umount); } else { snprintf(s->s_id, sizeof(s->s_id), "%pg", bdev); - shrinker_debugfs_rename(&s->s_shrink, "sb-%s:%s", + shrinker_debugfs_rename(s->s_shrink, "sb-%s:%s", fs_type->name, s->s_id); sb_set_blocksize(s, block_size(bdev)); error = fill_super(s, data, flags & SB_SILENT ? 1 : 0); diff --git a/include/linux/fs.h b/include/linux/fs.h index 53e0b5e98046..dd6f8ce28385 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1228,7 +1228,7 @@ struct super_block { const struct dentry_operations *s_d_op; /* default d_op for dentries */ - struct shrinker s_shrink; /* per-sb shrinker handle */ + struct shrinker *s_shrink; /* per-sb shrinker handle */ /* Number of inodes with nlink == 0 but still referenced */ atomic_long_t s_remove_count;