From patchwork Mon Aug 7 11:09:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13343694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C360BC41513 for ; Mon, 7 Aug 2023 11:18:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62EC46B007E; Mon, 7 Aug 2023 07:18:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B9CC8D0005; Mon, 7 Aug 2023 07:18:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40AE18D0003; Mon, 7 Aug 2023 07:18:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 294BD6B007E for ; Mon, 7 Aug 2023 07:18:46 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F263540978 for ; Mon, 7 Aug 2023 11:18:45 +0000 (UTC) X-FDA: 81097061010.10.AFE0E45 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) by imf24.hostedemail.com (Postfix) with ESMTP id 0E6C5180019 for ; Mon, 7 Aug 2023 11:18:43 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PfEVKvTz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691407124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AnO+VrCD4LojLy+8kmnCjGg0ZRt7Zy9NIxA02Cs4pZw=; b=3vh5cJyajCWcuKt1ajTjCu2/lvGaWF39qYCwvPrexZ4pQmGztmFBLLrFATMgcGyQcAu0H7 zzvK/NV5888FPp7JVZ1Rw/h5qGT9cc3/BDu54ms7jUduAFuWHcsHEr2fPJ2w7GhkXeRcYn OTjFmribHlaxtzCmp+pLvrrTjusuEBo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PfEVKvTz; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.177 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691407124; a=rsa-sha256; cv=none; b=hz/D4ygnWct0hSS+RtrlWTEKGrOH1i0QzI8QQPIgEYw1gFzMCYG+RZ/Enipe5NePZF2p0q iuX5+IqZEYL2RC8EQynRQ+eRNBrI856G/oz0v0n15V4Ez/cJFxUMIb5n0hKnpdHJP8uW0T kJsTvDkG2EqOW2WtJH9202PfPlUn/hw= Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-55b5a37acb6so362922a12.0 for ; Mon, 07 Aug 2023 04:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1691407123; x=1692011923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AnO+VrCD4LojLy+8kmnCjGg0ZRt7Zy9NIxA02Cs4pZw=; b=PfEVKvTzBWRKzDZcOs/Fep65btb/5j6s3WDHceANi3UdyfyE1ZYJjj/ojCsgHi3Vd1 UnyiA9UJ+WO0skTyzkklbevZFVt0QM5P026zpV9irULg/dt9elBIaT9aO85mKj85kzSK 8BL7WAzqEgJCUSOrld3eSg+Xi6Bj/nVLlSU4qlnz3r3ovJoXr2T2o0DtWh8ePmCGKfhu 8HAbYH+dcq7rqiPTVsEp3knKk+0ZUdZ34RURzik4J4PgZDJhHLpdrzH5JSXxHECtRx3a erZSmwbY1m1zt8qE5RzbmfCM7KzyvNBKjG86UwibulZ8+wpz72tXfP+ENX4A3BoRP40M UhJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691407123; x=1692011923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AnO+VrCD4LojLy+8kmnCjGg0ZRt7Zy9NIxA02Cs4pZw=; b=aOtn76NGPHEHS66B9a1f2YHjZCLiMdyCwBcDGf8XuNQl0PkXerKdQ+2jpzEjd6ImCL bEUHYdJgYczv/CT2ImlDK4KK9vsBUvh7x7lF2QDVP+16eNo9YGjg197wU0VIrpzYv5Nr pGQSDIeWJMRk0NqY/ge3UHMIeW1+tzGLPAd2UVjyyvWf5GSVnqOpDv1s1mtbjYdGN9Hy 4a/vHQTdnbGdhGlbHjjiKvdS3g6pm5eHd32pGcWbLBWo45tvK0J9dO6GtoZOJ5UsUmR5 VjyBfyH8i/OzxnvuT1dFMsFAkJEnnVhYqeZF8XnUcsaWKSNB4ZaoTgEz3ikFE/w6XxSg fg5w== X-Gm-Message-State: ABy/qLZ+MSTHWLbEnout4xbPamXokiTQOownk+xbTR///OU38QyIIq1U rVpbhm+EtWrBACHjmUzlTNSZ4w== X-Google-Smtp-Source: APBJJlGN7o+mPwW+D250kL/E75whSiIFGnI9vIACxkViJa+uI/EkAzzWgkZZOw/GfOlqJQ+tGCA4bg== X-Received: by 2002:a17:90a:3f08:b0:268:1e3b:14c9 with SMTP id l8-20020a17090a3f0800b002681e3b14c9mr23489214pjc.2.1691407122942; Mon, 07 Aug 2023 04:18:42 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id y13-20020a17090aca8d00b0025be7b69d73sm5861191pjt.12.2023.08.07.04.18.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Aug 2023 04:18:42 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, simon.horman@corigine.com, dlemoal@kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, linux-erofs@lists.ozlabs.org, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, linux-nfs@vger.kernel.org, linux-mtd@lists.infradead.org, rcu@vger.kernel.org, netdev@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, dm-devel@redhat.com, linux-raid@vger.kernel.org, linux-bcache@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-btrfs@vger.kernel.org, Qi Zheng , Muchun Song Subject: [PATCH v4 41/48] fs: super: dynamically allocate the s_shrink Date: Mon, 7 Aug 2023 19:09:29 +0800 Message-Id: <20230807110936.21819-42-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230807110936.21819-1-zhengqi.arch@bytedance.com> References: <20230807110936.21819-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0E6C5180019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: afd3mqsnw7iq14dprn9z9okgcjiicg6j X-HE-Tag: 1691407123-457234 X-HE-Meta: U2FsdGVkX1/FrPDiQ3Qgik4FSnVSvdLax0ODiFlKBbuq42Lh20kN4MyvWIi1MfzrYzfTlCSySpfAQ+0gQ7GxAiGiFc1YU35bUsgDDd6MRcvaCpi0NU0zcVrnCFGZrRPBKrbo3CG4/2pzBXb/FSh3MeVNdWc8j6J5hxAs4b667jnmEPpsEYGXiJJIqJQ8dP7FdbLZXRix+0F4O0PiA/+TN5UawC0eIKmcZB0XYM+Tw1ziU+3h4GxRG67r+SHtzt/8yt+69lvc1WkLHblvzl6SSjjPPrAZRmbOLndT8t384qnilyU869lllqa5cTB8qtCZgsQaY6N4UH7c00aXu7zbQWTjO0D4K6BPotI/pOohTmDVe2nARXl0vZPQmiwWP7n3N5NVv17ZfoX11y5j4MBGVAlcqmzqP0cTJE1C2WzVSUuv6NL1frH6ujzJVekVXRwoP8wY57G3/Jl/pC5uKxXpU2v3O96V11sP8/+K1zyAMgUW+HaNNTMcsObwSoi2cZsl5AORBTguKFu4lsbNQMArQaZU2TVYPLC0DX7SFo7DoyTP//j9lSs9zG7IH3UGmcAyssD6RJZ05g0v+ELt3L4ecX9F500Rh9iKsXiGhiPjSl8jbjIYmy5eUEsD3dBb/L+T0OmO7m/kC1lbPcvrRvoWO1QwLcQCrCaSxWDfNhtOF05IchvRZMt14t4Br/cmf9WzpGNdC34ThzchSe3d8J+36PfrzxSWVPpmJeb6yMqSvTL5lnx0tshyLqwYlyHwORJhpInuIhzm4n9Lf6Cti+smMEN24C2DfvkfQqyneWpD439eJSFmhY58x279wFYGqo9qT3GW02Ng+sNIj+jnlUEN2CrgTeaJWtRd4JYc8mH0TBGgCE7cCrvaNyFKAG5LllMtmTpSWvaQtpL0RTP6Xq8sLtB8pNWcs5A1fD+t5cJsfawxB8tznLczpR7NerEDG9xIYDdyclDhQ8sQYspsQso Yr27c5zP g0x59SXzdY0YFyXRY3WkyvXdVxzt8wnGwkKolH8lWkcICu0w12lG64yhaOZrKW+vcZFSpJzXRMftSojQxGBwIFj6VtDYfRBob2exxpGkG3h0gbS4Ue7nf2lsRglsF77RnO1ZTYLD9iNv6GDQlfnPiDNCXldX7nlqoAV3IjPrFUBca2tDnX7vL3tacOo2Gou82g/G+eRnpbZyUM9Ic8h6IFkQe+L1uciP/qn0iQMkCydVkiWkj7Q0SxU385CDfNXl7CVOs19pD0ZrGHvGWpysWTpbCQ6t+iaf8FV8pbneK4BKzoAr+UZ7MoEybxUUh08hsm9nofsDmypWrFS6xsbqerLbx8IYI170ulpbHDUtHFYj1V0xM6rJTqfWj5DdWz81ukcW8cIU35JWDNby+0s6hAaH4EG7rh/8RzoZVebHhdnnl5Hpm/r2e3QoRYXDvzf6yfAk3h5HHe6Mg68VQBqGnY4IIpQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for implementing lockless slab shrink, use new APIs to dynamically allocate the s_shrink, so that it can be freed asynchronously using kfree_rcu(). Then it doesn't need to wait for RCU read-side critical section when releasing the struct super_block. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- fs/btrfs/super.c | 2 +- fs/kernfs/mount.c | 2 +- fs/proc/root.c | 2 +- fs/super.c | 34 +++++++++++++++++++--------------- include/linux/fs.h | 2 +- 5 files changed, 23 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 5ca07b41b4cd..c6980fc6fe02 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1514,7 +1514,7 @@ static struct dentry *btrfs_mount_root(struct file_system_type *fs_type, snprintf(s->s_id, sizeof(s->s_id), "%pg", fs_devices->latest_dev->bdev); - shrinker_debugfs_rename(&s->s_shrink, "sb-%s:%s", fs_type->name, + shrinker_debugfs_rename(s->s_shrink, "sb-%s:%s", fs_type->name, s->s_id); fs_info->bdev_holder = s; error = btrfs_fill_super(s, fs_devices, data); diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index c4bf26142eec..79b96e74a8a0 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -265,7 +265,7 @@ static int kernfs_fill_super(struct super_block *sb, struct kernfs_fs_context *k sb->s_time_gran = 1; /* sysfs dentries and inodes don't require IO to create */ - sb->s_shrink.seeks = 0; + sb->s_shrink->seeks = 0; /* get root inode, initialize and unlock it */ down_read(&kf_root->kernfs_rwsem); diff --git a/fs/proc/root.c b/fs/proc/root.c index 9191248f2dac..b55dbc70287b 100644 --- a/fs/proc/root.c +++ b/fs/proc/root.c @@ -188,7 +188,7 @@ static int proc_fill_super(struct super_block *s, struct fs_context *fc) s->s_stack_depth = FILESYSTEM_MAX_STACK_DEPTH; /* procfs dentries and inodes don't require IO to create */ - s->s_shrink.seeks = 0; + s->s_shrink->seeks = 0; pde_get(&proc_root); root_inode = proc_get_inode(s, &proc_root); diff --git a/fs/super.c b/fs/super.c index 2354bcab6fff..a28193045345 100644 --- a/fs/super.c +++ b/fs/super.c @@ -67,7 +67,7 @@ static unsigned long super_cache_scan(struct shrinker *shrink, long dentries; long inodes; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * Deadlock avoidance. We may hold various FS locks, and we don't want @@ -120,7 +120,7 @@ static unsigned long super_cache_count(struct shrinker *shrink, struct super_block *sb; long total_objects = 0; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * We don't call trylock_super() here as it is a scalability bottleneck, @@ -182,7 +182,7 @@ static void destroy_unused_super(struct super_block *s) security_sb_free(s); put_user_ns(s->s_user_ns); kfree(s->s_subtype); - free_prealloced_shrinker(&s->s_shrink); + shrinker_free(s->s_shrink); /* no delays needed */ destroy_super_work(&s->destroy_work); } @@ -259,16 +259,20 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, s->s_time_min = TIME64_MIN; s->s_time_max = TIME64_MAX; - s->s_shrink.seeks = DEFAULT_SEEKS; - s->s_shrink.scan_objects = super_cache_scan; - s->s_shrink.count_objects = super_cache_count; - s->s_shrink.batch = 1024; - s->s_shrink.flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE; - if (prealloc_shrinker(&s->s_shrink, "sb-%s", type->name)) + s->s_shrink = shrinker_alloc(SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE, + "sb-%s", type->name); + if (!s->s_shrink) goto fail; - if (list_lru_init_memcg(&s->s_dentry_lru, &s->s_shrink)) + + s->s_shrink->seeks = DEFAULT_SEEKS; + s->s_shrink->scan_objects = super_cache_scan; + s->s_shrink->count_objects = super_cache_count; + s->s_shrink->batch = 1024; + s->s_shrink->private_data = s; + + if (list_lru_init_memcg(&s->s_dentry_lru, s->s_shrink)) goto fail; - if (list_lru_init_memcg(&s->s_inode_lru, &s->s_shrink)) + if (list_lru_init_memcg(&s->s_inode_lru, s->s_shrink)) goto fail; return s; @@ -326,7 +330,7 @@ void deactivate_locked_super(struct super_block *s) { struct file_system_type *fs = s->s_type; if (atomic_dec_and_test(&s->s_active)) { - unregister_shrinker(&s->s_shrink); + shrinker_free(s->s_shrink); fs->kill_sb(s); /* @@ -610,7 +614,7 @@ struct super_block *sget_fc(struct fs_context *fc, hlist_add_head(&s->s_instances, &s->s_type->fs_supers); spin_unlock(&sb_lock); get_filesystem(s->s_type); - register_shrinker_prepared(&s->s_shrink); + shrinker_register(s->s_shrink); return s; share_extant_sb: @@ -693,7 +697,7 @@ struct super_block *sget(struct file_system_type *type, hlist_add_head(&s->s_instances, &type->fs_supers); spin_unlock(&sb_lock); get_filesystem(type); - register_shrinker_prepared(&s->s_shrink); + shrinker_register(s->s_shrink); return s; } EXPORT_SYMBOL(sget); @@ -1312,7 +1316,7 @@ int setup_bdev_super(struct super_block *sb, int sb_flags, mutex_unlock(&bdev->bd_fsfreeze_mutex); snprintf(sb->s_id, sizeof(sb->s_id), "%pg", bdev); - shrinker_debugfs_rename(&sb->s_shrink, "sb-%s:%s", sb->s_type->name, + shrinker_debugfs_rename(sb->s_shrink, "sb-%s:%s", sb->s_type->name, sb->s_id); sb_set_blocksize(sb, block_size(bdev)); return 0; diff --git a/include/linux/fs.h b/include/linux/fs.h index 9515d3688f71..1464afd41164 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1263,7 +1263,7 @@ struct super_block { const struct dentry_operations *s_d_op; /* default d_op for dentries */ - struct shrinker s_shrink; /* per-sb shrinker handle */ + struct shrinker *s_shrink; /* per-sb shrinker handle */ /* Number of inodes with nlink == 0 but still referenced */ atomic_long_t s_remove_count;