From patchwork Thu Aug 24 03:42:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13363501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64E18C71145 for ; Thu, 24 Aug 2023 03:49:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00A5F2800AC; Wed, 23 Aug 2023 23:49:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFD878E0011; Wed, 23 Aug 2023 23:49:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D761D2800AC; Wed, 23 Aug 2023 23:49:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C3F388E0011 for ; Wed, 23 Aug 2023 23:49:30 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 938688018C for ; Thu, 24 Aug 2023 03:49:30 +0000 (UTC) X-FDA: 81157618500.07.0DEC44E Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) by imf10.hostedemail.com (Postfix) with ESMTP id CD222C0003 for ; Thu, 24 Aug 2023 03:49:27 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=I3XSiBJx; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692848967; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4pvJpzZ+JGddoFyhhdTQRQhhn5Ey+9a9fh8KQ3jqI+8=; b=7Z7v2p3bVs0JvtqTG1AL80KxBaZovdvUpZze3ibxB9zZhwU/RLAEfkyu2YnvoQntc/RE06 BpMlAZlobVOu5scPZtAflFEGUXVDG6OmClFEC6hfMkDPAf6G7T4Wq9DfBsEC1rKgGk2N+p EFiBtd04GBAbfJ+Bxb29ukosguTcAJg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692848967; a=rsa-sha256; cv=none; b=IIOrx3Ztcl2F9s4HaIKXfBy01SEXtxRaxZKZJMp6kJIQFyXrv91H1aw6eH2PfXbUV3P0fO i0XKCsBQcwK+7zX2XHI/kQQayp/1rogxoYoTu/BMAehsAytgE1uAVtDe88vqsCYoTp5MfI xU9CMK5q8pqN2SKuvqjWgoHG1xFTcQE= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=I3XSiBJx; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.161.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-57328758a72so32807eaf.1 for ; Wed, 23 Aug 2023 20:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692848967; x=1693453767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4pvJpzZ+JGddoFyhhdTQRQhhn5Ey+9a9fh8KQ3jqI+8=; b=I3XSiBJxFGTyV5v72oW8iYTiSCqWs4HZ1HZw8oZzwqWxapAobrNgMr8upBULWpii7y VR2axtb8bjGL7XqnnQpdat9npGu/rRtd1Fgchh92o30XtKGQss0no/eGxWyQbTc1WMx1 1YnguI2ZWqaEG8ObOf/2A9+7l+bPWm8QGpyq6VNnu7Jm/2nad6x1VHqoAUYkdlXiNgcT cEdDi8dcaQystHhf/K3a20Y6CNUZTBRKOiG8A4F6nPXP93ugh7YPdHiuGUqSFmHRe5rI jDI7/ioUA3WKm/fBXgl0YVy2WWa330giSCtLLFW761sWac9t2hy1dXUh9fjJ5Q2xW36a 0spw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692848967; x=1693453767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4pvJpzZ+JGddoFyhhdTQRQhhn5Ey+9a9fh8KQ3jqI+8=; b=XREJn9NqLZ7q+XUvJkEE/Wi8WZcAK72We5ouFZSoXAi6WTGnzJY7MvGDWmQpPhcOyq uMjMi9ZnkbHwlCPTgnJ/TuHX+ia2ennJ+kLSlqMZ/LVs77Writ1YMSo1sDyp7DaCUq1B UVeeUoFZQkpyanCFosx7BEWZL04mgB89b9rq0cFIwZyTSdVBwltoGEtiwEOVMCy62p61 PmXtnBDCasiryGRNqCJGetQt9GpMpvkqYw5BXkmMasnF5KR+ruUVrj8ctCYCrSnEe1Ty +qxPwyWX6SM20fKWDP23p9Ff2yDlXL4gIt4kBCf7MkGWc3N39wJpa+MxseeE+o1RWnr7 eU0A== X-Gm-Message-State: AOJu0Yz+hsLFs2f8Hsuww2KM0mlbemaU7gMELN4KtRd5TKwafG82dFM/ pjInUE0vlucv3yZWvvJsQg5E5A== X-Google-Smtp-Source: AGHT+IE8LNv4W4yBEr9dpKHpPUllD8RN42Yg87AfWyhrznuN0/qSlKWFsLw1CNWJr1EuA0HNZqRQ4g== X-Received: by 2002:a05:6808:1825:b0:3a3:a8d3:e01 with SMTP id bh37-20020a056808182500b003a3a8d30e01mr15808941oib.3.1692848966929; Wed, 23 Aug 2023 20:49:26 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id t6-20020a63b246000000b005579f12a238sm10533157pgo.86.2023.08.23.20.49.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 20:49:26 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Qi Zheng , Muchun Song , Chris Mason , Josef Bacik , David Sterba , Alexander Viro , linux-btrfs@vger.kernel.org Subject: [PATCH v5 38/45] fs: super: dynamically allocate the s_shrink Date: Thu, 24 Aug 2023 11:42:57 +0800 Message-Id: <20230824034304.37411-39-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230824034304.37411-1-zhengqi.arch@bytedance.com> References: <20230824034304.37411-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: CD222C0003 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: osg8q6mxw3pibeju4b3rj196qrztig77 X-HE-Tag: 1692848967-579469 X-HE-Meta: U2FsdGVkX1+PPaZ9fMCUmzg7EH6Tr5e9mFD8JjpvGToK+dji4T1JWB9Go6EHlQqSaCqx6D3JuntPyTOnXRG5u6vHbiorToZhd816AQfdMNizF76CsZUvxlWE/GW49ahhz4AFKkcD6z1Q4VfBZhCI4f+9jhGjbvYQF1OUXvPFslUGrK4WEKdFeFiJUDZMKT8ti3eeYi3lpp6bv5j/4v83sLq24i+U5qgaQwDPnBCuNnTX+goQH+7czPS+RzrQaS4YQ3LSdHOOJwzt5xCZOpO34OHMjY4gDniAXnHah0BwiRSepLWk70pywXIoVrc/ShnfylfELxZY2iHCQtTh/msIHOCmIG/WnFRpliadMji2TNuYedjWO87CapQBB3fx9v0lU8kGsTFZ35T647H6LjQDa/+Cbx079ybWpna1tUigF7r22lazaxJt3Aw6m/2M2XOM8JRi8JOAA6wKRzlptZPMRqJlT/SoaUo+i0e0WZ1epPGhaAVE0rRjN8Est4EeekBi1E4laZ/OVbB4PzMMAg56QGde9n1++/Rm3v5udBt30T5L1C2QUB1rzkvUO0daZ5J0mHtHsWhvR90xekSoIkR9jO8zpRqTtI1qfrCnI98X4SCMlLsOhhGxwxjD4cL8P/6c0GS4cxiITS0wT07pToUHVbWtOWTaBZxfTjieYnMzvAcVqn0xuBdYnfuyqsKHlZJLmp3JgjVCBq/eOZmZQjCWyu5nlcYvfQZhTGHUD5R7tDwivwCjQniIhphonGdvqkf6p3IihY9LM6YyMfrkz1dRZvmAXWAY4quF9MLJum3X/5V1+heOVcNN2a6aLT7U5qu3plYm3ZbxFKosuNmmznlcss60j7tVn+f/u0AThAf/qdxkNN/VS+aU+Y1nPs5EIBCYL/qQ9/kv0s6iN6Sor8biDDqrHQJkQIhH/LAB4jPtwZ7Hs2g43wNNEkjp40oV6sHBm3636WKgcp9HmepawYc o4nr92k0 JB5ec8DyI6gvCvRt7rbR/640dpZ5ejEResPBWa3TImbcaI8QkKTLalRNeJ/AA4cZjlBpTUv7g/nRsEws7+zXrlIWlM13dCzsfWwc9kbxPe/x7xjoAxowmQR6Ym8DiN4IwLTd8SToodPj9Q+anU8Lto06BUQ8dXWiELNBToG5ri/aJ5euWPw3SP/UgGPPCxl+b1/FhMzbc0NsrwFKlVil48hFnikuqlU+RCdtUe89EtJBycyz61bLl6/ytJliZ6BLp03Ezmb1QDe/VcG8X4idtR8qiH30WBHxbBG230peCFP7fXGwp6taroXTaYy2DPLRHtihMgru5HSu15vs7I5l4UCCLbLp5OPjgwXa6QncTGMph10RlwDkdPV5l53Fo7e8VRJO9mnzBR5tprdaw1MP8dJNrG7d+WiaJvw72282lb5hYsiVBqrMTjc7wYUM9g3UAS51qU5VLDvYNEcNANZ1vO0bzkean3P31h7OxL7ekVTKuwsBiy2QCfBkIcAtVPKJQWsw5lbICSF2sZ0BeqLWoZ/qiersZ4UsW+7QmKvg36nMnnhB+bB6ueHDnqAIu9649lHi9ov7zBDJ4guge9EK/AsCIQG6gNV96PTmvj42ftvCS5vKQ2XL7LOgfKxLrQD+L7QCJdsZYg7Tzb9uVSBYLnmF8gmhZnA5menNO6z0ZPcUOMGu1RwKUAmFUrC+i1iYwhRErovppfbbLuJjyrOrdLvaMmK80gHEEJN7/mYDTmIAik7HYpb3H2Jofh17Myt9VQcOE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for implementing lockless slab shrink, use new APIs to dynamically allocate the s_shrink, so that it can be freed asynchronously via RCU. Then it doesn't need to wait for RCU read-side critical section when releasing the struct super_block. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song CC: Chris Mason CC: Josef Bacik CC: David Sterba CC: Alexander Viro CC: Christian Brauner CC: linux-btrfs@vger.kernel.org --- fs/btrfs/super.c | 2 +- fs/kernfs/mount.c | 2 +- fs/proc/root.c | 2 +- fs/super.c | 34 +++++++++++++++++++--------------- include/linux/fs.h | 2 +- 5 files changed, 23 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 09bfe68d2ea3..3b165d9967bb 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1519,7 +1519,7 @@ static struct dentry *btrfs_mount_root(struct file_system_type *fs_type, error = -EBUSY; } else { snprintf(s->s_id, sizeof(s->s_id), "%pg", bdev); - shrinker_debugfs_rename(&s->s_shrink, "sb-%s:%s", fs_type->name, + shrinker_debugfs_rename(s->s_shrink, "sb-%s:%s", fs_type->name, s->s_id); btrfs_sb(s)->bdev_holder = fs_type; error = btrfs_fill_super(s, fs_devices, data); diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c index c4bf26142eec..79b96e74a8a0 100644 --- a/fs/kernfs/mount.c +++ b/fs/kernfs/mount.c @@ -265,7 +265,7 @@ static int kernfs_fill_super(struct super_block *sb, struct kernfs_fs_context *k sb->s_time_gran = 1; /* sysfs dentries and inodes don't require IO to create */ - sb->s_shrink.seeks = 0; + sb->s_shrink->seeks = 0; /* get root inode, initialize and unlock it */ down_read(&kf_root->kernfs_rwsem); diff --git a/fs/proc/root.c b/fs/proc/root.c index 9191248f2dac..b55dbc70287b 100644 --- a/fs/proc/root.c +++ b/fs/proc/root.c @@ -188,7 +188,7 @@ static int proc_fill_super(struct super_block *s, struct fs_context *fc) s->s_stack_depth = FILESYSTEM_MAX_STACK_DEPTH; /* procfs dentries and inodes don't require IO to create */ - s->s_shrink.seeks = 0; + s->s_shrink->seeks = 0; pde_get(&proc_root); root_inode = proc_get_inode(s, &proc_root); diff --git a/fs/super.c b/fs/super.c index 719b67cd4a6b..5ce0c51bb445 100644 --- a/fs/super.c +++ b/fs/super.c @@ -191,7 +191,7 @@ static unsigned long super_cache_scan(struct shrinker *shrink, long dentries; long inodes; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * Deadlock avoidance. We may hold various FS locks, and we don't want @@ -244,7 +244,7 @@ static unsigned long super_cache_count(struct shrinker *shrink, struct super_block *sb; long total_objects = 0; - sb = container_of(shrink, struct super_block, s_shrink); + sb = shrink->private_data; /* * We don't call super_trylock_shared() here as it is a scalability @@ -306,7 +306,7 @@ static void destroy_unused_super(struct super_block *s) security_sb_free(s); put_user_ns(s->s_user_ns); kfree(s->s_subtype); - free_prealloced_shrinker(&s->s_shrink); + shrinker_free(s->s_shrink); /* no delays needed */ destroy_super_work(&s->destroy_work); } @@ -383,16 +383,20 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, s->s_time_min = TIME64_MIN; s->s_time_max = TIME64_MAX; - s->s_shrink.seeks = DEFAULT_SEEKS; - s->s_shrink.scan_objects = super_cache_scan; - s->s_shrink.count_objects = super_cache_count; - s->s_shrink.batch = 1024; - s->s_shrink.flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE; - if (prealloc_shrinker(&s->s_shrink, "sb-%s", type->name)) + s->s_shrink = shrinker_alloc(SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE, + "sb-%s", type->name); + if (!s->s_shrink) goto fail; - if (list_lru_init_memcg(&s->s_dentry_lru, &s->s_shrink)) + + s->s_shrink->seeks = DEFAULT_SEEKS; + s->s_shrink->scan_objects = super_cache_scan; + s->s_shrink->count_objects = super_cache_count; + s->s_shrink->batch = 1024; + s->s_shrink->private_data = s; + + if (list_lru_init_memcg(&s->s_dentry_lru, s->s_shrink)) goto fail; - if (list_lru_init_memcg(&s->s_inode_lru, &s->s_shrink)) + if (list_lru_init_memcg(&s->s_inode_lru, s->s_shrink)) goto fail; return s; @@ -450,7 +454,7 @@ void deactivate_locked_super(struct super_block *s) { struct file_system_type *fs = s->s_type; if (atomic_dec_and_test(&s->s_active)) { - unregister_shrinker(&s->s_shrink); + shrinker_free(s->s_shrink); fs->kill_sb(s); /* @@ -808,7 +812,7 @@ struct super_block *sget_fc(struct fs_context *fc, hlist_add_head(&s->s_instances, &s->s_type->fs_supers); spin_unlock(&sb_lock); get_filesystem(s->s_type); - register_shrinker_prepared(&s->s_shrink); + shrinker_register(s->s_shrink); return s; share_extant_sb: @@ -891,7 +895,7 @@ struct super_block *sget(struct file_system_type *type, hlist_add_head(&s->s_instances, &type->fs_supers); spin_unlock(&sb_lock); get_filesystem(type); - register_shrinker_prepared(&s->s_shrink); + shrinker_register(s->s_shrink); return s; } EXPORT_SYMBOL(sget); @@ -1484,7 +1488,7 @@ int setup_bdev_super(struct super_block *sb, int sb_flags, mutex_unlock(&bdev->bd_fsfreeze_mutex); snprintf(sb->s_id, sizeof(sb->s_id), "%pg", bdev); - shrinker_debugfs_rename(&sb->s_shrink, "sb-%s:%s", sb->s_type->name, + shrinker_debugfs_rename(sb->s_shrink, "sb-%s:%s", sb->s_type->name, sb->s_id); sb_set_blocksize(sb, block_size(bdev)); return 0; diff --git a/include/linux/fs.h b/include/linux/fs.h index c8ff4156a0a1..82a0b5aca045 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1265,7 +1265,7 @@ struct super_block { const struct dentry_operations *s_d_op; /* default d_op for dentries */ - struct shrinker s_shrink; /* per-sb shrinker handle */ + struct shrinker *s_shrink; /* per-sb shrinker handle */ /* Number of inodes with nlink == 0 but still referenced */ atomic_long_t s_remove_count;