From patchwork Fri Jun 9 08:15:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13273421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F23EC83005 for ; Fri, 9 Jun 2023 08:17:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B196F8E0009; Fri, 9 Jun 2023 04:17:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA3058E0003; Fri, 9 Jun 2023 04:17:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 944198E0009; Fri, 9 Jun 2023 04:17:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7E34F8E0003 for ; Fri, 9 Jun 2023 04:17:38 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 49FB31C7318 for ; Fri, 9 Jun 2023 08:17:38 +0000 (UTC) X-FDA: 80882505396.02.CDA3FC4 Received: from out-10.mta0.migadu.com (out-10.mta0.migadu.com [91.218.175.10]) by imf15.hostedemail.com (Postfix) with ESMTP id 8525AA0012 for ; Fri, 9 Jun 2023 08:17:36 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=QnAzENdG; spf=pass (imf15.hostedemail.com: domain of qi.zheng@linux.dev designates 91.218.175.10 as permitted sender) smtp.mailfrom=qi.zheng@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686298656; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EMn+msnIJXvC+06grZ542lhB8pLG9zNhZSiyiHhRbaA=; b=1m4S4jmGqGd7GkjvrjHv6AyHX6UXj8jcIgKyoszZdxji0AUAEBbNe7vxo4YYhtnp760ANI OeRZnQ8lfGpnbOgZFuwQnqMDqDZeRgNTOCJ6yWX0eugRswap/+lj0034vFwPYwaWlz/xei tjbbAFmWZ5TPTu0pQDF4AUfNXe+nUmw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686298656; a=rsa-sha256; cv=none; b=pss/dAZHRU+r0/xmmWwKi0b67GFj0EqhN1H9cooshLjQli4as2E2qd1AtXBR2vmtIUofLe Zs/VLpcCEM/5pfN3QedhSQmm59CC2946Uuqz1e30ENTZYUzB4Plj5dOS0a4Kx4jBV9BYn7 ZwdVm1DDAtHu+S7quSaAuOAOP8fl6YI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=QnAzENdG; spf=pass (imf15.hostedemail.com: domain of qi.zheng@linux.dev designates 91.218.175.10 as permitted sender) smtp.mailfrom=qi.zheng@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1686298655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EMn+msnIJXvC+06grZ542lhB8pLG9zNhZSiyiHhRbaA=; b=QnAzENdGYWeYO57Spgjhk1adZxJY7gan2t4wYWPeZIaItdmJZDM8xOICqPyq/QELcJD9OY hG30tKtjBO4CWu/+8SDMmqFoQybaFEZQceY8dbZCLKigPc6SX9awgQJwPEusZAlihLEw5W Hw8FqnopH5nBR0yG+Weoe/3z4ob2VOY= From: Qi Zheng To: akpm@linux-foundation.org Cc: david@fromorbit.com, tkhai@ya.ru, roman.gushchin@linux.dev, vbabka@suse.cz, muchun.song@linux.dev, yujie.liu@intel.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Qi Zheng Subject: [PATCH 7/7] Revert "mm: vmscan: make global slab shrink lockless" Date: Fri, 9 Jun 2023 08:15:18 +0000 Message-Id: <20230609081518.3039120-8-qi.zheng@linux.dev> In-Reply-To: <20230609081518.3039120-1-qi.zheng@linux.dev> References: <20230609081518.3039120-1-qi.zheng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 8525AA0012 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: f95aq6hz41eohe493gbgekdo3xeni8ab X-HE-Tag: 1686298656-325687 X-HE-Meta: U2FsdGVkX1+VwDh9F/OQDS/6C1ppcSWY+97vzqb4XDuwiFQKjhCdCQoJ8XE+0ElLIlWWBzzAsHLntcPsNeSKAQIdGBlqBp3E3k3AYUFuQWFnagOgPqnzdsD+rwO/oVVtPgnquC9VH0LbofTvSLsEUfEi5/vG4j7vfvod7/UGZAu6mL4K8eAiBlA4inzF4zgxJeXXEh6iNtZC+r2yVqiYGrSU2VXbYfyXWEECgT3bBm9CnlLe0bXX2B3l13uUMd8TMuhJ+75lb3KWt0iEq1Bmj7UzFB/oP2BJ+s/C76vWizuqgwTozd9ZkFkgaakuIiQ0fh1IiHqif1dE8BCOHjCT0xnECIRPTF/a2sdlQLi9mwvSq01ULgLiWn3PGcgowTidpMK5ZPjZLp2zrduYA5PrEe4hxY+XACgnu5UM1bpcdX+ZezNp/usrQQZ+GoNGt1NEuogOHefdTUNffzO+m8vn36nfOdC7mVCWpt3iYK8dfZ7X45W69G9ulYFPZh58+hF48D28E0tGfvrXXNXeuHu3mGGWjnfjmuVU2GMVH33ZEhvUr2psrUBNzmDGl/d9faUz0u6hk4J29pYeSpNiHc1WM4BdndAQu959+QHCRkdvCkYvyorWIM5O2SaGy1r0qNfIxess1SdDiSp52eIOXc+djHrtU5etBG5lFY2MhKQxOjY4MKR/4/lXqJNxVnUaa/xPBqAQRiIVy4jG/hZUb6s5nY+Abq7wMVsYdgAdu4oQh1CTLhP7tX6fxVomV2jQXE6vm3TlGglxy88oObes3U9ayhLRu3F54fmP5ptGTHNwt9UYiSP8Q5AdvGoz0T2Am7Eg4GSr6z+tlhdtwIWiUIc/6NSvgQhBAxNL7IlAejlkNO9oiQNL1v5fpBczMSjE2nrf7JGwT4opQENmACbJXJkDXkxLO5xnTqA3qh5XAdRcoGt8FlqrqhIrCx8xTv4IU+EcBWxXTGUU/N3Dh9PmiFK WiMIUzhE gYJqmPayWhe+i6bf+WkBRfb3O5hq8n9hG4yOAE9zftJ/lzaxtqBf9aPBnoPsDnFfnW45bMR1caeSf487iThAzAdSXQh8VHPWJC8EpOnNOp2OoI8y1NcsVqeQIX2J04fThD8jqiAElgK6FTU38evGJEXkua2El4SuE7uixJO4mhH6ujG2E1n3s7+EXrnie3AzSND7L0Q/kkYVLxMfkUsYrhgixmPITZSQDddgM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Qi Zheng This reverts commit f95bdb700bc6bb74e1199b1f5f90c613e152cfa7. Kernel test robot reports -88.8% regression in stress-ng.ramfs.ops_per_sec test case [1], which is caused by commit f95bdb700bc6 ("mm: vmscan: make global slab shrink lockless"). The root cause is that SRCU has to be careful to not frequently check for SRCU read-side critical section exits. Therefore, even if no one is currently in the SRCU read-side critical section, synchronize_srcu() cannot return quickly. That's why unregister_shrinker() has become slower. After discussion, we will try to use the refcount+RCU method [2] proposed by Dave Chinner to continue to re-implement the lockless slab shrink. So revert the shrinker_srcu related changes first. [1]. https://lore.kernel.org/lkml/202305230837.db2c233f-yujie.liu@intel.com/ [2]. https://lore.kernel.org/lkml/ZIJhou1d55d4H1s0@dread.disaster.area/ Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-lkp/202305230837.db2c233f-yujie.liu@intel.com Signed-off-by: Qi Zheng --- mm/vmscan.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index a008d7f2d0fc..5bf98d0a22c9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -57,7 +57,6 @@ #include #include #include -#include #include #include @@ -191,7 +190,6 @@ int vm_swappiness = 60; LIST_HEAD(shrinker_list); DECLARE_RWSEM(shrinker_rwsem); -DEFINE_SRCU(shrinker_srcu); #ifdef CONFIG_MEMCG static int shrinker_nr_max; @@ -742,7 +740,7 @@ void free_prealloced_shrinker(struct shrinker *shrinker) void register_shrinker_prepared(struct shrinker *shrinker) { down_write(&shrinker_rwsem); - list_add_tail_rcu(&shrinker->list, &shrinker_list); + list_add_tail(&shrinker->list, &shrinker_list); shrinker->flags |= SHRINKER_REGISTERED; shrinker_debugfs_add(shrinker); up_write(&shrinker_rwsem); @@ -797,15 +795,13 @@ void unregister_shrinker(struct shrinker *shrinker) return; down_write(&shrinker_rwsem); - list_del_rcu(&shrinker->list); + list_del(&shrinker->list); shrinker->flags &= ~SHRINKER_REGISTERED; if (shrinker->flags & SHRINKER_MEMCG_AWARE) unregister_memcg_shrinker(shrinker); debugfs_entry = shrinker_debugfs_detach(shrinker, &debugfs_id); up_write(&shrinker_rwsem); - synchronize_srcu(&shrinker_srcu); - shrinker_debugfs_remove(debugfs_entry, debugfs_id); kfree(shrinker->nr_deferred); @@ -825,7 +821,6 @@ void synchronize_shrinkers(void) { down_write(&shrinker_rwsem); up_write(&shrinker_rwsem); - synchronize_srcu(&shrinker_srcu); } EXPORT_SYMBOL(synchronize_shrinkers); @@ -1036,7 +1031,6 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, { unsigned long ret, freed = 0; struct shrinker *shrinker; - int srcu_idx; /* * The root memcg might be allocated even though memcg is disabled @@ -1048,10 +1042,10 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg)) return shrink_slab_memcg(gfp_mask, nid, memcg, priority); - srcu_idx = srcu_read_lock(&shrinker_srcu); + if (!down_read_trylock(&shrinker_rwsem)) + goto out; - list_for_each_entry_srcu(shrinker, &shrinker_list, list, - srcu_read_lock_held(&shrinker_srcu)) { + list_for_each_entry(shrinker, &shrinker_list, list) { struct shrink_control sc = { .gfp_mask = gfp_mask, .nid = nid, @@ -1062,9 +1056,19 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, if (ret == SHRINK_EMPTY) ret = 0; freed += ret; + /* + * Bail out if someone want to register a new shrinker to + * prevent the registration from being stalled for long periods + * by parallel ongoing shrinking. + */ + if (rwsem_is_contended(&shrinker_rwsem)) { + freed = freed ? : 1; + break; + } } - srcu_read_unlock(&shrinker_srcu, srcu_idx); + up_read(&shrinker_rwsem); +out: cond_resched(); return freed; }