From patchwork Tue Jan 30 01:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90525C47DB3 for ; Tue, 30 Jan 2024 01:42:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628F16B00B4; Mon, 29 Jan 2024 20:42:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 589276B00B5; Mon, 29 Jan 2024 20:42:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DF326B00B6; Mon, 29 Jan 2024 20:42:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 28E7D6B00B4 for ; Mon, 29 Jan 2024 20:42:31 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0661E140651 for ; Tue, 30 Jan 2024 01:42:31 +0000 (UTC) X-FDA: 81734277702.12.EF99E40 Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf20.hostedemail.com (Postfix) with ESMTP id 32A651C000D for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="C3H9L/PT"; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.175 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=xFibc/+4QqSvIOE0Xe+y2BrIQPh/5Kew6qFA5+1jpuu04UxTi73eo3TmQT8WQnlEdRHCUk Nl5GA1Xh1DAQPN28T3uM7xnt34uWBSczmE8/AOhiWmudOnY25+LAFniVssMkoFHrIzE4yc IhijPh3e3RsesggK9kTcxHljQ/uXse8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="C3H9L/PT"; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.175 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578949; a=rsa-sha256; cv=none; b=F1jWE3kHx2EI4sjf1B74z2C7lqtUCjq1XctU2iBm3uJNfDjd82Xh4XhFRCz6Daha82evrv 6swCi6TTYzPMotO+JM3g5wzNzf/cFuOQhgWHEYyd8a/Qn7HfzUJxG6j42x1ChlFYC3RBFa 5ZQK/wLC8M7vZRRPyu5wuYuhVzu6jRc= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-42993124fa1so32817451cf.3 for ; Mon, 29 Jan 2024 17:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578948; x=1707183748; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=C3H9L/PT0XzpqK0vd/ULe5zwWR0aLptwHd/Rwa8z6D9cg9XE8f/1BJHeKfYUPOIU6G 9Vl1GpWGTrTnNk4iUgAhJQUQkKN3j9GQBeKNi0Ol+qy941RaRq7PCwCsllKILPE7TLbX POvKvvSPXL2y6dJslLYMXJvzu6bHNSmdYjhYvFtmKzDQxsofnKUqz4Gc0Htl+GjEw5og ASADwU/KzIR8631ST/naMeFdM9bZ+bfnXFVvhohZXs+JF6+kHoTPW/spGvb11Yt3adPH KwLZYEWdxtUl0N3p1zMJD/BNcT9z6U1TeiPwNFLZVinJB95hprUPgPU56eBTTAB6K5aF lCJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578948; x=1707183748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=s059i5S0Wo3SLZfhpQ80udbuZurlHGCzyuL5luXJZpbgDZ96GnFf1Btqp/gRhBJAlW 9qIcPuxxilZ7eAaidA20ayRlAgAVMhhnGRlLgsa9p4rIkH/bqnOGc/Xgj7PRIvAlAo5Z QtRBvITFmwZ3XwaAh7Ttnknw817CL83M+Edh6VjTJyAqbCFlMONuZA9fT5hj/bloy8ge AJ1Q5cn9/HA9EENRqUnV4Bm40Np3IFWFkYsqH155LgqM4oBzpm6kaY4q8OU9tMoP1/ce zpi5ciSDM2VEluHB+ofrESkfIFI+YWcifCJuLMKoDKMyTlDoqsfuoWcLG/O1RYj+XiXJ CArg== X-Gm-Message-State: AOJu0YyDcF0iGgHSTYNn14c7adRO16L0OrywXulzGxaiTehXqCEefXxH SANvwFCsKAsBLr9m84XhIkG9mGFsYEf34ajIUlWs2LvCACLV04fj1ig1a/3FHGk= X-Google-Smtp-Source: AGHT+IHN9SfcdSpe35E+NdhAeZAJBp2Tlz+ZnOTskDGGlYQh7cy5auJKHnbJEqJ5ObxjfCI7Rxu3Cw== X-Received: by 2002:ac8:5994:0:b0:42a:b152:c0fe with SMTP id e20-20020ac85994000000b0042ab152c0femr1228897qte.31.1706578948436; Mon, 29 Jan 2024 17:42:28 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ie7-20020a05622a698700b00427fabefe3bsm4023363qtb.52.2024.01.29.17.42.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:28 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/20] mm: zswap: function ordering: zswap_pools Date: Mon, 29 Jan 2024 20:36:48 -0500 Message-ID: <20240130014208.565554-13-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: fj5xd9d1ayzo3q556gbnjfmdxrk9h7cb X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 32A651C000D X-HE-Tag: 1706578949-325659 X-HE-Meta: U2FsdGVkX195QGDs53SaDL+rZFpfnY4lS48mYweGZQQAVKRw9Z48a+lx2r/e2Q9Rk5i2S+ua6y2050asXSOTVYUTpcXbyaPgZbJoONmtpjVLUtslZkkkC3BcNLRe9HVydjI26SoOXT4PTwfY6U16csGvJB8nbhdpslq3szug0+1V2nd0Yz60okf0fF6Iz/aY56mfbQAa0qqP0m2XE1u38tocgQ4yqlr6fwUPKY5Zzz+pXVkDQ9l0AABfa4+xYoWgUExiYeb8ZKJo6+rlMcSvd02/eeQz0xtL3nZqQfvSMGSkZo44MADop4RVoFFU5L7OUC5dXBeup6tuRi0uJm4gLZliiqV6BFYlpfEdDWDVdCwXOv3iA8n8lAGY04A4iHKMKSLoktWVwVOmxcOOfL4nC+N+QMfjou1MFZTt5/WJREEq9ETbkjgaMjTuW5nM0NRUItyD0Sv4HC4DMuwzL7TH/+b6mRkDnzyJiVIey5Wh/+a/zPrk/Cf5dkcFv+xDhdC3uzN3Gy/gCd5/P5X1++jKLlUCfsRZRrktgvucChigTU2kMqZmjD9iw+e4TVJx7ScC2cO0CziXqSb/PGltSREhdui5+LC9EoTQU+WyLx62SxwDDGKtP2K/8Gfa9XMdZ/CNo4nR4uFk4A3Mj71bQxN6HkvdEc8ox05+d/fYIfdkkHRZmNhHF+jhSqeXcyjRhiGDZXqUeWFscVdqgV0S4xjss8s3RZ8ihWQZc022k2x0bzvdeaEiK5o6kO+5bY6FpcOBVqz1ktBmMF3eYaIih7UJP5H04peztuMRDSnd96mSVROScq9EtxH8Ksc91ONPaP2o7tOMSm1bktfiA30hog5GL7XEGiaxWUvNAaqyWOUIn0RTJRAk1LM+oWMoieimrSekJN/qyL0ftRKIyP/TQvNi2ZjTypXMrNIXl1yRu0FT+3d6s5fSID6SKxrcpquTvWGUKElfIWDD65qBZ25cDdF s4Mee+H1 a8REYi8Oc4MFpYJP6QfqiLngJlAwFcnBOCQqwy+4eTPumvUQhLGyBKyM3unKP4TOxUdaI+ACaDCsbOwR8JC+20xrqG1ieF0v097Y5rQD2kqaTiju2bVCxnxXGJ+KELveOpR7nUm51cu69/CDxDVoXn/zzIstItu9kDUO73/c8H/dkQ+cjRbTcJR6R0fESg77Wzmacoie9G2teNYLNyQmR5iqSpjobZTbbBmTbEQGdVINJ2liVJ822i7YxJXTbIZC/0pWVJfL4RTN7rDoaBd0adVKXgMCWs8i6tO2s5fPBz9kkVM+8zdMZylJOJo8qHqR89f5GeJDDOoCJW4T0Y/Ekov/OmMWKIrmvY1DpkGRoBNzxcT5WFXNL/KFwRChEh8KVlox2Oth78iWrLY7P9plku5BDX+LO9NcfmGje12Clrk4wGlyRmzc0cuarGdAwocLHqJ1syWPoSQPyHemM3c6rsTUp5la+keUJiR0pX3CgHHzPikI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move the operations against the global zswap_pools list (current pool, last, find) to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 150 ++++++++++++++++++++++++++--------------------------- 1 file changed, 73 insertions(+), 77 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 33775f2224b7..168afd6767b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -517,6 +517,79 @@ static void zswap_pool_put(struct zswap_pool *pool) kref_put(&pool->kref, __zswap_pool_empty); } +static struct zswap_pool *__zswap_pool_current(void) +{ + struct zswap_pool *pool; + + pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); + WARN_ONCE(!pool && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + + return pool; +} + +static struct zswap_pool *zswap_pool_current(void) +{ + assert_spin_locked(&zswap_pools_lock); + + return __zswap_pool_current(); +} + +static struct zswap_pool *zswap_pool_current_get(void) +{ + struct zswap_pool *pool; + + rcu_read_lock(); + + pool = __zswap_pool_current(); + if (!zswap_pool_get(pool)) + pool = NULL; + + rcu_read_unlock(); + + return pool; +} + +static struct zswap_pool *zswap_pool_last_get(void) +{ + struct zswap_pool *pool, *last = NULL; + + rcu_read_lock(); + + list_for_each_entry_rcu(pool, &zswap_pools, list) + last = pool; + WARN_ONCE(!last && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + if (!zswap_pool_get(last)) + last = NULL; + + rcu_read_unlock(); + + return last; +} + +/* type and compressor must be null-terminated */ +static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) +{ + struct zswap_pool *pool; + + assert_spin_locked(&zswap_pools_lock); + + list_for_each_entry_rcu(pool, &zswap_pools, list) { + if (strcmp(pool->tfm_name, compressor)) + continue; + /* all zpools share the same type */ + if (strcmp(zpool_get_type(pool->zpools[0]), type)) + continue; + /* if we can't get it, it's about to be destroyed */ + if (!zswap_pool_get(pool)) + continue; + return pool; + } + + return NULL; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -938,83 +1011,6 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -/********************************* -* pool functions -**********************************/ - -static struct zswap_pool *__zswap_pool_current(void) -{ - struct zswap_pool *pool; - - pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); - WARN_ONCE(!pool && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - - return pool; -} - -static struct zswap_pool *zswap_pool_current(void) -{ - assert_spin_locked(&zswap_pools_lock); - - return __zswap_pool_current(); -} - -static struct zswap_pool *zswap_pool_current_get(void) -{ - struct zswap_pool *pool; - - rcu_read_lock(); - - pool = __zswap_pool_current(); - if (!zswap_pool_get(pool)) - pool = NULL; - - rcu_read_unlock(); - - return pool; -} - -static struct zswap_pool *zswap_pool_last_get(void) -{ - struct zswap_pool *pool, *last = NULL; - - rcu_read_lock(); - - list_for_each_entry_rcu(pool, &zswap_pools, list) - last = pool; - WARN_ONCE(!last && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - if (!zswap_pool_get(last)) - last = NULL; - - rcu_read_unlock(); - - return last; -} - -/* type and compressor must be null-terminated */ -static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) -{ - struct zswap_pool *pool; - - assert_spin_locked(&zswap_pools_lock); - - list_for_each_entry_rcu(pool, &zswap_pools, list) { - if (strcmp(pool->tfm_name, compressor)) - continue; - /* all zpools share the same type */ - if (strcmp(zpool_get_type(pool->zpools[0]), type)) - continue; - /* if we can't get it, it's about to be destroyed */ - if (!zswap_pool_get(pool)) - continue; - return pool; - } - - return NULL; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) {