From patchwork Fri Oct 18 06:47:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13841259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48F02D3C54C for ; Fri, 18 Oct 2024 06:48:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D900C6B00AC; Fri, 18 Oct 2024 02:48:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D40336B00AE; Fri, 18 Oct 2024 02:48:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C08646B00AF; Fri, 18 Oct 2024 02:48:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9F1926B00AC for ; Fri, 18 Oct 2024 02:48:11 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1B59814185B for ; Fri, 18 Oct 2024 06:47:59 +0000 (UTC) X-FDA: 82685793330.15.054C9DA Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by imf24.hostedemail.com (Postfix) with ESMTP id E718D180009 for ; Fri, 18 Oct 2024 06:48:06 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fJ0aDIcZ; spf=pass (imf24.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 192.198.163.14 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729233942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tfa3AhuC5v0Hnd+2+sLbYPM0N2OtZZfu9aqDB5NMoM8=; b=LsIR+aPk8ErOKuYkaiXwhVrPCN7rwbzFQoLDxeTzPpflHkUluTAtD2j81zc673lOIDwQCk csCeU0jClzLTu5Ayw8beVpCt0tQZuQ7u5atwLi+1b08eV5IFJaXjwN/PsQlJuipl64BgqF f7T32Y1VUA2F45p6Nnuvh1TcNykDnK0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729233942; a=rsa-sha256; cv=none; b=pSVIeGCwfBbieObF+cQtVYzYSA1HQDCimGu9yTSw0t9vQPKrJI6b8blICOzPnz1+LI9Ac5 ngECTurzPHDdtTDSKpxO6B6QikCbUjCGZgrhR0tOgzk7uh+zKp0ueoy/mPEPtprz0Z0A28 IhQ+CTLAPNh/xj7/+kG3Np+W7FG2e7U= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fJ0aDIcZ; spf=pass (imf24.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 192.198.163.14 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729234089; x=1760770089; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vgbP0deIxDuw/qX2EJ50fTnXNEzCSKwaUCbRMPN4Zeg=; b=fJ0aDIcZaTxjwPZtQzaCSLYiMdcN8th3q9avRVO5kKpDi7c1qeqLucaJ swWkpxixqYPJaWL8OAolLbtxvVh2FEKwzfYGEIUJPe3HPeVkphFFn5frX /yMb/50ySgx1tYmEbGCT9fENGAN8HmPsiHqbrne25+8K1/KdY964vzwrM CAkRjYuJwZeC49vaWa4djHhaYajkQzvXkTtOYGKCd3eilRLbu+Aj3/e15 6EXLAUI8SQ0cpQLymARW9o9sFMod+OIog2sjAWrnkntIx2QCBaBh1m6wH YQYma1ZfXyPnbsqva3eBb8uMXkTf9lP9ePscgY2klNjx3IbaKe8uks5Ne A==; X-CSE-ConnectionGUID: tbL/JpEhRHe9OLaUG9GxqQ== X-CSE-MsgGUID: FC8wbEC1Rzmxq3ygoX3+4A== X-IronPort-AV: E=McAfee;i="6700,10204,11228"; a="28963316" X-IronPort-AV: E=Sophos;i="6.11,212,1725346800"; d="scan'208";a="28963316" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Oct 2024 23:48:07 -0700 X-CSE-ConnectionGUID: UUU+zR47R3umRvDc5akGeQ== X-CSE-MsgGUID: nOsiFCOlRUqm01m/QxIO7g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,212,1725346800"; d="scan'208";a="82744499" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa003.fm.intel.com with ESMTP; 17 Oct 2024 23:48:06 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, hughd@google.com, willy@infradead.org, bfoster@redhat.com, dchinner@redhat.com, chrisl@kernel.org, david@redhat.com Cc: wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [RFC PATCH v1 1/7] mm: zswap: Config variable to enable zswap loads with decompress batching. Date: Thu, 17 Oct 2024 23:47:59 -0700 Message-Id: <20241018064805.336490-2-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20241018064805.336490-1-kanchana.p.sridhar@intel.com> References: <20241018064805.336490-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Stat-Signature: y11byyd56g5x53zo6y6ni84zcbny5836 X-Rspamd-Queue-Id: E718D180009 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1729234086-913555 X-HE-Meta: U2FsdGVkX19P2adk75bEVZw5hEcyq5Pf5KbwAjxHsJE2QdPSH0aHCufkhs6DRXJ8jDQ2822i3yVIBdY/95xvtsmKhiN2RtVMspfoFz3by8atGWzHyxKuZwoSem6zNpF3CzXnAIuzdKkGaEfk6mFNcGrlO/BBY8BMr0txrRwq9xR11RD5JhWA0r49mlNxoyRshWY6u6JuLu+YYHwx/vC5Wq6PIEx8UCn9Uszf1rl7ddxgMR6+d4m7nAFmy644he5SgCApcT/dTrcBYHpxabs9kPgO29bzNqs18Qh+Z/tIY0yUOlWkEVHHzHRmFmYF11wMxipqCNafuvAsuIW6D7Vv7ppArgnWKRBERIYIPH66ikZNoURIzoRgsB4oK6Tl2jKDK8kbj3+rseydsS4odU2gdkdPfnVGpdgQvfsCEJxd2Qw7oT4U8UJiWr3HQ+xhtluOLi3+NNoO18+CoEkY3uJUWhg4FF0r8Or5JT7Q8AVE7FCrmLtp18Q8635hpl34LflrkYtaXKurDgLLk+hswnhchEX/z7dfxEck3kGTvaX5yBarP3lgOWABq9h4B3+XOikwCP1buqwNpliTuVr0Mw5lZTiB3MULefdGfCQvYGXoko4BBwqbJBslpYBqBM/OGujZL3JTvniv/Vf1NjjROmLF+Ti9dPNslJZA+jMYizzaJDGnulB0Tez5Jh82sZEkyOPqUbGh8kgSiLb4By84/eM6FL/mfW8kl0iJLE9oJHKaNRuVkmF/pe68JdTJD5DSoYRd/REun6lSlM8d8PqYORjYtVp1A55An/w58uuHBsErT5xojtZ8ANHeF28mx8JYvWPcE3LmPfZO684G1BLaD9t6ZYZgOeNn8o2G3TzoDwVzWzTdw4KHmwRL1OOYNT5i6MlbChfVYDo8c5mkvD/cCvrbmAiy28fifwunk8MbU1AwQvZ8Vo2NXIfCxQiimSq+e9sxFyLKsB0bJTigrhmx38r 18wIXR6/ vjecvBXGPCACpT9PgcjT/QC5cHRLw2rlV7IDGky0l+iJDHotCLvO7qCD8MHKfFhWJjjOtLcgsVorS9QB6fAM8DbgPQDCCCwL16aWMSXyLRgI4do2mIv+U1U6n3F0r3HP5b74wlQPJQE1KxQsniiEnYQ1RV240FO1FMiJr7Z/61ggJJfjmhx3h3VOlXA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a new zswap config variable that controls whether zswap load will decompress a batch of 4K folios, for instance, the folios prefetched during swapin_readahead(): CONFIG_ZSWAP_LOAD_BATCHING_ENABLED The existing CONFIG_CRYPTO_DEV_IAA_CRYPTO variable added in commit ea7a5cbb4369 ("crypto: iaa - Add Intel IAA Compression Accelerator crypto driver core") is used to detect if the system has the Intel Analytics Accelerator (IAA), and the iaa_crypto module is available. If so, the kernel build will prompt for CONFIG_ZSWAP_LOAD_BATCHING_ENABLED. Hence, users have the ability to set CONFIG_ZSWAP_LOAD_BATCHING_ENABLED="y" only on systems that have Intel IAA. If CONFIG_ZSWAP_LOAD_BATCHING_ENABLED is enabled, and IAA is configured as the zswap compressor, the vm.page-cluster is used to prefetch up to 32 4K folios using swapin_readahead(). The readahead folios present in zswap are then loaded as a batch using IAA decompression batching. The patch also implements a zswap API that returns the status of this config variable. Signed-off-by: Kanchana P Sridhar --- include/linux/zswap.h | 8 ++++++++ mm/Kconfig | 13 +++++++++++++ mm/zswap.c | 12 ++++++++++++ 3 files changed, 33 insertions(+) diff --git a/include/linux/zswap.h b/include/linux/zswap.h index 328a1e09d502..294d13efbfb1 100644 --- a/include/linux/zswap.h +++ b/include/linux/zswap.h @@ -118,6 +118,9 @@ static inline void zswap_store_batch(struct swap_in_memory_cache_cb *simc) else __zswap_store_batch_single(simc); } + +bool zswap_load_batching_enabled(void); + unsigned long zswap_total_pages(void); bool zswap_store(struct folio *folio); bool zswap_load(struct folio *folio); @@ -145,6 +148,11 @@ static inline void zswap_store_batch(struct swap_in_memory_cache_cb *simc) { } +static inline bool zswap_load_batching_enabled(void) +{ + return false; +} + static inline bool zswap_store(struct folio *folio) { return false; diff --git a/mm/Kconfig b/mm/Kconfig index 26d1a5cee471..98e46a3cf0e3 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -137,6 +137,19 @@ config ZSWAP_STORE_BATCHING_ENABLED in the folio in hardware, thereby improving large folio compression throughput and reducing swapout latency. +config ZSWAP_LOAD_BATCHING_ENABLED + bool "Batching of zswap loads of 4K folios with Intel IAA" + depends on ZSWAP && CRYPTO_DEV_IAA_CRYPTO + default n + help + Enables zswap_load to swapin multiple 4K folios in batches of 8, + rather than a folio at a time, if the system has Intel IAA for hardware + acceleration of decompressions. swapin_readahead will be used to + prefetch a batch of folios to be swapped in along with the faulting + folio. If IAA is the zswap compressor, this will parallelize batch + decompression of upto 8 folios in hardware, thereby reducing swapin + and do_swap_page latency. + choice prompt "Default allocator" depends on ZSWAP diff --git a/mm/zswap.c b/mm/zswap.c index 68ce498ad000..fe7bc2a6672e 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -136,6 +136,13 @@ module_param_named(shrinker_enabled, zswap_shrinker_enabled, bool, 0644); static bool __zswap_store_batching_enabled = IS_ENABLED( CONFIG_ZSWAP_STORE_BATCHING_ENABLED); +/* + * Enable/disable batching of decompressions of multiple 4K folios, if + * the system has Intel IAA. + */ +static bool __zswap_load_batching_enabled = IS_ENABLED( + CONFIG_ZSWAP_LOAD_BATCHING_ENABLED); + bool zswap_is_enabled(void) { return zswap_enabled; @@ -246,6 +253,11 @@ __always_inline bool zswap_store_batching_enabled(void) return __zswap_store_batching_enabled; } +__always_inline bool zswap_load_batching_enabled(void) +{ + return __zswap_load_batching_enabled; +} + static void __zswap_store_batch_core( int node_id, struct folio **folios,