From patchwork Mon Sep 30 22:12:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70967CEB2C2 for ; Mon, 30 Sep 2024 22:12:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2081C280030; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C7EC28002D; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DED26280032; Mon, 30 Sep 2024 18:12:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B4B49280030 for ; Mon, 30 Sep 2024 18:12:27 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 710FD1C6343 for ; Mon, 30 Sep 2024 22:12:27 +0000 (UTC) X-FDA: 82622804334.05.A8D0001 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf21.hostedemail.com (Postfix) with ESMTP id 5E72B1C000D for ; Mon, 30 Sep 2024 22:12:25 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Q5xC+J2E; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734220; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gZv4LykG9QLKvMczgn2ESTNiDzzsQysM39q/qNvh7tc=; b=4ZvFEYjb4+kpHXSvCKzvbLq9flmxAdokIeO8YIFZvY2v2C6+49t0010ye+G+ieUSiGr+RV IPx+EW2ugyfMeqkO8jE55wIRm/z1Z5kljDRWe6hkGxNcWhv4GZZzXBPEIn/LcvA/WiZ/q5 ltZl9SPI/qdjsOfNAW/ZUr14KAxMjDs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734220; a=rsa-sha256; cv=none; b=cduEIHvf8mhsaEu/JEQppN1zXNcCg61FtNrnDoIbcvl/SNVvHkfDYm9ZxV1iodVI01o/3H h6K5EJhFGauPAiZpMGvDX5InxfFqK7qkwbxRH5RsKQxPfFpb/uSej5bShz00Zp8vC+hLsM +S0/TyOG040S0YnvB/FWVaA35hTnFAQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Q5xC+J2E; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734346; x=1759270346; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HXPnxaS0BG2kkNkoIrCoeh8TspMXuNCwKniqdIZtWWM=; b=Q5xC+J2E0rNC09FnSv85nvD7c4ZAha3EcvWBW31upbIt2+QspW39catZ ZPVtcqQErzyAUyrFdqYteN9Ru/+yBIJVc+sUbIxLaRHphUzcDdOBQdbyK WkU1sAlRRfUCUk7jlkj3imjR1jpkIe944/buSQdJvQC6GZieM4anxtg8E SmYqg+dhfa7G0metFBDhS8yXpesgABLeDz2GnOb7RN/vVN681yw2I4FLs ymO1d19jPab+xVsrRvGM7sprmGpZXsZDPIB0nkerdxBlnV07g3H1NGA0Q 75TRbX14XunejG+pGav12Zu9DCWgfR/rPdrpiVAReRnbibJBKcUxVIcgV A==; X-CSE-ConnectionGUID: 6D5QE2nvTLClYzi5ojpPeA== X-CSE-MsgGUID: Tb2gaWDgRAuFv1pQDerwnA== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368400" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368400" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:23 -0700 X-CSE-ConnectionGUID: W5E0QIJeR6SpPmgGvxsiuw== X-CSE-MsgGUID: myhOnlK6SsqNUZ7J0In4fQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985572" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:22 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 1/7] mm: Define obj_cgroup_get() if CONFIG_MEMCG is not defined. Date: Mon, 30 Sep 2024 15:12:15 -0700 Message-Id: <20240930221221.6981-2-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5E72B1C000D X-Stat-Signature: qf5b7cjeygu6h1x6rgq7s1ngrcpcfn4g X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727734345-972545 X-HE-Meta: U2FsdGVkX19scKtUhuozdwb7Z0OLyoU1w3BRlqFY/MA+J23pW9UBabrqwtJPOXNr57ZM5wENzRou/ppPwEubDt3SMsyebo0v0jMm8NrCXgASwszC2CkvcIrVUJpC14edDRy3B19Hcuf4e0V5QY8XJ6t/ximmVJbyK6qr7OoGSuXLgcq9OeoMcKACOYGw0U7Ea+NWJClP/ua1c18g2kfo36QrKedRn9iRhoGatXlP68MKcbiC1lmyGoPabJBtrhBGb7J9+jSASKu3vTgqaC185j0otY1Jqp+diaX3Zs8klHnnZNi8FbSiZyb8gVTuPCk3dzbi/DXggQ2q1KYefdXOMBK1Yyj6FYJzqpZtw/jltqtF8xtiD4dusuecI8REJ2c+1FUH9dRUEDiYXuPGWMTTz8pUjKd8R7s92kwyv+BQMUS9osifbkLEuch58lwpkLr8qrbVvlGbNSn30ZtE+6pSruoiLQoMiwJ9ZjaFmxWCr2gi9QTg5Btp2N3Co8kK8zgBO+JjUfm010wk6lCKljRpOFnnUXVhO2Xta3orhUXvSuQPs0mkhgIXoeqxjcUuneFzipw2/VbF0TGcJ0ewstuoazMFIAcnT5BJwHsbWvG2xZglHJIgyvss2B0KOdHpSkDMLnYd1Eq7xjN8ovZOyv2CLoNjfuQ5Az857ksqC9gjcnwiv2j7KDMQlKyUvlN+nw3adaVeDBPiMfUEXX2Oeb6aQ+VWCcnX1zfCNCbS/Ol5fYYjUHVeyjGIAtS//TwK/3oXXhU0NB30Vss2jOIEhM+0TO9TNmIjyvaAoul8LYU+mFq9J/vzCFGo7oXnKU+oohlEUeoukBjBDVyXRdJb9aCKmzLJxourhwcv4pFMDovyLXq3TIpVj/Yn7S0ULDBkxZEDA40P3QcWwabts/rzbiMf8P6Tc4pd93h7NjJlaoOSPmBh90ifLlCDEgVsBZupoNJSkkjZs8eCVvV+XtORylI QA/MzryI l1fihiKsA4k2G9dBOacj67D8Y6MdK9l2GdKwTHDIYrWSjNKBHxqxXE9RbLkhAkHKPT0Zip7U3JIpxOVwvTzW1h+8xKIm7twVo3T9lguyWJqQwacZYy8qxBFSaUCFCqtQyKlzX67pZ6jaDLJSvGD577terSYkdYFBm0/RUHymWZC7rwNBwPCakcix3fwsSUv+yelbFyZuZzfy2ijSJNQs0OehIAMtGrdQcliMs6eKmoq52v/7R4xjuKBj1JenAOnIO5/tsS4bfO54hMuMoVexJ4hxCA4nO8VVivnUdbkzLxvUPpEpApA7dGesS02c+1ndmYfx85H0KvWz081s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This resolves an issue with obj_cgroup_get() not being defined if CONFIG_MEMCG is not defined. Before this patch, we would see build errors if obj_cgroup_get() is called from code that is agnostic of CONFIG_MEMCG. The zswap_store() changes for large folios in subsequent commits will require the use of obj_cgroup_get() in zswap code that falls into this category. Signed-off-by: Kanchana P Sridhar Reviewed-by: Nhat Pham Reviewed-by: Yosry Ahmed Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner --- include/linux/memcontrol.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 34d2da05f2f1..15c2716f9aa3 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1282,6 +1282,10 @@ struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css) return NULL; } +static inline void obj_cgroup_get(struct obj_cgroup *objcg) +{ +} + static inline void obj_cgroup_put(struct obj_cgroup *objcg) { } From patchwork Mon Sep 30 22:12:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D40A8CEB2CB for ; Mon, 30 Sep 2024 22:12:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C42FF280032; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC9AF28002D; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1C00280032; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 856AB28002D for ; Mon, 30 Sep 2024 18:12:28 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3AE2F1C6343 for ; Mon, 30 Sep 2024 22:12:28 +0000 (UTC) X-FDA: 82622804376.19.AECB008 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf03.hostedemail.com (Postfix) with ESMTP id 16CB12000B for ; Mon, 30 Sep 2024 22:12:25 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Lv6I4vqV; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf03.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734243; a=rsa-sha256; cv=none; b=Azb56xE+oMHS+NMteYcOAl4WzkrqOpofa/olJ+2075zmSiNBpMiHUZht6XsJCZEepjCNP6 1YKoORQu68werQnHcGf7fJpKAccp5B8HggXyWwyj04obko8dtk3wAD+Hp6q5al9WacIXR7 V5mYluZlz9iHCc36wIgZL3QS2V/wgQQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Lv6I4vqV; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf03.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734243; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+SbmJ6A8PBY8eei7zu0r5gtPlPGA5g+RZGIYnfbGGqQ=; b=pnTBIVrN6porjinWU3F7+map6QadbZ+WcnmBfWz5PjstEXAicA3phJmxB++O622Q/gZoXO Tr01LhGOU9ppToSTLL6r2bqwncdVipqUcb1fYn3XtMQh3fHiaDaZmEg4bEQS72kQB/kxVP SGS5KPw51o6iyrUQsE+L9PTc7LM8jaY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734347; x=1759270347; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WnHMq8BNhME2EnXLEvjuvqDtcGOOpjFxIJHkgzLF8FI=; b=Lv6I4vqVkWVJ0BKl5JvVNj4i9moCaiCQ/JBQkKZRcxn91M/zwwQBvXEK fH4JWFMVA1fwgMJAJchptkk3K+rurEgvhqT99srtgqYJR0jY9v22NRGE9 bXo0PL6V3PjOXg3aTSNw8NVYv9Z5Ef0PgYBeASP8ofSbAWfsRKsi4yj9H qnExC2+4Mx51l2kgvfgDwJrqUk1OasGluQoHW0AnH7Jg6SJbtT/sZa0rc fSKDrH0at24M4hPXw7ITlu2GNlDW+7pGhZzTlBcSfchUeQnq2d7iYisPw CniuEdgNA+X/PbzFVKlkmrhiSvHQSJUACPnkn/USSfbUhp/ymn64Zzer8 g==; X-CSE-ConnectionGUID: pJNhU5abTq6RdqR22XuYkg== X-CSE-MsgGUID: sXNo5+t/QbWQvr3h8bjsiQ== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368406" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368406" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:23 -0700 X-CSE-ConnectionGUID: fR3kETJWS8C5+ue4fQbuyQ== X-CSE-MsgGUID: BlgiosatShO/iBvRb3rvGw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985575" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:22 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 2/7] mm: zswap: Modify zswap_compress() to accept a page instead of a folio. Date: Mon, 30 Sep 2024 15:12:16 -0700 Message-Id: <20240930221221.6981-3-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 16CB12000B X-Stat-Signature: iree4x844k1nziphrxw4t37qjqyqayo9 X-Rspam-User: X-HE-Tag: 1727734345-154295 X-HE-Meta: U2FsdGVkX1/Vj5aFtBVD5iKOC/BeS0g5MuhF7X874V0xRABPJQdCygHXpLRLUg4F2Zeyovdyx+a7D9AodEHASLaebMCOKauwFO+XmJu0DdqZ9AD+Q5Tp1DipeUXBePvS7b1CpZBMzI01IL45yjL8Ch/l9dIyLSrfsscxNI/d6htCs0xfm1Qiq0/7ei2zTl+bhxEQF0FkjbxRfrQoSPR8NyGZAjAXPETpd0T9H++HqZ/Ah/ymuYTGkJwqG5LLkxsGPTzLJuakraqVv6H0vMhhnhmypTNJzZ3o+PZn04eGVZi+3b/q5e3eK6TEDQS6jLb84Vd5Ao9K3o/PSqKzNA9z1I3Whsm8cIo9aAt4+r0gsU76hBJrZV2Uz54bIpShhmGwkPTCGz2XhNe3WuCeGUx2XhA8vU+KKKqYjO1qB1OoeA3q/J5ZgBUSMMgX/xWVGeE483guv0bwHtOkR0zou0ICKygkEFZ2Ysi8g9aGawTtLof0PBGzAosT0l/moJ10BL5vTi06kTDjk5spt28h05swLUN5mZsGECdW20Ksau/XKv7axT8cJcNYUa7Xjm8eWfhXe/nVcyWUyPRRlpfg7KCDfhN5gisJe8+H/fXqSRPOZWmBxDi1YexYVq9+ZKiFel8LUTb6aY1EDx312gBPJxsuo5hMdAk3w3odHJhyUFSXMsN0rWTyFV+IbOTadx06Azp+oZj0SYnh6kpjYe95fXYoQZhBNIYuUDYipJit1kjZoAa/8mVTUYOXjvzXLAPPyALMB/u7xyZYWR5GUNfqH3KJA+LPRXRGumwd8fZkDeWYkxT0C6DX81iLO5wjcA6duQepyviXKtLW/LM/xiZ8lDAFRQqqugkD2ZqqffBUrwzvcd5qTWRQOcFsniz/6P1ZSNWPSdp3D6N1+3XBYtHmVM0UDMRqgxbsstWfpWRWEUNhQv1DxK7ZqSFZkfKEECLpM/to7RkeY6rmHV+um/F0xUl w8zswxr1 /p6byABMbsLIKmwcCbW8Q6xpne6JZAKHfUTRuodxa/Dj9c6L8pzDBXKN0eyWmoTe28kUAo8JJ0m6TuteELDevMKIbI8YRNv3Sm9WPNF3hsqpLGnn9q7yEL2frAp6Tu2+QaAWMt3lSjlMoFy5crVFAklSPZ/xkW8WATLMA7h51UZRK3Jf6HUcjjJOcIncclELCPQtIAygb703TtGV/T1txbKOm0NQS6Co4gCYIO+Ag90+p0g3gDypym6TZIxnZIKZexNpm5Uy4t7bK8pdL3ntnKBTrXBhy5VjtWq2N6Ji3aIXcAJHTuvCDbiyabO19b4qpkS3BAt87A1sBm3KtBlm+XbuDGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For zswap_store() to be able to store a large folio by compressing it one page at a time, zswap_compress() needs to accept a page as input. This will allow us to iterate through each page in the folio in zswap_store(), compress it and store it in the zpool. Signed-off-by: Kanchana P Sridhar Reviewed-by: Nhat Pham Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner Acked-by: Yosry Ahmed --- mm/zswap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index efad4e941e44..fd7a8c14457a 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -875,7 +875,7 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +static bool zswap_compress(struct page *page, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; struct scatterlist input, output; @@ -893,7 +893,7 @@ static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) dst = acomp_ctx->buffer; sg_init_table(&input, 1); - sg_set_folio(&input, folio, PAGE_SIZE, 0); + sg_set_page(&input, page, PAGE_SIZE, 0); /* * We need PAGE_SIZE * 2 here since there maybe over-compression case, @@ -1456,7 +1456,7 @@ bool zswap_store(struct folio *folio) mem_cgroup_put(memcg); } - if (!zswap_compress(folio, entry)) + if (!zswap_compress(&folio->page, entry)) goto put_pool; entry->swpentry = swp; From patchwork Mon Sep 30 22:12:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B637CEB2C9 for ; Mon, 30 Sep 2024 22:12:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12381280033; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D85B280034; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8A35280033; Mon, 30 Sep 2024 18:12:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C617028002D for ; Mon, 30 Sep 2024 18:12:29 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7CC2AA0AEB for ; Mon, 30 Sep 2024 22:12:29 +0000 (UTC) X-FDA: 82622804418.19.0E27603 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf16.hostedemail.com (Postfix) with ESMTP id 4C86E18000F for ; Mon, 30 Sep 2024 22:12:27 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="ZXZtiPk/"; spf=pass (imf16.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734179; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5EdKqjOL7o3kOjKhQvu2azcBBD6RrwG0Yp5W/R+zxPE=; b=pgMXhRg+8vPJa6iLkGlXgI2grsrJ+6085a6NskMLF0vDLQj5ArexPsxOKdZmdiLB2B+qq1 xSfZDlv53utBNj2mtxddVRf2lrLOmx3PXPkweZIWtDd/SdhKTrVaU/Ghptuu9CnSsjegVh rI2QKzanfAr97df8PsjsliyMSvdikeg= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="ZXZtiPk/"; spf=pass (imf16.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734179; a=rsa-sha256; cv=none; b=tMGOuBLx37LEtGyX0XNS5oihTTtedTLC3oROuJ/p1FkTBE5YXXaAEj1aERVdglDiBrsvaX I6GYoJGBF6vpEZKHn3rZa08vuehyrCsTK+XKmbT35rbmptCaLEQm55A+aJ7MYM95J1vxm0 PMJimpEHdtYxuqUGOZbEEnBbAIlVxlw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734348; x=1759270348; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RXFigHVvqX66h0CgnEPMdbPUVdZNL13x+vDE/BJrhj8=; b=ZXZtiPk/2zQqpDrrylSaQKFTFpTX7L+YvAHp6QsP/Bt7A0voEWoRoqck 6XVgkn3OeLIgppJ1/rROqxFzbFiqzIr4VGVY+CfOnxai5RUWEvoHJkfal 9dRG2u+tq6OvRh/MpMgS8Tig+ZI8LoTs3G537rnF2yaJDmSrJCwvcRfzp FdyZW7fIu9nnJSTJGSUKtUGT79JORLQ3KYDSbMJiI0szrcd0jvFcNFOUu hGYsTvgB9P50mqHnjS4x3SJZVUquA4pswdIfmAV89jQAUQ86PiOUay3Q5 XUQQHoCQEYkr5ETMODSXA62ChmvRvusb4gZ72zX0d7Dwpk0WIZBI1U8mJ g==; X-CSE-ConnectionGUID: ZjbWNHL6R7Cm+/UddiWKIQ== X-CSE-MsgGUID: +V0TAZ7vRuCFZrLEAljKPw== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368417" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368417" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:24 -0700 X-CSE-ConnectionGUID: hJw5AqQZTyiDIxFOb3lmSA== X-CSE-MsgGUID: 6Tdm742SSzSKOCczqo9KXg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985579" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:23 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 3/7] mm: zswap: Rename zswap_pool_get() to zswap_pool_tryget(). Date: Mon, 30 Sep 2024 15:12:17 -0700 Message-Id: <20240930221221.6981-4-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4C86E18000F X-Stat-Signature: gahdefy8ax3q6qjf93kq8zqwm9kn18m9 X-Rspam-User: X-HE-Tag: 1727734347-250664 X-HE-Meta: U2FsdGVkX1+LxB3d+ppdWFeONhhwvAQX+QDGQFGSxTT5UPeTQMGyVJg9otzPdv19IULtPKq9JjPkY1eRfZrLGsHFI+JE1GcWMorhdq3c4ks+Pr0M43VvZW+NHaKANFMXst75WZRKV5l7DoFe+4kSmcfam0jaNLHy/ASUrIbgud06d3xGI2sGR0EvYGSYWDc3uQuH8/PJFuhCR27BQtiBssO8h/SZF/5dsfs6TN9Fto/+EZr1ZUKEhVM41/QbBmYpfs5q5ooz++hnwiNvGL6HfbQKhTkYhqrar7NhiEfnG/RjcOt8CpiEhyt95ubsO67ZjQjqJPZo4qkfzZQ4ZRMRqhGVF4vvjQCyvjjOypQTpPbA2+FSc1vMwA73AGRuKGR/fYAZdAHwU7Mq60bLkitqSH/LV7Ao1q/BAmDmzAmVE/DgFErwXhRK8BU6Vvs8PJYG1Ttbz5fXUMoT3PJ3GcXQpPIhfV2k+LlLQ7zpKR9h5x2YokOx3Sifc3PdiAew/QWnvLvLKLATwcNSb95sNiNDQ9MHIN3idDnQHkmbJa89VxekaoPvQwFo4buSV6QVoibGpn0zBE7obOkziIr/rOVnStodLYEyGTyAOXyY0QYsECPv0LYNkElN95xfaYvpkBIkE1WcoBw/dEoih5FrJ1/alvNGBahs+MZ/cgnost+fMu5ySqzwG3BMhmTfCyW2C/8d/8RMcndR21xYS7ZbsB7EVarjT9W9RjYYXUMA/jJ3ckQTl/JurDvZEJQP1+cReFBEk1TSlJW5+3dZ6LMwHVaR1VfAcSWpTtW3LFFnlJWF03CX32fyV4jVWFk8MvML145nTu32PKeV6V4H4JhGWf8nyz04SRJmch8Mn7q7TGYzTfe6rO+fpd0ifhZ7W1GtLiKwwAoqtGyAwHoA6z5+uCA1RRDOKUozlXbxn89bNS0rIWKF83kpqVG3ivAspA35/FKtBWbzwOLU/ndWkzD7OGq Pn5ffjFp 7oe8t7GeZ6DYAYGqdGotMBtu9vmah8qTngoJT8tbkobVNcGHqy7agRfV7bOcJSGtg3I985xzuEKGlgHWIpGF/g57uv4pKprb/LgdBUhgO7Xeeo1ZCr6Wb1/ielEUfsVIFnDU02+SusCStiQCRwNS8bOYo0S4zLo1TN2UalSZo19zQaRJxN816A1JBZMS5ZTe1W1vXSVFlfPCOmAOgWwFtHX+Stlb78p2XE3fAgfdmJQWx4bvnu9MHxLw3Oxh0Nq3OE4BCMmu3LMi2Qlupch3NKgwSTbnQLe0DWKPVc2XfZqjGnN+GVqzou/Bmr7mH8ByRXxLbBhuIZGlJOdc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Modify the name of the existing zswap_pool_get() to zswap_pool_tryget() to be representative of the call it makes to percpu_ref_tryget(). A subsequent patch will introduce a new zswap_pool_get() that calls percpu_ref_get(). The intent behind this change is for higher level zswap API such as zswap_store() to call zswap_pool_tryget() to check upfront if the pool's refcount is "0" (which means it could be getting destroyed) and to handle this as an error condition. zswap_store() would proceed only if zswap_pool_tryget() returns success, and any additional pool refcounts that need to be obtained for compressing sub-pages in a large folio could simply call zswap_pool_get(). Signed-off-by: Kanchana P Sridhar Acked-by: Yosry Ahmed Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index fd7a8c14457a..0f281e50a034 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -403,7 +403,7 @@ static void __zswap_pool_empty(struct percpu_ref *ref) spin_unlock_bh(&zswap_pools_lock); } -static int __must_check zswap_pool_get(struct zswap_pool *pool) +static int __must_check zswap_pool_tryget(struct zswap_pool *pool) { if (!pool) return 0; @@ -441,7 +441,7 @@ static struct zswap_pool *zswap_pool_current_get(void) rcu_read_lock(); pool = __zswap_pool_current(); - if (!zswap_pool_get(pool)) + if (!zswap_pool_tryget(pool)) pool = NULL; rcu_read_unlock(); @@ -462,7 +462,7 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) if (strcmp(zpool_get_type(pool->zpool), type)) continue; /* if we can't get it, it's about to be destroyed */ - if (!zswap_pool_get(pool)) + if (!zswap_pool_tryget(pool)) continue; return pool; } From patchwork Mon Sep 30 22:12:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09204CEB2CA for ; Mon, 30 Sep 2024 22:12:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64E7E280034; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D63E28002D; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 450F8280034; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 07A4228002D for ; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AA2D48071D for ; Mon, 30 Sep 2024 22:12:29 +0000 (UTC) X-FDA: 82622804418.09.C676615 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf21.hostedemail.com (Postfix) with ESMTP id 8BD4F1C0007 for ; Mon, 30 Sep 2024 22:12:27 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=h1G+C+gV; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4lJlNC7s/SkC12bTYrWldJ9+mlN3EU9TgO7LswTLiGA=; b=sjxKJX+dTnJ938T6DOMNWnNs3K8xks3IOJptlatO6Un8jh2CRSdA3VuO0Q+0pfuRSaYw43 vK+gKT3o7WHKdzda2tZmYtCXaaYFBZUochBO5PPJ1on1ZDrlhgZfMMaZX3BQ/U2kmiudSM ZkRDdS6aBNY9zySPWJ/JE9UBUM41KCM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734222; a=rsa-sha256; cv=none; b=T+uMey+QDrr6VP5+OL5/CdlJyWkv2pglbfZw7bzwRWu28KmIbxN9CO6xXgIQGGH9j7hYcY 2PODo+HbMapploYDNqNXv6mQExDKHvcGZzsdVwfUHxZv/rvaHPdcIoshszmuw7Na/HX0tK dMrj1DDnfrlHJPGf7XnH08NJow59CBk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=h1G+C+gV; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734348; x=1759270348; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sH8cQ3w+IHRFjDbPI6y+/ZvrGGUAI7tDyoePLrfKV9c=; b=h1G+C+gVod1avA5DC368myABjj2cfpxAs0yqymR4Y8AWdVCN3d1cfTDt pydCUkUpFQHuBjU3yQgyJdZftFpvQ2cNbYBwwh8KXOYUt7L7MEx8cfDIY kNn+xciSgdm3HOi5IiIAE9JzNoG0oUlz62FQLy0952ZuWUvHYLdSq+Lfz +d0o0jOUHkTS27xir9+TluwP98mdv/Gf9N97R9eiM76/k+6sXJuwW3gn6 2s/CjiZkrf4/VO0N5mlWLAJJgVQIHNiP3A2fMdj4GUJ6DZxWBweeQvAXC gCATrK2UmzcTTZ0z3MB+oANHspxK+iyPnRn52Gr47TZoB+XuBH9qA9QDd g==; X-CSE-ConnectionGUID: svzk8ibKTZ2zTPn7a6XhAg== X-CSE-MsgGUID: jM8jh9WQRwmqAp27Ihe1tQ== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368428" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368428" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:24 -0700 X-CSE-ConnectionGUID: JiNA1XIES2mHOMYswCZ+/w== X-CSE-MsgGUID: RS1ODkyVTHartxBgNoU0aQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985586" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:23 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 4/7] mm: Change count_objcg_event() to count_objcg_events() for batch event updates. Date: Mon, 30 Sep 2024 15:12:18 -0700 Message-Id: <20240930221221.6981-5-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8BD4F1C0007 X-Stat-Signature: xzi1toqtiw8dim9q9btfc438bafxnpr3 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727734347-267439 X-HE-Meta: U2FsdGVkX1/KocpU3e6T7dW0Ru3I+3xPfsMrLMMZ7vjLv40pNVJWBTZfOuupGHikfUSGPQ1eZdBq0d4z11TJUFLg1UCfTGbC32jpQJ5bSabd5GDdcQ65EXTeqeweeTq3BoISQfNiD9PAendI3j9vabR9aPLKt03hL1BTGa5AHeZPGpE9fNIu23Se4mBaMaHVyWZA25Ce+f2/b3H9SFHAk/k0ixKZs2geyXJ94zGIS6oYQkzxcPP/pq52IJAgShdc5Sx/DeEqHWKbDvoKUYhtdxQDPoOfl/H5q6RgPXbIUIrWpIkmhtkgHiGMeW/A7+ywDY8QPYmuUdVzBX3vn2Ip6xq3jEf/z32sUmyk5EDpAWFLpz+7JUnObuyYh7ZACyV7YPkUQXIKXGtzFVL5E+gAsGFmPWMsK/1sHGR7t7fVmNkRbHBAZj1tZfZDKey/fSFftBjKNG2xqHwDMOpJiqf2Vbr359pMHYzGjoC/1b/L8fIxfF+gNuOkmfZmb02bDi9surl3eGu/9nzyqjraJKYCRH4cTIo4zdDAK5kg4CQow3v0Tp7blVahHHux2uSbF2mOeAGYYbcT326uvngDOI6UJOlNBnabtr5XVUgNU06In/NW493sJ+KUBzcOwvIySG0zzt/F+5UEnES1TH2pmlxoXiGldq2HgUpDgfZiKQ0Nd3FDkseKmj9kMXTHoCiwH/eZ8+alyM8Z6BFnXESPbnDFgzgzJVZlL9GROMz9bau2381cP1MOw/OzLLHZK0TdTmsODqxkfNWw7NOyMN1dRqvK62S8UmGQ3G8hHqSAYMwQ3QSzmhAh6XRKbRj0m4NpM4WGJ0iXTIpbVwJo9omuUguzEaXp+5ayBR5UJaeTe4XTIWqbPk34GolbADrBZ+mcpSwuwimQ/epy0bjXvL+SHxGcyivsWxoNngYwYT9XZ+Q1+TVtrO6yuAjSzirYGZEmtjlMFUeA2DWpzS/Ozey/FGT nq5h9yAO Nsj/2M13S5Osl9tim0cvx5ngwrdlM85mH76GQFnVjI0MBVn3MvpdQQg3Uv9lnGOtAsYSmCkzuiy2//83cDXaOOJWiHj/uJJCamL08QD497SeuAChDBuyV9XZSl8UFmuC8u91irZSrl9Xz4t/0MzkiMMVUlQEcVRLBUY2UylVU1D4Trlkx+xqOChw68Em6YBfO1EB64VySqqSmIIMd0irO9HSQa9p0Jn8prnJ2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With the introduction of zswap_store() swapping out large folios, we need to efficiently update the objcg's memcg events once per successfully stored folio. For instance, the 'ZSWPOUT' event needs to be incremented by folio_nr_pages(). To facilitate this, the existing count_objcg_event() API is modified to be count_objcg_events() that additionally accepts a count parameter. The only existing calls to count_objcg_event() are in zswap.c - these have been modified to call count_objcg_events() with a count of 1. Signed-off-by: Kanchana P Sridhar Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham Acked-by: Johannes Weiner --- include/linux/memcontrol.h | 12 +++++++----- mm/zswap.c | 6 +++--- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 15c2716f9aa3..524006313b0d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1764,8 +1764,9 @@ static inline int memcg_kmem_id(struct mem_cgroup *memcg) struct mem_cgroup *mem_cgroup_from_slab_obj(void *p); -static inline void count_objcg_event(struct obj_cgroup *objcg, - enum vm_event_item idx) +static inline void count_objcg_events(struct obj_cgroup *objcg, + enum vm_event_item idx, + unsigned long count) { struct mem_cgroup *memcg; @@ -1774,7 +1775,7 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, rcu_read_lock(); memcg = obj_cgroup_memcg(objcg); - count_memcg_events(memcg, idx, 1); + count_memcg_events(memcg, idx, count); rcu_read_unlock(); } @@ -1829,8 +1830,9 @@ static inline struct mem_cgroup *mem_cgroup_from_slab_obj(void *p) return NULL; } -static inline void count_objcg_event(struct obj_cgroup *objcg, - enum vm_event_item idx) +static inline void count_objcg_events(struct obj_cgroup *objcg, + enum vm_event_item idx, + unsigned long count) { } diff --git a/mm/zswap.c b/mm/zswap.c index 0f281e50a034..69b9c025fd47 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1053,7 +1053,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, count_vm_event(ZSWPWB); if (entry->objcg) - count_objcg_event(entry->objcg, ZSWPWB); + count_objcg_events(entry->objcg, ZSWPWB, 1); zswap_entry_free(entry); @@ -1482,7 +1482,7 @@ bool zswap_store(struct folio *folio) if (objcg) { obj_cgroup_charge_zswap(objcg, entry->length); - count_objcg_event(objcg, ZSWPOUT); + count_objcg_events(objcg, ZSWPOUT, 1); } /* @@ -1576,7 +1576,7 @@ bool zswap_load(struct folio *folio) count_vm_event(ZSWPIN); if (entry->objcg) - count_objcg_event(entry->objcg, ZSWPIN); + count_objcg_events(entry->objcg, ZSWPIN, 1); if (swapcache) { zswap_entry_free(entry); From patchwork Mon Sep 30 22:12:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9681DCEB2C9 for ; Mon, 30 Sep 2024 22:12:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F666280035; Mon, 30 Sep 2024 18:12:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A83928002D; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1482280035; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C19D128002D for ; Mon, 30 Sep 2024 18:12:30 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 701781204CF for ; Mon, 30 Sep 2024 22:12:30 +0000 (UTC) X-FDA: 82622804460.12.B94345E Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf03.hostedemail.com (Postfix) with ESMTP id 52CDD20005 for ; Mon, 30 Sep 2024 22:12:28 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LvfGn5v+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf03.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734245; a=rsa-sha256; cv=none; b=CCAOC+2ceUdmg6RwqSjtLri4mIyVNCMbI1CVi9mDpLBvca8Uwr3R3YaiudgOckVsChMMAr Vb07iWCWG+pl1EC3YrNDR/M5yvKXVU5PNYIicoumFLvztAD/sSu6lfb7mTmQGOAAenCexg rBvacJylM1TCRkwOqsSQBDeLuogyIdQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LvfGn5v+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf03.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734245; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n3lpT5wzGF+b1pa3hoTUzlSMa9Vzwo4wZdujvqkC3ms=; b=ulxs+UzwYpPGynM55binXNlr5BdX0ukH/NBURg8U/ljq9ZOezNa+Zg86fmLcqAEzJeBVHC TCGp9VOOP7gW/XzVvo5/k6wBcExDc/sD6kq7B+Jx01eFMwqw2YSIpYLq2ZpnryFEkqnRLz eGteBhQTH83SfB2x+UIbDSAmYzcWFSI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734349; x=1759270349; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fh13koWj73Qksnj2l+56lXO4bpj9lkhBiLj2Y16CgXI=; b=LvfGn5v+eu0POTua5fGZb8EsIiR4WyNRtrUHtaXvXhcHxDN/OojTZk2d 1Nqzu+GQ3J/N4Ht0MLkHU6a4iFAlpvQSaFtGlt5Q1N9Ws0lDZ6neSRXUk 5aDqM5491tO/Ax7xh42ULFKz0AzEAypQNKJ+J6Sp8IlZxeVBsXMvLg7Cz kpjugdR23SFE/C2QAr6Yzn6DFg/tRGpQYAKR+3+QU5/oUrRPWFbtLDTTg qHaLyyUcNHNBMsW6hejvpRgVONfrLiQmznnFTbsqWn6FrOZ57X+R8eKUB FDY3+tS/JyNPVfSRw6L8ttPytgSIDzFV+KZ1k13hVu8s2PHQTTPYMLuIb A==; X-CSE-ConnectionGUID: 6Lwptl01Sl2Gz4X3hg1LgQ== X-CSE-MsgGUID: HvB9vmV1QQSWcq11Z0+zBw== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368437" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368437" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:25 -0700 X-CSE-ConnectionGUID: vRKOb2DBRnOhm6ekko/j6Q== X-CSE-MsgGUID: QaKdL+cPSRu5NCz+D5QRRQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985589" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:24 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 5/7] mm: zswap: Modify zswap_stored_pages to be atomic_long_t. Date: Mon, 30 Sep 2024 15:12:19 -0700 Message-Id: <20240930221221.6981-6-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 52CDD20005 X-Stat-Signature: qr3gy89qhb5gszx6c6rwnk8y4bsodu5r X-Rspam-User: X-HE-Tag: 1727734348-367602 X-HE-Meta: U2FsdGVkX19Wz54EfBEwixeMdicB18jY3Wn9FdZTkkMlbFMABL+TIywuVgetKGO4kHhu7fDDWmz0xjVLDkVnO9gwdyCupixmcKsn+5GStggPJsgoy1rTys6gg5olB9yFwp3z6E0L/QZSLIZ6w8T+ErCJfPjWFT515lKnsJcC88oJRQfV6Ee6EJJ20w1nMz2P49zMMyWC+aWIljMPTI+HsSQ4d5e/lCyqS9z9AWeHec78LDpctdL0AZ9mGAu3AzuJvW7vVe54TGEc4H7PHZMPqDk08Beex/9gUYepu575f1Yt3z6OZwN80ZPIWryVw+/13WNNUdqaxPHD7G+L8eiTUKr1xsmRekBgHxwxolQvrbYLt06w89FfAXNWDc8XtMGuDVIm3wtZ7nHk+qn1bjmRuXLFXJH9fKWWsSFhX8R1ggu7HDdJOe/nU9YCqZsnyIgWDTpqSSzgqAl6gh/cdnPUiQ0m9xYNbvkvpn+ZBeUwEqcDNCOqgCX3T0LTG0jXtGnOX/PvYZ9rKfMS8AulnPKCTHdD+DB9niD+gHXSJ39Qg8FAbjuX3b0XAsSDPLEWnulF5WVDE4fioCSj59x9zsQhT54HKHeL/ts+a72GO3u38UrArH8bvYQQqQ99WSXcwM/ztEdd7T8KXxTfsx4RE7Lj31QGOvQPDlKqDIwDDCEjcHPfKWFxpyLqM7guYrt83IiRf7Aust/FzPheLJ1fJh9rFSmr8dnqR70hMa2DqZ9gDsgfLUmYAROfWQuJDT3NElWyT1NgIH19vMWKPFxdQivYtdiKr0zcfXmzxjhzaoqA9zIH6Bs7dPPc9OPJ2vI8plqgxNgX7M/ycWXI3V33VTpp11CmyeWULkzoQBjED4MTFdhgUrcQCGDvUXDKlNoD+rgbD88mI/MxHsY6mowPhbMalLQwQP42NzsU1D/h7RRgTQEh1La2CYF8TR0yWhxvW7FN461n3TPlNM780TGS/k7 Q0UXErMN 9MlbzSS/CzjK+Nn0uOeK42WOKGrzldLSGd8CfkpqKoUwR0/yo+S9VOimidertWmwRbFgA5ZDUP4frGG0yfhLuE298I33oR9S77lgH5MPWGaflLO48+0vp4uWicpKxIfM9+fUB1HJFxZQ06knPbbHccusRR2T4mIUWIZbwGTbDI0vnhK3BB3g3wXkbGPCWoM3Lb5+RE12ODBpt4Osy1yv5Wyi3ARDZumjudSya/KouZ+tehZvR2uMZ9S5HhqYQ9b+sk3/MsOKbdTXN17k+bDFtzsbrnHCU55YbNZ53oVRsluTvSbvFKLzoGIWm83Ov8l7UeDJ1FAlOwldGd9A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For zswap_store() to support large folios, we need to be able to do a batch update of zswap_stored_pages upon successful store of all pages in the folio. For this, we need to add folio_nr_pages(), which returns a long, to zswap_stored_pages. Signed-off-by: Kanchana P Sridhar Acked-by: Yosry Ahmed Acked-by: Johannes Weiner Reviewed-by: Nhat Pham --- fs/proc/meminfo.c | 2 +- include/linux/zswap.h | 2 +- mm/zswap.c | 19 +++++++++++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 245171d9164b..8ba9b1472390 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -91,7 +91,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_ZSWAP show_val_kb(m, "Zswap: ", zswap_total_pages()); seq_printf(m, "Zswapped: %8lu kB\n", - (unsigned long)atomic_read(&zswap_stored_pages) << + (unsigned long)atomic_long_read(&zswap_stored_pages) << (PAGE_SHIFT - 10)); #endif show_val_kb(m, "Dirty: ", diff --git a/include/linux/zswap.h b/include/linux/zswap.h index 9cd1beef0654..d961ead91bf1 100644 --- a/include/linux/zswap.h +++ b/include/linux/zswap.h @@ -7,7 +7,7 @@ struct lruvec; -extern atomic_t zswap_stored_pages; +extern atomic_long_t zswap_stored_pages; #ifdef CONFIG_ZSWAP diff --git a/mm/zswap.c b/mm/zswap.c index 69b9c025fd47..2b8da50f6322 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -43,7 +43,7 @@ * statistics **********************************/ /* The number of compressed pages currently stored in zswap */ -atomic_t zswap_stored_pages = ATOMIC_INIT(0); +atomic_long_t zswap_stored_pages = ATOMIC_INIT(0); /* * The statistics below are not protected from concurrent access for @@ -802,7 +802,7 @@ static void zswap_entry_free(struct zswap_entry *entry) obj_cgroup_put(entry->objcg); } zswap_entry_cache_free(entry); - atomic_dec(&zswap_stored_pages); + atomic_long_dec(&zswap_stored_pages); } /********************************* @@ -1232,7 +1232,7 @@ static unsigned long zswap_shrinker_count(struct shrinker *shrinker, nr_stored = memcg_page_state(memcg, MEMCG_ZSWAPPED); } else { nr_backing = zswap_total_pages(); - nr_stored = atomic_read(&zswap_stored_pages); + nr_stored = atomic_long_read(&zswap_stored_pages); } if (!nr_stored) @@ -1501,7 +1501,7 @@ bool zswap_store(struct folio *folio) } /* update stats */ - atomic_inc(&zswap_stored_pages); + atomic_long_inc(&zswap_stored_pages); count_vm_event(ZSWPOUT); return true; @@ -1650,6 +1650,13 @@ static int debugfs_get_total_size(void *data, u64 *val) } DEFINE_DEBUGFS_ATTRIBUTE(total_size_fops, debugfs_get_total_size, NULL, "%llu\n"); +static int debugfs_get_stored_pages(void *data, u64 *val) +{ + *val = atomic_long_read(&zswap_stored_pages); + return 0; +} +DEFINE_DEBUGFS_ATTRIBUTE(stored_pages_fops, debugfs_get_stored_pages, NULL, "%llu\n"); + static int zswap_debugfs_init(void) { if (!debugfs_initialized()) @@ -1673,8 +1680,8 @@ static int zswap_debugfs_init(void) zswap_debugfs_root, &zswap_written_back_pages); debugfs_create_file("pool_total_size", 0444, zswap_debugfs_root, NULL, &total_size_fops); - debugfs_create_atomic_t("stored_pages", 0444, - zswap_debugfs_root, &zswap_stored_pages); + debugfs_create_file("stored_pages", 0444, + zswap_debugfs_root, NULL, &stored_pages_fops); return 0; } From patchwork Mon Sep 30 22:12:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A55CCEB2C2 for ; Mon, 30 Sep 2024 22:12:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D89F280037; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 58A46280036; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F230280037; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0FCA728002D for ; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B5374120BA2 for ; Mon, 30 Sep 2024 22:12:31 +0000 (UTC) X-FDA: 82622804502.22.88C824F Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf16.hostedemail.com (Postfix) with ESMTP id 94501180007 for ; Mon, 30 Sep 2024 22:12:29 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Z+BvAHDl; spf=pass (imf16.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734182; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YsKRmQoYoQzefhoLcurdjhvgw705R9YFJDO0bonL/ps=; b=IQT8aK8Z2KTLf9iSgLeeunrNbGgg+4kg74qGRie4nG4J7iPwz0T2vk22oYu5AiipfimWoc QTqG+rimpmdMF9BLG2ASJiN1HZr8TOd/WklloaCdCf80NwlQf7JcK/EDtuiXxZ95GFbCOX MVCoFroRGuoRqTbiPKpNY/2MlDdYjps= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Z+BvAHDl; spf=pass (imf16.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734182; a=rsa-sha256; cv=none; b=ZSfi5mYHehkGfWTGhNFG48sz8heRdC5OjQy31TgdVaCNAiAP3aHL9qWZA4uiwJIXP5kFhw QgaKcm6GRgPr8EP0UAieCv6ylmbFYF3wzz9X8g8KNYCa/698TER73WRiLJf3TjtXMGPN19 WWnXWXedtLitRTP4KPmq3YT4UzdcIN8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734350; x=1759270350; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rGYXZbQGx8nuXhyND0p/8h/6lHtYJuiMzkYh1FrNrkU=; b=Z+BvAHDl21z94tvXiJ4anmrAUrdUHGMU8OXCsbe5EZ/xNbZxj9AnNOjA brIl8pk6DddOIX3fNZ2wc1j1gawM4fvhB8xeAmJ0hiYuwrAhP8GdkX3a2 EL9N1PhnN5I+2BlAxa+ecQb52Uz6ukXCSISGY3nDuKx4tB4mu0KuGflPc ctJO0vLw+tuYGvECymkRzUuDDbePTFx2ZIyhDtmLnqR1Z7P1KU9yZBHMu 7eiZy7eT8PE5dvkZG0ZjzFnceT8m48ZgC0/vgYv+8SMfaghkFON8ZDqvv yntOUrnL3i/nGeNb0KKT82T8ILfTS02+lnRVV1iRWOqHcrtY1ncK8rBO9 g==; X-CSE-ConnectionGUID: rYB8Fw55QCW4g+2XFVY31w== X-CSE-MsgGUID: nMWoJfxSRxKKghp5Fd8ddw== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368446" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368446" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:25 -0700 X-CSE-ConnectionGUID: 6OCfoR3wTpiVDGXWLJhJ7w== X-CSE-MsgGUID: QqMGlGaTSySq4bW+2bm9jg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985592" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:24 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 6/7] mm: zswap: Support large folios in zswap_store(). Date: Mon, 30 Sep 2024 15:12:20 -0700 Message-Id: <20240930221221.6981-7-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 94501180007 X-Stat-Signature: ttcts4oztccedxb1gysech9kgmh4tizo X-Rspam-User: X-HE-Tag: 1727734349-591339 X-HE-Meta: U2FsdGVkX1/pmzn7n2xH49dD/RBi13y+qe7OdwmwrDGpZVJVJsjni9vGNeW+kG6HardKHvsL6D5HslVcsQS62hUVz4qYFcDUX85E3A2n288u92uBiUK8VUMaS4EU61rl+REqJq5ulMFW6nDN/k4njhd7R2q2aykkT6eEAhVs3m0E8c+3WdnaDiLMn9dUeJTtM4F5ZaOLXwMhmOzmUUTJKBFyP/Zjsuw+X3XQR+f5AhEYhndxAbjvUusYGoL5TIjmBflB+UMAC/MsLzi17MPaxT0dLDQHvTdiKAiSwaM03/ViC67QyeGKktZwF8/x0+6Ikh3dJAf4TcGnH8PmvV4Hpfn4YQjQpAsj3gJkJXHj6E5IwR5AOtRe3seTLM7isCrBl7AhWqU8D7dt497HGJeC9vXTrP1QJUW0LUOqPgp7chK7R1L/Cg1loYUIlJx3DNwcxkVipbbDti3TwZhFb4bv+F4QIwQUOu+9HDaQtvzreSRi0UVkjiwCXjx8HgYwvvWg4rWnuLmGNoO3Nip/tM0VMDajDl8RC6LbthaXcJigxDBVrPYvNw6DbVJuwChvezQ8pfjgoJCgEWYtSBoc2IDf9ZU5fsI7iiOzK7gzqU4tHneNDHE7d3Nz06GL6mfCpjizJHwCZihuqXTh+JExLIJUslxwLRD9xz9p4HdlWdbONANZfqK8eDZPqJHdFXt9nqh3rlPtz6RoLrFe5DHYpJzulLlKjQpOOgIjVBJ+NfCQLsNzw/EBA0iRvDA9ArT4zHEe/3ReZTgGFcu+lysepZqJqON/E5mB/1uIibSapNZQZiG43UvjZwLJ9RyhWspeeH+Lp7R7fMj8SqNT0ngD2Bdd/zlt2zmADgSv/EohsjHiYPXHC5XDXwLIsaRFq38i6Sq1s8iwA8xU+KK/U78RznEK5Ep/2OvMD+kytIPXuG7sxXAhaOWQq4hRR2WIpkaVigWpw12UcmGO4Nr2QL01BGc h8uQRxTD PLXmcOrDRsFBFVZ+hftfjr1TAx6+Th4Xpp8j0w4mCWwQQk/sHFDQKi4KA1g/nVOPJS6QKBKTI5YJUfrFpc7oCqIszGo0Lr1WHPQOXtmS6P90l86gGY7qzvS0U8UvfYELhs683Huw/pAsDS2/W8xIc7aPyJcQRDjJ8UvRWSzBMVX3hLuKA6QeWyUSWUTl5FDHksJAHSKZmhr7KLpjIpl7aYls/O5ACzX4JOvIaV9piA9qC08Aue1m22oP+pWhXSkPegcvyZl4wSJC2Se+YLJAHzntAcrsagsSe+4zLzKjhw/fteHBhWEZHvaS0YA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: zswap_store() will store large folios by compressing them page by page. This patch provides a sequential implementation of storing a large folio in zswap_store() by iterating through each page in the folio to compress and store it in the zswap zpool. zswap_store() calls the newly added zswap_store_page() function for each page in the folio. zswap_store_page() handles compressing and storing each page. We check the global and per-cgroup limits once at the beginning of zswap_store(), and only check that the limit is not reached yet. This is racy and inaccurate, but it should be sufficient for now. We also obtain initial references to the relevant objcg and pool to guarantee that subsequent references can be acquired by zswap_store_page(). A new function zswap_pool_get() is added to facilitate this. If these one-time checks pass, we compress the pages of the folio, while maintaining a running count of compressed bytes for all the folio's pages. If all pages are successfully compressed and stored, we do the cgroup zswap charging with the total compressed bytes, and batch update the zswap_stored_pages atomic/zswpout event stats with folio_nr_pages() once, before returning from zswap_store(). If an error is encountered during the store of any page in the folio, all pages in that folio currently stored in zswap will be invalidated. Thus, a folio is either entirely stored in zswap, or entirely not stored in zswap. The most important value provided by this patch is it enables swapping out large folios to zswap without splitting them. Furthermore, it batches some operations while doing so (cgroup charging, stats updates). This patch also forms the basis for building compress batching of pages in a large folio in zswap_store() by compressing up to say, 8 pages of the folio in parallel in hardware using the Intel In-Memory Analytics Accelerator (Intel IAA). This change reuses and adapts the functionality in Ryan Roberts' RFC patch [1]: "[RFC,v1] mm: zswap: Store large folios without splitting" [1] https://lore.kernel.org/linux-mm/20231019110543.3284654-1-ryan.roberts@arm.com/T/#u Also, addressed some of the RFC comments from the discussion in [1]. Co-developed-by: Ryan Roberts Signed-off-by: Signed-off-by: Kanchana P Sridhar Reviewed-by: Nhat Pham --- mm/zswap.c | 220 +++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 153 insertions(+), 67 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 2b8da50f6322..b74c8de99646 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -411,6 +411,12 @@ static int __must_check zswap_pool_tryget(struct zswap_pool *pool) return percpu_ref_tryget(&pool->ref); } +/* The caller must already have a reference. */ +static void zswap_pool_get(struct zswap_pool *pool) +{ + percpu_ref_get(&pool->ref); +} + static void zswap_pool_put(struct zswap_pool *pool) { percpu_ref_put(&pool->ref); @@ -1402,68 +1408,52 @@ static void shrink_worker(struct work_struct *w) /********************************* * main API **********************************/ -bool zswap_store(struct folio *folio) + +/* + * Stores the page at specified "index" in a folio. + * + * @page: The page to store in zswap. + * @objcg: The folio's objcg. Caller has a reference. + * @pool: The zswap_pool to store the compressed data for the page. + * The caller should have obtained a reference to a valid + * zswap_pool by calling zswap_pool_tryget(), to pass as this + * argument. + * @tree: The xarray for the @page's folio's swap. + * @compressed_bytes: The compressed entry->length value is added + * to this, so that the caller can get the total + * compressed lengths of all sub-pages in a folio. + */ +static bool zswap_store_page(struct page *page, + struct obj_cgroup *objcg, + struct zswap_pool *pool, + struct xarray *tree, + size_t *compressed_bytes) { - swp_entry_t swp = folio->swap; - pgoff_t offset = swp_offset(swp); - struct xarray *tree = swap_zswap_tree(swp); struct zswap_entry *entry, *old; - struct obj_cgroup *objcg = NULL; - struct mem_cgroup *memcg = NULL; - - VM_WARN_ON_ONCE(!folio_test_locked(folio)); - VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); - - /* Large folios aren't supported */ - if (folio_test_large(folio)) - return false; - - if (!zswap_enabled) - goto check_old; - - /* Check cgroup limits */ - objcg = get_obj_cgroup_from_folio(folio); - if (objcg && !obj_cgroup_may_zswap(objcg)) { - memcg = get_mem_cgroup_from_objcg(objcg); - if (shrink_memcg(memcg)) { - mem_cgroup_put(memcg); - goto reject; - } - mem_cgroup_put(memcg); - } - - if (zswap_check_limits()) - goto reject; /* allocate entry */ - entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio)); + entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(page_folio(page))); if (!entry) { zswap_reject_kmemcache_fail++; goto reject; } - /* if entry is successfully added, it keeps the reference */ - entry->pool = zswap_pool_current_get(); - if (!entry->pool) - goto freepage; + /* zswap_store() already holds a ref on 'objcg' and 'pool' */ + if (objcg) + obj_cgroup_get(objcg); + zswap_pool_get(pool); - if (objcg) { - memcg = get_mem_cgroup_from_objcg(objcg); - if (memcg_list_lru_alloc(memcg, &zswap_list_lru, GFP_KERNEL)) { - mem_cgroup_put(memcg); - goto put_pool; - } - mem_cgroup_put(memcg); - } + /* if entry is successfully added, it keeps the reference */ + entry->pool = pool; - if (!zswap_compress(&folio->page, entry)) - goto put_pool; + if (!zswap_compress(page, entry)) + goto put_pool_objcg; - entry->swpentry = swp; + entry->swpentry = page_swap_entry(page); entry->objcg = objcg; entry->referenced = true; - old = xa_store(tree, offset, entry, GFP_KERNEL); + old = xa_store(tree, swp_offset(entry->swpentry), entry, GFP_KERNEL); if (xa_is_err(old)) { int err = xa_err(old); @@ -1480,11 +1470,6 @@ bool zswap_store(struct folio *folio) if (old) zswap_entry_free(old); - if (objcg) { - obj_cgroup_charge_zswap(objcg, entry->length); - count_objcg_events(objcg, ZSWPOUT, 1); - } - /* * We finish initializing the entry while it's already in xarray. * This is safe because: @@ -1496,36 +1481,137 @@ bool zswap_store(struct folio *folio) * an incoherent entry. */ if (entry->length) { + *compressed_bytes += entry->length; INIT_LIST_HEAD(&entry->lru); zswap_lru_add(&zswap_list_lru, entry); } - /* update stats */ - atomic_long_inc(&zswap_stored_pages); - count_vm_event(ZSWPOUT); - + /* + * We shouldn't have any possibility of failure after the entry is + * added in the xarray. The pool/objcg refs obtained here will only + * be dropped if/when zswap_entry_free() gets called. + */ return true; store_failed: zpool_free(entry->pool->zpool, entry->handle); -put_pool: - zswap_pool_put(entry->pool); -freepage: +put_pool_objcg: + zswap_pool_put(pool); + obj_cgroup_put(objcg); zswap_entry_cache_free(entry); reject: + return false; +} + +bool zswap_store(struct folio *folio) +{ + long nr_pages = folio_nr_pages(folio); + swp_entry_t swp = folio->swap; + struct xarray *tree = swap_zswap_tree(swp); + struct obj_cgroup *objcg = NULL; + struct mem_cgroup *memcg = NULL; + struct zswap_pool *pool; + size_t compressed_bytes = 0; + bool ret = false; + long index; + + VM_WARN_ON_ONCE(!folio_test_locked(folio)); + VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); + + if (!zswap_enabled) + goto check_old; + + /* + * Check cgroup zswap limits: + * + * The cgroup zswap limit check is done once at the beginning of + * zswap_store(). The cgroup charging is done once, at the end + * of a successful folio store. What this means is, if the cgroup + * was within the zswap_max limit at the beginning of a large folio + * store, it could go over the limit by at most (HPAGE_PMD_NR - 1) + * pages due to this store. + */ + objcg = get_obj_cgroup_from_folio(folio); + if (objcg && !obj_cgroup_may_zswap(objcg)) { + memcg = get_mem_cgroup_from_objcg(objcg); + if (shrink_memcg(memcg)) { + mem_cgroup_put(memcg); + goto put_objcg; + } + mem_cgroup_put(memcg); + } + + /* + * Check zpool utilization against zswap limits: + * + * The zswap zpool utilization is also checked against the limits + * just once, at the start of zswap_store(). If the check passes, + * any breaches of the limits set by zswap_max_pages() or + * zswap_accept_thr_pages() that may happen while storing this + * folio, will only be detected during the next call to + * zswap_store() by any process. + */ + if (zswap_check_limits()) + goto put_objcg; + + pool = zswap_pool_current_get(); + if (!pool) + goto put_objcg; + + if (objcg) { + memcg = get_mem_cgroup_from_objcg(objcg); + if (memcg_list_lru_alloc(memcg, &zswap_list_lru, GFP_KERNEL)) { + mem_cgroup_put(memcg); + goto put_pool; + } + mem_cgroup_put(memcg); + } + + /* + * Store each page of the folio as a separate entry. If we fail to + * store a page, unwind by deleting all the pages for this folio + * currently in zswap. + */ + for (index = 0; index < nr_pages; ++index) { + if (!zswap_store_page(folio_page(folio, index), objcg, pool, tree, &compressed_bytes)) + goto put_pool; + } + + if (objcg) { + obj_cgroup_charge_zswap(objcg, compressed_bytes); + count_objcg_events(objcg, ZSWPOUT, nr_pages); + } + + atomic_long_add(nr_pages, &zswap_stored_pages); + count_vm_events(ZSWPOUT, nr_pages); + + ret = true; + +put_pool: + zswap_pool_put(pool); +put_objcg: obj_cgroup_put(objcg); - if (zswap_pool_reached_full) + if (!ret && zswap_pool_reached_full) queue_work(shrink_wq, &zswap_shrink_work); check_old: /* - * If the zswap store fails or zswap is disabled, we must invalidate the - * possibly stale entry which was previously stored at this offset. - * Otherwise, writeback could overwrite the new data in the swapfile. + * If the zswap store fails or zswap is disabled, we must invalidate + * the possibly stale entries which were previously stored at the + * offsets corresponding to each page of the folio. Otherwise, + * writeback could overwrite the new data in the swapfile. */ - entry = xa_erase(tree, offset); - if (entry) - zswap_entry_free(entry); - return false; + if (!ret) { + pgoff_t offset = swp_offset(swp); + struct zswap_entry *entry; + + for (index = 0; index < nr_pages; ++index) { + entry = xa_erase(tree, offset + index); + if (entry) + zswap_entry_free(entry); + } + } + + return ret; } bool zswap_load(struct folio *folio) From patchwork Mon Sep 30 22:12:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13817149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AD07CEB2C2 for ; Mon, 30 Sep 2024 22:12:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8DDA228002D; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8299D280038; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38E9D28002D; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 16B8E280036 for ; Mon, 30 Sep 2024 18:12:32 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CBE8D1C6343 for ; Mon, 30 Sep 2024 22:12:31 +0000 (UTC) X-FDA: 82622804502.12.54FC89C Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf21.hostedemail.com (Postfix) with ESMTP id C59731C0007 for ; Mon, 30 Sep 2024 22:12:29 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nH2SJM+y; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727734224; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9lDffjabhJBUzn5h9qpLxQIzx4SS88A69IWNt0BF5D0=; b=UKkIzPAXdtFIL3gOXMbNwKWZ4kMcwCLTT2UQDyg92tRH/HsXeZo8yBga12HmEA/tcfwKww BmcFJ7eZzXUEBTChILa4B/FdwaaQGjgdx0lgyAxkxFb9j/xYVe9uJsBTRliVIaAkB/mVmj pbIHYb/lQXOKOIbgSU+bW2Po6Vz8aTk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727734224; a=rsa-sha256; cv=none; b=2keocTFmZIOqkxJWWhWYSDqIzJvyL35Q89zEXqGB3SwVZKgbM9fQ7tgYPltaCFrBhIdM/E 997SXT/prLiBp3rVBLXFkms5CcWW66yu7LqzdzVrAOohCIYZXm/D5YZx/TG9BfEKzL4cdb xL/CXHtil396wyPBiKq7PPnQZv0MJRo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nH2SJM+y; spf=pass (imf21.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727734350; x=1759270350; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9WsgnlbaWxDYzoSeOHrRpgB8xi3C6kSbFeaymu5ESD4=; b=nH2SJM+yFKKS5sBT1n5XXWFWRxD0B3F6YK4tKKhyFPj7S0cPE4Sgib3g eX41ehwVOhiL6dnfTW/vkNXGXsN/hzuzWR7wgb1miNKAqkPiVu2ONYLgo chTVUVdBsIgdaC3v4ILYpLLjLYxR54GYSluTXfrBGcXGlISZk96NJ4j6G RDKRWhpi4cOAoci9YasrwDEwkUju6OCB/po6BEXli8e/FuRVwMIHh7HY7 rO+PhqiGNxs7aSroiu+QFWP4kDPRVGPUUy0s9oJNGiW+aprEA+LekbAh4 RHmBILhKUUdNBbwBVyzSKIaBtHZmoiYHaNsODkGuwj3kMT8YdYi1AltO8 A==; X-CSE-ConnectionGUID: JfeC2ayvSpmEIfA0KaCwLg== X-CSE-MsgGUID: nLSSj6UMQ0OG0nbdpDRnVw== X-IronPort-AV: E=McAfee;i="6700,10204,11211"; a="49368458" X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="49368458" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Sep 2024 15:12:25 -0700 X-CSE-ConnectionGUID: T9yraZJRTZCRQZahtHf09g== X-CSE-MsgGUID: prXEZfqVSHWeMdbJj+LIig== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,166,1725346800"; d="scan'208";a="77985597" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa004.fm.intel.com with ESMTP; 30 Sep 2024 15:12:24 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, usamaarif642@gmail.com, shakeel.butt@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org, willy@infradead.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v9 7/7] mm: swap: Count successful large folio zswap stores in hugepage zswpout stats. Date: Mon, 30 Sep 2024 15:12:21 -0700 Message-Id: <20240930221221.6981-8-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> References: <20240930221221.6981-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C59731C0007 X-Stat-Signature: yohskcg3tynyg6tgjz9hiyx3gnmeeug5 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727734349-810115 X-HE-Meta: U2FsdGVkX19GFQEQaOwXCJrzRbweqvSKSeGoGF8kigKV9SYc27dfDKPcuBw03aJ5WD0lg/QVuarm9DAYOfOQkcfQLPUk+tDrWwwLdyxQGlmuB3tfdaOazkkmrgPkSwnIRT0kZ7/0QW5cpgD6CuKtu4B4Zu+Fl4gUAf9Qxb3mkKddGtx9MU0+1icDxOTvdlpfZDKsyDhfu+OUYVTLKNHnw+cN8HSe6lvAMJY3E12gAxcfBSXt1D0yGApA8l8mbIvR+eDMWEdcPP6MvL8eUbjxGcJgvrigVl9kVGyA6nvVvw/15P5ocSRfnTfYoGyd6TBc68DlZJyrjGZuviQAXxUbGCAj8eaQ5O+apIr1JC9UKhe1cRaEtAMLEw/G2ZhE+JFI4TKk/22S/KC1HFIvSOzvdm3k3YJdiuSUa8nAXeWr0SL18+J/YoJzse93SrE2zvGODOxFiMRxk7ShC57UzSq9iykY4XBoJmCr+iUIOl7KfYjCu0cuhqgbniAqSb0ski/i87uN7P+rKJwpiW21wj72dXSWsvmx1cDZuY06m/3da7Fo91gL8ctMDnxymSIrUCQBAd+KjiYtkrVi9eoUYGY6XZMJTvLg9Cb8HtQ4wLhWzTJXN2YF9XLh3r8jYj8g/hDyvYARkdRZMBufV9Q/bvWIndY1aeEadwhDMYwAO84daSkQaPWdk3957bohwiJWz1yPruYqRiJRAg9F32WEvLtR7HPfbGNletHErhgnkwbgoN+akFI9NMhRL5/7XS6cqd8JRVyLdr0Y25jHAb0++0WhGCz0uVP2MTo8QWqGrIbmivtY29WpN+qgtJzvUBe4qIE2KWMujEryAqbGDxssQetSi87JsOfDDdJ5sGAxzkOK9LGqab6xF71mALBiEi4p6sfe2eYR6z5mBP7wg/3zatNO+ze/65nFnBtl+BSKIz/izCR2XEw6+f2vNgeNgcoeOHeaq+seJQP0a+/57n6fI7Q IWGr5dgF H5uBUFKU61tb1+5jomCXgK1hEaNfq1yAlsFg9maxgu1IwhfWgyI5dA+2MtdfdcPdS3QWKUrV/UFCuHVXB02T5seH2qfLL3A/Qdgc1gQzmrSAzc3LOqjCBrBa9Pw203O5HwVZlxwE5xXl1wMz2eSTBA18o3Xt9jLrGZYU+madMwa+7p64CxKGYlPzbZxtwG6ep0D1H7VgmjhP9wSk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Added a new MTHP_STAT_ZSWPOUT entry to the sysfs transparent_hugepage stats so that successful large folio zswap stores can be accounted under the per-order sysfs "zswpout" stats: /sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/zswpout Other non-zswap swap device swap-out events will be counted under the existing sysfs "swpout" stats: /sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/swpout Also, added documentation for the newly added sysfs per-order hugepage "zswpout" stats. The documentation clarifies that only non-zswap swapouts will be accounted in the existing "swpout" stats. Signed-off-by: Kanchana P Sridhar Reviewed-by: Nhat Pham --- Documentation/admin-guide/mm/transhuge.rst | 8 ++++++-- include/linux/huge_mm.h | 1 + mm/huge_memory.c | 3 +++ mm/page_io.c | 1 + 4 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index cfdd16a52e39..2a171ed5206e 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -530,10 +530,14 @@ anon_fault_fallback_charge instead falls back to using huge pages with lower orders or small pages even though the allocation was successful. -swpout - is incremented every time a huge page is swapped out in one +zswpout + is incremented every time a huge page is swapped out to zswap in one piece without splitting. +swpout + is incremented every time a huge page is swapped out to a non-zswap + swap device in one piece without splitting. + swpout_fallback is incremented if a huge page has to be split before swapout. Usually because failed to allocate some continuous swap space diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 5eb4b0376c7d..3eca60f3d512 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -119,6 +119,7 @@ enum mthp_stat_item { MTHP_STAT_ANON_FAULT_ALLOC, MTHP_STAT_ANON_FAULT_FALLBACK, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_ZSWPOUT, MTHP_STAT_SWPOUT, MTHP_STAT_SWPOUT_FALLBACK, MTHP_STAT_SHMEM_ALLOC, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 13bf59b84075..f596f57a3a90 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -611,6 +611,7 @@ static struct kobj_attribute _name##_attr = __ATTR_RO(_name) DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(zswpout, MTHP_STAT_ZSWPOUT); DEFINE_MTHP_STAT_ATTR(swpout, MTHP_STAT_SWPOUT); DEFINE_MTHP_STAT_ATTR(swpout_fallback, MTHP_STAT_SWPOUT_FALLBACK); #ifdef CONFIG_SHMEM @@ -629,6 +630,7 @@ static struct attribute *anon_stats_attrs[] = { &anon_fault_fallback_attr.attr, &anon_fault_fallback_charge_attr.attr, #ifndef CONFIG_SHMEM + &zswpout_attr.attr, &swpout_attr.attr, &swpout_fallback_attr.attr, #endif @@ -659,6 +661,7 @@ static struct attribute_group file_stats_attr_grp = { static struct attribute *any_stats_attrs[] = { #ifdef CONFIG_SHMEM + &zswpout_attr.attr, &swpout_attr.attr, &swpout_fallback_attr.attr, #endif diff --git a/mm/page_io.c b/mm/page_io.c index bc1183299a7d..4aa34862676f 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -269,6 +269,7 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) swap_zeromap_folio_clear(folio); } if (zswap_store(folio)) { + count_mthp_stat(folio_order(folio), MTHP_STAT_ZSWPOUT); folio_unlock(folio); return 0; }