From patchwork Mon Aug 19 02:16:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13767723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E99ADC5321D for ; Mon, 19 Aug 2024 02:16:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 751366B0085; Sun, 18 Aug 2024 22:16:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B2A26B0088; Sun, 18 Aug 2024 22:16:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B81C6B0089; Sun, 18 Aug 2024 22:16:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2F97A6B0085 for ; Sun, 18 Aug 2024 22:16:27 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D758CC0B77 for ; Mon, 19 Aug 2024 02:16:26 +0000 (UTC) X-FDA: 82467380772.12.71EB268 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf28.hostedemail.com (Postfix) with ESMTP id BE2A7C0018 for ; Mon, 19 Aug 2024 02:16:24 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Vb3sOVRb; spf=pass (imf28.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724033708; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HVbxgbaLuIkv/t+1nckM1WyTgPV4PHYSrdazdOCSElg=; b=jo1J3OFYLK6GtMcvZMVP7BQr5OCHgd/I0xUqfLWfYBalV/qsYrufXfVHCbKocXxcb3jKq6 fkKhIpccU9rPIk89EgDOq6sjqJaVGyY6bOpDAjoWs9+p/jcJ1nsiRVowgpup7Q/SLr7+ay J9mkdUnI8Wqy/qU3a+19xwlNd9p5YxI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724033708; a=rsa-sha256; cv=none; b=IjRCFb39hhIPKAh9dNo3FLVxAAUN2gTzPukTiMNQ8f16aKVOgRHP62rWQHDqdUlfly2vMT sHfXPdIeWmpfamB2eTH+iNb4hxFfDBkEYWdVuRoPVMPRGqSu9L76IolNMXhekLzmqsawBj Hs5TtDQnz1Wpg6+kK1SD50OFTMrXF4s= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Vb3sOVRb; spf=pass (imf28.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724033785; x=1755569785; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i9i0GU1tqbl+gZG9rdF1xrCDy8bSikrdWFFB2p3UFUw=; b=Vb3sOVRbDYJL29ClXUDUkuEtuuYV2XMA+jrynajBdvrdpCHqPwJLJQW5 3DgYgEniJsf/dMbIztkp2H5F6sl7rraKmzGmIN0FeawLXUBuwL0w7/E8+ Onz/Y8xv04QhyLEu14+lCJmv6OETE/afJzDz8t1yCX9lotQKlLQwcuaD1 ieUNi9tSTdxwFeMseWYnX11GJNmjYSyiGoZvCppHbSg3lsmx2G3igExvy d8yonF07F9qGM/7cOhSNFrQK0Hmz9MrKHSk6b25/mCK4OOYMt8Mv6eovw kIYYA8owJtulTtlTF59uK5YgElZaHDmtTExkir2QnLT1F4BdrVwFOfYf1 w==; X-CSE-ConnectionGUID: urbN5WG6RkyPTohU4iqdxw== X-CSE-MsgGUID: a5iDXdo1QVuRUDvDJrFoxQ== X-IronPort-AV: E=McAfee;i="6700,10204,11168"; a="44782960" X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="44782960" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 19:16:23 -0700 X-CSE-ConnectionGUID: wUHvjw1jTIGDkmypoq16Tw== X-CSE-MsgGUID: R4qRuZDVT/aLemdFg0NPEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="64610270" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa005.fm.intel.com with ESMTP; 18 Aug 2024 19:16:21 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v4 1/4] mm: zswap: zswap_is_folio_same_filled() takes an index in the folio. Date: Sun, 18 Aug 2024 19:16:18 -0700 Message-Id: <20240819021621.29125-2-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BE2A7C0018 X-Stat-Signature: taacad1b4nw3qt8s4x3cxnhwakik33kw X-HE-Tag: 1724033784-176173 X-HE-Meta: U2FsdGVkX1/3m7+WXF1+n05JmUQ60niAR3tbe7KqhesVM3OWZNGyn85A2mVTRYyitQx5ck3C0Vs/XLomVSNg4daGjlsvV+0oN3AVY2NdEH8inXN7YalkhmAk1uOLjwmXEWXV93MzX6/wkDLgdx7iOd2XpxIK/kIotw6wGAidZQsELh+Q4X9Sl5sbmTLA4z+RyRPpNS2mlJcvm97ELNyKBUb1fAY7Tv+IeycZptP3V3gqO1OTEgxfJasN8nfXCVFVSc60k/TVPeuzaHH7wniTcbRs4xWJR63a4d7uH9ewi/fqqQMc/RH+g5JdzHtoebvgox0CuGyYDDsnvaBkEJZnkU4/kVTJYDigEwMhhTbUoX4jz/4l/Kw0Z4EsL/JG5GaOlgGD1D4HdFzbEkyS7P5OcAg5WhdJbggeUfgq6r5J2AoGpK4GFPL+StGwNi+CP8sC1tHlO3dwwKl/LyziWtp6H9rTGAFiJVKCm9TbZKPP/Z/XLhsQBRgDaF8Z4Tok+HuyoQ9n8+Ql749eA+98YeAREGDcGvk5YqXTrt58oF/TBcjGBW8Yoxhe2u4hQWphvLg7weqMB5NOlru4HtWeoDlQKaI7VrfXeWbp7VPGLobbllJpmORYHlJvmXFYOm9HO/NiqpNd47WlQIw3GKszFj+iiZGS/+3nahNwnnIJgwsX/O4Nd8ZzVekjjMa5zYLCJy0Lbh0pFJUl7oklEhYL+LERdWuImrJoUlAEDhQv/y/GFCvuKMlM3IEOPGY8wmJFRaWShIFqnjODNmFXUtGYBokQL5kKD/Rona9gUYWF05V3As1NLDrEUcEidIux7g8wicZMlEpoR25mDCQoqtCVI2xM1I/H2U2WYqF9wsybfEm2h0WRFxoO4rmAg2zRNcJsAvcT0WOHKePoy8vzl0WB/qctfu3ltggRZpwAIIKdvk5K4bOXQu3ODzRCTEN9UCvxmWzclqaEN6Gx6lmmtdxFYkY 2yk1phym pI5hQ9NIQvgCpC+WEJ4TgT0SJgp2Rzmq//t56DKKwho6GhqRqWI+1Fa/wzhumZ9dB5lopBN22uVB6MJRj4XZxIeV02pM+hmC0JnRSVmIxc3xJGbg3QlQw5GZ0KHHeWIgBbPdrhhVZycvkPRD2qR7irWAZDl9KyAGRphttJM5hy/XAo5rnD24+OKw0ajxxr+jgx2A40U9uW9rJVg4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This change is being made so that zswap_store can process mTHP folios. Modified zswap_is_folio_same_filled() to work for any-order folios, by accepting an additional "index" parameter to arrive at the page within the folio to run the same-filled page check. Signed-off-by: Kanchana P Sridhar --- mm/zswap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index df66ab102d27..6434e76a4ea3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1419,14 +1419,14 @@ static void shrink_worker(struct work_struct *w) /********************************* * same-filled functions **********************************/ -static bool zswap_is_folio_same_filled(struct folio *folio, unsigned long *value) +static bool zswap_is_folio_same_filled(struct folio *folio, long index, unsigned long *value) { unsigned long *data; unsigned long val; unsigned int pos, last_pos = PAGE_SIZE / sizeof(*data) - 1; bool ret = false; - data = kmap_local_folio(folio, 0); + data = kmap_local_folio(folio, index * PAGE_SIZE); val = data[0]; if (val != data[last_pos]) @@ -1496,7 +1496,7 @@ bool zswap_store(struct folio *folio) goto reject; } - if (zswap_is_folio_same_filled(folio, &value)) { + if (zswap_is_folio_same_filled(folio, 0, &value)) { entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); From patchwork Mon Aug 19 02:16:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13767724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0CE3C531DF for ; Mon, 19 Aug 2024 02:16:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68F856B0088; Sun, 18 Aug 2024 22:16:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 617FD6B0089; Sun, 18 Aug 2024 22:16:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F51A6B008A; Sun, 18 Aug 2024 22:16:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 22CBD6B0088 for ; Sun, 18 Aug 2024 22:16:28 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C80ABC0EB5 for ; Mon, 19 Aug 2024 02:16:27 +0000 (UTC) X-FDA: 82467380814.22.4A9DE2D Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf24.hostedemail.com (Postfix) with ESMTP id A1862180012 for ; Mon, 19 Aug 2024 02:16:25 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=VaZwzird; spf=pass (imf24.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724033747; a=rsa-sha256; cv=none; b=mbH3oAv6hjdka9PKF++WDRD/HnfR0COTTxZ2CqnRXZZa8hzua86jYlrgAiDtjOaNg1PjW0 WNBPCTV8rDehayI3qrGDZe/qkF10xSZPQuKumKWoibY4zg9TEPS02B/UE5ECgixL401HU1 UcK9EfM8QfKRn1VAic0mQWfXTbspdqU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=VaZwzird; spf=pass (imf24.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724033747; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6JnaV89XwuiFN6KUU62xOsDunazbf/zlhDtReIUxJ4g=; b=hKfGMI72LIijQsAV4OR+AaYxYnoLorZEvdsRs6YJRwDzbkLN+oQ1oc/njF2o7wn0pvO/jF atyDHN3dgPe0JfL5TDCxtltFEWK8Blm4dqBZVYl6xPo9zi7TCLZjBQBEptdM0k1wX2lNuL z34FGR2QwaaNJPMKMm7N+7bIkCLouyM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724033786; x=1755569786; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Brs+d0RWd3m3x72bXC3nHyVxOKVWfAGnWhAdd32EUtA=; b=VaZwzirdo6K4Gs6C8njeV7f+3QVmMP8GhkG5KKw/0g/CHXZemVwA+vE8 pLfmWFN0nEWZFPiHpkLNSXz0niVBD7aSrBMVoWcGeoi0RudLSZT1zq2/s suilM6JBDOjZeXRUTmmo0rqhyTng13Y04f2+LyFnznNmry5shDTilxwoK XuFPJBtBWujPWB9WtPYVerGXPE+lIo5B9aB4AWEMP+VZAjCS2TS/W2oRW BQ9w07wMeJe60zpJm+GqleDWqeK5oeXEDAfvlY0bevrodb5q3WKiTMdFc 9RlsK4IPnJyTjxo9cfAsxAWWq9RNTM2tR8hHgGt42I+KRRVrUVVr/0idI Q==; X-CSE-ConnectionGUID: nrcZdKOwSnCy44Li5NpiPw== X-CSE-MsgGUID: Z18BKT6kQHyL5FSEmN/mGQ== X-IronPort-AV: E=McAfee;i="6700,10204,11168"; a="44782967" X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="44782967" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 19:16:23 -0700 X-CSE-ConnectionGUID: xF8wM+DER2KVerhf5nrDDQ== X-CSE-MsgGUID: l8IiLFcXQl21mePbv8afzQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="64610274" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa005.fm.intel.com with ESMTP; 18 Aug 2024 19:16:22 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v4 2/4] mm: zswap: zswap_store() extended to handle mTHP folios. Date: Sun, 18 Aug 2024 19:16:19 -0700 Message-Id: <20240819021621.29125-3-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Stat-Signature: uriax4tytiu4j1esk6i676mqg71kdq5q X-Rspamd-Queue-Id: A1862180012 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724033785-598296 X-HE-Meta: U2FsdGVkX1+yt/Xh299zVXeQGAVNl9AZIo9ZtTDDRhrc+zETryM/o4zBgzk+jUyPt5nuHRy6MMDjZdhsBqaaE+z/WbLF0bZlhTBHTUQ3c9nu1iMeNumQikd25PpkGxE6FbRrG48KdQrGdv4wurTqTGa4yI9Sp8N2gvNXHAsds8dRz+xxntOQ/AWV4oBdURmQFqW7aSvhTc3WaPvgBV75en3BdBUCJD3h0mDckaIN5W+v+L9XbffwoxnUmmJ4MGd3vsuwMnv4eyK6YAM/LHC3xwaBPFDw+if7UOQLElQelUYtZLv7NaoVCFXTtfw2hMM3RpSfsqn+FAEZCzErA+RCDRnP63AJ4oJAbB1sN0WMLiezsdwJ7oJWxFspxUqaTyMPo5fL8ty0wDtyeBRE/UcwRYVk+c5nLwytL0Dcdno2Fy/x+NfbiB1tv/oh2OfDrva6+f7ZHURoEdmS+sQdEeCK8qNL5XvD1XuRjAKNrZ5ZwiiVUg/GqVV+aHg7066Mm1Ez7a2hbXldpAmHGGzTpfVk8TUGe6TbjsFuGEVhu6tOUku+lvqnNGqgwozxBAq22L7GbaEXiFKhIcLwDHjMjfu8YORZkJErSPRl4Y1iqOf8Gy4eFmkAhr8iFkbBqc9ADwASQWSNxSyT9Vvx2+mZgt9od9uw1CW4LKgSZSb1H2Ha1sV4SOf6ptSt53zGh4H/aKAjrKvjMcgIXV3nnC7v2ZZvAG1tciVmkHHK7M9h25krp4CfW4Dq0qsOTAeWVwsyGQCE/c9mILFQJZet+tDH356wFr+uEVzkO5bURMK4HFiZtX6/qDD2fX8nD71OvK9GOxxa5hbimf5pQzwNGH/w5E6DR+NBck56dwWlKO+rHc3VpW/pRODo/ByUUINkCpwNCFWBR5soVKKIzHmsea282mCTpCcfHXE9W69fDvjYlnWQQ840PDwfzCE4uwSLzXd0omP2vJXF2JqS7RjZJ15Ctzd xiYcya+2 jypVCGqj2ONHEG/97oxiouL7bwa4TztLZXcMVKHxwduyoIR71OTlwOmWo8IYw9QR3jRb7mv6C7M8XLLZIVlRDSyUa7aSKHKASCFuE+qLzqFRQo+G3AbIP6lwwo2VQMxtsBXqmdcAU2BgEyI8+bZU2FMe/+xm9A/dROrhSUXMjR/9XvCTRjD6pSuLA+JKm3MLGqmvUpS1JRnTpsRvm6t60i3S6G84Am3Jzx6FgQLc0uG++2PUojDRqGoG4hgcgF+zaZoYu8D5c3Mga1vZV5b3KwueYTNSxSunjd4WVCVQe6LosOGOKCbAWdyBZcpkS4LsSUdf3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: zswap_store() will now process and store mTHP and PMD-size THP folios. This change reuses and adapts the functionality in Ryan Roberts' RFC patch [1]: "[RFC,v1] mm: zswap: Store large folios without splitting" [1] https://lore.kernel.org/linux-mm/20231019110543.3284654-1-ryan.roberts@arm.com/T/#u This patch provides a sequential implementation of storing an mTHP in zswap_store() by iterating through each page in the folio to compress and store it in the zswap zpool. Towards this goal, zswap_compress() is modified to take a page instead of a folio as input. Each page's swap offset is stored as a separate zswap entry. If an error is encountered during the store of any page in the mTHP, all previous pages/entries stored will be invalidated. Thus, an mTHP is either entirely stored in ZSWAP, or entirely not stored in ZSWAP. This forms the basis for building batching of pages during zswap store of large folios, by compressing batches of up to say, 8 pages in an mTHP in parallel in hardware, with the Intel In-Memory Analytics Accelerator (Intel IAA). Also, addressed some of the RFC comments from the discussion in [1]. Co-developed-by: Ryan Roberts Signed-off-by: Signed-off-by: Kanchana P Sridhar Reported-by: kernel test robot --- mm/zswap.c | 234 +++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 172 insertions(+), 62 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 6434e76a4ea3..9862a4cf0651 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -886,7 +886,7 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +static bool zswap_compress(struct page *page, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; struct scatterlist input, output; @@ -904,7 +904,7 @@ static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) dst = acomp_ctx->buffer; sg_init_table(&input, 1); - sg_set_folio(&input, folio, PAGE_SIZE, 0); + sg_set_page(&input, page, PAGE_SIZE, 0); /* * We need PAGE_SIZE * 2 here since there maybe over-compression case, @@ -1455,36 +1455,83 @@ static void zswap_fill_folio(struct folio *folio, unsigned long value) /********************************* * main API **********************************/ -bool zswap_store(struct folio *folio) + +/* + * Returns true if the entry was successfully + * stored in the xarray, and false otherwise. + */ +static bool zswap_store_entry(struct xarray *tree, + struct zswap_entry *entry) { - swp_entry_t swp = folio->swap; - pgoff_t offset = swp_offset(swp); - struct xarray *tree = swap_zswap_tree(swp); - struct zswap_entry *entry, *old; - struct obj_cgroup *objcg = NULL; - struct mem_cgroup *memcg = NULL; - unsigned long value; + struct zswap_entry *old; + pgoff_t offset = swp_offset(entry->swpentry); - VM_WARN_ON_ONCE(!folio_test_locked(folio)); - VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); + old = xa_store(tree, offset, entry, GFP_KERNEL); - /* Large folios aren't supported */ - if (folio_test_large(folio)) + if (xa_is_err(old)) { + int err = xa_err(old); + + WARN_ONCE(err != -ENOMEM, "unexpected xarray error: %d\n", err); + zswap_reject_alloc_fail++; return false; + } - if (!zswap_enabled) - goto check_old; + /* + * We may have had an existing entry that became stale when + * the folio was redirtied and now the new version is being + * swapped out. Get rid of the old. + */ + if (old) + zswap_entry_free(old); - /* Check cgroup limits */ - objcg = get_obj_cgroup_from_folio(folio); - if (objcg && !obj_cgroup_may_zswap(objcg)) { - memcg = get_mem_cgroup_from_objcg(objcg); - if (shrink_memcg(memcg)) { - mem_cgroup_put(memcg); - goto reject; - } - mem_cgroup_put(memcg); + return true; +} + +/* + * If the zswap store fails or zswap is disabled, we must invalidate the + * possibly stale entries which were previously stored at the offsets + * corresponding to each page of the folio. Otherwise, writeback could + * overwrite the new data in the swapfile. + * + * This is called after the store of the i-th offset in a large folio has + * failed. All zswap entries in the folio must be deleted. This helps make + * sure that a swapped-out mTHP is either entirely stored in zswap, or + * entirely not stored in zswap. + * + * This is also called if zswap_store() is invoked, but zswap is not enabled. + * All offsets for the folio are deleted from zswap in this case. + */ +static void zswap_delete_stored_offsets(struct xarray *tree, + pgoff_t offset, + long nr_pages) +{ + struct zswap_entry *entry; + long i; + + for (i = 0; i < nr_pages; ++i) { + entry = xa_erase(tree, offset + i); + if (entry) + zswap_entry_free(entry); } +} + +/* + * Stores the page at specified "index" in a folio. + */ +static bool zswap_store_page(struct folio *folio, long index, + struct obj_cgroup *objcg, + struct zswap_pool *pool) +{ + swp_entry_t swp = folio->swap; + int type = swp_type(swp); + pgoff_t offset = swp_offset(swp) + index; + struct page *page = folio_page(folio, index); + struct xarray *tree = swap_zswap_tree(swp); + struct zswap_entry *entry; + unsigned long value; + + if (objcg) + obj_cgroup_get(objcg); if (zswap_check_limits()) goto reject; @@ -1496,7 +1543,7 @@ bool zswap_store(struct folio *folio) goto reject; } - if (zswap_is_folio_same_filled(folio, 0, &value)) { + if (zswap_is_folio_same_filled(folio, index, &value)) { entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); @@ -1504,43 +1551,21 @@ bool zswap_store(struct folio *folio) } /* if entry is successfully added, it keeps the reference */ - entry->pool = zswap_pool_current_get(); - if (!entry->pool) + if (!zswap_pool_get(pool)) goto freepage; - if (objcg) { - memcg = get_mem_cgroup_from_objcg(objcg); - if (memcg_list_lru_alloc(memcg, &zswap_list_lru, GFP_KERNEL)) { - mem_cgroup_put(memcg); - goto put_pool; - } - mem_cgroup_put(memcg); - } + entry->pool = pool; - if (!zswap_compress(folio, entry)) + if (!zswap_compress(page, entry)) goto put_pool; store_entry: - entry->swpentry = swp; + entry->swpentry = swp_entry(type, offset); entry->objcg = objcg; entry->referenced = true; - old = xa_store(tree, offset, entry, GFP_KERNEL); - if (xa_is_err(old)) { - int err = xa_err(old); - - WARN_ONCE(err != -ENOMEM, "unexpected xarray error: %d\n", err); - zswap_reject_alloc_fail++; + if (!zswap_store_entry(tree, entry)) goto store_failed; - } - - /* - * We may have had an existing entry that became stale when - * the folio was redirtied and now the new version is being - * swapped out. Get rid of the old. - */ - if (old) - zswap_entry_free(old); if (objcg) { obj_cgroup_charge_zswap(objcg, entry->length); @@ -1574,7 +1599,7 @@ bool zswap_store(struct folio *folio) else { zpool_free(entry->pool->zpool, entry->handle); put_pool: - zswap_pool_put(entry->pool); + zswap_pool_put(pool); } freepage: zswap_entry_cache_free(entry); @@ -1582,16 +1607,101 @@ bool zswap_store(struct folio *folio) obj_cgroup_put(objcg); if (zswap_pool_reached_full) queue_work(shrink_wq, &zswap_shrink_work); -check_old: + + return false; +} + +/* + * Modified to store mTHP folios. Each page in the mTHP will be compressed + * and stored sequentially. + */ +bool zswap_store(struct folio *folio) +{ + long nr_pages = folio_nr_pages(folio); + swp_entry_t swp = folio->swap; + pgoff_t offset = swp_offset(swp); + struct xarray *tree = swap_zswap_tree(swp); + struct obj_cgroup *objcg = NULL; + struct mem_cgroup *memcg = NULL; + struct zswap_pool *pool; + bool ret = false; + long index; + + VM_WARN_ON_ONCE(!folio_test_locked(folio)); + VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); + + if (!zswap_enabled) + goto reject; + /* - * If the zswap store fails or zswap is disabled, we must invalidate the - * possibly stale entry which was previously stored at this offset. - * Otherwise, writeback could overwrite the new data in the swapfile. + * Check cgroup limits: + * + * The cgroup zswap limit check is done once at the beginning of an + * mTHP store, and not within zswap_store_page() for each page + * in the mTHP. We do however check the zswap pool limits at the + * start of zswap_store_page(). What this means is, the cgroup + * could go over the limits by at most (HPAGE_PMD_NR - 1) pages. + * However, the per-store-page zswap pool limits check should + * hopefully trigger the cgroup aware and zswap LRU aware global + * reclaim implemented in the shrinker. If this assumption holds, + * the cgroup exceeding the zswap limits could potentially be + * resolved before the next zswap_store, and if it is not, the next + * zswap_store would fail the cgroup zswap limit check at the start. */ - entry = xa_erase(tree, offset); - if (entry) - zswap_entry_free(entry); - return false; + objcg = get_obj_cgroup_from_folio(folio); + if (objcg && !obj_cgroup_may_zswap(objcg)) { + memcg = get_mem_cgroup_from_objcg(objcg); + if (shrink_memcg(memcg)) { + mem_cgroup_put(memcg); + goto put_objcg; + } + mem_cgroup_put(memcg); + } + + if (zswap_check_limits()) + goto put_objcg; + + pool = zswap_pool_current_get(); + if (!pool) + goto put_objcg; + + if (objcg) { + memcg = get_mem_cgroup_from_objcg(objcg); + if (memcg_list_lru_alloc(memcg, &zswap_list_lru, GFP_KERNEL)) { + mem_cgroup_put(memcg); + goto put_pool; + } + mem_cgroup_put(memcg); + } + + /* + * Store each page of the folio as a separate entry. If we fail to store + * a page, unwind by removing all the previous pages we stored. + */ + for (index = 0; index < nr_pages; ++index) { + if (!zswap_store_page(folio, index, objcg, pool)) + goto put_pool; + } + + ret = true; + +put_pool: + zswap_pool_put(pool); +put_objcg: + obj_cgroup_put(objcg); + if (zswap_pool_reached_full) + queue_work(shrink_wq, &zswap_shrink_work); +reject: + /* + * If the zswap store fails or zswap is disabled, we must invalidate + * the possibly stale entries which were previously stored at the + * offsets corresponding to each page of the folio. Otherwise, + * writeback could overwrite the new data in the swapfile. + */ + if (!ret) + zswap_delete_stored_offsets(tree, offset, nr_pages); + + return ret; } bool zswap_load(struct folio *folio) From patchwork Mon Aug 19 02:16:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13767725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD5A4C5321D for ; Mon, 19 Aug 2024 02:16:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84C676B008A; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C1C56B0093; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 493546B0092; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 23C266B008A for ; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A0A51160E96 for ; Mon, 19 Aug 2024 02:16:28 +0000 (UTC) X-FDA: 82467380856.09.8709E19 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf18.hostedemail.com (Postfix) with ESMTP id 83F441C001A for ; Mon, 19 Aug 2024 02:16:26 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DJLFl5sV; spf=pass (imf18.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724033709; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uty/QWuHBMaY/uR/NUz+WS/fVkoVt/Cbs3XD4FdGk8k=; b=qL7fxSDMlGi9OTmIDUvpGfDlg3NMz64Tf3g/hSo4X4jQMIvI56Rtw4+yfT0GsM+aGemrn1 gzyN6v5isbKE+71mNmt77mPWrRaW1Zh4ED34iNIJXoDRYs3NTIAiXGbn4O3Y0Fzt9DbwRl fUa02ZN/kZgYt+2qj/DhLXxteRMb5Tw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724033709; a=rsa-sha256; cv=none; b=Xqv13gfhMwGoREYcR6DVW1tumi+BrX/eywlCZH5YNTfrolXbgNKmwDXE6DYSE56jCbaRaR 2BKc8l5mzHXwwdIK4jQqPuJm1Ym5uAgYPjczZ4yguHYagkOEBDlc31AA+B+Kk95XNsfKPa eBerEN1gkZnvBdLo9KKZRNLtpF6RC5w= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DJLFl5sV; spf=pass (imf18.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724033787; x=1755569787; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pCqID+ibJQzADFduLDZJx6IU2iRBnr8hLVZlnSUbIa4=; b=DJLFl5sVEYLpX2SIVeHhTyfdmZoryX4ggQD2ezfTP2K7RqyEqjj08apw 9pFNUYAsdrvShmuJ5pUimOJ7BPAWxE43E1k71xb0XSKSQrcRG8a3CdPKC fhd5SCtUV3A7j4OgfvUQATlxluqH28eUQdUvUdHiDU4jnMTwEeVSOHi3g dh6zLJGEASnShBmT+SyIlHcHCAtMMtQ+e8oPF0C3bB0IKVu1qUQKVRQPa jzZytlJsHL/lu+ofwpGG6BR1H7LtbUYG4zWOp6IgnT67Egyi8UwbO8mo7 cjATS3pKjd2FJxRb0AV3QzH/4uw/zOfywt+ub5TJ7BLGszHleKRYH8vBe g==; X-CSE-ConnectionGUID: RY4ZVVAgSseK6P/+ccU2Uw== X-CSE-MsgGUID: AP7svo3FQii2vk9tbHnfow== X-IronPort-AV: E=McAfee;i="6700,10204,11168"; a="44782974" X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="44782974" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 19:16:24 -0700 X-CSE-ConnectionGUID: rdbxAd1TSN20Iehf1fk1wQ== X-CSE-MsgGUID: 2WIPODmDRpS0GbvMDmwXxQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="64610277" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa005.fm.intel.com with ESMTP; 18 Aug 2024 19:16:22 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v4 3/4] mm: Add MTHP_STAT_ZSWPOUT to sysfs per-order mthp stats. Date: Sun, 18 Aug 2024 19:16:20 -0700 Message-Id: <20240819021621.29125-4-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Stat-Signature: wa5tiniptmutbrod99i6si8csbg1s7af X-Rspamd-Queue-Id: 83F441C001A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1724033786-932794 X-HE-Meta: U2FsdGVkX1/+I3XMbfquJa0bDpTmlL78H5yg+TJ3XXQ0FKBq5sAL/F2hhAvqkTjRIjqImbAB6TgzFAPm6lnQUKwEhI5H3VZ99E+oIbe/VmAUnupG1rZnj7/ThkKoADDg/IlKZAStw0M0qVMdiFPUQqCu+xeSW0gGMXyhAT+oNRnbkcBxcPIakZeg8TRZQXq0/nrdtXtU/413i6LZVO8MfWFN+iZOK/82zHN6cpOuL7fWG93mPm7TTVnZvUctuxgrQfxdCWNw31xmaUp+HHtdakwSdil0THKDdX3b/E7d7KZB0Ff8rkwfTrWyEDwb0Z1mDTtm2DV+LmS2n8Gq3H+CDmYAGRhYxY8V/Hk3mAjsIMDHKRA1yrUxJvasCRKQA17vXTN1XeX9qc5H567HnXo845o8uZLeQ5ckJDI9KVNl1JpX7uQeABeJz1FaD7KXmALLjPtU8iyRLawJkWtiI7DEvdxgGl+LVku8J5WaGELD5nsb5zP22k0OsIuOtDUVYQ5qA+vkcdQJHjB5bl+CpGQ93rlyHSlBECns+/P+gAwnu9W5pPb69nwX2f4rh17MxKYFZnfBN1l/rlXK+aHs5/TudP+iWThUp8h61cshxZz0h7Or/jbZE01AFDSwNUeRX8Kw192h4qCF0rVlM5fK0pp6BSQS+NR6E4NK8WGPIAnGD86Lx3Y5nFjiqIDERYzLwRvUbtErkBOrjtw5KoSEFOoZToH/oucWxYjAqq1QYBKtAx5MHlSiwCVUmc8HQ99e2F5qz2IMiN1WSQqwwMEEVgCkE1BjMjjxslk0rAvtU9s1wSWqOrzYfq+MEfp/CHAM0CYZNsUGWm89GTKmXtB37tbSSSzKVA5ovsh0uBOcTK9hPuLu/IJVeyJTTf3XYkpPgYd3JokAbKZMXRqxyUuHF8DclknWzlNpKutUEuXCrRMjbuiy8PRDEgMtVZiz4cPrAJEVon7Gn3X8dFwRMpk6pH3 pXy3F0aB 9CdBSKnHu3RZChXIPBsNfcmIe440V6KMerqrGEXSDXcIv8JcuYV47FWzWZnPFoyOG0QkK7NCeKe/PAnauVBGP7uE/cv9C9Z4J90iieBTfDqygu1AvOd641PNndSkOIVYqwWD7fs6kKHXftPVw6gCZr1+hnO+V7FfEYU5swv+zVyAHZ67ijdWrniAQHazIyWtPNxtYKsvk4G82rXg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a new MTHP_STAT_ZSWPOUT entry to the sysfs mTHP stats so that per-order mTHP folio ZSWAP stores can be accounted. Signed-off-by: Kanchana P Sridhar --- include/linux/huge_mm.h | 1 + mm/huge_memory.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 4c32058cacfe..19d3e26eb276 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -118,6 +118,7 @@ enum mthp_stat_item { MTHP_STAT_ANON_FAULT_ALLOC, MTHP_STAT_ANON_FAULT_FALLBACK, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_ZSWPOUT, MTHP_STAT_SWPOUT, MTHP_STAT_SWPOUT_FALLBACK, MTHP_STAT_SHMEM_ALLOC, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index cf8e34f62976..73b8ad255d8b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -587,6 +587,7 @@ static struct kobj_attribute _name##_attr = __ATTR_RO(_name) DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(zswpout, MTHP_STAT_ZSWPOUT); DEFINE_MTHP_STAT_ATTR(swpout, MTHP_STAT_SWPOUT); DEFINE_MTHP_STAT_ATTR(swpout_fallback, MTHP_STAT_SWPOUT_FALLBACK); #ifdef CONFIG_SHMEM @@ -603,6 +604,7 @@ static struct attribute *anon_stats_attrs[] = { &anon_fault_fallback_attr.attr, &anon_fault_fallback_charge_attr.attr, #ifndef CONFIG_SHMEM + &zswpout_attr.attr, &swpout_attr.attr, &swpout_fallback_attr.attr, #endif @@ -633,6 +635,7 @@ static struct attribute_group file_stats_attr_grp = { static struct attribute *any_stats_attrs[] = { #ifdef CONFIG_SHMEM + &zswpout_attr.attr, &swpout_attr.attr, &swpout_fallback_attr.attr, #endif From patchwork Mon Aug 19 02:16:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13767726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37DB3C531DF for ; Mon, 19 Aug 2024 02:16:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFA156B008C; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE3E86B0092; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A38926B0095; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6711F6B008C for ; Sun, 18 Aug 2024 22:16:29 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 22115A873B for ; Mon, 19 Aug 2024 02:16:29 +0000 (UTC) X-FDA: 82467380898.05.B8142A9 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by imf28.hostedemail.com (Postfix) with ESMTP id 0184FC000F for ; Mon, 19 Aug 2024 02:16:26 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hmUL5Qdy; spf=pass (imf28.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724033710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OPBEYLfNaOw1f4NX9PtVSyPiQ1Xr1wZvUu5S5eRpF9c=; b=zP/yVIENMEIsar4g70ctIWZHLPD80BDOWIr6SB8AjZD40EXnNK1uzzufqsZXW+NihxD0LR 3H9PdNZo6ASDHc9Qyr2DkNdvxp91hehQ03Z7v3i9Wo8Ay6+gW+CejJxyaFx6XxZw+RERYG mjiMzyBPTWjNF/n7QomEEIUAqhUA5Xg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724033710; a=rsa-sha256; cv=none; b=ZoUCe3/T3UBGiHyCUQ6s9wn6y/9q30JMq3V7vKDZbeyCXgsL84uZt/KRJWFAA6GgBOPiUZ S905cF0ZV3yh1/OTekr741AjDr3/KY0R+NHkakkOb7iG4wCi3u/624rcMF0mYKhgh87U0m 33jpd4La1yoF/BKX2YiyClCu8qxOUjQ= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hmUL5Qdy; spf=pass (imf28.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.9 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724033787; x=1755569787; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e4dJeIMpZ7zVkmYhUNqheK2yMfNN01J1RMy+uwUX+JE=; b=hmUL5Qdy2y1jb8jpOUBu3NrsUCd/sP8mR7/QozxCEbnlwkhYG1QEXyaV 7ijpVQ+jPOxny04ETH/CcQgd4nNoH3Vpjfjg9WkTLi7pumDSawMOvrnc2 fVW/ahiIRBDawL1jDIcmaiI1ZgwhxIe+/Jervp/v9H5LA8P4gNIByZ9pM 4pmcE+mZnHoGuEMcm543m/kz1d3C9juvLniIaZ5A2wCEw4TYZcB5j+5G2 7YRgUeJ2fEQDMESMLsQ9NSEHR0PuMgh19BAn5MSakB/wOnK2AY6K5s9qf hC9yThvKXttMgi6jw8o2nzv5nqOpXCvE9IGBDucS+5CVgCVHMwPA3XEJ9 A==; X-CSE-ConnectionGUID: sXHgo2gVRwin8m2BTdjqUg== X-CSE-MsgGUID: JbChkcpXSFG3BzLzZ+zFVg== X-IronPort-AV: E=McAfee;i="6700,10204,11168"; a="44782981" X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="44782981" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 19:16:24 -0700 X-CSE-ConnectionGUID: Q7fqm6iGRiq5vh7mK/HcfA== X-CSE-MsgGUID: BTCZqOyyTrukqBfPvkuwiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="64610280" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by fmviesa005.fm.intel.com with ESMTP; 18 Aug 2024 19:16:23 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org Cc: nanhai.zou@intel.com, wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v4 4/4] mm: swap: Count successful mTHP ZSWAP stores in sysfs mTHP stats. Date: Sun, 18 Aug 2024 19:16:21 -0700 Message-Id: <20240819021621.29125-5-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0184FC000F X-Stat-Signature: tw4c185uaigfr4a4jr6h5j961r9jxym3 X-HE-Tag: 1724033786-567999 X-HE-Meta: U2FsdGVkX1+iBnfn/PMdrteGiCN0syJFU2ScVGJcLw5qG3jWNAlEZ4+F8Ep0pPvHmsEs8g7O9Q0dtSgQwMaV7kyGbqto+E8WZwYIUlxP55+/FdERL11zs8f+QUMVjMD+AGOPCoBVTEZUkuiHGp1MLStEmwVV5SA18Yt/5UcCj2lUAXgDC8nO3v1LxCnwuWG5dp01C+w4zfxyuOh9DX7cQiAL1i2dQjaguPu8qoOJE/WxFlTgJmwg0fZTV61006/kdMGMA6GSOovxr96llYxp5QFy29gkkeJZHq0EvDi/lgSYGl04tvpEblkuzJuQyWH0fYLJN+pPOof4LLsicUg+6YMTBCdiML0fJdL+fXBxMkAICVRkaRyZE65ZXrbxdJHjcd4EiwykRLNn2a3WOolXahr4AYnIxPGKXk4ngVPzypKrwFNNwMRhFwziZMmW7gZbpvdd+ZS+H+HabvEuMyaSNTKc/vkDSaNOoEoi3D72Hv8992NwVubgKYOcy173NFHGQoFhO/BZHBJqQYbC5R97e2NTPNps9iXg/xEaWRHBBL3yrXsZ9+yNcqNwHag8LNS24a1JuyQmKG40XLX/A2CQjzDOqcUb5suGjlR+DwthhOITQ6SP6xrVXfcspuuxXrD2qDuQXb+LALRucIr/zN88l8ou2hxPxH1gkFVt9anu0dMGFGQudz6+ng/Al3ie8WI4n1gCh0Z6jSdrOWvTB3GMq1Inlk78FxMhhw8flbVPe/VWExeM3U8WRt6ktN3D5BvPr33UBD4DDkpi18A1mjz31RT0MC/8oKytmUDyEnURFmGgDonY+PHKIMPDsX5kXiYzyi8ISH2F/b+7hAsUVjvDbxvMZzEPtXst1X0GURcEau/dQJegt+NBge8PZOBwhav7Vu5VMNGutqJDzJTjDlMEMU4dKz7NqOgbQDeUPmcMwmv0d7IDePobmI+23RUvTCpsDf9KybOs4CDBP7A4DI5 +3cg/l+m M4EqgRD6gqUqeEvMZ7/gSf5QMTrzjYdgXN0d2KZmcMj4o15fnq1kSak8GqFnvStOo6JNhynhEi3vIYmMkfZoYSed9etJsrMcXpfqDluZW4+NWCOaKzy5nN9kR5++Fxd6llaiSbGxkl9LTO6/zp58a+OrUtw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If zswap_store() successfully swaps out an mTHP, it will be counted under the per-order sysfs "zswpout" stats: /sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/zswpout Other block dev/fs mTHP swap-out events will be counted under the existing sysfs "swpout" stats: /sys/kernel/mm/transparent_hugepage/hugepages-*kB/stats/swpout Based on changes made in commit 61e751c01466ffef5dc72cb64349454a691c6bfe ("mm: cleanup count_mthp_stat() definition"), this patch also moves the call to count_mthp_stat() in count_swpout_vm_event() to be outside the "ifdef CONFIG_TRANSPARENT_HUGEPAGE". Signed-off-by: Kanchana P Sridhar --- mm/page_io.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/page_io.c b/mm/page_io.c index a00e2f615118..ac5954149eb9 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -196,6 +196,7 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) return ret; } if (zswap_store(folio)) { + count_mthp_stat(folio_order(folio), MTHP_STAT_ZSWPOUT); folio_unlock(folio); return 0; } @@ -215,8 +216,8 @@ static inline void count_swpout_vm_event(struct folio *folio) count_memcg_folio_events(folio, THP_SWPOUT, 1); count_vm_event(THP_SWPOUT); } - count_mthp_stat(folio_order(folio), MTHP_STAT_SWPOUT); #endif + count_mthp_stat(folio_order(folio), MTHP_STAT_SWPOUT); count_vm_events(PSWPOUT, folio_nr_pages(folio)); }