From patchwork Wed Jun 12 12:43:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13694919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8BF2C27C53 for ; Wed, 12 Jun 2024 12:47:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34CF66B0092; Wed, 12 Jun 2024 08:47:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FAD96B0093; Wed, 12 Jun 2024 08:47:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C38F6B0095; Wed, 12 Jun 2024 08:47:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F03516B0092 for ; Wed, 12 Jun 2024 08:47:55 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A7070141757 for ; Wed, 12 Jun 2024 12:47:55 +0000 (UTC) X-FDA: 82222213710.29.5C8764B Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) by imf19.hostedemail.com (Postfix) with ESMTP id E87541A0003 for ; Wed, 12 Jun 2024 12:47:53 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DjnKPm6+; spf=pass (imf19.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.128.172 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718196473; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=b9F+Cd+tL8ekTIBJ3rfuDvhFdQnCYmwqNON6SdmRqk0=; b=Zcf4DaOArF55dEuehDdO+qzryBnr2KO8i0odfS6kJUuvycFcNorVgH8qMdcWoZT+RsLoxz FtrtXWH/S8TxSBUPp2VwiVv4LlLIyhvwEhtzWgu0Y5Y3WIju/GPncxsfiWweYRn2SMIYJQ xFGg7TNqxfE5TJvc7Hkp2eiYvC9s508= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718196473; a=rsa-sha256; cv=none; b=Lr39jqH22lw/3foKmJeXRe2BrmcWRfLrKI/cB8YUQf9NOfvfEsTNUOR0vlVlzND1DUyfcs 2h1W5WVsnAnZHuEmX1ILK3eeA/l9A6QJwDA4mpnk3N7I91qTyu+5GXgd6gAd5ZqwDKQuux 6nKU1Hq+w/pfCNPV+lvclm4NQg7B8Xs= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DjnKPm6+; spf=pass (imf19.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.128.172 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-62fffd5d36bso5281267b3.2 for ; Wed, 12 Jun 2024 05:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718196473; x=1718801273; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=b9F+Cd+tL8ekTIBJ3rfuDvhFdQnCYmwqNON6SdmRqk0=; b=DjnKPm6+g+rmu71XVAkzFNLnzVzY6yCXd5Eaebv+Jxs42ztnpwnIzY349O+qMsPydI WHd6+ht+9RxbK28h077cNNx7eARSJz1M6HC8sbmclOFmyIDu0LtzkaxBvqKOhrrYWd6D MeLWz+EVB98bp+nKsQ8/JEpqRlBEc1IrCzTCiW6SBctsm7m9JWCdi5RwNNxh0efKJqT4 DgYwU9arkuK7uYkkVVrYlC3zQghE+cvM5+dFfbUV6MijhoHuUVDC9d3sacVoIRB7szn4 ajUFg7CXj1Mkvmhv4MYxRqpZ3Rg/X/9uKl5/uz2f2jyKSR/Xkw3N+SYBaw31JJMD6EUm fbZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718196473; x=1718801273; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b9F+Cd+tL8ekTIBJ3rfuDvhFdQnCYmwqNON6SdmRqk0=; b=LlRAbxr7uVhkMjk439al8QBWw1q97cTXhi50X71XmiGrBKFqPilHctk6jOmUt+JdRM S7EjzJJrX50xC0zyB9/XxUifjHjG6b8fqGHXW8Af17P9iRI5Ei6qYXbJm5lhm/pSonq1 Nn0Oni82BvpVlq1wuTi3bVpuQOFfw5UUncTie/8t3eYONIwH/ORsn+VxlOEqjFwd1cb8 zRmbfexjnqC519Ey1xrV6Wft1yGYKy6Yc7RR3VLLC+hs+b7worWP2OWpmYpxsRBobnM6 5uvfdF0Ag5UiT8Zsxh7v0NJF9pABo2zvIzL3pC3M/kaHuO8dLexxk7/HkBo+vSax5Sma jcmQ== X-Forwarded-Encrypted: i=1; AJvYcCW0uSK0J+EH/u982DNbfh9bn8mmeIFaHjwunkSFCPjeCO9RWKPef94WMQ+P1x0/ZVTVOJzS7x3R1QHFWpb8O0BAMLk= X-Gm-Message-State: AOJu0YzpScFx6FOTLOjIIk8liFWREWs7qbgYEn4NglbG5936uG4N5n1y JuYNp9DCFu1oP1yrnZaQEeWVA/kq1ZA84J8qhnU2+ViIulP86rM+ X-Google-Smtp-Source: AGHT+IFYufT33T1Uwmyw4Xl+z3+PXG7RCsDVkLNGUSJ+AUOrxPKBRCeoBLQ0B6MfLbwtpM8TJSwhyA== X-Received: by 2002:a25:df48:0:b0:dfe:4fb1:7b4a with SMTP id 3f1490d57ef6-dfe65f7414bmr1556605276.8.1718196472842; Wed, 12 Jun 2024 05:47:52 -0700 (PDT) Received: from localhost (fwdproxy-ash-113.fbsv.net. [2a03:2880:20ff:71::face:b00c]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4406075025dsm34780511cf.85.2024.06.12.05.47.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jun 2024 05:47:52 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, shakeel.butt@linux.dev, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif Subject: [PATCH v4 0/2] mm: store zero pages to be swapped out in a bitmap Date: Wed, 12 Jun 2024 13:43:34 +0100 Message-ID: <20240612124750.2220726-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Stat-Signature: gjkxe5pmncfny57qyh5bfrgyzdhbzzpt X-Rspamd-Queue-Id: E87541A0003 X-Rspam-User: X-HE-Tag: 1718196473-249021 X-HE-Meta: U2FsdGVkX1/e8eytWjrFGFu2lV+CAcEvUjSGXL5KKirwODDF6h4sJIz7/wA/yvNEgS10haaZ74PGYntJNOX1shYUjafEgGjbz/Zx1ZztKQv4M7dGlxjR6LoimW4TZpQ8QPCfQ4a0rqtadCUk/3dzrCsmLZBykR/muVfP7uHl3AnUhZNALO65SF8Bw2U6mR0lGupBHcpj+PJs/ECo6oRg+OmCEHiwCGRSFsLgYVQax9DCwk5t07zSSLIyEURMGDW6QDJ4IgkjB1ci/10uJ+Cw+5Q0gMQshS7Me8EqCE4Psp+nPG64y/wMLfrgK+qADUfiJvx1ufMettwRILbLVjIH7Y/iSm9tiswZKmDkaqWY+Y4RIGSuWxkreABWrNTneOgSKKfxRg3G7ptDbN+8xwFUyMdpOfgsa4JwXx5nWr+ASU84D3SPIrpAnB3m6npiSqwkbsTuDDH1B7pz+cX+Gy2qKR3MuGxVQXU5p9Ha1IwxKZa/saOmavNXHwOHD84GZx0sx4b2ZLMKThW2ypAotR/RtkQP/0P4nLOLjAkw+iVxsZInKV53JU61NLrEeYelrQzBbVgqssFThulLAvLsjP6qQBEtTm+V74rG4XdHn3N4CB1AUjts+yDH72POXWQh02onf8Ver8g1FaulOzcghNjChN8vJBVWTgoEedV2Rn2ExQhmhSzcAb+XyCYgFUB7PZpuL43P7awO0cbI9QTV2oWVw+PhJM2rypTOhF7SOZMi7FqypqdrlVdOFt6uIK72DfGywa7PlC0v+5rXO3kATWtLGu9ANaYiLFu7Z5QKmHyXs84A1d7PXxRX/5IuyAsThd+STE/Og37Kb7omcK62OR21uAXzjWNkd3e5uacmGEonBbA3DMC8YhjLCyNiJDJvSqRYWUXF2LiPuHVXBdx9NcQqTDXEpNZraQ/59HeFZLIiWhHGNPAgGdDxBh5OsT6C0erXtJ7QZAB13kuxz6RRVF3 QH1jD9g3 V7PpN5SfwRSefkMsUCnyppts5RzvR1m0ArxPpna/XWZWrB8lrb4fwzI05+KONxy0yGtp8w8ymlW3BEsBRnOSyMd3/k+c1zIbWtOL64QBnIpsdhL3Q0TaQjbGJ1arVE5UPHjie4lX2p745Z4C+s2r+P3v0HVyDXA4g456Y5bANu9DKz5bFt/eXFF3TAuh5s7ptSPfmJcTEo2ijiFgJBPbtWsXcDF1YnE3UXgOTQm/B6l5+wWp71IW/B0e7QtnGIDsB3/VwP5e+hjjX7HeQT/V1NN7EViRHLoN7Vbs9KIs0qizYoyEJfZ/7qt50i9OQDStdl5ljNuRC9eREfjsQ3D2ON6MXWK762jWtJIxflR8zgjK2gZo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As shown in the patchseries that introduced the zswap same-filled optimization [1], 10-20% of the pages stored in zswap are same-filled. This is also observed across Meta's server fleet. By using VM counters in swap_writepage (not included in this patchseries) it was found that less than 1% of the same-filled pages to be swapped out are non-zero pages. For conventional swap setup (without zswap), rather than reading/writing these pages to flash resulting in increased I/O and flash wear, a bitmap can be used to mark these pages as zero at write time, and the pages can be filled at read time if the bit corresponding to the page is set. When using zswap with swap, this also means that a zswap_entry does not need to be allocated for zero filled pages resulting in memory savings which would offset the memory used for the bitmap. A similar attempt was made earlier in [2] where zswap would only track zero-filled pages instead of same-filled. This patchseries adds zero-filled pages optimization to swap (hence it can be used even if zswap is disabled) and removes the same-filled code from zswap (as only 1% of the same-filled pages are non-zero), simplifying code. This patchseries is based on mm-unstable. [1] https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d03d1344dde9fce0@epcms5p1/ [2] https://lore.kernel.org/lkml/20240325235018.2028408-1-yosryahmed@google.com/ --- v3 -> v4: - remove folio_start/end_writeback when folio is zero filled at swap_writepage (Matthew) - check if a large folio is partially in zeromap and return without folio_mark_uptodate so that an IO error is emitted, rather than checking zswap/disk (Yosry) - clear zeromap in swap_free_cluster (Nhat) v2 -> v3: - Going back to the v1 version of the implementation (David and Shakeel) - convert unatomic bitmap_set/clear to atomic set/clear_bit (Johannes) - use clear_highpage instead of folio_page_zero_fill (Yosry) v1 -> v2: - instead of using a bitmap in swap, clear pte for zero pages and let do_pte_missing handle this page at page fault. (Yosry and Matthew) - Check end of page first when checking if folio is zero filled as it could lead to better performance. (Yosry) Usama Arif (2): mm: store zero pages to be swapped out in a bitmap mm: remove code to handle same filled pages include/linux/swap.h | 1 + mm/page_io.c | 114 ++++++++++++++++++++++++++++++++++++++++++- mm/swapfile.c | 24 ++++++++- mm/zswap.c | 86 +++----------------------------- 4 files changed, 144 insertions(+), 81 deletions(-)