From patchwork Fri Jun 14 10:07:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13698502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63615C27C6E for ; Fri, 14 Jun 2024 10:15:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D4016B00BB; Fri, 14 Jun 2024 06:09:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 332EF6B00BC; Fri, 14 Jun 2024 06:09:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D945D6B00BB; Fri, 14 Jun 2024 06:09:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9B85D6B014E for ; Fri, 14 Jun 2024 06:09:09 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 46A2FA0089 for ; Fri, 14 Jun 2024 10:09:09 +0000 (UTC) X-FDA: 82229071218.11.CADCB07 Received: from mail-vs1-f54.google.com (mail-vs1-f54.google.com [209.85.217.54]) by imf26.hostedemail.com (Postfix) with ESMTP id 81321140015 for ; Fri, 14 Jun 2024 10:09:07 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bYJ/PnLx"; spf=pass (imf26.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.217.54 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718359744; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=VWc3xb/HQ07XBx75U8zO/2sCP9wicZSlLE1I4NEfPds=; b=j7EajvvA8GIPjxtJkNyMTJL1ceCcNEyZtiBivmtxeNJ10guotdQCQ9cxTZxYIpY9Gu89Vs zbgTcwu8+7ewu6CLu1F4+AUTqvySdEXinSdbYmk8LsH6UFjdCJShG1jPkdAliByFr/U2so 18+VvsN56cqJbipFgG3YGe3QZCi7tgA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bYJ/PnLx"; spf=pass (imf26.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.217.54 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718359744; a=rsa-sha256; cv=none; b=oxz9HVNvhvss/zlIkl+lVP5307MnRsCU2d520QX2hMiUDfEpC2RgH81VuebZ4SzPGAG2bo Lk6Y1RZrDUFWT1hWE2Lzj+N7t1ayrseHgQBncxlNDZ29IjeA03ABCQDIvmqXNoXBJOAbTq QjreHJprW1UXkcOtoU0zFCDYDDrO5TU= Received: by mail-vs1-f54.google.com with SMTP id ada2fe7eead31-48c458b9aa7so726977137.2 for ; Fri, 14 Jun 2024 03:09:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718359746; x=1718964546; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=VWc3xb/HQ07XBx75U8zO/2sCP9wicZSlLE1I4NEfPds=; b=bYJ/PnLx7sldcTOsGhbB7WSjtR5yhMuAFPxNYF3+5xh1iQzO89h1xcSKn84z8kpZv0 4k5k0OOlp3kYF9zm6TbJwLenXadxa0OyKSUMeIvwdtqNaE2bbsbNyScJTQXV+FzB3DGU Vm/MwS95RU6Gwwkjq5mJ7XLEq2o9bT6HM61er9F6ETtgeMnN6nAq+xKPfTBckZUYS2sA x2d5VG7Ddhi2X3SEVrWzGARnA9zkerETxOl74+ebflJ8dsCIOepA5g3NTEv7R12g5gfT SYmStl3rTyrAMDzNcEtb6r6ci2fMO3flSBe7Pxs1NhviI5OO51TJUsy/8+efrIBfXlZY EZkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718359746; x=1718964546; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VWc3xb/HQ07XBx75U8zO/2sCP9wicZSlLE1I4NEfPds=; b=nGHNMXvy4MHTkc81FPMWEkeSeKv1Q+8aMbRoMhX8pREu5LiFEyPLHsY4uxQR0stu0j OTNO3N6gEvdsQ0IJqM4zZbDYWXfSE0M6LVPx97O+R1FqfEuPa/2NGiHRK4Rpc0nvqsHB uMjGw+stks/lQsiB8sUE1nxUhKRwBkqn21UKnJcyQJ0pRZhjRb+SRNK6xq5gq5FUk+hJ DRPQ4BvfDIS843LvPTdkYRK4rvL01u9+mUcOwHfD+75xDDaOy71vn3Zl0jmWTOJnd0wS dcNXo2X7ovOj6Cbmw0MwvgsbBpPUDayQp75YBJZCa8y8pemJSOxaGhFLCoL8vuH5gQ74 cpRA== X-Forwarded-Encrypted: i=1; AJvYcCV20Xf8at0GasewXS6toySrPPUtVFhC25AvmU5fcZtXkljQcrY8Rrt5fHZgoeD8qIM2nnc/jV/o232OGDVywFc2nYc= X-Gm-Message-State: AOJu0YzRo97VNNkVX8ouXmwCFZPr79e40exRMh10p3AeFMHcYimCeVwz EDxfkgL9kQMbsLf9beAzWuqYGsvhkGCr4B8o4cNr2JX1hcZAihVQ X-Google-Smtp-Source: AGHT+IFzbHBz0PdVxNeE2rB6k5bdThJ6O1EORN/j59be7zwxh0ZgAEI356Xg5ZdBQ2/cmO0wswbsIQ== X-Received: by 2002:a05:6102:9c3:b0:48d:893d:9415 with SMTP id ada2fe7eead31-48dae42ac06mr2225891137.32.1718359746320; Fri, 14 Jun 2024 03:09:06 -0700 (PDT) Received: from localhost (fwdproxy-ash-114.fbsv.net. [2a03:2880:20ff:72::face:b00c]) by smtp.gmail.com with ESMTPSA id af79cd13be357-798abe6b4easm129470685a.123.2024.06.14.03.09.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Jun 2024 03:09:05 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, shakeel.butt@linux.dev, david@redhat.com, ying.huang@intel.com, hughd@google.com, willy@infradead.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif Subject: [PATCH v5 0/2] mm: store zero pages to be swapped out in a bitmap Date: Fri, 14 Jun 2024 11:07:04 +0100 Message-ID: <20240614100902.3469724-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 81321140015 X-Stat-Signature: yg9et7ohck77g444a87ft1md7x1wj1en X-Rspam-User: X-HE-Tag: 1718359747-574504 X-HE-Meta: U2FsdGVkX1+ueX47W7wJDhTCDMcID/9OqMdmo5hEQCKc+JsIa/rMOQ/DBuFiAxx0oMsVhZlcPYHirl/aqfUgVHVADT2LOPQ+CtvuqZ4xYVNXQwR6530pxCShEJFVeto3W8C9YJlnF1rd5iAzab3WzDH4MXGHbOQ9kVs1zA89umNWDOyGoHl8BmBTygd5OFJT4M6sIVzxbpvk4DtnlKyvxPZD1Rmef2x2w0Ke/3SO2Ez3rCJHTQ0K0CuHvObD7HSuk4a0Ejre4ovpHzU4yK87L4nZXO+yrpEkKf9yA/EWWNvp1MoAWhzqY6HVoD7/Bqe03DERCUTucjtHPFMJQUebTY6+xLlEWVTn3aO7drH6xUQLrLlCYBAGSVvOxFiur3qkQTbUwYi5MhIyNk3UW4Nkd0n3rPDnH81rnJqvDW/sYAat7i0ECeLONm1A7izCBMqYxXGIGPURENUB01Qo3TJd6QEp7/4OAowe991514acubJqrmrZJaXoG42Z814vgo6KVJAKvtSKmpl9jjb6rrfB4UX8NQiZ/LUpMdTnd7LSOfUtLCundtyWYpCAjJLH46BbFGbtkjORpA+cGmUp6vubEpCTlebQyBiGG4nrO68SntGshGaoKypUTxKi9LCuqYUwlYjHOpcqCXN32zizlA8VgsyudwQX/l/uXuhykBKOExmYWw02PcgAT+wmbqpC3Y4zN5Es1MKOwISwvulU//YX7b2cQJHVn0n1l7yUciDzP9S4Av+WrpPWkz04OBjXgxrfQx4UHqJv+/vtItLz5Jw4djC9yg8GS/e2lhWo83fnsUoAbkrsfCTy0O60iRn6QM05/5OOiitZ0KFXF2DOq2kjnXb7PGs1sWQbkuF7dKBYmLg+9Vb95L75Mt1dgeHSB3Kl/2iqoO6MPoIulFOYsX3BYnIoZ9Qp6Bn8lZuMxOrIwM6smjbFNJ/1krUVu9p/i7+GQtifkA6Zw9Y2Bkf/+ZX pob6X1aJ UZ3Eoom3NwCll0J7oNHlIMia7YUwBVD49Ck/PO+Lx0xcN6DAasVKfh2Reml2QJxGyyKUI95vWszWiqVwioTeEXabnyF9LXsctbpn/0G79i8jPER2hF2npTIk4n11iBuW+Kl47UFUMvT8k7cSteKwsULYhDvoAqNsX1INlGQeCp1RHbLlZGL6ZsiSZmJkVN9POKGKJJq3SXPNL9TfwSbu8ODzmiSw52CNn2usw+3h+/qM+rrRIrTFSrUq5NxS1ZBzHB+8djgzEmXlQMCYquTXlTW5l/+7Yok0BUxauJ9iP5LQWzgicOKox7/IYTRq//jEQTWztXbpJaOLpCgCO1Oa2hcE/rsrybSinvsOeHP4Hw7i4wIQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As shown in the patchseries that introduced the zswap same-filled optimization [1], 10-20% of the pages stored in zswap are same-filled. This is also observed across Meta's server fleet. By using VM counters in swap_writepage (not included in this patchseries) it was found that less than 1% of the same-filled pages to be swapped out are non-zero pages. For conventional swap setup (without zswap), rather than reading/writing these pages to flash resulting in increased I/O and flash wear, a bitmap can be used to mark these pages as zero at write time, and the pages can be filled at read time if the bit corresponding to the page is set. When using zswap with swap, this also means that a zswap_entry does not need to be allocated for zero filled pages resulting in memory savings which would offset the memory used for the bitmap. A similar attempt was made earlier in [2] where zswap would only track zero-filled pages instead of same-filled. This patchseries adds zero-filled pages optimization to swap (hence it can be used even if zswap is disabled) and removes the same-filled code from zswap (as only 1% of the same-filled pages are non-zero), simplifying code. This patchseries is based on mm-unstable. [1] https://lore.kernel.org/all/20171018104832epcms5p1b2232e2236258de3d03d1344dde9fce0@epcms5p1/ [2] https://lore.kernel.org/lkml/20240325235018.2028408-1-yosryahmed@google.com/ --- v4 -> v5 (Yosry): - Correct comment about using clear_bit instead of bitmp_clear. - Remove clearing the zeromap from swap_cluster_schedule_discard and swap_do_scheduled_discard. v3 -> v4: - remove folio_start/end_writeback when folio is zero filled at swap_writepage (Matthew) - check if a large folio is partially in zeromap and return without folio_mark_uptodate so that an IO error is emitted, rather than checking zswap/disk (Yosry) - clear zeromap in swap_free_cluster (Nhat) v2 -> v3: - Going back to the v1 version of the implementation (David and Shakeel) - convert unatomic bitmap_set/clear to atomic set/clear_bit (Johannes) - use clear_highpage instead of folio_page_zero_fill (Yosry) v1 -> v2: - instead of using a bitmap in swap, clear pte for zero pages and let do_pte_missing handle this page at page fault. (Yosry and Matthew) - Check end of page first when checking if folio is zero filled as it could lead to better performance. (Yosry) Usama Arif (2): mm: store zero pages to be swapped out in a bitmap mm: remove code to handle same filled pages include/linux/swap.h | 1 + mm/page_io.c | 113 ++++++++++++++++++++++++++++++++++++++++++- mm/swapfile.c | 15 ++++++ mm/zswap.c | 86 +++----------------------------- 4 files changed, 136 insertions(+), 79 deletions(-)