From patchwork Sat Apr 13 02:24:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13628597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 470EEC4345F for ; Sat, 13 Apr 2024 02:24:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30EC26B0093; Fri, 12 Apr 2024 22:24:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BEDC6B0096; Fri, 12 Apr 2024 22:24:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 112486B0095; Fri, 12 Apr 2024 22:24:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DFE1A6B0092 for ; Fri, 12 Apr 2024 22:24:17 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 79EF51603F8 for ; Sat, 13 Apr 2024 02:24:17 +0000 (UTC) X-FDA: 82002914154.20.265EF3D Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf30.hostedemail.com (Postfix) with ESMTP id BC94B8000D for ; Sat, 13 Apr 2024 02:24:15 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2S3Ieg1K; spf=pass (imf30.hostedemail.com: domain of 3zuwZZgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3zuwZZgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712975055; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hZYUWNTiUhiZC6PVJ2X+TVQi5b4PqWXfNQ2Zo+Rl5yk=; b=u2sGCBs+JGGbUI9FuxulrYOMEFZvXTok12NLFc2BxtYZPz3q8i2NkmZb1K8bd1JYHqkgPd efZaCIExjSKAtqmcRRI/dNsNACYv38dQbbILePpTFjmaaS6QZZ7bCeHmfH6J2FHMjkeDeq URG4TB8GwLWgHlzQmXe7s/XjkmtRHV4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712975055; a=rsa-sha256; cv=none; b=x0IrBBh0zTH1jhCNI9SpaRRdYZYTJEU0TPYCJxv7iNmYhXh1yVa8iZdjsAdwoHorBIRZfN VEBaPDv4DSDGFTu+9Xjv4gRvA0ODrYxThkU8rH4e9iBZTStsfquEDSjkERj6O5nZrasDWo sclJyKPlo1C1JUJitkoFlcjVpfUfR4U= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2S3Ieg1K; spf=pass (imf30.hostedemail.com: domain of 3zuwZZgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3zuwZZgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dd933a044baso3039873276.0 for ; Fri, 12 Apr 2024 19:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712975055; x=1713579855; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hZYUWNTiUhiZC6PVJ2X+TVQi5b4PqWXfNQ2Zo+Rl5yk=; b=2S3Ieg1Kv+GklRX3izDWoACdFfScC3iFPdRirVggxNoE+cG15iSnA2rLMNc0Di3EwA 9ANKk1pnyxyU6YlpZUgRibn53kZqiqefPas26Cj3c38IpLHNHbxZJHeQnkIsJwDGlISg rbV0ltXKNiX6LCWvVlLyxBDCY8uLQ/hvOgT9pjD7h2zdLXhZurJjq6FtiMnh4hWdQH1I pO+v0J1eNNm04xEO21UN0B9lYtiO51xYes580l48ih7nBPc0liarbLs9L4ayiDdwEaRE sJG9osl8Gc/VWf7/yMIa5NYN1Oywo3nY94K8B0hCHXtFi6vgdX2oy3ZuCU8TrA+vo8sQ RJow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712975055; x=1713579855; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hZYUWNTiUhiZC6PVJ2X+TVQi5b4PqWXfNQ2Zo+Rl5yk=; b=Vxyc2y/P8YF9VTqKGGy3V7E/HZ/KB6vBrWhSZBBsTcecPxmv8xI7hGUiG+Ma9Y4B5j D9FN/wxzGIFsLO7k3VSf7UYri8GMwY9NveDAkA9mTl3g1sK55rxS1WxLGPukJDWov5tF y4h/w9vPPRhf5aBEbmkXkwTLW2cYqN3UDt4d20Qw9/9++9uWkqFHD6eA0czjuncQMeB5 02OWgTq+5Y9NPZT5rZjDzYLICQ+aGb1HL9RoM8ovOlApT/FSmpHL9oZ/CepYLS1CTlcL 0Y1QNnTUH88+N6O4dcqupfEz80xFSYbfgKejfJAH8f9peKrkni5khcll7q1P3MWB9SWZ BjAw== X-Forwarded-Encrypted: i=1; AJvYcCVjF+CdEzvbUGVDCFQ8ktawtUUVAIKbbo59kxkgmTTPn2s/vFfSZ0nm6zbkpd2d9McwQi7ym1kjboDbqjjNcQUvi5s= X-Gm-Message-State: AOJu0YyA0HXGGUgOTUXeDK4KrrJwtatAe0myAq1VLsTcuylgBzhlwa11 u524xvTe/+FxLkX6zojIrT4bqtqkOvh2hszdqKGs5+X3o1OmKaD41Q65IMB0RBQYvwVg/JGNq6M jG/S1cCSfdamiB5Porw== X-Google-Smtp-Source: AGHT+IGTrx1GnvC7Seu3eWAOb4tBFHfkiqWBD9kd67Bwq3v4nuuqOaFgb15AT9eRdbOLIz4wHqs3/FUeUlx91LlG X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:1243:b0:dbd:ee44:8908 with SMTP id t3-20020a056902124300b00dbdee448908mr1442080ybu.0.1712975054816; Fri, 12 Apr 2024 19:24:14 -0700 (PDT) Date: Sat, 13 Apr 2024 02:24:06 +0000 In-Reply-To: <20240413022407.785696-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20240413022407.785696-1-yosryahmed@google.com> X-Mailer: git-send-email 2.44.0.683.g7961c838ac-goog Message-ID: <20240413022407.785696-4-yosryahmed@google.com> Subject: [PATCH v3 3/4] mm: zswap: move more same-filled pages checks outside of zswap_store() From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Stat-Signature: 99669pq7hau8cenatm7y7rwdqhnrkoyo X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: BC94B8000D X-Rspam-User: X-HE-Tag: 1712975055-866064 X-HE-Meta: U2FsdGVkX1+wriET85wiZwWZ8WOUzmZkcrtIcp3qV7D8nNfdSmGbZXzC7MJUxPiMYfeTsATszTcIeLCS3uvVKFgIZBk/6fm35Gli5KT+t+/Ti1iuuzjNySHkEo3fD03G7UXCuM4VbMPWGXo6ci3F8lrJI/pjxb7D4g8x691wkfyTCVHfOySVIaM5d+8bVAQtL/pwVpKHCsk/SDKn9CNpfptrPozq09wrt5hElTgJYM/VvAhN+EeMVLqaTH6OPGQ5n93MW2vRPj8DaXi9uRKMroHcjUOO5RPQgEyP7vyOlDTUski0W9ScOdpHMrWxZfiHUhiJoCIvcZJLw1J1KaRbwF/9EXhO0v2FAABzT8mNZojPFolTVaYHe0CPApEt1AbOeWZIfnAKCJlWGFJwN9hbSr7bDppMYO/oqu3d+BrwsWbKZ1rI0uPvGGojfjkgoOEiJU3+Cdn0R13Do4tCYreRwQFrZBNAl+DiUsjE8rcfzxZVrFgQSdMb25C6/UUvoylnlhSa65RTqaH7ScM2+YCkLavJiJU5ZIxCW27T5S3FRd3k2qZIRA0Gdf6CTgaPytOvrvDQ/NfvrcYiO6N3HbkvpP0V+f1pwh0G1dL2QtsmO7uAcWClqBzry2chbOaO644jYinm/hRFhMbSEDo4uCN/wMewfvLMtLbApXho5oUjp2Y0mEvls50F+hMmy64vijgnPzDKuNgTgl0nN2ckpkU2wGVktmJfptbGMAc/gVJGD9ZqVIoaELdSu9sgWINJwgAKwv/BbBXjitOaYjj4ir21dkvM4NjdgNyfsbgh3roMjNXlWmu4Tcumn6BSwMHTyM4Dse7xhk/cEc4+m3t3dM6IqPKM7IZl3BqzkSLMTfSI9nRx7tEZMaeKaGNxKP5kzAt9CIKdMiNP+gp7uCBkoNb6SAcGyYHeOeNt1+e8LqnPm2eoL3yAJlCe60SemqJ/czuQ31p8CnfVse9tHy4aYqv rK+oSVCN qkytp9YQLLQvL6P3lUCIRJmNhRJpM/VIDHjaX0k3N8wOdE047Z1XyaM8vVswMR4iuPPWn1y2qZV9iPiURA0fIgdgZMcpXmSdtHbvaXMWrCGhNryggfjKUqE7YrlQkzC06/8gO7LZ5Iv9TPqRPVDwwHn/I40/fu06sSJMEc6WVZlpwZ9tAfuLQUCPDaoBBvvgjaCgrp8/B86fDLmre1a9Qk6A4cTsd6+00XoCwyFShbQzAFB3SHs1JhQSgQZxeKfrjwxxIuENBTj99xQJVVpx7T+cKK3A8lt8Jyu0W2UGQdHoJJwPaQNyuRrhYvpd/UMgmjYwvKgudQziBUxrad5PMNhMgZb9XiVzj6fVl87Idy0jJCcjPyIOS9T6eDExHZstPjCux0xBwzTI75wuk3uZ5jiqATwABs8Y2Y9dGS8lJZ+vO9WlvzvwdbVRdT0shPe2rzTv/zjm9slgXvsYGgHLVwRRyXcFMJNOP+NbEpSSl1+nXBS3XNhyCVUUZV2E1QPatl3FIGQbbSeXZZI+SvRF4u7zVCvN5blZGfb3thF5riLw2Xg7+QbQpoSnghgC8EHzgXvhbnpf7sDLu7oxYiTA8QxdJxXGKGSJhhHoODONPcdlWN3KpRrQCJgSEreeaf/Heqpwt9oRu5ALMVNCbPi+cc76TVgQFfu5xkQHp2owAm2lVCsQUGHP2ke8bUfbeKHAUrdQnzVQfVI4sYx8mH/iNUZ8Fw6Y8DhUhAYoq8UL5RGfDv+jImQw0VCK6sAmUIU4fKir6B6fawvRS+5c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, zswap_store() checks zswap_same_filled_pages_enabled, kmaps the folio, then calls zswap_is_page_same_filled() to check the folio contents. Move this logic into zswap_is_page_same_filled() as well (and rename it to use 'folio' while we are at it). This makes zswap_store() cleaner, and makes following changes to that logic contained within the helper. While we are at it: - Rename the insert_entry label to store_entry to match xa_store(). - Add comment headers for same-filled functions and the main API functions (load, store, invalidate, swapon, swapoff). No functional change intended. Signed-off-by: Yosry Ahmed Reviewed-by: Nhat Pham Reviewed-by: Chengming Zhou Acked-by: Johannes Weiner --- mm/zswap.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7cddbca3ac62c..f1d3204c604bd 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1376,26 +1376,35 @@ static void shrink_worker(struct work_struct *w) } while (zswap_total_pages() > thr); } -static int zswap_is_page_same_filled(void *ptr, unsigned long *value) +/********************************* +* same-filled functions +**********************************/ +static bool zswap_is_folio_same_filled(struct folio *folio, unsigned long *value) { unsigned long *page; unsigned long val; unsigned int pos, last_pos = PAGE_SIZE / sizeof(*page) - 1; + bool ret = false; - page = (unsigned long *)ptr; + if (!zswap_same_filled_pages_enabled) + return false; + + page = kmap_local_folio(folio, 0); val = page[0]; if (val != page[last_pos]) - return 0; + goto out; for (pos = 1; pos < last_pos; pos++) { if (val != page[pos]) - return 0; + goto out; } *value = val; - - return 1; + ret = true; +out: + kunmap_local(page); + return ret; } static void zswap_fill_page(void *ptr, unsigned long value) @@ -1406,6 +1415,9 @@ static void zswap_fill_page(void *ptr, unsigned long value) memset_l(page, value, PAGE_SIZE / sizeof(unsigned long)); } +/********************************* +* main API +**********************************/ bool zswap_store(struct folio *folio) { swp_entry_t swp = folio->swap; @@ -1414,6 +1426,7 @@ bool zswap_store(struct folio *folio) struct zswap_entry *entry, *old; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; + unsigned long value; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1446,19 +1459,11 @@ bool zswap_store(struct folio *folio) goto reject; } - if (zswap_same_filled_pages_enabled) { - unsigned long value; - u8 *src; - - src = kmap_local_folio(folio, 0); - if (zswap_is_page_same_filled(src, &value)) { - kunmap_local(src); - entry->length = 0; - entry->value = value; - atomic_inc(&zswap_same_filled_pages); - goto insert_entry; - } - kunmap_local(src); + if (zswap_is_folio_same_filled(folio, &value)) { + entry->length = 0; + entry->value = value; + atomic_inc(&zswap_same_filled_pages); + goto store_entry; } if (!zswap_non_same_filled_pages_enabled) @@ -1481,7 +1486,7 @@ bool zswap_store(struct folio *folio) if (!zswap_compress(folio, entry)) goto put_pool; -insert_entry: +store_entry: entry->swpentry = swp; entry->objcg = objcg;