From patchwork Fri Oct 18 10:48:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13841573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAA44D2FFEF for ; Fri, 18 Oct 2024 10:50:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 411766B008A; Fri, 18 Oct 2024 06:50:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 399CF6B008C; Fri, 18 Oct 2024 06:50:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2155A6B0092; Fri, 18 Oct 2024 06:50:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EFC0A6B008A for ; Fri, 18 Oct 2024 06:50:42 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CFA00121A05 for ; Fri, 18 Oct 2024 10:50:31 +0000 (UTC) X-FDA: 82686404220.01.4ABD913 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf02.hostedemail.com (Postfix) with ESMTP id 0198580009 for ; Fri, 18 Oct 2024 10:50:19 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bIe7KXk3; spf=pass (imf02.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729248494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q2+OQEgh0fCG5oZB9bP27eMnsujYhSchw3YP6zGdLsg=; b=wysqw2Qt2QaDDbSjbp2Cy+nIyOobUrlmh4S5umGxufpxlWWtu+Y6ea0VbtI71dyu79qd3g txmVJp2FLlYx2JeS2WlmFdMAdnsRCAV/fq9kOjCjEMcx+Nn2xiJFTBMaKMdVPQP6ssXANz QPLWSwHfW2zn02CWavTV/zr2z8qUUJk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729248494; a=rsa-sha256; cv=none; b=xwc1rcwdNt2tbnR7OdHjldtplQSuup7/AYrS4s4OyZY/Q37If5/TDRC38Uc32AoeZdrhJt CTZMAn/B/WKV3GEP+UEkzahstTMU5ZQEOgwlHo5Gf+yXRqE2iUnSubdjw1V0EfhcgpSohO rjl36kyUlZdWXhh5iCneRG+9c15F0Bo= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bIe7KXk3; spf=pass (imf02.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-7b155cca097so75147085a.1 for ; Fri, 18 Oct 2024 03:50:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729248640; x=1729853440; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q2+OQEgh0fCG5oZB9bP27eMnsujYhSchw3YP6zGdLsg=; b=bIe7KXk3owCrKMi2HzKtZNmm1DFBR8qy+TKCbRjc3Ve1OXJDmUy7oAQqPmf1RCvWrV 2PcNwLuj8q5ZZQJeUF4/ay9lyP6ewWgGGLo8ZEwpUbHFZlxy6B+oA//wZx3p6yKfBJd6 Bf8VzDFuDCy+FJ4bWuEIm5sI5TPGZpDKCl3/rkr6WxRSmKx2V3i5diu+tEeT+r8MKPSM kbd4p99WcucyDd6omEnMMm3ZxoQCEXTiRKJKoMrDbDHlZcfbw/7U7DZtcH0OI/a6jeFr YoOTaT0e8CyflS+0Ng1Z1yi4EKiMKFJdise3FL5RkkbYIgDm+DJ/jsDHYVxtQrRnnhFl Wfcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729248640; x=1729853440; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q2+OQEgh0fCG5oZB9bP27eMnsujYhSchw3YP6zGdLsg=; b=pKlNmPdail4nsERZs0pD28lCf9QHmTtHJ3Zn/XPcTF7FJMCIeJGhqo/C//meXx0jST rQOQIE3QFfhezPen3dPHpTdnc9/7wO5ODwnIavb0o8CPBuH3l8skYt++dcmk/6w+r6c+ M+VfVUepDkXAq2TDjrX9PIKF+PEbIEb+EOkRs/R5ab/70PGKibHFnSPUJWmlfAp/dZTJ dwQUmamDFg+qhQPwgDX9xp7lg9C83ENI0xBZmQLpY8LnLwKhLGTKPxlOGRpY2nWeyn8/ +hqA91sqHKYQ+S+z44k1nPmwHIhoUOVsMhodtVDlz8/PbCL2VV7/ZYGzO0Ttam0Xhf0k ROhA== X-Forwarded-Encrypted: i=1; AJvYcCWhvw4tziYAIIcfhuRY7OsEFkef3fOLsdOhn+HFjVLuRjE5ARntYFLos710QcX2FW+Hm46ON4XW8A==@kvack.org X-Gm-Message-State: AOJu0YwS3lQjSVoTh1CBlvbHKB4JE+KumisjZHHCRvFliOgr4SeIZnnP +kuocug6ThB2QfglvcUxdZpOzQdAseLpj7DFo5laOPaCJ/1ssJdL X-Google-Smtp-Source: AGHT+IExnPa9OX7aimLs3HML3KX6QH8HCRkuDhHElKsmz6WwfmgBG4EaxhA/mKXEbRMU3PnKMeQz8Q== X-Received: by 2002:a05:620a:4708:b0:7b1:3c19:a665 with SMTP id af79cd13be357-7b14d51d341mr914254285a.19.1729248639792; Fri, 18 Oct 2024 03:50:39 -0700 (PDT) Received: from localhost (fwdproxy-ash-017.fbsv.net. [2a03:2880:20ff:11::face:b00c]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b156fe5b6fsm57894985a.82.2024.10.18.03.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 03:50:39 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, david@redhat.com, willy@infradead.org, kanchana.p.sridhar@intel.com, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, riel@surriel.com, shakeel.butt@linux.dev, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Usama Arif Subject: [RFC 1/4] mm/zswap: skip swapcache for swapping in zswap pages Date: Fri, 18 Oct 2024 11:48:39 +0100 Message-ID: <20241018105026.2521366-2-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241018105026.2521366-1-usamaarif642@gmail.com> References: <20241018105026.2521366-1-usamaarif642@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0198580009 X-Stat-Signature: hubd9w3oxk8t4g8efcazsausz6qpw9jd X-HE-Tag: 1729248619-154767 X-HE-Meta: U2FsdGVkX1+vVQqnYse7PQ4rEtSkpWmqmpHYHKuU0gRfrQuZ95NwzNj6bmSSTEySsOUIPoB2pnEpvaFk7U0tLHU/FdFSNzAkjXiCy04PnknCFf0NIjPN7/ZC9E7AZm8V5bJK9obPnNJ+IfKuDRXI05NFMyKQGGLgJXmL09FIiB0cJyflHviGOMRfHrplzCK6WpWHP9GEquhZUL8+XaGbr3Dn0CDDMgMIW1s4lN7zjPraY/+ENdUusZ39Cozqo9exRocB0oz/3vrR5SynanVltbi/4XIcZF8CMCYEka4ECmCl6oY1ny8PpKYvCXR1+8iItXf1d+g3xs9PqYDz7HcWoxMGirtU2yakRqjVELxeddXCdwXjDuseD8zr/wGGbvTyRgpnI1eC9ihXSrswpWK/My5dCq/KX65YF+kCDPoeGyQRkyqezwS5XsYzrTJaQ736QIUjGwUD+Gt/49HKEDISdjyikznvkNqxCux8AXr71FGvN0iaY199byoaZMX0eOD0eCjXgOF8cIQZUWwBfCutB6f6wG3DtyNWpYFTB3i50Z1jrIJwtcOYV07+7ww712DH7aNanKcXJG+Li2PUrP2obKuRp+GN9Iq6kVdfzqehTrkBic5G5o7VH2PZ4/mZZO43BD3uCVKGFNS50O0VFjPfmRyp9OoDYd+4Q8BFm9k7EltwhbWVjyA+PYZEMx6eOsl2UC9t2KitcBv/yUHPYULti+Mo4NqxvCYLXfIeTWj7GeoVImG/viGuvHRt6jfnNbALIN17Vk8J55xxEXpRkCOk6i24hoo+FCzjJKNEFk92miNfqSpspRWJi34+FJj47urzZ2ZlR66OjM+AB4YAc1DDfVaiBxOQ1O8s376LEjnWTpWrSTqNop3xTHpsHcqdsu9RnDJcpVDY2j0+8u53ZLCcPObkk4RrJRWPO8jghf6Mj6gAD0eSnkiDigPkmc8Z7uOJfOm9TGsVXLcH4IX0aSF GN+p4VZP tdvwMdu/uEKweRbT3NIifFMCVCJJ+LnpMIdMwqGRtnqYbuo/kUsMOYjIRqitMxrOhhqJ8cXyvl7dXUXVbvqNpks0uLakQCrdbnC3smx5gbthG8enihzAtJKbsH3h+ZTNy0fd6ZpNjjfFSw/5L9FNLuxOhA1nvIuyUMKUKfVfCF4OXUlXFRzxQCXFHJvG9m9g1QNq24BLmPZyIJH27/i9uUHuoIeL1yJ/XEXW3wuc7dj3UU+v6ZnSEnYfmPoUz5sRYd+G1MoMmlDPbIYi4VAKtSDLEikWfmAMX7w01mauFdurLaPT3nNdB/7/GgmSgfDtd70QToUFZ1blMKDxRvtqPe33tLNuYQrbTQmTZMKlfj9Cj0qS8mZHbW5FiuwwQ6iRNCc8loNm2g9cY3OLYwwMw1nSvbvcgqjca44vB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As mentioned in [1], there is a significant improvement in no readahead swapin performance for super fast devices when skipping swapcache. With large folio zswapin support added in later patches, this will also mean this path will also act as "readahead" by swapping in multiple pages into large folios. further improving performance. [1] https://lore.kernel.org/all/1505886205-9671-5-git-send-email-minchan@kernel.org/T/#m5a792a04dfea20eb7af4c355d00503efe1c86a93 Signed-off-by: Usama Arif --- include/linux/zswap.h | 6 ++++++ mm/memory.c | 3 ++- mm/page_io.c | 1 - mm/zswap.c | 46 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 54 insertions(+), 2 deletions(-) diff --git a/include/linux/zswap.h b/include/linux/zswap.h index d961ead91bf1..e418d75db738 100644 --- a/include/linux/zswap.h +++ b/include/linux/zswap.h @@ -27,6 +27,7 @@ struct zswap_lruvec_state { unsigned long zswap_total_pages(void); bool zswap_store(struct folio *folio); bool zswap_load(struct folio *folio); +bool zswap_present_test(swp_entry_t swp, int nr_pages); void zswap_invalidate(swp_entry_t swp); int zswap_swapon(int type, unsigned long nr_pages); void zswap_swapoff(int type); @@ -49,6 +50,11 @@ static inline bool zswap_load(struct folio *folio) return false; } +static inline bool zswap_present_test(swp_entry_t swp, int nr_pages) +{ + return false; +} + static inline void zswap_invalidate(swp_entry_t swp) {} static inline int zswap_swapon(int type, unsigned long nr_pages) { diff --git a/mm/memory.c b/mm/memory.c index 03e5452dd0c0..49d243131169 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4289,7 +4289,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swapcache = folio; if (!folio) { - if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && + if ((data_race(si->flags & SWP_SYNCHRONOUS_IO) || + zswap_present_test(entry, 1)) && __swap_count(entry) == 1) { /* skip swapcache */ folio = alloc_swap_folio(vmf); diff --git a/mm/page_io.c b/mm/page_io.c index 4aa34862676f..2a15b197968a 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -602,7 +602,6 @@ void swap_read_folio(struct folio *folio, struct swap_iocb **plug) unsigned long pflags; bool in_thrashing; - VM_BUG_ON_FOLIO(!folio_test_swapcache(folio) && !synchronous, folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_uptodate(folio), folio); diff --git a/mm/zswap.c b/mm/zswap.c index 7f00cc918e7c..f4b03071b2fb 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1576,6 +1576,52 @@ bool zswap_store(struct folio *folio) return ret; } +static bool swp_offset_in_zswap(unsigned int type, pgoff_t offset) +{ + return (offset >> SWAP_ADDRESS_SPACE_SHIFT) < nr_zswap_trees[type]; +} + +/* Returns true if the entire folio is in zswap */ +bool zswap_present_test(swp_entry_t swp, int nr_pages) +{ + pgoff_t offset = swp_offset(swp), tree_max_idx; + int max_idx = 0, i = 0, tree_offset = 0; + unsigned int type = swp_type(swp); + struct zswap_entry *entry = NULL; + struct xarray *tree; + + while (i < nr_pages) { + tree_offset = offset + i; + /* Check if the tree exists. */ + if (!swp_offset_in_zswap(type, tree_offset)) + return false; + + tree = swap_zswap_tree(swp_entry(type, tree_offset)); + XA_STATE(xas, tree, tree_offset); + + tree_max_idx = tree_offset % SWAP_ADDRESS_SPACE_PAGES ? + ALIGN(tree_offset, SWAP_ADDRESS_SPACE_PAGES) : + ALIGN(tree_offset + 1, SWAP_ADDRESS_SPACE_PAGES); + max_idx = min(offset + nr_pages, tree_max_idx) - 1; + rcu_read_lock(); + xas_for_each(&xas, entry, max_idx) { + if (xas_retry(&xas, entry)) + continue; + i++; + } + rcu_read_unlock(); + /* + * If xas_for_each exits because entry is NULL and + * the number of entries checked are less then max idx, + * then zswap does not contain the entire folio. + */ + if (!entry && offset + i <= max_idx) + return false; + } + + return true; +} + bool zswap_load(struct folio *folio) { swp_entry_t swp = folio->swap;