From patchwork Mon Jun 12 09:38:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 13275957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3740C7EE45 for ; Mon, 12 Jun 2023 09:38:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2AE578E0008; Mon, 12 Jun 2023 05:38:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2152E8E0007; Mon, 12 Jun 2023 05:38:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 088F08E0008; Mon, 12 Jun 2023 05:38:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E32F18E0007 for ; Mon, 12 Jun 2023 05:38:31 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BBA6F1C75EF for ; Mon, 12 Jun 2023 09:38:31 +0000 (UTC) X-FDA: 80893595622.12.DAB8009 Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by imf05.hostedemail.com (Postfix) with ESMTP id D6C8410000C for ; Mon, 12 Jun 2023 09:38:29 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=sbtrlJfj; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of cerasuolodomenico@gmail.com designates 209.85.208.45 as permitted sender) smtp.mailfrom=cerasuolodomenico@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686562710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BtDbY3geJk5ZFtkgOT5O5EUWsoo6AICXqA8Xr4TghOM=; b=Hk5UBGg30q2hTCgSbbSTc97LQcHiAL/vqy1Jd0DSDLieSl+mFTERW1qxFEP9Othfx+LepA nlzqlEIpiu8gCQr3d6nt0rCh8gQ06E1SsdCG9DbD5VUWz1ckV8x5uCMiPFxB0BSZoR+Ls4 vkOfSd+xmrWCjlT5M7GN8JBFwZV9Cv4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=sbtrlJfj; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of cerasuolodomenico@gmail.com designates 209.85.208.45 as permitted sender) smtp.mailfrom=cerasuolodomenico@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686562710; a=rsa-sha256; cv=none; b=fBcKTJqtR2uHbyySHAewOqcv3aJCrN2AcREF5UivQYdN9QqAPDd7N9NDrwY1cgKFBpnQ5s ACuQ5dknfc+hhWghK8i9yqPhuVH7U7MbYiYTopb71BhQ88H/tT5qpSC4rLeTiRAmbuqF/B 4hdMw22LtLqAl6W0TF7yLYba0dlcgAo= Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5183101690cso3119468a12.0 for ; Mon, 12 Jun 2023 02:38:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686562708; x=1689154708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BtDbY3geJk5ZFtkgOT5O5EUWsoo6AICXqA8Xr4TghOM=; b=sbtrlJfjq4iOA2iKxVvlVUOSJqU+wWrzWL5qkr7NPaCf0qspebF+DperUWmXtMsLN7 awsXcdmmd3l5lMRpE1uWm4Giyfpswpz59osjNefCCyHA+bAac/onBHIcr7U7qMwjPYPn Y7held6fOw7FIBMb+tQOwpsJtbCGa9QQPho8tZFf1E97/Tmo1q9+MCfnzmdVYUHpWBX/ 9jjsSc9HNbCXfd+48kzRAY23T92961bkPMF7VoN50AjcHvTHkQ7Q+vTchaFUC3bbgbpK FefZWzCUGcr60mNZWhF4mhpksv5TtnPksEsoKqaTkB5qzYsVkIKSRxIsFKaQ2VQfvt71 mF5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686562708; x=1689154708; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BtDbY3geJk5ZFtkgOT5O5EUWsoo6AICXqA8Xr4TghOM=; b=lZqWghCrIgEGvt/VGHKodtlNjemVa2r3h1a6gdXM4Cm/yFOImalkye1IdJLIY7houS YHKhh8iWD4i/SMD9lmDWs0w3lmYYK1Ej/lKXUrG6VCOLuK86zuNvYys0N14gJYeMwz6S JryN4f5BBZd+aPNB2AsSCSOl8l38/oabW4527dCFnq4adzQELN5GtN//ZkXcgOP3HYo4 EDPrN+CWwZrn5eBkNtcIgT1w3ePu5Rm8QTYSD5qAVaUq5E6OEkK5Yrd1I2YpQjkUWpmr 5Dfjw3AREoq5UGJNzVI+zQl42MtuKGnda3JukEsxFBY18rqcDh3L/Hzq8F8K9PD/qIGR bW0A== X-Gm-Message-State: AC+VfDyjREKac77WFBvI7C6Xo/fCrTaNqc5asX5p3V7UbobywWhalMDD v3VPWusWfxvrO8iVr+dUlOs= X-Google-Smtp-Source: ACHHUZ5jdk574K53jAuDd36SfnD+TsI8d0fI044BN7IUN/jGQusjkNJ8HZapcVM3ykunUY7a3uRgmQ== X-Received: by 2002:a17:906:7953:b0:973:92a8:f611 with SMTP id l19-20020a170906795300b0097392a8f611mr8867651ejo.31.1686562708286; Mon, 12 Jun 2023 02:38:28 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-212-171-43-8.retail.telecomitalia.it. [212.171.43.8]) by smtp.gmail.com with ESMTPSA id ce23-20020a170906b25700b0097887b68c17sm4951358ejb.98.2023.06.12.02.38.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 02:38:27 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [PATCH v3 6/7] mm: zswap: simplify writeback function Date: Mon, 12 Jun 2023 11:38:14 +0200 Message-Id: <20230612093815.133504-7-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230612093815.133504-1-cerasuolodomenico@gmail.com> References: <20230612093815.133504-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D6C8410000C X-Stat-Signature: zft4dia7yzonqqg1zbno3aw97nnzwceo X-HE-Tag: 1686562709-871031 X-HE-Meta: U2FsdGVkX18zXzB2puV4/M5wTLQAe2v5ZTy/I3Zr4kbXJVllJAyGOfBijhXJ0mVeDR1MzisWTY7P8bkDlrN5OPViIbENSW2pv2IsSy2k89O2mOUr6KHOQscnhif0I5AhVY/2S/2knvX4/64w6tLJWB/KERH8UZdpl8Fae5nPzt3kUGZOKNsrAn/1ehw3rbe7lol8zp50cyRjtTmjeOcYdumjO9jkwFQTK6P5eTJmd7pmR7KRwDuj9iG1bQxfF/aHTt+mc7hNY1fGk/68exJXcR4tR99FTXU87IBZLl6CMHAvn4XWlRd74b3sOM9Medox0Duy4BS4/xBgdZGWnMHlOj9UPP7sqFWd3FJHle72t1wX2DU1jygXVj5GOtNootUz9IB0P8UQxC38QCAJz3Vh78xp2+1OvoQR3HijzsOUO0OWXRFogfblGN2LbGfS5XdiotEeou2Ag3nRnJIYAhFdsHnEJMgWLLA9/VyO7g35t/BFYz0pDq5lYTUKtPBWMMB0WAbnk7SjCKvbVS7/NsxxOfmOM3xqf7T0FHLkExYAQ675vXFWlr+lNF/IFw9xjS5rSpVkiQYUhQ8hr3GMgl1FWqXMPqevN0rCXxPO2Qmavps4Xro5Oyv7NLqlLw5OOh9xKN1OmnUA+HdF9dl8XsMgNZ7FS9Suhe+0SjmoO5TIKfH/tLIXzVFVo4itEgEESMb9oc/18QZez++IkbVW3inl00SMtdZdPQ2LKSfbf/uXdv0mC2TKKFwV5PWQVhiDDVgevlnwCOxQOQz9voOPhygK87Ht6JSa3iHmuOa1g9JtXDnAAVPko1ODkZ6SzWWgZztk3404hL6QVstUMu4YXAkQv+W/plzy1903zvc2sFPNHakeI/AciN8XWzwR/mSAvNvzsIXKLJTm0hB5DZJ7bEIETc8mfN+lU7Rctz6Qr1QNHGR1G0cWFZa8yYCOl7SwAiBhlnuIQ1hhFYurIccM3Ig VD5mlHWT v5dtI+fuU/AgSB1Cj9yXO1zudDwmhpZFjKJffsJEMSVPkcFXGLYaSFCaBv9MnValT8D/Jg2hMiTroB5Q2Q2ZFKzXAcBq+0+Vf5EM72VqpKUV3ERJsAs9MJKdIhnmYwmRlVS0w1uhq0348SfMt7sM6gpwSV4poHI5dH/3am23P+9vQ0PgndWcgR8+PrrLRcC8Nhj29dl62pEhBBSKWRyNez2xtrlyI90hsoDB1F4dW+L31NPWALXoEvkHlPdClLWBmLvfTachN/cz8fk4vXsbCqUBCPqOotEKAVmmsFYKKLwzRssnJ8qcd16n36eR8nOCeGPPowWbMm/gtV3erd/u7B2Dj8SmWOusyLNVWGy5tizLfulmxNXpVPsNZjM8+UFVkvpxG6x8hBvDJoukImTUJ/uCkVixmTt/LtxGgTaWqivnAEV79OA7zuJpW3vyvMArDD8g6QeY7qSAL/Ycvh1DkgkNvWA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: zswap_writeback_entry() used to be a callback for the backends, which don't know about struct zswap_entry. Now that the only user is the generic zswap LRU reclaimer, it can be simplified: pass the pinned zswap_entry directly, and consolidate the refcount management in the shrink function. Tested-by: Yosry Ahmed Signed-off-by: Domenico Cerasuolo Acked-by: Johannes Weiner --- mm/zswap.c | 70 +++++++++++++++--------------------------------------- 1 file changed, 19 insertions(+), 51 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index a4f8c20e161b..3a6b07a19262 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -254,7 +254,8 @@ static bool zswap_has_pool; pr_debug("%s pool %s/%s\n", msg, (p)->tfm_name, \ zpool_get_type((p)->zpool)) -static int zswap_writeback_entry(struct zpool *pool, unsigned long handle); +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, + struct zswap_tree *tree); static int zswap_pool_get(struct zswap_pool *pool); static void zswap_pool_put(struct zswap_pool *pool); @@ -635,7 +636,7 @@ static int zswap_reclaim_entry(struct zswap_pool *pool) zswap_entry_get(entry); spin_unlock(&tree->lock); - ret = zswap_writeback_entry(pool->zpool, entry->handle); + ret = zswap_writeback_entry(entry, zhdr, tree); spin_lock(&tree->lock); if (ret) { @@ -643,8 +644,17 @@ static int zswap_reclaim_entry(struct zswap_pool *pool) spin_lock(&pool->lru_lock); list_move(&entry->lru, &pool->lru); spin_unlock(&pool->lru_lock); + goto put_unlock; } + /* Check for invalidate() race */ + if (entry != zswap_rb_search(&tree->rbroot, swpoffset)) + goto put_unlock; + + /* Drop base reference */ + zswap_entry_put(tree, entry); + +put_unlock: /* Drop local reference */ zswap_entry_put(tree, entry); unlock: @@ -1045,16 +1055,14 @@ static int zswap_get_swap_cache_page(swp_entry_t entry, * the swap cache, the compressed version stored by zswap can be * freed. */ -static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, + struct zswap_tree *tree) { - struct zswap_header *zhdr; - swp_entry_t swpentry; - struct zswap_tree *tree; - pgoff_t offset; - struct zswap_entry *entry; + swp_entry_t swpentry = zhdr->swpentry; struct page *page; struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; + struct zpool *pool = entry->pool->zpool; u8 *src, *tmp = NULL; unsigned int dlen; @@ -1069,25 +1077,6 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) return -ENOMEM; } - /* extract swpentry from data */ - zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); - swpentry = zhdr->swpentry; /* here */ - tree = zswap_trees[swp_type(swpentry)]; - offset = swp_offset(swpentry); - zpool_unmap_handle(pool, handle); - - /* find and ref zswap entry */ - spin_lock(&tree->lock); - entry = zswap_entry_find_get(&tree->rbroot, offset); - if (!entry) { - /* entry was invalidated */ - spin_unlock(&tree->lock); - kfree(tmp); - return 0; - } - spin_unlock(&tree->lock); - BUG_ON(offset != entry->offset); - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1121,12 +1110,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; - zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + zhdr = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); src = (u8 *)zhdr + sizeof(struct zswap_header); if (!zpool_can_sleep_mapped(pool)) { memcpy(tmp, src, entry->length); src = tmp; - zpool_unmap_handle(pool, handle); + zpool_unmap_handle(pool, entry->handle); } mutex_lock(acomp_ctx->mutex); @@ -1141,7 +1130,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!zpool_can_sleep_mapped(pool)) kfree(tmp); else - zpool_unmap_handle(pool, handle); + zpool_unmap_handle(pool, entry->handle); BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1158,23 +1147,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) put_page(page); zswap_written_back_pages++; - spin_lock(&tree->lock); - /* drop local reference */ - zswap_entry_put(tree, entry); - - /* - * There are two possible situations for entry here: - * (1) refcount is 1(normal case), entry is valid and on the tree - * (2) refcount is 0, entry is freed and not on the tree - * because invalidate happened during writeback - * search the tree and free the entry if find entry - */ - if (entry == zswap_rb_search(&tree->rbroot, offset)) - zswap_entry_put(tree, entry); - spin_unlock(&tree->lock); - return ret; - fail: if (!zpool_can_sleep_mapped(pool)) kfree(tmp); @@ -1183,13 +1156,8 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we get here due to ZSWAP_SWAPCACHE_EXIST * a load may be happening concurrently. * it is safe and okay to not free the entry. - * if we free the entry in the following put * it is also okay to return !0 */ - spin_lock(&tree->lock); - zswap_entry_put(tree, entry); - spin_unlock(&tree->lock); - return ret; }