From patchwork Mon Nov 28 19:16:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13057890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A84FC433FE for ; Mon, 28 Nov 2022 19:16:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4A4A6B0075; Mon, 28 Nov 2022 14:16:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CFA416B0078; Mon, 28 Nov 2022 14:16:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB3A06B007B; Mon, 28 Nov 2022 14:16:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9AFDB6B0075 for ; Mon, 28 Nov 2022 14:16:20 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 80306140BC4 for ; Mon, 28 Nov 2022 19:16:20 +0000 (UTC) X-FDA: 80183806920.23.EA3BB41 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf29.hostedemail.com (Postfix) with ESMTP id 07796120012 for ; Mon, 28 Nov 2022 19:16:19 +0000 (UTC) Received: by mail-pf1-f173.google.com with SMTP id w79so11473913pfc.2 for ; Mon, 28 Nov 2022 11:16:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F/z8xBFicb3R2PeUO5QeKRD0fnuKCQ0SAaRvKkmAk3E=; b=MoXOV1pWeYXh6PbisJX01BZJce1uz4bNi3N8VaqG8qKv7jhznnybAwHouVcezuswdR 3rwSPKUFecPuAqXKIoX9s/uUVgU68mSe/sp8MPwWcYHzOZeybPexk8iEXBciozwzH8NG sa0DhlIdnpCghqfW9A+bG/zOqBnNUeb+tU4ahX2OTZ91qq4y9OFv6kqUavWrjuNyDrFC Ntg+DiGiJghENQ0wYrDIrlOZlrdtQL9QW4WWsFVFhBPrytQznW3ck7MRBQ2PjxbPzXsN 5UL0J2LZF9HCtnimSrb8Av0tYmDx4N2egUI/lwL6hOtm9JRmADw9kL8vrmt2zF0F77Ke Ha+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F/z8xBFicb3R2PeUO5QeKRD0fnuKCQ0SAaRvKkmAk3E=; b=gphXGFk1noxzQwouls4p39eUDw0mxLvpoL6saOgD5enu9Qf77NcRq2zknHTk+LejIt yXUQSAdp1ZfYQ3Q3NyK97G08lba/X3tA8WdvjRudcj0L6vFbWp50Vtafw3T/pPylK5Ph 5qP0e8zOd4wHSbYkA4KOBYEgB9D5zBNZUturSLIwLpDzSn4zLJvTL6JBH6a9jE0VQlAP RiK7E2q9TwMcJfOp0Ez/SDi8zq8HSt9z34VBWpiUKAEAYwZNBx4kIgyA+0VX2d5oiAM4 yrywK0nHX/7kHz0Hozl5FtkpyYn+gVr91vkyE2lyBZGKOsA+dtPC+uo91bgNXBjE4Rfp pxRA== X-Gm-Message-State: ANoB5pnRhizLVNrzYOKw5AsTJ9sGoFdU99MeAoKxN8fiqWptAQK19FFe tPNfw6IDjHx9R5ggwuhLL9K470yFVp5vsQ== X-Google-Smtp-Source: AA0mqf523izJHxzI+4A7CQUZVosROEwsf3PehmYpsrWuusjaoiqMgcmIdQp5MoYFCbbxgrsbbKH3JQ== X-Received: by 2002:a63:5a56:0:b0:46e:9bac:17f with SMTP id k22-20020a635a56000000b0046e9bac017fmr31414467pgm.420.1669662978660; Mon, 28 Nov 2022 11:16:18 -0800 (PST) Received: from localhost (fwdproxy-prn-013.fbsv.net. [2a03:2880:ff:d::face:b00c]) by smtp.gmail.com with ESMTPSA id 67-20020a630346000000b0046f1e8cb30dsm7187107pgd.26.2022.11.28.11.16.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Nov 2022 11:16:18 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v7 1/6] zswap: fix writeback lock ordering for zsmalloc Date: Mon, 28 Nov 2022 11:16:10 -0800 Message-Id: <20221128191616.1261026-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221128191616.1261026-1-nphamcs@gmail.com> References: <20221128191616.1261026-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669662980; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F/z8xBFicb3R2PeUO5QeKRD0fnuKCQ0SAaRvKkmAk3E=; b=H8eW6PXRkddcQBB7VSYUz58QnXbmHJbqFnPQvUTddTEgrHS9TxY033zDM3GiAL67FOd/VA JlX9J728y7+jMtjkYNNH30TTRKTYlW5y8rWE2pBDSRU05Gpb9fZ2t6UkkBm80KB1xuSe6y yGa0hI+0pnJV14CGhImUHw7RM/CACr4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MoXOV1pW; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669662980; a=rsa-sha256; cv=none; b=aqzKKbUDz0hp391BVlnotEqyqoFsn7IiGDisbsAgXrnqyxIcWvxwrwsaJ9qh4GzMCffkla EqeTwIKuQHWe9nmVPjaZccYiOsoTR4xnHuvVf7AWbUc5TfsOKiR1caudhYfO/V2wE4vESJ D8TdBC0Ad/C8+gNAZV4VbfxBsgwM2uE= X-Stat-Signature: dt3rc3d7it9cpkb16bjxdmmqwybuz53a X-Rspamd-Queue-Id: 07796120012 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MoXOV1pW; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1669662979-366216 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham Reviewed-by: Sergey Senozhatsky --- mm/zswap.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 3019f0bde194..f6c89049cf70 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }