From patchwork Wed Oct 26 20:06:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13021239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B67FC433FE for ; Wed, 26 Oct 2022 20:06:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 51D398E0003; Wed, 26 Oct 2022 16:06:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 456258E0001; Wed, 26 Oct 2022 16:06:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F7818E0003; Wed, 26 Oct 2022 16:06:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1E49B8E0001 for ; Wed, 26 Oct 2022 16:06:17 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id EA66314085B for ; Wed, 26 Oct 2022 20:06:16 +0000 (UTC) X-FDA: 80064182352.03.DF84A9A Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf18.hostedemail.com (Postfix) with ESMTP id 957F31C0042 for ; Wed, 26 Oct 2022 20:06:16 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id t10-20020a17090a4e4a00b0020af4bcae10so3215595pjl.3 for ; Wed, 26 Oct 2022 13:06:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=Hi53eiNft/uxfPCA9DiEpqEUXzmt5CELkUIG4MFbzeAEA0yq6pTdtJTWBHdXP5SCiS n2Ur0KkDeGzYyPPU9S1a9htUfQlaijj8CCXdjA4PC9kS7VA9vxREY3Cw/N9LFE9f1bnA fZiNG0Brr5sdQwx9wEuZveHTnAKm4SpFqqTBNfi1t7B+g8OZ9ipJcZZOWrfcjQcGjNDC uGdiBtgZuRgYQ8HrD+09R/mRUyEq/IVckBQ6SkfC1zPCm3EtYyYiOPvYovyty4U1n2lH dGHYFWO8wt3JAUfVkXbtouiJ1q1G+6Ual7nwO3I/SWEYqWiCnwLKxXhDETzKFL1OtyCF sHxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=C6iKf3nbz+DOKhYOYPwOyEZm+aqKHhmvQzfe1WrwDq2JVAJcXGvIMPXqTI3G/1qQb2 eBGn7YMoX3ACdGVWhCwxqAEelLwqXpRXxDn84WbzVDJmkRcxY8r0m/AGTK3VGtbe0ZBT 4XBAO7yGa5w81hfejRZRjCc/1P4Sdh2kVYJiq7oSdDTrlQmb3NHWQTt0GnwV4gkdokci IO4o0HbKEpB9AUvUzc/o30PTKdTrlp7Kzj2pCL/VDnf8WHFkRWOGRZEbsBSwO/AVlHM+ adWGcXE7PREQILsx7Yg/FGeXQBx4q5ySVOyy5eIa3JZ3GVq/DpLYmqgIOVSQetSlISbj PpEQ== X-Gm-Message-State: ACrzQf0cD0MUYGrnKwhAc6JpcxZ1EzAxUlUKJrDeyWBEMR7m4AfUrjbF reTCry9ufNSdHT9M9Jz4erw= X-Google-Smtp-Source: AMsMyM4iFxW9DroEwTO8biUJrN1XDhw8KUDpdW0Isf4jh0mJSFOpt+BANcnAvrvFY5OCPpE4+BCl/g== X-Received: by 2002:a17:902:f710:b0:184:7a4c:fdc1 with SMTP id h16-20020a170902f71000b001847a4cfdc1mr45443478plo.27.1666814775478; Wed, 26 Oct 2022 13:06:15 -0700 (PDT) Received: from localhost (fwdproxy-prn-015.fbsv.net. [2a03:2880:ff:f::face:b00c]) by smtp.gmail.com with ESMTPSA id y27-20020aa7943b000000b0056c47a5c34dsm1775845pfo.122.2022.10.26.13.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Oct 2022 13:06:15 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH 1/5] zswap: fix writeback lock ordering for zsmalloc Date: Wed, 26 Oct 2022 13:06:09 -0700 Message-Id: <20221026200613.1031261-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221026200613.1031261-1-nphamcs@gmail.com> References: <20221026200613.1031261-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666814776; a=rsa-sha256; cv=none; b=qy4WhUe/FVITlsK4zXfisvp5OZnqrS8HbuoEuBO0XCqFIIyIN81IPwC0TU6FnFO12VEerj ailo6KQXG4OmikWUBPUm4wnIk9j2NgcwEhiEukY7KhlCQCiqKXxRnvNL323D6xhaUbl0zG eNU7uiKFF9f2s6ITT5xglX9zdOAOMmE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hi53eiNf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666814776; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=L3lQGV+Z/wdD6jZB70BPo8pzRZrsH7gZCqVfkGB+bO+nW4jpbsbESojuwb8au42se6iH4M 7Y4j0lbJkiFZnpM5/H5ApXvm09jZPuvEE6uOOTdk1vzs6BIUZ70I3YP4kQ0y6X/aGybF50 ++C/pcApOLRIvWbEF3Fg+O3hYWmmqSA= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 957F31C0042 X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hi53eiNf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nphamcs@gmail.com X-Stat-Signature: rzcdg7wcx7i1g8rp7ko3aicft98gsif5 X-HE-Tag: 1666814776-569113 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }