From patchwork Thu Nov 17 16:38:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13047108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DD85C4332F for ; Thu, 17 Nov 2022 16:38:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81BCC6B0072; Thu, 17 Nov 2022 11:38:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CCD26B0073; Thu, 17 Nov 2022 11:38:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 694A08E0001; Thu, 17 Nov 2022 11:38:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 56EF86B0072 for ; Thu, 17 Nov 2022 11:38:45 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 24E72ABA18 for ; Thu, 17 Nov 2022 16:38:45 +0000 (UTC) X-FDA: 80143493010.05.4DB0AC4 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf11.hostedemail.com (Postfix) with ESMTP id B9B8D4000B for ; Thu, 17 Nov 2022 16:38:43 +0000 (UTC) Received: by mail-pl1-f180.google.com with SMTP id 4so2156836pli.0 for ; Thu, 17 Nov 2022 08:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=oUdeLYBGlvLG123q8GqQX48SG762gZkHyxwD/CQnrBoB4Vh1+p2Ck7lVhiNkqnu5q0 du9Xij/rT2JVxeyh9jH5BAwEhVln1dU0S3KA2SfSRSnQQG0IOlwjQHR7toI9DdBaEbeQ 9B1MH+AyYhr8Zxcxdi22Wm8dI5rSEieLZCPSN2sPD16X47ZsCxyHQp4Avg6axUIlyOim PBuesYanmIb4IOmdKWe6NhkCBviHMSzeYwiqsNlw1dNcmsQZP963LulvMKKMZhxHI5Rv bJlQHezRy1lbHMPvTEV9CPvM/QW6fWHqSCqpAhiY34OIqcdl621KD7NG/zqRW5tQZWJP Z6tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=rBo65GjyaEDei1jpqZzGx2WiejjOJmI2ZY8zYFtogKbZWRML1ExThhN2PBFUNaZEnp qkx5OzOT1gLUvbY5/yDc2zML43yfo16oCQF3PxGif6lf4Llb0T5laxhoXh3kg9dnsg2u za1xfmVzx2QOm5E70FhFRNbBOZWHv/MEgg2+ryBhQdQ5zDa13iXSc3+RfC3jeIRayCZK SMVUAi9PT+hkpgsept7Rm9tKumLhX9NSzDBiRZG82LgwUG7++xa6XE7LX3Qfg6SbiJTX mmHBsEEiYiLSDbPSqt1rnVwk088uWX95juXHO1eJQTcE0ayidyy3UZsLGQwcE9Mqmvem eE8Q== X-Gm-Message-State: ANoB5pm1Faz6Ol7g53HMSbL75jLXj/PJXs9Ggg2/bfw4pFnGyKNeOJ1Q vrGrk2sOl6FPQ1f/h4su62M= X-Google-Smtp-Source: AA0mqf4dCzk0UaidJ1LEguhOxPNhCf2SFFc2QYWKmgSXtPv/ga+VnLB8+OZZUwB8M6zR7N1hzOWP/g== X-Received: by 2002:a17:902:c409:b0:186:fd58:c706 with SMTP id k9-20020a170902c40900b00186fd58c706mr3658532plk.4.1668703122572; Thu, 17 Nov 2022 08:38:42 -0800 (PST) Received: from localhost (fwdproxy-prn-000.fbsv.net. [2a03:2880:ff::face:b00c]) by smtp.gmail.com with ESMTPSA id y65-20020a626444000000b0056d73ef41fdsm1377275pfb.75.2022.11.17.08.38.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Nov 2022 08:38:42 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v4 1/5] zswap: fix writeback lock ordering for zsmalloc Date: Thu, 17 Nov 2022 08:38:35 -0800 Message-Id: <20221117163839.230900-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221117163839.230900-1-nphamcs@gmail.com> References: <20221117163839.230900-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oUdeLYBG; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668703123; a=rsa-sha256; cv=none; b=KNUZk4Lk/mpv/43mQp6YvMQevCLRw/1HESxgPGpIMXMrFYowMnjfsDy6Gjy+7z1H9uu8ur tLEtmFJXL0xO2EU82++owrV0B7d4yQ4G/yw6af9BIY9KYeZAa/zdeBm7MAZnbgDwO/aKRi WAHtjoGzDVPdImeIbre2zJ/U9xqC0Yg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668703123; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=EOdt6p4D4EhOghBTYU08hDo4Lt1gx25dPeYe/x3onsMqZO774bvn0BYchgixO4zBN0TCZ/ C/jgsnGq/5DvTXoV2hZmgpg0kDUtVJKnx9sfTRMZPjt5Jhb5hzQsFKwVocdlt5a8VOlrtv Fv6zX0noIWOH76xz1+jdRwuZ0yPlcNQ= Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oUdeLYBG; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Rspam-User: X-Stat-Signature: sjo1s5wsgceizh1exgqkyn5i36tde3t1 X-Rspamd-Queue-Id: B9B8D4000B X-HE-Tag: 1668703123-788957 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }