From patchwork Sun Mar 24 21:04:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13600922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2BC7C47DD9 for ; Sun, 24 Mar 2024 21:04:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF1506B007B; Sun, 24 Mar 2024 17:04:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA1C16B0082; Sun, 24 Mar 2024 17:04:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 969296B0083; Sun, 24 Mar 2024 17:04:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 866186B007B for ; Sun, 24 Mar 2024 17:04:57 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1D65D80355 for ; Sun, 24 Mar 2024 21:04:57 +0000 (UTC) X-FDA: 81933162234.09.1EF3183 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf01.hostedemail.com (Postfix) with ESMTP id 2B1C240010 for ; Sun, 24 Mar 2024 21:04:55 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=kMjVit53; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.169 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711314295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=H/e3ZJBIRlaFRfZGVB5Bn/R7HM/dB89XQuSLzZy4Kjo=; b=FT9WcZ2a5LIHPpGkrEEZf9J/fkuQ3+tnnJhAVElxo+RRn03F5oQy9WtqAwOLs8WU7rQe4U dc7Gdv08xaRxRB7znaKiunf1o1qOxS9puPYpIfeZL0BE/e3wekH/sBQ9FqGSN3igg2l+aK AsZb0ofF5GwlgggSm5N0rV8n3RLeRlU= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=kMjVit53; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.169 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711314295; a=rsa-sha256; cv=none; b=ABiIQX3jhEzyOGZPQHYau4742NUAki2psbQ5A9tj63mZ/1RJRrxcLl0LJAD6amW89rNEzG WtznpF0x7hL2hf4VpnvQ7mY3sS0q24gpJzq/mE2esZVS3Z0Q79DFpCyxp+fHVA+qkGwC3x L2PsEUYnsgb29EXskVGS5DFIWcn45TA= Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-42a0ba5098bso25644181cf.0 for ; Sun, 24 Mar 2024 14:04:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1711314294; x=1711919094; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=H/e3ZJBIRlaFRfZGVB5Bn/R7HM/dB89XQuSLzZy4Kjo=; b=kMjVit53/Zq7SZ0GP38Nrv8j5crtFypJZAYiQL97c6naYe8acyPlONYnUuJpdfciKL YK1Q1bml26HNtNDns+vvB/sLqoyeUaAP2MwGbb27/eNm1zFPfaOg3C2EctH0LcJ0D53Z 7N4O5eJQMUGopVWXshmNhgoHROpxDN3l/flS36UeLGTE2E+/HD22SRY8pgcsJjNJIv3N v74wrtfUBg+1i8FAeVDW7B4JVeWq9pxjCiqLzWQVw7LW7zjojMTHoZIfj9/4gnj8ORQ1 ZKSRCmxlIHImpLAAvrPOFcSZymSOL3aZWYjd/BOlmmVlNDvekDXsuj5LlNdNraQXUP5Y 3Fcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711314294; x=1711919094; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H/e3ZJBIRlaFRfZGVB5Bn/R7HM/dB89XQuSLzZy4Kjo=; b=ApUyFZozcw1KQ5xdb19TugDmI7Rov5EKNlSMmyKQjUH+Od8jx35i81AgMjiA1fWrje 9t7RIbhzS7l84kdViwVec8tvNziJE7eqZZtdB/1VvDCGfHA008pVogVi1bLn2uRnHFUn d0E3c7Iy+So8YpO6Sv7MC/MSxwjuZOgn237hLMAZSnA94Uhpo5qpgaI8kwLNVk1NLWDR 89fBF1M56W53Z7uUByZ05tGsUH7/TeB24FNdoxI1wmjcEqCEXDp57Yxf+sRFXJm8sUBW HMC6RJWRnn53D7oWTON1wJYkjzuuFK44auRNro979ICGEsDvu4gZvgeByS/n7cBo8nC6 b3BA== X-Forwarded-Encrypted: i=1; AJvYcCUnV2jQVLS0jnaUWGG3GFsfxOYZVe6ze9Jtu58xPDd9f4LtjHQhKX4FSpcUEql5L3NOYuFxbJYi1gQfZkuc83FxPxE= X-Gm-Message-State: AOJu0YwfF2cqT7lVLm7WG5xLS+sMl86zav54K7La9q+BfRn0vdY8gNEf j5Wt3H0hZTOSE90ZFT+BIH7ZW4OuvCK0Qvw36t2qmQczEZtsYxXlQ1pYjHYdPGrHNgutf27I0Fb O X-Google-Smtp-Source: AGHT+IGNExr7L3QsItNVJAVJsqzbgVpSn2HCkqM9zXFxcJ0uA6chC1PR5+hBIS7f1AT1DGZ1cg9w/Q== X-Received: by 2002:ac8:7f89:0:b0:430:dca8:dd47 with SMTP id z9-20020ac87f89000000b00430dca8dd47mr5598250qtj.47.1711314294228; Sun, 24 Mar 2024 14:04:54 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:16be]) by smtp.gmail.com with ESMTPSA id bb28-20020a05622a1b1c00b0042f376886d2sm1314173qtb.36.2024.03.24.14.04.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Mar 2024 14:04:53 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Zhongkun He , Chengming Zhou , Yosry Ahmed , Barry Song <21cnbao@gmail.com>, Chris Li , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH] mm: zswap: fix data loss on SWP_SYNCHRONOUS_IO devices Date: Sun, 24 Mar 2024 17:04:47 -0400 Message-ID: <20240324210447.956973-1-hannes@cmpxchg.org> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 2B1C240010 X-Stat-Signature: huxm96hd16h1hgmhkgxqwk9ppx7qq77e X-Rspam-User: X-HE-Tag: 1711314295-942723 X-HE-Meta: U2FsdGVkX19GuZE2EGOGzi77kUW6iRRxHKzUjna01npifOwh35XUXaZGAzJNMKr+iScaLAR4xdL+YtFm+l16u+iF9R7psRLWRYA4a8lo0p0oKdJCmXzUHN+p+XMqugrrX+HwIeotobG8Iu4AKEj2VH2v2zbltDXwB/fRHsySEsKnDahFaIFKz+iyVbnpdFvxW/XYntEIk7ENavV8IR0+dxKr3hCTDggN9CNJHY8s3rVxsd3mVXEyKrVCzOyt8w8zZSbtfnL5sILf2ymBQ5dXUB8LHThwdBJ40ioaOk4sngOz9NBFjumMyKWF9xwhwMw4s0v9mjQ4jtm8guaVh11L7NSkoq0kVUjgZ4wcRyz84dgWvB1eABFttar8LU3h/DfiB8XIugwFp7rScEPed0k3fzk07ZHYvUkl1SszNVU9i1eUP4Mx1Huabkm0ZPOe/5UuDidFLanPgsIAMecNeOGu4M+EQe6XvHju/jiflwJALQqTGhPdvwhywlseqBrkI0Mz7IlIIoqlQMDZ0QvI6K3fWJU5KBbAD8nYm9ypCKRMzw7FBiZr4j0PLSU5D//VLA5Ry6b769BPIlPUhaNTqi1JB5epN0zJ+DW65rGrP8YL5gq0UYCYzk0wjNaIL89PHP/K26ifm99M/gWZOqMePLvo4rzZ+dSUq/MnzvCmql8Yj6Fns+yNuvOS/KwkyygJvEu/9w1tard0th/H6ySr1mtFiPvSjzM1O5I+NGt6jVoIvWLNB++RERX/UvoyKSCpA/YTV/2KhG64CHEdTLMRDcYqPjUScU3gKW/nCzeQyMRAtGIjldRKEN4K7w1Bop1gnOHs5/fKzj61aTTEbY4M1K81DTDXRyFAAzsF77AnaoTqeGmP5ZRy/nOwy0oKWkiwi/4hAgGZr9e9G0DX1BEHSiw9w+7yUh/lMoIpPRBCTMUNBquAb+lXTB5FgHmP+guAxOUw5PEw1E1wkCuQuGwqQh7 zhnqO8Kc eeccgMJyo19Bb8MoAJ6YK0/Iw8OIRjnG76KqFgizlJUy3Gwk2v6uXsK/1TAY1wAaMxWWMQWl9tB6ewWTLDiN/jUmpq4fKHUrKruDMWKyG/4dGFdgiDGTwHbwTidvAJVCRsUuGRmVtYUytlEETTcuCESN5legxXwgX21byyDXlVaxLz39Ox7kWhyRqX85BUhBP6MW9vrD+09qM5JjmPbcQpvAEOQ3NKYzlM3mwy/GlVjrOXvL9iRIUwJ4fLrd31WPKHbQS9tpq8dLGF3JtXClQxKlib8gJNSowa16Ur25QjGl+2CPhcnOTzHvwr4s/46cQ2UEl8wwHIiTqjVxuvK9XvZE6M3axMXQ5BdQlhvQ6jL4jNmJk6TtyDO43yW0dz/qi6a4RaV3YOG4DSuU9v6X6s8wscdjX3pP597hnJ5Rzzx/Xfuo7nYPHbp6lFcR+BQeF8HmzvSD7q/xOP4WJpWOwnu3HJUpwb2fEcR25sKjRbnexXAp4gZckFuU9JA1u2NYUw4yPBYpoyVQ4lboUyPy+rivGnEAaWxpo2gVAOyWG9j/cTBRU3JmRQdX52DQTu5I22oodcIQK5M+TXNQqhDEyRnaqDFHNX29mPpvhLaH5KlX/blwHlRg7eS6f+qBsqmx81CbRI74iJT22urDx//hn1+mlAO5GdUhfCpdrTDiLRBhAhV4//XWpRAAzAsy97DcKrVcL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Zhongkun He reports data corruption when combining zswap with zram. The issue is the exclusive loads we're doing in zswap. They assume that all reads are going into the swapcache, which can assume authoritative ownership of the data and so the zswap copy can go. However, zram files are marked SWP_SYNCHRONOUS_IO, and faults will try to bypass the swapcache. This results in an optimistic read of the swap data into a page that will be dismissed if the fault fails due to races. In this case, zswap mustn't drop its authoritative copy. Link: https://lore.kernel.org/all/CACSyD1N+dUvsu8=zV9P691B9bVq33erwOXNTmEaUbi9DrDeJzw@mail.gmail.com/ Reported-by: Zhongkun He Fixes: b9c91c43412f ("mm: zswap: support exclusive loads") Cc: stable@vger.kernel.org [6.5+] Signed-off-by: Johannes Weiner Tested-by: Zhongkun He Acked-by: Yosry Ahmed Reviewed-by: Chengming Zhou Acked-by: Barry Song Reviewed-by: Nhat Pham Acked-by: Chris Li --- mm/zswap.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 535c907345e0..41a1170f7cfe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1622,6 +1622,7 @@ bool zswap_load(struct folio *folio) swp_entry_t swp = folio->swap; pgoff_t offset = swp_offset(swp); struct page *page = &folio->page; + bool swapcache = folio_test_swapcache(folio); struct zswap_tree *tree = swap_zswap_tree(swp); struct zswap_entry *entry; u8 *dst; @@ -1634,7 +1635,20 @@ bool zswap_load(struct folio *folio) spin_unlock(&tree->lock); return false; } - zswap_rb_erase(&tree->rbroot, entry); + /* + * When reading into the swapcache, invalidate our entry. The + * swapcache can be the authoritative owner of the page and + * its mappings, and the pressure that results from having two + * in-memory copies outweighs any benefits of caching the + * compression work. + * + * (Most swapins go through the swapcache. The notable + * exception is the singleton fault on SWP_SYNCHRONOUS_IO + * files, which reads into a private page and may free it if + * the fault fails. We remain the primary owner of the entry.) + */ + if (swapcache) + zswap_rb_erase(&tree->rbroot, entry); spin_unlock(&tree->lock); if (entry->length) @@ -1649,9 +1663,10 @@ bool zswap_load(struct folio *folio) if (entry->objcg) count_objcg_event(entry->objcg, ZSWPIN); - zswap_entry_free(entry); - - folio_mark_dirty(folio); + if (swapcache) { + zswap_entry_free(entry); + folio_mark_dirty(folio); + } return true; }