From patchwork Fri May 12 23:57:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 13239991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 666BEC77B7C for ; Fri, 12 May 2023 23:58:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4F786B0072; Fri, 12 May 2023 19:58:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E26786B0074; Fri, 12 May 2023 19:58:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA0F26B0078; Fri, 12 May 2023 19:58:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BABA06B0072 for ; Fri, 12 May 2023 19:58:18 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 863C341094 for ; Fri, 12 May 2023 23:58:18 +0000 (UTC) X-FDA: 80783269476.03.FD10DD2 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf22.hostedemail.com (Postfix) with ESMTP id BE374C0005 for ; Fri, 12 May 2023 23:58:15 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="f5hclik/"; spf=pass (imf22.hostedemail.com: domain of 3ltJeZAMKCN8QDDHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--pcc.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3ltJeZAMKCN8QDDHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683935895; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=mFHTWqZDbChus+ybWMnSwB8QQiXQFh57TLiJRp9j4Z+Ay4w47SrjnfgfRKHmIv0qtpMhAb 6rtHx5rcZHP0aRKlR/gvMswY1LCPtfhcDeZsr5+134aTWPZUqI4iXJkbOGYDKt2FyYXo9q gSs2+zRWfAne/sC5k6/wtFCsn3ejoXg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683935895; a=rsa-sha256; cv=none; b=KI1XFn0aajwM+FCDI1jUtL6Wi7lpDhaY+PUswr26DZ+qPlKX2DNbBXjFZG/frW5b8pRaM/ XRLcsnmbycef+IeySaLwENoJdeqJifjBIOX8zwJrNat56A5Pq+34RvmDRZjNN37hULCw2U 0kRTQ4fEUDSkEIWdT16YhPnuN3FRC40= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="f5hclik/"; spf=pass (imf22.hostedemail.com: domain of 3ltJeZAMKCN8QDDHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--pcc.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3ltJeZAMKCN8QDDHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-55ffc3d2b63so107875907b3.2 for ; Fri, 12 May 2023 16:58:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935895; x=1686527895; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=f5hclik/Ew6KC4whYAG5VntaAWwKmaBWXX4bCpgRdYjBpbSj5Apr3PkSoK7/QAJd0N RpHxTIRW/bLimRvcbUmZjobDE/Eutpj6tvgC/Rdg9nOt+ZvW2UeGjSSNnpNfchKgE9qV bfsrP2dthFHNMRT6c5aCQw0Ff/JTyx0DIAtRow1fvLqu2M2UZFqFRteLQ4IPrCBhdwYv ZUWPKO0rlZI6hiVGzQF5WVmuVs1EGnbfoUeE09/HLSnj37gvPWFw6+cmVnJFfHK5WbAH YWYFfgDih13LoVhaHD+4rYuds+BE6rKkLJ/AoCA0fp0U2j35styXF7xvUDDBUZNd5sjd vJuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935895; x=1686527895; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LVFc+Brp/nv5bcVUuCiFrcDkBMn4WnVlAUQLAdw9K1k=; b=KBNvlxRJXyxqM+1p38ICO1inRi8AoF70IhipzfJUxABECjpwb5hapGnQovTbzCD8oA uUpZ2RPVrr1jW5ch4B8OfLoNWUSwHX5nzTnlzwaGTRYaF1KmcqD52LNed/msOftbOhvZ eQaC7HYsLkmadi1ocOBzgkHALjHMRIbT0DG5wRqKNKCauBXWq8CEbQWKQtG3oa2VMDz9 ZWjMF61ShfN29iAaueh2VFwyvhoAu2iRfNjZ6zM2ZVLuWxRq5rKqlzE4o/3HYs686QG5 BrBov48IpNEnsi88nroqnINW30AoE+7ZihTSY3mADnFdYNlvCrjTBnROxQ/sr8qUd7Q0 qCPg== X-Gm-Message-State: AC+VfDyPMCIXzs8ild9V8dzd56FZvc96n9ZEvP1anQKtl0S+9j++g/cv yJtx9/MeN72A7YGYbzbgLwyfDVk= X-Google-Smtp-Source: ACHHUZ7VMsxVospQ1XLBd1n7/FkUbi+qPyj+Q+Eihu+tg6ttj97Cmt/cOaDOuPcaz9q0znnK8MBI35s= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a81:b285:0:b0:559:f1b0:6eb with SMTP id q127-20020a81b285000000b00559f1b006ebmr16091480ywh.4.1683935894947; Fri, 12 May 2023 16:58:14 -0700 (PDT) Date: Fri, 12 May 2023 16:57:50 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-2-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 1/3] mm: Move arch_do_swap_page() call to before swap_free() From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo?= =?utf-8?b?5by16Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vhbi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Rspamd-Queue-Id: BE374C0005 X-Stat-Signature: j4zdskhraopgxd36m3ru3ig3irurohg7 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1683935895-659138 X-HE-Meta: U2FsdGVkX1+cRfKxSJ6t3y63deCjoc4Q6hhB4bcj6Y9waoGuMrd7MccuAOJx/tVShQMh/jbXBRRAuNSYKBRM/c+VFAs6d7E/eAwjGwsT1LWv18FB3kKL7oddNKy1TTyLRZtb9xKuIFaecuA5hw2kautI1jIsDQ4T4w8iyLvRdViuy4fcVAcLV8yesKMtlK8d4lh15b4HfQDZP/BKPJX04KrOknLUaTWAGRuUBmHK8Y7H3dysFiV9FZqR+kugta+/8i+WxGP+LMXJtxpH5nUc5PNPplOlKiszJWBcilTDggyhaUdne+/aKDlXKMhkkV0zx4yCAOr37fbswq2RBQt6KxNGF7Lro1+6ub1favxIsbogxwWpQdebWIQFpT2vxVB1YbK7xaIxktq08sDhiNc1O9J6y7lqLFvfn8S7GENnd+IK43kw8N2OwNRDm7n5y6SgRx8ncp5UW2JD0jTIJQOIumZAtZ2FKfdYuzjgbgBmEMrm9GR5sQZ2Vj0KJ/0/ZFtvljrBKlGH5LQLqh7Vhymjs0gXLzkZ4YPwnLTS2Kyz3eIArAinULFE+VNCiyaroaKpbGxJRt4IvQXGFCAEdoGZyZw03oUfQJ0zjihhoxnR3avk3/mnfOwt91RZ/vSMUB6xXXN4HSfcObXSmFjFyvfUAEYILkJWjE5afBB+CgyXN0neAyxcHZG3ahXGbgJlbmHdWl0GBd73kB4ojw32qBXQDfX3QXeR3JOqhe/7Yk0tZvolRm2p3YqUIggbgcFo5k2hhlcaZjJKoUV/kYGrb85XcPxoVNSchtg0jyPJY/EwyS/R+JMuGetUKgh2CTEqFLVE1sC2Kczzh8Ws72pVJGPVRq53MVigyE0vLiz7Cidogi5yuHA6BYJ6p75JroAMzSoM2an9H7naKUxAeIj5/EYM9fIskezerIl20+OXYwpcT9iciARvSPpOvgy2F+K0evjCdZ9Ju24pKFgeqiHDCKe 8kOMSMFF 8xrox8uc0tIag0rVGyxC4ICoaNKYM6mM428k+I1PdkBghvi/7LNFz39T5uMUGknS5qGg1MyNv1TehvtnKW1X3VwJ+WHdT9bJcUWcQuHqIz1R1N3PSjQkRWQsQvx8Gx/U19MBE0ofyY1JZ7hQXdqir16MBLSSAXGTtacGm4ZnQz68a2Sb3s7rLv4tYPvIBfGxHEFwS1w4d8c1+JFD/afZh4wckLeYv3uXwkRqbiCxX9Z9IZEbuvJX27fLzY7XsWMjC0TOnFmuX0nP5XxJT2lVwoNPl4cgTYqMdy9qBqXR3+jKEvZpaDABI5ibowCyEmqJThscYxX5Pe1DfH3+10A+ffErpY7xQ2KSmiYnqdcjmXAk74iTzykL1+GDyF+aqZ4FqlsB5GJu6yjtT0V+7F7PSjw58j507vtQQRkbMSA76dSdudm0tCaUrGtIZwKEGTYDOPn7weB/wxkZn6XdMi1cbneHGjIQW68O72D7/VYCp0wsw2J0tiVbxC5UEw/o6ChgE3GhJm/SmR9PWfnUmwpAkF1dfF3N9mBOzs3QtLkB6lwQNJTR5JvEPIgmYehvyOsCb8t6wmzkVCl0C/tUNfa5DeKl5erx9Z9XNXBSZLhno1U+Dq3OqZk5CFUEiVyySldx5SJ4lN4zjJpjgD3VL78CY8iRnR4D9YTDxNsLYXxer/n5E5bpuf8LeRBTjACmTcKcJzU6EB2sopYd056VEJQZdGZxV6ZKWLEk5E16q4DuETMp16CV/DMVYJZ+SDwvEtKov+awbSDM3QHTqHFG4Y7pQEZArPgfoBixsw8f6KaQyVHvCso+5VngybPb6IakyV7QUFsYarDwnnyZOcvZqHGC7/NZbkVj1uXRQROmE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit c145e0b47c77 ("mm: streamline COW logic in do_swap_page()") moved the call to swap_free() before the call to set_pte_at(), which meant that the MTE tags could end up being freed before set_pte_at() had a chance to restore them. One other possibility was to hook arch_do_swap_page(), but this had a number of problems: - The call to the hook was also after swap_free(). - The call to the hook was after the call to set_pte_at(), so there was a racy window where uninitialized metadata may be exposed to userspace. This likely also affects SPARC ADI, which implements this hook to restore tags. - As a result of commit 1eba86c096e3 ("mm: change page type prior to adding page table entry"), we were also passing the new PTE as the oldpte argument, preventing the hook from knowing the swap index. Fix all of these problems by moving the arch_do_swap_page() call before the call to free_page(), and ensuring that we do not set orig_pte until after the call. Signed-off-by: Peter Collingbourne Suggested-by: Catalin Marinas Link: https://linux-review.googlesource.com/id/I6470efa669e8bd2f841049b8c61020c510678965 Cc: # 6.1 Fixes: ca827d55ebaa ("mm, swap: Add infrastructure for saving page metadata on swap") Fixes: 1eba86c096e3 ("mm: change page type prior to adding page table entry") --- mm/memory.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 01a23ad48a04..83268d287ff1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3914,19 +3914,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } } - /* - * Remove the swap entry and conditionally try to free up the swapcache. - * We're already holding a reference on the page but haven't mapped it - * yet. - */ - swap_free(entry); - if (should_try_to_free_swap(folio, vma, vmf->flags)) - folio_free_swap(folio); - - inc_mm_counter(vma->vm_mm, MM_ANONPAGES); - dec_mm_counter(vma->vm_mm, MM_SWAPENTS); pte = mk_pte(page, vma->vm_page_prot); - /* * Same logic as in do_wp_page(); however, optimize for pages that are * certainly not shared either because we just allocated them without @@ -3946,8 +3934,21 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) pte = pte_mksoft_dirty(pte); if (pte_swp_uffd_wp(vmf->orig_pte)) pte = pte_mkuffd_wp(pte); + arch_do_swap_page(vma->vm_mm, vma, vmf->address, pte, vmf->orig_pte); vmf->orig_pte = pte; + /* + * Remove the swap entry and conditionally try to free up the swapcache. + * We're already holding a reference on the page but haven't mapped it + * yet. + */ + swap_free(entry); + if (should_try_to_free_swap(folio, vma, vmf->flags)) + folio_free_swap(folio); + + inc_mm_counter(vma->vm_mm, MM_ANONPAGES); + dec_mm_counter(vma->vm_mm, MM_SWAPENTS); + /* ksm created a completely new copy */ if (unlikely(folio != swapcache && swapcache)) { page_add_new_anon_rmap(page, vma, vmf->address); @@ -3959,7 +3960,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) VM_BUG_ON(!folio_test_anon(folio) || (pte_write(pte) && !PageAnonExclusive(page))); set_pte_at(vma->vm_mm, vmf->address, vmf->pte, pte); - arch_do_swap_page(vma->vm_mm, vma, vmf->address, pte, vmf->orig_pte); folio_unlock(folio); if (folio != swapcache && swapcache) { From patchwork Fri May 12 23:57:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 13239992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A125C7EE24 for ; Fri, 12 May 2023 23:58:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 108AF6B0074; Fri, 12 May 2023 19:58:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BAEB6B0078; Fri, 12 May 2023 19:58:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E75856B007B; Fri, 12 May 2023 19:58:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D71AA6B0074 for ; Fri, 12 May 2023 19:58:20 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B0DA8AD1D1 for ; Fri, 12 May 2023 23:58:20 +0000 (UTC) X-FDA: 80783269560.21.3179124 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf21.hostedemail.com (Postfix) with ESMTP id EC6BB1C0008 for ; Fri, 12 May 2023 23:58:17 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=xk2Ibde3; spf=pass (imf21.hostedemail.com: domain of 3mdJeZAMKCOITGGKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--pcc.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3mdJeZAMKCOITGGKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683935898; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=hoXQPwm6eKMpwlE/ffdnArAq29/vm+Ntek3jHZLQ0y3sdniGbUgdPT397pcAvaGvdiNCQG HtU4AiLOGu4lXWjQKSMs6UGhRs3wtYs8A+N23uwPNife9xDa0/UDzgxO8YgvNK8gGPsu2s OECidsMQ8VyAYn5Voa21JHqR1AxAafc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683935898; a=rsa-sha256; cv=none; b=jBMfun9JWkVrQIkwI5iDIVaKDxlsYlQzZXcVEoDX/2x3v7D9/ee9ycHgDVKQk3FNq4kOE5 y4zRWyzyRgfo2U1eU1moBI08TA4D7EFkNXYa4bjnKh4G/tPg4qZyd6icLa5gRmG6VX/23a UVHBtE+rG3S72XryAz99kDQVHjxbMg0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=xk2Ibde3; spf=pass (imf21.hostedemail.com: domain of 3mdJeZAMKCOITGGKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--pcc.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3mdJeZAMKCOITGGKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-ba237aec108so9950819276.3 for ; Fri, 12 May 2023 16:58:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935897; x=1686527897; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:from:to:cc:subject:date :message-id:reply-to; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=xk2Ibde3e8u/s1FXz5MjnaikjmFzTVuGbTz673xLExz96TfA9SY9JBEhrH4pDcD4kc lJcaoTYG0RcJgWFZFSCaY33HyZj2N01rZHV90M6ATudVtPlhyiI8LL94cSLE8hlDZsK1 ohmUhbm4cTQwXS7bDVKRWaW81vYw/Zs2XgvQtpjyOpS39KbdeTP0PlNP0LVOEJX0BiKd Gp/2PGuqyiqk4rIboKMupJ2lV3A59cCft+4hG9Pnr7dP/E1pUxUHHsAIktr3S1az7xY9 Y7C7SJ3J766Sa8Y3zSYkAveN2OM091uDtEnFY90+chuUo5dWOIKtYgSwr2SEMxwfaVJk l+/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935897; x=1686527897; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=XwMV83qzB00xUe1xka9Kex2X8Ec2jkmSilor+2INk6I=; b=Ca/CDFUCOEfhKsPYfMzTgvx9dHljnQZPCxciEMMU3Xz7tIHVfOgZod6zn66Z5Qa5Oh I4tf9AaUPbeO3plcB/quYo8Q0N4bfTw5ZUGDNgk2TAOiNt70imJt04i1VTJN06V1+IeI WfJbKLUXB7309K8Gym0msXZjleAhC3jw0K3JeZxfS5BBx9ddmPVN2fKyoGI2pHsnNg9B I6s5Lg1GpZ4EPlyU8+UG+0MfAItLkbJwAqXdhCra0uDyJHru9rt8ynp6RTEv6Sl8h/Ep N1Fz+2GgUAI4xaIN6TJEvZRehyzHUIbKVy1DurT8hKZFpXDzU2qIR4ZdBZp1emTKB9f6 NF5A== X-Gm-Message-State: AC+VfDwGv22uhlPrefZ5P6T1pkwanwkULKEMAemVDo2VTk5A1Uif3XfY jkhIt7mcJFWABYn2RoDlj/lgeoQ= X-Google-Smtp-Source: ACHHUZ5fNzBCdsm//w4GJKGotggr/NOqYy/clxbht/ilkykWu8XfrBS2AXyT1QiZkNq0D7bzCiaesqo= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a25:d18a:0:b0:ba7:29a9:a471 with SMTP id i132-20020a25d18a000000b00ba729a9a471mr1565248ybg.0.1683935897117; Fri, 12 May 2023 16:58:17 -0700 (PDT) Date: Fri, 12 May 2023 16:57:51 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-3-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 2/3] mm: Call arch_swap_restore() from arch_do_swap_page() and deprecate the latter From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo?= =?utf-8?b?5by16Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vhbi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Stat-Signature: ejpwperuygzbxr6yhyn66ocpgrx1nhaa X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EC6BB1C0008 X-Rspam-User: X-HE-Tag: 1683935897-694209 X-HE-Meta: U2FsdGVkX1/BaSZShXirjv+pVf2DhP4XuNZG8hE13qQvQUSwT6NZ/BxqXCPj8wFUHvOYLncYv+H88XN3rzucBc5CUt71mrVss1reVcqwc6HFhIvQzOE8kFD4qHAted5m7jiQuDMVhcic/zpxhK9iSOgn5URnrfaXVY5Nw3np9WLI66Opa6ecX0SqA4G3b/M3ePiq3GQFXLXjKG30syH/iGYD1Su1lPNFEt4+/QW/C05iPTAyUWCULejpPdNEFrM+rzf3HWKjpeuVQLuGtcXa1Aqc33hvuPppoUJhTAh7r0zF15PiMmrh5UTBJvChfmIirecrPD1RoYeNk+sUmoeS+sUKWtXCau5G/oBUwwrT831pzfTioo1N+t57M3FOgWQhnWt8yzW8iRt/pgAE2/AMxtjvp3cHAjuZ+CI2vT9dI2vDUyzIq4Wff3rCSjbKEmq+zWP75cF+c2QZlbqKIqz/NL1Q8Z6AZi1OCuJuyiJJq0m92n7fHqLZxkSZbjs/fvl3WihfbFLWDLoytUFCFUz5fG0Vu/3/1knZYkzOD2Dhrl47Kh0ALLRMXtpVkUM8pHvNAEOp1+UbJagAP4NxXzMfdinPEGQgm7uQfVnRfn6f2tWcRxryNCU44ORNL/SHYWi5OykiHkOQ5TJpVK8Qha9dzEFKqTTqMsdqBUP2ZhQSixZtfbKtSGJLxnqgF3Hp7p1DweS/Yzgpm9EZlJnHTL0MAKThpjXisUftDq/mIS7s+2JwdtwObH2Y5z8xsbd4kGryub23i8wIyk8Ec/RXk7cdBTZ7eur4IHx4BVHYfoLBicWsjOchaVqbS/H4g5CwxejX61y3aXpG9E8aoK3mTnh3o3wU2fO6hf8d3YahSlMsoceGxU3/ehXY03ZgRK3DtwsLg9Er7Ew3a5y29iKGTw6vk8pDKlMSAYlbqgjF1QIdfG3ITedKxDosNcn4RLP3+aVFf/I3c6aA6/tAeCzeznM TRNmK7Pt pzss8xsnY61hMRjWKTPGiGYryueeW/waApF2FA60JKrP0EM9HiwC+3u3X+lrZHJjF4Vo/s2lUOpy3E9UOZbeXUOoL/lUuSRf/jwoRjmMY8NTU6cMg9+opPx7hmXPoJyfNU7LlOKqujsOF6NC0jYkkZxrgHOJUvwd18svi2c1iracpTwgBDw5ILJpU2mVkxQ/X8yhkMZF6SnoUqEhVRYMkvY14kYihKyK3XLOqntj8Xoy+oqbjuFgD+Z2vt0nIFgsYXk5/pBgX28wFG8HI6hITpA/WW9Bpw5dZUvtnaUeHWahXiy7cJGeq8x9/pxdpo3PlERdpXtJB1ZJlunkPJc3htwO1EsBdFfqgdKmr7uE5Hw6tzQdHN3lvp6Zh8OetTYKII7qfo64a3595+mCnE2IadiFJlwzHogQ0dFIHDl+gVjy2/IGyl/Zp+Uy7pHh3E57EnXnlHAL28NHCrYEFX1WVDZpbPfnPTunGMZL/qYaM5b8hz775fA1PwX1Ww1aO1p1ZY02sbiBYph7JBQvoOGY3BXg80WBlV+j8p52vMAV7cGdLCZv9jGrTfEhXmUUPF7UXnKaD7rVYsU1v4B7bBvsBY4Ds5aKEWMVwxK4PQy7snsPR5nwb6Vkw8Ivv4dk1lHKyP/880vKs1VW5o3etrHGxPOe2yQWyw+RG3D6QmtZSacAplpqgXZLlIFZk7hISbprUtf+SjifNV4pfthcBydxMzj+GWbjFX2w4FYKUu3o7upFed+E95cmhF5oyYMDQ7E+evJweHdwYiTNjjS4d0zEttrvNHrdKLc2InkdXdszdNFPTl9tCwOEQ4T7XZCyApRYxIJCd4ZuUT+Y++nMNVJgB84BTKlac4B1+QzsG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The previous patch made it possible for MTE to restore tags before they are freed by hooking arch_do_swap_page(). However, the arch_do_swap_page() hook API is incompatible with swap restoration in circumstances where we do not have an mm or a vma, such as swapoff with swapped out shmem, and I expect that ADI will currently fail to restore tags in these circumstances. This implies that arch-specific metadata stores ought to be indexed by swap index, as MTE does, rather than by mm and vma, as ADI does, and we should discourage hooking arch_do_swap_page(), preferring to hook arch_swap_restore() instead, as MTE already does. Therefore, instead of directly hooking arch_do_swap_page() for MTE, deprecate that hook, change its default implementation to call arch_swap_restore() and rely on the existing implementation of the latter for MTE. Fixes: c145e0b47c77 ("mm: streamline COW logic in do_swap_page()") Link: https://linux-review.googlesource.com/id/Id2f1ad76eaf606ae210e1d2dd0b7fe287e5f7d87 Signed-off-by: Peter Collingbourne Reported-by: Qun-wei Lin (林群崴) Link: https://lore.kernel.org/all/5050805753ac469e8d727c797c2218a9d780d434.camel@mediatek.com/ Cc: # 6.1 --- include/linux/pgtable.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index c63cd44777ec..fc0259cf60fb 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -740,6 +740,12 @@ static inline int pgd_same(pgd_t pgd_a, pgd_t pgd_b) set_pgd(pgdp, pgd); \ }) +#ifndef __HAVE_ARCH_SWAP_RESTORE +static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio) +{ +} +#endif + #ifndef __HAVE_ARCH_DO_SWAP_PAGE /* * Some architectures support metadata associated with a page. When a @@ -748,14 +754,14 @@ static inline int pgd_same(pgd_t pgd_a, pgd_t pgd_b) * processors support an ADI (Application Data Integrity) tag for the * page as metadata for the page. arch_do_swap_page() can restore this * metadata when a page is swapped back in. + * + * This hook is deprecated. Architectures should hook arch_swap_restore() + * instead, because this hook is not called on all code paths that can + * swap in a page, particularly those where mm and vma are not available + * (e.g. swapoff for shmem pages). */ -static inline void arch_do_swap_page(struct mm_struct *mm, - struct vm_area_struct *vma, - unsigned long addr, - pte_t pte, pte_t oldpte) -{ - -} +#define arch_do_swap_page(mm, vma, addr, pte, oldpte) \ + arch_swap_restore(pte_to_swp_entry(oldpte), page_folio(pte_page(pte))) #endif #ifndef __HAVE_ARCH_UNMAP_ONE @@ -798,12 +804,6 @@ static inline void arch_swap_invalidate_area(int type) } #endif -#ifndef __HAVE_ARCH_SWAP_RESTORE -static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio) -{ -} -#endif - #ifndef __HAVE_ARCH_PGD_OFFSET_GATE #define pgd_offset_gate(mm, addr) pgd_offset(mm, addr) #endif From patchwork Fri May 12 23:57:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 13239993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73194C77B7F for ; Fri, 12 May 2023 23:58:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10964900002; Fri, 12 May 2023 19:58:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B9846B007D; Fri, 12 May 2023 19:58:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9C21900002; Fri, 12 May 2023 19:58:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DA46A6B007B for ; Fri, 12 May 2023 19:58:23 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A9777C0DB0 for ; Fri, 12 May 2023 23:58:23 +0000 (UTC) X-FDA: 80783269686.12.2BCAA85 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf23.hostedemail.com (Postfix) with ESMTP id AED97140011 for ; Fri, 12 May 2023 23:58:20 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=RfEuSDTL; spf=pass (imf23.hostedemail.com: domain of 3m9JeZAMKCOQVIIMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--pcc.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3m9JeZAMKCOQVIIMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683935900; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=xPxkh/k5FbLHJGPzpDqUyF4bvLKX0wjj8oD2WHTrtX8BwJpolnCa4qhTy2RZzgCPpQfkI4 dtWK8FwtdLmU+nO4QcOP3hF0n1ZKDDC30nOTBxb9EiDhnXjXalOMc1Kb/qpY/wkMUk/KaG luNOImgvsjjSD4Kpm+F348R/o0zECp0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=RfEuSDTL; spf=pass (imf23.hostedemail.com: domain of 3m9JeZAMKCOQVIIMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--pcc.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3m9JeZAMKCOQVIIMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--pcc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683935900; a=rsa-sha256; cv=none; b=dZ7GW/9hyq0D83Wz+s8QCie6D3nt5DH3WB0oTet25RGwsISBm9xnqlYsn4Ng2NZUuMPml3 VUiAUYyuxtnJzeUsUoAAFomB8qBeZ2tgS2lHsHYgNgC3Mc9b5dW0zE4ddSffzekOH/wPI5 +v9/F/eI5zgV4ie7xSKnyZnIGVVSsKo= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-517bad1b8c5so9902148a12.0 for ; Fri, 12 May 2023 16:58:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683935899; x=1686527899; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=RfEuSDTLMnNvVhco/REROey0gb2x3qNZHS7QsUUEezpvjObhi40SJ4Ivhu9qrO+zn9 KaL54Or+583uthg14uV25Du+x+MjKeSsXpP0N8kbyqmOimQ32HKrIuItJhzMEe7Z7r00 H3xPmIPgJeHVZwbgHFPY4EK7ygVj2/eXjW5vntomlpbdw4u2rW1l7DywwTG+JbEDqlqW a2pVysEP1gPh7M/+7mMP5dvhKaBf8GtLDWx16ApQeX5HQjqiHJ5Sq4w6ZHe7QyWEHes7 xcFButTqpKqyg6mne6Xbpv8LV/kx6mv3uBW1s8FPhdpT8SGS/NjZYbwltNiI2b33sVlg VO5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683935899; x=1686527899; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PFB1dkpGULUqWzGVGpWTfZD/wGkLnLUL2yYSj9UHJe8=; b=WB2SHWWR4wvPbj/Pny4BIo/qr1m0Tr0S8tdtXOKwqWhkHmd1AVXiIWrXZdq9Zfs2xq dIFkXj/7UBLl1tp0age9W/znQXRgcRzQnsWGyhyNu0/qky7cMMsNI6FExTqsD8zObe0g zDdsvPm4Urv4ORaRx+OdCUzeqzFw2aoaBk+p1SsOL5QelLQoIAB8Sup7jFPRpPCFGHGb Pss3fPo2tO6msTGAwYDJbpP5VLxS3nVqU7vlbBTPnIioFvJhrk1+6snsZs13NbfevsoG d/P/V58C4AQT+9osZ+jTp97c56eWmKEE2TxmYr31U9jNyFJhdgasni2g+aZwE8TV4yc9 klRg== X-Gm-Message-State: AC+VfDzv+DOet0a0iPmCzwkFdMV65UIxGvxTMM+3j7HI1OnHNTCLBIoy TCO0pNBLlHt1EbkZvow8UCRAKA0= X-Google-Smtp-Source: ACHHUZ4Om3weymLriZRqilqiW1Q7IzP8A/Hgna9MQFb9oy4dHoPlAMWv6ovNh/hRL0eehCAtf3LXCLo= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2d3:205:ff6:108b:739d:6a1c]) (user=pcc job=sendgmr) by 2002:a63:151:0:b0:52c:6149:f6be with SMTP id 78-20020a630151000000b0052c6149f6bemr7437419pgb.4.1683935899323; Fri, 12 May 2023 16:58:19 -0700 (PDT) Date: Fri, 12 May 2023 16:57:52 -0700 In-Reply-To: <20230512235755.1589034-1-pcc@google.com> Message-Id: <20230512235755.1589034-4-pcc@google.com> Mime-Version: 1.0 References: <20230512235755.1589034-1-pcc@google.com> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Subject: [PATCH 3/3] arm64: mte: Simplify swap tag restoration logic and fix uninitialized tag issue From: Peter Collingbourne To: Catalin Marinas Cc: Peter Collingbourne , " =?utf-8?b?UXVuLXdlaSBMaW4gKA==?= =?utf-8?b?5p6X576k5bS0KQ==?= " , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, "surenb@google.com" , "david@redhat.com" , " =?utf-8?b?Q2hpbndlbiBDaGFuZyAo?= =?utf-8?b?5by16Yym5paHKQ==?= " , "kasan-dev@googlegroups.com" , " =?utf-8?b?S3Vhbi1ZaW5nIExlZSAo5p2O5Yag56mOKQ==?= " , " =?utf-8?b?Q2FzcGVyIExpICjmnY7kuK3mpq4p?= " , "gregkh@linuxfoundation.org" , vincenzo.frascino@arm.com, Alexandru Elisei , will@kernel.org, eugenis@google.com, Steven Price , stable@vger.kernel.org X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: AED97140011 X-Stat-Signature: ghtpaqpgf3zzm5ay6j49bzcdkg7gtirc X-HE-Tag: 1683935900-165058 X-HE-Meta: U2FsdGVkX1+FsXsFmEiHqlZRaavuAE1XxzHRvRXGJC4GQ7rGaROlqoji3K74Rfpark8ENguznFB7GklgZp1HepkaAXKPVjivUdVJ4ypxgDiIP5m3ki+PyGwYFeI4vLDWY27Nol0Or4ohndl5UvXPVzxcBZ+/3AyS+FUrQzyjmnS1Klv0s4AeACP5MufBR4Z8HKIJzIP8dIyf4WI+/32xSet4OxEIRP4o8arJszBGjj46rHdk1jSB9HPbFf3U+odttlyDpkNYICJ0ETI+iJPyb0wZgOuN9xrZd15u5iOSTdAsL55HMx4xNG3H9gJgD/TdpTGN3w0c7PqWlGS4DCb0NG2SLPZ2Yx5Y1opR7SouusMhQbIs3nwqvz4mHLYeLEDIQnVcPabnrOrOL/Agz8vx61w9wSjvumVRNftu8rnAojI7ujiFnl4fkok9AoqVnUAa7rO6KomGQhHFCFtfgszxUOiBGhYSf0eDkCEFdfwmBtW2YTEsr1d6uWkSt3fGuvg5TLLdaAdMl98yQYLfDVYw8s1HwPpYqzZYRrMs975JYgqUVLTj1mFqQML8Czflmqott6Yz8yfxBePmJHICZ9SulNsZYHPRH6k1Ogk/ZPpzz7FwxKvPshBvgaanKWDxgHJCCUvV++K3ml+5pZgx5UiSmZC3VNq/7OMnpPxnyKLnG94Qrjwf26aoe7BpSlS/Xd/dekWcTzujTIt8WY4Fl4ls88MUXTnEiIbyd1goqKLqHayfJttiskZnU/ecMfjxoRJ/BeZZJJLml4DAaavDe5Paf7zPWwXwapS41YgWZHtWLpocUjghIF3RfZ5Bj4X/RuZpEukfKZPtuS2rM4OBpx4N7sWHMstDM1DENxmQQb6uVBsEkJR13/w7weBpn4hkFYOJ6Xkou2TvMnFWuyd+KKoKlJf2SQI5Og6e6/tWhJZRynmvZIXnITfbA01s30RUYgJz5JsEXoXstxBpLldhFRg v5dRqmzF JPSh/3/0iwLMkG9tATkST8bDeWq+rtjRU6imqPgEG8WYo3mExRsT0IkRz91mYXAZ/yMV8X6raV4FgeK4D+8y5huH7zBn8hbHI2ULznGpCaz9wWGVdV/YU0uwEk/uhAA9x63w/q9vYePt7vp88tcYWt0tpBDwKyNgnyXI4ZbbceufNXZ6x4UfovNMuf9Vh9FEvNnmM5Nawbg4wzxZTjSiAvT/n1412hf1TcjZClGe3FXOEU0/uaZ821+3aFYKeBSzz4IVGT3kl3vVA4IdDM3JcTtmKyFxGFbKo1NcreT4IPyihK0ZBrAqy5ElrVA+qmdpbIgdd+mT4bVRrgZ4nZdET1j/Q/pg78Iu/ujP/5+PvngDfGtjUpeGSqWR3mItm8e7ykPozXG4T5DbRgIUOaw2Bo40rnpw6jQU057FoOrW2IMWB9EELQ3Huh0GlmoFksP/97o5DibVNAnE7EnOeg9vuWWfIkSIMNc2nMKWWITSgXTJO6o/77tSfGsC0ILK5nbnmSLsXyrc7/2iKkwqqZLQpG8ZdqSfwlaZvcYTggl9P733JIAx0kCQLLim+AwmwrxGP+LSOEdNzvKcFyNRrkWPfZdZwjiewZA0zdZqy0WeJKV5N78nIYBJ0Y8bzWwPexlcl/rkB2BplPYbhH5fQlGcl5I92BiiIUYvXuleG/Yjf1fLQ8kpDrfT/CRUISIHTjA9IZc4qJBlcy1PI78Ow4pu+VI5q7naGIu8KeWG6mRIPCakv927NypAVSeb9uQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As a result of the previous two patches, there are no circumstances in which a swapped-in page is installed in a page table without first having arch_swap_restore() called on it. Therefore, we no longer need the logic in set_pte_at() that restores the tags, so remove it. Because we can now rely on the page being locked, we no longer need to handle the case where a page is having its tags restored by multiple tasks concurrently, so we can slightly simplify the logic in mte_restore_tags(). This patch also fixes an issue where a page can have PG_mte_tagged set with uninitialized tags. The issue is that the mte_sync_page_tags() function sets PG_mte_tagged if it initializes page tags. Then we return to mte_sync_tags(), which sets PG_mte_tagged again. At best, this is redundant. However, it is possible for mte_sync_page_tags() to return without having initialized tags for the page, i.e. in the case where check_swap is true (non-compound page), is_swap_pte(old_pte) is false and pte_is_tagged is false. So at worst, we set PG_mte_tagged on a page with uninitialized tags. This can happen if, for example, page migration causes a PTE for an untagged page to be replaced. If the userspace program subsequently uses mprotect() to enable PROT_MTE for that page, the uninitialized tags will be exposed to userspace. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/I8ad54476f3b2d0144ccd8ce0c1d7a2963e5ff6f3 Fixes: e059853d14ca ("arm64: mte: Fix/clarify the PG_mte_tagged semantics") Cc: # 6.1 --- The Fixes: tag (and the commit message in general) are written assuming that this patch is landed in a maintainer tree instead of "arm64: mte: Do not set PG_mte_tagged if tags were not initialized". arch/arm64/include/asm/mte.h | 4 ++-- arch/arm64/include/asm/pgtable.h | 14 ++------------ arch/arm64/kernel/mte.c | 32 +++----------------------------- arch/arm64/mm/mteswap.c | 7 +++---- 4 files changed, 10 insertions(+), 47 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index 20dd06d70af5..dfea486a6a85 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -90,7 +90,7 @@ static inline bool try_page_mte_tagging(struct page *page) } void mte_zero_clear_page_tags(void *addr); -void mte_sync_tags(pte_t old_pte, pte_t pte); +void mte_sync_tags(pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void mte_thread_init_user(void); void mte_thread_switch(struct task_struct *next); @@ -122,7 +122,7 @@ static inline bool try_page_mte_tagging(struct page *page) static inline void mte_zero_clear_page_tags(void *addr) { } -static inline void mte_sync_tags(pte_t old_pte, pte_t pte) +static inline void mte_sync_tags(pte_t pte) { } static inline void mte_copy_page_tags(void *kto, const void *kfrom) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b6ba466e2e8a..efdf48392026 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -337,18 +337,8 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, * don't expose tags (instruction fetches don't check tags). */ if (system_supports_mte() && pte_access_permitted(pte, false) && - !pte_special(pte)) { - pte_t old_pte = READ_ONCE(*ptep); - /* - * We only need to synchronise if the new PTE has tags enabled - * or if swapping in (in which case another mapping may have - * set tags in the past even if this PTE isn't tagged). - * (!pte_none() && !pte_present()) is an open coded version of - * is_swap_pte() - */ - if (pte_tagged(pte) || (!pte_none(old_pte) && !pte_present(old_pte))) - mte_sync_tags(old_pte, pte); - } + !pte_special(pte) && pte_tagged(pte)) + mte_sync_tags(pte); __check_safe_pte_update(mm, ptep, pte); diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index f5bcb0dc6267..c40728046fed 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -35,41 +35,15 @@ DEFINE_STATIC_KEY_FALSE(mte_async_or_asymm_mode); EXPORT_SYMBOL_GPL(mte_async_or_asymm_mode); #endif -static void mte_sync_page_tags(struct page *page, pte_t old_pte, - bool check_swap, bool pte_is_tagged) -{ - if (check_swap && is_swap_pte(old_pte)) { - swp_entry_t entry = pte_to_swp_entry(old_pte); - - if (!non_swap_entry(entry)) - mte_restore_tags(entry, page); - } - - if (!pte_is_tagged) - return; - - if (try_page_mte_tagging(page)) { - mte_clear_page_tags(page_address(page)); - set_page_mte_tagged(page); - } -} - -void mte_sync_tags(pte_t old_pte, pte_t pte) +void mte_sync_tags(pte_t pte) { struct page *page = pte_page(pte); long i, nr_pages = compound_nr(page); - bool check_swap = nr_pages == 1; - bool pte_is_tagged = pte_tagged(pte); - - /* Early out if there's nothing to do */ - if (!check_swap && !pte_is_tagged) - return; /* if PG_mte_tagged is set, tags have already been initialised */ for (i = 0; i < nr_pages; i++, page++) { - if (!page_mte_tagged(page)) { - mte_sync_page_tags(page, old_pte, check_swap, - pte_is_tagged); + if (try_page_mte_tagging(page)) { + mte_clear_page_tags(page_address(page)); set_page_mte_tagged(page); } } diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c index cd508ba80ab1..3a78bf1b1364 100644 --- a/arch/arm64/mm/mteswap.c +++ b/arch/arm64/mm/mteswap.c @@ -53,10 +53,9 @@ void mte_restore_tags(swp_entry_t entry, struct page *page) if (!tags) return; - if (try_page_mte_tagging(page)) { - mte_restore_page_tags(page_address(page), tags); - set_page_mte_tagged(page); - } + WARN_ON_ONCE(!try_page_mte_tagging(page)); + mte_restore_page_tags(page_address(page), tags); + set_page_mte_tagged(page); } void mte_invalidate_tags(int type, pgoff_t offset)