From patchwork Fri Jul 7 21:55:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42AF7C001E0 for ; Fri, 7 Jul 2023 21:55:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8601A6B0074; Fri, 7 Jul 2023 17:55:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 810B08D0002; Fri, 7 Jul 2023 17:55:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68C258D0001; Fri, 7 Jul 2023 17:55:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 570556B0074 for ; Fri, 7 Jul 2023 17:55:48 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 34A80160149 for ; Fri, 7 Jul 2023 21:55:48 +0000 (UTC) X-FDA: 80986173576.01.1614BD9 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf11.hostedemail.com (Postfix) with ESMTP id 70A7440002 for ; Fri, 7 Jul 2023 21:55:46 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=QMS3WZg3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 34YmoZA0KCMIi5mtzi0u200mvowwotm.kwutqv25-uus3iks.wzo@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=34YmoZA0KCMIi5mtzi0u200mvowwotm.kwutqv25-uus3iks.wzo@flex--axelrasmussen.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766946; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B70uVhggCX8fJNIY7gzg1xZOvXHfmALP4G5DZO4j1xc=; b=E3bA4O8wdXksccsA2MNhfS1ZUCpfBG0/KVKsAglEaaZ/g+4YpDXciUscakBdlHA0EagFeK r2DtUCnlllxMyg2MqivO4WGpaHLgPwSwG4sTwaheLT93UIDtjIzKUGZG/kqMSbMAIJixLc cqp5VR3yzRBwXmb16yEOJ/b4cKHS3CQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=QMS3WZg3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 34YmoZA0KCMIi5mtzi0u200mvowwotm.kwutqv25-uus3iks.wzo@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=34YmoZA0KCMIi5mtzi0u200mvowwotm.kwutqv25-uus3iks.wzo@flex--axelrasmussen.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766946; a=rsa-sha256; cv=none; b=Ls8axzLwd4lgwUqJADXn4KDyaajrgYMSokA23xJ67sKhZF06z+zWl6uo4iSLfHsfnhm3EQ 20rx2Kw+x8OfUjTvaMdxkTigT723fUOzAlVDS8SDH0OFYsQ+QU3wHFyCBsxpWf4JinFMHD CEosqEj9wLM9b1KzRKd90jJJP1RgBig= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-c4e77129acfso2658463276.3 for ; Fri, 07 Jul 2023 14:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766945; x=1691358945; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=B70uVhggCX8fJNIY7gzg1xZOvXHfmALP4G5DZO4j1xc=; b=QMS3WZg30DM3nklkucAdvPFP0qPy5/mGc8Rq4F6Mwo5PRAhSuGlTM9nPNReje+qJWC f1YvL6DSuMLBGxMiN9ZA+jOZADIPR58nXsdTRYufCEWE9hX/dgGQPjcnQHU/ptHtbA+9 K379J7XgV5wYOIAA5PCuZJq/1ezJ6yKLgiGrkY3x1i1vRjn3NPhlmy7BCmaa8YXJTdTC iU2C/h2e81QUxqrVVHcgF7b7xoU5eKKYm/ylPoAsrvZChDqpRSthCfTNtjv/ZaK4k4Vo 6icDFibzuF8Uix1QTjV4alO5LLAQ7WtSuUVkGB8I2FcVe15qGjiEueapzQTQ/lMJWfMt 9vuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766945; x=1691358945; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=B70uVhggCX8fJNIY7gzg1xZOvXHfmALP4G5DZO4j1xc=; b=N/Q0G3G+JZoDAMz4aeAX1P2r5bqnth8N9BgkGPSbIjFDyPcAzXR4oe09A6Je2INQ3e pPl9UW27X1+DgGf5pUhdYwAejq6AKW1EJ0gLD9hO1s3S+/676u9HBl0a6n+kpmHq0I7V Jo7tBapb0wJdOENS/MfJzSm3OeiuBPi1dymPpgege62YcZ5KdEJE5NKfDwIgIM6eQAKy I6qIx4XEGhREmIAaYem8KGxPJDIZ+nyK9I4EMFiBb9dHITzYSKDjoZeoNL9UXskM1PvB 3hrAp7xEiT4KXLk4MvlRwBmDtoJ5hfw+gmi0sL879uzM5iN315GagfwoVWrBruve/Nkd TR0w== X-Gm-Message-State: ABy/qLbPqf7wbF61hueRpuQCx1l6OK8qQAMpGtL410eMdlMjmsbuStXI +cOM/LVJORL3x8oyEp86p29eQyg2S/AZUxaXEQp7 X-Google-Smtp-Source: APBJJlFO+dy0OuzOto98GK6Ib07YvcCh2JXDJhvTyNfjxaAmDvRpFN8QngXCTxnAVEaKVcdA8r/3KCXJmrdI1PcwvNw1 X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a25:694e:0:b0:c72:2386:7d26 with SMTP id e75-20020a25694e000000b00c7223867d26mr3509ybc.0.1688766945429; Fri, 07 Jul 2023 14:55:45 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:33 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-2-axelrasmussen@google.com> Subject: [PATCH v4 1/8] mm: make PTE_MARKER_SWAPIN_ERROR more general From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Queue-Id: 70A7440002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: z13wqhk1sgusgufgfoige3wdxobdg9zw X-HE-Tag: 1688766946-431699 X-HE-Meta: U2FsdGVkX1+7efyNDlLTAhLW8z29CiaCjxJRCcPZ7GWSQA3cm420OcCfbeNsfpp6EzvXZd370voIci5f8dVpyTkNWgyGI+fcU1jyNH2/9pWdh8eGd+0LNTG8fVFxs0m5CLgkesZoT9lArE/bQzs16kQlPq7v1bEFR/UCb58m28nkE6s96AgLAEPgc+JYQ+6NnoU4r7xiolmoS3H6yW1JsFB3e3ZyTx5A0NK9bYf+IZCsqY8CsVhQSvbtXd9nD/8j3kYXOgYTvWOgk91TZOK9rJChqcGSutg+AzU43aW6cYrxIWtrRBCa1gr3ad2NGMyFRLzsu5TPqjYFZ+iVZVcg49L3T2S3UzZUJx8boNxjhnKoowOdSLlQerEu3t3i0Sc/CadR+UexAz6stmkhZo2El1aJPDBNnGSBlIee6hA8hsVuiCfpPTrQz1824vmqi2Tw+lRb7s9N2wEAAzMvo+c/C8LG9WeabQfFi/dnoroVhJ+D011j81bJJVA26804xiCVhfH+3dsCV2FxOoaql3rXJiJbAiAiE/4G6n/vkSRrQ8CJXQo4p6F71uIhXsTsIfSOURnuZ+fIWGTjQAihKGjK8nRU7EZfJYKJwvZOfDoHguHlLXcusFh9G7u659dTqhxXsi5qN14fRSyqTBpJ5qRtesokAD4mwPxEMx4WQW2stRgD12JIm/fwM93r5jvNrKfVCk3Cjo80PDlerTlmL2oxdpFYYixi9GoymFBAet/3bze5Ksccm/mZ4vpYjwp6i82GxMwZmHdJ/PXCmEn8xzDCbvgZ+yflZ83RoF+gl3Ga1c6D5HFMsqdAya2z2/ziopa24JWKGfqtrxdBM5SsBeiZ1XUXHiYvdwm8ywD7z6vKumsAw0IuYMF8vJpm0UCR0YvTHUZC/Z+4vbE18VJv5MnknyV6q+rA600WMK2kiIL8lU13Gr20QneoGeaj/g1VH+lnvkmtiDYMa8HOJKn2Ufz zHloxMyO MzZ11CTFfW3DhprB/ztCN/tq/U1rE3MVwIC9DJuaL3EvfPIHs6S+J+jSbFj+YZmi/vKBD5gIAXDgzKEUn4gQjcQz68ZX2UCrtCJ1AVQrOBq5nM4o8u7W8WbgRrHKUMJGd80gz9VqkOVfliEADSVN6m1JPHC8bRm8WUYFhd4Sl1iwLhvziGLReJTfV/pjNIcayF1H/BdnSVsyPEZwKtLP6otmyZYwFXCSAUmLWwQXNirHZ5duFy78J1hvsIRMxVQ571DMCMOqtb63Dg2RfObr+s3g70s7FyA8Zj9HVsmya9uQRXTXUWsMyyMBbAzjDI7ZZG+6m2N6r9b2vbj2XZRUUR85GxSIWnovT8yVoUuC73fB7PuCnskTqUVleE4ozrvQEKgZAgMFlAsIeEC93HJb/tIM5uKuhcVoM8wrVzuLIne62yGcAhUY82D6i3FW9PqWAIqeUOCbTHAnX4Rd48XL6U2PZ2kHfMJnBPl2gp4sVa/8LiuL/aBVVhYJiy462qiTM2ZeEUzUUjKFkW1r+wMEWAzoXss1Cwkjbsr4/llVsyEPO97XrtFfFb+U2kJAkHIp51bwOxzoq9F9pNew93dR9XsP2AWUmEEAmRt00Xzy28rSC6QYgVYrhS3be/5tuOTVquG2BzFo2IIFguKlg7nLRy750iZ8ZQz26qq1XsE8G+C5rM8xKj/YpwJUWSibtWCLeY2lqmFJkt208WnVvz4aB3Jfxww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Future patches will re-use PTE_MARKER_SWAPIN_ERROR to implement UFFDIO_POISON, so make some various preparations for that: First, rename it to just PTE_MARKER_POISONED. The "SWAPIN" can be confusing since we're going to re-use it for something not really related to swap. This can be particularly confusing for things like hugetlbfs, which doesn't support swap whatsoever. Also rename some various helper functions. Next, fix pte marker copying for hugetlbfs. Previously, it would WARN on seeing a PTE_MARKER_SWAPIN_ERROR, since hugetlbfs doesn't support swap. But, since we're going to re-use it, we want it to go ahead and copy it just like non-hugetlbfs memory does today. Since the code to do this is more complicated now, pull it out into a helper which can be re-used in both places. While we're at it, also make it slightly more explicit in its handling of e.g. uffd wp markers. For non-hugetlbfs page faults, instead of returning VM_FAULT_SIGBUS for an error entry, return VM_FAULT_HWPOISON. For most cases this change doesn't matter, e.g. a userspace program would receive a SIGBUS either way. But for UFFDIO_POISON, this change will let KVM guests get an MCE out of the box, instead of giving a SIGBUS to the hypervisor and requiring it to somehow inject an MCE. Finally, for hugetlbfs faults, handle PTE_MARKER_POISONED, and return VM_FAULT_HWPOISON_LARGE in such cases. Note that this can't happen today because the lack of swap support means we'll never end up with such a PTE anyway, but this behavior will be needed once such entries *can* show up via UFFDIO_POISON. Signed-off-by: Axel Rasmussen Acked-by: Peter Xu --- include/linux/mm_inline.h | 19 +++++++++++++++++++ include/linux/swapops.h | 15 ++++++++++----- mm/hugetlb.c | 32 +++++++++++++++++++++----------- mm/madvise.c | 2 +- mm/memory.c | 15 +++++++++------ mm/mprotect.c | 4 ++-- mm/shmem.c | 4 ++-- mm/swapfile.c | 2 +- 8 files changed, 65 insertions(+), 28 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 21d6c72bcc71..a86c84600787 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -523,6 +523,25 @@ static inline bool mm_tlb_flush_nested(struct mm_struct *mm) return atomic_read(&mm->tlb_flush_pending) > 1; } +/* + * Computes the pte marker to copy from the given source entry into dst_vma. + * If no marker should be copied, returns 0. + * The caller should insert a new pte created with make_pte_marker(). + */ +static inline pte_marker copy_pte_marker( + swp_entry_t entry, struct vm_area_struct *dst_vma) +{ + pte_marker srcm = pte_marker_get(entry); + /* Always copy error entries. */ + pte_marker dstm = srcm & PTE_MARKER_POISONED; + + /* Only copy PTE markers if UFFD register matches. */ + if ((srcm & PTE_MARKER_UFFD_WP) && userfaultfd_wp(dst_vma)) + dstm |= PTE_MARKER_UFFD_WP; + + return dstm; +} + /* * If this pte is wr-protected by uffd-wp in any form, arm the special pte to * replace a none pte. NOTE! This should only be called when *pte is already diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 4c932cb45e0b..bff1e8d97de0 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -393,7 +393,12 @@ static inline bool is_migration_entry_dirty(swp_entry_t entry) typedef unsigned long pte_marker; #define PTE_MARKER_UFFD_WP BIT(0) -#define PTE_MARKER_SWAPIN_ERROR BIT(1) +/* + * "Poisoned" here is meant in the very general sense of "future accesses are + * invalid", instead of referring very specifically to hardware memory errors. + * This marker is meant to represent any of various different causes of this. + */ +#define PTE_MARKER_POISONED BIT(1) #define PTE_MARKER_MASK (BIT(2) - 1) static inline swp_entry_t make_pte_marker_entry(pte_marker marker) @@ -421,15 +426,15 @@ static inline pte_t make_pte_marker(pte_marker marker) return swp_entry_to_pte(make_pte_marker_entry(marker)); } -static inline swp_entry_t make_swapin_error_entry(void) +static inline swp_entry_t make_poisoned_swp_entry(void) { - return make_pte_marker_entry(PTE_MARKER_SWAPIN_ERROR); + return make_pte_marker_entry(PTE_MARKER_POISONED); } -static inline int is_swapin_error_entry(swp_entry_t entry) +static inline int is_poisoned_swp_entry(swp_entry_t entry) { return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_SWAPIN_ERROR); + (pte_marker_get(entry) & PTE_MARKER_POISONED); } /* diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bce28cca73a1..66225b21c64e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -5101,15 +5102,12 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, entry = huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry); } else if (unlikely(is_pte_marker(entry))) { - /* No swap on hugetlb */ - WARN_ON_ONCE( - is_swapin_error_entry(pte_to_swp_entry(entry))); - /* - * We copy the pte marker only if the dst vma has - * uffd-wp enabled. - */ - if (userfaultfd_wp(dst_vma)) - set_huge_pte_at(dst, addr, dst_pte, entry); + pte_marker marker = copy_pte_marker( + pte_to_swp_entry(entry), dst_vma); + + if (marker) + set_huge_pte_at(dst, addr, dst_pte, + make_pte_marker(marker)); } else { entry = huge_ptep_get(src_pte); pte_folio = page_folio(pte_page(entry)); @@ -6090,14 +6088,26 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, } entry = huge_ptep_get(ptep); - /* PTE markers should be handled the same way as none pte */ - if (huge_pte_none_mostly(entry)) + if (huge_pte_none_mostly(entry)) { + if (is_pte_marker(entry)) { + pte_marker marker = + pte_marker_get(pte_to_swp_entry(entry)); + + if (marker & PTE_MARKER_POISONED) { + ret = VM_FAULT_HWPOISON_LARGE; + goto out_mutex; + } + } + /* + * Other PTE markers should be handled the same way as none PTE. + * * hugetlb_no_page will drop vma lock and hugetlb fault * mutex internally, which make us return immediately. */ return hugetlb_no_page(mm, vma, mapping, idx, address, ptep, entry, flags); + } ret = 0; diff --git a/mm/madvise.c b/mm/madvise.c index 886f06066622..a317aa0a92b8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -660,7 +660,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, free_swap_and_cache(entry); pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); } else if (is_hwpoison_entry(entry) || - is_swapin_error_entry(entry)) { + is_poisoned_swp_entry(entry)) { pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); } continue; diff --git a/mm/memory.c b/mm/memory.c index 0ae594703021..6309a4b9a79d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -860,8 +860,11 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, return -EBUSY; return -ENOENT; } else if (is_pte_marker_entry(entry)) { - if (is_swapin_error_entry(entry) || userfaultfd_wp(dst_vma)) - set_pte_at(dst_mm, addr, dst_pte, pte); + pte_marker marker = copy_pte_marker(entry, dst_vma); + + if (marker) + set_pte_at(dst_mm, addr, dst_pte, + make_pte_marker(marker)); return 0; } if (!userfaultfd_wp(dst_vma)) @@ -1500,7 +1503,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, !zap_drop_file_uffd_wp(details)) continue; } else if (is_hwpoison_entry(entry) || - is_swapin_error_entry(entry)) { + is_poisoned_swp_entry(entry)) { if (!should_zap_cows(details)) continue; } else { @@ -3647,7 +3650,7 @@ static vm_fault_t pte_marker_clear(struct vm_fault *vmf) * none pte. Otherwise it means the pte could have changed, so retry. * * This should also cover the case where e.g. the pte changed - * quickly from a PTE_MARKER_UFFD_WP into PTE_MARKER_SWAPIN_ERROR. + * quickly from a PTE_MARKER_UFFD_WP into PTE_MARKER_POISONED. * So is_pte_marker() check is not enough to safely drop the pte. */ if (pte_same(vmf->orig_pte, ptep_get(vmf->pte))) @@ -3693,8 +3696,8 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) return VM_FAULT_SIGBUS; /* Higher priority than uffd-wp when data corrupted */ - if (marker & PTE_MARKER_SWAPIN_ERROR) - return VM_FAULT_SIGBUS; + if (marker & PTE_MARKER_POISONED) + return VM_FAULT_HWPOISON; if (pte_marker_entry_uffd_wp(entry)) return pte_marker_handle_uffd_wp(vmf); diff --git a/mm/mprotect.c b/mm/mprotect.c index 6f658d483704..5c3112d92466 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -230,10 +230,10 @@ static long change_pte_range(struct mmu_gather *tlb, newpte = pte_swp_mkuffd_wp(newpte); } else if (is_pte_marker_entry(entry)) { /* - * Ignore swapin errors unconditionally, + * Ignore error swap entries unconditionally, * because any access should sigbus anyway. */ - if (is_swapin_error_entry(entry)) + if (is_poisoned_swp_entry(entry)) continue; /* * If this is uffd-wp pte marker and we'd like diff --git a/mm/shmem.c b/mm/shmem.c index 2f2e0e618072..ebfde8416bb3 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1707,7 +1707,7 @@ static void shmem_set_folio_swapin_error(struct inode *inode, pgoff_t index, swp_entry_t swapin_error; void *old; - swapin_error = make_swapin_error_entry(); + swapin_error = make_poisoned_swp_entry(); old = xa_cmpxchg_irq(&mapping->i_pages, index, swp_to_radix_entry(swap), swp_to_radix_entry(swapin_error), 0); @@ -1752,7 +1752,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, swap = radix_to_swp_entry(*foliop); *foliop = NULL; - if (is_swapin_error_entry(swap)) + if (is_poisoned_swp_entry(swap)) return -EIO; si = get_swap_device(swap); diff --git a/mm/swapfile.c b/mm/swapfile.c index 8e6dde68b389..3dbc6d37df60 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1773,7 +1773,7 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, swp_entry = make_hwpoison_entry(swapcache); page = swapcache; } else { - swp_entry = make_swapin_error_entry(); + swp_entry = make_poisoned_swp_entry(); } new_pte = swp_entry_to_pte(swp_entry); ret = 0; From patchwork Fri Jul 7 21:55:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A19BC00528 for ; Fri, 7 Jul 2023 21:55:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 860058D0001; Fri, 7 Jul 2023 17:55:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 811A16B0078; Fri, 7 Jul 2023 17:55:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63B7D8D0001; Fri, 7 Jul 2023 17:55:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 54ECE6B0075 for ; Fri, 7 Jul 2023 17:55:50 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 212031A015D for ; Fri, 7 Jul 2023 21:55:50 +0000 (UTC) X-FDA: 80986173660.22.3DAC0EE Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf16.hostedemail.com (Postfix) with ESMTP id 47189180007 for ; Fri, 7 Jul 2023 21:55:47 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=NOEsj6Ql; spf=pass (imf16.hostedemail.com: domain of 344moZA0KCMQk7ov1k2w422oxqyyqvo.mywvsx47-wwu5kmu.y1q@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=344moZA0KCMQk7ov1k2w422oxqyyqvo.mywvsx47-wwu5kmu.y1q@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766948; a=rsa-sha256; cv=none; b=WNzOCsLwNQiIN21SZ1HubQb71chGBwZ8Fm8nIVIoFsYNxS5tBl+3ZCWaiNM/mz4hYnta2X y92+Gut49PTokKQlmerhys8Z+5Aiy4DJRFRk1hvFee7TGsRIrm5P0LwudgdyQlfHLEKQmr FIsA+8zdEovETyr8IilJ8Nt30Go4AqY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=NOEsj6Ql; spf=pass (imf16.hostedemail.com: domain of 344moZA0KCMQk7ov1k2w422oxqyyqvo.mywvsx47-wwu5kmu.y1q@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=344moZA0KCMQk7ov1k2w422oxqyyqvo.mywvsx47-wwu5kmu.y1q@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766948; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NbMcVYI2pBRE777guUXN8FaAW75orbR1/eldPHt/4Qo=; b=0BQlu1NTDdYPL9ij8eW19oma8smY0EotCPpJ6KXW73D1wPD78ZfKVJUaOIYY225BvC4aiF G86ZrOGIUafSHtPQ+XGmJTp/70glddyyRgIF+VWKWPOh0N6wIS1JHkB4GEgIHXyJltTz70 LQTFE1uC9SucBWeayU6HGyT4icwxoug= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c5c8d00ce50so2499226276.1 for ; Fri, 07 Jul 2023 14:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766947; x=1691358947; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NbMcVYI2pBRE777guUXN8FaAW75orbR1/eldPHt/4Qo=; b=NOEsj6QlU33fkoMHjc+WtoODa05QBYjoZyIbXV7SYMyRK4hq0SF4tb/6rEhclUqRYC mOAiI18o28n3yIQcsYTOwNtyQs9I4Y75+l2ujaQgav71uvbNCaDX8ZAB71qsYgJ7+xTa uxTM2xxcQMPCK82+u5G0FIcYmJndcJvSC2j4rI/o0BwYIadvkUVSbCsJTgLi+uxz3KVd Q99KgDvVH0Fhf1f3UtgXY26LR8Gv38XOQazfPV3NexlCyCTv+Yqi8QmGof9JmvYLxs5o 0PxXYucNeE0EtG/ra5xrr2THGEnwJn3zj/1ct12e0dwMeVPDE0288jdN/lwXa8MEHOhK 4RFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766947; x=1691358947; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NbMcVYI2pBRE777guUXN8FaAW75orbR1/eldPHt/4Qo=; b=JhH5l3LarXJuwkfInzKVAaLHOXPS0jvkjv1tQNTvZBKdK2tP3wphg0ed/uOYSRW0Va yJS+syzWGq7mFIpWXKixxEGn5l03YVY4f1O/ztB8e0v3f/fT7wQKu/NW/grS0d8YacSP 8fzU46iatAMZsInI40a9undh6k40OYXdnyNHLhiXXhrzbWAzNxK7Me93DKis2ibYlMRb OdD9OuEfW4WlzyLxt0CNkcXFIdNVvHP6H2zRr+p0y9ILDEUklIMY/80TtHml7UO5EKWJ M5nuKvt9QXZ4deeoh6eWyL6z98nnR6kwED7/1wrySwIPrhMmsoYF50+G9FsS8mCdLkgg 4yGQ== X-Gm-Message-State: ABy/qLZHyaq3WTuNloISCKZ4nMZIQUumD6xq29MFujYXwpgwURJNhOwu eDMqktaUsqXWipo7AMSBh/xxILoWaX6fDE+6FIAP X-Google-Smtp-Source: APBJJlEoVC4taEJfd9VKK1x5AQ0VZDBfiUiNYwaYXwEPy1wLKPyTHC7hYYwTD01ltEHbhNSVCtGCPYme0YQ0PD4Gp2hC X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a25:ac9e:0:b0:c5b:c4:1789 with SMTP id x30-20020a25ac9e000000b00c5b00c41789mr33953ybi.1.1688766947243; Fri, 07 Jul 2023 14:55:47 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:34 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-3-axelrasmussen@google.com> Subject: [PATCH v4 2/8] mm: userfaultfd: check for start + len overflow in validate_range From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 47189180007 X-Stat-Signature: sh5wbddsfqkk6ezp9tmtr9azczte53df X-Rspam-User: X-HE-Tag: 1688766947-880307 X-HE-Meta: U2FsdGVkX1+JcFKjI8OssOIsDU0RF1go0A8b1zOSytUshGL5sy0nxrGNgvvOEsULIipbt29CLB/WkB1z/CMhZYOMsYPN+IkRhvSrXxHGx+eSV3dUxOxs95kb8znP5Iw7alJDhficSmxnXMB0J1E8erZNmzXQbzInyqutZQpc7gcUQcsXGirJ+6P7bt6u7nOp/158ElVZgAOO88/88NF0GvBy77rtyNMDRc+ipdoIl4qZ2k93UrRUzUjwvul5iNAxhRvTNjqfI6kyDT5mzhgiSnO2dLYfDVXEX27H/LdBNE0vC6Vh2LS0/votWZQU+FG9vKYXVVHR3ACj/cJBIUCfZFVOCjVBOroTMZSNIHG0IGKp8m7zCg20V9rSNEs5zXLxcQE2RX1CNOxY4SJzlGRmGFu212QA/Y17b9iSFR033IaQ+ZDapsEPn+wuKH3uhCFTak03sb6G7lJFdQ6uHU4tkvfT1dAiZIJ/el6W2q2Hb+7vcxdvRxE9ELNoOMvXYz8oD3KAsMSqkEZnGirqZaLvPUiWOlCx15ZVYMI1BpfAPk6nQkNslh+7khbrV6UXV/oULOlcznIEtsEVlmmjBkh82tm+PXvMLgeLE7MA41+XZgZ1DiHO8hlhGvNFdpVv2VW22kDTC6MmMerDmUCB/EOrgvaQ3hmG2eP/zeaP7+63uQIng/8rpmhQe+4nKu4zFuFgiK8BUlUVvsP0AePuzJsFNSJVqNhMaKVZsSICQAJ9NRhxjslZQJMyKXSyPGYdxQLy9y77QoVED2Xah/OBxbLyTLboAKCS0862Njg8SdvTrTbaYhKdcYt2QyauAuWJelKQFpXcpTxOR975U09zVjksMPVlkS9XKhQ++nyUBXdDA63P6lnzf2Q7tYHsUTzw4CCnYmOTinojY3mmmXaI5BAg/0cCMfb/6aJtdvm0XP6ob5mqHe8CidGQE7alGyegHExNbOHR+cK2vNgFSt3yKTA mwZ4aRyx yEd2VPhU6XO4eoWXgCcx9MQ6U54IEuSqIcilxrTaBLlJx+S8i/KnM9kTlaYgdb46fCRagByG3xibfeXpnx/wG0ezWEsOAfyqltdZ3hJqtxLybrfgkgCu5XiPUW0W8vUTx9qBWxv0Tk64z2BINdjMgQpHmEzUrxFP4SfNze12mZFeqD6nLbJ0DKtMCOSvb6wgJgJX+pIhCoZvehJM1PBpn/zzdxBrUTzSBxHXCPbefdfE3JcR6cUA0R910PfZoBuFlQICHjhGn9nrGU1QZqRqA064Lz3GETQpsvRwayKDB59P4TaNu4wS6D6nXUBN7E1ZoFEybbmMwMPXn33za0SR/8sSBluIeFqoQ7/ZxmrgY4I4CCi2DWI7TzwiHQ8T3KzgybzACe/lsYcQTn9+uuaGs2Aa+TZfkra764WCkGj5bHN9aqetRnr1O4Vt/LLavuCNJb1cUr02LWsg1w9UpvJuG/YPKjQoURJ8Ct3Cr7VMsUHaYSp5nqs5m43aPOWOEn8rHba3aTn+DTww7HNS5WuCV5fcbup1kIsi/qZ/DLQbXa6wwyP28a7uL85a3Kx9NZTguXJLCTxxfnLGSWV+fBc5o5bVvIiTnsOvnmFySshr+cCw6yBjQY7r8wPkcFwgdpzbwzzBDKs8eZNa7BeROGul1aDw/t4BUv3oAnpVM4sTCFBSEgzMv2WSLGtuFIyAxRqlYPNnwoC7U9IwBZL0srnnv60jN7tATEihKdwT0+Jbc7hWsgmo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Most userfaultfd ioctls take a `start + len` range as an argument. We have the validate_range helper to check that such ranges are valid. However, some (but not all!) ioctls *also* check that `start + len` doesn't wrap around (overflow). Just check for this in validate_range. This saves some repetitive code, and adds the check to some ioctls which weren't bothering to check for it before. Reviewed-by: Peter Xu Signed-off-by: Axel Rasmussen --- fs/userfaultfd.c | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 7cecd49e078b..2e84684c46f0 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1306,6 +1306,8 @@ static __always_inline int validate_range(struct mm_struct *mm, return -EINVAL; if (len > task_size - start) return -EINVAL; + if (start + len <= start) + return -EINVAL; return 0; } @@ -1760,14 +1762,8 @@ static int userfaultfd_copy(struct userfaultfd_ctx *ctx, ret = validate_range(ctx->mm, uffdio_copy.dst, uffdio_copy.len); if (ret) goto out; - /* - * double check for wraparound just in case. copy_from_user() - * will later check uffdio_copy.src + uffdio_copy.len to fit - * in the userland range. - */ + ret = -EINVAL; - if (uffdio_copy.src + uffdio_copy.len <= uffdio_copy.src) - goto out; if (uffdio_copy.mode & ~(UFFDIO_COPY_MODE_DONTWAKE|UFFDIO_COPY_MODE_WP)) goto out; if (uffdio_copy.mode & UFFDIO_COPY_MODE_WP) @@ -1927,11 +1923,6 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg) goto out; ret = -EINVAL; - /* double check for wraparound just in case. */ - if (uffdio_continue.range.start + uffdio_continue.range.len <= - uffdio_continue.range.start) { - goto out; - } if (uffdio_continue.mode & ~(UFFDIO_CONTINUE_MODE_DONTWAKE | UFFDIO_CONTINUE_MODE_WP)) goto out; From patchwork Fri Jul 7 21:55:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 228FDEB64D9 for ; Fri, 7 Jul 2023 21:55:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30A308D0002; Fri, 7 Jul 2023 17:55:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26CA46B0078; Fri, 7 Jul 2023 17:55:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E4F68D0002; Fri, 7 Jul 2023 17:55:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 015FE6B0075 for ; Fri, 7 Jul 2023 17:55:51 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id ADF9EC0122 for ; Fri, 7 Jul 2023 21:55:51 +0000 (UTC) X-FDA: 80986173702.09.167B040 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf26.hostedemail.com (Postfix) with ESMTP id EC04014000E for ; Fri, 7 Jul 2023 21:55:49 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Vna1SetL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 35ImoZA0KCMUl8pw2l3x533pyrzzrwp.nzxwty58-xxv6lnv.z2r@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35ImoZA0KCMUl8pw2l3x533pyrzzrwp.nzxwty58-xxv6lnv.z2r@flex--axelrasmussen.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766950; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5XpJBVMM9VGEbS89HlMJM/b+46oLjR8s/ahtpOvDxE4=; b=2BXwlC+gGToIEIPh1I5F4BphUIKqRizfyeH3XJYWgTCJvJjalHxgwzQxoKyBW2gndGXvBW J+1r/l9YM2ZGU4M8vZGQwIY+1YvXDo8GKfRx9YSIh3eoAef34luwyxiynFSiz+jp0BCzvu PeFu50pEtz5yAPjcFkYlgJk8Qtn7hm0= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Vna1SetL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 35ImoZA0KCMUl8pw2l3x533pyrzzrwp.nzxwty58-xxv6lnv.z2r@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35ImoZA0KCMUl8pw2l3x533pyrzzrwp.nzxwty58-xxv6lnv.z2r@flex--axelrasmussen.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766950; a=rsa-sha256; cv=none; b=C0BB7X0360ErFfZLSECLKkwU7JP9azpAbCRr4EQKP5jGk4d1PIY5CKYwdkP4DU2TG2/0Z1 V8GUPEzA4sg0T9G7SkU56muLS2lCZjRXE5+5Lh5La0rwSMyo2WIf2C/Fiv6f+0N4d/mmQX s4ubWL9ZkDMnMAgb+u760tt8PqqjL7g= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c0f35579901so2614756276.0 for ; Fri, 07 Jul 2023 14:55:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766949; x=1691358949; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5XpJBVMM9VGEbS89HlMJM/b+46oLjR8s/ahtpOvDxE4=; b=Vna1SetL1twavTmQ3g0qKEnlEJ8kJBZCi65Nnw+YcV5ZZCoEneQ4zTFVsFXUuvkRKr Y9wbkNV5IwnplqQXfIPLSaPB2IcZ307sSmUuIJy2rwTPrQUVsrk2CdliIB7gI42gWhdM 6gimHkR7Zsp03wFa9g3UVN+pHLvWeuzqesEJ2UoOZley50vyuTzor2gOnslHanzRO+2V zz+MOmoHULIYJJA/CUzyHkrN7bP3g2kVeV9cVUxa0wQEPONQ3+8KsB0PFHA1Sp9MPuuD 2F0o29XPzlT5FRPXTikiGPDnm68fX68i/xryJT4RapXWGGzNcDuVhIoxEk1qtV2lNW65 X5GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766949; x=1691358949; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5XpJBVMM9VGEbS89HlMJM/b+46oLjR8s/ahtpOvDxE4=; b=GlQrdBGuCB/KbGnfUfkfSN4hgov5+1XN48MsDQvI1OuBh3qs6a00LO9G9g/g0yFKa8 TlNkZYYQ75FEnaTq8PEJYpLeVpe0eHjV5a669hYkEOtvF0HBMX4YxaZ2g5OmlJ+APxVG ZLQbHPQ1Uckv7a1ZYAwRHv5igKDK4g+vjnnEKO66VZfSLiFYowzANcQjWzLAZVXpiQE+ f5XGuMWCgchRs7G3cDfczHm6guUCNb/rfGMZmx3+2SCYi9PGY42N1pm8y3UuWykn2dSO lBSTJ9rx8iBeFyrCLAckzNwpkNRLroegybJAm+q8Qy7COe9KXmcnvqSakzBQEBnsitot OV0g== X-Gm-Message-State: ABy/qLaETSL3LNWzSkHApFx/2EC4aFcMZiLOcPsdNZYobDgFOrEA04Pl +zpuGgtfioEowfuwdCCIl3pG4Hf3azlNBaZ/4HyU X-Google-Smtp-Source: APBJJlHNyRYQbB7ZObZHpnn7dAmDsn5jT+2xbl5BfWLddNXyMesuVTzhCBkP1TN4IxNkWiSdwYOjtvr2Xk3oVPP6sM47 X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a05:6902:350:b0:c4c:ef98:6252 with SMTP id e16-20020a056902035000b00c4cef986252mr34624ybs.9.1688766948974; Fri, 07 Jul 2023 14:55:48 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:35 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-4-axelrasmussen@google.com> Subject: [PATCH v4 3/8] mm: userfaultfd: extract file size check out into a helper From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EC04014000E X-Stat-Signature: 59666b8496qr8uw5bftqr9kghjdyzgak X-Rspam-User: X-HE-Tag: 1688766949-668845 X-HE-Meta: U2FsdGVkX19XpldF7AZKV318Ubw7QW11LjC4v9UlTAFqB+dTSC1oTt2QTEuxaAudJ2+PQMyD+aWOvLgKfH4R1xdQUDu8mbxVhTn8IjTdd8Kmq25P6TjSLm+w8sKyna2Np19WwXIea6Ah7crYCEBbICOVc2U5r0KXVduEGL7BawmrL5VOj6EwhrSKUMjijxbnMsRFp2vhc6lRE6+MqiMSMxFLdpvKqFFvT/hYCZD1DnfTKHpqb2r8JGMo0kb7T5uda9iM3oDdbkGTVRK75vohK95dRxYFuGtdzLHlyDEObsBltE9jFAlpBCZGQbAbkAPc44+KLfTWSB1z66it9hc/5UlnMy+rGUwrwQHGXm69kVOWhF1YFoblAPGuj/70clyd03QWnUW4hYkSOLU5lNPS491MQ447GieWJ3T3QkYEuzUbEawgrGgijWi5tcKGcL62QxgRz2WLsFeyiJVyKeP8oVp/Fg6ifuQiaq2y1yIRO3Lhof3OjE126RfkN9F4fE1rpYj0t2CSU3oRJJzBgn4qtjRSTPZDoVtO3LZ2rY2CiE3QN0ni2ZWZuweFvZQSb9rWzTXcEda0XZ4/SR1ziOAgqG1vdtHp6wX+dzaX6in/2vqU8QuWtbjWIsObioaObx1jOl3R0NQJ0b9xCjnMJ9DxPT+5TqkuOg4YuCAVjMzwYQShJIxhWQ51mYyuXYYhth6Liu5u1ssnWj5t9uaBXTBaIwn8nEW8pIjswXQRDSqeOELaZgBMEF/cvtioFhIbyApSNch6DchPFWWLkv+zJNA6Des73lzIbS+qA2gUgPZRrxAptf2vienc4ocWKcK5s+toty3Z4mJe4Gn69vVM24IH0exX4vKkB6gaGgCcr1CGfKZeUEyWPYWfRaW/xELUObdGM4eqnbAtEttU5mBpCejvr/rnInjXT8waEFwGR4JjReXbio2U1R3lviPmwoLaxrujBSPYwH/TKuO5ohWhYHh IKFuXwRT 8EVcEC9GdxdLBYAkFMd4ek7VV6rmQUBvF0uAu2+krUJEx8jN4Z1WGOyRFhqBO+CTPI65krWtmZe5xhwmXcPRr0QK29/qioxUd4TwdYVxirNETamheBJKrt28AFzHI90mNCt4R65Pu6BLqJSSi9TgUjjRNRn+9gLZJ2vNIy6HxVrOLR6Kk3cniM2/jwjdXKpT1LfEeIYVFQ+3Mdw+RjJKwsSDQEwYiDQ55zszWqGe6Is9dkMGheDzzFCZW46u3NLJv0g64ZEJmOyZe5klUUL+rHMjkrRhOeiLcYjk4PXeeG+KMSptzYlgm6D3or3vDiIq1cGYkjr4oFL0k17xQU/DrNfptZJMSgcoerPBL0ntDzPpLGgG8ziU84D+98MzzJC1OQy/5qd4BL4ojIPRMk7jQWvqAEYGCXositG+knEAkoluI3JYkpgjvkgI1fxSr0C8L35muWPmFlOH/5YNeuL6+8+4vRh8gEZwkxdyziVpJPhPaQW4TRJBd8IQbjFoaj2vi5U7f6p5Wi6KPHWrvPrR2lZynPpwphCD2U846sNyGEgLjN8jVcuJvy8Zeu66eRtOtEPwi5jwwh9obZquFBuiybrb3iz8Za9h9Mb+ReW5HVHG83o49RhqGPOuQhJoaOy3JOv73bqMXslgzxDETU700jedqQtFPkTS7l63Sek61BI+VL2OGkqBm2dJzyA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This code is already duplicated twice, and UFFDIO_POISON will do the same check a third time. So, it's worth extracting into a helper to save repetitive lines of code. Reviewed-by: Peter Xu Signed-off-by: Axel Rasmussen --- mm/userfaultfd.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index a2bf37ee276d..4244ca7ee903 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -45,6 +45,22 @@ struct vm_area_struct *find_dst_vma(struct mm_struct *dst_mm, return dst_vma; } +/* Check if dst_addr is outside of file's size. Must be called with ptl held. */ +static bool mfill_file_over_size(struct vm_area_struct *dst_vma, + unsigned long dst_addr) +{ + struct inode *inode; + pgoff_t offset, max_off; + + if (!dst_vma->vm_file) + return false; + + inode = dst_vma->vm_file->f_inode; + offset = linear_page_index(dst_vma, dst_addr); + max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); + return offset >= max_off; +} + /* * Install PTEs, to map dst_addr (within dst_vma) to page. * @@ -64,8 +80,6 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, bool page_in_cache = page_mapping(page); spinlock_t *ptl; struct folio *folio; - struct inode *inode; - pgoff_t offset, max_off; _dst_pte = mk_pte(page, dst_vma->vm_page_prot); _dst_pte = pte_mkdirty(_dst_pte); @@ -81,14 +95,9 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, if (!dst_pte) goto out; - if (vma_is_shmem(dst_vma)) { - /* serialize against truncate with the page table lock */ - inode = dst_vma->vm_file->f_inode; - offset = linear_page_index(dst_vma, dst_addr); - max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); + if (mfill_file_over_size(dst_vma, dst_addr)) { ret = -EFAULT; - if (unlikely(offset >= max_off)) - goto out_unlock; + goto out_unlock; } ret = -EEXIST; @@ -211,8 +220,6 @@ static int mfill_atomic_pte_zeropage(pmd_t *dst_pmd, pte_t _dst_pte, *dst_pte; spinlock_t *ptl; int ret; - pgoff_t offset, max_off; - struct inode *inode; _dst_pte = pte_mkspecial(pfn_pte(my_zero_pfn(dst_addr), dst_vma->vm_page_prot)); @@ -220,14 +227,9 @@ static int mfill_atomic_pte_zeropage(pmd_t *dst_pmd, dst_pte = pte_offset_map_lock(dst_vma->vm_mm, dst_pmd, dst_addr, &ptl); if (!dst_pte) goto out; - if (dst_vma->vm_file) { - /* the shmem MAP_PRIVATE case requires checking the i_size */ - inode = dst_vma->vm_file->f_inode; - offset = linear_page_index(dst_vma, dst_addr); - max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); + if (mfill_file_over_size(dst_vma, dst_addr)) { ret = -EFAULT; - if (unlikely(offset >= max_off)) - goto out_unlock; + goto out_unlock; } ret = -EEXIST; if (!pte_none(ptep_get(dst_pte))) From patchwork Fri Jul 7 21:55:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7875BC001E0 for ; Fri, 7 Jul 2023 21:55:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 10DDE8D0003; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BD506B0078; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E544D8D0003; Fri, 7 Jul 2023 17:55:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D778C6B0075 for ; Fri, 7 Jul 2023 17:55:54 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A680FAF660 for ; Fri, 7 Jul 2023 21:55:54 +0000 (UTC) X-FDA: 80986173828.28.8F032EB Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf14.hostedemail.com (Postfix) with ESMTP id D442310000D for ; Fri, 7 Jul 2023 21:55:51 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="MOZe/4J0"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 35omoZA0KCMcnAry4n5z755r0t11tyr.p1zyv07A-zzx8npx.14t@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35omoZA0KCMcnAry4n5z755r0t11tyr.p1zyv07A-zzx8npx.14t@flex--axelrasmussen.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766951; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=13fFKLFK5UOuV/VKaxpsoKuDeTvwdO4Zasj0G6nnvw4=; b=BA9r+f07U6l8GgH+/CogDKe+v0JhpVxcbuoeYLAzO66C3Nr4+0nRAGeihz9MXu6cVoD+I3 O4lRT2n+Cl5pKKoPMMJE0J2LhPeBtDmrOy2CU+YRg2ox8BEC+Ooelm9LSmGQLb3I73Jz2X go5rviEgO0cHVGpLmaHWJxq3ETPiN/Q= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="MOZe/4J0"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 35omoZA0KCMcnAry4n5z755r0t11tyr.p1zyv07A-zzx8npx.14t@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=35omoZA0KCMcnAry4n5z755r0t11tyr.p1zyv07A-zzx8npx.14t@flex--axelrasmussen.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766951; a=rsa-sha256; cv=none; b=SyBAFz4JRZwHoZlzcaimtydHMvPU4erXxgeIjVm/8rCf+NisRoOP7UDfANyKGbZ719511Z mjUXiW5M40TPONmY9vDA5O2sRJUFA5mcHlas2FbLgjAwJ6cIOyz7GojTmNxQc8o/sawMSJ BbOS0x7ZSEShRGfjPYESh2MRrBr8Oco= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c6dd0e46a52so1081518276.2 for ; Fri, 07 Jul 2023 14:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766951; x=1691358951; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=13fFKLFK5UOuV/VKaxpsoKuDeTvwdO4Zasj0G6nnvw4=; b=MOZe/4J0t2eY56FwP6+UOnOCO967WEJl1TbQmpEjUkORQByt+zNCfplkM8DFDsfn2O r8uJClY/Y8Z5xV1YyuYdtmUraPk6/0cTJYHNmUSZbe/IN5Fu/cCN2a5n6nood0PBJ5ub r+oK/nXF6cKIsISrdC7GbKJnOlHWU/fYaofkbzYDjk4RMoXaSDwF7Xm8k0rAsTnBwL6M iIH+0ynit7jmNdoNEYTTc4GCckiC0R5j2gDKsv3fFbP7GHFkZpeE0NTYI7CtMCGaLWtX veWaDAElReFjefLsUGuuOtLuP8wj+UPVmDqBKLb1/cz2GLGjcNUxr2o85TMD4Ekuetv1 Bkvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766951; x=1691358951; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=13fFKLFK5UOuV/VKaxpsoKuDeTvwdO4Zasj0G6nnvw4=; b=LMzCXxMk9pglNEVgWAqPgRWmrtdHVub6RiuirdQdMCC77IPQ1As3dZlJlagfqzxwiQ 1q2yAkwPHBBiNvOZX0aNVAoyz/qMKTPQnl4HWWf8Y0/ooHTxCwgI0nDBv5tAmYHLC7zK 0sFeEE7xbcOj4H6pA/IsGGq+/bORw6rJCiiTMVOsKTydPK8rVUNECIIDgfMkYTcVtcpx hB+psfHBGmbeVp45hIqxz4yHTLVAHHrk99eMm7EdBUTbCoCYMdnq0yQ+KYvjAy/IwLzA rnu+yzTxRhI88rDw7G4OSXpIIbdHezopIRRtSdCeohRRlVfY/sE74ZMo704wAgXJbO+V zlOA== X-Gm-Message-State: ABy/qLYZesTeKKO/O+/EiRUzSnpCs+CXTVeUZzG/PQ93CkuLqyVvPxta IEUwE1xygR0ndNSKhK0afZusM198ZdUuTASEsDR7 X-Google-Smtp-Source: APBJJlE6DEJsijlYNuZPo3I86aUhHhB3DQHO8fcNF8FmogVn4S+pUbHfGvuKc4J+MOTJwTKMVeH0FM48t2P4Yv+ogdSs X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a5b:c86:0:b0:c02:7c99:62e with SMTP id i6-20020a5b0c86000000b00c027c99062emr64646ybq.13.1688766950995; Fri, 07 Jul 2023 14:55:50 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:36 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-5-axelrasmussen@google.com> Subject: [PATCH v4 4/8] mm: userfaultfd: add new UFFDIO_POISON ioctl From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Queue-Id: D442310000D X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mfaug9om1hj1da7h4modqu1wydng16tt X-HE-Tag: 1688766951-118760 X-HE-Meta: U2FsdGVkX19j3PrU6NLLOBPeP/hdWPf7Tk9uUpkyNUJV2JrZsFmYiKZQwaAX5mmv1K9YevPyVZJwtAIeVbMuH+w4Z0F8z6x/rt8bBCmtsgsRPH9gOW3tXg2Cyj/5vYqKwgTIsf1B7VgdYiPBgZksmbpvHy6GzrWTEmkkbKwjc2zGiO17t9eU1vA1cO2IJEW75PNG5oGMvz/w336h/qDwaj+G6Dbc4x9Gv5RBlAWgCKBukxsMXpHhyjoJeAjwRFuGds3MYbBZ0Vh5pSNoVUiknwJp3WjRQxL6h1fU+mM588afWUMpH5Mqyma5DCkssHjl3NrECda4Kf1WnO8fWWd20Yd3i1qpgK1hmc3diUAye2j7Tgz0TsTF4/vzZ2jcw06BJENaqWRjba8VICrPwQXALtFXG/4PanzzEUF/XCP7SFqAHHxm9XmYjoqzI/4Cpbr/DT/nGzPnpQbbX6J5+5XYbQ3pRGrEuunwUvoSxN++8feZ3xQHfRfWY0J+Zxnw36RpP/OQJR74X5m2n1nLd7OKmyk+NGvekTnG+CxplMjz0K6ZCVs6DK5+4XFoHsDXmNwk7Xt0nI6b7dfRTwIKQgN6svzbBV2B/6w1eTgDDIqs+BxHqm8ADkY62tTOUi9/heWuhNJenR+P6DwUntJQU+eMReuuGbTgsYUnswogrTcKaZdnoYYchCcyECMZ2u68atS0Ad6e+zkVrCJLzI1kfV7QdRpVJJ93O3Fxnffhi17KHWAbTxLGuE7GkLjV+dHOac1EbAQ8RqWB1EMENTsT8ej9bpzf4zXgYfHuvzZqVkab228PYYjcCZXs59X088qbkAcT8mWzQK0m0SvL4by9ASQzQqTo1kO7/TroUWCWOCMGlDZr+DEu1dSgR3+sbvtj+ibdpBJ7h0uCpixA2W33TpxU78OU+EeuyNinYNg7Km3V6DL7w+8Kzf29jgNY1R09aKIWgDYii6B7D4OPxmHomi9 lQ+Bz4kG 7IbV6fyFGm+2pZVqErJkWkQ/szuak94yP4NgKKXq1VKBOdmqyqtUO7GsEWM90v7s6cms6p/gkyu/dlOKhmo3Xtyy5EaSblKoKzvXOwiNHIsT2PPKAI2kvQYkhWvsBIIhEwKT1zoxzPRBNomojTF/u1/X7cS8ut55NQmU57sNbLxOEbF6/JB/v4EXeiHStNlKrPkGU+KPm6e44szoBDGU4ihdAfnq7QyHbzpaOJJsbAFcgL8mYHCQkhFtN3Vjr9LZnrTp9q8rv0TozyY++z9854bCTWXy5BZ9W0soYIIyKnzyugNigVeOGr3Ttim6AdmGEjN8onJb2xKtOOFhWHgqWPppvk3rgVFn5AkMIcYTwg0iDlIQLWjPzk43KJPWFgA+OX+VMNVOuNaA6FmWs+B3jNnvmk0NPWubhmyrKaCfZE1B3CXTWKIoh4e+6UASI4cm6w/A16hO8Z3d4us8ZgJ98irtY8eRPmp66e+wtktjrngZmbCjlNm2kjn9ydOGI5aWwWEyuuv5M47/3JCbMMNSbUZSE+EN3hV0FO47Rf78vkq1QpD+k225VUYnGTu0+ktLFdqCBKxaSL7/Q3rfmi8BtDFzW36qkHQqjqWIFWQK40FRiQutOeGs0Px0i4Pu8K37osAfqJqzuURXYFSfMODeuU33BC92dqh/CK9D8lNCitecWFE+lE6YS1K0hpOGjaz+z0N3uz6WCihenU2/fbPJCqRHnzN1OpeN+Zz5NRSe/FjshQHo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The basic idea here is to "simulate" memory poisoning for VMs. A VM running on some host might encounter a memory error, after which some page(s) are poisoned (i.e., future accesses SIGBUS). They expect that once poisoned, pages can never become "un-poisoned". So, when we live migrate the VM, we need to preserve the poisoned status of these pages. When live migrating, we try to get the guest running on its new host as quickly as possible. So, we start it running before all memory has been copied, and before we're certain which pages should be poisoned or not. So the basic way to use this new feature is: - On the new host, the guest's memory is registered with userfaultfd, in either MISSING or MINOR mode (doesn't really matter for this purpose). - On any first access, we get a userfaultfd event. At this point we can communicate with the old host to find out if the page was poisoned. - If so, we can respond with a UFFDIO_POISON - this places a swap marker so any future accesses will SIGBUS. Because the pte is now "present", future accesses won't generate more userfaultfd events, they'll just SIGBUS directly. UFFDIO_POISON does not handle unmapping previously-present PTEs. This isn't needed, because during live migration we want to intercept all accesses with userfaultfd (not just writes, so WP mode isn't useful for this). So whether minor or missing mode is being used (or both), the PTE won't be present in any case, so handling that case isn't needed. Similarly, UFFDIO_POISON won't replace existing PTE markers. This might be okay to do, but it seems to be safer to just refuse to overwrite any existing entry (like a UFFD_WP PTE marker). Acked-by: Peter Xu Signed-off-by: Axel Rasmussen --- fs/userfaultfd.c | 58 ++++++++++++++++++++++++++++++++ include/linux/userfaultfd_k.h | 4 +++ include/uapi/linux/userfaultfd.h | 16 +++++++++ mm/userfaultfd.c | 48 +++++++++++++++++++++++++- 4 files changed, 125 insertions(+), 1 deletion(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 2e84684c46f0..53a7220c4679 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1956,6 +1956,61 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg) return ret; } +static inline int userfaultfd_poison(struct userfaultfd_ctx *ctx, unsigned long arg) +{ + __s64 ret; + struct uffdio_poison uffdio_poison; + struct uffdio_poison __user *user_uffdio_poison; + struct userfaultfd_wake_range range; + + user_uffdio_poison = (struct uffdio_poison __user *)arg; + + ret = -EAGAIN; + if (atomic_read(&ctx->mmap_changing)) + goto out; + + ret = -EFAULT; + if (copy_from_user(&uffdio_poison, user_uffdio_poison, + /* don't copy the output fields */ + sizeof(uffdio_poison) - (sizeof(__s64)))) + goto out; + + ret = validate_range(ctx->mm, uffdio_poison.range.start, + uffdio_poison.range.len); + if (ret) + goto out; + + ret = -EINVAL; + if (uffdio_poison.mode & ~UFFDIO_POISON_MODE_DONTWAKE) + goto out; + + if (mmget_not_zero(ctx->mm)) { + ret = mfill_atomic_poison(ctx->mm, uffdio_poison.range.start, + uffdio_poison.range.len, + &ctx->mmap_changing, 0); + mmput(ctx->mm); + } else { + return -ESRCH; + } + + if (unlikely(put_user(ret, &user_uffdio_poison->updated))) + return -EFAULT; + if (ret < 0) + goto out; + + /* len == 0 would wake all */ + BUG_ON(!ret); + range.len = ret; + if (!(uffdio_poison.mode & UFFDIO_POISON_MODE_DONTWAKE)) { + range.start = uffdio_poison.range.start; + wake_userfault(ctx, &range); + } + ret = range.len == uffdio_poison.range.len ? 0 : -EAGAIN; + +out: + return ret; +} + static inline unsigned int uffd_ctx_features(__u64 user_features) { /* @@ -2057,6 +2112,9 @@ static long userfaultfd_ioctl(struct file *file, unsigned cmd, case UFFDIO_CONTINUE: ret = userfaultfd_continue(ctx, arg); break; + case UFFDIO_POISON: + ret = userfaultfd_poison(ctx, arg); + break; } return ret; } diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index ac7b0c96d351..ac8c6854097c 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -46,6 +46,7 @@ enum mfill_atomic_mode { MFILL_ATOMIC_COPY, MFILL_ATOMIC_ZEROPAGE, MFILL_ATOMIC_CONTINUE, + MFILL_ATOMIC_POISON, NR_MFILL_ATOMIC_MODES, }; @@ -83,6 +84,9 @@ extern ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, extern ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long dst_start, unsigned long len, atomic_t *mmap_changing, uffd_flags_t flags); +extern ssize_t mfill_atomic_poison(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing, + uffd_flags_t flags); extern int mwriteprotect_range(struct mm_struct *dst_mm, unsigned long start, unsigned long len, bool enable_wp, atomic_t *mmap_changing); diff --git a/include/uapi/linux/userfaultfd.h b/include/uapi/linux/userfaultfd.h index 66dd4cd277bd..b5f07eacc697 100644 --- a/include/uapi/linux/userfaultfd.h +++ b/include/uapi/linux/userfaultfd.h @@ -71,6 +71,7 @@ #define _UFFDIO_ZEROPAGE (0x04) #define _UFFDIO_WRITEPROTECT (0x06) #define _UFFDIO_CONTINUE (0x07) +#define _UFFDIO_POISON (0x08) #define _UFFDIO_API (0x3F) /* userfaultfd ioctl ids */ @@ -91,6 +92,8 @@ struct uffdio_writeprotect) #define UFFDIO_CONTINUE _IOWR(UFFDIO, _UFFDIO_CONTINUE, \ struct uffdio_continue) +#define UFFDIO_POISON _IOWR(UFFDIO, _UFFDIO_POISON, \ + struct uffdio_poison) /* read() structure */ struct uffd_msg { @@ -225,6 +228,7 @@ struct uffdio_api { #define UFFD_FEATURE_EXACT_ADDRESS (1<<11) #define UFFD_FEATURE_WP_HUGETLBFS_SHMEM (1<<12) #define UFFD_FEATURE_WP_UNPOPULATED (1<<13) +#define UFFD_FEATURE_POISON (1<<14) __u64 features; __u64 ioctls; @@ -321,6 +325,18 @@ struct uffdio_continue { __s64 mapped; }; +struct uffdio_poison { + struct uffdio_range range; +#define UFFDIO_POISON_MODE_DONTWAKE ((__u64)1<<0) + __u64 mode; + + /* + * Fields below here are written by the ioctl and must be at the end: + * the copy_from_user will not read past here. + */ + __s64 updated; +}; + /* * Flags for the userfaultfd(2) system call itself. */ diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 4244ca7ee903..68157359dc34 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -288,6 +288,40 @@ static int mfill_atomic_pte_continue(pmd_t *dst_pmd, goto out; } +/* Handles UFFDIO_POISON for all non-hugetlb VMAs. */ +static int mfill_atomic_pte_poison(pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + uffd_flags_t flags) +{ + int ret; + struct mm_struct *dst_mm = dst_vma->vm_mm; + pte_t _dst_pte, *dst_pte; + spinlock_t *ptl; + + _dst_pte = make_pte_marker(PTE_MARKER_POISONED); + dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); + + if (mfill_file_over_size(dst_vma, dst_addr)) { + ret = -EFAULT; + goto out_unlock; + } + + ret = -EEXIST; + /* Refuse to overwrite any PTE, even a PTE marker (e.g. UFFD WP). */ + if (!pte_none(*dst_pte)) + goto out_unlock; + + set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + + /* No need to invalidate - it was non-present before */ + update_mmu_cache(dst_vma, dst_addr, dst_pte); + ret = 0; +out_unlock: + pte_unmap_unlock(dst_pte, ptl); + return ret; +} + static pmd_t *mm_alloc_pmd(struct mm_struct *mm, unsigned long address) { pgd_t *pgd; @@ -339,7 +373,8 @@ static __always_inline ssize_t mfill_atomic_hugetlb( * by THP. Since we can not reliably insert a zero page, this * feature is not supported. */ - if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE)) { + if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE) || + uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { mmap_read_unlock(dst_mm); return -EINVAL; } @@ -483,6 +518,9 @@ static __always_inline ssize_t mfill_atomic_pte(pmd_t *dst_pmd, if (uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { return mfill_atomic_pte_continue(dst_pmd, dst_vma, dst_addr, flags); + } else if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { + return mfill_atomic_pte_poison(dst_pmd, dst_vma, + dst_addr, flags); } /* @@ -704,6 +742,14 @@ ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long start, uffd_flags_set_mode(flags, MFILL_ATOMIC_CONTINUE)); } +ssize_t mfill_atomic_poison(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing, + uffd_flags_t flags) +{ + return mfill_atomic(dst_mm, start, 0, len, mmap_changing, + uffd_flags_set_mode(flags, MFILL_ATOMIC_POISON)); +} + long uffd_wp_range(struct vm_area_struct *dst_vma, unsigned long start, unsigned long len, bool enable_wp) { From patchwork Fri Jul 7 21:55:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79C23EB64D9 for ; Fri, 7 Jul 2023 21:55:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E05476B0075; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DDFE38D0005; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B45B76B007B; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A72EF6B0075 for ; Fri, 7 Jul 2023 17:55:55 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7BD0AC013C for ; Fri, 7 Jul 2023 21:55:55 +0000 (UTC) X-FDA: 80986173870.15.D69F54E Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf27.hostedemail.com (Postfix) with ESMTP id 9F6CB4000B for ; Fri, 7 Jul 2023 21:55:53 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=u4eWVZhz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 36ImoZA0KCMkpCt06p71977t2v33v0t.r310x29C-11zAprz.36v@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=36ImoZA0KCMkpCt06p71977t2v33v0t.r310x29C-11zAprz.36v@flex--axelrasmussen.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OLsbot8A3mLzPOTacWgWQym2WdRGSP5CUekvmW42V6U=; b=c9xF0+0kFPbRzc7/yeNelSimqdGR4fu3FpwOhjlVVoFDtamxUwCydsN0mROrFcHDIGn+oK DR96wq263rFwqHGj7Nvc7JoW+f1sBKgcpUTnG3as3QKvA7fn2NhZSXv4KiUUUbYB6tHRhs HLHQ+rbi2x35VGAOCgN7GCeVKKrzR4g= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=u4eWVZhz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 36ImoZA0KCMkpCt06p71977t2v33v0t.r310x29C-11zAprz.36v@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=36ImoZA0KCMkpCt06p71977t2v33v0t.r310x29C-11zAprz.36v@flex--axelrasmussen.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766953; a=rsa-sha256; cv=none; b=zia2xpZ1Ay4WrRk7VfWEOxqV9B7BKZeXRIGLxj+f1axi5G0pUsqmZc8Hw10K6In7lX6DYl GjaCSgxELyc15z04c9fM8ssP839DKlCx1kEOF5KzOdHA+KqpTSLFzS0pEd61r8bk8j9yCi VKf6BqgYlIgRTzNRsPPoNjtD5jzbePs= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-bacd408046cso2452106276.3 for ; Fri, 07 Jul 2023 14:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766953; x=1691358953; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=OLsbot8A3mLzPOTacWgWQym2WdRGSP5CUekvmW42V6U=; b=u4eWVZhzXkhr8wDK2s+0iTmAY7l1h7R8MMREKtd6LAlEQHntgig16nK2DmvVjFtAts msXCG4jhwlQLWJ7mMhcWd94nmZDBm9Cq+zUjd6HljvgRAmkhpgv0xVCKBFZiliIr4ZRU lJ8XlRPYDtAXOgFTgkEbA2VAMupk6A+/nyrhmDIHETRiNLXPxTJkdqTH3PtZeCK+4ys/ yja0ZfMTqDUhv3HkJ7Igyj0pyhydxjC+QDfmIGRe122yVx7v1/1OQSVN86AWXYuftoFV PzsEid3tTnR28UcWxblLYAu3XQZ8xKr3MUXV1i8PoVEZYRxu++znQoGIDktCzSa3JUOO wVnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766953; x=1691358953; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OLsbot8A3mLzPOTacWgWQym2WdRGSP5CUekvmW42V6U=; b=FnJS3iHsCZoqiZwyPhW43SLd7Zf5aO1rcGoejx5W+bvxpNvVZLQY5LyygnYHK0aGNG EOQhELrb0VjIWUppRnJjhm+aAtcJoxkAiMOSvs19t9SK2SXYpmNnaHcuxK/iH82l6a++ W+4GDjeEsb4dOkP9gdEsX2y8xnCnBhuw7euwH229qkUch7PCRvYSE7O2QZD+a/6mM95x WPFCDCvOcQsQ6hae+gelxEtX5mAZexUmpA8xc1sTfg8X99BUrPIq9qEUdUV018/Zbo+2 Q9idpShbBgZHJwvyHOPrrAp+MtZJ7GpLXQ7SE0ZFMOg+Jf5h5xwFnG4zEFAcjOjT52kP Nb2g== X-Gm-Message-State: ABy/qLYx991NORHXxcDu0usXydX7yHPVuZG8/MdIFs/60Pmab3EHkEzW Fu3x0yElq9ONrszLW6FgCwtBLC8KdaGGq8CFXM8M X-Google-Smtp-Source: APBJJlHL/U0DvDL04upfB3UAQ4yyr3JL0AGmb9MqqDsUkDgEZRrl5NMG/hgdzAz1NMr7770PTdyAART4pZ39J+ilp8hn X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a25:e442:0:b0:bd7:6810:6666 with SMTP id b63-20020a25e442000000b00bd768106666mr33172ybh.5.1688766952790; Fri, 07 Jul 2023 14:55:52 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:37 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-6-axelrasmussen@google.com> Subject: [PATCH v4 5/8] mm: userfaultfd: support UFFDIO_POISON for hugetlbfs From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspam-User: X-Stat-Signature: 361oazatsxsgsnycjp8nqqioj4dky9ph X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9F6CB4000B X-HE-Tag: 1688766953-701380 X-HE-Meta: U2FsdGVkX1+nx/54sPdvEYLUdCrrbCNRrryK57UYAOkRZnFgstuY6ca/H3yKK/I2aTPpY0NxZUIR6efuF2hDgPGMpW4oCh+6YpRVUqJtiB4ZXcXuJ0LZIjBaDt44241temcucgvBd9vQ0EHfseQ7sjGTUDp25hq3xo5w8nmASIhxtpnePrbq8vkmf9EEtB8vusAvP42TwFio8el87E/tfdIC9AZDb4j0t26ESHc3ghsV4+rihaVO97nrkwkRCfhvwEt0jAhrlQ1SUknMz0V179LJ/lxkZ2+LKAvpWZdGyvCpUyxaYwCeWg/wCA8kp4NWuEoFYEdTgoYO4lqCRWVeH7x3kt9KVGQFLh7ywSJuPQ0y/KuWkv7I94Dbg7AOcV6qMXn/46xwp2qqRpviXkJbxauj0lP1ZpFXJAUUGEjqa2tUsusZYIN/c1i+urVCNM/v6f64iU2cmIyghhTi8CCLZxd0E3V3AfEGitt3M7+byNOVl6m3lYH8sb+J6BWM8amuJOUDreJSQlh66VIQRufWnVq30zXbIe3ldMvF07goaN5YoDzL4gKNYFL6ZVxntpHfNuQgyoBy9Ug7X8lep0AC/ckMxE8d8tKE2XLiHDnEOTc2JYq/AN1tp9IjDTbd+l3sXiXgMOU2EL0x+7L/RvNIJLestIKk8FSPPD0Bu3CZQhz3zapxlZ2hZd8INTsOQeHSuT7lK9Vq4MWOxTclgpYhCsHzYarYJel71QdgjfVMkBESlP0PnApD7e3II7SmZPvxwCI8Krhm2K0akgVl84iKX8ge1tFWgylyc7LPBptMbQFysmId+QPsw5vNbwc+Un5qC8VV2Cq+0v/GntYLzud5vXASyf97/QWdUvPSnn/wuYcCZFvRrrVTEV/L+lWCSeTrd0WUF4FgclwHl33f+YCNmNUJ4PQFOdcGR4wEozD2uMVIFbVrREusWE4LfmfL/H+Uaej0fs+GA10Fg+J4ZyS 4tSepRkO WZWSjUsA7GU1CzFPdKkEcxFZ8Ree9IftALjwe0mCRkSKj8TkN3ruEWE5baCL75idE4bo6gnJnCx3cZ067MozOD9fL41AE6rsEh/xukObNZkQ2UXuQHcaZjJCP2q6r8wyIYWmo3oqihwm4AR5hejFxPIJPZNHgRTDyyRyTbi2tNyiev7HUn+9yafaUhuuRwTeJeXfZTp0Otkpq41p8fYhTLGlGOJ+JGQ6UG9KQ3PHxJzybsN+AEW7AnetKwHOTmeBEz+p5tncOFWAMqFETZA2DZtRELN6PphTxMegxmQGj1L2mv4XPK4e3IEgN9sFR2iCVxxeMCJ7zPF18tcsBK8aB12sZ+Esty8EHgXyS1wgqzKaf210gQWqPl9+j77uak0wEX2MLxPNBQwqnczokBAfF/OCZ97d8nf6gZYZyNgt38FqNklZKP2NFe6kSaDWXtW3saWMOurUeHKK23ogB7apzfc+Njt/v6cAONR00wP3WLkLJWHxe0OS48MzY12hFUZhCxvnJFNYWcra6dlWKS1NRsoNwHA/JMyosnUU7DfMm856yztAoDlFrJEhBaHOBQmB6XTETKeQYJq1t9aCxNF2mtOdNsQ5lf1o9CZk/OmevARIXIZlFvQrTaFIiSJ3GFif9SxE4BCKM2ij2T+psvk48eo4VNMhyla/VyTvl/Lahhnrvaxg8XfZqGvkGWCNfcYyZ6QXdqNA94PAp/cNdyFLGE45+AFJh4eDSRk3b7Rr5BJvLRNg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The behavior here is the same as it is for anon/shmem. This is done separately because hugetlb pte marker handling is a bit different. Acked-by: Peter Xu Signed-off-by: Axel Rasmussen --- mm/hugetlb.c | 19 +++++++++++++++++++ mm/userfaultfd.c | 3 +-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 66225b21c64e..8789101de88f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6263,6 +6263,25 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, int writable; bool folio_in_pagecache = false; + if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { + ptl = huge_pte_lock(h, dst_mm, dst_pte); + + /* Don't overwrite any existing PTEs (even markers) */ + if (!huge_pte_none(huge_ptep_get(dst_pte))) { + spin_unlock(ptl); + return -EEXIST; + } + + _dst_pte = make_pte_marker(PTE_MARKER_POISONED); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + + /* No need to invalidate - it was non-present before */ + update_mmu_cache(dst_vma, dst_addr, dst_pte); + + spin_unlock(ptl); + return 0; + } + if (is_continue) { ret = -EFAULT; folio = filemap_lock_folio(mapping, idx); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 68157359dc34..eaa59fb81719 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -373,8 +373,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb( * by THP. Since we can not reliably insert a zero page, this * feature is not supported. */ - if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE) || - uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { + if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE)) { mmap_read_unlock(dst_mm); return -EINVAL; } From patchwork Fri Jul 7 21:55:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69CEEEB64DA for ; Fri, 7 Jul 2023 21:55:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94BA68D0006; Fri, 7 Jul 2023 17:55:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FA2A8D0005; Fri, 7 Jul 2023 17:55:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79E338D0006; Fri, 7 Jul 2023 17:55:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 658208D0005 for ; Fri, 7 Jul 2023 17:55:57 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3A52514014D for ; Fri, 7 Jul 2023 21:55:57 +0000 (UTC) X-FDA: 80986173954.03.C1AA935 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf09.hostedemail.com (Postfix) with ESMTP id 6698F14000D for ; Fri, 7 Jul 2023 21:55:55 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=WinlA4xj; spf=pass (imf09.hostedemail.com: domain of 36omoZA0KCMsrEv28r93B99v4x55x2v.t532z4BE-331Crt1.58x@flex--axelrasmussen.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=36omoZA0KCMsrEv28r93B99v4x55x2v.t532z4BE-331Crt1.58x@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7OMVl3YxFVN2A+Ry/UxbO1YmCHHRZcxw7VDxaJg9ek4=; b=xcCYhQy61ku0UYmAAjp4K4DdZOfad7GIS6ekAuhyWdRIjrRZycN+gl+6ft6QP4pU/hZ2oi E1nTG/bf9m/S4DRDzxArijmbXRSW0dZFAVe5Q9V/tdmnbymebIUZU+v0wD7AGqX5e5Bnt5 x6rNLLsJ5h6wQadK1XBVWIat1UQ8XRA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766955; a=rsa-sha256; cv=none; b=PRucCZFuqG7u3/vf1x/p2d5cPazHSPEvutKTWu/jMweQRW0iygZW/itKON2vHOo0F3Wbkb UKF79UV4QJ+xHQqOLLopOFJBk8q9ODdjCjrJ5oCiqwGRO44Zc6PstF67islu3DEk0T5YT9 xWxlhhBtK4YS793K2ZjOJ86huFsJVYg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=WinlA4xj; spf=pass (imf09.hostedemail.com: domain of 36omoZA0KCMsrEv28r93B99v4x55x2v.t532z4BE-331Crt1.58x@flex--axelrasmussen.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=36omoZA0KCMsrEv28r93B99v4x55x2v.t532z4BE-331Crt1.58x@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-569e7aec37bso27544547b3.2 for ; Fri, 07 Jul 2023 14:55:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766954; x=1691358954; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7OMVl3YxFVN2A+Ry/UxbO1YmCHHRZcxw7VDxaJg9ek4=; b=WinlA4xjLrm42Y0LrkLO833bYnLkHqw90W20NHsoyLZQtg1RvWlssOMFOMjd6VvSn3 iYv6kZ6oeM6LcpvIDaTEPpfcc8gSW/pj+FpAaO2nVo/YQ4FKqs5+alRw0RQFBHZ5C9Li WwXNwLXfn0bdHjwb+FCiBwlKRuhOJzkac2hZ6zvt9v8rdPkWboNk13P08v0VELKTQZoF 0QcDn0IuZNiLfUlhg+XzK/7q28VwlIE5Lf3mwk8//FBbyC0xrc3lW7P3+clz4gUrnpYh S59eoTTQlzeW4FwWK37lEymavNQ2LCOR9UtKZW+BbCcuM6gOcPJpGzfDLhFZE3wyjxHA uahQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766954; x=1691358954; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7OMVl3YxFVN2A+Ry/UxbO1YmCHHRZcxw7VDxaJg9ek4=; b=f9v+UezewYT3YBjFrjPtBwmdaZtx6C4sQCZqt5DK84oGz08Ifq5ZNjlUj54bmgiizW 6X6xBESURlip5gqnyd2FKjgeKpnxXpaJJcKUolNtsve4dyB3XtE7b0/RWZ8Vf4MD5cPE yzybVYdhSPO3Thzv4YoEtxOeeUnDiWQEuJD9iz5t7YxATYNkl4TER7Nw4yunSXZv/oRl OP8JHkXeIrzLIdHT1b7vhz5dx3TFq4oTGX2HVU1XxUfs1Zx6IZ0jHAXtfhX61HVOQi71 4g+xGtavseDkh5TVAsuz2iBxdGf//mDcADinoE8tuHVAxX71PmlaDYBdgsBfA/M4iMNd dWMw== X-Gm-Message-State: ABy/qLZKlTyR5E9mSduOst14g0km7MeNUFJcRMeH0RzumXwNrjSa3t23 ByegB4U08mJF9t/jfT+HbP4mNnt+MhyYUIEnjBqk X-Google-Smtp-Source: APBJJlEb1nGvsIgNN71uOPifboK21axP2OJKOqwVdFOinGjePMT+S3RRgV5WMxsRC8ogPCTBFhgiiAFAPJrAtTi306IK X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a81:af1b:0:b0:573:6b9c:6fa1 with SMTP id n27-20020a81af1b000000b005736b9c6fa1mr41633ywh.1.1688766954512; Fri, 07 Jul 2023 14:55:54 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:38 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-7-axelrasmussen@google.com> Subject: [PATCH v4 6/8] mm: userfaultfd: document and enable new UFFDIO_POISON feature From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Queue-Id: 6698F14000D X-Rspam-User: X-Stat-Signature: e43bgqos8a9uogagzx5emyra87rs6m5w X-Rspamd-Server: rspam03 X-HE-Tag: 1688766955-931884 X-HE-Meta: U2FsdGVkX18eD+sJ1oUp7QoC1xsAs0Q3jnXLwAl40bkMy717IfPEL5aN3WevqgT9EGh9inCXsn6H4I21rdknUcH6v3fLE9G7U5O7NNf0ZIv9MLhS5CKWa9K3n6i8XORxVsM4VwTqO8CDBW5FBK57B/zlxlxNwkLgCpJwtWM3Sd8nH786usHB6liWSgUnoZS34FW8X3X+1PGBisoA+dQCISqhiImu2z8LstzYuhm30w+fGwa2e4jSIbsttI6LK67upDWYohDWfmEMCT5Nhufw9KvqGiXrWOTpGxuUkcDQ8bCkaghc7nq2Q9LHx1xz/5uqVBJIuFVHx1XKYTi7Fs4DBTXXjY1ukzZxdQWHCgnkbUIPIMnuMTQ3gy6xyLLJrm4Hbmct2OGEXwBqv+sOWNc3/LLx8W0C5TKDJIW5I59+ATSEpDsEAVPjc0SQKZYFafMmTouWHNaCce9C/QQdDqjtsDpax3bzcTSHm2ImVAvA11MZQj6z3BuXebE7wLv51oD2OcSYTfwnC9mQ9V1ETanKiCiVq0p5cGRtkPtcvqvgYlCCeoBf49L0DCzCMlaqubzz1STw1Emtf7N58k2fyE1tuSWd9cEPCYqDi72CBeKCvGiJttvKmPGsPlsVlpBWjFH00bVQl24TpQ81T6d9gCloXxjhYrkvjb2hKLFOtdqBpA3c9LQEH7pS0lhf97bU7xaGm3102f3bGBKk5LS3dIFhr+tTF0LN9VqoK9Zcdz3B/ICywJYmh/httPL4cj8Lja0JzDSL+e1I86mwutF+ZtdH1Q3P6/uHUBOkT7NAKmEwVQimkNaoGRMRBMdoy+Is0RjRmRDeSwQRiSbsziqDPV0hcwZGqOIjGB6wMxjeganeQDz4Y5EtRR3zSFXJ8eBQsC/TYSEFvQhBfTKJQdytXJU0KYEEDtywh3ebQ0HdBDAj6i5Te5FdsEhoCnUXNCMa/viEDgizNYoOB3JCZWSYRIC GKnM0Rdy qRMYS1LUnBWB6iNRlmNx48nI7dGOazOVuRU7kzYx7BXIrWzdt5gsSYMN0Cxq2cJFoJsPSRxszU2PTBufUl8KDfqpoRgmRAlj1UjQsKZHecWtvh28x/cQR1iRMpPJ6p4bngasb41CZg7/05HcfkPkiMttSdW8K8PMkMnrAL1dRIjbhIenwCPQZaPIWkaa8/OeP0tkwrBZiHN1yYFYBf2ELFCdR1gZ5VaGiUmSNA5BLvGTmIrzrlIN2e7U60r139gAUswbFvDC0YSrZWcHA1aC3RL5S+jeodtDxkFZ1fpqNcEzL0wPOW5k5k4H9qGf1lEU0+2SRQXqGRxxS9fHgEhwA9buMIzFb367pjridyz0rUItUqcrOp8spWqfIgxevrCB8Rq9asY6PVlXpBlcCkDbafoh0U447YXjaj1YDkRU9jA4TDpMS9liWxfwl89L0bPxOcSPQzSGKjhxXCf5Suc9vItX6IOTxWYwm7MUVyLKikRCVYkIYdKAW9CabnJ3/0KzbbSFY3lzDTC3idzHZBIxD9VMHZcsP1e1rX49m66issUqXXjAOvkZppjLCElnlkwCjRlW14aKVUKO8FzKZtRwulf5ocs0jJU/pmnLbXombWNlA2WttreAl372u7qDtyfyxjhxGQLI072cU6o5UJAWzQT+TlNvTSwZtOyHK9ouEN34oFBYwL4HM36GCJp8RsNChbRApDxSIS3v/iZFHvs1qEVWXXR2rVrbTPu+Z57fLDfHIvJM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Update the userfaultfd API to advertise this feature as part of feature flags and supported ioctls (returned upon registration). Add basic documentation describing the new feature. Acked-by: Peter Xu Signed-off-by: Axel Rasmussen --- Documentation/admin-guide/mm/userfaultfd.rst | 15 +++++++++++++++ include/uapi/linux/userfaultfd.h | 9 ++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/Documentation/admin-guide/mm/userfaultfd.rst b/Documentation/admin-guide/mm/userfaultfd.rst index 7c304e432205..4349a8c2b978 100644 --- a/Documentation/admin-guide/mm/userfaultfd.rst +++ b/Documentation/admin-guide/mm/userfaultfd.rst @@ -244,6 +244,21 @@ write-protected (so future writes will also result in a WP fault). These ioctls support a mode flag (``UFFDIO_COPY_MODE_WP`` or ``UFFDIO_CONTINUE_MODE_WP`` respectively) to configure the mapping this way. +Memory Poisioning Emulation +--------------------------- + +In response to a fault (either missing or minor), an action userspace can +take to "resolve" it is to issue a ``UFFDIO_POISON``. This will cause any +future faulters to either get a SIGBUS, or in KVM's case the guest will +receive an MCE as if there were hardware memory poisoning. + +This is used to emulate hardware memory poisoning. Imagine a VM running on a +machine which experiences a real hardware memory error. Later, we live migrate +the VM to another physical machine. Since we want the migration to be +transparent to the guest, we want that same address range to act as if it was +still poisoned, even though it's on a new physical host which ostensibly +doesn't have a memory error in the exact same spot. + QEMU/KVM ======== diff --git a/include/uapi/linux/userfaultfd.h b/include/uapi/linux/userfaultfd.h index b5f07eacc697..62151706c5a3 100644 --- a/include/uapi/linux/userfaultfd.h +++ b/include/uapi/linux/userfaultfd.h @@ -39,7 +39,8 @@ UFFD_FEATURE_MINOR_SHMEM | \ UFFD_FEATURE_EXACT_ADDRESS | \ UFFD_FEATURE_WP_HUGETLBFS_SHMEM | \ - UFFD_FEATURE_WP_UNPOPULATED) + UFFD_FEATURE_WP_UNPOPULATED | \ + UFFD_FEATURE_POISON) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -49,12 +50,14 @@ (__u64)1 << _UFFDIO_COPY | \ (__u64)1 << _UFFDIO_ZEROPAGE | \ (__u64)1 << _UFFDIO_WRITEPROTECT | \ - (__u64)1 << _UFFDIO_CONTINUE) + (__u64)1 << _UFFDIO_CONTINUE | \ + (__u64)1 << _UFFDIO_POISON) #define UFFD_API_RANGE_IOCTLS_BASIC \ ((__u64)1 << _UFFDIO_WAKE | \ (__u64)1 << _UFFDIO_COPY | \ + (__u64)1 << _UFFDIO_WRITEPROTECT | \ (__u64)1 << _UFFDIO_CONTINUE | \ - (__u64)1 << _UFFDIO_WRITEPROTECT) + (__u64)1 << _UFFDIO_POISON) /* * Valid ioctl command number range with this API is from 0x00 to From patchwork Fri Jul 7 21:55:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16549EB64D9 for ; Fri, 7 Jul 2023 21:56:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83D298D0007; Fri, 7 Jul 2023 17:55:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D8F88D0005; Fri, 7 Jul 2023 17:55:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A0DB8D0007; Fri, 7 Jul 2023 17:55:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4420D8D0005 for ; Fri, 7 Jul 2023 17:55:59 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F0FA1120149 for ; Fri, 7 Jul 2023 21:55:58 +0000 (UTC) X-FDA: 80986173996.21.37E1AFD Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf03.hostedemail.com (Postfix) with ESMTP id 304EB20009 for ; Fri, 7 Jul 2023 21:55:56 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=mSszojzK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of 37ImoZA0KCM0tGx4AtB5DBBx6z77z4x.v75416DG-553Etv3.7Az@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=37ImoZA0KCM0tGx4AtB5DBBx6z77z4x.v75416DG-553Etv3.7Az@flex--axelrasmussen.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766957; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gK2BPE6zTmiKkfK2gtxYAq2cBVrZ0W+ejTntqdgctKA=; b=Uvh8VmefIUGlMMgYFULQaI3tAf5QE28b2UaU98JB4J34shlSHKvgtNZMO0CT7mYNQbApCd BDeoeZqo/mhDco0aj6DVcIMIUpiZf7WBG7CBXfPJ+05MgOvV/3yI7rr3ySTti3569/U1Ua oUw8sIpDcM3bWwRGL6lRkPF2HaTbnXc= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=mSszojzK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of 37ImoZA0KCM0tGx4AtB5DBBx6z77z4x.v75416DG-553Etv3.7Az@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=37ImoZA0KCM0tGx4AtB5DBBx6z77z4x.v75416DG-553Etv3.7Az@flex--axelrasmussen.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766957; a=rsa-sha256; cv=none; b=63g6jR0vQsz16ztjzur6yNZW1EyOSgDGk5NvvWUV/hZdWypRMRf3CJNP4p7ghCV2mtRB0t ngz9LhOSBrPi6MjDPbAU1IAEC9nlFUoBGjqq9VI0r1D/QVgNkPi8aJYM1zYDNsYxyGMH5K WguxtRIj002ADOMwhYBhaDYPhLRUkew= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c56c5e61ae4so2480386276.0 for ; Fri, 07 Jul 2023 14:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766956; x=1691358956; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=gK2BPE6zTmiKkfK2gtxYAq2cBVrZ0W+ejTntqdgctKA=; b=mSszojzKEahxrcX9TmdDeq6BffcUFhBof3Wfa2NSLReCESkfHCMI7XDPflqSMkDGpn ouiK/1EZIjxsDpq0jMwqD9m1X62FyMq4nDzBgb6M7/ML2imJB1U5v1dc1l+ah8jMdnJe PB5xOLeeNwdXhcD53UzfTcq44+hoXfPErYaXguCDFgNeDDRdB2ZRmDfpfqDhBFUErms9 b4UKo5d1K8Kn+ry9fkThfTtz75Zrd1GT+xA0NJx4QctBIIrNfwI5B6zi4Oqa1PV6Le9z CZ6qPQtNTcVthIO6ODY++3/5NAu2trU9W9ps5V5xHMzx1D8Foq0j24NK0X2aUZ74o34R JWeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766956; x=1691358956; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gK2BPE6zTmiKkfK2gtxYAq2cBVrZ0W+ejTntqdgctKA=; b=P12AKOTc8tpOii3cqJ6zFzWJneUhKWX6f1s6IsrH2oXcDaGMHJ4Tc/ZBPgJtGO5ydQ wDi6g5//zwFLQFxQvMZTIlev5vi7QfpBCS4PTwdIk6CH8QSDroqnf8ifPf800SQ1xivv tzC5j9TXjGURLwD7kSkPF3x08TgOEvNclaSzoYUyRhJDajEcqS04K9ur+S65/bg8F3aI qkiIuUnMbtKWVjRRAmoBgXDEVqqLbfa6b3U8L5g+FhvQX01v97CGamXSiRgN8UxEPope tOvZHNTf7V+6s9cO3wRmmr8y5vPBnul7lL9YONfgz1vGWKjPHDVod070PnEqp5qt+zsp M4kw== X-Gm-Message-State: ABy/qLbK5sF+u8j7dXvw6MtlKLBU1giksouzMLtoIdwFkUZzAjJus7IM lTEowYyU+38nb2+Mb43qU4lMfbCtYem/ykBtptZc X-Google-Smtp-Source: APBJJlFFlaqfrCAHxFCqM3Bx4zG+dvuvfGgO5ZnIbPvaXcko0/wlrp48kEZGBI/2IRBZ3QqzAWog232RIOlbEqwMvlQB X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a25:f448:0:b0:bac:adb8:a605 with SMTP id p8-20020a25f448000000b00bacadb8a605mr43005ybe.2.1688766956303; Fri, 07 Jul 2023 14:55:56 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:39 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-8-axelrasmussen@google.com> Subject: [PATCH v4 7/8] selftests/mm: refactor uffd_poll_thread to allow custom fault handlers From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 304EB20009 X-Stat-Signature: xcwyqaqkm9a1jaizxzewytmkz8p9nnhu X-Rspam-User: X-HE-Tag: 1688766956-989525 X-HE-Meta: U2FsdGVkX18Y2TPL9Js1Gjgv+5LxUELeoTwE3H/QFL7PC4ePVUf5YTpl3zRx26TJt+LRZ4xuqUxAEK1cE+22zBWLWtiGTjyL2qUa+N3NWkq7HiTC5ckrOCBeX3yZgo9vwEcrEWuLNF8osBRFNWSyM7mgw1LQgCMxhlyR4qRljGYsY6VF5lKuKxqLof5D6cwvkZX3YWZL9qEEr1vfZO/9h4BfqzuPxbqGS3Cj0BTJBihM1bGMl8CACC9A0bmHH5REDSe/QI3/6hJ7vdSfjQhdMQ9NUgnMVRFKT0VtBCNoUuOqa4UXtb3dviwgx7i8Dqc0lqlETgk5dmWcZ09rroAJ2QMIKZtkYxWZzypoEEUBhUWqvwDmyB7lR9kZt11fh+DFs2ZFtCgrJI68AEAVqiJrYKEc7EhQk65eLkyqCzudIh58tkcXHoVXssdlPE14fIACL7vKnGIbAygTnjhQwRWUk28xtmxqQULsHPn4ygFHziYdnZmfY0cIwxZCXlR00ig+rsvjTGfDbS04UrnjQEC5XrhOWTBWIySY92QEpc9Zlpu8JiCwDBSDeu4hnzhAfoM4GdMhNSvuZ9L3j0xdlV+dXgRXTuqE+c0VeQGib6d54daN/woxpeuBbWXCamfTs5JuT7p7VzatJyXxXQ+CEyfyuGMI4o1i4aFW7xxebXTryh06qngPFpFRAzglWeFwieG/WvREUooPRlKNsgKyh6dlbVl6IeHFKbOU8BgIIbRNtQtnSmHhHkfCtyZHHaQYnoNEPNYdyeEBwFp+X+ZGPZeMkWVpwxM5AkXwfyfexsz/J+QthBWbWoAL9yDY42bSQgc4RC3h9OTVIskZFrxcagGNar+5d68UleoBrLI/18FzfFe+SM3YFwrtd5622JP2LM3dfGGrnDRSCGQd3UsFLXqiNPdhArJbTJAXk4F18JEVzzQjL6Sq3RNGoPVamUy/n4CmjKkpa1zN7/GTQPpQ/OS q1WOGQ2O d0dNi2YzH8PKDpceOs9YYHygchhHXVo2TPV2flGW57lW+j9fUvY4pVokccdC7WoczGZ88LLL6IRqVzdf0jjP8xqks8g3fTfV/G0dz87QR+b8GaJnC8XMFh+iLUaCY3SkGTzToHg8Mm48gOjWJP0BSeJMxuwxIduZmMSzEGgUP3kCwRbQJ/WvCVyb9xYAL5Nvr5PkIGV+79gPND3UjbHGVSMee9z97T8k656hGDC1bazg+WQNfxyCWJ37Bi62CGwHO9dSkH7lZvSe8vqpvR+tfeGrV+Bs526UqwDrHdLxLAuC6bbT7qbQblEGfoKSBN6Miahc1XQqT66U0zWKsskduTuiHbQmleKFlfepJjXIIHlX3piL/1gElRcI6lIR8f0yHi88zjmf/1Y1ry8PgEM+N2wzxfFGPP3U1HXtItV3fKPp9SE+yrbz26kc6n69Yr3xmDPYxhqucZGDqGNtKUTp2Vp/pH16oUpSWhw4EVg5htBAA2hZkq4fZrEezbltleW2CLwd9SAKCybf3I2JBLP6URtUSOdWEBw178cN2I7SekHIb6rZ3EM4SZu3UJUZKvKCnkAuZu5U4hFAx5ZQqihlLG90ZEhCL5WImWp2zj6lW6HVh+jG3YpOOFpnEej31ElXaehGr/TotMEa48Ox/VykJiENJL8bWMIFpDdzDayC+7ZQGuE2rpKicJ9Z3uocKlMOWJvsCCrgMwzjR0uDV70bdi7hASg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Previously, we had "one fault handler to rule them all", which used several branches to deal with all of the scenarios required by all of the various tests. In upcoming patches, I plan to add a new test, which has its own slightly different fault handling logic. Instead of continuing to add cruft to the existing fault handler, let's allow tests to define custom ones, separate from other tests. Signed-off-by: Axel Rasmussen Acked-by: Peter Xu --- tools/testing/selftests/mm/uffd-common.c | 5 ++++- tools/testing/selftests/mm/uffd-common.h | 3 +++ tools/testing/selftests/mm/uffd-stress.c | 8 ++++---- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c index ba20d7504022..02b89860e193 100644 --- a/tools/testing/selftests/mm/uffd-common.c +++ b/tools/testing/selftests/mm/uffd-common.c @@ -499,6 +499,9 @@ void *uffd_poll_thread(void *arg) int ret; char tmp_chr; + if (!args->handle_fault) + args->handle_fault = uffd_handle_page_fault; + pollfd[0].fd = uffd; pollfd[0].events = POLLIN; pollfd[1].fd = pipefd[cpu*2]; @@ -527,7 +530,7 @@ void *uffd_poll_thread(void *arg) err("unexpected msg event %u\n", msg.event); break; case UFFD_EVENT_PAGEFAULT: - uffd_handle_page_fault(&msg, args); + args->handle_fault(&msg, args); break; case UFFD_EVENT_FORK: close(uffd); diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h index 197f5262fe0d..7c4fa964c3b0 100644 --- a/tools/testing/selftests/mm/uffd-common.h +++ b/tools/testing/selftests/mm/uffd-common.h @@ -77,6 +77,9 @@ struct uffd_args { unsigned long missing_faults; unsigned long wp_faults; unsigned long minor_faults; + + /* A custom fault handler; defaults to uffd_handle_page_fault. */ + void (*handle_fault)(struct uffd_msg *msg, struct uffd_args *args); }; struct uffd_test_ops { diff --git a/tools/testing/selftests/mm/uffd-stress.c b/tools/testing/selftests/mm/uffd-stress.c index 995ff13e74c7..73ebb97c7026 100644 --- a/tools/testing/selftests/mm/uffd-stress.c +++ b/tools/testing/selftests/mm/uffd-stress.c @@ -189,10 +189,8 @@ static int stress(struct uffd_args *args) locking_thread, (void *)cpu)) return 1; if (bounces & BOUNCE_POLL) { - if (pthread_create(&uffd_threads[cpu], &attr, - uffd_poll_thread, - (void *)&args[cpu])) - return 1; + if (pthread_create(&uffd_threads[cpu], &attr, uffd_poll_thread, &args[cpu])) + err("uffd_poll_thread create"); } else { if (pthread_create(&uffd_threads[cpu], &attr, uffd_read_thread, @@ -250,6 +248,8 @@ static int userfaultfd_stress(void) struct uffd_args args[nr_cpus]; uint64_t mem_size = nr_pages * page_size; + memset(args, 0, sizeof(struct uffd_args) * nr_cpus); + if (uffd_test_ctx_init(UFFD_FEATURE_WP_UNPOPULATED, NULL)) err("context init failed"); From patchwork Fri Jul 7 21:55:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13305365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCC9AC0015E for ; Fri, 7 Jul 2023 21:56:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C30E28D0005; Fri, 7 Jul 2023 17:56:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BBB358E0001; Fri, 7 Jul 2023 17:56:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E4A28D0008; Fri, 7 Jul 2023 17:56:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 90CF78D0005 for ; Fri, 7 Jul 2023 17:56:01 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 663A114014B for ; Fri, 7 Jul 2023 21:56:01 +0000 (UTC) X-FDA: 80986174122.04.E38B9AF Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf09.hostedemail.com (Postfix) with ESMTP id 8458914000C for ; Fri, 7 Jul 2023 21:55:59 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OMdpoDHV; spf=pass (imf09.hostedemail.com: domain of 37omoZA0KCM8vIz6CvD7FDDz819916z.x97638FI-775Gvx5.9C1@flex--axelrasmussen.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=37omoZA0KCM8vIz6CvD7FDDz819916z.x97638FI-775Gvx5.9C1@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688766959; a=rsa-sha256; cv=none; b=xRWvlgJN7qPkVgIIkKAP7j+yU7ii7sXCnrlV4fjdp+VgJt4zHTOGpWgBOxE0wydILEK3KF 2czXnC/ewPGU+5YeieQwg89Y8TyAcxpPHmpYDf4qrZfFOBSjIGcNZBQryMaRVArQ4sSEeL cxzEbUzq7A+i41adYh3SkQVVb7ogACk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=OMdpoDHV; spf=pass (imf09.hostedemail.com: domain of 37omoZA0KCM8vIz6CvD7FDDz819916z.x97638FI-775Gvx5.9C1@flex--axelrasmussen.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=37omoZA0KCM8vIz6CvD7FDDz819916z.x97638FI-775Gvx5.9C1@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688766959; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vNyM+7c0est1wcVkLGeYyWS2zFzaKr32nANrKlDiGpI=; b=20PiMG6WRMm18RsJ4+Zoax18JqkQd3WHgLXy5U3ITX+Ijvx08AhvJMcIaZy9GHfswuG9mZ ydALlc05pozEzpFMnyD6OmWdQr6N6GB3ibDWD2pM0Vb4M3ratCSC0GNqbTPknzCjCX+OSi uARFxiI/hoXSVxJf9uzlyCxuhz65oFY= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-56942667393so26704297b3.2 for ; Fri, 07 Jul 2023 14:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688766958; x=1691358958; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=vNyM+7c0est1wcVkLGeYyWS2zFzaKr32nANrKlDiGpI=; b=OMdpoDHVwHWGjrcqQJPsCz18MHB25H0LJo2nJDpMTFaEBMMVUhi44V1ldpofEQheC0 m4JfYqAGr/gmOByQh659ri0DTCXPUN3wq0q9Y61OTOaapOrMdiNoHQh6afOT6Th9pdol 6+Ht5/zRVaIIhULOylK6yhUyyPHER4k+z45bkpq2bwwXlhD5GQG21YfPfmVH3RloLN6w wtstq9y90K39zITeLzb/scB5M7A5pCo8nSgwdCQYNkCCv8K/hLgyaE+soykDU5ln1WLL 7xsIGId5npFZRXxs0JphIddyCoDPMBn40haDcCt74ON/WBh0oWvk/3GCiN3vTTt9Y71N aqMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688766958; x=1691358958; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vNyM+7c0est1wcVkLGeYyWS2zFzaKr32nANrKlDiGpI=; b=N/S7gcx6pHeZXd/px8qeKD8XeB6usNoUDLNJSnPn0IkbyxrCTdr7939P+gMAyRx9+E Gbw6rQCaddhtSlfyh9oUVtN/LsOONfrAyF2+80sDSO4ZnduS5TeoVE2UuGD6/XJxLq2F ZIMRxCc5M3tzWgkKqy4AwRSYkRf2jogoh/gvWYa3KTMBX8w/YY4e4lMJCWSX0qR1XAka eEKn4ze3Rwx6rzVdT+eS1OQ1FDJ1HkrX5u+lxsoMXVYIC394LM28AgrSLqAyNYoOhRvk TowqdlKpZ2VRU8MWY/GfDsqtXqHA2Z/EWUPVg8fuh7vg9C5KjfP4TmzMzJTW34wuEX6W tUyQ== X-Gm-Message-State: ABy/qLYda0UYvHda9TUsM9lfR+9RqtEPfDQ81uxMmkkuiwUpTkP1m1Fh TFFxkCwaXcb4nPvULpkhITLhYqjEYh1u3OUhrNXV X-Google-Smtp-Source: APBJJlEkVp64R6YyNIYgkWWJLixkffDfIZz5S4mpdabnKl15q3RW7laRRo2Y50RlxbwI2hpdw7ztg2lX8JH5LcyqTH+L X-Received: from axel.svl.corp.google.com ([2620:15c:2a3:200:c201:5125:39d1:ef3f]) (user=axelrasmussen job=sendgmr) by 2002:a81:b3c3:0:b0:569:e04a:239d with SMTP id r186-20020a81b3c3000000b00569e04a239dmr45939ywh.0.1688766958488; Fri, 07 Jul 2023 14:55:58 -0700 (PDT) Date: Fri, 7 Jul 2023 14:55:40 -0700 In-Reply-To: <20230707215540.2324998-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230707215540.2324998-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230707215540.2324998-9-axelrasmussen@google.com> Subject: [PATCH v4 8/8] selftests/mm: add uffd unit test for UFFDIO_POISON From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Brian Geffon , Christian Brauner , David Hildenbrand , Gaosheng Cui , Huang Ying , Hugh Dickins , James Houghton , "Jan Alexander Steffens (heftig)" , Jiaqi Yan , Jonathan Corbet , Kefeng Wang , "Liam R. Howlett" , Miaohe Lin , Mike Kravetz , "Mike Rapoport (IBM)" , Muchun Song , Nadav Amit , Naoya Horiguchi , Peter Xu , Ryan Roberts , Shuah Khan , Suleiman Souhlal , Suren Baghdasaryan , "T.J. Alumbaugh" , Yu Zhao , ZhangPeng Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 8458914000C X-Stat-Signature: 41qach83f487onisopb43ugw49mp4fj7 X-Rspam-User: X-HE-Tag: 1688766959-165084 X-HE-Meta: U2FsdGVkX19cd1vKGymCmp/v29abqxRWfxh3pyq9X4pcEcGTVxMNwktTqQDwqLQpnDfyL3iaYB1bKpWXazqSmY3q9N3qgWkMAcNuQ0taJimNd+3yPOyZb+k3hYGGnlid00u1hCuegXo6QoN5btU5tYt4Z5rD/3dyp1VBYZCVAy6T8c2tCkg8Gm8ooPmgRIsBQTdgILAFjlId3t+7CCBPp7FkcFh2u4k34fLkhJEjaxmArLP0+KrIPrmJtLRg92puSsZ1+pu4SXSZn1XfSLem3PgcUtSKnxXC4iNcfmy9D801pLkei2o2DGVTuMG/3pKw0+gxtOrbchFOiEUD8dF9K+1FVnIF1caxfH3VPz71OqjBVvqlWGpq054LQS2cNPfHHdEIlCiamsOAAODyL2lt8u5qvozK0UTi0aHCcr8T7Eph2Vo5auOPoTWW1Hzx4XmGujQQPPTlwUuVdxceAIqOhIg4hwXdz1k3wlPtQKOU5QaSpet/fr2c45WDVObt2j2lw8ybIPETSJvXIq1g0rsY4k2Rb8mftHVLepNIgJ50GhItpbafFNfbfLg8h74nlvukWibdheT8vDkM7eiinGL7NiMfreM1mfHoUae5FSrrGRuxwoCEQEJ1GOMyFQJDIJAvX0N95PG78W529PXOJqEiDf0HYxsBbfXLvTWpd9pcu3YB5B59WGwa8qq4tVCibBpjK5+LppUULLketBKw5Mrt+niMI2L2lgSYCts60KbiUZIE4lUjd0ESmUZlnxS3tOewPr1EyEQ+Rpy09z8OmQdlYtfzrnMnByuP4kGPG+qYVZhkyeK++ylezVFlqZKa2VVCQBSVBmPPfn/t3tIbuxc7ioDz8LiKi0dj0VeVeDuAmHn4tZSN9j0llhjO/JrLoFreJSjawG0LdNW1BnIiCM2eGnWZcZ1a0ZSm0I2/UCrDgapKMHFKVM4gzArUVpzs2zwZ0wYE1ddzHvViycguzyu YilZqq7L UkUbxxziqpnqB6HmMyPy+74esFsY6/V2ZT7oLOI6xUvybw7btjz0f5jGqaw5VWqOZWLhRvHKJL+4vaAwMvLOZ6Q5CJTYU5r6E+8hObdZExpDSb74Wyz8Z3NAv1Epr7XpYAYe2XLYzYDWByiiL8Z62X4kiVX67OdTLBYgQhskBB5KknYwI+AroQSYQ2/IRCF6iUOGtc4mz1MmiMYSIJ0GCLyeOHWOqbv5Szp8br7Fxf0NI2VD/rfHSQxVMUEXC9coAzzKI1l5vU1lCrENPoAXha4exCs5VpIqCgcz/9Zmn44XlXT0W7Ucp20CWUlJk8WlK0WiEbWHEIzf4xQQU3OExLz+5nSKwAgiqh5ESaoBvGv0rwI/0hVUZ6Jjjsb8/Yt0aYT/8bELqWbKqQwhXZJJ+Kjqwu9DplwfjzmMJCnPzIiuDgQdUaIUztsosH27WJUPo+eItXZx9ZZeVqlV0hGg7sOk+qhwCMMIOTPdJXA4/V+dWP5oE6Q9RyWQDMYgVkeFQtV8AwvYjA+qdM8Sn+yyFUzBCqb1QAyVHUJW0Ygg/ItSNyUPT4bnWikJ1sNE6duxINoUYq75ajcCZtcwJpPv2XSG23m5LMeEHPvlX4M5fMrLoUM3daAE4ZHI1+aCSlp0Umo0wpQxIl3x2qGIdnc3m/8gp7AVoWmX2FIf7dThYxKtlrT1MhfF2A8waDGJVhY7jGARQ8Z60ZvIoaD/flTS/XtrJhjpiDdtUwrBZT1/r71IBSHg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The test is pretty basic, and exercises UFFDIO_POISON straightforwardly. We register a region with userfaultfd, in missing fault mode. For each fault, we either UFFDIO_COPY a zeroed page (odd pages) or UFFDIO_POISON (even pages). We do this mix to test "something like a real use case", where guest memory would be some mix of poisoned and non-poisoned pages. We read each page in the region, and assert that the odd pages are zeroed as expected, and the even pages yield a SIGBUS as expected. Why UFFDIO_COPY instead of UFFDIO_ZEROPAGE? Because hugetlb doesn't support UFFDIO_ZEROPAGE, and we don't want to have special case code. Acked-by: Peter Xu Signed-off-by: Axel Rasmussen --- tools/testing/selftests/mm/uffd-unit-tests.c | 117 +++++++++++++++++++ 1 file changed, 117 insertions(+) diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c index 04d91f144d1c..2709a34a39c5 100644 --- a/tools/testing/selftests/mm/uffd-unit-tests.c +++ b/tools/testing/selftests/mm/uffd-unit-tests.c @@ -951,6 +951,117 @@ static void uffd_zeropage_test(uffd_test_args_t *args) uffd_test_pass(); } +static void uffd_register_poison(int uffd, void *addr, uint64_t len) +{ + uint64_t ioctls = 0; + uint64_t expected = (1 << _UFFDIO_COPY) | (1 << _UFFDIO_POISON); + + if (uffd_register_with_ioctls(uffd, addr, len, true, + false, false, &ioctls)) + err("poison register fail"); + + if ((ioctls & expected) != expected) + err("registered area doesn't support COPY and POISON ioctls"); +} + +static void do_uffdio_poison(int uffd, unsigned long offset) +{ + struct uffdio_poison uffdio_poison = { 0 }; + int ret; + __s64 res; + + uffdio_poison.range.start = (unsigned long) area_dst + offset; + uffdio_poison.range.len = page_size; + uffdio_poison.mode = 0; + ret = ioctl(uffd, UFFDIO_POISON, &uffdio_poison); + res = uffdio_poison.updated; + + if (ret) + err("UFFDIO_POISON error: %"PRId64, (int64_t)res); + else if (res != page_size) + err("UFFDIO_POISON unexpected size: %"PRId64, (int64_t)res); +} + +static void uffd_poison_handle_fault( + struct uffd_msg *msg, struct uffd_args *args) +{ + unsigned long offset; + + if (msg->event != UFFD_EVENT_PAGEFAULT) + err("unexpected msg event %u", msg->event); + + if (msg->arg.pagefault.flags & + (UFFD_PAGEFAULT_FLAG_WP | UFFD_PAGEFAULT_FLAG_MINOR)) + err("unexpected fault type %llu", msg->arg.pagefault.flags); + + offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst; + offset &= ~(page_size-1); + + /* Odd pages -> copy zeroed page; even pages -> poison. */ + if (offset & page_size) + copy_page(uffd, offset, false); + else + do_uffdio_poison(uffd, offset); +} + +static void uffd_poison_test(uffd_test_args_t *targs) +{ + pthread_t uffd_mon; + char c; + struct uffd_args args = { 0 }; + struct sigaction act = { 0 }; + unsigned long nr_sigbus = 0; + unsigned long nr; + + fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK); + + uffd_register_poison(uffd, area_dst, nr_pages * page_size); + memset(area_src, 0, nr_pages * page_size); + + args.handle_fault = uffd_poison_handle_fault; + if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args)) + err("uffd_poll_thread create"); + + sigbuf = &jbuf; + act.sa_sigaction = sighndl; + act.sa_flags = SA_SIGINFO; + if (sigaction(SIGBUS, &act, 0)) + err("sigaction"); + + for (nr = 0; nr < nr_pages; ++nr) { + unsigned long offset = nr * page_size; + const char *bytes = (const char *) area_dst + offset; + const char *i; + + if (sigsetjmp(*sigbuf, 1)) { + /* + * Access below triggered a SIGBUS, which was caught by + * sighndl, which then jumped here. Count this SIGBUS, + * and move on to next page. + */ + ++nr_sigbus; + continue; + } + + for (i = bytes; i < bytes + page_size; ++i) { + if (*i) + err("nonzero byte in area_dst (%p) at %p: %u", + area_dst, i, *i); + } + } + + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); + if (pthread_join(uffd_mon, NULL)) + err("pthread_join()"); + + if (nr_sigbus != nr_pages / 2) + err("expected to receive %lu SIGBUS, actually received %lu", + nr_pages / 2, nr_sigbus); + + uffd_test_pass(); +} + /* * Test the returned uffdio_register.ioctls with different register modes. * Note that _UFFDIO_ZEROPAGE is tested separately in the zeropage test. @@ -1126,6 +1237,12 @@ uffd_test_case_t uffd_tests[] = { UFFD_FEATURE_PAGEFAULT_FLAG_WP | UFFD_FEATURE_WP_HUGETLBFS_SHMEM, }, + { + .name = "poison", + .uffd_fn = uffd_poison_test, + .mem_targets = MEM_ALL, + .uffd_feature_required = UFFD_FEATURE_POISON, + }, }; static void usage(const char *prog)