From patchwork Mon Nov 15 07:55:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12618849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57EADC433EF for ; Mon, 15 Nov 2021 07:56:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07BF363211 for ; Mon, 15 Nov 2021 07:56:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 07BF363211 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9736A6B0081; Mon, 15 Nov 2021 02:56:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FCA26B0082; Mon, 15 Nov 2021 02:56:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7777D6B0083; Mon, 15 Nov 2021 02:56:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 620256B0081 for ; Mon, 15 Nov 2021 02:56:12 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 234528249980 for ; Mon, 15 Nov 2021 07:56:12 +0000 (UTC) X-FDA: 78810406584.25.52A1BDC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 49B5510512DC for ; Mon, 15 Nov 2021 07:55:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1636962971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PmskQuQ5v+E/C6hWNn1p+tojqPkwHTmjUCaKkLoPmAQ=; b=Q9mocqAOCVZoQMRmqH3ztXxUGjz/qwKax4zWu4wdtyWK5ooracSzPSmCgSdYrU0Ze1icWn 9CjBL0MKhbAHe8jV5P1vwoN8aFwgtbLrpsrQuyqS2ALJLQvtvDoyPHUHIn9IuyQaVgaciz ls0FrXgSVsIodGevWaMj5q2ItcWMszI= Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-565-llEQGDM1OL2THjH8SmAQYQ-1; Mon, 15 Nov 2021 02:56:10 -0500 X-MC-Unique: llEQGDM1OL2THjH8SmAQYQ-1 Received: by mail-pl1-f197.google.com with SMTP id n13-20020a170902d2cd00b0014228ffc40dso5831537plc.4 for ; Sun, 14 Nov 2021 23:56:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PmskQuQ5v+E/C6hWNn1p+tojqPkwHTmjUCaKkLoPmAQ=; b=gzQ0yir4cdKdLhiIaith/eptUW0eSi/5IUo57TEfxbDwXug4pJKFn0npzsjS1qO1nO QUvobop9db3J8AcqtZinb5JFUvtVQN3KiWMKKxG9kSvLTmI7Z2nO94WnsLDacr44nWDo og3/XnfjwR8f1nVohPlNWgzUDhOvMMGSQO4QUnDycUlfo1F70fogJhptNSLDLlo0jA3p EZKQyAM05JL0+oSd+lTartPbu8XaSC19CV/3YzOFOTVxL1RIz/UmHpEQPHNuvbqSNRnH fT8m8vSDA066ANR17FPFMpLEmHe52JmDNTJ3UqVQSsyqa/evzoDyavSDyxcRoOacxLwG Wfjg== X-Gm-Message-State: AOAM531lSFXmjN8AInooKkLzaDwROIBMGp5kyXLhAsFQfkv0gzcT/f/V FxEoxDlgGLzBPmR2oBDDnov8UU+gw2b9SgfbZXIdCSFQPDVawJKLUlCbYE/E65DJiudz1DHu6kO ycsBA8G/1Ev/V2SoNph+7kGqojc6QNBFTJXt+yuVqsfMvVH0C992dESMX2ctN X-Received: by 2002:a05:6a00:1901:b0:44b:e041:f07f with SMTP id y1-20020a056a00190100b0044be041f07fmr31906460pfi.52.1636962968805; Sun, 14 Nov 2021 23:56:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJwO6oDN0fGNLF/myzmtv7RWKcD4B2LB1pQ6VJVj/upm3PYad0YwxcCTvIC/G3q3fUkWS4EYpg== X-Received: by 2002:a05:6a00:1901:b0:44b:e041:f07f with SMTP id y1-20020a056a00190100b0044be041f07fmr31906409pfi.52.1636962968263; Sun, 14 Nov 2021 23:56:08 -0800 (PST) Received: from localhost.localdomain ([191.101.132.223]) by smtp.gmail.com with ESMTPSA id e10sm15792796pfv.140.2021.11.14.23.56.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Nov 2021 23:56:07 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Axel Rasmussen , Nadav Amit , Mike Rapoport , Hugh Dickins , Mike Kravetz , "Kirill A . Shutemov" , Alistair Popple , Jerome Glisse , Matthew Wilcox , Andrew Morton , peterx@redhat.com, David Hildenbrand , Andrea Arcangeli Subject: [PATCH v6 04/23] mm/uffd: PTE_MARKER_UFFD_WP Date: Mon, 15 Nov 2021 15:55:03 +0800 Message-Id: <20211115075522.73795-5-peterx@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211115075522.73795-1-peterx@redhat.com> References: <20211115075522.73795-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Q9mocqAO; spf=none (imf13.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 49B5510512DC X-Stat-Signature: 8k3rwdgzwx1qfptgw3muhw7noh77i86o X-HE-Tag: 1636962959-845108 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch introduces the 1st user of pte marker: the uffd-wp marker. When the pte marker is installed with the uffd-wp bit set, it means this pte was wr-protected by uffd. We will use this special pte to arm the ptes that got either unmapped or swapped out for a file-backed region that was previously wr-protected. This special pte could trigger a page fault just like swap entries. This idea is greatly inspired by Hugh and Andrea in the discussion, which is referenced in the links below. Some helpers are introduced to detect whether a swap pte is uffd wr-protected. After the pte marker introduced, one swap pte can be wr-protected in two forms: either it is a normal swap pte and it has _PAGE_SWP_UFFD_WP set, or it's a pte marker that has PTE_MARKER_UFFD_WP set. Link: https://lore.kernel.org/lkml/20201126222359.8120-1-peterx@redhat.com/ Link: https://lore.kernel.org/lkml/20201130230603.46187-1-peterx@redhat.com/ Suggested-by: Andrea Arcangeli Suggested-by: Hugh Dickins Signed-off-by: Peter Xu --- include/linux/swapops.h | 3 ++- include/linux/userfaultfd_k.h | 38 +++++++++++++++++++++++++++++++++++ mm/Kconfig | 9 +++++++++ 3 files changed, 49 insertions(+), 1 deletion(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 5103d2a4ae38..2cec3ef355a7 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -249,7 +249,8 @@ static inline int is_writable_migration_entry(swp_entry_t entry) typedef unsigned long pte_marker; -#define PTE_MARKER_MASK (0) +#define PTE_MARKER_UFFD_WP BIT(0) +#define PTE_MARKER_MASK (PTE_MARKER_UFFD_WP) #ifdef CONFIG_PTE_MARKER diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 33cea484d1ad..7d7ffec53ddb 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -15,6 +15,8 @@ #include #include +#include +#include #include /* The set of all possible UFFD-related VM flags. */ @@ -236,4 +238,40 @@ static inline void userfaultfd_unmap_complete(struct mm_struct *mm, #endif /* CONFIG_USERFAULTFD */ +static inline bool is_pte_marker_uffd_wp(pte_t pte) +{ +#ifdef CONFIG_PTE_MARKER_UFFD_WP + swp_entry_t entry; + + if (!is_swap_pte(pte)) + return false; + + entry = pte_to_swp_entry(pte); + + return is_pte_marker_entry(entry) && + (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); +#else + return false; +#endif +} + +/* + * Returns true if this is a swap pte and was uffd-wp wr-protected in either + * forms (pte marker or a normal swap pte), false otherwise. + */ +static inline bool pte_swp_uffd_wp_any(pte_t pte) +{ +#ifdef CONFIG_PTE_MARKER_UFFD_WP + if (!is_swap_pte(pte)) + return false; + + if (pte_swp_uffd_wp(pte)) + return true; + + if (is_pte_marker_uffd_wp(pte)) + return true; +#endif + return false; +} + #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 66f23c6c2032..f01c8e0afadf 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -904,6 +904,15 @@ config PTE_MARKER help Allows to create marker PTEs for file-backed memory. +config PTE_MARKER_UFFD_WP + bool "Marker PTEs support for userfaultfd write protection" + depends on PTE_MARKER && HAVE_ARCH_USERFAULTFD_WP + + help + Allows to create marker PTEs for userfaultfd write protection + purposes. It is required to enable userfaultfd write protection on + file-backed memory types like shmem and hugetlbfs. + source "mm/damon/Kconfig" endmenu