From patchwork Fri Jan 22 21:29:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 12040661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 710DAC433E6 for ; Fri, 22 Jan 2021 21:29:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1D50723AF8 for ; Fri, 22 Jan 2021 21:29:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D50723AF8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9FA606B000C; Fri, 22 Jan 2021 16:29:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8E3136B000D; Fri, 22 Jan 2021 16:29:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EB766B000E; Fri, 22 Jan 2021 16:29:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 4C25E6B000C for ; Fri, 22 Jan 2021 16:29:35 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 12FD65008 for ; Fri, 22 Jan 2021 21:29:35 +0000 (UTC) X-FDA: 77734702710.22.ship49_3012b692756f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id E660D1810DA53 for ; Fri, 22 Jan 2021 21:29:34 +0000 (UTC) X-HE-Tag: ship49_3012b692756f X-Filterd-Recvd-Size: 8742 Received: from mail-qk1-f201.google.com (mail-qk1-f201.google.com [209.85.222.201]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 21:29:34 +0000 (UTC) Received: by mail-qk1-f201.google.com with SMTP id i82so5056812qke.19 for ; Fri, 22 Jan 2021 13:29:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=7ooyFXBx1fIP/mZWl+q42UsYzUb6W7T7o6Qp5Bwm9hw=; b=XsWnNNT3bc6NKjNDXbJftwz4CzNpoXWdn06wSFhE9hrt040cUk7xfYmqbrN5GIVZWi i86OXfhFvdsu6DVlgJxXEVtz+etIe2fK6hDTiOU2ZNuVWwA6OVjCJOjxR8tzuhJS3kMb xZwUld/tGUOekzt3iwU/YmDZXAE2VD1ZGIdEXGjiLDvRYKyG82TgkmUDJCd+0S5ZqW4I 70Z21Wp0GtmlzoRgbtQ1fXtAA9OaXyTuH1/uVDuqRWkKJd6Ql1sD6VTsoQGCNiMog+Ep B8d7aXwnk2MzbdVwMN6cCOQf9v6PYZL7IZEHfyC48t2yNf6yIQ+9ABBB60mxOeVc55Bm f3sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7ooyFXBx1fIP/mZWl+q42UsYzUb6W7T7o6Qp5Bwm9hw=; b=d8Aog/uFtnBqXT15jq22IvjjOKEXkb5chAF37iNR2CaF4dwJhCCFOQFctMLE8fSwpR UDhIZ6Xf8Oimg7dPqy4ULs8pVNDmHrksk1bH+s6nOJOp39UnqaLSkD9NSKML5Xn5ye0Z 5NOk2VIn7tAcz4LjwTUpg2OT627ZvrVgBi92u0095kLYmEs4lQ8he7M1IUj2fVbSYjOG yEe3nl3UF6wJrRS1Bu5gQDbKACkgNXIVmNX/XJjtmSL7mT3nABiBUH0JybsFXS7jcNWj /cXf+FQ4gE2T1UI3ULcfT9nPmpjj9L4drywVP5SaQZ8LTH8a+p+qxD2M6D5Qqj24WDkj 9ltA== X-Gm-Message-State: AOAM530ftA/9h4QKTepaCAmbGLzSPppaj71tngpCtCBoMEbNS7UnKpd6 gfdaPHlxNnLh8VwK6ee04kuoj8BMNjUCJ79o2xVq X-Google-Smtp-Source: ABdhPJxb/hwoxHIo8QflV3z+18lopvVFKCpUYIsbtpY3zEzFE4r6BQtxe0nvXRYR/UfBzES127pXZs6ua8xttRg5W3aC X-Received: from ajr0.svl.corp.google.com ([2620:15c:2cd:203:f693:9fff:feef:c8f8]) (user=axelrasmussen job=sendgmr) by 2002:a0c:9d9a:: with SMTP id s26mr6613401qvd.9.1611350973499; Fri, 22 Jan 2021 13:29:33 -0800 (PST) Date: Fri, 22 Jan 2021 13:29:19 -0800 In-Reply-To: <20210122212926.3457593-1-axelrasmussen@google.com> Message-Id: <20210122212926.3457593-3-axelrasmussen@google.com> Mime-Version: 1.0 References: <20210122212926.3457593-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.30.0.280.ga3ce27912f-goog Subject: [PATCH v2 2/9] hugetlb/userfaultfd: Forbid huge pmd sharing when uffd enabled From: Axel Rasmussen To: Alexander Viro , Alexey Dobriyan , Andrea Arcangeli , Andrew Morton , Anshuman Khandual , Catalin Marinas , Chinwen Chang , Huang Ying , Ingo Molnar , Jann Horn , Jerome Glisse , Lokesh Gidra , "Matthew Wilcox (Oracle)" , Michael Ellerman , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Michel Lespinasse , Mike Kravetz , Mike Rapoport , Nicholas Piggin , Peter Xu , Shaohua Li , Shawn Anastasio , Steven Rostedt , Steven Price , Vlastimil Babka Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Adam Ruprecht , Axel Rasmussen , Cannon Matthews , "Dr . David Alan Gilbert" , David Rientjes , Oliver Upton X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Huge pmd sharing could bring problem to userfaultfd. The thing is that userfaultfd is running its logic based on the special bits on page table entries, however the huge pmd sharing could potentially share page table entries for different address ranges. That could cause issues on either: - When sharing huge pmd page tables for an uffd write protected range, the newly mapped huge pmd range will also be write protected unexpectedly, or, - When we try to write protect a range of huge pmd shared range, we'll first do huge_pmd_unshare() in hugetlb_change_protection(), however that also means the UFFDIO_WRITEPROTECT could be silently skipped for the shared region, which could lead to data loss. Since at it, a few other things are done altogether: - Move want_pmd_share() from mm/hugetlb.c into linux/hugetlb.h, because that's definitely something that arch code would like to use too - ARM64 currently directly check against CONFIG_ARCH_WANT_HUGE_PMD_SHARE when trying to share huge pmd. Switch to the want_pmd_share() helper. Signed-off-by: Peter Xu Signed-off-by: Axel Rasmussen --- arch/arm64/mm/hugetlbpage.c | 3 +-- include/linux/hugetlb.h | 15 +++++++++++++++ include/linux/userfaultfd_k.h | 9 +++++++++ mm/hugetlb.c | 5 ++--- 4 files changed, 27 insertions(+), 5 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5b32ec888698..1a8ce0facfe8 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,8 +284,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, */ ptep = pte_alloc_map(mm, pmdp, addr); } else if (sz == PMD_SIZE) { - if (IS_ENABLED(CONFIG_ARCH_WANT_HUGE_PMD_SHARE) && - pud_none(READ_ONCE(*pudp))) + if (want_pmd_share(vma) && pud_none(READ_ONCE(*pudp))) ptep = huge_pmd_share(mm, addr, pudp); else ptep = (pte_t *)pmd_alloc(mm, pudp, addr); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1e0abb609976..4508136c8376 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -11,6 +11,7 @@ #include #include #include +#include struct ctl_table; struct user_struct; @@ -947,4 +948,18 @@ static inline __init void hugetlb_cma_check(void) } #endif +static inline bool want_pmd_share(struct vm_area_struct *vma) +{ +#ifdef CONFIG_USERFAULTFD + if (uffd_disable_huge_pmd_share(vma)) + return false; +#endif + +#ifdef CONFIG_ARCH_WANT_HUGE_PMD_SHARE + return true; +#else + return false; +#endif +} + #endif /* _LINUX_HUGETLB_H */ diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index a8e5f3ea9bb2..c63ccdae3eab 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -52,6 +52,15 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma, return vma->vm_userfaultfd_ctx.ctx == vm_ctx.ctx; } +/* + * Never enable huge pmd sharing on uffd-wp registered vmas, because uffd-wp + * protect information is per pgtable entry. + */ +static inline bool uffd_disable_huge_pmd_share(struct vm_area_struct *vma) +{ + return vma->vm_flags & VM_UFFD_WP; +} + static inline bool userfaultfd_missing(struct vm_area_struct *vma) { return vma->vm_flags & VM_UFFD_MISSING; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 07b23c81b1db..d46f50a99ff1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5371,7 +5371,7 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, *addr = ALIGN(*addr, HPAGE_SIZE * PTRS_PER_PTE) - HPAGE_SIZE; return 1; } -#define want_pmd_share() (1) + #else /* !CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ pte_t *huge_pmd_share(struct mm_struct *mm, unsigned long addr, pud_t *pud) { @@ -5388,7 +5388,6 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, unsigned long *start, unsigned long *end) { } -#define want_pmd_share() (0) #endif /* CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB @@ -5410,7 +5409,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, pte = (pte_t *)pud; } else { BUG_ON(sz != PMD_SIZE); - if (want_pmd_share() && pud_none(*pud)) + if (want_pmd_share(vma) && pud_none(*pud)) pte = huge_pmd_share(mm, addr, pud); else pte = (pte_t *)pmd_alloc(mm, pud, addr);