From patchwork Tue Mar 23 00:48:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12156477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AAD2C433C1 for ; Tue, 23 Mar 2021 00:49:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 99D6C6196C for ; Tue, 23 Mar 2021 00:49:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99D6C6196C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B98F46B00FD; Mon, 22 Mar 2021 20:49:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B48A96B0100; Mon, 22 Mar 2021 20:49:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 923C56B0103; Mon, 22 Mar 2021 20:49:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 659A46B00FD for ; Mon, 22 Mar 2021 20:49:21 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 27B2E1E0A for ; Tue, 23 Mar 2021 00:49:21 +0000 (UTC) X-FDA: 77949305322.29.6F2438E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf02.hostedemail.com (Postfix) with ESMTP id A2F6E407F8F3 for ; Tue, 23 Mar 2021 00:49:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616460559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+P1dWJGjbRulk7s1nJfmaHQsVvt1Wj4iX3MZXYNalzI=; b=JHphZv9WvNAXGi9DnPc+gQvIYuZzfgs2Xl0YIm7ep/c93VC7kIdaOWDCIAGigYzXfCmZO4 1pjWY+9dELuz3alS1ko+5vRSPjVjXk4FbdLEgDs2vVt/bS4O5ZOoUEC71TpqshuNd+2qIH Wn+Vym49QoWod51oa7Qa9pcj+RElE/A= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-XCkuk6bnPVy1i7AtYLZCKw-1; Mon, 22 Mar 2021 20:49:18 -0400 X-MC-Unique: XCkuk6bnPVy1i7AtYLZCKw-1 Received: by mail-qv1-f70.google.com with SMTP id b15so550854qvz.15 for ; Mon, 22 Mar 2021 17:49:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+P1dWJGjbRulk7s1nJfmaHQsVvt1Wj4iX3MZXYNalzI=; b=Fl5XSE0mwJt1kckXy0PpDhB87lU6Hv2EInHPh2YCbfWoggNK0M5Oaf8g3YHfuRiqcM ssQ0H1GSvcxNRhchi3etoJGvRhxw1C8yCFliMkV+H1o7BM9clFjJvmFagoehB01/6ltu FbF880HBlr4tdQjiUgbvzEhP0PxfYkUZ+BfhLEAVPxn+0x4SLUhWyE9Jr/5IDmDfGI0U KXBumNJ0eNnYC3S1UVBR8KojaUbzv9HbKn+mTyDPgauonZtCjTDRa5mfV+gnQiKjQqUZ sMiZGEYJWktC6F5W4lO0q8u9QQYKi4++/6MgHUhw5eI6xrebFpAvOfA2rujigvJCjbk9 orTw== X-Gm-Message-State: AOAM531lczo/79FiSMnTKqPdHQOO3bNZ7cb5677V5Jna8FIUwmZcjOrF eHbIS6g/UR+NW64gDzdRWoIsRW/NdIxvC4kfj4sydJUYKo5Jz3TJc+2uLfz6o67+ZcYQVKdzDuS 7FUH7vbEUNz4= X-Received: by 2002:a05:622a:1192:: with SMTP id m18mr2273147qtk.27.1616460557075; Mon, 22 Mar 2021 17:49:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqjFEqUS8mI2bN3hk02sEVq6e4Mt4cud7LDIj6h6LCxZV+FMUuvXeBvvd82VpC1CtDlxHBfA== X-Received: by 2002:a05:622a:1192:: with SMTP id m18mr2273126qtk.27.1616460556811; Mon, 22 Mar 2021 17:49:16 -0700 (PDT) Received: from localhost.localdomain (bras-base-toroon474qw-grc-82-174-91-135-175.dsl.bell.ca. [174.91.135.175]) by smtp.gmail.com with ESMTPSA id n6sm5031793qtx.22.2021.03.22.17.49.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Mar 2021 17:49:16 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Kirill A . Shutemov" , Jerome Glisse , Mike Kravetz , Matthew Wilcox , Andrew Morton , Axel Rasmussen , Hugh Dickins , peterx@redhat.com, Nadav Amit , Andrea Arcangeli , Mike Rapoport Subject: [PATCH 01/23] shmem/userfaultfd: Take care of UFFDIO_COPY_MODE_WP Date: Mon, 22 Mar 2021 20:48:50 -0400 Message-Id: <20210323004912.35132-2-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210323004912.35132-1-peterx@redhat.com> References: <20210323004912.35132-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Stat-Signature: 73jnsf8xyajagezsr6bqbc74df665841 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: A2F6E407F8F3 Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616460559-213897 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firstly, pass wp_copy into shmem_mfill_atomic_pte() through the stack. Then apply the UFFD_WP bit properly when the UFFDIO_COPY on shmem is with UFFDIO_COPY_MODE_WP. One thing to mention is that shmem_mfill_atomic_pte() needs to set the dirty bit in pte even if UFFDIO_COPY_MODE_WP is set. The reason is similar to dcf7fe9d8976 ("userfaultfd: shmem: UFFDIO_COPY: set the page dirty if VM_WRITE is not set") where we need to set page as dirty even if VM_WRITE is no there. It's just that shmem can drop the pte any time later, and if it's not dirty the data will be dropped. For uffd-wp, that could lead to data loss if without the dirty bit set. Note that shmem_mfill_zeropage_pte() will always call shmem_mfill_atomic_pte() with wp_copy==false because UFFDIO_ZEROCOPY does not support UFFDIO_COPY_MODE_WP. Signed-off-by: Peter Xu --- include/linux/shmem_fs.h | 5 +++-- mm/shmem.c | 18 ++++++++++++++---- mm/userfaultfd.c | 2 +- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index f0919c3722e7..dfd0369657d8 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -128,10 +128,11 @@ extern void shmem_uncharge(struct inode *inode, long pages); int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - enum mcopy_atomic_mode mode, struct page **pagep); + enum mcopy_atomic_mode mode, struct page **pagep, + bool wp_copy); #else /* !CONFIG_SHMEM */ #define shmem_mcopy_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, \ - src_addr, mode, pagep) ({ BUG(); 0; }) + src_addr, mode, pagep, wp_copy) ({ BUG(); 0; }) #endif /* CONFIG_SHMEM */ #endif /* CONFIG_USERFAULTFD */ diff --git a/mm/shmem.c b/mm/shmem.c index 5cfd2fb6e52b..e88aaabaeb27 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2364,7 +2364,8 @@ static struct inode *shmem_get_inode(struct super_block *sb, const struct inode int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, - enum mcopy_atomic_mode mode, struct page **pagep) + enum mcopy_atomic_mode mode, struct page **pagep, + bool wp_copy) { bool is_continue = (mode == MCOPY_ATOMIC_CONTINUE); struct inode *inode = file_inode(dst_vma->vm_file); @@ -2438,9 +2439,18 @@ int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, } _dst_pte = mk_pte(page, dst_vma->vm_page_prot); - if (dst_vma->vm_flags & VM_WRITE) - _dst_pte = pte_mkwrite(pte_mkdirty(_dst_pte)); - else { + if (dst_vma->vm_flags & VM_WRITE) { + if (wp_copy) + _dst_pte = pte_mkuffd_wp(pte_wrprotect(_dst_pte)); + else + _dst_pte = pte_mkwrite(_dst_pte); + /* + * Similar reason to set_page_dirty(), that we need to mark the + * pte dirty even if wp_copy==true here, otherwise the pte and + * its page could be dropped at anytime when e.g. swapped out. + */ + _dst_pte = pte_mkdirty(_dst_pte); + } else { /* * We don't set the pte dirty if the vma has no * VM_WRITE permission, so mark the page dirty or it diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index cbb7c8d79a4d..0963e0d9ed20 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -448,7 +448,7 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, } else { VM_WARN_ON_ONCE(wp_copy); err = shmem_mcopy_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - src_addr, mode, page); + src_addr, mode, page, wp_copy); } return err;