From patchwork Thu Sep 29 22:29:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 12994670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C69AC4167E for ; Thu, 29 Sep 2022 22:30:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 821DA8D000A; Thu, 29 Sep 2022 18:30:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77BCC8D000B; Thu, 29 Sep 2022 18:30:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61BDF8D000A; Thu, 29 Sep 2022 18:30:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 531C58D0001 for ; Thu, 29 Sep 2022 18:30:26 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2FE88121018 for ; Thu, 29 Sep 2022 22:30:26 +0000 (UTC) X-FDA: 79966568052.16.F700DC0 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by imf26.hostedemail.com (Postfix) with ESMTP id AADA414000B for ; Thu, 29 Sep 2022 22:30:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1664490625; x=1696026625; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=kQODslNH1nloZ+Qn0eQNHtmSgsYUq5sI4TpgOnIwj+g=; b=j61QdmZltfciumsEE5IbzAW5NQklml+Iyg/15nW2sKyz08dA8jfE/wBf kUOz+H+17yuDvuYRljAhYUhUSUeXJieSFa1oCStNnugkOGgbgTv2Ewg6O D1CwJweYGcRi9csVAqbUMVE8YgOKY5rdhBrs2wKP0eldN5EdYxc/squ0T PFhaIyu9VuaJqqSFQ7LuhyYiihLEce5d6z366Wrh8mOAkrDcKBayFMzWC WaVt0QmWKnxuKZw0lk+O7rfQ4fZRS9lnv10rMKiEvGMdaj7YtIgiif6nz AznVBTkKGtDSt3Urs2ALFbbULYsCFSRhC3eFxoEMtiYuLhArMJI3U3VMF w==; X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="328420480" X-IronPort-AV: E=Sophos;i="5.93,356,1654585200"; d="scan'208";a="328420480" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 15:30:23 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10485"; a="691016227" X-IronPort-AV: E=Sophos;i="5.93,356,1654585200"; d="scan'208";a="691016227" Received: from sergungo-mobl.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.251.25.88]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2022 15:30:18 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V . Shankar" , Weijiang Yang , "Kirill A . Shutemov" , joao.moreira@intel.com, John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com Cc: rick.p.edgecombe@intel.com, Yu-cheng Yu Subject: [PATCH v2 17/39] mm: Fixup places that call pte_mkwrite() directly Date: Thu, 29 Sep 2022 15:29:14 -0700 Message-Id: <20220929222936.14584-18-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220929222936.14584-1-rick.p.edgecombe@intel.com> References: <20220929222936.14584-1-rick.p.edgecombe@intel.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664490625; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=T5B82Q/4J9Kk/ESzzD5k/poEJjVrxDvqOShCeWflLFA=; b=HAYwMG0DtX36M9IMa9krInkwTfBMFHxyDwmoW/TfGunnRc0PA3LZVlpM7h7BIi8r0DaNf4 0nKNkSXaYj9Z2zsY4o7QDz9YZbI6Y+ci8SjeBYGpweD9IG1VnBOw05UtHi7FlwG2DAAPro OyvuZryX7CDpUD+W2kbCniC30lXHnKs= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=j61QdmZl; spf=pass (imf26.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664490625; a=rsa-sha256; cv=none; b=z94OwRcECk5kGx4VIPo89a41OR+/mvCMS0BTl7y6jD5mkGH7N7qm7Yv02eBZoYjAhxJQpE M+MXjyyeHE4XKNUiIyWfEbHe/OcB+ZEsn+3ZsrgEWc/CD/oMhGIVzHJquMo0xNO5QK45aK WGWBCBIZy1E5UVvH6fDUACfx5A9SvFE= Authentication-Results: imf26.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=j61QdmZl; spf=pass (imf26.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Stat-Signature: twih8xxdquczcbcsrdf6k9haj9845awo X-Rspamd-Queue-Id: AADA414000B X-Rspamd-Server: rspam08 X-HE-Tag: 1664490625-39064 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu-cheng Yu With the introduction of shadow stack memory there are two ways a pte can be writable: regular writable memory and shadow stack memory. In past patches, maybe_mkwrite() has been updated to apply pte_mkwrite() or pte_mkwrite_shstk() depending on the VMA flag. This covers most cases where a PTE is made writable. However, there are places where pte_mkwrite() is called directly and the logic should now also create a shadow stack PTE in the case of a shadow stack VMA. - do_anonymous_page() and migrate_vma_insert_page() check VM_WRITE directly and call pte_mkwrite(), which is the same as maybe_mkwrite() in logic and intention. Just change them to maybe_mkwrite(). - When userfaultfd is creating a PTE after userspace handles the fault it calls pte_mkwrite() directly. Teach it about pte_mkwrite_shstk() In other cases where pte_mkwrite() is called directly, the VMA will not be VM_SHADOW_STACK, and so shadow stack memory should not be created. - In the case of pte_savedwrite(), shadow stack VMA's are excluded. - In the case of the "dirty_accountable" optimization in mprotect(), shadow stack VMA's won't be VM_SHARED, so it is not nessary. Signed-off-by: Yu-cheng Yu Co-developed-by: Rick Edgecombe Signed-off-by: Rick Edgecombe Cc: Kees Cook Reviewed-by: Kees Cook --- v2: - Updated commit log with comment's from Dave Hansen - Dave also suggested (I understood) to maybe tweak vm_get_page_prot() to avoid having to call maybe_mkwrite(). After playing around with this I opted to *not* do this. Shadow stack memory memory is effectively writable, so having the default permissions be writable ended up mapping the zero page as writable and other surprises. So creating shadow stack memory needs to be done with manual logic like pte_mkwrite(). - Drop change in change_pte_range() because it couldn't actually trigger for shadow stack VMAs. - Clarify reasoning for skipped cases of pte_mkwrite(). Yu-cheng v25: - Apply same changes to do_huge_pmd_numa_page() as to do_numa_page(). mm/migrate_device.c | 3 +-- mm/userfaultfd.c | 10 +++++++--- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 27fb37d65476..eba3164736b3 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -606,8 +606,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate, goto abort; } entry = mk_pte(page, vma->vm_page_prot); - if (vma->vm_flags & VM_WRITE) - entry = pte_mkwrite(pte_mkdirty(entry)); + entry = maybe_mkwrite(pte_mkdirty(entry), vma); } ptep = pte_offset_map_lock(mm, pmdp, addr, &ptl); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 7327b2573f7c..b49372c7de41 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -63,6 +63,7 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, int ret; pte_t _dst_pte, *dst_pte; bool writable = dst_vma->vm_flags & VM_WRITE; + bool shstk = dst_vma->vm_flags & VM_SHADOW_STACK; bool vm_shared = dst_vma->vm_flags & VM_SHARED; bool page_in_cache = page->mapping; spinlock_t *ptl; @@ -83,9 +84,12 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, writable = false; } - if (writable) - _dst_pte = pte_mkwrite(_dst_pte); - else + if (writable) { + if (shstk) + _dst_pte = pte_mkwrite_shstk(_dst_pte); + else + _dst_pte = pte_mkwrite(_dst_pte); + } else /* * We need this to make sure write bit removed; as mk_pte() * could return a pte with write bit set.