From patchwork Sun Mar 19 00:15:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13180152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B23B6C76196 for ; Sun, 19 Mar 2023 00:16:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1C1C280016; Sat, 18 Mar 2023 20:16:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E7D8B280001; Sat, 18 Mar 2023 20:16:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5943280016; Sat, 18 Mar 2023 20:16:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A66DB280001 for ; Sat, 18 Mar 2023 20:16:37 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8AB581C6408 for ; Sun, 19 Mar 2023 00:16:37 +0000 (UTC) X-FDA: 80583731634.11.9A1654D Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf11.hostedemail.com (Postfix) with ESMTP id 8F6AC4000E for ; Sun, 19 Mar 2023 00:16:35 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hkc5btwE; spf=pass (imf11.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679184995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=SYx+P7Ic7HNiZnYw3nI9Ocezil5osIwuNxYJyZqEM2o=; b=A6rL4NasthllxKUXZf21V1lbmfjDeLCyW87lIhMxPnrBgpXrooR2x7IXtdhpF9AgLVZmtA dS/vfMxM9pb1PhLZFjOc2X240L9vpSwUB5PUiYrEd/NpCPs6RDj0BLh+sqURu4EDFZ3cKq s7AgzoL+Rw/cbK/4IIW2EMDPgvBLYM4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=hkc5btwE; spf=pass (imf11.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679184995; a=rsa-sha256; cv=none; b=aHqq52ddZuRgoFJ5paTNXIUlfJjsAxcX+bYFxOIALCovLRrI+CHgH0QFRWP35qIU3C1t/m kyUb5kxLeBVyPMRn+AfG1Qer1RX9AYNbVzCzeDEzVkTUwb3rYbzjJCbITI1r06agnsTa6I w897ZrRQC06/NIviLUeD7QjL7VytX2Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679184995; x=1710720995; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=opyjBN/VDdKGQFn13MiWP9WuORkQYDUhpnOpss/5QKc=; b=hkc5btwESL6le/ndhfzVkaWzkQYQL9Fh/wJlbOwCcfg4zS/YnfvuBW1j nYvoBMjJIukhhW6aWepzH5xx/0IfLeFCGur6/Dv3KLtzIOhLC8fbre3hU VntbOMrMLBFHi3C/aIjS91PHODh7PNeZYvzAXg4bTLbV2jmXVWrcK2yjK AGgSM/nz6ZBeW/iAavTN5wtcAnOpcKFwogL3lNNoMv8gcyT163/mYUL6A A3myPsabo1BFyTKf2RhdKE+17n2i/ReHWIY4/5vEN032exKjDO4NagTht KF6D9GLE2nRZFiXhuYGG/O8os4G4ZcFz+lrp1yljXuWQluj4rE6VmP0Zf A==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="338491268" X-IronPort-AV: E=Sophos;i="5.98,272,1673942400"; d="scan'208";a="338491268" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2023 17:16:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="749672879" X-IronPort-AV: E=Sophos;i="5.98,272,1673942400"; d="scan'208";a="749672879" Received: from bmahatwo-mobl1.gar.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.135.34.5]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2023 17:16:33 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v8 24/40] mm: Don't allow write GUPs to shadow stack memory Date: Sat, 18 Mar 2023 17:15:19 -0700 Message-Id: <20230319001535.23210-25-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230319001535.23210-1-rick.p.edgecombe@intel.com> References: <20230319001535.23210-1-rick.p.edgecombe@intel.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8F6AC4000E X-Rspam-User: X-Stat-Signature: icqshrren76px8tniyaq3gd3rjcfzcth X-HE-Tag: 1679184995-313021 X-HE-Meta: U2FsdGVkX1/MKDfSqKuiEEo5CX2nli5xq2meNbHIXfzZM02ppbFKgIxbI0ygA5JTjtHksE2aZXyWcx00UmkVAGnpvgTsqAlteI/VnGZ/IMqsudroYPWfEzcAnbvYEtpm9GowUmaUiKbbpmX8oPV04k4drIKIaccForpMYIwxufuMpPIa09UY8Z8zBJIAAe+I4xfr1tLAU8S01Nz+pR51ZOnVL9lbK7t9ZcyaTTO/VNYmNoFjcoVgYYQRcOt732hSvXZB1VPYdQsjaptKkbyMSGQJylx3Qf1PzIFdlS2+56SrWCp/JLr6aQo9Fhm4olF56hulWCd3pEXMNWOxaj4zHQb/YwBWwmYNFjZGqK2HUsVvkx1Qou4L+Ms5VRXxDOiUOugoczwQuWDiiAhaSfrTG8alRbZn0hQREt0EP4itBh0qIpPff+huFXztnu9/mRgpf2D0M/AzfFQ8fqXP4kO34WOz12/gX0gOZmxoMYltkeqhyzGu8mAa6y2BdDExMZgFn00wWYYEx592NEWT6WaW+q4a2z2dZLFagVEPAQHkHl2mLGzi6sNqVV5W5xI/cmgseRYzm/FMPpNhAIP5HHdApK8rcYkDukFWMA3ZpQ64Kh8flE1WBVTDeyN2/gTSPo2ATnyV3xRkU6TLJaBUnVmfZxVmN7E1iBWhrZ2JwzMaDSUGOhGadQsf0GrJLOgH5jdmvYCoJjXUIdfjHzEM6Wll+KAXQv2hZUo12lg6oMK8g6Symy3Ru6Rh45TXfDtD91/xk8/MQd0tPQnrITemxM3fDQG3KH7eu5jOJSnAGOmx8EfBIQBSMDiCAkM3yoIDd/JJhXcKjdTA06WPR2XVI11i7Zm0pYMkYp/ucd+sKGEgCSnhvkppk/vyfPHYs5ybOkMZuNSNJrUhitlwxt+R1SpsbIlVTskXsNxJzERKacDhlLQu6CVPH3LwBUnQxObfD0orIqktwxAZF6RUdWu8ef7 eK9gk9ar pThJBCxlff8skMl/zX7+1RpDTCI9xE88YqU1WVgw8LAis+2REvSQ9OasySNsGJtkS/9iPwqfFuK9vA7U70KqHYqUAvcnlGLmtmGZNYhXXFjz/w2/NK/7bj0i83mMsOkFy4y/9QmH6CrPC/baG9ALbkDBbKVnj+0rs4K2CrKAHAiJhQsjyAe4xe5kQg/mNBGeknAZMUrDLBDRryuZwogw1/GqULI+RzOpxhfWaInTSkPxsZKemj7bDS03vzV1yThFaIxYZOSkRRCOukuCMUZxuFO/1UeaFogFx2h4U74OZqfOd8jpNYSIgeNGHj8Vpqq8/sX7TgoTiaF34enb+TjD8U41rcreOXSmH+0a8dv4FaUdZ4m3MlNK09q3Y1g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The x86 Control-flow Enforcement Technology (CET) feature includes a new type of memory called shadow stack. This shadow stack memory has some unusual properties, which requires some core mm changes to function properly. In userspace, shadow stack memory is writable only in very specific, controlled ways. However, since userspace can, even in the limited ways, modify shadow stack contents, the kernel treats it as writable memory. As a result, without additional work there would remain many ways for userspace to trigger the kernel to write arbitrary data to shadow stacks via get_user_pages(, FOLL_WRITE) based operations. To help userspace protect their shadow stacks, make this a little less exposed by blocking writable get_user_pages() operations for shadow stack VMAs. Still allow FOLL_FORCE to write through shadow stack protections, as it does for read-only protections. This is required for debugging use cases. Signed-off-by: Rick Edgecombe Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Acked-by: David Hildenbrand Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- v8: - Update commit log verbiage (Boris, AndyL) v3: - Add comment in __pte_access_permitted() (Dave) - Remove unneeded shadow stack specific check in __pte_access_permitted() (Jann) --- arch/x86/include/asm/pgtable.h | 5 +++++ mm/gup.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index d81e7ec27507..2e3d8cca1195 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1638,6 +1638,11 @@ static inline bool __pte_access_permitted(unsigned long pteval, bool write) { unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER; + /* + * Write=0,Dirty=1 PTEs are shadow stack, which the kernel + * shouldn't generally allow access to, but since they + * are already Write=0, the below logic covers both cases. + */ if (write) need_pte_bits |= _PAGE_RW; diff --git a/mm/gup.c b/mm/gup.c index eab18ba045db..e7c7bcc0e268 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -978,7 +978,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) return -EFAULT; if (write) { - if (!(vm_flags & VM_WRITE)) { + if (!(vm_flags & VM_WRITE) || (vm_flags & VM_SHADOW_STACK)) { if (!(gup_flags & FOLL_FORCE)) return -EFAULT; /* hugetlb does not support FOLL_FORCE|FOLL_WRITE. */