From patchwork Mon Feb 27 22:29:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rick Edgecombe X-Patchwork-Id: 13154264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41E2CC7EE2E for ; Mon, 27 Feb 2023 22:32:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 930806B00A5; Mon, 27 Feb 2023 17:31:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8E00A6B00A6; Mon, 27 Feb 2023 17:31:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 732086B00A7; Mon, 27 Feb 2023 17:31:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 64E186B00A5 for ; Mon, 27 Feb 2023 17:31:59 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 291E3AAF96 for ; Mon, 27 Feb 2023 22:31:59 +0000 (UTC) X-FDA: 80514520758.26.0B141E4 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf28.hostedemail.com (Postfix) with ESMTP id 0715CC0010 for ; Mon, 27 Feb 2023 22:31:56 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=htfEXMEO; spf=pass (imf28.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677537117; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=2FtOU/X+ROob91Drvw/u5zl2nlpfMJbMpmIA9iRxoYU=; b=NRgZ3k2x5yFskWHh3BBbYGsOIgO0+/4J9x5O4/1WIP40yqK4gkx7A9o7II3AX0YSVAMxHH SeN1sKBAVOryGF3GcHWEaCaHym1YMjPJBBza8k16FDLnj7LSHWR4WRmyWtWFSfJykthr/d tasBtT+bkIJ36wfXF9dSLdR1eg365kI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=htfEXMEO; spf=pass (imf28.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677537117; a=rsa-sha256; cv=none; b=MswoNePfg6RJJPxcTNKLi8vmS6BpsNcvySv3YDreJIfqe4dLZilUv5JkoJtozy2lzcPPDL gKMeESYlBDh9zStkxeRC2ZTnh/t64oDBTcZUqHjQyZ6+f2zrmyTFXNZ2Mwwb3Kb4jzInAd SUUk4P4S56/K5sKjfeBHjzoa/IcLSu4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677537117; x=1709073117; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=0sN0qiJhpy5r3MZ28FxXbF8Pdb6RhgABRtgYTnCuxVM=; b=htfEXMEO+vgr93l0tjiPi9NCz8x5udGoDxEBnJvwstA8IpJQyRN9JA7s p0V1ODC4nztfKci4d/S9fI/9hXOWAuiS7KrIhE2iby1vFhnuK2zPLKkL4 izuyTD02fg7bSK8H2FJLjwrhwWd5M9V+DqK9l+ex3jzPhIVOPcoDbSayu /Vvx1+e0+fMguXDfZ/OlnMN9ijb9PCaU4u3sAoErqhL31U1ljr0NnA/ay uRwWCZxu/GpYoZ8zpTQz7Kv6WVsB/rfvysGtjJdxLMvSuKqFbXXW6N5Zy G0f0BmYUuNXW3V2zWeYbAlvj4XOExTBSvjaCoeyrQrHWoC6uHVs6f1BZo A==; X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="313657806" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="313657806" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:32 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="848024769" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="848024769" Received: from leonqu-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.72.19]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:32 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v7 33/41] x86/shstk: Introduce map_shadow_stack syscall Date: Mon, 27 Feb 2023 14:29:49 -0800 Message-Id: <20230227222957.24501-34-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230227222957.24501-1-rick.p.edgecombe@intel.com> References: <20230227222957.24501-1-rick.p.edgecombe@intel.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0715CC0010 X-Stat-Signature: shs6qwhcyibqtew3un1jzwb7qb1jkpk4 X-Rspam-User: X-HE-Tag: 1677537116-255029 X-HE-Meta: U2FsdGVkX183rKdEFHVJq1et7kVivvuBZ0X2p9ZA4XyG8UdPdQ8l1uvaTwWhHLBkpUp1bkGzFI5s1Bl+rQoawM651vCJgSYfUZiODaO1qtBUcY3pFS9C9DVRu5N8QHLc06zWOKFGmqc4JAH0wZKE+CXJmQB54vBad4XivGC+8Re+ivtQ3/s4yU3fWZv8TdMZCH8NWwiLoBQK5auXY9iTOCW9X7O8pvpxdf9koeIPRjxVeE0brRv9sP+DsoMhXW+oah9gdIIGNdGspAqALm8kAVQ5rXVtJ4W5yMENnp6LxJ8tY/NYm99pRQs6pmSgLg4WUtdPyAinqCNHdw6vNcwsncdyzDo7GLnxN/gAEWH53tzMX1TRtFHmBy4o4gEWeyImRxEbuJ5A8kLsn2pc/dYluv4AuJhac9lHDxnDDLYtWZAlfTaTO09ZK6mSoptv5UBuUHiqCpQnwU0xKXfja63fBbvbOG7SjiAvouWS72nnLa+AvAfbTbaDb0BrhW2s8I01DJ8ch1ZOybxIPiUbRSKopwx1KsRzDQSSZVNQB4PbKptMZQIBDbb0WvtlzfX+HawvY1oQ/49YUGk0fM/otx8o3N5LqTg3mGoIan2lQSuNiOnphPyGnC9/g1lkL24LPBTW5yp5x/G+GlzywupgUST4RQNaQuN5bvs5pLVaScXY+d8VnJyEf5ZxLCNdI3PFiRLEubZr29SpItVGqM9ApXcMEv66tADzuzbNJ7IKh++n9KocJLST3EurpJ0td7Ueo386VawJ0K99N0HIZMuHHs8Oirc2r0LtsxAL/sAl8Ty573CqP1g4EwYCmEk6o0SsJHvpOmEPrzcbJaBjGVARnDiYy8wux37ZM5ipjQ/XkUWqoLwe09iC3wwbnzfvN+Dj2bDvl2FUBimjzgHWo1ZS72UpQ8GjiIi37nUu5WWsmjeZgQBL8FmN0t/Y4VUasd2157+GDUECOdaK+scLjNJmMOE bn8Ss1vv mQrwBltYnkXOQDeqDLmKP2aY8AqVkpO94weWHpSFqZV50LMUZQEtEGPgO22aeHVxRsXsZce5HEjLBtQYZLbtJ0n7D5aawiG8endxnasRKhx7ilFtdlpWXTrjT0463bgLjjKobxsyyvcEJXBjjWwvQ6juyu7GjMGGf0BVYGgn26DsrNdK0bLLm9WdlPf4jWZd+w2BDMetV6cljONha1U8s236FXWnmJnKOXUjWCsv354leS3Cdq3b6VztqNQyb48zEskEafqZ3mVlCwFWla6f4f2NxU0/uvBXVVM8FiIc712U2lEo3wUOFH3GI2wZU+/PLrXSmvNksKQp9utl+8TKg9zMaGw7eH0cir15BLS1JtpXYiKWNTO8Ncte4fC1toWa80I3ihoKs8w4wz4wUxrBDASoKNQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When operating with shadow stacks enabled, the kernel will automatically allocate shadow stacks for new threads, however in some cases userspace will need additional shadow stacks. The main example of this is the ucontext family of functions, which require userspace allocating and pivoting to userspace managed stacks. Unlike most other user memory permissions, shadow stacks need to be provisioned with special data in order to be useful. They need to be setup with a restore token so that userspace can pivot to them via the RSTORSSP instruction. But, the security design of shadow stack's is that they should not be written to except in limited circumstances. This presents a problem for userspace, as to how userspace can provision this special data, without allowing for the shadow stack to be generally writable. Previously, a new PROT_SHADOW_STACK was attempted, which could be mprotect()ed from RW permissions after the data was provisioned. This was found to not be secure enough, as other thread's could write to the shadow stack during the writable window. The kernel can use a special instruction, WRUSS, to write directly to userspace shadow stacks. So the solution can be that memory can be mapped as shadow stack permissions from the beginning (never generally writable in userspace), and the kernel itself can write the restore token. First, a new madvise() flag was explored, which could operate on the PROT_SHADOW_STACK memory. This had a couple downsides: 1. Extra checks were needed in mprotect() to prevent writable memory from ever becoming PROT_SHADOW_STACK. 2. Extra checks/vma state were needed in the new madvise() to prevent restore tokens being written into the middle of pre-used shadow stacks. It is ideal to prevent restore tokens being added at arbitrary locations, so the check was to make sure the shadow stack had never been written to. 3. It stood out from the rest of the madvise flags, as more of direct action than a hint at future desired behavior. So rather than repurpose two existing syscalls (mmap, madvise) that don't quite fit, just implement a new map_shadow_stack syscall to allow userspace to map and setup new shadow stacks in one step. While ucontext is the primary motivator, userspace may have other unforeseen reasons to setup it's own shadow stacks using the WRSS instruction. Towards this provide a flag so that stacks can be optionally setup securely for the common case of ucontext without enabling WRSS. Or potentially have the kernel set up the shadow stack in some new way. The following example demonstrates how to create a new shadow stack with map_shadow_stack: void *shstk = map_shadow_stack(addr, stack_size, SHADOW_STACK_SET_TOKEN); Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook Acked-by: Mike Rapoport (IBM) Reviewed-by: Kees Cook Signed-off-by: Rick Edgecombe --- v5: - Fix addr/mapped_addr (Kees) - Switch to EOPNOTSUPP (Kees suggested ENOTSUPP, but checkpatch suggests this) - Return error for addresses below 4G v3: - Change syscall common -> 64 (Kees) - Use bit shift notation instead of 0x1 for uapi header (Kees) - Call do_mmap() with MAP_FIXED_NOREPLACE (Kees) - Block unsupported flags (Kees) - Require size >= 8 to set token (Kees) v2: - Change syscall to take address like mmap() for CRIU's usage v1: - New patch (replaces PROT_SHADOW_STACK). --- arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/x86/include/uapi/asm/mman.h | 3 ++ arch/x86/kernel/shstk.c | 59 ++++++++++++++++++++++---- include/linux/syscalls.h | 1 + include/uapi/asm-generic/unistd.h | 2 +- kernel/sys_ni.c | 1 + 6 files changed, 58 insertions(+), 9 deletions(-) diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl index c84d12608cd2..f65c671ce3b1 100644 --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -372,6 +372,7 @@ 448 common process_mrelease sys_process_mrelease 449 common futex_waitv sys_futex_waitv 450 common set_mempolicy_home_node sys_set_mempolicy_home_node +451 64 map_shadow_stack sys_map_shadow_stack # # Due to a historical design error, certain syscalls are numbered differently diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 5a0256e73f1e..8148bdddbd2c 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -13,6 +13,9 @@ ((key) & 0x8 ? VM_PKEY_BIT3 : 0)) #endif +/* Flags for map_shadow_stack(2) */ +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ + #include #endif /* _ASM_X86_MMAN_H */ diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 40f0a55762a9..0a3decab70ee 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -71,19 +72,31 @@ static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) return 0; } -static unsigned long alloc_shstk(unsigned long size) +static unsigned long alloc_shstk(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok) { int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; struct mm_struct *mm = current->mm; - unsigned long addr, unused; + unsigned long mapped_addr, unused; - mmap_write_lock(mm); - addr = do_mmap(NULL, 0, size, PROT_READ, flags, - VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + if (addr) + flags |= MAP_FIXED_NOREPLACE; + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); mmap_write_unlock(mm); - return addr; + if (!set_res_tok || IS_ERR_VALUE(mapped_addr)) + goto out; + + if (create_rstor_token(mapped_addr + token_offset, NULL)) { + vm_munmap(mapped_addr, size); + return -EINVAL; + } + +out: + return mapped_addr; } static unsigned long adjust_shstk_size(unsigned long size) @@ -134,7 +147,7 @@ static int shstk_setup(void) return -EOPNOTSUPP; size = adjust_shstk_size(0); - addr = alloc_shstk(size); + addr = alloc_shstk(0, size, 0, false); if (IS_ERR_VALUE(addr)) return PTR_ERR((void *)addr); @@ -178,7 +191,7 @@ int shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, return 0; size = adjust_shstk_size(stack_size); - addr = alloc_shstk(size); + addr = alloc_shstk(0, size, 0, false); if (IS_ERR_VALUE(addr)) return PTR_ERR((void *)addr); @@ -371,6 +384,36 @@ static int shstk_disable(void) return 0; } +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + bool set_tok = flags & SHADOW_STACK_SET_TOKEN; + unsigned long aligned_size; + + if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK)) + return -EOPNOTSUPP; + + if (flags & ~SHADOW_STACK_SET_TOKEN) + return -EINVAL; + + /* If there isn't space for a token */ + if (set_tok && size < 8) + return -EINVAL; + + if (addr && addr <= 0xFFFFFFFF) + return -EINVAL; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + aligned_size = PAGE_ALIGN(size); + if (aligned_size < size) + return -EOVERFLOW; + + return alloc_shstk(addr, aligned_size, size, set_tok); +} + long shstk_prctl(struct task_struct *task, int option, unsigned long features) { if (option == ARCH_SHSTK_LOCK) { diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index 33a0ee3bcb2e..392dc11e3556 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -1058,6 +1058,7 @@ asmlinkage long sys_memfd_secret(unsigned int flags); asmlinkage long sys_set_mempolicy_home_node(unsigned long start, unsigned long len, unsigned long home_node, unsigned long flags); +asmlinkage long sys_map_shadow_stack(unsigned long addr, unsigned long size, unsigned int flags); /* * Architecture-specific system calls diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index 45fa180cc56a..b12940ec5926 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -887,7 +887,7 @@ __SYSCALL(__NR_futex_waitv, sys_futex_waitv) __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node) #undef __NR_syscalls -#define __NR_syscalls 451 +#define __NR_syscalls 452 /* * 32 bit systems traditionally used different diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index 860b2dcf3ac4..cb9aebd34646 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -381,6 +381,7 @@ COND_SYSCALL(vm86old); COND_SYSCALL(modify_ldt); COND_SYSCALL(vm86); COND_SYSCALL(kexec_file_load); +COND_SYSCALL(map_shadow_stack); /* s390 */ COND_SYSCALL(s390_pci_mmio_read);