From patchwork Tue Jun 13 00:10:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13277745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9295EC7EE2E for ; Tue, 13 Jun 2023 00:13:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A2FD8E001F; Mon, 12 Jun 2023 20:12:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92C5D8E000B; Mon, 12 Jun 2023 20:12:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 694B38E001F; Mon, 12 Jun 2023 20:12:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 500218E000B for ; Mon, 12 Jun 2023 20:12:38 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 256FD14036E for ; Tue, 13 Jun 2023 00:12:38 +0000 (UTC) X-FDA: 80895798396.08.E6C15AB Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf19.hostedemail.com (Postfix) with ESMTP id 243791A0007 for ; Tue, 13 Jun 2023 00:12:35 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DGyIIn1+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686615156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CCO8Fv6gzg+48ugIh6HaSUWv+RYPVlqFMXpZfWmPIDc=; b=CrgljBdXVbVWkS7v0FQWbJRLyDIGtp886lC9xExgNn8p2jE1jXuFoqvqgcbfJubG8EB0Tu 9bmfxzcwANcjJxak3lokA0nhF9/mkir12xqpiYqgGBGmdZT8E/SBiVEzUOWCRpekQe4REc xnx+D1ryU4Ax9AfObRVamFG5IfkM85Y= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DGyIIn1+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf19.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686615156; a=rsa-sha256; cv=none; b=NxLZUbpqXFuxFEkYWwDq3rlhmB1RvM3Pl1pC9m8Ptyoke8ZTuwB8a7HZH2v+tFu3LxqN5S H0Ka9yx2AtDc9NQysJMvYZi7E9E81/vfA3n2V/XY1WjGbR/i1bFJtbanaroWZBFm/ckl3W 6kqdRwq3i67xEtrl703nqeLlmo56EFY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686615156; x=1718151156; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JRlODCr89ZW4tpE3v7Z5UBl8Adk88XgPugy9MazljtY=; b=DGyIIn1+KbQ5CzN2LRgX5+BUJjK9/Cc5Ea+rSij5mfNtqhQxM7YD7REa 8GD9V7NKI3mnrDfX83P3uLGjWU2DM8dXuAbZVge8FVvFq6T4uh8QX1VbB EzbWBNrWMeelhbRaew9ztwYU44gvXGtIC4MYf+Z4KYP7Xvc7hYK+IEVG1 nBHXSt1um9OMDb1pONxircQ6U17wSw/J1nYp3SmJTKtI5AGcnXNpli2XP /2pjQjCUtUyRPWFqyuoZGmhe6UxaJctxy70j5L41troxgD4ymlv4og+uk 6kBJyNRoo5fP7iQjghGQmdxEy7XGGqbG4sAF6lD/mr12cHQsxCdya/47K Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="361557375" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="361557375" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="835671105" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="835671105" Received: from almeisch-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.amr.corp.intel.com) ([10.209.42.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:33 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, broonie@kernel.org Cc: rick.p.edgecombe@intel.com, Yu-cheng Yu , Pengfei Xu Subject: [PATCH v9 30/42] x86/shstk: Introduce routines modifying shstk Date: Mon, 12 Jun 2023 17:10:56 -0700 Message-Id: <20230613001108.3040476-31-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> References: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 243791A0007 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: c5sfuzy4kojsnb38b6i1xz7jty8mafaw X-HE-Tag: 1686615155-973177 X-HE-Meta: U2FsdGVkX19khmEmqkDQPYXt4GSZcvP8AfL0PtM6W+qgMdl4e7ZSU1CFfG0t74OeXVxm5X8LQ+2Xsd+IQ3JO38mVZumAg948gmeSz2z/zbxunUMWG5uk3UDddenkNIb7/PwUdmhopeHL6oGqGw0qBm3OsBNl6SH/i6D4AjqecqfOHMkfrFrTSPBud7PT8fAJHAqxxTwcmFBFEpIDg3gSwOM94ob1hcQqLDnv0ucvgYpJuu8EHViZnBKXHl8MvEfsHcqW6WHU43+fO/9tdfzq3NFU5XcriBJZpoz8pv2V98TMqM+GK06cbtMQWzZ4KOKAShk71DbykSefoJ62ZlUA78jE4H0AfA5ZXbRIziiK6Rma/iqDzqU+M5NvgsQseaPJleqRto7d4hM16D64hI+ni7OFmmxCBSWvYMWiH/LloiAweaR+1PQ+G2zh2g1ERugtW6fjTjLLHiXGv7dBaOkWyI+6XxzU1CKBH5oFfa5srCiX3rvvFSpnFXAzor3bsgK+mrZ8M9ib/LWvKGf4smaly19P7r+TssqD0khax+0FJrBRAOWSF2z2MklOQnCBrWq5wTdK3NZ4UdFhj0DCfzh2rJR3lG1X2Tdvsk9pqKMIqCNcVIrxmAUv2zIEs/Y8F0H+252UIXo0xiBCukZUxejNYXzxqhdkID6+xVrWsmF3aDm+3CzoC58TLutgtSU54vG3a30uKHMgohkbp4v7lQpqFHzWUIiwnzRTz0Z/bcDxauELEXsMiEvSs+O7ZyP6zph4X9va/sFLlOnTEB533m5bXMwDlppXC2+8rgFUUS3G8dZwdAYstRJd1UmGWuT4TKQUd+Qugggvwc6DYV3ygdhyRVJWeHTVeaiycHCQL0p3om/hd3fbeenhLfyDIoF+0gNw1rRRnUW1fInXysB8PwkyP299XpiLaVZufCf3zXLqGQ7kkkPWbJHtUnhurBiI13OvRcONMegRcPIY8xV3nn6 9+YlXXHF R8N2Es//ERMzNAq8ZEze/RAXigTEa9HedHZRU2QPp2GN+PStb6LoOj1En41Cb+xokwHkuCYRRMfW4BkIrvjMZLk8k44DqZfm+idVeWbt1aiOj+3KqL9xZeDn/zptS4lLYeg48aV/u6X2csU5j7WVg85156l8H9MFsNoyj1lNvo8lcSeUCNEdow+FnG3hnwRflei17PgKe9ZjCf/RYI8NM/1GAAXUPVGIbd7i9rsKHju4nCLCde/HdUEy++DYgYha4c2xYQWvjvR6pVyN5AU2VuPhi+pcA+ZSXx7ejMnwo4FGEQ3bnbVoWE/EHYn7YrCLyzkMYyBxCb8Jv1KCJ4vfTd7yAFLQlEaJy1jYkoXrI0xU4Lm88dim6pk2U6Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Shadow stacks are normally written to via CALL/RET or specific CET instructions like RSTORSSP/SAVEPREVSSP. However, sometimes the kernel will need to write to the shadow stack directly using the ring-0 only WRUSS instruction. A shadow stack restore token marks a restore point of the shadow stack, and the address in a token must point directly above the token, which is within the same shadow stack. This is distinctively different from other pointers on the shadow stack, since those pointers point to executable code area. Introduce token setup and verify routines. Also introduce WRUSS, which is a kernel-mode instruction but writes directly to user shadow stack. In future patches that enable shadow stack to work with signals, the kernel will need something to denote the point in the stack where sigreturn may be called. This will prevent attackers calling sigreturn at arbitrary places in the stack, in order to help prevent SROP attacks. To do this, something that can only be written by the kernel needs to be placed on the shadow stack. This can be accomplished by setting bit 63 in the frame written to the shadow stack. Userspace return addresses can't have this bit set as it is in the kernel range. It also can't be a valid restore token. Co-developed-by: Yu-cheng Yu Signed-off-by: Yu-cheng Yu Signed-off-by: Rick Edgecombe Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- arch/x86/include/asm/special_insns.h | 13 +++++ arch/x86/kernel/shstk.c | 75 ++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index de48d1389936..d6cd9344f6c7 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -202,6 +202,19 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } +#ifdef CONFIG_X86_USER_SHADOW_STACK +static inline int write_user_shstk_64(u64 __user *addr, u64 val) +{ + asm_volatile_goto("1: wrussq %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} +#endif /* CONFIG_X86_USER_SHADOW_STACK */ + #define nop() asm volatile ("nop") static inline void serialize(void) diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index bd9cdc3a7338..e22928c63ffc 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -25,6 +25,8 @@ #include #include +#define SS_FRAME_SIZE 8 + static bool features_enabled(unsigned long features) { return current->thread.features & features; @@ -40,6 +42,35 @@ static void features_clr(unsigned long features) current->thread.features &= ~features; } +/* + * Create a restore token on the shadow stack. A token is always 8-byte + * and aligned to 8. + */ +static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) +{ + unsigned long addr; + + /* Token must be aligned */ + if (!IS_ALIGNED(ssp, 8)) + return -EINVAL; + + addr = ssp - SS_FRAME_SIZE; + + /* + * SSP is aligned, so reserved bits and mode bit are a zero, just mark + * the token 64-bit. + */ + ssp |= BIT(0); + + if (write_user_shstk_64((u64 __user *)addr, (u64)ssp)) + return -EFAULT; + + if (token_addr) + *token_addr = addr; + + return 0; +} + static unsigned long alloc_shstk(unsigned long size) { int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; @@ -157,6 +188,50 @@ unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long cl return addr + size; } +static unsigned long get_user_shstk_addr(void) +{ + unsigned long long ssp; + + fpregs_lock_and_load(); + + rdmsrl(MSR_IA32_PL3_SSP, ssp); + + fpregs_unlock(); + + return ssp; +} + +#define SHSTK_DATA_BIT BIT(63) + +static int put_shstk_data(u64 __user *addr, u64 data) +{ + if (WARN_ON_ONCE(data & SHSTK_DATA_BIT)) + return -EINVAL; + + /* + * Mark the high bit so that the sigframe can't be processed as a + * return address. + */ + if (write_user_shstk_64(addr, data | SHSTK_DATA_BIT)) + return -EFAULT; + return 0; +} + +static int get_shstk_data(unsigned long *data, unsigned long __user *addr) +{ + unsigned long ldata; + + if (unlikely(get_user(ldata, addr))) + return -EFAULT; + + if (!(ldata & SHSTK_DATA_BIT)) + return -EINVAL; + + *data = ldata & ~SHSTK_DATA_BIT; + + return 0; +} + void shstk_free(struct task_struct *tsk) { struct thread_shstk *shstk = &tsk->thread.shstk;