From patchwork Tue Jun 13 00:10:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rick Edgecombe X-Patchwork-Id: 13277722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 846CCCA9EB3 for ; Tue, 13 Jun 2023 00:12:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E85058E0009; Mon, 12 Jun 2023 20:12:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBF998E0008; Mon, 12 Jun 2023 20:12:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEB618E0009; Mon, 12 Jun 2023 20:12:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A7A538E0003 for ; Mon, 12 Jun 2023 20:12:18 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 78D7B40385 for ; Tue, 13 Jun 2023 00:12:18 +0000 (UTC) X-FDA: 80895797556.15.082A9AD Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf08.hostedemail.com (Postfix) with ESMTP id 4CBD7160011 for ; Tue, 13 Jun 2023 00:12:16 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="R3/b+BSD"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686615136; a=rsa-sha256; cv=none; b=a3PF+MP/crOdj681Ge/bKtD7JInLuwmovjXPpS0zUss58Ki6W4RwSrAcoWFekXdUfLLpkA AOm+aXQFAO6Y1OSz7+goZJdY5yxsqet5dudcPj11TJVvjpvZaiGj2t+tX/H1gT6E7AcPqb 8Aye2TS1vaf4efzCdKcVA9yDBSkbuw4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="R3/b+BSD"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686615136; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4x3914htyKBGWYyTzQcGb3BHP/ZAi07IgspQYvXolFs=; b=4aV2fWJPhRxlH9R+XOPkkaQO5+rtq3U3K9HJd4sJlcFFqd523xV9Bb/F9myyZUoz/mIO7t bF/9Tlzwnv/xGp689QKvawAvGHO1y4rEONTr8oA5WhV1t8lw5hGPHiBDPMYYpEuatUHXcw Jmdc2tWa38VteKtK/hxUo6isx2J/60E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686615136; x=1718151136; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1T/qgZrUQhYBEmir0bddzuZOwE3WskkI/v/mGEi34h4=; b=R3/b+BSDam0NqDlAT1HhuUrsopGL40H9zZf3quXKnqZM8STTQFcNglZ/ 9GVzOcJb/LA1ZIXm2HoBRfGZP69oQEndy38eukFBmj5Xw22YEHG3fWV2G V5Grxd7Uqp7qmOALzhjSEIIEKr0Kod1F0p3vGVgK22hn/fZ7UDhYX+wJI RsI+XOuVWRl5V+fPTnY9sYcNONw0hBPOSXta27XwFUrM3qvTHfebL/DXu EOq5y0Y5MPn25nC3rAQWSNogSRfcxNYmDQsiE9OyTagnSCXOLMyidMPHH wnMwndWjUKzgkuW/uaaketD1g4GXc2dRny1y/ZokKMefHTXN/l7Uxw9y7 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="361556810" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="361556810" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="835670993" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="835670993" Received: from almeisch-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.amr.corp.intel.com) ([10.209.42.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:11 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, broonie@kernel.org Cc: rick.p.edgecombe@intel.com, Pengfei Xu Subject: [PATCH v9 07/42] x86/traps: Move control protection handler to separate file Date: Mon, 12 Jun 2023 17:10:33 -0700 Message-Id: <20230613001108.3040476-8-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> References: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4CBD7160011 X-Stat-Signature: 4sz8bexexkmzz53a8xm1hqhqwo9u3tf5 X-HE-Tag: 1686615136-483452 X-HE-Meta: U2FsdGVkX1/AXJ+fjK2T0zGlMAiTv0d9ie60+NNZ5ZGt1dCmeNQldhgT7smimGtf1C9IHgJshef+++IeF/ae/wswnAwj+Q9XZzzxaa65ANgWEw/4yhz5mULwZn1NYTTMCvhcXzBN4IhzL3lAkTRB5ghs4k9NF0KvJeQ72BMIQGsNX9iD/XzG87vlfQNM5W6golb9XcvF1hW3vTIP2NlrEAiJuxVv6k5u8CiNwAi/yYBYjxEpjWDggGo2vFQ+5WgnAcb7Q/RskyDIxfRuO4KrcK46V9ohmqED8B8TOwKrzHAV8RFSuk6sDmVFtjXIiC2w2rvwyBIM0WSfS1dV0aD3y9r7TdO4wKgtAbakl66Vq+g7pgj0vxMaraVjL/py1BsrEfGGZa7xsUQbSAZ56bdeYf6SOhGDNKODP5l2YIosaSgpY0iI7pbaaYoJstZXAJcwiYqZZaxjkiBiHLIXz4s7LjOZgLEP+kt6xasz90FEn1ZqL27HV5nMjOYrN96ODgbFkEeyZcDeltseiRTGOp/rujcxYkihxtlS07dB2AsgvvLGjtuJRafMQoMDQPAuOk08NQ4+/QxbuLWfXLmnjhBQ+oyvWA619yruZHUVoTSy7HcSTZHX78fCEborfHMZBTVAVi8ThH1QCm5vbnUGrCePSfIAmL6xAA4NOOWUWrYHSrVezsxybCSGqivQzhfqarQyk508mTV8l5uCB3tMyAiATdohaoh+bHZu90M+2jPzADuXcjvxepVfCHfkU5HcrElgy2QupMzSvZaqxaJjwDSc6G0iIM1bSUoctAwYTNK3IFGIU4jI6OYBBMoP+NsEJcMdmIGHS7Y+G27Zl/uNoCcqKABmxK9fxKhBjurch4ou1MoeS7aMG15XMk3jpKKjLAFly+b7GA8RGLnBCoDcUZDONqiNKll4OI81+EMLd50HPyvTGjIaXJftTWlZHCvjG6TcSpIY0GosJRYbBDPZPJq jZJm/g17 C+RXOLybM0spbfkV96Cu8wau9yOG+1NclTQ37dQ4UvhNHsDPEB8M/htwfjzn6WR0GmtM7nk8PqELCDJLZOk2XXdeOLamtaZIISIgGruSLzMbkMYhTUk2myJ6gaJgsyZVIma1H4OUJ9obZlmn67k4jOhQQ/HH7NHUv85lYHTVdvdHqygGy9RW8n+YIlWICCh2zAB65IJ3txUJpW5/CXpqFngkN6suE7bJuGkxPA+fsrvkJBG+61dOYlNJ1Q8aqxzaK1XCu1rpCbcvAjV6ll9DHASqqhpaGG9oqx0g1WY0EBhNrrUN64qPb2YIXGzp7I+2Chs+3KH0oPBsTL1zV5ZKbzlbFw3t1U8UnktKveD63byltPwJg5P8SIaBVmQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Today the control protection handler is defined in traps.c and used only for the kernel IBT feature. To reduce ifdeffery, move it to it's own file. In future patches, functionality will be added to make this handler also handle user shadow stack faults. So name the file cet.c. No functional change. Signed-off-by: Rick Edgecombe Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- arch/x86/kernel/Makefile | 2 ++ arch/x86/kernel/cet.c | 76 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 75 --------------------------------------- 3 files changed, 78 insertions(+), 75 deletions(-) create mode 100644 arch/x86/kernel/cet.c diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 4070a01c11b7..abee0564b750 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -145,6 +145,8 @@ obj-$(CONFIG_CFI_CLANG) += cfi.o obj-$(CONFIG_CALL_THUNKS) += callthunks.o +obj-$(CONFIG_X86_CET) += cet.o + ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c new file mode 100644 index 000000000000..7ad22b705b64 --- /dev/null +++ b/arch/x86/kernel/cet.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +static __ro_after_init bool ibt_fatal = true; + +extern void ibt_selftest_ip(void); /* code label defined in asm below */ + +enum cp_error_code { + CP_EC = (1 << 15) - 1, + + CP_RET = 1, + CP_IRET = 2, + CP_ENDBR = 3, + CP_RSTRORSSP = 4, + CP_SETSSBSY = 5, + + CP_ENCL = 1 << 15, +}; + +DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) +{ + if (!cpu_feature_enabled(X86_FEATURE_IBT)) { + pr_err("Unexpected #CP\n"); + BUG(); + } + + if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) + return; + + if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { + regs->ax = 0; + return; + } + + pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); + if (!ibt_fatal) { + printk(KERN_DEFAULT CUT_HERE); + __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); + return; + } + BUG(); +} + +/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ +noinline bool ibt_selftest(void) +{ + unsigned long ret; + + asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" + ANNOTATE_RETPOLINE_SAFE + " jmp *%%rax\n\t" + "ibt_selftest_ip:\n\t" + UNWIND_HINT_FUNC + ANNOTATE_NOENDBR + " nop\n\t" + + : "=a" (ret) : : "memory"); + + return !ret; +} + +static int __init ibt_setup(char *str) +{ + if (!strcmp(str, "off")) + setup_clear_cpu_cap(X86_FEATURE_IBT); + + if (!strcmp(str, "warn")) + ibt_fatal = false; + + return 1; +} + +__setup("ibt=", ibt_setup); diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 58b1f208eff5..6f666dfa97de 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -213,81 +213,6 @@ DEFINE_IDTENTRY(exc_overflow) do_error_trap(regs, 0, "overflow", X86_TRAP_OF, SIGSEGV, 0, NULL); } -#ifdef CONFIG_X86_KERNEL_IBT - -static __ro_after_init bool ibt_fatal = true; - -extern void ibt_selftest_ip(void); /* code label defined in asm below */ - -enum cp_error_code { - CP_EC = (1 << 15) - 1, - - CP_RET = 1, - CP_IRET = 2, - CP_ENDBR = 3, - CP_RSTRORSSP = 4, - CP_SETSSBSY = 5, - - CP_ENCL = 1 << 15, -}; - -DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) -{ - if (!cpu_feature_enabled(X86_FEATURE_IBT)) { - pr_err("Unexpected #CP\n"); - BUG(); - } - - if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) - return; - - if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { - regs->ax = 0; - return; - } - - pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); - if (!ibt_fatal) { - printk(KERN_DEFAULT CUT_HERE); - __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); - return; - } - BUG(); -} - -/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ -noinline bool ibt_selftest(void) -{ - unsigned long ret; - - asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" - ANNOTATE_RETPOLINE_SAFE - " jmp *%%rax\n\t" - "ibt_selftest_ip:\n\t" - UNWIND_HINT_FUNC - ANNOTATE_NOENDBR - " nop\n\t" - - : "=a" (ret) : : "memory"); - - return !ret; -} - -static int __init ibt_setup(char *str) -{ - if (!strcmp(str, "off")) - setup_clear_cpu_cap(X86_FEATURE_IBT); - - if (!strcmp(str, "warn")) - ibt_fatal = false; - - return 1; -} - -__setup("ibt=", ibt_setup); - -#endif /* CONFIG_X86_KERNEL_IBT */ - #ifdef CONFIG_X86_F00F_BUG void handle_invalid_op(struct pt_regs *regs) #else