From patchwork Mon Feb 27 22:29:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13154238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D0BCC64ED8 for ; Mon, 27 Feb 2023 22:31:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B05706B0080; Mon, 27 Feb 2023 17:31:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB6686B0081; Mon, 27 Feb 2023 17:31:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86A8A6B0082; Mon, 27 Feb 2023 17:31:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7918D6B0080 for ; Mon, 27 Feb 2023 17:31:36 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3D1C4AAEAF for ; Mon, 27 Feb 2023 22:31:36 +0000 (UTC) X-FDA: 80514519792.02.9D289A0 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf13.hostedemail.com (Postfix) with ESMTP id 478982000A for ; Mon, 27 Feb 2023 22:31:34 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Xs7f2G5A; spf=pass (imf13.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677537094; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=tsJuTqRpMXZtpcwHwPCHx0KxUqD4hID8NTn5X+g1d6E=; b=piFQBjV8vvk4pE64dTbMZnOJ8em3W8fXEOiDlDdz7opHijQXunmMiqPyNJUdUau9rCtE62 wpxxX62838CSHjeJNpIwDtNw9Ha0/79DH3Hz0UxXg+CVOIVh5IIz1mB+J8aPgCWEyo0Oh7 qLFpANqDdopRTVrZE4aGloTBVaAGMhE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Xs7f2G5A; spf=pass (imf13.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677537094; a=rsa-sha256; cv=none; b=OXMkgMNOdIJC5NoFPaifebsxdhjZOmYES50uAg23c7j6sMhwoFC63pZuqlJlSL4zBdBJZL /15fNNYRBnsV15/VJAKpLeH/EewqqFIfFvbY97btSAOUEjFDiAiSPI5tMD+D4D/qZu4mMX ukmC2L/oudAECExSqUa2qJCE8GRZu2I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677537094; x=1709073094; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=2jjnDaU5BZRKVaf5H9IX0n9VX/dF6zxdHp6xtU3HToE=; b=Xs7f2G5AkvO/Vv4tQ2/XhYtS7IjVNxEhADiaDTBVSzlo8LKjsOfNYQgP a4nx0xpFrz9FCl5qSufcjE5zR8R2+/kjP9LNPo98KaSBEF4nY2HcHhdgh NljEXR/aqc090QEem4vIAiA1ZLcSizyZL3oEc8g3Ff7NPtniJsi//1q9j NoGrqAbzjv/vgXWghdi2I8jrDNXNwi+qj0qV+/yM06JiWrch+BHe0Qhaj minqhjn39W9ZwqZxckv1pIoQhdUDV31u+BjcZWED4KCEetuX8+RljGKeA u0fzRK3jobg01iKTQq1FUokzxfAP5DwK1I1AI/Mlpwx46ykQaaMEbSqne w==; X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="313657157" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="313657157" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:12 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="848024403" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="848024403" Received: from leonqu-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.72.19]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:12 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v7 07/41] x86: Move control protection handler to separate file Date: Mon, 27 Feb 2023 14:29:23 -0800 Message-Id: <20230227222957.24501-8-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230227222957.24501-1-rick.p.edgecombe@intel.com> References: <20230227222957.24501-1-rick.p.edgecombe@intel.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 478982000A X-Stat-Signature: 48dyd8jy89e8wq7wphon93h7x1mxjxgo X-Rspam-User: X-HE-Tag: 1677537094-732176 X-HE-Meta: U2FsdGVkX1+6g1YiStY4NNKiAZdNqcBQIGvW/NX7pL+w2alOJY0ftGArsvujhaPYCLS/xVuNtS40j741BD7hd7U/6VhsdLd0bqZ6qVkSmazzpaE/Dn29dzP6nGKdYzCzeD3sCog/euWlewp26C11DXFJ1Y9zZjKjEgyxqGo5givIMWisa5yisEt7z+cDJuRhw2IdFpzINV8kRhRL/uqNC2mtCDU5WQy2aBqtV2BZpOLC6GSCL9mNkPm85Qz3DNyb1aFoEyuDei0rCkZXKOvrJw/4srt/TsD3/orQC1O9FFRY3A+rxyobx3xjo98xNim0m3r9F3i0knP7IVFx3aj3CZthslCouJSsUXzVA1JVY9TUElHQ9q5QlJlU1aJzblreMHaEuXRIYgPv1IXQRdND1BmbgvXa5cebWuO6ConjArnl7SnhIjSbMMX3+INewAkPbYq5REZgE9VcBBhM3oSaAI7Ao1NdSlMAAqeM3F4kktcBbwMpQBObnyrux7NaKzqEIPgDz48nUG1ozxTczX09A2m0fAL2XdXc7KOKR7/ah+ZBS+dhRNl3nEoPXqyGF+5jSOZV293ocYAfH4HOxA6jTFUlJBpSXKC9CDCvka0teYajnFgfjczzbxEXQKKMmHohrFOyc71+rsejfB5lSyGunrRwqN4RBm2DIO2olAbNUn9MaOEXxIDXUNANtS15Ad/ixFg7/0PjZjzJejtaw36QXZ4x9jDF7ycFMjWQafpR0QiBruiN1VgItt+gUL+UlGWDxeOOcpTw07XXvs89AiUPhhy95kfMG1pNrE2jbzT/Dk906XHlKK3ttgkPjiXrTp4FwkFAHL2TkZPMJw4OUcdbnN+gusXM+u8tId749yn8V+fcSDnjJat1xhJSYjycQ7ZDyI7FqCShGo7IcCwa2LTzw80yZZHZMCa7hDGFkp+Oon4HLEhjzRygAiFvamGzxc/e5UMb6c7nQVGPuEPTnCV oxbWSa0E 11NIoq57Q/qrMxRcqYD1jTZqRQPy7JG0Zr6hdQC/cshPdpnK0dgv6rbZ54ZIxOyVfgFdYzExwY1MFauvwcPW/NcEmwR8SYrrcH+R3QshX1y0FORq397cMryQE65fD4EddvDbUbl+9ApUAUuM8i5fgbrEWk7Rx45vNcFjHz4xLNME7hVLjx6nZ3KwY+rYUiHj2Nk0wFAr0onPMxx1XEkc0PD5C6ubI+HZHCOCvvYIo8g/DJzCB98kQPchf3i0uwnH+6ziFfNofzO5Gfh9WV0RBO2avv/fAy6R7BfmgI6GORKBYRoq0k0b6rY/4/LfLWHufsA3h3ciE232X8YLSUCZfQ53n5L2Gzyco9d+A X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Today the control protection handler is defined in traps.c and used only for the kernel IBT feature. To reduce ifdeffery, move it to it's own file. In future patches, functionality will be added to make this handler also handle user shadow stack faults. So name the file cet.c. No functional change. Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook Acked-by: Mike Rapoport (IBM) Reviewed-by: Kees Cook Signed-off-by: Rick Edgecombe --- v6: - Split move to cet.c and shadow stack enhancements to fault handler to separate files. (Kees) --- arch/x86/kernel/Makefile | 2 ++ arch/x86/kernel/cet.c | 76 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 75 --------------------------------------- 3 files changed, 78 insertions(+), 75 deletions(-) create mode 100644 arch/x86/kernel/cet.c diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index dd61752f4c96..92446f1dedd7 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -144,6 +144,8 @@ obj-$(CONFIG_CFI_CLANG) += cfi.o obj-$(CONFIG_CALL_THUNKS) += callthunks.o +obj-$(CONFIG_X86_CET) += cet.o + ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c new file mode 100644 index 000000000000..7ad22b705b64 --- /dev/null +++ b/arch/x86/kernel/cet.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +static __ro_after_init bool ibt_fatal = true; + +extern void ibt_selftest_ip(void); /* code label defined in asm below */ + +enum cp_error_code { + CP_EC = (1 << 15) - 1, + + CP_RET = 1, + CP_IRET = 2, + CP_ENDBR = 3, + CP_RSTRORSSP = 4, + CP_SETSSBSY = 5, + + CP_ENCL = 1 << 15, +}; + +DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) +{ + if (!cpu_feature_enabled(X86_FEATURE_IBT)) { + pr_err("Unexpected #CP\n"); + BUG(); + } + + if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) + return; + + if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { + regs->ax = 0; + return; + } + + pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); + if (!ibt_fatal) { + printk(KERN_DEFAULT CUT_HERE); + __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); + return; + } + BUG(); +} + +/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ +noinline bool ibt_selftest(void) +{ + unsigned long ret; + + asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" + ANNOTATE_RETPOLINE_SAFE + " jmp *%%rax\n\t" + "ibt_selftest_ip:\n\t" + UNWIND_HINT_FUNC + ANNOTATE_NOENDBR + " nop\n\t" + + : "=a" (ret) : : "memory"); + + return !ret; +} + +static int __init ibt_setup(char *str) +{ + if (!strcmp(str, "off")) + setup_clear_cpu_cap(X86_FEATURE_IBT); + + if (!strcmp(str, "warn")) + ibt_fatal = false; + + return 1; +} + +__setup("ibt=", ibt_setup); diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d317dc3d06a3..cc223e60aba2 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -213,81 +213,6 @@ DEFINE_IDTENTRY(exc_overflow) do_error_trap(regs, 0, "overflow", X86_TRAP_OF, SIGSEGV, 0, NULL); } -#ifdef CONFIG_X86_KERNEL_IBT - -static __ro_after_init bool ibt_fatal = true; - -extern void ibt_selftest_ip(void); /* code label defined in asm below */ - -enum cp_error_code { - CP_EC = (1 << 15) - 1, - - CP_RET = 1, - CP_IRET = 2, - CP_ENDBR = 3, - CP_RSTRORSSP = 4, - CP_SETSSBSY = 5, - - CP_ENCL = 1 << 15, -}; - -DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) -{ - if (!cpu_feature_enabled(X86_FEATURE_IBT)) { - pr_err("Unexpected #CP\n"); - BUG(); - } - - if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) - return; - - if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { - regs->ax = 0; - return; - } - - pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); - if (!ibt_fatal) { - printk(KERN_DEFAULT CUT_HERE); - __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); - return; - } - BUG(); -} - -/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ -noinline bool ibt_selftest(void) -{ - unsigned long ret; - - asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" - ANNOTATE_RETPOLINE_SAFE - " jmp *%%rax\n\t" - "ibt_selftest_ip:\n\t" - UNWIND_HINT_FUNC - ANNOTATE_NOENDBR - " nop\n\t" - - : "=a" (ret) : : "memory"); - - return !ret; -} - -static int __init ibt_setup(char *str) -{ - if (!strcmp(str, "off")) - setup_clear_cpu_cap(X86_FEATURE_IBT); - - if (!strcmp(str, "warn")) - ibt_fatal = false; - - return 1; -} - -__setup("ibt=", ibt_setup); - -#endif /* CONFIG_X86_KERNEL_IBT */ - #ifdef CONFIG_X86_F00F_BUG void handle_invalid_op(struct pt_regs *regs) #else