From patchwork Sat Feb 18 21:13:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13145646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0658AC6FA9D for ; Sat, 18 Feb 2023 21:16:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C16C280005; Sat, 18 Feb 2023 16:16:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 321B6280004; Sat, 18 Feb 2023 16:16:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1749B280005; Sat, 18 Feb 2023 16:16:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 035BB280004 for ; Sat, 18 Feb 2023 16:16:08 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C50371C5F88 for ; Sat, 18 Feb 2023 21:16:07 +0000 (UTC) X-FDA: 80481670374.05.C62E042 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf16.hostedemail.com (Postfix) with ESMTP id D3445180007 for ; Sat, 18 Feb 2023 21:16:05 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SNm3Gp7f; spf=pass (imf16.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676754966; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=rkA83GGmNHLrIYmiZJYoU8PC+1rjmj46jPgBfejrGnc=; b=716kMbCGrKVVaYXHAl4P5bcu4ldeFCFgnEctCJmlujFs0142UDJE7t3LL+WaGeoCNVBja+ Rm2ZL8/mt/jet5b+h3bg9A8eMKAwYXz30OCsjkABepqU61hPLM2CfkN03wo/B3mxIxIsCN MQxcnxkbqvHQwmRSNrBN/F6wK4RhqmY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SNm3Gp7f; spf=pass (imf16.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676754966; a=rsa-sha256; cv=none; b=1Yqtl04v4d84LPx/CQq6fWKmTHtqxrQIVib7sZWI6JEJZrIdJHCo09hOFFJdscpPSCFzyv 66QAKMk88k4eM3m1WstXwrmWtqOb+2xJz+sSowtjiGRKYLi6yMoBXqhOKgt/1ac9QB1F9M u2wCchq2B0+49SngEB6ElRUq5VWcFlQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676754966; x=1708290966; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=I2MatxMVauhJIHVRqc02oKKO+hrjMuY4waDrBBWGcyY=; b=SNm3Gp7fULaPBudLEhjazim347iec2mfmTq4X+j34EBtza40xjfYQh3R Lw9vfTqpHqW3TzS29HIGGmKf9K7nK0SqkDfMzPxJI550ojPj5BXrzgyf0 bc2qu3gslFVbUu0LOnDdmTuGcq/9NlML6nCOJzRtSdHW5+RAdo09ozSed SaVGpWgzpKZsAaPHotU8zHSEjMVcY49mVcu5wg3a3GIWoYu+oVylo6a3N 5NFpJFQHrZcb1DJHOUyHpUPBIysZO9rNPvDYXMJ3I1f4je2D2LvpcIoy1 oxwZExXSyQX8ndyoRersyLpa952onQxpCE7LUKngDVvndSheNBxautyii w==; X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="418427234" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="418427234" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:16:02 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="664241598" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="664241598" Received: from adityava-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.80.223]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:16:01 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v6 07/41] x86: Move control protection handler to separate file Date: Sat, 18 Feb 2023 13:13:59 -0800 Message-Id: <20230218211433.26859-8-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230218211433.26859-1-rick.p.edgecombe@intel.com> References: <20230218211433.26859-1-rick.p.edgecombe@intel.com> X-Rspamd-Queue-Id: D3445180007 X-Stat-Signature: 3bu4ta6miwcwkr7mh95f1f5ocs3ey6mk X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1676754965-725518 X-HE-Meta: U2FsdGVkX1+g19WytvP1tP01V4HE3eWNKQVit+DYYvEHr5jdtR6L6iC7dUZ/GPUMH5nkptD9Rn4ZcBPi2b5EAt5w3hce151jg8KOXckOvPEdFenrAGs8KxEHer0JMpi+jyFNBU6a3JAscRgVQGteNUjF39B4bd7CSey7bM0NUCtXK1e1k7J1ADIXVwBmLkg7EJXYhqfC9R2V/n2SkP3HazKm9LdXI8fCIh0i5Ry8r+Y2+KXmRr8/DADi+NTQLlmW7pT4Tf7uUFFIMruwza7GaNLx7rIHyZFq11SITJ15FLj2MzvHtGQvmU1pntL30cyFCESKke20jjzl1OoM7iquf41rUm6jVF3CZkFPj1H4TZ+35wNdfvZj4GETYeazGDbsjaqoGeC9trLYER8qIVQdWjsujGmtMkWlgdgOUQtiLvSbhfstkob2BoTEVj5M7VU3OIYrdelpZqpOTJ9Z+NqDZDDnT7ec9cXt/2Ywy8R5fOJOQOYJJ0dUDqQylSykjDoUkcAqwU2rRv68A3a5DryRfhQKb1hAjGsZEK77GeDxD6PSO0ZmOn9Yng8FbORlTbO8ruX8PQvUDO5QDSjd/3QlYDp5X+HX5Zz0z7GAbVxhXsRd92j6F58gfhBNn7ThpHydyMS6mSDqDtK4lcFwrtAqLkTc1cm+qsUXh2byTUmu/xp50a4vAA9w53MqvST9MuctQT9dewcJqPr6lafJNpbEmpJW+QZjRoMvbBjbgfqq4PAccefHrts3Iuee2HohUqv6gmw8SGVrK3xCHd1gFX3wDwIcv7n5inGQT1Ql8TaNXwHHSxPGQYQ3dySpUgb3AaUhI99+QKDbCxckGFlwz/Q9yRJTBsmAwy5kPkwqYJBxkNy3rRI81WBPYq5qt5sUaYLffUkNLGFEy1fWyaE7IMH3K6Bz0VUX6lsIKmoGSef6WA5GNYGXasDq54PKGBsN9fOxdqrSdm9TR3z/3WXuAvd qBQ1T8cJ AIAeGgHl9LluRyPRz8o0IQa1hqPMAkwX97Dci8nGDV7VHOTp9QMvM0eM3PambsbM0RIrAvi8ae9ATbUauE5RJCN3QZtEN5a723XJIv5Mw9xbhCf/gQXZUvE0iaocuJnWHdRbvrdt4O9WAZShYhWG504bKmvYmqO1x+EMdCDR0IUoHdCn7u/zFLs5reel13ONjW7uX2V/xhJjN0MP/+d1X0PeXuC3veOGvLkvXXlgV0K/pbzcuEQS6uq0eUZvG8rwak8KFuzyQUwTEUgE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Today the control protection handler is defined in traps.c and used only for the kernel IBT feature. To reduce ifdeffery, move it to it's own file. In future patches, functionality will be added to make this handler also handle user shadow stack faults. So name the file cet.c. No functional change. Tested-by: Pengfei Xu Reviewed-by: Kees Cook Signed-off-by: Rick Edgecombe --- v6: - Split move to cet.c and shadow stack enhancements to fault handler to separate files. (Kees) --- arch/x86/kernel/Makefile | 2 ++ arch/x86/kernel/cet.c | 76 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 75 --------------------------------------- 3 files changed, 78 insertions(+), 75 deletions(-) create mode 100644 arch/x86/kernel/cet.c diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index dd61752f4c96..92446f1dedd7 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -144,6 +144,8 @@ obj-$(CONFIG_CFI_CLANG) += cfi.o obj-$(CONFIG_CALL_THUNKS) += callthunks.o +obj-$(CONFIG_X86_CET) += cet.o + ### # 64 bit specific files ifeq ($(CONFIG_X86_64),y) diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c new file mode 100644 index 000000000000..7ad22b705b64 --- /dev/null +++ b/arch/x86/kernel/cet.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +static __ro_after_init bool ibt_fatal = true; + +extern void ibt_selftest_ip(void); /* code label defined in asm below */ + +enum cp_error_code { + CP_EC = (1 << 15) - 1, + + CP_RET = 1, + CP_IRET = 2, + CP_ENDBR = 3, + CP_RSTRORSSP = 4, + CP_SETSSBSY = 5, + + CP_ENCL = 1 << 15, +}; + +DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) +{ + if (!cpu_feature_enabled(X86_FEATURE_IBT)) { + pr_err("Unexpected #CP\n"); + BUG(); + } + + if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) + return; + + if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { + regs->ax = 0; + return; + } + + pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); + if (!ibt_fatal) { + printk(KERN_DEFAULT CUT_HERE); + __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); + return; + } + BUG(); +} + +/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ +noinline bool ibt_selftest(void) +{ + unsigned long ret; + + asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" + ANNOTATE_RETPOLINE_SAFE + " jmp *%%rax\n\t" + "ibt_selftest_ip:\n\t" + UNWIND_HINT_FUNC + ANNOTATE_NOENDBR + " nop\n\t" + + : "=a" (ret) : : "memory"); + + return !ret; +} + +static int __init ibt_setup(char *str) +{ + if (!strcmp(str, "off")) + setup_clear_cpu_cap(X86_FEATURE_IBT); + + if (!strcmp(str, "warn")) + ibt_fatal = false; + + return 1; +} + +__setup("ibt=", ibt_setup); diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d317dc3d06a3..cc223e60aba2 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -213,81 +213,6 @@ DEFINE_IDTENTRY(exc_overflow) do_error_trap(regs, 0, "overflow", X86_TRAP_OF, SIGSEGV, 0, NULL); } -#ifdef CONFIG_X86_KERNEL_IBT - -static __ro_after_init bool ibt_fatal = true; - -extern void ibt_selftest_ip(void); /* code label defined in asm below */ - -enum cp_error_code { - CP_EC = (1 << 15) - 1, - - CP_RET = 1, - CP_IRET = 2, - CP_ENDBR = 3, - CP_RSTRORSSP = 4, - CP_SETSSBSY = 5, - - CP_ENCL = 1 << 15, -}; - -DEFINE_IDTENTRY_ERRORCODE(exc_control_protection) -{ - if (!cpu_feature_enabled(X86_FEATURE_IBT)) { - pr_err("Unexpected #CP\n"); - BUG(); - } - - if (WARN_ON_ONCE(user_mode(regs) || (error_code & CP_EC) != CP_ENDBR)) - return; - - if (unlikely(regs->ip == (unsigned long)&ibt_selftest_ip)) { - regs->ax = 0; - return; - } - - pr_err("Missing ENDBR: %pS\n", (void *)instruction_pointer(regs)); - if (!ibt_fatal) { - printk(KERN_DEFAULT CUT_HERE); - __warn(__FILE__, __LINE__, (void *)regs->ip, TAINT_WARN, regs, NULL); - return; - } - BUG(); -} - -/* Must be noinline to ensure uniqueness of ibt_selftest_ip. */ -noinline bool ibt_selftest(void) -{ - unsigned long ret; - - asm (" lea ibt_selftest_ip(%%rip), %%rax\n\t" - ANNOTATE_RETPOLINE_SAFE - " jmp *%%rax\n\t" - "ibt_selftest_ip:\n\t" - UNWIND_HINT_FUNC - ANNOTATE_NOENDBR - " nop\n\t" - - : "=a" (ret) : : "memory"); - - return !ret; -} - -static int __init ibt_setup(char *str) -{ - if (!strcmp(str, "off")) - setup_clear_cpu_cap(X86_FEATURE_IBT); - - if (!strcmp(str, "warn")) - ibt_fatal = false; - - return 1; -} - -__setup("ibt=", ibt_setup); - -#endif /* CONFIG_X86_KERNEL_IBT */ - #ifdef CONFIG_X86_F00F_BUG void handle_invalid_op(struct pt_regs *regs) #else