From patchwork Mon Jul 31 06:32:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83C5BC04FDF for ; Mon, 31 Jul 2023 07:02:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230413AbjGaHCI (ORCPT ); Mon, 31 Jul 2023 03:02:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230339AbjGaHCA (ORCPT ); Mon, 31 Jul 2023 03:02:00 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1B3B139; Mon, 31 Jul 2023 00:01:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786919; x=1722322919; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6FF6EgUyznIqvHXwn5+F1gytlgvayKLTt5Pzm0+v43k=; b=VXdDp7uBNRnY6vdVoib8N9PN4nsVBAoVqJdXDpMjE5LNLjvQuEuH0CHI GIFf+l8sBXRDuPZkgtr1pKmfWsIoSCwjKS545D8Czj9I8xn/Um0A15Jnd 6p2XWNWR/MAI/ksnh+7D75+dtFgd1BJI85dvxM3HHN4J+GMAH0Rr3LnUc qkUUJH7ea1gnmgfKgxau1o/pYAsJAu15FCs+Dm4KrtSi3pQecJ2C63cM3 JbeBf6JgynJshGKJaXiBjC6sU4GY8h2RIJs1aanj8qmXEviwAb+mnPvdN UlPtObjx49bctD66nFRedcJsODBaU8HBxnNkQdPj+u/tDX2J+3CPnSAgM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648599" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648599" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543335" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:51 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 01/36] Documentation/x86/64: Add documentation for FRED Date: Sun, 30 Jul 2023 23:32:42 -0700 Message-Id: <20230731063317.3720-2-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Briefly introduce FRED, its advantages compared to IDT, and its Linux enabling. Signed-off-by: Xin Li --- Documentation/arch/x86/x86_64/fred.rst | 102 ++++++++++++++++++++++++ Documentation/arch/x86/x86_64/index.rst | 1 + 2 files changed, 103 insertions(+) create mode 100644 Documentation/arch/x86/x86_64/fred.rst diff --git a/Documentation/arch/x86/x86_64/fred.rst b/Documentation/arch/x86/x86_64/fred.rst new file mode 100644 index 000000000000..27c980e882ba --- /dev/null +++ b/Documentation/arch/x86/x86_64/fred.rst @@ -0,0 +1,102 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +Flexible Return and Event Delivery (FRED) +========================================= + +Overview +======== + +The FRED architecture defines simple new transitions that change +privilege level (ring transitions). The FRED architecture was +designed with the following goals: + +1) Improve overall performance and response time by replacing event + delivery through the interrupt descriptor table (IDT event + delivery) and event return by the IRET instruction with lower + latency transitions. + +2) Improve software robustness by ensuring that event delivery + establishes the full supervisor context and that event return + establishes the full user context. + +The new transitions defined by the FRED architecture are FRED event +delivery and, for returning from events, two FRED return instructions. +FRED event delivery can effect a transition from ring 3 to ring 0, but +it is used also to deliver events incident to ring 0. One FRED +instruction (ERETU) effects a return from ring 0 to ring 3, while the +other (ERETS) returns while remaining in ring 0. Collectively, FRED +event delivery and the FRED return instructions are FRED transitions. + +In addition to these transitions, the FRED architecture defines a new +instruction (LKGS) for managing the state of the GS segment register. +The LKGS instruction can be used by 64-bit operating systems that do +not use the new FRED transitions. + +Software based event dispatching +================================ + +FRED operates differently from IDT in terms of event handling. Instead +of directly dispatching an event to its handler based on the event +vector, FRED requires the software to dispatch an event to its handler +based on both the event's type and vector. Therefore, an event +dispatch framework must be implemented to facilitate the +event-to-handler dispatch process. The FRED event dispatch framework +assumes control once an event is delivered, starting from two FRED +entry points, after which several event dispatch tables are introduced +to facilitate the dispatching. + +The first level dispatching is event type based, and two tables need +to be defined, one for ring 3 event dispatching, and the other +for ring 0. + +The second level dispatching is event vector based, and +several tables need to be defined, e.g., an exception handler table +for exception dispatching. + +Full supervisor/user context +============================ + +FRED event delivery atomically save and restore full supervisor/user +context upon event delivery and return. Thus it avoids the problem of +transient states due to %cr2 and/or %dr6, thus it is no longer needed +to handle all the ugly corner cases caused by half baked CPU states. + +FRED allows explicit unblock of NMI with new event return instructions +ERETS/ERETU, avoiding the mess caused by IRET which unconditionally +unblocks NMI, when an exception happens during NMI handling. + +FRED always restores the full value of %rsp, thus ESPFIX is no longer +needed when FRED is enabled. + +LKGS +==== + +LKGS behaves like the MOV to GS instruction except that it loads the +base address into the IA32_KERNEL_GS_BASE MSR instead of the GS +segment’s descriptor cache, which is exactly what Linux kernel does +to load user level GS base. With LKGS, it ends up with avoiding +mucking with kernel GS. + +Because FRED event delivery from ring 3 swaps the value of the GS base +address and that of the IA32_KERNEL_GS_BASE MSR, and ERETU swaps the +value of the GS base address and that of the IA32_KERNEL_GS_BASE MSR, +plus the introduction of LKGS instruction, the SWAPGS instruction is +no longer needed when FRED is enabled, thus is disallowed (#UD). + +Stack levels +============ + +4 stack levels 0~3 are introduced to replace the un-reentrant IST for +handling events. Each stack level could be configured to use a +dedicated stack. + +The current stack level could be unchanged or go higher upon FRED +event delivery. If unchanged, the CPU keeps using the current event +stack. If higher, the CPU switches to a new stack specified by the +stack MSR of the new stack level. + +Only execution of a FRED return instruction ERETU or ERETS could lower +the current stack level, causing the CPU to switch back to the stack +it was on before a previous event delivery. +satck. diff --git a/Documentation/arch/x86/x86_64/index.rst b/Documentation/arch/x86/x86_64/index.rst index a56070fc8e77..ad15e9bd623f 100644 --- a/Documentation/arch/x86/x86_64/index.rst +++ b/Documentation/arch/x86/x86_64/index.rst @@ -15,3 +15,4 @@ x86_64 Support cpu-hotplug-spec machinecheck fsgs + fred From patchwork Mon Jul 31 06:32:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBA68C05051 for ; Mon, 31 Jul 2023 07:02:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230364AbjGaHCD (ORCPT ); Mon, 31 Jul 2023 03:02:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230349AbjGaHCB (ORCPT ); Mon, 31 Jul 2023 03:02:01 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A09ED184; Mon, 31 Jul 2023 00:01:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786919; x=1722322919; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7IEqijDovSKiraXkZB5AveXiayuWDixrga5p5CTJEmM=; b=eNffPAdqZYVpE0SQIcUnjPMlISiEnefqsWNzEhUsGvoBWMCcTZepKuXK h1qF9aXTJa6dsD+b6nBQKSUFG+zpKDzoJD+B2hSryCKx0LP6kYpHkAgYM 8zGiD4e/lPOU4A9Q+iqIk2PV+djrBToaH7fVl6q4m18vA4Gp4LjslFVEB oQSfUOo6E7NCy1yO9aFlkFqtX95Yt7UzhNyRh6XOcBs02A53wlsOk/ep4 nj7I/AXCZphQil9T/atxPHWoR1CCGhBoKjBX1lzo1wgOl/4bd/75O7C/N ealV3JIWpoOxp+NoS47P33ry8kSM6XVFVeV+5C4qXjrmpCvWUJc7ZvEhg Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648633" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648633" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543342" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:52 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 02/36] x86/fred: Add Kconfig option for FRED (CONFIG_X86_FRED) Date: Sun, 30 Jul 2023 23:32:43 -0700 Message-Id: <20230731063317.3720-3-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add the configuration option CONFIG_X86_FRED to enable FRED. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/Kconfig | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7422db409770..700d94cb8330 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -494,6 +494,15 @@ config X86_CPU_RESCTRL Say N if unsure. +config X86_FRED + bool "Flexible Return and Event Delivery" + depends on X86_64 + help + When enabled, try to use Flexible Return and Event Delivery + instead of the legacy SYSCALL/SYSENTER/IDT architecture for + ring transitions and exception/interrupt handling if the + system supports. + if X86_32 config X86_BIGSMP bool "Support for big SMP systems with more than 8 CPUs" From patchwork Mon Jul 31 06:32:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED878C07E8D for ; Mon, 31 Jul 2023 07:02:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230387AbjGaHCF (ORCPT ); Mon, 31 Jul 2023 03:02:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230350AbjGaHCB (ORCPT ); Mon, 31 Jul 2023 03:02:01 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C557D136; Mon, 31 Jul 2023 00:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786921; x=1722322921; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ok7xj4KwdkLOXDJ1J3mT1oirCcERqdhXdoxphTKdFWY=; b=ODVAz6KV9OSUIPjAQ+K3PHDSVgZJRel5UoXfIn9QkRAsDVZwQOqZegT0 P4eE179UfN4lxr6TquDXhUp7wgpz+Xqa+o/qf+RRk2wYlELsrPtBsH/TC J7aoSnu7aCY3L/N3JPXB718jLLIgwHFoYE+KqT+o6agsLVPHkNbqnd2uZ clAB+kB+B6oTGDCbYUeMy5msz5YmeSiTOoMn+6YI1WxLHbhrYcXjrmm5v WIqxCwtWDtVj78TLx21APElZxa13AK5+4nOB3pAxMFiY7GvxMx8xfbiH3 7Edv0oKY7HNx3LzepeeiyclCU225zFOfF3yApDlN/88fGxKd/wgIgzwka g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648669" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648669" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543348" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:53 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 03/36] x86/fred: Disable FRED support if CONFIG_X86_FRED is disabled Date: Sun, 30 Jul 2023 23:32:44 -0700 Message-Id: <20230731063317.3720-4-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add CONFIG_X86_FRED to to make cpu_feature_enabled() work correctly with FRED. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/disabled-features.h | 8 +++++++- tools/arch/x86/include/asm/disabled-features.h | 8 +++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index fafe9be7a6f4..85fd67c67ce1 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -105,6 +105,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -122,7 +128,7 @@ #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING) #define DISABLED_MASK12 (DISABLE_LAM) -#define DISABLED_MASK13 0 +#define DISABLED_MASK13 (DISABLE_FRED) #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP| \ diff --git a/tools/arch/x86/include/asm/disabled-features.h b/tools/arch/x86/include/asm/disabled-features.h index fafe9be7a6f4..85fd67c67ce1 100644 --- a/tools/arch/x86/include/asm/disabled-features.h +++ b/tools/arch/x86/include/asm/disabled-features.h @@ -105,6 +105,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -122,7 +128,7 @@ #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING) #define DISABLED_MASK12 (DISABLE_LAM) -#define DISABLED_MASK13 0 +#define DISABLED_MASK13 (DISABLE_FRED) #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP| \ From patchwork Mon Jul 31 06:32:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31266C07E8C for ; Mon, 31 Jul 2023 07:02:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230433AbjGaHCK (ORCPT ); Mon, 31 Jul 2023 03:02:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230356AbjGaHCD (ORCPT ); Mon, 31 Jul 2023 03:02:03 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AC84185; Mon, 31 Jul 2023 00:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786922; x=1722322922; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tXI/C++YYSo9nyQMGnRVNwF3YsuprVE9KCho8shF7bg=; b=Y3tYMwS/pkB5m52h6paapMyOucykhHc/Vh0VfyWqM+luw5Ukq00nMK3L p2AgU9NPD02TN8liUQOnFQ35OVes2GV0l0OtIuxpA15PUod25jn8Y1Nlw KbSkvGrBQZFG8BFblXmuk6EXwazMRTp0cvBRBX5xX4MJzEfYQaPKEvWMO sPaiJeILKl2l3KJoZpPLbovzRku3U6u4CmFNsYrIbM3O2T2886WnP0Ibn AfOPEyDvzhe8r2sr5ti9ukceD7SELtvw1poRbuIg7iY2bClQ8qdNbnN6J o+syfgkbAaFcjzZSvKsb8WkzPjKWAXlOeA25w0kG+O8rIcwVALNfmjvCd w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648706" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648706" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543352" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:54 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 04/36] x86/cpufeatures: Add the cpu feature bit for FRED Date: Sun, 30 Jul 2023 23:32:45 -0700 Message-Id: <20230731063317.3720-5-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add the CPU feature bit for FRED. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index cb8ca46213be..fd3ddd5c0283 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -317,6 +317,7 @@ #define X86_FEATURE_FZRM (12*32+10) /* "" Fast zero-length REP MOVSB */ #define X86_FEATURE_FSRS (12*32+11) /* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12) /* "" Fast short REP {CMPSB,SCASB} */ +#define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMADD52[H,L]UQ */ diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/include/asm/cpufeatures.h index cb8ca46213be..fd3ddd5c0283 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -317,6 +317,7 @@ #define X86_FEATURE_FZRM (12*32+10) /* "" Fast zero-length REP MOVSB */ #define X86_FEATURE_FSRS (12*32+11) /* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12) /* "" Fast short REP {CMPSB,SCASB} */ +#define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMADD52[H,L]UQ */ From patchwork Mon Jul 31 06:32:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D771C05052 for ; Mon, 31 Jul 2023 07:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230456AbjGaHCN (ORCPT ); Mon, 31 Jul 2023 03:02:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230377AbjGaHCF (ORCPT ); Mon, 31 Jul 2023 03:02:05 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8D52139; Mon, 31 Jul 2023 00:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786924; x=1722322924; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=F/SemwGNBG6weEZopYj3iIwsf6sy0XGLv5PanieJeUc=; b=Vw2WEwXXcICQqdNT2ae4LolsYzfchslshNDfrPHKeIwSBeIM0fghfq4G sz2FutcpKHx2VUJIEkKWyHQgQ8yGI4vrNPHlVpjPQxECJqKTFL7F03coX hZ9fQeKzySNZD6hrTb6WbYLpkcw8b1FwPvq/2NnyLwjd4eN4X+xOe+HLD Ep4p6hZ+R2A/XGkhFndH+Ipk/I5NDo+iMtKnuPEt1wVK/UzKX/kDfvJ+h cOyOgLfA3QC/IGE9YntxaH8p6yIGRINi4R/gYkvOS5BeIMXjWwZJzTarG LTMa7sZflz/xd/VHes9y97uXLpXLSUurLOMpWVxhKj0cmp4RN3OU3neok A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648744" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648744" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543358" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:55 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 05/36] x86/opcode: Add ERETU, ERETS instructions to x86-opcode-map Date: Sun, 30 Jul 2023 23:32:46 -0700 Message-Id: <20230731063317.3720-6-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add instruction opcodes used by FRED ERETU/ERETS to x86-opcode-map. Opcode numbers are per FRED spec v5.0. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li Reviewed-by: Masami Hiramatsu (Google) --- arch/x86/lib/x86-opcode-map.txt | 2 +- tools/arch/x86/lib/x86-opcode-map.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt index 5168ee0360b2..7a269e269dc0 100644 --- a/arch/x86/lib/x86-opcode-map.txt +++ b/arch/x86/lib/x86-opcode-map.txt @@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv diff --git a/tools/arch/x86/lib/x86-opcode-map.txt b/tools/arch/x86/lib/x86-opcode-map.txt index 5168ee0360b2..7a269e269dc0 100644 --- a/tools/arch/x86/lib/x86-opcode-map.txt +++ b/tools/arch/x86/lib/x86-opcode-map.txt @@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv From patchwork Mon Jul 31 06:32:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9133CC04E69 for ; Mon, 31 Jul 2023 07:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230486AbjGaHCQ (ORCPT ); Mon, 31 Jul 2023 03:02:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230349AbjGaHCG (ORCPT ); Mon, 31 Jul 2023 03:02:06 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F220186; Mon, 31 Jul 2023 00:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786925; x=1722322925; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+857ZiZ3kYB9kvyULHXUrCarfExj5Gil+GQ461ib92M=; b=NE4em+KwgK+XoH/U7xVPtrVo6rpE0Dy4VRKTigLSgq2pLWRSRYyeqyT3 zk9R/n0uB1yqFlMwabpgzPknIwroSMZGdFeDA9P4INOL8lZO5uj4+qg2f uQ/q13Nh8u59aehP0x2GChul1f6nXuab2LXKaekVEnDfUUDvH0mJu0f5d ASibBM4CfGnxm+gbGWJnjDnaUsNuPOpwEn/sFWJtYmoPslM9l6j6HiXTF opnY/E8H8M2V1fd8NhPA6NqaSybSK9paXNHhujmFdTHmlW4D82ybYB/+5 tUQCzMUtQCMjk6gvtq/d9eg1cI5o+9RYcg4N9LdnCIjRevhybKUFqivzN Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648781" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648781" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543364" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:56 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 06/36] x86/objtool: Teach objtool about ERETU and ERETS Date: Sun, 30 Jul 2023 23:32:47 -0700 Message-Id: <20230731063317.3720-7-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Update the objtool decoder to know about the ERETU and ERETS instructions (type INSN_CONTEXT_SWITCH). Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- tools/objtool/arch/x86/decode.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c index 2e1caabecb18..a486485cff20 100644 --- a/tools/objtool/arch/x86/decode.c +++ b/tools/objtool/arch/x86/decode.c @@ -509,11 +509,20 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *sec if (op2 == 0x01) { - if (modrm == 0xca) - insn->type = INSN_CLAC; - else if (modrm == 0xcb) - insn->type = INSN_STAC; - + switch (insn_last_prefix_id(&ins)) { + case INAT_PFX_REPE: + case INAT_PFX_REPNE: + if (modrm == 0xca) + /* eretu/erets */ + insn->type = INSN_CONTEXT_SWITCH; + break; + default: + if (modrm == 0xca) + insn->type = INSN_CLAC; + else if (modrm == 0xcb) + insn->type = INSN_STAC; + break; + } } else if (op2 >= 0x80 && op2 <= 0x8f) { insn->type = INSN_JUMP_CONDITIONAL; From patchwork Mon Jul 31 06:32:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D10A5C05052 for ; Mon, 31 Jul 2023 07:02:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230469AbjGaHCS (ORCPT ); Mon, 31 Jul 2023 03:02:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230411AbjGaHCI (ORCPT ); Mon, 31 Jul 2023 03:02:08 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 678551A5; Mon, 31 Jul 2023 00:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786926; x=1722322926; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ks9fQ3QR6UqRMocjQpLz+yk1iYvQPVTbiFl2BDd8yPg=; b=FJ6c1HRtn6LGcysL1uA2OSv4/K/7VnmXSE4ZnZBZEB0V525Psxg8LIv4 HOpM0/sVyOWNFp/HyGernH33Igw9EAl/Q2pfGd0MxnSI94zfhoRUoYwp8 hOgFT9uL9oRdAj1es5CPJ2G8OSpn+hdeis4R2kPAkuPnDpMEfGWCJolfl 9lZGt1aCgWf5le/dHhWxQHlFcByLDuL4Gm7fk/u8NJxL3vJag3ZUMljFJ /Kvn1OxuVWCQ7+hvGGplMMzqwsDHSOeTx239IuWfskJp1MTvhyedufhRI BVzKxQEt6YDroTIG/Hy4n1cFSbxd91MSzl8hMdt1VAyvZmVkOZBv/Os3C Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648821" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648821" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543368" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:57 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 07/36] x86/cpu: Add X86_CR4_FRED macro Date: Sun, 30 Jul 2023 23:32:48 -0700 Message-Id: <20230731063317.3720-8-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add X86_CR4_FRED macro for the FRED bit in %cr4. This bit should be a pinned bit, not to be changed after initialization. CR4 macros are defined in arch/x86/include/uapi/asm/processor-flags.h, which is uapi, and thus cannot depend on CONFIG_X86_64. Using _BITUL() causes build errors on 32 bits, and there is no guarantee that user space applications (e.g. something like Qemu) might not want to use this declaration even when building for i386 or x32. However, %cr4 is a machine word (unsigned long), so to avoid build warnings on 32 bits, explicitly cast the value to unsigned long, truncating upper 32 bits. The other alternative would be to use CONFIG_X86_64 around the definition of cr4_pinned_mask. It is probably not desirable to make cr4_pinned_mask non-const. Another option, which may be preferable, to be honest: explicitly enumerate the CR4 bits which *may* be changed (a whitelist), instead of the ones that may not. That would be a separate, pre-FRED, patch, and would automatically resolve this problem as a side effect. The following flags probably should have been in this set all along, as they are all controls affecting the kernel runtime environment as opposed to user space: X86_CR4_DE, X86_CR4_PAE, X86_CR4_PSE, X86_CR4_MCE, X86_CR4_PGE, X86_CR4_OSFXSR, X86_CR4_OSXMMEXCPT, X86_CR4_LA57, X86_CR4_PCIDE, X86_CR4_LAM_SUP Possibly X86_CR4_VMXE as well, which seems harmless even if KVM is not loaded; X86_CR4_PKE can be fixed as long as the PKE configuration registers are at least initialized to disabled. It is relatively simple to do an audit of which flags are allowed to be modified at runtime and whitelist only those. There is no reason why we should allow bits in CR4 to be toggled by default. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/uapi/asm/processor-flags.h | 2 ++ arch/x86/kernel/cpu/common.c | 5 +++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/uapi/asm/processor-flags.h b/arch/x86/include/uapi/asm/processor-flags.h index d898432947ff..ce08c2ca70b5 100644 --- a/arch/x86/include/uapi/asm/processor-flags.h +++ b/arch/x86/include/uapi/asm/processor-flags.h @@ -138,6 +138,8 @@ #define X86_CR4_CET _BITUL(X86_CR4_CET_BIT) #define X86_CR4_LAM_SUP_BIT 28 /* LAM for supervisor pointers */ #define X86_CR4_LAM_SUP _BITUL(X86_CR4_LAM_SUP_BIT) +#define X86_CR4_FRED_BIT 32 /* enable FRED kernel entry */ +#define X86_CR4_FRED _BITULL(X86_CR4_FRED_BIT) /* * x86-64 Task Priority Register, CR8 diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 0ba1067f4e5f..331b06d19f7f 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -402,8 +402,9 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) /* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask = - X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | - X86_CR4_FSGSBASE | X86_CR4_CET; + (unsigned long) + (X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | + X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED); static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning); static unsigned long cr4_pinned_bits __ro_after_init; From patchwork Mon Jul 31 06:32:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FB9CC04A94 for ; Mon, 31 Jul 2023 07:02:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230491AbjGaHCU (ORCPT ); Mon, 31 Jul 2023 03:02:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230418AbjGaHCJ (ORCPT ); Mon, 31 Jul 2023 03:02:09 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1F67E75; Mon, 31 Jul 2023 00:02:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786928; x=1722322928; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4BQfFZvfF2FO6NJuqP2rESZLyKeNKRqaVIW2rUp/ej0=; b=cF7XRDwAKTjuuVKtgbseu/ZpyezQjKakUB+gzqfjSXcV9KQb647sywGP FeD3RIoGZZ4DA/61OVfF1M0dGTmv+FA+ebd7NdcTnZZhPhnKaUrmOTnJc DddYi9KRe8L+8nE5ha8yif34TsWufvoBVHteUxKgaQbePrXDxVx9uU+hy 3Kwu2WbYIodq0FO3iAmubdKoqal+QOpESg/Miowha8tL9VjSe9s8bHaiu lVQQCrZhI7dwlzxdcW8JMAez7veZmQUD1TI6MnfwdzfYtQdTpMbckCNKF h4UkfbmRC7PCuJFdaIdqKYRVfqLgpEc+zgm21uIKY/SuWBaGBQVKNLzpu A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648855" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648855" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543375" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:58 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 08/36] x86/cpu: Add MSR numbers for FRED configuration Date: Sun, 30 Jul 2023 23:32:49 -0700 Message-Id: <20230731063317.3720-9-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add MSR numbers for the FRED configuration registers. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/msr-index.h | 13 ++++++++++++- tools/arch/x86/include/asm/msr-index.h | 13 ++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index a00a53e15ab7..111fb76f6dbe 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -36,8 +36,19 @@ #define EFER_FFXSR (1<<_EFER_FFXSR) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc /* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd /* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce /* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf /* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0 /* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP /* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1 /* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2 /* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3 /* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4 /* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT) diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index 3aedae61af4f..565cade0785a 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -36,8 +36,19 @@ #define EFER_FFXSR (1<<_EFER_FFXSR) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc /* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd /* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce /* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf /* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0 /* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP /* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1 /* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2 /* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3 /* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4 /* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT) From patchwork Mon Jul 31 06:32:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AFD2C001DF for ; Mon, 31 Jul 2023 07:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230416AbjGaHCY (ORCPT ); Mon, 31 Jul 2023 03:02:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230425AbjGaHCK (ORCPT ); Mon, 31 Jul 2023 03:02:10 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2DCB185; Mon, 31 Jul 2023 00:02:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786929; x=1722322929; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f1KjXTBSCMC/3m47AKFmo72XHXtCtx2wNvJSmQVR8WU=; b=kqBHMwGjigCkQXGhyJ0SsSXb5L9lppOMXrViqJkaj3h7QUqCppXx0/E8 vsXLwYzVeZKZuopG2nYUFHL8ETOGY4xgO4Gz80PoAVYDekBrt/I8plwnZ T7xtFi6tKPINsNWFni4h2Lz1SBLumAQx+lVTa9TPT6CFa7a802cDFX6FG mO27iAqqZVQNiMEXCGdh+tDiLkjj21TAqZSSYi4UEF9o8ST1/v0ZOHCtW UfuhmTUurOSK2p56gJui0wm/GyCcdf+IfvAfxI9b7a65LIwp30FKLjora ZMLH56vHrX6cB6KPUkRXtpYYHVo7cS7LN+sM5QufmF4ChsG21J9BcO+0g Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648891" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648891" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543380" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:01:59 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 09/36] x86/fred: Make unions for the cs and ss fields in struct pt_regs Date: Sun, 30 Jul 2023 23:32:50 -0700 Message-Id: <20230731063317.3720-10-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Make the cs and ss fields in struct pt_regs unions between the actual selector and the unsigned long stack slot. FRED uses this space to store additional flags. The printk changes are simply due to the cs and ss fields changed to unsigned short from unsigned long. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Reflect stack frame definition changes from FRED spec 3.0 to 5.0. * Use __packed instead of __attribute__((__packed__)) (Borislav Petkov). * Put all comments above the members, like the rest of the file does (Borislav Petkov). Changes since v3: * Rename csl/ssl of the pt_regs structure to csx/ssx (x for extended) (Andrew Cooper). --- arch/x86/entry/vsyscall/vsyscall_64.c | 2 +- arch/x86/include/asm/ptrace.h | 57 +++++++++++++++++++++++++-- arch/x86/kernel/process_64.c | 2 +- 3 files changed, 56 insertions(+), 5 deletions(-) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c index e0ca8120aea8..a3c0df11d0e6 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -76,7 +76,7 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, if (!show_unhandled_signals) return; - printk_ratelimited("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n", + printk_ratelimited("%s%s[%d] %s ip:%lx cs:%x sp:%lx ax:%lx si:%lx di:%lx\n", level, current->comm, task_pid_nr(current), message, regs->ip, regs->cs, regs->sp, regs->ax, regs->si, regs->di); diff --git a/arch/x86/include/asm/ptrace.h b/arch/x86/include/asm/ptrace.h index f4db78b09c8f..f1690beffd15 100644 --- a/arch/x86/include/asm/ptrace.h +++ b/arch/x86/include/asm/ptrace.h @@ -80,15 +80,66 @@ struct pt_regs { /* * On syscall entry, this is syscall#. On CPU exception, this is error code. * On hw interrupt, it's IRQ number: + * + * A FRED stack frame starts here: + * 1) It _always_ includes an error code; + * 2) The return frame for eretu/erets starts here. */ unsigned long orig_ax; /* Return frame for iretq */ unsigned long ip; - unsigned long cs; + union { +/* CS extended: CS + any fields above it */ + unsigned long csx; + struct { +/* CS selector proper */ + unsigned short cs; +/* The stack level (SL) at the time the event occurred */ + unsigned int sl : 2; +/* Set to indicate that indirect branch tracker in WAIT_FOR_ENDBRANCH state */ + unsigned int wfe : 1; + unsigned int __csx_resv1: 13; + unsigned int __csx_resv2: 32; + } __packed; + }; unsigned long flags; unsigned long sp; - unsigned long ss; -/* top of stack page */ + union { +/* SS extended: SS + any fields above it */ + unsigned long ssx; + struct { +/* SS selector proper */ + unsigned short ss; +/* Set to indicate that interrupt blocking by STI was in effect */ + unsigned int sti : 1; +/* For SYSCALL, SYSENTER, or INT n (for any value of n) */ + unsigned int sys : 1; + unsigned int nmi : 1; + unsigned int __ssx_resv1: 13; +/* Event information fields, ignored by the FRED return instructions */ + unsigned int vector : 8; + unsigned int __ssx_resv2: 8; + unsigned int type : 4; + unsigned int __ssx_resv3: 4; +/* Set to indicate that the event was incident to enclave execution */ + unsigned int enc : 1; +/* Set to indicate that the logical processor had been in 64-bit mode */ + unsigned int l : 1; +/* + * Set to indicate the event is a nested exception encountered during FRED + * event delivery of another event. This bit is not set if the event is + * double fault (#DF). + */ + unsigned int nst : 1; + unsigned int __ssx_resv4: 1; +/* The length of the instruction causing the event */ + unsigned int instr_len : 4; + } __packed; + }; +/* + * Top of stack page on IDT systems, while FRED systems have extra fields + * defined above, see . + */ }; #endif /* !__i386__ */ diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 3d181c16a2f6..265ab8fcb146 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -117,7 +117,7 @@ void __show_regs(struct pt_regs *regs, enum show_regs_mode mode, printk("%sFS: %016lx(%04x) GS:%016lx(%04x) knlGS:%016lx\n", log_lvl, fs, fsindex, gs, gsindex, shadowgs); - printk("%sCS: %04lx DS: %04x ES: %04x CR0: %016lx\n", + printk("%sCS: %04x DS: %04x ES: %04x CR0: %016lx\n", log_lvl, regs->cs, ds, es, cr0); printk("%sCR2: %016lx CR3: %016lx CR4: %016lx\n", log_lvl, cr2, cr3, cr4); From patchwork Mon Jul 31 06:32:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 716B8C04E69 for ; Mon, 31 Jul 2023 07:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230377AbjGaHC1 (ORCPT ); Mon, 31 Jul 2023 03:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230434AbjGaHCL (ORCPT ); Mon, 31 Jul 2023 03:02:11 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EBE71A5; Mon, 31 Jul 2023 00:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786929; x=1722322929; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2rbYs2KdLMV2D8jndr60eRf402MI8mOow/QviwqPNSc=; b=nNlM8I/yp9KSLBodDD1PTyH9qE+LuGseHi6TLUO0EgSF7ynBulOQ980z hKyqnJOhlN35rV1tXbsW8zTuytbj+ZOqA64PKx5mYlSNILWMpyTEGQ5QZ 9DYGqo/2tYMZg92aRAFtHRts8WL7TpOJSVA7OYvVLHnUfpEyhelxgEDod 240C+kS0XixnCNdBBFO0NjU7Jgk1F8cwf46SlpIZ61638nkZYu8CZYbs6 XXXIhvH9KFGak2xriDXAKnLCZVuesckWMPH4DzcTXvZBx8iEatGq3nFJp O6bZE9l7BLBkzROFJrTNyRA4IuONWmcufhg9HJnbfNP+fx4n+2YJXvTz7 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648930" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648930" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:01:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543389" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:00 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 10/36] x86/fred: Add a new header file for FRED definitions Date: Sun, 30 Jul 2023 23:32:51 -0700 Message-Id: <20230731063317.3720-11-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add a header file for FRED prototypes and definitions. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v6: * Replace pt_regs csx flags prefix FRED_CSL_ with FRED_CSX_. --- arch/x86/include/asm/fred.h | 104 ++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 arch/x86/include/asm/fred.h diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h new file mode 100644 index 000000000000..d76e681a806f --- /dev/null +++ b/arch/x86/include/asm/fred.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Macros for Flexible Return and Event Delivery (FRED) + */ + +#ifndef ASM_X86_FRED_H +#define ASM_X86_FRED_H + +#include +#include + +/* + * FRED return instructions + * + * Replace with "ERETS"/"ERETU" once binutils support FRED return instructions. + * The binutils version supporting FRED instructions is still TBD, and will + * update once we have it. + */ +#define ERETS _ASM_BYTES(0xf2,0x0f,0x01,0xca) +#define ERETU _ASM_BYTES(0xf3,0x0f,0x01,0xca) + +/* + * RSP is aligned to a 64-byte boundary before used to push a new stack frame + */ +#define FRED_STACK_FRAME_RSP_MASK _AT(unsigned long, (~0x3f)) + +/* + * Event stack level macro for the FRED_STKLVLS MSR. + * Usage example: FRED_STKLVL(X86_TRAP_DF, 3) + * Multiple values can be ORd together. + */ +#define FRED_STKLVL(v,l) (_AT(unsigned long, l) << (2*(v))) + +/* FRED_CONFIG MSR */ +#define FRED_CONFIG_CSL_MASK 0x3 +/* + * Used for the return address for call emulation during code patching, + * and measured in 64-byte cache lines. + */ +#define FRED_CONFIG_REDZONE_AMOUNT 1 +#define FRED_CONFIG_REDZONE (_AT(unsigned long, FRED_CONFIG_REDZONE_AMOUNT) << 6) +#define FRED_CONFIG_INT_STKLVL(l) (_AT(unsigned long, l) << 9) +#define FRED_CONFIG_ENTRYPOINT(p) _AT(unsigned long, (p)) + +/* + * FRED event type and vector bit width and counts. + * + * There is space in the stack frame making it possible to extend event type + * and vector fields in the future. + */ +#define FRED_EVENT_TYPE_BITS 3 +#define FRED_EVENT_TYPE_COUNT _BITUL(FRED_EVENT_TYPE_BITS) +#define FRED_EVENT_VECTOR_BITS 8 +#define FRED_EVENT_VECTOR_COUNT _BITUL(FRED_EVENT_VECTOR_BITS) + +/* FRED EVENT_TYPE_OTHER vector numbers */ +#define FRED_SYSCALL 1 +#define FRED_SYSENTER 2 +#define FRED_NUM_OTHER_VECTORS 3 + +/* Flags above the SS selector (regs->ssx) */ +#define FRED_SSX_INTERRUPT_SHADOW_BIT 16 +#define FRED_SSX_INTERRUPT_SHADOW _BITUL(FRED_SSX_INTERRUPT_SHADOW_BIT) +#define FRED_SSX_SOFTWARE_INITIATED_BIT 17 +#define FRED_SSX_SOFTWARE_INITIATED _BITUL(FRED_SSX_SOFTWARE_INITIATED_BIT) +#define FRED_SSX_NMI_BIT 18 +#define FRED_SSX_NMI _BITUL(FRED_SSX_NMI_BIT) +#define FRED_SSX_64_BIT_MODE_BIT 57 +#define FRED_SSX_64_BIT_MODE _BITUL(FRED_SSX_64_BIT_MODE_BIT) + +#ifdef CONFIG_X86_FRED + +#ifndef __ASSEMBLY__ + +#include +#include + +struct fred_info { + /* Event data: CR2, DR6, ... */ + unsigned long edata; + unsigned long resv; +}; + +/* Full format of the FRED stack frame */ +struct fred_frame { + struct pt_regs regs; + struct fred_info info; +}; + +static __always_inline struct fred_info *fred_info(struct pt_regs *regs) +{ + return &container_of(regs, struct fred_frame, regs)->info; +} + +static __always_inline unsigned long fred_event_data(struct pt_regs *regs) +{ + return fred_info(regs)->edata; +} + +#endif /* __ASSEMBLY__ */ + +#endif /* CONFIG_X86_FRED */ + +#endif /* ASM_X86_FRED_H */ From patchwork Mon Jul 31 06:32:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE6E6C001DC for ; Mon, 31 Jul 2023 07:02:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230510AbjGaHCf (ORCPT ); Mon, 31 Jul 2023 03:02:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230438AbjGaHCM (ORCPT ); Mon, 31 Jul 2023 03:02:12 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65720E68; Mon, 31 Jul 2023 00:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786931; x=1722322931; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+L7aUhS6u+4qwVp6754zs3SWJDqRpOJYTCoF5qb+V6E=; b=gJMz/dMGuP1m4jyrrNIs7kDXMReMW9GUsFuYhSD2sJqUi6ZG/9ZhOFMN uoGJex+I/vUGt3g9RPzC+SsBOUA5iwGeUPhsxRyMXVluNflpzvyCjrOYj K52IVUxWJoxxRVpf6Camsuev/Hew9nlYyyNN8yQ32DyDgKRcbDdplwtmo UhkgqDFdBhkN2+S6xsZmL7OsGnGWn5uLyDTPajfqMCk9Cxe4/7fW+5XuN 0zYLbWfmk2XzXlFKuq4r+QIY8bywrp7IrIIFJFNxjgXPLuEKM6ImglCJ0 gOLbrPKuUArGjHNXsbE8oqgdW/w7/jVLPuZkkFcM9DbeobaD1Yc6Gfs/4 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648963" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648963" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543395" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:01 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 11/36] x86/fred: Reserve space for the FRED stack frame Date: Sun, 30 Jul 2023 23:32:52 -0700 Message-Id: <20230731063317.3720-12-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" When using FRED, reserve space at the top of the stack frame, just like i386 does. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/thread_info.h | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index d63b02940747..089cab875cba 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -31,7 +31,9 @@ * In vm86 mode, the hardware frame is much longer still, so add 16 * bytes to make room for the real-mode segments. * - * x86_64 has a fixed-length stack frame. + * x86-64 has a fixed-length stack frame, but it depends on whether + * or not FRED is enabled. Future versions of FRED might make this + * dynamic, but for now it is always 2 words longer. */ #ifdef CONFIG_X86_32 # ifdef CONFIG_VM86 @@ -39,8 +41,12 @@ # else # define TOP_OF_KERNEL_STACK_PADDING 8 # endif -#else -# define TOP_OF_KERNEL_STACK_PADDING 0 +#else /* x86-64 */ +# ifdef CONFIG_X86_FRED +# define TOP_OF_KERNEL_STACK_PADDING (2*8) +# else +# define TOP_OF_KERNEL_STACK_PADDING 0 +# endif #endif /* From patchwork Mon Jul 31 06:32:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 670E8C001DF for ; Mon, 31 Jul 2023 07:02:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230479AbjGaHCb (ORCPT ); Mon, 31 Jul 2023 03:02:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230470AbjGaHCO (ORCPT ); Mon, 31 Jul 2023 03:02:14 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43F11E7B; Mon, 31 Jul 2023 00:02:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786932; x=1722322932; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lZsnH9xZ+sSI0HJfWdXFi2jVveN7fME2POBe5mQbKKE=; b=J0LdpU5rSmCuBwzkdRgVcfNkCXN4YDZW9i6UpATzZXkMhv3H3dFMbCdL u+CgWOHx4YPkeXuXx6fqMo9KGymbbBnaY6mut4yWQWO/B40B+qroLttVa ShuknVTU1EOwSB0+ih+HfernHtcZhNmBbB27+FxFeoEz0Ge0wGV76Tz3R BmJONRngwVM15yn6k9QjIc/42TGDnty9o1DnAnM9V0B5NzNbUQWxvmDsu JMA5/10NjT5i4UMF6NO4TvZdeWkc3mbYcuuy0Grcs3yIBQNkqTf8gPZLh fZZwOzrJFPuK4aeuajYFSiK7W+1PdMiFwJ+yE1HY3XNplW9ecEwOVqU5a w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371648999" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371648999" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543400" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:02 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 12/36] x86/fred: Update MSR_IA32_FRED_RSP0 during task switch Date: Sun, 30 Jul 2023 23:32:53 -0700 Message-Id: <20230731063317.3720-13-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" MSR_IA32_FRED_RSP0 is used during ring 3 event delivery, and needs to be updated to point to the top of next task stack during task switch. Update MSR_IA32_FRED_RSP0 with WRMSR instruction for now, and will use WRMSRNS/WRMSRLIST for performance once it gets upstreamed. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/switch_to.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h index f42dbf17f52b..6c911fd400b2 100644 --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -70,9 +70,16 @@ static inline void update_task_stack(struct task_struct *task) #ifdef CONFIG_X86_32 this_cpu_write(cpu_tss_rw.x86_tss.sp1, task->thread.sp0); #else - /* Xen PV enters the kernel on the thread stack. */ - if (cpu_feature_enabled(X86_FEATURE_XENPV)) + if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* + * Will use WRMSRNS/WRMSRLIST for performance once it's upstreamed. + */ + wrmsrl(MSR_IA32_FRED_RSP0, + (unsigned long)task_stack_page(task) + THREAD_SIZE); + } else if (cpu_feature_enabled(X86_FEATURE_XENPV)) { + /* Xen PV enters the kernel on the thread stack. */ load_sp0(task_top_of_stack(task)); + } #endif } From patchwork Mon Jul 31 06:32:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E16DFC07E8F for ; Mon, 31 Jul 2023 07:03:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231175AbjGaHDY (ORCPT ); Mon, 31 Jul 2023 03:03:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230451AbjGaHCR (ORCPT ); Mon, 31 Jul 2023 03:02:17 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A9FE184; Mon, 31 Jul 2023 00:02:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786934; x=1722322934; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=90z+Db4H8sAaozlh1y54siqgX9Yc5fCU6uoHEAi25BE=; b=Ayh6kAJ3XqvBJ/mtAVxKzE+igNID/C9D+m58DZHwdkigg0qxcR4cLiIz Ae7EqFTgibZypG+m2/VkqHL3iSbF40Ea0LlQ0F7+JPKqL7B4xMM4Dr4gr /YCibwM3fE5n1/dV8yPeseudDS5GvaB+lxuU2IQkurKycW9GHEmCNRHxy 1dRhVciB5sHkov+DMfF8LMQriwPrSvYrZFEewjrqngvFNWuJnRhOVAO/2 joBN8kexk45Swjmfd2qcvqMK98ZVj6fWsN4xF0edxWJJlMjoBhP3sAHVx 4NSQGiSvcxLadG6V+ASy+F+QP2tPLVS2ADGdDJnRCte3fyoe12ZC0l5fa w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649035" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649035" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543416" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:03 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 13/36] x86/fred: Let ret_from_fork_asm() jmp to fred_exit_user when FRED is enabled Date: Sun, 30 Jul 2023 23:32:54 -0700 Message-Id: <20230731063317.3720-14-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Let ret_from_fork_asm() jmp to fred_exit_user when FRED is enabled, otherwise the existing IDT code is chosen. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/entry/entry_64.S | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 43606de22511..8069151176f2 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -309,7 +309,13 @@ SYM_CODE_START(ret_from_fork_asm) * and unwind should work normally. */ UNWIND_HINT_REGS + +#ifdef CONFIG_X86_FRED + ALTERNATIVE "jmp swapgs_restore_regs_and_return_to_usermode", \ + "jmp fred_exit_user", X86_FEATURE_FRED +#else jmp swapgs_restore_regs_and_return_to_usermode +#endif SYM_CODE_END(ret_from_fork_asm) .popsection From patchwork Mon Jul 31 06:32:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5391FC001DC for ; Mon, 31 Jul 2023 07:03:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231196AbjGaHDd (ORCPT ); Mon, 31 Jul 2023 03:03:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231135AbjGaHDX (ORCPT ); Mon, 31 Jul 2023 03:03:23 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EF1DE7B; Mon, 31 Jul 2023 00:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786937; x=1722322937; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sycVsd87zmfA3yJCdcgOgrxd4mm6nSq5VT1UdIRGngQ=; b=Sb11QzFc6ceq0l6D2+ruUClNO8ujAszWBJDj6SztcYOU1DfsTYSJv42N okTd5WK5y1TjFX3HdMy4gJ0pvccjcIuK4WXCl5uvbKFMaos440Jq9SGLI AdkwaP3FXrq2DhhHYUQNfgE0GYyBsvl3XnFUeP3KWtbXppxLQApZguX0T v7hAcwpjwjwndNX6kVO1qYkmARBsrdmABgb4rD1bk7TvhNDznKBI0zler hxTSo91zpuelReQhxkAJMLFrmOabj2QQ1YirIZgSQ+jowleKi5L8gPKsz 0yOQLD+OyLNzzHxNXXbQvfpxbir9n6oaMNFFTN/5rPWSM0p423iALYeaz A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649068" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649068" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543425" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:04 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 14/36] x86/fred: Disallow the swapgs instruction when FRED is enabled Date: Sun, 30 Jul 2023 23:32:55 -0700 Message-Id: <20230731063317.3720-15-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" The FRED architecture establishes the full supervisor/user through: 1) FRED event delivery from ring 3 swaps the value of the GS base address and that of the IA32_KERNEL_GS_BASE MSR. 2) ERETU swaps the value of the GS base address and that of the IA32_KERNEL_GS_BASE MSR. 3) LKGS is already upstreamed and automatically enabled with FRED to load the GS base address directly into the IA32_KERNEL_GS_BASE MSR instead of the GS segment’s descriptor cache. As a result, there is no need to SWAPGS away from the kernel GS base, i.e., the swapgs instruction is no longer needed when FRED is enabled, thus is disallowed. Otherwise it causes #UD. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Explain why writing directly to the IA32_KERNEL_GS_BASE MSR is doing the right thing (Thomas Gleixner). --- arch/x86/kernel/process_64.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 265ab8fcb146..6d5fed29f552 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -166,7 +166,8 @@ static noinstr unsigned long __rdgsbase_inactive(void) lockdep_assert_irqs_disabled(); - if (!cpu_feature_enabled(X86_FEATURE_XENPV)) { + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + !cpu_feature_enabled(X86_FEATURE_XENPV)) { native_swapgs(); gsbase = rdgsbase(); native_swapgs(); @@ -191,7 +192,8 @@ static noinstr void __wrgsbase_inactive(unsigned long gsbase) { lockdep_assert_irqs_disabled(); - if (!cpu_feature_enabled(X86_FEATURE_XENPV)) { + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + !cpu_feature_enabled(X86_FEATURE_XENPV)) { native_swapgs(); wrgsbase(gsbase); native_swapgs(); From patchwork Mon Jul 31 06:32:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 597D3C001DC for ; Mon, 31 Jul 2023 07:03:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231225AbjGaHDw (ORCPT ); Mon, 31 Jul 2023 03:03:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231174AbjGaHDY (ORCPT ); Mon, 31 Jul 2023 03:03:24 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10C4810CC; Mon, 31 Jul 2023 00:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786937; x=1722322937; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YTgEgmA3f1uzY8PacyZ6XmpZcGIgDW5/dB0vDrivilc=; b=lAVTQf24ixp5T1w9ciVKyDsB92RZPT15jaNOCW4k825EFYeIZmxtPpwh GsXDKyGUj3YeNW7L4oaXlqCpvfgY8pgfU2PMLxy8BJiXS5OXYuLFF6Jt7 FC7VnxL8Y4gKw9QqLv5vbg8aEKosOPRYYkWCLHUeTxIXtSOhwll0MHYvD Kyssu0T9v0NN+mgoutjpYdUfOsPT/NBiml87e7/aQ+B6j88oFu8m1jHuT kj5lWZDS+bqh3T8/sN+EbYiPX7Hu2AUWBT93d1w/j03ABzUsGqs9ph7v5 Hbt1PXYiaA7qHsOeo33C1/qkxjZjkXBKc/9rK71b15jSkiCJkWrdGJEQ3 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649104" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649104" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543430" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:05 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 15/36] x86/fred: No ESPFIX needed when FRED is enabled Date: Sun, 30 Jul 2023 23:32:56 -0700 Message-Id: <20230731063317.3720-16-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Because FRED always restores the full value of %rsp, ESPFIX is no longer needed when it's enabled. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/espfix_64.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kernel/espfix_64.c b/arch/x86/kernel/espfix_64.c index 16f9814c9be0..48d133a54f45 100644 --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -106,6 +106,10 @@ void __init init_espfix_bsp(void) pgd_t *pgd; p4d_t *p4d; + /* FRED systems don't need ESPFIX */ + if (cpu_feature_enabled(X86_FEATURE_FRED)) + return; + /* Install the espfix pud into the kernel page directory */ pgd = &init_top_pgt[pgd_index(ESPFIX_BASE_ADDR)]; p4d = p4d_alloc(&init_mm, pgd, ESPFIX_BASE_ADDR); @@ -129,6 +133,10 @@ void init_espfix_ap(int cpu) void *stack_page; pteval_t ptemask; + /* FRED systems don't need ESPFIX */ + if (cpu_feature_enabled(X86_FEATURE_FRED)) + return; + /* We only have to do this once... */ if (likely(per_cpu(espfix_stack, cpu))) return; /* Already initialized */ From patchwork Mon Jul 31 06:32:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CE0DC04A94 for ; Mon, 31 Jul 2023 07:03:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230449AbjGaHDr (ORCPT ); Mon, 31 Jul 2023 03:03:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231172AbjGaHDY (ORCPT ); Mon, 31 Jul 2023 03:03:24 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E1C91712; Mon, 31 Jul 2023 00:02:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786938; x=1722322938; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R/F68dINSJ894CWgsWAeNvE3sYeHzSDBLZY+Pavzuys=; b=lKSYGrEu8T1GGKWNCOhRk0Xue8TyAjY+4Iev4p1UGDxMGV2/jmlBWAcS MmCPCT+dzXqJh3kDhnzMoELkhvcbpkbNgo/BlgKf65ja3dDt+tlDZ2daw T5OuyrfJR6tesxvz29mSMEqoEtDPCBNxmFqrvlHQ/SwTKKgYLmTsDtryc +sVWotjcqC/axyEkzJEysDA62kHSTrKseXTlbzpTIrlDFcN0AFyYJFyYb OrMzr30KnKALnEpnLRWc9Cy/MtGV0erQNxiE3Htyfe6RoEiOczlfrhkTa gckTFOy1ORX5/nMcEkhjrZW3D0tEh86VSuGdFnBZsqo2siHNxFcPDsq+h A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649144" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649144" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543437" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:06 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 16/36] x86/fred: Allow single-step trap and NMI when starting a new task Date: Sun, 30 Jul 2023 23:32:57 -0700 Message-Id: <20230731063317.3720-17-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Entering a new task is logically speaking a return from a system call (exec, fork, clone, etc.). As such, if ptrace enables single stepping a single step exception should be allowed to trigger immediately upon entering user space. This is not optional. NMI should *never* be disabled in user space. As such, this is an optional, opportunistic way to catch errors. Allow single-step trap and NMI when starting a new task, thus once the new task enters user space, single-step trap and NMI are both enabled immediately. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Use high-order 48 bits above the lowest 16 bit SS only when FRED is enabled (Thomas Gleixner). --- arch/x86/kernel/process_64.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 6d5fed29f552..0b47871a6141 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -56,6 +56,7 @@ #include #include #include +#include #ifdef CONFIG_IA32_EMULATION /* Not included via unistd.h */ #include @@ -507,8 +508,18 @@ void x86_gsbase_write_task(struct task_struct *task, unsigned long gsbase) static void start_thread_common(struct pt_regs *regs, unsigned long new_ip, unsigned long new_sp, - unsigned int _cs, unsigned int _ss, unsigned int _ds) + u16 _cs, u16 _ss, u16 _ds) { + /* + * Paranoia: High-order 48 bits above the lowest 16 bit SS are + * discarded by the legacy IRET instruction on all Intel, AMD, + * and Cyrix/Centaur/VIA CPUs, thus can be set unconditionally, + * even when FRED is not enabled. But we choose the safer side + * to use these bits only when FRED is enabled. + */ + const unsigned long ssx_flags = cpu_feature_enabled(X86_FEATURE_FRED) ? + (FRED_SSX_SOFTWARE_INITIATED | FRED_SSX_NMI) : 0; + WARN_ON_ONCE(regs != current_pt_regs()); if (static_cpu_has(X86_BUG_NULL_SEG)) { @@ -522,11 +533,11 @@ start_thread_common(struct pt_regs *regs, unsigned long new_ip, loadsegment(ds, _ds); load_gs_index(0); - regs->ip = new_ip; - regs->sp = new_sp; - regs->cs = _cs; - regs->ss = _ss; - regs->flags = X86_EFLAGS_IF; + regs->ip = new_ip; + regs->sp = new_sp; + regs->csx = _cs; + regs->ssx = _ss | ssx_flags; + regs->flags = X86_EFLAGS_IF | X86_EFLAGS_FIXED; } void From patchwork Mon Jul 31 06:32:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56F51C04FDF for ; Mon, 31 Jul 2023 07:04:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231245AbjGaHET (ORCPT ); Mon, 31 Jul 2023 03:04:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231221AbjGaHDv (ORCPT ); Mon, 31 Jul 2023 03:03:51 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6699B1BDF; Mon, 31 Jul 2023 00:02:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786948; x=1722322948; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1h9fTPNcW28O8bu/nclSFH4GxPR5CW0Rl9/PbASKNuY=; b=QxWDADlZ9vCgE4aVAKNS/AeTeBChHMPRiv2UfhInUOBOBiZRn9VcrOxU NHOLWKuWC0OTEMbis8lTw1C/lC80oHjsZEWMp8e7mJsyzpezjU8G9isfG OORGn5H11MMLUE9v8uxtzjOXWmwFV3mr2fuoDG6db6cD6ErgasS1Ts56/ qD9BebHbfTWG1/dANuykxcrK/9ujx5szNraRUeYuKB6lOplGjwItjInP7 btstGsDPnCwGSS+wdMbCkn1cVeP6Lf/De6ZVyyA/GGrXSE282riN/ndDr L1J54GrEjgTaQqvl2lRLkMmlDOWv0bCxGOg+fGxTunwjzjCFANWCIYWEY Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649178" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649178" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543443" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:07 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 17/36] x86/fred: Define a common function type fred_handler Date: Sun, 30 Jul 2023 23:32:58 -0700 Message-Id: <20230731063317.3720-18-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org FRED event delivery establishes a full supervisor context by saving the essential information about an event to a FRED stack frame, e.g., the faulting linear address of a #PF is saved as event data of a FRED stack frame. Thus a struct pt_regs has all the needed data to handle an event and it's the only input argument of a FRED event handler. Define fred_handler, a common function type used in the FRED event dispatch framework, which makes it easier to find the entry points (via grep), allows the prototype to change if necessary without requiring changing changes everywhere, and makes sure that all the entry points have the proper decorations (currently noinstr, but could change in the future.) Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index d76e681a806f..b45c1bea5b7f 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -68,6 +68,19 @@ #define FRED_SSX_64_BIT_MODE_BIT 57 #define FRED_SSX_64_BIT_MODE _BITUL(FRED_SSX_64_BIT_MODE_BIT) +/* + * FRED event delivery establishes a full supervisor context by + * saving the essential information about an event to a FRED + * stack frame, e.g., the faulting linear address of a #PF is + * saved as event data of a FRED #PF stack frame. Thus a struct + * pt_regs has all the needed data to handle an event and it's + * the only input argument of a FRED event handler. + * + * FRED handlers need to be placed in the noinstr text section. + */ +#define DECLARE_FRED_HANDLER(f) void f (struct pt_regs *regs) +#define DEFINE_FRED_HANDLER(f) noinstr DECLARE_FRED_HANDLER(f) + #ifdef CONFIG_X86_FRED #ifndef __ASSEMBLY__ @@ -97,6 +110,8 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) return fred_info(regs)->edata; } +typedef DECLARE_FRED_HANDLER((*fred_handler)); + #endif /* __ASSEMBLY__ */ #endif /* CONFIG_X86_FRED */ From patchwork Mon Jul 31 06:32:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31D8AC001DC for ; Mon, 31 Jul 2023 07:04:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230425AbjGaHEV (ORCPT ); Mon, 31 Jul 2023 03:04:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231223AbjGaHDv (ORCPT ); Mon, 31 Jul 2023 03:03:51 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A27B10D3; Mon, 31 Jul 2023 00:02:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786949; x=1722322949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=61pP2ZrTnfM4AWC4LzTY02SWj0Yo4OSnGRh0vm8RTJ0=; b=AKeecPh6ZKhoTu3YEFeY8q3jmeeMrJmrdBNVSQwmX4s3mN6arBBJkQaQ NEGDbtl5xFNAVnm9z+sd0NcsYS9Sir4GGDk7c0xyzBWseZW6N3THs7Uck 8VhpCRaSkFwtKzUzOaJJ76fkEUl2Mpsycze5fzKiAcukpEwe7sKSjkq/h NLXhF6I6Qj5xD1Vm8MNtRfec1F+MSLonn1XmT00oq+X+5WWTnlqCfD1AA JsmnStzsx8iKrzEgz0+PzhXXtfTcu4c0rOeC+f+hzxjEw2b55mmSPQ2OH qKwaSfnyRlrjbjWjiTg30U6SEFTUhsVRFUni2z23wIkVfOx01wFjrdZpR g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649212" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649212" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543449" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:09 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 18/36] x86/fred: Add a page fault entry stub for FRED Date: Sun, 30 Jul 2023 23:32:59 -0700 Message-Id: <20230731063317.3720-19-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add a page fault entry stub for FRED. On a FRED system, the faulting address (CR2) is passed on the stack, to avoid the problem of transient state. Thus we get the page fault address from the stack instead of CR2. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 2 ++ arch/x86/mm/fault.c | 18 ++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index b45c1bea5b7f..fb8e7b4f2d38 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -112,6 +112,8 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_page_fault); + #endif /* __ASSEMBLY__ */ #endif /* CONFIG_X86_FRED */ diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index e8711b2cafaf..dd3df092d0f2 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -34,6 +34,7 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include #define CREATE_TRACE_POINTS #include @@ -1495,9 +1496,10 @@ handle_page_fault(struct pt_regs *regs, unsigned long error_code, } } -DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) +static __always_inline void page_fault_common(struct pt_regs *regs, + unsigned int error_code, + unsigned long address) { - unsigned long address = read_cr2(); irqentry_state_t state; prefetchw(¤t->mm->mmap_lock); @@ -1544,3 +1546,15 @@ DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) irqentry_exit(regs, state); } + +DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault) +{ + page_fault_common(regs, error_code, read_cr2()); +} + +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_page_fault) +{ + page_fault_common(regs, regs->orig_ax, fred_event_data(regs)); +} +#endif From patchwork Mon Jul 31 06:33:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2669BC04A94 for ; Mon, 31 Jul 2023 07:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231178AbjGaHEX (ORCPT ); Mon, 31 Jul 2023 03:04:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231230AbjGaHDw (ORCPT ); Mon, 31 Jul 2023 03:03:52 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63A8F1BF0; Mon, 31 Jul 2023 00:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786952; x=1722322952; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=j/3qauZAla4FAGFWojUuQZ7eYbLWgN4OkDYWsZA4A8I=; b=QoESSR6yjKqehqmFxb2gBIKWoJGQSpGC2P5vNOZln6dfoEuJwzn/pBWF sn2hTaeLCBthJ6+hFTFo81ufQq/m62r/yUaqTtZu9rWlpOAQd3SMZgdTl r3KTQH7xh7j5hK6Emmn4BMmrFZlzsyVoBHQBZb+aBCa2P2gg40S/DJtT2 0zcfjPFHGtwT85u8Qzm+h99ks1z7we1Fk6WOw4YuFDB4FUVuX3rVRM3nO NBgMNHDHXDrR9cuqIvYfcQy1QbvOliQYhG2VCGbgn5/osyke/B6ZIiDaF F9EABHH/1G7xSlJkK6beVRbQgStSQtm1nN3vZhnjk8t9pdUR4DQgHqm8d A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649244" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649244" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543456" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:10 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 19/36] x86/fred: Add a debug fault entry stub for FRED Date: Sun, 30 Jul 2023 23:33:00 -0700 Message-Id: <20230731063317.3720-20-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" Add a debug fault entry stub for FRED. On a FRED system, the debug trap status information (DR6) is passed on the stack, to avoid the problem of transient state. Furthermore, FRED transitions avoid a lot of ugly corner cases the handling of which can, and should be, skipped. The FRED debug trap status information saved on the stack differs from DR6 in both stickiness and polarity; it is exactly what debug_read_clear_dr6() returns, and exc_debug_user()/exc_debug_kernel() expect. Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v1: * call irqentry_nmi_{enter,exit}() in both IDT and FRED debug fault kernel handler (Peter Zijlstra). --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/traps.c | 56 +++++++++++++++++++++++++++---------- 2 files changed, 42 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index fb8e7b4f2d38..ad7b79130b1e 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -112,6 +112,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 4a817d20ce3b..b10464966a81 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -47,6 +47,7 @@ #include #include #include +#include #include #include #include @@ -1021,21 +1022,9 @@ static bool notify_debug(struct pt_regs *regs, unsigned long *dr6) return false; } -static __always_inline void exc_debug_kernel(struct pt_regs *regs, - unsigned long dr6) +static __always_inline void debug_kernel_common(struct pt_regs *regs, + unsigned long dr6) { - /* - * Disable breakpoints during exception handling; recursive exceptions - * are exceedingly 'fun'. - * - * Since this function is NOKPROBE, and that also applies to - * HW_BREAKPOINT_X, we can't hit a breakpoint before this (XXX except a - * HW_BREAKPOINT_W on our stack) - * - * Entry text is excluded for HW_BP_X and cpu_entry_area, which - * includes the entry stack is excluded for everything. - */ - unsigned long dr7 = local_db_save(); irqentry_state_t irq_state = irqentry_nmi_enter(regs); instrumentation_begin(); @@ -1063,7 +1052,8 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs, * Catch SYSENTER with TF set and clear DR_STEP. If this hit a * watchpoint at the same time then that will still be handled. */ - if ((dr6 & DR_STEP) && is_sysenter_singlestep(regs)) + if (!cpu_feature_enabled(X86_FEATURE_FRED) && + (dr6 & DR_STEP) && is_sysenter_singlestep(regs)) dr6 &= ~DR_STEP; /* @@ -1091,7 +1081,25 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs, out: instrumentation_end(); irqentry_nmi_exit(regs, irq_state); +} +static __always_inline void exc_debug_kernel(struct pt_regs *regs, + unsigned long dr6) +{ + /* + * Disable breakpoints during exception handling; recursive exceptions + * are exceedingly 'fun'. + * + * Since this function is NOKPROBE, and that also applies to + * HW_BREAKPOINT_X, we can't hit a breakpoint before this (XXX except a + * HW_BREAKPOINT_W on our stack) + * + * Entry text is excluded for HW_BP_X and cpu_entry_area, which + * includes the entry stack is excluded for everything. + */ + unsigned long dr7 = local_db_save(); + + debug_kernel_common(regs, dr6); local_db_restore(dr7); } @@ -1180,6 +1188,24 @@ DEFINE_IDTENTRY_DEBUG_USER(exc_debug) { exc_debug_user(regs, debug_read_clear_dr6()); } + +# ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_debug) +{ + /* + * The FRED debug information saved onto stack differs from + * DR6 in both stickiness and polarity; it is exactly what + * debug_read_clear_dr6() returns. + */ + unsigned long dr6 = fred_event_data(regs); + + if (user_mode(regs)) + exc_debug_user(regs, dr6); + else + debug_kernel_common(regs, dr6); +} +# endif /* CONFIG_X86_FRED */ + #else /* 32 bit does not have separate entry points. */ DEFINE_IDTENTRY_RAW(exc_debug) From patchwork Mon Jul 31 06:33:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39CF1C001DF for ; Mon, 31 Jul 2023 07:04:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231236AbjGaHEy (ORCPT ); Mon, 31 Jul 2023 03:04:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231266AbjGaHEV (ORCPT ); Mon, 31 Jul 2023 03:04:21 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3311C2114; Mon, 31 Jul 2023 00:02:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786974; x=1722322974; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CDXH8O7ex78Bl5bAFBaH8EJ8EpWaGJ2TaZytElzEDrs=; b=fgJbMHWyIAxMx0W0A4zvc5fU565rnjoLyu/2GAB3zs4onEND+V2jI+Z6 qQqDgQgLKVhPnuwcXkJn26sbp0GtnvomvfMYqVuAtL6VmSQjoYTHzCK7y zr20qHMhNR44yQeQAbkeEMslXUZ8LGf62059eD8xZY37xcDVRgZrFf4R8 6Pz5mW6CgT8H6GepeYRGAjvNd2NYlNLK2KkNCpKB0lwIVj7jiVRTqgS/O ozci8LvRMYA0GpR/ABTpYKj7Hk2OyfDS4Qp6HgFmGbzS2higOy6fW/LN1 4EuhKsERBMFdP+LZ2l1VVeWlS5ZzQG28zaLzU3IHCxIoDwIBhcy5H7A1h A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649279" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649279" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543462" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:11 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 20/36] x86/fred: Add a NMI entry stub for FRED Date: Sun, 30 Jul 2023 23:33:01 -0700 Message-Id: <20230731063317.3720-21-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" On a FRED system, NMIs nest both with themselves and faults, transient information is saved into the stack frame, and NMI unblocking only happens when the stack frame indicates that so should happen. Thus, the NMI entry stub for FRED is really quite small... Signed-off-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/nmi.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index ad7b79130b1e..2a7c47dfd733 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -112,6 +112,7 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs) typedef DECLARE_FRED_HANDLER((*fred_handler)); +DECLARE_FRED_HANDLER(fred_exc_nmi); DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index a0c551846b35..f803e2bcd024 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -34,6 +34,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -643,6 +644,24 @@ void nmi_backtrace_stall_check(const struct cpumask *btp) #endif +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_nmi) +{ + /* + * With FRED, CR2 and DR6 are pushed atomically on faults, + * so we don't have to worry about saving and restoring them. + * Breakpoint faults nest, so assume it is OK to leave DR7 + * enabled. + */ + irqentry_state_t irq_state = irqentry_nmi_enter(regs); + + inc_irq_stat(__nmi_count); + default_do_nmi(regs); + + irqentry_nmi_exit(regs, irq_state); +} +#endif + void stop_nmi(void) { ignore_nmis++; From patchwork Mon Jul 31 06:33:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D165AC04FE2 for ; Mon, 31 Jul 2023 07:04:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231274AbjGaHE5 (ORCPT ); Mon, 31 Jul 2023 03:04:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231219AbjGaHEW (ORCPT ); Mon, 31 Jul 2023 03:04:22 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA3762121; Mon, 31 Jul 2023 00:02:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786975; x=1722322975; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dQPbDjEqQWsCazdenr4wm1/cp/YJX7zOnAx3X5niG4Q=; b=Lw2pc903eR3UGc6g01tROhrFb+5GwhmXEOuvGxEvrHRuYXMIEFsUq8WA jg1EOxSF3Ifo0r0yvk29eHO6TH8RecOVlSA1BJGtlYxGh2EOs0ocKHCPn Z5FcCH/kk7OUZbs3+9lPyFjDzQ5/d24AylkpyQbi9L/7dgF5on7qxPAFX h8mORaX57azrlYcnd299iImF5JjlNT4n8n8zBxfsmByEo2Bh8LF9C80ox yF1Kj4aWmcJHdu4t1J8Luyg4mLebj7/stiHFZYROURlTtVaenbddzsmJQ WO05eyx/DLaP2iNgaJm1FwwAFFax4+Tv6WMCQaaU+phJtDK16x1aKQ+FF A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649313" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649313" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543470" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:12 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 21/36] x86/fred: Add a machine check entry stub for FRED Date: Sun, 30 Jul 2023 23:33:02 -0700 Message-Id: <20230731063317.3720-22-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a machine check entry stub for FRED. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v5: * Disallow #DB inside #MCE for robustness sake (Peter Zijlstra). --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/cpu/mce/core.c | 15 +++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 2a7c47dfd733..f559dd9dc4f2 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -115,6 +115,7 @@ typedef DECLARE_FRED_HANDLER((*fred_handler)); DECLARE_FRED_HANDLER(fred_exc_nmi); DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); +DECLARE_FRED_HANDLER(fred_exc_machine_check); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index b8ad5a5b4026..98456e20f155 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -52,6 +52,7 @@ #include #include #include +#include #include "internal.h" @@ -2118,6 +2119,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check) exc_machine_check_user(regs); local_db_restore(dr7); } + +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_machine_check) +{ + unsigned long dr7; + + dr7 = local_db_save(); + if (user_mode(regs)) + exc_machine_check_user(regs); + else + exc_machine_check_kernel(regs); + local_db_restore(dr7); +} +#endif #else /* 32bit unified entry point */ DEFINE_IDTENTRY_RAW(exc_machine_check) From patchwork Mon Jul 31 06:33:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DD19C001DF for ; Mon, 31 Jul 2023 07:05:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231285AbjGaHE7 (ORCPT ); Mon, 31 Jul 2023 03:04:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231172AbjGaHEX (ORCPT ); Mon, 31 Jul 2023 03:04:23 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98D27212B; Mon, 31 Jul 2023 00:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690786976; x=1722322976; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QeZUtDgMAPwFIrdo9TmaG72+ignP6GGIZpxe8B2aw6k=; b=jdf62DHeNx3GUGi4/Rmb3ifKkK2J/LQxQsM73UALjV9lAVzRty0g5hNa ly0iDGcLnrKt45yqiAx9Bn1TE9IfYJ0/aMUVfDFStOzWWjNLb+iDWHK2y cytjav7MSAkJiVmx8Gvi+68JqPdHe/j5p6ewBIeQvsKdPQS/CT0jSR30C VWkiwu+3TO/qLOp+4BGGR6Wut0ARGvGmKvdLog7vQDTiSsZFY28FdOIRO JUoEqwOkwW5YwKthwoLjTO6WtoxrdsZhQqJv8D1faNVAEfsWpBY0iiu+T a5uc7YaYGjid3YJ7bAZYW65tSCVbReeCvTMdbqkx4IRdYvxYJWmrC5I0Y Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649351" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649351" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543474" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:13 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 22/36] x86/fred: Add a double fault entry stub for FRED Date: Sun, 30 Jul 2023 23:33:03 -0700 Message-Id: <20230731063317.3720-23-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The IDT event delivery of a double fault pushes an error code into the orig_ax member of the pt_regs structure, and the error code is passed as the second argument of its C-handler exc_double_fault(), although the pt_regs structure is already passed as the first argument. The existing IDT double fault asm entry code does the following movq ORIG_RAX(%rsp), %rsi /* get error code into 2nd argument*/ movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */ to set the orig_ax member to -1 just before calling the C-handler. X86_TRAP_TS, X86_TRAP_NP, X86_TRAP_SS, X86_TRAP_GP, X86_TRAP_AC and X86_TRAP_CP are all handled in the same way because the IDT event delivery pushes an error code into their stack frame for them. The commit d99015b1abbad ("x86: move entry_64.S register saving out of the macros") introduced the changes to set orig_ax to -1, but I can't see why. Our tests with FRED seem fine if orig_ax is left unchanged instead of set to -1. It's probably cleaner and simpler to remove the second argument from exc_double_fault() while leave orig_ax unchanged to pass the error code inside the first argument, at least on native x86_64. That would be a separate, pre-FRED, patch. For now just add a double fault entry stub for FRED, which simply calls the existing exc_double_fault(). Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/fred.h | 1 + arch/x86/kernel/traps.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index f559dd9dc4f2..bd701ac87528 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -116,6 +116,7 @@ DECLARE_FRED_HANDLER(fred_exc_nmi); DECLARE_FRED_HANDLER(fred_exc_debug); DECLARE_FRED_HANDLER(fred_exc_page_fault); DECLARE_FRED_HANDLER(fred_exc_machine_check); +DECLARE_FRED_HANDLER(fred_exc_double_fault); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index b10464966a81..49dd92458eb0 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -555,6 +555,13 @@ DEFINE_IDTENTRY_DF(exc_double_fault) instrumentation_end(); } +#ifdef CONFIG_X86_FRED +DEFINE_FRED_HANDLER(fred_exc_double_fault) +{ + exc_double_fault(regs, regs->orig_ax); +} +#endif + DEFINE_IDTENTRY(exc_bounds) { if (notify_die(DIE_TRAP, "bounds", regs, 0, From patchwork Mon Jul 31 06:33:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBD79C04A94 for ; Mon, 31 Jul 2023 07:05:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231300AbjGaHF0 (ORCPT ); Mon, 31 Jul 2023 03:05:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231209AbjGaHFK (ORCPT ); Mon, 31 Jul 2023 03:05:10 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB69C2D77; Mon, 31 Jul 2023 00:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787005; x=1722323005; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e41GdB5DEWVT44n/+24ZLrn6jlcg49oHxwygKL8Pbqk=; b=X2mzvXjmolKwlEgNOjaKGfMGowdEtWHeYO4ynZA4/kB93d/kcSSCIrR/ BsO0qIntBIpO9/vaTSdfdHdG1PbM04ulClb8FNGx43QgsZsB6H0+2zvBU OcHZTbTB1Q0aTYrW1tJ/SG8/XmPtkCB9D04nJ0LDPw2y4AiHMXlH7dGk4 f90VLD01jw6MOQEG+7JTOwEC63NKMuWxQD4WpyZPxFbCPpzNiuMK3i7z1 CuhFL2i72E5GGPjpCvjjC9h9NZuvNivl6ObjirpfaUojMl6nC8HJzWoJt 4SAICEzcAp35aysx/lndR4vbWMBEO3Uulwmlvwau4bubSTb+BKUim+6cE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649389" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649389" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543481" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:14 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 23/36] x86/entry: Remove idtentry_sysvec from entry_{32,64}.S Date: Sun, 30 Jul 2023 23:33:04 -0700 Message-Id: <20230731063317.3720-24-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org idtentry_sysvec is really just DECLARE_IDTENTRY defined in , no need to define it separately. Signed-off-by: Xin Li --- arch/x86/entry/entry_32.S | 4 ---- arch/x86/entry/entry_64.S | 8 -------- arch/x86/include/asm/idtentry.h | 2 +- 3 files changed, 1 insertion(+), 13 deletions(-) diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S index 6e6af42e044a..e0f22ad8ff7e 100644 --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -649,10 +649,6 @@ SYM_CODE_START_LOCAL(asm_\cfunc) SYM_CODE_END(asm_\cfunc) .endm -.macro idtentry_sysvec vector cfunc - idtentry \vector asm_\cfunc \cfunc has_error_code=0 -.endm - /* * Include the defines which emit the idt entries which are shared * shared between 32 and 64 bit and emit the __irqentry_text_* markers diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 8069151176f2..44f14b990597 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -438,14 +438,6 @@ SYM_CODE_END(\asmsym) idtentry \vector asm_\cfunc \cfunc has_error_code=1 .endm -/* - * System vectors which invoke their handlers directly and are not - * going through the regular common device interrupt handling code. - */ -.macro idtentry_sysvec vector cfunc - idtentry \vector asm_\cfunc \cfunc has_error_code=0 -.endm - /** * idtentry_mce_db - Macro to generate entry stubs for #MC and #DB * @vector: Vector number diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index cd5c10a74071..6817c0f8e323 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -447,7 +447,7 @@ __visible noinstr void func(struct pt_regs *regs, \ /* System vector entries */ #define DECLARE_IDTENTRY_SYSVEC(vector, func) \ - idtentry_sysvec vector func + DECLARE_IDTENTRY(vector, func) #ifdef CONFIG_X86_64 # define DECLARE_IDTENTRY_MCE(vector, func) \ From patchwork Mon Jul 31 06:33:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333744 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62427C0729B for ; Mon, 31 Jul 2023 07:05:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230438AbjGaHF2 (ORCPT ); Mon, 31 Jul 2023 03:05:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231292AbjGaHFL (ORCPT ); Mon, 31 Jul 2023 03:05:11 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D32442D7C; Mon, 31 Jul 2023 00:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787006; x=1722323006; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=noUm+C4jbhw0mC8rdAqNdlrDhDQPWQ7nu1SRrIUqW1s=; b=YqjGgIpxHd+MyCCKsj/PXXANmFffEjhIol5/Iwsj97J/WB5+w4bjgnfN A448lQRdmWSHBAqncR6wPqQIwvjj7B4Kg6cJ7+QwkytIEpaVUWFtUAd5t xMq9T1ReaTc8L2I/u6ROhDRYnIZKC/xoXosrmo6V6cEQ9e4ize5YXEKeO oxNg1iE7mOgmo5RhOGZegwFldHSrw5YWLM3aIjipGOyqH5kkbXVdQnOg1 IcGOgwkd9Jj1x9Hh/fV1cZa9LkfLQljQ0OEH23/9zrKems8agx+/ikx1L 0tEaNr+uCZOzefE1Ofef1EO2HfQJHLrMUYU8Xp31I0vnGIL1GINsjhtid Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649423" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649423" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543485" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:15 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 24/36] x86/idtentry: Incorporate definitions/declarations of the FRED external interrupt handler type Date: Sun, 30 Jul 2023 23:33:05 -0700 Message-Id: <20230731063317.3720-25-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org FRED operates differently from IDT in terms of interrupt handling. Instead of directly dispatching an interrupt to its handler based on the interrupt vector, FRED requires the software to dispatch an event to its handler based on both the event's type and vector. Therefore, an event dispatch framework must be implemented to facilitate the event-to-handler dispatch process. The FRED event dispatch framework assumes control once an event is delivered, starting from two FRED entry points, after which several event dispatch tables are introduced to facilitate the dispatching. The first level dispatching is event type based, and two tables need to be defined, one for ring 3 event dispatching, and the other for ring 0. The second level dispatching is event vector based, and several tables need to be defined, e.g., an exception handler table for exception dispatching. Handlers in these tables are typically noinstr. However for external interrupt dispatching, irqentry_{enter,exit}() and instrumentation_{begin,end}() can be extracted from respective interrupt handler to the dispatch framework. As a result, FRED external interrupt handlers don't need to be noinstr. Incorporate definitions/declarations of FRED external interrupt handler types into the IDT entry macros. It is probably better to rename idtentry as event_entry. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Put IDTENTRY changes in a separate patch (Thomas Gleixner). --- arch/x86/include/asm/idtentry.h | 91 +++++++++++++++++++++++++++++---- 1 file changed, 82 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 6817c0f8e323..e67d111bf932 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -167,17 +167,22 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) /** * DECLARE_IDTENTRY_IRQ - Declare functions for device interrupt IDT entry - * points (common/spurious) + * points (common/spurious) and their corresponding + * software based dispatch handlers in the non-noinstr + * text section * @vector: Vector number (ignored for C) * @func: Function name of the entry point * - * Maps to DECLARE_IDTENTRY_ERRORCODE() + * Maps to DECLARE_IDTENTRY_ERRORCODE(), plus a dispatch function prototype */ #define DECLARE_IDTENTRY_IRQ(vector, func) \ - DECLARE_IDTENTRY_ERRORCODE(vector, func) + DECLARE_IDTENTRY_ERRORCODE(vector, func); \ + void dispatch_##func(struct pt_regs *regs, unsigned long error_code) /** * DEFINE_IDTENTRY_IRQ - Emit code for device interrupt IDT entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @func: Function name of the entry point * * The vector number is pushed by the low level entry stub and handed @@ -187,6 +192,11 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) * irq_enter/exit_rcu() are invoked before the function body and the * KVM L1D flush request is set. Stack switching to the interrupt stack * has to be done in the function body if necessary. + * + * dispatch_func() is a software based dispatch handler in the non-noinstr + * text section, assuming the irqentry_{enter,exit}() and + * instrumentation_{begin,end}() helpers are invoked in the external + * interrupt dispatch framework before and after dispatch_func(). */ #define DEFINE_IDTENTRY_IRQ(func) \ static void __##func(struct pt_regs *regs, u32 vector); \ @@ -204,31 +214,68 @@ __visible noinstr void func(struct pt_regs *regs, \ irqentry_exit(regs, state); \ } \ \ +__visible void dispatch_##func(struct pt_regs *regs, \ + unsigned long error_code) \ +{ \ + u32 vector = (u32)(u8)error_code; \ + \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_irq_on_irqstack_cond(__##func, regs, vector); \ +} \ + \ static noinline void __##func(struct pt_regs *regs, u32 vector) +/* + * Define a function type system_interrupt_handler as the element type of + * the table system_interrupt_handlers. + * + * System interrupt handlers don't take any interrupt vector number, or + * any interrupt error code as arguments, as a system interrupt handler + * is defined to handle a specific interrupt vector, and no error code + * is defined for external interrupts. It takes only one argument of type + * struct pt_regs *. + */ +#define DECLARE_SYSTEM_INTERRUPT_HANDLER(f) \ + void f (struct pt_regs *regs) +#define DEFINE_SYSTEM_INTERRUPT_HANDLER(f) \ + __visible DECLARE_SYSTEM_INTERRUPT_HANDLER(f) +typedef DECLARE_SYSTEM_INTERRUPT_HANDLER((*system_interrupt_handler)); + /** * DECLARE_IDTENTRY_SYSVEC - Declare functions for system vector entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @vector: Vector number (ignored for C) * @func: Function name of the entry point * - * Declares three functions: + * Declares four functions: * - The ASM entry point: asm_##func * - The XEN PV trap entry point: xen_##func (maybe unused) * - The C handler called from the ASM entry point + * - The C handler used in the system interrupt handler table * - * Maps to DECLARE_IDTENTRY(). + * Maps to DECLARE_IDTENTRY(), plus a dispatch table function prototype */ #define DECLARE_IDTENTRY_SYSVEC(vector, func) \ - DECLARE_IDTENTRY(vector, func) + DECLARE_IDTENTRY(vector, func); \ + DECLARE_SYSTEM_INTERRUPT_HANDLER(dispatch_table_##func) /** * DEFINE_IDTENTRY_SYSVEC - Emit code for system vector IDT entry points + * and their corresponding software based dispatch + * handlers in the non-noinstr text section * @func: Function name of the entry point * * irqentry_enter/exit() and irq_enter/exit_rcu() are invoked before the * function body. KVM L1D flush request is set. * - * Runs the function on the interrupt stack if the entry hit kernel mode + * Runs the function on the interrupt stack if the entry hit kernel mode. + * + * dispatch_table_func() is used to fill the system interrupt handler table + * for system interrupts dispatching, assuming the irqentry_{enter,exit}() + * and instrumentation_{begin,end}() helpers are invoked in the external + * interrupt dispatch framework before and after dispatch_table_func(), + * thus in the non-noinstr text section. */ #define DEFINE_IDTENTRY_SYSVEC(func) \ static void __##func(struct pt_regs *regs); \ @@ -244,11 +291,19 @@ __visible noinstr void func(struct pt_regs *regs) \ irqentry_exit(regs, state); \ } \ \ +DEFINE_SYSTEM_INTERRUPT_HANDLER(dispatch_table_##func) \ +{ \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_sysvec_on_irqstack_cond(__##func, regs); \ +} \ + \ static noinline void __##func(struct pt_regs *regs) /** * DEFINE_IDTENTRY_SYSVEC_SIMPLE - Emit code for simple system vector IDT - * entry points + * entry points and their corresponding + * software based dispatch handlers in + * the non-noinstr text section * @func: Function name of the entry point * * Runs the function on the interrupted stack. No switch to IRQ stack and @@ -256,6 +311,12 @@ static noinline void __##func(struct pt_regs *regs) * * Only use for 'empty' vectors like reschedule IPI and KVM posted * interrupt vectors. + * + * dispatch_table_func() is used to fill the system interrupt handler table + * for system interrupts dispatching, assuming the irqentry_{enter,exit}() + * and instrumentation_{begin,end}() helpers are invoked in the external + * interrupt dispatch framework before and after dispatch_table_func(), + * thus in the non-noinstr text section. */ #define DEFINE_IDTENTRY_SYSVEC_SIMPLE(func) \ static __always_inline void __##func(struct pt_regs *regs); \ @@ -273,6 +334,14 @@ __visible noinstr void func(struct pt_regs *regs) \ irqentry_exit(regs, state); \ } \ \ +DEFINE_SYSTEM_INTERRUPT_HANDLER(dispatch_table_##func) \ +{ \ + __irq_enter_raw(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + __##func (regs); \ + __irq_exit_raw(); \ +} \ + \ static __always_inline void __##func(struct pt_regs *regs) /** @@ -647,7 +716,11 @@ DECLARE_IDTENTRY_SYSVEC(X86_PLATFORM_IPI_VECTOR, sysvec_x86_platform_ipi); #endif #ifdef CONFIG_SMP -DECLARE_IDTENTRY(RESCHEDULE_VECTOR, sysvec_reschedule_ipi); +/* + * Use DECLARE_IDTENTRY_SYSVEC instead of DECLARE_IDTENTRY to add a + * software based dispatch handler declaration for RESCHEDULE_VECTOR. + */ +DECLARE_IDTENTRY_SYSVEC(RESCHEDULE_VECTOR, sysvec_reschedule_ipi); DECLARE_IDTENTRY_SYSVEC(REBOOT_VECTOR, sysvec_reboot); DECLARE_IDTENTRY_SYSVEC(CALL_FUNCTION_SINGLE_VECTOR, sysvec_call_function_single); DECLARE_IDTENTRY_SYSVEC(CALL_FUNCTION_VECTOR, sysvec_call_function); From patchwork Mon Jul 31 06:33:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E6F6C04A94 for ; Mon, 31 Jul 2023 07:05:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231338AbjGaHFa (ORCPT ); Mon, 31 Jul 2023 03:05:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231293AbjGaHFL (ORCPT ); Mon, 31 Jul 2023 03:05:11 -0400 Received: from mgamail.intel.com (unknown [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6804E30C2; Mon, 31 Jul 2023 00:03:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787007; x=1722323007; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X4MZTo9+yUn1bPdeMYVQLWvNe1DyIq6KwUj9R7l5YRg=; b=QocNXTlIXc34bawMb/E6KEk5BOVfTXd5aiGhQ8x+s6OXgz3JN+pMLZDe f0zoordHLjPtlYLmo2qSnu5I8JuVFADeMcZADUIEObyOPDlIgXJ2bj+0X y/kXP2Dz7kBDK1G7S4DGi1lGfJYJq5nOnQqWZoQzaw7k7OAxApxk6wbfQ gXCZEU5IqEki7e+b2zytPuA+AtINAMMpbE91HcN+Xag4SKVD4tQ8aKjIG 4lZ02HJUxwCytys+rVfeko6TqwCUwusgUPTPNGp5Njp5iYnHBz5RQaHEV V2M1D6x7Tn4GDJuIdOqLA+mcjg5fKxlc22Dg18NT+XZYHX6FnfElLqDMu A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="371649458" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="371649458" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:02:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="871543489" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 31 Jul 2023 00:02:16 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 25/36] x86/traps: Add a system interrupt handler table for system interrupt dispatch Date: Sun, 30 Jul 2023 23:33:06 -0700 Message-Id: <20230731063317.3720-26-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731063317.3720-1-xin3.li@intel.com> References: <20230731063317.3720-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" On x86, external interrupts can be categorized into two groups: 1) System interrupts 2) External device interrupts All external device interrupts are directed to the common_interrupt(), which, in turn, dispatches these external device interrupts using a per-CPU external device interrupt dispatch table vector_irq. To handle system interrupts, a system interrupt handler table needs to be introduced. This table enables the direct dispatching of a system interrupt to its corresponding handler. As a result, a software-based dispatch function will be implemented as: void external_interrupt(struct pt_regs *regs) { u8 vector = regs->vector; if (is_system_interrupt(vector)) system_interrupt_handlers[vector_to_sysvec(vector)](regs); else /* external device interrupt */ common_interrupt(regs); } Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Remove junk code that assumes no local APIC on x86_64 (Thomas Gleixner). Changes since v5: * Initialize system_interrupt_handlers with dispatch_table_spurious_interrupt() instead of NULL to get rid of any NULL check (Peter Zijlstra). --- arch/x86/kernel/traps.c | 50 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 49dd92458eb0..e430a8c47931 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1488,6 +1488,56 @@ DEFINE_IDTENTRY_SW(iret_error) } #endif +#ifdef CONFIG_X86_64 + +static void dispatch_table_spurious_interrupt(struct pt_regs *regs) +{ + dispatch_spurious_interrupt(regs, regs->vector); +} + +#define SYSV(x,y) [(x) - FIRST_SYSTEM_VECTOR] = y + +static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { + [0 ... NR_SYSTEM_VECTORS-1] = dispatch_table_spurious_interrupt, +#ifdef CONFIG_SMP + SYSV(RESCHEDULE_VECTOR, dispatch_table_sysvec_reschedule_ipi), + SYSV(CALL_FUNCTION_VECTOR, dispatch_table_sysvec_call_function), + SYSV(CALL_FUNCTION_SINGLE_VECTOR, dispatch_table_sysvec_call_function_single), + SYSV(REBOOT_VECTOR, dispatch_table_sysvec_reboot), +#endif + +#ifdef CONFIG_X86_THERMAL_VECTOR + SYSV(THERMAL_APIC_VECTOR, dispatch_table_sysvec_thermal), +#endif + +#ifdef CONFIG_X86_MCE_THRESHOLD + SYSV(THRESHOLD_APIC_VECTOR, dispatch_table_sysvec_threshold), +#endif + +#ifdef CONFIG_X86_MCE_AMD + SYSV(DEFERRED_ERROR_VECTOR, dispatch_table_sysvec_deferred_error), +#endif + +#ifdef CONFIG_X86_LOCAL_APIC + SYSV(LOCAL_TIMER_VECTOR, dispatch_table_sysvec_apic_timer_interrupt), + SYSV(X86_PLATFORM_IPI_VECTOR, dispatch_table_sysvec_x86_platform_ipi), +# ifdef CONFIG_HAVE_KVM + SYSV(POSTED_INTR_VECTOR, dispatch_table_sysvec_kvm_posted_intr_ipi), + SYSV(POSTED_INTR_WAKEUP_VECTOR, dispatch_table_sysvec_kvm_posted_intr_wakeup_ipi), + SYSV(POSTED_INTR_NESTED_VECTOR, dispatch_table_sysvec_kvm_posted_intr_nested_ipi), +# endif +# ifdef CONFIG_IRQ_WORK + SYSV(IRQ_WORK_VECTOR, dispatch_table_sysvec_irq_work), +# endif + SYSV(SPURIOUS_APIC_VECTOR, dispatch_table_sysvec_spurious_apic_interrupt), + SYSV(ERROR_APIC_VECTOR, dispatch_table_sysvec_error_interrupt), +#endif +}; + +#undef SYSV + +#endif /* CONFIG_X86_64 */ + void __init trap_init(void) { /* Init cpu_entry_area before IST entries are set up */ From patchwork Mon Jul 31 06:40:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6D4AC04E69 for ; Mon, 31 Jul 2023 07:11:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229912AbjGaHLv (ORCPT ); Mon, 31 Jul 2023 03:11:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229666AbjGaHLa (ORCPT ); Mon, 31 Jul 2023 03:11:30 -0400 Received: from mgamail.intel.com (unknown [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02573210C; Mon, 31 Jul 2023 00:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787363; x=1722323363; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=0Vsn6bDzpRkXYLGAG2NP2KMCccOE1uQI/p8GcVukrL0=; b=UwiW2QVw9Ogi5/7ebMqH0f/lzptRFLkpNDCtuT7EMADADnKu9GP67vWk Q6+4QrXcvCfNQV9ycTebE9j0j/OIT4v5tXcKxD5FDWgKlxGgzdjhgbQ7N /ZWb81pvC9JJ71yMdK2o9bnhcBdI3cSgFKBZxcuh15BacXOxe4n3nW7Gp f7t/u5LNR35R7BMr8SmfTm7NK6T8btghj3bX71HHchG0NQ8UniK58HCNu rlTtex5VhQJlfXhixhK/xwnz0hpnwUMiOKUAp6PlyZIHQPZOsJWBCNmts GaY9BjKbnawx+HrJTDYKKubtABuv6FkmQK1tfv0YA2lFpoM09rrEmCAf2 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="348538894" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="348538894" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:08:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="851922998" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="851922998" Received: from unknown (HELO fred..) ([172.25.112.68]) by orsmga004.jf.intel.com with ESMTP; 31 Jul 2023 00:08:56 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 26/36] x86/traps: Add sysvec_install() to install a system interrupt handler Date: Sun, 30 Jul 2023 23:40:25 -0700 Message-Id: <20230731064025.3819-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add sysvec_install() to install a system interrupt handler into both the IDT and system_interrupt_handlers. The latter is used to dispatch system interrupts to their respective handlers when FRED is enabled. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Introduce a macro sysvec_install() to derive the asm handler name from a C handler, which simplifies the code and avoids an ugly typecast (Thomas Gleixner). --- arch/x86/include/asm/traps.h | 19 +++++++++++++++++++ arch/x86/kernel/cpu/acrn.c | 5 +++-- arch/x86/kernel/cpu/mshyperv.c | 16 ++++++++-------- arch/x86/kernel/kvm.c | 2 +- arch/x86/kernel/traps.c | 6 ++++++ drivers/xen/events/events_base.c | 3 ++- 6 files changed, 39 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 47ecfff2c83d..cba3e4dfc329 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -47,4 +47,23 @@ void __noreturn handle_stack_overflow(struct pt_regs *regs, struct stack_info *info); #endif +#ifdef CONFIG_X86_64 +inline void set_sysvec_handler(unsigned int i, system_interrupt_handler func); + +static inline void sysvec_setup_fred(unsigned int vector, system_interrupt_handler func) +{ + BUG_ON(vector < FIRST_SYSTEM_VECTOR); + set_sysvec_handler(vector - FIRST_SYSTEM_VECTOR, func); +} +#else +static inline void sysvec_setup_fred(unsigned int vector, system_interrupt_handler func) +{ +} +#endif + +#define sysvec_install(vector, func) { \ + sysvec_setup_fred(vector, func); \ + alloc_intr_gate(vector, asm_##func); \ +} + #endif /* _ASM_X86_TRAPS_H */ diff --git a/arch/x86/kernel/cpu/acrn.c b/arch/x86/kernel/cpu/acrn.c index 485441b7f030..a879b4b87740 100644 --- a/arch/x86/kernel/cpu/acrn.c +++ b/arch/x86/kernel/cpu/acrn.c @@ -18,6 +18,7 @@ #include #include #include +#include static u32 __init acrn_detect(void) { @@ -26,8 +27,8 @@ static u32 __init acrn_detect(void) static void __init acrn_init_platform(void) { - /* Setup the IDT for ACRN hypervisor callback */ - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_acrn_hv_callback); + /* Install system interrupt handler for ACRN hypervisor callback */ + sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_acrn_hv_callback); x86_platform.calibrate_tsc = acrn_get_tsc_khz; x86_platform.calibrate_cpu = acrn_get_tsc_khz; diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index c7969e806c64..134830a7f575 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -480,19 +481,18 @@ static void __init ms_hyperv_init_platform(void) */ x86_platform.apic_post_init = hyperv_init; hyperv_setup_mmu_ops(); - /* Setup the IDT for hypervisor callback */ - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_hyperv_callback); - /* Setup the IDT for reenlightenment notifications */ + /* Install system interrupt handler for hypervisor callback */ + sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_hyperv_callback); + + /* Install system interrupt handler for reenlightenment notifications */ if (ms_hyperv.features & HV_ACCESS_REENLIGHTENMENT) { - alloc_intr_gate(HYPERV_REENLIGHTENMENT_VECTOR, - asm_sysvec_hyperv_reenlightenment); + sysvec_install(HYPERV_REENLIGHTENMENT_VECTOR, sysvec_hyperv_reenlightenment); } - /* Setup the IDT for stimer0 */ + /* Install system interrupt handler for stimer0 */ if (ms_hyperv.misc_features & HV_STIMER_DIRECT_MODE_AVAILABLE) { - alloc_intr_gate(HYPERV_STIMER0_VECTOR, - asm_sysvec_hyperv_stimer0); + sysvec_install(HYPERV_STIMER0_VECTOR, sysvec_hyperv_stimer0); } # ifdef CONFIG_SMP diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 1cceac5984da..12c799412c5d 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -829,7 +829,7 @@ static void __init kvm_guest_init(void) if (kvm_para_has_feature(KVM_FEATURE_ASYNC_PF_INT) && kvmapf) { static_branch_enable(&kvm_async_pf_enabled); - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_kvm_asyncpf_interrupt); + sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_kvm_asyncpf_interrupt); } #ifdef CONFIG_SMP diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index e430a8c47931..9040c7f01c93 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1536,6 +1536,12 @@ static system_interrupt_handler system_interrupt_handlers[NR_SYSTEM_VECTORS] = { #undef SYSV +void set_sysvec_handler(unsigned int i, system_interrupt_handler func) +{ + BUG_ON(i >= NR_SYSTEM_VECTORS); + system_interrupt_handlers[i] = func; +} + #endif /* CONFIG_X86_64 */ void __init trap_init(void) diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index c7715f8bd452..16d51338e1f8 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #endif @@ -2249,7 +2250,7 @@ static __init void xen_alloc_callback_vector(void) return; pr_info("Xen HVM callback vector for event delivery is enabled\n"); - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_xen_hvm_callback); + sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_xen_hvm_callback); } #else void xen_setup_callback_vector(void) {} From patchwork Mon Jul 31 06:40:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF530C04E69 for ; Mon, 31 Jul 2023 07:12:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231377AbjGaHMS (ORCPT ); Mon, 31 Jul 2023 03:12:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231296AbjGaHMC (ORCPT ); Mon, 31 Jul 2023 03:12:02 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE68B30CB; Mon, 31 Jul 2023 00:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787397; x=1722323397; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=BTa7pFRtc5hN79memTk6tlfkpBhDUr274mj/lMgq9SQ=; b=Yv1w90SrZXZ9nk+W9ghSL2J1rAe8eUVXVgVCPA8+1KH5BH8Q0+gbeafu BTQXAC8QLqBw37L/j9ZL8hSd0YONUMYeXnNbcu9Pqu54e+uqH3wKCUoT9 SHfeiB/x2DHIInv6tuCnqT3t6NVlX2ACLCVQYlPuBeZvf9U70r5rL33Gs Kgpn4VNqArHQJrT4frmmi+P+FzSiRvM2RuR8IRIe6NcNL+0zMUMjirsuQ NyDAyU0FQY/ZwulA3Up8HOXVu4mcD/DI2MXlhN1rB4JuNg/SIATc+IcTA X59ekSdjrPbiHDo2R149TsA4qb5a6eyhhKjov9l65GJ7yn4akA56gNVhp A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432749875" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432749875" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:09:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="731491567" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="731491567" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga007.fm.intel.com with ESMTP; 31 Jul 2023 00:09:26 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 27/36] x86/traps: Add external_interrupt() to dispatch external interrupts Date: Sun, 30 Jul 2023 23:40:55 -0700 Message-Id: <20230731064055.3842-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" external_interrupt() dispatches all external interrupts: it checks if an external interrupt is a system interrupt, if yes it dipatches it through the system_interrupt_handlers table, otherwise to dispatch_common_interrupt(). Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Reword the patch description, which was confusing (Thomas Gleixner). Changes since v5: * Initialize system_interrupt_handlers with dispatch_table_spurious_interrupt() instead of NULL to get rid of a branch (Peter Zijlstra). --- arch/x86/include/asm/traps.h | 2 ++ arch/x86/kernel/traps.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index cba3e4dfc329..48daa78ee88c 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -66,4 +66,6 @@ static inline void sysvec_setup_fred(unsigned int vector, system_interrupt_handl alloc_intr_gate(vector, asm_##func); \ } +int external_interrupt(struct pt_regs *regs); + #endif /* _ASM_X86_TRAPS_H */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 9040c7f01c93..90fdfcccee7a 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1542,6 +1542,24 @@ void set_sysvec_handler(unsigned int i, system_interrupt_handler func) system_interrupt_handlers[i] = func; } +int external_interrupt(struct pt_regs *regs) +{ + unsigned int vector = regs->vector; + unsigned int sysvec = vector - FIRST_SYSTEM_VECTOR; + + if (unlikely(vector < FIRST_EXTERNAL_VECTOR)) { + pr_err("invalid external interrupt vector %d\n", vector); + return -EINVAL; + } + + if (sysvec < NR_SYSTEM_VECTORS) + system_interrupt_handlers[sysvec](regs); + else + dispatch_common_interrupt(regs, vector); + + return 0; +} + #endif /* CONFIG_X86_64 */ void __init trap_init(void) From patchwork Mon Jul 31 06:41:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC669C41513 for ; Mon, 31 Jul 2023 07:12:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229777AbjGaHMV (ORCPT ); Mon, 31 Jul 2023 03:12:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229718AbjGaHMD (ORCPT ); Mon, 31 Jul 2023 03:12:03 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E0F130D4; Mon, 31 Jul 2023 00:09:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787397; x=1722323397; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Aa5CvA7MmnPHyksJQgqEteFVOvaio1lA17Ke5qS+vL0=; b=AbdaPJ+VIc7cPfldf602/AZUYaW7Unuiw0w18eM/ivyOPxxIEicCESIQ zltxFd6hK6QC8vVXIPJL0bLwZQfDiqloZoVV2zxeFEqjG2Zxzc9E4R5xy sT8YBCEJTEX9wZLBTxXEYNYmtSBmYCqMYo/bRkeKJOzb1G6OXTe/1fRee PcqDvC+XK92gmvRDHDApoFBmTy+p9PyrChzVMQjP1aBH4IB5lyfnEOGS3 wUirEAMdbNJysaCYMYFnEAi1CDCsJfUxZjpjorzGOZTBLKO0gH4rxDrCS sTY08frLXJ/RsSTxNvEq4asKuE5jCt1eZ5IACI2V8+hvdE+7wbTExSirp w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432749914" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432749914" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:09:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="731491679" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="731491679" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga007.fm.intel.com with ESMTP; 31 Jul 2023 00:09:35 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 28/36] x86/idtentry: Incorporate declaration/definition of the FRED exception handler type Date: Sun, 30 Jul 2023 23:41:05 -0700 Message-Id: <20230731064105.3857-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The existing IDT exception C-handlers of X86_TRAP_TS, X86_TRAP_NP, X86_TRAP_SS, X86_TRAP_GP, X86_TRAP_AC and X86_TRAP_CP take an error code as the second argument, thus their FRED version handlers simply call the corresponding existing IDT handlers with orig_ax from the pt_regs structure as the second argument. Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/include/asm/idtentry.h | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index e67d111bf932..3b743c3fbe91 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -11,6 +11,7 @@ #include #include +#include #include /** @@ -67,13 +68,16 @@ static __always_inline void __##func(struct pt_regs *regs) /** * DECLARE_IDTENTRY_ERRORCODE - Declare functions for simple IDT entry points + * and their corresponding software based + * dispatch handler * Error code pushed by hardware * @vector: Vector number (ignored for C) * @func: Function name of the entry point * - * Declares three functions: + * Declares four functions: * - The ASM entry point: asm_##func * - The XEN PV trap entry point: xen_##func (maybe unused) + * - The C handler called from the FRED event dispatch framework * - The C handler called from the ASM entry point * * Same as DECLARE_IDTENTRY, but has an extra error_code argument for the @@ -82,14 +86,19 @@ static __always_inline void __##func(struct pt_regs *regs) #define DECLARE_IDTENTRY_ERRORCODE(vector, func) \ asmlinkage void asm_##func(void); \ asmlinkage void xen_asm_##func(void); \ + __visible DECLARE_FRED_HANDLER(fred_##func); \ __visible void func(struct pt_regs *regs, unsigned long error_code) /** * DEFINE_IDTENTRY_ERRORCODE - Emit code for simple IDT entry points + * and their corresponding software based + * dispatch handler * Error code pushed by hardware * @func: Function name of the entry point * - * Same as DEFINE_IDTENTRY, but has an extra error_code argument + * Same as DEFINE_IDTENTRY, but has an extra error_code argument. The + * fred_func() simply calls func() with passing orig_ax as its second + * argument. */ #define DEFINE_IDTENTRY_ERRORCODE(func) \ static __always_inline void __##func(struct pt_regs *regs, \ @@ -106,6 +115,11 @@ __visible noinstr void func(struct pt_regs *regs, \ irqentry_exit(regs, state); \ } \ \ +__visible DEFINE_FRED_HANDLER(fred_##func) \ +{ \ + func(regs, regs->orig_ax); \ +} \ + \ static __always_inline void __##func(struct pt_regs *regs, \ unsigned long error_code) From patchwork Mon Jul 31 06:41:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D754C001DF for ; Mon, 31 Jul 2023 07:12:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231165AbjGaHMX (ORCPT ); Mon, 31 Jul 2023 03:12:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231194AbjGaHME (ORCPT ); Mon, 31 Jul 2023 03:12:04 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 580ED30E2; Mon, 31 Jul 2023 00:09:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787398; x=1722323398; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=mV4jSKbY2d+wZLYFM+7NRxftfBvXd0NC2fevZMSM55Q=; b=mu/aBl+LQaZ69kHgXasqfa9KbuguzEUTvbq9emU7DVr4EB8qENq696hu eQcsRdHD96wQ/fN/MjNtD/JsPXN+BfOXOdwbTRxbzAQkSPIdUo2f4lq/O cmw8NeJwRsy+QMs6CnXN2/XfopC2MqljpAp5bxWZHES+1vEREEFAqNQKz 9N0c+t8g20RVvlaEJzyco8ZPja0n/vAf/mpeqDH5Q9HwWa20ZuJ4//61d gKXWPiZLbAdRI6Va95RP/rWobD6crh5wWAMta8+i+b6/wbafNA5adnIto 6NYY49XbPxoJ6YsdGGZjh7jCRjj1TIkirqAk9OlHM9bvBR2p9VN9E314W A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750007" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750007" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:09:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="731491787" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="731491787" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga007.fm.intel.com with ESMTP; 31 Jul 2023 00:09:50 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 29/36] x86/fred: FRED entry/exit and dispatch code Date: Sun, 30 Jul 2023 23:41:19 -0700 Message-Id: <20230731064119.3870-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" The code to actually handle kernel and event entry/exit using FRED. It is split up into two files thus: - entry_64_fred.S contains the actual entrypoints and exit code, and saves and restores registers. - entry_fred.c contains the two-level event dispatch code for FRED. The first-level dispatch is on the event type, and the second-level is on the event vector. Originally-by: Megha Dey Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Don't do syscall early out in fred_entry_from_user() before there are proper performance numbers and justifications (Thomas Gleixner). * Add the control exception handler to the FRED exception handler table (Thomas Gleixner). * Add ENDBR to the FRED_ENTER asm macro. * Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack. Changes since v1: * Initialize a FRED exception handler to fred_bad_event() instead of NULL if no FRED handler defined for an exception vector (Peter Zijlstra). * Push calling irqentry_{enter,exit}() and instrumentation_{begin,end}() down into individual FRED exception handlers, instead of in the dispatch framework (Peter Zijlstra). --- arch/x86/entry/Makefile | 5 +- arch/x86/entry/entry_64_fred.S | 53 +++++++ arch/x86/entry/entry_fred.c | 220 ++++++++++++++++++++++++++ arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/include/asm/fred.h | 4 + arch/x86/include/asm/idtentry.h | 4 + 6 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 arch/x86/entry/entry_64_fred.S create mode 100644 arch/x86/entry/entry_fred.c diff --git a/arch/x86/entry/Makefile b/arch/x86/entry/Makefile index ca2fe186994b..c93e7f5c2a06 100644 --- a/arch/x86/entry/Makefile +++ b/arch/x86/entry/Makefile @@ -18,6 +18,9 @@ obj-y += vdso/ obj-y += vsyscall/ obj-$(CONFIG_PREEMPTION) += thunk_$(BITS).o +CFLAGS_entry_fred.o += -fno-stack-protector +CFLAGS_REMOVE_entry_fred.o += -pg $(CC_FLAGS_FTRACE) +obj-$(CONFIG_X86_FRED) += entry_64_fred.o entry_fred.o + obj-$(CONFIG_IA32_EMULATION) += entry_64_compat.o syscall_32.o obj-$(CONFIG_X86_X32_ABI) += syscall_x32.o - diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S new file mode 100644 index 000000000000..4ae12d557db3 --- /dev/null +++ b/arch/x86/entry/entry_64_fred.S @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * The actual FRED entry points. + */ + +#include + +#include "calling.h" + + .code64 + .section ".noinstr.text", "ax" + +.macro FRED_ENTER + UNWIND_HINT_END_OF_STACK + ENDBR + PUSH_AND_CLEAR_REGS + movq %rsp, %rdi /* %rdi -> pt_regs */ +.endm + +.macro FRED_EXIT + UNWIND_HINT_REGS + POP_REGS +.endm + +/* + * The new RIP value that FRED event delivery establishes is + * IA32_FRED_CONFIG & ~FFFH for events that occur in ring 3. + * Thus the FRED ring 3 entry point must be 4K page aligned. + */ + .align 4096 + +SYM_CODE_START_NOALIGN(fred_entrypoint_user) + FRED_ENTER + call fred_entry_from_user +SYM_INNER_LABEL(fred_exit_user, SYM_L_GLOBAL) + FRED_EXIT + ERETU +SYM_CODE_END(fred_entrypoint_user) + +.fill fred_entrypoint_kernel - ., 1, 0xcc + +/* + * The new RIP value that FRED event delivery establishes is + * (IA32_FRED_CONFIG & ~FFFH) + 256 for events that occur in + * ring 0, i.e., fred_entrypoint_user + 256. + */ + .org fred_entrypoint_user+256 +SYM_CODE_START_NOALIGN(fred_entrypoint_kernel) + FRED_ENTER + call fred_entry_from_kernel + FRED_EXIT + ERETS +SYM_CODE_END(fred_entrypoint_kernel) diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c new file mode 100644 index 000000000000..1688e7e09370 --- /dev/null +++ b/arch/x86/entry/entry_fred.c @@ -0,0 +1,220 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * This contains the dispatch functions called from the entry point + * assembly. + */ + +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +static DEFINE_FRED_HANDLER(fred_bad_event) +{ + irqentry_state_t irq_state = irqentry_nmi_enter(regs); + + instrumentation_begin(); + + /* Panic on events from a high stack level */ + if (regs->sl > 0) { + pr_emerg("PANIC: invalid or fatal FRED event; event type %u " + "vector %u error 0x%lx aux 0x%lx at %04x:%016lx\n", + regs->type, regs->vector, regs->orig_ax, + fred_event_data(regs), regs->cs, regs->ip); + die("invalid or fatal FRED event", regs, regs->orig_ax); + panic("invalid or fatal FRED event"); + } else { + unsigned long flags = oops_begin(); + int sig = SIGKILL; + + pr_alert("BUG: invalid or fatal FRED event; event type %u " + "vector %u error 0x%lx aux 0x%lx at %04x:%016lx\n", + regs->type, regs->vector, regs->orig_ax, + fred_event_data(regs), regs->cs, regs->ip); + + if (__die("Invalid or fatal FRED event", regs, regs->orig_ax)) + sig = 0; + + oops_end(flags, regs, sig); + } + + instrumentation_end(); + irqentry_nmi_exit(regs, irq_state); +} + +static DEFINE_FRED_HANDLER(fred_exception) +{ + /* + * Exceptions that cannot happen on FRED h/w are set to fred_bad_event(). + */ + static const fred_handler exception_handlers[NUM_EXCEPTION_VECTORS] = { + [0 ... NUM_EXCEPTION_VECTORS-1] = fred_bad_event, + + [X86_TRAP_DE] = exc_divide_error, + [X86_TRAP_DB] = fred_exc_debug, + [X86_TRAP_BP] = exc_int3, + [X86_TRAP_OF] = exc_overflow, + [X86_TRAP_BR] = exc_bounds, + [X86_TRAP_UD] = exc_invalid_op, + [X86_TRAP_NM] = exc_device_not_available, + [X86_TRAP_DF] = fred_exc_double_fault, + [X86_TRAP_TS] = fred_exc_invalid_tss, + [X86_TRAP_NP] = fred_exc_segment_not_present, + [X86_TRAP_SS] = fred_exc_stack_segment, + [X86_TRAP_GP] = fred_exc_general_protection, + [X86_TRAP_PF] = fred_exc_page_fault, + [X86_TRAP_MF] = exc_coprocessor_error, + [X86_TRAP_AC] = fred_exc_alignment_check, + [X86_TRAP_MC] = fred_exc_machine_check, + [X86_TRAP_XF] = exc_simd_coprocessor_error, + [X86_TRAP_CP] = fred_exc_control_protection, + }; + + exception_handlers[regs->vector](regs); +} + +static __always_inline void fred_emulate_trap(struct pt_regs *regs) +{ + regs->orig_ax = 0; + fred_exception(regs); +} + +static __always_inline void fred_emulate_fault(struct pt_regs *regs) +{ + regs->ip -= regs->instr_len; + fred_emulate_trap(regs); +} + +static DEFINE_FRED_HANDLER(fred_sw_interrupt_user) +{ + /* + * In compat mode INT $0x80 (32bit system call) is + * performance-critical. Handle it first. + */ + if (IS_ENABLED(CONFIG_IA32_EMULATION) && + likely(regs->vector == IA32_SYSCALL_VECTOR)) { + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + return do_int80_syscall_32(regs); + } + + /* + * Some software exceptions can also be triggered as + * int instructions, for historical reasons. + */ + switch (regs->vector) { + case X86_TRAP_BP: + case X86_TRAP_OF: + fred_emulate_trap(regs); + break; + default: + regs->vector = X86_TRAP_GP; + fred_emulate_fault(regs); + break; + } +} + +static DEFINE_FRED_HANDLER(fred_other_default) +{ + regs->vector = X86_TRAP_UD; + fred_emulate_fault(regs); +} + +static DEFINE_FRED_HANDLER(fred_syscall) +{ + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + do_syscall_64(regs, regs->orig_ax); +} + +#if IS_ENABLED(CONFIG_IA32_EMULATION) +/* + * Emulate SYSENTER if applicable. This is not the preferred system + * call in 32-bit mode under FRED, rather int $0x80 is preferred and + * exported in the vdso. + */ +static DEFINE_FRED_HANDLER(fred_sysenter) +{ + regs->orig_ax = regs->ax; + regs->ax = -ENOSYS; + do_fast_syscall_32(regs); +} +#else +#define fred_sysenter fred_other_default +#endif + +static DEFINE_FRED_HANDLER(fred_other) +{ + static const fred_handler user_other_handlers[FRED_NUM_OTHER_VECTORS] = + { + /* + * Vector 0 of the other event type is not used + * per FRED spec 5.0. + */ + [0] = fred_other_default, + [FRED_SYSCALL] = fred_syscall, + [FRED_SYSENTER] = fred_sysenter + }; + + user_other_handlers[regs->vector](regs); +} + +static DEFINE_FRED_HANDLER(fred_hw_interrupt) +{ + irqentry_state_t state = irqentry_enter(regs); + + instrumentation_begin(); + external_interrupt(regs); + instrumentation_end(); + irqentry_exit(regs, state); +} + +__visible noinstr void fred_entry_from_user(struct pt_regs *regs) +{ + static const fred_handler user_handlers[FRED_EVENT_TYPE_COUNT] = + { + [EVENT_TYPE_HWINT] = fred_hw_interrupt, + [EVENT_TYPE_RESERVED] = fred_bad_event, + [EVENT_TYPE_NMI] = fred_exc_nmi, + [EVENT_TYPE_SWINT] = fred_sw_interrupt_user, + [EVENT_TYPE_HWFAULT] = fred_exception, + [EVENT_TYPE_SWFAULT] = fred_exception, + [EVENT_TYPE_PRIVSW] = fred_exception, + [EVENT_TYPE_OTHER] = fred_other + }; + + /* + * FRED employs a two-level event dispatch mechanism, with the + * first-level on the type of an event and the second-level on + * its vector. Here is the first-level dispatch for ring 3 events. + */ + user_handlers[regs->type](regs); +} + +__visible noinstr void fred_entry_from_kernel(struct pt_regs *regs) +{ + static const fred_handler kernel_handlers[FRED_EVENT_TYPE_COUNT] = + { + [EVENT_TYPE_HWINT] = fred_hw_interrupt, + [EVENT_TYPE_RESERVED] = fred_bad_event, + [EVENT_TYPE_NMI] = fred_exc_nmi, + [EVENT_TYPE_SWINT] = fred_bad_event, + [EVENT_TYPE_HWFAULT] = fred_exception, + [EVENT_TYPE_SWFAULT] = fred_exception, + [EVENT_TYPE_PRIVSW] = fred_exception, + [EVENT_TYPE_OTHER] = fred_bad_event + }; + + /* + * FRED employs a two-level event dispatch mechanism, with the + * first-level on the type of an event and the second-level on + * its vector. Here is the first-level dispatch for ring 0 events. + */ + kernel_handlers[regs->type](regs); +} diff --git a/arch/x86/include/asm/asm-prototypes.h b/arch/x86/include/asm/asm-prototypes.h index b1a98fa38828..36505b991f88 100644 --- a/arch/x86/include/asm/asm-prototypes.h +++ b/arch/x86/include/asm/asm-prototypes.h @@ -13,6 +13,7 @@ #include #include #include +#include #ifndef CONFIG_X86_CMPXCHG64 extern void cmpxchg8b_emu(void); diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index bd701ac87528..3c91f0eae62e 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -118,6 +118,10 @@ DECLARE_FRED_HANDLER(fred_exc_page_fault); DECLARE_FRED_HANDLER(fred_exc_machine_check); DECLARE_FRED_HANDLER(fred_exc_double_fault); +/* The actual assembly entry point for ring 3 and 0 */ +extern asmlinkage __visible void fred_entrypoint_user(void); +extern asmlinkage __visible void fred_entrypoint_kernel(void); + #endif /* __ASSEMBLY__ */ #endif /* CONFIG_X86_FRED */ diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 3b743c3fbe91..0df3a3cc7e0f 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -661,6 +661,8 @@ DECLARE_IDTENTRY_RAW(X86_TRAP_MC, exc_machine_check); #ifdef CONFIG_XEN_PV DECLARE_IDTENTRY_RAW(X86_TRAP_MC, xenpv_exc_machine_check); #endif +#else +#define fred_exc_machine_check fred_bad_event #endif /* NMI */ @@ -699,6 +701,8 @@ DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault); /* #CP */ #ifdef CONFIG_X86_KERNEL_IBT DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_CP, exc_control_protection); +#else +#define fred_exc_control_protection fred_bad_event #endif /* #VC */ From patchwork Mon Jul 31 06:41:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333806 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAA1EC001DF for ; Mon, 31 Jul 2023 07:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231425AbjGaHMv (ORCPT ); Mon, 31 Jul 2023 03:12:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229666AbjGaHMP (ORCPT ); Mon, 31 Jul 2023 03:12:15 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 652B02D64; Mon, 31 Jul 2023 00:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787415; x=1722323415; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=U03M3kZ8BDHSBlTSXJIlVDmlKpE5VAGPdolj0bAVLvI=; b=hqS7oJxNq6ZZZzeV+NUfK51rzbHMkZOfqLmedvkViRR3x5xA7qmLmkps 17CMtf08yfUiWvthJ/CcBw4vgvHfI4S9w9caabFoP2bOxaLQTUSLH+YB0 xJCyqU8tKrStMFdlZcrXWJoqdfHz3LitJ5dGL82sZOV4fJN/5FH8BRAvu uki/5SwBUOEHnhx9wzHBnNJZvMqX6TuCo8YezwGO2gNw+XaNq9rp6NwiC qnkxYGzCPtTY3WCcN9L5lBAysR3cgv0A8VwcfG+LPnfBfM9TCqnsiVLzo HBMQPq16W338ry5SnkBnIQ4L5PZLGcQAx8TjT6zwMFXZsRexf8JR/+HAO w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750071" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750071" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157778" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157778" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:05 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 30/36] x86/fred: Fixup fault on ERETU by jumping to fred_entrypoint_user Date: Sun, 30 Jul 2023 23:41:27 -0700 Message-Id: <20230731064133.3881-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org If the stack frame contains an invalid user context (e.g. due to invalid SS, a non-canonical RIP, etc.) the ERETU instruction will trap (#SS or #GP). From a Linux point of view, this really should be considered a user space failure, so use the standard fault fixup mechanism to intercept the fault, fix up the exception frame, and redirect execution to fred_entrypoint_user. The end result is that it appears just as if the hardware had taken the exception immediately after completing the transition to user space. Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack. Changes since v6: * Add a comment to explain why it is safe to write to the previous FRED stack frame. (Lai Jiangshan). Changes since v5: * Move the NMI bit from an invalid stack frame, which caused ERETU to fault, to the fault handler's stack frame, thus to unblock NMI ASAP if NMI is blocked (Lai Jiangshan). --- arch/x86/entry/entry_64_fred.S | 5 +- arch/x86/include/asm/extable_fixup_types.h | 4 +- arch/x86/mm/extable.c | 79 ++++++++++++++++++++++ 3 files changed, 86 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index 4ae12d557db3..d24bf7f10ac8 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -3,6 +3,7 @@ * The actual FRED entry points. */ +#include #include #include "calling.h" @@ -34,7 +35,9 @@ SYM_CODE_START_NOALIGN(fred_entrypoint_user) call fred_entry_from_user SYM_INNER_LABEL(fred_exit_user, SYM_L_GLOBAL) FRED_EXIT - ERETU +1: ERETU + + _ASM_EXTABLE_TYPE(1b, fred_entrypoint_user, EX_TYPE_ERETU) SYM_CODE_END(fred_entrypoint_user) .fill fred_entrypoint_kernel - ., 1, 0xcc diff --git a/arch/x86/include/asm/extable_fixup_types.h b/arch/x86/include/asm/extable_fixup_types.h index 991e31cfde94..1585c798a02f 100644 --- a/arch/x86/include/asm/extable_fixup_types.h +++ b/arch/x86/include/asm/extable_fixup_types.h @@ -64,6 +64,8 @@ #define EX_TYPE_UCOPY_LEN4 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(4)) #define EX_TYPE_UCOPY_LEN8 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(8)) -#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ +#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ + +#define EX_TYPE_ERETU 21 #endif diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 271dcb2deabc..0874f29e85ef 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -223,6 +224,80 @@ static bool ex_handler_ucopy_len(const struct exception_table_entry *fixup, return ex_handler_uaccess(fixup, regs, trapnr, fault_address); } +#ifdef CONFIG_X86_FRED +static bool ex_handler_eretu(const struct exception_table_entry *fixup, + struct pt_regs *regs, unsigned long error_code) +{ + struct pt_regs *uregs = (struct pt_regs *) + (regs->sp - offsetof(struct pt_regs, orig_ax)); + unsigned short ss = uregs->ss; + unsigned short cs = uregs->cs; + + /* + * Move the NMI bit from the invalid stack frame, which caused ERETU + * to fault, to the fault handler's stack frame, thus to unblock NMI + * with the fault handler's ERETS instruction ASAP if NMI is blocked. + */ + regs->nmi = uregs->nmi; + + /* + * Sync event information to uregs, i.e., the ERETU return frame, but + * is it safe to write to the ERETU return frame which is just above + * current event stack frame? + * + * The RSP used by FRED to push a stack frame is not the value in %rsp, + * it is calculated from %rsp with the following 2 steps: + * 1) RSP = %rsp - (IA32_FRED_CONFIG & 0x1c0) // Reserve N*64 bytes + * 2) RSP = RSP & ~0x3f // Align to a 64-byte cache line + * when an event delivery doesn't trigger a stack level change. + * + * Here is an example with N*64 (N=1) bytes reserved: + * + * 64-byte cache line ==> ______________ + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| + * 64-byte cache line ==> |__Error_code__| <== ERETU return frame + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * 64-byte cache line ==> |______________| <== RSP after step 1) and 2) + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| + * 64-byte cache line ==> |__Error_code__| <== ERETS return frame + * + * Thus a new FRED stack frame will always be pushed below a previous + * FRED stack frame ((N*64) bytes may be reserved between), and it is + * safe to write to a previous FRED stack frame as they never overlap. + */ + fred_info(uregs)->edata = fred_event_data(regs); + uregs->ssx = regs->ssx; + uregs->ss = ss; + /* The NMI bit was moved away above */ + uregs->nmi = 0; + uregs->csx = regs->csx; + uregs->sl = 0; + uregs->wfe = 0; + uregs->cs = cs; + uregs->orig_ax = error_code; + + return ex_handler_default(fixup, regs); +} +#endif + int ex_get_fixup_type(unsigned long ip) { const struct exception_table_entry *e = search_exception_tables(ip); @@ -300,6 +375,10 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, return ex_handler_ucopy_len(e, regs, trapnr, fault_addr, reg, imm); case EX_TYPE_ZEROPAD: return ex_handler_zeropad(e, regs, fault_addr); +#ifdef CONFIG_X86_FRED + case EX_TYPE_ERETU: + return ex_handler_eretu(e, regs, error_code); +#endif } BUG(); } From patchwork Mon Jul 31 06:41:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF2D5C04FDF for ; Mon, 31 Jul 2023 07:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231372AbjGaHMy (ORCPT ); Mon, 31 Jul 2023 03:12:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231124AbjGaHMQ (ORCPT ); Mon, 31 Jul 2023 03:12:16 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CCF53C13; Mon, 31 Jul 2023 00:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787416; x=1722323416; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QJo+FdezOu/7r5CtwRnG6EayDGJs/agjxSi0oClRDik=; b=EpL73frxbIyK4PWC9dg1wWHyD3sDjHmz5im5yHjY1peTwyx5RfHV3aTG 4p79kW5D5+PmQNgFAuts919Fc5pZzobSGCthBXuBZ8G0vduAiwGKTa5HV OsxMH1NOBoRX4cBTuVP03cNFV9NRXpsf7QZIPbESet/od5WY3kd2+iaRv 3VYjCp4eI/4j1CdN+SIAZHg4q4JPhyAbXx7xB6WQsRXY0SkOsGV1r1/hc UG62zJPrcoUgYYWva+kKLsXv9wTHEKbABvv3UPQ68U6h2CXylN9M3utm4 SFmP0PAZ7ZpGleYywH3HKxo4HoW6XXUH6FpG14o1HtQS9Ky2BxNru2u4d w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750124" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750124" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157781" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157781" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:07 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 31/36] x86/traps: Export external_interrupt() for handling IRQ in IRQ induced VM exits Date: Sun, 30 Jul 2023 23:41:28 -0700 Message-Id: <20230731064133.3881-2-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Export external_interrupt() for handling IRQ in IRQ induced VM exits. Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/traps.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 90fdfcccee7a..6143ad56008e 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1560,6 +1560,11 @@ int external_interrupt(struct pt_regs *regs) return 0; } +#if IS_ENABLED(CONFIG_KVM_INTEL) +/* For KVM VMX to handle IRQs in IRQ induced VM exits. */ +EXPORT_SYMBOL_GPL(external_interrupt); +#endif + #endif /* CONFIG_X86_64 */ void __init trap_init(void) From patchwork Mon Jul 31 06:41:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A369BC04A94 for ; Mon, 31 Jul 2023 07:13:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230161AbjGaHNa (ORCPT ); Mon, 31 Jul 2023 03:13:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231382AbjGaHMS (ORCPT ); Mon, 31 Jul 2023 03:12:18 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58B6D359F; Mon, 31 Jul 2023 00:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787423; x=1722323423; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0k0T0+NBS4NOJOU0xtkOh6eqcDmcObEt/EaW35o1kMc=; b=UmQcSKwC3EMO/CubajJwt74gKWtWPL2GtqkG7H0gEJjykDP0/RDhoKYA Yb3RrQ4mtTk4ZLVopP21ijWMa76BBNi91KKFFJwigYbUMfVqRmb/ogTQJ 5fg8A/U2G5EMVVTuA53zuJxde8DJYz793K4tL8DCeSHkr4gZJlpmlp16T GabUCG55V92LbT7zldScIL8B+UYHcI62wauioi0k6Q3Y7T4BEJ8IojW7K pm8cq5Em/M5N0TsCXdVMV40h/GxS+xD6LEOMrjwv9I2MacxQ4HD8/XrIs 6347JJ8/Kc7x70rx7ISjM15HnXA9ZiwwT7IaIqAh+oVieIpcJQJpFu3XN g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750163" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750163" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157785" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157785" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:08 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 32/36] x86/fred: Export fred_entrypoint_kernel() for handling NMI in NMI induced VM exits Date: Sun, 30 Jul 2023 23:41:29 -0700 Message-Id: <20230731064133.3881-3-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Export fred_entrypoint_kernel() for handling NMI in NMI induced VM exits. Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/entry/entry_64_fred.S | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index d24bf7f10ac8..12063267d2ac 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -4,6 +4,7 @@ */ #include +#include #include #include "calling.h" @@ -54,3 +55,4 @@ SYM_CODE_START_NOALIGN(fred_entrypoint_kernel) FRED_EXIT ERETS SYM_CODE_END(fred_entrypoint_kernel) +EXPORT_SYMBOL(fred_entrypoint_kernel) From patchwork Mon Jul 31 06:41:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333808 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD46CC04E69 for ; Mon, 31 Jul 2023 07:13:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229827AbjGaHN0 (ORCPT ); Mon, 31 Jul 2023 03:13:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231316AbjGaHMT (ORCPT ); Mon, 31 Jul 2023 03:12:19 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B345D3C26; Mon, 31 Jul 2023 00:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787423; x=1722323423; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nRz3qqnddKXYmfF22m/o97+CO0o46xEgUK6odjAvUmY=; b=Sgo4CRgMb+0zOJuQqZBNFyzvFz/NhlGktqACW/W0Utjj+k/OhFu++vmI Pq6jVZk7wE6VvwFcRUpUAXPUBIjipbkhyKX6hqRGuAv72jXxz/vBTN++h 8pR8JRlQTiW5PNz1T3NppJNXPu59u5sYw4AGGEZ7oWSZrczhHv3UgHiCj P26DHMzNcnxkd0cv6eJvyUDcHzZelxzpBSg8lngpGD2gTOjpMCe94w/pp LKtWkOTD3Z+SpFeuEHH1K30eLP8dzxl2teUZg92B1q8zcmejFIH+T1XIz Dumegt2Vowgr/3INoCZ4rxf7uA8LtOwYoiX/oKMw+WZISMTGpgjFfRsDT Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750214" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750214" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157787" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157787" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:09 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 33/36] KVM: VMX: Add VMX_DO_FRED_EVENT_IRQOFF for IRQ/NMI handling Date: Sun, 30 Jul 2023 23:41:30 -0700 Message-Id: <20230731064133.3881-4-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Compared to an IDT stack frame, a FRED stack frame has extra 16 bytes of information pushed at the regular stack top and 8 bytes of error code _always_ pushed at the regular stack bottom, add VMX_DO_FRED_EVENT_IRQOFF to generate FRED stack frames with event type and vector properly set. Thus, IRQ/NMI can be handled with the existing approach when FRED is enabled. For IRQ handling, general purpose registers are pushed to the stack to form a pt_regs structure, which is then used to call external_interrupt(). As a result, IRQ handling no longer re-enters the noinstr code. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Add a new macro VMX_DO_FRED_EVENT_IRQOFF for FRED instead of refactoring VMX_DO_EVENT_IRQOFF (Sean Christopherson). * Do NOT use a trampoline, just LEA+PUSH the return RIP, PUSH the error code, and jump to the FRED kernel entry point for NMI or call external_interrupt() for IRQs (Sean Christopherson). * Call external_interrupt() only when FRED is enabled, and convert the non-FRED handling to external_interrupt() after FRED lands (Sean Christopherson). --- arch/x86/kvm/vmx/vmenter.S | 88 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 19 ++++++-- 2 files changed, 104 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index 07e927d4d099..5ee6a57b59a5 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -2,12 +2,14 @@ #include #include #include +#include #include #include #include #include #include "kvm-asm-offsets.h" #include "run_flags.h" +#include "../../entry/calling.h" #define WORD_SIZE (BITS_PER_LONG / 8) @@ -31,6 +33,80 @@ #define VCPU_R15 __VCPU_REGS_R15 * WORD_SIZE #endif +#ifdef CONFIG_X86_FRED +.macro VMX_DO_FRED_EVENT_IRQOFF branch_insn branch_target nmi=0 + /* + * Unconditionally create a stack frame, getting the correct RSP on the + * stack (for x86-64) would take two instructions anyways, and RBP can + * be used to restore RSP to make objtool happy (see below). + */ + push %_ASM_BP + mov %_ASM_SP, %_ASM_BP + + /* + * Don't check the FRED stack level, the call stack leading to this + * helper is effectively constant and shallow (relatively speaking). + * + * Emulate the FRED-defined redzone and stack alignment. + */ + sub $(FRED_CONFIG_REDZONE_AMOUNT << 6), %rsp + and $FRED_STACK_FRAME_RSP_MASK, %rsp + + /* + * A FRED stack frame has extra 16 bytes of information pushed at the + * regular stack top compared to an IDT stack frame. + */ + push $0 /* Reserved by FRED, must be 0 */ + push $0 /* FRED event data, 0 for NMI and external interrupts */ + + shl $32, %rdi /* FRED event type and vector */ + .if \nmi + bts $FRED_SSX_NMI_BIT, %rdi /* Set the NMI bit */ + .endif + bts $FRED_SSX_64_BIT_MODE_BIT, %rdi /* Set the 64-bit mode */ + or $__KERNEL_DS, %rdi + push %rdi + push %rbp + pushf + mov $__KERNEL_CS, %rax + push %rax + + /* + * Unlike the IDT event delivery, FRED _always_ pushes an error code + * after pushing the return RIP, thus the CALL instruction CANNOT be + * used here to push the return RIP, otherwise there is no chance to + * push an error code before invoking the IRQ/NMI handler. + * + * Use LEA to get the return RIP and push it, then push an error code. + */ + lea 1f(%rip), %rax + push %rax + push $0 /* FRED error code, 0 for NMI and external interrupts */ + + .if \nmi == 0 + PUSH_REGS + mov %rsp, %rdi + .endif + + \branch_insn \branch_target + + .if \nmi == 0 + POP_REGS + .endif + +1: + /* + * "Restore" RSP from RBP, even though IRET has already unwound RSP to + * the correct value. objtool doesn't know the callee will IRET and, + * without the explicit restore, thinks the stack is getting walloped. + * Using an unwind hint is problematic due to x86-64's dynamic alignment. + */ + mov %_ASM_BP, %_ASM_SP + pop %_ASM_BP + RET +.endm +#endif + .macro VMX_DO_EVENT_IRQOFF call_insn call_target /* * Unconditionally create a stack frame, getting the correct RSP on the @@ -299,6 +375,12 @@ SYM_INNER_LABEL_ALIGN(vmx_vmexit, SYM_L_GLOBAL) SYM_FUNC_END(__vmx_vcpu_run) +#ifdef CONFIG_X86_FRED +SYM_FUNC_START(vmx_do_fred_nmi_irqoff) + VMX_DO_FRED_EVENT_IRQOFF jmp fred_entrypoint_kernel nmi=1 +SYM_FUNC_END(vmx_do_fred_nmi_irqoff) +#endif + SYM_FUNC_START(vmx_do_nmi_irqoff) VMX_DO_EVENT_IRQOFF call asm_exc_nmi_kvm_vmx SYM_FUNC_END(vmx_do_nmi_irqoff) @@ -357,6 +439,12 @@ SYM_FUNC_START(vmread_error_trampoline) SYM_FUNC_END(vmread_error_trampoline) #endif +#ifdef CONFIG_X86_FRED +SYM_FUNC_START(vmx_do_fred_interrupt_irqoff) + VMX_DO_FRED_EVENT_IRQOFF call external_interrupt +SYM_FUNC_END(vmx_do_fred_interrupt_irqoff) +#endif + SYM_FUNC_START(vmx_do_interrupt_irqoff) VMX_DO_EVENT_IRQOFF CALL_NOSPEC _ASM_ARG1 SYM_FUNC_END(vmx_do_interrupt_irqoff) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0ecf4be2c6af..4e90c69a92bf 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6890,6 +6890,14 @@ static void vmx_apicv_post_state_restore(struct kvm_vcpu *vcpu) memset(vmx->pi_desc.pir, 0, sizeof(vmx->pi_desc.pir)); } +#ifdef CONFIG_X86_FRED +void vmx_do_fred_interrupt_irqoff(unsigned int vector); +void vmx_do_fred_nmi_irqoff(unsigned int vector); +#else +#define vmx_do_fred_interrupt_irqoff(x) BUG() +#define vmx_do_fred_nmi_irqoff(x) BUG() +#endif + void vmx_do_interrupt_irqoff(unsigned long entry); void vmx_do_nmi_irqoff(void); @@ -6932,14 +6940,16 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) { u32 intr_info = vmx_get_intr_info(vcpu); unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; - gate_desc *desc = (gate_desc *)host_idt_base + vector; if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, "unexpected VM-Exit interrupt info: 0x%x", intr_info)) return; kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); - vmx_do_interrupt_irqoff(gate_offset(desc)); + if (cpu_feature_enabled(X86_FEATURE_FRED)) + vmx_do_fred_interrupt_irqoff(vector); /* Event type is 0 */ + else + vmx_do_interrupt_irqoff(gate_offset((gate_desc *)host_idt_base + vector)); kvm_after_interrupt(vcpu); vcpu->arch.at_instruction_boundary = true; @@ -7225,7 +7235,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, if ((u16)vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI && is_nmi(vmx_get_intr_info(vcpu))) { kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); - vmx_do_nmi_irqoff(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) + vmx_do_fred_nmi_irqoff((EVENT_TYPE_NMI << 16) | NMI_VECTOR); + else + vmx_do_nmi_irqoff(); kvm_after_interrupt(vcpu); } From patchwork Mon Jul 31 06:41:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333812 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CE65C04E69 for ; Mon, 31 Jul 2023 07:14:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229892AbjGaHOz (ORCPT ); Mon, 31 Jul 2023 03:14:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231389AbjGaHMT (ORCPT ); Mon, 31 Jul 2023 03:12:19 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BDBD3C2F; Mon, 31 Jul 2023 00:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787425; x=1722323425; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g76n0qqOm69A/O2uudI0FugS1H+Frufn6xIPQbl7rb0=; b=jMH4gTLRnsxoiPbhogqlJMq1DJBR7igoG12cNvXjV7ZVulpLOKH0o1+W 1Xzl+n3dO50lcZjAbaKCufhe3xHOyPfIpwakzmf+/wwxaYku11GWzJ7yP SJSpyDJwIR9hbUD3oDFWhdN+SW9qxR2WeV/zcUbhXcbMcUyPRHZv7J+gf 17GTZWMZZfvSe/ETEbhTzDhQO/kHTUgDWLVF0yxJy3RJ4oFWFfl6lAxUI 9gke3crME7gla6vaOddT5JFkpM9TXaaLlvmqIfefe5J6kNHrY6D3GEBi/ DYvwMt8SHUWYHSEFXJnepjpPenMKrBPEaybm+8ug13xyjFAx2eFfSehyN A==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750225" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750225" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157799" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157799" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:09 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 34/36] x86/syscall: Split IDT syscall setup code into idt_syscall_init() Date: Sun, 30 Jul 2023 23:41:31 -0700 Message-Id: <20230731064133.3881-5-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Split IDT syscall setup code into idt_syscall_init() to make it cleaner to add FRED syscall setup code. Suggested-by: Thomas Gleixner Tested-by: Shan Kang Signed-off-by: Xin Li --- arch/x86/kernel/cpu/common.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 331b06d19f7f..bb03dacc5fb8 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2027,10 +2027,8 @@ static void wrmsrl_cstar(unsigned long val) wrmsrl(MSR_CSTAR, val); } -/* May not be marked __init: used by software suspend */ -void syscall_init(void) +static inline void idt_syscall_init(void) { - wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS); wrmsrl(MSR_LSTAR, (unsigned long)entry_SYSCALL_64); #ifdef CONFIG_IA32_EMULATION @@ -2064,6 +2062,15 @@ void syscall_init(void) X86_EFLAGS_AC|X86_EFLAGS_ID); } +/* May not be marked __init: used by software suspend */ +void syscall_init(void) +{ + /* The default user and kernel segments */ + wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS); + + idt_syscall_init(); +} + #else /* CONFIG_X86_64 */ #ifdef CONFIG_STACKPROTECTOR From patchwork Mon Jul 31 06:41:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9273BC001DF for ; Mon, 31 Jul 2023 07:13:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231158AbjGaHNj (ORCPT ); Mon, 31 Jul 2023 03:13:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231420AbjGaHMv (ORCPT ); Mon, 31 Jul 2023 03:12:51 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB64A172A; Mon, 31 Jul 2023 00:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787441; x=1722323441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lojZodI3hwQLsEtOWQ3jf2ThLdccjOvBbgNRJld8Dhw=; b=SvtA/vfQB5NX0XRBwdH6ta+Xy62vUzNj40Irn79rF8bqNWocxLqi9/CZ 23U6jXEVjwV+c4pD8AKFS/NRyAWfmnJifqQrBWTqjlGeDovNM4HfRt6ZF lcWIrgrgUgWeEW9Bk4Fu+2iCOY6XZfjUgpo2962dtX26kKkTsXiXynbX8 49MERUUBU+/6Io7JHuc/u2Hn1KNZ5YCscB5S63hGgYMel+3Yqhsh983Hv 1ICxpSacI5sncZJhM+N5fNNt4IjO+7u+6qClUJoJqKnu61kC0YqFVX/fc 6MHiM1cq93FMMQbbWEE7tXzBKfhmZ/keruAvkfFwWH80D66GE0/Lalm9U g==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750231" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750231" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157807" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157807" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:12 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 35/36] x86/fred: FRED initialization code Date: Sun, 30 Jul 2023 23:41:32 -0700 Message-Id: <20230731064133.3881-6-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: "H. Peter Anvin (Intel)" The code to initialize FRED when it's available and _not_ disabled. cpu_init_fred_exceptions() is the core function to initialize FRED, which 1. Sets up FRED entrypoints for events happening in ring 0 and 3. 2. Sets up a default stack for event handling. 3. Sets up dedicated event stacks for DB/NMI/MC/DF, equivalent to the IDT IST stacks. 4. Forces 32-bit system calls to use "int $0x80" only. 5. Enables FRED and invalidtes IDT. When the FRED is used, cpu_init_exception_handling() initializes FRED through calling cpu_init_fred_exceptions(), otherwise it sets up TSS IST and loads IDT. As FRED uses the ring 3 FRED entrypoint for SYSCALL and SYSENTER, it skips setting up SYSCALL/SYSENTER related MSRs, e.g., MSR_LSTAR. Signed-off-by: H. Peter Anvin (Intel) Co-developed-by: Xin Li Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Move this patch after all required changes are in place (Thomas Gleixner). Changes since v5: * Add a comment for FRED stack level settings (Lai Jiangshan). * Define #DB/NMI/#MC/#DF stack levels using macros. --- arch/x86/include/asm/fred.h | 28 ++++++++++++++++ arch/x86/include/asm/traps.h | 4 ++- arch/x86/kernel/Makefile | 1 + arch/x86/kernel/cpu/common.c | 28 +++++++++++++--- arch/x86/kernel/fred.c | 64 ++++++++++++++++++++++++++++++++++++ arch/x86/kernel/irqinit.c | 7 +++- arch/x86/kernel/traps.c | 11 ++++++- 7 files changed, 135 insertions(+), 8 deletions(-) create mode 100644 arch/x86/kernel/fred.c diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 3c91f0eae62e..6031138b778c 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -68,6 +68,19 @@ #define FRED_SSX_64_BIT_MODE_BIT 57 #define FRED_SSX_64_BIT_MODE _BITUL(FRED_SSX_64_BIT_MODE_BIT) +/* #DB in the kernel would imply the use of a kernel debugger. */ +#define FRED_DB_STACK_LEVEL 1 +#define FRED_NMI_STACK_LEVEL 2 +#define FRED_MC_STACK_LEVEL 2 +/* + * #DF is the highest level because a #DF means "something went wrong + * *while delivering an exception*." The number of cases for which that + * can happen with FRED is drastically reduced and basically amounts to + * "the stack you pointed me to is broken." Thus, always change stacks + * on #DF, which means it should be at the highest level. + */ +#define FRED_DF_STACK_LEVEL 3 + /* * FRED event delivery establishes a full supervisor context by * saving the essential information about an event to a FRED @@ -122,8 +135,23 @@ DECLARE_FRED_HANDLER(fred_exc_double_fault); extern asmlinkage __visible void fred_entrypoint_user(void); extern asmlinkage __visible void fred_entrypoint_kernel(void); +void cpu_init_fred_exceptions(void); +void fred_setup_apic(void); + #endif /* __ASSEMBLY__ */ +#else +#ifndef __ASSEMBLY__ +static inline void cpu_init_fred_exceptions(void) +{ + BUG(); +} + +static inline void fred_setup_apic(void) +{ + BUG(); +} +#endif #endif /* CONFIG_X86_FRED */ #endif /* ASM_X86_FRED_H */ diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 48daa78ee88c..da7e8ab1d66d 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -49,6 +49,7 @@ void __noreturn handle_stack_overflow(struct pt_regs *regs, #ifdef CONFIG_X86_64 inline void set_sysvec_handler(unsigned int i, system_interrupt_handler func); +bool is_sysvec_used(unsigned int i); static inline void sysvec_setup_fred(unsigned int vector, system_interrupt_handler func) { @@ -63,7 +64,8 @@ static inline void sysvec_setup_fred(unsigned int vector, system_interrupt_handl #define sysvec_install(vector, func) { \ sysvec_setup_fred(vector, func); \ - alloc_intr_gate(vector, asm_##func); \ + if (!cpu_feature_enabled(X86_FEATURE_FRED)) \ + alloc_intr_gate(vector, asm_##func); \ } int external_interrupt(struct pt_regs *regs); diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 4070a01c11b7..46d8daa11c17 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -48,6 +48,7 @@ obj-y += platform-quirks.o obj-y += process_$(BITS).o signal.o signal_$(BITS).o obj-y += traps.o idt.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o dumpstack.o nmi.o +obj-$(CONFIG_X86_FRED) += fred.o obj-$(CONFIG_MODIFY_LDT_SYSCALL) += ldt.o obj-y += setup.o x86_init.o i8259.o irqinit.o obj-$(CONFIG_JUMP_LABEL) += jump_label.o diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index bb03dacc5fb8..b34a8a138755 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -62,6 +62,7 @@ #include #include #include +#include #include #include #include @@ -2062,13 +2063,24 @@ static inline void idt_syscall_init(void) X86_EFLAGS_AC|X86_EFLAGS_ID); } +static inline void fred_syscall_init(void) +{ + /* Both sysexit and sysret cause #UD when FRED is enabled */ + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); +} + /* May not be marked __init: used by software suspend */ void syscall_init(void) { /* The default user and kernel segments */ wrmsr(MSR_STAR, 0, (__USER32_CS << 16) | __KERNEL_CS); - idt_syscall_init(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) + fred_syscall_init(); + else + idt_syscall_init(); } #else /* CONFIG_X86_64 */ @@ -2184,8 +2196,6 @@ void cpu_init_exception_handling(void) /* paranoid_entry() gets the CPU number from the GDT */ setup_getcpu(cpu); - /* IST vectors need TSS to be set up. */ - tss_setup_ist(tss); tss_setup_io_bitmap(tss); set_tss_desc(cpu, &get_cpu_entry_area(cpu)->tss.x86_tss); @@ -2194,8 +2204,16 @@ void cpu_init_exception_handling(void) /* GHCB needs to be setup to handle #VC. */ setup_ghcb(); - /* Finally load the IDT */ - load_current_idt(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* Set up FRED exception handling */ + cpu_init_fred_exceptions(); + } else { + /* IST vectors need TSS to be set up. */ + tss_setup_ist(tss); + + /* Finally load the IDT */ + load_current_idt(); + } } /* diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c new file mode 100644 index 000000000000..7fdf79c964a8 --- /dev/null +++ b/arch/x86/kernel/fred.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include + +#include +#include +#include +#include + +void cpu_init_fred_exceptions(void) +{ + wrmsrl(MSR_IA32_FRED_CONFIG, + /* Reserve for CALL emulation */ + FRED_CONFIG_REDZONE | + FRED_CONFIG_INT_STKLVL(0) | + FRED_CONFIG_ENTRYPOINT(fred_entrypoint_user)); + + /* + * The purpose of separate stacks for NMI, #DB and #MC *in the kernel* + * (remember that user space faults are always taken on stack level 0) + * is to avoid overflowing the kernel stack. + */ + wrmsrl(MSR_IA32_FRED_STKLVLS, + FRED_STKLVL(X86_TRAP_DB, FRED_DB_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_NMI, FRED_NMI_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_MC, FRED_MC_STACK_LEVEL) | + FRED_STKLVL(X86_TRAP_DF, FRED_DF_STACK_LEVEL)); + + /* The FRED equivalents to IST stacks... */ + wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB)); + wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI)); + wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF)); + + /* Not used with FRED */ + wrmsrl(MSR_LSTAR, 0ULL); + wrmsrl(MSR_CSTAR, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG); + wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL); + + /* Enable FRED */ + cr4_set_bits(X86_CR4_FRED); + /* Any further IDT use is a bug */ + idt_invalidate(); + + /* Use int $0x80 for 32-bit system calls in FRED mode */ + setup_clear_cpu_cap(X86_FEATURE_SYSENTER32); + setup_clear_cpu_cap(X86_FEATURE_SYSCALL32); +} + +/* + * Initialize system vectors from a FRED perspective, so + * lapic_assign_system_vectors() can do its job. + */ +void __init fred_setup_apic(void) +{ + int i; + + for (i = 0; i < FIRST_EXTERNAL_VECTOR; i++) + set_bit(i, system_vectors); + + for (i = 0; i < NR_SYSTEM_VECTORS; i++) + if (is_sysvec_used(i)) + set_bit(i + FIRST_SYSTEM_VECTOR, system_vectors); +} diff --git a/arch/x86/kernel/irqinit.c b/arch/x86/kernel/irqinit.c index c683666876f1..2a510f72dd11 100644 --- a/arch/x86/kernel/irqinit.c +++ b/arch/x86/kernel/irqinit.c @@ -28,6 +28,7 @@ #include #include #include +#include #include /* @@ -96,7 +97,11 @@ void __init native_init_IRQ(void) /* Execute any quirks before the call gates are initialised: */ x86_init.irqs.pre_vector_init(); - idt_setup_apic_and_irq_gates(); + if (cpu_feature_enabled(X86_FEATURE_FRED)) + fred_setup_apic(); + else + idt_setup_apic_and_irq_gates(); + lapic_assign_system_vectors(); if (!acpi_ioapic && !of_ioapic && nr_legacy_irqs()) { diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 6143ad56008e..21eeba7b188f 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1542,6 +1542,12 @@ void set_sysvec_handler(unsigned int i, system_interrupt_handler func) system_interrupt_handlers[i] = func; } +bool is_sysvec_used(unsigned int i) +{ + BUG_ON(i >= NR_SYSTEM_VECTORS); + return system_interrupt_handlers[i] != dispatch_table_spurious_interrupt; +} + int external_interrupt(struct pt_regs *regs) { unsigned int vector = regs->vector; @@ -1577,7 +1583,10 @@ void __init trap_init(void) /* Initialize TSS before setting up traps so ISTs work */ cpu_init_exception_handling(); + /* Setup traps as cpu_init() might #GP */ - idt_setup_traps(); + if (!cpu_feature_enabled(X86_FEATURE_FRED)) + idt_setup_traps(); + cpu_init(); } From patchwork Mon Jul 31 06:41:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 13333810 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D32E3C001DC for ; Mon, 31 Jul 2023 07:13:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231130AbjGaHNg (ORCPT ); Mon, 31 Jul 2023 03:13:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231348AbjGaHMu (ORCPT ); Mon, 31 Jul 2023 03:12:50 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D48834233; Mon, 31 Jul 2023 00:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787442; x=1722323442; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gEY4ZGGaaQpe3vscglEO/Z/ZNgQa2ejE/Ek77q9vAj0=; b=jPwwv3fJ7pH//o7VOsuljtRrYcYDxNc2+pMYZ40uHoi4AWgNJcrIURWN W5o/nZ0truTZHU4seKuLlTz5s2EJ9isHD3dhJOBvRi9OE3N+lE8qGOhcp IJaziGlvZAozZl3KASINswr9oof0kyn4ODle4D0i/bBXbO2Q+2oAukp3z N1kKebduF0BOCpg8MZMEP8Q05x5TybghaG2kAD7aAwEh4IiA9aDLg6njO a2Cbr/qdJx9O9JqGecnKYp2/TH4qILXY/fX2t3nfAR2rDeoISoh7qGvbs zPQ8sq6kh7Ck6lhpClm7ey3KAcgBpH43Q9lUSYuPKBAewL2Hb9dU3nslh w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750289" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750289" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157816" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157816" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:14 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 36/36] x86/fred: Disable FRED by default in its early stage Date: Sun, 30 Jul 2023 23:41:33 -0700 Message-Id: <20230731064133.3881-7-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230731064133.3881-1-xin3.li@intel.com> References: <20230731064133.3881-1-xin3.li@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Disable FRED by default in its early stage. To enable FRED, a new kernel command line option "fred" needs to be added. Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v7: * Add a log message when FRED is enabled. --- Documentation/admin-guide/kernel-parameters.txt | 4 ++++ arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/fred.c | 3 +++ 3 files changed, 10 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index a1457995fd41..cb12decfcdc0 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1513,6 +1513,10 @@ Warning: use of this parameter will taint the kernel and may cause unknown problems. + fred + Forcefully enable flexible return and event delivery, + which is otherwise disabled by default. + ftrace=[tracer] [FTRACE] will set and start the specified tracer as early as possible in order to facilitate early diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index b34a8a138755..38cf4f64a56e 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1451,6 +1451,9 @@ static void __init cpu_parse_early_param(void) char *argptr = arg, *opt; int arglen, taint = 0; + if (!cmdline_find_option_bool(boot_command_line, "fred")) + setup_clear_cpu_cap(X86_FEATURE_FRED); + #ifdef CONFIG_X86_32 if (cmdline_find_option_bool(boot_command_line, "no387")) #ifdef CONFIG_MATH_EMULATION diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c index 7fdf79c964a8..a4a726ea9fc2 100644 --- a/arch/x86/kernel/fred.c +++ b/arch/x86/kernel/fred.c @@ -8,6 +8,9 @@ void cpu_init_fred_exceptions(void) { + /* When FRED is enabled by default, this log message may not needed */ + pr_info("Initialize FRED on CPU%d\n", smp_processor_id()); + wrmsrl(MSR_IA32_FRED_CONFIG, /* Reserve for CALL emulation */ FRED_CONFIG_REDZONE |