From patchwork Wed Sep 13 20:27:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C950EE020B for ; Wed, 13 Sep 2023 20:28:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601582.937626 (Exim 4.92) (envelope-from ) id 1qgWTH-00058q-RK; Wed, 13 Sep 2023 20:28:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601582.937626; Wed, 13 Sep 2023 20:28:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTH-00058j-Nb; Wed, 13 Sep 2023 20:28:23 +0000 Received: by outflank-mailman (input) for mailman id 601582; Wed, 13 Sep 2023 20:28:22 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTF-0004dA-Vi for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:22 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 126b00d6-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:20 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 126b00d6-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636900; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7dZ3kUnhKwuOh+sk9OEhdNHhMXUmtOUWD4tuDVNnFdc=; b=SdbD0hC5bbVQO4rz2172/WJNCXEnQTC8BuK5LVhW4td1bVjyBGkn0aqD yZpMiLAyAng1ssaC7/keOuTyygcBRaGZUy3+uxTlY0KyYAwLQa5MHSN1z 8W0MLntL3QzKZ1BqZ/sxy3pJPRrcRK+PhmKXFIW8cyVMbjY5MAlVaHfG4 A=; X-CSE-ConnectionGUID: OTY/1Nd8T/+vvQCCeXLd1A== X-CSE-MsgGUID: 8f5KVhqHTNyPCN9Cu/9H9g== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213279 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:Q5MTA6rNRozPK3QSr6fL2jHl49VeBmIOZRIvgKrLsJaIsI4StFCzt garIBnUOKvcZTTye9lyPomy8EMAvJLRyYcwHQA6rCtnQilBoJuZCYyVIHmrMnLJJKUvbq7FA +Y2MYCccZ9uHhcwgj/3b9ANeFEljfngqoLUUbOCYmYpA1Y8FE/NsDo788YhmIlknNOlNA2Ev NL2sqX3NUSsnjV5KQr40YrawP9UlKq04GlwUmAWP6gR5wePziJNV/rzGInqR5fGatgMdgKFb 76rIIGRpgvx4xorA9W5pbf3GmVirmn6ZFXmZtJ+AsBOszAazsAA+v9T2Mk0MC+7vw6hjdFpo OihgLTrIesf0g8gr8xGO/VQO3kW0aSrY9YrK1Dn2SCY5xWun3cBX5yCpaz5VGEV0r8fPI1Ay RAXABcxX1faoe7m/PX4VPdVre8uHO3nZapK7xmMzRmBZRonaZXKQqGM7t5ExjYgwMtJGJ4yZ eJAN2ApNk6ZJUQSZBFOUslWcOSA3xETdxVxrl6PqLVxyG/U1AFri5DmMcbPe8zMTsJQ9qqdj jucpj6gWUtFZbRzzxLV/C+PuO/WvR/dSbo5MuyT67k7oAyckzl75Bo+CgLg/KjRZlSFc8JSL QkY9zQjqYA29Ve3VZ/tUhugunmGsxUAHd1KHIUSyiuA167V6AaxHXUfQ3hKb9lOnNAybSwn0 BmOhdyBONB0mOTLEzTHrO7S9G7sf3FPdgfueBPoUyMi3+ik+ac/3i6IDch9KYWeg+3RFWjZl mXiQDcFu50fissC1qOe9F/Bgi6xqpWhcjPZ9jk7TUr+sFonOdfNi5iArAGCsK0edNrxokyp5 iBspiSI0AwZ4XhhfgSpSf5FIrym7u3t3Nb00Q82RMlJG9hAFheekWFsDNNWfhcB3iUsI2WBj KrvVeR5vcU7AZdSRfUrC79d8uxzpUQaKfzrV+rPcv1FaYVreQmM8UlGPBDBgz62yBBwz/xha P93lPpA6l5AVcxaIMeeHb9BgdfHOAhgrY8seXwL50v+iufPDJJkYbwELEGPfogEAFCs+W3oH yJkH5LSkX13CbSuChQ7BKZPdTjm21BnX8GpwyGWH8bfSjdb9JYJV6OOme54K9w990mX/8+Rl kyAtoZj4AKXrRX6xc+iMBiPtJuHsU5DkE8G IronPort-HdrOrdr: A9a23:1aMZqq5UXd20tBoalAPXwPDXdLJyesId70hD6qhwISY6TiX+rb HWoB17726TtN9/YhEdcLy7VJVoBEmskKKdgrNhWotKPjOW21dARbsKheCJrgEIWReOktK1vZ 0QC5SWY+eQMbEVt6nHCXGDYrQd/OU= X-Talos-CUID: 9a23:yjT9uG4a60us0e99y9ss03IlXZkDfi3hizTOPXaTImtZbLHMYArF X-Talos-MUID: 9a23:ozwCrgx3S4h1z2qMDPUg5nWmINyaqPSgVWMzkYwmh++VDQdWahyAy2WXE4Byfw== X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213279" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 1/8] x86/spec-ctrl: Fix confusion between SPEC_CTRL_EXIT_TO_XEN{,_IST} Date: Wed, 13 Sep 2023 21:27:51 +0100 Message-ID: <20230913202758.508225-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 c/s 3fffaf9c13e9 ("x86/entry: Avoid using alternatives in NMI/#MC paths") dropped the only user, leaving behind the (incorrect) implication that Xen had split exit paths. Delete the unused SPEC_CTRL_EXIT_TO_XEN and rename SPEC_CTRL_EXIT_TO_XEN_IST to SPEC_CTRL_EXIT_TO_XEN for consistency. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 8 +------- xen/arch/x86/x86_64/entry.S | 2 +- 2 files changed, 2 insertions(+), 8 deletions(-) base-commit: 6aa25c32180ab59081c73bae4c568367d9133a1f diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index f48f9e75e8dc..14ec40e8d32f 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -67,7 +67,6 @@ * - SPEC_CTRL_ENTRY_FROM_PV * - SPEC_CTRL_ENTRY_FROM_INTR * - SPEC_CTRL_ENTRY_FROM_INTR_IST - * - SPEC_CTRL_EXIT_TO_XEN_IST * - SPEC_CTRL_EXIT_TO_XEN * - SPEC_CTRL_EXIT_TO_PV * @@ -256,11 +255,6 @@ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), \ X86_FEATURE_SC_MSR_PV -/* Use when exiting to Xen context. */ -#define SPEC_CTRL_EXIT_TO_XEN \ - ALTERNATIVE "", \ - DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV - /* Use when exiting to PV guest context. */ #define SPEC_CTRL_EXIT_TO_PV \ ALTERNATIVE "", \ @@ -328,7 +322,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): .endm /* Use when exiting to Xen in IST context. */ -.macro SPEC_CTRL_EXIT_TO_XEN_IST +.macro SPEC_CTRL_EXIT_TO_XEN /* * Requires %rbx=stack_end * Clobbers %rax, %rcx, %rdx diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 81dd2c74b876..a1c860f56949 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -673,7 +673,7 @@ UNLIKELY_START(ne, exit_cr3) UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN_IST /* Req: %rbx=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %rbx=end, Clob: acd */ RESTORE_ALL adj=8 iretq From patchwork Wed Sep 13 20:27:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2DFA8EE020D for ; Wed, 13 Sep 2023 20:28:49 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601590.937685 (Exim 4.92) (envelope-from ) id 1qgWTN-0006Wj-SX; Wed, 13 Sep 2023 20:28:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601590.937685; Wed, 13 Sep 2023 20:28:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTN-0006Tp-G3; Wed, 13 Sep 2023 20:28:29 +0000 Received: by outflank-mailman (input) for mailman id 601590; Wed, 13 Sep 2023 20:28:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTM-0004dA-0e for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:28 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 1768456a-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:26 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1768456a-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636905; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jx4SOK+9S/IjqodWIlucBKoLHCAUVljbJYSnUX0zZyE=; b=KqoALMNqnBHPFSEGzbk7Z0Yy0R6I55TxlI4L7yUE634VynszxbZ1fW5c Jkq3KOfXLzcJNRqCRcMpJ3Etdo0hvZ8DSVggPgKgtZ7zhc59DR3oatP2M ALqe9jcaNQOx3HB8ypV3Sx+I2TAQiU8JonG6ca4qHXfIPQj0hUGTdmD94 w=; X-CSE-ConnectionGUID: tqLoj7ejQSK+P4DSBC/0tA== X-CSE-MsgGUID: Ve2Ajt9KQAWORAnfabEMKg== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125280891 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:9AOFYqwl9Z0HFCInghd6t+chxirEfRIJ4+MujC+fZmUNrF6WrkUEz WcbXGyGb/iCNDOhL9h2O4yzpB9Sup6HyIMxGwVtpCAxQypGp/SeCIXCJC8cHc8wwu7rFxs7s ppEOrEsCOhuExcwcz/0auCJQUFUjPzOHvykTrecZkidfCc8IA85kxVvhuUltYBhhNm9Emult Mj75sbSIzdJ4RYtWo4vw/zF8EkHUMja4mtC5QRvPKsT5zcyqlFOZH4hDfDpR5fHatE88t6SH 47r0Ly/92XFyBYhYvvNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ai87XAME0e0ZP4whlqvgqo Dl7WT5cfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQq2pYjqhljJBheAGEWxgp4KX1X0 fsadz0PVyugxNrq7ImRYa40mu12eaEHPKtH0p1h5TTQDPJgSpHfWaTao9Rf2V/chOgXQ6yYP ZBAL2MyMlKZOUYn1lQ/UfrSmM+BgHXlfiIeg1WSvactuEDYzRBr0airO93QEjCPbZwPwxzI/ jKXpgwVBDkkOYa2yybU0UuFucj1tCX1UYg/MZ63o6sCbFq7mTVIVUx+uUGAiem0jAuyVsxSL 2QQ+zEytu4i+UqzVN7/Uhak5nmesXY0efBdDuk74wGl0bfP7kCSAW1sZiFFQMwrsokxXzNC6 7OSt4q3X3o16uTTEC/DsO7O9lteJBT5M0cefAsDT1QI4eDYg8IL0j70ZP0kSLKc24id9S7L/ xiGqy03hrM2hMEN1rmm8V2vvw9AtqQlXSZuuFyJAzvNAhdRIdf8Otf2sQSzAeNodt7xc7WXg JQTdyFyBsgqBIrFqiGCSf5l8FqBt6fca220bbKC8vAcG9WRF5yLJ9A4DNJWfh0B3iM4ldjBO RW7hO+pzMUPVEZGlIcuC25LN+wkzLL7CfPuXe3OY9xFb/BZLVHWoH81PRbMgT2zwSDAdJ3T3 r/BLa6R4YsyU/w7nFJauc9DuVPU+szO7TyKHs2qp/hW+bGfeGSUWd843KimN4gEAFe/iFyNq b53bpLaoyizpcWiOkE7B6ZPdwFVRZX6bLiqw/FqmhmrflY5QDx4W6+JqV7jEqQ895loei7z1 inVcidlJJDX1BUr9S3ihqhfVY7S IronPort-HdrOrdr: A9a23:zYTrlK1FUvlG6PzvXu3uKQqjBIokLtp133Aq2lEZdPRUGvb3qy nIpoVj6faUskd2ZJhOo7C90cW7LU80sKQFhLX5Xo3SOzUO2lHYT72KhLGKq1aLdhEWtNQtsZ uIG5IOcOEYZmIasS+V2maF+q4bsbu6zJw= X-Talos-CUID: 9a23:CoiE/GrCSoCEODMpS+izFZXmUdw8Vnrfk3rSGnSbSl41ZIa/YhjI5Ioxxg== X-Talos-MUID: 9a23:/ObK0A0TCPcZP15Y9NPuwlvItTUj3an0IUIRwLY/kvajDwV2NT6XoRGUTdpy X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="125280891" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 2/8] x86/spec-ctrl: Fold DO_SPEC_CTRL_EXIT_TO_XEN into it's single user Date: Wed, 13 Sep 2023 21:27:52 +0100 Message-ID: <20230913202758.508225-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 With the SPEC_CTRL_EXIT_TO_XEN{,_IST} confusion fixed, it's now obvious that there's only a single EXIT_TO_XEN path. Fold DO_SPEC_CTRL_EXIT_TO_XEN into SPEC_CTRL_EXIT_TO_XEN to simplify further fixes. When merging labels, switch the name to .L\@_skip_sc_msr as "skip" on its own is going to be too generic shortly. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 40 ++++++++++-------------- 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 14ec40e8d32f..dd7c5e8bfc79 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -199,27 +199,6 @@ wrmsr .endm -.macro DO_SPEC_CTRL_EXIT_TO_XEN -/* - * Requires %rbx=stack_end - * Clobbers %rax, %rcx, %rdx - * - * When returning to Xen context, look to see whether SPEC_CTRL shadowing is - * in effect, and reload the shadow value. This covers race conditions which - * exist with an NMI/MCE/etc hitting late in the return-to-guest path. - */ - xor %edx, %edx - - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) - jz .L\@_skip - - mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax - mov $MSR_SPEC_CTRL, %ecx - wrmsr - -.L\@_skip: -.endm - .macro DO_SPEC_CTRL_EXIT_TO_GUEST /* * Requires %eax=spec_ctrl, %rsp=regs/cpuinfo @@ -328,11 +307,24 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): * Clobbers %rax, %rcx, %rdx */ testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) - jz .L\@_skip + jz .L\@_skip_sc_msr - DO_SPEC_CTRL_EXIT_TO_XEN + /* + * When returning to Xen context, look to see whether SPEC_CTRL shadowing + * is in effect, and reload the shadow value. This covers race conditions + * which exist with an NMI/MCE/etc hitting late in the return-to-guest + * path. + */ + xor %edx, %edx -.L\@_skip: + testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + jz .L\@_skip_sc_msr + + mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax + mov $MSR_SPEC_CTRL, %ecx + wrmsr + +.L\@_skip_sc_msr: .endm #endif /* __ASSEMBLY__ */ From patchwork Wed Sep 13 20:27:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17D54EE020D for ; Wed, 13 Sep 2023 20:28:41 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601591.937703 (Exim 4.92) (envelope-from ) id 1qgWTP-000765-9U; Wed, 13 Sep 2023 20:28:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601591.937703; Wed, 13 Sep 2023 20:28:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTO-000755-W6; Wed, 13 Sep 2023 20:28:30 +0000 Received: by outflank-mailman (input) for mailman id 601591; Wed, 13 Sep 2023 20:28:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTN-0004dA-0e for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:29 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 18084ff1-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:27 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 18084ff1-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636906; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zKE4TU+LF26IXN40K9i0T1Kvr1B0IL2da7a6adH2ZkM=; b=S1qExoHxfV/EvjWKQmrK/OhI5jMmGOI3qbdfMh/yZ7llee/jjf2AXlzE /YlMEcue5uaeZr4fk+MmeEIw9OfcRGEZrVYodB0eti4bccCnlh8TEJDrw +/boftSJZbuEO2Fv4WQnn5ze+ZsD4DsdWmbW29OOiZK9t4l/UumQ/eu89 o=; X-CSE-ConnectionGUID: tqLoj7ejQSK+P4DSBC/0tA== X-CSE-MsgGUID: eGHVzTYyREumYktvs7pQkA== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125280892 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:8berC6ilp7e3uL8lL1hZSd1UX161dhAKZh0ujC45NGQN5FlHY01je htvWG/UaP+JamvwKoskO96woR4Gv5/QzoM2S1Ft+XhhFHwb9cadCdqndUqhZCn6wu8v7q5Ex 55HNoSfdpBcolv0/ErF3m3J9CEkvU2wbuOhTraCYmYoHVMMpB4J0XpLg/Q+jpNjne+3CgaMv cKai8DEMRqu1iUc3lg8sspvkzsx+qyo0N8klgZmP6sT7QaFzyJ94K83fsldEVOpGuG4IcbiL wrz5OnR1n/U+R4rFuSknt7TGqHdauePVeQmoiM+t5mK2nCulARrukoIHKN0hXNsoyeIh7hMJ OBl7vRcf+uL0prkw4zxWzEAe8130DYvFLXveRBTuuTLp6HKnueFL1yDwyjaMKVBktubD12i+ tQEMRQKbE+OhduN4/GDFsVXrO9kdNXCadZ3VnFIlVk1DN4jSJHHBa7L+cVZzHE7gcUm8fT2P pRDL2A1NVKZPkMJYw1MYH49tL7Aan3XWjtUsl+K44Ew5HDe1ldZ27nxKtvFPNeNQK25m27B/ D+crzypU0ty2Nq3xBWlzGCQt/71sD7+aYEOCqbn7sdpqQjGroAUIEJPDgbqyRWjsWauVtQaJ 0EK9y4Gqakp6FftXtT7Rwe/onOPolgbQdU4O88Q5RyJy6HUyx2EHWVCRTlEAPQ5sOcmSDps0 UWG9+4FHhQ27ufTEyjEsO7J83XrY3N9wXI+iTEsURIh5Nu8rYMKtTGMdopPK4SeyYXpMGSlq 9yVlxQWi7IWhM8N8qy0+1Hbnj6hzqT0oh4JChb/BTz8sF4gDGKxT8nxsAWAs64cRGqMZgPZ1 EXojfRy+wzn4XulsCWWCNsAE7iyjxpuGG2N2AU/d3XNGtnExpJCQWyyyGsuTKuKGpxeEdMMX KM0kVoLjKK/xFPwMcdKj3uZUqzGN5TIG9X/TezzZdFTeJV3fwLv1HgwNBLOhDi1zBZxz/BX1 XKnnSGEVypyNEia5GDuG7d1PUEDmEjSOl8/tbiklk/6gNJylVaeSKsfMUvmUwzKxPrsnekhy P4Gb5Hi40wGAIXDjtz/rdZ7waYicSJqWvgbaqV/Koa+H+aRMD19VKeMmONxI+SIXc19z4/1w 510YWcAoHKXuJENAVTihqxLAF83YatCkA== IronPort-HdrOrdr: A9a23:AQDBuao+mdozNWIpurug2hwaV5oTeYIsimQD101hICG8cqSj+f xG+85rrCMc6QxhPk3I9urhBEDtex/hHNtOkOws1NSZLW7bUQmTXeJfBOLZqlWKcUDDH6xmpM NdmsBFeaXN5DNB7PoSjjPWLz9Z+qjkzJyV X-Talos-CUID: 9a23:oVwCaWtTsT/xnBK8CwvVeiKJ6Is5a1nazijdIXaDKiVRFJGuY3TXoL9Nxp8= X-Talos-MUID: 9a23:6Y2HCglQRYQHL6p+P6MqdnpvG+dW8ZifGXogjJA3uNWaLnNoAjS02WE= X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="125280892" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 3/8] x86/spec-ctrl: Turn the remaining SPEC_CTRL_{ENTRY,EXIT}_* into asm macros Date: Wed, 13 Sep 2023 21:27:53 +0100 Message-ID: <20230913202758.508225-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 These have grown more complex over time, with some already having been converted. Provide full Requires/Clobbers comments, otherwise missing at this level of indirection. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 37 ++++++++++++++++++------ 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index dd7c5e8bfc79..ee75f2bced42 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -219,26 +219,45 @@ .endm /* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */ -#define SPEC_CTRL_ENTRY_FROM_PV \ +.macro SPEC_CTRL_ENTRY_FROM_PV +/* + * Requires %rsp=regs/cpuinfo, %rdx=0 + * Clobbers %rax, %rcx, %rdx + */ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_COND_IBPB maybexen=0), \ - X86_FEATURE_IBPB_ENTRY_PV; \ - ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \ + X86_FEATURE_IBPB_ENTRY_PV + + ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV + ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), \ X86_FEATURE_SC_MSR_PV +.endm /* Use in interrupt/exception context. May interrupt Xen or PV context. */ -#define SPEC_CTRL_ENTRY_FROM_INTR \ +.macro SPEC_CTRL_ENTRY_FROM_INTR +/* + * Requires %rsp=regs, %r14=stack_end, %rdx=0 + * Clobbers %rax, %rcx, %rdx + */ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_COND_IBPB maybexen=1), \ - X86_FEATURE_IBPB_ENTRY_PV; \ - ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \ + X86_FEATURE_IBPB_ENTRY_PV + + ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV + ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), \ X86_FEATURE_SC_MSR_PV +.endm /* Use when exiting to PV guest context. */ -#define SPEC_CTRL_EXIT_TO_PV \ - ALTERNATIVE "", \ - DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV; \ +.macro SPEC_CTRL_EXIT_TO_PV +/* + * Requires %rax=spec_ctrl, %rsp=regs/info + * Clobbers %rcx, %rdx + */ + ALTERNATIVE "", DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV + DO_SPEC_CTRL_COND_VERW +.endm /* * Use in IST interrupt/exception context. May interrupt Xen or PV context. From patchwork Wed Sep 13 20:27:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B71E3EE020B for ; Wed, 13 Sep 2023 20:28:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601583.937636 (Exim 4.92) (envelope-from ) id 1qgWTJ-0005OL-2a; Wed, 13 Sep 2023 20:28:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601583.937636; Wed, 13 Sep 2023 20:28:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTI-0005OA-VY; Wed, 13 Sep 2023 20:28:24 +0000 Received: by outflank-mailman (input) for mailman id 601583; Wed, 13 Sep 2023 20:28:23 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTH-0004dA-6k for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:23 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 152dfb69-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:22 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 152dfb69-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636902; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2vjOrVTuSvMwiX/obR22m8asNoVNui9kb4bQVY0kxGo=; b=AsQFWJH+/qtcE17b+oYjABZWQEcbIGLs8Qw/IkeIZ3qmvmNKOzFYqDug kOQCwwIP4I9cTXQwvhYwbRMCcrOFFrvaynNHOif15yIhCUYT8zKuJ5igM dhqiVn/Kl+Xf6+Mf4Z5R2SOp30XQl9zsmXBY84uHA0TZdQcoivXR5dSZX o=; X-CSE-ConnectionGUID: OTY/1Nd8T/+vvQCCeXLd1A== X-CSE-MsgGUID: QHz6PAKDSW64SprJkqRDWA== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213280 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:XLSYTa79CCQz/aMlhQJNqQxRtC7HchMFZxGqfqrLsTDasY5as4F+v jZNCjiPOaqDazf8f9h0b9m1pE4Fv5PVx9ZhTwRlpH9hHi5G8cbLO4+Ufxz6V8+wwm8vb2o8t plDNYOQRCwQZiWBzvt4GuG59RGQ7YnRG/ykTraCY3gtLeNdYH9JoQp5nOIkiZJfj9G8Agec0 fv/uMSaM1K+s9JOGjt8B5mr9lU35pwehBtC5gZlPaES5weF/5UoJMl3yZ+ZfiOQrrZ8RoZWd 86bpJml82XQ+QsaC9/Nut4XpWVTH9Y+lSDX4pZnc/DKbipq/0Te4Y5iXBYoUm9Fii3hojxE4 I4lWapc6+seFvakdOw1C3G0GszlVEFM0OevzXOX6aR/w6BaGpdFLjoH4EweZOUlFuhL7W5m0 sApb2krYkC/wKGo3ouQYeRcosh6BZy+VG8fkikIITDxCP8nRdbIQrnQ5M8e1zA17ixMNa+AP YxDM2MpNUmeJUQVYT/7C7pn9AusrlD5fydVtxS+oq0v7nKI5AdwzKLsIJzefdniqcB9xxzB/ DudpTShav0cHOW/22S921GAvdCRwRjjQqARDYeUydc/1TV/wURMUUZLBDNXu8KRmkO4Ht5SN UEQ0i4vtrQpslymSMHnWB+1q2LCuQQTM/JyOeAn7ACGyoLP/h2UQGMDS1Zpd9gOpMIwAzsw2 Te0c8jBXGI19ufPEDTEq+nS9GnpUcQIEYMcTQg4fExcxdWznMZwkAz2Zf1uIIq51/SgTFkc3 Au2hCQ5grwSi+sC2KO64U3LjlqQm3TZcuImzl6JBzz4t2uVcKbgPtX1sgaDsZ6sOa7DFjG8U G44d99yBQzkJbWEj2SzTeoEB9lFDN7VYWSH0TaD83TMnglBGkJPn6gKsFmSx28zaK7onAMFh 2eK42tsCGd7ZifCUEOOS9vZ5z4W5abhD8/5cfvfc8BDZJN8HCfeonAyOxXIhzyzzRZ9+U3aB Xt8WZzwZUv29Iw9lGbmLwvj+eFDKt8CKZP7GsmgkkXPPUu2b3+JU7YVWGZinchghJ5oVD79q o4FX+PTkkU3bQELSnWPmWLlBQxQfCdT6FGfg5A/S9Nv1SI6RD55VqaBketJlk4Mt/09q9okN 0qVAidwoGcTT1WcQelWQhiPsI/SYKs= IronPort-HdrOrdr: A9a23:Vzs0fKw8qCl+/TdN12LmKrPwFL1zdoMgy1knxilNoRw8SKKlfq eV7Y0mPH7P+VAssR4b+exoVJPtfZqYz+8R3WBzB8bEYOCFghrKEGgK1+KLqFeMJ8S9zJ846U 4JSdkHNDSaNzlHZKjBjzVQa+xQouW6zA== X-Talos-CUID: 9a23:ZZ3JQ2E5i3i+gj+fqmJAyH9TQ+YpIkGaj2nOGmSTBFZNROS8HAo= X-Talos-MUID: 9a23:Yu9bFQzBD//SNicouwFW4FLI43qaqJ2OGh41uLYqgPbaaRdoIRWelQ6bfaZyfw== X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213280" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 4/8] x86/spec-ctrl: Extend all SPEC_CTRL_{ENTER,EXIT}_* comments Date: Wed, 13 Sep 2023 21:27:54 +0100 Message-ID: <20230913202758.508225-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... to better explain how they're used. Doing so highlights that SPEC_CTRL_EXIT_TO_XEN is missing a VERW flush for the corner case when e.g. an NMI hits late in an exit-to-guest path. Leave a TODO, which will be addressed in subsequent patches which arrange for DO_COND_VERW to be safe within SPEC_CTRL_EXIT_TO_XEN. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu This was decided not to be XSA-worthy, as guests can't usefully control when IST events occur. --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 35 ++++++++++++++++++++---- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index ee75f2bced42..77f6c35bb9c5 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -218,7 +218,10 @@ wrmsr .endm -/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */ +/* + * Used after a synchronous entry from PV context. SYSCALL, SYSENTER, INT, + * etc. Will always interrupt a guest speculation context. + */ .macro SPEC_CTRL_ENTRY_FROM_PV /* * Requires %rsp=regs/cpuinfo, %rdx=0 @@ -233,7 +236,11 @@ X86_FEATURE_SC_MSR_PV .endm -/* Use in interrupt/exception context. May interrupt Xen or PV context. */ +/* + * Used after a synchronous interrupt or exception. May interrupt Xen or PV + * context, but will not interrupt Xen with a guest speculation context, + * outside of fatal error cases. + */ .macro SPEC_CTRL_ENTRY_FROM_INTR /* * Requires %rsp=regs, %r14=stack_end, %rdx=0 @@ -248,7 +255,10 @@ X86_FEATURE_SC_MSR_PV .endm -/* Use when exiting to PV guest context. */ +/* + * Used when exiting from any entry context, back to PV context. This + * includes from an IST entry which moved onto the primary stack. + */ .macro SPEC_CTRL_EXIT_TO_PV /* * Requires %rax=spec_ctrl, %rsp=regs/info @@ -260,7 +270,12 @@ .endm /* - * Use in IST interrupt/exception context. May interrupt Xen or PV context. + * Used after an IST entry (i.e. needs special care, consider to be fully + * asynchronous with finding sane state). May interrupt PV or Xen context, + * including other SPEC_CTRL_{ENTRY,EXIT}_* regions with unsanitised state. + * + * An IST entry which interrupts PV context moves onto the primary stack and + * leaves via SPEC_CTRL_EXIT_TO_PV, *not* SPEC_CTRL_EXIT_TO_XEN. */ .macro SPEC_CTRL_ENTRY_FROM_INTR_IST /* @@ -319,7 +334,14 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): UNLIKELY_END(\@_serialise) .endm -/* Use when exiting to Xen in IST context. */ +/* + * Use when exiting from any entry context, back to Xen context. This + * includes returning to other SPEC_CTRL_{ENTRY,EXIT}_* regions with + * unsanitised state. + * + * Because we might have interrupted Xen beyond SPEC_CTRL_EXIT_TO_$GUEST, we + * must treat this as if it were an EXIT_TO_$GUEST case too. + */ .macro SPEC_CTRL_EXIT_TO_XEN /* * Requires %rbx=stack_end @@ -344,6 +366,9 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): wrmsr .L\@_skip_sc_msr: + + /* TODO VERW */ + .endm #endif /* __ASSEMBLY__ */ From patchwork Wed Sep 13 20:27:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA42BEE020E for ; Wed, 13 Sep 2023 20:28:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601587.937660 (Exim 4.92) (envelope-from ) id 1qgWTL-0005zA-T5; Wed, 13 Sep 2023 20:28:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601587.937660; Wed, 13 Sep 2023 20:28:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTL-0005yE-Pd; Wed, 13 Sep 2023 20:28:27 +0000 Received: by outflank-mailman (input) for mailman id 601587; Wed, 13 Sep 2023 20:28:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTK-0004dA-0L for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:26 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 1695771d-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:24 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1695771d-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636904; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EAYwPucKCgFfpPtFZ5OzqdU5toziSCvVykYftRRfIdA=; b=ZfMMRej2INx492TkPHB1KBJeDDTJuGBKZBlP1zZpcyDGpBg0MJ6BRUNJ Y/QfDz/HYnbSw3Z/ubB81eY5BNnAcOdAtQoifT3AqvyDUz1KMCikZc8p3 BikipgQPTAvAgEESJ3znnIQKAMJdSKgz6eemxCX2CY6/uOcZthz3tJOb+ Q=; X-CSE-ConnectionGUID: hN7dthOmQouNNVMZfTFHmw== X-CSE-MsgGUID: woHOhxVzToi4V/sI0hrRwg== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213283 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:7aegra46JQllLXF5vj2pDgxRtCTHchMFZxGqfqrLsTDasY5as4F+v jAXXTyOPPzZMGv9fNskbI+19kICusLRx9dhSgJs/iE8Hi5G8cbLO4+Ufxz6V8+wwm8vb2o8t plDNYOQRCwQZiWBzvt4GuG59RGQ7YnRG/ykTraCY3gtLeNdYH9JoQp5nOIkiZJfj9G8Agec0 fv/uMSaM1K+s9JOGjt8B5mr9lU35pwehBtC5gZlPaES5weF/5UoJMl3yZ+ZfiOQrrZ8RoZWd 86bpJml82XQ+QsaC9/Nut4XpWVTH9Y+lSDX4pZnc/DKbipq/0Te4Y5iXBYoUm9Fii3hojxE4 I4lWapc6+seFvakdOw1C3G0GszlVEFM0OevzXOX6aR/w6BaGpdFLjoH4EweZOUlFuhL7W5m9 dpHJCwNSRW6gP+Q5oyeeOdQqOQmM5y+VG8fkikIITDxCP8nRdbIQrnQ5M8e1zA17ixMNa+AP YxDM2MpNUmeJUQVYT/7C7pn9AusrlD5fydVtxS+oq0v7nKI5AdwzKLsIJzefdniqcB9xxzB/ DudpTSiav0cHMOY6R2F4ky+uujOwR28Q6NKFKeC6fE/1TV/wURMUUZLBDNXu8KRmkO4Ht5SN UEQ0i4vtrQpslymSMHnWB+1q2LCuQQTM/JyOeAn7ACGyoLP/h2UQGMDS1Zpd9gOpMIwAzsw2 Te0c8jBXGI19ufPEDTEq+nS9GnpUcQIEYMcTS8iEgcZ4/7mmYYuqDTxa+RtLICVi8KgTFkc3 Au2hCQ5grwSi+sC2KO64U3LjlqQm3TZcuImzl6JBzz4t2uVcKbgPtX1sgaDsZ6sOa7DFjG8U G44d99yBQzkJbWEj2SzTeoEB9lFDN7VYWSH0TaD83TMnglBGkJPn6gKsFmSx28zaK7onAMFh 2eK4mtsCGd7ZifCUEOOS9vZ5z4W5abhD8/5cfvfc8BDZJN8HCfeonAyPBfJgzC1yxR0+U3aB Xt9WZz8ZZr9If47pAdaus9HieN7rszA7Ty7qW/HI+SPjuPFOS/9pUYtO1qSdOEphJ5oUy2Mm +uzw/Cikk0FOMWnO3m/zGLmBQxSRZTNLcys+pM/my/qClYOJVzN/NeNnex5JtU1z/w9eyWh1 ijVZ3K0AWHX3RXvQThmoFg5AF8zdf6TdU4GABE= IronPort-HdrOrdr: A9a23:HUJxKa7Oz8A1nM8TKQPXwOnXdLJyesId70hD6qkRc20vTiX8ra qTdZsgpHvJYVoqKRQdcLO7Scq9qBHnhPhICOAqVN/IPWnbUUSTXeNfBODZskTd8kPFh5ZgPG RbH5SWyuecMXFKyej95ge8H5IbzN6L/LvtrfvCzh5WPGZXgm1bgDuRwzz0LnFL X-Talos-CUID: 9a23:O82ACWh9hFkAae30pIinBlfwajJucluCwGvQBEqBDntPboOwe1Gs1oNhjJ87 X-Talos-MUID: 9a23:Xb4ANw2hs1kQgb1r2phz+h7ljjUj/KOOL0Adr4c6vMjYC2trGm+dlg6oTdpy X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213283" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 5/8] x86/entry: Adjust restore_all_xen to hold stack_end in %r14 Date: Wed, 13 Sep 2023 21:27:55 +0100 Message-ID: <20230913202758.508225-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 All other SPEC_CTRL_{ENTRY,EXIT}_* helpers hold stack_end in %r14. Adjust it for consistency, freeing up %rbx to be used differently. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 8 ++++---- xen/arch/x86/x86_64/entry.S | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 77f6c35bb9c5..acdb526d292d 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -344,10 +344,10 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ .macro SPEC_CTRL_EXIT_TO_XEN /* - * Requires %rbx=stack_end + * Requires %r14=stack_end * Clobbers %rax, %rcx, %rdx */ - testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) jz .L\@_skip_sc_msr /* @@ -358,10 +358,10 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ xor %edx, %edx - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) jz .L\@_skip_sc_msr - mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax + mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%r14), %eax mov $MSR_SPEC_CTRL, %ecx wrmsr diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index a1c860f56949..525877e97330 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -665,15 +665,15 @@ restore_all_xen: * Check whether we need to switch to the per-CPU page tables, in * case we return to late PV exit code (from an NMI or #MC). */ - GET_STACK_END(bx) - cmpb $0, STACK_CPUINFO_FIELD(use_pv_cr3)(%rbx) + GET_STACK_END(14) + cmpb $0, STACK_CPUINFO_FIELD(use_pv_cr3)(%r14) UNLIKELY_START(ne, exit_cr3) - mov STACK_CPUINFO_FIELD(pv_cr3)(%rbx), %rax + mov STACK_CPUINFO_FIELD(pv_cr3)(%r14), %rax mov %rax, %cr3 UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN /* Req: %rbx=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %r14=end, Clob: acd */ RESTORE_ALL adj=8 iretq From patchwork Wed Sep 13 20:27:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31FC7EE020D for ; Wed, 13 Sep 2023 20:28:36 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601585.937646 (Exim 4.92) (envelope-from ) id 1qgWTK-0005eh-B4; Wed, 13 Sep 2023 20:28:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601585.937646; Wed, 13 Sep 2023 20:28:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTK-0005eY-88; Wed, 13 Sep 2023 20:28:26 +0000 Received: by outflank-mailman (input) for mailman id 601585; Wed, 13 Sep 2023 20:28:25 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTJ-0004dA-06 for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:25 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 15e34a42-5274-11ee-8787-cb3800f73035; Wed, 13 Sep 2023 22:28:23 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15e34a42-5274-11ee-8787-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mZLU7yqbhnu8RFdttdE7JP+JDv6CudSB/eLutF8Nbbo=; b=Q34pzGydG57qG0EdZqqvWKl0k9W3sRafNLAljV5QNJocfPhg/RrlWNnM 5D3AbrGqa6cfuQ/xTYgEkY5xZDys22KkjpEkLz6MLyeQnIIogcHYGtQdN dA6zMg4TURrpDDH15PJFHf4CwfnqHFU7lAtHa1w2JvIyD2+sH1jNlqsyU g=; X-CSE-ConnectionGUID: OTY/1Nd8T/+vvQCCeXLd1A== X-CSE-MsgGUID: xmbwu7F0R1OtFqblUXH+wQ== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213281 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:mSAOkq3qv5C9ipOMt/bD5cVxkn2cJEfYwER7XKvMYLTBsI5bpzEAz GsfXDvUaKmIZ2umeYhybtjn9UMAv5bdxoVmSAVtpC1hF35El5HIVI+TRqvS04F+DeWYFR46s J9OAjXkBJppJpMJjk71atANlVEliOfQAOK6UbaYUsxIbVcMYD87jh5+kPIOjIdtgNyoayuAo tq3qMDEULOf82cc3lk8teTb8nuDgNyo4GlD5g1nPqgQ1LPjvyJ94Kw3dPnZw0TQGuG4LsbiL 87fwbew+H/u/htFIrtJRZ6iLyXm6paLVeS/oiI+t5qK23CulQRrukoPD9IOaF8/ttm8t4sZJ OOhF3CHYVxB0qXkwIzxWvTDes10FfUuFLTveRBTvSEPpqFvnrSFL/hGVSkL0YMkFulfJl1/s qU+JGAxKVOdmdud67+LbuU1r5F2RCXrFNt3VnBIyDjYCbAtQIzZQrWM7thdtNsyrpkQR7CEP ZNfMGcxKk2aOHWjOX9OYH46tM6uimPybHtzr1WNqLBsy2PS0BZwwP7mN9+9ltmiHJ8MxB/D/ DKfl4j/KgEfPsOClhTdyCuLmsHWlyfGaKJRC4Tto5aGh3XMnzdOWXX6T2CTsfS/z0KzRd9bA 0gV4TY167g/8lSxSdvwVAH+p2SL1jY+cddNF+wx6CmW17HZpQ2eAwAsUTppeNEg8sgsSlQXO kShxo2zQ2Y16fvMFCzbr+3Pxd+vBcQLBUg+VGhdawdV356gvp8KpTnCfvJvSYfg27UZBgrM6 zyNqSE/gZAagsgKy7i38Dj7vt68mnTaZlVrv1uKBwpJ+is8Pdf4PNLwtTA3+N4adO6kok+9U G/ociR0xMQHFtmzmSOEW43h95n5tq/eYFUwbbOCdqTNFghBGVb5Lei8Axkkfi+F1/ronhewO ifuVft5vsM7AZdTRfYfj3iNI8or17P8Mt/uS+rZaNFDCrAoKl7bono2PRLOgj68+KTJrU3YE c3HGftA8F5AUfg3pNZIb7l1PUAXKtAWmjqIGMGTI+WP2ruCfn+FIYrpw3PXBt3VGJis+V2Pm /4Gbpvi9vmqeLGmCsUh2dJJfA9iwLlSLcyelvG7gcbfflU+RThxW6e5LHFIU9UNopm5X9zgp hmVMnK0AnKj7ZEbAW1mskxeVY4= IronPort-HdrOrdr: A9a23:6EGM1KAi2/IDXCTlHemN55DYdb4zR+YMi2TDtnoBMiC9F/bzqy nApoV+6faZskd1ZJhCo6HiBEDjewK7yXcd2+B4VtbDYODIghrLEGgI1/qa/9SPIVyHygef78 tdmmpFZeHYPBxVi8D15QX9KdomzdWdtIi1mOa29QYLceinUc5dBs5CZDqmLg== X-Talos-CUID: 9a23:fEtew2HGnyiNxZmIqmJAyH9TQ+YpIkGaj2nOGmSTBFZNROS8HAo= X-Talos-MUID: 9a23:/rN/EAgN4R/qTlJ2BfMVXMMpJMJi/LnxJhg3yLIaldCOMRFAG3SCtWHi X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213281" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 6/8] x86/entry: Track the IST-ness of an entry for the exit paths Date: Wed, 13 Sep 2023 21:27:56 +0100 Message-ID: <20230913202758.508225-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Use %r12 to hold an ist_exit boolean. This register is zero elsewhere in the entry/exit asm, so it only needs setting in the IST path. As this is subtle and fragile, add check_ist_exit() to be used in debugging builds to cross-check that the ist_exit boolean matches the entry vector. Write check_ist_exit() it in C, because it's debug only and the logic more complicated than I care about maintaining in asm. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/traps.c | 13 +++++++++++++ xen/arch/x86/x86_64/compat/entry.S | 9 ++++++++- xen/arch/x86/x86_64/entry.S | 23 ++++++++++++++++++++--- 3 files changed, 41 insertions(+), 4 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index dead728ce329..0a005f088bca 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -2259,6 +2259,19 @@ void asm_domain_crash_synchronous(unsigned long addr) do_softirq(); } +#ifdef CONFIG_DEBUG +void check_ist_exit(const struct cpu_user_regs *regs, bool ist_exit) +{ + const unsigned int ist_mask = + (1U << X86_EXC_NMI) | (1U << X86_EXC_DB) | + (1U << X86_EXC_DF) | (1U << X86_EXC_MC); + uint8_t ev = regs->entry_vector; + bool is_ist = (ev < X86_EXC_NUM) && ((1U << ev) & ist_mask); + + ASSERT(is_ist == ist_exit); +} +#endif + /* * Local variables: * mode: C diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S index bd5abd8040bd..7504bfb4f326 100644 --- a/xen/arch/x86/x86_64/compat/entry.S +++ b/xen/arch/x86/x86_64/compat/entry.S @@ -117,8 +117,15 @@ compat_process_trap: call compat_create_bounce_frame jmp compat_test_all_events -/* %rbx: struct vcpu, interrupts disabled */ +/* %rbx: struct vcpu, %r12: ist_exit, interrupts disabled */ ENTRY(compat_restore_all_guest) + +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + ASSERT_INTERRUPTS_DISABLED mov $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), %r11d and UREGS_eflags(%rsp),%r11d diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 525877e97330..da084a7e8e54 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -142,10 +142,16 @@ process_trap: .section .text.entry, "ax", @progbits -/* %rbx: struct vcpu, interrupts disabled */ +/* %rbx: struct vcpu, %r12: ist_exit, interrupts disabled */ restore_all_guest: - ASSERT_INTERRUPTS_DISABLED +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + + ASSERT_INTERRUPTS_DISABLED /* Stash guest SPEC_CTRL value while we can read struct vcpu. */ mov VCPU_arch_msrs(%rbx), %rdx mov VCPUMSR_spec_ctrl_raw(%rdx), %r15d @@ -659,8 +665,15 @@ ENTRY(early_page_fault) .section .text.entry, "ax", @progbits ALIGN -/* No special register assumptions. */ +/* %r12=ist_exit */ restore_all_xen: + +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + /* * Check whether we need to switch to the per-CPU page tables, in * case we return to late PV exit code (from an NMI or #MC). @@ -1087,6 +1100,10 @@ handle_ist_exception: .L_ist_dispatch_done: mov %r15, STACK_CPUINFO_FIELD(xen_cr3)(%r14) mov %bl, STACK_CPUINFO_FIELD(use_pv_cr3)(%r14) + + /* This is an IST exit */ + mov $1, %r12 + cmpb $X86_EXC_NMI, UREGS_entry_vector(%rsp) jne ret_from_intr From patchwork Wed Sep 13 20:27:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 64C10EE020B for ; Wed, 13 Sep 2023 20:28:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601588.937668 (Exim 4.92) (envelope-from ) id 1qgWTM-00065N-HG; Wed, 13 Sep 2023 20:28:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601588.937668; Wed, 13 Sep 2023 20:28:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTM-00064L-5n; Wed, 13 Sep 2023 20:28:28 +0000 Received: by outflank-mailman (input) for mailman id 601588; Wed, 13 Sep 2023 20:28:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTK-0004yg-Ub for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:26 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 15f63a70-5274-11ee-9b0d-b553b5be7939; Wed, 13 Sep 2023 22:28:24 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15f63a70-5274-11ee-9b0d-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636904; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2OX3DxkMU6J5CjLFRkPzVMla4/tm4f+X2kaevcyHDIQ=; b=Tkd7e31bd3fAo5f/ivGt1lQq+AF4Cb52xSqaCD9LXpxkE27taasi7PfZ GXax0h1B2ubdf0hYMmtB9Ros/7mjNviKjq9TH8zpBrsuocx9Zvq94zNOq u9fcJA2f72cuuddmLyjHCHzltu/uoGwndvtFdWpAKdaxbLgs3raxVSD1n Y=; X-CSE-ConnectionGUID: hN7dthOmQouNNVMZfTFHmw== X-CSE-MsgGUID: 5pkOILBvSXyUEX0ggRl1Tw== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213284 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:g6lX8KAc4rcXbBVW/zXjw5YqxClBgxIJ4kV8jS/XYbTApD4rg2QAz GYdXTzTaKrYazDyKo92bNu+/ENQ75CBytA3QQY4rX1jcSlH+JHPbTi7wuUcHAvJd5GeExg3h yk6QoOdRCzhZiaE/n9BCpC48D8kk/nOH+KgYAL9EngZbRd+Tys8gg5Ulec8g4p56fC0GArIs t7pyyHlEAbNNwVcbCRMscpvlDs15K6p4GJB5QRlDRx2lAS2e0c9Xcp3yZ6ZdxMUcqEMdsamS uDKyq2O/2+x13/B3fv8z94X2mVTKlLjFVDmZkh+AsBOsTAbzsAG6Y4pNeJ0VKtio27hc+ada jl6ncfYpQ8BZsUgkQmGOvVSO3kW0aZuoNcrLZUj2CA6IoKvn3bEmp1T4E8K0YIw+sAoDX1O0 vkhLy0rSC6mteyK7pK+Vbw57igjBJGD0II3v3hhyXfSDOo8QICFSKLPjTNa9G5u3IYUR6+YP pdHL2M1N3wsYDUWUrsTILs4kP2lmT/UdDpApUjOjaE2/3LS3Ep6172F3N/9I4bTH5wOxBbEz o7A1znhATY7EPaB8gqUz1+c3uDApyeiZJ1HQdVU8dY12QbOlwT/EiY+RVa95PW0lEO6c9ZeM FAPvDojq7Ao806mRcW7WAe3yFamlBMBX9tbE8Uh9RqAjKHT5m6xGWwsXjNHLts8u6cLqScCj wHT2YmzXHo27ePTECjGnluJkd+sESgzcmMeZQgBdAwYxf/dhbM9sjzvS8k2RcZZkebJMT33x jmLqg03iLMSkdMH2s2HwLzXv96/jsOXF1Bov207Skrgt1okP9D9O+RE/HCBtZ59wJClok5tV ZTus+yX96gwAJ6Ej0Rhq81dTejyt55p3NAx6GOD/qXNFRz3qxZPnqgKulmSwXuF1e5dIlfUj Lf741852XOqFCLCgVVLS4ywEd826qPrCM7oUPvZBvIXPMktLVbeoHE2PhLIt4wIrKTKuftvU Xt8WZzwZUv29Iw9lGbmLwvj+eBDKt8CKZP7GsmgkkXPPUu2b3+JU7YVWGZinchghJ5oVD79q o4FX+PTkkU3bQELSnWPmWLlBQxQfCdT6FGfg5A/S9Nv1SI6RD55VqaBketJlk4Mt/09q9okN 0qVAidwoGcTT1WdQelWQhiPsI/SYKs= IronPort-HdrOrdr: A9a23:UARoG66DfrkgqFCANwPXwPDXdLJyesId70hD6qhwISY6TiX+rb HWoB17726TtN9/YhEdcLy7VJVoBEmskKKdgrNhWotKPjOW21dARbsKheCJrgEIWReOktK1vZ 0QC5SWY+eQMbEVt6nHCXGDYrQd/OU= X-Talos-CUID: 9a23:MiGufGrNSqKXV4FP4jNIfZLmUcADLXHBzFDVGkilL0BXbY+bdXGw/bwxxg== X-Talos-MUID: 9a23:6xHzcQVe4wxmJBTq/GL0hxZDEO122JS/NW09yZoJgc3dCTMlbg== X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213284" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 7/8] x86/spec-ctrl: Issue VERW during IST exit to Xen Date: Wed, 13 Sep 2023 21:27:57 +0100 Message-ID: <20230913202758.508225-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There is a corner case where e.g. an NMI hitting an exit-to-guest path after SPEC_CTRL_EXIT_TO_* would have run the entire NMI handler *after* the VERW flush to scrub potentially sensitive data from uarch buffers. In order to compensate, issue VERW when exiting to Xen from an IST entry. SPEC_CTRL_EXIT_TO_XEN already has two reads of spec_ctrl_flags off the stack, and we're about to add a third. Load the field into %ebx, and list the register as clobbered. %r12 has been arranged to be the ist_exit signal, so add this as an input dependency and use it to identify when to issue a VERW. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 20 +++++++++++++++----- xen/arch/x86/x86_64/entry.S | 2 +- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index acdb526d292d..9740697114ad 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -344,10 +344,12 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ .macro SPEC_CTRL_EXIT_TO_XEN /* - * Requires %r14=stack_end - * Clobbers %rax, %rcx, %rdx + * Requires %r12=ist_exit, %r14=stack_end + * Clobbers %rax, %rbx, %rcx, %rdx */ - testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) + movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %ebx + + testb $SCF_ist_sc_msr, %bl jz .L\@_skip_sc_msr /* @@ -358,7 +360,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ xor %edx, %edx - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) + testb $SCF_use_shadow, %bl jz .L\@_skip_sc_msr mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%r14), %eax @@ -367,8 +369,16 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): .L\@_skip_sc_msr: - /* TODO VERW */ + test %r12, %r12 + jz .L\@_skip_ist_exit + + /* Logically DO_SPEC_CTRL_COND_VERW but without the %rsp=cpuinfo dependency */ + testb $SCF_verw, %bl + jz .L\@_verw_skip + verw STACK_CPUINFO_FIELD(verw_sel)(%r14) +.L\@_verw_skip: +.L\@_skip_ist_exit: .endm #endif /* __ASSEMBLY__ */ diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index da084a7e8e54..f70752fa36c1 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -686,7 +686,7 @@ UNLIKELY_START(ne, exit_cr3) UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN /* Req: %r14=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %r12=ist_exit %r14=end, Clob: abcd */ RESTORE_ALL adj=8 iretq From patchwork Wed Sep 13 20:27:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13383762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 146BAEE020C for ; Wed, 13 Sep 2023 20:28:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.601586.937656 (Exim 4.92) (envelope-from ) id 1qgWTL-0005vE-JF; Wed, 13 Sep 2023 20:28:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 601586.937656; Wed, 13 Sep 2023 20:28:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTL-0005v5-Ff; Wed, 13 Sep 2023 20:28:27 +0000 Received: by outflank-mailman (input) for mailman id 601586; Wed, 13 Sep 2023 20:28:25 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qgWTJ-0004yg-UM for xen-devel@lists.xenproject.org; Wed, 13 Sep 2023 20:28:25 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 15412c50-5274-11ee-9b0d-b553b5be7939; Wed, 13 Sep 2023 22:28:23 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15412c50-5274-11ee-9b0d-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694636903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6Uh1DvjUFaFWcf8oaS6mdJk87ZcViax08R9IgITT/VU=; b=XMNdKkEffjgAfXJXib7+MVRpcYUavQSHTykGpCrmuBYLRRXLZjzQuws6 cvgXZu/Yh7Nhl198BMuuwNPgp7wgTkbXWUMnvzw0GnS3QzFGKVligb7Wu b09MTW+PJb9cToefyXNRhmOLTRfL60tZtMNc5r64qOoG5hjWvBaxkaTx5 A=; X-CSE-ConnectionGUID: OTY/1Nd8T/+vvQCCeXLd1A== X-CSE-MsgGUID: C11+nIAcQkOzLcw8YJc38g== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121213282 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:mbX+UKxiLCk3CFAANtF6t+chxirEfRIJ4+MujC+fZmUNrF6WrkVUz GdNDG2Ab/mIMGWjfdt1atm2oE5VscfTnd81GgE5rSAxQypGp/SeCIXCJC8cHc8wwu7rFxs7s ppEOrEsCOhuExcwcz/0auCJQUFUjPzOHvykTrecZkidfCc8IA85kxVvhuUltYBhhNm9Emult Mj75sbSIzdJ4RYtWo4vw/zF8EkHUMja4mtC5QRvPKsT5zcyqlFOZH4hDfDpR5fHatE88t6SH 47r0Ly/92XFyBYhYvvNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ai87XAME0e0ZP4whlqvgqo Dl7WT5cfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQq2pYjqhljJBheAGEWxgp4KUZyx LtBDjxdUlfAmfyP65m2es1g3+12eaEHPKtH0p1h5TTQDPJgSpHfWaTao9Rf2V/chOgXQ6yYP ZBAL2MyMlKZOUYn1lQ/UfrSmM+BgHXlfiIeg1WSvactuEDYzRBr0airO93QEjCPbZwPxxzB/ TiWpQwVBDkxbYXczxus+Uuvof3rljyraaAZT+23o6sCbFq7mTVIVUx+uUGAiem0jAuyVsxSL 2QQ+zEytu4i+UqzVN7/Uhak5nmesXY0efBdDuk74wGl0bfP7kCSAW1sZiFFQMwrsokxXzNC6 7OSt4q3X3o16uTTEC/DsO7O9lteJBT5M0clWR5DFC0Js+Wk458ttznkQ+ZHHPec24id9S7L/ xiGqy03hrM2hMEN1rmm8V2vvw9AtqQlXSZuuFyJAzvNAhdRIdf8Otf2sQSzAeNodt7xc7WXg JQTdyFyBsgqBIrFqiGCSf5l8FqBt6fca220bbKC8vAcG9WRF5yLJ9A4DNJWfh0B3iM4ldjBO RW7hO+pzMUPVEZGlIcuC25LN+wkzLL7CfPuXe3OY9xFb/BZLVHWoH81PRbMgT2zwSDAdJ3T3 r/BLa6R4YsyU/w7nFJauc9DuVPU+szO7TyKHs2qp/hW+bGfeGSUWd843KimN4gEAFe/iFyNq b53bpLaoyizpcWiOkE7B6ZPdwFVRZX6bLiqw/FqmhmrflY5QDx4W6+JqV7jEqQ895loei7z1 inVcidlJJDX3xUr9S3ihqhfVY7S IronPort-HdrOrdr: A9a23:aArYoqFfQ+SCHEtQpLqE0MeALOsnbusQ8zAXP0AYc3Jom6uj5r mTdZUgpHnJYVkqOE3I9ertBEDEewK4yXcX2/h3AV7BZniEhILAFugLhuGO/9SjIVybygc079 YZT0EUMrzN5DZB4voSmDPIceod/A== X-Talos-CUID: 9a23:KoX2Y2kUoehyM3vgS5IWtFX+hyHXOVrh/TCJBUPoNSEqVqOsYnSdyIdoyuM7zg== X-Talos-MUID: 9a23:rPcumw8rV7iMmZ8qEZUM9OiQf+wxuueoKWkGqJYhqe2PKnxBGTzG3DviFw== X-IronPort-AV: E=Sophos;i="6.02,144,1688443200"; d="scan'208";a="121213282" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 8/8] x86/spec-ctrl: Mitigate the Zen1 DIV leakge Date: Wed, 13 Sep 2023 21:27:58 +0100 Message-ID: <20230913202758.508225-9-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230913202758.508225-1-andrew.cooper3@citrix.com> References: <20230913202758.508225-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 In the Zen1 microarchitecure, there is one divider in the pipeline which services uops from both threads. In the case of #DE, the latched result from the previous DIV to execute will be forwarded speculatively. This is an interesting covert channel that allows two threads to communicate without any system calls. In also allows userspace to obtain the result of the most recent DIV instruction executed (even speculatively) in the core, which can be from a higher privilege context. Scrub the buffers in the divider unit by executing a non-faulting divide. This needs performing on the exit-to-guest paths, and ist_exit-to-Xen. This is XSA-439 / CVE-2023-20588. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu No embargo - this is already public. XSA paperwork to follow. --- docs/misc/xen-command-line.pandoc | 6 +++- xen/arch/x86/hvm/svm/entry.S | 1 + xen/arch/x86/include/asm/cpufeatures.h | 2 +- xen/arch/x86/include/asm/spec_ctrl_asm.h | 16 +++++++++ xen/arch/x86/spec_ctrl.c | 45 ++++++++++++++++++++++-- 5 files changed, 66 insertions(+), 4 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index f88e6a70aed6..7acd68885656 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2353,7 +2353,7 @@ By default SSBD will be mitigated at runtime (i.e `ssbd=runtime`). > {msr-sc,rsb,md-clear,ibpb-entry}=|{pv,hvm}=, > bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,psfd, > eager-fpu,l1d-flush,branch-harden,srb-lock, -> unpriv-mmio,gds-mit}= ]` +> unpriv-mmio,gds-mit,div-scrub}= ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -2475,6 +2475,10 @@ has elected not to lock the configuration, Xen will use GDS_CTRL to mitigate GDS with. Otherwise, Xen will mitigate by disabling AVX, which blocks the use of the AVX2 Gather instructions. +On all hardware, the `div-scrub=` option can be used to force or prevent Xen +from mitigating the DIV-leakage vulnerability. By default, Xen will mitigate +DIV-leakage on hardware believed to be vulnerable. + ### sync_console > `= ` diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S index 9effd2199ba0..c52528fed4cf 100644 --- a/xen/arch/x86/hvm/svm/entry.S +++ b/xen/arch/x86/hvm/svm/entry.S @@ -72,6 +72,7 @@ __UNLIKELY_END(nsvm_hap) 1: /* No Spectre v1 concerns. Execution will hit VMRUN imminently. */ .endm ALTERNATIVE "", svm_vmentry_spec_ctrl, X86_FEATURE_SC_MSR_HVM + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV pop %r15 pop %r14 diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/asm/cpufeatures.h index da0593de8542..724de2e11db4 100644 --- a/xen/arch/x86/include/asm/cpufeatures.h +++ b/xen/arch/x86/include/asm/cpufeatures.h @@ -35,7 +35,7 @@ XEN_CPUFEATURE(SC_RSB_HVM, X86_SYNTH(19)) /* RSB overwrite needed for HVM XEN_CPUFEATURE(XEN_SELFSNOOP, X86_SYNTH(20)) /* SELFSNOOP gets used by Xen itself */ XEN_CPUFEATURE(SC_MSR_IDLE, X86_SYNTH(21)) /* Clear MSR_SPEC_CTRL on idle */ XEN_CPUFEATURE(XEN_LBR, X86_SYNTH(22)) /* Xen uses MSR_DEBUGCTL.LBR */ -/* Bits 23 unused. */ +XEN_CPUFEATURE(SC_DIV, X86_SYNTH(25)) /* DIV scrub needed */ XEN_CPUFEATURE(SC_RSB_IDLE, X86_SYNTH(24)) /* RSB overwrite needed for idle. */ XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */ XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */ diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 9740697114ad..10e57780f08b 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -165,6 +165,18 @@ .L\@_verw_skip: .endm +.macro DO_SPEC_CTRL_DIV +/* + * Requires nothing + * Clobbers %rax + * + * Issue a DIV for its flushing side effect (Zen1 uarch specific). Any + * non-faulting DIV will do, and a byte DIV has least latency. + */ + mov $1, %eax + div %al +.endm + .macro DO_SPEC_CTRL_ENTRY maybexen:req /* * Requires %rsp=regs (also cpuinfo if !maybexen) @@ -267,6 +279,8 @@ ALTERNATIVE "", DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV DO_SPEC_CTRL_COND_VERW + + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV .endm /* @@ -378,6 +392,8 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): verw STACK_CPUINFO_FIELD(verw_sel)(%r14) .L\@_verw_skip: + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV + .L\@_skip_ist_exit: .endm diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 9b8fdb5303ad..5332dba3f659 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -67,6 +67,7 @@ static int8_t __initdata opt_srb_lock = -1; static bool __initdata opt_unpriv_mmio; static bool __ro_after_init opt_fb_clear_mmio; static int8_t __initdata opt_gds_mit = -1; +static int8_t __initdata opt_div_scrub = -1; static int __init cf_check parse_spec_ctrl(const char *s) { @@ -121,6 +122,7 @@ static int __init cf_check parse_spec_ctrl(const char *s) opt_srb_lock = 0; opt_unpriv_mmio = false; opt_gds_mit = 0; + opt_div_scrub = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -273,6 +275,8 @@ static int __init cf_check parse_spec_ctrl(const char *s) opt_unpriv_mmio = val; else if ( (val = parse_boolean("gds-mit", s, ss)) >= 0 ) opt_gds_mit = val; + else if ( (val = parse_boolean("div-scrub", s, ss)) >= 0 ) + opt_div_scrub = val; else rc = -EINVAL; @@ -473,7 +477,7 @@ static void __init print_details(enum ind_thunk thunk) "\n"); /* Settings for Xen's protection, irrespective of guests. */ - printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s\n", + printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s%s\n", thunk == THUNK_NONE ? "N/A" : thunk == THUNK_RETPOLINE ? "RETPOLINE" : thunk == THUNK_LFENCE ? "LFENCE" : @@ -498,7 +502,8 @@ static void __init print_details(enum ind_thunk thunk) opt_l1d_flush ? " L1D_FLUSH" : "", opt_md_clear_pv || opt_md_clear_hvm || opt_fb_clear_mmio ? " VERW" : "", - opt_branch_harden ? " BRANCH_HARDEN" : ""); + opt_branch_harden ? " BRANCH_HARDEN" : "", + opt_div_scrub ? " DIV" : ""); /* L1TF diagnostics, printed if vulnerable or PV shadowing is in use. */ if ( cpu_has_bug_l1tf || opt_pv_l1tf_hwdom || opt_pv_l1tf_domu ) @@ -955,6 +960,40 @@ static void __init srso_calculations(bool hw_smt_enabled) setup_force_cpu_cap(X86_FEATURE_SRSO_NO); } +/* + * Div leakage is specific to the AMD Zen1 microarchitecure. Use STIBP as a + * heuristic to select between Zen1 and Zen2 uarches. + */ +static bool __init has_div_vuln(void) +{ + if ( !(boot_cpu_data.x86_vendor & + (X86_VENDOR_AMD | X86_VENDOR_HYGON)) ) + return false; + + if ( (boot_cpu_data.x86 != 0x17 && boot_cpu_data.x86 != 0x18) || + !boot_cpu_has(X86_FEATURE_AMD_STIBP) ) + return false; + + return true; +} + +static void __init div_calculations(bool hw_smt_enabled) +{ + bool cpu_bug_div = has_div_vuln(); + + if ( opt_div_scrub == -1 ) + opt_div_scrub = cpu_bug_div; + + if ( opt_div_scrub ) + setup_force_cpu_cap(X86_FEATURE_SC_DIV); + + if ( opt_smt == -1 && cpu_bug_div && hw_smt_enabled ) + warning_add( + "Booted on leaky-DIV hardware with SMT/Hyperthreading\n" + "enabled. Please assess your configuration and choose an\n" + "explicit 'smt=' setting. See XSA-439.\n"); +} + static void __init ibpb_calculations(void) { bool def_ibpb_entry = false; @@ -1714,6 +1753,8 @@ void __init init_speculation_mitigations(void) ibpb_calculations(); + div_calculations(hw_smt_enabled); + /* Check whether Eager FPU should be enabled by default. */ if ( opt_eager_fpu == -1 ) opt_eager_fpu = should_use_eager_fpu();