From patchwork Fri Sep 15 15:00:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D539DEED600 for ; Fri, 15 Sep 2023 15:01:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603208.940152 (Exim 4.92) (envelope-from ) id 1qhAJX-0005Ku-SK; Fri, 15 Sep 2023 15:00:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603208.940152; Fri, 15 Sep 2023 15:00:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJX-0005Kn-NY; Fri, 15 Sep 2023 15:00:59 +0000 Received: by outflank-mailman (input) for mailman id 603208; Fri, 15 Sep 2023 15:00:58 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJW-0004UV-80 for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:00:58 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id abef32b7-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:00:57 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: abef32b7-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790057; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mCnqiLXy0cb8cqZh1FcVLN8e9wmVJ7m7NThYBUa7SXs=; b=fyXZJFbzx9/XLIDBIyb+yYkoSkKrgptY3/qKpciLjutuD5pSEg5T45sv 9+JM07j7D1UBPY2nhwM69CqLxWnMfl9qxzg63lWEIBShmjwjINBCyfUG4 ogaiI1cQ6YCegVWO26mSSRDexHq6T/gvJ4UcQOKxUUm32elq/59wUQjNR w=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: 7qOK/haVQ8qBVL4Lvh+2eA== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509273 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:9VNr0qsNzeTluX89C7THC199p+fnVEleMUV32f8akzHdYApBsoF/q tZmKT2HOavca2SkfYolb4u2oRxUuZLcndNmTgI9rXw1RC1A+JbJXdiXEBz9bniYRiHhoOCLz O1FM4Wdc5pkJpP4jk3wWlQ0hSAkjclkfpKlVaicfHg3HFc4IMsYoUoLs/YjhYJ1isSODQqIu Nfjy+XSI1bg0DNvWo4uw/vrRChH4rKq4lv0gnRkPaoQ5A6HyiFPZH4iDfrZw0XQE9E88tGSH 44v/JnhlkvF8hEkDM+Sk7qTWiXmlZaLYGBiIlIPM0STqkAqSh4ai87XB9JFAatjsB2bnsgZ9 Tl4ncfYpTHFnEH7sL91vxFwS0mSNEDdkVPNCSDXXce7lyUqf5ZwqhnH4Y5f0YAwo45K7W9yG fMwFG5KQRCzhuWM+7/lccdt2t96cJXIBdZK0p1g5Wmx4fcORJnCR+PB5MNC3Sd2jcdLdRrcT 5NHM3w1Nk2GOkARfA5NU/rSn8/x7pX7WxRepEiYuuwc5G/LwRYq+LPsLMDUapqBQsA9ckOw/ zibojyoWE5BXDCZ4WWc1VHrhubxpHPicrw2SJmm9r0wjXTGkwT/DzVJDADm8JFVkHWWS99Zb kAZ5Ccqhawz71CwCMnwWQWip3yJtQJaXMBfe8U44gyQzqvf4y6CG3MJCDVGbbQOq8seVTEsk FiTkLvU6SdH6ePPDyjHr/HN8G30YHJORYMfWcMaZTdbpPbjkIA0tCnCEO1NEfKu14GpOgill lhmsxMCr7kUiMcK0YCy8lbGny+gq/D1c+Il2unEdjn7t10kPeZJc6TtsAGGtqgYcO51W3Hb5 BA5d96iAPfi5H1nvAiEW60zEb6g/J5p2xWM0Ac0T/HNG9lAkkNPnLy8AhkkdC+F0e5eI1cFh XM/XisIv/du0IOCN/MfXm5II51CIVLcPdrkTOvISdFFf4J8cgSKlAk3Ox/Mhjm0zBh0yv1nU Xt+TSpKJSxKYUiA5GDqL9rxLJdxnnxurY8tbc+TI+ubPUq2OyfOFOZt3KqmZeEl9qKUyDg5A P4GX/ZmPy53CbWkCgGOqN57ELz/BSRjbXwAg5ANJ7Hrz8sPMD1JNsI9Npt6J9Q/xf0EzLaYl px/M2cBoGfCabT8AV3iQhhehHnHB/6TcVpT0fQQAGuV IronPort-HdrOrdr: A9a23:I5YfkaB7wNnMRxflHemW55DYdb4zR+YMi2TDgXoBLiC9Ffbo9P xG/c566faasl0ssR0b8+xoW5PgfZq/z/FICNIqTNOftWDd0QOVxedZgLcKqAePJ8SRzIJgPQ gLSdkZNDVdZ2IK7/oTQWODYrMd/OU= X-Talos-CUID: 9a23:nMSP/Wl0Ss/lLHOtALZFCWPi+zHXOUGDi1jQE06/M11kFoa7SAOLof84sPM7zg== X-Talos-MUID: 9a23:7h0LEg6NdKM0iDV68lCoC3T4xow3+oONT0AEr6xWouO6EXBUJTfe1jmeF9o= X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509273" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 1/9] x86/spec-ctrl: Fix confusion between SPEC_CTRL_EXIT_TO_XEN{,_IST} Date: Fri, 15 Sep 2023 16:00:30 +0100 Message-ID: <20230915150038.602577-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 c/s 3fffaf9c13e9 ("x86/entry: Avoid using alternatives in NMI/#MC paths") dropped the only user, leaving behind the (incorrect) implication that Xen had split exit paths. Delete the unused SPEC_CTRL_EXIT_TO_XEN and rename SPEC_CTRL_EXIT_TO_XEN_IST to SPEC_CTRL_EXIT_TO_XEN for consistency. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v2: * Tweak comment. --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 10 ++-------- xen/arch/x86/x86_64/entry.S | 2 +- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index f48f9e75e8dc..cfba35560333 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -67,7 +67,6 @@ * - SPEC_CTRL_ENTRY_FROM_PV * - SPEC_CTRL_ENTRY_FROM_INTR * - SPEC_CTRL_ENTRY_FROM_INTR_IST - * - SPEC_CTRL_EXIT_TO_XEN_IST * - SPEC_CTRL_EXIT_TO_XEN * - SPEC_CTRL_EXIT_TO_PV * @@ -256,11 +255,6 @@ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), \ X86_FEATURE_SC_MSR_PV -/* Use when exiting to Xen context. */ -#define SPEC_CTRL_EXIT_TO_XEN \ - ALTERNATIVE "", \ - DO_SPEC_CTRL_EXIT_TO_XEN, X86_FEATURE_SC_MSR_PV - /* Use when exiting to PV guest context. */ #define SPEC_CTRL_EXIT_TO_PV \ ALTERNATIVE "", \ @@ -327,8 +321,8 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): UNLIKELY_END(\@_serialise) .endm -/* Use when exiting to Xen in IST context. */ -.macro SPEC_CTRL_EXIT_TO_XEN_IST +/* Use when exiting to Xen context. */ +.macro SPEC_CTRL_EXIT_TO_XEN /* * Requires %rbx=stack_end * Clobbers %rax, %rcx, %rdx diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 81dd2c74b876..a1c860f56949 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -673,7 +673,7 @@ UNLIKELY_START(ne, exit_cr3) UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN_IST /* Req: %rbx=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %rbx=end, Clob: acd */ RESTORE_ALL adj=8 iretq From patchwork Fri Sep 15 15:00:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74623EED602 for ; Fri, 15 Sep 2023 15:01:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603209.940162 (Exim 4.92) (envelope-from ) id 1qhAJZ-0005bH-2k; Fri, 15 Sep 2023 15:01:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603209.940162; Fri, 15 Sep 2023 15:01:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJZ-0005b8-00; Fri, 15 Sep 2023 15:01:01 +0000 Received: by outflank-mailman (input) for mailman id 603209; Fri, 15 Sep 2023 15:00:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJX-0004UV-MC for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:00:59 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ad71710e-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:00:58 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ad71710e-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Pf82kjUXJ6jZJAJI5DPaVOzFlKCNTwtgYRVi04CNhmw=; b=WZ6ghBkDxbMesPnGG3WRo9zSvieQB42p83V2eoEcSOR+apHVmVozBPOJ 383kYrnSKlE8uSkhHoYwHnQ8BGB96PshmIANyqviBXMQvg1VxM2+U7X5b EuMgFxRJvBJ1+gNHoU/JBBJEwTmmynJ6AIzkjpj3kfIlgs+JqOKCZuvAI A=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: XvE0CHgRTJij4wJOTrLyEw== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509276 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:rnYqYaih39rx7pHhYPaMPSFmX161fxAKZh0ujC45NGQN5FlHY01je htvXT/VOvrYM2Gnco1zbd60pEkP78TRy4dkGQRu/CwwHyMb9cadCdqndUqhZCn6wu8v7q5Ex 55HNoSfdpBcolv0/ErF3m3J9CEkvU2wbuOhTraCYmYoHVMMpB4J0XpLg/Q+jpNjne+3CgaMv cKai8DEMRqu1iUc3lg8sspvkzsx+qyo0N8klgZmP6sT7QaDzyN94K83fsldEVOpGuG4IcbiL wrz5OnR1n/U+R4rFuSknt7TGqHdauePVeQmoiM+t5mK2nCulARrukoIHKN0hXNsoyeIh7hMJ OBl7vRcf+uL0prkw4zxWzEAe8130DYvFLXveRBTuuTLp6HKnueFL1yDwyjaMKVBktubD12i+ tQkdncibiCCgNiNzeyAZOgwnZ1/cOrSadZ3VnFIlVk1DN4jSJHHBa7L+cVZzHE7gcUm8fT2P pRDL2A1NVKZPkMJYw1MYH49tL7Aan3XWjtUsl+K44Ew5HDe1ldZ27nxKtvFPNeNQK25m27B/ D+bpzWjXU9y2Nq3+SSoqlmMidXzvyL5BJ0wM4Gi0u9HnwjGroAUIEJPDgbqyRWjsWauVtQaJ 0EK9y4Gqakp6FftXtT7Rwe/onOPolgbQdU4O+8w5RyJy6HUyx2EHWVCRTlEAPQ5sOcmSDps0 UWG9+4FHhQ27ufTEyjEsO7J83XrY3N9wXI+iTEscVVCzNnToYYJtR+RdfhiTLXs0oHWIGSlq 9yVlxQWi7IWhM8N8qy0+1Hbnj6hzqT0oh4JChb/BTz8sF4gDGKxT8nxsAWAs64cRGqMZgPZ1 EXojfRy+wzn4XulsCWWCNsAE7iyjxpuGG2N2AU/d3XNGtnExpJCQWyyyGsuTKuKGpxeEdMMX KM0kVoKjKK/xFPwMcdKj3uZUqzGN5TIG9X/TezzZdFTeJV3fwLv1HgwNBLJhjmxwBB3zflX1 XKnnSCEVyZy5UNPlmbeegvg+eVzmnBWKZ37GvgXMChLIZLBPSXIGN/pwXOFb/wj7bPsnekm2 483Cid+8D0GCLeWSnCOoeYuwaUicSBT6Wbe95YGKYZu42NORAkcNhMm6eh9I9Y/xvUPx7egE 7PUchYw9WcTTEbvcW2iAk2PopuyNXqjhRrX5RARAGs= IronPort-HdrOrdr: A9a23:y5DG+qnuLm++I8yHzvsl/6B+S4TpDfLT3DAbv31ZSRFFG/Fw9v re5cjzuiWE7Qr5NEtQ++xoW5PwIk80l6QFhbX5VI3KNGKN1VdAR7sSircKrQeQfREWNdQz6U 6jScRD4RHLbGRSvILC2y+fPO8H4P67mZrY/dv2/jNVVgdtZLhn7wBlTiimMmAefng8ObMJUK OG4MxJvjyhfmlSSP+aKD0qY8jvzue77q4PR3Y9dmIaAc21/E6VAXfBfXil4is= X-Talos-CUID: 9a23:NVMriWMKTVk4sO5DZHcg204kG8cfSXCD91nRf1ThVTpKcejA X-Talos-MUID: 9a23:TXBhhAb5YlfkOOBThyHB3BpCO/1UwaWsVF8vtqkcn9ipKnkl X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509276" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 2/9] x86/spec-ctrl: Fold DO_SPEC_CTRL_EXIT_TO_XEN into it's single user Date: Fri, 15 Sep 2023 16:00:31 +0100 Message-ID: <20230915150038.602577-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 With the SPEC_CTRL_EXIT_TO_XEN{,_IST} confusion fixed, it's now obvious that there's only a single EXIT_TO_XEN path. Fold DO_SPEC_CTRL_EXIT_TO_XEN into SPEC_CTRL_EXIT_TO_XEN to simplify further fixes. When merging labels, switch the name to .L\@_skip_sc_msr as "skip" on its own is going to be too generic shortly. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 40 ++++++++++-------------- 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index cfba35560333..72e7046f70d6 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -199,27 +199,6 @@ wrmsr .endm -.macro DO_SPEC_CTRL_EXIT_TO_XEN -/* - * Requires %rbx=stack_end - * Clobbers %rax, %rcx, %rdx - * - * When returning to Xen context, look to see whether SPEC_CTRL shadowing is - * in effect, and reload the shadow value. This covers race conditions which - * exist with an NMI/MCE/etc hitting late in the return-to-guest path. - */ - xor %edx, %edx - - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) - jz .L\@_skip - - mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax - mov $MSR_SPEC_CTRL, %ecx - wrmsr - -.L\@_skip: -.endm - .macro DO_SPEC_CTRL_EXIT_TO_GUEST /* * Requires %eax=spec_ctrl, %rsp=regs/cpuinfo @@ -328,11 +307,24 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): * Clobbers %rax, %rcx, %rdx */ testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) - jz .L\@_skip + jz .L\@_skip_sc_msr - DO_SPEC_CTRL_EXIT_TO_XEN + /* + * When returning to Xen context, look to see whether SPEC_CTRL shadowing + * is in effect, and reload the shadow value. This covers race conditions + * which exist with an NMI/MCE/etc hitting late in the return-to-guest + * path. + */ + xor %edx, %edx -.L\@_skip: + testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + jz .L\@_skip_sc_msr + + mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax + mov $MSR_SPEC_CTRL, %ecx + wrmsr + +.L\@_skip_sc_msr: .endm #endif /* __ASSEMBLY__ */ From patchwork Fri Sep 15 15:00:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A4659EED603 for ; Fri, 15 Sep 2023 15:01:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603210.940171 (Exim 4.92) (envelope-from ) id 1qhAJa-0005sL-AV; Fri, 15 Sep 2023 15:01:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603210.940171; Fri, 15 Sep 2023 15:01:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJa-0005sB-7h; Fri, 15 Sep 2023 15:01:02 +0000 Received: by outflank-mailman (input) for mailman id 603210; Fri, 15 Sep 2023 15:01:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJY-0004UV-S2 for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:01:00 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ae4d3cbc-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:01:00 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ae4d3cbc-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/u8QeVsYfOmIrdRheduf3X7k6ZQUSH9FsIiCo9BxkLA=; b=CfPY0ppBplhvOvn7O3J2038gUwCE4xWlYCNw9VpjcInUavq+VLrj2ryk 8qQId048x1w22aFZHteyy3Jp74luL9+fytw++zBXRW7T5D/w3Gnrc633X D5GLVGl31/VqRHgpUIV02Pjrf0wOW/ye+5mNbNdLU5yMKdtGfBmJfPhyI 4=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: XGE2t4/OQE6ysVOHiFaVsg== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509277 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:gtXlvas+kuKMG4dtr5Z9gtnTD+fnVEleMUV32f8akzHdYApBsoF/q tZmKT3TOKuPY2vxKt4iPY22oUlXvJHRn4UwSgFoqX9jFiNE+JbJXdiXEBz9bniYRiHhoOCLz O1FM4Wdc5pkJpP4jk3wWlQ0hSAkjclkfpKlVaicfHg3HFc4IMsYoUoLs/YjhYJ1isSODQqIu Nfjy+XSI1bg0DNvWo4uw/vrRChH4rKq4lv0gnRkPaoQ5A6HyiFPZH4iDfrZw0XQE9E88tGSH 44v/JnhlkvF8hEkDM+Sk7qTWiXmlZaLYGBiIlIPM0STqkAqSh4ai87XB9JFAatjsB2bnsgZ9 Tl4ncfYpTHFnEH7sL91vxFwS0mSNEDdkVPNCSDXXce7lyUqf5ZwqhnH4Y5f0YAwo45K7W9yG fMwFG5KQRCzhuWM+7/lccdt2t96cJXIBdZK0p1g5Wmx4fcORJnCR+PB5MNC3Sd2jcdLdRrcT 5NHM3w1Nk2GOkARfA5NU/rSn8/x7pX7WxRepEiYuuwc5G/LwRYq+LPsLMDUapqBQsA9ckOw/ zibojyoWE5FXDCZ4Qqq3yiw2qjopA6lSZksM5252PdUm1LGkwT/DzVJDADm8JFVkHWWS99Zb kAZ5Ccqhawz71CwCMnwWQWip3yJtQJaXMBfe8U44gyQzqvf4y6CG3MJCDVGbbQOq8seVTEsk FiTkLvU6SdH6ePPDyjHr/HN8G30YHJORYMfWcMaZTdbpPbjkIA0tCnCEO1NEfKu14GpOgill lhmsxMCr7kUiMcK0YCy8lbGny+gq/D1c+Il2unEdjn7t10kPeZJc6TtsAGGtqgYcO51W3Hb5 BA5d96iAPfi5H1nvAiEW60zEb6g/J5p2xWM0Ac0T/HNG9lAkkNPnLy8AhkkdC+F0e5eI1cFh XM/XisIv/du0IOCN/MfXm5II51CIVLcPdrkTOvISdFFf4J8cgSKlAk3Ox/Mhjm0zBh0yv1nU Xt+TSpKJSxKYUiA5GDqL9rxLJdxnnxurY8tbc+TI+ubPUq2OyfOFOZt3KqmZeEl9qKUyDg5A P4GX/ZmPy53CbWkCgGOqN57ELz/BSRjbXwAg5ANJ7Hrz8sPMD1JNsI9Npt6J9Q/xf0EzLaYl px/M2cBoGfCabT8AV3iQhhehHnHAP6TcVpT0fQQAGuV IronPort-HdrOrdr: A9a23:7VgBSaiZwza4JWTwWCan3Gf5x3BQXiUji2hC6mlwRA09TyX4rb HWoB11726QtN98YgBDpTniAtjifZq/z+8Q3WB5B97LN2OKhILCFvAE0WKN+UyHJ8Q8zIFgPG VbH5SWxOeQMXFKyf/Axi+WKvAB5+TvytHRuc7ui053Swdkcqdh6Bo8JDq6PyRNNXJ7LKt8LY Gb4MVfoTqmZDAwVeSUQlc4f8WrnaywqHrBCSR2eyLPLDP+8A9AIYSVcySl4g== X-Talos-CUID: 9a23:0tIXV2j2HNmrEu3yvd0dKEr+WTJuTiOa43fgIkiCMms3c5OUFUTJpP9Hqp87 X-Talos-MUID: 9a23:jhIdBwoixFhryifEX3cezwFNGZ10v+OJMW1TgI44nOmFGyVOIDjI2Q== X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509277" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 3/9] x86/spec-ctrl: Turn the remaining SPEC_CTRL_{ENTRY,EXIT}_* into asm macros Date: Fri, 15 Sep 2023 16:00:32 +0100 Message-ID: <20230915150038.602577-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 These have grown more complex over time, with some already having been converted. Provide full Requires/Clobbers comments, otherwise missing at this level of indirection. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 37 ++++++++++++++++++------ 1 file changed, 28 insertions(+), 9 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 72e7046f70d6..f768b0f48a0b 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -219,26 +219,45 @@ .endm /* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */ -#define SPEC_CTRL_ENTRY_FROM_PV \ +.macro SPEC_CTRL_ENTRY_FROM_PV +/* + * Requires %rsp=regs/cpuinfo, %rdx=0 + * Clobbers %rax, %rcx, %rdx + */ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_COND_IBPB maybexen=0), \ - X86_FEATURE_IBPB_ENTRY_PV; \ - ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \ + X86_FEATURE_IBPB_ENTRY_PV + + ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV + ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=0), \ X86_FEATURE_SC_MSR_PV +.endm /* Use in interrupt/exception context. May interrupt Xen or PV context. */ -#define SPEC_CTRL_ENTRY_FROM_INTR \ +.macro SPEC_CTRL_ENTRY_FROM_INTR +/* + * Requires %rsp=regs, %r14=stack_end, %rdx=0 + * Clobbers %rax, %rcx, %rdx + */ ALTERNATIVE "", __stringify(DO_SPEC_CTRL_COND_IBPB maybexen=1), \ - X86_FEATURE_IBPB_ENTRY_PV; \ - ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV; \ + X86_FEATURE_IBPB_ENTRY_PV + + ALTERNATIVE "", DO_OVERWRITE_RSB, X86_FEATURE_SC_RSB_PV + ALTERNATIVE "", __stringify(DO_SPEC_CTRL_ENTRY maybexen=1), \ X86_FEATURE_SC_MSR_PV +.endm /* Use when exiting to PV guest context. */ -#define SPEC_CTRL_EXIT_TO_PV \ - ALTERNATIVE "", \ - DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV; \ +.macro SPEC_CTRL_EXIT_TO_PV +/* + * Requires %rax=spec_ctrl, %rsp=regs/info + * Clobbers %rcx, %rdx + */ + ALTERNATIVE "", DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV + DO_SPEC_CTRL_COND_VERW +.endm /* * Use in IST interrupt/exception context. May interrupt Xen or PV context. From patchwork Fri Sep 15 15:00:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A2FF8EED603 for ; Fri, 15 Sep 2023 15:01:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603206.940129 (Exim 4.92) (envelope-from ) id 1qhAJO-0004YO-8b; Fri, 15 Sep 2023 15:00:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603206.940129; Fri, 15 Sep 2023 15:00:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJO-0004Xp-1p; Fri, 15 Sep 2023 15:00:50 +0000 Received: by outflank-mailman (input) for mailman id 603206; Fri, 15 Sep 2023 15:00:48 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJM-0004UV-Mu for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:00:48 +0000 Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com [216.71.145.142]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a67cbca2-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:00:46 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a67cbca2-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790046; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=izsyaKGJeUnhPk6j2osd2bXLpPVhEb5F2wtxQKRG81U=; b=WHixYv+Xe4E8hpMIAIMP7O1QjxRjIKCQdzOmaSd7YmbIlKAMeew5xnjB +cQNF8u1ZfOyBsnUUbwvXm4Kt3ofiIdASb773OsPIw3sy8cuSLH1e7LdW cFv32Gz658fyygle7FWwGEMmTeTBYeVuQ02aICkbvZ9giZlbGR1G84pzm c=; X-CSE-ConnectionGUID: X/kd13lbTuuvW1qLdo0dGQ== X-CSE-MsgGUID: EbxX6fuzQ8a5rCXSy6+RKw== Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 123201210 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:8OAk86NtW1zWR3fvrR26l8FynXyQoLVcMsEvi/4bfWQNrUohhTQAm DdLC22CMq3fajGmLtoibI++8EhT6pTTz4BgQQto+SlhQUwRpJueD7x1DKtS0wC6dZSfER09v 63yTvGacajYm1eF/k/F3oDJ9CQ6jefQAOOkVIYoAwgpLSd8UiAtlBl/rOAwh49skLCRDhiE/ Nj/uKUzAnf8s9JPGjxSs/jrRC9H5qyo42tJ5wFmP5ingXeF/5UrJMNHTU2OByOQrrl8RoaSW +vFxbelyWLVlz9F5gSNy+uTnuUiG9Y+DCDW4pZkc/HKbitq/0Te5p0TJvsEAXq7vh3S9zxHJ HehgrTrIeshFvWkdO3wyHC0GQkmVUFN0OevzXRSLaV/ZqAJGpfh66wGMa04AWEX0tYwAm0Xr vgSFC0QcC/ansa6x+q3ZdA506zPLOGzVG8eknRpzDWfBvc6W5HTBa7N4Le03h9p2JoIR6yHI ZNEN3w2Nk+ojx5nYz/7DLoXmuuyi2a5WDpfsF+P/oI84nTJzRw327/oWDbQUoXQHp8NxxzB/ goq+UzUCAEqatKC2wPV+y+2rd/0jH24YqENQejQGvlC3wTImz175ActfUu2p7y1h1CzX/pbK lcI4Ww+oK4q7kupQ9LhGRqirxasnDQRRt5RGO0S8xyWx+zf5APxLncAZi5MbpohrsBeeNAx/ gbXxZWzX2Up6eDLDyvHrd94sA9eJwAlamwOPHQARzcY/sfc+ZgXghWXQvlaRfvdYsLOJRn8x DWDrS4bjroVjNIW26jTwW0rkw5AtbCSEFdru1y/snaNq1ogOdX7P9DABU3zt64oEWqPcrWWU JHoceC65ftGM5yCnTflrA4lTODwvKbt3NExbDdS83gdG9aFoS7LkWN4umsWyKJV3iEsIGWBX aMrkVkNjKK/xVPzBUONX6q/Ct4x0Y/rHsn/W/bfY7JmO8YgKFDeono1NBHJjwgBdXTAd4llY f93lu72Vh4n5VlPlmLqF4/xL5d1rszB+Y8jbc+ilEn2uVZvTHWUVa0EIDOzghMRtcu5TPHu2 48HbaOikkwPONASlwGLqeb/23hWdylkbX03wuQLHtO+zv1OQz15UaWIkOlwK+SIXc19z4/1w 510YWcAoHKXuJENAVzihqxLAF83YatCkA== IronPort-HdrOrdr: A9a23:UCQYH6k018Pm5S6EpkOjOzXFnSLpDfLo3DAbv31ZSRFFG/Fw9/ rCoB17726QtN91YhsdcL+7V5VoLUmzyXcX2/hyAV7BZmnbUQKTRekP0WKL+Vbd8kbFh41gPM lbEpSXCLfLfCJHZcSR2njELz73quP3jJxBho3lvghQpRkBUdAF0+/gYDzranGfQmN9dP0EPa vZ3OVrjRy6d08aa8yqb0N1JNQq97Xw5fTbiQdtPW9f1DWz X-Talos-CUID: 9a23:bIgqRGMU0z1ZMu5DaG5k+BdPHsoZb2DD8y7JB0O/UWpGcejA X-Talos-MUID: 9a23:6kwvBgS7jObFTYURRXTBjiFSasl18Z3+AR0XuqU9qpfHDw9vbmI= X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="123201210" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 4/9] x86/spec-ctrl: Improve all SPEC_CTRL_{ENTER,EXIT}_* comments Date: Fri, 15 Sep 2023 16:00:33 +0100 Message-ID: <20230915150038.602577-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 ... to better explain how they're used. Doing so highlights that SPEC_CTRL_EXIT_TO_XEN is missing a VERW flush for the corner case when e.g. an NMI hits late in an exit-to-guest path. Leave a TODO, which will be addressed in subsequent patches which arrange for DO_COND_VERW to be safe within SPEC_CTRL_EXIT_TO_XEN. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu This was decided not to be XSA-worthy, as guests can't usefully control when IST events occur. v2: * Rewrite. --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 36 ++++++++++++++++++++---- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index f768b0f48a0b..8996fe3fc0ef 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -218,7 +218,10 @@ wrmsr .endm -/* Use after an entry from PV context (syscall/sysenter/int80/int82/etc). */ +/* + * Used after an entry from PV context: SYSCALL, SYSENTER, INT, + * etc. There is always a guest speculation state in context. + */ .macro SPEC_CTRL_ENTRY_FROM_PV /* * Requires %rsp=regs/cpuinfo, %rdx=0 @@ -233,7 +236,11 @@ X86_FEATURE_SC_MSR_PV .endm -/* Use in interrupt/exception context. May interrupt Xen or PV context. */ +/* + * Used after an exception or maskable interrupt, hitting Xen or PV context. + * There will either be a guest speculation context, or (baring fatal + * exceptions) a well-formed Xen speculation context. + */ .macro SPEC_CTRL_ENTRY_FROM_INTR /* * Requires %rsp=regs, %r14=stack_end, %rdx=0 @@ -248,7 +255,10 @@ X86_FEATURE_SC_MSR_PV .endm -/* Use when exiting to PV guest context. */ +/* + * Used when exiting from any entry context, back to PV context. This + * includes from an IST entry which moved onto the primary stack. + */ .macro SPEC_CTRL_EXIT_TO_PV /* * Requires %rax=spec_ctrl, %rsp=regs/info @@ -260,7 +270,13 @@ .endm /* - * Use in IST interrupt/exception context. May interrupt Xen or PV context. + * Used after an IST entry hitting Xen or PV context. Special care is needed, + * because when hitting Xen context, there may not a well-formed speculation + * context. (i.e. it can hit in the middle of SPEC_CTRL_{ENTRY,EXIT}_* + * regions.) + * + * An IST entry which hits PV context moves onto the primary stack and leaves + * via SPEC_CTRL_EXIT_TO_PV, *not* SPEC_CTRL_EXIT_TO_XEN. */ .macro SPEC_CTRL_ENTRY_FROM_INTR_IST /* @@ -319,7 +335,14 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): UNLIKELY_END(\@_serialise) .endm -/* Use when exiting to Xen context. */ +/* + * Use when exiting from any entry context, back to Xen context. This + * includes returning to other SPEC_CTRL_{ENTRY,EXIT}_* regions with an + * incomplete speculation context. + * + * Because we might have interrupted Xen beyond SPEC_CTRL_EXIT_TO_$GUEST, we + * need to treat this as if it were an EXIT_TO_$GUEST case too. + */ .macro SPEC_CTRL_EXIT_TO_XEN /* * Requires %rbx=stack_end @@ -344,6 +367,9 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): wrmsr .L\@_skip_sc_msr: + + /* TODO VERW */ + .endm #endif /* __ASSEMBLY__ */ From patchwork Fri Sep 15 15:00:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 827E1EED604 for ; Fri, 15 Sep 2023 15:01:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603213.940186 (Exim 4.92) (envelope-from ) id 1qhAJc-0006Dm-50; Fri, 15 Sep 2023 15:01:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603213.940186; Fri, 15 Sep 2023 15:01:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJb-0006D0-Tp; Fri, 15 Sep 2023 15:01:03 +0000 Received: by outflank-mailman (input) for mailman id 603213; Fri, 15 Sep 2023 15:01:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJa-0005sI-J8 for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:01:02 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ad8409d6-53d8-11ee-9b0d-b553b5be7939; Fri, 15 Sep 2023 17:01:00 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ad8409d6-53d8-11ee-9b0d-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KznqDoscyqMnpcQNfSJy4Gab/IynVVymbTjqUVUKxtA=; b=aI8O9h79Gd7i8LSnwgm7YhfgnXuyQ7yr3nwOAbeFOj48kG9/5A6caX8b wA9uB97hCXHbZ511KKFyi2oWXg1GxVhwXPq4Uue60Q64cFCA1937jtgDC qmoDt05amFYIa1owMbfWkvheZbcA8q8czbgHbO65TrkBndF//gswNCuhl c=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: LsywUawbQEWBwf9B3F4Crw== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509278 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:YIwosq3HpgvEUo08mvbD5cxxkn2cJEfYwER7XKvMYLTBsI5bpzBSx jdMXjvUP62INzfzeYp1PYrk9h8Fv5aGz9dlTAJopC1hF35El5HIVI+TRqvS04F+DeWYFR46s J9OAjXkBJppJpMJjk71atANlVEliOfQAOK6UbaYUsxIbVcMYD87jh5+kPIOjIdtgNyoayuAo tq3qMDEULOf82cc3lk8teTb8nuDgNyo4GlD5g1nOKgR1LPjvyJ94Kw3dPnZw0TQGuG4LsbiL 87fwbew+H/u/htFIrtJRZ6iLyXm6paLVeS/oiI+t5qK23CulQRrukoPD9IOaF8/ttm8t4sZJ OOhF3CHYVxB0qXkwIzxWvTDes10FfUuFLTveRBTvSEPpqFvnrSFL/hGVSkL0YMkFulfPT8J2 vwoKTc3VR3dvMy3nKniFrVPv5F2RCXrFNt3VnBIyDjYCbAtQIzZQrWM7thdtNsyrpkQR7CEP ZNfMGcxKk2aOHWjOX9OYH46tM6uimPybHtzr1WNqLBsy2PS0BZwwP7mN9+9ltmiHJ8Iwx7J/ D2Wl4j/KjUBJcOiwCWPy1+TrPDBzAmgdt45DqLto5aGh3XMnzdOWXX6T2CTsfS/z0KzRd9bA 0gV4TY167g/8lSxSdvwVAH+p2SL1jYeUddNF+wx6CmW17HZpQ2eAwAsUTppeNEg8sgsSlQXO kShxo2zQ2Y16fvMFCzbr+3Pxd+vBcQLBVNcIgAEfQoC2eLu/rodj07UF4o/M5fg27UZBgrM6 zyNqSE/gZAagsgKy7i38Dj7vt68mnTaZlVrv1uKBwpJ+is8Pdf4PNLwtTA3+N4adO6kok+9U G/ociR0xMQHFtmzmSOEW43h95n5tq/eYFUwbbOCdqTNFghBGVb5Lei8Axkkfi+F1/ronhewO yfuVft5vsM7AZdTRfYfj3iNI8or17P8Mt/uS+rZaNFDCrAoKl7bpXg3OR7IhDy1+KTJrU3ZE c3LGSpLJSxGYZmLMRLsH7tNuVPV7n9WKZzvqWDTkE38jOv2iI+9QrYZKlqeBt3VH4vdyDg5B +13bpPQoz0GCb2WX8Ui2dJLRbz8BSRhVM+eRg0+XrLrHzeK70l6V6eLm+pwId0190mX/8+Rl kyAtoZj4AKXrRX6xc+iNxiPtJuHsU5DkE8G IronPort-HdrOrdr: A9a23:yBwV3amg4zxKdk+ZgZeEwzgWYW7pDfLT3DAbv31ZSRFFG/Fw9v re5cjzuiWE7Qr5NEtQ++xoW5PwIk80l6QFhbX5VI3KNGKN1VdAR7sSircKrQeQfREWNdQz6U 6jScRD4RHLbGRSvILC2y+fPO8H4P67mZrY/dv2/jNVVgdtZLhn7wBlTiimMmAefng8ObMJUK OG4MxJvjyhfmlSSP+aKD0qY8jvzue77q4PR3Y9dmIaAc21/E6VAXfBfXil4is= X-Talos-CUID: 9a23:tK2yqWielXFeHm3gfnNSoDAFxTJuTiOa43fgIkiCMms3c5OUFUTJpP9Hqp87 X-Talos-MUID: 9a23:4HaHKwwurkl3+uDgSWtwXaW7KAyaqIC1GV8wmIoMguS7HzN+ah3GomicWJByfw== X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509278" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 5/9] x86/entry: Adjust restore_all_xen to hold stack_end in %r14 Date: Fri, 15 Sep 2023 16:00:34 +0100 Message-ID: <20230915150038.602577-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 All other SPEC_CTRL_{ENTRY,EXIT}_* helpers hold stack_end in %r14. Adjust it for consistency. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 8 ++++---- xen/arch/x86/x86_64/entry.S | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 8996fe3fc0ef..b696033240e4 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -345,10 +345,10 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ .macro SPEC_CTRL_EXIT_TO_XEN /* - * Requires %rbx=stack_end + * Requires %r14=stack_end * Clobbers %rax, %rcx, %rdx */ - testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) jz .L\@_skip_sc_msr /* @@ -359,10 +359,10 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ xor %edx, %edx - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%rbx) + testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) jz .L\@_skip_sc_msr - mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%rbx), %eax + mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%r14), %eax mov $MSR_SPEC_CTRL, %ecx wrmsr diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index a1c860f56949..525877e97330 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -665,15 +665,15 @@ restore_all_xen: * Check whether we need to switch to the per-CPU page tables, in * case we return to late PV exit code (from an NMI or #MC). */ - GET_STACK_END(bx) - cmpb $0, STACK_CPUINFO_FIELD(use_pv_cr3)(%rbx) + GET_STACK_END(14) + cmpb $0, STACK_CPUINFO_FIELD(use_pv_cr3)(%r14) UNLIKELY_START(ne, exit_cr3) - mov STACK_CPUINFO_FIELD(pv_cr3)(%rbx), %rax + mov STACK_CPUINFO_FIELD(pv_cr3)(%r14), %rax mov %rax, %cr3 UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN /* Req: %rbx=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %r14=end, Clob: acd */ RESTORE_ALL adj=8 iretq From patchwork Fri Sep 15 15:00:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71FA0EED603 for ; Fri, 15 Sep 2023 15:01:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603207.940133 (Exim 4.92) (envelope-from ) id 1qhAJO-0004fc-G7; Fri, 15 Sep 2023 15:00:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603207.940133; Fri, 15 Sep 2023 15:00:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJO-0004dm-AU; Fri, 15 Sep 2023 15:00:50 +0000 Received: by outflank-mailman (input) for mailman id 603207; Fri, 15 Sep 2023 15:00:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJN-0004UV-01 for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:00:49 +0000 Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com [216.71.145.142]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a741277f-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:00:48 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a741277f-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790048; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TtLZP+3qIQQSlR+qOYXKHN3/07wVBW9kau99ogGnBZk=; b=Mkex/qgXmBpIT9rvDQGxQ4NzAr/1lcl5kg5XXxGrZsX6RuawAXJG5ZmJ +GLiYXN/VtzkcwTyL1HqSsAc0Ws3h0QiJyy3+kwLCqaFxGJw4dhYZ48LJ R5UfibZvlUbH/bdqkdH2XthWxAzAyH9V8S/S5Wt7CWgUZb51t49o4KtWg o=; X-CSE-ConnectionGUID: X/kd13lbTuuvW1qLdo0dGQ== X-CSE-MsgGUID: eyR7kQ8bR6y6ixMxkAXS4w== Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 123201211 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:hVEh0a96e4iFdMqnzRk6DrUDn36TJUtcMsCJ2f8bNWPcYEJGY0x3x jRKWz+Ha6zcNGD1L98kbI6x8x5Q6p7XzNcwSVQ+rno8E34SpcT7XtnIdU2Y0wF+jCHgZBk+s 5hBMImowOQcFCK0SsKFa+C5xZVE/fjVAOK6UKidYnwZqTZMEE8JkQhkl/MynrlmiN24BxLlk d7pqojUNUTNNwRcawr40Ird7ks11BjOkGlA5AdmNKkW5AW2e0Q9V/rzG4ngdxMUfaEMdgKKb 76r5K20+Grf4yAsBruN+losWhRXKlJ6FVHmZkt+A8BOsDAbzsAB+v9T2M4nQVVWk120c+VZk 72hg3ASpTABZcUgkMxFO/VR/roX0aduoNcrKlDn2SCfItGvn9IBDJyCAWlvVbD09NqbDkl+s v8UcWsJZCyemvyJmaufRu40iusaeZyD0IM34hmMzBncBPciB5vCX7/L9ZlT2zJYasJmRKiEI ZBDMHw2MUqGOkcUUrsUIMtWcOOAr3/zaTBH7nmSorI6+TP7xw1tyrn9dtHSf7RmQO0MxB3C/ jucpj2R7hcyOc2lljm8oniW2P6RnDPeSIE7JIbk6as/6LGU7jNKU0BHPbehmtGph0j7V99BJ kg8/is1sbN05EGtVsP6XRCzvDiDpBF0c/h6HvA+6QqN4rHJ+AvfDW8BJhZebPQ2uclwQiYlv mJlhPuwW2Yp6ufMDyvAqPHN92ja1TUpwXEqVWgHQVJe59PfvZwtsUvWatpkSKKat4igcd3v+ AyioC87jrQVqMcE0aSn4FzK6w6RSoj1oh0dvVuOAD/8hu9tTMv8PtHztwCHhRpVBNzBJmRtq kTojCR3AAomKZiW3BKAT+wWdF1Cz6bUaWaM6bKD8nRIythMx5JBVdoAiN2dDB0zWirhRdMOS BaJ0T69HLcJYBOXgVZfOupd8fgCw6n6DsjCXfvJdNdIaZUZXFbZrX0xPBDLhDGyyRhEfUQD1 XGzK57E4ZEyU/8P8dZLb71Fje9DKt4Wmws/uqwXPzz4iOHDNRZ5uJ8OMUeUb/BR0U93iFy9z jqrDOPTk083eLSnMkHqHXs7cQhiwY4TWcqn9KS6t4erfmJbJY3WI6SNmel7It0/zvQ9eyWh1 ijVZ3K0AWHX3RXvQThmoFg6AF8zdf6TdU4GABE= IronPort-HdrOrdr: A9a23:vbMVKqrjiZtVGwPSTliQoVMaV5rveYIsimQD101hICG9Evb0qy nOpoV/6faQslwssR4b9uxoVJPvfZq+z+8W3WByB9eftWDd0QPFEGgL1+DfKlbbak7DH4BmtJ uJc8JFeafN5VoRt7eG3OFveexQvOVu88qT9JjjJ28Gd3APV0n5hT0JcjpyFCdNNW57LKt8Lr WwzOxdqQGtfHwGB/7LfUXsD4D41rv2fIuNW29+OyIa X-Talos-CUID: 9a23:cC+Yq27vzzBFUhJF79ssxAkXFpl+dnfh0WrOHBahLWpuELG6cArF X-Talos-MUID: 9a23:q9Z/AgowkSQu2mPbEvcezzxzDphr2vSkMh4cz5QWntmHHBxLZyjI2Q== X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="123201211" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 6/9] x86/entry: Track the IST-ness of an entry for the exit paths Date: Fri, 15 Sep 2023 16:00:35 +0100 Message-ID: <20230915150038.602577-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Use %r12 to hold an ist_exit boolean. This register is zero elsewhere in the entry/exit asm, so it only needs setting in the IST path. As this is subtle and fragile, add check_ist_exit() to be used in debugging builds to cross-check that the ist_exit boolean matches the entry vector. Write check_ist_exit() it in C, because it's debug only and the logic more complicated than I care to maintain in asm. For now, we only need to use this signal in the exit-to-Xen path, but some exit-to-guest paths happen in IST context too. Check the correctness in all exit paths to avoid the logic bitrotting. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v2: * %r12 -> %r12d * Extend commit message * Tweak surrounding context --- xen/arch/x86/traps.c | 13 +++++++++++++ xen/arch/x86/x86_64/compat/entry.S | 9 ++++++++- xen/arch/x86/x86_64/entry.S | 22 ++++++++++++++++++++-- 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index dead728ce329..0a005f088bca 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -2259,6 +2259,19 @@ void asm_domain_crash_synchronous(unsigned long addr) do_softirq(); } +#ifdef CONFIG_DEBUG +void check_ist_exit(const struct cpu_user_regs *regs, bool ist_exit) +{ + const unsigned int ist_mask = + (1U << X86_EXC_NMI) | (1U << X86_EXC_DB) | + (1U << X86_EXC_DF) | (1U << X86_EXC_MC); + uint8_t ev = regs->entry_vector; + bool is_ist = (ev < X86_EXC_NUM) && ((1U << ev) & ist_mask); + + ASSERT(is_ist == ist_exit); +} +#endif + /* * Local variables: * mode: C diff --git a/xen/arch/x86/x86_64/compat/entry.S b/xen/arch/x86/x86_64/compat/entry.S index bd5abd8040bd..7504bfb4f326 100644 --- a/xen/arch/x86/x86_64/compat/entry.S +++ b/xen/arch/x86/x86_64/compat/entry.S @@ -117,8 +117,15 @@ compat_process_trap: call compat_create_bounce_frame jmp compat_test_all_events -/* %rbx: struct vcpu, interrupts disabled */ +/* %rbx: struct vcpu, %r12: ist_exit, interrupts disabled */ ENTRY(compat_restore_all_guest) + +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + ASSERT_INTERRUPTS_DISABLED mov $~(X86_EFLAGS_IOPL | X86_EFLAGS_VM), %r11d and UREGS_eflags(%rsp),%r11d diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index 525877e97330..e5055e5bbf9f 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -142,8 +142,15 @@ process_trap: .section .text.entry, "ax", @progbits -/* %rbx: struct vcpu, interrupts disabled */ +/* %rbx: struct vcpu, %r12: ist_exit, interrupts disabled */ restore_all_guest: + +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + ASSERT_INTERRUPTS_DISABLED /* Stash guest SPEC_CTRL value while we can read struct vcpu. */ @@ -659,8 +666,15 @@ ENTRY(early_page_fault) .section .text.entry, "ax", @progbits ALIGN -/* No special register assumptions. */ +/* %r12=ist_exit */ restore_all_xen: + +#ifdef CONFIG_DEBUG + mov %rsp, %rdi + mov %r12, %rsi + call check_ist_exit +#endif + /* * Check whether we need to switch to the per-CPU page tables, in * case we return to late PV exit code (from an NMI or #MC). @@ -1087,6 +1101,10 @@ handle_ist_exception: .L_ist_dispatch_done: mov %r15, STACK_CPUINFO_FIELD(xen_cr3)(%r14) mov %bl, STACK_CPUINFO_FIELD(use_pv_cr3)(%r14) + + /* This is an IST exit */ + mov $1, %r12d + cmpb $X86_EXC_NMI, UREGS_entry_vector(%rsp) jne ret_from_intr From patchwork Fri Sep 15 15:00:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 07C99EED605 for ; Fri, 15 Sep 2023 15:01:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603214.940192 (Exim 4.92) (envelope-from ) id 1qhAJc-0006KH-II; Fri, 15 Sep 2023 15:01:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603214.940192; Fri, 15 Sep 2023 15:01:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJc-0006HS-AW; Fri, 15 Sep 2023 15:01:04 +0000 Received: by outflank-mailman (input) for mailman id 603214; Fri, 15 Sep 2023 15:01:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJb-0005sI-87 for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:01:03 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id adc97a79-53d8-11ee-9b0d-b553b5be7939; Fri, 15 Sep 2023 17:01:00 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: adc97a79-53d8-11ee-9b0d-b553b5be7939 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/GM8AdBIr2NA9MDy77t7kyfRHQ78VMf2N1XB8hJaZKs=; b=UiAxU11rOKC/kl1Di1c7Y6aFSWtmVjfoYJxhIzkEW5KxTUgAYIpFX4F+ +fo7UPBNQd/Nd++LCE5p9s47jyNRVsS4qcYjnaOWXK3MDxKpWzIQOgOxX 5Ui2/1dnzWHNGP3v9rN5WKVCJ3D6VmkC0fxVVyvIn4F1iB+ewbmhxAgct k=; X-CSE-ConnectionGUID: vAIIvUM8SCaECUCG0mQOuQ== X-CSE-MsgGUID: U1c+vOBkTMS643nOsobRsQ== Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 121436365 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:fqf5paq9isA7+37WV1E8v4IhgApeBmIHZRIvgKrLsJaIsI4StFCzt garIBmPbKmDYzSjL4wnb9vioRwBscLVxt9hTAFk/iE3QiMUopuZCYyVIHmrMnLJJKUvbq7FA +Y2MYCccZ9uHhcwgj/3b9ANeFEljfngqoLUUbOCYmYpA1Y8FE/NsDo788YhmIlknNOlNA2Ev NL2sqX3NUSsnjV5KQr40YrawP9UlKq04GlwUmAWP6gR5wePziRNVvrzGInqR5fGatgMdgKFb 76rIIGRpgvx4xorA9W5pbf3GmVirmn6ZFXmZtJ+AsBOszAazsAA+v9T2Mk0MC+7vw6hjdFpo OihgLTrIesf0g8gr8xGO/VQO3kW0aSrY9YrK1Dn2SCY5xWun3cBX5yCpaz5VGEV0r8fPI1Ay RAXAC4kTzGdusfm+5myZ8JRret7KfDMIbpK7xmMzRmBZRonaZXKQqGM7t5ExjYgwMtJGJ4yZ eJAN2ApNk6ZJUQSZBFOUslWcOSA3xETdxVxrl6PqLVxyG/U1AFri5DmMcbPe8zMTsJQ9qqdj jucpjiiXEpEabRzzxK4q3nvmMjlnh/eY9xOT7uf7flgukaqkzl75Bo+CgLg/KjRZlSFc8JSL QkY9zQjqYA29Ve3VZ/tUhugunmGsxUAHd1KHIUS6guA167V6AaxHXUfQ3hKb9lOnNAybSwn0 BmOhdyBONB0mOTLEzTHrO7S9G7sf3FPdgfueBPoUyMbyvjCvrwW3inkR/tILYWa1tjMPizJl mXiQDcFu50fissC1qOe9F/Bgi6xqpWhcjPZ9jk7TUr+sFonOdfNi5iArAGCsK0edNrxokyp5 iBspiSI0AwZ4XhhfgSpSf5FIrym7u3t3Nb00Q82RMlJG9hAFheekWFsDNNWfhcB3iUsI2WBj KrvVeR5vcU7AZdSRfUrC79d8uxzpUQaKfzrV+rPcv1FaYVreQmM8UlGPBDBgz+0zhN3wP5ma f93lPpA615AUcyLKxLsHI8gPUIDnHhilQs/u7ilp/hY7VZuTCHMEupUWLd/Rus48LmFsG3oH yV3bqO3J+FkeLSmOEH/qNdDRW3m2FBnXfgaXeQLLL/cSuencUl9Y8LsLUQJINA9xf4Mx72Zr hlQmCZwkTLCuJEOEi3SAlgLVV8ldcwXQa4TVcD0AWuV5g== IronPort-HdrOrdr: A9a23:LgWkGq5y6/toTLUnhgPXwOfXdLJyesId70hD6qkRc202TiX8ra rCoB1173PJYVoqN03I4OrwXZVoGEmskaKdgrNhXotKPjOGhILAFugLhrcKpQeQfREWndQ86U 4PScZD4ZLLfD9HZTWR2njALz9Z+qj8zElev5ai85/UJzsaEJ2IRj0JcjqmLg== X-Talos-CUID: 9a23:yVMdVWkDsvJMbrCPdjcHyC/PDkfXOWP07VbOHmroMk1gZZaoYnDI/ZpDjvM7zg== X-Talos-MUID: 9a23:laeG1AR2PZZyoLTXRXTR3TohKuBC4JiLMx5RypMhoM6ODzdJbmI= X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="121436365" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 7/9] x86/spec-ctrl: Issue VERW during IST exit to Xen Date: Fri, 15 Sep 2023 16:00:36 +0100 Message-ID: <20230915150038.602577-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 There is a corner case where e.g. an NMI hitting an exit-to-guest path after SPEC_CTRL_EXIT_TO_* would have run the entire NMI handler *after* the VERW flush to scrub potentially sensitive data from uarch buffers. In order to compensate, issue VERW when exiting to Xen from an IST entry. SPEC_CTRL_EXIT_TO_XEN already has two reads of spec_ctrl_flags off the stack, and we're about to add a third. Load the field into %ebx, and list the register as clobbered. %r12 has been arranged to be the ist_exit signal, so add this as an input dependency and use it to identify when to issue a VERW. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu Note to reviewers: .L\@_skip_verw and .L\@_skip_ist_exit are separate to reduce the churn in the following patch. v2: * Rename .L\@_skip_verw --- xen/arch/x86/include/asm/spec_ctrl_asm.h | 20 +++++++++++++++----- xen/arch/x86/x86_64/entry.S | 2 +- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index b696033240e4..9a27e3170347 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -345,10 +345,12 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ .macro SPEC_CTRL_EXIT_TO_XEN /* - * Requires %r14=stack_end - * Clobbers %rax, %rcx, %rdx + * Requires %r12=ist_exit, %r14=stack_end + * Clobbers %rax, %rbx, %rcx, %rdx */ - testb $SCF_ist_sc_msr, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) + movzbl STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14), %ebx + + testb $SCF_ist_sc_msr, %bl jz .L\@_skip_sc_msr /* @@ -359,7 +361,7 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): */ xor %edx, %edx - testb $SCF_use_shadow, STACK_CPUINFO_FIELD(spec_ctrl_flags)(%r14) + testb $SCF_use_shadow, %bl jz .L\@_skip_sc_msr mov STACK_CPUINFO_FIELD(shadow_spec_ctrl)(%r14), %eax @@ -368,8 +370,16 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): .L\@_skip_sc_msr: - /* TODO VERW */ + test %r12, %r12 + jz .L\@_skip_ist_exit + + /* Logically DO_SPEC_CTRL_COND_VERW but without the %rsp=cpuinfo dependency */ + testb $SCF_verw, %bl + jz .L\@_skip_verw + verw STACK_CPUINFO_FIELD(verw_sel)(%r14) +.L\@_skip_verw: +.L\@_skip_ist_exit: .endm #endif /* __ASSEMBLY__ */ diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S index e5055e5bbf9f..988ef6cbc628 100644 --- a/xen/arch/x86/x86_64/entry.S +++ b/xen/arch/x86/x86_64/entry.S @@ -687,7 +687,7 @@ UNLIKELY_START(ne, exit_cr3) UNLIKELY_END(exit_cr3) /* WARNING! `ret`, `call *`, `jmp *` not safe beyond this point. */ - SPEC_CTRL_EXIT_TO_XEN /* Req: %r14=end, Clob: acd */ + SPEC_CTRL_EXIT_TO_XEN /* Req: %r12=ist_exit %r14=end, Clob: abcd */ RESTORE_ALL adj=8 iretq From patchwork Fri Sep 15 15:00:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 88486EED600 for ; Fri, 15 Sep 2023 15:01:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603212.940182 (Exim 4.92) (envelope-from ) id 1qhAJb-0006AG-P4; Fri, 15 Sep 2023 15:01:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603212.940182; Fri, 15 Sep 2023 15:01:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJb-0006A7-Km; Fri, 15 Sep 2023 15:01:03 +0000 Received: by outflank-mailman (input) for mailman id 603212; Fri, 15 Sep 2023 15:01:02 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJa-0004UV-8S for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:01:02 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id af01a903-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:01:01 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: af01a903-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790061; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iqRrxOP3pxe6jZWttoS6DHQOzltrAeSnz0bv2Vp0Iyo=; b=eQQAFigzmhqxVeht+7Agb10UcXChbBH4VJ2Z+8oyCHf/kEAqeadxW9Xm ++cK0Lnn31fMQ9VT1Jyw5pytyCxjTZqkQbR6jU7g/l6WncEFLziA2olvg G2m6jdF7LvQlQbjTutNZZKYKGTSsTBEt13YVVyksisVh10kcJQyigstGJ I=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: CDhin9m1QGyvT2GKHUBMJQ== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509279 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:D5UCQ6ysCzvRT1q0Qv16t+coxirEfRIJ4+MujC+fZmUNrF6WrkVRy jdOXGrXaareZWfxfd11atvn/E8DvpDUm99mHgdv+SAxQypGp/SeCIXCJC8cHc8wwu7rFxs7s ppEOrEsCOhuExcwcz/0auCJQUFUjPzOHvykTrecZkidfCc8IA85kxVvhuUltYBhhNm9Emult Mj75sbSIzdJ4RYtWo4vw/zF8EkHUMja4mtC5QRvPK0T5jcyqlFOZH4hDfDpR5fHatE88t6SH 47r0Ly/92XFyBYhYvvNfmHTKxBirhb6ZGBiu1IOM0SQqkEqSh8ai87XAME0e0ZP4whlqvgqo Dl7WT5cfi9yVkHEsLx1vxC1iEiSN4UekFPMCSDXXcB+UyQq2pYjqhljJBheAGEWxgp4KV0Qs tMYGDEKVCuO2dmYwe6gE7Uwpf12eaEHPKtH0p1h5TTQDPJgSpHfWaTao9Rf2V/chOgXQ6yYP ZBAL2MyMlKZOUYn1lQ/UfrSmM+BgHXlfiIeg1WSvactuEDYzRBr0airO93QEjCPbZwPwxvA9 ziZrgwVBDkxGNyez2uAq0uDl/HzxwzYeL84MZ+Bo6sCbFq7mTVIVUx+uUGAiem0jAuyVsxSL 2QQ+zEytu4i+UqzVN7/Uhak5nmesXY0efBdDuk74wGl0bfP7kCSAW1sZiFFQMwrsokxXzNC6 7OSt4q3X3o16uTTEC/DsO7O9lteJBT5M0c+O2grTTsE5OPYrd8IoBOTVY8/TI2M24id9S7L/ xiGqy03hrM2hMEN1rmm8V2vvw9AtqQlXSZuuFyJAzvNAhdRIdf8Otf2sQSzAeNodt7xc7WXg JQTdyFyBsgqBIrFqiGCSf5l8FqBt6fca220bbKC8vAcG9WRF5yLJ9A4DNJWfh0B3iM4ldjBO RS7hO+pzMUPVEZGlIcuC25LN+wkzLL7CfPuXe3OY9xFb/BZLVHWoHg3PBLAhzuxyCDAdJ3T3 7/BIa5A6l5AVcxaIMeeHb9BgdfHOAhhrY8seXwL50v+iufPDJJkYbwELEGPfogEAFCs+W3oH yJkH5LSkX13CbSuChQ7BKZPdTjm21BnX8GpwyGWH8bfSjdb9JYJUKOJm+l4KtY0z8y4VI7gp xmAZ6OR83Kn7VWvFOlAQi0LhG/HNXqnkU8GAA== IronPort-HdrOrdr: A9a23:QNS2g63KRC48Pi88Pt/udgqjBJ0kLtp133Aq2lEZdPU1SL37qy nKpp536faaslossR0b9uxoQZPwJE80rKQFhbX5Xo3SPzUO2lHIEGgK1+KLqAEIfReOkNK1vp 0BT0ERMrPN5BdB/KHHCPrTKadY/DD+ytHTuQ4o9QYRcenTAZsQlDuRozzranFLeA== X-Talos-CUID: 9a23:eDw+oGhb2Wh7EEo6TBnavLAYpzJuTiOa43fgIkiCMms3c5OUFUTJpP9Hqp87 X-Talos-MUID: 9a23:h0P4rAiCZpFCuxOzQEOpc8MpH+to4qL/DRk0voQYh5OrFgxNERK/tWHi X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509279" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 8/9] x86/amd: Introduce is_zen{1,2}_uarch() predicates Date: Fri, 15 Sep 2023 16:00:37 +0100 Message-ID: <20230915150038.602577-9-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 We already have 3 cases using STIBP as a Zen1/2 heuristic, and are about to introduce a 4th. Wrap the heuristic into a pair of predictes rather than opencoding it, and the explaination of the heursitic, at each usage site. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu v2: * New --- xen/arch/x86/cpu/amd.c | 18 ++++-------------- xen/arch/x86/include/asm/amd.h | 11 +++++++++++ 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c index bbf7887f2e1d..4f27187f92ec 100644 --- a/xen/arch/x86/cpu/amd.c +++ b/xen/arch/x86/cpu/amd.c @@ -882,15 +882,13 @@ void amd_set_legacy_ssbd(bool enable) * non-branch instructions to be ignored. It is to be set unilaterally in * newer microcode. * - * This chickenbit is something unrelated on Zen1, and Zen1 vs Zen2 isn't a - * simple model number comparison, so use STIBP as a heuristic to separate the - * two uarches in Fam17h(AMD)/18h(Hygon). + * This chickenbit is something unrelated on Zen1. */ void amd_init_spectral_chicken(void) { uint64_t val, chickenbit = 1 << 1; - if (cpu_has_hypervisor || !boot_cpu_has(X86_FEATURE_AMD_STIBP)) + if (cpu_has_hypervisor || !is_zen2_uarch()) return; if (rdmsr_safe(MSR_AMD64_DE_CFG2, val) == 0 && !(val & chickenbit)) @@ -939,11 +937,8 @@ void amd_check_zenbleed(void) * With the Fam17h check above, most parts getting here are * Zen1. They're not affected. Assume Zen2 ones making it * here are affected regardless of microcode version. - * - * Zen1 vs Zen2 isn't a simple model number comparison, so use - * STIBP as a heuristic to distinguish. */ - if (!boot_cpu_has(X86_FEATURE_AMD_STIBP)) + if (is_zen1_uarch()) return; good_rev = ~0U; break; @@ -1298,12 +1293,7 @@ static int __init cf_check zen2_c6_errata_check(void) */ s_time_t delta; - /* - * Zen1 vs Zen2 isn't a simple model number comparison, so use STIBP as - * a heuristic to separate the two uarches in Fam17h. - */ - if (cpu_has_hypervisor || boot_cpu_data.x86 != 0x17 || - !boot_cpu_has(X86_FEATURE_AMD_STIBP)) + if (cpu_has_hypervisor || boot_cpu_data.x86 != 0x17 || !is_zen2_uarch()) return 0; /* diff --git a/xen/arch/x86/include/asm/amd.h b/xen/arch/x86/include/asm/amd.h index 09ee52dc1c09..d862cb7972a1 100644 --- a/xen/arch/x86/include/asm/amd.h +++ b/xen/arch/x86/include/asm/amd.h @@ -140,6 +140,17 @@ AMD_MODEL_RANGE(0x11, 0x0, 0x0, 0xff, 0xf), \ AMD_MODEL_RANGE(0x12, 0x0, 0x0, 0xff, 0xf)) +/* + * The Zen1 and Zen2 microarchitectures are implemented by AMD (Fam17h) and + * Hygon (Fam18h) but without simple model number rules. Instead, use STIBP + * as a heuristic that distinguishes the two. + * + * The caller is required to perform the appropriate vendor/family checks + * first. + */ +#define is_zen1_uarch() (!boot_cpu_has(X86_FEATURE_AMD_STIBP)) +#define is_zen2_uarch() boot_cpu_has(X86_FEATURE_AMD_STIBP) + struct cpuinfo_x86; int cpu_has_amd_erratum(const struct cpuinfo_x86 *, int, ...); From patchwork Fri Sep 15 15:00:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13387143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 33677EED600 for ; Fri, 15 Sep 2023 15:01:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.603215.940200 (Exim 4.92) (envelope-from ) id 1qhAJd-0006VN-4D; Fri, 15 Sep 2023 15:01:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 603215.940200; Fri, 15 Sep 2023 15:01:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJc-0006Tc-Sn; Fri, 15 Sep 2023 15:01:04 +0000 Received: by outflank-mailman (input) for mailman id 603215; Fri, 15 Sep 2023 15:01:03 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qhAJb-0004UV-8R for xen-devel@lists.xenproject.org; Fri, 15 Sep 2023 15:01:03 +0000 Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ae6378ef-53d8-11ee-8788-cb3800f73035; Fri, 15 Sep 2023 17:01:01 +0200 (CEST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ae6378ef-53d8-11ee-8788-cb3800f73035 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1694790061; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G13uxDI+Vf2EUoeXWUQgkocQAX3E/eken8QVmw9pr80=; b=CtnVVfzePCFH5UK/W9icaPHtFN1rgv6w1SJwlnEZmaCX0AnVaPDsDo4j 7aS8mS9HIH8MX2adzgG4shfC2dZFXdgDY5qcDYBeihzZtmruwmX+dPjLk yqjDvwS9+UI9V5Vn63esuhZMVtOgwlm+oVqceT7dO3z7S5bX43YTJqgdt k=; X-CSE-ConnectionGUID: T6/GhXioRSe6SFi4s144KQ== X-CSE-MsgGUID: 2jZM/k87SiyEhEWEqg5jag== Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none X-SBRS: 4.0 X-MesageID: 125509281 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.123 X-Policy: $RELAYED X-ThreatScanner-Verdict: Negative IronPort-Data: A9a23:H+M9LKlYZ4uTeowJepNbCnzo5gyeJkRdPkR7XQ2eYbSJt1+Wr1Gzt xIaXzuPbKnbZDD9KItwa47no0pVvJHSmN9kGgJsri0wECMWpZLJC+rCIxarNUt+DCFhoGFPt JxCN4aafKjYaleG+39B55C49SEUOZmgH+e6UKicfHkpGWeIcQ954Tp7gek1n4V0ttawBgKJq LvartbWfVSowFaYCEpNg064gE0p5K6aVA8w5ARkPqga5QKGzRH5MbpETU2PByqgKmVrNrbSq 9brlNmR4m7f9hExPdKp+p6TnpoiG+O60aCm0xK6aoD66vRwjnVaPpUTbZLwXXx/mTSR9+2d/ f0W3XCGpaXFCYWX8AgVe0Ew/yiTpsSq8pefSZS0mZT7I0Er7xIAahihZa07FdRwxwp5PY1B3 cBGbx8NXhGEvdm7nomVS7V335h/D/C+aevzulk4pd3YJfMvQJSFSKTW/95Imjw3g6iiH96HO ZBfM2A2Kk2dPVsWYAx/5JEWxY9EglHWdTFCpU3Tjq0w+2XJlyR60aT3McqTcduPLSlQth/C+ z+erj6pW3n2MvTFywigtXH82dTOjA77db0/DbLl9Pdl1Qj7Kms7V0RNCArTTeOCol6zXZdTJ lIZ/gIqrLMu7wq7Q9/lRRq6rXWY+BkGVLJ4Mcc39QWMwar8+BuCCy4PSTspQMMinN87Q3otz FDhoj/yLWUx6vvPEyvbr+rK62roYkD5MFPuewc4E0wq4+nip7sIg0z0YNc/C/fp0v7MTGSYL y+xkAAygLAajMgu3qq9/Ezajz/EmqUlXjLZ9S2MADv7s1oRiJqNItXxtAOFtaoowJOxFAHpg ZQSpySJAAni57mpnTfFfugCFarBCx2tYGyF2g4H83XMGl2QF5+fkWJ4umEWyKRBaJxsldrVj Kj74Fg52XOrFCH2BZKbmqroYyjQ8YDuFM7+StffZcdUb556eWevpX8/ORfJhjy9yBNwycnT3 Kt3l+72XR727ow9kFKLqxo1i+d3lkjSO0uMLXwE8/hX+eXHPyPEIVv0GFCPcvo4/Mu5TPb9q r5i2z+x40wHCoXWO3CHmbP/2HhWdRDX87iq8Z0IHgNCSyI6cFwc5wj5mu95IdY8zv4Exo8lP BiVAydl9bY2vlWfQS3iV5ypQOiHsUpXxZ7jARERAA== IronPort-HdrOrdr: A9a23:mA6Q+KG1BoxfIOWopLqEx8eALOsnbusQ8zAXPiFKOH5om6mj/a 2TdZsguSMc5Ax/ZJhYo6HkBEDYewK5yXcX2/hzAV7BZmjbUQKTRekI0WKF+UyHJ8SUzI5gPM lbHZSWR+eAamSTfqzBkXCF+q4bsaO6GVeT9JrjJ88Gd3AQV0jI1XYANjqm X-Talos-CUID: 9a23:YP5kG24hxNP8CAXtA9ssyBBTNcAVc3rn93qOGGG5VX0wFOG2cArF X-Talos-MUID: 9a23:7hQ8sA1Wewob9y7Up/v59lNonzUjuaacLQcOwco9utipNQFPFi6S0Ta+Tdpy X-IronPort-AV: E=Sophos;i="6.02,149,1688443200"; d="scan'208";a="125509281" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH 9/9] x86/spec-ctrl: Mitigate the Zen1 DIV leakge Date: Fri, 15 Sep 2023 16:00:38 +0100 Message-ID: <20230915150038.602577-10-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915150038.602577-1-andrew.cooper3@citrix.com> References: <20230915150038.602577-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 In the Zen1 microarchitecure, there is one divider in the pipeline which services uops from both threads. In the case of #DE, the latched result from the previous DIV to execute will be forwarded speculatively. This is an interesting covert channel that allows two threads to communicate without any system calls. In also allows userspace to obtain the result of the most recent DIV instruction executed (even speculatively) in the core, which can be from a higher privilege context. Scrub the result from the divider by executing a non-faulting divide. This needs performing on the exit-to-guest paths, and ist_exit-to-Xen. This is XSA-439 / CVE-2023-20588. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Wei Liu No embargo - this is already public. XSA paperwork to follow. v2: * Rebase over the introduction of is_zen1_uarch(). * Fix the SC_DIV bit not to alias SC_VERW_IDLE. * Extend comments. --- docs/misc/xen-command-line.pandoc | 6 ++- xen/arch/x86/hvm/svm/entry.S | 1 + xen/arch/x86/include/asm/cpufeatures.h | 2 +- xen/arch/x86/include/asm/spec_ctrl_asm.h | 17 ++++++++ xen/arch/x86/spec_ctrl.c | 49 +++++++++++++++++++++++- 5 files changed, 72 insertions(+), 3 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc index f88e6a70aed6..7acd68885656 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2353,7 +2353,7 @@ By default SSBD will be mitigated at runtime (i.e `ssbd=runtime`). > {msr-sc,rsb,md-clear,ibpb-entry}=|{pv,hvm}=, > bti-thunk=retpoline|lfence|jmp, {ibrs,ibpb,ssbd,psfd, > eager-fpu,l1d-flush,branch-harden,srb-lock, -> unpriv-mmio,gds-mit}= ]` +> unpriv-mmio,gds-mit,div-scrub}= ]` Controls for speculative execution sidechannel mitigations. By default, Xen will pick the most appropriate mitigations based on compiled in support, @@ -2475,6 +2475,10 @@ has elected not to lock the configuration, Xen will use GDS_CTRL to mitigate GDS with. Otherwise, Xen will mitigate by disabling AVX, which blocks the use of the AVX2 Gather instructions. +On all hardware, the `div-scrub=` option can be used to force or prevent Xen +from mitigating the DIV-leakage vulnerability. By default, Xen will mitigate +DIV-leakage on hardware believed to be vulnerable. + ### sync_console > `= ` diff --git a/xen/arch/x86/hvm/svm/entry.S b/xen/arch/x86/hvm/svm/entry.S index 9effd2199ba0..c52528fed4cf 100644 --- a/xen/arch/x86/hvm/svm/entry.S +++ b/xen/arch/x86/hvm/svm/entry.S @@ -72,6 +72,7 @@ __UNLIKELY_END(nsvm_hap) 1: /* No Spectre v1 concerns. Execution will hit VMRUN imminently. */ .endm ALTERNATIVE "", svm_vmentry_spec_ctrl, X86_FEATURE_SC_MSR_HVM + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV pop %r15 pop %r14 diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/asm/cpufeatures.h index da0593de8542..c3aad21c3b43 100644 --- a/xen/arch/x86/include/asm/cpufeatures.h +++ b/xen/arch/x86/include/asm/cpufeatures.h @@ -35,7 +35,7 @@ XEN_CPUFEATURE(SC_RSB_HVM, X86_SYNTH(19)) /* RSB overwrite needed for HVM XEN_CPUFEATURE(XEN_SELFSNOOP, X86_SYNTH(20)) /* SELFSNOOP gets used by Xen itself */ XEN_CPUFEATURE(SC_MSR_IDLE, X86_SYNTH(21)) /* Clear MSR_SPEC_CTRL on idle */ XEN_CPUFEATURE(XEN_LBR, X86_SYNTH(22)) /* Xen uses MSR_DEBUGCTL.LBR */ -/* Bits 23 unused. */ +XEN_CPUFEATURE(SC_DIV, X86_SYNTH(23)) /* DIV scrub needed */ XEN_CPUFEATURE(SC_RSB_IDLE, X86_SYNTH(24)) /* RSB overwrite needed for idle. */ XEN_CPUFEATURE(SC_VERW_IDLE, X86_SYNTH(25)) /* VERW used by Xen for idle */ XEN_CPUFEATURE(XEN_SHSTK, X86_SYNTH(26)) /* Xen uses CET Shadow Stacks */ diff --git a/xen/arch/x86/include/asm/spec_ctrl_asm.h b/xen/arch/x86/include/asm/spec_ctrl_asm.h index 9a27e3170347..5c5b3b6f5324 100644 --- a/xen/arch/x86/include/asm/spec_ctrl_asm.h +++ b/xen/arch/x86/include/asm/spec_ctrl_asm.h @@ -165,6 +165,19 @@ .L\@_verw_skip: .endm +.macro DO_SPEC_CTRL_DIV +/* + * Requires nothing + * Clobbers %rax + * + * Issue a DIV for its flushing side effect (Zen1 uarch specific). Any + * non-faulting DIV will do; a byte DIV has least latency, and doesn't clobber + * %rdx. + */ + mov $1, %eax + div %al +.endm + .macro DO_SPEC_CTRL_ENTRY maybexen:req /* * Requires %rsp=regs (also cpuinfo if !maybexen) @@ -267,6 +280,8 @@ ALTERNATIVE "", DO_SPEC_CTRL_EXIT_TO_GUEST, X86_FEATURE_SC_MSR_PV DO_SPEC_CTRL_COND_VERW + + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV .endm /* @@ -379,6 +394,8 @@ UNLIKELY_DISPATCH_LABEL(\@_serialise): verw STACK_CPUINFO_FIELD(verw_sel)(%r14) .L\@_skip_verw: + ALTERNATIVE "", DO_SPEC_CTRL_DIV, X86_FEATURE_SC_DIV + .L\@_skip_ist_exit: .endm diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index 9b8fdb5303ad..4c510094498e 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -67,6 +67,7 @@ static int8_t __initdata opt_srb_lock = -1; static bool __initdata opt_unpriv_mmio; static bool __ro_after_init opt_fb_clear_mmio; static int8_t __initdata opt_gds_mit = -1; +static int8_t __initdata opt_div_scrub = -1; static int __init cf_check parse_spec_ctrl(const char *s) { @@ -121,6 +122,7 @@ static int __init cf_check parse_spec_ctrl(const char *s) opt_srb_lock = 0; opt_unpriv_mmio = false; opt_gds_mit = 0; + opt_div_scrub = 0; } else if ( val > 0 ) rc = -EINVAL; @@ -273,6 +275,8 @@ static int __init cf_check parse_spec_ctrl(const char *s) opt_unpriv_mmio = val; else if ( (val = parse_boolean("gds-mit", s, ss)) >= 0 ) opt_gds_mit = val; + else if ( (val = parse_boolean("div-scrub", s, ss)) >= 0 ) + opt_div_scrub = val; else rc = -EINVAL; @@ -473,7 +477,7 @@ static void __init print_details(enum ind_thunk thunk) "\n"); /* Settings for Xen's protection, irrespective of guests. */ - printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s\n", + printk(" Xen settings: BTI-Thunk %s, SPEC_CTRL: %s%s%s%s%s, Other:%s%s%s%s%s%s\n", thunk == THUNK_NONE ? "N/A" : thunk == THUNK_RETPOLINE ? "RETPOLINE" : thunk == THUNK_LFENCE ? "LFENCE" : @@ -498,6 +502,7 @@ static void __init print_details(enum ind_thunk thunk) opt_l1d_flush ? " L1D_FLUSH" : "", opt_md_clear_pv || opt_md_clear_hvm || opt_fb_clear_mmio ? " VERW" : "", + opt_div_scrub ? " DIV" : "", opt_branch_harden ? " BRANCH_HARDEN" : ""); /* L1TF diagnostics, printed if vulnerable or PV shadowing is in use. */ @@ -955,6 +960,46 @@ static void __init srso_calculations(bool hw_smt_enabled) setup_force_cpu_cap(X86_FEATURE_SRSO_NO); } +/* + * The Div leakage issue is specific to the AMD Zen1 microarchitecure. + * + * However, there's no $FOO_NO bit defined, so if we're virtualised we have no + * hope of spotting the case where we might move to vulnerable hardware. We + * also can't make any useful conclusion about SMT-ness. + * + * Don't check the hypervisor bit, so at least we do the safe thing when + * booting on something that looks like a Zen1 CPU. + */ +static bool __init has_div_vuln(void) +{ + if ( !(boot_cpu_data.x86_vendor & + (X86_VENDOR_AMD | X86_VENDOR_HYGON)) ) + return false; + + if ( (boot_cpu_data.x86 != 0x17 && boot_cpu_data.x86 != 0x18) || + !is_zen1_uarch() ) + return false; + + return true; +} + +static void __init div_calculations(bool hw_smt_enabled) +{ + bool cpu_bug_div = has_div_vuln(); + + if ( opt_div_scrub == -1 ) + opt_div_scrub = cpu_bug_div; + + if ( opt_div_scrub ) + setup_force_cpu_cap(X86_FEATURE_SC_DIV); + + if ( opt_smt == -1 && !cpu_has_hypervisor && cpu_bug_div && hw_smt_enabled ) + warning_add( + "Booted on leaky-DIV hardware with SMT/Hyperthreading\n" + "enabled. Please assess your configuration and choose an\n" + "explicit 'smt=' setting. See XSA-439.\n"); +} + static void __init ibpb_calculations(void) { bool def_ibpb_entry = false; @@ -1714,6 +1759,8 @@ void __init init_speculation_mitigations(void) ibpb_calculations(); + div_calculations(hw_smt_enabled); + /* Check whether Eager FPU should be enabled by default. */ if ( opt_eager_fpu == -1 ) opt_eager_fpu = should_use_eager_fpu();