From patchwork Thu May 25 12:15:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Lagerwall X-Patchwork-Id: 9748313 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 788526032C for ; Thu, 25 May 2017 12:18:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 677E827D4A for ; Thu, 25 May 2017 12:18:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BB8B27EE2; Thu, 25 May 2017 12:18:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6461A27E5A for ; Thu, 25 May 2017 12:18:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dDrg8-0000Tl-2C; Thu, 25 May 2017 12:15:44 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dDrg6-0000TZ-Cp for xen-devel@lists.xen.org; Thu, 25 May 2017 12:15:42 +0000 Received: from [85.158.137.68] by server-16.bemta-3.messagelabs.com id 36/B2-29088-DEAC6295; Thu, 25 May 2017 12:15:41 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNLMWRWlGSWpSXmKPExsXitHSDve7bU2q RBut2MVks+biYxYHR4+ju30wBjFGsmXlJ+RUJrBnTD0kU3FCoeLj/CXsD4xPJLkZODgkBf4nm Y+uZQGw2AQOJW5e+M4PYIgKyEqu75rB3MXJxMAt8ZJR4cOwFK0hCWCBQYuWTyewgNouAqsTqp vdADRwcvAJ2Er97fCFmykks3X4dbI6QgJrE2+VnWEBsXgFBiZMzn4DZzAISEgdfvGCewMg9C0 lqFpLUAkamVYzqxalFZalFuuZ6SUWZ6RkluYmZObqGBsZ6uanFxYnpqTmJScV6yfm5mxiBgcA ABDsYG787HWKU5GBSEuUVn6sWKcSXlJ9SmZFYnBFfVJqTWnyIUYaDQ0mCt/AkUE6wKDU9tSIt MwcYkjBpCQ4eJRHeu8eB0rzFBYm5xZnpEKlTjIpS4rxbTgAlBEASGaV5cG2wOLjEKCslzMsId IgQT0FqUW5mCar8K0ZxDkYlYd6ZINt5MvNK4Ka/AlrMBLTY9a4yyOKSRISUVAPj1tvLLgaabZ uda/L6/U6FjZGLs1/xlAkdjhJsODvlaviRbTYSnALtz+1LpNISrA+V16tHFpm2W/RM0E+YLs5 Skv9rNrNV4UuxnVmnWp0+7zQSauVzeR56ae05oXiNqmU8RepOOdpTHy5SW8PY/POai+r8lrMP 7h6aKMgTktz5OGD6sl+npkQosRRnJBpqMRcVJwIAzMdZpn4CAAA= X-Env-Sender: prvs=311f2fc21=ross.lagerwall@citrix.com X-Msg-Ref: server-9.tower-31.messagelabs.com!1495714539!46996148!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 7647 invoked from network); 25 May 2017 12:15:40 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-9.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 25 May 2017 12:15:40 -0000 X-IronPort-AV: E=Sophos;i="5.38,391,1491264000"; d="scan'208";a="434005283" From: Ross Lagerwall To: Date: Thu, 25 May 2017 13:15:33 +0100 Message-ID: <20170525121533.32409-1-ross.lagerwall@citrix.com> X-Mailer: git-send-email 2.9.4 MIME-Version: 1.0 Cc: Sergey Dyasli , Kevin Tian , Jun Nakajima , Andrew Cooper , Ross Lagerwall , Jan Beulich Subject: [Xen-devel] [PATCH] x86/vmx: Fix vmentry failure because of invalid LER on Broadwell X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Occasionally, the top three bits of MSR_IA32_LASTINTTOIP (MSR_LER_TO_LIP) may be incorrect, as though the MSR is using the LBR_FORMAT_EIP_FLAGS_TSX format. The MSR should contain an offset into the current code segment according to the Intel documentation. It is not clear why this happens. It may be due to erratum BDF14, or some other errata. The result is a vmentry failure. Workaround the issue by sign-extending into bits 61:63 for this MSR on Broadwell CPUs. --- xen/arch/x86/hvm/vmx/vmx.c | 29 +++++++++++++++++++++++++++++ xen/include/asm-x86/hvm/vmx/vmcs.h | 1 + 2 files changed, 30 insertions(+) diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index c8ef18a..7d729af 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -2434,6 +2434,7 @@ static void pi_notification_interrupt(struct cpu_user_regs *regs) } static void __init lbr_tsx_fixup_check(void); +static void __init ler_bdw_fixup_check(void); const struct hvm_function_table * __init start_vmx(void) { @@ -2499,6 +2500,7 @@ const struct hvm_function_table * __init start_vmx(void) setup_vmcs_dump(); lbr_tsx_fixup_check(); + ler_bdw_fixup_check(); return &vmx_function_table; } @@ -2790,8 +2792,10 @@ enum }; #define LBR_FROM_SIGNEXT_2MSB ((1ULL << 59) | (1ULL << 60)) +#define LER_TO_SIGNEXT_3MSB (LBR_FROM_SIGNEXT_2MSB | (1ULL << 58)) static bool __read_mostly lbr_tsx_fixup_needed; +static bool __read_mostly ler_bdw_fixup_needed; static uint32_t __read_mostly lbr_from_start; static uint32_t __read_mostly lbr_from_end; static uint32_t __read_mostly lbr_lastint_from; @@ -2828,6 +2832,13 @@ static void __init lbr_tsx_fixup_check(void) } } +static void __init ler_bdw_fixup_check(void) +{ + /* Broadwell E5-2600 v4 processors need to work around erratum BDF14. */ + if ( boot_cpu_data.x86 == 6 && boot_cpu_data.x86_model == 79 ) + ler_bdw_fixup_needed = true; +} + static int is_last_branch_msr(u32 ecx) { const struct lbr_info *lbr = last_branch_msr_get(); @@ -3089,6 +3100,8 @@ static int vmx_msr_write_intercept(unsigned int msr, uint64_t msr_content) vmx_disable_intercept_for_msr(v, lbr->base + i, MSR_TYPE_R | MSR_TYPE_W); v->arch.hvm_vmx.lbr_tsx_fixup_enabled = lbr_tsx_fixup_needed; + v->arch.hvm_vmx.ler_bdw_fixup_enabled = + ler_bdw_fixup_needed; } } @@ -4174,6 +4187,20 @@ static void lbr_tsx_fixup(void) msr->data |= ((LBR_FROM_SIGNEXT_2MSB & msr->data) << 2); } +static void ler_bdw_fixup(void) +{ + struct vmx_msr_entry *msr; + + /* + * Occasionally, the top three bits of MSR_IA32_LASTINTTOIP may be + * incorrect (possibly due to BDF14), as though the MSR is using the + * LBR_FORMAT_EIP_FLAGS_TSX format. This is incorrect and causes a vmentry + * failure -- the MSR should contain an offset into the current code + * segment. Fix it up by sign-extending into bits 61:63. */ + if ( (msr = vmx_find_msr(MSR_IA32_LASTINTTOIP, VMX_GUEST_MSR)) != NULL ) + msr->data |= ((LER_TO_SIGNEXT_3MSB & msr->data) << 3); +} + void vmx_vmenter_helper(const struct cpu_user_regs *regs) { struct vcpu *curr = current; @@ -4232,6 +4259,8 @@ void vmx_vmenter_helper(const struct cpu_user_regs *regs) out: if ( unlikely(curr->arch.hvm_vmx.lbr_tsx_fixup_enabled) ) lbr_tsx_fixup(); + if ( unlikely(curr->arch.hvm_vmx.ler_bdw_fixup_enabled) ) + ler_bdw_fixup(); HVMTRACE_ND(VMENTRY, 0, 1/*cycles*/, 0, 0, 0, 0, 0, 0, 0); diff --git a/xen/include/asm-x86/hvm/vmx/vmcs.h b/xen/include/asm-x86/hvm/vmx/vmcs.h index 9507bd2..aedef82 100644 --- a/xen/include/asm-x86/hvm/vmx/vmcs.h +++ b/xen/include/asm-x86/hvm/vmx/vmcs.h @@ -137,6 +137,7 @@ struct arch_vmx_struct { uint8_t vmx_emulate; bool lbr_tsx_fixup_enabled; + bool ler_bdw_fixup_enabled; /* Bitmask of segments that we can't safely use in virtual 8086 mode */ uint16_t vm86_segment_mask;