From patchwork Wed Oct 25 20:53:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 13436784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80CCBC25B6B for ; Wed, 25 Oct 2023 20:54:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235059AbjJYUyA (ORCPT ); Wed, 25 Oct 2023 16:54:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234602AbjJYUxj (ORCPT ); Wed, 25 Oct 2023 16:53:39 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3508BD4F; Wed, 25 Oct 2023 13:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698267209; x=1729803209; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=ZayczDqX1KVxTZCaG/Mto+AoTwEtKqxRVewv1TUrl9g=; b=B7z6gfUR0iRhssWVQRgUdvY70iijnpQcZd5BjLhHUuUxPCdFyJPaOi5G N8ijmU9RM5zJjru0U3QruA2A64Md9E91SVRFNn5LVRPDHU6s9265dongw bQHO9SbKIbC+UkTtzc/1k+zH5UaTx6hju9PV2JXkIWFZqWzg0m5qQxaxI SNSGcsxMQX1RFg99lTpEuWjYZbuax2v3++KEp2luYOGoz4po2NUfFrEZF SsgR33ifSADt1rYKArw/IpU6bBi5VXKFXdXhVyzL6eElGseYmrgWwFfkD G713srsINhFY7Ic9h717qeb9BAKM+GyRAA2/uB535BuG0ZX9zytuKfCOj A==; X-IronPort-AV: E=McAfee;i="6600,9927,10874"; a="367610052" X-IronPort-AV: E=Sophos;i="6.03,250,1694761200"; d="scan'208";a="367610052" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2023 13:53:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10874"; a="902683314" X-IronPort-AV: E=Sophos;i="6.03,250,1694761200"; d="scan'208";a="902683314" Received: from kkomeyli-mobl.amr.corp.intel.com (HELO desk) ([10.251.29.139]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2023 13:51:03 -0700 Date: Wed, 25 Oct 2023 13:53:24 -0700 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Pawan Gupta Subject: [PATCH v3 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20231025-delay-verw-v3-6-52663677ee35@linux.intel.com> X-Mailer: b4 0.12.3 References: <20231025-delay-verw-v3-0-52663677ee35@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231025-delay-verw-v3-0-52663677ee35@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 10 +++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index b3b13ec04bac..139960deb736 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 24e8694b83fc..2d149589cf5b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7226,13 +7226,17 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed inspite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) + /* MMIO mitigation is mutually exclusive with MDS mitigation later in asm */ mds_clear_cpu_buffers(); vmx_disable_fb_clear(vmx);