From patchwork Sun Mar 19 08:49:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0CD2C7618E for ; Sun, 19 Mar 2023 08:49:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229689AbjCSItl (ORCPT ); Sun, 19 Mar 2023 04:49:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229619AbjCSIti (ORCPT ); Sun, 19 Mar 2023 04:49:38 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACDB91A48F for ; Sun, 19 Mar 2023 01:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215777; x=1710751777; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qvxOPplvEv1zyAd/qa0KOc29zbye8sCt8fS0alfd2Vc=; b=j8fZXclDp8nQcUEHgNtALGrmGb+oKDpIFNycbbBpzk41WKEIpjMb+2It IQksq0o8DOw6zY9z6XyBfkWCljomnBjZ+2+uMhFmQsRegw1w2sBrLx4Hi fveTxTkx0lbFL1DUpWKEWxSFtGDzORWic9sq5oPE1ti14ku3tbeQU7ybZ 6RjclXuTh0dxyIqzlHE52Sz3Z0PV3q1pjB6aFcsoHN7tQbIsDUk50g+rE PJirH6WxCsLCbNQof7Q3YvUWEXVLHWbhDr3/z9QocUPJY10HCOXK5dynp urYWeLbYSnCvSpVnDlcnrZYDKJWRX99iO5G7uif2ISXazJmcszstpKKnK w==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767836" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767836" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146235" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146235" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:35 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 1/7] KVM: x86: Explicitly cast ulong to bool in kvm_set_cr3() Date: Sun, 19 Mar 2023 16:49:21 +0800 Message-Id: <20230319084927.29607-2-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Robert Hoo kvm_read_cr4_bits() returns ulong, explicitly cast it bool when assign to a bool variable. Signed-off-by: Robert Hoo Signed-off-by: Binbin Wu --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 810cbe3b3ba6..410327e7eb55 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1238,7 +1238,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) bool skip_tlb_flush = false; unsigned long pcid = 0; #ifdef CONFIG_X86_64 - bool pcid_enabled = kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE); + bool pcid_enabled = !!kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE); if (pcid_enabled) { skip_tlb_flush = cr3 & X86_CR3_PCID_NOFLUSH; From patchwork Sun Mar 19 08:49:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74919C6FD1F for ; Sun, 19 Mar 2023 08:49:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229878AbjCSItn (ORCPT ); Sun, 19 Mar 2023 04:49:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230223AbjCSItl (ORCPT ); Sun, 19 Mar 2023 04:49:41 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41C391A496 for ; Sun, 19 Mar 2023 01:49:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215780; x=1710751780; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xNrunTdo0vvnnwWCoVCBRwV5lHAA0UcXRvs5A3f4bu8=; b=SH6ZguCi/q0MpA9m68z39itENCOAAVQgwq6upgHO+sT+Kq0oN1KLAlQM xPb5K6LHmE9bsiM6Kj2EW6S0AhD1uh8cQnIpO16X/25l1U//GztuTj/3H Vlk5t1XW754bmBc0bhWd82LSjYfWEo+0nIumHdz7FcdM1bVxIGEnq2qsW SGNXy8hdhKxxM7BuM5loALVr2Mh2gSLAfoZW56YuRjiz7p4UTAVdU75tC XGdTNet4bkWji6uoTyaiwoZz8popL0/GZY8u0b7agZAAflVcAbyEbR9Z+ eLABbY50mEq/JIkIXg+R0sF78TthTjSpuTSy6ujkMKlfl3wgBvpJV98sI Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767844" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767844" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146273" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146273" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:38 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 2/7] KVM: VMX: Use is_64_bit_mode() to check 64-bit mode Date: Sun, 19 Mar 2023 16:49:22 +0800 Message-Id: <20230319084927.29607-3-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org get_vmx_mem_address() and sgx_get_encls_gva() use is_long_mode() to check 64-bit mode. Should use is_64_bit_mode() instead. Fixes: f9eb4af67c9d ("KVM: nVMX: VMX instructions: add checks for #GP/#SS exceptions") Fixes: 70210c044b4e ("KVM: VMX: Add SGX ENCLS[ECREATE] handler to enforce CPUID restrictions") Signed-off-by: Binbin Wu Reviewed-by: Kai Huang --- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/vmx/sgx.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 557b9c468734..0f84cc05f57c 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4959,7 +4959,7 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification, /* Checks for #GP/#SS exceptions. */ exn = false; - if (is_long_mode(vcpu)) { + if (is_64_bit_mode(vcpu)) { /* * The virtual/linear address is never truncated in 64-bit * mode, e.g. a 32-bit address size can yield a 64-bit virtual diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index aa53c98034bf..0574030b071f 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -29,14 +29,14 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset, /* Skip vmcs.GUEST_DS retrieval for 64-bit mode to avoid VMREADs. */ *gva = offset; - if (!is_long_mode(vcpu)) { + if (!is_64_bit_mode(vcpu)) { vmx_get_segment(vcpu, &s, VCPU_SREG_DS); *gva += s.base; } if (!IS_ALIGNED(*gva, alignment)) { fault = true; - } else if (likely(is_long_mode(vcpu))) { + } else if (likely(is_64_bit_mode(vcpu))) { fault = is_noncanonical_address(*gva, vcpu); } else { *gva &= 0xffffffff; From patchwork Sun Mar 19 08:49:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71462C7618E for ; Sun, 19 Mar 2023 08:50:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230266AbjCSItt (ORCPT ); Sun, 19 Mar 2023 04:49:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229968AbjCSItp (ORCPT ); Sun, 19 Mar 2023 04:49:45 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 345AB1ABD1 for ; Sun, 19 Mar 2023 01:49:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215783; x=1710751783; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6woDpsARbqGdJJRxLiAQ9w4hoqV7TtoFViGE/ANx9bY=; b=bDsUYNyZfiZHwknzyNtF0Pd5VMXVgxJb0c/Tb01SdcqM9fdSCTwRk6qe qPRIhC6xtCSofLjuL6Asy2xKVNIrxGuJtsWT8bf9VLsFvKWhGvRaAGdvJ 9OmHAm4bnbLMac+BlBFKM3I0sz2vvM/Z2WBsA5qilITnFlnMapToT/bSh ROC5hiQo9KfbahpxyiSP3FMa+K42EhiWNT1MUhZblD9ysLGqTAuvCiYlH jkFHxHtLt7RDkx121OJHYYOIlKomXvadhq2fRgB4dm72MZO1WpwWxlqfh 7btrk9Wle2dcefWyKz26Zbz7UbUl0okBSVM3eKasi1zQOSe5kl4e3bilX w==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767853" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767853" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146291" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146291" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:41 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 3/7] KVM: x86: Virtualize CR4.LAM_SUP Date: Sun, 19 Mar 2023 16:49:23 +0800 Message-Id: <20230319084927.29607-4-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Robert Hoo Allow setting of CR4.LAM_SUP (bit 28) by guest if vCPU supports LAM, and intercept the bit (as it already is). LAM uses CR4.LAM_SUP to configure LAM masking on supervisor mode address. To virtualize that, move CR4.LAM_SUP out of CR4_RESERVED_BITS and its reservation depends on vCPU has LAM feature or not. CR4.LAM_SUP is allowed to be set even not in 64-bit mode, but it will not take effect since LAM only applies to 64-bit linear address. Leave the bit intercepted to avoid vmread every time when KVM fetches its value, with the expectation that guest won't toggle the bit frequently. Hardware is not required to do TLB flush when CR4.LAM_SUP toggled, so KVM doesn't need to emulate TLB flush based on it. There's no other features/vmx_exec_controls connection, therefore no code need to be complemented in kvm/vmx_set_cr4(). Signed-off-by: Robert Hoo Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu Reviewed-by: Chao Gao --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/vmx/vmx.c | 3 +++ arch/x86/kvm/x86.h | 2 ++ 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 8b38a4cb2e29..742fd84c7997 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -125,7 +125,8 @@ | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \ | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \ - | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP)) + | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \ + | X86_CR4_LAM_SUP)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index bbf60bda877e..66a50224293e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7617,6 +7617,9 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu) cr4_fixed1_update(X86_CR4_UMIP, ecx, feature_bit(UMIP)); cr4_fixed1_update(X86_CR4_LA57, ecx, feature_bit(LA57)); + entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1); + cr4_fixed1_update(X86_CR4_LAM_SUP, eax, feature_bit(LAM)); + #undef cr4_fixed1_update } diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9de72586f406..fe32554e0da6 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -475,6 +475,8 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, u32 type); __reserved_bits |= X86_CR4_VMXE; \ if (!__cpu_has(__c, X86_FEATURE_PCID)) \ __reserved_bits |= X86_CR4_PCIDE; \ + if (!__cpu_has(__c, X86_FEATURE_LAM)) \ + __reserved_bits |= X86_CR4_LAM_SUP; \ __reserved_bits; \ }) From patchwork Sun Mar 19 08:49:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D82AC7618A for ; Sun, 19 Mar 2023 08:50:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230118AbjCSIuM (ORCPT ); Sun, 19 Mar 2023 04:50:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230263AbjCSItt (ORCPT ); Sun, 19 Mar 2023 04:49:49 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB25418B29 for ; Sun, 19 Mar 2023 01:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215786; x=1710751786; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AF53uty/FJI81fea3GlOtXMqGAV27RC5qpM/OnDRXXE=; b=Q0TALjFrmP527rlZalUhjDwBVWvxEK7nxAg/GjC1OY5mwnD5SW9Vph3o 1HCjeYbxfAaeIOW/eUUTqZCgQ76joGUCK4by5tiG4yovrYlyD93UT/A7F c2kydT5R0Q1ocGbhLydfrhsPVCOUMfJmHHCtsSJ7u/a2t/jlsh1Kly0Fs 3ajh7YWZLeQJxaMM/SfGZ4aIY6KjUuLNN+y0acyE/M5q6hK9BmhM9mbW2 4/22jPEI19XfWCl8AM/GwgBZeViJuh7yi/u2FDOyGkwAjGp7x3mAEhse6 YbYwcyXqfixPjZaF5K6X8goa/nm0oEOMbQT4ZTIncf8uAo2Bc4g+HzHOP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767862" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767862" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146306" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146306" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:44 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 4/7] KVM: x86: Virtualize CR3.LAM_{U48,U57} Date: Sun, 19 Mar 2023 16:49:24 +0800 Message-Id: <20230319084927.29607-5-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Robert Hoo LAM uses CR3.LAM_U48 (bit 62) and CR3.LAM_U57 (bit 61) to configure LAM masking for user mode pointers. When EPT is on: CR3 is fully under control of guest, guest LAM is thus transparent to KVM. When EPT is off (shadow paging): KVM needs to handle guest CR3.LAM_U48 and CR3.LAM_U57 toggles. The two bits don't participate in page table walking. They should be masked to get the base address of page table. When shadow paging is used, the two bits should be kept as they are in the shadow CR3. To be generic, introduce a field 'cr3_ctrl_bits' in kvm_vcpu_arch to record the bits used to control supported features related to CR3 (e.g. LAM). - Supported control bits are set to cr3_ctrl_bits. - Add kvm_vcpu_is_legal_cr3() to validate CR3, allow setting of the control bits for the supported features. - cr3_ctrl_bits is used to mask the control bits when calculate the base address of page table from mmu::get_guest_pgd(). - Add kvm_get_active_cr3_ctrl_bits() to get the active control bits to form a new guest CR3 (in vmx_load_mmu_pgd()). - For only control bits toggle cases, it is unnecessary to make new pgd, but just make request of load pgd. Especially, for ONLY-LAM-bits toggle cases, skip TLB flush since hardware is not required to flush TLB when CR3 LAM bits toggled. Signed-off-by: Robert Hoo Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu --- arch/x86/include/asm/kvm_host.h | 7 +++++++ arch/x86/kvm/cpuid.h | 5 +++++ arch/x86/kvm/mmu.h | 5 +++++ arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/vmx/nested.c | 6 +++--- arch/x86/kvm/vmx/vmx.c | 6 +++++- arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++------ 8 files changed, 50 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 742fd84c7997..2174ad27013b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -730,6 +730,13 @@ struct kvm_vcpu_arch { unsigned long cr0_guest_owned_bits; unsigned long cr2; unsigned long cr3; + /* + * Bits in CR3 used to enable certain features. These bits don't + * participate in page table walking. They should be masked to + * get the base address of page table. When shadow paging is + * used, these bits should be kept as they are in the shadow CR3. + */ + u64 cr3_ctrl_bits; unsigned long cr4; unsigned long cr4_guest_owned_bits; unsigned long cr4_guest_rsvd_bits; diff --git a/arch/x86/kvm/cpuid.h b/arch/x86/kvm/cpuid.h index b1658c0de847..ef8e1b912d7d 100644 --- a/arch/x86/kvm/cpuid.h +++ b/arch/x86/kvm/cpuid.h @@ -42,6 +42,11 @@ static inline int cpuid_maxphyaddr(struct kvm_vcpu *vcpu) return vcpu->arch.maxphyaddr; } +static inline bool kvm_vcpu_is_legal_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) +{ + return !((cr3 & vcpu->arch.reserved_gpa_bits) & ~vcpu->arch.cr3_ctrl_bits); +} + static inline bool kvm_vcpu_is_legal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa) { return !(gpa & vcpu->arch.reserved_gpa_bits); diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 168c46fd8dd1..29985eeb8e12 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -142,6 +142,11 @@ static inline unsigned long kvm_get_active_pcid(struct kvm_vcpu *vcpu) return kvm_get_pcid(vcpu, kvm_read_cr3(vcpu)); } +static inline u64 kvm_get_active_cr3_ctrl_bits(struct kvm_vcpu *vcpu) +{ + return kvm_read_cr3(vcpu) & vcpu->arch.cr3_ctrl_bits; +} + static inline void kvm_mmu_load_pgd(struct kvm_vcpu *vcpu) { u64 root_hpa = vcpu->arch.mmu->root.hpa; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index aeb240b339f5..e0b86ace7326 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3722,7 +3722,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) int quadrant, i, r; hpa_t root; - root_pgd = mmu->get_guest_pgd(vcpu); + root_pgd = mmu->get_guest_pgd(vcpu) & ~vcpu->arch.cr3_ctrl_bits; root_gfn = root_pgd >> PAGE_SHIFT; if (mmu_check_root(vcpu, root_gfn)) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index e5662dbd519c..8887615534b0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -324,7 +324,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, trace_kvm_mmu_pagetable_walk(addr, access); retry_walk: walker->level = mmu->cpu_role.base.level; - pte = mmu->get_guest_pgd(vcpu); + pte = mmu->get_guest_pgd(vcpu) & ~vcpu->arch.cr3_ctrl_bits; have_ad = PT_HAVE_ACCESSED_DIRTY(mmu); #if PTTYPE == 64 diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 0f84cc05f57c..2eb258992d63 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1079,7 +1079,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool nested_ept, bool reload_pdptrs, enum vm_entry_failure_code *entry_failure_code) { - if (CC(kvm_vcpu_is_illegal_gpa(vcpu, cr3))) { + if (CC(!kvm_vcpu_is_legal_cr3(vcpu, cr3))) { *entry_failure_code = ENTRY_FAIL_DEFAULT; return -EINVAL; } @@ -1101,7 +1101,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, kvm_init_mmu(vcpu); if (!nested_ept) - kvm_mmu_new_pgd(vcpu, cr3); + kvm_mmu_new_pgd(vcpu, cr3 & ~vcpu->arch.cr3_ctrl_bits); return 0; } @@ -2907,7 +2907,7 @@ static int nested_vmx_check_host_state(struct kvm_vcpu *vcpu, if (CC(!nested_host_cr0_valid(vcpu, vmcs12->host_cr0)) || CC(!nested_host_cr4_valid(vcpu, vmcs12->host_cr4)) || - CC(kvm_vcpu_is_illegal_gpa(vcpu, vmcs12->host_cr3))) + CC(!kvm_vcpu_is_legal_cr3(vcpu, vmcs12->host_cr3))) return -EINVAL; if (CC(is_noncanonical_address(vmcs12->host_ia32_sysenter_esp, vcpu)) || diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 66a50224293e..9638a3000256 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3390,7 +3390,8 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, update_guest_cr3 = false; vmx_ept_load_pdptrs(vcpu); } else { - guest_cr3 = root_hpa | kvm_get_active_pcid(vcpu); + guest_cr3 = root_hpa | kvm_get_active_pcid(vcpu) | + kvm_get_active_cr3_ctrl_bits(vcpu); } if (update_guest_cr3) @@ -7750,6 +7751,9 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) vmx->msr_ia32_feature_control_valid_bits &= ~FEAT_CTL_SGX_LC_ENABLED; + if (guest_cpuid_has(vcpu, X86_FEATURE_LAM)) + vcpu->arch.cr3_ctrl_bits |= X86_CR3_LAM_U48 | X86_CR3_LAM_U57; + /* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap(vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 410327e7eb55..e74af72f53ec 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1236,7 +1236,7 @@ static void kvm_invalidate_pcid(struct kvm_vcpu *vcpu, unsigned long pcid) int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) { bool skip_tlb_flush = false; - unsigned long pcid = 0; + unsigned long pcid = 0, old_cr3; #ifdef CONFIG_X86_64 bool pcid_enabled = !!kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE); @@ -1247,8 +1247,9 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) } #endif + old_cr3 = kvm_read_cr3(vcpu); /* PDPTRs are always reloaded for PAE paging. */ - if (cr3 == kvm_read_cr3(vcpu) && !is_pae_paging(vcpu)) + if (cr3 == old_cr3 && !is_pae_paging(vcpu)) goto handle_tlb_flush; /* @@ -1256,14 +1257,30 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) * stuff CR3, e.g. for RSM emulation, and there is no guarantee that * the current vCPU mode is accurate. */ - if (kvm_vcpu_is_illegal_gpa(vcpu, cr3)) + if (!kvm_vcpu_is_legal_cr3(vcpu, cr3)) return 1; if (is_pae_paging(vcpu) && !load_pdptrs(vcpu, cr3)) return 1; - if (cr3 != kvm_read_cr3(vcpu)) - kvm_mmu_new_pgd(vcpu, cr3); + if (cr3 != old_cr3) { + if ((cr3 ^ old_cr3) & ~vcpu->arch.cr3_ctrl_bits) { + kvm_mmu_new_pgd(vcpu, cr3 & ~vcpu->arch.cr3_ctrl_bits); + } else { + /* + * Though only control (LAM) bits changed, make the + * request to force an update on guest CR3 because the + * control (LAM) bits are stale + */ + kvm_make_request(KVM_REQ_LOAD_MMU_PGD, vcpu); + /* + * HW is not required to flush TLB when CR3 LAM bits toggled. + * Currently only LAM bits in cr3_ctrl_bits, if more bits added in + * the future, need to check whether to skip TLB flush or not. + */ + skip_tlb_flush = true; + } + } vcpu->arch.cr3 = cr3; kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3); @@ -11305,7 +11322,7 @@ static bool kvm_is_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) */ if (!(sregs->cr4 & X86_CR4_PAE) || !(sregs->efer & EFER_LMA)) return false; - if (kvm_vcpu_is_illegal_gpa(vcpu, sregs->cr3)) + if (!kvm_vcpu_is_legal_cr3(vcpu, sregs->cr3)) return false; } else { /* From patchwork Sun Mar 19 08:49:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78EF8C6FD1F for ; Sun, 19 Mar 2023 08:50:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230240AbjCSIuN (ORCPT ); Sun, 19 Mar 2023 04:50:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230280AbjCSItu (ORCPT ); Sun, 19 Mar 2023 04:49:50 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF9C64EF9 for ; Sun, 19 Mar 2023 01:49:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215788; x=1710751788; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dAcsoSCcNW+4yrJIcgA3nPHYIpfL/iT+WlimwslqWHA=; b=IAKQmAGWBLeNOx0Aex65UFr8ehNXd/0BRTwW+dI3JQJcJBYq0ZM8xaaG nQG5IaSodHkbvyl6BQJhIfoDakfXJAY9l/cgRVFJRCi689DYFrk1sUWX9 y5Gl7K141X/9cxcdMK32d48bigKSHUbNM6e6cDicrPfi0K4/zrwEacS6n jLoYcHP7LVhww8JeyQgEJBTsomRuesc7bhqCIUvScieU8VuWeDVtTnW28 4mjZSBTqqVzJGVgoU+7QKp8EHqL44iW8LSAqHSP94lWPvGiKkwQy3nA8Q 1Du13RmcANoEB2u99GP9oK9fXcnHydaow6g2n/7KE2y6GMXKvilo2Yr89 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767869" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767869" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146346" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146346" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:46 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 5/7] KVM: x86: Introduce untag_addr() in kvm_x86_ops Date: Sun, 19 Mar 2023 16:49:25 +0800 Message-Id: <20230319084927.29607-6-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce a new interface untag_addr() to kvm_x86_ops to untag the metadata from linear address. Implement LAM version in VMX and dummy version in SVM. When enabled feature like Intel Linear Address Masking or AMD Upper Address Ignore, linear address may be tagged with metadata. Linear address should be checked for modified canonicality and untagged in instrution emulations or vmexit handlings if LAM or UAI is applicable. Introduce untag_addr() to kvm_x86_ops to hide the code related to vendor specific details. - For VMX, LAM version is implemented. LAM has a modified canonical check when applicable: * LAM_S48 : [ 1 ][ metadata ][ 1 ] 63 47 * LAM_U48 : [ 0 ][ metadata ][ 0 ] 63 47 * LAM_S57 : [ 1 ][ metadata ][ 1 ] 63 56 * LAM_U57 + 5-lvl paging : [ 0 ][ metadata ][ 0 ] 63 56 * LAM_U57 + 4-lvl paging : [ 0 ][ metadata ][ 0...0 ] 63 56..47 If LAM is applicable to certain address, untag the metadata bits and replace them with the value of bit 47 (LAM48) or bit 56 (LAM57). Later the untagged address will do legacy canonical check. So that LAM canonical check and mask can be covered by "untag + legacy canonical check". For cases LAM is not applicable, 'flags' is passed to the interface to skip untag. - For SVM, add a dummy version to do nothing, but return the original address. Signed-off-by: Binbin Wu --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 5 +++ arch/x86/kvm/svm/svm.c | 7 ++++ arch/x86/kvm/vmx/vmx.c | 60 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.h | 2 + 5 files changed, 75 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 8dc345cc6318..7d63d1b942ac 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -52,6 +52,7 @@ KVM_X86_OP(cache_reg) KVM_X86_OP(get_rflags) KVM_X86_OP(set_rflags) KVM_X86_OP(get_if_flag) +KVM_X86_OP(untag_addr) KVM_X86_OP(flush_tlb_all) KVM_X86_OP(flush_tlb_current) KVM_X86_OP_OPTIONAL(tlb_remote_flush) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 2174ad27013b..dd34041e3a4a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -69,6 +69,9 @@ #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS (KVM_X86_NOTIFY_VMEXIT_ENABLED | \ KVM_X86_NOTIFY_VMEXIT_USER) +/* flags for kvm_x86_ops::untag_addr() */ +#define KVM_X86_UNTAG_ADDR_SKIP_LAM _BITULL(0) + /* x86-specific vcpu->requests bit members */ #define KVM_REQ_MIGRATE_TIMER KVM_ARCH_REQ(0) #define KVM_REQ_REPORT_TPR_ACCESS KVM_ARCH_REQ(1) @@ -1597,6 +1600,8 @@ struct kvm_x86_ops { void (*set_rflags)(struct kvm_vcpu *vcpu, unsigned long rflags); bool (*get_if_flag)(struct kvm_vcpu *vcpu); + u64 (*untag_addr)(struct kvm_vcpu *vcpu, u64 la, u64 flags); + void (*flush_tlb_all)(struct kvm_vcpu *vcpu); void (*flush_tlb_current)(struct kvm_vcpu *vcpu); int (*tlb_remote_flush)(struct kvm *kvm); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d13cf53e7390..48ce80235728 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4696,6 +4696,11 @@ static int svm_vm_init(struct kvm *kvm) return 0; } +static u64 svm_untag_addr(struct kvm_vcpu *vcpu, u64 addr, u64 flags) +{ + return addr; +} + static struct kvm_x86_ops svm_x86_ops __initdata = { .name = KBUILD_MODNAME, @@ -4745,6 +4750,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .set_rflags = svm_set_rflags, .get_if_flag = svm_get_if_flag, + .untag_addr = svm_untag_addr, + .flush_tlb_all = svm_flush_tlb_current, .flush_tlb_current = svm_flush_tlb_current, .flush_tlb_gva = svm_flush_tlb_gva, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 9638a3000256..736544f20709 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8124,6 +8124,64 @@ static void vmx_vm_destroy(struct kvm *kvm) free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm)); } + +#define LAM_S57_EN_MASK (X86_CR4_LAM_SUP | X86_CR4_LA57) + +static int lam_sign_extend_bit(bool user, struct kvm_vcpu *vcpu) +{ + u64 cr3, cr4; + + if (user) { + cr3 = kvm_read_cr3(vcpu); + if (cr3 & X86_CR3_LAM_U57) + return 56; + if (cr3 & X86_CR3_LAM_U48) + return 47; + } else { + cr4 = kvm_read_cr4_bits(vcpu, LAM_S57_EN_MASK); + if (cr4 == LAM_S57_EN_MASK) + return 56; + if (cr4 & X86_CR4_LAM_SUP) + return 47; + } + return -1; +} + +/* + * Only called in 64-bit mode. + * + * Mask metadata in pointers by sign-extending the value of bit 47 (LAM48) or + * 56 (LAM57). Metadata are bits [62:48] in LAM48 and are [62:57] in LAM57. + * The resulting address after untagging isn't guaranteed to be canonical. + * Callers should perform the original canonical check and raise #GP/#SS if the + * address is non-canonical. + */ +u64 vmx_untag_addr(struct kvm_vcpu *vcpu, u64 addr, u64 flags) +{ + int sign_ext_bit; + + /* + * Instead of calling relatively expensive guest_cpuid_has(), just check + * LAM_U48 in cr3_ctrl_bits. If not set, vCPU doesn't supports LAM. + */ + if (!(vcpu->arch.cr3_ctrl_bits & X86_CR3_LAM_U48) || + (flags & KVM_X86_UNTAG_ADDR_SKIP_LAM)) + return addr; + + if(!is_64_bit_mode(vcpu)){ + WARN_ONCE(1, "Only be called in 64-bit mode"); + return addr; + } + + sign_ext_bit = lam_sign_extend_bit(!(addr >> 63), vcpu); + + if (sign_ext_bit < 0) + return addr; + + return (sign_extend64(addr, sign_ext_bit) & ~BIT_ULL(63)) | + (addr & BIT_ULL(63)); +} + static struct kvm_x86_ops vmx_x86_ops __initdata = { .name = KBUILD_MODNAME, @@ -8172,6 +8230,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .set_rflags = vmx_set_rflags, .get_if_flag = vmx_get_if_flag, + .untag_addr = vmx_untag_addr, + .flush_tlb_all = vmx_flush_tlb_all, .flush_tlb_current = vmx_flush_tlb_current, .flush_tlb_gva = vmx_flush_tlb_gva, diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index a3da84f4ea45..023d9b359ded 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type); u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu); u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu); +u64 vmx_untag_addr(struct kvm_vcpu *vcpu, u64 addr, u64 flags); + static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 msr, int type, bool value) { From patchwork Sun Mar 19 08:49:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57B55C7618E for ; Sun, 19 Mar 2023 08:50:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230244AbjCSIuO (ORCPT ); Sun, 19 Mar 2023 04:50:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230306AbjCSIty (ORCPT ); Sun, 19 Mar 2023 04:49:54 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C29DA15C9A for ; Sun, 19 Mar 2023 01:49:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215791; x=1710751791; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8uCV2A81ab+R8cDhygTv/C4AdyqKeKY6KFi+lhcY65Q=; b=lJNNO4y3Za4Vm4ivGpNM6CC4RkGKq0sks0H37TRjlH2OY8Nu11PsEZhs THgZz7kSpcUcKmrZLWgo1ZLE311uMI1ibE08B/yzdFHvXfqyTTq/sFt2l jBoiVG43y85RCineDbpa1icFGD5Dwa3Q3+FKDN8b87HII37C2Og7vFHYL sbiTU7EorfqGTh2t4ggmwld/uPpmnmkH5UAMRXy3vfgOifIy3oOojqNmi o/zXdETljyrj6mH9reYXfraOUPBSq5O6o1hZGMX7B9ZyAc4oSTeyLU9aj IAnedQyjzqPGhJqSxsGLn2rzYrIc9yXRiANburVpJj6rOND3QR5qdokhX Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767880" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767880" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146368" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146368" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:49 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com Subject: [PATCH v6 6/7] KVM: x86: Untag address when LAM applicable Date: Sun, 19 Mar 2023 16:49:26 +0800 Message-Id: <20230319084927.29607-7-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Untag address for 64-bit memory/mmio operand in instruction emulations and vmexit handlers when LAM is applicable. For instruction emulation, untag address in __linearize() before canonical check. LAM doesn't apply to instruction fetch and invlpg, use KVM_X86_UNTAG_ADDR_SKIP_LAM to skip LAM untag. For vmexit handlings related to 64-bit linear address: - Cases need to untag address Operand(s) of VMX instructions and INVPCID Operand(s) of SGX ENCLS Linear address in INVVPID descriptor. - Cases LAM doesn't apply to (no change needed) Operand of INVLPG Linear address in INVPCID descriptor Co-developed-by: Robert Hoo Signed-off-by: Robert Hoo Signed-off-by: Binbin Wu --- arch/x86/kvm/emulate.c | 25 +++++++++++++++++-------- arch/x86/kvm/vmx/nested.c | 2 ++ arch/x86/kvm/vmx/sgx.c | 1 + arch/x86/kvm/x86.c | 4 ++++ 4 files changed, 24 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index a630c5db971c..c46f0162498e 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -688,7 +688,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt, struct segmented_address addr, unsigned *max_size, unsigned size, bool write, bool fetch, - enum x86emul_mode mode, ulong *linear) + enum x86emul_mode mode, ulong *linear, + u64 untag_flags) { struct desc_struct desc; bool usable; @@ -701,9 +702,10 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt, *max_size = 0; switch (mode) { case X86EMUL_MODE_PROT64: - *linear = la; + *linear = static_call(kvm_x86_untag_addr)(ctxt->vcpu, la, untag_flags); + va_bits = ctxt_virt_addr_bits(ctxt); - if (!__is_canonical_address(la, va_bits)) + if (!__is_canonical_address(*linear, va_bits)) goto bad; *max_size = min_t(u64, ~0u, (1ull << va_bits) - la); @@ -757,8 +759,8 @@ static int linearize(struct x86_emulate_ctxt *ctxt, ulong *linear) { unsigned max_size; - return __linearize(ctxt, addr, &max_size, size, write, false, - ctxt->mode, linear); + return __linearize(ctxt, addr, &max_size, size, false, false, + ctxt->mode, linear, 0); } static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst) @@ -771,7 +773,9 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst) if (ctxt->op_bytes != sizeof(unsigned long)) addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1); - rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &linear); + /* skip LAM untag for instruction */ + rc = __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, + &linear, KVM_X86_UNTAG_ADDR_SKIP_LAM); if (rc == X86EMUL_CONTINUE) ctxt->_eip = addr.ea; return rc; @@ -906,9 +910,11 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctxt *ctxt, int op_size) * __linearize is called with size 0 so that it does not do any * boundary check itself. Instead, we use max_size to check * against op_size. + * + * skip LAM untag for instruction */ rc = __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode, - &linear); + &linear, KVM_X86_UNTAG_ADDR_SKIP_LAM); if (unlikely(rc != X86EMUL_CONTINUE)) return rc; @@ -3433,8 +3439,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt) { int rc; ulong linear; + unsigned max_size; - rc = linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear); + /* skip untag for invlpg since LAM is not applied to invlpg */ + rc = __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false, + ctxt->mode, &linear, KVM_X86_UNTAG_ADDR_SKIP_LAM); if (rc == X86EMUL_CONTINUE) ctxt->ops->invlpg(ctxt, linear); /* Disable writeback. */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 2eb258992d63..dd1d28a0d147 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4970,6 +4970,7 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification, else *ret = off; + *ret = vmx_untag_addr(vcpu, *ret, 0); /* Long mode: #GP(0)/#SS(0) if the memory address is in a * non-canonical form. This is the only check on the memory * destination for long mode! @@ -5787,6 +5788,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu) vpid02 = nested_get_vpid02(vcpu); switch (type) { case VMX_VPID_EXTENT_INDIVIDUAL_ADDR: + operand.gla = vmx_untag_addr(vcpu, operand.gla, 0); if (!operand.vpid || is_noncanonical_address(operand.gla, vcpu)) return nested_vmx_fail(vcpu, diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index 0574030b071f..527f1a902c65 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -37,6 +37,7 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset, if (!IS_ALIGNED(*gva, alignment)) { fault = true; } else if (likely(is_64_bit_mode(vcpu))) { + *gva = vmx_untag_addr(vcpu, *gva, 0); fault = is_noncanonical_address(*gva, vcpu); } else { *gva &= 0xffffffff; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e74af72f53ec..d85f87a19f58 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -13233,6 +13233,10 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva) switch (type) { case INVPCID_TYPE_INDIV_ADDR: + /* + * LAM doesn't apply to the linear address in the descriptor, + * still need to be canonical + */ if ((!pcid_enabled && (operand.pcid != 0)) || is_noncanonical_address(operand.gla, vcpu)) { kvm_inject_gp(vcpu, 0); From patchwork Sun Mar 19 08:49:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13180275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86E58C7618A for ; Sun, 19 Mar 2023 08:50:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230249AbjCSIuP (ORCPT ); Sun, 19 Mar 2023 04:50:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230335AbjCSIt6 (ORCPT ); Sun, 19 Mar 2023 04:49:58 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43011E1AF for ; Sun, 19 Mar 2023 01:49:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679215795; x=1710751795; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zjfiq805HsblC/KH/UQG7FEhBjf125By5Er7e8elbeA=; b=GWx5iWji2MXKEO/fmZ8Ox89m1f22Ipj+ofCNLi7VNx8lRtUERvaU5sdU HfVl/4/Ajqkcnf8ncXiIm6W+qTnmfGNg9/pDlLv29Pc6IXYDH6BRG9D62 wMNsd1P/fUR71NdUHoGaZr26QZjnwjhMFMADDG2aSRYP/QK41kbkBG2Qh fqn1JwHs/IBXqclryDe7aI9GMMSX6aFXu56MANIKz8J5YvRpMcEIISyvf HxCpO28KO3GzvTCPRbcvth4RPROWRtfkMGBPw4iXlLG4zdPkU+A2MbkJh meuEf7jVMe5zt1S5LmWHt18sM3GFqWV8Ey+ihDYZdvQ3AFk6GTMA5MWAZ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="424767890" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="424767890" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="683146397" X-IronPort-AV: E=Sophos;i="5.98,273,1673942400"; d="scan'208";a="683146397" Received: from binbinwu-mobl.ccr.corp.intel.com ([10.254.209.111]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2023 01:49:52 -0700 From: Binbin Wu To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com Cc: chao.gao@intel.com, robert.hu@linux.intel.com, binbin.wu@linux.intel.com, Jingqi Liu Subject: [PATCH v6 7/7] KVM: x86: Expose LAM feature to userspace VMM Date: Sun, 19 Mar 2023 16:49:27 +0800 Message-Id: <20230319084927.29607-8-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230319084927.29607-1-binbin.wu@linux.intel.com> References: <20230319084927.29607-1-binbin.wu@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Robert Hoo LAM feature is enumerated by CPUID.7.1:EAX.LAM[bit 26]. Expose the feature to guest as the final step after the following supports: - CR4.LAM_SUP virtualization - CR3.LAM_U48 and CR3.LAM_U57 virtualization - Check and untag 64-bit linear address when LAM applies in instruction emulations and vmexit handlers. Signed-off-by: Robert Hoo Signed-off-by: Binbin Wu Reviewed-by: Jingqi Liu Reviewed-by: Chao Gao --- arch/x86/kvm/cpuid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 8f8edeaf8177..13840ef897c7 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -670,7 +670,7 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_mask(CPUID_7_1_EAX, F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(FZRM) | F(FSRS) | F(FSRC) | - F(AMX_FP16) | F(AVX_IFMA) + F(AMX_FP16) | F(AVX_IFMA) | F(LAM) ); kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX,