From patchwork Mon Oct 28 13:20:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13853462 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CE6C1DC06B; Mon, 28 Oct 2024 13:20:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121633; cv=none; b=ALFe+g08U6DE3yTZxBOCApro4DLDJ769zmYpaaWGEOjWPVlFUr74rw2dqZMI5J5ssE40oMAWUzIFN5ghFtW+EDkmNKJRFQJexch3S/OV7+6nfcSdGc4TvtRAb9U8VUk9bdJo81o80FbxbgFWeCsJzRhcUbNaZif6N/Ii8YR2zO8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121633; c=relaxed/simple; bh=fAAeiIXSaCtiwEUxKsyPTFXz0tDGtTakDZBaBMRw4mI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uW7ucgMwzQBt9uFb4mlbmRSUM2X2DEwTSuTEITUEs0q2wmhXKBgQjvzRzsF/EocgeddlpcOq466VJNJOEDSwLRRQ9LOtB2cDbdKhbyedaRmtEsYq5DKzkJBrlIqxP51Argu61GelkMb3V+e57A2mriy6YmrO5x8Vcv13h+6xZ4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jDdBLuVE; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jDdBLuVE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730121631; x=1761657631; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fAAeiIXSaCtiwEUxKsyPTFXz0tDGtTakDZBaBMRw4mI=; b=jDdBLuVEyo7wVkZ+IRj0DGsYD+FTK5BIZJzpb3M7lWjeZ4OY9HtFqm8O r0KvqT+Vn1jat2TPI0F7wfdJCdlFC732vviurBbByGv58QJnfv90MRt8P R8tybusWqLRgrxTwFncyiuGJIWQvr7whzwfZHiy9JXQBH3CzLE4SwlbZ5 3zLK4E8djIY1wLwknq6oWAUbIP1rHE/sJswhcz7Y2p8oilslxS7U0KSN/ lamjS1aDnoC5vJHPwGRNTHh8soij9fCx9U/DdDDgazVlyTARccL6QECgR CHjjXG2gSXTwpK2zUriMNx0wx/rw2bcL6fGvjpF8cv6qgYVWuF2seVP7Z w==; X-CSE-ConnectionGUID: 4LuEB4YST7qBVzKdfsWO0g== X-CSE-MsgGUID: qkKJQPQCQZWEChpP+88mQQ== X-IronPort-AV: E=McAfee;i="6700,10204,11238"; a="29820967" X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="29820967" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:31 -0700 X-CSE-ConnectionGUID: WA+vo9OHS0G502fm1b0hxg== X-CSE-MsgGUID: pMSxz1dwTKmYHbBGtLQX3Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="86397223" Received: from gargmani-mobl1.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.124.222.169]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:28 -0700 From: Kai Huang To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org, rick.p.edgecombe@intel.com Cc: isaku.yamahata@intel.com, reinette.chatre@intel.com, binbin.wu@linux.intel.com, xiaoyao.li@intel.com, yan.y.zhao@intel.com, adrian.hunter@intel.com, tony.lindgren@intel.com, kristen@linux.intel.com, linux-kernel@vger.kernel.org, Kai Huang Subject: [PATCH 1/3] KVM: VMX: Refactor VMX module init/exit functions Date: Tue, 29 Oct 2024 02:20:14 +1300 Message-ID: <6e15578c9c2fd14b9a8fd02fda4df3843267dd13.1730120881.git.kai.huang@intel.com> X-Mailer: git-send-email 2.46.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add vt_init() and vt_exit() as the new module init/exit functions and refactor existing vmx_init()/vmx_exit() as helper to make room for TDX specific initialization and teardown. To support TDX, KVM will need to enabling TDX during KVM module loading time. Enabling TDX requires enabling hardware virtualization first so that all online CPUs (and the new CPU going online) are in post-VMXON state. Currently, the vmx_init() flow is: 1) hv_init_evmcs(), 2) kvm_x86_vendor_init(), 3) Other VMX specific initialization, 4) kvm_init() The kvm_x86_vendor_init() invokes kvm_x86_init_ops::hardware_setup() to do VMX specific hardware setup and calls kvm_update_ops() to initialize kvm_x86_ops to VMX's version. TDX will have its own version for most of kvm_x86_ops callbacks. It would be nice if kvm_x86_init_ops::hardware_setup() could also be used for TDX, but in practice it cannot. The reason is, as mentioned above, TDX initialization requires hardware virtualization having been enabled, which must happen after kvm_update_ops(), but hardware_setup() is done before that. Also, TDX is based on VMX, and it makes sense to only initialize TDX after VMX has been initialized. If VMX fails to initialize, TDX is likely broken anyway. So the new flow of KVM module init function will be: 1) Current VMX initialization code in vmx_init() before kvm_init(), 2) TDX initialization, 3) kvm_init() Split vmx_init() into two parts based above 1) and 3) so that TDX initialization can fit in between. Make part 1) as the new helper vmx_init(). Introduce vt_init() as the new module init function which calls vmx_init() and kvm_init(). TDX initialization will be added later. Do the same thing for vmx_exit()/vt_exit(). Signed-off-by: Kai Huang --- arch/x86/kvm/vmx/main.c | 32 ++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 23 ++--------------------- arch/x86/kvm/vmx/vmx.h | 3 +++ 3 files changed, 37 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 7668e2fb8043..433ecbd90905 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -166,3 +166,35 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .runtime_ops = &vt_x86_ops, .pmu_ops = &intel_pmu_ops, }; + +static void vt_exit(void) +{ + kvm_exit(); + vmx_exit(); +} +module_exit(vt_exit); + +static int __init vt_init(void) +{ + int r; + + r = vmx_init(); + if (r) + return r; + + /* + * Common KVM initialization _must_ come last, after this, /dev/kvm is + * exposed to userspace! + */ + r = kvm_init(sizeof(struct vcpu_vmx), __alignof__(struct vcpu_vmx), + THIS_MODULE); + if (r) + goto err_kvm_init; + + return 0; + +err_kvm_init: + vmx_exit(); + return r; +} +module_init(vt_init); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f6900bec4874..976fe6579f62 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8587,23 +8587,16 @@ static void vmx_cleanup_l1d_flush(void) l1tf_vmx_mitigation = VMENTER_L1D_FLUSH_AUTO; } -static void __vmx_exit(void) +void vmx_exit(void) { allow_smaller_maxphyaddr = false; vmx_cleanup_l1d_flush(); -} -static void vmx_exit(void) -{ - kvm_exit(); - __vmx_exit(); kvm_x86_vendor_exit(); - } -module_exit(vmx_exit); -static int __init vmx_init(void) +int __init vmx_init(void) { int r, cpu; @@ -8647,21 +8640,9 @@ static int __init vmx_init(void) if (!enable_ept) allow_smaller_maxphyaddr = true; - /* - * Common KVM initialization _must_ come last, after this, /dev/kvm is - * exposed to userspace! - */ - r = kvm_init(sizeof(struct vcpu_vmx), __alignof__(struct vcpu_vmx), - THIS_MODULE); - if (r) - goto err_kvm_init; - return 0; -err_kvm_init: - __vmx_exit(); err_l1d_flush: kvm_x86_vendor_exit(); return r; } -module_init(vmx_init); diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 40303b43da6c..ad9efe41e691 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -755,4 +755,7 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) vmx->segment_cache.bitmask = 0; } +int vmx_init(void); +void vmx_exit(void); + #endif /* __KVM_X86_VMX_H */ From patchwork Mon Oct 28 13:20:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13853463 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CF941DD0FE; Mon, 28 Oct 2024 13:20:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121636; cv=none; b=FhYcfwa5NvjgWajs0ZxmEUQU4jN5LUFtxMSj22w5orO41soOLinjcOF5XjOyw5uZCyOSXIyjPDJqMaS+0l6XdiBYBsntPWfPucOTVVSFJb+Ts+tisnhStPVmxu6//g32gqE+2wgAddRDCD0VbEONvw/g3OArH7HTCIiv+pSuqNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121636; c=relaxed/simple; bh=13344vxsdqZxXQjsXwnFs3FcFrd0V5S/i7X+xJ4Un9E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bPluzXrwU9B+0Xlz8fRQrJWq96q8IOYGlYrWc2IAVBukI9FSykgynYgwpELOOQVeXJyOuBOp1zH3rIC1i4VAhgFu2+VPUsPgtcnxaIr2bbttZzKN46VxO7NVTkS0ZQGxvC2lDM5tQLLQxuvRtDTSKQ1yOOOC1X3RkByYIKbn0kY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fNcN5dlF; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fNcN5dlF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730121634; x=1761657634; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=13344vxsdqZxXQjsXwnFs3FcFrd0V5S/i7X+xJ4Un9E=; b=fNcN5dlF6dL59UF0KeqEt4wyO2QHWA9pFXpYTBJwEjUZid7UShVkyrcq JrNfkUfSbIGryTdNN7A9Y0CArnLFXixIm1XA8/bcDTj5y+sbScOoKg60h Ywsl35LYmuqbhiXRtvIbhBu0okoKyKEInhsRqm3PRxMM7UVkbhk7Kx/xk hEkYsrOulaS7u6RJjPPVjebftBj1dbIitId4Ycgf+PJ8LILpmwSxV3PyL /y3ngXcp0U26TkrqPd9hm/8y9BWNrXi8aD67YHErwnj9xJxmReH6S9RHu 0CgVpcoMbmvzdX4J4Wx8OhGzWCXXXiiCsjWOSgW/nb8VCbtZLQxfCSafy w==; X-CSE-ConnectionGUID: zClUHYLkTFyunJlwPvrd0g== X-CSE-MsgGUID: 6WkClmRpSkSt4Vuc35M5bA== X-IronPort-AV: E=McAfee;i="6700,10204,11238"; a="29820973" X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="29820973" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:34 -0700 X-CSE-ConnectionGUID: iO0Inx3ARvyetf5lNBxWnw== X-CSE-MsgGUID: TmTGDDFTQJ2FBkDFrusY4Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="86397239" Received: from gargmani-mobl1.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.124.222.169]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:32 -0700 From: Kai Huang To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org, rick.p.edgecombe@intel.com Cc: isaku.yamahata@intel.com, reinette.chatre@intel.com, binbin.wu@linux.intel.com, xiaoyao.li@intel.com, yan.y.zhao@intel.com, adrian.hunter@intel.com, tony.lindgren@intel.com, kristen@linux.intel.com, linux-kernel@vger.kernel.org, Kai Huang Subject: [PATCH 2/3] KVM: Export hardware virtualization enabling/disabling functions Date: Tue, 29 Oct 2024 02:20:15 +1300 Message-ID: <4388864f91eb661a698bedf8bd910d18fda0daa6.1730120881.git.kai.huang@intel.com> X-Mailer: git-send-email 2.46.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 To support TDX, KVM will need to enabling TDX during KVM module loading time. Enabling TDX requires enabling hardware virtualization first so that all online CPUs (and the new CPU going online) are in post-VMXON state. KVM by default enables hardware virtualization but that is done in kvm_init(), which must be the last step after all initialization is done thus is too late for enabling TDX. Export functions to enabling/disable hardware virtualization so that TDX code can use them to handle hardware virtualization enabling before kvm_init(). Signed-off-by: Kai Huang --- include/linux/kvm_host.h | 8 ++++++++ virt/kvm/kvm_main.c | 18 ++++-------------- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 02f0206fd2dc..024c91eef7db 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2557,4 +2557,12 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, struct kvm_pre_fault_memory *range); #endif +#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING +int kvm_enable_virtualization(void); +void kvm_disable_virtualization(void); +#else +static inline int kvm_enable_virtualization(void) { return 0; } +static inline void kvm_disable_virtualization(void) { } +#endif + #endif diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1b10dc408a0..a0117390ea7f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -143,8 +143,6 @@ static int kvm_no_compat_open(struct inode *inode, struct file *file) #define KVM_COMPAT(c) .compat_ioctl = kvm_no_compat_ioctl, \ .open = kvm_no_compat_open #endif -static int kvm_enable_virtualization(void); -static void kvm_disable_virtualization(void); static void kvm_io_bus_destroy(struct kvm_io_bus *bus); @@ -5523,7 +5521,7 @@ static struct syscore_ops kvm_syscore_ops = { .shutdown = kvm_shutdown, }; -static int kvm_enable_virtualization(void) +int kvm_enable_virtualization(void) { int r; @@ -5568,8 +5566,9 @@ static int kvm_enable_virtualization(void) --kvm_usage_count; return r; } +EXPORT_SYMBOL_GPL(kvm_enable_virtualization); -static void kvm_disable_virtualization(void) +void kvm_disable_virtualization(void) { guard(mutex)(&kvm_usage_lock); @@ -5580,6 +5579,7 @@ static void kvm_disable_virtualization(void) cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); kvm_arch_disable_virtualization(); } +EXPORT_SYMBOL_GPL(kvm_disable_virtualization); static int kvm_init_virtualization(void) { @@ -5595,21 +5595,11 @@ static void kvm_uninit_virtualization(void) kvm_disable_virtualization(); } #else /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */ -static int kvm_enable_virtualization(void) -{ - return 0; -} - static int kvm_init_virtualization(void) { return 0; } -static void kvm_disable_virtualization(void) -{ - -} - static void kvm_uninit_virtualization(void) { From patchwork Mon Oct 28 13:20:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13853464 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35D501DD872; Mon, 28 Oct 2024 13:20:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121639; cv=none; b=PEzGGIaBVPpZ/5Mf0UALwp3h606MqL8zlbFWgwpV+sylD7K7K1uQmStNNNlMNAK7vglZTo9HMtqLBJxBSi9K5ZcxgAdK6JPTRtX3zlVtkteXZUscI8V5RoECN+awAg76rvkIsKflTITOOqc8zHTOmI024qMIFEZgWGyLwTf5RBo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730121639; c=relaxed/simple; bh=yjSEcZCTd05Yt7ubS1sHvNhzEiGxhRqRhxD7zM4nrxo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BsFAHMujrU+hD+QAXXnYN/kHbZuQJ0ObgI/4WGifpNepkeUNkzgdTDZdtmeFZEdc8pvqhLiUB8LQSCOeEnUibd5cI6W4u6DnCumKBEMpD619sSH0QqcRDP96GkJUHTyaIIo+icQ5JxOkAm9Kt0H9563E10cqaA5cSlXm75Sn50Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZM9Guy5t; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZM9Guy5t" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730121637; x=1761657637; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yjSEcZCTd05Yt7ubS1sHvNhzEiGxhRqRhxD7zM4nrxo=; b=ZM9Guy5tFjaQPiRaGzxgHoBF/TEugJu2NRCXwDzrCb1jA/ngrUDm34Nl qgQFlqysrBFFUrnH58eKrkA0Gkxzj/YoTAyqsk3BSyRqbTvrzHqCtBYK2 abw+jO2+QKX9/qwB83QBv41YrrrQ/GfpEfmcQxFrcbc1KxCfpIP0D6riq k5NyE9dBk/5/UxZ9xFmYbXX/SCBgkZ6fyCUZCSPX4xWeIXPQebeso+R2P JOskGxD/EppS938x/8094/m/rLlAUpvf1oqz6pRDdfYwUjv2xjtcqg7Ig 3aTSJAK7Wn3d+iMZPtD8a51hI04TbZPF0JXygZh33awtsjZLjr3KRsHvc Q==; X-CSE-ConnectionGUID: M7T0UVHgTIGx6Can7avk1g== X-CSE-MsgGUID: 5xoCxHObT7eC30oVwiQAFg== X-IronPort-AV: E=McAfee;i="6700,10204,11238"; a="29820983" X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="29820983" Received: from orviesa005.jf.intel.com ([10.64.159.145]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:37 -0700 X-CSE-ConnectionGUID: qiJkstXASuSMyxbV2DxLPw== X-CSE-MsgGUID: N+xiRrQtRkqOLzEOyl+Mng== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,239,1725346800"; d="scan'208";a="86397267" Received: from gargmani-mobl1.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.124.222.169]) by orviesa005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Oct 2024 06:20:35 -0700 From: Kai Huang To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org, rick.p.edgecombe@intel.com Cc: isaku.yamahata@intel.com, reinette.chatre@intel.com, binbin.wu@linux.intel.com, xiaoyao.li@intel.com, yan.y.zhao@intel.com, adrian.hunter@intel.com, tony.lindgren@intel.com, kristen@linux.intel.com, linux-kernel@vger.kernel.org, Kai Huang Subject: [PATCH 3/3] KVM: VMX: Initialize TDX during KVM module load Date: Tue, 29 Oct 2024 02:20:16 +1300 Message-ID: X-Mailer: git-send-email 2.46.2 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Before KVM can use TDX to create and run TDX guests, TDX needs to be initialized from two perspectives: 1) TDX module must be initialized properly to a working state; 2) A per-cpu TDX initialization, a.k.a the TDH.SYS.LP.INIT SEAMCALL must be done on any logical cpu before it can run any other TDX SEAMCALLs. The TDX host core-kernel provides two functions to do above two respectively: tdx_enable() and tdx_cpu_enable(). There are two options in terms of when to initialize TDX: initialize TDX at KVM module loading time, or when creating the first TDX guest. Choose to initialize TDX during KVM module loading time: Initializing TDX module is both memory and CPU time consuming: 1) the kernel needs to allocate a non-trivial size(~1/256) of system memory as metadata used by TDX module to track each TDX-usable memory page's status; 2) the TDX module needs to initialize this metadata, one entry for each TDX-usable memory page. Also, the kernel uses alloc_contig_pages() to allocate those metadata chunks, because they are large and need to be physically contiguous. alloc_contig_pages() can fail. If initializing TDX when creating the first TDX guest, then there's chance that KVM won't be able to run any TDX guests albeit KVM _declares_ to be able to support TDX. This isn't good to the user. On the other hand, initializing TDX at KVM module loading time can make sure KVM is providing a consistent view of whether KVM can support TDX to the user. Always only try to initialize TDX after VMX has been initialized. TDX is based on VMX, and if VMX fails to initialize then TDX is likely to be broken anyway. Also, in practice, supporting TDX will require part of VMX and common x86 infrastructure in working order, so TDX cannot be sololy w/o VMX support. Specifically, initialize TDX after VMX has been initialized and before kvm_init() in vt_init(). Don't fail the whole vt_init() if TDX fails to initialize, since in this case KVM can still support normal VMX guests. Because TDX costs additional memory, don't enable TDX by default. Add a new module parameter 'enable_tdx' to allow the user to opt-in. Register a new TDX-specific cpuhp callback to run tdx_cpu_enable(), and call tdx_enable() after that to ensure tdx_cpu_enable() has been done for all online CPUs before making the tdx_enable(). Use a dynamic cpuhp state for TDX so that KVM's cpuhp callback to enable VMX on new online CPU can happen before tdx_cpu_enable(). Note, the name tdx_init() has already been taken by the early boot code. Use tdx_bringup() for initializing TDX (and tdx_cleanup() since KVM doesn't actually teardown TDX). They don't match vt_init()/vt_exit(), vmx_init()/vmx_exit() etc but it's not end of the world. Also, once initialized, the TDX module cannot be disabled and enabled again w/o the TDX module runtime update, which isn't supported by the kernel. After TDX is enabled, nothing needs to be done when KVM disables hardware virtualization, e.g., when offlining CPU, or during suspend/resume. Signed-off-by: Kai Huang --- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 6 +++ arch/x86/kvm/vmx/tdx.c | 115 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 12 +++++ 4 files changed, 134 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index f9dddb8cb466..fec803aff7ad 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -20,6 +20,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 433ecbd90905..053294939eb1 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -6,6 +6,7 @@ #include "nested.h" #include "pmu.h" #include "posted_intr.h" +#include "tdx.h" #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLED) | \ @@ -170,6 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { static void vt_exit(void) { kvm_exit(); + tdx_cleanup(); vmx_exit(); } module_exit(vt_exit); @@ -182,6 +184,9 @@ static int __init vt_init(void) if (r) return r; + /* tdx_init() has been taken */ + tdx_bringup(); + /* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! @@ -194,6 +199,7 @@ static int __init vt_init(void) return 0; err_kvm_init: + tdx_cleanup(); vmx_exit(); return r; } diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c new file mode 100644 index 000000000000..8651599822d5 --- /dev/null +++ b/arch/x86/kvm/vmx/tdx.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include "capabilities.h" +#include "tdx.h" + +#undef pr_fmt +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +static bool enable_tdx __ro_after_init; +module_param_named(tdx, enable_tdx, bool, 0444); + +static enum cpuhp_state tdx_cpuhp_state; + +static int tdx_online_cpu(unsigned int cpu) +{ + unsigned long flags; + int r; + + /* Sanity check CPU is already in post-VMXON */ + WARN_ON_ONCE(!(cr4_read_shadow() & X86_CR4_VMXE)); + + /* tdx_cpu_enable() must be called with IRQ disabled */ + local_irq_save(flags); + r = tdx_cpu_enable(); + local_irq_restore(flags); + + return r; +} + +static void __do_tdx_cleanup(void) +{ + /* + * Once TDX module is initialized, it cannot be disabled and + * re-initialized again w/o runtime update (which isn't + * supported by kernel). In fact the kernel doesn't support + * disable (shut down) TDX module, so only need to remove the + * cpuhp state. + */ + WARN_ON_ONCE(!tdx_cpuhp_state); + cpuhp_remove_state_nocalls(tdx_cpuhp_state); + tdx_cpuhp_state = 0; +} + +static int __init __do_tdx_bringup(void) +{ + int r; + + /* + * TDX-specific cpuhp callback to call tdx_cpu_enable() on all + * online CPUs before calling tdx_enable(), and on any new + * going-online CPU to make sure it is ready for TDX guest. + */ + r = cpuhp_setup_state_cpuslocked(CPUHP_AP_ONLINE_DYN, + "kvm/cpu/tdx:online", + tdx_online_cpu, NULL); + if (r < 0) + return r; + + tdx_cpuhp_state = r; + + /* tdx_enable() must be called with cpus_read_lock() */ + r = tdx_enable(); + if (r) + __do_tdx_cleanup(); + + return r; +} + +static int __init __tdx_bringup(void) +{ + int r; + + if (!enable_ept) { + pr_err("Cannot enable TDX with EPT disabled.\n"); + return -EINVAL; + } + + /* + * Enabling TDX requires enabling hardware virtualization first, + * as making SEAMCALLs requires CPU being in post-VMXON state. + */ + r = kvm_enable_virtualization(); + if (r) + return r; + + cpus_read_lock(); + r = __do_tdx_bringup(); + cpus_read_unlock(); + + if (r) + goto tdx_bringup_err; + + /* + * Leave hardware virtualization enabled after TDX is enabled + * successfully. TDX CPU hotplug depends on this. + */ + return 0; +tdx_bringup_err: + kvm_disable_virtualization(); + return r; +} + +void tdx_cleanup(void) +{ + if (enable_tdx) { + __do_tdx_cleanup(); + kvm_disable_virtualization(); + } +} + +void __init tdx_bringup(void) +{ + enable_tdx = enable_tdx && !__tdx_bringup(); +} diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h new file mode 100644 index 000000000000..766a6121f670 --- /dev/null +++ b/arch/x86/kvm/vmx/tdx.h @@ -0,0 +1,12 @@ +#ifndef __KVM_X86_VMX_TDX_H +#define __KVM_X86_VMX_TDX_H + +#ifdef CONFIG_INTEL_TDX_HOST +void tdx_bringup(void); +void tdx_cleanup(void); +#else +static inline void tdx_bringup(void) {} +static inline void tdx_cleanup(void) {} +#endif + +#endif