From patchwork Mon Feb 13 11:59:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13138310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BD67C636D7 for ; Mon, 13 Feb 2023 12:01:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA6B46B0081; Mon, 13 Feb 2023 07:01:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D31076B0082; Mon, 13 Feb 2023 07:01:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A49F26B0083; Mon, 13 Feb 2023 07:01:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 927D76B0081 for ; Mon, 13 Feb 2023 07:01:40 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6888FC1556 for ; Mon, 13 Feb 2023 12:01:40 +0000 (UTC) X-FDA: 80462129160.10.32FCFFC Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by imf11.hostedemail.com (Postfix) with ESMTP id 52B444001A for ; Mon, 13 Feb 2023 12:01:38 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="TCiv8V/S"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf11.hostedemail.com: domain of kai.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=kai.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676289698; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HLE1+USv0XakWKwN8DkEY8FJMz/7FDyxsLIh3uDhmX0=; b=6dGcQiUX2q9XP8n5gRR/OT41MOZvlv1t1t9tdkIhIXD4d/fJff+HjSR604x4PkjudRxOmC 5Mhic/VGw9zIeTm+DzEwtDW9UniKBQx1qF6DXbDRo8x+qjtjidbCq+CMegF6+HKG9chKJA qcVpypH4EkKawTPCiAgxHYP6IXErDR4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="TCiv8V/S"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf11.hostedemail.com: domain of kai.huang@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=kai.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676289698; a=rsa-sha256; cv=none; b=f5osHXNtZikmlmdFZcWQv74qzJBCNFOjMhn5dkp9Q3ExHNU0YaokpR7aC6exaGnSZHgNA/ 2nNGqQ12zlAdG08pbtCu2ng13mW2sBcALn9GuXGiMXLABd0o3P/qmXIPBc2M9nFTeRGfXQ yL03Xa14m9gb75As3CuW/Hu1rSgbDSQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676289698; x=1707825698; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gV7imOVz4HnigCq8Nw4/g/zyr45AbaY82hHOJD6E8Zc=; b=TCiv8V/Sc0Yn3PuBHNbFYpohNt6b1fP5ihQWhY439mr2LpZ8Nattk0up XEDnZjqoz91YcHYzYB+qcu3PdSZY9DfIgAzCt8qaWvdZVihSp9BPg2WaU Gg6glDEPB9n73j6Zpmcm+UTA7W3rtIK5x+A1KS8Y5UiBTFBWnOYZ/RnIV v643RiC1gmlazGAg62YJOYOjhEpvXlCnkIhZa/PjMtcgXQbfMKwVf6Lch ue977jxTSjrJSlCFQ3dZstHruMWlVJisMS8jArBVcsolPz+RMbdC5XW1T SicwbrBgf/sF5eYgXwNolGkYSIXBJbpqvlk+x8Ns3NyII84+QOmojWl0l g==; X-IronPort-AV: E=McAfee;i="6500,9779,10619"; a="358283483" X-IronPort-AV: E=Sophos;i="5.97,293,1669104000"; d="scan'208";a="358283483" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Feb 2023 04:01:32 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10619"; a="701243521" X-IronPort-AV: E=Sophos;i="5.97,293,1669104000"; d="scan'208";a="701243521" Received: from wonger-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.209.188.34]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Feb 2023 04:01:27 -0800 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, dave.hansen@intel.com, peterz@infradead.org, tglx@linutronix.de, seanjc@google.com, pbonzini@redhat.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, kirill.shutemov@linux.intel.com, ying.huang@intel.com, reinette.chatre@intel.com, len.brown@intel.com, tony.luck@intel.com, ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, david@redhat.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v9 15/18] x86/virt/tdx: Configure global KeyID on all packages Date: Tue, 14 Feb 2023 00:59:22 +1300 Message-Id: X-Mailer: git-send-email 2.39.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 52B444001A X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: hhugz5jgnfw94y4cuk5f6jgn69jykprd X-HE-Tag: 1676289698-276305 X-HE-Meta: U2FsdGVkX1+lQvu3iO2PBTUBayGzxZ1YK1a5jcR/4OR4JOC/e2kTXaAz4ltE7fddW+izHrexR14NeVxeRZ/s2rbJj78NZtLRr3Xe87ldOEi75FvQvmypL+ADUCadMvdNfN2eGDYff1vTT3dAlKS+Qr388Y09683ABIys07sREE9eSr7ehLieXnZwG5xodxvOsyL9dvMss7nONOgkdEbRBcDNE+OuQjAW0c+q8ebXhRAqG1yJdHMfSrvyH7WNfSOyRAxenoGX3rKU7vSdxEGVeKpn2acduQRCIb2KZteegQYqjOM3Tl4MR3itKmTgyb6Q2r5w6c7XIDmw8eJ2MQu7AIp3hjDzLOC+SKmcVbuxF3Scqz/ym09gMs13VLAKOgf5/evjCmnZArrBC96Nce+hivxgZiJYfcvXpdmhtoYV6a5jiX2zPqzgvpwQ/ilZT5PfVT514hhE731W3zamcQuFLYtUgyaMlUbdBzagk+pJByQhk9AuV5jqN6qx7x7QQJd/rNvHWarX0vEaso5A8NEOdx9MlrNDLzdTO22JNLG9LvTM9UImnvGpQEvKG+GqWPoKB4eSz0SU7OM7dw2q4i7f9U1obOmB9JWsya+MZXnLEBn8VXV8cEB3kGz8idwG7DfhIrZPDUpGhYRRMVcbvbe7iIDCvQeb4oxk/+lYtZy5QPvoMYR5wBMmfT9VjWD7PwOHfqSTMLRc/VRaO06+Xwy3/BK2i8cZPY0Hak4C+CkJ7qaL/GHgdP3OnTp8KYfaXnv2u1ipg/YFLzWSbjrKZ/h56a8WCtEFfnBcdJxK1UiZvlwjAPTL+Xu/yXcWXDNXNuwq3XxkvzgRRphG//iM7mGgqtA65FEFti3XI0RW4YghDJAUZLqoIt8gPKWh8rpcK+qzpq3Ngd9zBB+I9WCrKzj8JZnyDNyTSnjhiUjOOr1KrTLf+veC0OE5byvbKSB+bhqfSOcQQRnpscip47SfAma 8o5Ao5bj tKqVfHi6fo2U+lcNo9wBqdb8+6LgumowjbjzzwmGKEAGvGE8xx1+6/Uy39LPVpWvBIPdotzr+n5G8rlV5a/jsnORVxTfVmc/BsocnOOpCLwwuAlxeTlPQ7mdlDkWPi/gPGRNEFLNzgk6OIw4rhzKE+U9oIPo0MBQOnnZhVxw20cYD4Bk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After the list of TDMRs and the global KeyID are configured to the TDX module, the kernel needs to configure the key of the global KeyID on all packages using TDH.SYS.KEY.CONFIG. Just use the helper, which conditionally calls function on all online cpus, to configure the global KeyID on all packages. Loop all online cpus, keep track which packages have been called and skip all cpus for those already called packages. To keep things simple, this implementation takes no affirmative steps to online cpus to make sure there's at least one cpu for each package. The callers (aka. KVM) can ensure success by ensuring that. Intel hardware doesn't guarantee cache coherency across different KeyIDs. The PAMTs are transitioning from being used by the kernel mapping (KeyId 0) to the TDX module's "global KeyID" mapping. This means that the kernel must flush any dirty KeyID-0 PAMT cachelines before the TDX module uses the global KeyID to access the PAMT. Otherwise, if those dirty cachelines were written back, they would corrupt the TDX module's metadata. Aside: This corruption would be detected by the memory integrity hardware on the next read of the memory with the global KeyID. The result would likely be fatal to the system but would not impact TDX security. Following the TDX module specification, flush cache before configuring the global KeyID on all packages. Given the PAMT size can be large (~1/256th of system RAM), just use WBINVD on all CPUs to flush. Note if any TDH.SYS.KEY.CONFIG fails, the TDX module may already have used the global KeyID to write any PAMT. Therefore, use WBINVD to flush cache before freeing the PAMTs back to the kernel. Signed-off-by: Kai Huang Reviewed-by: Isaku Yamahata --- v8 -> v9: - Improved changelog (Dave). - Improved comments to explain the function to configure global KeyID "takes no affirmative action to online any cpu". (Dave). - Improved other comments suggested by Dave. v7 -> v8: (Dave) - Changelog changes: - Point out this is the step of "multi-steps" of init_tdx_module(). - Removed MOVDIR64B part. - Other changes due to removing TDH.SYS.SHUTDOWN and TDH.SYS.LP.INIT. - Changed to loop over online cpus and use smp_call_function_single() directly as the patch to shut down TDX module has been removed. - Removed MOVDIR64B part in comment. v6 -> v7: - Improved changelong and comment to explain why MOVDIR64B isn't used when returning PAMTs back to the kernel. --- arch/x86/virt/vmx/tdx/tdx.c | 92 ++++++++++++++++++++++++++++++++++--- arch/x86/virt/vmx/tdx/tdx.h | 1 + 2 files changed, 86 insertions(+), 7 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 5a4163d40f58..ff6f2c9d9838 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -1031,6 +1031,61 @@ static int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid) return ret; } +static int smp_func_config_global_keyid(void *data) +{ + cpumask_var_t packages = data; + int pkg, ret; + + pkg = topology_physical_package_id(smp_processor_id()); + + /* + * TDH.SYS.KEY.CONFIG may fail with entropy error (which is a + * recoverable error). Assume this is exceedingly rare and + * just return error if encountered instead of retrying. + * + * All '0's are just unused parameters. + */ + ret = seamcall(TDH_SYS_KEY_CONFIG, 0, 0, 0, 0, NULL, NULL); + if (!ret) + cpumask_set_cpu(pkg, packages); + + return ret; +} + +static bool skip_func_config_global_keyid(int cpu, void *data) +{ + cpumask_var_t packages = data; + + return cpumask_test_cpu(topology_physical_package_id(cpu), + packages); +} + +/* + * Attempt to configure the global KeyID on all physical packages. + * + * This requires running code on at least one CPU in each package. If a + * package has no online CPUs, that code will not run and TDX module + * initialization (TDMR initialization) will fail. + * + * This code takes no affirmative steps to online CPUs. Callers (aka. + * KVM) can ensure success by ensuring sufficient CPUs are online for + * this to succeed. + */ +static int config_global_keyid(void) +{ + cpumask_var_t packages; + int ret; + + if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) + return -ENOMEM; + + ret = tdx_on_each_cpu_cond(smp_func_config_global_keyid, packages, + skip_func_config_global_keyid, packages); + + free_cpumask_var(packages); + return ret; +} + static int init_tdx_module(void) { static DECLARE_PADDED_STRUCT(tdsysinfo_struct, tdsysinfo, @@ -1100,10 +1155,24 @@ static int init_tdx_module(void) if (ret) goto out_free_pamts; + /* + * Hardware doesn't guarantee cache coherency across different + * KeyIDs. The kernel needs to flush PAMT's dirty cachelines + * (associated with KeyID 0) before the TDX module can use the + * global KeyID to access the PAMT. Given PAMTs are potentially + * large (~1/256th of system RAM), just use WBINVD on all cpus + * to flush the cache. + */ + wbinvd_on_all_cpus(); + + /* Config the key of global KeyID on all packages */ + ret = config_global_keyid(); + if (ret) + goto out_free_pamts; + /* * TODO: * - * - Configure the global KeyID on all packages. * - Initialize all TDMRs. * * Return error before all steps are done. @@ -1111,8 +1180,18 @@ static int init_tdx_module(void) ret = -EINVAL; out_free_pamts: - if (ret) + if (ret) { + /* + * Part of PAMT may already have been initialized by the + * TDX module. Flush cache before returning PAMT back + * to the kernel. + * + * No need to worry about integrity checks here. KeyID + * 0 has integrity checking disabled. + */ + wbinvd_on_all_cpus(); tdmrs_free_pamt_all(&tdx_tdmr_list); + } else pr_info("%lu KBs allocated for PAMT.\n", tdmrs_count_pamt_pages(&tdx_tdmr_list) * 4); @@ -1166,11 +1245,7 @@ static int __tdx_enable(void) */ static void disable_tdx_module(void) { - /* - * TODO: module clean up in reverse to steps in - * init_tdx_module(). Remove this comment after - * all steps are done. - */ + wbinvd_on_all_cpus(); tdmrs_free_pamt_all(&tdx_tdmr_list); free_tdmr_list(&tdx_tdmr_list); free_tdx_memlist(&tdx_memlist); @@ -1228,6 +1303,9 @@ static int __tdx_enable_online_cpus(void) * This function internally calls cpus_read_lock()/unlock() to prevent * any cpu from going online and offline. * + * This function requires there's at least one online cpu for each CPU + * package to succeed. + * * This function assumes all online cpus are already in VMX operation. * * This function can be called in parallel by multiple callers. diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index 7b34ac257b9a..ca4e2edbf4bc 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -16,6 +16,7 @@ /* * TDX module SEAMCALL leaf functions */ +#define TDH_SYS_KEY_CONFIG 31 #define TDH_SYS_INFO 32 #define TDH_SYS_CONFIG 45