From patchwork Wed Oct 26 23:16:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13021355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B94C5C38A2D for ; Wed, 26 Oct 2022 23:17:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 131968E0005; Wed, 26 Oct 2022 19:17:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BA868E0001; Wed, 26 Oct 2022 19:17:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9DA78E0005; Wed, 26 Oct 2022 19:17:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DB41E8E0001 for ; Wed, 26 Oct 2022 19:17:30 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A9C94120614 for ; Wed, 26 Oct 2022 23:17:30 +0000 (UTC) X-FDA: 80064664260.23.A841217 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf20.hostedemail.com (Postfix) with ESMTP id 1BC371C000B for ; Wed, 26 Oct 2022 23:17:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666826250; x=1698362250; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8+SpCN+dQ22o1M5BWK9wviXLnwdQVF8MwFp55TnMGLY=; b=aSkkkr6xbiXXBuGLqKjuJ2Nsj0fYDRveSexVwIC4OVNELH5XtUS5BHCC RmMkLB99KEQCxBQrVIav+KE9sOczg6q6oZeV+AWHiOqV2w94T9YB1sh9R PZ0z3ag8kUUzHi+oJH+3Xxbruj5g9oO+mIoKOb3Kszt7qYtkJfEdgmm+M 8hC12ymTABy9lDTFYtZpyKICjLl9PWs+bO52jtYvhA6qiw25SA4iwC2EP CJQW5plIgvTrVsok7ftx2vAHd3rFED3AhEGr1Kda2dkUqlIpfcekk26ym fwR7rF9yC/kiq8sIu70VgFOtITGbto5tWmr5/q9YsBp0SrpWWkxOuC+53 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10512"; a="394400321" X-IronPort-AV: E=Sophos;i="5.95,215,1661842800"; d="scan'208";a="394400321" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Oct 2022 16:17:29 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10512"; a="737446315" X-IronPort-AV: E=Sophos;i="5.95,215,1661842800"; d="scan'208";a="737446315" Received: from fordon1x-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.212.24.177]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Oct 2022 16:17:25 -0700 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, kirill.shutemov@linux.intel.com, reinette.chatre@intel.com, len.brown@intel.com, tony.luck@intel.com, peterz@infradead.org, ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v6 07/21] x86/virt/tdx: Shut down TDX module in case of error Date: Thu, 27 Oct 2022 12:16:06 +1300 Message-Id: <3f61504e985ade3acd2e36cfa33aa00d3c9ce16f.1666824663.git.kai.huang@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666826250; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BpoYGe7KyfauTC7bLzKAgHyYUCzdnrqAL9bTIFQfbIQ=; b=KBsLLBpw/L1M25x7rDma98lRrzIowtIri1fMyXw91iKJT37K8kCwQ6ymJurDsqrjtLT6yL T6uSZ9yrZ9Q2Diw4V2/bJakAaSy5MwJi74NyEnJ9pqfceFJt0R0pCreqU8h0EYXWmhpwxe EQjddzhCI6W2K8h2DJtuP//RQnw5Awo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=aSkkkr6x; spf=pass (imf20.hostedemail.com: domain of kai.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=kai.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666826250; a=rsa-sha256; cv=none; b=Nu18eqxDb5YXseG82PFLS0wRj0t/RbFzTVWivpVQwY7PKEwMILW2MlEUwZbGQECd0LmEOD T0l8kT15MlBvIWvklVWLw1GoZmiUMP9sijLVKHXF+kWVxg7jJiwflYmIl6h7M9f+wsEE0N fmTSjYjF++4EqOlNJH/vS1QmxQtKM1U= X-Rspamd-Queue-Id: 1BC371C000B Authentication-Results: imf20.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=aSkkkr6x; spf=pass (imf20.hostedemail.com: domain of kai.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=kai.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Stat-Signature: qwcnu5yp94yqnhezhd9ooij5pm6k6tx7 X-HE-Tag: 1666826249-644283 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: TDX supports shutting down the TDX module at any time during its lifetime. After the module is shut down, no further TDX module SEAMCALL leaf functions can be made to the module on any logical cpu. Shut down the TDX module in case of any error during the initialization process. It's pointless to leave the TDX module in some middle state. Shutting down the TDX module requires calling TDH.SYS.LP.SHUTDOWN on all BIOS-enabled CPUs, and the SEMACALL can run concurrently on different CPUs. Implement a mechanism to run SEAMCALL concurrently on all online CPUs and use it to shut down the module. Later logical-cpu scope module initialization will use it too. Reviewed-by: Isaku Yamahata Signed-off-by: Kai Huang --- v5 -> v6: - Removed the seamcall() wrapper to previous patch (Dave). - v3 -> v5 (no feedback on v4): - Added a wrapper of __seamcall() to print error code if SEAMCALL fails. - Made the seamcall_on_each_cpu() void. - Removed 'seamcall_ret' and 'tdx_module_out' from 'struct seamcall_ctx', as they must be local variable. - Added the comments to tdx_init() and one paragraph to changelog to explain the caller should handle VMXON. - Called out after shut down, no "TDX module" SEAMCALL can be made. --- arch/x86/virt/vmx/tdx/tdx.c | 43 +++++++++++++++++++++++++++++++++---- arch/x86/virt/vmx/tdx/tdx.h | 5 +++++ 2 files changed, 44 insertions(+), 4 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index f1154ef15549..5246335abe07 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #include #include #include @@ -267,15 +269,27 @@ bool platform_tdx_enabled(void) return !!tdx_keyid_num; } +/* + * Data structure to make SEAMCALL on multiple CPUs concurrently. + * @err is set to -EFAULT when SEAMCALL fails on any cpu. + */ +struct seamcall_ctx { + u64 fn; + u64 rcx; + u64 rdx; + u64 r8; + u64 r9; + atomic_t err; +}; + /* * Wrapper of __seamcall() to convert SEAMCALL leaf function error code * to kernel error code. @seamcall_ret and @out contain the SEAMCALL * leaf function return code and the additional output respectively if * not NULL. */ -static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, - u64 *seamcall_ret, - struct tdx_module_output *out) +static int seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, + u64 *seamcall_ret, struct tdx_module_output *out) { u64 sret; @@ -309,6 +323,25 @@ static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, } } +static void seamcall_smp_call_function(void *data) +{ + struct seamcall_ctx *sc = data; + int ret; + + ret = seamcall(sc->fn, sc->rcx, sc->rdx, sc->r8, sc->r9, NULL, NULL); + if (ret) + atomic_set(&sc->err, -EFAULT); +} + +/* + * Call the SEAMCALL on all online CPUs concurrently. Caller to check + * @sc->err to determine whether any SEAMCALL failed on any cpu. + */ +static void seamcall_on_each_cpu(struct seamcall_ctx *sc) +{ + on_each_cpu(seamcall_smp_call_function, sc, true); +} + /* * Detect and initialize the TDX module. * @@ -324,7 +357,9 @@ static int init_tdx_module(void) static void shutdown_tdx_module(void) { - /* TODO: Shut down the TDX module */ + struct seamcall_ctx sc = { .fn = TDH_SYS_LP_SHUTDOWN }; + + seamcall_on_each_cpu(&sc); } static int __tdx_enable(void) diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index 92a8de957dc7..215cc1065d78 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -12,6 +12,11 @@ /* MSR to report KeyID partitioning between MKTME and TDX */ #define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087 +/* + * TDX module SEAMCALL leaf functions + */ +#define TDH_SYS_LP_SHUTDOWN 44 + /* * Do not put any hardware-defined TDX structure representations below * this comment!