From patchwork Mon Mar 6 14:13:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13161222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42954C678D4 for ; Mon, 6 Mar 2023 14:15:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D50596B0078; Mon, 6 Mar 2023 09:15:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CFF82280003; Mon, 6 Mar 2023 09:15:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC7B5280001; Mon, 6 Mar 2023 09:15:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A96826B0078 for ; Mon, 6 Mar 2023 09:15:08 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 78E7F1406BB for ; Mon, 6 Mar 2023 14:15:08 +0000 (UTC) X-FDA: 80538670296.09.70D34E0 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf05.hostedemail.com (Postfix) with ESMTP id 599AB100016 for ; Mon, 6 Mar 2023 14:15:05 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZRS2DN4m; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf05.hostedemail.com: domain of kai.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=kai.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678112105; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=922nywOa+e7tb6QjHPwrsrM9xYheyIddKPYc/Ze9SJ0=; b=WMbtlQcyHERaPegRYyumZpiF2L3jXWjlB/JaPrDpxb5m3i7xdeEiqR9H/i1oXcsQEMlqob IeUsic9mftp5xRC25hxAKjNtFX+tfcYOKo9QBEyl1lgrb28FENnJA39eBn60ZVUkm8Gsrn 7H2xjN9ECzkA5dTA9Lg+L4fO0x7RmP8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZRS2DN4m; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf05.hostedemail.com: domain of kai.huang@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=kai.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678112105; a=rsa-sha256; cv=none; b=ioiN3xlr4t5JX6eaLY6vwg+BsY2zyywJmnsVtJ58wtW0+swTvxI4GcC5m+kCZ6G0ByWpkE JJM0Go6uV5emSZ78+Pm2RSIGV+mBqJzRv+wls90IqkxjIk2nvGo80BA242wF2IATPSdri+ p7Dxm1ISobMaFdLC6JxyTmF3GGs+WXQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678112105; x=1709648105; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k87SClNK/ZMlCe0BUnMjRVRbHwBgYZY0XiVTxfQ3zsA=; b=ZRS2DN4mFwmEqrgK3uZY9svliHPjs21T29ve1Kex1Lu8zsz3e0IEsxL+ ztsJeUSyA5PkNU5F5/nUZH5a/YbXc8aEBmDQ6CsU7Rn+XZNg46QEQbRql /Xl2Y1AL01yLNJTJC7K5v51DQtRPgwynFVno2f7a2CTbDoTy9VU+85zmK PjvOCVHmhZ6t6U6C2T48mPeSOMgmWkrFQRvXvncAtsMzQvwvn/WJxudRG FHw+llXKSMxp0zCdnbkW7OSN/0AgjxWhJO/xUq4pIf3RasQ8Uwe7IMtNz yK1NSDpoUSaKd80MdDxhBdvQEBzkCXxcp18dG0tilA3arjF5yhABzlT1F Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10641"; a="337080151" X-IronPort-AV: E=Sophos;i="5.98,238,1673942400"; d="scan'208";a="337080151" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Mar 2023 06:15:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10641"; a="765232141" X-IronPort-AV: E=Sophos;i="5.98,238,1673942400"; d="scan'208";a="765232141" Received: from jwhisle1-mobl2.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.212.92.57]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Mar 2023 06:15:00 -0800 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, dave.hansen@intel.com, peterz@infradead.org, tglx@linutronix.de, seanjc@google.com, pbonzini@redhat.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, kirill.shutemov@linux.intel.com, ying.huang@intel.com, reinette.chatre@intel.com, len.brown@intel.com, tony.luck@intel.com, ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, david@redhat.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v10 07/16] x86/virt/tdx: Use all system memory when initializing TDX module as TDX memory Date: Tue, 7 Mar 2023 03:13:52 +1300 Message-Id: <54fce4052e700a78843b5672c5380ffaf3879902.1678111292.git.kai.huang@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 599AB100016 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: yzk4igdy3d9qhakn3xjc61euhyj8f61j X-HE-Tag: 1678112105-624706 X-HE-Meta: U2FsdGVkX18v6QHOoyI+I6Ek4GgqLPfhRPcHZcmzXmsAwUvMWRXIrp5NGmpwZJZhQE3w7RdH2IOuPVjiocsZQYNLKkjEaALu6q/7qbA2HvT88bMIdmkDanb1c/aAXWIvYb7T2+Zd+GbEBxixzjWVnRe0sPCg8qqR2ehZoKt4Ggo7qqWb6oaTFajB8dyTvjGhhgdd4Y9Hyfpst1wHAfwoIXJ9Mhx3V0iktOJlK61YszqxJX1Rz4vpxLRcTTN8KMp3+pfzNkoHmtKtDKEpSXsNq3ZrXmQ3ycUIEb5+eSlAQ+7CIAquLTQYmt7p55o+yW2rkpbSoS+C+Fi9Pkkw19OPH1nxOalcQeO4LYt6kTSxiuqfSbxk1OiP3b/RQ6NLG824GRHiWc/hz5j+0B3RLikMgiO2e8Awj6xqD5+YZwujKUSVTkq/S1RgQuR2dSBFXa4yVuiwAskcS7zQZT2zM0vRMAyBhqe1+9C3z7/w2Z8kCp+11+BUgfyBHQWNIbBbXJ6WpmnOuKpctRvNKmIE1YoIy+vd3VvOl50d5HlqIiDCAhB0ibJE9k8rOqH+4xfl83MiIkd7chnCuYoL00sKni6PFuQ2XzTSRMkjSNUIBBAsYqAonI0S3tXQ38N40VNHNnH3Z0EZrKv+CYzbK73hKip+oLK0TtWe5T0b9X2WBL+Uy7qzXA4lY5o0eUVGWpg+goVie8+o4+RBgNJ7w4naoTFMkZcxLPDNtdgBeX+oKfNPBeIQE0oRfU26meR36PosY4KZ2++faqNSES6S4pOEor1Ap4tSv1y3bKAAvjbrsK1Gu5erMKsfaQueV7xpwUAyU6lf/IrhX+HRDbU+mdQTL8H6d4v/Ag1XtsxRWUDrATABvLvWUAe6vLkPH6tTI/6LrfZYCXcDXpUxo8UMs2vi1PXiC61yoOMDIxCcaB26o4A2Pkk2kRAOK6D8ekYhLfMZ8zoD40gzOulngVRsD655GFP 2bWqoioC MtpCrEl+rmWN8wawF2F/6Pq8xCYCQHspU9qcOcKC5rDEZ8OqMoZmDsUqsPRdfI4Lf3Ue+7zlVnkZy7Dk5wGj+HdHXzrXzT/DHVImN1FY+PWggAAOX6LtSm++VhxtxvX7C2bY10fy4fsI/HshHdn0jcQ4rSWs90mm+TNW/s6U5lewsAQQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As a step of initializing the TDX module, the kernel needs to tell the TDX module which memory regions can be used by the TDX module as TDX guest memory. TDX reports a list of "Convertible Memory Region" (CMR) to tell the kernel which memory is TDX compatible. The kernel needs to build a list of memory regions (out of CMRs) as "TDX-usable" memory and pass them to the TDX module. Once this is done, those "TDX-usable" memory regions are fixed during module's lifetime. To keep things simple, assume that all TDX-protected memory will come from the page allocator. Make sure all pages in the page allocator *are* TDX-usable memory. As TDX-usable memory is a fixed configuration, take a snapshot of the memory configuration from memblocks at the time of module initialization (memblocks are modified on memory hotplug). This snapshot is used to enable TDX support for *this* memory configuration only. Use a memory hotplug notifier to ensure that no other RAM can be added outside of this configuration. This approach requires all memblock memory regions at the time of module initialization to be TDX convertible memory to work, otherwise module initialization will fail in a later SEAMCALL when passing those regions to the module. This approach works when all boot-time "system RAM" is TDX convertible memory, and no non-TDX-convertible memory is hot-added to the core-mm before module initialization. For instance, on the first generation of TDX machines, both CXL memory and NVDIMM are not TDX convertible memory. Using kmem driver to hot-add any CXL memory or NVDIMM to the core-mm before module initialization will result in failure to initialize the module. The SEAMCALL error code will be available in the dmesg to help user to understand the failure. Signed-off-by: Kai Huang Reviewed-by: "Huang, Ying" Reviewed-by: Isaku Yamahata --- v9 -> v10: - Moved empty @tdx_memlist check out of is_tdx_memory() to make the logic better. - Added Ying's Reviewed-by. v8 -> v9: - Replace "The initial support ..." with timeless sentence in both changelog and comments(Dave). - Fix run-on sentence in changelog, and senstence to explain why to stash off memblock (Dave). - Tried to improve why to choose this approach and how it work in changelog based on Dave's suggestion. - Many other comments enhancement (Dave). v7 -> v8: - Trimed down changelog (Dave). - Changed to use PHYS_PFN() and PFN_PHYS() throughout this series (Ying). - Moved memory hotplug handling from add_arch_memory() to memory_notifier (Dan/David). - Removed 'nid' from 'struct tdx_memblock' to later patch (Dave). - {build|free}_tdx_memory() -> {build|}free_tdx_memlist() (Dave). - Removed pfn_covered_by_cmr() check as no code to trim CMRs now. - Improve the comment around first 1MB (Dave). - Added a comment around reserve_real_mode() to point out TDX code relies on first 1MB being reserved (Ying). - Added comment to explain why the new online memory range cannot cross multiple TDX memory blocks (Dave). - Improved other comments (Dave). --- arch/x86/Kconfig | 1 + arch/x86/kernel/setup.c | 2 + arch/x86/virt/vmx/tdx/tdx.c | 165 +++++++++++++++++++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx.h | 6 ++ 4 files changed, 172 insertions(+), 2 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 6dd5d5586099..f23bc540778a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1958,6 +1958,7 @@ config INTEL_TDX_HOST depends on X86_64 depends on KVM_INTEL depends on X86_X2APIC + select ARCH_KEEP_MEMBLOCK help Intel Trust Domain Extensions (TDX) protects guest VMs from malicious host and certain physical attacks. This option enables necessary TDX diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 88188549647c..a8a119a9b48c 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1165,6 +1165,8 @@ void __init setup_arch(char **cmdline_p) * * Moreover, on machines with SandyBridge graphics or in setups that use * crashkernel the entire 1M is reserved anyway. + * + * Note the host kernel TDX also requires the first 1MB being reserved. */ x86_platform.realmode_reserve(); diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 981e11492d0e..9149144cd7e7 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -17,6 +17,13 @@ #include #include #include +#include +#include +#include +#include +#include +#include +#include #include #include #include @@ -39,6 +46,9 @@ static DEFINE_PER_CPU(unsigned int, tdx_lp_init_status); static enum tdx_module_status_t tdx_module_status; static DEFINE_MUTEX(tdx_module_lock); +/* All TDX-usable memory regions. Protected by mem_hotplug_lock. */ +static LIST_HEAD(tdx_memlist); + /* * Use tdx_global_keyid to indicate that TDX is uninitialized. * This is used in TDX initialization error paths to take it from @@ -77,6 +87,54 @@ static int __init record_keyid_partitioning(u32 *tdx_keyid_start, return 0; } +static bool is_tdx_memory(unsigned long start_pfn, unsigned long end_pfn) +{ + struct tdx_memblock *tmb; + + /* + * This check assumes that the start_pfn<->end_pfn range does not + * cross multiple @tdx_memlist entries. A single memory online + * event across multiple memblocks (from which @tdx_memlist + * entries are derived at the time of module initialization) is + * not possible. This is because memory offline/online is done + * on granularity of 'struct memory_block', and the hotpluggable + * memory region (one memblock) must be multiple of memory_block. + */ + list_for_each_entry(tmb, &tdx_memlist, list) { + if (start_pfn >= tmb->start_pfn && end_pfn <= tmb->end_pfn) + return true; + } + return false; +} + +static int tdx_memory_notifier(struct notifier_block *nb, unsigned long action, + void *v) +{ + struct memory_notify *mn = v; + + if (action != MEM_GOING_ONLINE) + return NOTIFY_OK; + + /* + * Empty list means TDX isn't enabled. Allow any memory + * to go online. + */ + if (list_empty(&tdx_memlist)) + return NOTIFY_OK; + + /* + * The TDX memory configuration is static and can not be + * changed. Reject onlining any memory which is outside of + * the static configuration whether it supports TDX or not. + */ + return is_tdx_memory(mn->start_pfn, mn->start_pfn + mn->nr_pages) ? + NOTIFY_OK : NOTIFY_BAD; +} + +static struct notifier_block tdx_memory_nb = { + .notifier_call = tdx_memory_notifier, +}; + static int __init tdx_init(void) { u32 tdx_keyid_start, nr_tdx_keyids; @@ -107,6 +165,13 @@ static int __init tdx_init(void) goto no_tdx; } + err = register_memory_notifier(&tdx_memory_nb); + if (err) { + pr_info("initialization failed: register_memory_notifier() failed (%d)\n", + err); + goto no_tdx; + } + tdx_guest_keyid_start = tdx_keyid_start; tdx_nr_guest_keyids = nr_tdx_keyids; @@ -316,6 +381,79 @@ static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo, return 0; } +/* + * Add a memory region as a TDX memory block. The caller must make sure + * all memory regions are added in address ascending order and don't + * overlap. + */ +static int add_tdx_memblock(struct list_head *tmb_list, unsigned long start_pfn, + unsigned long end_pfn) +{ + struct tdx_memblock *tmb; + + tmb = kmalloc(sizeof(*tmb), GFP_KERNEL); + if (!tmb) + return -ENOMEM; + + INIT_LIST_HEAD(&tmb->list); + tmb->start_pfn = start_pfn; + tmb->end_pfn = end_pfn; + + /* @tmb_list is protected by mem_hotplug_lock */ + list_add_tail(&tmb->list, tmb_list); + return 0; +} + +static void free_tdx_memlist(struct list_head *tmb_list) +{ + /* @tmb_list is protected by mem_hotplug_lock */ + while (!list_empty(tmb_list)) { + struct tdx_memblock *tmb = list_first_entry(tmb_list, + struct tdx_memblock, list); + + list_del(&tmb->list); + kfree(tmb); + } +} + +/* + * Ensure that all memblock memory regions are convertible to TDX + * memory. Once this has been established, stash the memblock + * ranges off in a secondary structure because memblock is modified + * in memory hotplug while TDX memory regions are fixed. + */ +static int build_tdx_memlist(struct list_head *tmb_list) +{ + unsigned long start_pfn, end_pfn; + int i, ret; + + for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, NULL) { + /* + * The first 1MB is not reported as TDX convertible memory. + * Although the first 1MB is always reserved and won't end up + * to the page allocator, it is still in memblock's memory + * regions. Skip them manually to exclude them as TDX memory. + */ + start_pfn = max(start_pfn, PHYS_PFN(SZ_1M)); + if (start_pfn >= end_pfn) + continue; + + /* + * Add the memory regions as TDX memory. The regions in + * memblock has already guaranteed they are in address + * ascending order and don't overlap. + */ + ret = add_tdx_memblock(tmb_list, start_pfn, end_pfn); + if (ret) + goto err; + } + + return 0; +err: + free_tdx_memlist(tmb_list); + return ret; +} + static int init_tdx_module(void) { static DECLARE_PADDED_STRUCT(tdsysinfo_struct, tdsysinfo, @@ -329,10 +467,25 @@ static int init_tdx_module(void) if (ret) return ret; + /* + * To keep things simple, assume that all TDX-protected memory + * will come from the page allocator. Make sure all pages in the + * page allocator are TDX-usable memory. + * + * Build the list of "TDX-usable" memory regions which cover all + * pages in the page allocator to guarantee that. Do it while + * holding mem_hotplug_lock read-lock as the memory hotplug code + * path reads the @tdx_memlist to reject any new memory. + */ + get_online_mems(); + + ret = build_tdx_memlist(&tdx_memlist); + if (ret) + goto out; + /* * TODO: * - * - Build the list of TDX-usable memory regions. * - Construct a list of "TD Memory Regions" (TDMRs) to cover * all TDX-usable memory regions. * - Configure the TDMRs and the global KeyID to the TDX module. @@ -341,7 +494,15 @@ static int init_tdx_module(void) * * Return error before all steps are done. */ - return -EINVAL; + ret = -EINVAL; +out: + /* + * @tdx_memlist is written here and read at memory hotplug time. + * Lock out memory hotplug code while building it. + */ + put_online_mems(); + + return ret; } static int __tdx_enable(void) diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index 2f2d8737a364..6518024fcb68 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -101,6 +101,12 @@ enum tdx_module_status_t { TDX_MODULE_ERROR }; +struct tdx_memblock { + struct list_head list; + unsigned long start_pfn; + unsigned long end_pfn; +}; + struct tdx_module_output; u64 __seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, struct tdx_module_output *out);