From patchwork Thu Mar 17 13:58:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiaoyao Li X-Patchwork-Id: 12784111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69CECC433EF for ; Thu, 17 Mar 2022 14:13:29 +0000 (UTC) Received: from localhost ([::1]:39894 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nUqsa-0008IL-HX for qemu-devel@archiver.kernel.org; Thu, 17 Mar 2022 10:13:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56904) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nUqfk-0000H2-Fn for qemu-devel@nongnu.org; Thu, 17 Mar 2022 10:00:13 -0400 Received: from mga12.intel.com ([192.55.52.136]:24925) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nUqfi-0004Dr-CU for qemu-devel@nongnu.org; Thu, 17 Mar 2022 10:00:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647525610; x=1679061610; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q1Bv0zE7QJL5HRu3M4Mn9ubF+qQKJfAicxrXrBFlJxo=; b=ltFplh8UEvbrkg0xw61Cl/aT+15xoakYygA7E053pKm/1mznXogz9LCD omuX26pa08LyBzpDeyW5gbXBSp3KVDkhnghZX0/4HMHlkDr6yeQArvApx Go1yF3QUkkDLxD0/bUB71AnEDWiSV9aeCQSXDjz4RAs0xPnORI1PvqLfb KvsCMcpC4nSoVplidYF5fzqi1JE4cTTtnOh2+3KknErcTRB5ihuf5ekHb rgerZyiVRzhUuaV952dzAKXjOYozKL6a5Zu2eIz+S1iICrrXi2iHBLRwy 7oFtwxzqPqZDw5TXhmL8pwIclaMxARu6cCK3PCjYQaM1B+UQ7A62tafwJ A==; X-IronPort-AV: E=McAfee;i="6200,9189,10288"; a="236816769" X-IronPort-AV: E=Sophos;i="5.90,188,1643702400"; d="scan'208";a="236816769" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Mar 2022 07:00:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,188,1643702400"; d="scan'208";a="541378080" Received: from lxy-dell.sh.intel.com ([10.239.159.55]) by orsmga007.jf.intel.com with ESMTP; 17 Mar 2022 07:00:03 -0700 From: Xiaoyao Li To: Paolo Bonzini , =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= , Richard Henderson , "Michael S. Tsirkin" , Marcel Apfelbaum , Cornelia Huck , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Marcelo Tosatti , Laszlo Ersek , Gerd Hoffmann , Eric Blake Subject: [RFC PATCH v3 11/36] i386/tdx: Initialize TDX before creating TD vcpus Date: Thu, 17 Mar 2022 21:58:48 +0800 Message-Id: <20220317135913.2166202-12-xiaoyao.li@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220317135913.2166202-1-xiaoyao.li@intel.com> References: <20220317135913.2166202-1-xiaoyao.li@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.136; envelope-from=xiaoyao.li@intel.com; helo=mga12.intel.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HK_RANDOM_ENVFROM=0.998, HK_RANDOM_FROM=0.998, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: isaku.yamahata@intel.com, kvm@vger.kernel.org, Connor Kuehl , seanjc@google.com, xiaoyao.li@intel.com, qemu-devel@nongnu.org, erdemaktas@google.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Invoke KVM_TDX_INIT in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT configures global TD state, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs. Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM and tie x86cpu->enable_pmu with TD's attributes. Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init(). Signed-off-by: Xiaoyao Li --- accel/kvm/kvm-all.c | 9 ++++++- target/i386/kvm/kvm.c | 3 +++ target/i386/kvm/tdx-stub.c | 5 ++++ target/i386/kvm/tdx.c | 49 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 4 ++++ 5 files changed, 69 insertions(+), 1 deletion(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index a4bb449737a6..fceb6b618b04 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -465,10 +465,17 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp) trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu)); + /* + * tdx_pre_create_vcpu() may call cpu_x86_cpuid(). It in turn may call + * kvm_vm_ioctl(). Set cpu->kvm_state in advance to avoid NULL pointer + * dereference. + */ + cpu->kvm_state = s; ret = kvm_arch_pre_create_vcpu(cpu); if (ret < 0) { error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_arch_pre_create_vcpu() failed"); + cpu->kvm_state = NULL; goto err; } @@ -476,11 +483,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp) if (ret < 0) { error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)", kvm_arch_vcpu_id(cpu)); + cpu->kvm_state = NULL; goto err; } cpu->kvm_fd = ret; - cpu->kvm_state = s; cpu->vcpu_dirty = true; cpu->dirty_pages = 0; diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 02849f6ef142..f2d71359b59d 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -2112,6 +2112,9 @@ int kvm_arch_init_vcpu(CPUState *cs) int kvm_arch_pre_create_vcpu(CPUState *cpu) { + if (is_tdx_vm()) + return tdx_pre_create_vcpu(cpu); + return 0; } diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c index 1df24735201e..2871de9d7b56 100644 --- a/target/i386/kvm/tdx-stub.c +++ b/target/i386/kvm/tdx-stub.c @@ -7,3 +7,8 @@ int tdx_kvm_init(MachineState *ms, Error **errp) { return -EINVAL; } + +int tdx_pre_create_vcpu(CPUState *cpu) +{ + return -EINVAL; +} diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c index e4ee55f30c79..a5cc187edbde 100644 --- a/target/i386/kvm/tdx.c +++ b/target/i386/kvm/tdx.c @@ -18,6 +18,7 @@ #include "sysemu/kvm.h" #include "hw/i386/x86.h" +#include "kvm_i386.h" #include "tdx.h" static TdxGuest *tdx_guest; @@ -149,6 +150,52 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg, } } +int tdx_pre_create_vcpu(CPUState *cpu) +{ + struct { + struct kvm_cpuid2 cpuid; + struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES]; + } cpuid_data; + + /* + * The kernel defines these structs with padding fields so there + * should be no extra padding in our cpuid_data struct. + */ + QEMU_BUILD_BUG_ON(sizeof(cpuid_data) != + sizeof(struct kvm_cpuid2) + + sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES); + + MachineState *ms = MACHINE(qdev_get_machine()); + X86CPU *x86cpu = X86_CPU(cpu); + CPUX86State *env = &x86cpu->env; + struct kvm_tdx_init_vm init_vm; + int r = 0; + + qemu_mutex_lock(&tdx_guest->lock); + if (tdx_guest->initialized) { + goto out; + } + + memset(&cpuid_data, 0, sizeof(cpuid_data)); + cpuid_data.cpuid.nent = kvm_x86_arch_cpuid(env, cpuid_data.entries, 0); + + init_vm.cpuid = (__u64)(&cpuid_data); + init_vm.max_vcpus = ms->smp.cpus; + init_vm.attributes = tdx_guest->attributes; + + r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, &init_vm); + if (r < 0) { + error_report("KVM_TDX_INIT_VM failed %s", strerror(-r)); + goto out; + } + + tdx_guest->initialized = true; + +out: + qemu_mutex_unlock(&tdx_guest->lock); + return r; +} + /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest, @@ -161,6 +208,8 @@ static void tdx_guest_init(Object *obj) { TdxGuest *tdx = TDX_GUEST(obj); + qemu_mutex_init(&tdx->lock); + tdx->attributes = 0; } diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h index 06599b65b827..46a24ee8c7cc 100644 --- a/target/i386/kvm/tdx.h +++ b/target/i386/kvm/tdx.h @@ -17,6 +17,9 @@ typedef struct TdxGuestClass { typedef struct TdxGuest { ConfidentialGuestSupport parent_obj; + QemuMutex lock; + + bool initialized; uint64_t attributes; /* TD attributes */ } TdxGuest; @@ -29,5 +32,6 @@ bool is_tdx_vm(void); int tdx_kvm_init(MachineState *ms, Error **errp); void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg, uint32_t *ret); +int tdx_pre_create_vcpu(CPUState *cpu); #endif /* QEMU_I386_TDX_H */