From patchwork Sat Feb 22 01:42:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986477 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 575F21E2847; Sat, 22 Feb 2025 01:40:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188458; cv=none; b=CuNSaj9x115N/RJcmYUfHm76ggHOHj6bYn6lBg3JOUzGZ5A5UaJ7CaeQb5VXz6QzyBAXPGF8LwOa+4Y4reEoerG50Ce+lTOekXEZbKny+ojGYN4zjQaHdhs17dZMwOhc0pSo/g1RTiWVZBP68JQNbZeaDeHQOHOp4tP3LuMx/9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188458; c=relaxed/simple; bh=cMGVQk/B4sh5x7ZgVUEmqPIMM1GzcauaGKpabgrjf9Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P4BKXrkz/g9E63ccx9d4Ivhf1VlWwwtRNMXp6FzaI9rvBl6gdmPriECkukE0pQ8qRlm9j5sQufuU9wL0ZmUm4dgea/tA5L3xLQAS+ByNRBEMD5Ru9tYAoe7XK5kDw1OCxUzov0+qhNF7yXPYZes87S4o3ZqkAiexdTb8nWt0UUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Bn7SRBpZ; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Bn7SRBpZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188456; x=1771724456; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cMGVQk/B4sh5x7ZgVUEmqPIMM1GzcauaGKpabgrjf9Q=; b=Bn7SRBpZH6tqFycPY1DJzCIGBcvd+amjmG3FB1MH0eZ8mT+PP7O4EPOO JmBu3DFHFZtCVQKPqzPR+Ulz8cjEl7OvDHhshy7sGroQeZZ1Kz9Bd+eco orZhvYwRHWfTN48XpK3WSX8lGFwlDE6TN6OAXig0tLlDxVJ7xwAmEeC0S b4r0b30mBAWUPfXH3aL+AhYuwSRKdmTG91q3UaRqCz/o9HGsF3pVnCXQr xHVPlJucZ4MNX/73rdTdes84uKOoHSdPvGXF/06Vh7HaT89GurlJ/+0Xq mXany81cX1X6CLQDlx29ojeSvntm1usvprEwof9sgNWUOLKxWgNNu/2Fv w==; X-CSE-ConnectionGUID: 2o//q4wLQuuzUYOeyKziJA== X-CSE-MsgGUID: vHIDQ8IbRFi6E3I78XgvsQ== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893250" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893250" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:40:56 -0800 X-CSE-ConnectionGUID: Pktr7dBZQg6sorfNrTgZrA== X-CSE-MsgGUID: PknHIJhURECoi7jVnMx5eQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370229" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:40:53 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 1/9] KVM: x86: Have ____kvm_emulate_hypercall() read the GPRs Date: Sat, 22 Feb 2025 09:42:17 +0800 Message-ID: <20250222014225.897298-2-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Have ____kvm_emulate_hypercall() read the GPRs instead of passing them in via the macro. When emulating KVM hypercalls via TDVMCALL, TDX will marshall registers of TDVMCALL ABI into KVM's x86 registers to match the definition of KVM hypercall ABI _before_ ____kvm_emulate_hypercall() gets called. Therefore, ____kvm_emulate_hypercall() can just read registers internally based on KVM hypercall ABI, and those registers can be removed from the __kvm_emulate_hypercall() macro. Also, op_64_bit can be determined inside ____kvm_emulate_hypercall(), remove it from the __kvm_emulate_hypercall() macro as well. No functional change intended. Suggested-by: Sean Christopherson Signed-off-by: Binbin Wu Reviewed-by: Kai Huang --- Hypercalls exit to userspace v3: - Add RB from Kai. --- arch/x86/kvm/x86.c | 15 ++++++++------- arch/x86/kvm/x86.h | 26 +++++++++----------------- 2 files changed, 17 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0bf1727d9f72..62dded70932d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10022,13 +10022,16 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } -int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, int (*complete_hypercall)(struct kvm_vcpu *)) { unsigned long ret; + unsigned long nr = kvm_rax_read(vcpu); + unsigned long a0 = kvm_rbx_read(vcpu); + unsigned long a1 = kvm_rcx_read(vcpu); + unsigned long a2 = kvm_rdx_read(vcpu); + unsigned long a3 = kvm_rsi_read(vcpu); + int op_64_bit = is_64_bit_hypercall(vcpu); ++vcpu->stat.hypercalls; @@ -10131,9 +10134,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) if (kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); - return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi, - is_64_bit_hypercall(vcpu), - kvm_x86_call(get_cpl)(vcpu), + return __kvm_emulate_hypercall(vcpu, kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 91e50a513100..8b27f70c6321 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -621,25 +621,17 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, int cpl, int (*complete_hypercall)(struct kvm_vcpu *)); -#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ -({ \ - int __ret; \ - \ - __ret = ____kvm_emulate_hypercall(_vcpu, \ - kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ - kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ - kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ - complete_hypercall); \ - \ - if (__ret > 0) \ - __ret = complete_hypercall(_vcpu); \ - __ret; \ +#define __kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall) \ +({ \ + int __ret; \ + __ret = ____kvm_emulate_hypercall(_vcpu, cpl, complete_hypercall); \ + \ + if (__ret > 0) \ + __ret = complete_hypercall(_vcpu); \ + __ret; \ }) int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); From patchwork Sat Feb 22 01:42:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986478 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D77FF1FAC42; Sat, 22 Feb 2025 01:40:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188461; cv=none; b=LFYn5JqkOi/1nYZ7lGwrJ3DLMrHWfqeIF8AwHodt7lpHI+90JwiIwxy/rP+s+IpO0F4+l8bRNPLRhsPdPzSvlioW0fXAqRGtnK6YtO628CEJ9t8OKz4aYmk0s/6M4PUE2dv2MMBlsa9MD5PNOAuUp7yiSeqynRoQZm+Ce7ic/ps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188461; c=relaxed/simple; bh=ySYH9hbc9/2+JlRdr+9eFJsonG3NTdIu0QBzzBYw10M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fD85rlxmqPs02nsFZkBlwVmcuhmQj7Ng8l1lnItwwGzBBp9fSFkTJqUKkQ+1B2Rb+4bUY+6qx2lnZTiLqaLozTNRj3llfJxFrSUPeptUiarQu2/ZDLLVaDlBoo/75BbG5Vxdq5F8qiyNkS82XiWtlcgAjskoh0xh0FOA6xqvujg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R5bXlJYq; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R5bXlJYq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188460; x=1771724460; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ySYH9hbc9/2+JlRdr+9eFJsonG3NTdIu0QBzzBYw10M=; b=R5bXlJYqYYjZI/XhJ3sH0Bo80adRzo55UVcRh/iTjtsN1689ZvqtCHQs 5innFvBEjdZTmxkWg3Lo2p2WC+b4B7AVS/9jYJx2MhjzKrskkCQeH43m3 BxnoAyd1yiSz4t0T1PUO5cZyc5n0VlPZMpBPwCphIBHTtqXRQr+D2lks8 zDoZ1S0i+ycYI5et4dArghpU3iaprio0e2eM8ZE2xbiEqddMuXMFV0BSU kKgEaNGdXvVtRwhYLAY5p+2H4sE1e5t+wwtSwALMcZrDFDOwjV1SHoKMf sYPj56bS901W+MKLpHTFkE8Osm0sx8pjuWrIUUp5sveggaPaOVljkkou8 A==; X-CSE-ConnectionGUID: k6xzC0iFR4Wk/b2fZvmXeA== X-CSE-MsgGUID: YBvnLFxAQjeODrNUVI0Y/A== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893256" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893256" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:40:59 -0800 X-CSE-ConnectionGUID: ECe0ivhOTha0/Hs8hDk7xg== X-CSE-MsgGUID: hWZ/yPtEQTmdZlO9tMeRGw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370235" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:40:56 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 2/9] KVM: x86: Move pv_unhaulted check out of kvm_vcpu_has_events() Date: Sat, 22 Feb 2025 09:42:18 +0800 Message-ID: <20250222014225.897298-3-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Move pv_unhaulted check out of kvm_vcpu_has_events(), check pv_unhaulted explicitly when handling PV unhalt and expose kvm_vcpu_has_events(). kvm_vcpu_has_events() returns true if pv_unhalted is set, and pv_unhalted is only cleared on transitions to KVM_MP_STATE_RUNNABLE. If the guest initiates a spurious wakeup, pv_unhalted could be left set in perpetuity. Currently, this is not problematic because kvm_vcpu_has_events() is only called when handling PV unhalt. However, if kvm_vcpu_has_events() is used for other purposes in the future, it could return the unexpected results. Export kvm_vcpu_has_events() for its usage in broader contexts. Suggested-by: Sean Christopherson Signed-off-by: Binbin Wu --- Hypercalls exit to userspace v3: - New added. --- arch/x86/kvm/x86.c | 11 +++++------ include/linux/kvm_host.h | 1 + 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 62dded70932d..8877d6db9b84 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11133,7 +11133,7 @@ static bool kvm_vcpu_running(struct kvm_vcpu *vcpu) !vcpu->arch.apf.halted); } -static bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) +bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) { if (!list_empty_careful(&vcpu->async_pf.done)) return true; @@ -11142,9 +11142,6 @@ static bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) kvm_apic_init_sipi_allowed(vcpu)) return true; - if (vcpu->arch.pv.pv_unhalted) - return true; - if (kvm_is_exception_pending(vcpu)) return true; @@ -11182,10 +11179,12 @@ static bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) return false; } +EXPORT_SYMBOL_GPL(kvm_vcpu_has_events); int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) { - return kvm_vcpu_running(vcpu) || kvm_vcpu_has_events(vcpu); + return kvm_vcpu_running(vcpu) || vcpu->arch.pv.pv_unhalted || + kvm_vcpu_has_events(vcpu); } /* Called within kvm->srcu read side. */ @@ -11321,7 +11320,7 @@ static int __kvm_emulate_halt(struct kvm_vcpu *vcpu, int state, int reason) */ ++vcpu->stat.halt_exits; if (lapic_in_kernel(vcpu)) { - if (kvm_vcpu_has_events(vcpu)) + if (kvm_vcpu_has_events(vcpu) || vcpu->arch.pv.pv_unhalted) vcpu->arch.pv.pv_unhalted = false; else vcpu->arch.mp_state = state; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3bfe3140f444..ed1968f6f841 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1609,6 +1609,7 @@ void kvm_arch_disable_virtualization(void); int kvm_arch_enable_virtualization_cpu(void); void kvm_arch_disable_virtualization_cpu(void); #endif +bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu); int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu); bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu); int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu); From patchwork Sat Feb 22 01:42:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986479 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10DE21FFC4A; Sat, 22 Feb 2025 01:41:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188466; cv=none; b=Zl4ac2QVc26LrrOb7JzQWCsVR+tliVBk9zgCC0QA4A7eDEYHRe+6CIgOjp3h04NYHqgzZtqWf4dXFwWo9sg2LK5ajIdCYbd/lSJmGq6Z+4l2Ep4KaAulWJst/XYmv7VHgL4KlSR7eXLKgnYjgwvp6ZumnrUuBoerl/X1DB4giuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188466; c=relaxed/simple; bh=lIDHlItnHSZG5XjLQuQWbhAdmVw8446SmnAN+G1i2/8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Epxmk25KmDYlrwPDhXd61ntRBSI7ul7jc1Amv10AV4db/sYY12xsl6wUZ5DXsuxhafOUouXtHjgqZXXgP5Xip43JzpUkIgzPtUzk074WEF+c9QTqYRENeDsJxhcLBAFDMuM/8qdWbMPAtqvl/5mBN4oa+vD+A3ZAbIgx4KzBM6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LKPDCrCL; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LKPDCrCL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188464; x=1771724464; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lIDHlItnHSZG5XjLQuQWbhAdmVw8446SmnAN+G1i2/8=; b=LKPDCrCL4CprgO4RDlRxZ62+baaVgzyFRvCb1wSBTRVNSiCVIEQTI0Xj FyvgAbSv/tYRZ82kM08ISfPewTvbQ2E/8vWl8kT5jyNOg3wzjYAGGgPIB 0FY6Wdr7fRk4euIyKqew3rz1ryuMWvLCsBCCPkXufggyydquWvqO3L510 /g6U0sx5Q5iF129Hx2mqlkVaZ2LDAzvLn9TO0FpwpxJg4cPOaZ6PYOaSu qZUkJuvpAGFOWYzF6QJOW2w8t8vye8PQlIHnvJmAdwKa/DGpcMZthQ8Eq 2HHxQDTGESYHCC+M4JjbLs5zIC7zxsjiyQuFssIcVdfouhuq2JLBcwmSG A==; X-CSE-ConnectionGUID: /aQuXq3fQTqHwx4Jmmdnyw== X-CSE-MsgGUID: aRICxRMDSUGRiGAElwPwKQ== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893261" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893261" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:03 -0800 X-CSE-ConnectionGUID: glWJJeIhRjSE/EZnXGaHcQ== X-CSE-MsgGUID: pJVQJj0ESJSIfoxlE6bkrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370242" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:00 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 3/9] KVM: TDX: Add a place holder to handle TDX VM exit Date: Sat, 22 Feb 2025 09:42:19 +0800 Message-ID: <20250222014225.897298-4-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Introduce the wiring for handling TDX VM exits by implementing the callbacks .get_exit_info(), .get_entry_info(), and .handle_exit(). Additionally, add error handling during the TDX VM exit flow, and add a place holder to handle various exit reasons. Store VMX exit reason and exit qualification in struct vcpu_vt for TDX, so that TDX/VMX can use the same helpers to get exit reason and exit qualification. Store extended exit qualification and exit GPA info in struct vcpu_tdx because they are used by TDX code only. Contention Handling: The TDH.VP.ENTER operation may contend with TDH.MEM.* operations due to secure EPT or TD EPOCH. If the contention occurs, the return value will have TDX_OPERAND_BUSY set, prompting the vCPU to attempt re-entry into the guest with EXIT_FASTPATH_EXIT_HANDLED, not EXIT_FASTPATH_REENTER_GUEST, so that the interrupts pending during IN_GUEST_MODE can be delivered for sure. Otherwise, the requester of KVM_REQ_OUTSIDE_GUEST_MODE may be blocked endlessly. Error Handling: - TDX_SW_ERROR: This includes #UD caused by SEAMCALL instruction if the CPU isn't in VMX operation, #GP caused by SEAMCALL instruction when TDX isn't enabled by the BIOS, and TDX_SEAMCALL_VMFAILINVALID when SEAM firmware is not loaded or disabled. - TDX_ERROR: This indicates some check failed in the TDX module, preventing the vCPU from running. - Failed VM Entry: Exit to userspace with KVM_EXIT_FAIL_ENTRY. Handle it separately before handling TDX_NON_RECOVERABLE because when off-TD debug is not enabled, TDX_NON_RECOVERABLE is set. - TDX_NON_RECOVERABLE: Set by the TDX module when the error is non-recoverable, indicating that the TDX guest is dead or the vCPU is disabled. A special case is triple fault, which also sets TDX_NON_RECOVERABLE but exits to userspace with KVM_EXIT_SHUTDOWN, aligning with the VMX case. - Any unhandled VM exit reason will also return to userspace with KVM_EXIT_INTERNAL_ERROR. Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu Reviewed-by: Chao Gao --- Hypercalls exit to userspace v3: - No change. Hypercalls exit to userspace v2: - Record vmx exit reason and exit_qualification in struct vcpu_vt for TDX, so that TDX/VMX can use the same helpers to get exit reason and exit qualification. (Sean) - Handle failed vmentry separately by KVM_EXIT_FAIL_ENTRY. (Xiaoyao) - Remove the print of hkid & set_hkid_to_hpa() for TDX_ERROR or TDX_NON_RECOVERABLE case. (Xiaoyao) - Handle EXIT_REASON_TRIPLE_FAULT in switch case, and drop the helper tdx_handle_triple_fault(), open code it. (Sean) - intr_info should be 0 for the case VMX exit reason is invalid in tdx_get_exit_info(). (Chao) - Combine TDX_OPERAND_BUSY for TDX_OPERAND_ID_TD_EPOCH and TDX_OPERAND_ID_SEPT, use EXIT_FASTPATH_EXIT_HANDLED instead of EXIT_FASTPATH_REENTER_GUEST. Updated comments. - Use helper tdx_operand_busy(). - Add vt_get_entry_info() to implement .get_entry_info() for TDX. Hypercalls exit to userspace v1: - Dropped Paolo's Reviewed-by since the change is not subtle. - Mention addition of .get_exit_info() handler in changelog. (Binbin) - tdh_sept_seamcall() -> tdx_seamcall_sept() in comments. (Binbin) - Do not open code TDX_ERROR_SEPT_BUSY. (Binbin) - "TDH.VP.ENTRY" -> "TDH.VP.ENTER". (Binbin) - Remove the use of union tdx_exit_reason. (Sean) https://lore.kernel.org/kvm/ZfSExlemFMKjBtZb@google.com/ - Add tdx_check_exit_reason() to check a VMX exit reason against the status code of TDH.VP.ENTER. - Move the handling of TDX_ERROR_SEPT_BUSY and (TDX_OPERAND_BUSY | TDX_OPERAND_ID_TD_EPOCH) into fast path, and add a helper function tdx_exit_handlers_fastpath(). - Remove the warning on TDX_SW_ERROR in fastpath, but return without further handling. - Call kvm_machine_check() for EXIT_REASON_MCE_DURING_VMENTRY, align with VMX case. - On failed_vmentry in fast path, return without further handling. - Exit to userspace for #UD and #GP. - Fix whitespace in tdx_get_exit_info() - Add a comment in tdx_handle_exit() to describe failed_vmentry case is handled by TDX_NON_RECOVERABLE handling. - Move the code of handling NMI, exception and external interrupts out of the patch, i.e., the NMI handling in tdx_vcpu_enter_exit() and the wiring of .handle_exit_irqoff() are removed. - Drop the check for VCPU_TD_STATE_INITIALIZED in tdx_handle_exit() because it has been checked in tdx_vcpu_pre_run(). - Update changelog. --- arch/x86/include/asm/tdx.h | 1 + arch/x86/kvm/vmx/main.c | 38 +++++++++- arch/x86/kvm/vmx/tdx.c | 141 ++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 2 + arch/x86/kvm/vmx/tdx_errno.h | 3 + arch/x86/kvm/vmx/x86_ops.h | 8 ++ 6 files changed, 189 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index 54e9e0fbfe27..e6b003fe7f5e 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -18,6 +18,7 @@ * TDX module. */ #define TDX_ERROR _BITUL(63) +#define TDX_NON_RECOVERABLE _BITUL(62) #define TDX_SW_ERROR (TDX_ERROR | GENMASK_ULL(47, 40)) #define TDX_SEAMCALL_VMFAILINVALID (TDX_SW_ERROR | _UL(0xFFFF0000)) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 41bdaa9932c5..75fd11ae6481 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -181,6 +181,15 @@ static fastpath_t vt_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) return vmx_vcpu_run(vcpu, force_immediate_exit); } +static int vt_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath) +{ + if (is_td_vcpu(vcpu)) + return tdx_handle_exit(vcpu, fastpath); + + return vmx_handle_exit(vcpu, fastpath); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -228,6 +237,29 @@ static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, vmx_load_mmu_pgd(vcpu, root_hpa, pgd_level); } +static void vt_get_entry_info(struct kvm_vcpu *vcpu, u32 *intr_info, u32 *error_code) +{ + *intr_info = 0; + *error_code = 0; + + if (is_td_vcpu(vcpu)) + return; + + vmx_get_entry_info(vcpu, intr_info, error_code); +} + +static void vt_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) +{ + if (is_td_vcpu(vcpu)) { + tdx_get_exit_info(vcpu, reason, info1, info2, intr_info, + error_code); + return; + } + + vmx_get_exit_info(vcpu, reason, info1, info2, intr_info, error_code); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -323,7 +355,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_pre_run = vt_vcpu_pre_run, .vcpu_run = vt_vcpu_run, - .handle_exit = vmx_handle_exit, + .handle_exit = vt_handle_exit, .skip_emulated_instruction = vmx_skip_emulated_instruction, .update_emulated_instruction = vmx_update_emulated_instruction, .set_interrupt_shadow = vmx_set_interrupt_shadow, @@ -357,8 +389,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .set_identity_map_addr = vmx_set_identity_map_addr, .get_mt_mask = vmx_get_mt_mask, - .get_exit_info = vmx_get_exit_info, - .get_entry_info = vmx_get_entry_info, + .get_exit_info = vt_get_exit_info, + .get_entry_info = vt_get_entry_info, .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 809ca013484d..4e46ebfb849a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -799,17 +799,70 @@ static bool tdx_guest_state_is_invalid(struct kvm_vcpu *vcpu) !guest_cpu_cap_has(vcpu, X86_FEATURE_XSAVES)); } +static __always_inline u32 tdx_to_vmx_exit_reason(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + switch (tdx->vp_enter_ret & TDX_SEAMCALL_STATUS_MASK) { + case TDX_SUCCESS: + case TDX_NON_RECOVERABLE_VCPU: + case TDX_NON_RECOVERABLE_TD: + case TDX_NON_RECOVERABLE_TD_NON_ACCESSIBLE: + case TDX_NON_RECOVERABLE_TD_WRONG_APIC_MODE: + break; + default: + return -1u; + } + + return tdx->vp_enter_ret; +} + static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); + struct vcpu_vt *vt = to_vt(vcpu); guest_state_enter_irqoff(); tdx->vp_enter_ret = tdh_vp_enter(&tdx->vp, &tdx->vp_enter_args); + vt->exit_reason.full = tdx_to_vmx_exit_reason(vcpu); + + vt->exit_qualification = tdx->vp_enter_args.rcx; + tdx->ext_exit_qualification = tdx->vp_enter_args.rdx; + tdx->exit_gpa = tdx->vp_enter_args.r8; + vt->exit_intr_info = tdx->vp_enter_args.r9; + guest_state_exit_irqoff(); } +static bool tdx_failed_vmentry(struct kvm_vcpu *vcpu) +{ + return vmx_get_exit_reason(vcpu).failed_vmentry && + vmx_get_exit_reason(vcpu).full != -1u; +} + +static fastpath_t tdx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) +{ + u64 vp_enter_ret = to_tdx(vcpu)->vp_enter_ret; + + /* + * TDX_OPERAND_BUSY could be returned for SEPT due to 0-step mitigation + * or for TD EPOCH due to contention with TDH.MEM.TRACK on TDH.VP.ENTER. + * + * When KVM requests KVM_REQ_OUTSIDE_GUEST_MODE, which has both + * KVM_REQUEST_WAIT and KVM_REQUEST_NO_ACTION set, it requires target + * vCPUs leaving fastpath so that interrupt can be enabled to ensure the + * IPIs can be delivered. Return EXIT_FASTPATH_EXIT_HANDLED instead of + * EXIT_FASTPATH_REENTER_GUEST to exit fastpath, otherwise, the + * requester may be blocked endlessly. + */ + if (unlikely(tdx_operand_busy(vp_enter_ret))) + return EXIT_FASTPATH_EXIT_HANDLED; + + return EXIT_FASTPATH_NONE; +} + #define TDX_REGS_UNSUPPORTED_SET (BIT(VCPU_EXREG_RFLAGS) | \ BIT(VCPU_EXREG_SEGMENTS)) @@ -855,9 +908,18 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) vcpu->arch.regs_avail &= ~TDX_REGS_UNSUPPORTED_SET; + if (unlikely((tdx->vp_enter_ret & TDX_SW_ERROR) == TDX_SW_ERROR)) + return EXIT_FASTPATH_NONE; + + if (unlikely(vmx_get_exit_reason(vcpu).basic == EXIT_REASON_MCE_DURING_VMENTRY)) + kvm_machine_check(); + trace_kvm_exit(vcpu, KVM_ISA_VMX); - return EXIT_FASTPATH_NONE; + if (unlikely(tdx_failed_vmentry(vcpu))) + return EXIT_FASTPATH_NONE; + + return tdx_exit_handlers_fastpath(vcpu); } void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) @@ -1173,6 +1235,83 @@ int tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn, return tdx_sept_drop_private_spte(kvm, gfn, level, page); } +int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + u64 vp_enter_ret = tdx->vp_enter_ret; + union vmx_exit_reason exit_reason = vmx_get_exit_reason(vcpu); + + if (fastpath != EXIT_FASTPATH_NONE) + return 1; + + /* + * Handle TDX SW errors, including TDX_SEAMCALL_UD, TDX_SEAMCALL_GP and + * TDX_SEAMCALL_VMFAILINVALID. + */ + if (unlikely((vp_enter_ret & TDX_SW_ERROR) == TDX_SW_ERROR)) { + KVM_BUG_ON(!kvm_rebooting, vcpu->kvm); + goto unhandled_exit; + } + + if (unlikely(tdx_failed_vmentry(vcpu))) { + /* + * If the guest state is protected, that means off-TD debug is + * not enabled, TDX_NON_RECOVERABLE must be set. + */ + WARN_ON_ONCE(vcpu->arch.guest_state_protected && + !(vp_enter_ret & TDX_NON_RECOVERABLE)); + vcpu->run->exit_reason = KVM_EXIT_FAIL_ENTRY; + vcpu->run->fail_entry.hardware_entry_failure_reason = exit_reason.full; + vcpu->run->fail_entry.cpu = vcpu->arch.last_vmentry_cpu; + return 0; + } + + if (unlikely(vp_enter_ret & (TDX_ERROR | TDX_NON_RECOVERABLE)) && + exit_reason.basic != EXIT_REASON_TRIPLE_FAULT) { + kvm_pr_unimpl("TD vp_enter_ret 0x%llx\n", vp_enter_ret); + goto unhandled_exit; + } + + WARN_ON_ONCE(exit_reason.basic != EXIT_REASON_TRIPLE_FAULT && + (vp_enter_ret & TDX_SEAMCALL_STATUS_MASK) != TDX_SUCCESS); + + switch (exit_reason.basic) { + case EXIT_REASON_TRIPLE_FAULT: + vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN; + vcpu->mmio_needed = 0; + return 0; + default: + break; + } + +unhandled_exit: + vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; + vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON; + vcpu->run->internal.ndata = 2; + vcpu->run->internal.data[0] = vp_enter_ret; + vcpu->run->internal.data[1] = vcpu->arch.last_vmentry_cpu; + return 0; +} + +void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + *reason = tdx->vt.exit_reason.full; + if (*reason != -1u) { + *info1 = vmx_get_exit_qual(vcpu); + *info2 = tdx->ext_exit_qualification; + *intr_info = vmx_get_intr_info(vcpu); + } else { + *info1 = 0; + *info2 = 0; + *intr_info = 0; + } + + *error_code = 0; +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { const struct tdx_sys_info_td_conf *td_conf = &tdx_sysinfo->td_conf; diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 5650691ddd14..3b1cfa737b70 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -47,6 +47,8 @@ enum vcpu_tdx_state { struct vcpu_tdx { struct kvm_vcpu vcpu; struct vcpu_vt vt; + u64 ext_exit_qualification; + gpa_t exit_gpa; struct tdx_module_args vp_enter_args; struct tdx_vp vp; diff --git a/arch/x86/kvm/vmx/tdx_errno.h b/arch/x86/kvm/vmx/tdx_errno.h index f9dbb3a065cc..6ff4672c4181 100644 --- a/arch/x86/kvm/vmx/tdx_errno.h +++ b/arch/x86/kvm/vmx/tdx_errno.h @@ -10,6 +10,9 @@ * TDX SEAMCALL Status Codes (returned in RAX) */ #define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_NON_RECOVERABLE_TD 0x4000000200000000ULL +#define TDX_NON_RECOVERABLE_TD_NON_ACCESSIBLE 0x6000000500000000ULL +#define TDX_NON_RECOVERABLE_TD_WRONG_APIC_MODE 0x6000000700000000ULL #define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL #define TDX_OPERAND_INVALID 0xC000010000000000ULL #define TDX_OPERAND_BUSY 0x8000020000000000ULL diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index eccba018386a..bc76f5b60b0e 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -136,6 +136,10 @@ int tdx_vcpu_pre_run(struct kvm_vcpu *vcpu); fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit); void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); void tdx_vcpu_put(struct kvm_vcpu *vcpu); +int tdx_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath); +void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -170,6 +174,10 @@ static inline fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediat } static inline void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_put(struct kvm_vcpu *vcpu) {} +static inline int tdx_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath) { return 0; } +static inline void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, + u64 *info2, u32 *intr_info, u32 *error_code) {} static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Sat Feb 22 01:42:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986480 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9595B200130; Sat, 22 Feb 2025 01:41:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188469; cv=none; b=Oei0EbD/eJ7aU5/u5CsV3yX1rheAnqTEz0/4bQSKY8CboSxogVK7Ixsg/Y31Y96RI7k/sI2P3xM0QGJ6CUijUZDba0B0JKGomMkUDhD6bocbnVfI8mDTMvAx/1stEOpti2VwG8KGRokJ9YsXpYuEAwGw/SMP8UWduaPKEwMzvJY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188469; c=relaxed/simple; bh=za4bNwA2okJnYR/YSYB+oyMuCgwn6f1EQ8xQRtbHG0U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Suz/AbzadkPvQzvo5ZCHjyYtnqW7sLMdPN6V4a9vkIl0j4vIsx85oc4g9dnNaAcXKufjL4C2W64Rm+2vQIMWevVBwrlAIvzvf4bJ4gipHMH/qt5gx//sMuDape/n+PfyAYYbOurNu51nvSSvOCqV/MxIfJr672EDduHnyGbxY9U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gs93dp8M; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gs93dp8M" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188467; x=1771724467; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=za4bNwA2okJnYR/YSYB+oyMuCgwn6f1EQ8xQRtbHG0U=; b=gs93dp8M+J52PEVF8GbcaU/h5DwR6HVKjYbgQevm2C70sqKGSekzxIcD RcWL1CIMDre7f9LxkDAuaeRh+D2o40qaU1ImeHNoKVaWLC6XsnTDMdcI4 ddxkaQ/pxg8y0GyxO3Udc0VDWLqvEEB080noVWm+CycP0/m0AZwrJpToy Q0QniRWVG2aoiy0l9tf2vBblnDugMu/NBpreDx6UzgxGIamgNkeYiVRUU 5nu9ms0VtZz0xX4LKEeQjTohP/SruZ1VPd3o37nSRHC5vUmHbIhfrjHEx y3vroGdz7qrlT+gyuRTFV1eyiQSjGreSBVFOnfBSLpBc0beIb384hQuvA Q==; X-CSE-ConnectionGUID: CpPlL0+PTlG6tILbmzy9UQ== X-CSE-MsgGUID: Ud/CrJ4ZRu+i9/IATgH0cg== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893272" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893272" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:07 -0800 X-CSE-ConnectionGUID: u+K80Uy6RvyjIfrWOJTEBQ== X-CSE-MsgGUID: DzAoefyURqGOTEj5octNDA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370246" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:04 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 4/9] KVM: TDX: Add a place holder for handler of TDX hypercalls (TDG.VP.VMCALL) Date: Sat, 22 Feb 2025 09:42:20 +0800 Message-ID: <20250222014225.897298-5-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add a place holder and related helper functions for preparation of TDG.VP.VMCALL handling. The TDX module specification defines TDG.VP.VMCALL API (TDVMCALL for short) for the guest TD to call hypercall to VMM. When the guest TD issues a TDVMCALL, the guest TD exits to VMM with a new exit reason. The arguments from the guest TD and returned values from the VMM are passed in the guest registers. The guest RCX register indicates which registers are used. Define helper functions to access those registers. A new VMX exit reason TDCALL is added to indicate the exit is due to TDVMCALL from the guest TD. Define the TDCALL exit reason and add a place holder to handle such exit. Some leafs of TDCALL will be morphed to another VMX exit reason instead of EXIT_REASON_TDCALL, add a helper tdcall_to_vmx_exit_reason() as a place holder to do the conversion. Suggested-by: Sean Christopherson Co-developed-by: Xiaoyao Li Signed-off-by: Xiaoyao Li Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu Reviewed-by: Chao Gao --- Hypercalls exit to userspace v3: - Add missing blank new lines. (Chao) - Do not morph all leafs less than 0x10000 unconditionally. (Chao) Hypercalls exit to userspace v2: - Get/set tdvmcall inputs/outputs from/to vp_enter_args. - Morph the guest requested exit reason (via TDVMCALL) to KVM's tracked exit reason when it could, i.e. when the TDVMCALL leaf number is less than 0x10000. (Sean) - Drop helpers for read/write a0~a3. Hypercalls exit to userspace v1: - Update changelog. - Drop the unused tdx->tdvmcall. (Chao) - Use TDVMCALL_STATUS prefix for TDX call status codes (Binbin) --- arch/x86/include/uapi/asm/vmx.h | 4 ++- arch/x86/kvm/vmx/tdx.c | 60 ++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h index a5faf6d88f1b..6a9f268a2d2c 100644 --- a/arch/x86/include/uapi/asm/vmx.h +++ b/arch/x86/include/uapi/asm/vmx.h @@ -92,6 +92,7 @@ #define EXIT_REASON_TPAUSE 68 #define EXIT_REASON_BUS_LOCK 74 #define EXIT_REASON_NOTIFY 75 +#define EXIT_REASON_TDCALL 77 #define VMX_EXIT_REASONS \ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \ @@ -155,7 +156,8 @@ { EXIT_REASON_UMWAIT, "UMWAIT" }, \ { EXIT_REASON_TPAUSE, "TPAUSE" }, \ { EXIT_REASON_BUS_LOCK, "BUS_LOCK" }, \ - { EXIT_REASON_NOTIFY, "NOTIFY" } + { EXIT_REASON_NOTIFY, "NOTIFY" }, \ + { EXIT_REASON_TDCALL, "TDCALL" } #define VMX_EXIT_REASON_FLAGS \ { VMX_EXIT_REASONS_FAILED_VMENTRY, "FAILED_VMENTRY" } diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 4e46ebfb849a..7a5f375d976a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -224,6 +224,28 @@ static bool tdx_operand_busy(u64 err) */ static DEFINE_PER_CPU(struct list_head, associated_tdvcpus); +static __always_inline unsigned long tdvmcall_exit_type(struct kvm_vcpu *vcpu) +{ + return to_tdx(vcpu)->vp_enter_args.r10; +} + +static __always_inline unsigned long tdvmcall_leaf(struct kvm_vcpu *vcpu) +{ + return to_tdx(vcpu)->vp_enter_args.r11; +} + +static __always_inline void tdvmcall_set_return_code(struct kvm_vcpu *vcpu, + long val) +{ + to_tdx(vcpu)->vp_enter_args.r10 = val; +} + +static __always_inline void tdvmcall_set_return_val(struct kvm_vcpu *vcpu, + unsigned long val) +{ + to_tdx(vcpu)->vp_enter_args.r11 = val; +} + static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx) { tdx_guest_keyid_free(kvm_tdx->hkid); @@ -799,9 +821,20 @@ static bool tdx_guest_state_is_invalid(struct kvm_vcpu *vcpu) !guest_cpu_cap_has(vcpu, X86_FEATURE_XSAVES)); } +static __always_inline u32 tdcall_to_vmx_exit_reason(struct kvm_vcpu *vcpu) +{ + switch (tdvmcall_leaf(vcpu)) { + default: + break; + } + + return EXIT_REASON_TDCALL; +} + static __always_inline u32 tdx_to_vmx_exit_reason(struct kvm_vcpu *vcpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); + u32 exit_reason; switch (tdx->vp_enter_ret & TDX_SEAMCALL_STATUS_MASK) { case TDX_SUCCESS: @@ -814,7 +847,19 @@ static __always_inline u32 tdx_to_vmx_exit_reason(struct kvm_vcpu *vcpu) return -1u; } - return tdx->vp_enter_ret; + exit_reason = tdx->vp_enter_ret; + + switch (exit_reason) { + case EXIT_REASON_TDCALL: + if (tdvmcall_exit_type(vcpu)) + return EXIT_REASON_VMCALL; + + return tdcall_to_vmx_exit_reason(vcpu); + default: + break; + } + + return exit_reason; } static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu) @@ -922,6 +967,17 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) return tdx_exit_handlers_fastpath(vcpu); } +static int handle_tdvmcall(struct kvm_vcpu *vcpu) +{ + switch (tdvmcall_leaf(vcpu)) { + default: + break; + } + + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND); + return 1; +} + void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { u64 shared_bit = (pgd_level == 5) ? TDX_SHARED_BIT_PWL_5 : @@ -1280,6 +1336,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN; vcpu->mmio_needed = 0; return 0; + case EXIT_REASON_TDCALL: + return handle_tdvmcall(vcpu); default: break; } From patchwork Sat Feb 22 01:42:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986481 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29BB5201270; Sat, 22 Feb 2025 01:41:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188472; cv=none; b=puaRQsOealM8BwOlRgsHPf1UN0HXsjvsV+6bjs0nP7XHJrhOfocJ8KZ2tgfhQy64pNWoSF5hXYHTu75TwrUbH3GwYLuBPPXb6CHwj/aOw/ypBnhY5ENTam2PsYLINNrdAkNTvdF2qWmkT+PUR46Y5w5FFopxpFo7wwYKoxEIz+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188472; c=relaxed/simple; bh=Zkm1Wv929Y7ynRBsU/THIgczu5F+uiDlIs8XolmKdwg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JzQL2C8Ve3WJrF4ggx1C9A+zogrUSPROErzxrXLJO/UezhWqM/n2dU5BxsC51R0Ne2FG3T/jeJP4tgC8+BbBryn7SlUVPIrg9Y6cgGh58mKejxqxdJFTzplMfpCAm5z5QqCPgs1j0R9YKwL61Jj7EpBDYZ+T0tYVGBafONrEsVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XV7fS8lX; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XV7fS8lX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188471; x=1771724471; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Zkm1Wv929Y7ynRBsU/THIgczu5F+uiDlIs8XolmKdwg=; b=XV7fS8lXuOzHIM/r/CSiQjaMWEnvq1R9J0a7zNqmhjsYd7OAsTKhx14r MhlxagPUtNP3s7JmnJV6PDKDVn+BEmexmQ3f5U4kbHp4dvSAPaGDY/SwN 9CWyoMMH9ppX66jcihNLi1uN//oZniD5I9UrKj1cCxdcNVmxrpAtuSpH7 /5ZcwL3D5QZ6mVfwpnTf5CCZG1s/IkzjXeXhCgzFs8PTXOUu322XFttYT LF2T84Gs/MkZ9d8nFoeRdLluqSDhnj06cHebXzy6W5Zc1yFfgI41Qzpmq KrEJY0EdZZ8Cxhh4W9P3Pl9+ELZVwC1zTjdHRsDH9CM/NN2qr6AHESLCh g==; X-CSE-ConnectionGUID: 6LQPFOomRW+W83ufiLx0Jw== X-CSE-MsgGUID: FW8hfV4fSkiz5RH0pAhBpw== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893278" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893278" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:10 -0800 X-CSE-ConnectionGUID: 1DP9U+hWT6CNBp1mX1Qyww== X-CSE-MsgGUID: USt8ZWJSTsOXVOptAOmgbQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370251" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:07 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 5/9] KVM: TDX: Handle KVM hypercall with TDG.VP.VMCALL Date: Sat, 22 Feb 2025 09:42:21 +0800 Message-ID: <20250222014225.897298-6-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Handle KVM hypercall for TDX according to TDX Guest-Host Communication Interface (GHCI) specification. The TDX GHCI specification defines the ABI for the guest TD to issue hypercalls. When R10 is non-zero, it indicates the TDG.VP.VMCALL is vendor-specific. KVM uses R10 as KVM hypercall number and R11-R14 as 4 arguments, while the error code is returned in R10. Morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL and marshall r10~r14 from vp_enter_args to the appropriate x86 registers for KVM hypercall handling. Signed-off-by: Isaku Yamahata Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu --- Hypercalls exit to userspace v3: - No change. Hypercalls exit to userspace v2: - Morph the TDG.VP.VMCALL with KVM hypercall to EXIT_REASON_VMCALL. - Marshall values to the appropriate x86 registers for KVM hypercall handling. Hypercalls exit to userspace v1: - Renamed from "KVM: TDX: handle KVM hypercall with TDG.VP.VMCALL" to "KVM: TDX: Handle KVM hypercall with TDG.VP.VMCALL". - Update the change log. - Rebased on Sean's "Prep KVM hypercall handling for TDX" patch set. https://lore.kernel.org/kvm/20241128004344.4072099-1-seanjc@google.com - Use the right register (i.e. R10) to set the return code after returning back from userspace. --- arch/x86/kvm/vmx/tdx.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7a5f375d976a..4f659ca50469 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -967,6 +967,23 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit) return tdx_exit_handlers_fastpath(vcpu); } +static int complete_hypercall_exit(struct kvm_vcpu *vcpu) +{ + tdvmcall_set_return_code(vcpu, vcpu->run->hypercall.ret); + return 1; +} + +static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) +{ + kvm_rax_write(vcpu, to_tdx(vcpu)->vp_enter_args.r10); + kvm_rbx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r11); + kvm_rcx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r12); + kvm_rdx_write(vcpu, to_tdx(vcpu)->vp_enter_args.r13); + kvm_rsi_write(vcpu, to_tdx(vcpu)->vp_enter_args.r14); + + return __kvm_emulate_hypercall(vcpu, 0, complete_hypercall_exit); +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { @@ -1338,6 +1355,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return 0; case EXIT_REASON_TDCALL: return handle_tdvmcall(vcpu); + case EXIT_REASON_VMCALL: + return tdx_emulate_vmcall(vcpu); default: break; } From patchwork Sat Feb 22 01:42:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986482 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CF272036F0; Sat, 22 Feb 2025 01:41:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188476; cv=none; b=L0hYB8tXjSN8PIBTXSx/lKAWPO3JCtbR7wZFMvPJ5eE6xLv/DHUs3+iP7tdVYY/HjksJKTRzlhLm59iLB1kNf2bDge6ej+MULIJTpUjm3bo5YaozBPf29eZqZ6cVPwqPUUo5VqvVNh0+zL1e44sCoO9v2FxkAt+6j5GA8mdWy7M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188476; c=relaxed/simple; bh=sJSadycwUGmpWEJ2a38Clddht63dvw6WzhYR8Ge6ZQY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U7L6N3SmTQh5+ftqvP7FWViRq3RA6Sk1YOU9qUtLDV0yghXqfKDZwymQ49+eK3hRn/WrAD5dCk5QDvHKNZEnanDgNU+ZPUNW8Ir9Rvf025ooF+VenswLeJRRSNLGyl/MZP5yTNnwLe4ZDZqcwKGS1M6UEgM+cisstFKkAQjS4KU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UHd3C8gG; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UHd3C8gG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188474; x=1771724474; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sJSadycwUGmpWEJ2a38Clddht63dvw6WzhYR8Ge6ZQY=; b=UHd3C8gGVjZwJonYqKSeOPcTyEbd/Ms145ySLaeM3KNc6yZw21Uyiej4 xr3Oj5FCBM6AJTicsRYyc+0MGl/E+t7GWdVyFaOHH67xKhWOpuTwdDzUu pR4Ud+hZIzS3I3SEExxJvPEiiBXR7oWzR5zQDMEwW8HqGWX+QOpxXZni3 eCSorn44AXmrNkIChSWWhLOziFCrGpBLIykRWmqPJekNXEjKxh0Kb3KaE LiEc9sMlPCJfIAgHlpplMP+YSuoQR8yHq8vfkdlzfJ2gcl6qgMacQYc/0 CQ0tbpVLceWTzIARAdmWuEuSSq6SJxrbHSypKb25rucwHTiDQ5LdQsIxI g==; X-CSE-ConnectionGUID: ipDtoUIFT8+eU8XAF1TMxg== X-CSE-MsgGUID: yicOxNnwT06+Irn0UjpkUQ== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893292" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893292" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:14 -0800 X-CSE-ConnectionGUID: lw6sw9LSSbuid4ifAK1WrQ== X-CSE-MsgGUID: VDkaFwo1SVqV/ZniQ9uAgA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370264" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:11 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 6/9] KVM: TDX: Handle TDG.VP.VMCALL Date: Sat, 22 Feb 2025 09:42:22 +0800 Message-ID: <20250222014225.897298-7-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Convert TDG.VP.VMCALL to KVM_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE and forward it to userspace for handling. MapGPA is used by TDX guest to request to map a GPA range as private or shared memory. It needs to exit to userspace for handling. KVM has already implemented a similar hypercall KVM_HC_MAP_GPA_RANGE, which will exit to userspace with exit reason KVM_EXIT_HYPERCALL. Do sanity checks, convert TDVMCALL_MAP_GPA to KVM_HC_MAP_GPA_RANGE and forward the request to userspace. To prevent a TDG.VP.VMCALL call from taking too long, the MapGPA range is split into 2MB chunks and check interrupt pending between chunks. This allows for timely injection of interrupts and prevents issues with guest lockup detection. TDX guest should retry the operation for the GPA starting at the address specified in R11 when the TDVMCALL return TDVMCALL_RETRY as status code. Note userspace needs to enable KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set for TD VM. Suggested-by: Sean Christopherson Signed-off-by: Binbin Wu --- Hypercalls exit to userspace v3: - Use kvm_vcpu_has_events() to replace the open code of NMI/interrupt checks. (Sean) The check of pending RVI, i.e., bit 0 of TDX_VCPU_STATE_DETAILS_INTR_PENDING is skipped for non halted cases in tdx_protected_apic_has_interrupt() in later section. Hypercalls exit to userspace v2: - Skip setting of return code as TDVMCALL_STATUS_SUCCESS. - Use vp_enter_args instead of x86 registers. - Remove unnecessary comments. - Zero run->hypercall.ret in __tdx_map_gpa() following the pattern of Paolo's patch, the feedback of adding a helper is still pending. (Rick) https://lore.kernel.org/kvm/20241213194137.315304-1-pbonzini@redhat.com Hypercalls exit to userspace v1: - New added. Implement one of the hypercalls need to exit to userspace for handling after dropping "KVM: TDX: Add KVM Exit for TDX TDG.VP.VMCALL", which tries to resolve Sean's comment. https://lore.kernel.org/kvm/Zg18ul8Q4PGQMWam@google.com/ - Check interrupt pending between chunks suggested by Sean. https://lore.kernel.org/kvm/ZleJvmCawKqmpFIa@google.com/ - Use TDVMCALL_STATUS prefix for TDX call status codes (Binbin) - Use vt_is_tdx_private_gpa() --- arch/x86/include/asm/shared/tdx.h | 1 + arch/x86/kvm/vmx/tdx.c | 111 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 3 + 3 files changed, 115 insertions(+) diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h index 4aedab1f2a1a..f23657350d28 100644 --- a/arch/x86/include/asm/shared/tdx.h +++ b/arch/x86/include/asm/shared/tdx.h @@ -77,6 +77,7 @@ #define TDVMCALL_STATUS_SUCCESS 0x0000000000000000ULL #define TDVMCALL_STATUS_RETRY 0x0000000000000001ULL #define TDVMCALL_STATUS_INVALID_OPERAND 0x8000000000000000ULL +#define TDVMCALL_STATUS_ALIGN_ERROR 0x8000000000000002ULL /* * Bitmasks of exposed registers (with VMM). diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 4f659ca50469..7b2f4c32f01b 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -984,9 +984,120 @@ static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) return __kvm_emulate_hypercall(vcpu, 0, complete_hypercall_exit); } +/* + * Split into chunks and check interrupt pending between chunks. This allows + * for timely injection of interrupts to prevent issues with guest lockup + * detection. + */ +#define TDX_MAP_GPA_MAX_LEN (2 * 1024 * 1024) +static void __tdx_map_gpa(struct vcpu_tdx *tdx); + +static int tdx_complete_vmcall_map_gpa(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + if (vcpu->run->hypercall.ret) { + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND); + tdx->vp_enter_args.r11 = tdx->map_gpa_next; + return 1; + } + + tdx->map_gpa_next += TDX_MAP_GPA_MAX_LEN; + if (tdx->map_gpa_next >= tdx->map_gpa_end) + return 1; + + /* + * Stop processing the remaining part if there is a pending interrupt, + * which could be qualified to deliver. Skip checking pending RVI for + * TDVMCALL_MAP_GPA. + * TODO: Add a comment to link the reason when the target function is + * implemented. + */ + if (kvm_vcpu_has_events(vcpu)) { + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_RETRY); + tdx->vp_enter_args.r11 = tdx->map_gpa_next; + return 1; + } + + __tdx_map_gpa(tdx); + return 0; +} + +static void __tdx_map_gpa(struct vcpu_tdx *tdx) +{ + u64 gpa = tdx->map_gpa_next; + u64 size = tdx->map_gpa_end - tdx->map_gpa_next; + + if (size > TDX_MAP_GPA_MAX_LEN) + size = TDX_MAP_GPA_MAX_LEN; + + tdx->vcpu.run->exit_reason = KVM_EXIT_HYPERCALL; + tdx->vcpu.run->hypercall.nr = KVM_HC_MAP_GPA_RANGE; + /* + * In principle this should have been -KVM_ENOSYS, but userspace (QEMU <=9.2) + * assumed that vcpu->run->hypercall.ret is never changed by KVM and thus that + * it was always zero on KVM_EXIT_HYPERCALL. Since KVM is now overwriting + * vcpu->run->hypercall.ret, ensuring that it is zero to not break QEMU. + */ + tdx->vcpu.run->hypercall.ret = 0; + tdx->vcpu.run->hypercall.args[0] = gpa & ~gfn_to_gpa(kvm_gfn_direct_bits(tdx->vcpu.kvm)); + tdx->vcpu.run->hypercall.args[1] = size / PAGE_SIZE; + tdx->vcpu.run->hypercall.args[2] = vt_is_tdx_private_gpa(tdx->vcpu.kvm, gpa) ? + KVM_MAP_GPA_RANGE_ENCRYPTED : + KVM_MAP_GPA_RANGE_DECRYPTED; + tdx->vcpu.run->hypercall.flags = KVM_EXIT_HYPERCALL_LONG_MODE; + + tdx->vcpu.arch.complete_userspace_io = tdx_complete_vmcall_map_gpa; +} + +static int tdx_map_gpa(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + u64 gpa = tdx->vp_enter_args.r12; + u64 size = tdx->vp_enter_args.r13; + u64 ret; + + /* + * Converting TDVMCALL_MAP_GPA to KVM_HC_MAP_GPA_RANGE requires + * userspace to enable KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE + * bit set. If not, the error code is not defined in GHCI for TDX, use + * TDVMCALL_STATUS_INVALID_OPERAND for this case. + */ + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) { + ret = TDVMCALL_STATUS_INVALID_OPERAND; + goto error; + } + + if (gpa + size <= gpa || !kvm_vcpu_is_legal_gpa(vcpu, gpa) || + !kvm_vcpu_is_legal_gpa(vcpu, gpa + size - 1) || + (vt_is_tdx_private_gpa(vcpu->kvm, gpa) != + vt_is_tdx_private_gpa(vcpu->kvm, gpa + size - 1))) { + ret = TDVMCALL_STATUS_INVALID_OPERAND; + goto error; + } + + if (!PAGE_ALIGNED(gpa) || !PAGE_ALIGNED(size)) { + ret = TDVMCALL_STATUS_ALIGN_ERROR; + goto error; + } + + tdx->map_gpa_end = gpa + size; + tdx->map_gpa_next = gpa; + + __tdx_map_gpa(tdx); + return 0; + +error: + tdvmcall_set_return_code(vcpu, ret); + tdx->vp_enter_args.r11 = gpa; + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { + case TDVMCALL_MAP_GPA: + return tdx_map_gpa(vcpu); default: break; } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 3b1cfa737b70..70da4e6a93e8 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -58,6 +58,9 @@ struct vcpu_tdx { u64 vp_enter_ret; enum vcpu_tdx_state state; + + u64 map_gpa_next; + u64 map_gpa_end; }; void tdh_vp_rd_failed(struct vcpu_tdx *tdx, char *uclass, u32 field, u64 err); From patchwork Sat Feb 22 01:42:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986483 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20CB92040AB; Sat, 22 Feb 2025 01:41:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188480; cv=none; b=HvPBU/M0FWvo7+duB5bQAjNupodKQCzr6E0rg9I2qGGITk8It8Gbqqm5bsoF1EpDz0LUgdYRV5B5UPcVtA+R5RJ3Nlh14dmE10Wxpe0MW0iyElQK2FtCa1rt0FX6IE4y+oj4OIJG53uCdpmmO7oLzXnzBj0s4QXfAR2NriTkJTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188480; c=relaxed/simple; bh=YWhNX5t2bn0dywpZOM7YmQtLt1z/ih4TjdvqvxCs8Cg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SiVTtb/JsnrLi8wVoi/0d6LFCHrzrsWbTpRTrhj/iyNwYD2GBXJWXzaX4sJfNkX4qGwFI6CHB7Do9PUCsYURk5FhVQtkPnCYud2hpTE8d/KBQ9zlDM/JkXsTLs2UUete6ykYJmudw4P2Uyd60XBAjLhUasuYlHLN6mSAl2q5MqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XNynLVyM; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XNynLVyM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188478; x=1771724478; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YWhNX5t2bn0dywpZOM7YmQtLt1z/ih4TjdvqvxCs8Cg=; b=XNynLVyMgFgmagqdiI16B5d/ymYyfQQNEldkez7L8pkOQVwFjdx3+DOO tyWGsDu/0Fv4pznCq/R/ZeFom6ZdGYgJXUWTI+VsvGPchQeFE1qlEp1Tp zVVNtZaQXT8nv3+wAf/h9vKH6EYS3lijqBrFTOO7QXyXta5Hn6hECBnRi bQ9tNOF7PXPzY9cAcp2AIRhp8bSSw0Ch0eGF1TOhx8uiq0k4NdYGS42VR YI3mvruZeAvKfJo+/Tm09xjjgTHnK86cBcRRC3YKD/jMU4ZPGRIXxQqhL r+5imvxAUwhPkuFFp10RoXc3a4s3buOU4vxOjC1Qf+9zdjD6iFMQOwmgn w==; X-CSE-ConnectionGUID: Q9PyuTGHTkaVaW2ebrLOYQ== X-CSE-MsgGUID: LXfvwHTXRRyLxNE0zuQUqw== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893304" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893304" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:17 -0800 X-CSE-ConnectionGUID: i/5xb73fSjO5cfKb+R148Q== X-CSE-MsgGUID: x1r+jbNbQvSLOsZBAGsOEg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370269" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:14 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 7/9] KVM: TDX: Handle TDG.VP.VMCALL Date: Sat, 22 Feb 2025 09:42:23 +0800 Message-ID: <20250222014225.897298-8-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Convert TDG.VP.VMCALL to KVM_EXIT_SYSTEM_EVENT with a new type KVM_SYSTEM_EVENT_TDX_FATAL and forward it to userspace for handling. TD guest can use TDG.VP.VMCALL to report the fatal error it has experienced. This hypercall is special because TD guest is requesting a termination with the error information, KVM needs to forward the hypercall to userspace anyway, KVM doesn't do parsing or conversion, it just dumps the 16 general-purpose registers to userspace and let userspace decide what to do. Signed-off-by: Binbin Wu --- Hypercalls exit to userspace v3: - Dump all 16 general-purpose registers to userspace. (Sean) Hypercalls exit to userspace v2: - Use vp_enter_args instead of x86 registers. - vcpu->run->system_event.ndata is not hardcoded to 10. (Xiaoyao) - Undefine COPY_REG after use. (Yilun) - Updated the document about KVM_SYSTEM_EVENT_TDX_FATAL. (Chao) Hypercalls exit to userspace v1: - New added. Implement one of the hypercalls need to exit to userspace for handling after reverting "KVM: TDX: Add KVM Exit for TDX TDG.VP.VMCALL", which tries to resolve Sean's comment. https://lore.kernel.org/kvm/Zg18ul8Q4PGQMWam@google.com/ - Use TDVMCALL_STATUS prefix for TDX call status codes (Binbin) --- Documentation/virt/kvm/api.rst | 9 ++++++--- arch/x86/kvm/vmx/tdx.c | 28 ++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 35 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 2b52eb77e29c..2d75edc9db4f 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6823,6 +6823,7 @@ should put the acknowledged interrupt vector into the 'epr' field. #define KVM_SYSTEM_EVENT_WAKEUP 4 #define KVM_SYSTEM_EVENT_SUSPEND 5 #define KVM_SYSTEM_EVENT_SEV_TERM 6 + #define KVM_SYSTEM_EVENT_TDX_FATAL 7 __u32 type; __u32 ndata; __u64 data[16]; @@ -6849,9 +6850,11 @@ Valid values for 'type' are: reset/shutdown of the VM. - KVM_SYSTEM_EVENT_SEV_TERM -- an AMD SEV guest requested termination. The guest physical address of the guest's GHCB is stored in `data[0]`. - - KVM_SYSTEM_EVENT_WAKEUP -- the exiting vCPU is in a suspended state and - KVM has recognized a wakeup event. Userspace may honor this event by - marking the exiting vCPU as runnable, or deny it and call KVM_RUN again. + - KVM_SYSTEM_EVENT_TDX_FATAL -- a TDX guest reported a fatal error state. + KVM doesn't do any parsing or conversion, it just dumps 16 general-purpose + registers to userspace, in ascending order of the 4-bit indices for x86-64 + general-purpose registers in instruction encoding, as defined in the Intel + SDM. - KVM_SYSTEM_EVENT_SUSPEND -- the guest has requested a suspension of the VM. diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7b2f4c32f01b..7c8356299a25 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1093,11 +1093,39 @@ static int tdx_map_gpa(struct kvm_vcpu *vcpu) return 1; } +static int tdx_report_fatal_error(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + u64 *regs = vcpu->run->system_event.data; + u64 *module_regs = &tdx->vp_enter_args.r8; + int index = VCPU_REGS_RAX; + + vcpu->run->exit_reason = KVM_EXIT_SYSTEM_EVENT; + vcpu->run->system_event.type = KVM_SYSTEM_EVENT_TDX_FATAL; + vcpu->run->system_event.ndata = 16; + + /* Dump 16 general-purpose registers to userspace in ascending order. */ + regs[index++] = tdx->vp_enter_ret; + regs[index++] = tdx->vp_enter_args.rcx; + regs[index++] = tdx->vp_enter_args.rdx; + regs[index++] = tdx->vp_enter_args.rbx; + regs[index++] = 0; + regs[index++] = 0; + regs[index++] = tdx->vp_enter_args.rsi; + regs[index] = tdx->vp_enter_args.rdi; + for (index = 0; index < 8; index++) + regs[VCPU_REGS_R8 + index] = module_regs[index]; + + return 0; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { case TDVMCALL_MAP_GPA: return tdx_map_gpa(vcpu); + case TDVMCALL_REPORT_FATAL_ERROR: + return tdx_report_fatal_error(vcpu); default: break; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 45e6d8fca9b9..937400350317 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -375,6 +375,7 @@ struct kvm_run { #define KVM_SYSTEM_EVENT_WAKEUP 4 #define KVM_SYSTEM_EVENT_SUSPEND 5 #define KVM_SYSTEM_EVENT_SEV_TERM 6 +#define KVM_SYSTEM_EVENT_TDX_FATAL 7 __u32 type; __u32 ndata; union { From patchwork Sat Feb 22 01:42:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986484 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE16B2045B1; Sat, 22 Feb 2025 01:41:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188483; cv=none; b=tl+6NSrnI+ubeqhztZETeJaEdpxQo0GVugrpaJM+fpopK7YlbbI4aHMPM2iJe1MXiOxIgIG+HamY+1AscnzU5Smehjks+YdoQkkkdxH1jgmTJqmv+nVBKOfjnbpN4L59YIpn36SHrBzoMDVwQdYiXiv2lhqmmoL7jg5hIL0Px70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188483; c=relaxed/simple; bh=3AGn768TJORBzQ6jY1KUaQ4pguOi9rpUjt4CBaaIank=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q6iAg6ML6kFqPbfveAO52cwHrF+Bo5rIJ9FJcU1l/5zRT6KvIflUfc9LSQ5zucasod6KDNVAcoznougX69jxQ01Q779sGcCJVarCpEsVdr8TTU8d7H3XegMtSVfoTvoYLgClyLvIQPwlU/khDVq0xouOePsDJW71Dw0lTl3XSvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MGNTf/UP; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MGNTf/UP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188482; x=1771724482; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3AGn768TJORBzQ6jY1KUaQ4pguOi9rpUjt4CBaaIank=; b=MGNTf/UPwt64wxos6xKgQETlBrRIgHRuHSorW8RYlYNDfu/qmkpXs7g9 g9/NdIMo6eJsN3HyeaeESZP91xxYuNnwmOzpVo0+GTL1iPqo3bV+AWPi9 nRHVpqo3csSENo1t92YGUVPG5TjM20jKBAk2UvZSpjYFpKO5lJeUMdJmZ 6Dn+i0n1pt0PVUi6c6V0/0yViEHrBlWQdihrXKf3ebpPNT2p71Lek6hIV PO1Lcg6XPGwblAsDnw1tb/Fl+Ymwjklvnzi2InEplbZqdTcdv2kkOq0yO R9YWpDgfn21Dqqf8B9nVRrItr2wEe0R+K1atwWrzQBhdC7ub+16+5ClRu w==; X-CSE-ConnectionGUID: MQfPE/MYRQCOJSV0wSQMgg== X-CSE-MsgGUID: 4GmnKcxRQLC7rqEai3H4tg== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893307" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893307" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:21 -0800 X-CSE-ConnectionGUID: fShLCHfAQkOrtnH3M5HvHA== X-CSE-MsgGUID: E/YlF498SAKdKdGuosHT3A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370273" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:18 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 8/9] KVM: TDX: Handle TDX PV port I/O hypercall Date: Sat, 22 Feb 2025 09:42:24 +0800 Message-ID: <20250222014225.897298-9-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Emulate port I/O requested by TDX guest via TDVMCALL with leaf Instruction.IO (same value as EXIT_REASON_IO_INSTRUCTION) according to TDX Guest Host Communication Interface (GHCI). All port I/O instructions inside the TDX guest trigger the #VE exception. On #VE triggered by I/O instructions, TDX guest can call TDVMCALL with leaf Instruction.IO to request VMM to emulate I/O instructions. Similar to normal port I/O emulation, try to handle the port I/O in kernel first, if kernel can't support it, forward the request to userspace. Note string I/O operations are not supported in TDX. Guest should unroll them before calling the TDVMCALL. Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu Reviewed-by: Paolo Bonzini --- Hypercalls exit to userspace v3: - Rebased to use tdcall_to_vmx_exit_reason(). Hypercalls exit to userspace v2: - Morph PV port I/O hypercall to EXIT_REASON_IO_INSTRUCTION. (Sean) - Use vp_enter_args instead of x86 registers. - Check write is either 0 or 1. (Chao) - Skip setting return code as TDVMCALL_STATUS_SUCCESS. (Sean) Hypercalls exit to userspace v1: - Renamed from "KVM: TDX: Handle TDX PV port io hypercall" to "KVM: TDX: Handle TDX PV port I/O hypercall". - Update changelog. - Add missing curly brackets. - Move reset of pio.count to tdx_complete_pio_out() and remove the stale comment. (binbin) - Use TDVMCALL_STATUS prefix for TDX call status codes (Binbin) - Set status code to TDVMCALL_STATUS_SUCCESS when PIO is handled in kernel. - Don't write to R11 when it is a write operation for output. v18: - Fix out case to set R10 and R11 correctly when user space handled port out. --- arch/x86/kvm/vmx/tdx.c | 62 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7c8356299a25..3a4437520868 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -824,6 +824,8 @@ static bool tdx_guest_state_is_invalid(struct kvm_vcpu *vcpu) static __always_inline u32 tdcall_to_vmx_exit_reason(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { + case EXIT_REASON_IO_INSTRUCTION: + return tdvmcall_leaf(vcpu); default: break; } @@ -1119,6 +1121,64 @@ static int tdx_report_fatal_error(struct kvm_vcpu *vcpu) return 0; } +static int tdx_complete_pio_out(struct kvm_vcpu *vcpu) +{ + vcpu->arch.pio.count = 0; + return 1; +} + +static int tdx_complete_pio_in(struct kvm_vcpu *vcpu) +{ + struct x86_emulate_ctxt *ctxt = vcpu->arch.emulate_ctxt; + unsigned long val = 0; + int ret; + + ret = ctxt->ops->pio_in_emulated(ctxt, vcpu->arch.pio.size, + vcpu->arch.pio.port, &val, 1); + + WARN_ON_ONCE(!ret); + + tdvmcall_set_return_val(vcpu, val); + + return 1; +} + +static int tdx_emulate_io(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + struct x86_emulate_ctxt *ctxt = vcpu->arch.emulate_ctxt; + unsigned long val = 0; + unsigned int port; + u64 size, write; + int ret; + + ++vcpu->stat.io_exits; + + size = tdx->vp_enter_args.r12; + write = tdx->vp_enter_args.r13; + port = tdx->vp_enter_args.r14; + + if ((write != 0 && write != 1) || (size != 1 && size != 2 && size != 4)) { + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND); + return 1; + } + + if (write) { + val = tdx->vp_enter_args.r15; + ret = ctxt->ops->pio_out_emulated(ctxt, size, port, &val, 1); + } else { + ret = ctxt->ops->pio_in_emulated(ctxt, size, port, &val, 1); + } + + if (!ret) + vcpu->arch.complete_userspace_io = write ? tdx_complete_pio_out : + tdx_complete_pio_in; + else if (!write) + tdvmcall_set_return_val(vcpu, val); + + return ret; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { @@ -1496,6 +1556,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return handle_tdvmcall(vcpu); case EXIT_REASON_VMCALL: return tdx_emulate_vmcall(vcpu); + case EXIT_REASON_IO_INSTRUCTION: + return tdx_emulate_io(vcpu); default: break; } From patchwork Sat Feb 22 01:42:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Binbin Wu X-Patchwork-Id: 13986485 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 878DC204F7E; Sat, 22 Feb 2025 01:41:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188487; cv=none; b=FVs/9tA+5FU2rifp4HsFD53C77DabrzlHo8lt6gZFTfiiTa6Vvg5nPl/r3Ad/Bj4wnXWEHV3Fj8JyVlncJ8WRaKQvC70QSMNL/Y/ITFXVqtU51qwnT+DFWlvYuBy2nVIUl7WaFfZu5aOhSxnBnJ/vkiVGFVvK3WKIbvZahFKOvY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740188487; c=relaxed/simple; bh=2DEd476Bpk+oX75xhAgahoRfnU3DcwQcOYOgDaGwJS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YI3TtJoYjrHCOQhJ6JqJZlRdTSb7VS/FWnd5cBZhMeU4AdWoZnZO9ubjNcDfauR3x8npo1VIy6Fk/CWpG0ZmpCKydvZ+r/8DPeaPoCbe2/tWWYhnWy3IYqWe3V2a5ciyQ93lgBB50pqa+N6cGsu8IHa3BNkO1qd/APpg5QHEyog= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PrbjoOdZ; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PrbjoOdZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740188485; x=1771724485; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2DEd476Bpk+oX75xhAgahoRfnU3DcwQcOYOgDaGwJS8=; b=PrbjoOdZSGUfhT7gNt07pGGZdPVnMV5AD392M9GcgG4HNCvf8O+aEqXI jhtKdAezdX6CZ28IRD1j3SHzvk+ET4saZ01KFXTi8wuFwYdrKY71sD8/+ xDzu5qLfNwhT0RzY+sI+lrJOvjDb/LNJNTQ3Gh2ayLQgtXFMUajrWon9y t0pg6fO32JeQ8X+LRLzlm0Ue/8Fja1BEI6Rq4oBI0qolMZvuxQMKLmjL5 mDf8WxaZHb/C5+vk3t+qaO8djpT+UrUI77m4zWeGkooCgqmgqcQ6IAQJN MIU4PfVmLk7lxrNf2LKJ+0B9xAAxL9Fi/2fTh8e+f8RGZ+ukHruDTKrkK Q==; X-CSE-ConnectionGUID: bjx9645oSSSjNZkHaytM7w== X-CSE-MsgGUID: tttUJF1gSzKZvWh9TKqZsw== X-IronPort-AV: E=McAfee;i="6700,10204,11352"; a="40893312" X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="40893312" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:25 -0800 X-CSE-ConnectionGUID: XhRuXY38Q2W432JcNuKAQA== X-CSE-MsgGUID: 4L9cjS1yRaeZOcagpXJmLg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,306,1732608000"; d="scan'208";a="146370281" Received: from litbin-desktop.sh.intel.com ([10.239.156.93]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Feb 2025 17:41:22 -0800 From: Binbin Wu To: pbonzini@redhat.com, seanjc@google.com, kvm@vger.kernel.org Cc: rick.p.edgecombe@intel.com, kai.huang@intel.com, adrian.hunter@intel.com, reinette.chatre@intel.com, xiaoyao.li@intel.com, tony.lindgren@intel.com, isaku.yamahata@intel.com, yan.y.zhao@intel.com, chao.gao@intel.com, linux-kernel@vger.kernel.org, binbin.wu@linux.intel.com Subject: [PATCH v3 9/9] KVM: TDX: Handle TDX PV MMIO hypercall Date: Sat, 22 Feb 2025 09:42:25 +0800 Message-ID: <20250222014225.897298-10-binbin.wu@linux.intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20250222014225.897298-1-binbin.wu@linux.intel.com> References: <20250222014225.897298-1-binbin.wu@linux.intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Handle TDX PV MMIO hypercall when TDX guest calls TDVMCALL with the leaf #VE.RequestMMIO (same value as EXIT_REASON_EPT_VIOLATION) according to TDX Guest Host Communication Interface (GHCI) spec. For TDX guests, VMM is not allowed to access vCPU registers and the private memory, and the code instructions must be fetched from the private memory. So MMIO emulation implemented for non-TDX VMs is not possible for TDX guests. In TDX the MMIO regions are instead configured by VMM to trigger a #VE exception in the guest. The #VE handling is supposed to emulate the MMIO instruction inside the guest and convert it into a TDVMCALL with the leaf #VE.RequestMMIO, which equals to EXIT_REASON_EPT_VIOLATION. The requested MMIO address must be in shared GPA space. The shared bit is stripped after check because the existing code for MMIO emulation is not aware of the shared bit. The MMIO GPA shouldn't have a valid memslot, also the attribute of the GPA should be shared. KVM could do the checks before exiting to userspace, however, even if KVM does the check, there still will be race conditions between the check in KVM and the emulation of MMIO access in userspace due to a memslot hotplug, or a memory attribute conversion. If userspace doesn't check the attribute of the GPA and the attribute happens to be private, it will not pose a security risk or cause an MCE, but it can lead to another issue. E.g., in QEMU, treating a GPA with private attribute as shared when it falls within RAM's range can result in extra memory consumption during the emulation to the access to the HVA of the GPA. There are two options: 1) Do the check both in KVM and userspace. 2) Do the check only in QEMU. This patch chooses option 2, i.e. KVM omits the memslot and attribute checks, and expects userspace to do the checks. Similar to normal MMIO emulation, try to handle the MMIO in kernel first, if kernel can't support it, forward the request to userspace. Export needed symbols used for MMIO handling. Fragments handling is not needed for TDX PV MMIO because GPA is provided, if a MMIO access crosses page boundary, it should be continuous in GPA. Also, the size is limited to 1, 2, 4, 8 bytes. No further split needed. Allow cross page access because no extra handling needed after checking both start and end GPA are shared GPAs. Suggested-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Co-developed-by: Binbin Wu Signed-off-by: Binbin Wu Reviewed-by: Paolo Bonzini --- Hypercalls exit to userspace v3: - Rebased to use tdcall_to_vmx_exit_reason() Hypercalls exit to userspace v2: - Morph PV MMIO hypercall to EXIT_REASON_EPT_MISCONFIG. (Sean) - Skip setting return code as TDVMCALL_STATUS_SUCCESS, as a result, the complete_userspace_io() callback for write is not needed. (Sean) - Remove the code for reading/writing APIC mmio,since TDX guest supports x2APIC only. - Use vp_enter_args directly instead of helpers. Hypercalls exit to userspace v1: - Update the changelog. - Remove the check of memslot for GPA. - Allow MMIO access crossing page boundary. - Move the tracepoint for KVM_TRACE_MMIO_WRITE earlier so the tracepoint handles the cases both for kernel and userspace. (Isaku) - Set TDVMCALL return code when back from userspace, which is missing in v19. - Move fast MMIO write into tdx_mmio_write() - Check GPA is shared GPA. (Binbin) - Remove extra check for size > 8u. (Binbin) - Removed KVM_BUG_ON() in tdx_complete_mmio() and tdx_emulate_mmio() - Removed vcpu->mmio_needed code since it's not used after removing KVM_BUG_ON(). - Use TDVMCALL_STATUS prefix for TDX call status codes (Binbin) - Use vt_is_tdx_private_gpa() --- arch/x86/kvm/vmx/tdx.c | 105 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 1 + virt/kvm/kvm_main.c | 1 + 3 files changed, 107 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 3a4437520868..c023e99de294 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -826,6 +826,8 @@ static __always_inline u32 tdcall_to_vmx_exit_reason(struct kvm_vcpu *vcpu) switch (tdvmcall_leaf(vcpu)) { case EXIT_REASON_IO_INSTRUCTION: return tdvmcall_leaf(vcpu); + case EXIT_REASON_EPT_VIOLATION: + return EXIT_REASON_EPT_MISCONFIG; default: break; } @@ -1179,6 +1181,107 @@ static int tdx_emulate_io(struct kvm_vcpu *vcpu) return ret; } +static int tdx_complete_mmio_read(struct kvm_vcpu *vcpu) +{ + unsigned long val = 0; + gpa_t gpa; + int size; + + gpa = vcpu->mmio_fragments[0].gpa; + size = vcpu->mmio_fragments[0].len; + + memcpy(&val, vcpu->run->mmio.data, size); + tdvmcall_set_return_val(vcpu, val); + trace_kvm_mmio(KVM_TRACE_MMIO_READ, size, gpa, &val); + return 1; +} + +static inline int tdx_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, int size, + unsigned long val) +{ + if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) { + trace_kvm_fast_mmio(gpa); + return 0; + } + + trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, size, gpa, &val); + if (kvm_io_bus_write(vcpu, KVM_MMIO_BUS, gpa, size, &val)) + return -EOPNOTSUPP; + + return 0; +} + +static inline int tdx_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, int size) +{ + unsigned long val; + + if (kvm_io_bus_read(vcpu, KVM_MMIO_BUS, gpa, size, &val)) + return -EOPNOTSUPP; + + tdvmcall_set_return_val(vcpu, val); + trace_kvm_mmio(KVM_TRACE_MMIO_READ, size, gpa, &val); + return 0; +} + +static int tdx_emulate_mmio(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + int size, write, r; + unsigned long val; + gpa_t gpa; + + size = tdx->vp_enter_args.r12; + write = tdx->vp_enter_args.r13; + gpa = tdx->vp_enter_args.r14; + val = write ? tdx->vp_enter_args.r15 : 0; + + if (size != 1 && size != 2 && size != 4 && size != 8) + goto error; + if (write != 0 && write != 1) + goto error; + + /* + * TDG.VP.VMCALL allows only shared GPA, it makes no sense to + * do MMIO emulation for private GPA. + */ + if (vt_is_tdx_private_gpa(vcpu->kvm, gpa) || + vt_is_tdx_private_gpa(vcpu->kvm, gpa + size - 1)) + goto error; + + gpa = gpa & ~gfn_to_gpa(kvm_gfn_direct_bits(vcpu->kvm)); + + if (write) + r = tdx_mmio_write(vcpu, gpa, size, val); + else + r = tdx_mmio_read(vcpu, gpa, size); + if (!r) + /* Kernel completed device emulation. */ + return 1; + + /* Request the device emulation to userspace device model. */ + vcpu->mmio_is_write = write; + if (!write) + vcpu->arch.complete_userspace_io = tdx_complete_mmio_read; + + vcpu->run->mmio.phys_addr = gpa; + vcpu->run->mmio.len = size; + vcpu->run->mmio.is_write = write; + vcpu->run->exit_reason = KVM_EXIT_MMIO; + + if (write) { + memcpy(vcpu->run->mmio.data, &val, size); + } else { + vcpu->mmio_fragments[0].gpa = gpa; + vcpu->mmio_fragments[0].len = size; + trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, size, gpa, NULL); + } + return 0; + +error: + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND); + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { switch (tdvmcall_leaf(vcpu)) { @@ -1558,6 +1661,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return tdx_emulate_vmcall(vcpu); case EXIT_REASON_IO_INSTRUCTION: return tdx_emulate_io(vcpu); + case EXIT_REASON_EPT_MISCONFIG: + return tdx_emulate_mmio(vcpu); default: break; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 8877d6db9b84..7b5f5f603ceb 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -14012,6 +14012,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_mmio); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_page_fault); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3187b2ae0e5b..da5ef7773b05 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5843,6 +5843,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r = __kvm_io_bus_read(vcpu, bus, &range, val); return r < 0 ? r : 0; } +EXPORT_SYMBOL_GPL(kvm_io_bus_read); int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, int len, struct kvm_io_device *dev)