From patchwork Wed May 22 02:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670196 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1B1156444 for ; Wed, 22 May 2024 02:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344914; cv=none; b=r8pAlnhfitC8PXuaZ7IOs3yE52BxLzkSN8zKLJJezC39MuvzX/iup6nFvA+7SCBWqIlygmE737XyIxLKGqEbTRhDj8h+m0nT2xapoL1Gbo6eidQJh/Lu9rwn/ydHgTHJGKEbsXk4IdEVgfXpioGWVYG/nb/03gE71/22xNpwrn0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344914; c=relaxed/simple; bh=OEUt2h0mBOqIq2hy4jSpEIVHUPabTFaHVJ651yVSDWk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BRfcTgFqf8bvwvHguCVvNMKM4ZUtHb7/m3e6mN6dwZp6dnU8aNCyl+mX4Zo4uT+PbppmarXRzeIxmBjpFmoyUztyo+VBEN94nSYOqgbJG8xNwRqkxjugcZAc/JGuc6JzClLAQ1OBtzlcxw/lYQcS5Mt3OOgspE5eprQBJTpps0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QwRHV1hb; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QwRHV1hb" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-627e6fe0303so3148627b3.2 for ; Tue, 21 May 2024 19:28:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344912; x=1716949712; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=mfIvjgKueQKVKfXuZvg9684V73mkUfXcfkr0mjRx2Pg=; b=QwRHV1hbnAWmDpmQrrGNZTpBCgkBJn0d+rJ2cj3HwiFe8ADrKXNB+e7c0LEbc0v5OZ W0iWedD9rGEc4w/PuELJvtIHJqeLVHHCWc9z5gVvRhC5uE1STZ2WDgpoPOFX4UzNI5Vu vV5fdZsY+K7EdnW48p+yligYaOHa727279VwBFgHE+gY+M9gFRjd1wWwLme9ynJc4gTR 3DdAVur1ngEhul99dsvnTdv0X72KHHncneWG6sXzAOsyXT1VPHU2n1jDZSlrXIVEnmO6 D24KNdoqNioc1/LiWPHZsdB7NSCUCaWiFnKEe/fq3tELjB2EIAYggcNgsONzyQ7gyhtw g+gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344912; x=1716949712; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mfIvjgKueQKVKfXuZvg9684V73mkUfXcfkr0mjRx2Pg=; b=psc81vZCudLTnxL1QlBl0Hwt3BjNZ7R7EdzrVzCMaUp8Ni9Vyswn5mxxE6Eqws+yvt cMI/MTkI3vB2Ys4k5sMmvDNVcDdHiTPofqVkSssogW8SNFsfOtmZeSPvSSokJKDPJiZw FKHbVJblx1QdwM4vCO4nFuuEFu2KBKASbUzDxz/f+6ADVH1a7swEI98BXZIJE0sh+1MS GIi3KpzdEnmVu7kQq1r38/9yUqscYXbSvLlXtxfgl1EpP81ad1nojLYO5doBG3tlC0g3 bb3ZGe9zo+Rm0Qg2gTBAV0k75JZJkrzJFYxOpiE7D3Q/MlMQUbG9xNXQnQ1hRZcIdpPv 9y8Q== X-Gm-Message-State: AOJu0YwaIlZFU5xV9b4E+e5WP3gVNpuV+Yz0fA5H//9F9Fq/S+3H3QV6 E0z6IrQ/IjToNEAZzuPhZy+Gv8iOxtduSXSD2NubJRwo+24H7CaM0AU6B3vPRRO6Eq9KliyRsjW OXA== X-Google-Smtp-Source: AGHT+IEYKgn0GPmk4EO5+2RJPVYqTGONODYkJ+ratdiZJJv2DqyQAom8Xj6BnX0RZi+f9FusFpTZashCgWU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:d78e:0:b0:61b:7912:6cad with SMTP id 00721157ae682-627e46cfe0emr2133237b3.2.1716344911878; Tue, 21 May 2024 19:28:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:22 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-2-seanjc@google.com> Subject: [PATCH v2 1/6] KVM: Register cpuhp and syscore callbacks when enabling hardware From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Register KVM's cpuhp and syscore callback when enabling virtualization in hardware instead of registering the callbacks during initialization, and let the CPU up/down framework invoke the inner enable/disable functions. Registering the callbacks during initialization makes things more complex than they need to be, as KVM needs to be very careful about handling races between enabling CPUs being onlined/offlined and hardware being enabled/disabled. Intel TDX support will require KVM to enable virtualization during KVM initialization, i.e. will add another wrinkle to things, at which point sorting out the potential races with kvm_usage_count would become even more complex. Use a dedicated mutex to guard kvm_usage_count, as taking kvm_lock outside cpu_hotplug_lock is disallowed. Ideally, KVM would *always* take kvm_lock outside cpu_hotplug_lock, but KVM x86 takes kvm_lock in several notifiers that may be called under cpus_read_lock(). kvmclock_cpufreq_notifier() in particular has callchains that are infeasible to guarantee will never be called with cpu_hotplug_lock held. And practically speaking, using a dedicated mutex is a non-issue as the cost is a few bytes for all of KVM. Note, using the cpuhp framework has a subtle behavioral change: enabling will be done serially across all CPUs, whereas KVM currently sends an IPI to all CPUs in parallel. While serializing virtualization enabling could create undesirable latency, the issue is limited to creation of KVM's first VM, and even that can be mitigated, e.g. by letting userspace force virtualization to be enabled when KVM is initialized. Cc: Chao Gao Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 189 ++++++++++++++++---------------------------- 1 file changed, 69 insertions(+), 120 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a1756d5077ee..97783d6987e9 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5499,9 +5499,10 @@ __visible bool kvm_rebooting; EXPORT_SYMBOL_GPL(kvm_rebooting); static DEFINE_PER_CPU(bool, hardware_enabled); +static DEFINE_MUTEX(kvm_usage_lock); static int kvm_usage_count; -static int __hardware_enable_nolock(void) +static int hardware_enable_nolock(void) { if (__this_cpu_read(hardware_enabled)) return 0; @@ -5516,34 +5517,18 @@ static int __hardware_enable_nolock(void) return 0; } -static void hardware_enable_nolock(void *failed) -{ - if (__hardware_enable_nolock()) - atomic_inc(failed); -} - static int kvm_online_cpu(unsigned int cpu) { - int ret = 0; - /* * Abort the CPU online process if hardware virtualization cannot * be enabled. Otherwise running VMs would encounter unrecoverable * errors when scheduled to this CPU. */ - mutex_lock(&kvm_lock); - if (kvm_usage_count) - ret = __hardware_enable_nolock(); - mutex_unlock(&kvm_lock); - return ret; + return hardware_enable_nolock(); } static void hardware_disable_nolock(void *junk) { - /* - * Note, hardware_disable_all_nolock() tells all online CPUs to disable - * hardware, not just CPUs that successfully enabled hardware! - */ if (!__this_cpu_read(hardware_enabled)) return; @@ -5554,78 +5539,10 @@ static void hardware_disable_nolock(void *junk) static int kvm_offline_cpu(unsigned int cpu) { - mutex_lock(&kvm_lock); - if (kvm_usage_count) - hardware_disable_nolock(NULL); - mutex_unlock(&kvm_lock); + hardware_disable_nolock(NULL); return 0; } -static void hardware_disable_all_nolock(void) -{ - BUG_ON(!kvm_usage_count); - - kvm_usage_count--; - if (!kvm_usage_count) - on_each_cpu(hardware_disable_nolock, NULL, 1); -} - -static void hardware_disable_all(void) -{ - cpus_read_lock(); - mutex_lock(&kvm_lock); - hardware_disable_all_nolock(); - mutex_unlock(&kvm_lock); - cpus_read_unlock(); -} - -static int hardware_enable_all(void) -{ - atomic_t failed = ATOMIC_INIT(0); - int r; - - /* - * Do not enable hardware virtualization if the system is going down. - * If userspace initiated a forced reboot, e.g. reboot -f, then it's - * possible for an in-flight KVM_CREATE_VM to trigger hardware enabling - * after kvm_reboot() is called. Note, this relies on system_state - * being set _before_ kvm_reboot(), which is why KVM uses a syscore ops - * hook instead of registering a dedicated reboot notifier (the latter - * runs before system_state is updated). - */ - if (system_state == SYSTEM_HALT || system_state == SYSTEM_POWER_OFF || - system_state == SYSTEM_RESTART) - return -EBUSY; - - /* - * When onlining a CPU, cpu_online_mask is set before kvm_online_cpu() - * is called, and so on_each_cpu() between them includes the CPU that - * is being onlined. As a result, hardware_enable_nolock() may get - * invoked before kvm_online_cpu(), which also enables hardware if the - * usage count is non-zero. Disable CPU hotplug to avoid attempting to - * enable hardware multiple times. - */ - cpus_read_lock(); - mutex_lock(&kvm_lock); - - r = 0; - - kvm_usage_count++; - if (kvm_usage_count == 1) { - on_each_cpu(hardware_enable_nolock, &failed, 1); - - if (atomic_read(&failed)) { - hardware_disable_all_nolock(); - r = -EBUSY; - } - } - - mutex_unlock(&kvm_lock); - cpus_read_unlock(); - - return r; -} - static void kvm_shutdown(void) { /* @@ -5648,27 +5565,25 @@ static int kvm_suspend(void) { /* * Secondary CPUs and CPU hotplug are disabled across the suspend/resume - * callbacks, i.e. no need to acquire kvm_lock to ensure the usage count - * is stable. Assert that kvm_lock is not held to ensure the system - * isn't suspended while KVM is enabling hardware. Hardware enabling - * can be preempted, but the task cannot be frozen until it has dropped - * all locks (userspace tasks are frozen via a fake signal). + * callbacks, i.e. no need to acquire kvm_usage_lock to ensure the usage + * count is stable. Assert that kvm_usage_lock is not held to ensure + * the system isn't suspended while KVM is enabling hardware. Hardware + * enabling can be preempted, but the task cannot be frozen until it has + * dropped all locks (userspace tasks are frozen via a fake signal). */ - lockdep_assert_not_held(&kvm_lock); + lockdep_assert_not_held(&kvm_usage_lock); lockdep_assert_irqs_disabled(); - if (kvm_usage_count) - hardware_disable_nolock(NULL); + hardware_disable_nolock(NULL); return 0; } static void kvm_resume(void) { - lockdep_assert_not_held(&kvm_lock); + lockdep_assert_not_held(&kvm_usage_lock); lockdep_assert_irqs_disabled(); - if (kvm_usage_count) - WARN_ON_ONCE(__hardware_enable_nolock()); + WARN_ON_ONCE(hardware_enable_nolock()); } static struct syscore_ops kvm_syscore_ops = { @@ -5676,6 +5591,60 @@ static struct syscore_ops kvm_syscore_ops = { .resume = kvm_resume, .shutdown = kvm_shutdown, }; + +static int hardware_enable_all(void) +{ + int r; + + guard(mutex)(&kvm_usage_lock); + + if (kvm_usage_count++) + return 0; + + r = cpuhp_setup_state(CPUHP_AP_KVM_ONLINE, "kvm/cpu:online", + kvm_online_cpu, kvm_offline_cpu); + if (r) + goto err_cpuhp; + + register_syscore_ops(&kvm_syscore_ops); + + /* + * Undo virtualization enabling and bail if the system is going down. + * If userspace initiated a forced reboot, e.g. reboot -f, then it's + * possible for an in-flight operation to enable virtualization after + * syscore_shutdown() is called, i.e. without kvm_shutdown() being + * invoked. Note, this relies on system_state being set _before_ + * kvm_shutdown(), e.g. to ensure either kvm_shutdown() is invoked + * or this CPU observes the impending shutdown. Which is why KVM uses + * a syscore ops hook instead of registering a dedicated reboot + * notifier (the latter runs before system_state is updated). + */ + if (system_state == SYSTEM_HALT || system_state == SYSTEM_POWER_OFF || + system_state == SYSTEM_RESTART) { + r = -EBUSY; + goto err_rebooting; + } + + return 0; + +err_rebooting: + unregister_syscore_ops(&kvm_syscore_ops); + cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); +err_cpuhp: + --kvm_usage_count; + return r; +} + +static void hardware_disable_all(void) +{ + guard(mutex)(&kvm_usage_lock); + + if (--kvm_usage_count) + return; + + unregister_syscore_ops(&kvm_syscore_ops); + cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); +} #else /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */ static int hardware_enable_all(void) { @@ -6381,15 +6350,6 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module) int r; int cpu; -#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING - r = cpuhp_setup_state_nocalls(CPUHP_AP_KVM_ONLINE, "kvm/cpu:online", - kvm_online_cpu, kvm_offline_cpu); - if (r) - return r; - - register_syscore_ops(&kvm_syscore_ops); -#endif - /* A kmem cache lets us meet the alignment requirements of fx_save. */ if (!vcpu_align) vcpu_align = __alignof__(struct kvm_vcpu); @@ -6400,10 +6360,8 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module) offsetofend(struct kvm_vcpu, stats_id) - offsetof(struct kvm_vcpu, arch), NULL); - if (!kvm_vcpu_cache) { - r = -ENOMEM; - goto err_vcpu_cache; - } + if (!kvm_vcpu_cache) + return -ENOMEM; for_each_possible_cpu(cpu) { if (!alloc_cpumask_var_node(&per_cpu(cpu_kick_mask, cpu), @@ -6460,11 +6418,6 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module) for_each_possible_cpu(cpu) free_cpumask_var(per_cpu(cpu_kick_mask, cpu)); kmem_cache_destroy(kvm_vcpu_cache); -err_vcpu_cache: -#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING - unregister_syscore_ops(&kvm_syscore_ops); - cpuhp_remove_state_nocalls(CPUHP_AP_KVM_ONLINE); -#endif return r; } EXPORT_SYMBOL_GPL(kvm_init); @@ -6486,10 +6439,6 @@ void kvm_exit(void) kmem_cache_destroy(kvm_vcpu_cache); kvm_vfio_ops_exit(); kvm_async_pf_deinit(); -#ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING - unregister_syscore_ops(&kvm_syscore_ops); - cpuhp_remove_state_nocalls(CPUHP_AP_KVM_ONLINE); -#endif kvm_irqfd_exit(); } EXPORT_SYMBOL_GPL(kvm_exit); From patchwork Wed May 22 02:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670197 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A65856A01E for ; Wed, 22 May 2024 02:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344916; cv=none; b=bXXhhQfuv9uZ5Dgi/dD2eCBczHWwN0BYnM/3J/PxO295pYxu3OV/KIpuT8ueVqfAWiCoUfuQQw8TtHlSLzN9GfK84xSVIKBZyUAiKnrlGY+vCzXFutQBRIFOttzQZk0VU88iBhtJvB/75kCNc0fAy7ojk9x+KPNCui0KaqGsNOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344916; c=relaxed/simple; bh=GC+UWTGloN0JKxcGq9mam/rQZ0km64cIv4PDPR1uo+A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uuo3GNmegJhXn/HNerllDQ9/SdKn+jec4QEtQyyIwdqVp8ywkg0QCqKpRwmfzJWJrkCkEGbPIRLavl7LNr4SvdxWY8LTCkX2SlhcHnD0Uyd3RSRkEObL7kCKTGkYMfezniP1t5EeBHsXNvzhPazHC4QDeWPft7NJpHeinqOuXKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Poqc/1zn; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Poqc/1zn" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61be1fcf9abso196085307b3.1 for ; Tue, 21 May 2024 19:28:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344913; x=1716949713; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wkvURyajJ4AL9oNmd9DQ4HvztWk8u7XSsmW3pg5AGnY=; b=Poqc/1znnNeXjbLuqTZTRNZaHQY//NcqfDGV/z8D1E36CGeMx/7pdMliwcRJ6kvKjq XXD8RlTYPiGAFlmKgW0sAMCYqEHkN0evrmWXAvRZxmRO3JBsGF/8vZvgQfyDI8USMJEY YG+Q15ql8z4czenswE5X7cegE8zNHwyNmuq9p/TduwdGLhopuEuu9ZvdkKbezTKpUAoE BraNDLJFhB5Imfr1afEfboVdiuAbAPfL14J8327Z5Y65Ma3S8Oly/OOeuiA61CunQbk1 H6JR1/575D2LA0zJ3pnXRYHk/aEUOCnR8hXSCfiD/DkJrYbHcF1InXujE1qZyR8S00mV 55ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344913; x=1716949713; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wkvURyajJ4AL9oNmd9DQ4HvztWk8u7XSsmW3pg5AGnY=; b=l9A8rsiA4xDS17Vgt7TJBX4+kCL/TWWJ2/0jp5B6LK3PcgzZTTiBgpg5H0xXfj8fZm oohue3Yq/cPL6n6+3CSUeruF/ZjXB+juK96rN/mqo8R+LlaPBa+jCEsrb6Dr/zPR4zQE GQo73IwKzSz+dsmSn81vcdB4ZX5VKSKEH+MmgNkhbyAyr5ecjBQ1x8lnXkMKT68IPQbp 0M86QK8ZtQgQiAAPj6h82N0+7JIxbzOAZSqn72K2uXsuGKHJfIQY71gwnXtNdu9BycLi qM/VyRClWrD+N6HLYco7XqSNtSONUtywYKLQAdubNPER3dpvRDDq0ZDMTa6f7Q3Z5O07 GGZw== X-Gm-Message-State: AOJu0YxWMxexM9cJKfgm8BnEFx/Gdh30ChNVx+1DQHmNL3ckobrZuJko yw2Segx/6UhxJ627H9TBmVMmVUUK60FWCU5W8F8YAkX1EKqcKVxDDH2maCPmOi3/qRVPamy6JNg 9iA== X-Google-Smtp-Source: AGHT+IE/M+gvR3qTPtUEzY/NuesrQrEgXujVA/JmkMkK46KNhqMBG88SboKXrNUqByP0iGpLt5/XAB15gLM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a0d:d5d1:0:b0:627:c0ac:a9e7 with SMTP id 00721157ae682-627e48733ffmr2436997b3.5.1716344913718; Tue, 21 May 2024 19:28:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:23 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-3-seanjc@google.com> Subject: [PATCH v2 2/6] KVM: Rename functions related to enabling virtualization hardware From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Rename the various functions that enable virtualization to prepare for upcoming changes, and to clean up artifacts of KVM's previous behavior, which required manually juggling locks around kvm_usage_count. Drop the "nolock" qualifier from per-CPU functions now that there are no "nolock" implementations of the "all" variants, i.e. now that calling a non-nolock function from a nolock function isn't confusing (unlike this sentence). Drop "all" from the outer helpers as they no longer manually iterate over all CPUs, and because it might not be obvious what "all" refers to. Instead, use double-underscores to communicate that the per-CPU functions are helpers to the outer APIs. Opportunistically prepend "kvm" to all functions to help make it clear that they are KVM helpers, but mostly there's no reason not to. Lastly, use "virtualization" instead of "hardware", because while the functions do enable virtualization in hardware, there are a _lot_ of things that KVM enables in hardware. Signed-off-by: Sean Christopherson Reviewed-by: Chao Gao Reviewed-by: Kai Huang --- virt/kvm/kvm_main.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 97783d6987e9..8ba2861e7788 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -139,8 +139,8 @@ static int kvm_no_compat_open(struct inode *inode, struct file *file) #define KVM_COMPAT(c) .compat_ioctl = kvm_no_compat_ioctl, \ .open = kvm_no_compat_open #endif -static int hardware_enable_all(void); -static void hardware_disable_all(void); +static int kvm_enable_virtualization(void); +static void kvm_disable_virtualization(void); static void kvm_io_bus_destroy(struct kvm_io_bus *bus); @@ -1216,7 +1216,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname) if (r) goto out_err_no_arch_destroy_vm; - r = hardware_enable_all(); + r = kvm_enable_virtualization(); if (r) goto out_err_no_disable; @@ -1259,7 +1259,7 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname) mmu_notifier_unregister(&kvm->mmu_notifier, current->mm); #endif out_err_no_mmu_notifier: - hardware_disable_all(); + kvm_disable_virtualization(); out_err_no_disable: kvm_arch_destroy_vm(kvm); out_err_no_arch_destroy_vm: @@ -1354,7 +1354,7 @@ static void kvm_destroy_vm(struct kvm *kvm) #endif kvm_arch_free_vm(kvm); preempt_notifier_dec(); - hardware_disable_all(); + kvm_disable_virtualization(); mmdrop(mm); } @@ -5502,7 +5502,7 @@ static DEFINE_PER_CPU(bool, hardware_enabled); static DEFINE_MUTEX(kvm_usage_lock); static int kvm_usage_count; -static int hardware_enable_nolock(void) +static int __kvm_enable_virtualization(void) { if (__this_cpu_read(hardware_enabled)) return 0; @@ -5524,10 +5524,10 @@ static int kvm_online_cpu(unsigned int cpu) * be enabled. Otherwise running VMs would encounter unrecoverable * errors when scheduled to this CPU. */ - return hardware_enable_nolock(); + return __kvm_enable_virtualization(); } -static void hardware_disable_nolock(void *junk) +static void __kvm_disable_virtualization(void *ign) { if (!__this_cpu_read(hardware_enabled)) return; @@ -5539,7 +5539,7 @@ static void hardware_disable_nolock(void *junk) static int kvm_offline_cpu(unsigned int cpu) { - hardware_disable_nolock(NULL); + __kvm_disable_virtualization(NULL); return 0; } @@ -5558,7 +5558,7 @@ static void kvm_shutdown(void) */ pr_info("kvm: exiting hardware virtualization\n"); kvm_rebooting = true; - on_each_cpu(hardware_disable_nolock, NULL, 1); + on_each_cpu(__kvm_disable_virtualization, NULL, 1); } static int kvm_suspend(void) @@ -5574,7 +5574,7 @@ static int kvm_suspend(void) lockdep_assert_not_held(&kvm_usage_lock); lockdep_assert_irqs_disabled(); - hardware_disable_nolock(NULL); + __kvm_disable_virtualization(NULL); return 0; } @@ -5583,7 +5583,7 @@ static void kvm_resume(void) lockdep_assert_not_held(&kvm_usage_lock); lockdep_assert_irqs_disabled(); - WARN_ON_ONCE(hardware_enable_nolock()); + WARN_ON_ONCE(__kvm_enable_virtualization()); } static struct syscore_ops kvm_syscore_ops = { @@ -5592,7 +5592,7 @@ static struct syscore_ops kvm_syscore_ops = { .shutdown = kvm_shutdown, }; -static int hardware_enable_all(void) +static int kvm_enable_virtualization(void) { int r; @@ -5635,7 +5635,7 @@ static int hardware_enable_all(void) return r; } -static void hardware_disable_all(void) +static void kvm_disable_virtualization(void) { guard(mutex)(&kvm_usage_lock); @@ -5646,12 +5646,12 @@ static void hardware_disable_all(void) cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); } #else /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */ -static int hardware_enable_all(void) +static int kvm_enable_virtualization(void) { return 0; } -static void hardware_disable_all(void) +static void kvm_disable_virtualization(void) { } From patchwork Wed May 22 02:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670198 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1245756444 for ; Wed, 22 May 2024 02:28:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344917; cv=none; b=fYs1sc/4AKz+RbPI3x5w+enixTXnpM7VywcLO0g7lR9gqlRFkDBiYba0zSd68mFy67z+N+3xlAwuHmlCqczHCNrZDC/PIqMVgPyVfOAkGZN3CFtgoHZJRPAyLt9qglG91kTws9lRL0aKeDjDhCNg8BH9O2O3WN00sic1d9wKlNk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344917; c=relaxed/simple; bh=UxQT87o44GpQPdmmQ9RElOqqQyGOJ8CmTiYc4Jwt3vQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=L/wvtheGkXMIE3iKPrZP63qop/Ov0Ac1z0wX77BF49mUnENnQDXHcobLaTmkEAdvRbccVuF5FMVv0VfYWmIVvKXZ+q++I6w4Pnl+w+4fYH+3ma2M/T1ZH6WtKtFBp1OZ+ZcEqdG18hIQQSK+FPzBuzs2uDGSdRigjlPonzAjz+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kBtaFWYS; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kBtaFWYS" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ba0fd5142dso348949a91.1 for ; Tue, 21 May 2024 19:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344915; x=1716949715; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=l54COnjdm6bBuuKu60Lhz71aEXp/tIUjbNfI6k631P8=; b=kBtaFWYS8RucusTik4JurLeY8XShRomSc8eoNC1fD840mH+Xhv8eIuI0QeZdd4OIfv ClPAvFrITdUlVMunJxDd75SgZkHfGHJjyHXQcs5zSyyvsbCbMn678xU0mCkG+ze1j/yB DwvftSUOOhvRBKXapWb0UKG2Ol0VfT1xabVKnd/GjJFff5mOjnoTntOJPe/oauS6JM5f sD1mUZp7vaUoVJfNdAf2RXCQPe4r9aUkUQdunjZZnVTwa9uJ+WavuZpBHAh/MsiV7kce R/vDi2d3wrbJMXocxYe6k6KkWif5FyZl9kByrXrrDS4GCCCV7MAnM/JfW2PMMaFHA1tF 0k0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344915; x=1716949715; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=l54COnjdm6bBuuKu60Lhz71aEXp/tIUjbNfI6k631P8=; b=osiWDHvMHNxk83P/TotN/0Jl/GeS/MVNwfW2JSL50CIAbEMT4gfaCKEAqgxDdhlFHE rmD7KbReRJ4x3sNfkaZ1ST/G/L92bJYUSIirFv2F3PxCPqux/vKa1riLlzTCgtrifOzT LPS0momhyEAx1uRayfgQkPVzJQe/odKAWYbt2/vBNRJxWN4NsDcgyjlecnhejpWINAew j+AG869mWr2uDxOd3QOr+Cb9Okg93UFcubEubdslXMUm2vQ6yAd1/xENmFR92g+5+GU0 l8+oW+GgxZKNthNkaUWX0VASf0zH9PEx4ITbKhkwsOVl2AElRbmBCf0xn2WMwdlaOqkW 3FTQ== X-Gm-Message-State: AOJu0Ywayv64Oe/+TuAUqIbvOtgVZs6hEyUlgBJ6Nb7r35XVQVfoIvDp qdkZTpJM83wmmEGk4wDf+B4ciy8m/QEEyzPYNNORvcgerSNZcymAKkiELcKBTCZGmrmuHUJBRc3 YNQ== X-Google-Smtp-Source: AGHT+IEggpUCE1TXWoxuVSp1D45sXZttcemhdyFN0WeD/tg/jKWqCjQCD1n159cl7D3wuZPwQTJigyvwb+s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:1903:b0:2b6:4220:159d with SMTP id 98e67ed59e1d1-2bd9f5db1a2mr11666a91.3.1716344915357; Tue, 21 May 2024 19:28:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:24 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-4-seanjc@google.com> Subject: [PATCH v2 3/6] KVM: Add a module param to allow enabling virtualization when KVM is loaded From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Add an off-by-default module param, enable_virt_at_load, to let userspace force virtualization to be enabled in hardware when KVM is initialized, i.e. just before /dev/kvm is exposed to userspace. Enabling virtualization during KVM initialization allows userspace to avoid the additional latency when creating/destroying the first/last VM. Now that KVM uses the cpuhp framework to do per-CPU enabling, the latency could be non-trivial as the cpuhup bringup/teardown is serialized across CPUs, e.g. the latency could be problematic for use case that need to spin up VMs quickly. Enabling virtualizaton during initialization will also allow KVM to setup the Intel TDX Module, which requires VMX to be fully enabled, without needing additional APIs to temporarily enable virtualization. Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8ba2861e7788..8d331ab3aef7 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5495,6 +5495,9 @@ static struct miscdevice kvm_dev = { }; #ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING +static bool enable_virt_at_load; +module_param(enable_virt_at_load, bool, 0444); + __visible bool kvm_rebooting; EXPORT_SYMBOL_GPL(kvm_rebooting); @@ -5645,15 +5648,41 @@ static void kvm_disable_virtualization(void) unregister_syscore_ops(&kvm_syscore_ops); cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); } + +static int kvm_init_virtualization(void) +{ + if (enable_virt_at_load) + return kvm_enable_virtualization(); + + return 0; +} + +static void kvm_uninit_virtualization(void) +{ + if (enable_virt_at_load) + kvm_disable_virtualization(); + + WARN_ON(kvm_usage_count); +} #else /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */ static int kvm_enable_virtualization(void) { return 0; } +static int kvm_init_virtualization(void) +{ + return 0; +} + static void kvm_disable_virtualization(void) { +} + +static void kvm_uninit_virtualization(void) +{ + } #endif /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */ @@ -6395,6 +6424,10 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module) kvm_gmem_init(module); + r = kvm_init_virtualization(); + if (r) + goto err_virt; + /* * Registration _must_ be the very last thing done, as this exposes * /dev/kvm to userspace, i.e. all infrastructure must be setup! @@ -6408,6 +6441,8 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module) return 0; err_register: + kvm_uninit_virtualization(); +err_virt: kvm_vfio_ops_exit(); err_vfio: kvm_async_pf_deinit(); @@ -6433,6 +6468,8 @@ void kvm_exit(void) */ misc_deregister(&kvm_dev); + kvm_uninit_virtualization(); + debugfs_remove_recursive(kvm_debugfs_dir); for_each_possible_cpu(cpu) free_cpumask_var(per_cpu(cpu_kick_mask, cpu)); From patchwork Wed May 22 02:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670199 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 238AD77620 for ; Wed, 22 May 2024 02:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344919; cv=none; b=H+c5X4v4SN9a0aZBf/MAJbV6L+woUxQ2K1aATxnn3z2K8QD6cixEKFWdnQQKz+BbuBYIl63G9Ws3jwL16aIYQoHi9L2L9nrDux9olybfcAQdMj7FqLqvJIf+hbbJtOOhk1Bg9MHOxdxbcm0WDA5Q29PVcV7LHuNUQ3v/dlAB9hM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344919; c=relaxed/simple; bh=KseKQYsb65izy3p8V2zuq91stQnrQDY996NaeIaADEw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IL9wjhHq9rMmgCIFimqmGEYpyiD9L9fs3EBYESPQYIcPJoe1jWdEvHBY3qQXIT+k+0TXJnAouzBN2uvVHVxZI3M2kJsyWGsynsCPRp0oop2rmlkb6OiLuzGYHVZHDOiwf6Zms01DB2mm3tEjvZRmoXtUtiQs7ReX+/yE1euo2Yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZDMlOaiO; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZDMlOaiO" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61510f72bb3so258580537b3.0 for ; Tue, 21 May 2024 19:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344917; x=1716949717; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=UTESscmE2xSGNRqbvGygf0O1f1TGKn697WGvKKO3Gn4=; b=ZDMlOaiOr5AEy584EEzaoBwvO0dJKfmGacKAtIK7k3cdRPqHbLjajoUKGzS0ngI+Wk /Zo2jxdfxxnj67Dngk4azuu4ePxsxWH5iP7GaBDZq94Aia5yYFwenEsLCVCRzO1bE/Bg ApPNqAut0+FRd5QQhVeYtUn3+rI3SDiN6T14QJ66k5Q9VXz3r4fA11/5LPtXG35SEmPH 962yOJsmKRk7LMAvXL/1KtF9FIjB1XVwCS6OvOu+BV4MzHqiZ5S7syKMbynJpi1spNOF fxxvQn+wgslriGNcDyN8yKIuhlyQ+6bcdX6I50Fn12ST4nZhhZhmMrzrPSmsZCFD8iQh +DOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344917; x=1716949717; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UTESscmE2xSGNRqbvGygf0O1f1TGKn697WGvKKO3Gn4=; b=YqrXalIHQIO7FXT50f5FB2j5XLb3kwS3cTshS3ho1bcI3XA9Kmj3p1w8FJund4p0Sx yIJAJtMzfqfrgptfRli5h9Oo8aLRefBg5KVXbme0c9Tv+kIctmeEQJ+9B9yE8SHz5KJX b8hXcDFYo83oxBQf3PT+ouI+UakdPhy1IHe19K9cw6aiZL0zZFoNJfD5igPpp0HCXo0B Ouwc26uW5f3qF63scWIDBIdv6qp2R5/MuERpTChLZUSYIdelH64aUfSHdwFDji0LZOCK 091vycl4Xh4XC+qvk7sJFXbZsaK+Ryyr0AEEewDbwRAqLn3/bqNnDUv6hvwxucz9HNRv pfcA== X-Gm-Message-State: AOJu0YwbDmiFa4tnRYsVjaW86POzfExJKWrMwtm8dYWtEdy/Hwokipl/ mVOt5P7GpIvAWTP1HRmo0N+zBunz3Tvrdg3WT4xf7LWF5CWqx6tQ4SBXhTGraoFA8z/EuKRg99y yag== X-Google-Smtp-Source: AGHT+IFj3m2fqiWDNTeWfp9mIyuoi+I7yJ0zJE2+dXnbjZcTREYleLX/EXiF+L/nTpNHlZFB0o0jcqdFrk0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:690c:6612:b0:61b:e2e7:e127 with SMTP id 00721157ae682-627e46855bamr2380847b3.1.1716344917175; Tue, 21 May 2024 19:28:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:25 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-5-seanjc@google.com> Subject: [PATCH v2 4/6] KVM: Add arch hooks for enabling/disabling virtualization From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Add arch hooks that are invoked when KVM enables/disable virtualization. x86 will use the hooks to register an "emergency disable" callback, which is essentially an x86-specific shutdown notifier that is used when the kernel is doing an emergency reboot/shutdown/kexec. Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang Reviewed-by: Chao Gao --- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 7b57878c8c18..701b16290932 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1514,6 +1514,8 @@ static inline void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) {} #endif #ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING +void kvm_arch_enable_virtualization(void); +void kvm_arch_disable_virtualization(void); int kvm_arch_hardware_enable(void); void kvm_arch_hardware_disable(void); #endif diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8d331ab3aef7..16adeb6b5bef 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5505,6 +5505,16 @@ static DEFINE_PER_CPU(bool, hardware_enabled); static DEFINE_MUTEX(kvm_usage_lock); static int kvm_usage_count; +__weak void kvm_arch_enable_virtualization(void) +{ + +} + +__weak void kvm_arch_disable_virtualization(void) +{ + +} + static int __kvm_enable_virtualization(void) { if (__this_cpu_read(hardware_enabled)) @@ -5604,6 +5614,8 @@ static int kvm_enable_virtualization(void) if (kvm_usage_count++) return 0; + kvm_arch_enable_virtualization(); + r = cpuhp_setup_state(CPUHP_AP_KVM_ONLINE, "kvm/cpu:online", kvm_online_cpu, kvm_offline_cpu); if (r) @@ -5634,6 +5646,7 @@ static int kvm_enable_virtualization(void) unregister_syscore_ops(&kvm_syscore_ops); cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); err_cpuhp: + kvm_arch_disable_virtualization(); --kvm_usage_count; return r; } @@ -5647,6 +5660,7 @@ static void kvm_disable_virtualization(void) unregister_syscore_ops(&kvm_syscore_ops); cpuhp_remove_state(CPUHP_AP_KVM_ONLINE); + kvm_arch_disable_virtualization(); } static int kvm_init_virtualization(void) From patchwork Wed May 22 02:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670200 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B929378C87 for ; Wed, 22 May 2024 02:28:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344921; cv=none; b=OIAciR80SAGWNWYd7/9/ERDvJ4kSR+B0Ca2rjHME5H2tZjiO/ShqqvGR/HNR8KHJcX/jKaMxUhLxE99uwEugfTSt8rs66UE5mLaURj8tw2L32IZKR5jxnCAtiiXRDOEz9WyS+aPBjrSbSjF77m5BHZyQ6nIRGNEThcDwvIiV6r0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344921; c=relaxed/simple; bh=anPM5N+OHWuLglN6BHbuA0ZyXvZryJxlwQUZs1Mzw2k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=C6H565hl5W8Pqc90oIOwpVrrfa6V5xPJEedi/Xzv7+oqVXEVTcXQzrIWzvZFgMHM4vH5KZxoFElqY0dIR5gJ2wguZOy2UKQK8cm4bzlllA6yRcwuysuJ3l0w5K2tjAw03HvrLeHTCcwORAnJqiWKZLJ7mnzHv2XdyzvLfnb+skc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1p1pLXDr; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1p1pLXDr" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2b364c4c4b0so13253905a91.1 for ; Tue, 21 May 2024 19:28:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344919; x=1716949719; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=w+U7wzbNqJl9ii7+XfVEw3LB69135VJ3H5IvnuX0LIo=; b=1p1pLXDrHSnLRUSnh0ne5QbMLkP0u/LYjrV6a7iE4z8n6MfMhrKpGkhZYaAgZnhNDd Svnqap3lGuq/kMPIEwn2hboW9mjj5WOZyeyeR0/3qB8JXxsuiXJPcwBu2A9IeQsGzoZx 1MThP9c7hrrJBPx59l01fb34C1M3M+1wh21INVWeaHfpEbgsSG4vPMXQMrDCRowJfX+Q 4VrJtiHe8kqhzqkrZBxq08EUSTzJKIVHV48tkKajcv+3AvVp5Z+QqHTS+5cwPLTs2/Vg qpOkUdnHdiRKBL9RSpyYUFdydPFnFA2HKf2gVxfX0AqLmySa4G4QwhN4zJ4NHPUfZIti sn+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344919; x=1716949719; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=w+U7wzbNqJl9ii7+XfVEw3LB69135VJ3H5IvnuX0LIo=; b=RDvJ54RgxaU0QOfr6oJsjDrGg+wgZXtf+C4ps71RkcuRRRZLutUV9RlKCKpeIOfbI5 uS8h4cAo0J5D92RixBMyZWRPteawGY/nlJatfpx6NIJVe8toxbbf2X8iGizk6mUEnKjG dstdNv/2q/6E1juSu35lcMFALkpZ5qBPOz/gh991jNO0SdpMJ6ZeM+iC4HFuEAGIAm9D a01ae9Zp522UzEF571pyyR5yQag5c9wHUzrwCFxIuPKydXlTGOkiyvR49RkadFU5C4sJ n39pNQKZ2izZ6KxQIeS3EvtRJZHF62EC7YoBfdcbixu2gnQRHNy0RLgvOEQfTDs44Aj9 pkRA== X-Gm-Message-State: AOJu0YynIl5mM/DIcFpQe8UQnebu1wvGjWeTCeizudGRMcGgW3aYLI0I 3VJB5oZnY7xNM7mlRLtTpUDfge1tuh+y7Nu45mY55NOk8NRMIDB1v38e7KaY9VPPJxlOXQ4fHOL maA== X-Google-Smtp-Source: AGHT+IEHUZ71Ny+Ioure2hruFjw2oRt0QpKd4bMsocwR7aZc6kl7/SQ67avYLUaT2vnq64tKh1daPSDmX5I= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:4a0f:b0:2a5:dd7:1c35 with SMTP id 98e67ed59e1d1-2bd9f6458d9mr9574a91.8.1716344919071; Tue, 21 May 2024 19:28:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:26 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-6-seanjc@google.com> Subject: [PATCH v2 5/6] x86/reboot: Unconditionally define cpu_emergency_virt_cb typedef From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Define cpu_emergency_virt_cb even if the kernel is being built without KVM support so that KVM can reference the typedef in asm/kvm_host.h without needing yet more #ifdefs. No functional change intended. Signed-off-by: Sean Christopherson Acked-by: Kai Huang Reviewed-by: Chao Gao --- arch/x86/include/asm/reboot.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index 6536873f8fc0..d0ef2a678d66 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,8 +25,8 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 -#if IS_ENABLED(CONFIG_KVM_INTEL) || IS_ENABLED(CONFIG_KVM_AMD) typedef void (cpu_emergency_virt_cb)(void); +#if IS_ENABLED(CONFIG_KVM_INTEL) || IS_ENABLED(CONFIG_KVM_AMD) void cpu_emergency_register_virt_callback(cpu_emergency_virt_cb *callback); void cpu_emergency_unregister_virt_callback(cpu_emergency_virt_cb *callback); void cpu_emergency_disable_virtualization(void); From patchwork Wed May 22 02:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13670201 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A7F67580D for ; Wed, 22 May 2024 02:28:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344922; cv=none; b=ji6V5YavSJPqaHH3sjazR7xQT0lUrTpXiCjOx0yvq4W0eHCDXqmxcA8z57wvcSuytxkNvP8EDr184j/nVqRC9Jl2BFoFdl6VFjszsnh3tELw9aVOtnbGBeUkaVD/WFYswsiVqEF5/s7UP0kupxX7AMQXNL6kFGc7maA6mhbxldw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716344922; c=relaxed/simple; bh=bL1+vHMHhm9cMRQczr5G+NHO3Zy1qclzvEFtBGp3ICc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZJLJksBmRHam9q7i132Le7qeyH/mDSKJrV4LtJhUd/ey39kxc4KodhdI0p64okneylswwdkuTo5cU02FcjngJw6orxPqgPUW9fKFYomXLPrtIAYEnt5lSYBqTbPjV1tLI0VqFxx8Kw/KkiOv3p88xIk0KIMWIf4eyOJcG2H7fZE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g0DwTBaD; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g0DwTBaD" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-6efef492e79so12887634b3a.1 for ; Tue, 21 May 2024 19:28:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716344920; x=1716949720; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3UQR7L7IeR8qz/nRM58REudhth61YMdGJiKJChX9kbI=; b=g0DwTBaDEmunazG8orN0zzc096h3jZFKCyA3z3zAZscPkfk/g+2tETd7bpzHOsJ+aa rIC9n6BWWtRATT7wJTL9Uskfz3FeR9QYg0JbKiXxzszUOs3iB/A/h2pTweo9Xfs7dgG/ Nz/6fL8T4kaDVCrtVmQV9+c5MD+nnQysxiHoMG5olk7E44CpOANl53DTVDEONZRSP9FT +UDU7tWFy0sWj6mhZi2HjhTN95SkY0MCqzVeFs27wS5ivB1/CuhMWvzzwIPIXDaWyKHX tkP7KoPV/Eq074iJE5wu3kdQmNbsyXyorEzoqJf9OufYKB1B3xa5HTTgQDCFlWR/AEev vFew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716344920; x=1716949720; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3UQR7L7IeR8qz/nRM58REudhth61YMdGJiKJChX9kbI=; b=iP7Ql9Tla68A2z+SL38Uy+jLM29jYlJBNwuUYZN15dBa75MBI8TQI8CRj3amC2YA04 1Bn3zuHf39A+sz2x7IYySeMHRJahUeUpGlcCOhtoTqpLJ9LQI8cRCdQNJogXHmaHv2X+ tKfe2xzQcTcymBGPfyuaqDrGXz9ZeK5bViY6c1yvedwYmD9iVFiAxVG4YhdFURPFMxgP mxocfDrD9VlZ6UzVQbKpU41cgsd7b41MAtLmr1kTmQTsVNwo6GA27pD6OcDTV5OTg66K vOeCNQnVRfM4URXI8Qugq6bYNki4hgvSORHQ/YG9zZWndam7d+Kmqvrf2GxrQHSlJzSk MX5A== X-Gm-Message-State: AOJu0YzwKHOOg0AOIP2a+0ioDDeMkQB1naWi9Z5c9d/Q3sPBorreKZzh N6U9Y3fN76G0TJ3l9RABe5w+ouI6Pep15Z+q+Dtpy9u036wh3ELbi8ZeNSij1KleAytPXrg0SAB qkg== X-Google-Smtp-Source: AGHT+IFOo0EomRnvSty2EF4AepPhMj/lMUO8x2DSRQ64LdBqHmI5HFTl/IGL+l18xs+e4laGXFK2xAMdhpc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d04:b0:6f3:ead3:c280 with SMTP id d2e1a72fcca58-6f6d60c1e1cmr35160b3a.2.1716344920604; Tue, 21 May 2024 19:28:40 -0700 (PDT) Reply-To: Sean Christopherson Date: Tue, 21 May 2024 19:28:27 -0700 In-Reply-To: <20240522022827.1690416-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240522022827.1690416-1-seanjc@google.com> X-Mailer: git-send-email 2.45.0.215.g3402c0e53f-goog Message-ID: <20240522022827.1690416-7-seanjc@google.com> Subject: [PATCH v2 6/6] KVM: x86: Register "emergency disable" callbacks when virt is enabled From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Chao Gao , Kai Huang Register the "disable virtualization in an emergency" callback just before KVM enables virtualization in hardware, as there is no functional need to keep the callbacks registered while KVM happens to be loaded, but is inactive, i.e. if KVM hasn't enabled virtualization. Note, unregistering the callback every time the last VM is destroyed could have measurable latency due to the synchronize_rcu() needed to ensure all references to the callback are dropped before KVM is unloaded. But the latency should be a small fraction of the total latency of disabling virtualization across all CPUs, and userspace can set enable_virt_at_load to completely eliminate the runtime overhead. Add a pointer in kvm_x86_ops to allow vendor code to provide its callback. There is no reason to force vendor code to do the registration, and either way KVM would need a new kvm_x86_ops hook. Suggested-by: Kai Huang Signed-off-by: Sean Christopherson Reviewed-by: Kai Huang Reviewed-by: Chao Gao --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/svm/svm.c | 5 +---- arch/x86/kvm/vmx/main.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 6 +----- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 10 ++++++++++ 6 files changed, 18 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index aabf1648a56a..66698f5bcc85 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -36,6 +36,7 @@ #include #include #include +#include #define __KVM_HAVE_ARCH_VCPU_DEBUGFS @@ -1613,6 +1614,8 @@ struct kvm_x86_ops { int (*hardware_enable)(void); void (*hardware_disable)(void); + cpu_emergency_virt_cb *emergency_disable; + void (*hardware_unsetup)(void); bool (*has_emulated_msr)(struct kvm *kvm, u32 index); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 3d0549ca246f..9c55d0c9cb59 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4952,6 +4952,7 @@ static void svm_vcpu_unblocking(struct kvm_vcpu *vcpu) static struct kvm_x86_ops svm_x86_ops __initdata = { .name = KBUILD_MODNAME, + .emergency_disable = svm_emergency_disable, .check_processor_compatibility = svm_check_processor_compat, .hardware_unsetup = svm_hardware_unsetup, @@ -5389,8 +5390,6 @@ static struct kvm_x86_init_ops svm_init_ops __initdata = { static void __svm_exit(void) { kvm_x86_vendor_exit(); - - cpu_emergency_unregister_virt_callback(svm_emergency_disable); } static int __init svm_init(void) @@ -5406,8 +5405,6 @@ static int __init svm_init(void) if (r) return r; - cpu_emergency_register_virt_callback(svm_emergency_disable); - /* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 7c546ad3e4c9..3f423afc263b 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -24,6 +24,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_enable = vmx_hardware_enable, .hardware_disable = vmx_hardware_disable, + .emergency_disable = vmx_emergency_disable, + .has_emulated_msr = vmx_has_emulated_msr, .vm_size = sizeof(struct kvm_vmx), diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 51b2cd13250a..eac505299a7b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -753,7 +753,7 @@ static int kvm_cpu_vmxoff(void) return -EIO; } -static void vmx_emergency_disable(void) +void vmx_emergency_disable(void) { int cpu = raw_smp_processor_id(); struct loaded_vmcs *v; @@ -8613,8 +8613,6 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr = false; - cpu_emergency_unregister_virt_callback(vmx_emergency_disable); - vmx_cleanup_l1d_flush(); } @@ -8661,8 +8659,6 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } - cpu_emergency_register_virt_callback(vmx_emergency_disable); - vmx_check_vmcs12_offsets(); /* diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 502704596c83..afddfe3747dd 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -15,6 +15,7 @@ void vmx_hardware_unsetup(void); int vmx_check_processor_compat(void); int vmx_hardware_enable(void); void vmx_hardware_disable(void); +void vmx_emergency_disable(void); int vmx_vm_init(struct kvm *kvm); void vmx_vm_destroy(struct kvm *kvm); int vmx_vcpu_precreate(struct kvm *kvm); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d750546ec934..84b34696a76c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12464,6 +12464,16 @@ void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) } EXPORT_SYMBOL_GPL(kvm_vcpu_deliver_sipi_vector); +void kvm_arch_enable_virtualization(void) +{ + cpu_emergency_register_virt_callback(kvm_x86_ops.emergency_disable); +} + +void kvm_arch_disable_virtualization(void) +{ + cpu_emergency_unregister_virt_callback(kvm_x86_ops.emergency_disable); +} + int kvm_arch_hardware_enable(void) { struct kvm *kvm;