From patchwork Thu Mar 18 16:02:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Kuznetsov X-Patchwork-Id: 12148497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0294DC433DB for ; Thu, 18 Mar 2021 16:09:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A8EE864F01 for ; Thu, 18 Mar 2021 16:09:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8EE864F01 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lMvD0-0005l5-NK for qemu-devel@archiver.kernel.org; Thu, 18 Mar 2021 12:09:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lMv7B-00011i-KP for qemu-devel@nongnu.org; Thu, 18 Mar 2021 12:03:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44144) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lMv74-0001tH-9p for qemu-devel@nongnu.org; Thu, 18 Mar 2021 12:03:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1616083384; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ldqxknpOfcgNFPksS+U4LnfshA8OqWCXYgfo0WQE7lo=; b=NdzwcgbqDL157mGSKWaB8EdiZptoimxZ92OZVyFoIqaVy8mveSETgdG6B+HBKwmCICZv+Z ZRvc13mES7cUZT+H0/Nw5VL2YM33h7JAcb6FJtuhGQCjgC2PWODEjSDh6n/cI3tSbvYXzw DgP2ITCmX/qNx2ke76jF3dL121Tl9Q0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-158-_GRv6wlUMRKAldZY0jI80Q-1; Thu, 18 Mar 2021 12:03:02 -0400 X-MC-Unique: _GRv6wlUMRKAldZY0jI80Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E41518189DB; Thu, 18 Mar 2021 16:03:01 +0000 (UTC) Received: from vitty.brq.redhat.com (unknown [10.40.194.127]) by smtp.corp.redhat.com (Postfix) with ESMTP id 289481045EBE; Thu, 18 Mar 2021 16:02:59 +0000 (UTC) From: Vitaly Kuznetsov To: qemu-devel@nongnu.org Subject: [PATCH 3/3] i386: Make sure kvm_arch_set_tsc_khz() succeeds on migration when 'hv-reenlightenment' was exposed Date: Thu, 18 Mar 2021 17:02:49 +0100 Message-Id: <20210318160249.1084178-4-vkuznets@redhat.com> In-Reply-To: <20210318160249.1084178-1-vkuznets@redhat.com> References: <20210318160249.1084178-1-vkuznets@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=vkuznets@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=vkuznets@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.249, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Marcelo Tosatti , Richard Henderson , Eduardo Habkost , "Dr . David Alan Gilbert" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" KVM doesn't fully support Hyper-V reenlightenment notifications on migration. In particular, it doesn't support emulating TSC frequency of the source host by trapping all TSC accesses so unless TSC scaling is supported on the destination host and KVM_SET_TSC_KHZ succeeds, it is unsafe to proceed with migration. Normally, we only require KVM_SET_TSC_KHZ to succeed when 'user_tsc_khz' was set and just 'try' KVM_SET_TSC_KHZ without otherwise. Introduce a new vmstate section (which is added when the guest has reenlightenment feature enabled) and add env.tsc_khz to it. We already have env.tsc_khz packed in 'cpu/tsc_khz' but we don't want to be dependent on the section order. Signed-off-by: Vitaly Kuznetsov --- target/i386/kvm/hyperv.h | 1 + target/i386/kvm/kvm.c | 11 +++++++++++ target/i386/machine.c | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+) diff --git a/target/i386/kvm/hyperv.h b/target/i386/kvm/hyperv.h index 67543296c3a4..c65e5c85c4d3 100644 --- a/target/i386/kvm/hyperv.h +++ b/target/i386/kvm/hyperv.h @@ -20,6 +20,7 @@ #ifdef CONFIG_KVM int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit); +int kvm_hv_tsc_frequency_loaded(X86CPU *cpu); #endif int hyperv_x86_synic_add(X86CPU *cpu); diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 7fe9f527103c..f6c4093778e9 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -1460,6 +1460,17 @@ static int hyperv_init_vcpu(X86CPU *cpu) return 0; } +int kvm_hv_tsc_frequency_loaded(X86CPU *cpu) +{ + CPUState *cs = CPU(cpu); + + /* + * KVM doens't fully support re-enlightenment notifications so we need to + * make sure TSC frequency doesn't change upon migration. + */ + return kvm_arch_set_tsc_khz(cs); +} + static Error *invtsc_mig_blocker; #define KVM_MAX_CPUID_ENTRIES 100 diff --git a/target/i386/machine.c b/target/i386/machine.c index 715620c58809..369a8f1e7a7a 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -896,6 +896,42 @@ static const VMStateDescription vmstate_msr_hyperv_reenlightenment = { VMSTATE_END_OF_LIST() } }; + +static bool hyperv_tsc_frequency_needed(void *opaque) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + + return env->tsc_khz != 0 && (env->msr_hv_reenlightenment_control || + env->msr_hv_tsc_emulation_control); +} + +static int hyperv_tsc_frequency_post_load(void *opaque, int version_id) +{ + X86CPU *cpu = opaque; + int r; + + r = kvm_hv_tsc_frequency_loaded(cpu); + if (r) { + error_report("Failed to set the desired TSC frequency and " + "reenlightenment was exposed"); + return -EINVAL; + } + + return 0; +} + +static const VMStateDescription vmstate_msr_hyperv_tsc_frequency = { + .name = "cpu/msr_hyperv_tsc_frequency", + .version_id = 1, + .minimum_version_id = 1, + .needed = hyperv_tsc_frequency_needed, + .post_load = hyperv_tsc_frequency_post_load, + .fields = (VMStateField[]) { + VMSTATE_INT64(env.tsc_khz, X86CPU), + VMSTATE_END_OF_LIST() + } +}; #endif static bool avx512_needed(void *opaque) @@ -1495,6 +1531,7 @@ VMStateDescription vmstate_x86_cpu = { &vmstate_msr_hyperv_synic, &vmstate_msr_hyperv_stimer, &vmstate_msr_hyperv_reenlightenment, + &vmstate_msr_hyperv_tsc_frequency, #endif &vmstate_avx512, &vmstate_xss,