From patchwork Thu May 12 21:45:09 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olaf Hering X-Patchwork-Id: 9086341 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id D751EBF29F for ; Thu, 12 May 2016 21:47:35 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A6FB12022D for ; Thu, 12 May 2016 21:47:34 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A4B6520225 for ; Thu, 12 May 2016 21:47:33 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b0yPz-0003wt-Ti; Thu, 12 May 2016 21:45:15 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b0yPx-0003wl-P8 for xen-devel@lists.xen.org; Thu, 12 May 2016 21:45:14 +0000 Received: from [193.109.254.147] by server-7.bemta-14.messagelabs.com id 59/86-03757-969F4375; Thu, 12 May 2016 21:45:13 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrMIsWRWlGSWpSXmKPExsUSuHLSLd2Mnyb hBs/ny1ks+biYxYHR4+ju30wBjFGsmXlJ+RUJrBl3n+1gLGgzqJh2dC17A+NFtS5GTg4WgeXM EqdPmIPYEgK5EtfP9bJ0MXIA2SIST/6ndTFycQgJnGaSuPZ6PjtIjZBAusSbE7+YIXpVJVYcO MIEYrMJKEnsPXicEcQWEfCQ+HKrmw3EFhZwlei+epYFxOYV0JVoX/AFao6uxNNV55gg4oISJ2 c+AathFlCX+DPvEjPIDcwC0hLL/3GAhDkF9CReHtsGVi4qICtxccJR1gmMArOQdM9C0j0LoXs BI/MqRo3i1KKy1CJdQyO9pKLM9IyS3MTMHF1DQxO93NTi4sT01JzEpGK95PzcTYzAwGQAgh2M Z6c5H2KU5GBSEuW1/WASLsSXlJ9SmZFYnBFfVJqTWnyIUYaDQ0mCN/AHUE6wKDU9tSItMwcYI zBpCQ4eJRFee5A0b3FBYm5xZjpE6hSjopQ4rxhIQgAkkVGaB9cGi8tLjLJSwryMQIcI8RSkFu VmlqDKv2IU52BUEubVB5nCk5lXAjf9FdBiJqDF1deNQBaXJCKkpBoYk/U676xpDtvTve7v+nk SZuk6XBX7tb47u2/lPbt2xh9e5Waml2wt9Ta/Ajzr78+INRc78Ums44u8kn54y9adjNHbJP/X e3l0nHc4kHFV88zFNU/+z19hsjY2U5Szl7Fb72lw29E7h99xSH1yXnzN4/JMx69JWtGhEWdTX tz3kezxbw/6W39HiaU4I9FQi7moOBEAI9rFN8YCAAA= X-Env-Sender: olaf@aepfle.de X-Msg-Ref: server-6.tower-27.messagelabs.com!1463089512!41274866!1 X-Originating-IP: [81.169.146.218] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 10523 invoked from network); 12 May 2016 21:45:12 -0000 Received: from mo4-p00-ob.smtp.rzone.de (HELO mo4-p00-ob.smtp.rzone.de) (81.169.146.218) by server-6.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 12 May 2016 21:45:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1463089512; l=5882; s=domk; d=aepfle.de; h=In-Reply-To:Content-Disposition:Content-Type:MIME-Version: References:Subject:To:From:Date; bh=FFOxwn4muYPZ5dOpOfC4D/YCMG/xntL67kIs+Cmu1qE=; b=OhpihhB/kzAwdaqGtURri9rI2zsxfSemqmijknF4WeNnuKq9ORJJjhVzjOHAwFJmU9R H+G1yNGarFBjYMm/NQUDgS0KDrtlY8DO6YisMwidmnW4i+9P8sJrG/R/msP5DS7RQA4yi bqpPW89abLO0GEEVw7gojBWIaMnWc1mvSuA= X-RZG-AUTH: :P2EQZWCpfu+qG7CngxMFH1J+yackYocTD1iAi8x+OWi/zfN1cLnDYUgpSVCBn4G9yZOI1CDA7cptljA4kA5oNYg0FTT5fg== X-RZG-CLASS-ID: mo00 Received: from probook ([2001:a62:1264:81ff:3915:34bf:c2ea:b403]) by smtp.strato.de (RZmta 37.26 AUTH) with ESMTPSA id U0572bs4CLj9bF1 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Thu, 12 May 2016 23:45:09 +0200 (CEST) Received: by probook (Postfix, from userid 1000) id 7BC1E50AFF; Thu, 12 May 2016 23:45:09 +0200 (CEST) Date: Thu, 12 May 2016 23:45:09 +0200 From: Olaf Hering To: xen-devel@lists.xen.org, Stefano Stabellini Message-ID: <20160512214509.GA5248@aepfle.de> References: <20160512154813.GB2960@aepfle.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20160512154813.GB2960@aepfle.de> User-Agent: Mutt/1.6.1 (6645) Subject: Re: [Xen-devel] live migrating hvm from 4.4 to 4.5 fails due to kvmvapic X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, May 12, Olaf Hering wrote: > One thing to fix it in staging-4.5 is to introduce a dummy device which > handles a section named "kvm-tpr-opt". I already have a hack which does > that, and the migration proceeds. I will propose a patch to deal with > this part of the bug. Something like shown below. > Unfortunately later the VM appears to be alive, but nothing happens in > it. I assume it gets no further interrupts. Guess I need help with this > part of the bug. Olaf --- hw/i386/Makefile.objs | 1 hw/i386/xen44_kvmvapic.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++ xen-hvm.c | 3 3 files changed, 151 insertions(+) --- a/hw/i386/Makefile.objs +++ b/hw/i386/Makefile.objs @@ -5,6 +5,7 @@ obj-y += pc_sysfw.o obj-$(CONFIG_XEN) += ../xenpv/ xen/ obj-y += kvmvapic.o +obj-y += xen44_kvmvapic.o obj-y += acpi-build.o obj-y += bios-linker-loader.o hw/i386/acpi-build.o: hw/i386/acpi-build.c hw/i386/acpi-dsdt.hex \ --- /dev/null +++ b/hw/i386/xen44_kvmvapic.c @@ -0,0 +1,147 @@ +/* + * Copy of kvmvapic to allow migration from xen-4.4 qemu-xen with "kvm-tpr-opt" + * + * Copyright (C) 2007-2008 Qumranet Technologies + * Copyright (C) 2012 Jan Kiszka, Siemens AG + * + * This work is licensed under the terms of the GNU GPL version 2, or + * (at your option) any later version. See the COPYING file in the + * top-level directory. + */ +#include "sysemu/sysemu.h" +#include "sysemu/cpus.h" +#include "sysemu/kvm.h" +#include "hw/i386/apic_internal.h" +#include "hw/sysbus.h" +#include "hw/xen/xen.h" + +typedef struct VAPICHandlers { + uint32_t set_tpr; + uint32_t set_tpr_eax; + uint32_t get_tpr[8]; + uint32_t get_tpr_stack; +} QEMU_PACKED VAPICHandlers; + +typedef struct GuestROMState { + char signature[8]; + uint32_t vaddr; + uint32_t fixup_start; + uint32_t fixup_end; + uint32_t vapic_vaddr; + uint32_t vapic_size; + uint32_t vcpu_shift; + uint32_t real_tpr_addr; + VAPICHandlers up; + VAPICHandlers mp; +} QEMU_PACKED GuestROMState; + +typedef struct VAPICROMState { + SysBusDevice busdev; + MemoryRegion io; + MemoryRegion rom; + uint32_t state; + uint32_t rom_state_paddr; + uint32_t rom_state_vaddr; + uint32_t vapic_paddr; + uint32_t real_tpr_addr; + GuestROMState rom_state; + size_t rom_size; + bool rom_mapped_writable; +} VAPICROMState; + +#define TYPE_XEN_KVMVAPIC "xen_kvmvapic" /* copy of "kvmvapic" */ + +static void xen44_vapic_realize(DeviceState *dev, Error **errp) +{ + fprintf(stderr, "%s(%u) dev %p\n", __func__, __LINE__, dev); +} + +static int xen44_vapic_post_load(void *opaque, int version_id) +{ + if (xen_enabled()) { + int i; + fprintf(stderr, "%s(%u) %p 0x%x\n", __func__, __LINE__, opaque, version_id); + for (i = 12; i > 0; i--) { + sleep(1); + fprintf(stderr, "%s(%u) sleep %d %ld\n", __func__, __LINE__, i, (long)getpid()); + } + } + return 0; +} + +static const VMStateDescription vmstate_handlers = { + .name = "kvmvapic-handlers", + .version_id = 1, + .minimum_version_id = 1, + .minimum_version_id_old = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT32(set_tpr, VAPICHandlers), + VMSTATE_UINT32(set_tpr_eax, VAPICHandlers), + VMSTATE_UINT32_ARRAY(get_tpr, VAPICHandlers, 8), + VMSTATE_UINT32(get_tpr_stack, VAPICHandlers), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription vmstate_guest_rom = { + .name = "kvmvapic-guest-rom", + .version_id = 1, + .minimum_version_id = 1, + .minimum_version_id_old = 1, + .fields = (VMStateField[]) { + VMSTATE_UNUSED(8), /* signature */ + VMSTATE_UINT32(vaddr, GuestROMState), + VMSTATE_UINT32(fixup_start, GuestROMState), + VMSTATE_UINT32(fixup_end, GuestROMState), + VMSTATE_UINT32(vapic_vaddr, GuestROMState), + VMSTATE_UINT32(vapic_size, GuestROMState), + VMSTATE_UINT32(vcpu_shift, GuestROMState), + VMSTATE_UINT32(real_tpr_addr, GuestROMState), + VMSTATE_STRUCT(up, GuestROMState, 0, vmstate_handlers, VAPICHandlers), + VMSTATE_STRUCT(mp, GuestROMState, 0, vmstate_handlers, VAPICHandlers), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription vmstate_xen44_vapic = { + .name = "kvm-tpr-opt", /* compatible with qemu-kvm VAPIC */ + .version_id = 1, + .minimum_version_id = 1, + .minimum_version_id_old = 1, + .post_load = xen44_vapic_post_load, + .fields = (VMStateField[]) { + VMSTATE_STRUCT(rom_state, VAPICROMState, 0, vmstate_guest_rom, + GuestROMState), + VMSTATE_UINT32(state, VAPICROMState), + VMSTATE_UINT32(real_tpr_addr, VAPICROMState), + VMSTATE_UINT32(rom_state_vaddr, VAPICROMState), + VMSTATE_UINT32(vapic_paddr, VAPICROMState), + VMSTATE_UINT32(rom_state_paddr, VAPICROMState), + VMSTATE_END_OF_LIST() + } +}; + +static void xen44_vapic_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + fprintf(stderr, "%s(%u) klass %p data %p\n", __func__, __LINE__, klass, data); + dc->vmsd = &vmstate_xen44_vapic; + dc->realize = xen44_vapic_realize; +} + +static const TypeInfo xen44_vapic_type = { + .name = TYPE_XEN_KVMVAPIC, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(VAPICROMState), + .class_init = xen44_vapic_class_init, +}; + +static void xen44_vapic_register(void) +{ + fprintf(stderr, "%s(%u)\n", __func__, __LINE__); + type_register_static(&xen44_vapic_type); +} + +type_init(xen44_vapic_register); + --- a/xen-hvm.c +++ b/xen-hvm.c @@ -15,6 +15,7 @@ #include "hw/i386/apic-msidef.h" #include "hw/xen/xen_common.h" #include "hw/xen/xen_backend.h" +#include "hw/sysbus.h" #include "qmp-commands.h" #include "sysemu/char.h" @@ -161,6 +162,8 @@ static void xen_set_irq(void *opaque, in qemu_irq *xen_interrupt_controller_init(void) { + DeviceState *vapic = sysbus_create_simple("kvmvapic", -1, NULL); + return qemu_allocate_irqs(xen_set_irq, NULL, 16); }