Message ID | 201211281339.17589.vrozenfe@redhat.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > On Tuesday, November 27, 2012 11:13:12 PM George-Cristian Bîrzan wrote: >> On Tue, Nov 27, 2012 at 10:38 PM, Vadim Rozenfeld <vrozenfe@redhat.com> > wrote: >> > I have some code which do both reference time and invariant TSC but it >> > will not work after migration. I will send it later today. >> >> Do you mean migrating guests? This is not an issue for us. > OK, but don't say I didn't warn you :) > > There are two patches, one for kvm and another one for qemu. > you will probably need to rebase them. > Add "hv_tsc" cpu parameter to activate this feature. > you will probably need to deactivate hpet by adding "-no-hpet" > parameter as well. I've also added +hv_relaxed since then, but this is the command I'm using now and there's no change: /usr/bin/qemu-kvm -name b691546e-79f8-49c6-a293-81067503a6ad -S -M pc-1.2 -enable-kvm -m 16384 -smp 9,sockets=1,cores=9,threads=1 -uuid b691546e-79f8-49c6-a293-81067503a6ad -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/b691546e-79f8-49c6-a293-81067503a6ad.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/dis-magnetics-2-223101/d8b233c6-8424-4de9-ae3c-7c9a60288514,if=none,id=drive-virtio-disk0,format=qcow2,cache=writeback,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=22:2e:fb:a2:36:be,bus=pci.0,addr=0x3 -netdev tap,fd=40,id=hostnet1,vhost=on,vhostfd=41 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=22:94:44:5a:cb:24,bus=pci.0,addr=0x4 -vnc 127.0.0.1:0,password -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -cpu host,hv_tsc I compiled qemu-1.2.0-24 after applying your patch, used the head for KVM, and I see no difference. I've tried setting windows' useplatformclock on and off, no change either. Other than that, was looking into a profiling trace of the software running and a lot of time (60%?) is spent calling two functions from hal.dll, HalpGetPmTimerSleepModePerfCounter when I disable HPET, and HalpHPETProgramRolloverTimer which do point at something related to the timers. Any other thing I can try? -- George-Cristian Bîrzan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote:
> There are two patches, one for kvm and another one for qemu.
I just realised this. I was supposed to use qemu, or qemu-kvm? I used qemu
--
George-Cristian Bîrzan
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 28, 2012 at 09:18:38PM +0200, George-Cristian Bîrzan wrote: > On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > > There are two patches, one for kvm and another one for qemu. > > I just realised this. I was supposed to use qemu, or qemu-kvm? I used qemu > Does not matter, but you need to also recompile kernel with the first patch. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 28, 2012 at 9:56 PM, Gleb Natapov <gleb@redhat.com> wrote: > On Wed, Nov 28, 2012 at 09:18:38PM +0200, George-Cristian Bîrzan wrote: >> On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: >> > There are two patches, one for kvm and another one for qemu. >> >> I just realised this. I was supposed to use qemu, or qemu-kvm? I used qemu >> > Does not matter, but you need to also recompile kernel with the first patch. Do I have to recompile the kernel, or just the module? I followed the instructions at http://www.linux-kvm.org/page/Code#building_an_external_module_with_older_kernels but I guess I can do the whole kernel, if it might help. -- George-Cristian Bîrzan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 28, 2012 at 10:01:04PM +0200, George-Cristian Bîrzan wrote: > On Wed, Nov 28, 2012 at 9:56 PM, Gleb Natapov <gleb@redhat.com> wrote: > > On Wed, Nov 28, 2012 at 09:18:38PM +0200, George-Cristian Bîrzan wrote: > >> On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > >> > There are two patches, one for kvm and another one for qemu. > >> > >> I just realised this. I was supposed to use qemu, or qemu-kvm? I used qemu > >> > > Does not matter, but you need to also recompile kernel with the first patch. > > Do I have to recompile the kernel, or just the module? I followed the > instructions at > http://www.linux-kvm.org/page/Code#building_an_external_module_with_older_kernels > but I guess I can do the whole kernel, if it might help. > Module is enough, but kvm-kmod is not what you want. Just rebuild the whole kernel if you do not know how to rebuild only the module for your distribution's kernel. -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wednesday, November 28, 2012 09:09:29 PM George-Cristian Bîrzan wrote: > On Wed, Nov 28, 2012 at 1:39 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > > On Tuesday, November 27, 2012 11:13:12 PM George-Cristian Bîrzan wrote: > >> On Tue, Nov 27, 2012 at 10:38 PM, Vadim Rozenfeld <vrozenfe@redhat.com> > > > > wrote: > >> > I have some code which do both reference time and invariant TSC but it > >> > will not work after migration. I will send it later today. > >> > >> Do you mean migrating guests? This is not an issue for us. > > > > OK, but don't say I didn't warn you :) > > > > There are two patches, one for kvm and another one for qemu. > > you will probably need to rebase them. > > Add "hv_tsc" cpu parameter to activate this feature. > > you will probably need to deactivate hpet by adding "-no-hpet" > > parameter as well. > > I've also added +hv_relaxed since then, but this is the command I'm I would suggest activating relaxed timing for all W2K8R2/Win7 guests. > using now and there's no change: > > /usr/bin/qemu-kvm -name b691546e-79f8-49c6-a293-81067503a6ad -S -M > pc-1.2 -enable-kvm -m 16384 -smp 9,sockets=1,cores=9,threads=1 -uuid > b691546e-79f8-49c6-a293-81067503a6ad -no-user-config -nodefaults > -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/b691546e-79f8-49c6-a293-8 > 1067503a6ad.monitor,server,nowait -mon > chardev=charmonitor,id=monitor,mode=control -rtc base=utc > -no-hpet -no-shutdown -device > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive > file=/var/lib/libvirt/images/dis-magnetics-2-223101/d8b233c6-8424-4de9-ae3c > -7c9a60288514,if=none,id=drive-virtio-disk0,format=qcow2,cache=writeback,ai > o=native -device > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=vir > tio-disk0,bootindex=1 -netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 > -device > virtio-net-pci,netdev=hostnet0,id=net0,mac=22:2e:fb:a2:36:be,bus=pci.0,addr > =0x3 -netdev tap,fd=40,id=hostnet1,vhost=on,vhostfd=41 -device > virtio-net-pci,netdev=hostnet1,id=net1,mac=22:94:44:5a:cb:24,bus=pci.0,addr > =0x4 -vnc 127.0.0.1:0,password -vga cirrus -device > virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -cpu host,hv_tsc > > I compiled qemu-1.2.0-24 after applying your patch, used the head for > KVM, and I see no difference. I've tried setting windows' > useplatformclock on and off, no change either. > > > Other than that, was looking into a profiling trace of the software > running and a lot of time (60%?) is spent calling two functions from > hal.dll, HalpGetPmTimerSleepModePerfCounter when I disable HPET, and > HalpHPETProgramRolloverTimer which do point at something related to > the timers. > It means that hyper-v time stamp source was not activated. > Any other thing I can try? > > > -- > George-Cristian Bîrzan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 29, 2012 at 1:56 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: >> I've also added +hv_relaxed since then, but this is the command I'm > > I would suggest activating relaxed timing for all W2K8R2/Win7 guests. Is there any place I can read up on the downsides of this for Linux, or is Just Better? >>>> Other than that, was looking into a profiling trace of the software >> running and a lot of time (60%?) is spent calling two functions from >> hal.dll, HalpGetPmTimerSleepModePerfCounter when I disable HPET, and >> HalpHPETProgramRolloverTimer which do point at something related to >> the timers. >> > It means that hyper-v time stamp source was not activated. I recompiled the whole kernel, with your patch, and while I cannot check at 70Mbps now, a test stream of 20 seems to do better. Also, now I don't see any of those functions, which used to account ~60% of the time spent by the program. I'm waiting for the customer to come back and start the 'real' stream, but from my tests, time spent in hal.dll is now an order of magnitude smaller. -- George-Cristian Bîrzan -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 29, 2012 at 03:45:52PM +0200, George-Cristian Bîrzan wrote: > On Thu, Nov 29, 2012 at 1:56 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > >> I've also added +hv_relaxed since then, but this is the command I'm > > > > I would suggest activating relaxed timing for all W2K8R2/Win7 guests. > > Is there any place I can read up on the downsides of this for Linux, > or is Just Better? > You shouldn't use hyper-v flags for Linux guests. In theory Linux should just ignore them, in practice there may be bugs that will prevent Linux from detecting that it runs as a guest and disable optimizations. > >>>> Other than that, was looking into a profiling trace of the software > >> running and a lot of time (60%?) is spent calling two functions from > >> hal.dll, HalpGetPmTimerSleepModePerfCounter when I disable HPET, and > >> HalpHPETProgramRolloverTimer which do point at something related to > >> the timers. > >> > > It means that hyper-v time stamp source was not activated. > > I recompiled the whole kernel, with your patch, and while I cannot > check at 70Mbps now, a test stream of 20 seems to do better. Also, now > I don't see any of those functions, which used to account ~60% of the > time spent by the program. I'm waiting for the customer to come back > and start the 'real' stream, but from my tests, time spent in hal.dll > is now an order of magnitude smaller. > > -- > George-Cristian Bîrzan -- Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thursday, November 29, 2012 03:56:10 PM Gleb Natapov wrote: > On Thu, Nov 29, 2012 at 03:45:52PM +0200, George-Cristian Bîrzan wrote: > > On Thu, Nov 29, 2012 at 1:56 PM, Vadim Rozenfeld <vrozenfe@redhat.com> wrote: > > >> I've also added +hv_relaxed since then, but this is the command I'm > > > > > > I would suggest activating relaxed timing for all W2K8R2/Win7 guests. > > > > Is there any place I can read up on the downsides of this for Linux, > > or is Just Better? > > You shouldn't use hyper-v flags for Linux guests. In theory Linux should > just ignore them, in practice there may be bugs that will prevent Linux > from detecting that it runs as a guest and disable optimizations. > As Gleb said, hyper-v flag are relevant to the Windows guests only. IIRC spinlocks and vapic should work for Vista and higher. Relaxed timing and partition reference time work for Win7/W2K8R2. > > >>>> Other than that, was looking into a profiling trace of the software > > >> > > >> running and a lot of time (60%?) is spent calling two functions from > > >> hal.dll, HalpGetPmTimerSleepModePerfCounter when I disable HPET, and > > >> HalpHPETProgramRolloverTimer which do point at something related to > > >> the timers. > > > > > > It means that hyper-v time stamp source was not activated. > > > > I recompiled the whole kernel, with your patch, and while I cannot > > check at 70Mbps now, a test stream of 20 seems to do better. Also, now > > I don't see any of those functions, which used to account ~60% of the > > time spent by the program. I'm waiting for the customer to come back > > and start the 'real' stream, but from my tests, time spent in hal.dll > > is now an order of magnitude smaller. > > > > -- > > George-Cristian Bîrzan > > -- > Gleb. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/target-i386/cpu.c b/target-i386/cpu.c index f3708e6..ad77b72 100644 --- a/target-i386/cpu.c +++ b/target-i386/cpu.c @@ -1250,6 +1250,8 @@ static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *cpu_model) hyperv_enable_relaxed_timing(true); } else if (!strcmp(featurestr, "hv_vapic")) { hyperv_enable_vapic_recommended(true); + } else if (!strcmp(featurestr, "hv_tsc")) { + hyperv_enable_tsc_recommended(true); } else { fprintf(stderr, "feature string `%s' not in format (+feature|-feature|feature=xyz)\n", featurestr); goto error; diff --git a/target-i386/hyperv.c b/target-i386/hyperv.c index f284e99..bd581a1 100644 --- a/target-i386/hyperv.c +++ b/target-i386/hyperv.c @@ -15,6 +15,12 @@ static bool hyperv_vapic; static bool hyperv_relaxed_timing; static int hyperv_spinlock_attempts = HYPERV_SPINLOCK_NEVER_RETRY; +static bool hyperv_tsc; + +void hyperv_enable_tsc_recommended(bool val) +{ + hyperv_tsc = val; +} void hyperv_enable_vapic_recommended(bool val) { @@ -42,12 +48,18 @@ bool hyperv_enabled(void) bool hyperv_hypercall_available(void) { if (hyperv_vapic || + hyperv_tsc || (hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_RETRY)) { return true; } return false; } +bool hyperv_tsc_recommended(void) +{ + return hyperv_tsc; +} + bool hyperv_vapic_recommended(void) { return hyperv_vapic; diff --git a/target-i386/hyperv.h b/target-i386/hyperv.h index bacb1d4..94c2d6e 100644 --- a/target-i386/hyperv.h +++ b/target-i386/hyperv.h @@ -27,10 +27,12 @@ #endif #if !defined(CONFIG_USER_ONLY) && defined(CONFIG_KVM) +void hyperv_enable_tsc_recommended(bool val); void hyperv_enable_vapic_recommended(bool val); void hyperv_enable_relaxed_timing(bool val); void hyperv_set_spinlock_retries(int val); #else +static inline void hyperv_enable_tsc_recommended(bool val) { } static inline void hyperv_enable_vapic_recommended(bool val) { } static inline void hyperv_enable_relaxed_timing(bool val) { } static inline void hyperv_set_spinlock_retries(int val) { } @@ -38,6 +40,7 @@ static inline void hyperv_set_spinlock_retries(int val) { } bool hyperv_enabled(void); bool hyperv_hypercall_available(void); +bool hyperv_tsc_recommended(void); bool hyperv_vapic_recommended(void); bool hyperv_relaxed_timing_enabled(void); int hyperv_get_spinlock_retries(void); diff --git a/target-i386/kvm.c b/target-i386/kvm.c index 5b18383..dc7f259 100644 --- a/target-i386/kvm.c +++ b/target-i386/kvm.c @@ -390,13 +390,17 @@ int kvm_arch_init_vcpu(CPUX86State *env) c = &cpuid_data.entries[cpuid_i++]; memset(c, 0, sizeof(*c)); c->function = KVM_CPUID_SIGNATURE; - if (!hyperv_enabled()) { - memcpy(signature, "KVMKVMKVM\0\0\0", 12); - c->eax = 0; - } else { - memcpy(signature, "Microsoft Hv", 12); + memcpy(signature, "KVMKVMKVM\0\0\0", 12); + if (hyperv_enabled()) { c->eax = HYPERV_CPUID_MIN; } +// if (!hyperv_enabled()) { +// memcpy(signature, "KVMKVMKVM\0\0\0", 12); +// c->eax = 0; +// } else { +// memcpy(signature, "Microsoft Hv", 12); +// c->eax = HYPERV_CPUID_MIN; +// } c->ebx = signature[0]; c->ecx = signature[1]; c->edx = signature[2]; @@ -427,7 +431,11 @@ int kvm_arch_init_vcpu(CPUX86State *env) c->eax |= HV_X64_MSR_HYPERCALL_AVAILABLE; c->eax |= HV_X64_MSR_APIC_ACCESS_AVAILABLE; } - + if (hyperv_tsc_recommended()) { + c->eax |= HV_X64_MSR_HYPERCALL_AVAILABLE; + c->eax |= HV_X64_MSR_TIME_REF_COUNT_AVAILABLE; + c->eax |= 0x200; + } c = &cpuid_data.entries[cpuid_i++]; memset(c, 0, sizeof(*c)); c->function = HYPERV_CPUID_ENLIGHTMENT_INFO; @@ -445,14 +453,14 @@ int kvm_arch_init_vcpu(CPUX86State *env) c->eax = 0x40; c->ebx = 0x40; - c = &cpuid_data.entries[cpuid_i++]; - memset(c, 0, sizeof(*c)); - c->function = KVM_CPUID_SIGNATURE_NEXT; - memcpy(signature, "KVMKVMKVM\0\0\0", 12); - c->eax = 0; - c->ebx = signature[0]; - c->ecx = signature[1]; - c->edx = signature[2]; +// c = &cpuid_data.entries[cpuid_i++]; +// memset(c, 0, sizeof(*c)); +// c->function = KVM_CPUID_SIGNATURE_NEXT; +// memcpy(signature, "KVMKVMKVM\0\0\0", 12); +// c->eax = 0; +// c->ebx = signature[0]; +// c->ecx = signature[1]; +// c->edx = signature[2]; } has_msr_async_pf_en = c->eax & (1 << KVM_FEATURE_ASYNC_PF);