| Message ID | 1494551525-4100-1-git-send-email-wanpeng.li@hotmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 12/05/2017 03:12, Wanpeng Li wrote: > From: Wanpeng Li <wanpeng.li@hotmail.com> > > BUG: using __this_cpu_read() in preemptible [00000000] code: qemu-system-x86/2809 > caller is __this_cpu_preempt_check+0x13/0x20 > CPU: 2 PID: 2809 Comm: qemu-system-x86 Not tainted 4.11.0+ #13 > Call Trace: > dump_stack+0x99/0xce > check_preemption_disabled+0xf5/0x100 > __this_cpu_preempt_check+0x13/0x20 > get_kvmclock_ns+0x6f/0x110 [kvm] > get_time_ref_counter+0x5d/0x80 [kvm] > kvm_hv_process_stimers+0x2a1/0x8a0 [kvm] > ? kvm_hv_process_stimers+0x2a1/0x8a0 [kvm] > ? kvm_arch_vcpu_ioctl_run+0xac9/0x1ce0 [kvm] > kvm_arch_vcpu_ioctl_run+0x5bf/0x1ce0 [kvm] > kvm_vcpu_ioctl+0x384/0x7b0 [kvm] > ? kvm_vcpu_ioctl+0x384/0x7b0 [kvm] > ? __fget+0xf3/0x210 > do_vfs_ioctl+0xa4/0x700 > ? __fget+0x114/0x210 > SyS_ioctl+0x79/0x90 > entry_SYSCALL_64_fastpath+0x23/0xc2 > RIP: 0033:0x7f9d164ed357 > ? __this_cpu_preempt_check+0x13/0x20 > > This can be reproduced by run kvm-unit-tests/hyperv_stimer.flat w/ > CONFIG_PREEMPT and CONFIG_DEBUG_PREEMPT enabled. > > Safe access to per-CPU data requires a couple of constraints, though: the > thread working with the data cannot be preempted and it cannot be migrated > while it manipulates per-CPU variables. If the thread is preempted, the > thread that replaces it could try to work with the same variables; migration > to another CPU could also cause confusion. However there is no preemption > disable when reads host per-CPU tsc rate to calculate the current kvmclock > timestamp. > > This patch fixes it by utilizing get_cpu/put_cpu pair to guarantee both > __this_cpu_read() and rdtsc() are not preempted. > > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Radim Krčmář <rkrcmar@redhat.com> > Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> > --- > v2 -> v3: > * update patch description > v1 -> v2: > * utilize get_cpu/put_cpu pair to protect both __this_cpu_read and rdtsc() > > arch/x86/kvm/x86.c | 10 +++++++++- > 1 file changed, 9 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index b54125b..3b5fc7e 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -1763,6 +1763,7 @@ u64 get_kvmclock_ns(struct kvm *kvm) > { > struct kvm_arch *ka = &kvm->arch; > struct pvclock_vcpu_time_info hv_clock; > + u64 ret; > > spin_lock(&ka->pvclock_gtod_sync_lock); > if (!ka->use_master_clock) { > @@ -1774,10 +1775,17 @@ u64 get_kvmclock_ns(struct kvm *kvm) > hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset; > spin_unlock(&ka->pvclock_gtod_sync_lock); > > + /* both __this_cpu_read() and rdtsc() should be on the same cpu */ > + get_cpu(); > + > kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, > &hv_clock.tsc_shift, > &hv_clock.tsc_to_system_mul); > - return __pvclock_read_cycles(&hv_clock, rdtsc()); > + ret = __pvclock_read_cycles(&hv_clock, rdtsc()); > + > + put_cpu(); > + > + return ret; > } > > static void kvm_setup_pvclock_page(struct kvm_vcpu *v) > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Cc: stable@vger.kernel.org
Ping, 2017-05-12 15:44 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>: > > > On 12/05/2017 03:12, Wanpeng Li wrote: >> From: Wanpeng Li <wanpeng.li@hotmail.com> >> >> BUG: using __this_cpu_read() in preemptible [00000000] code: qemu-system-x86/2809 >> caller is __this_cpu_preempt_check+0x13/0x20 >> CPU: 2 PID: 2809 Comm: qemu-system-x86 Not tainted 4.11.0+ #13 >> Call Trace: >> dump_stack+0x99/0xce >> check_preemption_disabled+0xf5/0x100 >> __this_cpu_preempt_check+0x13/0x20 >> get_kvmclock_ns+0x6f/0x110 [kvm] >> get_time_ref_counter+0x5d/0x80 [kvm] >> kvm_hv_process_stimers+0x2a1/0x8a0 [kvm] >> ? kvm_hv_process_stimers+0x2a1/0x8a0 [kvm] >> ? kvm_arch_vcpu_ioctl_run+0xac9/0x1ce0 [kvm] >> kvm_arch_vcpu_ioctl_run+0x5bf/0x1ce0 [kvm] >> kvm_vcpu_ioctl+0x384/0x7b0 [kvm] >> ? kvm_vcpu_ioctl+0x384/0x7b0 [kvm] >> ? __fget+0xf3/0x210 >> do_vfs_ioctl+0xa4/0x700 >> ? __fget+0x114/0x210 >> SyS_ioctl+0x79/0x90 >> entry_SYSCALL_64_fastpath+0x23/0xc2 >> RIP: 0033:0x7f9d164ed357 >> ? __this_cpu_preempt_check+0x13/0x20 >> >> This can be reproduced by run kvm-unit-tests/hyperv_stimer.flat w/ >> CONFIG_PREEMPT and CONFIG_DEBUG_PREEMPT enabled. >> >> Safe access to per-CPU data requires a couple of constraints, though: the >> thread working with the data cannot be preempted and it cannot be migrated >> while it manipulates per-CPU variables. If the thread is preempted, the >> thread that replaces it could try to work with the same variables; migration >> to another CPU could also cause confusion. However there is no preemption >> disable when reads host per-CPU tsc rate to calculate the current kvmclock >> timestamp. >> >> This patch fixes it by utilizing get_cpu/put_cpu pair to guarantee both >> __this_cpu_read() and rdtsc() are not preempted. >> >> Cc: Paolo Bonzini <pbonzini@redhat.com> >> Cc: Radim Krčmář <rkrcmar@redhat.com> >> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com> >> --- >> v2 -> v3: >> * update patch description >> v1 -> v2: >> * utilize get_cpu/put_cpu pair to protect both __this_cpu_read and rdtsc() >> >> arch/x86/kvm/x86.c | 10 +++++++++- >> 1 file changed, 9 insertions(+), 1 deletion(-) >> >> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >> index b54125b..3b5fc7e 100644 >> --- a/arch/x86/kvm/x86.c >> +++ b/arch/x86/kvm/x86.c >> @@ -1763,6 +1763,7 @@ u64 get_kvmclock_ns(struct kvm *kvm) >> { >> struct kvm_arch *ka = &kvm->arch; >> struct pvclock_vcpu_time_info hv_clock; >> + u64 ret; >> >> spin_lock(&ka->pvclock_gtod_sync_lock); >> if (!ka->use_master_clock) { >> @@ -1774,10 +1775,17 @@ u64 get_kvmclock_ns(struct kvm *kvm) >> hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset; >> spin_unlock(&ka->pvclock_gtod_sync_lock); >> >> + /* both __this_cpu_read() and rdtsc() should be on the same cpu */ >> + get_cpu(); >> + >> kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, >> &hv_clock.tsc_shift, >> &hv_clock.tsc_to_system_mul); >> - return __pvclock_read_cycles(&hv_clock, rdtsc()); >> + ret = __pvclock_read_cycles(&hv_clock, rdtsc()); >> + >> + put_cpu(); >> + >> + return ret; >> } >> >> static void kvm_setup_pvclock_page(struct kvm_vcpu *v) >> > > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> > Cc: stable@vger.kernel.org
2017-05-19 05:53+0800, Wanpeng Li:
> Ping,
Applying for 4.12-rc2, thanks.
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b54125b..3b5fc7e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1763,6 +1763,7 @@ u64 get_kvmclock_ns(struct kvm *kvm) { struct kvm_arch *ka = &kvm->arch; struct pvclock_vcpu_time_info hv_clock; + u64 ret; spin_lock(&ka->pvclock_gtod_sync_lock); if (!ka->use_master_clock) { @@ -1774,10 +1775,17 @@ u64 get_kvmclock_ns(struct kvm *kvm) hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset; spin_unlock(&ka->pvclock_gtod_sync_lock); + /* both __this_cpu_read() and rdtsc() should be on the same cpu */ + get_cpu(); + kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); - return __pvclock_read_cycles(&hv_clock, rdtsc()); + ret = __pvclock_read_cycles(&hv_clock, rdtsc()); + + put_cpu(); + + return ret; } static void kvm_setup_pvclock_page(struct kvm_vcpu *v)