| Message ID | 1583823679-17648-1-git-send-email-wanpengli@tencent.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: X86: Don't load/put guest FPU context for sleeping AP | expand |
On Tue, Mar 10, 2020 at 03:01:19PM +0800, Wanpeng Li wrote: > From: Wanpeng Li <wanpengli@tencent.com> > > kvm_load_guest_fpu() and kvm_put_guest_fpu() each consume more than 14us > observed by ftrace, the qemu userspace FPU is swapped out for the guest > FPU context for the duration of the KVM_RUN ioctl even if sleeping AP, > we shouldn't load/put guest FPU context for this case especially for > serverless scenario which sensitives to boot time. > > Signed-off-by: Wanpeng Li <wanpengli@tencent.com> > --- > arch/x86/kvm/x86.c | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 5de2006..080ffa4 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -8680,7 +8680,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > > vcpu_load(vcpu); > kvm_sigset_activate(vcpu); > - kvm_load_guest_fpu(vcpu); > > if (unlikely(vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED)) { > if (kvm_run->immediate_exit) { > @@ -8718,12 +8717,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > } > } > > + kvm_load_guest_fpu(vcpu); Ugh, so this isn't safe on MPX capable CPUs, kvm_apic_accept_events() can trigger kvm_vcpu_reset() with @init_event=true and try to unload guest_fpu. We could hack around that issue, but it'd be ugly, and I'm also concerned that calling vmx_vcpu_reset() without guest_fpu loaded will be problematic in the future with all the things that are getting managed by XSAVE. > + > if (unlikely(vcpu->arch.complete_userspace_io)) { > int (*cui)(struct kvm_vcpu *) = vcpu->arch.complete_userspace_io; > vcpu->arch.complete_userspace_io = NULL; > r = cui(vcpu); > if (r <= 0) > - goto out; > + goto out_fpu; > } else > WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed); > > @@ -8732,8 +8733,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > else > r = vcpu_run(vcpu); > > -out: > +out_fpu: > kvm_put_guest_fpu(vcpu); > +out: > if (vcpu->run->kvm_valid_regs) > store_regs(vcpu); > post_kvm_run_save(vcpu); > -- > 2.7.4 >
On Wed, 11 Mar 2020 at 00:01, Sean Christopherson <sean.j.christopherson@intel.com> wrote: > > On Tue, Mar 10, 2020 at 03:01:19PM +0800, Wanpeng Li wrote: > > From: Wanpeng Li <wanpengli@tencent.com> > > > > kvm_load_guest_fpu() and kvm_put_guest_fpu() each consume more than 14us > > observed by ftrace, the qemu userspace FPU is swapped out for the guest > > FPU context for the duration of the KVM_RUN ioctl even if sleeping AP, > > we shouldn't load/put guest FPU context for this case especially for > > serverless scenario which sensitives to boot time. > > > > Signed-off-by: Wanpeng Li <wanpengli@tencent.com> > > --- > > arch/x86/kvm/x86.c | 8 +++++--- > > 1 file changed, 5 insertions(+), 3 deletions(-) > > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > index 5de2006..080ffa4 100644 > > --- a/arch/x86/kvm/x86.c > > +++ b/arch/x86/kvm/x86.c > > @@ -8680,7 +8680,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > > > > vcpu_load(vcpu); > > kvm_sigset_activate(vcpu); > > - kvm_load_guest_fpu(vcpu); > > > > if (unlikely(vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED)) { > > if (kvm_run->immediate_exit) { > > @@ -8718,12 +8717,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > > } > > } > > > > + kvm_load_guest_fpu(vcpu); > > Ugh, so this isn't safe on MPX capable CPUs, kvm_apic_accept_events() can > trigger kvm_vcpu_reset() with @init_event=true and try to unload guest_fpu. Right. > > We could hack around that issue, but it'd be ugly, and I'm also concerned > that calling vmx_vcpu_reset() without guest_fpu loaded will be problematic > in the future with all the things that are getting managed by XSAVE. > > > + > > if (unlikely(vcpu->arch.complete_userspace_io)) { > > int (*cui)(struct kvm_vcpu *) = vcpu->arch.complete_userspace_io; > > vcpu->arch.complete_userspace_io = NULL; > > r = cui(vcpu); > > if (r <= 0) > > - goto out; > > + goto out_fpu; > > } else > > WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed); > > > > @@ -8732,8 +8733,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) > > else > > r = vcpu_run(vcpu); > > > > -out: > > +out_fpu: > > kvm_put_guest_fpu(vcpu); > > +out: > > if (vcpu->run->kvm_valid_regs) > > store_regs(vcpu); > > post_kvm_run_save(vcpu); > > -- > > 2.7.4 > >
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5de2006..080ffa4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8680,7 +8680,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) vcpu_load(vcpu); kvm_sigset_activate(vcpu); - kvm_load_guest_fpu(vcpu); if (unlikely(vcpu->arch.mp_state == KVM_MP_STATE_UNINITIALIZED)) { if (kvm_run->immediate_exit) { @@ -8718,12 +8717,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) } } + kvm_load_guest_fpu(vcpu); + if (unlikely(vcpu->arch.complete_userspace_io)) { int (*cui)(struct kvm_vcpu *) = vcpu->arch.complete_userspace_io; vcpu->arch.complete_userspace_io = NULL; r = cui(vcpu); if (r <= 0) - goto out; + goto out_fpu; } else WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed); @@ -8732,8 +8733,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) else r = vcpu_run(vcpu); -out: +out_fpu: kvm_put_guest_fpu(vcpu); +out: if (vcpu->run->kvm_valid_regs) store_regs(vcpu); post_kvm_run_save(vcpu);