[v4,0/3] KVM: SVM: Refactor vcpu_load/put to use vmload/vmsave for host state

Message

Michael Roth Feb. 2, 2021, 7:01 p.m. UTC

Hi Sean, Paolo,

Following up from previous v3 discussion:

  https://lore.kernel.org/kvm/[e-mail]/

I got bit in internal testing by a bug in v3 of this series that Sean had
already pointed out in v3 comments, so I thought it might be good to go
ahead and send a v4 with those fixes included. I also saw that Sean's vmsave
helpers are now in kvm/queue, so I've rebased these on top of those, and
made use of the new vmsave/vmload helpers:

  https://lore.kernel.org/kvm/[e-mail]/

Thanks!

-Mike

= Overview =

This series re-works the SVM KVM implementation to use vmload/vmsave to
handle saving/restoring additional host MSRs rather than explicit MSR
read/writes, resulting in a significant performance improvement for some
specific workloads and simplifying some of the save/load code (PATCH 1).

With those changes some commonalities emerge between SEV-ES and normal
vcpu_load/vcpu_put paths, which we then take advantage of to share more code,
as well as refactor them in a way that more closely aligns with the VMX
implementation (PATCH 2 and 3).

v4:
 - rebased on kvm/queue
 - use sme_page_pa() when accessing save area (Sean)
 - make sure vmload during host reboot is handled (Sean)
 - introduce vmload() helper like we have with vmsave(), use that instead
   of moving the introduce to ASM (Sean)

v3:
 - rebased on kvm-next
 - remove uneeded braces from host MSR save/load loops (Sean)
 - use page_to_phys() in place of page_to_pfn() and shifting (Sean)
 - use stack instead of struct field to cache host save area outside of
   per-cpu storage, and pass as an argument to __svm_vcpu_run() to
   handle the VMLOAD in ASM code rather than inlining ASM (Sean/Andy)
 - remove now-uneeded index/sev_es_restored fields from
   host_save_user_msrs list
 - move host-saving/guest-loading of registers to prepare_guest_switch(),
   and host-loading of registers to prepare_host_switch, for both normal
   and sev-es paths (Sean)

v2:
 - rebase on latest kvm/next
 - move VMLOAD to just after vmexit so we can use it to handle all FS/GS
   host state restoration and rather than relying on loadsegment() and
   explicit write to MSR_GS_BASE (Andy)
 - drop 'host' field from struct vcpu_svm since it is no longer needed
   for storing FS/GS/LDT state (Andy)

 arch/x86/kvm/svm/sev.c     |  30 +-----------------------------
 arch/x86/kvm/svm/svm.c     | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------
 arch/x86/kvm/svm/svm.h     |  29 +++++------------------------
 arch/x86/kvm/svm/svm_ops.h |   5 +++++
 4 files changed, 67 insertions(+), 104 deletions(-)

Comments

Paolo Bonzini Feb. 3, 2021, 8:10 a.m. UTC | #1

On 02/02/21 20:01, Michael Roth wrote:
> Hi Sean, Paolo,
> 
> Following up from previous v3 discussion:
> 
>    https://lore.kernel.org/kvm/[e-mail]/
> 
> I got bit in internal testing by a bug in v3 of this series that Sean had
> already pointed out in v3 comments, so I thought it might be good to go
> ahead and send a v4 with those fixes included. I also saw that Sean's vmsave
> helpers are now in kvm/queue, so I've rebased these on top of those, and
> made use of the new vmsave/vmload helpers:
> 
>    https://lore.kernel.org/kvm/[e-mail]/
> 
> Thanks!
> 
> -Mike
> 
> = Overview =
> 
> This series re-works the SVM KVM implementation to use vmload/vmsave to
> handle saving/restoring additional host MSRs rather than explicit MSR
> read/writes, resulting in a significant performance improvement for some
> specific workloads and simplifying some of the save/load code (PATCH 1).
> 
> With those changes some commonalities emerge between SEV-ES and normal
> vcpu_load/vcpu_put paths, which we then take advantage of to share more code,
> as well as refactor them in a way that more closely aligns with the VMX
> implementation (PATCH 2 and 3).

Queued, thanks.

Paolo

> v4:
>   - rebased on kvm/queue
>   - use sme_page_pa() when accessing save area (Sean)
>   - make sure vmload during host reboot is handled (Sean)
>   - introduce vmload() helper like we have with vmsave(), use that instead
>     of moving the introduce to ASM (Sean)
> 
> v3:
>   - rebased on kvm-next
>   - remove uneeded braces from host MSR save/load loops (Sean)
>   - use page_to_phys() in place of page_to_pfn() and shifting (Sean)
>   - use stack instead of struct field to cache host save area outside of
>     per-cpu storage, and pass as an argument to __svm_vcpu_run() to
>     handle the VMLOAD in ASM code rather than inlining ASM (Sean/Andy)
>   - remove now-uneeded index/sev_es_restored fields from
>     host_save_user_msrs list
>   - move host-saving/guest-loading of registers to prepare_guest_switch(),
>     and host-loading of registers to prepare_host_switch, for both normal
>     and sev-es paths (Sean)
> 
> v2:
>   - rebase on latest kvm/next
>   - move VMLOAD to just after vmexit so we can use it to handle all FS/GS
>     host state restoration and rather than relying on loadsegment() and
>     explicit write to MSR_GS_BASE (Andy)
>   - drop 'host' field from struct vcpu_svm since it is no longer needed
>     for storing FS/GS/LDT state (Andy)
> 
>   arch/x86/kvm/svm/sev.c     |  30 +-----------------------------
>   arch/x86/kvm/svm/svm.c     | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------
>   arch/x86/kvm/svm/svm.h     |  29 +++++------------------------
>   arch/x86/kvm/svm/svm_ops.h |   5 +++++
>   4 files changed, 67 insertions(+), 104 deletions(-)
> 
>