[RFC,for-4.20,v1,0/1] x86/hvm: Introduce Xen-wide ASID allocator

Message

Vaishali Thakkar May 24, 2024, 12:31 p.m. UTC

Motivation:
-----------
This is part of the effort to enable AMD SEV technologies in Xen. For
AMD SEV support, we need a fixed ASID associated with all vcpus of the
same domain throughout the domain's lifetime. This is because for SEV/
SEV-{ES,SNP} VM, the ASID is the index which is associated with the
encryption key.

Currently, ASID generation and management is done per-PCPU in Xen. And
at the time of each VMENTER, the ASID associated with vcpus of the
domain is changed. This implementation is incompatible with SEV
technologies for the above mentioned reasons. In a discussion with
Andrew Cooper, it came up that it’ll be nice to have fixed ASIDs not
only for SEV VMs but also for all VMs. Because it opens up the
opportunity to use instructions like TLBSYNC and INVLPGB (Section
5.5.3 in AMD Architecture manual[0]) for broadcasting the TLB
Invalidations.

Why is this RFC?
----------------
This is only tested on AMD SVM at the moment. There are a few points
that I would like to discuss and get a feedback on from the community
before further development and testing. I’ve also submitted a design
session for this RFC to discuss further at the Xen Summit.

Points of discussion:
---------------------
1. I’m not sure how this should be handled for the nestedhvm. To start
with, at the moment all the values seem to be handled via struct
nestedvcpu. Do we want to keep it that way or do we want to have
something like nestedhvm_domain to associate certain values like asid?
I’ve not handled this as part of this RFC as I would like to know the
opinions and plans of those working on nested virtualization.

2. I’m doing initialization of xen-wide asids at the moment in setup.c
but is that the right place to do it? I’m asking this because I’ve
been seeing a weird bug with the code in this RFC. Dom0 is able to
have a fixed asid through the lifecycle of it. But if I start a domU
with 2/4 vcpus via xl, sometimes it only brings up the one vcpu and
shows ‘tsc: Unable to calibrate against PIT’ while booting the kernel.

Notes:
-----
1. Currently the RFC doesn’t demonstrate the use of TLBSYNC and INVLPGB.
It can further be added if required. I'm not sure if it should be part
of the same patch series or not.

2. This is a basic RFC to start the discussion on the above points but
I further plan to add a logic to reclaim the asids that are no longer
in use and add a check to pick the asid from such stack before doing
hvm_asid_flush_all.

[0] https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/24593.pdf



Vaishali Thakkar (1):
  x86/hvm: Introduce Xen-wide ASID Allocator

 xen/arch/x86/flushtlb.c                |  1 -
 xen/arch/x86/hvm/asid.c                | 61 ++++++++++++++------------
 xen/arch/x86/hvm/emulate.c             |  3 --
 xen/arch/x86/hvm/hvm.c                 |  2 +-
 xen/arch/x86/hvm/nestedhvm.c           |  4 +-
 xen/arch/x86/hvm/svm/asid.c            | 28 +++++++-----
 xen/arch/x86/hvm/svm/nestedsvm.c       |  2 +-
 xen/arch/x86/hvm/svm/svm.c             | 33 ++++++--------
 xen/arch/x86/hvm/svm/svm.h             |  1 -
 xen/arch/x86/hvm/vmx/vmcs.c            |  2 +-
 xen/arch/x86/hvm/vmx/vmx.c             | 15 +++----
 xen/arch/x86/hvm/vmx/vvmx.c            |  6 +--
 xen/arch/x86/include/asm/hvm/asid.h    | 19 ++++----
 xen/arch/x86/include/asm/hvm/domain.h  |  1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  2 +-
 xen/arch/x86/include/asm/hvm/svm/svm.h |  1 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  6 +--
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  3 +-
 xen/arch/x86/mm/hap/hap.c              |  4 +-
 xen/arch/x86/mm/p2m.c                  |  6 +--
 xen/arch/x86/mm/paging.c               |  2 +-
 xen/arch/x86/setup.c                   |  7 +++
 22 files changed, 108 insertions(+), 101 deletions(-)