mbox series

[00/13] KVM: x86: Add a cap to disable NX hugepages on a VM

Message ID 20220310164532.1821490-1-bgardon@google.com (mailing list archive)
Headers show
Series KVM: x86: Add a cap to disable NX hugepages on a VM | expand

Message

Ben Gardon March 10, 2022, 4:45 p.m. UTC
Given the high cost of NX hugepages in terms of TLB performance, it may
be desirable to disable the mitigation on a per-VM basis. In the case of public
cloud providers with many VMs on a single host, some VMs may be more trusted
than others. In order to maximize performance on critical VMs, while still
providing some protection to the host from iTLB Multihit, allow the mitigation
to be selectively disabled.

Disabling NX hugepages on a VM is relatively straightforward, but I took this
as an opportunity to add some NX hugepages test coverage and clean up selftests
infrastructure a bit.

Patches 1-2 add some library calls for accessing stats via the binary stats API.
Patches 3-5 improve memslot ID handling in the KVM util library.
Patch 6 is a misc logging improvement.
Patches 7 and 13 implement an NX hugepages test.
Patches 8, 9, 10, and 12 implement disabling NX on a VM.
Patch 11 is a small cleanup of a bad merge.

This series was tested with the new selftest and the rest of the KVM selftests
on an Intel Haswell machine.

The following tests failed, but I do not believe that has anything to do with
this series:
	userspace_io_test
	vmx_nested_tsc_scaling_test
	vmx_preemption_timer_test

Ben Gardon (13):
  selftests: KVM: Dump VM stats in binary stats test
  selftests: KVM: Test reading a single stat
  selftests: KVM: Wrap memslot IDs in a struct for readability
  selftests: KVM: Add memslot parameter to VM vaddr allocation
  selftests: KVM: Add memslot parameter to elf_load
  selftests: KVM: Improve error message in vm_phy_pages_alloc
  selftests: KVM: Add NX huge pages test
  KVM: x86/MMU: Factor out updating NX hugepages state for a VM
  KVM: x86/MMU: Track NX hugepages on a per-VM basis
  KVM: x86/MMU: Allow NX huge pages to be disabled on a per-vm basis
  KVM: x86: Fix errant brace in KVM capability handling
  KVM: x86/MMU: Require reboot permission to disable NX hugepages
  selftests: KVM: Test disabling NX hugepages on a VM

 arch/x86/include/asm/kvm_host.h               |   3 +
 arch/x86/kvm/mmu.h                            |   9 +-
 arch/x86/kvm/mmu/mmu.c                        |  23 +-
 arch/x86/kvm/mmu/spte.c                       |   7 +-
 arch/x86/kvm/mmu/spte.h                       |   3 +-
 arch/x86/kvm/mmu/tdp_mmu.c                    |   3 +-
 arch/x86/kvm/x86.c                            |  24 +-
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../selftests/kvm/aarch64/psci_cpu_on_test.c  |   2 +-
 .../selftests/kvm/dirty_log_perf_test.c       |   7 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |  45 +--
 .../selftests/kvm/hardware_disable_test.c     |   2 +-
 .../selftests/kvm/include/kvm_util_base.h     |  57 ++--
 .../selftests/kvm/include/x86_64/vmx.h        |   4 +-
 .../selftests/kvm/kvm_binary_stats_test.c     |   6 +
 .../selftests/kvm/kvm_page_table_test.c       |   9 +-
 .../selftests/kvm/lib/aarch64/processor.c     |   7 +-
 tools/testing/selftests/kvm/lib/elf.c         |   5 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    | 311 +++++++++++++++---
 .../selftests/kvm/lib/kvm_util_internal.h     |   2 +-
 .../selftests/kvm/lib/perf_test_util.c        |   4 +-
 .../selftests/kvm/lib/riscv/processor.c       |   5 +-
 .../selftests/kvm/lib/s390x/processor.c       |   9 +-
 .../kvm/lib/x86_64/nx_huge_pages_guest.S      |  45 +++
 .../selftests/kvm/lib/x86_64/processor.c      |  11 +-
 tools/testing/selftests/kvm/lib/x86_64/svm.c  |   8 +-
 tools/testing/selftests/kvm/lib/x86_64/vmx.c  |  26 +-
 .../selftests/kvm/max_guest_memory_test.c     |   6 +-
 .../kvm/memslot_modification_stress_test.c    |   6 +-
 .../testing/selftests/kvm/memslot_perf_test.c |  11 +-
 .../selftests/kvm/set_memory_region_test.c    |   8 +-
 tools/testing/selftests/kvm/steal_time.c      |   3 +-
 tools/testing/selftests/kvm/x86_64/amx_test.c |   6 +-
 .../testing/selftests/kvm/x86_64/cpuid_test.c |   2 +-
 .../kvm/x86_64/emulator_error_test.c          |   2 +-
 .../selftests/kvm/x86_64/hyperv_clock.c       |   2 +-
 .../selftests/kvm/x86_64/hyperv_features.c    |   6 +-
 .../selftests/kvm/x86_64/kvm_clock_test.c     |   2 +-
 .../selftests/kvm/x86_64/mmu_role_test.c      |   3 +-
 .../selftests/kvm/x86_64/nx_huge_pages_test.c | 149 +++++++++
 .../kvm/x86_64/nx_huge_pages_test.sh          |  25 ++
 .../selftests/kvm/x86_64/set_boot_cpu_id.c    |   2 +-
 tools/testing/selftests/kvm/x86_64/smm_test.c |   2 +-
 .../selftests/kvm/x86_64/vmx_dirty_log_test.c |  10 +-
 .../selftests/kvm/x86_64/xapic_ipi_test.c     |   2 +-
 .../selftests/kvm/x86_64/xen_shinfo_test.c    |   4 +-
 .../selftests/kvm/x86_64/xen_vmcall_test.c    |   2 +-
 48 files changed, 704 insertions(+), 190 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/nx_huge_pages_guest.S
 create mode 100644 tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.c
 create mode 100755 tools/testing/selftests/kvm/x86_64/nx_huge_pages_test.sh

Comments

Sean Christopherson March 10, 2022, 5:58 p.m. UTC | #1
On Thu, Mar 10, 2022, Ben Gardon wrote:
>   selftests: KVM: Wrap memslot IDs in a struct for readability
>   selftests: KVM: Add memslot parameter to VM vaddr allocation
>   selftests: KVM: Add memslot parameter to elf_load

I really, really, don't want to go down this path of proliferating memslot crud
throughout the virtual memory allocators.  I would much rather we solve this by
teaching the VM creation helpers to (optionally) use hugepages.  The amount of
churn required just so that one test can back code with hugepages is absurd, and
there's bound to be tests in the future that want to force hugepages as well.
Ben Gardon March 10, 2022, 7:29 p.m. UTC | #2
On Thu, Mar 10, 2022 at 11:58 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Thu, Mar 10, 2022, Ben Gardon wrote:
> >   selftests: KVM: Wrap memslot IDs in a struct for readability
> >   selftests: KVM: Add memslot parameter to VM vaddr allocation
> >   selftests: KVM: Add memslot parameter to elf_load
>
> I really, really, don't want to go down this path of proliferating memslot crud
> throughout the virtual memory allocators.  I would much rather we solve this by
> teaching the VM creation helpers to (optionally) use hugepages.  The amount of
> churn required just so that one test can back code with hugepages is absurd, and
> there's bound to be tests in the future that want to force hugepages as well.

I agree that proliferating the memslots argument isn't strictly
required for this test, but doing so makes it much easier to make
assertions about hugepage counts and such because you don't have your
stacks and page tables backed with hugepages.

Those patches are a lot of churn, but at least to me, they make the
code much more readable. Currently there are many functions which just
pass along 0 for the memslot, and often have multiple other numerical
arguments, which makes it hard to understand what the function is
doing.

I don't think explicitly specifying memslots really adds that much
overhead to the tests, and I'd rather have control over that than
implicitly cramming everything into memslot 0.

If you have a better way to manage the memslots and create virtual
mappings for / load code into other memslots, I'm open to it, but we
should do something about it.
Sean Christopherson March 11, 2022, 2:19 a.m. UTC | #3
On Thu, Mar 10, 2022, Ben Gardon wrote:
> Those patches are a lot of churn, but at least to me, they make the
> code much more readable. Currently there are many functions which just
> pass along 0 for the memslot, and often have multiple other numerical
> arguments, which makes it hard to understand what the function is
> doing.

Yeah, my solution for that was to rip out all the params.  E.g. the most used
function I ended up with is

  static inline struct kvm_vm *vm_create_with_one_vcpu(struct kvm_vcpu **vcpu,
						     void *guest_code)
  {
	return __vm_create_with_one_vcpu(vcpu, 0, guest_code);
  }

and then the usage is

	vm = vm_create_with_one_vcpu(&vcpu, guest_main);

	supp_cpuid = kvm_get_supported_cpuid();
	cpuid2 = vcpu_get_cpuid(vcpu);

My overarching complaint with the selftests is that they make the hard things hard,
and the easy things harder.  If a test wants to be backed by hugepages, it shouldn't
have to manually specify a memslot.

Let me post my selftests rework as RFC (_very_ RFC at this point).  I was hoping to
do more than compile test before posting anything, but it's going to be multiple
weeks before I'll get back to it.  Hopefully it'll start a discussion on actually
rewriting the framework so that writing new tests is less painful, and so that every
new thing that comes along doesn't require poking at 50 different tests.
Ben Gardon March 15, 2022, 11:12 p.m. UTC | #4
Okay, I'll hold off on the memslot refactor in v2, but if folks have
feedback on the other aspects of the v1 patch series, I'd appreciate
it.
If not, I'll try to get a v2 sent out.
I think that the commits adding utility functions for the binary stats
interface to the binary stats test could be queued separately from the
rest of this series and will be helpful for other folks working on new
selftests.

On Thu, Mar 10, 2022 at 8:19 PM Sean Christopherson <seanjc@google.com> wrote:
>
> On Thu, Mar 10, 2022, Ben Gardon wrote:
> > Those patches are a lot of churn, but at least to me, they make the
> > code much more readable. Currently there are many functions which just
> > pass along 0 for the memslot, and often have multiple other numerical
> > arguments, which makes it hard to understand what the function is
> > doing.
>
> Yeah, my solution for that was to rip out all the params.  E.g. the most used
> function I ended up with is
>
>   static inline struct kvm_vm *vm_create_with_one_vcpu(struct kvm_vcpu **vcpu,
>                                                      void *guest_code)
>   {
>         return __vm_create_with_one_vcpu(vcpu, 0, guest_code);
>   }
>
> and then the usage is
>
>         vm = vm_create_with_one_vcpu(&vcpu, guest_main);
>
>         supp_cpuid = kvm_get_supported_cpuid();
>         cpuid2 = vcpu_get_cpuid(vcpu);
>
> My overarching complaint with the selftests is that they make the hard things hard,
> and the easy things harder.  If a test wants to be backed by hugepages, it shouldn't
> have to manually specify a memslot.
>
> Let me post my selftests rework as RFC (_very_ RFC at this point).  I was hoping to
> do more than compile test before posting anything, but it's going to be multiple
> weeks before I'll get back to it.  Hopefully it'll start a discussion on actually
> rewriting the framework so that writing new tests is less painful, and so that every
> new thing that comes along doesn't require poking at 50 different tests.