[v3,00/10] Independent per-CPU data section for nVHE

Message

David Brazdil Sept. 16, 2020, 5:34 p.m. UTC

Introduce '.hyp.data..percpu' as part of ongoing effort to make nVHE
hyp code self-contained and independent of the rest of the kernel.

Main benefits:
 * independent nVHE per-CPU data section that can be unmapped from host,
 * more robust linking of nVHE hyp code,
 * no need for hyp-specific macros to access per-CPU variables.

The series is structured as follows:

 - patch 1: Improve existing hyp build rules. This could be sent and merged
    independently of per-CPU but this series builds on it.

 - patches 2-3: Minor cleanups.

 - patches 4-5: Replace hyp helpers for accessing per-CPU variables
     with common helpers modified to work correctly in hyp. Per-CPU
     variables can now be accessed with one API anywhere.

 - patches 6-8: Where VHE and nVHE use per-CPU variables defined in
     kernel proper, move their definitions to hyp/ where they are
     duplicated and owned by VHE/nVHE, respectively. Non-VHE hyp code
     now refers only to per-CPU variables defined in its source files.
     Helpers are added so that kernel proper can continue to access
     nVHE hyp variables, same way as it does with other nVHE symbols.

 - patches 9-11: Introduce '.hyp.data..percpu' ELF section and allocate
     memory for every CPU core during KVM init. All nVHE per-CPU state
     is now grouped together in ELF and in memory. Introducing a new
     per-CPU variable does not require adding new memory mappings any
     more. nVHE hyp code cannot accidentally refer to kernel-proper
     per-CPU data as it only has the pointer to its own per-CPU memory.

Patches are rebased on v5.9-rc5 and available in branch 'topic/percpu-v3' at:
    https://android-kvm.googlesource.com/linux

For maintainers: In case of interest, there are patches that remove the need
for redefining macros under DEBUG_PREEMPT available at the same repo, branch
'topic/percpu-v3-debug-preempt'. Since they are non-trivial, I am not going
to post them here so late in the 5.10 window. I plan to post them for 5.11
when they will also be useful for other patches.

Changes v2 -> v3:
 * Use PERCPU_INPUT in hyp.ld instead of modifying PERCPU_SECTION
 * Only pass linker script once to LD (fix error message)
 * Renamed '.hyp.o' to '.nvhe.o'
 * Use __KVM_VHE_HYPERVISOR__ to select TPIDR_EL2 instead of alternatives
 * Move all prefixing-related macros to hyp_image.h

Changes v1 -> v2:
 * partially link hyp code, add linker script

David Brazdil (11):
  kvm: arm64: Partially link nVHE hyp code, simplify HYPCOPY
  kvm: arm64: Move nVHE hyp namespace macros to hyp_image.h
  kvm: arm64: Only define __kvm_ex_table for CONFIG_KVM
  kvm: arm64: Remove __hyp_this_cpu_read
  kvm: arm64: Remove hyp_adr/ldr_this_cpu
  kvm: arm64: Add helpers for accessing nVHE hyp per-cpu vars
  kvm: arm64: Duplicate arm64_ssbd_callback_required for nVHE hyp
  kvm: arm64: Create separate instances of kvm_host_data for VHE/nVHE
  kvm: arm64: Mark hyp stack pages reserved
  kvm: arm64: Set up hyp percpu data for nVHE
  kvm: arm64: Remove unnecessary hyp mappings

 arch/arm64/include/asm/assembler.h        |  27 ++++--
 arch/arm64/include/asm/hyp_image.h        |  36 +++++++
 arch/arm64/include/asm/kvm_asm.h          |  82 ++++++++--------
 arch/arm64/include/asm/kvm_host.h         |   2 +-
 arch/arm64/include/asm/kvm_mmu.h          |  19 ++--
 arch/arm64/include/asm/percpu.h           |  33 ++++++-
 arch/arm64/include/asm/sections.h         |   1 +
 arch/arm64/kernel/image-vars.h            |   4 -
 arch/arm64/kernel/vmlinux.lds.S           |  13 +++
 arch/arm64/kvm/arm.c                      | 109 +++++++++++++++++++---
 arch/arm64/kvm/hyp/hyp-entry.S            |   2 +-
 arch/arm64/kvm/hyp/include/hyp/debug-sr.h |   4 +-
 arch/arm64/kvm/hyp/include/hyp/switch.h   |   8 +-
 arch/arm64/kvm/hyp/nvhe/Makefile          |  60 ++++++------
 arch/arm64/kvm/hyp/nvhe/hyp.lds.S         |  19 ++++
 arch/arm64/kvm/hyp/nvhe/switch.c          |   8 +-
 arch/arm64/kvm/hyp/vhe/switch.c           |   5 +-
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c        |   4 +-
 arch/arm64/kvm/pmu.c                      |  13 ++-
 19 files changed, 320 insertions(+), 129 deletions(-)
 create mode 100644 arch/arm64/include/asm/hyp_image.h
 create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp.lds.S