mbox series

[v3,00/17] KVM: arm64: Preliminary NV patches

Message ID 20200706125425.1671020-1-maz@kernel.org (mailing list archive)
Headers show
Series KVM: arm64: Preliminary NV patches | expand

Message

Marc Zyngier July 6, 2020, 12:54 p.m. UTC
Hi all,

In order not to repeat the 90+ patch series that resulted in a
deafening silence last time, I've extracted a smaller set of patches
that form the required dependencies that allow the rest of the 65 NV
patches to be added on top. Yes, it is that bad.

The one real feature here is support for the ARMv8.4-TTL extension at
Stage-2 only. The reason to support it is that it helps the hypervisor
a lot when it comes to finding out how much to invalidate. It is thus
always "supported" with NV.

The rest doesn't contain any functionality change. Most of it reworks
existing data structures and adds new accessors for the things that
get moved around. The reason for this is that:

- With NV, we end-up with multiple Stage-2 MMU contexts per VM instead
  of a single one. This requires we divorce struct kvm from the S2 MMU
  configuration. Of course, we stick with a single MMU context for now.

- With ARMv8.4-NV, a number of system register accesses are turned
  into memory accesses into the so-called VNCR page. It is thus
  convenient to make this VNCR page part of the vcpu context and avoid
  copying data back and forth. For this to work, we need to make sure
  that all the VNCR-aware sysregs are moved into our per-vcpu sys_regs
  array instead of leaving in other data structures (the timers, for
  example). The VNCR page itself isn't introduced with these patches.

- As some of these data structures change, we need a way to isolate
  the userspace ABI from such change.

There is also a number of cleanups that were in the full fat series
that I decided to move early to get them out of the way.

The whole this is a bit of a mix of vaguely unrelated "stuff", but it
all comes together if you look at the final series. This applies on
top of David Brazdil's series splitting the VHE and nVHE objects.

I plan on taking this early into v5.9, and I really mean it this time!

Catalin: How do you want to proceed for patches 2, 3, and 4? I could
make a stable branch that gets you pull into the arm64 tree, or the
other way around. Just let me know.

Thanks,

	M.

* From v2:
  - Rebased on top of David's el2-obj series
  - Fixed the terrible __kvm_tlb_flush_local_vmid bug
  - Renamed TLBI_TTL_PS_* to TLBI_TTL_TG_* (Alex)
  - Fixed a misleading comment in mmu.c (Alex)
  - Fixed the debug patch commit log
  - Collected Alex's RBs, with thanks.

* From v1:
  - A bunch of patches have been merged. These are the current leftovers.
  - Rebased on top of v5.8-rc1, and it wasn't fun.

Christoffer Dall (1):
  KVM: arm64: Factor out stage 2 page table data from struct kvm

Marc Zyngier (16):
  arm64: Detect the ARMv8.4 TTL feature
  arm64: Document SW reserved PTE/PMD bits in Stage-2 descriptors
  arm64: Add level-hinted TLB invalidation helper
  KVM: arm64: Use TTL hint in when invalidating stage-2 translations
  KVM: arm64: Introduce accessor for ctxt->sys_reg
  KVM: arm64: hyp: Use ctxt_sys_reg/__vcpu_sys_reg instead of raw
    sys_regs access
  KVM: arm64: sve: Use __vcpu_sys_reg() instead of raw sys_regs access
  KVM: arm64: pauth: Use ctxt_sys_reg() instead of raw sys_regs access
  KVM: arm64: debug: Drop useless vpcu parameter
  KVM: arm64: Make struct kvm_regs userspace-only
  KVM: arm64: Move ELR_EL1 to the system register array
  KVM: arm64: Move SP_EL1 to the system register array
  KVM: arm64: Disintegrate SPSR array
  KVM: arm64: Move SPSR_EL1 to the system register array
  KVM: arm64: timers: Rename kvm_timer_sync_hwstate to
    kvm_timer_sync_user
  KVM: arm64: timers: Move timer registers to the sys_regs file

 arch/arm64/include/asm/cpucaps.h           |   3 +-
 arch/arm64/include/asm/kvm_asm.h           |   8 +-
 arch/arm64/include/asm/kvm_emulate.h       |  37 +--
 arch/arm64/include/asm/kvm_host.h          |  71 ++++--
 arch/arm64/include/asm/kvm_mmu.h           |  16 +-
 arch/arm64/include/asm/pgtable-hwdef.h     |   2 +
 arch/arm64/include/asm/stage2_pgtable.h    |   9 +
 arch/arm64/include/asm/sysreg.h            |   1 +
 arch/arm64/include/asm/tlbflush.h          |  45 ++++
 arch/arm64/kernel/asm-offsets.c            |   3 +-
 arch/arm64/kernel/cpufeature.c             |  11 +
 arch/arm64/kvm/arch_timer.c                | 157 +++++++++---
 arch/arm64/kvm/arm.c                       |  40 +--
 arch/arm64/kvm/fpsimd.c                    |   6 +-
 arch/arm64/kvm/guest.c                     |  79 +++++-
 arch/arm64/kvm/hyp/entry.S                 |   3 +-
 arch/arm64/kvm/hyp/include/hyp/debug-sr.h  |  22 +-
 arch/arm64/kvm/hyp/include/hyp/switch.h    |  38 +--
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 152 ++++++-----
 arch/arm64/kvm/hyp/nvhe/switch.c           |   6 +-
 arch/arm64/kvm/hyp/nvhe/tlb.c              |  36 +--
 arch/arm64/kvm/hyp/vhe/switch.c            |   2 +-
 arch/arm64/kvm/hyp/vhe/tlb.c               |  29 ++-
 arch/arm64/kvm/inject_fault.c              |   2 +-
 arch/arm64/kvm/mmu.c                       | 281 ++++++++++++---------
 arch/arm64/kvm/regmap.c                    |  37 ++-
 arch/arm64/kvm/reset.c                     |   2 +-
 arch/arm64/kvm/sys_regs.c                  |   2 +
 arch/arm64/kvm/trace_arm.h                 |   8 +-
 include/kvm/arm_arch_timer.h               |  13 +-
 30 files changed, 696 insertions(+), 425 deletions(-)

Comments

Catalin Marinas July 6, 2020, 5:27 p.m. UTC | #1
On Mon, Jul 06, 2020 at 01:54:08PM +0100, Marc Zyngier wrote:
> Catalin: How do you want to proceed for patches 2, 3, and 4? I could
> make a stable branch that gets you pull into the arm64 tree, or the
> other way around. Just let me know.

Please create a separate branch for the S2 TTL patches (ideally based on
no later than -rc3). I plan to queue the rest of Zhenyu's patches on
top.

https://lore.kernel.org/linux-arm-kernel/20200625080314.230-1-yezhenyu2@huawei.com/

Thanks.
Marc Zyngier July 7, 2020, 8:51 a.m. UTC | #2
Hi Catalin,

On Mon, 06 Jul 2020 18:27:48 +0100,
Catalin Marinas <catalin.marinas@arm.com> wrote:
> 
> On Mon, Jul 06, 2020 at 01:54:08PM +0100, Marc Zyngier wrote:
> > Catalin: How do you want to proceed for patches 2, 3, and 4? I could
> > make a stable branch that gets you pull into the arm64 tree, or the
> > other way around. Just let me know.
> 
> Please create a separate branch for the S2 TTL patches (ideally based on
> no later than -rc3). I plan to queue the rest of Zhenyu's patches on
> top.
> 
> https://lore.kernel.org/linux-arm-kernel/20200625080314.230-1-yezhenyu2@huawei.com/

I've now pushed out this branch[1], containing the three patches in
isolation. Unless you tell me otherwise, I will push this into -next
today, together with the rest of the KVM/arm64 queue.

Thanks,

	M.

[1] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/ttl-for-arm64
Alexandru Elisei July 7, 2020, 11:24 a.m. UTC | #3
Hi,

On 7/6/20 1:54 PM, Marc Zyngier wrote:
> Hi all,
>
> In order not to repeat the 90+ patch series that resulted in a
> deafening silence last time, I've extracted a smaller set of patches
> that form the required dependencies that allow the rest of the 65 NV
> patches to be added on top. Yes, it is that bad.
>
> The one real feature here is support for the ARMv8.4-TTL extension at
> Stage-2 only. The reason to support it is that it helps the hypervisor
> a lot when it comes to finding out how much to invalidate. It is thus
> always "supported" with NV.
>
> The rest doesn't contain any functionality change. Most of it reworks
> existing data structures and adds new accessors for the things that
> get moved around. The reason for this is that:
>
> - With NV, we end-up with multiple Stage-2 MMU contexts per VM instead
>   of a single one. This requires we divorce struct kvm from the S2 MMU
>   configuration. Of course, we stick with a single MMU context for now.
>
> - With ARMv8.4-NV, a number of system register accesses are turned
>   into memory accesses into the so-called VNCR page. It is thus
>   convenient to make this VNCR page part of the vcpu context and avoid
>   copying data back and forth. For this to work, we need to make sure
>   that all the VNCR-aware sysregs are moved into our per-vcpu sys_regs
>   array instead of leaving in other data structures (the timers, for
>   example). The VNCR page itself isn't introduced with these patches.
>
> - As some of these data structures change, we need a way to isolate
>   the userspace ABI from such change.
>
> There is also a number of cleanups that were in the full fat series
> that I decided to move early to get them out of the way.
>
> The whole this is a bit of a mix of vaguely unrelated "stuff", but it
> all comes together if you look at the final series. This applies on
> top of David Brazdil's series splitting the VHE and nVHE objects.
>
> I plan on taking this early into v5.9, and I really mean it this time!
>
> Catalin: How do you want to proceed for patches 2, 3, and 4? I could
> make a stable branch that gets you pull into the arm64 tree, or the
> other way around. Just let me know.
>
> Thanks,
>
> 	M.
>
> * From v2:
>   - Rebased on top of David's el2-obj series

I tried to apply the patches on top of v5.8-rc1, but I get a conflict on the very
first patch. I guess it's because I don't have the el2-obj series. Is that v4 of
"Split off nVHE hyp code" patches from David Brazil?

Thanks,
Alex
Alexandru Elisei July 7, 2020, 11:48 a.m. UTC | #4
Hi,

On 7/7/20 12:24 PM, Alexandru Elisei wrote:
> Hi,
>
> On 7/6/20 1:54 PM, Marc Zyngier wrote:
>> Hi all,
>>
>> In order not to repeat the 90+ patch series that resulted in a
>> deafening silence last time, I've extracted a smaller set of patches
>> that form the required dependencies that allow the rest of the 65 NV
>> patches to be added on top. Yes, it is that bad.
>>
>> The one real feature here is support for the ARMv8.4-TTL extension at
>> Stage-2 only. The reason to support it is that it helps the hypervisor
>> a lot when it comes to finding out how much to invalidate. It is thus
>> always "supported" with NV.
>>
>> The rest doesn't contain any functionality change. Most of it reworks
>> existing data structures and adds new accessors for the things that
>> get moved around. The reason for this is that:
>>
>> - With NV, we end-up with multiple Stage-2 MMU contexts per VM instead
>>   of a single one. This requires we divorce struct kvm from the S2 MMU
>>   configuration. Of course, we stick with a single MMU context for now.
>>
>> - With ARMv8.4-NV, a number of system register accesses are turned
>>   into memory accesses into the so-called VNCR page. It is thus
>>   convenient to make this VNCR page part of the vcpu context and avoid
>>   copying data back and forth. For this to work, we need to make sure
>>   that all the VNCR-aware sysregs are moved into our per-vcpu sys_regs
>>   array instead of leaving in other data structures (the timers, for
>>   example). The VNCR page itself isn't introduced with these patches.
>>
>> - As some of these data structures change, we need a way to isolate
>>   the userspace ABI from such change.
>>
>> There is also a number of cleanups that were in the full fat series
>> that I decided to move early to get them out of the way.
>>
>> The whole this is a bit of a mix of vaguely unrelated "stuff", but it
>> all comes together if you look at the final series. This applies on
>> top of David Brazdil's series splitting the VHE and nVHE objects.
>>
>> I plan on taking this early into v5.9, and I really mean it this time!
>>
>> Catalin: How do you want to proceed for patches 2, 3, and 4? I could
>> make a stable branch that gets you pull into the arm64 tree, or the
>> other way around. Just let me know.
>>
>> Thanks,
>>
>> 	M.
>>
>> * From v2:
>>   - Rebased on top of David's el2-obj series
> I tried to apply the patches on top of v5.8-rc1, but I get a conflict on the very
> first patch. I guess it's because I don't have the el2-obj series. Is that v4 of
> "Split off nVHE hyp code" patches from David Brazil?

Nevermind, figured it out and used your kvm-arm64/el2-obj-4.1 branch as a base.

Thanks,
Alex
Marc Zyngier July 7, 2020, 11:49 a.m. UTC | #5
On 2020-07-07 12:24, Alexandru Elisei wrote:
> Hi,
> 
> On 7/6/20 1:54 PM, Marc Zyngier wrote:
>> Hi all,
>> 
>> In order not to repeat the 90+ patch series that resulted in a
>> deafening silence last time, I've extracted a smaller set of patches
>> that form the required dependencies that allow the rest of the 65 NV
>> patches to be added on top. Yes, it is that bad.
>> 
>> The one real feature here is support for the ARMv8.4-TTL extension at
>> Stage-2 only. The reason to support it is that it helps the hypervisor
>> a lot when it comes to finding out how much to invalidate. It is thus
>> always "supported" with NV.
>> 
>> The rest doesn't contain any functionality change. Most of it reworks
>> existing data structures and adds new accessors for the things that
>> get moved around. The reason for this is that:
>> 
>> - With NV, we end-up with multiple Stage-2 MMU contexts per VM instead
>>   of a single one. This requires we divorce struct kvm from the S2 MMU
>>   configuration. Of course, we stick with a single MMU context for 
>> now.
>> 
>> - With ARMv8.4-NV, a number of system register accesses are turned
>>   into memory accesses into the so-called VNCR page. It is thus
>>   convenient to make this VNCR page part of the vcpu context and avoid
>>   copying data back and forth. For this to work, we need to make sure
>>   that all the VNCR-aware sysregs are moved into our per-vcpu sys_regs
>>   array instead of leaving in other data structures (the timers, for
>>   example). The VNCR page itself isn't introduced with these patches.
>> 
>> - As some of these data structures change, we need a way to isolate
>>   the userspace ABI from such change.
>> 
>> There is also a number of cleanups that were in the full fat series
>> that I decided to move early to get them out of the way.
>> 
>> The whole this is a bit of a mix of vaguely unrelated "stuff", but it
>> all comes together if you look at the final series. This applies on
>> top of David Brazdil's series splitting the VHE and nVHE objects.
>> 
>> I plan on taking this early into v5.9, and I really mean it this time!
>> 
>> Catalin: How do you want to proceed for patches 2, 3, and 4? I could
>> make a stable branch that gets you pull into the arm64 tree, or the
>> other way around. Just let me know.
>> 
>> Thanks,
>> 
>> 	M.
>> 
>> * From v2:
>>   - Rebased on top of David's el2-obj series
> 
> I tried to apply the patches on top of v5.8-rc1, but I get a conflict
> on the very
> first patch. I guess it's because I don't have the el2-obj series. Is 
> that v4 of
> "Split off nVHE hyp code" patches from David Brazil?

You need the slightly amended version (kvm-arm6/el2-obj-v4.1 from my 
tree).
Otherwise, just pick kvmarm/next, which has everything put together
in one scary lot.

Thanks,

         M.