| Message ID | 20201126155421.14901-7-dbrazdil@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Opt-in always-on nVHE hypervisor | expand |
On Thu, Nov 26, 2020 at 03:54:04PM +0000, David Brazdil wrote: > Add an early parameter that allows users to opt into protected KVM mode > when using the nVHE hypervisor. In this mode, guest state will be kept > private from the host. This will primarily involve enabling stage-2 > address translation for the host, restricting DMA to host memory, and > filtering host SMCs. > > Capability ARM64_PROTECTED_KVM is set if the param is passed, CONFIG_KVM > is enabled and the kernel was not booted with VHE. > > Signed-off-by: David Brazdil <dbrazdil@google.com> > --- > .../admin-guide/kernel-parameters.txt | 5 ++++ > arch/arm64/include/asm/cpucaps.h | 3 +- > arch/arm64/include/asm/virt.h | 8 +++++ > arch/arm64/kernel/cpufeature.c | 29 +++++++++++++++++++ > arch/arm64/kvm/arm.c | 4 ++- > 5 files changed, 47 insertions(+), 2 deletions(-) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index 526d65d8573a..06c89975c29c 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -2259,6 +2259,11 @@ > for all guests. > Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. > > + kvm-arm.protected= > + [KVM,ARM] Allow spawning protected guests whose state > + is kept private from the host. Only valid for non-VHE. > + Default is 0 (disabled). > + Sorry for being pedantic. Can we reword this to say valid for !CONFIG_ARM64_VHE ? I read this as valid only for non-VHE hardware, it may be just me, but if you agree please update so that it doesn't give remote idea that it is not valid on VHE enabled hardware. I was trying to run this on the hardware and was trying to understand the details on how to do that.
Hey Sudeep, > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > index 526d65d8573a..06c89975c29c 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -2259,6 +2259,11 @@ > > for all guests. > > Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. > > > > + kvm-arm.protected= > > + [KVM,ARM] Allow spawning protected guests whose state > > + is kept private from the host. Only valid for non-VHE. > > + Default is 0 (disabled). > > + > > Sorry for being pedantic. Can we reword this to say valid for > !CONFIG_ARM64_VHE ? I read this as valid only for non-VHE hardware, it may > be just me, but if you agree please update so that it doesn't give remote > idea that it is not valid on VHE enabled hardware. > > I was trying to run this on the hardware and was trying to understand the > details on how to do that. I see what you're saying, but !CONFIG_ARM64_VHE isn't accurate either. The option makes sense if: 1) all cores booted in EL2 == is_hyp_mode_available() 2) ID_AA64MMFR1_EL1.VH=0 or !CONFIG_ARM64_VHE == !is_kernel_in_hyp_mode() The former feels implied for KVM, the latter could be 'Valid if the kernel is running in EL1'? WDYT? -David
On Tue, Dec 01, 2020 at 01:19:13PM +0000, David Brazdil wrote: > Hey Sudeep, > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > > index 526d65d8573a..06c89975c29c 100644 > > > --- a/Documentation/admin-guide/kernel-parameters.txt > > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > > @@ -2259,6 +2259,11 @@ > > > for all guests. > > > Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. > > > > > > + kvm-arm.protected= > > > + [KVM,ARM] Allow spawning protected guests whose state > > > + is kept private from the host. Only valid for non-VHE. > > > + Default is 0 (disabled). > > > + > > > > Sorry for being pedantic. Can we reword this to say valid for > > !CONFIG_ARM64_VHE ? I read this as valid only for non-VHE hardware, it may > > be just me, but if you agree please update so that it doesn't give remote > > idea that it is not valid on VHE enabled hardware. > > > > I was trying to run this on the hardware and was trying to understand the > > details on how to do that. > > I see what you're saying, but !CONFIG_ARM64_VHE isn't accurate either. The > option makes sense if: > 1) all cores booted in EL2 > == is_hyp_mode_available() > 2) ID_AA64MMFR1_EL1.VH=0 or !CONFIG_ARM64_VHE > == !is_kernel_in_hyp_mode() > > The former feels implied for KVM, the latter could be 'Valid if the kernel > is running in EL1'? WDYT? I reckon we can avoid the restriction if we instead add an early stub like with have for KASLR. That way we could parse the command line early, and if necessary re-initialize EL2 and drop to EL1 before the main kernel has to make any decisions about how to initialize things. That would allow us to have a more general kvm-arm.mode option where a single kernel Image could support: * "protected" mode on nVHE or VHE HW * "nvhe" mode on nVHE or VHE HW * "vhe" mode on VHE HW ... defaulting to VHE/nVHE modes depending on HW support. That would also be somewhat future-proof if we have to add other variants of protected mode in future, as we could extend the mode option with parameters for each mode. Thanks, Mark.
> > > be just me, but if you agree please update so that it doesn't give remote > > > idea that it is not valid on VHE enabled hardware. > > > > > > I was trying to run this on the hardware and was trying to understand the > > > details on how to do that. > > > > I see what you're saying, but !CONFIG_ARM64_VHE isn't accurate either. The > > option makes sense if: > > 1) all cores booted in EL2 > > == is_hyp_mode_available() > > 2) ID_AA64MMFR1_EL1.VH=0 or !CONFIG_ARM64_VHE > > == !is_kernel_in_hyp_mode() > > > > The former feels implied for KVM, the latter could be 'Valid if the kernel > > is running in EL1'? WDYT? > > I reckon we can avoid the restriction if we instead add an early stub > like with have for KASLR. That way we could parse the command line > early, and if necessary re-initialize EL2 and drop to EL1 before the > main kernel has to make any decisions about how to initialize things. > That would allow us to have a more general kvm-arm.mode option where a > single kernel Image could support: > > * "protected" mode on nVHE or VHE HW > * "nvhe" mode on nVHE or VHE HW > * "vhe" mode on VHE HW > > ... defaulting to VHE/nVHE modes depending on HW support. > > That would also be somewhat future-proof if we have to add other > variants of protected mode in future, as we could extend the mode option > with parameters for each mode. Agreed that 'mode' is a more future-proof flag and I would very much love to have an option to force nVHE on VHE HW. I however expect that the early stub would not be a trivial addition and would not want to get into that in this series. Could we agree on 'protected' as the only supported value for the time being? David
On Tue, Dec 01, 2020 at 02:43:49PM +0000, David Brazdil wrote: > > > > be just me, but if you agree please update so that it doesn't give remote > > > > idea that it is not valid on VHE enabled hardware. > > > > > > > > I was trying to run this on the hardware and was trying to understand the > > > > details on how to do that. > > > > > > I see what you're saying, but !CONFIG_ARM64_VHE isn't accurate either. The > > > option makes sense if: > > > 1) all cores booted in EL2 > > > == is_hyp_mode_available() > > > 2) ID_AA64MMFR1_EL1.VH=0 or !CONFIG_ARM64_VHE > > > == !is_kernel_in_hyp_mode() > > > > > > The former feels implied for KVM, the latter could be 'Valid if the kernel > > > is running in EL1'? WDYT? > > > > I reckon we can avoid the restriction if we instead add an early stub > > like with have for KASLR. That way we could parse the command line > > early, and if necessary re-initialize EL2 and drop to EL1 before the > > main kernel has to make any decisions about how to initialize things. > > That would allow us to have a more general kvm-arm.mode option where a > > single kernel Image could support: > > > > * "protected" mode on nVHE or VHE HW > > * "nvhe" mode on nVHE or VHE HW > > * "vhe" mode on VHE HW > > > > ... defaulting to VHE/nVHE modes depending on HW support. > > > > That would also be somewhat future-proof if we have to add other > > variants of protected mode in future, as we could extend the mode option > > with parameters for each mode. > > Agreed that 'mode' is a more future-proof flag and I would very much love to > have an option to force nVHE on VHE HW. I however expect that the early stub > would not be a trivial addition and would not want to get into that in this > series. Could we agree on 'protected' as the only supported value for the time > being? Sure, that works for me. Thanks, Mark.
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 526d65d8573a..06c89975c29c 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2259,6 +2259,11 @@ for all guests. Default is 1 (enabled) if in 64-bit or 32-bit PAE mode. + kvm-arm.protected= + [KVM,ARM] Allow spawning protected guests whose state + is kept private from the host. Only valid for non-VHE. + Default is 0 (disabled). + kvm-arm.vgic_v3_group0_trap= [KVM,ARM] Trap guest accesses to GICv3 group-0 system registers diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 162539d4c8cd..9fab6cbffce2 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -66,7 +66,8 @@ #define ARM64_HAS_TLB_RANGE 56 #define ARM64_MTE 57 #define ARM64_WORKAROUND_1508412 58 +#define ARM64_PROTECTED_KVM 59 -#define ARM64_NCAPS 59 +#define ARM64_NCAPS 60 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 6069be50baf9..2fde1186b962 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -97,6 +97,14 @@ static __always_inline bool has_vhe(void) return cpus_have_final_cap(ARM64_HAS_VIRT_HOST_EXTN); } +static __always_inline bool is_protected_kvm_enabled(void) +{ + if (is_vhe_hyp_code()) + return false; + else + return cpus_have_final_cap(ARM64_PROTECTED_KVM); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6f36c4f62f69..dd5bc0f0cf0d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1709,6 +1709,29 @@ static void cpu_enable_mte(struct arm64_cpu_capabilities const *cap) } #endif /* CONFIG_ARM64_MTE */ +#ifdef CONFIG_KVM +static bool enable_protected_kvm; + +static bool has_protected_kvm(const struct arm64_cpu_capabilities *entry, int __unused) +{ + if (!enable_protected_kvm) + return false; + + if (is_kernel_in_hyp_mode()) { + pr_warn("Protected KVM not available with VHE\n"); + return false; + } + + return true; +} + +static int __init early_protected_kvm_cfg(char *buf) +{ + return strtobool(buf, &enable_protected_kvm); +} +early_param("kvm-arm.protected", early_protected_kvm_cfg); +#endif /* CONFIG_KVM */ + /* Internal helper functions to match cpu capability type */ static bool cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap) @@ -1822,6 +1845,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .field_pos = ID_AA64PFR0_EL1_SHIFT, .min_field_value = ID_AA64PFR0_EL1_32BIT_64BIT, }, + { + .desc = "Protected KVM", + .capability = ARM64_PROTECTED_KVM, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_protected_kvm, + }, #endif { .desc = "Kernel page table isolation (KPTI)", diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 2d0a37c75cda..b25035dc0478 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1818,7 +1818,9 @@ int kvm_arch_init(void *opaque) if (err) goto out_hyp; - if (in_hyp_mode) + if (is_protected_kvm_enabled()) + kvm_info("Protected nVHE mode initialized successfully\n"); + else if (in_hyp_mode) kvm_info("VHE mode initialized successfully\n"); else kvm_info("Hyp mode initialized successfully\n");
Add an early parameter that allows users to opt into protected KVM mode when using the nVHE hypervisor. In this mode, guest state will be kept private from the host. This will primarily involve enabling stage-2 address translation for the host, restricting DMA to host memory, and filtering host SMCs. Capability ARM64_PROTECTED_KVM is set if the param is passed, CONFIG_KVM is enabled and the kernel was not booted with VHE. Signed-off-by: David Brazdil <dbrazdil@google.com> --- .../admin-guide/kernel-parameters.txt | 5 ++++ arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/virt.h | 8 +++++ arch/arm64/kernel/cpufeature.c | 29 +++++++++++++++++++ arch/arm64/kvm/arm.c | 4 ++- 5 files changed, 47 insertions(+), 2 deletions(-)