| Message ID | 20230612042559.375660-8-michael.roth@amd.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support | expand |
On 6/11/23 21:25, Michael Roth wrote: > + /* > + * Calculate the amount the memory that must be reserved by the BIOS to > + * address the whole RAM, including the bookkeeping area. The RMP itself > + * must also be covered. > + */ > + max_rmp_pfn = max_pfn; > + if (PHYS_PFN(rmp_end) > max_pfn) > + max_rmp_pfn = PHYS_PFN(rmp_end); Could you say a little here about how this deals with memory hotplug?
On Mon, Jun 12, 2023 at 08:34:02AM -0700, Dave Hansen wrote: > On 6/11/23 21:25, Michael Roth wrote: > > + /* > > + * Calculate the amount the memory that must be reserved by the BIOS to > > + * address the whole RAM, including the bookkeeping area. The RMP itself > > + * must also be covered. > > + */ > > + max_rmp_pfn = max_pfn; > > + if (PHYS_PFN(rmp_end) > max_pfn) > > + max_rmp_pfn = PHYS_PFN(rmp_end); > > Could you say a little here about how this deals with memory hotplug? Does SNP hw even support memory hotplug? I think in order to support that, you'd need some special dance because of the RMP table etc...
On Sun, Jun 11, 2023 at 11:25:15PM -0500, Michael Roth wrote: > From: Brijesh Singh <brijesh.singh@amd.com> > > The memory integrity guarantees of SEV-SNP are enforced through a new > structure called the Reverse Map Table (RMP). The RMP is a single data > structure shared across the system that contains one entry for every 4K > page of DRAM that may be used by SEV-SNP VMs. APM2 section 15.36 details Rather say 'APM v2, section "Secure Nested Paging (SEV-SNP)"' because the numbering is more likely to change than the name in the future. With the name, people can find it faster. > a number of steps needed to detect/enable SEV-SNP and RMP table support > on the host: > > - Detect SEV-SNP support based on CPUID bit > - Initialize the RMP table memory reported by the RMP base/end MSR > registers and configure IOMMU to be compatible with RMP access > restrictions > - Set the MtrrFixDramModEn bit in SYSCFG MSR > - Set the SecureNestedPagingEn and VMPLEn bits in the SYSCFG MSR > - Configure IOMMU > > RMP table entry format is non-architectural and it can vary by > processor. It is defined by the PPR. Restrict SNP support to CPU > models/families which are compatible with the current RMP table entry > format to guard against any undefined behavior when running on other > system types. Future models/support will handle this through an > architectural mechanism to allow for broader compatibility. I'm guessing this is all for live migration between SNP hosts. If so, then there will have to be a guest API to handle the differences. > SNP host code depends on CONFIG_KVM_AMD_SEV config flag, which may be > enabled even when CONFIG_AMD_MEM_ENCRYPT isn't set, so update the > SNP-specific IOMMU helpers used here to rely on CONFIG_KVM_AMD_SEV > instead of CONFIG_AMD_MEM_ENCRYPT. Does that mean that even on CONFIG_AMD_MEM_ENCRYPT=n kernels, host SNP can function? Do we even want that? I'd expect that a host SNP kernel should have SME enabled too even though it is not absolutely necessary. > Co-developed-by: Ashish Kalra <ashish.kalra@amd.com> > Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> > Co-developed-by: Tom Lendacky <thomas.lendacky@amd.com> > Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> > Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> > [mdr: rework commit message to be clearer about what patch does, squash > in early_rmptable_check() handling from Tom] > Signed-off-by: Michael Roth <michael.roth@amd.com> > --- > arch/x86/coco/Makefile | 1 + > arch/x86/coco/sev/Makefile | 3 + > arch/x86/coco/sev/host.c | 212 +++++++++++++++++++++++ > arch/x86/include/asm/disabled-features.h | 8 +- > arch/x86/include/asm/msr-index.h | 11 +- > arch/x86/include/asm/sev.h | 2 + > arch/x86/kernel/cpu/amd.c | 19 ++ > drivers/iommu/amd/init.c | 2 +- > include/linux/amd-iommu.h | 2 +- > 9 files changed, 256 insertions(+), 4 deletions(-) > create mode 100644 arch/x86/coco/sev/Makefile > create mode 100644 arch/x86/coco/sev/host.c Ignored review comments here: https://lore.kernel.org/r/Y9ubi0i4Z750gdMm@zn.tnic Ignoring this one for now too.
On 6/21/23 02:15, Borislav Petkov wrote: > On Mon, Jun 12, 2023 at 08:34:02AM -0700, Dave Hansen wrote: >> On 6/11/23 21:25, Michael Roth wrote: >>> + /* >>> + * Calculate the amount the memory that must be reserved by the BIOS to >>> + * address the whole RAM, including the bookkeeping area. The RMP itself >>> + * must also be covered. >>> + */ >>> + max_rmp_pfn = max_pfn; >>> + if (PHYS_PFN(rmp_end) > max_pfn) >>> + max_rmp_pfn = PHYS_PFN(rmp_end); >> Could you say a little here about how this deals with memory hotplug? > Does SNP hw even support memory hotplug? > > I think in order to support that, you'd need some special dance because > of the RMP table etc... Yep, there's the hardware side and then there are fun nuggets like using mem= and then doing a software-only hot-add later after boot. Also, if the hardware doesn't support any kind of hotplug, it would be great to point to the place in the spec where it says that.
On 6/21/23 04:42, Borislav Petkov wrote: > On Sun, Jun 11, 2023 at 11:25:15PM -0500, Michael Roth wrote: >> From: Brijesh Singh <brijesh.singh@amd.com> >> >> The memory integrity guarantees of SEV-SNP are enforced through a new >> structure called the Reverse Map Table (RMP). The RMP is a single data >> structure shared across the system that contains one entry for every 4K >> page of DRAM that may be used by SEV-SNP VMs. APM2 section 15.36 details > > Rather say 'APM v2, section "Secure Nested Paging (SEV-SNP)"' because > the numbering is more likely to change than the name in the future. With > the name, people can find it faster. > >> a number of steps needed to detect/enable SEV-SNP and RMP table support >> on the host: >> >> - Detect SEV-SNP support based on CPUID bit >> - Initialize the RMP table memory reported by the RMP base/end MSR >> registers and configure IOMMU to be compatible with RMP access >> restrictions >> - Set the MtrrFixDramModEn bit in SYSCFG MSR >> - Set the SecureNestedPagingEn and VMPLEn bits in the SYSCFG MSR >> - Configure IOMMU >> >> RMP table entry format is non-architectural and it can vary by >> processor. It is defined by the PPR. Restrict SNP support to CPU >> models/families which are compatible with the current RMP table entry >> format to guard against any undefined behavior when running on other >> system types. Future models/support will handle this through an >> architectural mechanism to allow for broader compatibility. > > I'm guessing this is all for live migration between SNP hosts. If so, > then there will have to be a guest API to handle the differences. > >> SNP host code depends on CONFIG_KVM_AMD_SEV config flag, which may be >> enabled even when CONFIG_AMD_MEM_ENCRYPT isn't set, so update the >> SNP-specific IOMMU helpers used here to rely on CONFIG_KVM_AMD_SEV >> instead of CONFIG_AMD_MEM_ENCRYPT. > > Does that mean that even on CONFIG_AMD_MEM_ENCRYPT=n kernels, host SNP > can function? Yes, because CONFIG_AMD_MEM_ENCRYPT is mainly for dealing with the encryption bit. > > Do we even want that? We support that today with SEV and SEV-ES guests. The host/hypervisor kernel does not need CONFIG_AMD_MEM_ENCRYPT=y in order to run SEV guests. > > I'd expect that a host SNP kernel should have SME enabled too even > though it is not absolutely necessary. I recommend using TSME over SME. Thanks, Tom > >> Co-developed-by: Ashish Kalra <ashish.kalra@amd.com> >> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> >> Co-developed-by: Tom Lendacky <thomas.lendacky@amd.com> >> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> >> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> >> [mdr: rework commit message to be clearer about what patch does, squash >> in early_rmptable_check() handling from Tom] >> Signed-off-by: Michael Roth <michael.roth@amd.com> >> --- >> arch/x86/coco/Makefile | 1 + >> arch/x86/coco/sev/Makefile | 3 + >> arch/x86/coco/sev/host.c | 212 +++++++++++++++++++++++ >> arch/x86/include/asm/disabled-features.h | 8 +- >> arch/x86/include/asm/msr-index.h | 11 +- >> arch/x86/include/asm/sev.h | 2 + >> arch/x86/kernel/cpu/amd.c | 19 ++ >> drivers/iommu/amd/init.c | 2 +- >> include/linux/amd-iommu.h | 2 +- >> 9 files changed, 256 insertions(+), 4 deletions(-) >> create mode 100644 arch/x86/coco/sev/Makefile >> create mode 100644 arch/x86/coco/sev/host.c > > Ignored review comments here: > > https://lore.kernel.org/r/Y9ubi0i4Z750gdMm@zn.tnic > > Ignoring this one for now too. >
On Wed, Jun 21, 2023 at 07:31:34AM -0700, Dave Hansen wrote: > Yep, there's the hardware side and then there are fun nuggets like using > mem= and then doing a software-only hot-add later after boot. Ah, right, Mike, I think we need to check whether mem= has any effect on SNP. > Also, if the hardware doesn't support any kind of hotplug, it would be > great to point to the place in the spec where it says that. I honestly don't know. But I can't recall ever hearing about hardware memory hotplug so something worth to check too.
Hello Boris, On 6/21/2023 4:42 AM, Borislav Petkov wrote: > On Sun, Jun 11, 2023 at 11:25:15PM -0500, Michael Roth wrote: >> From: Brijesh Singh <brijesh.singh@amd.com> >> >> The memory integrity guarantees of SEV-SNP are enforced through a new >> structure called the Reverse Map Table (RMP). The RMP is a single data >> structure shared across the system that contains one entry for every 4K >> page of DRAM that may be used by SEV-SNP VMs. APM2 section 15.36 details > > Rather say 'APM v2, section "Secure Nested Paging (SEV-SNP)"' because > the numbering is more likely to change than the name in the future. With > the name, people can find it faster. > >> a number of steps needed to detect/enable SEV-SNP and RMP table support >> on the host: >> >> - Detect SEV-SNP support based on CPUID bit >> - Initialize the RMP table memory reported by the RMP base/end MSR >> registers and configure IOMMU to be compatible with RMP access >> restrictions >> - Set the MtrrFixDramModEn bit in SYSCFG MSR >> - Set the SecureNestedPagingEn and VMPLEn bits in the SYSCFG MSR >> - Configure IOMMU >> >> RMP table entry format is non-architectural and it can vary by >> processor. It is defined by the PPR. Restrict SNP support to CPU >> models/families which are compatible with the current RMP table entry >> format to guard against any undefined behavior when running on other >> system types. Future models/support will handle this through an >> architectural mechanism to allow for broader compatibility. > > I'm guessing this is all for live migration between SNP hosts. If so, > then there will have to be a guest API to handle the differences. This is basically for the RMP table entry format/structure definition in arch/x86/coco/sev/host.c, as this is non-architectural it is defined in a .c file instead of a header file, so that the structure remains private (and restricted to that file) to the SNP host code and not exposed to the rest of the kernel. As mentioned in the comments above, future CPU models may support RMP table accesses in an architectural way. > >> SNP host code depends on CONFIG_KVM_AMD_SEV config flag, which may be >> enabled even when CONFIG_AMD_MEM_ENCRYPT isn't set, so update the >> SNP-specific IOMMU helpers used here to rely on CONFIG_KVM_AMD_SEV >> instead of CONFIG_AMD_MEM_ENCRYPT. > > Does that mean that even on CONFIG_AMD_MEM_ENCRYPT=n kernels, host SNP > can function? > Yes, host SNP is supposed to function with CONFIG_AMD_MEM_ENCRYPT=n. CONFIG_AMD_MEM_ENCRYPT=y is needed for SNP guest. > Do we even want that? > > I'd expect that a host SNP kernel should have SME enabled too even > though it is not absolutely necessary. Yes, we typically test host SNP kernel with SME enabled. Thanks, Ashish > >> Co-developed-by: Ashish Kalra <ashish.kalra@amd.com> >> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> >> Co-developed-by: Tom Lendacky <thomas.lendacky@amd.com> >> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> >> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> >> [mdr: rework commit message to be clearer about what patch does, squash >> in early_rmptable_check() handling from Tom] >> Signed-off-by: Michael Roth <michael.roth@amd.com> >> --- >> arch/x86/coco/Makefile | 1 + >> arch/x86/coco/sev/Makefile | 3 + >> arch/x86/coco/sev/host.c | 212 +++++++++++++++++++++++ >> arch/x86/include/asm/disabled-features.h | 8 +- >> arch/x86/include/asm/msr-index.h | 11 +- >> arch/x86/include/asm/sev.h | 2 + >> arch/x86/kernel/cpu/amd.c | 19 ++ >> drivers/iommu/amd/init.c | 2 +- >> include/linux/amd-iommu.h | 2 +- >> 9 files changed, 256 insertions(+), 4 deletions(-) >> create mode 100644 arch/x86/coco/sev/Makefile >> create mode 100644 arch/x86/coco/sev/host.c > > Ignored review comments here: > > https://lore.kernel.org/r/Y9ubi0i4Z750gdMm@zn.tnic > > Ignoring this one for now too. >
On Sun, Jun 11, 2023 at 11:25:15PM -0500, Michael Roth wrote: > From: Brijesh Singh <brijesh.singh@amd.com> > > The memory integrity guarantees of SEV-SNP are enforced through a new > structure called the Reverse Map Table (RMP). The RMP is a single data > structure shared across the system that contains one entry for every 4K > page of DRAM that may be used by SEV-SNP VMs. APM2 section 15.36 details > a number of steps needed to detect/enable SEV-SNP and RMP table support > on the host: > > - Detect SEV-SNP support based on CPUID bit > - Initialize the RMP table memory reported by the RMP base/end MSR > registers and configure IOMMU to be compatible with RMP access > restrictions > - Set the MtrrFixDramModEn bit in SYSCFG MSR > - Set the SecureNestedPagingEn and VMPLEn bits in the SYSCFG MSR > - Configure IOMMU > > RMP table entry format is non-architectural and it can vary by > processor. It is defined by the PPR. Restrict SNP support to CPU > models/families which are compatible with the current RMP table entry > format to guard against any undefined behavior when running on other > system types. Future models/support will handle this through an > architectural mechanism to allow for broader compatibility. > > SNP host code depends on CONFIG_KVM_AMD_SEV config flag, which may be > enabled even when CONFIG_AMD_MEM_ENCRYPT isn't set, so update the > SNP-specific IOMMU helpers used here to rely on CONFIG_KVM_AMD_SEV > instead of CONFIG_AMD_MEM_ENCRYPT. > > Co-developed-by: Ashish Kalra <ashish.kalra@amd.com> > Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> > Co-developed-by: Tom Lendacky <thomas.lendacky@amd.com> > Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> > Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> > [mdr: rework commit message to be clearer about what patch does, squash > in early_rmptable_check() handling from Tom] > Signed-off-by: Michael Roth <michael.roth@amd.com> > --- > arch/x86/coco/Makefile | 1 + > arch/x86/coco/sev/Makefile | 3 + > arch/x86/coco/sev/host.c | 212 +++++++++++++++++++++++ > arch/x86/include/asm/disabled-features.h | 8 +- > arch/x86/include/asm/msr-index.h | 11 +- > arch/x86/include/asm/sev.h | 2 + > arch/x86/kernel/cpu/amd.c | 19 ++ > drivers/iommu/amd/init.c | 2 +- > include/linux/amd-iommu.h | 2 +- > 9 files changed, 256 insertions(+), 4 deletions(-) > create mode 100644 arch/x86/coco/sev/Makefile > create mode 100644 arch/x86/coco/sev/host.c > > diff --git a/arch/x86/coco/Makefile b/arch/x86/coco/Makefile > index 6aa52e719bf5..6a7d876130e2 100644 > --- a/arch/x86/coco/Makefile > +++ b/arch/x86/coco/Makefile > @@ -6,3 +6,4 @@ CFLAGS_core.o += -fno-stack-protector > obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) += core.o > > obj-$(CONFIG_INTEL_TDX_GUEST) += tdx/ > +obj-$(CONFIG_KVM_AMD_SEV) += sev/ > diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile > new file mode 100644 > index 000000000000..27c0500d75c8 > --- /dev/null > +++ b/arch/x86/coco/sev/Makefile > @@ -0,0 +1,3 @@ > +# SPDX-License-Identifier: GPL-2.0 > + > +obj-y += host.o > diff --git a/arch/x86/coco/sev/host.c b/arch/x86/coco/sev/host.c > new file mode 100644 > index 000000000000..6907ce887b23 > --- /dev/null > +++ b/arch/x86/coco/sev/host.c > @@ -0,0 +1,212 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * AMD SVM-SEV Host Support. > + * > + * Copyright (C) 2023 Advanced Micro Devices, Inc. > + * > + * Author: Ashish Kalra <ashish.kalra@amd.com> > + * > + */ > + > +#include <linux/cc_platform.h> > +#include <linux/printk.h> > +#include <linux/mm_types.h> > +#include <linux/set_memory.h> > +#include <linux/memblock.h> > +#include <linux/kernel.h> > +#include <linux/mm.h> > +#include <linux/cpumask.h> > +#include <linux/iommu.h> > +#include <linux/amd-iommu.h> > + > +#include <asm/sev.h> > +#include <asm/processor.h> > +#include <asm/setup.h> > +#include <asm/svm.h> > +#include <asm/smp.h> > +#include <asm/cpu.h> > +#include <asm/apic.h> > +#include <asm/cpuid.h> > +#include <asm/cmdline.h> > +#include <asm/iommu.h> > + > +/* > + * The first 16KB from the RMP_BASE is used by the processor for the > + * bookkeeping, the range needs to be added during the RMP entry lookup. > + */ > +#define RMPTABLE_CPU_BOOKKEEPING_SZ 0x4000 > + > +static unsigned long rmptable_start __ro_after_init; > +static unsigned long rmptable_end __ro_after_init; > + > +#undef pr_fmt > +#define pr_fmt(fmt) "SEV-SNP: " fmt > + > +static int __mfd_enable(unsigned int cpu) > +{ > + u64 val; > + > + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) > + return 0; > + > + rdmsrl(MSR_AMD64_SYSCFG, val); > + > + val |= MSR_AMD64_SYSCFG_MFDM; > + > + wrmsrl(MSR_AMD64_SYSCFG, val); > + > + return 0; > +} > + > +static __init void mfd_enable(void *arg) > +{ > + __mfd_enable(smp_processor_id()); > +} > + > +static int __snp_enable(unsigned int cpu) > +{ > + u64 val; > + > + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) > + return 0; > + > + rdmsrl(MSR_AMD64_SYSCFG, val); > + > + val |= MSR_AMD64_SYSCFG_SNP_EN; > + val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN; > + > + wrmsrl(MSR_AMD64_SYSCFG, val); > + > + return 0; > +} > + > +static __init void snp_enable(void *arg) > +{ > + __snp_enable(smp_processor_id()); > +} > + > +bool snp_get_rmptable_info(u64 *start, u64 *len) > +{ > + u64 max_rmp_pfn, calc_rmp_sz, rmp_sz, rmp_base, rmp_end; > + > + rdmsrl(MSR_AMD64_RMP_BASE, rmp_base); > + rdmsrl(MSR_AMD64_RMP_END, rmp_end); > + > + if (!rmp_base || !rmp_end) { > + pr_err("Memory for the RMP table has not been reserved by BIOS\n"); > + return false; > + } > + > + rmp_sz = rmp_end - rmp_base + 1; > + > + /* > + * Calculate the amount the memory that must be reserved by the BIOS to > + * address the whole RAM, including the bookkeeping area. The RMP itself > + * must also be covered. > + */ > + max_rmp_pfn = max_pfn; > + if (PHYS_PFN(rmp_end) > max_pfn) > + max_rmp_pfn = PHYS_PFN(rmp_end); > + > + calc_rmp_sz = (max_rmp_pfn << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ; > + > + if (calc_rmp_sz > rmp_sz) { > + pr_err("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n", > + calc_rmp_sz, rmp_sz); > + return false; > + } > + > + *start = rmp_base; > + *len = rmp_sz; > + > + return true; > +} > + > +static __init int __snp_rmptable_init(void) > +{ > + u64 rmp_base, sz; > + void *start; > + u64 val; > + > + if (!snp_get_rmptable_info(&rmp_base, &sz)) > + return 1; > + > + pr_info("RMP table physical address [0x%016llx - 0x%016llx]\n", > + rmp_base, rmp_base + sz - 1); > + > + start = memremap(rmp_base, sz, MEMREMAP_WB); > + if (!start) { > + pr_err("Failed to map RMP table addr 0x%llx size 0x%llx\n", rmp_base, sz); > + return 1; > + } > + > + /* > + * Check if SEV-SNP is already enabled, this can happen in case of > + * kexec boot. > + */ > + rdmsrl(MSR_AMD64_SYSCFG, val); > + if (val & MSR_AMD64_SYSCFG_SNP_EN) > + goto skip_enable; > + > + /* Initialize the RMP table to zero */ > + memset(start, 0, sz); > + > + /* Flush the caches to ensure that data is written before SNP is enabled. */ > + wbinvd_on_all_cpus(); > + > + /* MFDM must be enabled on all the CPUs prior to enabling SNP. */ > + on_each_cpu(mfd_enable, NULL, 1); > + > + /* Enable SNP on all CPUs. */ > + on_each_cpu(snp_enable, NULL, 1); > + > +skip_enable: > + rmptable_start = (unsigned long)start; > + rmptable_end = rmptable_start + sz - 1; > + > + return 0; > +} > + > +static int __init snp_rmptable_init(void) > +{ > + int family, model; > + > + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) > + return 0; > + > + family = boot_cpu_data.x86; > + model = boot_cpu_data.x86_model; > + > + /* > + * RMP table entry format is not architectural and it can vary by processor and > + * is defined by the per-processor PPR. Restrict SNP support on the known CPU > + * model and family for which the RMP table entry format is currently defined for. > + */ > + if (!(family == 0x19 && model <= 0xaf) && !(family == 0x1a && model <= 0xf)) > + goto nosnp; > + > + if (amd_iommu_snp_enable()) > + goto nosnp; > + > + if (__snp_rmptable_init()) > + goto nosnp; > + > + cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/rmptable_init:online", __snp_enable, NULL); > + > + return 0; > + > +nosnp: > + setup_clear_cpu_cap(X86_FEATURE_SEV_SNP); > + return -ENOSYS; > +} > + > +/* > + * This must be called after the PCI subsystem. This is because amd_iommu_snp_enable() > + * is called to ensure the IOMMU supports the SEV-SNP feature, which can only be > + * called after subsys_initcall(). > + * > + * NOTE: IOMMU is enforced by SNP to ensure that hypervisor cannot program DMA > + * directly into guest private memory. In case of SNP, the IOMMU ensures that > + * the page(s) used for DMA are hypervisor owned. > + */ > +fs_initcall(snp_rmptable_init); > diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h > index 5dfa4fb76f4b..0a9938aea305 100644 > --- a/arch/x86/include/asm/disabled-features.h > +++ b/arch/x86/include/asm/disabled-features.h > @@ -99,6 +99,12 @@ > # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) > #endif > > +#ifdef CONFIG_KVM_AMD_SEV > +# define DISABLE_SEV_SNP 0 > +#else > +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) > +#endif > + > /* > * Make sure to add features to the correct mask > */ > @@ -123,7 +129,7 @@ > DISABLE_ENQCMD) > #define DISABLED_MASK17 0 > #define DISABLED_MASK18 0 > -#define DISABLED_MASK19 0 > +#define DISABLED_MASK19 (DISABLE_SEV_SNP) > #define DISABLED_MASK20 0 > #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21) > > diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h > index ad35355ee43e..db0f3a041930 100644 > --- a/arch/x86/include/asm/msr-index.h > +++ b/arch/x86/include/asm/msr-index.h > @@ -574,6 +574,8 @@ > #define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT) > #define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT) > #define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) > +#define MSR_AMD64_RMP_BASE 0xc0010132 > +#define MSR_AMD64_RMP_END 0xc0010133 > > /* SNP feature bits enabled by the hypervisor */ > #define MSR_AMD64_SNP_VTOM BIT_ULL(3) > @@ -675,7 +677,14 @@ > #define MSR_K8_TOP_MEM2 0xc001001d > #define MSR_AMD64_SYSCFG 0xc0010010 > #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 > -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) > +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) > +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 > +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) > +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 > +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT) > +#define MSR_AMD64_SYSCFG_MFDM_BIT 19 > +#define MSR_AMD64_SYSCFG_MFDM BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT) > + > #define MSR_K8_INT_PENDING_MSG 0xc0010055 > /* C1E active bits in int pending message */ > #define K8_INTP_C1E_ACTIVE_MASK 0x18000000 > diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h > index ebc271bb6d8e..d34c46db7dd1 100644 > --- a/arch/x86/include/asm/sev.h > +++ b/arch/x86/include/asm/sev.h > @@ -197,6 +197,7 @@ void snp_set_wakeup_secondary_cpu(void); > bool snp_init(struct boot_params *bp); > void __init __noreturn snp_abort(void); > int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); > +bool snp_get_rmptable_info(u64 *start, u64 *len); > #else > static inline void sev_es_ist_enter(struct pt_regs *regs) { } > static inline void sev_es_ist_exit(void) { } > @@ -221,6 +222,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in > { > return -ENOTTY; > } > +static inline bool snp_get_rmptable_info(u64 *start, u64 *len) { return false; } > #endif > > #endif > diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c > index a79774181f22..1493ddf89fdf 100644 > --- a/arch/x86/kernel/cpu/amd.c > +++ b/arch/x86/kernel/cpu/amd.c > @@ -20,6 +20,7 @@ > #include <asm/delay.h> > #include <asm/debugreg.h> > #include <asm/resctrl.h> > +#include <asm/sev.h> > > #ifdef CONFIG_X86_64 > # include <asm/mmconfig.h> > @@ -546,6 +547,20 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) > resctrl_cpu_detect(c); > } > > +static bool early_rmptable_check(void) > +{ > + u64 rmp_base, rmp_size; > + > + /* > + * For early BSP initialization, max_pfn won't be set up yet, wait until > + * it is set before performing the RMP table calculations. > + */ > + if (!max_pfn) > + return true; > + > + return snp_get_rmptable_info(&rmp_base, &rmp_size); > +} > + When CONFIG_AMD_MEM_ENCRYPT=y && CONFIG_KVM=n (=> CONFIG_KVM_AMD_SEV=n) this results in an undefined reference to snp_get_rmptable_info when linking this file. The header provides a stub when AMD_MEM_ENCRYPT=n but the definition is only compiled in when KVM_AMD_SEV=y Jeremi
diff --git a/arch/x86/coco/Makefile b/arch/x86/coco/Makefile index 6aa52e719bf5..6a7d876130e2 100644 --- a/arch/x86/coco/Makefile +++ b/arch/x86/coco/Makefile @@ -6,3 +6,4 @@ CFLAGS_core.o += -fno-stack-protector obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) += core.o obj-$(CONFIG_INTEL_TDX_GUEST) += tdx/ +obj-$(CONFIG_KVM_AMD_SEV) += sev/ diff --git a/arch/x86/coco/sev/Makefile b/arch/x86/coco/sev/Makefile new file mode 100644 index 000000000000..27c0500d75c8 --- /dev/null +++ b/arch/x86/coco/sev/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-y += host.o diff --git a/arch/x86/coco/sev/host.c b/arch/x86/coco/sev/host.c new file mode 100644 index 000000000000..6907ce887b23 --- /dev/null +++ b/arch/x86/coco/sev/host.c @@ -0,0 +1,212 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * AMD SVM-SEV Host Support. + * + * Copyright (C) 2023 Advanced Micro Devices, Inc. + * + * Author: Ashish Kalra <ashish.kalra@amd.com> + * + */ + +#include <linux/cc_platform.h> +#include <linux/printk.h> +#include <linux/mm_types.h> +#include <linux/set_memory.h> +#include <linux/memblock.h> +#include <linux/kernel.h> +#include <linux/mm.h> +#include <linux/cpumask.h> +#include <linux/iommu.h> +#include <linux/amd-iommu.h> + +#include <asm/sev.h> +#include <asm/processor.h> +#include <asm/setup.h> +#include <asm/svm.h> +#include <asm/smp.h> +#include <asm/cpu.h> +#include <asm/apic.h> +#include <asm/cpuid.h> +#include <asm/cmdline.h> +#include <asm/iommu.h> + +/* + * The first 16KB from the RMP_BASE is used by the processor for the + * bookkeeping, the range needs to be added during the RMP entry lookup. + */ +#define RMPTABLE_CPU_BOOKKEEPING_SZ 0x4000 + +static unsigned long rmptable_start __ro_after_init; +static unsigned long rmptable_end __ro_after_init; + +#undef pr_fmt +#define pr_fmt(fmt) "SEV-SNP: " fmt + +static int __mfd_enable(unsigned int cpu) +{ + u64 val; + + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) + return 0; + + rdmsrl(MSR_AMD64_SYSCFG, val); + + val |= MSR_AMD64_SYSCFG_MFDM; + + wrmsrl(MSR_AMD64_SYSCFG, val); + + return 0; +} + +static __init void mfd_enable(void *arg) +{ + __mfd_enable(smp_processor_id()); +} + +static int __snp_enable(unsigned int cpu) +{ + u64 val; + + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) + return 0; + + rdmsrl(MSR_AMD64_SYSCFG, val); + + val |= MSR_AMD64_SYSCFG_SNP_EN; + val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN; + + wrmsrl(MSR_AMD64_SYSCFG, val); + + return 0; +} + +static __init void snp_enable(void *arg) +{ + __snp_enable(smp_processor_id()); +} + +bool snp_get_rmptable_info(u64 *start, u64 *len) +{ + u64 max_rmp_pfn, calc_rmp_sz, rmp_sz, rmp_base, rmp_end; + + rdmsrl(MSR_AMD64_RMP_BASE, rmp_base); + rdmsrl(MSR_AMD64_RMP_END, rmp_end); + + if (!rmp_base || !rmp_end) { + pr_err("Memory for the RMP table has not been reserved by BIOS\n"); + return false; + } + + rmp_sz = rmp_end - rmp_base + 1; + + /* + * Calculate the amount the memory that must be reserved by the BIOS to + * address the whole RAM, including the bookkeeping area. The RMP itself + * must also be covered. + */ + max_rmp_pfn = max_pfn; + if (PHYS_PFN(rmp_end) > max_pfn) + max_rmp_pfn = PHYS_PFN(rmp_end); + + calc_rmp_sz = (max_rmp_pfn << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ; + + if (calc_rmp_sz > rmp_sz) { + pr_err("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n", + calc_rmp_sz, rmp_sz); + return false; + } + + *start = rmp_base; + *len = rmp_sz; + + return true; +} + +static __init int __snp_rmptable_init(void) +{ + u64 rmp_base, sz; + void *start; + u64 val; + + if (!snp_get_rmptable_info(&rmp_base, &sz)) + return 1; + + pr_info("RMP table physical address [0x%016llx - 0x%016llx]\n", + rmp_base, rmp_base + sz - 1); + + start = memremap(rmp_base, sz, MEMREMAP_WB); + if (!start) { + pr_err("Failed to map RMP table addr 0x%llx size 0x%llx\n", rmp_base, sz); + return 1; + } + + /* + * Check if SEV-SNP is already enabled, this can happen in case of + * kexec boot. + */ + rdmsrl(MSR_AMD64_SYSCFG, val); + if (val & MSR_AMD64_SYSCFG_SNP_EN) + goto skip_enable; + + /* Initialize the RMP table to zero */ + memset(start, 0, sz); + + /* Flush the caches to ensure that data is written before SNP is enabled. */ + wbinvd_on_all_cpus(); + + /* MFDM must be enabled on all the CPUs prior to enabling SNP. */ + on_each_cpu(mfd_enable, NULL, 1); + + /* Enable SNP on all CPUs. */ + on_each_cpu(snp_enable, NULL, 1); + +skip_enable: + rmptable_start = (unsigned long)start; + rmptable_end = rmptable_start + sz - 1; + + return 0; +} + +static int __init snp_rmptable_init(void) +{ + int family, model; + + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) + return 0; + + family = boot_cpu_data.x86; + model = boot_cpu_data.x86_model; + + /* + * RMP table entry format is not architectural and it can vary by processor and + * is defined by the per-processor PPR. Restrict SNP support on the known CPU + * model and family for which the RMP table entry format is currently defined for. + */ + if (!(family == 0x19 && model <= 0xaf) && !(family == 0x1a && model <= 0xf)) + goto nosnp; + + if (amd_iommu_snp_enable()) + goto nosnp; + + if (__snp_rmptable_init()) + goto nosnp; + + cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/rmptable_init:online", __snp_enable, NULL); + + return 0; + +nosnp: + setup_clear_cpu_cap(X86_FEATURE_SEV_SNP); + return -ENOSYS; +} + +/* + * This must be called after the PCI subsystem. This is because amd_iommu_snp_enable() + * is called to ensure the IOMMU supports the SEV-SNP feature, which can only be + * called after subsys_initcall(). + * + * NOTE: IOMMU is enforced by SNP to ensure that hypervisor cannot program DMA + * directly into guest private memory. In case of SNP, the IOMMU ensures that + * the page(s) used for DMA are hypervisor owned. + */ +fs_initcall(snp_rmptable_init); diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 5dfa4fb76f4b..0a9938aea305 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -99,6 +99,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_KVM_AMD_SEV +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -123,7 +129,7 @@ DISABLE_ENQCMD) #define DISABLED_MASK17 0 #define DISABLED_MASK18 0 -#define DISABLED_MASK19 0 +#define DISABLED_MASK19 (DISABLE_SEV_SNP) #define DISABLED_MASK20 0 #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index ad35355ee43e..db0f3a041930 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -574,6 +574,8 @@ #define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT) #define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT) #define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_RMP_BASE 0xc0010132 +#define MSR_AMD64_RMP_END 0xc0010133 /* SNP feature bits enabled by the hypervisor */ #define MSR_AMD64_SNP_VTOM BIT_ULL(3) @@ -675,7 +677,14 @@ #define MSR_K8_TOP_MEM2 0xc001001d #define MSR_AMD64_SYSCFG 0xc0010010 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT) +#define MSR_AMD64_SYSCFG_MFDM_BIT 19 +#define MSR_AMD64_SYSCFG_MFDM BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT) + #define MSR_K8_INT_PENDING_MSG 0xc0010055 /* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK 0x18000000 diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index ebc271bb6d8e..d34c46db7dd1 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -197,6 +197,7 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); +bool snp_get_rmptable_info(u64 *start, u64 *len); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -221,6 +222,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } +static inline bool snp_get_rmptable_info(u64 *start, u64 *len) { return false; } #endif #endif diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index a79774181f22..1493ddf89fdf 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -20,6 +20,7 @@ #include <asm/delay.h> #include <asm/debugreg.h> #include <asm/resctrl.h> +#include <asm/sev.h> #ifdef CONFIG_X86_64 # include <asm/mmconfig.h> @@ -546,6 +547,20 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) resctrl_cpu_detect(c); } +static bool early_rmptable_check(void) +{ + u64 rmp_base, rmp_size; + + /* + * For early BSP initialization, max_pfn won't be set up yet, wait until + * it is set before performing the RMP table calculations. + */ + if (!max_pfn) + return true; + + return snp_get_rmptable_info(&rmp_base, &rmp_size); +} + static void early_detect_mem_encrypt(struct cpuinfo_x86 *c) { u64 msr; @@ -587,6 +602,9 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c) if (!(msr & MSR_K7_HWCR_SMMLOCK)) goto clear_sev; + if (cpu_has(c, X86_FEATURE_SEV_SNP) && !early_rmptable_check()) + goto clear_snp; + return; clear_all: @@ -594,6 +612,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c) clear_sev: setup_clear_cpu_cap(X86_FEATURE_SEV); setup_clear_cpu_cap(X86_FEATURE_SEV_ES); +clear_snp: setup_clear_cpu_cap(X86_FEATURE_SEV_SNP); } } diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 19a46b9f7357..33ea62d93540 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -3665,7 +3665,7 @@ int amd_iommu_pc_set_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 return iommu_pc_get_set_reg(iommu, bank, cntr, fxn, value, true); } -#ifdef CONFIG_AMD_MEM_ENCRYPT +#ifdef CONFIG_KVM_AMD_SEV int amd_iommu_snp_enable(void) { /* diff --git a/include/linux/amd-iommu.h b/include/linux/amd-iommu.h index 953e6f12fa1c..8f0cde2d451c 100644 --- a/include/linux/amd-iommu.h +++ b/include/linux/amd-iommu.h @@ -206,7 +206,7 @@ int amd_iommu_pc_get_reg(struct amd_iommu *iommu, u8 bank, u8 cntr, u8 fxn, u64 *value); struct amd_iommu *get_amd_iommu(unsigned int idx); -#ifdef CONFIG_AMD_MEM_ENCRYPT +#ifdef CONFIG_KVM_AMD_SEV int amd_iommu_snp_enable(void); #endif