| Message ID | 20240701095505.165383-1-steven.price@arm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm64: Support for running as a guest in Arm CCA | expand |
Hi Steven, On Mon, Jul 01, 2024 at 10:54:50AM +0100, Steven Price wrote: > This series adds support for running Linux in a protected VM under the > Arm Confidential Compute Architecture (CCA). This has been updated > following the feedback from the v3 posting[1]. Thanks for the feedback! > Individual patches have a change log. But things to highlight: Hold onto your hat, I'm going to dust off our pKVM protected guest changes and see what we can share here! I've left a few comments on the series, but the main differences seem to be: - You try to probe really early - You have that horrible split IPA space thing from the RSI spec but some of the mechanisms are broadly similar (e.g. implementing the set_memory_*crypted() API). Hopefully I can give your GIC changes a spin, too. Just one minor (but probably annoying) comment: > arch/arm64/Kconfig | 3 + > arch/arm64/include/asm/fixmap.h | 2 +- > arch/arm64/include/asm/io.h | 8 +- > arch/arm64/include/asm/mem_encrypt.h | 17 ++ > arch/arm64/include/asm/pgtable-hwdef.h | 6 - > arch/arm64/include/asm/pgtable-prot.h | 3 + > arch/arm64/include/asm/pgtable.h | 13 +- > arch/arm64/include/asm/rsi.h | 64 ++++++ > arch/arm64/include/asm/rsi_cmds.h | 134 +++++++++++ > arch/arm64/include/asm/rsi_smc.h | 142 ++++++++++++ > arch/arm64/include/asm/set_memory.h | 3 + > arch/arm64/kernel/Makefile | 3 +- > arch/arm64/kernel/efi.c | 2 +- > arch/arm64/kernel/rsi.c | 104 +++++++++ > arch/arm64/kernel/setup.c | 8 + > arch/arm64/mm/init.c | 10 +- > arch/arm64/mm/pageattr.c | 76 ++++++- > drivers/firmware/psci/psci.c | 25 +++ > drivers/irqchip/irq-gic-v3-its.c | 142 +++++++++--- > drivers/virt/coco/Kconfig | 2 + > drivers/virt/coco/Makefile | 1 + > drivers/virt/coco/arm-cca-guest/Kconfig | 11 + > drivers/virt/coco/arm-cca-guest/Makefile | 2 + > .../virt/coco/arm-cca-guest/arm-cca-guest.c | 211 ++++++++++++++++++ > include/linux/psci.h | 5 + > 25 files changed, 953 insertions(+), 44 deletions(-) > create mode 100644 arch/arm64/include/asm/mem_encrypt.h > create mode 100644 arch/arm64/include/asm/rsi.h > create mode 100644 arch/arm64/include/asm/rsi_cmds.h > create mode 100644 arch/arm64/include/asm/rsi_smc.h > create mode 100644 arch/arm64/kernel/rsi.c > create mode 100644 drivers/virt/coco/arm-cca-guest/Kconfig > create mode 100644 drivers/virt/coco/arm-cca-guest/Makefile > create mode 100644 drivers/virt/coco/arm-cca-guest/arm-cca-guest.c Any chance of some documentation, please? Cheers, Will
On Mon, Jul 01, 2024 at 10:54:50AM +0100, Steven Price wrote: > This series adds support for running Linux in a protected VM under the > Arm Confidential Compute Architecture (CCA). This has been updated > following the feedback from the v3 posting[1]. Thanks for the feedback! > Individual patches have a change log. But things to highlight: > > * a new patch ("firmware/psci: Add psci_early_test_conduit()") to > prevent SMC calls being made on systems which don't support them - > i.e. systems without EL2/EL3 - thanks Jean-Philippe! > > * two patches dropped (overriding set_fixmap_io). Instead > FIXMAP_PAGE_IO is modified to include PROT_NS_SHARED. When support > for assigning hardware devices to a realm guest is added this will > need to be brought back in some form. But for now it's just adding > complixity and confusion for no gain. > > * a new patch ("arm64: mm: Avoid TLBI when marking pages as valid") > which avoids doing an extra TLBI when doing the break-before-make. > Note that this changes the behaviour in other cases when making > memory valid. This should be safe (and saves a TLBI for those cases), > but it's a separate patch in case of regressions. > > * GIC ITT allocation now uses a custom genpool-based allocator. I > expect this will be replaced with a generic way of allocating > decrypted memory (see [4]), but for now this gets things working > without wasting too much memory. > > The ABI to the RMM from a realm (the RSI) is based on the final RMM v1.0 > (EAC 5) specification[2]. Future RMM specifications will be backwards > compatible so a guest using the v1.0 specification (i.e. this series) > will be able to run on future versions of the RMM without modification. > > This series is based on v6.10-rc1. It is also available as a git > repository: > > https://gitlab.arm.com/linux-arm/linux-cca cca-guest/v4 > > This series (the guest side) should be in a good state so please review > with the intention that this could be merged soon. The host side will > require more iteration so the versioning of the series will diverge - > so for now continue to use v3 for the host support. > > Introduction (unchanged from v2 posting) > ============ > A more general introduction to Arm CCA is available on the Arm > website[3], and links to the other components involved are available in > the overall cover letter. > > Arm Confidential Compute Architecture adds two new 'worlds' to the > architecture: Root and Realm. A new software component known as the RMM > (Realm Management Monitor) runs in Realm EL2 and is trusted by both the > Normal World and VMs running within Realms. This enables mutual > distrust between the Realm VMs and the Normal World. > > Virtual machines running within a Realm can decide on a (4k) > page-by-page granularity whether to share a page with the (Normal World) > host or to keep it private (protected). This protection is provided by > the hardware and attempts to access a page which isn't shared by the > Normal World will trigger a Granule Protection Fault. > > Realm VMs can communicate with the RMM via another SMC interface known > as RSI (Realm Services Interface). This series adds wrappers for the > full set of RSI commands and uses them to manage the Realm IPA State > (RIPAS) and to discover the configuration of the realm. > > The VM running within the Realm needs to ensure that memory that is > going to use is marked as 'RIPAS_RAM' (i.e. protected memory accessible > only to the guest). This could be provided by the VMM (and subject to > measurement to ensure it is setup correctly) or the VM can set it > itself. This series includes a patch which will iterate over all > described RAM and set the RIPAS. This is a relatively cheap operation, > and doesn't require memory donation from the host. Instead, memory can > be dynamically provided by the host on fault. An alternative would be to > update booting.rst and state this as a requirement, but this would > reduce the flexibility of the VMM to manage the available memory to the > guest (as the initial RIPAS state is part of the guest's measurement). > > Within the Realm the most-significant active bit of the IPA is used to > select whether the access is to protected memory or to memory shared > with the host. This series treats this bit as if it is attribute bit in > the page tables and will modify it when sharing/unsharing memory with > the host. > > This top bit usage also necessitates that the IPA width is made more > dynamic in the guest. The VMM will choose a width (and therefore which > bit controls the shared flag) and the guest must be able to identify > this bit to mask it out when necessary. PHYS_MASK_SHIFT/PHYS_MASK are > therefore made dynamic. > > To allow virtio to communicate with the host the shared buffers must be > placed in memory which has this top IPA bit set. This is achieved by > implementing the set_memory_{encrypted,decrypted} APIs for arm64 and > forcing the use of bounce buffers. For now all device access is > considered to required the memory to be shared, at this stage there is > no support for real devices to be assigned to a realm guest - obviously > if device assignment is added this will have to change. > > Finally the GIC is (largely) emulated by the (untrusted) host. The RMM > provides some management (including register save/restore) but the > ITS buffers must be placed into shared memory for the host to emulate. > There is likely to be future work to harden the GIC driver against a > malicious host (along with any other drivers used within a Realm guest). > > [1] https://lore.kernel.org/lkml/20240605093006.145492-1-steven.price%40arm.com > [2] https://developer.arm.com/documentation/den0137/1-0eac5/ > [3] https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture > [4] https://lore.kernel.org/lkml/ZmNJdSxSz-sYpVgI%40arm.com > > Jean-Philippe Brucker (1): > firmware/psci: Add psci_early_test_conduit() > > Sami Mujawar (2): > arm64: rsi: Interfaces to query attestation token > virt: arm-cca-guest: TSM_REPORT support for realms > > Steven Price (7): > arm64: realm: Query IPA size from the RMM > arm64: Mark all I/O as non-secure shared > arm64: Make the PHYS_MASK_SHIFT dynamic > arm64: Enforce bounce buffers for realm DMA > arm64: mm: Avoid TLBI when marking pages as valid > irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor > irqchip/gic-v3-its: Rely on genpool alignment > > Suzuki K Poulose (5): > arm64: rsi: Add RSI definitions > arm64: Detect if in a realm and set RIPAS RAM > arm64: Enable memory encrypt for Realms > arm64: Force device mappings to be non-secure shared > efi: arm64: Map Device with Prot Shared > > arch/arm64/Kconfig | 3 + > arch/arm64/include/asm/fixmap.h | 2 +- > arch/arm64/include/asm/io.h | 8 +- > arch/arm64/include/asm/mem_encrypt.h | 17 ++ > arch/arm64/include/asm/pgtable-hwdef.h | 6 - > arch/arm64/include/asm/pgtable-prot.h | 3 + > arch/arm64/include/asm/pgtable.h | 13 +- > arch/arm64/include/asm/rsi.h | 64 ++++++ > arch/arm64/include/asm/rsi_cmds.h | 134 +++++++++++ > arch/arm64/include/asm/rsi_smc.h | 142 ++++++++++++ > arch/arm64/include/asm/set_memory.h | 3 + > arch/arm64/kernel/Makefile | 3 +- > arch/arm64/kernel/efi.c | 2 +- > arch/arm64/kernel/rsi.c | 104 +++++++++ > arch/arm64/kernel/setup.c | 8 + > arch/arm64/mm/init.c | 10 +- > arch/arm64/mm/pageattr.c | 76 ++++++- > drivers/firmware/psci/psci.c | 25 +++ > drivers/irqchip/irq-gic-v3-its.c | 142 +++++++++--- > drivers/virt/coco/Kconfig | 2 + > drivers/virt/coco/Makefile | 1 + > drivers/virt/coco/arm-cca-guest/Kconfig | 11 + > drivers/virt/coco/arm-cca-guest/Makefile | 2 + > .../virt/coco/arm-cca-guest/arm-cca-guest.c | 211 ++++++++++++++++++ > include/linux/psci.h | 5 + > 25 files changed, 953 insertions(+), 44 deletions(-) > create mode 100644 arch/arm64/include/asm/mem_encrypt.h > create mode 100644 arch/arm64/include/asm/rsi.h > create mode 100644 arch/arm64/include/asm/rsi_cmds.h > create mode 100644 arch/arm64/include/asm/rsi_smc.h > create mode 100644 arch/arm64/kernel/rsi.c > create mode 100644 drivers/virt/coco/arm-cca-guest/Kconfig > create mode 100644 drivers/virt/coco/arm-cca-guest/Makefile > create mode 100644 drivers/virt/coco/arm-cca-guest/arm-cca-guest.c > > -- > 2.34.1 > > Hello, I tested this series on QEMU/KVM with the CCA patches(v2) and the FVP model. I could not find any evident issue. Tested-by: Matias Ezequiel Vara Larsen <mvaralar@redhat.com>
Hi Steven, On 7/12/24 03:54, Matias Ezequiel Vara Larsen wrote: > On Mon, Jul 01, 2024 at 10:54:50AM +0100, Steven Price wrote: >> This series adds support for running Linux in a protected VM under the >> Arm Confidential Compute Architecture (CCA). This has been updated >> following the feedback from the v3 posting[1]. Thanks for the feedback! >> Individual patches have a change log. But things to highlight: >> >> * a new patch ("firmware/psci: Add psci_early_test_conduit()") to >> prevent SMC calls being made on systems which don't support them - >> i.e. systems without EL2/EL3 - thanks Jean-Philippe! >> >> * two patches dropped (overriding set_fixmap_io). Instead >> FIXMAP_PAGE_IO is modified to include PROT_NS_SHARED. When support >> for assigning hardware devices to a realm guest is added this will >> need to be brought back in some form. But for now it's just adding >> complixity and confusion for no gain. >> >> * a new patch ("arm64: mm: Avoid TLBI when marking pages as valid") >> which avoids doing an extra TLBI when doing the break-before-make. >> Note that this changes the behaviour in other cases when making >> memory valid. This should be safe (and saves a TLBI for those cases), >> but it's a separate patch in case of regressions. >> >> * GIC ITT allocation now uses a custom genpool-based allocator. I >> expect this will be replaced with a generic way of allocating >> decrypted memory (see [4]), but for now this gets things working >> without wasting too much memory. >> >> The ABI to the RMM from a realm (the RSI) is based on the final RMM v1.0 >> (EAC 5) specification[2]. Future RMM specifications will be backwards >> compatible so a guest using the v1.0 specification (i.e. this series) >> will be able to run on future versions of the RMM without modification. >> >> This series is based on v6.10-rc1. It is also available as a git >> repository: >> >> https://gitlab.arm.com/linux-arm/linux-cca cca-guest/v4 Which cca-host branch should I use for testing cca-guest/v4? I'm getting compilation errors with cca-host/v3 and cca-guest/v4, is there any known WAR or fix to resolve this issue? arch/arm64/kvm/rme.c: In function ‘kvm_realm_reset_id_aa64dfr0_el1’: ././include/linux/compiler_types.h:487:45: error: call to ‘__compiletime_assert_650’ declared with attribute error: FIELD_PREP: value too large for the field 487 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) | ^ ././include/linux/compiler_types.h:468:25: note: in definition of macro ‘__compiletime_assert’ 468 | prefix ## suffix(); \ | ^~~~~~ ././include/linux/compiler_types.h:487:9: note: in expansion of macro ‘_compiletime_assert’ 487 | _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__) | ^~~~~~~~~~~~~~~~~~~ ./include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’ 39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg) | ^~~~~~~~~~~~~~~~~~ ./include/linux/bitfield.h:68:17: note: in expansion of macro ‘BUILD_BUG_ON_MSG’ 68 | BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ? \ | ^~~~~~~~~~~~~~~~ ./include/linux/bitfield.h:115:17: note: in expansion of macro ‘__BF_FIELD_CHECK’ 115 | __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: "); \ | ^~~~~~~~~~~~~~~~ arch/arm64/kvm/rme.c:315:16: note: in expansion of macro ‘FIELD_PREP’ 315 | val |= FIELD_PREP(ID_AA64DFR0_EL1_BRPs_MASK, bps - 1) | | ^~~~~~~~~~ make[5]: *** [scripts/Makefile.build:244: arch/arm64/kvm/rme.o] Error 1 make[4]: *** [scripts/Makefile.build:485: arch/arm64/kvm] Error 2 make[3]: *** [scripts/Makefile.build:485: arch/arm64] Error 2 make[3]: *** Waiting for unfinished jobs.... I'm using gcc-13.3.0 compiler and cross-compiling on X86 machine. -Shanker
On 15/08/2024 23:16, Shanker Donthineni wrote: > Hi Steven, > > On 7/12/24 03:54, Matias Ezequiel Vara Larsen wrote: >> On Mon, Jul 01, 2024 at 10:54:50AM +0100, Steven Price wrote: >>> This series adds support for running Linux in a protected VM under the >>> Arm Confidential Compute Architecture (CCA). This has been updated >>> following the feedback from the v3 posting[1]. Thanks for the feedback! >>> Individual patches have a change log. But things to highlight: >>> >>> * a new patch ("firmware/psci: Add psci_early_test_conduit()") to >>> prevent SMC calls being made on systems which don't support them - >>> i.e. systems without EL2/EL3 - thanks Jean-Philippe! >>> >>> * two patches dropped (overriding set_fixmap_io). Instead >>> FIXMAP_PAGE_IO is modified to include PROT_NS_SHARED. When support >>> for assigning hardware devices to a realm guest is added this will >>> need to be brought back in some form. But for now it's just adding >>> complixity and confusion for no gain. >>> >>> * a new patch ("arm64: mm: Avoid TLBI when marking pages as valid") >>> which avoids doing an extra TLBI when doing the break-before-make. >>> Note that this changes the behaviour in other cases when making >>> memory valid. This should be safe (and saves a TLBI for those >>> cases), >>> but it's a separate patch in case of regressions. >>> >>> * GIC ITT allocation now uses a custom genpool-based allocator. I >>> expect this will be replaced with a generic way of allocating >>> decrypted memory (see [4]), but for now this gets things working >>> without wasting too much memory. >>> >>> The ABI to the RMM from a realm (the RSI) is based on the final RMM v1.0 >>> (EAC 5) specification[2]. Future RMM specifications will be backwards >>> compatible so a guest using the v1.0 specification (i.e. this series) >>> will be able to run on future versions of the RMM without modification. >>> >>> This series is based on v6.10-rc1. It is also available as a git >>> repository: >>> >>> https://gitlab.arm.com/linux-arm/linux-cca cca-guest/v4 > > Which cca-host branch should I use for testing cca-guest/v4? > > I'm getting compilation errors with cca-host/v3 and cca-guest/v4, is there > any known WAR or fix to resolve this issue? cca-host/v3 should work with cca-guest/v4. I've been working on rebasing/updating the branches and should be able to post v4/v5 series next week. > > arch/arm64/kvm/rme.c: In function ‘kvm_realm_reset_id_aa64dfr0_el1’: > ././include/linux/compiler_types.h:487:45: error: call to > ‘__compiletime_assert_650’ declared with attribute error: FIELD_PREP: > value too large for the field > 487 | _compiletime_assert(condition, msg, > __compiletime_assert_, __COUNTER__) > | ^ > ././include/linux/compiler_types.h:468:25: note: in definition of macro > ‘__compiletime_assert’ > 468 | prefix ## > suffix(); \ > | ^~~~~~ > ././include/linux/compiler_types.h:487:9: note: in expansion of macro > ‘_compiletime_assert’ > 487 | _compiletime_assert(condition, msg, > __compiletime_assert_, __COUNTER__) > | ^~~~~~~~~~~~~~~~~~~ > ./include/linux/build_bug.h:39:37: note: in expansion of macro > ‘compiletime_assert’ > 39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), > msg) > | ^~~~~~~~~~~~~~~~~~ > ./include/linux/bitfield.h:68:17: note: in expansion of macro > ‘BUILD_BUG_ON_MSG’ > 68 | BUILD_BUG_ON_MSG(__builtin_constant_p(_val) > ? \ > | ^~~~~~~~~~~~~~~~ > ./include/linux/bitfield.h:115:17: note: in expansion of macro > ‘__BF_FIELD_CHECK’ > 115 | __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: > "); \ > | ^~~~~~~~~~~~~~~~ > arch/arm64/kvm/rme.c:315:16: note: in expansion of macro ‘FIELD_PREP’ > 315 | val |= FIELD_PREP(ID_AA64DFR0_EL1_BRPs_MASK, bps - 1) | > | ^~~~~~~~~~ > make[5]: *** [scripts/Makefile.build:244: arch/arm64/kvm/rme.o] Error 1 > make[4]: *** [scripts/Makefile.build:485: arch/arm64/kvm] Error 2 > make[3]: *** [scripts/Makefile.build:485: arch/arm64] Error 2 > make[3]: *** Waiting for unfinished jobs.... > > I'm using gcc-13.3.0 compiler and cross-compiling on X86 machine. I'm not sure quite how this happens. The 'value' (bps - 1) shouldn't be considered constant, so I don't see how the compiler has decided to complain here - the __builtin_constant_p() should really be evaluating to 0. The only thing I can think of is if the compiler has somehow determined that rmm_feat_reg0 is 0 - which in theory it could do if it knew that kvm_init_rme() cannot succeed (rmi_features() would never be called, so the variable will never be set). Which makes me wonder if you're building with a PAGE_SIZE other than 4k? Obviously the code should still build if that's the case (so this would be a bug) but we don't currently support CCA with PAGE_SIZE != 4k. Steve
On 8/16/24 11:06, Steven Price wrote: > External email: Use caution opening links or attachments > > > On 15/08/2024 23:16, Shanker Donthineni wrote: >> Hi Steven, >> >> On 7/12/24 03:54, Matias Ezequiel Vara Larsen wrote: >>> On Mon, Jul 01, 2024 at 10:54:50AM +0100, Steven Price wrote: >>>> This series adds support for running Linux in a protected VM under the >>>> Arm Confidential Compute Architecture (CCA). This has been updated >>>> following the feedback from the v3 posting[1]. Thanks for the feedback! >>>> Individual patches have a change log. But things to highlight: >>>> >>>> * a new patch ("firmware/psci: Add psci_early_test_conduit()") to >>>> prevent SMC calls being made on systems which don't support them - >>>> i.e. systems without EL2/EL3 - thanks Jean-Philippe! >>>> >>>> * two patches dropped (overriding set_fixmap_io). Instead >>>> FIXMAP_PAGE_IO is modified to include PROT_NS_SHARED. When support >>>> for assigning hardware devices to a realm guest is added this will >>>> need to be brought back in some form. But for now it's just adding >>>> complixity and confusion for no gain. >>>> >>>> * a new patch ("arm64: mm: Avoid TLBI when marking pages as valid") >>>> which avoids doing an extra TLBI when doing the break-before-make. >>>> Note that this changes the behaviour in other cases when making >>>> memory valid. This should be safe (and saves a TLBI for those >>>> cases), >>>> but it's a separate patch in case of regressions. >>>> >>>> * GIC ITT allocation now uses a custom genpool-based allocator. I >>>> expect this will be replaced with a generic way of allocating >>>> decrypted memory (see [4]), but for now this gets things working >>>> without wasting too much memory. >>>> >>>> The ABI to the RMM from a realm (the RSI) is based on the final RMM v1.0 >>>> (EAC 5) specification[2]. Future RMM specifications will be backwards >>>> compatible so a guest using the v1.0 specification (i.e. this series) >>>> will be able to run on future versions of the RMM without modification. >>>> >>>> This series is based on v6.10-rc1. It is also available as a git >>>> repository: >>>> >>>> https://gitlab.arm.com/linux-arm/linux-cca cca-guest/v4 >> >> Which cca-host branch should I use for testing cca-guest/v4? >> >> I'm getting compilation errors with cca-host/v3 and cca-guest/v4, is there >> any known WAR or fix to resolve this issue? > > cca-host/v3 should work with cca-guest/v4. I've been working on > rebasing/updating the branches and should be able to post v4/v5 series > next week. > >> >> arch/arm64/kvm/rme.c: In function ‘kvm_realm_reset_id_aa64dfr0_el1’: >> ././include/linux/compiler_types.h:487:45: error: call to >> ‘__compiletime_assert_650’ declared with attribute error: FIELD_PREP: >> value too large for the field >> 487 | _compiletime_assert(condition, msg, >> __compiletime_assert_, __COUNTER__) >> | ^ >> ././include/linux/compiler_types.h:468:25: note: in definition of macro >> ‘__compiletime_assert’ >> 468 | prefix ## >> suffix(); \ >> | ^~~~~~ >> ././include/linux/compiler_types.h:487:9: note: in expansion of macro >> ‘_compiletime_assert’ >> 487 | _compiletime_assert(condition, msg, >> __compiletime_assert_, __COUNTER__) >> | ^~~~~~~~~~~~~~~~~~~ >> ./include/linux/build_bug.h:39:37: note: in expansion of macro >> ‘compiletime_assert’ >> 39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), >> msg) >> | ^~~~~~~~~~~~~~~~~~ >> ./include/linux/bitfield.h:68:17: note: in expansion of macro >> ‘BUILD_BUG_ON_MSG’ >> 68 | BUILD_BUG_ON_MSG(__builtin_constant_p(_val) >> ? \ >> | ^~~~~~~~~~~~~~~~ >> ./include/linux/bitfield.h:115:17: note: in expansion of macro >> ‘__BF_FIELD_CHECK’ >> 115 | __BF_FIELD_CHECK(_mask, 0ULL, _val, "FIELD_PREP: >> "); \ >> | ^~~~~~~~~~~~~~~~ >> arch/arm64/kvm/rme.c:315:16: note: in expansion of macro ‘FIELD_PREP’ >> 315 | val |= FIELD_PREP(ID_AA64DFR0_EL1_BRPs_MASK, bps - 1) | >> | ^~~~~~~~~~ >> make[5]: *** [scripts/Makefile.build:244: arch/arm64/kvm/rme.o] Error 1 >> make[4]: *** [scripts/Makefile.build:485: arch/arm64/kvm] Error 2 >> make[3]: *** [scripts/Makefile.build:485: arch/arm64] Error 2 >> make[3]: *** Waiting for unfinished jobs.... >> >> I'm using gcc-13.3.0 compiler and cross-compiling on X86 machine. > > I'm not sure quite how this happens. The 'value' (bps - 1) shouldn't be > considered constant, so I don't see how the compiler has decided to > complain here - the __builtin_constant_p() should really be evaluating to 0. > > The only thing I can think of is if the compiler has somehow determined > that rmm_feat_reg0 is 0 - which in theory it could do if it knew that > kvm_init_rme() cannot succeed (rmi_features() would never be called, so > the variable will never be set). Which makes me wonder if you're > building with a PAGE_SIZE other than 4k? > > Obviously the code should still build if that's the case (so this would > be a bug) but we don't currently support CCA with PAGE_SIZE != 4k. > I've encountered this error multiple times with both 4K and 64K, but it's currently not reproducible. I'll update if the issue reappears. In the meantime, I've verified the host-v3 and guest-v4 patches using v6.11.rc3, tested Realm boot, CCA-KVM-UNIT-TESTs, and normal VM boot (without CCA). No issues have been observed. Additionally, I've validated Realm and CCA-KVM-UNIT-TESTs on a host with PSZ=64K. For testing purposes, I modified KVM64 and KVM-UNIT-TESTS to support PSZ=64K. Tested-by: Shanker Donthineni <sdonthineni@nvidia.com> > Steve >
This series adds support for running Linux in a protected VM under the Arm Confidential Compute Architecture (CCA). This has been updated following the feedback from the v3 posting[1]. Thanks for the feedback! Individual patches have a change log. But things to highlight: * a new patch ("firmware/psci: Add psci_early_test_conduit()") to prevent SMC calls being made on systems which don't support them - i.e. systems without EL2/EL3 - thanks Jean-Philippe! * two patches dropped (overriding set_fixmap_io). Instead FIXMAP_PAGE_IO is modified to include PROT_NS_SHARED. When support for assigning hardware devices to a realm guest is added this will need to be brought back in some form. But for now it's just adding complixity and confusion for no gain. * a new patch ("arm64: mm: Avoid TLBI when marking pages as valid") which avoids doing an extra TLBI when doing the break-before-make. Note that this changes the behaviour in other cases when making memory valid. This should be safe (and saves a TLBI for those cases), but it's a separate patch in case of regressions. * GIC ITT allocation now uses a custom genpool-based allocator. I expect this will be replaced with a generic way of allocating decrypted memory (see [4]), but for now this gets things working without wasting too much memory. The ABI to the RMM from a realm (the RSI) is based on the final RMM v1.0 (EAC 5) specification[2]. Future RMM specifications will be backwards compatible so a guest using the v1.0 specification (i.e. this series) will be able to run on future versions of the RMM without modification. This series is based on v6.10-rc1. It is also available as a git repository: https://gitlab.arm.com/linux-arm/linux-cca cca-guest/v4 This series (the guest side) should be in a good state so please review with the intention that this could be merged soon. The host side will require more iteration so the versioning of the series will diverge - so for now continue to use v3 for the host support. Introduction (unchanged from v2 posting) ============ A more general introduction to Arm CCA is available on the Arm website[3], and links to the other components involved are available in the overall cover letter. Arm Confidential Compute Architecture adds two new 'worlds' to the architecture: Root and Realm. A new software component known as the RMM (Realm Management Monitor) runs in Realm EL2 and is trusted by both the Normal World and VMs running within Realms. This enables mutual distrust between the Realm VMs and the Normal World. Virtual machines running within a Realm can decide on a (4k) page-by-page granularity whether to share a page with the (Normal World) host or to keep it private (protected). This protection is provided by the hardware and attempts to access a page which isn't shared by the Normal World will trigger a Granule Protection Fault. Realm VMs can communicate with the RMM via another SMC interface known as RSI (Realm Services Interface). This series adds wrappers for the full set of RSI commands and uses them to manage the Realm IPA State (RIPAS) and to discover the configuration of the realm. The VM running within the Realm needs to ensure that memory that is going to use is marked as 'RIPAS_RAM' (i.e. protected memory accessible only to the guest). This could be provided by the VMM (and subject to measurement to ensure it is setup correctly) or the VM can set it itself. This series includes a patch which will iterate over all described RAM and set the RIPAS. This is a relatively cheap operation, and doesn't require memory donation from the host. Instead, memory can be dynamically provided by the host on fault. An alternative would be to update booting.rst and state this as a requirement, but this would reduce the flexibility of the VMM to manage the available memory to the guest (as the initial RIPAS state is part of the guest's measurement). Within the Realm the most-significant active bit of the IPA is used to select whether the access is to protected memory or to memory shared with the host. This series treats this bit as if it is attribute bit in the page tables and will modify it when sharing/unsharing memory with the host. This top bit usage also necessitates that the IPA width is made more dynamic in the guest. The VMM will choose a width (and therefore which bit controls the shared flag) and the guest must be able to identify this bit to mask it out when necessary. PHYS_MASK_SHIFT/PHYS_MASK are therefore made dynamic. To allow virtio to communicate with the host the shared buffers must be placed in memory which has this top IPA bit set. This is achieved by implementing the set_memory_{encrypted,decrypted} APIs for arm64 and forcing the use of bounce buffers. For now all device access is considered to required the memory to be shared, at this stage there is no support for real devices to be assigned to a realm guest - obviously if device assignment is added this will have to change. Finally the GIC is (largely) emulated by the (untrusted) host. The RMM provides some management (including register save/restore) but the ITS buffers must be placed into shared memory for the host to emulate. There is likely to be future work to harden the GIC driver against a malicious host (along with any other drivers used within a Realm guest). [1] https://lore.kernel.org/lkml/20240605093006.145492-1-steven.price%40arm.com [2] https://developer.arm.com/documentation/den0137/1-0eac5/ [3] https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture [4] https://lore.kernel.org/lkml/ZmNJdSxSz-sYpVgI%40arm.com Jean-Philippe Brucker (1): firmware/psci: Add psci_early_test_conduit() Sami Mujawar (2): arm64: rsi: Interfaces to query attestation token virt: arm-cca-guest: TSM_REPORT support for realms Steven Price (7): arm64: realm: Query IPA size from the RMM arm64: Mark all I/O as non-secure shared arm64: Make the PHYS_MASK_SHIFT dynamic arm64: Enforce bounce buffers for realm DMA arm64: mm: Avoid TLBI when marking pages as valid irqchip/gic-v3-its: Share ITS tables with a non-trusted hypervisor irqchip/gic-v3-its: Rely on genpool alignment Suzuki K Poulose (5): arm64: rsi: Add RSI definitions arm64: Detect if in a realm and set RIPAS RAM arm64: Enable memory encrypt for Realms arm64: Force device mappings to be non-secure shared efi: arm64: Map Device with Prot Shared arch/arm64/Kconfig | 3 + arch/arm64/include/asm/fixmap.h | 2 +- arch/arm64/include/asm/io.h | 8 +- arch/arm64/include/asm/mem_encrypt.h | 17 ++ arch/arm64/include/asm/pgtable-hwdef.h | 6 - arch/arm64/include/asm/pgtable-prot.h | 3 + arch/arm64/include/asm/pgtable.h | 13 +- arch/arm64/include/asm/rsi.h | 64 ++++++ arch/arm64/include/asm/rsi_cmds.h | 134 +++++++++++ arch/arm64/include/asm/rsi_smc.h | 142 ++++++++++++ arch/arm64/include/asm/set_memory.h | 3 + arch/arm64/kernel/Makefile | 3 +- arch/arm64/kernel/efi.c | 2 +- arch/arm64/kernel/rsi.c | 104 +++++++++ arch/arm64/kernel/setup.c | 8 + arch/arm64/mm/init.c | 10 +- arch/arm64/mm/pageattr.c | 76 ++++++- drivers/firmware/psci/psci.c | 25 +++ drivers/irqchip/irq-gic-v3-its.c | 142 +++++++++--- drivers/virt/coco/Kconfig | 2 + drivers/virt/coco/Makefile | 1 + drivers/virt/coco/arm-cca-guest/Kconfig | 11 + drivers/virt/coco/arm-cca-guest/Makefile | 2 + .../virt/coco/arm-cca-guest/arm-cca-guest.c | 211 ++++++++++++++++++ include/linux/psci.h | 5 + 25 files changed, 953 insertions(+), 44 deletions(-) create mode 100644 arch/arm64/include/asm/mem_encrypt.h create mode 100644 arch/arm64/include/asm/rsi.h create mode 100644 arch/arm64/include/asm/rsi_cmds.h create mode 100644 arch/arm64/include/asm/rsi_smc.h create mode 100644 arch/arm64/kernel/rsi.c create mode 100644 drivers/virt/coco/arm-cca-guest/Kconfig create mode 100644 drivers/virt/coco/arm-cca-guest/Makefile create mode 100644 drivers/virt/coco/arm-cca-guest/arm-cca-guest.c