[v1,00/14] Add Nested Translation Support for SMMUv3

Message

Nicolin Chen March 9, 2023, 10:53 a.m. UTC

Hi all,

This series of patches add nested translation support for ARM SMMUv3.

Eric Auger made a huge effort previously with the VFIO uAPIs, and sent
his v16 a year ago. Now, the nested translation should follow the new
IOMMUFD uAPIs design. So, most of the key features are ported from the
privous VFIO solution, and then rebuilt on top of the IOMMUFD nesting
infrastructure.

This series is rebased on top of the Intel VT-d nesting changes, so as
to reduce merge conflicts at the uapi header updates.

The essential parts in the driver to support a nested translation are
->hw_info, ->domain_alloc_user and ->invalidate_cache_user ops. So this
series fundamentally adds these three functions in the SMMUv3 driver,
along with several preparations and cleanups for them.

One unique requirement for SMMUv3 nested translation support is the MSI
doorbell address translation, which is a 2-stage translation too. And,
to working with the ITS driver, an msi_cookie needs to be setup on the
kernel-managed domain, the stage-2 domain of the nesting setup. And the
same msi_cookie will be fetched, via iommu_get_domain_for_dev(), in the
iommu core to allocate and creates IOVA mappings for the MSI doorbell
page(s). However, with the nesting design, the device is attached to a
user-managed domain, the stage-1 domain. So both the setup and fetching
of the msi_cookie would not work at the level of stage-2 domain. Thus,
on both sides, the msi_cookie setup and fetching require a redirection
of the domain pointer. It's easy to do so in iommufd core, but needs a
new op in the iommu core and driver.

You can also find this series on the Github:
https://github.com/nicolinc/iommufd/commits/iommufd_nesting

The kernel branch is tested with this QEMU branch:
https://github.com/nicolinc/qemu/commits/wip/iommufd_rfcv3+nesting+smmuv3

Thanks!
Nicolin Chen

Eric Auger (2):
  iommu/arm-smmu-v3: Unset corresponding STE fields when s2_cfg is NULL
  iommu/arm-smmu-v3: Add STRTAB_STE_0_CFG_NESTED for 2-stage translation

Nicolin Chen (12):
  iommu: Add iommu_get_unmanaged_domain helper
  iommufd: Add nesting related data structures for ARM SMMUv3
  iommufd/device: Setup MSI on kernel-managed domains
  iommu/arm-smmu-v3: Add arm_smmu_hw_info
  iommu/arm-smmu-v3: Remove ARM_SMMU_DOMAIN_NESTED
  iommu/arm-smmu-v3: Prepare for nested domain support
  iommu/arm-smmu-v3: Implement arm_smmu_get_unmanaged_domain
  iommu/arm-smmu-v3: Pass in user_cfg to arm_smmu_domain_finalise
  iommu/arm-smmu-v3: Add arm_smmu_domain_alloc_user
  iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED type of allocations
  iommu/arm-smmu-v3: Add CMDQ_OP_TLBI_NH_VAA and CMDQ_OP_TLBI_NH_ALL
  iommu/arm-smmu-v3: Add arm_smmu_cache_invalidate_user

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 267 ++++++++++++++++----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   7 +-
 drivers/iommu/dma-iommu.c                   |   5 +-
 drivers/iommu/iommu-priv.h                  |  15 ++
 drivers/iommu/iommufd/device.c              |   5 +-
 drivers/iommu/iommufd/hw_pagetable.c        |   4 +
 drivers/iommu/iommufd/main.c                |   1 +
 include/linux/iommu.h                       |   2 +
 include/uapi/linux/iommufd.h                |  64 +++++
 9 files changed, 323 insertions(+), 47 deletions(-)