[RFC,00/34] iommu: Move iommu_group setup to IOMMU core code

Message

Joerg Roedel April 7, 2020, 6:37 p.m. UTC

Hi,

here is a patch-set to remove all calls of iommu_group_get_for_dev() from
the IOMMU drivers and move the per-device group setup and default domain
allocation into the IOMMU core code.

This eliminates some ugly back and forth between IOMMU core code and the
IOMMU drivers, where the driver called iommu_group_get_for_dev() which itself
called back into the driver.

The patch-set started as a "quick" Friday afternoon project to split the
IOMMU group creation and the allocation of the default domain, so that the
default domain is not allocated before all devices are added to the group.
In the end it took 1.5 weeks to get this in a reasonable shape, but now the
code (during bus probing) first adds all devices to their respective IOMMU
group before it determines the default domain type and then allocates it for
the group.

It turned out that this required to remove the calls of
iommu_group_get_for_dev() from the IOMMU drivers. While at it, the calls to
iommu_device_link()/unlink() where also moved out of the drivers, which
required a different interface than add_device()/remove_device(). The result
is the new probe_device()/release_device() interface, where the driver just
does its own setup and then returns the iommu_device which belongs to the
device being probed.

There is certainly more room for cleanups, but I think this is a good start
to simplify the code flow during IOMMU device probing.  It is also a more
robust base for the pending patch-sets which implement per-group default
domain types and the removal of the private domains from the Intel VT-d
driver.

With regards to testing, I verified this code works on three IOMMUs:

	- AMD-Vi
	- Intel VT-d (but there might be breakages on some hardware, the
	  patches to remove the private domain handling from the VT-d driver
	  should be rebased to these patches)
	- ARM-SMMU-v3 (as emulated by QEMU)

Most driver conversions to the probe_device()/release_device() interface
were trivial, but there were also some hard nuts, which I am not sure still
work. The more difficult drivers were:

	- ARM-SMMU-v2
	- OMAP
	- Renesas
	- Mediatek IOMMU v1
	- Exynos

It would be great if the changes could be tested (and possibly fixed) on
those IOMMUs, as I can't do testing on them.

The patches are based on the current iommu/next branch, I will rebase them
to v5.7-rc1 when it comes out. A branch with these patches applied can be
found here:

	https://git.kernel.org/pub/scm/linux/kernel/git/joro/linux.git/log/?h=iommu-probe-device

Please review and test these changes and let me know what breaks.

Thanks,

	Joerg

Joerg Roedel (33):
  iommu: Move default domain allocation to separate function
  iommu/amd: Implement iommu_ops->def_domain_type call-back
  iommu/vt-d: Wire up iommu_ops->def_domain_type
  iommu/amd: Remove dma_mask check from check_device()
  iommu/amd: Return -ENODEV in add_device when device is not handled by
    IOMMU
  iommu: Add probe_device() and remove_device() call-backs
  iommu: Move default domain allocation to iommu_probe_device()
  iommu: Keep a list of allocated groups in __iommu_probe_device()
  iommu: Move new probe_device path to separate function
  iommu: Split off default domain allocation from group assignment
  iommu: Move iommu_group_create_direct_mappings() out of
    iommu_group_add_device()
  iommu: Export bus_iommu_probe() and make is safe for re-probing
  iommu/amd: Remove dev_data->passthrough
  iommu/amd: Convert to probe/release_device() call-backs
  iommu/vt-d: Convert to probe/release_device() call-backs
  iommu/arm-smmu: Store device instead of group in arm_smmu_s2cr
  iommu/arm-smmu: Convert to probe/release_device() call-backs
  iommu/pamu: Convert to probe/release_device() call-backs
  iommu/s390: Convert to probe/release_device() call-backs
  iommu/virtio: Convert to probe/release_device() call-backs
  iommu/msm: Convert to probe/release_device() call-backs
  iommu/mediatek: Convert to probe/release_device() call-backs
  iommu/mediatek-v1 Convert to probe/release_device() call-backs
  iommu/qcom: Convert to probe/release_device() call-backs
  iommu/rockchip: Convert to probe/release_device() call-backs
  iommu/tegra: Convert to probe/release_device() call-backs
  iommu/renesas: Convert to probe/release_device() call-backs
  iommu/omap: Remove orphan_dev tracking
  iommu/omap: Convert to probe/release_device() call-backs
  iommu/exynos: Create iommu_device in struct exynos_iommu_owner
  iommu/exynos: Convert to probe/release_device() call-backs
  iommu: Remove add_device()/remove_device() code-paths
  iommu: Unexport iommu_group_get_for_dev()

Sai Praneeth Prakhya (1):
  iommu: Add def_domain_type() callback in iommu_ops

 drivers/iommu/amd_iommu.c       |  97 ++++----
 drivers/iommu/amd_iommu_types.h |   1 -
 drivers/iommu/arm-smmu-v3.c     |  42 +---
 drivers/iommu/arm-smmu.c        |  44 ++--
 drivers/iommu/exynos-iommu.c    | 113 ++++++---
 drivers/iommu/fsl_pamu_domain.c |  22 +-
 drivers/iommu/intel-iommu.c     |  68 +-----
 drivers/iommu/iommu.c           | 391 +++++++++++++++++++++++++-------
 drivers/iommu/ipmmu-vmsa.c      |  60 ++---
 drivers/iommu/msm_iommu.c       |  34 +--
 drivers/iommu/mtk_iommu.c       |  24 +-
 drivers/iommu/mtk_iommu_v1.c    |  50 ++--
 drivers/iommu/omap-iommu.c      |  99 ++------
 drivers/iommu/qcom_iommu.c      |  24 +-
 drivers/iommu/rockchip-iommu.c  |  26 +--
 drivers/iommu/s390-iommu.c      |  22 +-
 drivers/iommu/tegra-gart.c      |  24 +-
 drivers/iommu/tegra-smmu.c      |  31 +--
 drivers/iommu/virtio-iommu.c    |  41 +---
 include/linux/iommu.h           |  21 +-
 20 files changed, 600 insertions(+), 634 deletions(-)