[v6,00/29] Update SMMUv3 to the modern iommu API (part 2/3)

Message

Jason Gunthorpe March 27, 2024, 6:07 p.m. UTC

Continuing the work of part 1 this focuses on the CD, PASID and SVA
components:

 - attach_dev failure does not change the HW configuration.

 - Full PASID API support including:
    - S1/SVA domains attached to PASIDs
    - IDENTITY/BLOCKED/S1 attached to RID
    - Change of the RID domain while PASIDs are attached

 - Streamlined SVA support using the core infrastructure

 - Hitless, whenever possible, change between two domains

Making the CD programming work like the new STE programming allows
untangling some of the confusing SVA flows. From there the focus is on
building out the core infrastructure for dealing with PASID and CD
entries, then keeping track of unique SSID's for ATS invalidation.

The ATS ordering is generalized so that the PASID flow can use it and put
into a form where it is fully hitless, whenever possible. Care is taken to
ensure that ATC flushes are present after any change in translation.

Finally we simply kill the entire outdated SVA mmu_notifier implementation
in one shot and switch it over to the newly created generic PASID & CD
code. This avoids the messy and confusing approach of trying to
incrementally untangle this in place. The new code is small and simple
enough this is much better than trying to figure out smaller steps.

Once SVA is resting on the right CD code it is straightforward to make the
PASID interface functionally complete.

It achieves the same goals as the several series from Michael and the S1DSS
series from Nicolin that were trying to improve portions of the API.

This is on github:
https://github.com/jgunthorpe/linux/commits/smmuv3_newapi

v6:
 - Rebase on v6.9-rc1
 - Remove arm_smmu_entry_writer_ops->num_entry_qwords and just use
   NUM_ENTRY_QWORDS for CD & STE in all places
 - Split arm_smmu_get_cd_ptr() into arm_smmu_alloc_cd_ptr() and call the
   allocation one only from attach paths
 - Remove cd_table.used_sid
 - Fix order of EPD1 in arm_smmu_write_cd_entry
 - Do not double invalidate during domain free by removing
   iommu_flush_ops->tlb_flush_all. Consolidate all the ID invalidation
   code to arm_smmu_tlb_inv_context()
 - Use the right old domain in arm_smmu_attach_commit() for PASID cases
 - Rename arm_smmu_domain_free() to arm_smmu_domain_free_paging()
 - ssid should be an ioasid_t not a u16
 - Scrub the CD target instead of the no_used_check temporary
 - Reorder the patches around arm_smmu_alloc/get_cd_ptr()
 - Add the temporary arm_smmu_write_cd_entry() calls to
   arm_smmu_sva_set_dev_pasid() with error handling
 - Move some of the hunks for the in_set/etc tracking around so that
   use_sid can go away. Use in_ste instead of arm_smmu_is_s1_domain()
v5: https://lore.kernel.org/r/[e-mail]
 - Rebase on v6.8-rc7 & Will's tree
 - Accomdate the SVA rc patch removing the master list iteration
 - Move the kfree(to_smmu_domain(domain)) hunk to the right patch
 - Move S1DSS get_used hunk to "Allow IDENTITY/BLOCKED to be set while
   PASID is used"
v4: https://lore.kernel.org/r/[e-mail]
 - Rebase on v6.8-rc1, adjust to use mm_get_enqcmd_pasid() and eventually
   remove all references from ARM. Move the new ARM_SMMU_FEAT_STALL_FORCE
   stuff to arm_smmu_make_sva_cd()
 - Adjust to use the new shared STE/CD writer logic. Disable some of the
   sanity checks for the interior of the series
 - Return ERR_PTR from domain_alloc functions
 - Move the ATS disablement flow into arm_smmu_attach_prepare()/commit()
   which lets all the STE update flows use the same sequence. This is
   needed for nesting in part 3
 - Put ssid in attach_state
 - Replace to_smmu_domain_safe() with to_smmu_domain_devices()
v3: https://lore.kernel.org/r/[e-mail]
 - Rebase on the latest part 1
 - update comments and commit messages
 - Fix error exit in arm_smmu_set_pasid()
 - Fix inverted logic for btm_invalidation
 - Add missing ATC invalidation on mm release
 - Add a big comment explaining that BTM is not enabled and what is
   missing to enable it.
v2: https://lore.kernel.org/r/[e-mail]
 - Rebased on iommmufd + Joerg's tree
 - Use sid_smmu_domain consistently to refer to the domain attached to the
   device (eg the PCIe RID)
 - Rework how arm_smmu_attach_*() and callers flow to be more careful
   about ordering around ATC invalidation. The ATC must be invalidated
   after it is impossible to establish stale entires.
 - ATS disable is now entirely part of arm_smmu_attach_dev_ste(), which is
   the only STE type that ever disables ATS.
 - Remove the 'existing_master_domain' optimization, the code is
   functionally fine without it.
 - Whitespace, spelling, and checkpatch related items
 - Fixed wrong value stored in the xa for the BTM flows
 - Use pasid more consistently instead of id
v1: https://lore.kernel.org/r/[e-mail]

Jason Gunthorpe (29):
  iommu: Validate the PASID in iommu_attach_device_pasid()
  iommu/arm-smmu-v3: Add cpu_to_le64() around STRTAB_STE_0_V
  iommu/arm-smmu-v3: Do not allow a SVA domain to be set on the wrong
    PASID
  iommu/arm-smmu-v3: Do not ATC invalidate the entire domain
  iommu/arm-smmu-v3: Add a type for the CD entry
  iommu/arm-smmu-v3: Add an ops indirection to the STE code
  iommu/arm-smmu-v3: Make CD programming use arm_smmu_write_entry()
  iommu/arm-smmu-v3: Move the CD generation for S1 domains into a
    function
  iommu/arm-smmu-v3: Consolidate clearing a CD table entry
  iommu/arm-smmu-v3: Make arm_smmu_alloc_cd_ptr()
  iommu/arm-smmu-v3: Allocate the CD table entry in advance
  iommu/arm-smmu-v3: Move the CD generation for SVA into a function
  iommu/arm-smmu-v3: Build the whole CD in arm_smmu_make_s1_cd()
  iommu/arm-smmu-v3: Start building a generic PASID layer
  iommu/arm-smmu-v3: Make smmu_domain->devices into an allocated list
  iommu/arm-smmu-v3: Make changing domains be hitless for ATS
  iommu/arm-smmu-v3: Add ssid to struct arm_smmu_master_domain
  iommu/arm-smmu-v3: Do not use master->sva_enable to restrict attaches
  iommu/arm-smmu-v3: Thread SSID through the arm_smmu_attach_*()
    interface
  iommu/arm-smmu-v3: Make SVA allocate a normal arm_smmu_domain
  iommu/arm-smmu-v3: Keep track of arm_smmu_master_domain for SVA
  iommu: Add ops->domain_alloc_sva()
  iommu/arm-smmu-v3: Put the SVA mmu notifier in the smmu_domain
  iommu/arm-smmu-v3: Consolidate freeing the ASID/VMID
  iommu/arm-smmu-v3: Move the arm_smmu_asid_xa to per-smmu like vmid
  iommu/arm-smmu-v3: Bring back SVA BTM support
  iommu/arm-smmu-v3: Allow IDENTITY/BLOCKED to be set while PASID is
    used
  iommu/arm-smmu-v3: Allow a PASID to be set when RID is
    IDENTITY/BLOCKED
  iommu/arm-smmu-v3: Allow setting a S1 domain to a PASID

 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  637 +++++-----
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 1118 +++++++++++------
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   76 +-
 drivers/iommu/iommu-sva.c                     |   16 +-
 drivers/iommu/iommu.c                         |   11 +-
 include/linux/iommu.h                         |    3 +
 6 files changed, 1086 insertions(+), 775 deletions(-)


base-commit: 4cece764965020c22cff7665b18a012006359095

Comments

Will Deacon April 9, 2024, 12:54 p.m. UTC | #1

On Wed, 27 Mar 2024 15:07:46 -0300, Jason Gunthorpe wrote:
> Continuing the work of part 1 this focuses on the CD, PASID and SVA
> components:
> 
>  - attach_dev failure does not change the HW configuration.
> 
>  - Full PASID API support including:
>     - S1/SVA domains attached to PASIDs
>     - IDENTITY/BLOCKED/S1 attached to RID
>     - Change of the RID domain while PASIDs are attached
> 
> [...]

In the interests of making a dent (albeit a tiny one!) in this series
while review is ongoing, I've picked up 3-5 as they look ready to go.

Applied 3-5 to will (for-joerg/arm-smmu/updates), thanks!

[03/29] iommu/arm-smmu-v3: Do not allow a SVA domain to be set on the wrong PASID
        https://git.kernel.org/will/c/fdc69d39e77f
[04/29] iommu/arm-smmu-v3: Do not ATC invalidate the entire domain
        https://git.kernel.org/will/c/86e5ca098dd9
[05/29] iommu/arm-smmu-v3: Add a type for the CD entry
        https://git.kernel.org/will/c/e8e4398d53f9

Cheers,

Jason Gunthorpe April 9, 2024, 11:55 p.m. UTC | #2

On Tue, Apr 09, 2024 at 01:54:59PM +0100, Will Deacon wrote:
> On Wed, 27 Mar 2024 15:07:46 -0300, Jason Gunthorpe wrote:
> > Continuing the work of part 1 this focuses on the CD, PASID and SVA
> > components:
> > 
> >  - attach_dev failure does not change the HW configuration.
> > 
> >  - Full PASID API support including:
> >     - S1/SVA domains attached to PASIDs
> >     - IDENTITY/BLOCKED/S1 attached to RID
> >     - Change of the RID domain while PASIDs are attached
> > 
> > [...]
> 
> In the interests of making a dent (albeit a tiny one!) in this series
> while review is ongoing, I've picked up 3-5 as they look ready to go.

Great, thanks!

Jason