diff mbox series

[v1,06/14] iommu/arm-smmu-v3: Unset corresponding STE fields when s2_cfg is NULL

Message ID 995e48fe6eb9e31c71dbe8bb80d445aa34a51819.1678348754.git.nicolinc@nvidia.com (mailing list archive)
State New, archived
Headers show
Series Add Nested Translation Support for SMMUv3 | expand

Commit Message

Nicolin Chen March 9, 2023, 10:53 a.m. UTC 
From: Eric Auger <eric.auger@redhat.com>

Despite the spec does not seem to mention this, on some implementations,
when the STE configuration switches from an S1+S2 cfg to an S1 only one,
a C_BAD_STE error would happen if dst[3] (S2TTB) is not reset.

Explicitly reset those two higher 64b fields, to prevent that.

Note that this is not a bug at this moment, since a 2-stage translation
setup is not yet enabled, until the following patches add its support.

Reported-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
 1 file changed, 3 insertions(+)

Comments

Robin Murphy March 9, 2023, 1:13 p.m. UTC | #1 
On 2023-03-09 10:53, Nicolin Chen wrote:
> From: Eric Auger <eric.auger@redhat.com>
> 
> Despite the spec does not seem to mention this, on some implementations,
> when the STE configuration switches from an S1+S2 cfg to an S1 only one,
> a C_BAD_STE error would happen if dst[3] (S2TTB) is not reset.

Can you provide more details, since it's not clear whether this is a 
hardware erratum workaround or a bodge around the driver itself doing 
something wrong like not doing a proper break-before-make transition of 
the STE. The architecture explicitly states that all the STE.S2* fields 
except S2VMID and potentially S2S are ignored when Stage 2 is bypassed.

Thanks,
Robin.

> Explicitly reset those two higher 64b fields, to prevent that.
> 
> Note that this is not a bug at this moment, since a 2-stage translation
> setup is not yet enabled, until the following patches add its support.
> 
> Reported-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---
>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index c5616145e2a3..29e36448d23b 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -1361,6 +1361,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
>   		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
>   
>   		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
> +	}  else {
> +		dst[2] = 0;
> +		dst[3] = 0;
>   	}
>   
>   	if (master->ats_enabled)
Shameer Kolothum March 9, 2023, 6:24 p.m. UTC | #2 
> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: 09 March 2023 13:13
> To: Nicolin Chen <nicolinc@nvidia.com>; jgg@nvidia.com; will@kernel.org
> Cc: eric.auger@redhat.com; kevin.tian@intel.com; baolu.lu@linux.intel.com;
> joro@8bytes.org; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>; jean-philippe@linaro.org;
> linux-arm-kernel@lists.infradead.org; iommu@lists.linux.dev;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v1 06/14] iommu/arm-smmu-v3: Unset corresponding
> STE fields when s2_cfg is NULL
> 
> On 2023-03-09 10:53, Nicolin Chen wrote:
> > From: Eric Auger <eric.auger@redhat.com>
> >
> > Despite the spec does not seem to mention this, on some implementations,
> > when the STE configuration switches from an S1+S2 cfg to an S1 only one,
> > a C_BAD_STE error would happen if dst[3] (S2TTB) is not reset.
> 
> Can you provide more details, since it's not clear whether this is a
> hardware erratum workaround or a bodge around the driver itself doing
> something wrong like not doing a proper break-before-make transition of
> the STE. The architecture explicitly states that all the STE.S2* fields
> except S2VMID and potentially S2S are ignored when Stage 2 is bypassed.

Took a while to locate the email thread where this was discussed,
https://patchwork.kernel.org/cover/11449895/#23244457

This was observed on a HiSilicon implementation where, if the SMMUv3 is configured with
both Stage 1 and Stage 2 (nested) mode once, then it is not possible to configure it back
for Stage 1 mode for the same device(stream id).

IIRC, the SMMUv3 implementation on these boards expects to set the S2TTB field in STE to zero
when using S1, otherwise it reports C_BAD_STE error. :(

You are right that the specification doesn't demand this and I am not sure there are any other
Hardware that requires this.

Could we please have this with a comment added in the code?

Thanks,
Shameer
Nicolin Chen March 10, 2023, 1:54 a.m. UTC | #3 
On Thu, Mar 09, 2023 at 06:24:29PM +0000, Shameerali Kolothum Thodi wrote:
> External email: Use caution opening links or attachments
> 
> 
> > -----Original Message-----
> > From: Robin Murphy [mailto:robin.murphy@arm.com]
> > Sent: 09 March 2023 13:13
> > To: Nicolin Chen <nicolinc@nvidia.com>; jgg@nvidia.com; will@kernel.org
> > Cc: eric.auger@redhat.com; kevin.tian@intel.com; baolu.lu@linux.intel.com;
> > joro@8bytes.org; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>; jean-philippe@linaro.org;
> > linux-arm-kernel@lists.infradead.org; iommu@lists.linux.dev;
> > linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH v1 06/14] iommu/arm-smmu-v3: Unset corresponding
> > STE fields when s2_cfg is NULL
> >
> > On 2023-03-09 10:53, Nicolin Chen wrote:
> > > From: Eric Auger <eric.auger@redhat.com>
> > >
> > > Despite the spec does not seem to mention this, on some implementations,
> > > when the STE configuration switches from an S1+S2 cfg to an S1 only one,
> > > a C_BAD_STE error would happen if dst[3] (S2TTB) is not reset.
> >
> > Can you provide more details, since it's not clear whether this is a
> > hardware erratum workaround or a bodge around the driver itself doing
> > something wrong like not doing a proper break-before-make transition of
> > the STE. The architecture explicitly states that all the STE.S2* fields
> > except S2VMID and potentially S2S are ignored when Stage 2 is bypassed.
> 
> Took a while to locate the email thread where this was discussed,
> https://patchwork.kernel.org/cover/11449895/#23244457
> 
> This was observed on a HiSilicon implementation where, if the SMMUv3 is configured with
> both Stage 1 and Stage 2 (nested) mode once, then it is not possible to configure it back
> for Stage 1 mode for the same device(stream id).
> 
> IIRC, the SMMUv3 implementation on these boards expects to set the S2TTB field in STE to zero
> when using S1, otherwise it reports C_BAD_STE error. :(
> 
> You are right that the specification doesn't demand this and I am not sure there are any other
> Hardware that requires this.
> 
> Could we please have this with a comment added in the code?

Yes, I can add that, and put that link in the commit message too.

Thanks
Nicolin
diff mbox series

Patch

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index c5616145e2a3..29e36448d23b 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1361,6 +1361,9 @@  static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 		dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK);
 
 		val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS);
+	}  else {
+		dst[2] = 0;
+		dst[3] = 0;
 	}
 
 	if (master->ats_enabled)