| Message ID | 20200626200042.13713-5-jcrouse@codeaurora.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | iommu/arm-smmu: Enable split pagetable support | expand |
On Fri, Jun 26, 2020 at 02:00:38PM -0600, Jordan Crouse wrote: > Add a link to the pointer to the struct device that is attached to a > domain. This makes it easy to get the pointer if it is needed in the > implementation specific code. > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> > --- > > drivers/iommu/arm-smmu.c | 6 ++++-- > drivers/iommu/arm-smmu.h | 1 + > 2 files changed, 5 insertions(+), 2 deletions(-) > > diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c > index 048de2681670..060139452c54 100644 > --- a/drivers/iommu/arm-smmu.c > +++ b/drivers/iommu/arm-smmu.c > @@ -668,7 +668,8 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) > } > > static int arm_smmu_init_domain_context(struct iommu_domain *domain, > - struct arm_smmu_device *smmu) > + struct arm_smmu_device *smmu, > + struct device *dev) > { > int irq, start, ret = 0; > unsigned long ias, oas; > @@ -801,6 +802,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, > cfg->asid = cfg->cbndx; > > smmu_domain->smmu = smmu; > + smmu_domain->dev = dev; > > pgtbl_cfg = (struct io_pgtable_cfg) { > .pgsize_bitmap = smmu->pgsize_bitmap, > @@ -1190,7 +1192,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > return ret; > > /* Ensure that the domain is finalised */ > - ret = arm_smmu_init_domain_context(domain, smmu); > + ret = arm_smmu_init_domain_context(domain, smmu, dev); > if (ret < 0) > goto rpm_put; > > diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h > index 5f2de20e883b..d33cfe26b2f5 100644 > --- a/drivers/iommu/arm-smmu.h > +++ b/drivers/iommu/arm-smmu.h > @@ -345,6 +345,7 @@ struct arm_smmu_domain { > struct mutex init_mutex; /* Protects smmu pointer */ > spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ > struct iommu_domain domain; > + struct device *dev; /* Device attached to this domain */ This really doesn't feel right to me -- you can generally have multiple devices attached to a domain and they can come and go without the domain being destroyed. Perhaps you could instead identify the GPU during cfg_probe() and squirrel that information away somewhere? The rest of the series looks ok to me. Will
On Mon, Jul 13, 2020 at 04:09:02PM +0100, Will Deacon wrote: > On Fri, Jun 26, 2020 at 02:00:38PM -0600, Jordan Crouse wrote: > > Add a link to the pointer to the struct device that is attached to a > > domain. This makes it easy to get the pointer if it is needed in the > > implementation specific code. > > > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> > > --- > > > > drivers/iommu/arm-smmu.c | 6 ++++-- > > drivers/iommu/arm-smmu.h | 1 + > > 2 files changed, 5 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c > > index 048de2681670..060139452c54 100644 > > --- a/drivers/iommu/arm-smmu.c > > +++ b/drivers/iommu/arm-smmu.c > > @@ -668,7 +668,8 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) > > } > > > > static int arm_smmu_init_domain_context(struct iommu_domain *domain, > > - struct arm_smmu_device *smmu) > > + struct arm_smmu_device *smmu, > > + struct device *dev) > > { > > int irq, start, ret = 0; > > unsigned long ias, oas; > > @@ -801,6 +802,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, > > cfg->asid = cfg->cbndx; > > > > smmu_domain->smmu = smmu; > > + smmu_domain->dev = dev; > > > > pgtbl_cfg = (struct io_pgtable_cfg) { > > .pgsize_bitmap = smmu->pgsize_bitmap, > > @@ -1190,7 +1192,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > > return ret; > > > > /* Ensure that the domain is finalised */ > > - ret = arm_smmu_init_domain_context(domain, smmu); > > + ret = arm_smmu_init_domain_context(domain, smmu, dev); > > if (ret < 0) > > goto rpm_put; > > > > diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h > > index 5f2de20e883b..d33cfe26b2f5 100644 > > --- a/drivers/iommu/arm-smmu.h > > +++ b/drivers/iommu/arm-smmu.h > > @@ -345,6 +345,7 @@ struct arm_smmu_domain { > > struct mutex init_mutex; /* Protects smmu pointer */ > > spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ > > struct iommu_domain domain; > > + struct device *dev; /* Device attached to this domain */ > > This really doesn't feel right to me -- you can generally have multiple > devices attached to a domain and they can come and go without the domain > being destroyed. Perhaps you could instead identify the GPU during > cfg_probe() and squirrel that information away somewhere? I need some help here. The SMMU device (qcom,adreno-smmu) will have at least two stream ids from two different platform devices (GPU and GMU) and I need to configure split-pagetable and stall/terminate differently on the two domains. I couldn't figure out a way to identify the platform device before it attached itself with iommu_attach_device. I tried poking around in fwspec but got lost. If there is a way we can uniquely identify the devices (by stream id maybe) then we could use that though I have reservations about hard coding stream IDs in the impl driver. That said, the stream IDs have never changed in the life of the GPU so maybe it's not a problem that needs solving. Jordan > The rest of the series looks ok to me. > > Will > _______________________________________________ > Freedreno mailing list > Freedreno@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/freedreno
On Mon, Jul 13, 2020 at 11:19:17AM -0600, Jordan Crouse wrote: > On Mon, Jul 13, 2020 at 04:09:02PM +0100, Will Deacon wrote: > > On Fri, Jun 26, 2020 at 02:00:38PM -0600, Jordan Crouse wrote: > > > diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h > > > index 5f2de20e883b..d33cfe26b2f5 100644 > > > --- a/drivers/iommu/arm-smmu.h > > > +++ b/drivers/iommu/arm-smmu.h > > > @@ -345,6 +345,7 @@ struct arm_smmu_domain { > > > struct mutex init_mutex; /* Protects smmu pointer */ > > > spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ > > > struct iommu_domain domain; > > > + struct device *dev; /* Device attached to this domain */ > > > > This really doesn't feel right to me -- you can generally have multiple > > devices attached to a domain and they can come and go without the domain > > being destroyed. Perhaps you could instead identify the GPU during > > cfg_probe() and squirrel that information away somewhere? > > I need some help here. The SMMU device (qcom,adreno-smmu) will have at least two > stream ids from two different platform devices (GPU and GMU) and I need to > configure split-pagetable and stall/terminate differently on the two domains. Hmm. How does the GPU driver know which context bank is assigned to the GPU and which one is assigned to the GMU? I assume it needs this information so that it can play its nasty tricks with the TTBR registers? I ask because if we need to guarantee stability of the context-bank assignment, then you could match on that in the ->init_context() callback, but now I worry that it currently works by luck :/ Do we need to add an extra callback to allocate the context bank? Will
On Thu, Jul 16, 2020 at 1:51 AM Will Deacon <will@kernel.org> wrote: > > On Mon, Jul 13, 2020 at 11:19:17AM -0600, Jordan Crouse wrote: > > On Mon, Jul 13, 2020 at 04:09:02PM +0100, Will Deacon wrote: > > > On Fri, Jun 26, 2020 at 02:00:38PM -0600, Jordan Crouse wrote: > > > > diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h > > > > index 5f2de20e883b..d33cfe26b2f5 100644 > > > > --- a/drivers/iommu/arm-smmu.h > > > > +++ b/drivers/iommu/arm-smmu.h > > > > @@ -345,6 +345,7 @@ struct arm_smmu_domain { > > > > struct mutex init_mutex; /* Protects smmu pointer */ > > > > spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ > > > > struct iommu_domain domain; > > > > + struct device *dev; /* Device attached to this domain */ > > > > > > This really doesn't feel right to me -- you can generally have multiple > > > devices attached to a domain and they can come and go without the domain > > > being destroyed. Perhaps you could instead identify the GPU during > > > cfg_probe() and squirrel that information away somewhere? > > > > I need some help here. The SMMU device (qcom,adreno-smmu) will have at least two > > stream ids from two different platform devices (GPU and GMU) and I need to > > configure split-pagetable and stall/terminate differently on the two domains. > > Hmm. How does the GPU driver know which context bank is assigned to the GPU > and which one is assigned to the GMU? I assume it needs this information so > that it can play its nasty tricks with the TTBR registers? > > I ask because if we need to guarantee stability of the context-bank > assignment, then you could match on that in the ->init_context() callback, > but now I worry that it currently works by luck :/ Yeah, it is basically by luck right now.. some sort of impl callback which was passed the dev into impl->init_context() might do the trick (ie. then the impl could match on compat string) BR, -R > Do we need to add an extra callback to allocate the context bank? > > Will > _______________________________________________ > Freedreno mailing list > Freedreno@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/freedreno
On Thu, Jul 16, 2020 at 09:50:53AM +0100, Will Deacon wrote: > On Mon, Jul 13, 2020 at 11:19:17AM -0600, Jordan Crouse wrote: > > On Mon, Jul 13, 2020 at 04:09:02PM +0100, Will Deacon wrote: > > > On Fri, Jun 26, 2020 at 02:00:38PM -0600, Jordan Crouse wrote: > > > > diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h > > > > index 5f2de20e883b..d33cfe26b2f5 100644 > > > > --- a/drivers/iommu/arm-smmu.h > > > > +++ b/drivers/iommu/arm-smmu.h > > > > @@ -345,6 +345,7 @@ struct arm_smmu_domain { > > > > struct mutex init_mutex; /* Protects smmu pointer */ > > > > spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ > > > > struct iommu_domain domain; > > > > + struct device *dev; /* Device attached to this domain */ > > > > > > This really doesn't feel right to me -- you can generally have multiple > > > devices attached to a domain and they can come and go without the domain > > > being destroyed. Perhaps you could instead identify the GPU during > > > cfg_probe() and squirrel that information away somewhere? > > > > I need some help here. The SMMU device (qcom,adreno-smmu) will have at least two > > stream ids from two different platform devices (GPU and GMU) and I need to > > configure split-pagetable and stall/terminate differently on the two domains. > > Hmm. How does the GPU driver know which context bank is assigned to the GPU > and which one is assigned to the GMU? I assume it needs this information so > that it can play its nasty tricks with the TTBR registers? > > I ask because if we need to guarantee stability of the context-bank > assignment, then you could match on that in the ->init_context() callback, > but now I worry that it currently works by luck :/ Its more like "educated" luck. If we assign the domains in the right order we know that the GPU will be on context bank 0 but you are entirely right that if everything doesn't go exactly the way we need things go badly. Some targets do have the ability to reprogram which context bank is used but that would require a domain attribute from the SMMU driver to communicate that information back to the GPU driver. > Do we need to add an extra callback to allocate the context bank? That seems like a reasonable option. That seems like it would work with legacy targets and rely less on luck. Jordan
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 048de2681670..060139452c54 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -668,7 +668,8 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) } static int arm_smmu_init_domain_context(struct iommu_domain *domain, - struct arm_smmu_device *smmu) + struct arm_smmu_device *smmu, + struct device *dev) { int irq, start, ret = 0; unsigned long ias, oas; @@ -801,6 +802,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, cfg->asid = cfg->cbndx; smmu_domain->smmu = smmu; + smmu_domain->dev = dev; pgtbl_cfg = (struct io_pgtable_cfg) { .pgsize_bitmap = smmu->pgsize_bitmap, @@ -1190,7 +1192,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return ret; /* Ensure that the domain is finalised */ - ret = arm_smmu_init_domain_context(domain, smmu); + ret = arm_smmu_init_domain_context(domain, smmu, dev); if (ret < 0) goto rpm_put; diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h index 5f2de20e883b..d33cfe26b2f5 100644 --- a/drivers/iommu/arm-smmu.h +++ b/drivers/iommu/arm-smmu.h @@ -345,6 +345,7 @@ struct arm_smmu_domain { struct mutex init_mutex; /* Protects smmu pointer */ spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ struct iommu_domain domain; + struct device *dev; /* Device attached to this domain */ }; static inline u32 arm_smmu_lpae_tcr(struct io_pgtable_cfg *cfg)
Add a link to the pointer to the struct device that is attached to a domain. This makes it easy to get the pointer if it is needed in the implementation specific code. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/iommu/arm-smmu.c | 6 ++++-- drivers/iommu/arm-smmu.h | 1 + 2 files changed, 5 insertions(+), 2 deletions(-)