| Message ID | 1565216500-28506-1-git-send-email-jcrouse@codeaurora.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | iommu/arm-smmu: Split pagetable support | expand |
On Wed, Aug 07, 2019 at 04:21:38PM -0600, Jordan Crouse wrote: > (Sigh, resend. I freaked out my SMTP server) > > This is part of an ongoing evolution for enabling split pagetable support for > arm-smmu. Previous versions can be found [1]. > > In the discussion for v2 Robin pointed out that this is a very Adreno specific > use case and that is exactly true. Not only do we want to configure and use a > pagetable in the TTBR1 space, we also want to configure the TTBR0 region but > not allocate a pagetable for it or touch it until the GPU hardware does so. As > much as I want it to be a generic concept it really isn't. > > This revision leans into that idea. Most of the same io-pgtable code is there > but now it is wrapped as an Adreno GPU specific format that is selected by the > compatible string in the arm-smmu device. > > Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable > to save on wasted memory. > > This isn't as clean as I would like it to be but I think that this is a better > direction than trying to pretend that the generic format would work. > > I'm tempting fate by posting this and then taking some time off, but I wanted > to try to kick off a conversation or at least get some flames so I can try to > refine this again next week. Please take a look and give some advice on the > direction. Will, Robin - Modulo the impl changes from Robin, do you think that using a dedicated pagetable format is the right approach for supporting split pagetables for the Adreno GPU? If so, then is adding the changes to io-pgtable-arm.c possible for 5.4 and then add the implementation specific code on top of Robin's stack later or do you feel they should come as part of a package deal? Jordan > Jordan Crouse (2): > iommu/io-pgtable-arm: Add support for ARM_ADRENO_GPU_LPAE io-pgtable > format > iommu/arm-smmu: Add support for Adreno GPU pagetable formats > > drivers/iommu/arm-smmu.c | 8 +- > drivers/iommu/io-pgtable-arm.c | 214 ++++++++++++++++++++++++++++++++++++++--- > drivers/iommu/io-pgtable.c | 1 + > include/linux/io-pgtable.h | 2 + > 4 files changed, 209 insertions(+), 16 deletions(-) > > -- > 2.7.4 > > _______________________________________________ > Freedreno mailing list > Freedreno@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/freedreno
Hi Jordan, On 15/08/2019 16:33, Jordan Crouse wrote: > On Wed, Aug 07, 2019 at 04:21:38PM -0600, Jordan Crouse wrote: >> (Sigh, resend. I freaked out my SMTP server) >> >> This is part of an ongoing evolution for enabling split pagetable support for >> arm-smmu. Previous versions can be found [1]. >> >> In the discussion for v2 Robin pointed out that this is a very Adreno specific >> use case and that is exactly true. Not only do we want to configure and use a >> pagetable in the TTBR1 space, we also want to configure the TTBR0 region but >> not allocate a pagetable for it or touch it until the GPU hardware does so. As >> much as I want it to be a generic concept it really isn't. >> >> This revision leans into that idea. Most of the same io-pgtable code is there >> but now it is wrapped as an Adreno GPU specific format that is selected by the >> compatible string in the arm-smmu device. >> >> Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable >> to save on wasted memory. >> >> This isn't as clean as I would like it to be but I think that this is a better >> direction than trying to pretend that the generic format would work. >> >> I'm tempting fate by posting this and then taking some time off, but I wanted >> to try to kick off a conversation or at least get some flames so I can try to >> refine this again next week. Please take a look and give some advice on the >> direction. > > Will, Robin - > > Modulo the impl changes from Robin, do you think that using a dedicated > pagetable format is the right approach for supporting split pagetables for the > Adreno GPU? How many different Adreno drivers would benefit from sharing it? The more I come back to this, the more I'm convinced that io-pgtable should focus on the heavy lifting of pagetable management - the code that nobody wants to have to write at all, let alone more than once - and any subtleties which aren't essential to that should be pushed back into whichever callers actually care. Consider that already, literally no caller actually uses an unmodified stage 1 TCR value as provided in the io_pgtable_cfg. I feel it would be most productive to elaborate further in the form of patches, so let me get right on that and try to bash something out before I go home tonight... Robin. > If so, then is adding the changes to io-pgtable-arm.c possible for 5.4 and then > add the implementation specific code on top of Robin's stack later or do you > feel they should come as part of a package deal? > > Jordan > >> Jordan Crouse (2): >> iommu/io-pgtable-arm: Add support for ARM_ADRENO_GPU_LPAE io-pgtable >> format >> iommu/arm-smmu: Add support for Adreno GPU pagetable formats >> >> drivers/iommu/arm-smmu.c | 8 +- >> drivers/iommu/io-pgtable-arm.c | 214 ++++++++++++++++++++++++++++++++++++++--- >> drivers/iommu/io-pgtable.c | 1 + >> include/linux/io-pgtable.h | 2 + >> 4 files changed, 209 insertions(+), 16 deletions(-) >> >> -- >> 2.7.4 >> >> _______________________________________________ >> Freedreno mailing list >> Freedreno@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/freedreno >
On Fri, Aug 16, 2019 at 9:58 AM Robin Murphy <robin.murphy@arm.com> wrote: > > Hi Jordan, > > On 15/08/2019 16:33, Jordan Crouse wrote: > > On Wed, Aug 07, 2019 at 04:21:38PM -0600, Jordan Crouse wrote: > >> (Sigh, resend. I freaked out my SMTP server) > >> > >> This is part of an ongoing evolution for enabling split pagetable support for > >> arm-smmu. Previous versions can be found [1]. > >> > >> In the discussion for v2 Robin pointed out that this is a very Adreno specific > >> use case and that is exactly true. Not only do we want to configure and use a > >> pagetable in the TTBR1 space, we also want to configure the TTBR0 region but > >> not allocate a pagetable for it or touch it until the GPU hardware does so. As > >> much as I want it to be a generic concept it really isn't. > >> > >> This revision leans into that idea. Most of the same io-pgtable code is there > >> but now it is wrapped as an Adreno GPU specific format that is selected by the > >> compatible string in the arm-smmu device. > >> > >> Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable > >> to save on wasted memory. > >> > >> This isn't as clean as I would like it to be but I think that this is a better > >> direction than trying to pretend that the generic format would work. > >> > >> I'm tempting fate by posting this and then taking some time off, but I wanted > >> to try to kick off a conversation or at least get some flames so I can try to > >> refine this again next week. Please take a look and give some advice on the > >> direction. > > > > Will, Robin - > > > > Modulo the impl changes from Robin, do you think that using a dedicated > > pagetable format is the right approach for supporting split pagetables for the > > Adreno GPU? > > How many different Adreno drivers would benefit from sharing it? Hypothetically everything back to a3xx, so I *could* see usefulness of this in qcom_iommu (or maybe even msm-iommu). OTOH maybe with "modularizing" arm-smmu we could re-combine qcom_iommu and arm-smmu. And as a practical matter, I'm not sure if anyone will get around to backporting per-context pagetables as far back as a3xx. BR, -R > The more I come back to this, the more I'm convinced that io-pgtable > should focus on the heavy lifting of pagetable management - the code > that nobody wants to have to write at all, let alone more than once - > and any subtleties which aren't essential to that should be pushed back > into whichever callers actually care. Consider that already, literally > no caller actually uses an unmodified stage 1 TCR value as provided in > the io_pgtable_cfg. > > I feel it would be most productive to elaborate further in the form of > patches, so let me get right on that and try to bash something out > before I go home tonight... > > Robin. > > > If so, then is adding the changes to io-pgtable-arm.c possible for 5.4 and then > > add the implementation specific code on top of Robin's stack later or do you > > feel they should come as part of a package deal? > > > > Jordan > > > >> Jordan Crouse (2): > >> iommu/io-pgtable-arm: Add support for ARM_ADRENO_GPU_LPAE io-pgtable > >> format > >> iommu/arm-smmu: Add support for Adreno GPU pagetable formats > >> > >> drivers/iommu/arm-smmu.c | 8 +- > >> drivers/iommu/io-pgtable-arm.c | 214 ++++++++++++++++++++++++++++++++++++++--- > >> drivers/iommu/io-pgtable.c | 1 + > >> include/linux/io-pgtable.h | 2 + > >> 4 files changed, 209 insertions(+), 16 deletions(-) > >> > >> -- > >> 2.7.4 > >> > >> _______________________________________________ > >> Freedreno mailing list > >> Freedreno@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/freedreno > > > _______________________________________________ > Freedreno mailing list > Freedreno@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/freedreno
On 16/08/2019 19:12, Rob Clark wrote: > On Fri, Aug 16, 2019 at 9:58 AM Robin Murphy <robin.murphy@arm.com> wrote: >> >> Hi Jordan, >> >> On 15/08/2019 16:33, Jordan Crouse wrote: >>> On Wed, Aug 07, 2019 at 04:21:38PM -0600, Jordan Crouse wrote: >>>> (Sigh, resend. I freaked out my SMTP server) >>>> >>>> This is part of an ongoing evolution for enabling split pagetable support for >>>> arm-smmu. Previous versions can be found [1]. >>>> >>>> In the discussion for v2 Robin pointed out that this is a very Adreno specific >>>> use case and that is exactly true. Not only do we want to configure and use a >>>> pagetable in the TTBR1 space, we also want to configure the TTBR0 region but >>>> not allocate a pagetable for it or touch it until the GPU hardware does so. As >>>> much as I want it to be a generic concept it really isn't. >>>> >>>> This revision leans into that idea. Most of the same io-pgtable code is there >>>> but now it is wrapped as an Adreno GPU specific format that is selected by the >>>> compatible string in the arm-smmu device. >>>> >>>> Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable >>>> to save on wasted memory. >>>> >>>> This isn't as clean as I would like it to be but I think that this is a better >>>> direction than trying to pretend that the generic format would work. >>>> >>>> I'm tempting fate by posting this and then taking some time off, but I wanted >>>> to try to kick off a conversation or at least get some flames so I can try to >>>> refine this again next week. Please take a look and give some advice on the >>>> direction. >>> >>> Will, Robin - >>> >>> Modulo the impl changes from Robin, do you think that using a dedicated >>> pagetable format is the right approach for supporting split pagetables for the >>> Adreno GPU? >> >> How many different Adreno drivers would benefit from sharing it? > > Hypothetically everything back to a3xx, so I *could* see usefulness of > this in qcom_iommu (or maybe even msm-iommu). OTOH maybe with > "modularizing" arm-smmu we could re-combine qcom_iommu and arm-smmu. Indeed, that's certainly something I'm planning to investigate as a future refactoring step. > And as a practical matter, I'm not sure if anyone will get around to > backporting per-context pagetables as far back as a3xx. > > BR, > -R > >> The more I come back to this, the more I'm convinced that io-pgtable >> should focus on the heavy lifting of pagetable management - the code >> that nobody wants to have to write at all, let alone more than once - >> and any subtleties which aren't essential to that should be pushed back >> into whichever callers actually care. Consider that already, literally >> no caller actually uses an unmodified stage 1 TCR value as provided in >> the io_pgtable_cfg. >> >> I feel it would be most productive to elaborate further in the form of >> patches, so let me get right on that and try to bash something out >> before I go home tonight... ...and now there's a rough WIP branch here: http://linux-arm.org/git?p=linux-rm.git;a=shortlog;h=refs/heads/iommu/pgtable I'll finish testing and polishing those patches at some point next week, probably, but hopefully they're sufficiently illustrative for the moment. Robin.
On Fri, Aug 16, 2019 at 08:43:53PM +0100, Robin Murphy wrote: > On 16/08/2019 19:12, Rob Clark wrote: > >On Fri, Aug 16, 2019 at 9:58 AM Robin Murphy <robin.murphy@arm.com> wrote: > >> > >>Hi Jordan, > >> > >>On 15/08/2019 16:33, Jordan Crouse wrote: > >>>On Wed, Aug 07, 2019 at 04:21:38PM -0600, Jordan Crouse wrote: > >>>>(Sigh, resend. I freaked out my SMTP server) > >>>> > >>>>This is part of an ongoing evolution for enabling split pagetable support for > >>>>arm-smmu. Previous versions can be found [1]. > >>>> > >>>>In the discussion for v2 Robin pointed out that this is a very Adreno specific > >>>>use case and that is exactly true. Not only do we want to configure and use a > >>>>pagetable in the TTBR1 space, we also want to configure the TTBR0 region but > >>>>not allocate a pagetable for it or touch it until the GPU hardware does so. As > >>>>much as I want it to be a generic concept it really isn't. > >>>> > >>>>This revision leans into that idea. Most of the same io-pgtable code is there > >>>>but now it is wrapped as an Adreno GPU specific format that is selected by the > >>>>compatible string in the arm-smmu device. > >>>> > >>>>Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable > >>>>to save on wasted memory. > >>>> > >>>>This isn't as clean as I would like it to be but I think that this is a better > >>>>direction than trying to pretend that the generic format would work. > >>>> > >>>>I'm tempting fate by posting this and then taking some time off, but I wanted > >>>>to try to kick off a conversation or at least get some flames so I can try to > >>>>refine this again next week. Please take a look and give some advice on the > >>>>direction. > >>> > >>>Will, Robin - > >>> > >>>Modulo the impl changes from Robin, do you think that using a dedicated > >>>pagetable format is the right approach for supporting split pagetables for the > >>>Adreno GPU? > >> > >>How many different Adreno drivers would benefit from sharing it? > > > >Hypothetically everything back to a3xx, so I *could* see usefulness of > >this in qcom_iommu (or maybe even msm-iommu). OTOH maybe with > >"modularizing" arm-smmu we could re-combine qcom_iommu and arm-smmu. > > Indeed, that's certainly something I'm planning to investigate as a future > refactoring step. > > >And as a practical matter, I'm not sure if anyone will get around to > >backporting per-context pagetables as far back as a3xx. > > > >BR, > >-R > > > >>The more I come back to this, the more I'm convinced that io-pgtable > >>should focus on the heavy lifting of pagetable management - the code > >>that nobody wants to have to write at all, let alone more than once - > >>and any subtleties which aren't essential to that should be pushed back > >>into whichever callers actually care. Consider that already, literally > >>no caller actually uses an unmodified stage 1 TCR value as provided in > >>the io_pgtable_cfg. > >> > >>I feel it would be most productive to elaborate further in the form of > >>patches, so let me get right on that and try to bash something out > >>before I go home tonight... > > ...and now there's a rough WIP branch here: > > http://linux-arm.org/git?p=linux-rm.git;a=shortlog;h=refs/heads/iommu/pgtable > > I'll finish testing and polishing those patches at some point next week, > probably, but hopefully they're sufficiently illustrative for the moment. This looks great so far. I can see where the TTBR1 stuff would fit in and I like it a lot. I'll try to have some patches ready when you are done polishing. Jordan > Robin.
(Sigh, resend. I freaked out my SMTP server) This is part of an ongoing evolution for enabling split pagetable support for arm-smmu. Previous versions can be found [1]. In the discussion for v2 Robin pointed out that this is a very Adreno specific use case and that is exactly true. Not only do we want to configure and use a pagetable in the TTBR1 space, we also want to configure the TTBR0 region but not allocate a pagetable for it or touch it until the GPU hardware does so. As much as I want it to be a generic concept it really isn't. This revision leans into that idea. Most of the same io-pgtable code is there but now it is wrapped as an Adreno GPU specific format that is selected by the compatible string in the arm-smmu device. Additionally, per Robin's suggestion we are skipping creating a TTBR0 pagetable to save on wasted memory. This isn't as clean as I would like it to be but I think that this is a better direction than trying to pretend that the generic format would work. I'm tempting fate by posting this and then taking some time off, but I wanted to try to kick off a conversation or at least get some flames so I can try to refine this again next week. Please take a look and give some advice on the direction. [1] https://patchwork.freedesktop.org/series/63403/ Jordan Jordan Crouse (2): iommu/io-pgtable-arm: Add support for ARM_ADRENO_GPU_LPAE io-pgtable format iommu/arm-smmu: Add support for Adreno GPU pagetable formats drivers/iommu/arm-smmu.c | 8 +- drivers/iommu/io-pgtable-arm.c | 214 ++++++++++++++++++++++++++++++++++++++--- drivers/iommu/io-pgtable.c | 1 + include/linux/io-pgtable.h | 2 + 4 files changed, 209 insertions(+), 16 deletions(-)