| Message ID | 1580249770-1088-3-git-send-email-jcrouse@codeaurora.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | iommu/arm-smmu: Auxiliary domain and per instance pagetables | expand |
On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > Support auxiliary domains for arm-smmu-v2 to initialize and support > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > hardware doesn't have any built in support for switching the pagetable > base it is left as an exercise to the caller to actually use the pagetable. > > Aux domains are supported if split pagetable (TTBR1) support has been > enabled on the master domain. Each auxiliary domain will reuse the > configuration of the master domain. By default the a domain with TTBR1 > support will have the TTBR0 region disabled so the first attached aux > domain will enable the TTBR0 region in the hardware and conversely the > last domain to be detached will disable TTBR0 translations. All subsequent > auxiliary domains create a pagetable but not touch the hardware. > > The leaf driver will be able to query the physical address of the > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > address with whatever means it has to switch the pagetable base. > > Following is a pseudo code example of how a domain can be created > > /* Check to see if aux domains are supported */ > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > iommu = iommu_domain_alloc(...); > > if (iommu_aux_attach_device(domain, dev)) > return FAIL; > > /* Save the base address of the pagetable for use by the driver > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > } I'm not really understanding what the pagetable base gets used for here and, to be honest with you, the whole thing feels like a huge layering violation with the way things are structured today. Why doesn't the caller just interface with io-pgtable directly? Finally, if we need to support context-switching TTBR0 for a live domain then that code really needs to live inside the SMMU driver because the ASID and TLB management necessary to do that safely doesn't belong anywhere else. Will
On Wed, Mar 18, 2020 at 3:48 PM Will Deacon <will@kernel.org> wrote: > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > hardware doesn't have any built in support for switching the pagetable > > base it is left as an exercise to the caller to actually use the pagetable. > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > enabled on the master domain. Each auxiliary domain will reuse the > > configuration of the master domain. By default the a domain with TTBR1 > > support will have the TTBR0 region disabled so the first attached aux > > domain will enable the TTBR0 region in the hardware and conversely the > > last domain to be detached will disable TTBR0 translations. All subsequent > > auxiliary domains create a pagetable but not touch the hardware. > > > > The leaf driver will be able to query the physical address of the > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > address with whatever means it has to switch the pagetable base. > > > > Following is a pseudo code example of how a domain can be created > > > > /* Check to see if aux domains are supported */ > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > iommu = iommu_domain_alloc(...); > > > > if (iommu_aux_attach_device(domain, dev)) > > return FAIL; > > > > /* Save the base address of the pagetable for use by the driver > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > } > > I'm not really understanding what the pagetable base gets used for here and, > to be honest with you, the whole thing feels like a huge layering violation > with the way things are structured today. Why doesn't the caller just > interface with io-pgtable directly? > > Finally, if we need to support context-switching TTBR0 for a live domain > then that code really needs to live inside the SMMU driver because the > ASID and TLB management necessary to do that safely doesn't belong anywhere > else. Hi Will, We do in fact need live domain switching, that is really the whole point. The GPU CP (command processor/parser) is directly updating TTBR0 and triggering TLB flush, asynchronously from the CPU. And I think the answer about ASID is easy (on current hw).. it must be zero[*]. BR, -R [*] my rough theory/plan there, and to solve the issue with drm/msm getting dma-iommu ops when it really would rather not (since blacklisting idea wasn't popular and I couldn't figure out a way to deal with case where device gets attached before driver shows up) is to invent some API that drm/msm can call to unhook the dma-iommu ops and detatch the DMA domain. Hopefully that at least gets us closer to the point where, when drm/msm attaches it's UNMANAGED domain, we get cbidx/asid zero.
On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > On Wed, Mar 18, 2020 at 3:48 PM Will Deacon <will@kernel.org> wrote: > > > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > > hardware doesn't have any built in support for switching the pagetable > > > base it is left as an exercise to the caller to actually use the pagetable. > > > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > > enabled on the master domain. Each auxiliary domain will reuse the > > > configuration of the master domain. By default the a domain with TTBR1 > > > support will have the TTBR0 region disabled so the first attached aux > > > domain will enable the TTBR0 region in the hardware and conversely the > > > last domain to be detached will disable TTBR0 translations. All subsequent > > > auxiliary domains create a pagetable but not touch the hardware. > > > > > > The leaf driver will be able to query the physical address of the > > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > > address with whatever means it has to switch the pagetable base. > > > > > > Following is a pseudo code example of how a domain can be created > > > > > > /* Check to see if aux domains are supported */ > > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > > iommu = iommu_domain_alloc(...); > > > > > > if (iommu_aux_attach_device(domain, dev)) > > > return FAIL; > > > > > > /* Save the base address of the pagetable for use by the driver > > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > > } > > > > I'm not really understanding what the pagetable base gets used for here and, > > to be honest with you, the whole thing feels like a huge layering violation > > with the way things are structured today. Why doesn't the caller just > > interface with io-pgtable directly? > > > > Finally, if we need to support context-switching TTBR0 for a live domain > > then that code really needs to live inside the SMMU driver because the > > ASID and TLB management necessary to do that safely doesn't belong anywhere > > else. > > Hi Will, > > We do in fact need live domain switching, that is really the whole > point. The GPU CP (command processor/parser) is directly updating > TTBR0 and triggering TLB flush, asynchronously from the CPU. Right. This is entirely done in hardware with a GPU that has complete access to the context bank registers. All the driver does is send the PTBASE to the command stream see [1] and especially [2] (look for CP_SMMU_TABLE_UPDATE). As for interacting with the io-pgtable directly I would love to do that but it would need some new infrastructure to either pull the io-pgtable from the aux domain or to create an io-pgtable ourselves and pass it for use by the aux domain. I'm not sure if that is better for the layering violation. > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. Right now the GPU microcode still uses TLBIALL. I want to assign each new aux domain its own ASID in the hopes that we could some day change that but for now having a uinque ASID doesn't help. Jordan [1] https://patchwork.freedesktop.org/patch/351089/ [2] https://patchwork.freedesktop.org/patch/351090/
On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > On Wed, Mar 18, 2020 at 3:48 PM Will Deacon <will@kernel.org> wrote: > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > > hardware doesn't have any built in support for switching the pagetable > > > base it is left as an exercise to the caller to actually use the pagetable. > > > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > > enabled on the master domain. Each auxiliary domain will reuse the > > > configuration of the master domain. By default the a domain with TTBR1 > > > support will have the TTBR0 region disabled so the first attached aux > > > domain will enable the TTBR0 region in the hardware and conversely the > > > last domain to be detached will disable TTBR0 translations. All subsequent > > > auxiliary domains create a pagetable but not touch the hardware. > > > > > > The leaf driver will be able to query the physical address of the > > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > > address with whatever means it has to switch the pagetable base. > > > > > > Following is a pseudo code example of how a domain can be created > > > > > > /* Check to see if aux domains are supported */ > > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > > iommu = iommu_domain_alloc(...); > > > > > > if (iommu_aux_attach_device(domain, dev)) > > > return FAIL; > > > > > > /* Save the base address of the pagetable for use by the driver > > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > > } > > > > I'm not really understanding what the pagetable base gets used for here and, > > to be honest with you, the whole thing feels like a huge layering violation > > with the way things are structured today. Why doesn't the caller just > > interface with io-pgtable directly? > > > > Finally, if we need to support context-switching TTBR0 for a live domain > > then that code really needs to live inside the SMMU driver because the > > ASID and TLB management necessary to do that safely doesn't belong anywhere > > else. > > We do in fact need live domain switching, that is really the whole > point. The GPU CP (command processor/parser) is directly updating > TTBR0 and triggering TLB flush, asynchronously from the CPU. > > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. Using ASID zero is really bad, because it means that you will end up sharing TLB entries with whichever device is using context bank 0. Is the SMMU only used by the GPU in your SoC? Will
On Mon, May 18, 2020 at 8:18 AM Will Deacon <will@kernel.org> wrote: > > On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > > On Wed, Mar 18, 2020 at 3:48 PM Will Deacon <will@kernel.org> wrote: > > > On Tue, Jan 28, 2020 at 03:16:06PM -0700, Jordan Crouse wrote: > > > > Support auxiliary domains for arm-smmu-v2 to initialize and support > > > > multiple pagetables for a single SMMU context bank. Since the smmu-v2 > > > > hardware doesn't have any built in support for switching the pagetable > > > > base it is left as an exercise to the caller to actually use the pagetable. > > > > > > > > Aux domains are supported if split pagetable (TTBR1) support has been > > > > enabled on the master domain. Each auxiliary domain will reuse the > > > > configuration of the master domain. By default the a domain with TTBR1 > > > > support will have the TTBR0 region disabled so the first attached aux > > > > domain will enable the TTBR0 region in the hardware and conversely the > > > > last domain to be detached will disable TTBR0 translations. All subsequent > > > > auxiliary domains create a pagetable but not touch the hardware. > > > > > > > > The leaf driver will be able to query the physical address of the > > > > pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the > > > > address with whatever means it has to switch the pagetable base. > > > > > > > > Following is a pseudo code example of how a domain can be created > > > > > > > > /* Check to see if aux domains are supported */ > > > > if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { > > > > iommu = iommu_domain_alloc(...); > > > > > > > > if (iommu_aux_attach_device(domain, dev)) > > > > return FAIL; > > > > > > > > /* Save the base address of the pagetable for use by the driver > > > > iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); > > > > } > > > > > > I'm not really understanding what the pagetable base gets used for here and, > > > to be honest with you, the whole thing feels like a huge layering violation > > > with the way things are structured today. Why doesn't the caller just > > > interface with io-pgtable directly? > > > > > > Finally, if we need to support context-switching TTBR0 for a live domain > > > then that code really needs to live inside the SMMU driver because the > > > ASID and TLB management necessary to do that safely doesn't belong anywhere > > > else. > > > > We do in fact need live domain switching, that is really the whole > > point. The GPU CP (command processor/parser) is directly updating > > TTBR0 and triggering TLB flush, asynchronously from the CPU. > > > > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. > > Using ASID zero is really bad, because it means that you will end up sharing > TLB entries with whichever device is using context bank 0. > > Is the SMMU only used by the GPU in your SoC? > yes, the snapdragon SoCs have two SMMU instances, one used by the GPU, where ASID0/cb0 is the gpu itself, and another cb is the GMU (basically power control for the gpu), and the second SMMU is everything else. BR, -R
On Mon, May 18, 2020 at 08:50:27AM -0700, Rob Clark wrote: > On Mon, May 18, 2020 at 8:18 AM Will Deacon <will@kernel.org> wrote: > > On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > > > We do in fact need live domain switching, that is really the whole > > > point. The GPU CP (command processor/parser) is directly updating > > > TTBR0 and triggering TLB flush, asynchronously from the CPU. > > > > > > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. > > > > Using ASID zero is really bad, because it means that you will end up sharing > > TLB entries with whichever device is using context bank 0. > > > > Is the SMMU only used by the GPU in your SoC? > > > > yes, the snapdragon SoCs have two SMMU instances, one used by the GPU, > where ASID0/cb0 is the gpu itself, and another cb is the GMU > (basically power control for the gpu), and the second SMMU is > everything else. Right, in which case I'm starting to think that we should treat this GPU SMMU instance specially. Give it its own compatible string (looks like you need this for HUPCFG anyway) and hook in via arm_smmu_impl_init(). You can then set IO_PGTABLE_QUIRK_ARM_TTBR1 when talking to the io-pgtable code without having to add a domain attribute. With that. you'll need to find a way to allow the GPU driver to call into your own hooks for getting at the TTBR0 tables -- given that you're programming these in the hardware, I don't think it makes sense to expose that in the IOMMU API, since most devices won't be able to do anything with that data. Perhaps you could install a couple of function pointers (subdomain_alloc/subdomain_free) in the GPU device when you see it appear from the SMMU driver? Alternatively, you could make an io_pgtable_cfg available so that the GPU driver can interface with io-pgtable directly. Yes, it's ugly, but I don't think it's worth trying to abstract this. Thoughts? It's taken me a long time to figure out what's going on here, so sorry if it feels like I'm leading you round the houses. Will
On Wed, May 20, 2020 at 01:57:01PM +0100, Will Deacon wrote: > On Mon, May 18, 2020 at 08:50:27AM -0700, Rob Clark wrote: > > On Mon, May 18, 2020 at 8:18 AM Will Deacon <will@kernel.org> wrote: > > > On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > > > > We do in fact need live domain switching, that is really the whole > > > > point. The GPU CP (command processor/parser) is directly updating > > > > TTBR0 and triggering TLB flush, asynchronously from the CPU. > > > > > > > > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. > > > > > > Using ASID zero is really bad, because it means that you will end up sharing > > > TLB entries with whichever device is using context bank 0. > > > > > > Is the SMMU only used by the GPU in your SoC? > > > > > > > yes, the snapdragon SoCs have two SMMU instances, one used by the GPU, > > where ASID0/cb0 is the gpu itself, and another cb is the GMU > > (basically power control for the gpu), and the second SMMU is > > everything else. > > Right, in which case I'm starting to think that we should treat this GPU > SMMU instance specially. Give it its own compatible string (looks like you > need this for HUPCFG anyway) and hook in via arm_smmu_impl_init(). You can > then set IO_PGTABLE_QUIRK_ARM_TTBR1 when talking to the io-pgtable code > without having to add a domain attribute. If we did this via a special GPU SMMU instance then we could also create and register a dummy TTBR0 instance along with the TTBR1 instance and then we wouldn't need to worry about the aux domains at all. > With that. you'll need to find a way to allow the GPU driver to call into > your own hooks for getting at the TTBR0 tables -- given that you're > programming these in the hardware, I don't think it makes sense to expose > that in the IOMMU API, since most devices won't be able to do anything with > that data. Perhaps you could install a couple of function pointers > (subdomain_alloc/subdomain_free) in the GPU device when you see it appear > from the SMMU driver? Alternatively, you could make an io_pgtable_cfg > available so that the GPU driver can interface with io-pgtable directly. I don't want to speak for Rob but I think that this is the same direction we've landed on. If we use the implementation specific code to initialize the base pagetables then the GPU driver can use io-pgtable directly. We can easily construct an io_pgtable_cfg. This feature will only be available for opt-in GPU targets that will have a known configuration. The only gotcha is TLB maintenance but Rob and I have ideas about coordinating with the GPU hardware (which has to do a TLBIALL during a switch anyway) and we can always use the iommu_tlb_flush_all() hammer from software if we really need it. It might take a bit of thought, but it is doable. > Yes, it's ugly, but I don't think it's worth trying to abstract this. I'm not sure how ugly it is. I've always operated under the assumption that the GPU SMMU was special (though it had generic registers) just because of where it was and how it it was used. In the long run baking in a implementation specific solution would probably be preferable to lots of domain attributes and aux domains that would never be used except by us. > Thoughts? It's taken me a long time to figure out what's going on here, > so sorry if it feels like I'm leading you round the houses. I'll hack on this and try to get something in place. It might be dumber on the GPU side than we would like but it would at least spur some more conversation. Jordan > Will
On Wed, May 20, 2020 at 8:13 AM Jordan Crouse <jcrouse@codeaurora.org> wrote: > > On Wed, May 20, 2020 at 01:57:01PM +0100, Will Deacon wrote: > > On Mon, May 18, 2020 at 08:50:27AM -0700, Rob Clark wrote: > > > On Mon, May 18, 2020 at 8:18 AM Will Deacon <will@kernel.org> wrote: > > > > On Wed, Mar 18, 2020 at 04:43:07PM -0700, Rob Clark wrote: > > > > > We do in fact need live domain switching, that is really the whole > > > > > point. The GPU CP (command processor/parser) is directly updating > > > > > TTBR0 and triggering TLB flush, asynchronously from the CPU. > > > > > > > > > > And I think the answer about ASID is easy (on current hw).. it must be zero[*]. > > > > > > > > Using ASID zero is really bad, because it means that you will end up sharing > > > > TLB entries with whichever device is using context bank 0. > > > > > > > > Is the SMMU only used by the GPU in your SoC? > > > > > > > > > > yes, the snapdragon SoCs have two SMMU instances, one used by the GPU, > > > where ASID0/cb0 is the gpu itself, and another cb is the GMU > > > (basically power control for the gpu), and the second SMMU is > > > everything else. > > > > Right, in which case I'm starting to think that we should treat this GPU > > SMMU instance specially. Give it its own compatible string (looks like you > > need this for HUPCFG anyway) and hook in via arm_smmu_impl_init(). You can > > then set IO_PGTABLE_QUIRK_ARM_TTBR1 when talking to the io-pgtable code > > without having to add a domain attribute. > > If we did this via a special GPU SMMU instance then we could also create and > register a dummy TTBR0 instance along with the TTBR1 instance and then we > wouldn't need to worry about the aux domains at all. > > > With that. you'll need to find a way to allow the GPU driver to call into > > your own hooks for getting at the TTBR0 tables -- given that you're > > programming these in the hardware, I don't think it makes sense to expose > > that in the IOMMU API, since most devices won't be able to do anything with > > that data. Perhaps you could install a couple of function pointers > > (subdomain_alloc/subdomain_free) in the GPU device when you see it appear > > from the SMMU driver? Alternatively, you could make an io_pgtable_cfg > > available so that the GPU driver can interface with io-pgtable directly. > > I don't want to speak for Rob but I think that this is the same direction we've > landed on. If we use the implementation specific code to initialize the base > pagetables then the GPU driver can use io-pgtable directly. We can easily > construct an io_pgtable_cfg. This feature will only be available for opt-in > GPU targets that will have a known configuration. Agreed about using io-pgtable helpers directly.. the gpu's use-case is pretty far different from anything normal/sane, and I don't think it is worth designing some generic iommu interfaces with precisely one user[*]. We just need enough in arm-smmu(/-impl) to bootstrap things when we power up the gpu. BR, -R [*] all the other gpu's that I've seen so far, even if they sit behind an iommu, they have their own internal mmu > The only gotcha is TLB maintenance but Rob and I have ideas about coordinating > with the GPU hardware (which has to do a TLBIALL during a switch anyway) and we > can always use the iommu_tlb_flush_all() hammer from software if we really need > it. It might take a bit of thought, but it is doable. > > > Yes, it's ugly, but I don't think it's worth trying to abstract this. > > I'm not sure how ugly it is. I've always operated under the assumption that the > GPU SMMU was special (though it had generic registers) just because of where it > was and how it it was used. In the long run baking in a implementation specific > solution would probably be preferable to lots of domain attributes and aux > domains that would never be used except by us. > > > Thoughts? It's taken me a long time to figure out what's going on here, > > so sorry if it feels like I'm leading you round the houses. > > I'll hack on this and try to get something in place. It might be dumber on the > GPU side than we would like but it would at least spur some more conversation. > > Jordan > > > Will > > -- > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 23b22fa..85a6773 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -91,6 +91,8 @@ struct arm_smmu_cb { u32 tcr[2]; u32 mair[2]; struct arm_smmu_cfg *cfg; + atomic_t aux; + atomic_t refcount; }; struct arm_smmu_master_cfg { @@ -533,6 +535,7 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain, struct arm_smmu_cb *cb = &smmu_domain->smmu->cbs[cfg->cbndx]; bool stage1 = cfg->cbar != CBAR_TYPE_S2_TRANS; + atomic_inc(&cb->refcount); cb->cfg = cfg; /* TCR */ @@ -671,6 +674,91 @@ static void arm_smmu_write_context_bank(struct arm_smmu_device *smmu, int idx) arm_smmu_cb_write(smmu, idx, ARM_SMMU_CB_SCTLR, reg); } +/* + * Update the context context bank to enable TTBR0. Assumes AARCH64 S1 + * configuration. + */ +static void arm_smmu_context_set_ttbr0(struct arm_smmu_cb *cb, + struct io_pgtable_cfg *pgtbl_cfg) +{ + u32 tcr = cb->tcr[0]; + + /* Add the TCR configuration from the new pagetable config */ + tcr |= arm_smmu_lpae_tcr(pgtbl_cfg); + + /* Make sure that both TTBR0 and TTBR1 are enabled */ + tcr &= ~(ARM_SMMU_TCR_EPD0 | ARM_SMMU_TCR_EPD1); + + /* Udate the TCR register */ + cb->tcr[0] = tcr; + + /* Program the new TTBR0 */ + cb->ttbr[0] = pgtbl_cfg->arm_lpae_s1_cfg.ttbr; + cb->ttbr[0] |= FIELD_PREP(ARM_SMMU_TTBRn_ASID, cb->cfg->asid); +} + +/* + * Thus function assumes that the current model only allows aux domains for + * AARCH64 S1 configurations + */ +static int arm_smmu_aux_init_domain_context(struct iommu_domain *domain, + struct arm_smmu_device *smmu, struct arm_smmu_cfg *master) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *pgtbl_ops; + struct io_pgtable_cfg pgtbl_cfg; + + mutex_lock(&smmu_domain->init_mutex); + + /* Copy the configuration from the master */ + memcpy(&smmu_domain->cfg, master, sizeof(smmu_domain->cfg)); + + smmu_domain->flush_ops = &arm_smmu_s1_tlb_ops; + smmu_domain->smmu = smmu; + + pgtbl_cfg = (struct io_pgtable_cfg) { + .pgsize_bitmap = smmu->pgsize_bitmap, + .ias = smmu->va_size, + .oas = smmu->ipa_size, + .coherent_walk = smmu->features & ARM_SMMU_FEAT_COHERENT_WALK, + .tlb = smmu_domain->flush_ops, + .iommu_dev = smmu->dev, + .quirks = 0, + }; + + if (smmu_domain->non_strict) + pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT; + + pgtbl_ops = alloc_io_pgtable_ops(ARM_64_LPAE_S1, &pgtbl_cfg, + smmu_domain); + if (!pgtbl_ops) { + mutex_unlock(&smmu_domain->init_mutex); + return -ENOMEM; + } + + domain->pgsize_bitmap = pgtbl_cfg.pgsize_bitmap; + + domain->geometry.aperture_end = (1UL << smmu->va_size) - 1; + domain->geometry.force_aperture = true; + + /* + * Add a reference to the context bank to make sure it stays programmed + * wile we are attached + */ + atomic_inc(&smmu->cbs[master->cbndx].refcount); + + /* enable TTBR0 when the the first aux domain is attached */ + if (atomic_inc_return(&smmu->cbs[master->cbndx].aux) == 1) { + arm_smmu_context_set_ttbr0(&smmu->cbs[master->cbndx], + &pgtbl_cfg); + arm_smmu_write_context_bank(smmu, master->cbndx); + } + + smmu_domain->ptbase = pgtbl_cfg.arm_lpae_s1_cfg.ttbr; + smmu_domain->pgtbl_ops = pgtbl_ops; + return 0; +} + static int arm_smmu_init_domain_context(struct iommu_domain *domain, struct arm_smmu_device *smmu) { @@ -882,36 +970,70 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, return ret; } +static void +arm_smmu_destroy_aux_domain_context(struct arm_smmu_domain *smmu_domain) +{ + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_cfg *cfg = &smmu_domain->cfg; + int ret; + + /* + * If this is the last aux domain to be freed, disable TTBR0 by turning + * off translations and clearing TTBR0 + */ + if (atomic_dec_return(&smmu->cbs[cfg->cbndx].aux) == 0) { + /* Clear out the T0 region */ + smmu->cbs[cfg->cbndx].tcr[0] &= ~GENMASK(15, 0); + /* Disable TTBR0 translations */ + smmu->cbs[cfg->cbndx].tcr[0] |= ARM_SMMU_TCR_EPD0; + /* Clear the TTBR0 pagetable address */ + smmu->cbs[cfg->cbndx].ttbr[0] = + FIELD_PREP(ARM_SMMU_TTBRn_ASID, cfg->asid); + + ret = arm_smmu_rpm_get(smmu); + if (!ret) { + arm_smmu_write_context_bank(smmu, cfg->cbndx); + arm_smmu_rpm_put(smmu); + } + } + +} + static void arm_smmu_destroy_domain_context(struct iommu_domain *domain) { struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_cfg *cfg = &smmu_domain->cfg; - int ret, irq; if (!smmu || domain->type == IOMMU_DOMAIN_IDENTITY) return; - ret = arm_smmu_rpm_get(smmu); - if (ret < 0) - return; + if (smmu_domain->aux) + arm_smmu_destroy_aux_domain_context(smmu_domain); - /* - * Disable the context bank and free the page tables before freeing - * it. - */ - smmu->cbs[cfg->cbndx].cfg = NULL; - arm_smmu_write_context_bank(smmu, cfg->cbndx); + /* Check if the last user is done with the context bank */ + if (atomic_dec_return(&smmu->cbs[cfg->cbndx].refcount) == 0) { + int ret = arm_smmu_rpm_get(smmu); + int irq; - if (cfg->irptndx != ARM_SMMU_INVALID_IRPTNDX) { - irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - devm_free_irq(smmu->dev, irq, domain); + if (ret < 0) + return; + + /* Disable the context bank */ + smmu->cbs[cfg->cbndx].cfg = NULL; + arm_smmu_write_context_bank(smmu, cfg->cbndx); + + if (cfg->irptndx != ARM_SMMU_INVALID_IRPTNDX) { + irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; + devm_free_irq(smmu->dev, irq, domain); + } + + __arm_smmu_free_bitmap(smmu->context_map, cfg->cbndx); + arm_smmu_rpm_put(smmu); } + /* Destroy the pagetable */ free_io_pgtable_ops(smmu_domain->pgtbl_ops); - __arm_smmu_free_bitmap(smmu->context_map, cfg->cbndx); - - arm_smmu_rpm_put(smmu); } static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) @@ -1181,6 +1303,73 @@ static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain, return 0; } +static bool arm_smmu_dev_has_feat(struct device *dev, + enum iommu_dev_features feat) +{ + if (feat != IOMMU_DEV_FEAT_AUX) + return false; + + return true; +} + +static int arm_smmu_dev_enable_feat(struct device *dev, + enum iommu_dev_features feat) +{ + /* aux domain support is always available */ + if (feat == IOMMU_DEV_FEAT_AUX) + return 0; + + return -ENODEV; +} + +static int arm_smmu_dev_disable_feat(struct device *dev, + enum iommu_dev_features feat) +{ + return -EBUSY; +} + +static int arm_smmu_aux_attach_dev(struct iommu_domain *domain, + struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + struct arm_smmu_device *smmu = fwspec_smmu(fwspec); + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_cb *cb; + int idx, i, ret, cbndx = -1; + + /* Try to find the context bank configured for this device */ + for_each_cfg_sme(fwspec, i, idx) { + if (idx != INVALID_SMENDX) { + cbndx = smmu->s2crs[idx].cbndx; + break; + } + } + + if (cbndx == -1) + return -ENODEV; + + cb = &smmu->cbs[cbndx]; + + /* Aux domains are only supported for AARCH64 configurations */ + if (cb->cfg->fmt != ARM_SMMU_CTX_FMT_AARCH64) + return -EINVAL; + + /* Make sure that TTBR1 is enabled in the hardware */ + if ((cb->tcr[0] & ARM_SMMU_TCR_EPD1)) + return -EINVAL; + + smmu_domain->aux = true; + + ret = arm_smmu_rpm_get(smmu); + if (ret < 0) + return ret; + + ret = arm_smmu_aux_init_domain_context(domain, smmu, cb->cfg); + + arm_smmu_rpm_put(smmu); + return ret; +} + static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret; @@ -1549,6 +1738,11 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain, case DOMAIN_ATTR_SPLIT_TABLES: *(int *)data = smmu_domain->split_pagetables; return 0; + case DOMAIN_ATTR_PTBASE: + if (!smmu_domain->aux) + return -EINVAL; + *(u64 *)data = smmu_domain->ptbase; + return 0; default: return -ENODEV; } @@ -1667,6 +1861,10 @@ static struct iommu_ops arm_smmu_ops = { .of_xlate = arm_smmu_of_xlate, .get_resv_regions = arm_smmu_get_resv_regions, .put_resv_regions = generic_iommu_put_resv_regions, + .dev_has_feat = arm_smmu_dev_has_feat, + .dev_enable_feat = arm_smmu_dev_enable_feat, + .dev_disable_feat = arm_smmu_dev_disable_feat, + .aux_attach_dev = arm_smmu_aux_attach_dev, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; diff --git a/drivers/iommu/arm-smmu.h b/drivers/iommu/arm-smmu.h index 53053fd..8266b29 100644 --- a/drivers/iommu/arm-smmu.h +++ b/drivers/iommu/arm-smmu.h @@ -345,6 +345,9 @@ struct arm_smmu_domain { spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ struct iommu_domain domain; bool split_pagetables; + bool aux; + /* Shadow of the pagetable base for aux domains */ + u64 ptbase; }; static inline u32 arm_smmu_lpae_tcr(struct io_pgtable_cfg *cfg)
Support auxiliary domains for arm-smmu-v2 to initialize and support multiple pagetables for a single SMMU context bank. Since the smmu-v2 hardware doesn't have any built in support for switching the pagetable base it is left as an exercise to the caller to actually use the pagetable. Aux domains are supported if split pagetable (TTBR1) support has been enabled on the master domain. Each auxiliary domain will reuse the configuration of the master domain. By default the a domain with TTBR1 support will have the TTBR0 region disabled so the first attached aux domain will enable the TTBR0 region in the hardware and conversely the last domain to be detached will disable TTBR0 translations. All subsequent auxiliary domains create a pagetable but not touch the hardware. The leaf driver will be able to query the physical address of the pagetable with the DOMAIN_ATTR_PTBASE attribute so that it can use the address with whatever means it has to switch the pagetable base. Following is a pseudo code example of how a domain can be created /* Check to see if aux domains are supported */ if (iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_AUX)) { iommu = iommu_domain_alloc(...); if (iommu_aux_attach_device(domain, dev)) return FAIL; /* Save the base address of the pagetable for use by the driver iommu_domain_get_attr(domain, DOMAIN_ATTR_PTBASE, &ptbase); } Then 'domain' can be used like any other iommu domain to map and unmap iova addresses in the pagetable. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/iommu/arm-smmu.c | 230 +++++++++++++++++++++++++++++++++++++++++++---- drivers/iommu/arm-smmu.h | 3 + 2 files changed, 217 insertions(+), 16 deletions(-)