| Message ID | 20190117092718.1396-3-vivek.gautam@codeaurora.org (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | iommu/arm: Add support for non-coherent page tables | expand |
On Thu, Jan 17, 2019 at 02:57:18PM +0530, Vivek Gautam wrote: > Adding a device tree option for arm smmu to enable non-cacheable > memory for page tables. > We already enable a smmu feature for coherent walk based on > whether the smmu device is dma-coherent or not. Have an option > to enable non-cacheable page table memory to force set it for > particular smmu devices. Hmm, I must be missing something here. What is the difference between this new property, and simply omitting dma-coherent on the SMMU? Will
Hi Will, On Sun, Jan 20, 2019 at 5:31 AM Will Deacon <will.deacon@arm.com> wrote: > > On Thu, Jan 17, 2019 at 02:57:18PM +0530, Vivek Gautam wrote: > > Adding a device tree option for arm smmu to enable non-cacheable > > memory for page tables. > > We already enable a smmu feature for coherent walk based on > > whether the smmu device is dma-coherent or not. Have an option > > to enable non-cacheable page table memory to force set it for > > particular smmu devices. > > Hmm, I must be missing something here. What is the difference between this > new property, and simply omitting dma-coherent on the SMMU? So, this is what I understood from the email thread for Last level cache support - Robin pointed to the fact that we may need to add support for setting non-cacheable mappings in the TCR. Currently, we don't do that for SMMUs that omit dma-coherent. We rely on the interconnect to handle the configuration set in TCR, and let interconnect ignore the cacheability if it can't support. Moreover, Robin suggested that we should take care of SMMUs, for which removing snoop latency on walks by making mappings as non-cacheable outweighs the cost of cache maintenance on PTE updates. So, this change adds another property to do this non-cacheable mappings explicitly. As I pointed, omitting 'dma-coherent', and corresponding IO_PGTABLE_QUIRK_NO_DMA' does takes care of few things. Should we handle the TCR settings too with this quirk? Regards Vivek > > Will > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
On Mon, Jan 21, 2019 at 11:35:30AM +0530, Vivek Gautam wrote: > On Sun, Jan 20, 2019 at 5:31 AM Will Deacon <will.deacon@arm.com> wrote: > > On Thu, Jan 17, 2019 at 02:57:18PM +0530, Vivek Gautam wrote: > > > Adding a device tree option for arm smmu to enable non-cacheable > > > memory for page tables. > > > We already enable a smmu feature for coherent walk based on > > > whether the smmu device is dma-coherent or not. Have an option > > > to enable non-cacheable page table memory to force set it for > > > particular smmu devices. > > > > Hmm, I must be missing something here. What is the difference between this > > new property, and simply omitting dma-coherent on the SMMU? > > So, this is what I understood from the email thread for Last level > cache support - > Robin pointed to the fact that we may need to add support for setting > non-cacheable > mappings in the TCR. > Currently, we don't do that for SMMUs that omit dma-coherent. > We rely on the interconnect to handle the configuration set in TCR, > and let interconnect > ignore the cacheability if it can't support. I think that's a bug. With that fixed, can you get what you want by omitting "dma-coherent"? Will
Hi Will, On Tue, Jan 22, 2019 at 11:14 AM Will Deacon <will.deacon@arm.com> wrote: > > On Mon, Jan 21, 2019 at 11:35:30AM +0530, Vivek Gautam wrote: > > On Sun, Jan 20, 2019 at 5:31 AM Will Deacon <will.deacon@arm.com> wrote: > > > On Thu, Jan 17, 2019 at 02:57:18PM +0530, Vivek Gautam wrote: > > > > Adding a device tree option for arm smmu to enable non-cacheable > > > > memory for page tables. > > > > We already enable a smmu feature for coherent walk based on > > > > whether the smmu device is dma-coherent or not. Have an option > > > > to enable non-cacheable page table memory to force set it for > > > > particular smmu devices. > > > > > > Hmm, I must be missing something here. What is the difference between this > > > new property, and simply omitting dma-coherent on the SMMU? > > > > So, this is what I understood from the email thread for Last level > > cache support - > > Robin pointed to the fact that we may need to add support for setting > > non-cacheable > > mappings in the TCR. > > Currently, we don't do that for SMMUs that omit dma-coherent. > > We rely on the interconnect to handle the configuration set in TCR, > > and let interconnect > > ignore the cacheability if it can't support. > > I think that's a bug. With that fixed, can you get what you want by omitting > "dma-coherent"? Based on the discussion on the first patch in this series [1], I can update the series. First thing can be - if QUIRK_NO_DMA is set (i.e. the IOMMU _is_ coherent) then we use a cacheable TCR; So, we may need an additional check for this when setting the TCR. For the second case - IOMMUs that are *not* coherent, i.e ones that are omitting 'dma-coherent' property, anyways have to access the page table directly from memory. We take care of the CPU side of this by allocating non-coherent memory, and making sure that we sync the PTEs from map call. Shouldn't we mark TCR for these IOMMUs as non-cacheable for inner and outer cacheability attribute? [1] https://lore.kernel.org/patchwork/patch/1032939/ Regards Vivek > > Will
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index af18a7e7f917..7ebbcf1b2eb3 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -188,6 +188,7 @@ struct arm_smmu_device { u32 features; #define ARM_SMMU_OPT_SECURE_CFG_ACCESS (1 << 0) +#define ARM_SMMU_OPT_PGTBL_NON_COHERENT (1 << 1) u32 options; enum arm_smmu_arch_version version; enum arm_smmu_implementation model; @@ -273,6 +274,7 @@ static bool using_legacy_binding, using_generic_binding; static struct arm_smmu_option_prop arm_smmu_options[] = { { ARM_SMMU_OPT_SECURE_CFG_ACCESS, "calxeda,smmu-secure-config-access" }, + { ARM_SMMU_OPT_PGTBL_NON_COHERENT, "arm,smmu-pgtable-non-coherent" }, { 0, NULL}, }; @@ -902,6 +904,11 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, if (smmu_domain->non_strict) pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT; + /* Non coherent page table mappings only for Stage-1 */ + if (smmu->options & ARM_SMMU_OPT_PGTBL_NON_COHERENT && + smmu_domain->stage == ARM_SMMU_DOMAIN_S1) + pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_COHERENT; + smmu_domain->smmu = smmu; pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain); if (!pgtbl_ops) {
Adding a device tree option for arm smmu to enable non-cacheable memory for page tables. We already enable a smmu feature for coherent walk based on whether the smmu device is dma-coherent or not. Have an option to enable non-cacheable page table memory to force set it for particular smmu devices. Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org> --- drivers/iommu/arm-smmu.c | 7 +++++++ 1 file changed, 7 insertions(+)