From patchwork Tue Sep 12 12:01:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Gautam X-Patchwork-Id: 9949061 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9958560325 for ; Tue, 12 Sep 2017 12:02:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88E87285B0 for ; Tue, 12 Sep 2017 12:02:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D87628F5C; Tue, 12 Sep 2017 12:02:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A2170285B0 for ; Tue, 12 Sep 2017 12:02:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=z1sdVwWdj7OSu/agxWIli5NfvZHpfHVTyIVpOlx/0Ek=; b=Lsz 58HDZ7i92qs3E3Jaf8YDTbHE8o8xeA9ykosroQ9iB+4dvwWdjxWXctIwKHrILh6fwEsxdlBr712TA 0GB/t0YcnGnJfYC1Dm/Cn/cumK0MLQiBg1ox1qnR3Qm90lVn4bAhsaC1kYv25LD9ZDBByArldoT9b NuuWlxUz54uNqMHfm/FM12+4q/MNMv2dFE6CMuhF8JnrtWJC5JjkSUHA9Q1LEKeDArL8Yf5O9GH9O HU2EJkhLuGKuwYtMZwvx6nHS2uljB/LxCwzVPRoH/tirZujfRZq45jGU/4PaYsJFDaZpAzL1mQPRt K7OyUdV3QxiNbb/WWL4n4dA14uCwJyQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1drjt2-0000et-Fq; Tue, 12 Sep 2017 12:01:52 +0000 Received: from smtp.codeaurora.org ([198.145.29.96]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1drjst-00006c-I1 for linux-arm-kernel@lists.infradead.org; Tue, 12 Sep 2017 12:01:49 +0000 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 962E66087E; Tue, 12 Sep 2017 12:01:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1505217682; bh=bRn3nulZdTG8faFdW74ncecQD4PmPBBs+STOltctnl8=; h=From:To:Cc:Subject:Date:From; b=EpvSuOdSgaO/QPH4c6MqH7QWc2QfOZZJnvsT6s5znlHE8BN/abqPi7fxm6S2t0bIr ud2XtejyK5W9I0dUgB538c3esxkXhRKvvV+lmDWGWFpRWaXCoiPTvG2cQUAs2CwFAg jpcJOiXylVfKOZvXAqSkrqSIKSvyIdGP0Byy+tGk= Received: from blr-ubuntu-41.ap.qualcomm.com (blr-bdr-fw-01_globalnat_allzones-outside.qualcomm.com [103.229.18.19]) (using TLSv1.1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: vivek.gautam@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 79005607DC; Tue, 12 Sep 2017 12:01:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1505217680; bh=bRn3nulZdTG8faFdW74ncecQD4PmPBBs+STOltctnl8=; h=From:To:Cc:Subject:Date:From; b=N9MLK9nUEDVAGDj5jR+drLKRvM2jJMn5rPvE9ivoknnI8sHRjCYG39m1DDk/ipxI7 dxKkBdJYVfz2YNa76E4laVfp7Nzmh6VbPZmDvg4CRx/pNULW7lY7i2571G74WUvKqL e6dLoeOfty25TjjSf+uU6Y42MhJwC9glKNZJc55o= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 79005607DC Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=vivek.gautam@codeaurora.org From: Vivek Gautam To: robin.murphy@arm.com, will.deacon@arm.com, robh+dt@kernel.org, iommu@lists.linux-foundation.org, devicetree@vger.kernel.org Subject: [RFC PATCH 1/1] iommu/arm-smmu: Add support for multiple TBU child devices Date: Tue, 12 Sep 2017 17:31:07 +0530 Message-Id: <1505217667-31184-1-git-send-email-vivek.gautam@codeaurora.org> X-Mailer: git-send-email 1.9.1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170912_050143_793505_2246FCA9 X-CRM114-Status: GOOD ( 22.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, linux-arm-msm@vger.kernel.org, joro@8bytes.org, linux-kernel@vger.kernel.org, Vivek Gautam , linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP ARM MMU-500 implements a TBU (uTLB) for each connected master besides a single TCU which controls and manages the address translations. Each of these TBUs can either be in the same power domain as the master, or they can have a independent power switch. This design addresses the challenges to control TBU power. Adding child devices for each TBUs present in the configuration lets us control the power and clocks to TLBs having individual power domains. The device link between master devices (such as, display, and GPU) and TBU devices ensures that the master takes care of powering the smmu as long as it's available. When the master is not available, the TBUs are identified with sid and powered on. Signed-off-by: Vivek Gautam --- - The idea behind this patch is to handle the distributed smmu architectures, similar to MMU-500. - Untested yet. - There are still few instances where the correct tbu device has to be referenced and thus powered on to handle TLB maintenance operations. .../devicetree/bindings/iommu/arm,smmu.txt | 27 +++ drivers/iommu/arm-smmu.c | 191 +++++++++++++++++++-- 2 files changed, 205 insertions(+), 13 deletions(-) diff --git a/Documentation/devicetree/bindings/iommu/arm,smmu.txt b/Documentation/devicetree/bindings/iommu/arm,smmu.txt index d97a6bc8e608..7cf67e75022e 100644 --- a/Documentation/devicetree/bindings/iommu/arm,smmu.txt +++ b/Documentation/devicetree/bindings/iommu/arm,smmu.txt @@ -98,6 +98,18 @@ conditions. accessed before master gets enabled and linked to its SMMU. +- child nodes: ARM MMU-500 implements a TBU (page table cache or TLB) for + each connected master besides a single TCU that controls + and manages the address translations. + Each of the child nodes represents a TBU that is attached to + the master. This child node will have following property: + + - compatibe: must be "arm,mmu-500-tbu" for TBU child nodes of arm,mmu-500 + smmu. + - stream-id-range: array representing the starting stream id and the number + of supported stream-ids. This gives information about + the range of stream-ids that are supported by this TBU. + ** Deprecated properties: - mmu-masters (deprecated in favour of the generic "iommus" binding) : @@ -187,3 +199,18 @@ conditions. <&mmcc SMMU_MDP_AHB_CLK>; clock-names = "bus", "iface"; }; + + smmu4: iommu { + compatible = "arm,mmu-500"; + reg = <0xd00000 0x10000>; + ... + ... + #address-cells = <1>; + #size-cells = <0>; + + tbu: tbu@0 { + compatible = "arm,mmu-500-tbu"; + reg = <0>; + stream-id-range = <0x0 0x400>; + }; + }; diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 14137e389071..ba84693e2de3 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -224,6 +224,8 @@ struct arm_smmu_device { /* IOMMU core code handle */ struct iommu_device iommu; + + struct list_head tbu_list; }; enum arm_smmu_context_fmt { @@ -333,6 +335,33 @@ static int __find_legacy_master_phandle(struct device *dev, void *data) static struct platform_driver arm_smmu_driver; static struct iommu_ops arm_smmu_ops; +struct arm_smmu500_tbu { + struct device *dev; + u32 sid_range[2]; + struct list_head list; + struct arm_smmu_device *smmu; +}; + +static struct device *find_tbu(struct device *parent, u16 sid) +{ + struct arm_smmu_device *smmu = dev_get_drvdata(parent); + struct arm_smmu500_tbu *tbu; + u32 start, end; + struct device *dev = NULL; + + list_for_each_entry(tbu, &smmu->tbu_list, list) { + start = tbu->sid_range[0]; + end = start + tbu->sid_range[1]; + + if (start <= sid && sid < end) { + dev = tbu->dev; + break; + } + } + + return dev; +} + static int arm_smmu_register_legacy_master(struct device *dev, struct arm_smmu_device **smmu) { @@ -927,6 +956,7 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain) if (!smmu || domain->type == IOMMU_DOMAIN_IDENTITY) return; + /* TODO Get the TBU dev power it on; or turn on all the TBUs? */ ret = pm_runtime_get_sync(smmu->dev); if (ret) return; @@ -1379,6 +1409,8 @@ static int arm_smmu_add_device(struct device *dev) struct iommu_fwspec *fwspec = dev->iommu_fwspec; struct device_link *link; int i, ret; + u16 sid; + struct device *tbu_dev; if (using_legacy_binding) { ret = arm_smmu_register_legacy_master(dev, &smmu); @@ -1425,29 +1457,42 @@ static int arm_smmu_add_device(struct device *dev) while (i--) cfg->smendx[i] = INVALID_SMENDX; - ret = pm_runtime_get_sync(smmu->dev); + /* + * Find out the correct TBU for this master based on the + * steamID, and then create the device link between them. + */ + /* TODO: Iterate over all the SIDs, and turn them on? */ + sid = fwspec->ids[0]; + tbu_dev = find_tbu(smmu->dev, sid); + if (!tbu_dev) + tbu_dev = smmu->dev; + + ret = pm_runtime_get_sync(tbu_dev); if (ret) goto out_cfg_free; ret = arm_smmu_master_alloc_smes(dev); if (ret) { - pm_runtime_put_sync(smmu->dev); + pm_runtime_put_sync(tbu_dev); goto out_cfg_free; } iommu_device_link(&smmu->iommu, dev); - pm_runtime_put_sync(smmu->dev); + pm_runtime_put_sync(tbu_dev); /* * Establish the link between smmu and master, so that the * smmu gets runtime enabled/disabled as per the master's * needs. + * TBU devices are children to the smmu device, so creating link + * between TBU devices and master takes care of turning on the smmu + * as well. */ - link = device_link_add(dev, smmu->dev, DL_FLAG_PM_RUNTIME); + link = device_link_add(dev, tbu_dev, DL_FLAG_PM_RUNTIME); if (!link) dev_warn(smmu->dev, "Unable to create device link between %s and %s\n", - dev_name(smmu->dev), dev_name(dev)); + dev_name(tbu_dev), dev_name(dev)); return 0; @@ -1464,6 +1509,8 @@ static void arm_smmu_remove_device(struct device *dev) struct arm_smmu_master_cfg *cfg; struct arm_smmu_device *smmu; int ret; + u16 sid; + struct device *tbu_dev; if (!fwspec || fwspec->ops != &arm_smmu_ops) return; @@ -1476,15 +1523,20 @@ static void arm_smmu_remove_device(struct device *dev) * smmu is already purged at this point. * So enable the power to smmu explicitly. */ + sid = fwspec->ids[0]; - ret = pm_runtime_get_sync(smmu->dev); + tbu_dev = find_tbu(smmu->dev, sid); + if (!tbu_dev) + tbu_dev = smmu->dev; + + ret = pm_runtime_get_sync(tbu_dev); if (ret) return; iommu_device_unlink(&smmu->iommu, dev); arm_smmu_master_free_smes(fwspec); - pm_runtime_put_sync(smmu->dev); + pm_runtime_put_sync(tbu_dev); iommu_group_remove_device(dev); kfree(fwspec->iommu_priv); @@ -2012,6 +2064,32 @@ struct arm_smmu_match_data { }; MODULE_DEVICE_TABLE(of, arm_smmu_of_match); +static const struct of_device_id arm_smmu_tbu_of_match[] = { + { .compatible = "arm,mmu-500-tbu", }, + {}, +}; +MODULE_DEVICE_TABLE(of, arm_smmu_tbu_of_match); + +static int arm_smmu_tbu_register(struct device *dev, void *cookie) +{ + struct arm_smmu_device *smmu = cookie; + struct arm_smmu500_tbu *tbu; + + if (!dev->driver) { + dev_err(dev, "TBU failed probe, SMMUV500 cannot continue!\n"); + return -EINVAL; + } + + tbu = dev_get_drvdata(dev); + + INIT_LIST_HEAD(&tbu->list); + tbu->smmu = smmu; + list_add_tail(&tbu->list, &smmu->tbu_list); + + return 0; +} + + #ifdef CONFIG_ACPI static int acpi_smmu_get_data(u32 model, struct arm_smmu_device *smmu) { @@ -2140,6 +2218,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev) resource_size_t ioaddr; struct arm_smmu_device *smmu; struct device *dev = &pdev->dev; + struct arm_smmu500_tbu *tbu; int num_irqs, i, err; smmu = devm_kzalloc(dev, sizeof(*smmu), GFP_KERNEL); @@ -2157,6 +2236,8 @@ static int arm_smmu_device_probe(struct platform_device *pdev) if (err) return err; + INIT_LIST_HEAD(&smmu->tbu_list); + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); ioaddr = res->start; smmu->base = devm_ioremap_resource(dev, res); @@ -2194,16 +2275,32 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->irqs[i] = irq; } + /* MMU 500 has one TCU and multiple TBU config */ + if (of_device_is_compatible(dev->of_node, "arm,mmu-500")) { + err = of_platform_populate(dev->of_node, NULL, NULL, dev); + if (err) { + dev_err(dev, "failed to register TBU children\n"); + return err; + } + + err = device_for_each_child(dev, smmu, arm_smmu_tbu_register); + if (err) + return -EPROBE_DEFER; + } + err = arm_smmu_init_clocks(smmu); if (err) return err; platform_set_drvdata(pdev, smmu); + pm_runtime_enable(dev); - err = pm_runtime_get_sync(dev); - if (err) - return err; + /* enable all TBUs */ + list_for_each_entry(tbu, &smmu->tbu_list, list) + err = pm_runtime_get_sync(tbu->dev); + if (err) + return err; err = arm_smmu_device_cfg_probe(smmu); if (err) @@ -2248,7 +2345,9 @@ static int arm_smmu_device_probe(struct platform_device *pdev) arm_smmu_device_reset(smmu); arm_smmu_test_smr_masks(smmu); - pm_runtime_put_sync(dev); + + list_for_each_entry(tbu, &smmu->tbu_list, list) + pm_runtime_put_sync(tbu->dev); /* * For ACPI and generic DT bindings, an SMMU will be probed before @@ -2278,6 +2377,7 @@ static int arm_smmu_legacy_bus_init(void) static int arm_smmu_device_remove(struct platform_device *pdev) { struct arm_smmu_device *smmu = platform_get_drvdata(pdev); + struct arm_smmu500_tbu *tbu; if (!smmu) return -ENODEV; @@ -2287,7 +2387,9 @@ static int arm_smmu_device_remove(struct platform_device *pdev) /* Turn the thing off */ writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); - pm_runtime_force_suspend(smmu->dev); + + list_for_each_entry(tbu, &smmu->tbu_list, list) + pm_runtime_force_suspend(tbu->dev); return 0; } @@ -2336,7 +2438,70 @@ static int __maybe_unused arm_smmu_suspend(struct device *dev) .remove = arm_smmu_device_remove, .shutdown = arm_smmu_device_shutdown, }; -module_platform_driver(arm_smmu_driver); + +static int arm_smmu_tbu_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct arm_smmu500_tbu *tbu; + int err; + + tbu = devm_kzalloc(dev, sizeof(*tbu), GFP_KERNEL); + if (!tbu) + return -ENOMEM; + + tbu->dev = &pdev->dev; + + err = of_property_read_u32_array(dev->of_node, "stream-id-range", + tbu->sid_range, 2); + if (err) + dev_err(dev, "failed to read stream id range\n"); + + pm_runtime_enable(dev); + + dev_set_drvdata(dev, tbu); + + return 0; +} + +static int arm_smmu_tbu_remove(struct platform_device *pdev) +{ + pm_runtime_disable(&pdev->dev); + + return 0; +} + +static struct platform_driver arm_smmu_tbu_driver = { + .driver = { + .name = "arm-mmu-500-tbu", + .of_match_table = of_match_ptr(arm_smmu_tbu_of_match), + }, + .probe = arm_smmu_tbu_probe, + .remove = arm_smmu_tbu_remove, +}; + +static int __init arm_smmu_init(void) +{ + int ret; + + ret = platform_driver_register(&arm_smmu_tbu_driver); + if (ret) + return ret; + + ret = platform_driver_register(&arm_smmu_driver); + if (ret) + platform_driver_unregister(&arm_smmu_tbu_driver); + + return ret; +} + +static void __init arm_smmu_exit(void) +{ + platform_driver_unregister(&arm_smmu_driver); + platform_driver_unregister(&arm_smmu_tbu_driver); +} + +module_init(arm_smmu_init); +module_exit(arm_smmu_exit); IOMMU_OF_DECLARE(arm_smmuv1, "arm,smmu-v1", NULL); IOMMU_OF_DECLARE(arm_smmuv2, "arm,smmu-v2", NULL);