From patchwork Sun Mar 24 12:29:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Peng Fan (OSS)" X-Patchwork-Id: 13600807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FB8ACD11DD for ; Sun, 24 Mar 2024 12:21:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Cc:To:In-Reply-To: References:Message-Id:Subject:Date:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Tp6oJ9YtCtOOcBW5E96hL6hYzwyXqEIafLT1U9xka+s=; b=r4ms0orASoB1Vi2qX+medxQAMd BlOZYrcX4deZTaYPC2Bh38BRZueP7iqpXRxLvVFGIGA+KJ6V4Vewl+K5zIlJBL2vyXWbM2RvF6tYS 9IQqNscKt67RT7/Wu40DW0cawR0sfaEMy1FOB2P7ojBiad7BYFwscHQt+BtNguJAlF3tw9UUYQWWw yd7+yoqlwLOUOqw32UvV2HfC7HG/gqXUmj9clSm1ge5vtWCErD83w/F5xGwgRtbqVwLMOB1wqI2Lz dpeuZ+Um20BiOGyg4n5qiRZK6IYfKWIgqWyVYjiXptFYijvG6Yh/WwOuHNLRg6VJlm9YLyGIYVTTU FL7p1Clw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1roMqi-0000000CXDB-47SH; Sun, 24 Mar 2024 12:21:17 +0000 Received: from mail-db8eur05on2062d.outbound.protection.outlook.com ([2a01:111:f400:7e1a::62d] helo=EUR05-DB8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1roMqc-0000000CX5K-1ZLc for linux-arm-kernel@lists.infradead.org; Sun, 24 Mar 2024 12:21:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TLb3H0vDf3V0thM6p380iCjhw6zJT6RQ2VC1oZTC9HwyJ8d5QiONn2Oi1VtaYvLgsDU83Y81mxpgYTcc635meQHzh5cufb4RvNnBhPs9JbmKXKheyuoA1OaIo9FrPIxZon71etM6mixeCvcYH4krHvTLdDAswqwtg9nwXZHvoJm4sNDObj0C3Ajysm40u3vf5QLI396Yns5r/DEJ5jUPPS/5LdHXIXlberbsCnLcm3ayPmolpUdYZ6oiX7bI0M6CNtEGghCVdRK4VmSFKY5tfZSrkM69QKxyi/jpP0S2Qc7hRsK5hpZfQw47NF8/v09MJ787gyKi7fYjlo4XVWBtwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qSBBP8CdkRt26BWQnEn0Twnx5706NESRFGp7EcaHaUU=; b=itQGpBekUwekhdfcuY/Z0S8L79SKdJQx5cbDM+L+Gychuk64U929lg5fpvL+HmejXLj2EQMWhetx0btSZZRnpAL+RANEiICepk0QaXG5IEVVBTMU4+YKOWhVsOelV4h2Tj9Hnnl3kgMSo1SIyQ32BWad67I+b44iKheTgZH2Q4RwxyPfuQQOXc7H1mWAq+nmlMFpFfBnBPo3U2qBZcw03tbPpR4UHIynDJAha2+Z6Xc1rFbxRfOGs82jKhoS5UZ8b8v9duDS+cYgjyqnElVKG/YPQPGLxGGyMFb6VUsiEy0JwQlpQ5jPtJgfSwI5LZfE8s9dvA/S8yRhpgEG9/bXog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector2-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qSBBP8CdkRt26BWQnEn0Twnx5706NESRFGp7EcaHaUU=; b=VdyJAsNvtnZipZdR/9LnxouoYjStzer8lMGCaGkXnUvEnCc+aTuGLwK72z3ltJ1nEcuFKYU4udwhpKUNbgGx3rtwEy4WIrOIIHuRBkQl6jfvW30yLnvuI2hBGD+tiftl9vpXvZuPzk7WONOXqZv5RndhikbDyyJItgSu6kH7InU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11) by GVXPR04MB9901.eurprd04.prod.outlook.com (2603:10a6:150:113::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7409.30; Sun, 24 Mar 2024 12:20:57 +0000 Received: from DU0PR04MB9417.eurprd04.prod.outlook.com ([fe80::d30b:44e7:e78e:662d]) by DU0PR04MB9417.eurprd04.prod.outlook.com ([fe80::d30b:44e7:e78e:662d%4]) with mapi id 15.20.7386.037; Sun, 24 Mar 2024 12:20:57 +0000 From: "Peng Fan (OSS)" Date: Sun, 24 Mar 2024 20:29:00 +0800 Subject: [PATCH 3/3] iommu/arm-smmu-v3: support suspend/resume Message-Id: <20240324-smmu-v3-v1-3-11bc96e156a5@nxp.com> References: <20240324-smmu-v3-v1-0-11bc96e156a5@nxp.com> In-Reply-To: <20240324-smmu-v3-v1-0-11bc96e156a5@nxp.com> To: Will Deacon , Robin Murphy , Joerg Roedel , Thomas Gleixner , Marc Zyngier Cc: Bixuan Cui , linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Peng Fan X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1711283345; l=4688; i=peng.fan@nxp.com; s=20230812; h=from:subject:message-id; bh=GiSd4PlTexX48iZrMpUZF0JbBWkoDDMT3nVKob6ALg0=; b=Qw1qNbRzkiQXnGwnSyX4pIUbPqNQ+Aqr0AQUC1bi66PWMtK0gor0N4PGwcC4SOnugOY6F4ikt w95Ia4FqP/ZAr5L4tw8jvUhfY9eLH6pR31cONXOtIWJhWquCksA4kjg X-Developer-Key: i=peng.fan@nxp.com; a=ed25519; pk=I4sJg7atIT1g63H7bb5lDRGR2gJW14RKDD0wFL8TT1g= X-ClientProxiedBy: SI2PR01CA0017.apcprd01.prod.exchangelabs.com (2603:1096:4:191::19) To DU0PR04MB9417.eurprd04.prod.outlook.com (2603:10a6:10:358::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU0PR04MB9417:EE_|GVXPR04MB9901:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c4dc8a9-8c4c-418d-92d6-08dc4bfcdbee X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qbSN8UJVE/j0W3BcbLbZzXAMmXmn0i+6BuhfLJxNNiB0tFftNS3BLRVChqFTQOq8LFO3Fo71ddekqFRXhkTK8jn2qdCZhpzGktGhQHqkmt3m+DzWodh0gO3VsBHhdWqjks5+J0ZWYnzghW6ZuVvkojVEoIxRGCKUEFxXt7toFf0VxAMGRx8Ekx3sDB8rnTFuAIYTvbSDh5E0OHMTl5LpmU4aK0O8WDQqI7hiEVwxEWVvpzg0uY/65b46JQi0nA2ibUtnAVPYNrdCcnOOYwI1ApwbgeFQjRyzA9r/EIjLosDTD6pTTdHhasO3ryJ2Jv55sLy7g3BHd3+CWWgWkCZFxeI2WusaA8Sja7RUwT99e5r8LaAp1u8SW9JeBBfxkw+TAN+3Wb+46cbqjWehc7owVMhFT5qeo5Ino3xECsCUUKBlcAXXvNhG+PMeHgrCGu7uHNyzAAWG8zk74k7JBW5c+xjIc1QlObw50pxnKK15NkGPEPfgDxevywbSM/qSRetrDeONe+9s5ReqfWUkDThGqiO3ej0kbWwuXD0QAUyh149Wnr5N6PgspgbedS0noNbOIame82E3HlzPUv1PstFQ2HIk/HtQoy74fQEiDiIiI1jiw9DizY+6kYW1taKEdGN9AHZchbkRo0mRfcKTGqX3omlZdqZJzlrk5anwrYKG2VcD32aFcftePWOH4iiKGE11ScEX7Gi8rhAwWZduRBkR9g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU0PR04MB9417.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(52116005)(1800799015)(376005)(366007)(38350700005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?/xs2dMA3aRDe1AaKR2jSqVsv9XP0?= =?utf-8?q?je32Tc3wE+VuvphFKa1t3d6lqCC7CHZdqFUeSNPT5Rf92nY/N5zgvXp8cl5fB8om8?= =?utf-8?q?biqzUm1QClKoN0J/okxT5ermC0T2FXi80xQ1vt18luo0wzVKttlFCznzyhd3SLjMH?= =?utf-8?q?KGnUd0PHfw6nYbzya57k8rcKv5aweRfSvgp9zb58D83YoYrJyyXtosyam3ScTM/Yf?= =?utf-8?q?eeVcJv5XAQ5UsTTL2lQ/9ZCKzAO8gKIdVFIRNgn1kDRAdUx1IORF08gqcfths0Mad?= =?utf-8?q?By2NdviDze/JCoAIQc/0ddql5+JPped4Sp4IrQ9+Z0Dk5S/KwLY+ciseyA2zqPxrf?= =?utf-8?q?1c87F3cLy5Ei0c9E5HBwHhfo0DSwV9lf36bwEu4LoeVe0vuMuExp8lP3NVspsC8sL?= =?utf-8?q?GaPS2jt9uCOvmZUarj/GELW2oPcWR5QCZDAzWeYjTWQHg203dozyhiSINoSbLAoV+?= =?utf-8?q?fuAzydRXi7uFepz5jrGZUmDsp83vjq7hPNactKducBbTU/rtYUn4dgU7IV4184pEz?= =?utf-8?q?PNjmFUyuV/bubQusROUifc2pQ1Lx+byqE15hHeaE3d87eq2ReYDR/tuHrU4SMtAES?= =?utf-8?q?6eYRe/aNbes07/XJq1llZqtYDYnS1cHuyaBeCVLx7VxTfzbGa/nre5skG+RAqa/Pu?= =?utf-8?q?Kf1QA8S/3yQEPK3krmGaFBcgFewRDkloU+NKe+h4TBXmuyViVSlP9DhfNazl3rMWv?= =?utf-8?q?0HQ5LJdKXw29siDoOPzYhoyHexy2sPQ4nMw9QwNmzVo6OlZXW7Z2vw3xT5Nv5xyGS?= =?utf-8?q?E7kHJWYgvk0rLd5HfIMG2DDt33z0+JB+BV9Jf8uNWkLxQbEwZDz2x8g2OHhVVMHuo?= =?utf-8?q?a0M4P8tsGwsvMlx92YhKPas52K/AEuWO7yH7xQEf0FvXHf7wo/ik7n7zS7WMlZwJP?= =?utf-8?q?EqTtamrMq5h3kxHo7olA37CYVTwPewINQpk27Ma1rDYS/0qU1zRJ+KOkKJwOQMJBA?= =?utf-8?q?kQxDH9fHPCtgys8vSYQ0B70vBRUvZopNhptE6YDFlbtMQhH6lZffuU1GfIuIXeLLp?= =?utf-8?q?0sc31WLMa6XwMPJSP6auX/AWCYGQFBCNE7MBqCyeCw2In9cbPgX1TNiCEANRmJERA?= =?utf-8?q?vMdqVJdbwJUEI3xGgeINeSDwB8X+wDJXX1Xe2/A553pfn4QNRwTcVVfnBXlox62Rg?= =?utf-8?q?JPl5XZJRdkkLDtsLQVA7XR5KUtH3WBFPksJlKf/o5XWqOayRdELW8J0fGeFanWWji?= =?utf-8?q?8XfK532uveLlWOUlE7QIjF6/TrIOG7Btd5envGR39KLHBIZZw/9wLB5l6KfB+lJRv?= =?utf-8?q?s+06ybNZJzOkGBRfl8bR61jZCvTaSz7LRgynJwjRwzS5tsZng027yuhF/bf+tGhGo?= =?utf-8?q?TGvXTlbdl3PVi2FbmoY/1WSbktPZ73a11Io671gXodFwqV9dRpotST+0ed/D/7c7h?= =?utf-8?q?LHVs1y7uJCM5TZvX2lZEFFzALbbnXTbJ8+J/CjrA+0Dx5UViGmJ0zKkWTNebgUXJu?= =?utf-8?q?iyaXNxOKWmXBBPcrYqrVcRyJ1xt1D18WTTmSD4H555ImGzzxofX/PMzagKaXvD3NZ?= =?utf-8?q?0ujysUSuOJL9?= X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c4dc8a9-8c4c-418d-92d6-08dc4bfcdbee X-MS-Exchange-CrossTenant-AuthSource: DU0PR04MB9417.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Mar 2024 12:20:57.4250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: s9/e38xJbknM2qqmXUpu0i/uYfs9dzOI0vXhCzH/hX3alae8DbLEMZOX1JWka3ccylgXJzjIIbl3q1BKjEIsGQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR04MB9901 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240324_052110_749391_C337DDF7 X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Peng Fan smmu maybe power-gated, and the registers are cleared. So need to restore its registers with arm_smmu_device_reset and resume the msi interrupt settings in resume callback. Signed-off-by: Peng Fan --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 71 ++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a8a569573c2f..2bfe4b3d0ba1 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3396,6 +3396,36 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) devm_add_action(dev, arm_smmu_free_msis, dev); } +static void arm_smmu_resume_unique_irqs(struct arm_smmu_device *smmu) +{ + struct device *dev = smmu->dev; + struct msi_desc *desc; + struct msi_msg msg; + + if (!dev->msi.domain) + return; + + desc = irq_get_msi_desc(smmu->evtq.q.irq); + if (desc) { + get_cached_msi_msg(smmu->evtq.q.irq, &msg); + arm_smmu_write_msi_msg(desc, &msg); + } + + desc = irq_get_msi_desc(smmu->gerr_irq); + if (desc) { + get_cached_msi_msg(smmu->gerr_irq, &msg); + arm_smmu_write_msi_msg(desc, &msg); + } + + if (smmu->features & ARM_SMMU_FEAT_PRI) { + desc = irq_get_msi_desc(smmu->priq.q.irq); + if (desc) { + get_cached_msi_msg(smmu->priq.q.irq, &msg); + arm_smmu_write_msi_msg(desc, &msg); + } + } +} + static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu) { int irq, ret; @@ -3442,7 +3472,7 @@ static void arm_smmu_setup_unique_irqs(struct arm_smmu_device *smmu) } } -static int arm_smmu_setup_irqs(struct arm_smmu_device *smmu) +static int arm_smmu_setup_irqs(struct arm_smmu_device *smmu, bool resume) { int ret, irq; u32 irqen_flags = IRQ_CTRL_EVTQ_IRQEN | IRQ_CTRL_GERROR_IRQEN; @@ -3456,7 +3486,7 @@ static int arm_smmu_setup_irqs(struct arm_smmu_device *smmu) } irq = smmu->combined_irq; - if (irq) { + if (irq && !resume) { /* * Cavium ThunderX2 implementation doesn't support unique irq * lines. Use a single irq line for all the SMMUv3 interrupts. @@ -3468,8 +3498,12 @@ static int arm_smmu_setup_irqs(struct arm_smmu_device *smmu) "arm-smmu-v3-combined-irq", smmu); if (ret < 0) dev_warn(smmu->dev, "failed to enable combined irq\n"); - } else - arm_smmu_setup_unique_irqs(smmu); + } else { + if (resume) + arm_smmu_resume_unique_irqs(smmu); + else + arm_smmu_setup_unique_irqs(smmu); + } if (smmu->features & ARM_SMMU_FEAT_PRI) irqen_flags |= IRQ_CTRL_PRIQ_IRQEN; @@ -3494,7 +3528,7 @@ static int arm_smmu_device_disable(struct arm_smmu_device *smmu) return ret; } -static int arm_smmu_device_reset(struct arm_smmu_device *smmu) +static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool resume) { int ret; u32 reg, enables; @@ -3602,7 +3636,7 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu) } } - ret = arm_smmu_setup_irqs(smmu); + ret = arm_smmu_setup_irqs(smmu, resume); if (ret) { dev_err(smmu->dev, "failed to setup irqs\n"); return ret; @@ -4086,7 +4120,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev) arm_smmu_rmr_install_bypass_ste(smmu); /* Reset the device */ - ret = arm_smmu_device_reset(smmu); + ret = arm_smmu_device_reset(smmu, false); if (ret) return ret; @@ -4124,12 +4158,34 @@ static void arm_smmu_device_shutdown(struct platform_device *pdev) arm_smmu_device_disable(smmu); } +static int __maybe_unused arm_smmu_suspend(struct device *dev) +{ + struct arm_smmu_device *smmu = dev_get_drvdata(dev); + + arm_smmu_device_disable(smmu); + + return 0; +} + +static int __maybe_unused arm_smmu_resume(struct device *dev) +{ + struct arm_smmu_device *smmu = dev_get_drvdata(dev); + + arm_smmu_device_reset(smmu, true); + + return 0; +} + static const struct of_device_id arm_smmu_of_match[] = { { .compatible = "arm,smmu-v3", }, { }, }; MODULE_DEVICE_TABLE(of, arm_smmu_of_match); +static const struct dev_pm_ops arm_smmu_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(arm_smmu_suspend, arm_smmu_resume) +}; + static void arm_smmu_driver_unregister(struct platform_driver *drv) { arm_smmu_sva_notifier_synchronize(); @@ -4141,6 +4197,7 @@ static struct platform_driver arm_smmu_driver = { .name = "arm-smmu-v3", .of_match_table = arm_smmu_of_match, .suppress_bind_attrs = true, + .pm = &arm_smmu_pm_ops, }, .probe = arm_smmu_device_probe, .remove_new = arm_smmu_device_remove,