From patchwork Tue Feb 28 04:21:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ganapatrao Kulkarni X-Patchwork-Id: 13154403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7495C64ED6 for ; Tue, 28 Feb 2023 04:23:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j1UpZd56v//qMMbeLFGMojXiPKNTGrwqRF+EANjySvg=; b=AA4NfPZpTT94ft ByafaZu5+e84ua6+P7eCpOf6eoD1zFoXd8ukW9XBd6F4i2LRkiUegNS17Ji69KZM24gFhZ8VhRlCE Dnl5G829JC58Wc8w23StUbe1Q3I/5KDs+JSJ1svHIVgogoaDotNdJ1s5q2Cwds6CE62+Sr+enfXb5 RJG16M1uCzpWya2toGoABCdyAlCg7ygZHgr80s4/d3JSB3IaU8wza3/0iM51qTkv7yyNQAfaFibZl w6MaaEPYatPLC76geg7XfLZ6LB4zByJfbfCmdG+DR2rMetNqJxWGtGWauKfzZofMpUSX3S47IbXWa 2VtInCDYMSdeU+T3z+zw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pWrVE-00BzSY-MA; Tue, 28 Feb 2023 04:22:12 +0000 Received: from mail-bn8nam12on2070e.outbound.protection.outlook.com ([2a01:111:f400:fe5b::70e] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pWrV4-00BzPC-DS for linux-arm-kernel@lists.infradead.org; Tue, 28 Feb 2023 04:22:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K25u0kyDiART4BZYuzP8dB5e6UmBaa9gqVZdlg+QG3hWfV6gvKC0DhTUdNWdaLTN0Wae2KSyJzIRDt2qw2rR6GNWvRkYhJVMDUQi6U6g+QjdjdVGaiA7I7p1KsuOf2gEvSWFgwPWcP2xKg/FiefEyqQ0rsMA883vsXGkeLj8ab2BEWri2cdwHOTLfQQWalxrqOQOahexi6+mAqgh+3fQsCjICRmYeGRw4892EK6OaE4hZUrH8VNM4HMC6pHdAVLN3NCzkgkcJafSiMH/hTYiOACNH7Fu0lIWtYZo6XM5tdWplsaqOfYmzsQvGgXe6RQbOH9WArkRLzvrn+YVfZR4Mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zAt2Jdw+BFCu/M780DOdKTlmXUiMtgpBo93yRj4aDVQ=; b=F6Z40zQkhBYvvkGLM6/54Da7ZdbwkHbO5KjEziYth++UJyO6HFqVY/rhRRhsePPjJyjAdYL1ClpsMa+Q732g6OA5R+esZ+SIFs+FO2fAYCA9RG7pm70QgPTb8hljSzvW8D6vVUHMZdA4VWrwMfqAUuyLTRkP7D6Sniw+hYB+edKFy9MlZljESJrnQ4yvF79bL/urTEqVwGfs58lBgMJ9qoqYfIGFi7aqIiEImHeHch+aK9+7imeItIsUDX+lhtWWUBOEHSQW4OZBCFDZ0SeYd6NKW2UlfKrtg2CLGLJ3Crg3aP5pyEW4EmqM1+hrFAIv8/U7jsF3w+KAHEaNOmZVUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zAt2Jdw+BFCu/M780DOdKTlmXUiMtgpBo93yRj4aDVQ=; b=bIJ4aXpa1CYEUHux2GLyheShEA0viOWStNR66RBitab3R4zy7yocq2CjMFG6GQPIpB4iqHPyr9WRUGjbSAp4reAgXWC0sEksfjwVH9O+F/eYM2fitLwgsQbZ+85kpw1JznWjDGAZ1jptxhn10ruz1Gf/m5bALYWoKzYGVQ0OqOk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from DM8PR01MB6824.prod.exchangelabs.com (2603:10b6:8:23::24) by MN2PR01MB5422.prod.exchangelabs.com (2603:10b6:208:112::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.19; Tue, 28 Feb 2023 04:21:55 +0000 Received: from DM8PR01MB6824.prod.exchangelabs.com ([fe80::fc8b:5e5e:a850:7f0a]) by DM8PR01MB6824.prod.exchangelabs.com ([fe80::fc8b:5e5e:a850:7f0a%6]) with mapi id 15.20.6134.029; Tue, 28 Feb 2023 04:21:54 +0000 From: Ganapatrao Kulkarni To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, joro@8bytes.org, bhelgaas@google.com, robin.murphy@arm.com, will@kernel.org Cc: jean-philippe@linaro.org, sathyanarayanan.kuppuswamy@linux.intel.com, darren@os.amperecomputing.com, scott@os.amperecomputing.com, gankulkarni@os.amperecomputing.com Subject: [PATCH v2 2/2] iommu/arm-smmu-v3: Configure STU of a PF if ATS is not enabled. Date: Mon, 27 Feb 2023 20:21:37 -0800 Message-Id: <20230228042137.1941024-3-gankulkarni@os.amperecomputing.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230228042137.1941024-1-gankulkarni@os.amperecomputing.com> References: <20230228042137.1941024-1-gankulkarni@os.amperecomputing.com> X-ClientProxiedBy: CH0PR04CA0109.namprd04.prod.outlook.com (2603:10b6:610:75::24) To DM8PR01MB6824.prod.exchangelabs.com (2603:10b6:8:23::24) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM8PR01MB6824:EE_|MN2PR01MB5422:EE_ X-MS-Office365-Filtering-Correlation-Id: 44c0fe17-16a2-4c15-5fc2-08db194352ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PBLiy04fbeHDYfkru8NSzeZf8QrFMJHsfLWgyXlh9h5BahmSfdy+AqZjvifzkYd6vT0IucQ4ZHuB470el2oQ+WlZIFVm3jg8wubtLD6Q9uKPsSrL28IzISexj3RfiPRZBc6KLJjOeU4X2pRI4rcgI7H009R0rtjNeXbMvltoKB4p4R6VJAIP905Z3T38lJw+n+Aco6t/AFnVgQ8leniAQuJXNowLbT7T1GB1887Se+fPA1T9gNhGRhJt7n1ANp2qjca8vvXm2Cm6KWKieEk40KFkMTu3YweVLnLfBvjeyOqSJF/j4Acy2JiUqa3QhqO/5QHZnFxFXnAr5JA3vn4IOxutat7dc1A0hiSxMfm7wY6kuB7c3QF5w8QzV6XGjuOrIxYkJoDgnNMGIyx9oTV06l7SXGVk2KRQm1P+89B0z34zvHELSHjhjWG7+i7rvs4FkNu87PifbjnBxxhBHYnW3AXiSXiMQH+8vAW6QosViuI/LbzrlTq9DKnFJ8TMDCMN9JR4jKn6hou6OICrErI9AyzKdqFNAFFSwf4AuPzvuzh6a7SMnfslog8JCb9IjAiBpR0VUBBoP+IiWKZ00EEizP13k7Pwzq+ixT2lVlXxSmCDCcAQ6nkPclHiAe/Nu1JJ0IJvih6bOK9ZuDRpFSBiNUUiAV/Ju8tdb6x2nO6lJGcHp2QGn3uRh8bhBWLvECOSgBscdV2wP5Tp2/Z1n1MDUw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM8PR01MB6824.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(39850400004)(396003)(136003)(346002)(376002)(451199018)(6486002)(107886003)(83380400001)(7416002)(2906002)(5660300002)(1076003)(41300700001)(6506007)(478600001)(86362001)(52116002)(6666004)(4326008)(8936002)(38350700002)(38100700002)(66476007)(6512007)(66556008)(316002)(26005)(2616005)(186003)(8676002)(66946007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NHYHiG2t5zoe8fCtwZF/4PY3Rrng3s0OrEd3xiW1vT7Vv35wLR+wDlwDfG4I5LlENUDjxSDYq/m1MXmkYSy2xr9uq5Ie5hMD/3VGoCMe98XbxS1SH+qao1WrbB5j8V7emUdenCE8HeSVk/KjyCE0o0Ojb+1I57w8gKkeOkNSft5lf1IAO4KY8ifK+M65M1hmB8DH8NC8l0qPzvACWQaG2N0V/+BUNkke3BLWL5C5OVRySAVYAHYdy087y77ADoLaoC8fnv7VH3KX+r0da8UF/8bhldl6m/2X4Cjvp1L6Z9P1z2jZunDxjdZQ+ibbq3cH94i5BoQH5SNYR/7t1XOfkINbvnu2ijVEGcR1ppe97xjcUR43zOFwuqKMuloht9TZZmxu2so+Zp7Vv4RBzL2g9D/o6JA3aw5lBPDidZ3anEBMUto5kCVkD1/PGzSyaQO9YgEaZYx9necFuoxicoFPt264ZhuOV0rkUbufR5CBCrbNE1YEDmxrw1vz57JkL8bGCUdKy+7GHBQz13om7Mc5tKSo43PzCsnLS2L7EeLUUBVUEulBubnQo6QSMP253V9t36BIbkx3TyUYTGlvQyPCrOziCvK+ODd1QHtNHcO455NK82RaW33gl4HWG+maGfBzm2c9j0YGwHWqPUKAXsuQ3MJNhYGGdYV7iHR1mvszYX0PBkd/srb3kab/JN7d1CNQS2a5jYerNX4rXELSbtoDC3RiP7G8PQhLMeV1GNCPvyyer8Du63hlrx0FJEqtxIz8p3C1tsF2HImvqSsz9K9aft8TwtxoDH3WQpAfrdmcDYnW4o/cxUUgNIxkFaIMtEsHK12gx0zUc9qmQqTaNWdInFhDErV9ExXLLbQoOUvIZxBH3fa7EdUdiejwO/clfiIbl/0RVJ0YaEBuP9KmOFFme39dQN4D5XAiPGU6CaHqgXftvcM1ScurS6RkLScJtDyu+3KGLfB7IKBPZ1Kpv8pbk3vNcnqvTZzjFzfKwKFMGQoNHV7uXzMunyjnLAHe1Iv6zW96fYG4Rhc+JlgG36Z2VJ2uC5K7EFJ2q5i7louOXbXMAX2/6UKYGPizFgGO5ylHsJ8LJFUdoiaB2A7ozGjb+J/G4A+jQPYMwiXtwP2d8VmfenjICG13zsUlFlBpQAtthLxJq9c1Xu33b6bEuwOo5DTpnDl0HoPV1hPKZC2jEMvlsVZPRqqqnyLLDqxUYFHxQypcrtz5fN0S4jvHZroLIFDLJmF8/CCybiFPLaTf0JKOT+deYoXM1Hj2PXc62eobSmxDIx46dHERTFd1RM0zZT8tduMILRiQDf0ITSW5Koc3IGZiw+NzJzLcYuTd3DizNZWa5f1RHi/L9GjGHeJT0Zi5zQBbozqaJNUKKI9rL3k9eSPEl4+KyCVzI2ZWjjcg+AsPr1+oehlPpWWLsunDPKGVrvv/AYE8vTmuz6KELHl4tb0auQjvB8H9hb0Z6Ckuu2FsJKsoAjIubg9jjT/4Vuxz2Sv9pNLmz40fIhHJzVd6/kFfVpJ+hZYAPpu3EnavlJj445fcntSqgK8RbgehOyyjAp1Dej6lwm0LITjALbOabmESHa9ZS/2kgAfN/OGcyDFa5ctviosjmtzPLN7paMm4JHhd+O/DcMQgPbQRALA= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 44c0fe17-16a2-4c15-5fc2-08db194352ec X-MS-Exchange-CrossTenant-AuthSource: DM8PR01MB6824.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2023 04:21:54.8889 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bVkl9GROhYaqbxgKC0MP6/sh8896988AxNGtsZLLTTs7HbitWG9rgDxKa3qk/8AlHpkP/rzNxPVGKq9Z+rqSxSTsc3dU9Li/SIPTd1qRpkKaYg2BdnGmg95XV9kIV5Uo X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR01MB5422 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230227_202202_477559_9A40F6E4 X-CRM114-Status: GOOD ( 14.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the host kernel is booted with iommu passthrough mode, PF and VFs are enumerated with iommu/smmu domain set to bypass mode. In bypass mode, ATS is not enabled on all VFs and associated PF. When VFs are attached to a VM, the corresponding iommu domain is set to SMMU translation mode and smmu-v3 driver try to enable the ATS and fails due to invalid STU of a PF. Adding a fix to configure STU of a PF while enumerating in passthrough mode. Signed-off-by: Ganapatrao Kulkarni --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 ++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index f2425b0f0cd6..b218ef0bf001 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2292,6 +2292,23 @@ static bool arm_smmu_ats_supported(struct arm_smmu_master *master) return dev_is_pci(dev) && pci_ats_supported(to_pci_dev(dev)); } +static void arm_smmu_ats_stu_init(struct arm_smmu_master *master) +{ + size_t stu; + struct pci_dev *pdev; + struct arm_smmu_device *smmu = master->smmu; + + if (master->ats_enabled) + return; + + /* Smallest Translation Unit: log2 of the smallest supported granule */ + stu = __ffs(smmu->pgsize_bitmap); + pdev = to_pci_dev(master->dev); + + if (pci_ats_stu_configure(pdev, stu)) + dev_err(master->dev, "Failed to configure ATS STU (%zu)\n", stu); +} + static void arm_smmu_enable_ats(struct arm_smmu_master *master) { size_t stu; @@ -2404,6 +2421,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master; + bool ats_supported; if (!fwspec) return -ENOENT; @@ -2446,9 +2464,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } master->domain = smmu_domain; + ats_supported = arm_smmu_ats_supported(master); if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) - master->ats_enabled = arm_smmu_ats_supported(master); + master->ats_enabled = ats_supported; arm_smmu_install_ste_for_dev(master); @@ -2458,6 +2477,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_enable_ats(master); + /* Configure ATS STU of a PF in passthrough */ + if (!master->ats_enabled && ats_supported) + arm_smmu_ats_stu_init(master); + out_unlock: mutex_unlock(&smmu_domain->init_mutex); return ret;