From patchwork Thu Dec 5 15:40:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13895623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F7DFE7716D for ; Thu, 5 Dec 2024 15:41:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-ID:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=2TPvgyTw3LZW4e0wTKdQTd3s8HG+STLY+zBXPkj4Tik=; b=2UpzapoXNJoRRVF1mZ9+sBwRZA fLf3mMFqDgi+7S2ItN/bBu4OjWYQ29DN0XN1gOq1q5cQOnhWmRRcws0RK2r/C/s3e+dyfFdg2WjWz QiDUsRTJTZdCV5nvn1Vil7deIzGA3wq7Wn0G5dzLgKEAoEZInYEWDSodsl6pqTmwPh+bOfWjLhccd to/TVTha308eT3mUgqFL7nW6ycMhDr/jflprOiSSqJM/AAEo7w9n5afSXhttM+6a5KzxDNNAQpz+Q oWzFsoAlbkkQLLRV0jE+lZBL5jqibqRVsSusqfAxBYziPjI7Lm8zc48pbfSXT+5nDSQnvof/L0u8q 5D9X5rGg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tJDz8-0000000Gb6w-2hD1; Thu, 05 Dec 2024 15:41:46 +0000 Received: from mail-bn7nam10on20619.outbound.protection.outlook.com ([2a01:111:f403:2009::619] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tJDxo-0000000Garr-3lRO for linux-arm-kernel@lists.infradead.org; Thu, 05 Dec 2024 15:40:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vVtSTJh0Z3mLLzEMlZs2t/2vhvFDTD57bKuGzWq7/ja8E79G7cAeksuzgqZvr5CuqeZe69C8I89bK8Fg7z0Uv2AgC4ROLvGdbzDpe2vAvsYq7fvTZ2zhSJ6iQsMhG5PsnF5q5XoJoBuLT34i5MejUHf8OrlgXwYQV9fc5Sm4xdcwlmY9p2NsZ30h5iklWxTKQIpAFLmtMOTux65CJzgvkI+S8eYo1l7XtJRLy4HIA4PMoYqrf075vY3WS8Boo92MICYrwsYsuamSjNkMzKyGpmvvZdUUc03+6Pzq+vSLuTWslbyOG//pJcbhDDNmKYtrrMPpHHDmvxEFZvwyw/V+bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2TPvgyTw3LZW4e0wTKdQTd3s8HG+STLY+zBXPkj4Tik=; b=MOEJQbGK+qu/H6to5c425425bPDB7+NC+cStY/+Z0QcdxD04+9T/unadIYrE3jF3i4VQs/eOx7nAZ4tcKSAldhBNTAY4WffJxIAnq+KickpxJJz3ew+F1uHJVqqGCzEi5zfgjB3GhwtdAopgsAKJIJF5Box6xlSK8x8DA9kSJbI8FlVCXb3A24PaftzuPozOXhMJa+cKg27GStMjwrKoL1LZl1tTVhCGu55UDxXfMyx9io4O8Ec0kJ8ulKAK3J8NjWmy1MwUA+/xvUFrRo1UljCiXdMcGQg7xlfUp1lP7Mx1WZy31t7CusoOHw2VbiUbOivS3Y8HRjeeUH0yu8MWbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2TPvgyTw3LZW4e0wTKdQTd3s8HG+STLY+zBXPkj4Tik=; b=hMI+YVKej9A9WRCW3r9VZVM9iS3HX3WIHTGZlZ6BBS/sCIt+4Wesznts+nuxLO1mYtMk6vplE5djXF3ndpk1B9Xy07pG7SZ2b9lPBGCiqt+0gjsj4IOxfO2fV1lwUyFhsHaZzh6ifWphp4zhQNBs04mAObzagViR3zcHDHrCqZOu1Xs1GXikNgjq6urgLRlmzugpiFvX2FLqGChW2bBT1mVLpCRZjdqjlqtRr1HOOi3bPdYHB+p6ziy9up2Kj/Nbb7kp/dBcW5a+EASCRR7/kYUlvi7AnPxtyj5mVeXxsisFpxtEF/FYPmFyO29loSQfwMAOb0PDCTOXTSY6sCg54A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by CH2PR12MB4248.namprd12.prod.outlook.com (2603:10b6:610:7a::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8230.11; Thu, 5 Dec 2024 15:40:17 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%5]) with mapi id 15.20.8230.010; Thu, 5 Dec 2024 15:40:17 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy Cc: Jerry Snitselaar , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameer Kolothum , Will Deacon Subject: [PATCH] iommu/arm-smmuv3: Update comments about ATS and bypass Date: Thu, 5 Dec 2024 11:40:15 -0400 Message-ID: <0-v1-f27174f44f39+27a33-smmuv3_ats_note_jgg@nvidia.com> X-ClientProxiedBy: BN9PR03CA0797.namprd03.prod.outlook.com (2603:10b6:408:13f::22) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|CH2PR12MB4248:EE_ X-MS-Office365-Filtering-Correlation-Id: 397ee47f-5eae-4683-60f7-08dd15431e21 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: jOj1yS7QyVzrM7jThCJZ6hV9vV68/zHXmWncW+gEQLDcDXX54l+CERKL9IhLkHdLPrdIfbjaUS7fxHUbfG6ECEJlDcEUzjSkYrBfEN98Q6Se8eKGVCwawn3D5NV+bfdP9KwETI2L8yIQOeY7avqe2kIyF+j9FbdFfvkfxMibbgWAUKB4meT7EjK8GMYGgUUvISWodFtYMBk+AtECZyll//p1wANdbJQ7UTEtXb301A3C7uR2EJeZZZI4sywYhs4AUwrxv+tJa7H68Y+uYJYxdQb3n0rLg/zUIqeG0A+cty1mLkTEWH1In1+P3GjCydhHZRCFaUaoA3NhGk+nNMG/0fambESr0T9+2kmV2yn2qEVZk64OGNCH2TFj8NJJ3Zhn3KklHCc8H1rVfNb80+s0i8N6tT/vbPmrQODThrLGwfcy4Ayr7WqT2cRXgyBO0F24AxO+TrAnSpxa/gdRCSXmrqH5o2zizIibYM7dGjcXizlUq0UugVzJy4ZoM94AnAcGu766YByrSqbVlENM4HlE9hH9q3NHGRKaFVJcdb5x9OmjaitdxWmwg0jbABQQI+0nZ4S/aWm0PbDpWsMTiYeIT4JkqXaqrgiBzX4iDqroHsywJWGSzipibxJYuIgWvdjj3UskQQpMuIpQcoB2hCSYgpuGaFrl+EudWUFBgwLJRvpdYB3VplQ9yGYLQF1+TN1SNH5YypcEI3b2kutmOaJ8ohMFLePzKQWWwrlcLgavnxrLqUq4nHk5mUoycnARk3YLwVaoGYSAO3Iib7mY4GrZSyG04o/aKhFL0B7+1pepLFcCXZY+Jn/c9B+kmjF7E29yHgGkgQT8fSLPDQmok3pGDEQJ9rplf28CvKx2luQ2+MxTkk6cWSMJavV52z/pNM++HpIQRQ8C/rE/RUi72vq7sf79uYR0vgGZgAhAKAb38Vndo0wWwq+b6QMHulKxdznMBNpF2ocDPCVL6YUdSouLLj8SKScu0sKbOk1YF+h+AVBXDLaC2LaEQjGqQyxrCbyFh9tRfgxhrbTFEsXI5VB/EpdYEUYBp5lDEgIja2OFrtKuE6zRAcqIqhIz5i4qpCV/rBONWL9rl9pqht6ahD8CJdLEGxd5SYFV2vB5GYlCWT3ac/kQNs3yNdw0weCrPNhzpPA/jy9c1EjxO3wzO7aTf9pltBGMLkWnbrpgQ9zXMQhVFAsxF5QqYrH25A/spYdqOvLD4XcgD0atXX/9mBZoIn4VJ9bEB1tSGJQ+elO9QwU6zDMLC5miOa2Rx1MUtK/W1HwPxtLVInvSOTFupQ7MupgmNS85Z1QmRRrn0npTc/SbuzyAFl5EWtJm2ebTK5b+sRFyGFIm2yMUxWd+Mo4gOA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KCqJjRyW+TME/iDW13+tKzoYca5Ve8BCDz3NCZJAdiXkDvWK1pM1fdprviA5J13WiLX1Zxxa5bbKskBG7hKIRC6ZbwxaXvYtHafF5DmKFCj6PfZsOk3K5cdFeL08QiWABGD0sa5+kyZCmFlAAACRrL4HRqFpYlkO4X86zMwsFjKY7qbn3juXkjfeTwT51AWfQN3pGVMA3aJ5k09VuN+l6mk9W8r5owg3JVgB1+124dmZrTD6+id1EXP3PhAmuLh622vPAwhsrNYD4l8nulYnw8DsxBkulbSbSh9y7mub59anLs5AlMTMRqfwpxlM7XWWH3FX0xFfbUIQekLYmETXUBdoq/dIhSDq5xWMKJAwNtr63ca3IpFcvcKW2ofPzJiucYKuu1XrOBrYbO3Rcko+BM4ytLayvSJIuZLig/kQvA/DGpngp537QEwI6JqW0+bUZdDewOTEgtDtm8ABMrfLXA1fmAE95QN/570XFZ1l+1eJOLUFgeFaR0ISkCG3uqqhNyTF2dY1+q+/5p1Mk40ufZGV2uh63B72KZ6Rqyh758CnEm/B9SYDKFQai8eo3iq2HyUpLZ8uQ0xZGIN+W4X9U8MobyjQnNYP6N5eialXIbHsb+w9wj5uEfs/N46baLC8HaJFZXW3skJLS3JrZglbwgCb+FgKh36lOSZ5hufWyal2FZ8GqVX8VSWOyFUWqIbX3guVG1jACDJtv4wODR1wVeJEOg2HZGZp1hPzal9HOxmaQiAMkWq1kaN99ihRmev5hr+NPf00TYOOGTUhrI6s+qUYq2oQIOcf64SCQFqLEqA/2C8IJSgJPnr6KvdPq8ITti9+/bMNz8VqY/6t0nKZ7v4tn1198Wbm1PLX2Qb0z5HvnhY8swIDIZhKE0snQz35jeJPmA5hYa3ru/BU7i2PImMMP5gV/RFWR5HEuv9wz3U9X1SCvVudO1oP/0plU2V9I9q1LeL7Pe4kY4CHWXTBY7tjh7+EsaDMeLYiLbxZW9aJvq2/1SAhGJ60J/DcXKj0BD0UgZ19OAFgQmJkiTqCeqdWPtbii/oU3nkaagHPvmPDHJ2mFWZdrctyz28y6zGuOZxezrFqfBtEEU8WB6Dd2EcmxKxNpTa13+nu0n2KZOQ6Jckk/eyi0WG9x32K5lcZjPa1NlDB25bu3nKXHHgHYRM8uGiyqJnYwqG4MdcTrF7c2+wZ9HiO8MKRdDere+eW5+xgmfOKWb/2SmqF+599f/D0zca3cFlhbY14JrGCCZ+xmqaXhbN/iEpMMeR92XjWXbPmxgrRcBsIjT13pastpg2VW7/cp63o4W4SZZkBk2kFIEFqw5NtBDTlJow+E+UnNWPfcf0tMJB98LiCPvfWRUu78TZ4U2nttY99OnuCVK2pKhp659M5KW1DaqF//hVD/k2JCoEuDPYPVjq1LsOug3Vhm+KXnk5GXtZzy+vyFu0qxMtvpHgZT9TcQ+NMJpvN4/HLGJFvfzn0Tv7+kcRFpqllQOXci3shGvY9HaTQ7qEN2FZblHdXGQfgmcdmQHRxpyDZ8h7U7WNZmiw9e+wn/nwZ0ZEfvPOiglJWRK9tI564cypf+F/C6RwfkwJkyhhZ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 397ee47f-5eae-4683-60f7-08dd15431e21 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2024 15:40:17.0609 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /IMGpQAIKygXhT2XiNK+I6u73owcvaDe1cUHHibJossTUNbU3eVtd346uMAHSkFz X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4248 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241205_074024_937852_09BF493A X-CRM114-Status: GOOD ( 15.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The SMMUv3 spec has a note that BYPASS and ATS don't work together under the STE EATS field definition. However there is another section "13.6.4 Full ATS skipping stage 1" that explains under certain conditions BYPASS and ATS do work together if the STE is using S1DSS to select BYPASS and the CD table has the possibility for a substream. When these comments were written the understanding was that all forms of BYPASS just didn't work and this was to be a future problem to solve. It turns out that ATS and IDENTITY will always work just fine: - If STE.Config = BYPASS then the PCI ATS is disabled - If a PASID domain is attached then S1DSS = BYPASS and ATS will be enabled. This meets the requirements of 13.6.4 to automatically generate 1:1 ATS replies on the RID. Update the comments to reflect this. Fixes: 7497f4211f4f ("iommu/arm-smmu-v3: Make changing domains be hitless for ATS") Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) base-commit: 40384c840ea1944d7c5a392e8975ed088ecf0b37 diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e4ebd9e12ad468..7605b264fbf983 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2745,9 +2745,14 @@ int arm_smmu_attach_prepare(struct arm_smmu_attach_state *state, * Translation Requests and Translated transactions are denied * as though ATS is disabled for the stream (STE.EATS == 0b00), * causing F_BAD_ATS_TREQ and F_TRANSL_FORBIDDEN events - * (IHI0070Ea 5.2 Stream Table Entry). Thus ATS can only be - * enabled if we have arm_smmu_domain, those always have page - * tables. + * (IHI0070Ea 5.2 Stream Table Entry). + * + * However, if we have installed a CD table and are using S1DSS + * then ATS will work in S1DSS bypass. See "13.6.4 Full ATS + * skipping stage 1". + * + * Disable ATS if we are going to create a normal 0b100 bypass + * STE. */ state->ats_enabled = !state->disable_ats && arm_smmu_ats_supported(master); @@ -3070,8 +3075,10 @@ static void arm_smmu_attach_dev_ste(struct iommu_domain *domain, if (arm_smmu_ssids_in_use(&master->cd_table)) { /* * If a CD table has to be present then we need to run with ATS - * on even though the RID will fail ATS queries with UR. This is - * because we have no idea what the PASID's need. + * on because we have to assume a PASID is using ATS. For + * IDENTITY this will setup things so that S1DSS=bypass which + * follows the explanation in "13.6.4 Full ATS skipping stage 1" + * and allows for ATS on the RID to work. */ state.cd_needs_ats = true; arm_smmu_attach_prepare(&state, domain);