From patchwork Tue Feb 6 15:12:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13547571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A284DC4829B for ; Tue, 6 Feb 2024 16:16:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w5x0rw0OmMzSCxq6pStMpKwufRSv8suyAslKWW28FNM=; b=QNtuOglGxfXjW1 dm24t+nH1BMxFhHCczUJDhYsfRdYRqyHXjx/wGhu7/JBO3M4uSUge2gATw1yQdI1S325LSTog4i1+ tXv9nY+ize5oZuSpPi367jFL8ju9hYZESDxcG/0OcfX1JtvgUm6KDALr+NPfUXGZkFzSI7bptrN+i Mo0QLvP5bJR6FDsmVjDlB37qUFodUizqzY4/htws8Wu/xq4zYRKcqe2sPMtiMC5qC0m0k6St1Ags7 lMORjuTifx1HsSMzoLLllZzBldu5iUm4bwfkymBwFVOyCu62/qIxh9xF0oMpcDwBgSFj8MjFdbf/N 1RxXjvpbumzFFl/sXnhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rXO70-00000008E22-2MoJ; Tue, 06 Feb 2024 16:15:54 +0000 Received: from mail-bn8nam12on20601.outbound.protection.outlook.com ([2a01:111:f403:2418::601] helo=NAM12-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rXN8a-000000081SH-2eQS for linux-arm-kernel@lists.infradead.org; Tue, 06 Feb 2024 15:13:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c4m60eudJXHaLWmRfCDReouaZUqPXgppU5mhzrmpiv3nLiwy97quMIo/BZ1p9zvT8j7cyvmbc/1fSocyXIy+QjTovraeArWxjBhvp4zpTQQtLy57BMMF4f3x/dPJwayGGk7NooH1M3I6krw7T4493XEUGlgJQn8f3SWhMqKhZNFgcfwEiWeFX1E8If1OdGHigcfTLls+cRBJqs6cPSa2XrKFTrB8eGq4aM2xFJ+f2PfLQBsLFfl0pmiipVZaQrjaF4r6GYh9Qx0Zi5nU3M6bfaVzDaDyc9umt0xhdyWmO7r2kO3Dgs3uKzVoTG6FiQ8uWIkwu+a+3xkjkw+MG7DmJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=neQ3Ln94OMYkFN41cp+XL2Ok9wGJ1AglbofXvFxu0uI=; b=kpRa+fa0LTd5QYt2c7B3IFaGcKENi4+v4E9Okf6UKXB8SXRu1bd/BMeIMbXbyAlZ30aCIzrBceyWWPQNO992MC8yeDEkmNwb4s4uw6bWPFnMkmJN6sntvCRviNYqjNXTJjxX1vd5WPtF+vTZ53EH5qns0matf3eH5ENgw6UtFEVl9oVrCk+Sie02vYtTKpVXlV728ibh2F6tUE0rtz85WM2QI3Lmw/yRUSmByDQ6S6iAz6E40qh2O3rLuGpd9FEZTPW5ZrjCvbUOQ/FcZ9+amuc2NUzOXKoQMtuwL1gWp3im1aolV1txfbFdcHLRalaevCSYz3rIzzbkBJwOrIgOQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=neQ3Ln94OMYkFN41cp+XL2Ok9wGJ1AglbofXvFxu0uI=; b=IMXoIWFSFt974ZhuoEb5sct+5/sV0yspZYo7g2NHD3v0DfFduXaGzFmxLNKobUgkyxXsTDZqY7GFcgtY62+ngCEBuFxkOkB9gHix4lYaSHITK9H5qK7imS7Yd3eZ5w/bVeR7ffM7sNa7V1Vw1IlxbwtxXAVWhCEgZ+QSFvXvMDcpg0Z9wCDUgnR5hee+R+J+wXFIZxAt4hp/1xuCGmYoi9Ksj0HhE6//ti6aeE6dD5XmsoOoyTZ19V1pDd2Ykowp1babtYCuGWYquAX0+yy11BT7SAFSysv9a0ARVrnmw+KL1iyMp/IH0mBUTwcEOUBy7IJMClOAm08sytzAWl17PA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by SJ0PR12MB6733.namprd12.prod.outlook.com (2603:10b6:a03:477::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.16; Tue, 6 Feb 2024 15:13:06 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::96dd:1160:6472:9873]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::96dd:1160:6472:9873%6]) with mapi id 15.20.7270.012; Tue, 6 Feb 2024 15:13:06 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Lu Baolu , Jean-Philippe Brucker , Joerg Roedel , Moritz Fischer , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameer Kolothum , Mostafa Saleh , Zhangfei Gao Subject: [PATCH v5 13/17] iommu/arm-smmu-v3: Add a global static IDENTITY domain Date: Tue, 6 Feb 2024 11:12:50 -0400 Message-ID: <13-v5-cd1be8dd9c71+3fa-smmuv3_newapi_p1_jgg@nvidia.com> In-Reply-To: <0-v5-cd1be8dd9c71+3fa-smmuv3_newapi_p1_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR13CA0015.namprd13.prod.outlook.com (2603:10b6:208:160::28) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|SJ0PR12MB6733:EE_ X-MS-Office365-Filtering-Correlation-Id: b4dafb2d-3e26-4961-e89e-08dc2726192e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iqxyVAIJxmazbk21KRSI+t07eRhVbHTUJ3Uv8zkWYi7faet1w36T47ZDr48xfKDsrTmxkXx7Plll9XnFRiQ+8P0PUo5Jiq/Q/mCB3FjW2CSX3EMyBZPJ2L8bnU+AMQx7WNuj1QB3KaF8ehgqcbbbRFOL1NZ+mJ9cjXImJ48VZgIiFwpIBjEeWVMBqVE+GM/nQEJPI45NRS1OSwlHHnkySwaDn/+flkZ6WZnvSY27XbRwqoVF6Cx42cJwI1OPU81DboCdA9EHHDoAgyuZf7OFJEK8fPhYMe5q/8MWxdTwbeZpKQ3JP8dENMWPYvm6zQeMNqxB31ltsrGSiVT8jZXi40gEqY8EVnDDELddy3AJBzEnukLQweblb8IIi5NN6EbvfBv8anX3Y2CTwOoEebPxwsW466ztra1oWoytKiW+MIbVU3bz9izIh3cq/lK5uaRKG8V3q6sqSWXQwB4YAD+MwIroioYo92KevRR64+5WhRMHrd/HGidbCOpiZAZfdDVRGXjodP+ED7DcrlrPzc+EqhavDFbt0Dkg/512upvncCm/aRlWk67p9Dh1y1UoVe7I X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39860400002)(366004)(376002)(396003)(136003)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(54906003)(316002)(83380400001)(110136005)(38100700002)(2616005)(6506007)(6666004)(26005)(2906002)(7416002)(6512007)(6486002)(86362001)(5660300002)(478600001)(8676002)(66946007)(66556008)(4326008)(8936002)(36756003)(66476007)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hblDwt2yvKqjcpZ7+MBppk/WXC3KJRpnAXAAtrfd+Lwov2a9ubyVGgwEC27+4MAdnEWwL24lpacTO0ipCdrhNYodKsIKc/n0+MyfHnVrNhnxGE3wGfG7FSyIW53cymhvsHl91s/qbkd96DNZmRII6iCPn6Von+a6KP0hfh6fkjmsEeCHBUgZm6GnRbOseaSKPXW8ow5LFC71I0PeAvE6k09eBfoMw3snLRMcMo1IGwHxHOXs2BI1BlCqksGykE6Bzpn9Q6onRUM9at8dsRydzmhEg+drZ6OsXD4LgboVHBiZX6eG+fyUX+3AGsnctlAclolsY8boJZSgzzWrfb80Gjeer4TTGTy5dJOMsoc66Unh0RfpqZmKWUw//zPR/TB37tZiiCR+wlclBfLyRx/SOlrRH2PrwQaCBIhDnfDATsKGTE4jw5Tnq+qwiHDJi4/rfNQbTVKsPM9zqhNr9eP7rj1Tr+4vRVSJ7lGCb8kVuaiIPx51g8srilj6jPduKXxezQQHz/0PyQJygXz+dI+k2KLLvqUp4Ly1TTy7MU8BT59GSK0NpnE1qAxNtW7e1hkoR3ebEsNuDUTR0GrhGEh+q+cPkT+5be8Q2jT3w2YvwyGPMwHCDz8K8zJUVSMRoBh+HwsRW5XlILErXrPQDITa9MThD0TlIl80bZ8vhXETRpqHJCbW/sXdkkVMrzmuJDCTRcxwDdmY+JFtmVmPImHR2CeP0T2iAtcwU9VAgkF4TgAjhfuvqDVZ1LHMhknFTsnuOfLtp7qFJ/ek9W8zieDK+VQaNk3v6gHi0l3s9VWF9WsCe26yrIfYwoD2VzjpyMEAzPAkgWkvrwVstqg4RC8o2IWf/Yun/xsatuWn0YWVXqcDdSTMTeS59AfDJxDBxWZm+Zi8yx+8lx7duuOrIqLC71I+ICx+ZpZzNfEZMoIgD2HDlHsjN1RbLRRmimQHVy6TkfD7XgU7H1TEuX+BokcxkiqoMC00HpN9EgPlAjGlbhnLDlqLgKkwS+6hI0w4wvwg0zD4Jfe9YAFOVSFjL0FTHYPUxxH+xLQQWr6OaOlVna89Cu0hYSYGWf2IqRF2cB9saWBsP3nciNHDiCIt1069BVIOc1baQTw53EopyXtG2Pf21XNqAhC2oI2hFVj/RQknTDt1qv2TNfubX7It7MG+8qN8a5z0grsAK9xDDqdNwGxCAUp22Ywi5+gLVJlc15EI/u8yHolFiSfw4PUJj+6himLCv5lLupIMlmeUqIBw4hR+nFC7AWNX8gIzQ0FL7hpD01q/84RKXeH53jOcNRJK2qEILT+T31pVj05tC/+bTvNl8Y5G/AYEQHNaTDpFBCJ/QKNLwIVSaP2kg1WR21FFU2B9ApxPIaMuPRssHUlyagXoBPtJnJLcxJyLpAQ5xlo1UAtkYteET0s3tgneEetzoe/bC0iCTLxGqUf3h4fpodqWjfrAepwoNOHbY/BfpeyRTtpqHDlcnyAGP6zcsGnotwOOHFK+r/mWhgkfgLOlbLC80X6NISCogAViGjF6NnJu3Z9m/vLRo3vFKTPXJoGwcS64qd5XBqcDztVLgbkALucxPXZ9Z4XsQ1gI2AktqOaW X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b4dafb2d-3e26-4961-e89e-08dc2726192e X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2024 15:12:56.5763 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qa/uK79N9VHqxdSblSqo3A6MWWrmBKzSG6IsMZjvpAUKClPdg553+4v9EdLVr/Dp X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6733 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240206_071329_135135_088B089D X-CRM114-Status: GOOD ( 20.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move to the new static global for identity domains. Move all the logic out of arm_smmu_attach_dev into an identity only function. Reviewed-by: Michael Shavit Reviewed-by: Nicolin Chen Tested-by: Shameer Kolothum Tested-by: Nicolin Chen Tested-by: Moritz Fischer Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 82 +++++++++++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 - 2 files changed, 58 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index a98707cd1efccb..a940b5ff96843c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2253,8 +2253,7 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return arm_smmu_sva_domain_alloc(); if (type != IOMMU_DOMAIN_UNMANAGED && - type != IOMMU_DOMAIN_DMA && - type != IOMMU_DOMAIN_IDENTITY) + type != IOMMU_DOMAIN_DMA) return NULL; /* @@ -2362,11 +2361,6 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; - if (domain->type == IOMMU_DOMAIN_IDENTITY) { - smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS; - return 0; - } - /* Restrict the stage to what we can actually support */ if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1)) smmu_domain->stage = ARM_SMMU_DOMAIN_S2; @@ -2564,7 +2558,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) struct arm_smmu_domain *smmu_domain; unsigned long flags; - if (!domain) + if (!domain || !(domain->type & __IOMMU_DOMAIN_PAGING)) return; smmu_domain = to_smmu_domain(domain); @@ -2627,15 +2621,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_detach_dev(master); - /* - * The SMMU does not support enabling ATS with bypass. When the STE is - * in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests and - * Translated transactions are denied as though ATS is disabled for the - * stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and - * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). - */ - if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) - master->ats_enabled = arm_smmu_ats_supported(master); + master->ats_enabled = arm_smmu_ats_supported(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_add(&master->domain_head, &smmu_domain->devices); @@ -2672,13 +2658,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); break; - case ARM_SMMU_DOMAIN_BYPASS: - arm_smmu_make_bypass_ste(&target); - arm_smmu_install_ste_for_dev(master, &target); - if (master->cd_table.cdtab) - arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, - NULL); - break; } arm_smmu_enable_ats(master, smmu_domain); @@ -2694,6 +2673,60 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return ret; } +static int arm_smmu_attach_dev_ste(struct device *dev, + struct arm_smmu_ste *ste) +{ + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + + if (arm_smmu_master_sva_enabled(master)) + return -EBUSY; + + /* + * Do not allow any ASID to be changed while are working on the STE, + * otherwise we could miss invalidations. + */ + mutex_lock(&arm_smmu_asid_lock); + + /* + * The SMMU does not support enabling ATS with bypass/abort. When the + * STE is in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests + * and Translated transactions are denied as though ATS is disabled for + * the stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and + * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). + */ + arm_smmu_detach_dev(master); + + arm_smmu_install_ste_for_dev(master, ste); + mutex_unlock(&arm_smmu_asid_lock); + + /* + * This has to be done after removing the master from the + * arm_smmu_domain->devices to avoid races updating the same context + * descriptor from arm_smmu_share_asid(). + */ + if (master->cd_table.cdtab) + arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); + return 0; +} + +static int arm_smmu_attach_dev_identity(struct iommu_domain *domain, + struct device *dev) +{ + struct arm_smmu_ste ste; + + arm_smmu_make_bypass_ste(&ste); + return arm_smmu_attach_dev_ste(dev, &ste); +} + +static const struct iommu_domain_ops arm_smmu_identity_ops = { + .attach_dev = arm_smmu_attach_dev_identity, +}; + +static struct iommu_domain arm_smmu_identity_domain = { + .type = IOMMU_DOMAIN_IDENTITY, + .ops = &arm_smmu_identity_ops, +}; + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -3083,6 +3116,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) } static struct iommu_ops arm_smmu_ops = { + .identity_domain = &arm_smmu_identity_domain, .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, .probe_device = arm_smmu_probe_device, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 587f99701ad30f..23d8ab9a937aa6 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -713,7 +713,6 @@ struct arm_smmu_master { enum arm_smmu_domain_stage { ARM_SMMU_DOMAIN_S1 = 0, ARM_SMMU_DOMAIN_S2, - ARM_SMMU_DOMAIN_BYPASS, }; struct arm_smmu_domain {