From patchwork Tue Dec 5 19:14:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13480660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D076AC10DCE for ; Tue, 5 Dec 2023 19:15:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GRDAqOXUJ07dZkSsaHLQgDS0Al9VPHVMF+mqTKyTcEk=; b=AXBPDK/2ugpTJO SyktjyNj5bFW/X/cKST/60uWysrxCpop5WgF6KrgfQ6Wt8arFryDU3aL3P8Ledbf4zAPDRfhDsAAh kq4XeMC/+vcYyZdD7dXCPZrfFVGk7w05myCbrrmqSPcjS9LzkU/3upvzopsBfexMoBlxjlzNkX/Kd KDF4tUnH8hurXQyGy8Fp2CXvgkD/NSzHeKlKtG3DU/y/0AVF/ljHfKyuiAX2t6jS8EbDo72EywjAe xZ0PZeQRRXVODXZCYYy3qkfOLlD1bQ1CRlMvVKaI1m3bEa5wyBqal+/DKbgl4mF7N0K9zDpzjs5dV mpn5OOLoyS1zksYGyekw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rAat8-008FKa-30; Tue, 05 Dec 2023 19:15:22 +0000 Received: from mail-mw2nam12on2062d.outbound.protection.outlook.com ([2a01:111:f400:fe5a::62d] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rAass-008F29-1Q for linux-arm-kernel@lists.infradead.org; Tue, 05 Dec 2023 19:15:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VMrlOk8wLhMDb3C/5LPt4qxRDJr0DyDkrhbaH3tjrLn6qtnLsAAAYrRRL0YeC3trnWErYv09KpIqXpdHWnw2KT9Mn2KgVzRAZKKXrP3zUcOa/6q6gasXNOzlPyBRbqVaE6NIivPXLLbkSP7rXV3w6NH6UuO5fXJvszsqw66ls1rmljv3uELSy9laMQlgZZ7Bo1E10Vqqg55kxksGNYOHxQCC8h525yb8WLfHB7mKtAN/JSGbodP/Co4dbPyes2pNwe0RxIXGM66IfQ5qYk7da0sGtYoAS6/6HIrbR7+i1psncIlhwtI8EozRfEF0bBTmEvC/1tKPmKZB0W/b2iPUKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XQERdPUsIWx6+TUpXLhG5WwgEbG0x7s28cYpuD/80IA=; b=K2X52WYATBmYlYN0AbUCh/b89mvjZ2sVzJbmUPJQYHS5QEN57xg2AoLoqhTh5O5HU2vNdLBbgE2wHRQOApFLhJF3AsFABWmforbG+biFmAe+HXQaTXYqT3H8eyOxw+YiRl1M6L79tDS8/UgGggcvInvfKeUHSP3ZH/9p1SK1v+4uZ7Y8rfwSD0imTfiaw5QtnkrNjd9SZZ9h7aFBC3UFsxtkJTdaPymx7hS+G/efa71aHMpGsmQyDx/0XSAG26OQjBnDmlgn5SXUVejs7er+BefO8F+xqBpwcAScuf061GQfMyAa8Pj9BDifOMXQzxwLf2o3PzQ2HYVQdL48LlN+xQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XQERdPUsIWx6+TUpXLhG5WwgEbG0x7s28cYpuD/80IA=; b=bO+ODFD4Bh/LqFnee7CSl4p1bnk3Dtf7ZZLLrMtHrS55GV2sIMc0hDHsWZgb2rzBxA1Oij6+TlnOQyMtwGhW4jnyzdH3pHLlk5PkswtIRPkToDdCjsv/jc6aWOVu3YgzoE50VNnDcLTYgkI+aY1414KIyMEtGcPXXx/EbmEaQl2EjKW9mB4BRO7O8pkt95v89nSPSClmpsYculHrR1VgAB3f5mYuXOxZcSisAytOBzq6mavh4VdrDFV9CIDU5J7aKBg1/JHSc0jh7h7sYqfjSAe+yMIgSIIpGx9hT3mUg6VUDJ2neA9/SZLIPlWQQvsZvQUzE6wpShTcXzSLBEw+mw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by PH7PR12MB6588.namprd12.prod.outlook.com (2603:10b6:510:210::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Tue, 5 Dec 2023 19:14:55 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::60d4:c1e3:e1aa:8f93%4]) with mapi id 15.20.7046.034; Tue, 5 Dec 2023 19:14:55 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Eric Auger , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameer Kolothum Subject: [PATCH v3 15/19] iommu/arm-smmu-v3: Add a global static IDENTITY domain Date: Tue, 5 Dec 2023 15:14:47 -0400 Message-ID: <15-v3-d794f8d934da+411a-smmuv3_newapi_p1_jgg@nvidia.com> In-Reply-To: <0-v3-d794f8d934da+411a-smmuv3_newapi_p1_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0130.namprd03.prod.outlook.com (2603:10b6:208:32e::15) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|PH7PR12MB6588:EE_ X-MS-Office365-Filtering-Correlation-Id: fc1138fa-8c2f-46d4-5057-08dbf5c67521 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z/jxdFMyHpMyeG6sH0ALAw/Q4dtHkwnYFIpMbK6fRHLNXtL2S88lQFe0DzKpgCN+GcwpW6LxLUj/fx5pVnyaQpt8aNY6TVGHijDqoKGNMkSK9G3+/e1DunQ2EiYQN3+VO6vzTZJWLif0zY6hHDzJ0F0s4L1C3R0Wx2nsba7FxLLC7dlHqSa244lvEhQ93qmkZa6p/8hBuxcYJJlozo/EzYQsJK6xjXFr1fw0kLv7VFyMcIgAcMCT0F1SZpa0jZ4ILMiXs92lWvHSbL+g783VCu/bN6Kpj3SOj0vckY5aDBL++TBxXu62am9DKlEpM3SR83sz5LGX+LxNV1qEYEnImJOIrJosKYuypvxQ67Nway0uhpQjGIgAVI1r1oB0/us3sevnlbR4WAMwP8ROq/inwZJM8uaJCSg6Gzi9/4lsacG+t7HEuY9/ZtFGcEk5ACHQ5ov0rXpBoNdf+GHeq36QIvh2BrwdYjs4kp/bdT4W/Dplugv2tnhJw1ObJ1sabLNOjnPv5IscqRNDK3eNjwEOTTNk31EYv5C1TG4swDDjHtyp2K+q47pQRh93R2IFnk5n X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(136003)(396003)(346002)(376002)(230922051799003)(1800799012)(451199024)(64100799003)(186009)(478600001)(6512007)(6506007)(8936002)(6486002)(2616005)(26005)(6666004)(54906003)(66476007)(110136005)(66556008)(8676002)(66946007)(4326008)(316002)(83380400001)(38100700002)(7416002)(36756003)(5660300002)(2906002)(86362001)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9EinjE74chJ8OkK79C2SMjR1uxSoilk1DYsyMv0iA67l6HkurkghN7Qpx2vZnPy3eqe6INjGOhmgWj4X49UHmPr+gllT9+vsjBMqWo5AOnma3BJ2IzsIcOTcK3aB9fHXlX+PsBaaOE7Rh8Tbp7T6t+9OYhWnCXN9IHWqUJIaByVxgiH/Ob738RJ98jE4LtSlhVX7bzmfBy7KdIQ0TzaDBqYxNGsdnm6Cj/CdlQOZjGXdxkXSfeWO5ND+VjgWYtzktuXBP/fgf/K0HGgvWQqpkikq+vHTJKebyloe1Fze5DGaHs78cwhOy6H2tiR9KhV2wmeIcmfMaGEuKTlp31ASoM0qXRPhl8HxiQecoB65P3AMCvfrSWJCypQ1vl/ZemQsYFQrVZGfb3DMQcQurnYXtTFmmj1Gysgq38rGbhZU1uiT+YAhiD8CBo2k6yFUdnjNV8y+TzeT5HNiYXL2Lytj2BCaZa4UzjId6RUYM93j6GW6aGdvuAAFrEUlm+9sgb4rudc2JKnKlbt6lFfs+OfaJG4BVVextOrazpKOUIM0XejxdyCTIcvtNqACdscoYuprai7m/SDj2eNkjNnAC2nijWaUwgIESCB2LoklWnTO+uyMN+xiZbBSCO8+8q6W0SQPKPrR2eHQVs3WH19/jFWOD4MojYOh5NVl+OXuIBiKvskbtRI+e6O/IgzzYrfA66W5MJwlfrU1TkDfg+VsZHXO94JvN2sAjU9/GggSglOXnQvuwQLdF7Qa87Ht6aDaO7rUTNV3qMi6vLR2gEX6/nY8tXsotGJCueNj5fZmPG8ZDmRWg1MBbosr90bAY9DCvJjvC0u1mxEI0U7zBIcaW6hcBUccm7KKunWgdxxy0wVze9cfH2R4Hiv8S4mym9BVXuoMh4mdcai7TmYX7l7niE8DDYowD0WQAK3yVRFYZXThfUERuII6k4FNCwM32AjsRuEVC8wDAXagGqfWPaftgj+qcUOJI01dBTX4g0P6oP2qMXODEjIDtFlQvgtgBfRZh4KF6nFVwxIKtb4ZGW405CGx7ockRYVgcl8FqtEgIXcwrYqy3IHqINqgXXV4CoBteTZkiHn09aX00KD2vrxMeW8gBeNfubzLGSIvLeLcjlfMWp0gOM4KbfRTAKc2NJqfMuTR1zGDBqb6xbTtb/3l1Gk2DXQ92R1y75qNrn5PWN2Qwb1YjZGLTC4eh/UbSn8o5MZiEpBwlqLPNmSCi+AI7dYkqZZ1V9RsUJ/YM5Dr1PJZchhtfsuDKwROlEc+eiogPLBJmEt54/T9GFxaKVEUJuu20yu+dZmO3GBUIi0IfgSv+8wcitK/JUlZNrZmUUH8IfLc0xPFyOQuZC6XTqJPz7czXwrYDlVrtjbLrm731AmIQdGwoEW67gCN0jrVPccRDQIswUBMhZnDNwPlLWxQbKZKjY50ef66y/ofYS98ZWgGFL+b1Hk53vpVfoLVR5kRxd1gIPilJ5L8pwHf0NuJn8E1trj8jkn8G+Clup2mkJf/4svJ66RL+72nW7p3c1YMURhmw67rboIiuZbjnjChL+Ha6JbvREbIqaZXKLUsxUhD9d4n9h0wQUKR2f7r3hz4eWAr X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fc1138fa-8c2f-46d4-5057-08dbf5c67521 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 19:14:52.1774 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8GE+ajxudAhjU3bIJNjsxlXvMJP2rMAbFoIfZAJiyncH+nC7wjkGY5wITeAQFiyY X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6588 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231205_111506_510243_8FABD1D6 X-CRM114-Status: GOOD ( 20.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move to the new static global for identity domains. Move all the logic out of arm_smmu_attach_dev into an identity only function. Reviewed-by: Michael Shavit Reviewed-by: Nicolin Chen Tested-by: Shameer Kolothum Tested-by: Nicolin Chen Signed-off-by: Jason Gunthorpe --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 82 +++++++++++++++------ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 - 2 files changed, 58 insertions(+), 25 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 525048a79e8a90..9899caeabc8744 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2173,8 +2173,7 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) return arm_smmu_sva_domain_alloc(); if (type != IOMMU_DOMAIN_UNMANAGED && - type != IOMMU_DOMAIN_DMA && - type != IOMMU_DOMAIN_IDENTITY) + type != IOMMU_DOMAIN_DMA) return NULL; /* @@ -2282,11 +2281,6 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; - if (domain->type == IOMMU_DOMAIN_IDENTITY) { - smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS; - return 0; - } - /* Restrict the stage to what we can actually support */ if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1)) smmu_domain->stage = ARM_SMMU_DOMAIN_S2; @@ -2484,7 +2478,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) struct arm_smmu_domain *smmu_domain; unsigned long flags; - if (!domain) + if (!domain || !(domain->type & __IOMMU_DOMAIN_PAGING)) return; smmu_domain = to_smmu_domain(domain); @@ -2547,15 +2541,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_detach_dev(master); - /* - * The SMMU does not support enabling ATS with bypass. When the STE is - * in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests and - * Translated transactions are denied as though ATS is disabled for the - * stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and - * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). - */ - if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) - master->ats_enabled = arm_smmu_ats_supported(master); + master->ats_enabled = arm_smmu_ats_supported(master); spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_add(&master->domain_head, &smmu_domain->devices); @@ -2592,13 +2578,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); break; - case ARM_SMMU_DOMAIN_BYPASS: - arm_smmu_make_bypass_ste(&target); - arm_smmu_install_ste_for_dev(master, &target); - if (master->cd_table.cdtab) - arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, - NULL); - break; } arm_smmu_enable_ats(master, smmu_domain); @@ -2614,6 +2593,60 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return ret; } +static int arm_smmu_attach_dev_ste(struct device *dev, + struct arm_smmu_ste *ste) +{ + struct arm_smmu_master *master = dev_iommu_priv_get(dev); + + if (arm_smmu_master_sva_enabled(master)) + return -EBUSY; + + /* + * Do not allow any ASID to be changed while are working on the STE, + * otherwise we could miss invalidations. + */ + mutex_lock(&arm_smmu_asid_lock); + + /* + * The SMMU does not support enabling ATS with bypass/abort. When the + * STE is in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests + * and Translated transactions are denied as though ATS is disabled for + * the stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and + * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry). + */ + arm_smmu_detach_dev(master); + + arm_smmu_install_ste_for_dev(master, ste); + mutex_unlock(&arm_smmu_asid_lock); + + /* + * This has to be done after removing the master from the + * arm_smmu_domain->devices to avoid races updating the same context + * descriptor from arm_smmu_share_asid(). + */ + if (master->cd_table.cdtab) + arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); + return 0; +} + +static int arm_smmu_attach_dev_identity(struct iommu_domain *domain, + struct device *dev) +{ + struct arm_smmu_ste ste; + + arm_smmu_make_bypass_ste(&ste); + return arm_smmu_attach_dev_ste(dev, &ste); +} + +static const struct iommu_domain_ops arm_smmu_identity_ops = { + .attach_dev = arm_smmu_attach_dev_identity, +}; + +static struct iommu_domain arm_smmu_identity_domain = { + .type = IOMMU_DOMAIN_IDENTITY, + .ops = &arm_smmu_identity_ops, +}; + static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t pgsize, size_t pgcount, int prot, gfp_t gfp, size_t *mapped) @@ -3007,6 +3040,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) } static struct iommu_ops arm_smmu_ops = { + .identity_domain = &arm_smmu_identity_domain, .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, .probe_device = arm_smmu_probe_device, diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 21f2f73501019a..154808f96718df 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -712,7 +712,6 @@ struct arm_smmu_master { enum arm_smmu_domain_stage { ARM_SMMU_DOMAIN_S1 = 0, ARM_SMMU_DOMAIN_S2, - ARM_SMMU_DOMAIN_BYPASS, }; struct arm_smmu_domain {