From patchwork Tue Jun 11 00:31:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 13692715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F208BC27C55 for ; Tue, 11 Jun 2024 00:31:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PLPFT5AAgWpenaMl3GICWNRiFq5W1YEZt/19t4OrKlM=; b=HEGs3NQO+2ethH GK6r4OxFevbKDSu+ol3dHK/fyPJrhAxxpkxdZUr+WM6A4j52/xnR7hecw6Yg+z9V3BiWXlpKc3Cri FEumVcDxpjgDUAm4SqZY1tbE94I5OT3wB0MlVMk+n60n2GpyAc07dfUWkiGjhON4uhb3AqVi/TG++ LSYLEkJ6k1+sRYmGv+afm+bk9UEBzCcpOfyqS27cJSOi0EB+DCGeZ33nJZkDUIM0BW4hg+G4HF5p9 ahYUWaRVEx2ylIy4Zox4TX1dO0PgX+L2PLPgYnxD6qL6p0weB6W4rRE1idyOG2OruyPbFkQdfHN45 2vBewGDpColbLz+sefeg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGpQK-00000006yTL-1djB; Tue, 11 Jun 2024 00:31:40 +0000 Received: from mail-mw2nam12on20629.outbound.protection.outlook.com ([2a01:111:f403:200a::629] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGpQB-00000006yMq-3dtp for linux-arm-kernel@lists.infradead.org; Tue, 11 Jun 2024 00:31:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gtibeu8Nu7C42YsEqKBLMHnxGbNQuMDy1RauwpbqxHDnG66ebLRGncVHSzJiUyXIVv8Wm4pPI+VCpq2dGEzoj3nf7vUvzfzDazaMEctyp1xEoJZlAX6FFwJ3j2J31l6CcBGFCv5+2RWWtLCZ8TD3X3yLG4+aCPvpTRH09hvpCInsDXAm0GR41MuetkmrSWmN0kaZFM71sp9QWmvpRvmj0fvMk2NK8psmF28I233/MVFiCrZdWOirHZVbKwbkdAzvo11OzPfouUygWZQ15EKbQSt16wd6MCcjkZztq98eQuTL6E2klriw8YLeEWCNxSn3PVt3hXawJJkXnCiA0AiGxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9wicn8xpR+3lz8Lwtw6mtwa1S5KQEjSdyhRq590rrAw=; b=Ji/+2ttlfBU0bLGbLfC0o8A6hXjZO+49V/7OujSOMGjGkSHxWOvW4elSYhKdYkJc8RzYcx9ZKrctuoH/JlYdhl82oBOuRf5rrL9MN3ARqjER8DVEo3O5cUAzd3alu70MPjB/5sDioTw5RNDlYLTQX/XsYVwBemOPwlXuY6HnpXlXq/1zF6t/3woxsv6Scx/RvFl+VThI+tfcdECBof/Z8KkPpuRksVVZt/+0LqxiUSk8mGF8Sw46hQe8mFrDi9vSjk7jfpRdB6dSa4qNI1HTngCqhYK8Pp4gJ4Otn2Enm5MxKUbWExUgAemJRKwAKB/b8Encp2uD9Gann3MAZebGlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9wicn8xpR+3lz8Lwtw6mtwa1S5KQEjSdyhRq590rrAw=; b=gjSwGUMvGOccspGbFjwoiUJXi9lkjePMZ03G73UCjfUFoLvC7WBYky0gOjcsIBuWkHmpRCrKvJYcVHL4d4GeIlXuMDDFOlC2EQSHl2MGoTYb01xUtbG4MWF5LrqpF5nkHEl7XTc5GZTwAVienyCTJUcssSQgVovPY/8vAhSkHRl80P4pvti6nWd8gXqUgtv8MWRHoPwvdt8v7LNMbsmUFokXotkXElGl7Y8Jq8vRcoFqKzeFuLPs5nURlgaI/HGjm2EOdwBlQc2sQDeFncCjiafFp2T8haWBMQ3CjnndXe96EeJEPlpG6KiI/t9K4YY6+IwIGqi1b+hWyC6CZVvmGg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by MN6PR12MB8565.namprd12.prod.outlook.com (2603:10b6:208:47d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.36; Tue, 11 Jun 2024 00:31:22 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%4]) with mapi id 15.20.7633.036; Tue, 11 Jun 2024 00:31:21 +0000 From: Jason Gunthorpe To: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Will Deacon Cc: Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Ryan Roberts , Mostafa Saleh Subject: [PATCH v2 02/10] iommu/arm-smmu-v3: Shrink the strtab l1_desc array Date: Mon, 10 Jun 2024 21:31:11 -0300 Message-ID: <2-v2-318ed5f6983b+198f-smmuv3_tidy_jgg@nvidia.com> In-Reply-To: <0-v2-318ed5f6983b+198f-smmuv3_tidy_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR19CA0060.namprd19.prod.outlook.com (2603:10b6:208:19b::37) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|MN6PR12MB8565:EE_ X-MS-Office365-Filtering-Correlation-Id: 86da12e4-7ba3-4a3d-7380-08dc89add145 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|366007; X-Microsoft-Antispam-Message-Info: WeJxJ/Oi9WOXbgjSo29BDhYwHL2fqgx3qjUHXF0L6WrSCkXf++S8kWc5gC/TaLQjKWX/1yU5EIYsZq9ee/qkMf7l3lXCOtUrzUPGFKE6zLbNDJvMZzFp9U1epni/nDdy9R7eWeztaI/Wm99VWLI9Tcc81BxpPrJkKnlscLxu3buFrWRhTUswIy0UQFLKgcDt/JWhXGDlOSuNIRZSUiLl9zSTG9tr8bBJI/zTJoEu6eS+sIFArwmxFSfZlVxoaYSBqSSknH4oqn6kNoiRL+X3yTaDTxCejNI3qG0xv0lK2P8QUtZr/6opraXztdHzxH0Pi2paCeCDnyQo7HX+KiFBUHj21Eh/1izknkurzKh01BSara8RcNZ0nljmHOVC8zhzeUHU4hbKucTK4SNK7yBeIvtqWakE8CrxapLNarE/t8DkhzxfJ3SrCZ0v0QTvgzYFabVzLCQzz6dtrUcVK8fGOS9fO73CV8sJF9zX6A7wgF8FPLefp4IwGfGiOV4tGmYAEwfdoSfgq5UfFKqtORnw+RzG17alip+ngxWS+XqIpCCjQJwGLKxF4KLuxCyOVpFiiYR893wp26HDxnqTJL+SwxfQ3KLSgUn2YuTdA07ZSJf3TzBWvm7lqKGm/vB0OJP7CfZp1cQRhGCVNkQfox3y2QaY0+EamdnwaTmpRtOpgdKhRjGmSwmiIwKPsJl6fr5YRYDXOS4FxJ9V6vIuFdQ/VBDf5p9jTnS2BWoWGorlbWuuN513hphbDWAO6Gzb6pv4QOwrAOlC9kPfBzWn/qH836VNYZh1jdN8PgvPyyx2qVUsM3FzIelULAF9joqQVFP91ooq3hdCxEvu5MrbRYoLG4AMtutujLbLDA137/3SzN7SBlvtfBjcKWHiFlHF6BNkNw66EBhPUBm8OEV6WBLQjOy9dijtMyuX5p+8FClEEmxQImc/4f8cDbbUTBX6k52IjjIZHBFO+y92vZ2h8v2mO3ywP2JuLkhjDTZ+ztg5CoNE0tbSFx2qPd7cPHcK0kvjyGVZdTIvgnQzsnjrYN96zmNJcuXT8xAG9DO5isTBt1gcIbN2eEReHSKZRp8yV0c0Z9vEoDjPjfXDovltEcrujzQiK12Q7aimLdEm9jooJ2DYUR2UR4HbiQT/5s9QtcK+5K30v1EflSVyKQWlYHaaQx7xGygGh+t9HmNknZ4w4ycdayw2tPzZ3wc3XHpnVUtgWoGNpNy7jDcCEsk9YHKVh+BbqxN9MX4gwr4ab5yybpsLcaKA9fUUhQgqAjcg0LAydNbsbHEu5V00Y4a6HWxArHoquGU7XQdR6XL9pMUvudE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376005)(1800799015)(366007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: yanyrZ/UDtyxPs7LvFCj6vYqKWIuYp1C3lDL2D174Otae0Q/ThpMn2hZBfDX1Q0bZEsSNQCN83KgLWVOlvDHISkdb+PACH/PORAwhF96c6BEYuZ4g4QIh/ziNF+oLSUIzwr4Lua44PTrC4iCl19zm4ddyHdc2Q7QbPb7EsUpTYVNfq6BZNAauv/m+L+L0ASyNwrL7zyO4nQwSQaS72tQxrw24B8nwZqwZMeoecE8m9pze6KVD9B9yey/jsr90OgOOKrEeDMBCTP4iMHdU76ppERd0jgCySWiP76vWhtz2XD1zhMxNCN86K4kojpIoYJwvuK8a6WM3HrCWdmM5pADlH4vyJZIMEyNMzbaN/qKulXFD/QTWM4KsHsSM+OiE82kH4ngvO16jJ73SSjytv6Nga2oxCh+EzyLSxy1VNofA65Bdbis1HvFcbWOBI2p3WxFEE3qFTMy7DjjKxUy4aKaGjNc+tYilj6SdYL2NHUkXQ+pzg1355xgYdPIU4+JcV6IAF3Q3QbdZvxko6N2fJI130QacK/pDKcy9JPdl9VNMNmt/gFS2wMq8VZgrr3ciz5bsMngAqHQ0Tts52l2jUxbZzMSp8S58VdBFrYLuvRbDpGXsUldfvk/WLHCWBJy3Y+wM0tu7KvPwEu0PDtkkAMQ0faHm73+8QXFE2/ZyLqD3eddcmmFfWLzwtY/8BGpKvf65mWm8TZ/pp1E0uIl7MOhMvxumaf3oqa0FCYBlcCp/BJtN/ZcafCIbRUTSAZdxf3Lft+nvtWT1qEZcEZ9LAAio7rsCtqATNjN2R5zgAtJPGbU2/OdUUCkPwYQcTahRbR5DQ8UwkVC2tzgezm3GPDVOz9VCfrDuC+JNLdiSAhouaSe7ZePy3lDv2ct3iBcYUrClqbZCEHahogN6E4ZpyHJIhxCYFB6NYAkT9iigGcqjb7gNvwupMKLZwHrXI6CiWdfsV7IsbHgEMFhq+k16JIfpPg4YalUh7Khc+B0MiSAvHz+Up7GxGqtLpj/yF14Q26+0FFHt25I5z+I2mAsJgwub5MWtqnifyWaR3gtBl44ticscx32b5S/gKVbSPGs6b+6+pG9475WqztuQpEHmegnkQcVjNzWarY22H0UposhEzpKN8Eu2+NlZnFtwAZ9kBDJaBduZ6gSsaNOz1PaYR8aWB3GtmWrKS/PJH0gFdDUyNeUzCUGAbv6CpXg7rcPLL+183pYHZb+Vo7drNyLb/7V95w9/xjlKEI0PFVDuswbhnDFoujdoLNkYxhay3Pg9TEA5aop9U6bQIbbBV401NuZT88F6nSdj9Phrv+Y/lShsSfs+lqxb+odbu5DJQcUMGOA1qap9MBE4ULXyoF15BO7xCCH58fv0UW+UtM9sktbCFdV4cWGrugglMOVM/Q95PPRdRK+lauREccuxCpRXjh8plcH0bx1x1A2F23zeITvDM/bAwxVtIZqg89VBKRdOlSsdfjO3uG4BgyzW6V/DeQHvRYIuohWfSt/hjpuST2wHq3Oe9DyGQW+9OEW+h5sAfpkGOVJyDunHnLOCR7vLKVMeXr1Vp3J7PvJzerh27okfQfUnob9qLRhfWuwiU+mDRrj X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 86da12e4-7ba3-4a3d-7380-08dc89add145 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2024 00:31:21.4513 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: K1ilnvDcbZ+6siYuILljdIC6B7UOHJ19sw6iTSlfd8odHiOOlu0d5WSN7Te1uH81 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8565 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240610_173131_961332_8BBA5C8F X-CRM114-Status: GOOD ( 12.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The top of the 2 level stream table is (at most) 128k entries big, and two high order allocations are required. One of __le64 which is programmed into the HW (1M), and one of struct arm_smmu_strtab_l1_desc which holds the CPU pointer (3M). There is no reason to store the l2ptr_dma as nothing reads it. devm stores a copy of it and the DMA memory will be freed via devm mechanisms. span is a constant of 8+1. Remove both. This removes 16 bytes from each arm_smmu_l1_ctx_desc and saves up to 2M of memory per iommu instance. Tested-by: Nicolin Chen Reviewed-by: Mostafa Saleh Signed-off-by: Jason Gunthorpe Reviewed-by: Nicolin Chen --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 13 ++++++------- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 --- 2 files changed, 6 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d56e19ca6ef6c5..95351c134c7c45 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1447,13 +1447,12 @@ bool arm_smmu_free_asid(struct arm_smmu_ctx_desc *cd) } /* Stream table manipulation functions */ -static void -arm_smmu_write_strtab_l1_desc(__le64 *dst, struct arm_smmu_strtab_l1_desc *desc) +static void arm_smmu_write_strtab_l1_desc(__le64 *dst, dma_addr_t l2ptr_dma) { u64 val = 0; - val |= FIELD_PREP(STRTAB_L1_DESC_SPAN, desc->span); - val |= desc->l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK; + val |= FIELD_PREP(STRTAB_L1_DESC_SPAN, STRTAB_SPLIT + 1); + val |= l2ptr_dma & STRTAB_L1_DESC_L2PTR_MASK; /* The HW has 64 bit atomicity with stores to the L2 STE table */ WRITE_ONCE(*dst, cpu_to_le64(val)); @@ -1655,6 +1654,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) { size_t size; void *strtab; + dma_addr_t l2ptr_dma; struct arm_smmu_strtab_cfg *cfg = &smmu->strtab_cfg; struct arm_smmu_strtab_l1_desc *desc = &cfg->l1_desc[sid >> STRTAB_SPLIT]; @@ -1664,8 +1664,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) size = 1 << (STRTAB_SPLIT + ilog2(STRTAB_STE_DWORDS) + 3); strtab = &cfg->strtab[(sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS]; - desc->span = STRTAB_SPLIT + 1; - desc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &desc->l2ptr_dma, + desc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &l2ptr_dma, GFP_KERNEL); if (!desc->l2ptr) { dev_err(smmu->dev, @@ -1675,7 +1674,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) } arm_smmu_init_initial_stes(desc->l2ptr, 1 << STRTAB_SPLIT); - arm_smmu_write_strtab_l1_desc(strtab, desc); + arm_smmu_write_strtab_l1_desc(strtab, l2ptr_dma); return 0; } diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index 1242a086c9f948..087733797f9087 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -579,10 +579,7 @@ struct arm_smmu_priq { /* High-level stream table and context descriptor structures */ struct arm_smmu_strtab_l1_desc { - u8 span; - struct arm_smmu_ste *l2ptr; - dma_addr_t l2ptr_dma; }; struct arm_smmu_ctx_desc {