From patchwork Wed Mar 23 16:08:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 12789884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 656B9C433F5 for ; Wed, 23 Mar 2022 16:08:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C998A10E759; Wed, 23 Mar 2022 16:08:34 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id 407C010E260 for ; Wed, 23 Mar 2022 16:08:29 +0000 (UTC) Received: from dude03.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::39]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1nX3X8-0001yD-0N; Wed, 23 Mar 2022 17:08:26 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH 1/4] drm/etnaviv: check for reaped mapping in etnaviv_iommu_unmap_gem Date: Wed, 23 Mar 2022 17:08:22 +0100 Message-Id: <20220323160825.3858619-1-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: patchwork-lst@pengutronix.de, kernel@pengutronix.de, dri-devel@lists.freedesktop.org, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" When the mapping is already reaped the unmap must be a no-op, as we would otherwise try to remove the mapping twice, corrupting the involved data structures. Cc: stable@vger.kernel.org # 5.4 Signed-off-by: Lucas Stach Reviewed-by: Philipp Zabel Tested-by: Guido Günther Acked-by: Guido Günther --- drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index 9fb1a2aadbcb..aabb997a74eb 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -286,6 +286,12 @@ void etnaviv_iommu_unmap_gem(struct etnaviv_iommu_context *context, mutex_lock(&context->lock); + /* Bail if the mapping has been reaped by another thread */ + if (!mapping->context) { + mutex_unlock(&context->lock); + return; + } + /* If the vram node is on the mm, unmap and remove the node */ if (mapping->vram_node.mm == &context->mm) etnaviv_iommu_remove_mapping(context, mapping); From patchwork Wed Mar 23 16:08:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 12789885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E121C433F5 for ; Wed, 23 Mar 2022 16:08:40 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 76A4810E75D; Wed, 23 Mar 2022 16:08:35 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id 383DF10E749 for ; Wed, 23 Mar 2022 16:08:30 +0000 (UTC) Received: from dude03.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::39]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1nX3X8-0001yD-FL; Wed, 23 Mar 2022 17:08:26 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH 2/4] drm/etnaviv: move MMU context ref/unref into map/unmap_gem Date: Wed, 23 Mar 2022 17:08:23 +0100 Message-Id: <20220323160825.3858619-2-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220323160825.3858619-1-l.stach@pengutronix.de> References: <20220323160825.3858619-1-l.stach@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: patchwork-lst@pengutronix.de, kernel@pengutronix.de, dri-devel@lists.freedesktop.org, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This makes it a little more clear that the mapping holds a reference to the context once the buffer has been successfully mapped into that context and simplifies the error handling a bit. Signed-off-by: Lucas Stach Reviewed-by: Philipp Zabel --- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 11 +++-------- drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 3 +++ 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c index d5314aa28ff7..a68e6a17505e 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c @@ -294,18 +294,15 @@ struct etnaviv_vram_mapping *etnaviv_gem_mapping_get( list_del(&mapping->obj_node); } - mapping->context = etnaviv_iommu_context_get(mmu_context); mapping->use = 1; ret = etnaviv_iommu_map_gem(mmu_context, etnaviv_obj, mmu_context->global->memory_base, mapping, va); - if (ret < 0) { - etnaviv_iommu_context_put(mmu_context); + if (ret < 0) kfree(mapping); - } else { + else list_add_tail(&mapping->obj_node, &etnaviv_obj->vram_list); - } out: mutex_unlock(&etnaviv_obj->lock); @@ -498,10 +495,8 @@ void etnaviv_gem_free_object(struct drm_gem_object *obj) WARN_ON(mapping->use); - if (context) { + if (context) etnaviv_iommu_unmap_gem(context, mapping); - etnaviv_iommu_context_put(context); - } list_del(&mapping->obj_node); kfree(mapping); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index aabb997a74eb..3957b9a752f5 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -245,6 +245,7 @@ int etnaviv_iommu_map_gem(struct etnaviv_iommu_context *context, iova = sg_dma_address(sgt->sgl) - memory_base; if (iova < 0x80000000 - sg_dma_len(sgt->sgl)) { mapping->iova = iova; + mapping->context = etnaviv_iommu_context_get(context); list_add_tail(&mapping->mmu_node, &context->mappings); ret = 0; goto unlock; @@ -271,6 +272,7 @@ int etnaviv_iommu_map_gem(struct etnaviv_iommu_context *context, goto unlock; } + mapping->context = etnaviv_iommu_context_get(context); list_add_tail(&mapping->mmu_node, &context->mappings); context->flush_seq++; unlock: @@ -299,6 +301,7 @@ void etnaviv_iommu_unmap_gem(struct etnaviv_iommu_context *context, list_del(&mapping->mmu_node); context->flush_seq++; mutex_unlock(&context->lock); + etnaviv_iommu_context_put(context); } static void etnaviv_iommu_context_free(struct kref *kref) From patchwork Wed Mar 23 16:08:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 12789882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D8E3C433F5 for ; Wed, 23 Mar 2022 16:08:31 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 56B7410E74C; Wed, 23 Mar 2022 16:08:30 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id BB14310E749 for ; Wed, 23 Mar 2022 16:08:29 +0000 (UTC) Received: from dude03.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::39]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1nX3X8-0001yD-U9; Wed, 23 Mar 2022 17:08:27 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH 3/4] drm/etnaviv: move flush_seq increment into etnaviv_iommu_map/unmap Date: Wed, 23 Mar 2022 17:08:24 +0100 Message-Id: <20220323160825.3858619-3-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220323160825.3858619-1-l.stach@pengutronix.de> References: <20220323160825.3858619-1-l.stach@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: patchwork-lst@pengutronix.de, kernel@pengutronix.de, dri-devel@lists.freedesktop.org, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The flush sequence is a marker that the page tables have been changed and any affected TLBs need to be flushed. Move the flush_seq increment a little further down the call stack to place it next to the actual page table manipulation. Not functional change. Signed-off-by: Lucas Stach Signed-off-by: Lucas Stach Reviewed-by: Philipp Zabel --- drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index 3957b9a752f5..d41295208102 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -92,6 +92,8 @@ static int etnaviv_iommu_map(struct etnaviv_iommu_context *context, u32 iova, da += bytes; } + context->flush_seq++; + return 0; fail: @@ -117,6 +119,8 @@ static void etnaviv_iommu_unmap(struct etnaviv_iommu_context *context, u32 iova, da += bytes; } + + context->flush_seq++; } static void etnaviv_iommu_remove_mapping(struct etnaviv_iommu_context *context, @@ -274,7 +278,6 @@ int etnaviv_iommu_map_gem(struct etnaviv_iommu_context *context, mapping->context = etnaviv_iommu_context_get(context); list_add_tail(&mapping->mmu_node, &context->mappings); - context->flush_seq++; unlock: mutex_unlock(&context->lock); @@ -299,7 +302,6 @@ void etnaviv_iommu_unmap_gem(struct etnaviv_iommu_context *context, etnaviv_iommu_remove_mapping(context, mapping); list_del(&mapping->mmu_node); - context->flush_seq++; mutex_unlock(&context->lock); etnaviv_iommu_context_put(context); } From patchwork Wed Mar 23 16:08:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 12789883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19320C433EF for ; Wed, 23 Mar 2022 16:08:36 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D5E1810E75B; Wed, 23 Mar 2022 16:08:34 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id 31FBF10E75B for ; Wed, 23 Mar 2022 16:08:31 +0000 (UTC) Received: from dude03.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::39]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1nX3X9-0001yD-Cn; Wed, 23 Mar 2022 17:08:27 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH 4/4] drm/etnaviv: reap idle softpin mappings when necessary Date: Wed, 23 Mar 2022 17:08:25 +0100 Message-Id: <20220323160825.3858619-4-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220323160825.3858619-1-l.stach@pengutronix.de> References: <20220323160825.3858619-1-l.stach@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::39 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: patchwork-lst@pengutronix.de, kernel@pengutronix.de, dri-devel@lists.freedesktop.org, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Right now the only point where softpin mappings get removed from the MMU context is when the mapped GEM object is destroyed. However, userspace might want to reuse that address space before the object is destroyed, which is a valid usage, as long as all mapping in that region of the address space are no longer used by any GPU jobs. Implement reaping of idle MMU mappings that would otherwise prevent the insertion of a softpin mapping. Signed-off-by: Lucas Stach Tested-by: Guido Günther Acked-by: Guido Günther Reviewed-by: Christian Gmeiner --- drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 39 +++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c index d41295208102..dc1aa738c4f1 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c @@ -223,8 +223,47 @@ static int etnaviv_iommu_find_iova(struct etnaviv_iommu_context *context, static int etnaviv_iommu_insert_exact(struct etnaviv_iommu_context *context, struct drm_mm_node *node, size_t size, u64 va) { + struct etnaviv_vram_mapping *m, *n; + struct drm_mm_node *scan_node; + LIST_HEAD(scan_list); + int ret; + lockdep_assert_held(&context->lock); + ret = drm_mm_insert_node_in_range(&context->mm, node, size, 0, 0, va, + va + size, DRM_MM_INSERT_LOWEST); + if (ret != -ENOSPC) + return ret; + + /* + * When we can't insert the node, due to a existing mapping blocking + * the address space, there are two possible reasons: + * 1. Userspace genuinely messed up and tried to reuse address space + * before the last job using this VMA has finished executing. + * 2. The existing buffer mappings are idle, but the buffers are not + * destroyed yet (likely due to being referenced by another context) in + * which case the mappings will not be cleaned up and we must reap them + * here to make space for the new mapping. + */ + + drm_mm_for_each_node_in_range(scan_node, &context->mm, va, va + size) { + m = container_of(scan_node, struct etnaviv_vram_mapping, + vram_node); + + if (m->use) + return -ENOSPC; + + list_add(&m->scan_node, &scan_list); + } + + list_for_each_entry_safe(m, n, &scan_list, scan_node) { + etnaviv_iommu_remove_mapping(context, m); + etnaviv_iommu_context_put(m->context); + m->context = NULL; + list_del_init(&m->mmu_node); + list_del_init(&m->scan_node); + } + return drm_mm_insert_node_in_range(&context->mm, node, size, 0, 0, va, va + size, DRM_MM_INSERT_LOWEST); }