From patchwork Wed Feb 1 15:26:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 13124594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7197C636D3 for ; Wed, 1 Feb 2023 15:26:17 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B271010E414; Wed, 1 Feb 2023 15:26:15 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7F31810E40F for ; Wed, 1 Feb 2023 15:26:13 +0000 (UTC) Received: from dude02.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::28]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1pNEzy-0000ja-7B; Wed, 01 Feb 2023 16:26:10 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH v3 1/3] drm/scheduler: track GPU active time per entity Date: Wed, 1 Feb 2023 16:26:07 +0100 Message-Id: <20230201152609.1395525-1-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::28 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@pengutronix.de, dri-devel@lists.freedesktop.org, patchwork-lst@pengutronix.de, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Track the accumulated time that jobs from this entity were active on the GPU. This allows drivers using the scheduler to trivially implement the DRM fdinfo when the hardware doesn't provide more specific information than signalling job completion anyways. Signed-off-by: Lucas Stach Reviewed-by: Andrey Grodzovsky --- drivers/gpu/drm/scheduler/sched_main.c | 6 ++++++ include/drm/gpu_scheduler.h | 7 +++++++ 2 files changed, 13 insertions(+) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 31f3a1267be4..cc5e77a30a66 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -895,6 +895,12 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) spin_unlock(&sched->job_list_lock); + if (job) { + job->entity->elapsed_ns += ktime_to_ns( + ktime_sub(job->s_fence->finished.timestamp, + job->s_fence->scheduled.timestamp)); + } + return job; } diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index ca857ec9e7eb..f88a3fc14fb1 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -228,6 +228,13 @@ struct drm_sched_entity { */ struct rb_node rb_tree_node; + /** + * @elapsed_ns + * + * Records the amount of time where jobs from this entity were active + * on the GPU. + */ + uint64_t elapsed_ns; }; /** From patchwork Wed Feb 1 15:26:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 13124593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AF9ECC636CD for ; Wed, 1 Feb 2023 15:26:15 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EDE8B10E411; Wed, 1 Feb 2023 15:26:14 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id D077F10E40F for ; Wed, 1 Feb 2023 15:26:13 +0000 (UTC) Received: from dude02.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::28]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1pNEzy-0000ja-If; Wed, 01 Feb 2023 16:26:10 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH v3 2/3] drm/etnaviv: allocate unique ID per drm_file Date: Wed, 1 Feb 2023 16:26:08 +0100 Message-Id: <20230201152609.1395525-2-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230201152609.1395525-1-l.stach@pengutronix.de> References: <20230201152609.1395525-1-l.stach@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::28 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@pengutronix.de, dri-devel@lists.freedesktop.org, patchwork-lst@pengutronix.de, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Allows to easily track if several fd are pointing to the same execution context due to being dup'ed. Signed-off-by: Lucas Stach Reviewed-by: Philipp Zabel --- v3: use xarray to track the active contexts to avoid issues on rollover --- drivers/gpu/drm/etnaviv/etnaviv_drv.c | 11 +++++++++++ drivers/gpu/drm/etnaviv/etnaviv_drv.h | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c index 1d2b4fb4bcf8..31a7f59ccb49 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c @@ -56,6 +56,11 @@ static int etnaviv_open(struct drm_device *dev, struct drm_file *file) if (!ctx) return -ENOMEM; + ret = xa_alloc_cyclic(&priv->active_contexts, &ctx->id, ctx, + xa_limit_32b, &priv->next_context_id, GFP_KERNEL); + if (ret < 0) + goto out_free; + ctx->mmu = etnaviv_iommu_context_init(priv->mmu_global, priv->cmdbuf_suballoc); if (!ctx->mmu) { @@ -99,6 +104,8 @@ static void etnaviv_postclose(struct drm_device *dev, struct drm_file *file) etnaviv_iommu_context_put(ctx->mmu); + xa_erase(&priv->active_contexts, ctx->id); + kfree(ctx); } @@ -514,6 +521,8 @@ static int etnaviv_bind(struct device *dev) dma_set_max_seg_size(dev, SZ_2G); + xa_init_flags(&priv->active_contexts, XA_FLAGS_ALLOC); + mutex_init(&priv->gem_lock); INIT_LIST_HEAD(&priv->gem_list); priv->num_gpus = 0; @@ -563,6 +572,8 @@ static void etnaviv_unbind(struct device *dev) etnaviv_cmdbuf_suballoc_destroy(priv->cmdbuf_suballoc); + xa_destroy(&priv->active_contexts); + drm->dev_private = NULL; kfree(priv); diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.h b/drivers/gpu/drm/etnaviv/etnaviv_drv.h index 0b311af04f1d..b3eb1662e90c 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.h +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.h @@ -29,6 +29,7 @@ struct etnaviv_iommu_global; #define ETNAVIV_SOFTPIN_START_ADDRESS SZ_4M /* must be >= SUBALLOC_SIZE */ struct etnaviv_file_private { + int id; struct etnaviv_iommu_context *mmu; struct drm_sched_entity sched_entity[ETNA_MAX_PIPES]; }; @@ -41,6 +42,9 @@ struct etnaviv_drm_private { struct etnaviv_cmdbuf_suballoc *cmdbuf_suballoc; struct etnaviv_iommu_global *mmu_global; + struct xarray active_contexts; + u32 next_context_id; + /* list of GEM objects: */ struct mutex gem_lock; struct list_head gem_list; From patchwork Wed Feb 1 15:26:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas Stach X-Patchwork-Id: 13124595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A393DC636CD for ; Wed, 1 Feb 2023 15:26:20 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 13CA010E415; Wed, 1 Feb 2023 15:26:16 +0000 (UTC) Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0F86310E40F for ; Wed, 1 Feb 2023 15:26:14 +0000 (UTC) Received: from dude02.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::28]) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1pNEzy-0000ja-U4; Wed, 01 Feb 2023 16:26:10 +0100 From: Lucas Stach To: etnaviv@lists.freedesktop.org Subject: [PATCH v3 3/3] drm/etnaviv: export client GPU usage statistics via fdinfo Date: Wed, 1 Feb 2023 16:26:09 +0100 Message-Id: <20230201152609.1395525-3-l.stach@pengutronix.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230201152609.1395525-1-l.stach@pengutronix.de> References: <20230201152609.1395525-1-l.stach@pengutronix.de> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 2a0a:edc0:0:1101:1d::28 X-SA-Exim-Mail-From: l.stach@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@pengutronix.de, dri-devel@lists.freedesktop.org, patchwork-lst@pengutronix.de, Russell King Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This exposes a accumulated GPU active time per client via the fdinfo infrastructure. Signed-off-by: Lucas Stach Reviewed-by: Philipp Zabel --- v3: handle NPU cores --- drivers/gpu/drm/etnaviv/etnaviv_drv.c | 43 ++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c index 31a7f59ccb49..44ca803237a5 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c @@ -22,6 +22,7 @@ #include "etnaviv_gem.h" #include "etnaviv_mmu.h" #include "etnaviv_perfmon.h" +#include "common.xml.h" /* * DRM operations: @@ -475,7 +476,47 @@ static const struct drm_ioctl_desc etnaviv_ioctls[] = { ETNA_IOCTL(PM_QUERY_SIG, pm_query_sig, DRM_RENDER_ALLOW), }; -DEFINE_DRM_GEM_FOPS(fops); +static void etnaviv_fop_show_fdinfo(struct seq_file *m, struct file *f) +{ + struct drm_file *file = f->private_data; + struct drm_device *dev = file->minor->dev; + struct etnaviv_drm_private *priv = dev->dev_private; + struct etnaviv_file_private *ctx = file->driver_priv; + + /* + * For a description of the text output format used here, see + * Documentation/gpu/drm-usage-stats.rst. + */ + seq_printf(m, "drm-driver:\t%s\n", dev->driver->name); + seq_printf(m, "drm-client-id:\t%u\n", ctx->id); + + for (int i = 0; i < ETNA_MAX_PIPES; i++) { + struct etnaviv_gpu *gpu = priv->gpu[i]; + char engine[10] = "UNK"; + int cur = 0; + + if (!gpu) + continue; + + if (gpu->identity.features & chipFeatures_PIPE_2D) + cur = snprintf(engine, sizeof(engine), "2D"); + if (gpu->identity.features & chipFeatures_PIPE_3D) + cur = snprintf(engine + cur, sizeof(engine) - cur, + "%s3D", cur ? "/" : ""); + if (gpu->identity.nn_core_count > 0) + cur = snprintf(engine + cur, sizeof(engine) - cur, + "%sNN", cur ? "/" : ""); + + seq_printf(m, "drm-engine-%s:\t%llu ns\n", engine, + ctx->sched_entity[i].elapsed_ns); + } +} + +static const struct file_operations fops = { + .owner = THIS_MODULE, + DRM_GEM_FOPS, + .show_fdinfo = etnaviv_fop_show_fdinfo, +}; static const struct drm_driver etnaviv_drm_driver = { .driver_features = DRIVER_GEM | DRIVER_RENDER,