From patchwork Sun Jan 26 20:11:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dmitry Osipenko X-Patchwork-Id: 13950871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C5F5C0218F for ; Sun, 26 Jan 2025 20:13:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tc90U-0001RZ-22; Sun, 26 Jan 2025 15:13:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tc90S-0001Qw-2N for qemu-devel@nongnu.org; Sun, 26 Jan 2025 15:13:20 -0500 Received: from sender4-pp-f112.zoho.com ([136.143.188.112]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tc90Q-0005wK-3f for qemu-devel@nongnu.org; Sun, 26 Jan 2025 15:13:19 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1737922383; cv=none; d=zohomail.com; s=zohoarc; b=fyHo0Jy9wbCmBl5d5+/6u41wTwSJVsY+B2lS9VO0X532FPezTInKkxF6sgfiZRueqxbTlF5cUvGGvQRNN8OFocyQr7TzSVzztJrxJBxlGRmO9N7l+ed8nLNE3QWxU+VZAmXZdbHla5rt/sM5WspW+hMlz9djpyDwkCUKUkGE99s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1737922383; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=nCPpmOfiN8ibsN75k6SFRxFac79Fjndz6GC2LH7rXds=; b=QGlgrUzxooJcK4HSv1VgFB6Th/ZRqi4cLsNVlDp3g307hyHobNiFHwX7JEIkvaTc9SMZkPb8iSvszQ+C0TU+U+zh/5S7N8FbBujYru6ZBfj7TG2fjb/IXtc+HHtPPhH3dt8YMY2HlBDuwBXytIupjCh/oAclZhIVqQ4KJv2zSNE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=collabora.com; spf=pass smtp.mailfrom=dmitry.osipenko@collabora.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1737922383; s=zohomail; d=collabora.com; i=dmitry.osipenko@collabora.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:In-Reply-To:References:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=nCPpmOfiN8ibsN75k6SFRxFac79Fjndz6GC2LH7rXds=; b=UtimWG4ptzkhKNgZXtvuJIsY+lWWCWc2cFWUCW2QgeOVthbNoydjwHslI/ipByc1 h0fW2jwTm8EaeRWjqatEw0t4IdoR35gnHy8kP0L7NfdXa72OE7mT+zhB8ZcyPvotyMO eoHGWldkINGsmDa0vapYd812pu1kSB9xLdBeWq4o= Received: by mx.zohomail.com with SMTPS id 1737922379064580.2851453623653; Sun, 26 Jan 2025 12:12:59 -0800 (PST) From: Dmitry Osipenko To: Akihiko Odaki , Huang Rui , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Gerd Hoffmann , =?utf-8?q?Alex_Benn=C3=A9e?= , "Michael S . Tsirkin" , Paolo Bonzini Cc: Gert Wollny , qemu-devel@nongnu.org, Gurchetan Singh , Alyssa Ross , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Alex Deucher , Stefano Stabellini , =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= , Xenia Ragiadakou , Pierre-Eric Pelloux-Prayer , Honglei Huang , Julia Zhang , Chen Jiqian , Rob Clark , Yiwei Zhang , Sergio Lopez Pascual Subject: [PATCH v6 04/10] virtio-gpu: Support asynchronous fencing Date: Sun, 26 Jan 2025 23:11:15 +0300 Message-ID: <20250126201121.470990-5-dmitry.osipenko@collabora.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250126201121.470990-1-dmitry.osipenko@collabora.com> References: <20250126201121.470990-1-dmitry.osipenko@collabora.com> MIME-Version: 1.0 X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.112; envelope-from=dmitry.osipenko@collabora.com; helo=sender4-pp-f112.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.01, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Support asynchronous fencing feature of virglrenderer. It allows Qemu to handle fence as soon as it's signalled instead of periodically polling the fence status. This feature is required for enabling DRM context support in Qemu because legacy fencing mode isn't supported for DRM contexts in virglrenderer. Reviewed-by: Akihiko Odaki Acked-by: Michael S. Tsirkin Tested-by: Alex Bennée Signed-off-by: Dmitry Osipenko Reviewed-by: Alex Bennée --- hw/display/virtio-gpu-gl.c | 3 + hw/display/virtio-gpu-virgl.c | 147 +++++++++++++++++++++++++++++++-- include/hw/virtio/virtio-gpu.h | 13 +++ 3 files changed, 154 insertions(+), 9 deletions(-) diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c index 683fad3bf8a8..d9bb50ac1d4a 100644 --- a/hw/display/virtio-gpu-gl.c +++ b/hw/display/virtio-gpu-gl.c @@ -169,6 +169,9 @@ static void virtio_gpu_gl_device_unrealize(DeviceState *qdev) if (gl->renderer_state >= RS_INITED) { #if VIRGL_VERSION_MAJOR >= 1 qemu_bh_delete(gl->cmdq_resume_bh); + + virtio_gpu_virgl_reset_async_fences(g); + qemu_bh_delete(gl->async_fence_bh); #endif if (virtio_gpu_stats_enabled(g->parent_obj.conf)) { timer_free(gl->print_stats); diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c index 2eb6aaab4e84..ee896eced67c 100644 --- a/hw/display/virtio-gpu-virgl.c +++ b/hw/display/virtio-gpu-virgl.c @@ -871,6 +871,7 @@ static void virgl_cmd_set_scanout_blob(VirtIOGPU *g, void virtio_gpu_virgl_process_cmd(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd) { + VirtIOGPUGL *gl = VIRTIO_GPU_GL(g); bool cmd_suspended = false; int ret; @@ -972,15 +973,34 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g, trace_virtio_gpu_fence_ctrl(cmd->cmd_hdr.fence_id, cmd->cmd_hdr.type); - /* - * Unlike other virglrenderer functions, this one returns a positive - * error code. - */ - ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0); - if (ret) { - qemu_log_mask(LOG_GUEST_ERROR, - "%s: virgl_renderer_create_fence error: %s", - __func__, strerror(ret)); + if (gl->context_fence_enabled && + (cmd->cmd_hdr.flags & VIRTIO_GPU_FLAG_INFO_RING_IDX)) { +#if VIRGL_VERSION_MAJOR >= 1 + uint32_t flags = 0; + + ret = virgl_renderer_context_create_fence(cmd->cmd_hdr.ctx_id, flags, + cmd->cmd_hdr.ring_idx, + cmd->cmd_hdr.fence_id); + if (ret) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: virgl_renderer_context_create_fence error: %s", + __func__, strerror(-ret)); + } +#else + /* gl->context_fence_enabled cannot be set with older virglrenderer */ + g_assert_not_reached(); +#endif + } else { + /* + * Unlike other virglrenderer functions, this one returns a positive + * error code. + */ + ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0); + if (ret) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: virgl_renderer_create_fence error: %s", + __func__, strerror(ret)); + } } } @@ -1008,6 +1028,102 @@ static void virgl_write_fence(void *opaque, uint32_t fence) } } +void virtio_gpu_virgl_reset_async_fences(VirtIOGPU *g) +{ + struct virtio_gpu_virgl_context_fence *f; + VirtIOGPUGL *gl = VIRTIO_GPU_GL(g); + + while (!QSLIST_EMPTY(&gl->async_fenceq)) { + f = QSLIST_FIRST(&gl->async_fenceq); + + QSLIST_REMOVE_HEAD(&gl->async_fenceq, next); + + g_free(f); + } +} + +#if VIRGL_VERSION_MAJOR >= 1 +static void virtio_gpu_virgl_async_fence_bh(void *opaque) +{ + QSLIST_HEAD(, virtio_gpu_virgl_context_fence) async_fenceq; + struct virtio_gpu_ctrl_command *cmd, *tmp; + struct virtio_gpu_virgl_context_fence *f; + VirtIOGPU *g = opaque; + VirtIOGPUGL *gl = VIRTIO_GPU_GL(g); + + QSLIST_MOVE_ATOMIC(&async_fenceq, &gl->async_fenceq); + + while (!QSLIST_EMPTY(&async_fenceq)) { + f = QSLIST_FIRST(&async_fenceq); + + QSLIST_REMOVE_HEAD(&async_fenceq, next); + + QTAILQ_FOREACH_SAFE(cmd, &g->fenceq, next, tmp) { + /* + * the guest can end up emitting fences out of order + * so we should check all fenced cmds not just the first one. + */ + if (cmd->cmd_hdr.fence_id > f->fence_id) { + continue; + } + if (cmd->cmd_hdr.flags & VIRTIO_GPU_FLAG_INFO_RING_IDX) { + if (cmd->cmd_hdr.ring_idx != f->ring_idx) { + continue; + } + if (cmd->cmd_hdr.ctx_id != f->ctx_id) { + continue; + } + } else if (f->ring_idx >= 0) { + /* ctx0 GL-query fences don't have ring info */ + continue; + } + virtio_gpu_ctrl_response_nodata(g, cmd, VIRTIO_GPU_RESP_OK_NODATA); + QTAILQ_REMOVE(&g->fenceq, cmd, next); + g_free(cmd); + } + + trace_virtio_gpu_fence_resp(f->fence_id); + g_free(f); + g->inflight--; + if (virtio_gpu_stats_enabled(g->parent_obj.conf)) { + trace_virtio_gpu_dec_inflight_fences(g->inflight); + } + } +} + +static void +virtio_gpu_virgl_push_async_fence(VirtIOGPU *g, uint32_t ctx_id, + int64_t ring_idx, uint64_t fence_id) +{ + struct virtio_gpu_virgl_context_fence *f; + VirtIOGPUGL *gl = VIRTIO_GPU_GL(g); + + f = g_new(struct virtio_gpu_virgl_context_fence, 1); + f->ctx_id = ctx_id; + f->ring_idx = ring_idx; + f->fence_id = fence_id; + + QSLIST_INSERT_HEAD_ATOMIC(&gl->async_fenceq, f, next); + + qemu_bh_schedule(gl->async_fence_bh); +} + +static void virgl_write_async_fence(void *opaque, uint32_t fence) +{ + VirtIOGPU *g = opaque; + + virtio_gpu_virgl_push_async_fence(g, 0, -1, fence); +} + +static void virgl_write_async_context_fence(void *opaque, uint32_t ctx_id, + uint32_t ring_idx, uint64_t fence) +{ + VirtIOGPU *g = opaque; + + virtio_gpu_virgl_push_async_fence(g, ctx_id, ring_idx, fence); +} +#endif + static virgl_renderer_gl_context virgl_create_context(void *opaque, int scanout_idx, struct virgl_renderer_gl_ctx_param *params) @@ -1095,6 +1211,8 @@ void virtio_gpu_virgl_reset_scanout(VirtIOGPU *g) dpy_gfx_replace_surface(g->parent_obj.scanout[i].con, NULL); dpy_gl_scanout_disable(g->parent_obj.scanout[i].con); } + + virtio_gpu_virgl_reset_async_fences(g); } void virtio_gpu_virgl_reset(VirtIOGPU *g) @@ -1112,6 +1230,13 @@ int virtio_gpu_virgl_init(VirtIOGPU *g) if (qemu_egl_display) { virtio_gpu_3d_cbs.version = 4; virtio_gpu_3d_cbs.get_egl_display = virgl_get_egl_display; +#if VIRGL_VERSION_MAJOR >= 1 + virtio_gpu_3d_cbs.write_fence = virgl_write_async_fence; + virtio_gpu_3d_cbs.write_context_fence = virgl_write_async_context_fence; + flags |= VIRGL_RENDERER_ASYNC_FENCE_CB; + flags |= VIRGL_RENDERER_THREAD_SYNC; + gl->context_fence_enabled = true; +#endif } #endif #ifdef VIRGL_RENDERER_D3D11_SHARE_TEXTURE @@ -1145,6 +1270,10 @@ int virtio_gpu_virgl_init(VirtIOGPU *g) gl->cmdq_resume_bh = aio_bh_new(qemu_get_aio_context(), virtio_gpu_virgl_resume_cmdq_bh, g); + + gl->async_fence_bh = aio_bh_new(qemu_get_aio_context(), + virtio_gpu_virgl_async_fence_bh, + g); #endif return 0; diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h index a42957c4e2cc..bd2cccdc60d7 100644 --- a/include/hw/virtio/virtio-gpu.h +++ b/include/hw/virtio/virtio-gpu.h @@ -230,6 +230,13 @@ struct VirtIOGPUClass { Error **errp); }; +struct virtio_gpu_virgl_context_fence { + uint32_t ctx_id; + int64_t ring_idx; + uint64_t fence_id; + QSLIST_ENTRY(virtio_gpu_virgl_context_fence) next; +}; + /* VirtIOGPUGL renderer states */ typedef enum { RS_START, /* starting state */ @@ -247,6 +254,11 @@ struct VirtIOGPUGL { QEMUTimer *print_stats; QEMUBH *cmdq_resume_bh; + + QEMUBH *async_fence_bh; + QSLIST_HEAD(, virtio_gpu_virgl_context_fence) async_fenceq; + + bool context_fence_enabled; }; struct VhostUserGPU { @@ -376,5 +388,6 @@ void virtio_gpu_virgl_reset_scanout(VirtIOGPU *g); void virtio_gpu_virgl_reset(VirtIOGPU *g); int virtio_gpu_virgl_init(VirtIOGPU *g); GArray *virtio_gpu_virgl_get_capsets(VirtIOGPU *g); +void virtio_gpu_virgl_reset_async_fences(VirtIOGPU *g); #endif