From patchwork Thu Feb 27 11:03:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67D88C282D0 for ; Fri, 28 Feb 2025 11:42:34 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 783E310EC6E; Fri, 28 Feb 2025 11:42:28 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=rz.uni-freiburg.de header.i=@rz.uni-freiburg.de header.b="GftgFe1I"; dkim-atps=neutral Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 60CD710EAA1 for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654231; x=1742468632; bh=phxh/JgzX4NWg7lRuEb+hes0uG8A2Nrn YcWnGVlVQMQ=; b=GftgFe1IOePSB28Wu/9WJfYQT5kD2ShxqaSj8sxBcnElDrEV lvTPjTRKZYgl/yHRBOBOdfwlfePlV1zwYB8Mxa1H5BQydH0RbHtPrBobe/yimzLc yI3gdJtyWey1++3ugc2ub7U40N5dNQKRVVhAQcZcBK5g/nO8WMZrkpHb/GM= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 2DE542C012D for ; Thu, 27 Feb 2025 12:03:51 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276272; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy , Jin Chung Teng , Hazwan Arif Mazlan Subject: [PATCH 01/10] gstreamer-encoder: Use a h/w based encoder with Intel GPUs if possible (v5) Date: Thu, 27 Feb 2025 12:03:23 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy Once it is determined that an Intel GPU is available/active (after looking into udev's database), we try to see if there is a h/w based encoder (element) available (in Gstreamer's registry cache) for the user selected video codec. In other words, if we find that the Intel Media SDK Gstreamer plugin (libgstmsdk.so) and associated libraries (such as va or vaapi) are all installed properly, we add the appropriate h/w based encoder and post-processor/converter elements to the pipeline (along with any relevant options) instead of the s/w based elements. For example, if the user selects h264 as the preferred codec format, msdkh264enc and vapostproc will be preferred instead of x264enc and videoconvert. v2: (addressed some review comments from Frediano) - Moved the udev helper into spice-common - Refactored the code to choose plugins in order msdk > va > vaapi v3: (Frediano) - Added relevant encoder options for mjpeg and vp9 codecs (Jin Chung) v4: (Fixups from Frediano) - Free the encoder when we cannot find vpp - Change type and find plugins array length using G_N_ELEMENTS - Fix gstenc_name UAF by freeing it at the end of the function - Use g_str_has_prefix instead of strstr - Include the string "_hw_" in function names that deal with h/w based plugins - Rebase on master v5: rebase on master Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy Co-developed-by: Jin Chung Teng Co-developed-by: Hazwan Arif Mazlan --- server/gstreamer-encoder.c | 123 ++++++++++++++++++++++++++++++++++++- 1 file changed, 122 insertions(+), 1 deletion(-) diff --git a/server/gstreamer-encoder.c b/server/gstreamer-encoder.c index 40882f69..be8e3111 100644 --- a/server/gstreamer-encoder.c +++ b/server/gstreamer-encoder.c @@ -38,6 +38,7 @@ #include "red-common.h" #include "video-encoder.h" #include "utils.h" +#include "common/udev.h" #define SPICE_GST_DEFAULT_FPS 30 @@ -861,6 +862,116 @@ static const gchar* get_gst_codec_name(const SpiceGstEncoder *encoder) } } +static const char video_codecs[][8] = { + { "" }, + { "mjpeg" }, + { "vp8" }, + { "h264" }, + { "vp9" }, + { "h265" }, +}; + +static bool gst_features_lookup(const gchar *feature_name) +{ + GstRegistry *registry; + GstPluginFeature *feature; + + registry = gst_registry_get(); + if (!registry) { + return false; + } + + feature = gst_registry_lookup_feature(registry, feature_name); + if (!feature) { + return false; + } + + gst_object_unref(feature); + return true; +} + +static gchar *find_best_hw_plugin(const gchar *codec_name) +{ + static const char plugins[][8] = {"msdk", "va", "vaapi"}; + gchar *feature_name; + int i; + + for (i = 0; i < G_N_ELEMENTS(plugins); i++) { + feature_name = !codec_name ? g_strconcat(plugins[i], "postproc", NULL) : + g_strconcat(plugins[i], codec_name, "enc", NULL); + if (!gst_features_lookup(feature_name)) { + g_free(feature_name); + feature_name = NULL; + continue; + } + break; + } + return feature_name; +} + +static gchar *get_hw_gstenc_opts(gchar *encoder, const gchar *codec_name) +{ + gchar *gstenc_opts; + + if (strcmp(codec_name, "mjpeg") == 0) { + return g_strdup(""); + } + + if (g_str_has_prefix(encoder, "msdk")) { + if (strcmp(codec_name, "vp9") == 0) { + gstenc_opts = g_strdup("async-depth=1 b-frames=0 rate-control=3 target-usage=7"); + } else { + gstenc_opts = g_strdup("async-depth=1 rate-control=3 gop-size=1 tune=16 b-frames=0 target-usage=7 min-qp=15 max-qp=35"); + } + } else if (g_str_has_prefix(encoder, "vaapi")) { + if (strcmp(codec_name, "vp9") == 0) { + gstenc_opts = g_strdup("tune=3 rate-control=1"); + } else { + gstenc_opts = g_strdup("rate-control=cqp max-bframes=0 min-qp=15 max-qp=35"); + } + } else { + if (strcmp(codec_name, "vp9") == 0) { + gstenc_opts = g_strdup("min-qp=15 max-qp=35 rate-control=16 ref-frames=0 target-usage=7"); + } else { + gstenc_opts = g_strdup("rate-control=16 b-frames=0 target-usage=7 min-qp=15 max-qp=35"); + } + } + return gstenc_opts; +} + +static void try_intel_hw_plugins(const gchar *codec_name, gchar **converter, + gchar **gstenc_name, gchar **gstenc_opts) +{ + gchar *encoder, *vpp; + + if (strcmp(codec_name, "vp8") == 0) { + return; + } + + encoder = find_best_hw_plugin(codec_name); + if (!encoder) { + return; + } + vpp = find_best_hw_plugin(NULL); + if (!vpp) { + g_free(encoder); + return; + } + + g_free(*converter); + g_free(*gstenc_name); + g_free(*gstenc_opts); + *gstenc_name = encoder; + *gstenc_opts = get_hw_gstenc_opts(encoder, codec_name); + + if (g_str_has_prefix(vpp, "vaapi")) { + *converter = g_strconcat(vpp, " ! video/x-raw(memory:VASurface),format=NV12", NULL); + } else { + *converter = g_strconcat(vpp, " ! video/x-raw(memory:VAMemory),format=NV12", NULL); + } + g_free(vpp); +} + /* At this time, only the following formats are supported by x264enc. */ static const char valid_formats[][10] = { { "Y444" }, @@ -900,7 +1011,7 @@ static gchar *get_gst_converter(void) static gboolean create_pipeline(SpiceGstEncoder *encoder) { - const gchar* gstenc_name = get_gst_codec_name(encoder); + gchar* gstenc_name = g_strdup(get_gst_codec_name(encoder)); if (!gstenc_name) { return FALSE; } @@ -947,10 +1058,18 @@ static gboolean create_pipeline(SpiceGstEncoder *encoder) default: /* gstreamer_encoder_new() should have rejected this codec type */ spice_warning("unsupported codec type %d", encoder->base.codec_type); + g_free(gstenc_name); g_free(converter); return FALSE; } + const char *codec_name = video_codecs[encoder->base.codec_type]; + GpuVendor vendor = spice_udev_detect_gpu(INTEL_VENDOR_ID); + if (vendor == VENDOR_GPU_DETECTED) { + try_intel_hw_plugins(codec_name, &converter, &gstenc_name, + &gstenc_opts); + } + GError *err = NULL; gchar *desc = g_strdup_printf("appsrc is-live=true format=time do-timestamp=true name=src !" " %s ! %s name=encoder %s ! appsink name=sink", @@ -967,6 +1086,7 @@ static gboolean create_pipeline(SpiceGstEncoder *encoder) gst_object_unref(encoder->pipeline); encoder->pipeline = NULL; } + g_free(gstenc_name); return FALSE; } encoder->appsrc = GST_APP_SRC(gst_bin_get_by_name(GST_BIN(encoder->pipeline), "src")); @@ -1003,6 +1123,7 @@ static gboolean create_pipeline(SpiceGstEncoder *encoder) SPICE_GST_VIDEO_PIPELINE_BITRATE | SPICE_GST_VIDEO_PIPELINE_CAPS); + g_free(gstenc_name); return TRUE; } From patchwork Thu Feb 27 11:03:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 192F4C282D0 for ; Fri, 28 Feb 2025 11:42:41 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 19F0C10EC78; Fri, 28 Feb 2025 11:42:33 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=rz.uni-freiburg.de header.i=@rz.uni-freiburg.de header.b="XrJpBqmC"; dkim-atps=neutral Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5FB8510EAA0 for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654232; x=1742468633; bh=nNe+wr4gCcvck0Cc7fjEsivdn53BdJTH YGszCMg7gls=; b=XrJpBqmCQCE96GxuZrGtmqfVvB7eTb043JYGjGCpOWIqQd4F DT7LKJBJi7EJXfvhiDOPahB/A3uWhmuBrnlutVOIaahFdv1IxZETvP0PnfMVQzcC 7Aa6QsZ9tFqU5/715OsTlucJgQcvvnUsuE0c8IVduUE5cOaLwKZ/YbtFsC8= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id E48902C0130 for ; Thu, 27 Feb 2025 12:03:51 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276637; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy Subject: [PATCH 02/10] dcc: Check to see if the client supports multiple codecs (v2) Date: Thu, 27 Feb 2025 12:03:24 +0100 Message-ID: <7596dfeff23c76c0309097e67e2207365cef680e.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy We need to determine if the client is new enough to support multiple codecs -- which might include any of the Gstreamer based ones. v2: (suggestions and fixups from Frediano) - Add is_gl_client() method to DisplayChannelClient instead of a dcc_is_gl_client() function. - Avoid the usage of XXX_CAST macro. Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy Acked-by: Frediano Ziglio --- server/dcc.cpp | 22 ++++++++++++---------- server/dcc.h | 6 ++++++ 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/server/dcc.cpp b/server/dcc.cpp index b04e5bc0..edec429d 100644 --- a/server/dcc.cpp +++ b/server/dcc.cpp @@ -494,29 +494,31 @@ RedSurfaceDestroyItem::RedSurfaceDestroyItem(uint32_t surface_id) RedPipeItemPtr dcc_gl_scanout_item_new(RedChannelClient *rcc, void *data, int num) { - /* FIXME: on !unix peer, start streaming with a video codec */ - if (!red_stream_is_plain_unix(rcc->get_stream()) || - !rcc->test_remote_cap(SPICE_DISPLAY_CAP_GL_SCANOUT)) { + auto dcc = static_cast(rcc); + + if (dcc->is_gl_client()) { + return red::make_shared(); + } else if (rcc->test_remote_cap(SPICE_DISPLAY_CAP_MULTI_CODEC)) { + return RedPipeItemPtr(); + } else { red_channel_warning(rcc->get_channel(), - "FIXME: client does not support GL scanout"); + "Client does not support GL scanout or multiple codecs"); rcc->disconnect(); return RedPipeItemPtr(); } - - return red::make_shared(); } XXX_CAST(RedChannelClient, DisplayChannelClient, DISPLAY_CHANNEL_CLIENT); RedPipeItemPtr dcc_gl_draw_item_new(RedChannelClient *rcc, void *data, int num) { - DisplayChannelClient *dcc = DISPLAY_CHANNEL_CLIENT(rcc); + auto dcc = static_cast(rcc); auto draw = static_cast(data); - if (!red_stream_is_plain_unix(rcc->get_stream()) || - !rcc->test_remote_cap(SPICE_DISPLAY_CAP_GL_SCANOUT)) { + if (!dcc->is_gl_client() && + !rcc->test_remote_cap(SPICE_DISPLAY_CAP_MULTI_CODEC)) { red_channel_warning(rcc->get_channel(), - "FIXME: client does not support GL scanout"); + "Client does not support GL scanout or multiple codecs"); rcc->disconnect(); return RedPipeItemPtr(); } diff --git a/server/dcc.h b/server/dcc.h index bb441905..4e7150d5 100644 --- a/server/dcc.h +++ b/server/dcc.h @@ -47,6 +47,12 @@ public: spice_wan_compression_t zlib_glz_state); virtual void disconnect() override; + bool is_gl_client() + { + return red_stream_is_plain_unix(get_stream()) && + test_remote_cap(SPICE_DISPLAY_CAP_GL_SCANOUT); + } + protected: virtual bool handle_message(uint16_t type, uint32_t size, void *msg) override; virtual bool config_socket() override; From patchwork Thu Feb 27 11:03:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7069CC282CD for ; Fri, 28 Feb 2025 11:42:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6EB5010EC65; Fri, 28 Feb 2025 11:42:27 +0000 (UTC) Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id BF53B10EA64 for ; Thu, 27 Feb 2025 11:13:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654232; x=1742468633; bh=6U6NZwdvUjXNW/ku7Nb0YKhGLqApZqKa vWhVNwpENzg=; b=oTS7UKeHqSRyhEfjFIUMzZndMY3oOmELhpyr8JCC8KYgyacg 942b+Y2aCJZpc7kUol5GRxJ1lzRV1kjn7k+HArfRwFYWOsWUDT+9mcTJqyjjb9zI lqmX3PfpkU4JkQw28QjnVkPMeKdHoY1aoAsqOlybsOMf4tTgop6AIYsz96k= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 7C1832C0133 for ; Thu, 27 Feb 2025 12:03:52 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46275184; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy Subject: [PATCH 03/10] dcc: Create a stream associated with gl_draw for non-gl clients (v6) Date: Thu, 27 Feb 2025 12:03:25 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy For non-gl/remote clients, if there is no stream associated with the DisplayChannel, then we create a new stream. Otherwise, we just update the current stream's timestamp. v2: (suggestions and fixups from Frediano) - Moved the gl_draw_stream object from DCC to DC - Moved the stream initialization code from display_channel_create_stream() into a separate function that is reused when creating gl_draw_stream v3: - Create a new primary surface whenever a new stream gets created v4: - Use nullptr instead of NULL and true instead of TRUE (Frediano) - Create the stream as part of gl scanout instead of gl draw operation so that if would be easily possible to obtain key params such as stride, flags, etc - Store key params such as fd, flags, stride, etc in the stream so that we do not have to look at scanout again v5: (Frediano) - No need to pass scanout object to create_gl_draw_stream as it is always NULL - Don't compute the stream's last_time if drawable is valid - Let the default input_fps be MAX_FPS - Use uint32_t type for stride - Make sure that a newly created stream is provided to all connected clients - When the scanout's drm_dma_buf_fd is associated with a stream, take an additional reference on the fd to ensure that it is not closed when the stream might still be using it v6: - Add a new helper is_new_stream_needed() to determine when to create new stream - Add more checks to see if the encoder is created or not - Don't share the dmabuf fd with the stream and instead keep the ownerhip of the fd with the scanout Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy --- server/dcc.cpp | 9 ++ server/display-channel-private.h | 1 + server/display-channel.cpp | 1 + server/video-stream.cpp | 194 ++++++++++++++++++++++++++----- server/video-stream.h | 4 + 5 files changed, 180 insertions(+), 29 deletions(-) diff --git a/server/dcc.cpp b/server/dcc.cpp index edec429d..58170681 100644 --- a/server/dcc.cpp +++ b/server/dcc.cpp @@ -523,6 +523,15 @@ RedPipeItemPtr dcc_gl_draw_item_new(RedChannelClient *rcc, void *data, int num) return RedPipeItemPtr(); } + if (!dcc->is_gl_client()) { + if (!display_channel_update_gl_draw_stream(dcc, draw)) { + red_channel_warning(rcc->get_channel(), + "Cannot update GL stream"); + rcc->disconnect(); + return RedPipeItemPtr(); + } + } + dcc->priv->gl_draw_ongoing = TRUE; auto item = red::make_shared(); item->draw = *draw; diff --git a/server/display-channel-private.h b/server/display-channel-private.h index 04ac2c0d..e0693f54 100644 --- a/server/display-channel-private.h +++ b/server/display-channel-private.h @@ -110,6 +110,7 @@ struct DisplayChannelPrivate uint32_t stream_count; std::array streams_buf; VideoStream *free_streams; + VideoStream *gl_draw_stream; Ring streams; std::array items_trace; uint32_t next_item_trace; diff --git a/server/display-channel.cpp b/server/display-channel.cpp index 4bd0cf41..53a4724d 100644 --- a/server/display-channel.cpp +++ b/server/display-channel.cpp @@ -2172,6 +2172,7 @@ display_channel_new(RedsState *reds, QXLInstance *qxl, video_codecs, n_surfaces); if (display) { display_channel_set_stream_video(display.get(), stream_video); + display->priv->gl_draw_stream = nullptr; } return display; } diff --git a/server/video-stream.cpp b/server/video-stream.cpp index e453f11d..691c50a3 100644 --- a/server/video-stream.cpp +++ b/server/video-stream.cpp @@ -20,6 +20,7 @@ #include "display-channel-private.h" #include "main-channel-client.h" #include "red-client.h" +#include "red-qxl.h" #define FPS_TEST_INTERVAL 1 #define FOREACH_STREAMS(display, item) \ @@ -366,7 +367,53 @@ static VideoStream *display_channel_stream_try_new(DisplayChannel *display) return stream; } -static void display_channel_create_stream(DisplayChannel *display, Drawable *drawable) +static void display_channel_init_stream(DisplayChannel *display, + VideoStream *stream, + Drawable *drawable, + SpiceRect* src_rect) +{ + SpiceBitmap *bitmap; + uint64_t duration; + + ring_add(&display->priv->streams, &stream->link); + stream->current = drawable; + stream->last_time = drawable ? drawable->creation_time : + spice_get_monotonic_time_ns(); + stream->width = src_rect->right - src_rect->left; + stream->height = src_rect->bottom - src_rect->top; + stream->dest_area = *src_rect; + stream->refs = 1; + stream->input_fps = MAX_FPS; + stream->num_input_frames = 0; + stream->input_fps_start_time = stream->last_time; + display->priv->streams_size_total += stream->width * stream->height; + display->priv->stream_count++; + + if (drawable) { + drawable->stream = stream; + stream->dest_area = drawable->red_drawable->bbox; + bitmap = &drawable->red_drawable->u.copy.src_bitmap->u.bitmap; + stream->top_down = !!(bitmap->flags & SPICE_BITMAP_FLAGS_TOP_DOWN); + + /* Provide an fps estimate the video encoder can use when initializing + * based on the frames that lead to the creation of the stream. Round + * to the nearest integer, for instance 24 for 23.976. + */ + duration = drawable->creation_time - drawable->first_frame_time; + if (duration > NSEC_PER_SEC * drawable->frames_count / MAX_FPS) { + stream->input_fps = (NSEC_PER_SEC * drawable->frames_count + duration / 2) / duration; + } + } + + spice_debug("stream %d %dx%d (%d, %d) (%d, %d) %u fps", + display_channel_get_video_stream_id(display, stream), stream->width, + stream->height, stream->dest_area.left, stream->dest_area.top, + stream->dest_area.right, stream->dest_area.bottom, + stream->input_fps); +} + +static void display_channel_create_stream(DisplayChannel *display, + Drawable *drawable) { DisplayChannelClient *dcc; VideoStream *stream; @@ -381,38 +428,127 @@ static void display_channel_create_stream(DisplayChannel *display, Drawable *dra spice_assert(drawable->red_drawable->type == QXL_DRAW_COPY); src_rect = &drawable->red_drawable->u.copy.src_area; - ring_add(&display->priv->streams, &stream->link); - stream->current = drawable; - stream->last_time = drawable->creation_time; - stream->width = src_rect->right - src_rect->left; - stream->height = src_rect->bottom - src_rect->top; - stream->dest_area = drawable->red_drawable->bbox; - stream->refs = 1; - SpiceBitmap *bitmap = &drawable->red_drawable->u.copy.src_bitmap->u.bitmap; - stream->top_down = !!(bitmap->flags & SPICE_BITMAP_FLAGS_TOP_DOWN); - drawable->stream = stream; - /* Provide an fps estimate the video encoder can use when initializing - * based on the frames that lead to the creation of the stream. Round to - * the nearest integer, for instance 24 for 23.976. - */ - uint64_t duration = drawable->creation_time - drawable->first_frame_time; - if (duration > NSEC_PER_SEC * drawable->frames_count / MAX_FPS) { - stream->input_fps = (NSEC_PER_SEC * drawable->frames_count + duration / 2) / duration; - } else { - stream->input_fps = MAX_FPS; + display_channel_init_stream(display, stream, drawable, src_rect); + FOREACH_DCC(display, dcc) { + dcc_create_stream(dcc, stream); } - stream->num_input_frames = 0; - stream->input_fps_start_time = drawable->creation_time; - display->priv->streams_size_total += stream->width * stream->height; - display->priv->stream_count++; +} + +bool display_channel_create_gl_draw_stream(DisplayChannel *display) +{ + QXLInstance* qxl = display->priv->qxl; + SpiceMsgDisplayGlScanoutUnix *scanout = red_qxl_get_gl_scanout(qxl); + DisplayChannelClient *dcc; + VideoStream *stream; + bool ret = false; + + if (!scanout) { + return false; + } + SpiceRect dest_area = { + .left = 0, + .top = 0, + .right = scanout->width, + .bottom = scanout->height + }; + + if (display->priv->surfaces[0]) { + display_channel_surface_id_unref(display, 0); + } + if (!display_channel_create_surface(display, 0, scanout->width, + scanout->height, scanout->stride, + SPICE_SURFACE_FMT_32_xRGB, + nullptr, true, true)) { + goto err; + } + + display_channel_set_monitors_config_to_primary(display); + display_channel_push_monitors_config(display); + display->pipes_add_empty_msg(SPICE_MSG_DISPLAY_MARK); + display->push(); + + if (!(stream = display_channel_stream_try_new(display))) { + goto err; + } + + ret = true; + display_channel_init_stream(display, stream, nullptr, &dest_area); + stream->top_down = (scanout->flags & SPICE_GL_SCANOUT_FLAGS_Y0TOP) ? 0 : 1; + stream->stride = scanout->stride; + /* This is the upper bound; it should be possible to stream at 60 FPS + * with a hardware based encoder. + */ + stream->input_fps = MAX_FPS * 2; + + display->priv->gl_draw_stream = stream; FOREACH_DCC(display, dcc) { dcc_create_stream(dcc, stream); } - spice_debug("stream %d %dx%d (%d, %d) (%d, %d) %u fps", - display_channel_get_video_stream_id(display, stream), stream->width, - stream->height, stream->dest_area.left, stream->dest_area.top, - stream->dest_area.right, stream->dest_area.bottom, - stream->input_fps); +err: + red_qxl_put_gl_scanout(qxl, scanout); + return ret; +} + +bool is_new_stream_needed(DisplayChannel *display) +{ + VideoStream *stream = display->priv->gl_draw_stream; + if (!stream) { + return true; + } + + QXLInstance* qxl = display->priv->qxl; + SpiceMsgDisplayGlScanoutUnix *scanout = red_qxl_get_gl_scanout(qxl); + uint32_t width, height, top_down; + + if (!scanout) { + return true; + } + + width = scanout->width; + height = scanout->height; + top_down = (scanout->flags & SPICE_GL_SCANOUT_FLAGS_Y0TOP) ? 0 : 1; + red_qxl_put_gl_scanout(qxl, scanout); + + if (stream->width == width && + stream->height == height && + stream->top_down == top_down) { + return false; + } + return true; +} + +bool display_channel_update_gl_draw_stream(DisplayChannelClient *dcc, + const SpiceMsgDisplayGlDraw *draw) +{ + DisplayChannel *display = DCC_TO_DC(dcc); + SpiceRect dest_area = { + .left = draw->x, + .top = draw->y, + .right = draw->w, + .bottom = draw->h + }; + + if (is_new_stream_needed(display)) { + if (!display_channel_create_gl_draw_stream(display)) { + return false; + } + } + + VideoStream *stream = display->priv->gl_draw_stream; + int stream_id = display_channel_get_video_stream_id(display, stream); + VideoStreamAgent *agent = dcc_get_video_stream_agent(dcc, stream_id); + + /* Just a sanity check to make sure that an encoder was created */ + if (!agent->video_encoder) { + dcc_create_stream(dcc, stream); + if (!agent->video_encoder) { + return false; + } + } + + stream->dest_area = dest_area; + stream->last_time = spice_get_monotonic_time_ns(); + return true; } // returns whether a stream was created diff --git a/server/video-stream.h b/server/video-stream.h index fc5b03dd..d3f25bb3 100644 --- a/server/video-stream.h +++ b/server/video-stream.h @@ -112,6 +112,7 @@ struct VideoStream { red_time_t last_time; int width; int height; + uint32_t stride; SpiceRect dest_area; int top_down; VideoStream *next; @@ -123,6 +124,9 @@ struct VideoStream { }; void display_channel_init_video_streams(DisplayChannel *display); +bool display_channel_create_gl_draw_stream(DisplayChannel *display); +bool display_channel_update_gl_draw_stream(DisplayChannelClient *dcc, + const SpiceMsgDisplayGlDraw *draw); void video_stream_stop(DisplayChannel *display, VideoStream *stream); void video_stream_trace_update(DisplayChannel *display, Drawable *drawable); void video_stream_maintenance(DisplayChannel *display, Drawable *candidate, From patchwork Thu Feb 27 11:03:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A1FAC282CD for ; Fri, 28 Feb 2025 11:42:40 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6BA8210EC61; Fri, 28 Feb 2025 11:42:27 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=rz.uni-freiburg.de header.i=@rz.uni-freiburg.de header.b="AadxIHYG"; dkim-atps=neutral Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id C115610EAA0 for ; Thu, 27 Feb 2025 11:14:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654233; x=1742468634; bh=QSKou7xcZ5p7NC7E+19lDLqUDEdEEQZi vjX761QpEAc=; b=AadxIHYGYFzxQg7p5K6TOm2YsZmOY/rdt1/tDCvROpgKnYTb xF+cr591x0XY2OGVHy3H69BawryaZvXZhYxQ+ywtqkhG8yfbEKZlcjZXot5qcxlu Xv0odE9OLeUGaJDQ17gKW38+8unu9j5YOk2O+k8qEPFd4bDtlWHCytZtjX8= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 5A0092C013A for ; Thu, 27 Feb 2025 12:03:53 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276591; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy Subject: [PATCH 04/10] dcc-send: Encode and send gl_draw stream data to the remote client (v4) Date: Thu, 27 Feb 2025 12:03:26 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy For remote (or non-gl) clients, if a valid gl_draw stream exists, then we first extract the dmabuf fd associated with the scanout and share it with the encoder along with other key parameters such as stride, width and height. Once the encoder finishes creating an encoded buffer (using the dmabuf fd as input), we then send it over to the client. And, as soon as the encoder notifies that it is no longer using the dmabuf fd, we send a gl_draw_done async to the application. v2: (suggestions and fixups from Frediano) - Moved the DisplayStreamData initialization code from red_marshall_stream_data() into a separate function that is reused when marshalling gl_draw_stream. - Used new/delete instead of g_new/g_free for creating and destroying dmabuf_data object - s/notify_mem_free/free s/red_gst_mem_free_cb/red_free_fb - Removed the usage of opaque from red_free_cb v3: - Obtain the key params such as fd, stride, etc from the stream instead of the scanout - Replace the switch with if in red_marshall_gl_draw_stream() to avoid printing a warning (and spamming the console) when a frame is dropped v4: - Slightly improve the readability of red_marshall_gl_draw_stream() by adding checks at the start Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy --- server/dcc-send.cpp | 158 +++++++++++++++++++++++++++++++++-------- server/video-encoder.h | 13 ++++ 2 files changed, 143 insertions(+), 28 deletions(-) diff --git a/server/dcc-send.cpp b/server/dcc-send.cpp index 2c40a231..c418f375 100644 --- a/server/dcc-send.cpp +++ b/server/dcc-send.cpp @@ -1637,6 +1637,60 @@ static void red_release_video_encoder_buffer(uint8_t *data, void *opaque) buffer->free(buffer); } +static void red_init_display_stream_data(DisplayChannelClient *dcc, + SpiceMarshaller *base_marshaller, + Drawable *drawable, int stream_id, + VideoBuffer *outbuf, int is_sized) +{ + SpiceMsgDisplayStreamData stream_data; + SpiceMsgDisplayStreamDataSized stream_data_sized; + SpiceStreamDataHeader *base; + SpiceCopy *copy; + uint32_t frame_mm_time = reds_get_mm_time(); + uint16_t msg_type = is_sized ? SPICE_MSG_DISPLAY_STREAM_DATA_SIZED : + SPICE_MSG_DISPLAY_STREAM_DATA; + + dcc->init_send_data(msg_type); + + base = is_sized ? &stream_data_sized.base : &stream_data.base; + base->id = stream_id; + base->multi_media_time = frame_mm_time; + stream_data.data_size = outbuf->size; + + if (is_sized) { + copy = &drawable->red_drawable->u.copy; + frame_mm_time = drawable->red_drawable->mm_time ? + drawable->red_drawable->mm_time : + reds_get_mm_time(); + + stream_data_sized.base.multi_media_time = frame_mm_time; + stream_data_sized.width = copy->src_area.right - copy->src_area.left; + stream_data_sized.height = copy->src_area.bottom - copy->src_area.top; + stream_data_sized.dest = drawable->red_drawable->bbox; + stream_data_sized.data_size = outbuf->size; + + spice_debug("stream %d: sized frame: dest ==> ", + stream_data_sized.base.id); + rect_debug(&stream_data_sized.dest); + spice_marshall_msg_display_stream_data_sized(base_marshaller, + &stream_data_sized); + } else { + spice_marshall_msg_display_stream_data(base_marshaller, &stream_data); + } + + spice_marshaller_add_by_ref_full(base_marshaller, outbuf->data, + outbuf->size, + &red_release_video_encoder_buffer, + outbuf); +#ifdef STREAM_STATS + VideoStreamAgent *agent = &dcc->priv->stream_agents[stream_id]; + + agent->stats.num_frames_sent++; + agent->stats.size_sent += outbuf->size; + agent->stats.end = frame_mm_time; +#endif +} + static bool red_marshall_stream_data(DisplayChannelClient *dcc, SpiceMarshaller *base_marshaller, Drawable *drawable) @@ -1693,41 +1747,81 @@ static bool red_marshall_stream_data(DisplayChannelClient *dcc, return FALSE; } - if (!is_sized) { - SpiceMsgDisplayStreamData stream_data; + red_init_display_stream_data(dcc, base_marshaller, drawable, + stream_id, outbuf, is_sized); + return TRUE; +} - dcc->init_send_data(SPICE_MSG_DISPLAY_STREAM_DATA); +static void red_free_cb(VideoEncoderDmabufData *dmabuf_data) +{ + auto dcc = static_cast(dmabuf_data->dcc); + DisplayChannel *display = DCC_TO_DC(dcc); - stream_data.base.id = stream_id; - stream_data.base.multi_media_time = frame_mm_time; - stream_data.data_size = outbuf->size; + dcc->priv->gl_draw_ongoing = false; + display_channel_gl_draw_done(display); + delete dmabuf_data; +} - spice_marshall_msg_display_stream_data(base_marshaller, &stream_data); - } else { - SpiceMsgDisplayStreamDataSized stream_data; +static void red_marshall_gl_draw_stream(DisplayChannelClient *dcc, + SpiceMarshaller *base_marshaller) +{ + DisplayChannel *display = DCC_TO_DC(dcc); + VideoStream *stream = display->priv->gl_draw_stream; + int stream_id = display_channel_get_video_stream_id(display, stream); + VideoStreamAgent *agent = &dcc->priv->stream_agents[stream_id]; - dcc->init_send_data(SPICE_MSG_DISPLAY_STREAM_DATA_SIZED); + if (!agent->video_encoder || !agent->video_encoder->encode_dmabuf) { + spice_warning("No video encoder available for this stream"); + return; + } - stream_data.base.id = stream_id; - stream_data.base.multi_media_time = frame_mm_time; - stream_data.data_size = outbuf->size; - stream_data.width = copy->src_area.right - copy->src_area.left; - stream_data.height = copy->src_area.bottom - copy->src_area.top; - stream_data.dest = drawable->red_drawable->bbox; + VideoEncoderDmabufData *dmabuf_data = new VideoEncoderDmabufData; + if (!dmabuf_data) { + spice_warning("Cannot create memory for dmabuf data"); + return; + } - spice_debug("stream %d: sized frame: dest ==> ", stream_data.base.id); - rect_debug(&stream_data.dest); - spice_marshall_msg_display_stream_data_sized(base_marshaller, &stream_data); + QXLInstance* qxl = display->priv->qxl; + SpiceMsgDisplayGlScanoutUnix *scanout = red_qxl_get_gl_scanout(qxl); + if (!scanout) { + spice_warning("Cannot access scanout"); + delete dmabuf_data; + return; } - spice_marshaller_add_by_ref_full(base_marshaller, outbuf->data, outbuf->size, - &red_release_video_encoder_buffer, outbuf); + + dmabuf_data->drm_dma_buf_fd = scanout->drm_dma_buf_fd; + dmabuf_data->width = stream->width; + dmabuf_data->height = stream->height; + dmabuf_data->stride = stream->stride; + dmabuf_data->dcc = dcc; + dmabuf_data->free = red_free_cb; + red_qxl_put_gl_scanout(qxl, scanout); + + VideoBuffer *outbuf; + VideoEncodeResults ret; + + ret = agent->video_encoder->encode_dmabuf(agent->video_encoder, + reds_get_mm_time(), + dmabuf_data, &outbuf); + + if (ret != VIDEO_ENCODER_FRAME_ENCODE_DONE) { + if (ret == VIDEO_ENCODER_FRAME_DROP) { #ifdef STREAM_STATS - agent->stats.num_frames_sent++; - agent->stats.size_sent += outbuf->size; - agent->stats.end = frame_mm_time; + agent->stats.num_drops_fps++; #endif + } else { + spice_warning("bad ret value (%d) from VideoEncoder::encode_dmabuf", + ret); + } - return TRUE; + dcc->priv->gl_draw_ongoing = false; + display_channel_gl_draw_done(display); + delete dmabuf_data; + return; + } + + red_init_display_stream_data(dcc, base_marshaller, nullptr, + stream_id, outbuf, false); } static inline void marshall_inval_palette(RedChannelClient *rcc, @@ -2126,6 +2220,8 @@ static void marshall_stream_start(DisplayChannelClient *dcc, if (stream->current) { RedDrawable *red_drawable = stream->current->red_drawable.get(); stream_create.clip = red_drawable->clip; + } else if (stream == DCC_TO_DC(dcc)->priv->gl_draw_stream){ + stream_create.clip.type = SPICE_CLIP_TYPE_NONE; } else { stream_create.clip.type = SPICE_CLIP_TYPE_RECTS; clip_rects.num_rects = 0; @@ -2275,14 +2371,20 @@ static void marshall_gl_scanout(DisplayChannelClient *dcc, red_qxl_put_gl_scanout(qxl, scanout); } -static void marshall_gl_draw(RedChannelClient *rcc, +static void marshall_gl_draw(DisplayChannelClient *dcc, SpiceMarshaller *m, RedPipeItem *item) { auto p = static_cast(item); - rcc->init_send_data(SPICE_MSG_DISPLAY_GL_DRAW); - spice_marshall_msg_display_gl_draw(m, &p->draw); + if (dcc->is_gl_client()) { + dcc->init_send_data(SPICE_MSG_DISPLAY_GL_DRAW); + spice_marshall_msg_display_gl_draw(m, &p->draw); + } else if (DCC_TO_DC(dcc)->priv->gl_draw_stream) { + red_marshall_gl_draw_stream(dcc, m); + } else { + spice_warning("nothing to send to the client"); + } } diff --git a/server/video-encoder.h b/server/video-encoder.h index c2bdc811..0261bfca 100644 --- a/server/video-encoder.h +++ b/server/video-encoder.h @@ -56,6 +56,15 @@ typedef struct VideoEncoderStats { double avg_quality; } VideoEncoderStats; +typedef struct VideoEncoderDmabufData { + int drm_dma_buf_fd; + uint32_t width; + uint32_t height; + uint32_t stride; + void *dcc; + void (*free)(struct VideoEncoderDmabufData*); +} VideoEncoderDmabufData; + typedef struct VideoEncoder VideoEncoder; struct VideoEncoder { /* Releases the video encoder's resources */ @@ -84,6 +93,10 @@ struct VideoEncoder { const SpiceRect *src, int top_down, gpointer bitmap_opaque, VideoBuffer** outbuf); + VideoEncodeResults (*encode_dmabuf)(VideoEncoder *encoder, uint32_t frame_mm_time, + VideoEncoderDmabufData *dmabuf_data, + VideoBuffer** outbuf); + /* * Bit rate control methods. */ From patchwork Thu Feb 27 11:03:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AC57C282CD for ; Fri, 28 Feb 2025 11:42:29 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AA53C10EC64; Fri, 28 Feb 2025 11:42:16 +0000 (UTC) Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id C52B510EAA2 for ; Thu, 27 Feb 2025 11:13:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654234; x=1742468635; bh=N6AEi+fnhRFgvL28LNIVxMFMTDYriLQT 7EBnDkqopFM=; b=HxMKR8+Iy9ziOzYRxK6TCliKU7QcnGTWpdjzpWxJzxvVyG8T 2HeACTB/rkmZpvr89ugQClRS2xK5ADQhuqshkEuReo1JxYqjv2YIItE8XUl/9IPR sNI8bIpMCxkTewjwu/BD4BmEyG+4BPbyb2qcefXwvRiXaewxgNP902iBnHk= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 183A42C0118 for ; Thu, 27 Feb 2025 12:03:54 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276640; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy Subject: [PATCH 05/10] gstreamer-encoder: Add an encoder function that takes dmabuf fd as input (v3) Date: Thu, 27 Feb 2025 12:03:27 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy This patch adds a new function to enable the creation of Gst memory with the dmabuf fd as the source by using a dmabuf allocator. And, it also adds a mechanism to register and invoke any callbacks once the Gst memory object is no longer used by the pipeline. This patch also ensures that the source_fps value is always non-zero. v2: (suggestions from Frediano) - Moved the code associated with add_frame() and pipeline configuration into separate functions that are used when encoding dmabuf fd v3: - Add the new gstreamer-allocators dependency in autoconf as well (Frediano) - Ensure that VIDEO_ENCODER_FRAME_UNSUPPORTED is returned when an error is encountered in spice_gst_encoder_encode_dmabuf() Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy --- configure.ac | 2 +- meson.build | 2 +- server/gstreamer-encoder.c | 165 ++++++++++++++++++++++++++++++------- 3 files changed, 137 insertions(+), 32 deletions(-) diff --git a/configure.ac b/configure.ac index ff13ee3a..5ad947ca 100644 --- a/configure.ac +++ b/configure.ac @@ -99,7 +99,7 @@ AC_ARG_ENABLE(gstreamer, [enable_gstreamer="auto"]) if test "x$enable_gstreamer" != "xno" && test "x$enable_gstreamer" != "x0.10"; then - SPICE_CHECK_GSTREAMER(GSTREAMER_1_0, 1.0, [gstreamer-1.0 gstreamer-base-1.0 gstreamer-app-1.0 gstreamer-video-1.0], + SPICE_CHECK_GSTREAMER(GSTREAMER_1_0, 1.0, [gstreamer-1.0 gstreamer-base-1.0 gstreamer-app-1.0 gstreamer-video-1.0 gstreamer-allocators-1.0], [enable_gstreamer="1.0" SPICE_CHECK_GSTREAMER_ELEMENTS($GST_INSPECT_1_0, [gst-plugins-base 1.0], [appsrc videoconvert appsink]) SPICE_CHECK_GSTREAMER_ELEMENTS($GST_INSPECT_1_0, [gstreamer-libav 1.0], [avenc_mjpeg]) diff --git a/meson.build b/meson.build index b1237e61..d6aea60a 100644 --- a/meson.build +++ b/meson.build @@ -131,7 +131,7 @@ endforeach spice_server_has_gstreamer = false spice_server_gst_version = get_option('gstreamer') if spice_server_gst_version != 'no' - gst_deps = ['gstreamer', 'gstreamer-base', 'gstreamer-app', 'gstreamer-video'] + gst_deps = ['gstreamer', 'gstreamer-base', 'gstreamer-app', 'gstreamer-video', 'gstreamer-allocators'] foreach dep : gst_deps dep = '@0@-@1@'.format(dep, spice_server_gst_version) spice_server_deps += dependency(dep) diff --git a/server/gstreamer-encoder.c b/server/gstreamer-encoder.c index be8e3111..d4efe732 100644 --- a/server/gstreamer-encoder.c +++ b/server/gstreamer-encoder.c @@ -27,6 +27,7 @@ # pragma GCC diagnostic ignored "-Wunused-const-variable" #endif #include +#include #include #include #include @@ -283,6 +284,7 @@ typedef struct SpiceGstEncoder { /* How many frames were dropped by the server since the last encoded frame. */ uint32_t server_drops; + GstAllocator *allocator; } SpiceGstEncoder; @@ -318,8 +320,12 @@ static inline double get_mbps(uint64_t bit_rate) */ static uint32_t get_source_fps(const SpiceGstEncoder *encoder) { - return encoder->cbs.get_source_fps ? - encoder->cbs.get_source_fps(encoder->cbs.opaque) : SPICE_GST_DEFAULT_FPS; + uint32_t source_fps = 0; + + if (encoder->cbs.get_source_fps) { + source_fps = encoder->cbs.get_source_fps(encoder->cbs.opaque); + } + return source_fps ? source_fps : SPICE_GST_DEFAULT_FPS; } static uint32_t get_network_latency(const SpiceGstEncoder *encoder) @@ -1523,6 +1529,7 @@ static void spice_gst_encoder_destroy(VideoEncoder *video_encoder) { SpiceGstEncoder *encoder = (SpiceGstEncoder*)video_encoder; + gst_object_unref(encoder->allocator); free_pipeline(encoder); pthread_mutex_destroy(&encoder->outbuf_mutex); pthread_cond_destroy(&encoder->outbuf_cond); @@ -1533,35 +1540,45 @@ static void spice_gst_encoder_destroy(VideoEncoder *video_encoder) g_free(encoder); } -static VideoEncodeResults -spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, - uint32_t frame_mm_time, - const SpiceBitmap *bitmap, - const SpiceRect *src, int top_down, - gpointer bitmap_opaque, - VideoBuffer **outbuf) +static void spice_gst_encoder_add_frame(SpiceGstEncoder *encoder, + VideoBuffer **outbuf, + uint64_t start, + uint32_t frame_mm_time) { - SpiceGstEncoder *encoder = (SpiceGstEncoder*)video_encoder; - g_return_val_if_fail(outbuf != NULL, VIDEO_ENCODER_FRAME_UNSUPPORTED); - *outbuf = NULL; + uint32_t last_mm_time = get_last_frame_mm_time(encoder); - /* Unref the last frame's bitmap_opaque structures if any */ - clear_zero_copy_queue(encoder, FALSE); + add_frame(encoder, frame_mm_time, spice_get_monotonic_time_ns() - start, + (*outbuf)->size); + + int32_t refill = encoder->bit_rate * (frame_mm_time - last_mm_time) / MSEC_PER_SEC / 8; + encoder->vbuffer_free = MIN(encoder->vbuffer_free + refill, + encoder->vbuffer_size) - (*outbuf)->size; + + server_increase_bit_rate(encoder, frame_mm_time); + update_next_frame_mm_time(encoder); +} + +static VideoEncodeResults +spice_gst_encoder_configure_pipeline(SpiceGstEncoder *encoder, + uint32_t width, uint32_t height, + const SpiceBitmap *bitmap, + uint32_t frame_mm_time) +{ + SpiceBitmapFmt format = bitmap ? (SpiceBitmapFmt) bitmap->format : + SPICE_BITMAP_FMT_32BIT; - uint32_t width = src->right - src->left; - uint32_t height = src->bottom - src->top; if (width != encoder->width || height != encoder->height || - encoder->spice_format != bitmap->format) { + encoder->spice_format != format) { spice_debug("video format change: width %d -> %d, height %d -> %d, format %d -> %d", encoder->width, width, encoder->height, height, - encoder->spice_format, bitmap->format); - encoder->format = map_format((SpiceBitmapFmt) bitmap->format); + encoder->spice_format, format); + encoder->format = map_format(format); if (encoder->format == GSTREAMER_FORMAT_INVALID) { - spice_warning("unable to map format type %d", bitmap->format); + spice_warning("unable to map format type %d", format); encoder->errors = 4; return VIDEO_ENCODER_FRAME_UNSUPPORTED; } - encoder->spice_format = (SpiceBitmapFmt) bitmap->format; + encoder->spice_format = format; encoder->width = width; encoder->height = height; if (encoder->bit_rate == 0) { @@ -1600,8 +1617,36 @@ spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, return VIDEO_ENCODER_FRAME_UNSUPPORTED; } + return VIDEO_ENCODER_FRAME_ENCODE_DONE; +} + +static VideoEncodeResults +spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, + uint32_t frame_mm_time, + const SpiceBitmap *bitmap, + const SpiceRect *src, int top_down, + gpointer bitmap_opaque, + VideoBuffer **outbuf) +{ + SpiceGstEncoder *encoder = (SpiceGstEncoder*)video_encoder; + g_return_val_if_fail(outbuf != NULL, VIDEO_ENCODER_FRAME_UNSUPPORTED); + VideoEncodeResults rc; + *outbuf = NULL; + + /* Unref the last frame's bitmap_opaque structures if any */ + clear_zero_copy_queue(encoder, FALSE); + + uint32_t width = src->right - src->left; + uint32_t height = src->bottom - src->top; + + rc = spice_gst_encoder_configure_pipeline(encoder, width, height, + bitmap, frame_mm_time); + if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { + return rc; + } + uint64_t start = spice_get_monotonic_time_ns(); - VideoEncodeResults rc = push_raw_frame(encoder, bitmap, src, top_down, bitmap_opaque); + rc = push_raw_frame(encoder, bitmap, src, top_down, bitmap_opaque); if (rc == VIDEO_ENCODER_FRAME_ENCODE_DONE) { rc = pull_compressed_buffer(encoder, outbuf); if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { @@ -1621,17 +1666,75 @@ spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { return rc; } - uint32_t last_mm_time = get_last_frame_mm_time(encoder); - add_frame(encoder, frame_mm_time, spice_get_monotonic_time_ns() - start, - (*outbuf)->size); - int32_t refill = encoder->bit_rate * (frame_mm_time - last_mm_time) / MSEC_PER_SEC / 8; - encoder->vbuffer_free = MIN(encoder->vbuffer_free + refill, - encoder->vbuffer_size) - (*outbuf)->size; + spice_gst_encoder_add_frame(encoder, outbuf, start, frame_mm_time); + return rc; +} - server_increase_bit_rate(encoder, frame_mm_time); - update_next_frame_mm_time(encoder); +static void +spice_gst_mem_free_cb(VideoEncoderDmabufData *dmabuf_data, GstMiniObject *obj) +{ + if (dmabuf_data->free) { + dmabuf_data->free(dmabuf_data); + } +} + +static VideoEncodeResults +spice_gst_encoder_encode_dmabuf(VideoEncoder *video_encoder, + uint32_t frame_mm_time, + VideoEncoderDmabufData *dmabuf_data, + VideoBuffer **outbuf) +{ + SpiceGstEncoder *encoder = (SpiceGstEncoder*)video_encoder; + g_return_val_if_fail(outbuf != NULL, VIDEO_ENCODER_FRAME_UNSUPPORTED); + g_return_val_if_fail(dmabuf_data != NULL, VIDEO_ENCODER_FRAME_UNSUPPORTED); + VideoEncodeResults rc; + + rc = spice_gst_encoder_configure_pipeline(encoder, dmabuf_data->width, + dmabuf_data->height, NULL, + frame_mm_time); + if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { + return rc; + } + + gsize size = dmabuf_data->stride * dmabuf_data->height; + uint64_t start = spice_get_monotonic_time_ns(); + GstBuffer *buffer; + GstMemory *mem; + *outbuf = NULL; + rc = VIDEO_ENCODER_FRAME_UNSUPPORTED; + + mem = gst_dmabuf_allocator_alloc_with_flags(encoder->allocator, + dmabuf_data->drm_dma_buf_fd, + size, + GST_FD_MEMORY_FLAG_DONT_CLOSE); + if (!mem) { + return rc; + } + buffer = gst_buffer_new(); + gst_buffer_append_memory(buffer, mem); + gst_mini_object_weak_ref(GST_MINI_OBJECT(mem), + (GstMiniObjectNotify)spice_gst_mem_free_cb, + dmabuf_data); + GstFlowReturn ret = gst_app_src_push_buffer(encoder->appsrc, buffer); + if (ret != GST_FLOW_OK) { + spice_warning("GStreamer error: unable to push source buffer (%d)", ret); + return rc; + } + + rc = pull_compressed_buffer(encoder, outbuf); + if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { + /* The input buffer will be stuck in the pipeline, preventing + * later ones from being processed. Furthermore something went + * wrong with this pipeline, so it may be safer to rebuild it + * from scratch. + */ + free_pipeline(encoder); + encoder->errors++; + return rc; + } + spice_gst_encoder_add_frame(encoder, outbuf, start, frame_mm_time); return rc; } @@ -1825,6 +1928,7 @@ VideoEncoder *gstreamer_encoder_new(SpiceVideoCodecType codec_type, SpiceGstEncoder *encoder = g_new0(SpiceGstEncoder, 1); encoder->base.destroy = spice_gst_encoder_destroy; encoder->base.encode_frame = spice_gst_encoder_encode_frame; + encoder->base.encode_dmabuf = spice_gst_encoder_encode_dmabuf; encoder->base.client_stream_report = spice_gst_encoder_client_stream_report; encoder->base.notify_server_frame_drop = spice_gst_encoder_notify_server_frame_drop; encoder->base.get_bit_rate = spice_gst_encoder_get_bit_rate; @@ -1837,6 +1941,7 @@ VideoEncoder *gstreamer_encoder_new(SpiceVideoCodecType codec_type, encoder->bitmap_ref = bitmap_ref; encoder->bitmap_unref = bitmap_unref; encoder->format = GSTREAMER_FORMAT_INVALID; + encoder->allocator = gst_dmabuf_allocator_new(); pthread_mutex_init(&encoder->outbuf_mutex, NULL); pthread_cond_init(&encoder->outbuf_cond, NULL); From patchwork Thu Feb 27 11:03:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF46BC19776 for ; Fri, 28 Feb 2025 11:42:21 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 547DF10EC6F; Fri, 28 Feb 2025 11:42:16 +0000 (UTC) Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5F54010EA9F for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:content-type :content-type:mime-version:references:in-reply-to:x-mailer :message-id:date:date:subject:subject:from:from:received; s=s1; t=1740654235; x=1742468636; bh=8RATFbMj7UpggREzTgYkMjC6tR6fuPcG 7aMqjpD43OA=; b=CR9rXQADcC1yKPqFh384CxIqFI8Wf5VcrGtsVIbgN+8MqMaR QVpZsi81w0Oam1L2yFGE1qFBpNrQDlyfJr0PibRRATt/jYYB8L1Qly6QM1Pk798w RtBHot/svG6NJ3DQZi9JA/C6Gs0APOZMA+IZcOJddTmUQ0gUC7hKJ8Cg7o4= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 0E92C2C013C for ; Thu, 27 Feb 2025 12:03:55 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276635; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Frediano Ziglio , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Dongwon Kim , Vivek Kasireddy Subject: [PATCH 06/10] video-stream: Don't stop a stream associated with gl_draw (v2) Date: Thu, 27 Feb 2025 12:03:28 +0100 Message-ID: <6b56536ad288075bfeda2bd0511401eba2edb65e.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy We do not want to stop a stream associated with gl_draw as a result of timeout because we may not get another opportunity to create a new stream if the current one gets stopped. However, when the stream does get stopped for other reasons, we need to clear the gl_draw_stream pointer associated with the relevant DC. v2: (suggestions from Frediano) - Don't stop the stream regardless of whether gl_draw is ongoing or not Cc: Frediano Ziglio Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Dongwon Kim Signed-off-by: Vivek Kasireddy Acked-by: Frediano Ziglio --- server/video-stream.cpp | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/server/video-stream.cpp b/server/video-stream.cpp index 691c50a3..8418eed3 100644 --- a/server/video-stream.cpp +++ b/server/video-stream.cpp @@ -116,6 +116,10 @@ void video_stream_stop(DisplayChannel *display, VideoStream *stream) } dcc->pipe_add(video_stream_destroy_item_new(stream_agent)); video_stream_agent_stats_print(stream_agent); + + if (stream == display->priv->gl_draw_stream) { + display->priv->gl_draw_stream = nullptr; + } } display->priv->streams_size_total -= stream->width * stream->height; ring_remove(&stream->link); @@ -1034,7 +1038,8 @@ void video_stream_timeout(DisplayChannel *display) while (item) { VideoStream *stream = SPICE_CONTAINEROF(item, VideoStream, link); item = ring_next(ring, item); - if (now >= (stream->last_time + RED_STREAM_TIMEOUT)) { + if (now >= (stream->last_time + RED_STREAM_TIMEOUT) && + stream != display->priv->gl_draw_stream) { detach_video_stream_gracefully(display, stream, nullptr); video_stream_stop(display, stream); } From patchwork Thu Feb 27 11:03:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BDB0C282D0 for ; Fri, 28 Feb 2025 11:42:26 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9CAA710EC6C; Fri, 28 Feb 2025 11:42:16 +0000 (UTC) Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 60CEC10EAA2 for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received; s=s1; t=1740654235; x=1742468636; bh=YoSH+mt4pZZAw6M5OviugL2nAqCFIHJqxgF1lqJ50Qo=; b=eodo+yaryiSX y7FwqmK6LPzB3/2YLZN6lb5O7EpindcdWG1loLodWYGC1ZKb4O6EexumDXqe6kXV cazFUODWY7g8qCgEylEwRmJZ9J9HLvk4BRkVjoAB04yYU78oU2chA1oTupv54RaZ tuVX0jgGo3LcG2yVHYlWjnRG0zFRSOM= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 8CED72C0132 for ; Thu, 27 Feb 2025 12:03:55 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276643; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle , Vivek Kasireddy Subject: [PATCH 07/10] gstreamer-encoder: Map the drm format to appropriate Gstreamer format Date: Thu, 27 Feb 2025 12:03:29 +0100 Message-ID: <4088d64b91bfa44191129f1e92ea1a6137534076.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Vivek Kasireddy We need to convert the scanout's drm format to the correct Gstreamer format while configuring the pipeline. This can be done using gst_video_dma_drm_fourcc_to_format() API, which will take the drm fourcc value and return the appropriate Gst format. Signed-off-by: Vivek Kasireddy --- server/dcc-send.cpp | 1 + server/gstreamer-encoder.c | 103 +++++++++++++++++++++++++++++++------ server/video-encoder.h | 1 + 3 files changed, 88 insertions(+), 17 deletions(-) diff --git a/server/dcc-send.cpp b/server/dcc-send.cpp index c418f375..7bc20a22 100644 --- a/server/dcc-send.cpp +++ b/server/dcc-send.cpp @@ -1790,6 +1790,7 @@ static void red_marshall_gl_draw_stream(DisplayChannelClient *dcc, } dmabuf_data->drm_dma_buf_fd = scanout->drm_dma_buf_fd; + dmabuf_data->drm_fourcc_format = scanout->drm_fourcc_format; dmabuf_data->width = stream->width; dmabuf_data->height = stream->height; dmabuf_data->stride = stream->stride; diff --git a/server/gstreamer-encoder.c b/server/gstreamer-encoder.c index d4efe732..196b27ab 100644 --- a/server/gstreamer-encoder.c +++ b/server/gstreamer-encoder.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #if defined(__GNUC__) && (__GNUC__ >= 6) # pragma GCC diagnostic pop @@ -40,19 +41,26 @@ #include "video-encoder.h" #include "utils.h" #include "common/udev.h" - +#include "drm/drm_fourcc.h" #define SPICE_GST_DEFAULT_FPS 30 typedef struct { SpiceBitmapFmt spice_format; + uint32_t drm_format; uint32_t bpp; char format[8]; GstVideoFormat gst_format; } SpiceFormatForGStreamer; +#define __FMT_DESC(spice_format, drm_format, bpp, format, gst_format) \ + { spice_format, drm_format, bpp, format, gst_format } + #define FMT_DESC(spice_format, bpp, format, gst_format) \ - { spice_format, bpp, format, gst_format } + __FMT_DESC(spice_format, DRM_FORMAT_INVALID, bpp, format, gst_format) + +#define DRM_FMT_DESC(drm_format, bpp) \ + __FMT_DESC(SPICE_BITMAP_FMT_INVALID, drm_format, bpp, "", GST_VIDEO_FORMAT_UNKNOWN) typedef struct SpiceGstVideoBuffer { VideoBuffer base; @@ -93,6 +101,7 @@ typedef struct SpiceGstEncoder { uint32_t height; const SpiceFormatForGStreamer *format; SpiceBitmapFmt spice_format; + uint32_t drm_format; /* Number of consecutive frame encoding errors. */ uint32_t errors; @@ -784,6 +793,47 @@ static const SpiceFormatForGStreamer *map_format(SpiceBitmapFmt format) return GSTREAMER_FORMAT_INVALID; } +static SpiceFormatForGStreamer drm_format_map[] = { + DRM_FMT_DESC(DRM_FORMAT_INVALID, 0), + DRM_FMT_DESC(DRM_FORMAT_XRGB8888, 32), + DRM_FMT_DESC(DRM_FORMAT_XBGR8888, 32), + DRM_FMT_DESC(DRM_FORMAT_RGBX8888, 32), + DRM_FMT_DESC(DRM_FORMAT_BGRX8888, 32), + DRM_FMT_DESC(DRM_FORMAT_ARGB8888, 32), + DRM_FMT_DESC(DRM_FORMAT_ABGR8888, 32), + DRM_FMT_DESC(DRM_FORMAT_RGBA8888, 32), + DRM_FMT_DESC(DRM_FORMAT_BGRA8888, 32), +}; +#define GSTREAMER_DRM_FORMAT_INVALID (&drm_format_map[0]) + +static const SpiceFormatForGStreamer *map_drm_format(uint32_t fourcc) +{ + int i; + + for (i = 0; i < G_N_ELEMENTS(drm_format_map); i++) { + if (drm_format_map[i].drm_format == fourcc) { + if (drm_format_map[i].gst_format == GST_VIDEO_FORMAT_UNKNOWN) { + int format_size = sizeof(drm_format_map[i].format); + GstVideoFormat gst_format; + + gst_format = gst_video_dma_drm_fourcc_to_format(fourcc); + if (gst_format == GST_VIDEO_FORMAT_UNKNOWN) { + break; + } + + drm_format_map[i].gst_format = gst_format; + strncpy(drm_format_map[i].format, + gst_video_format_to_string(gst_format), + format_size - 1); + drm_format_map[i].format[format_size - 1] = '\0'; + } + return &drm_format_map[i]; + } + } + + return GSTREAMER_DRM_FORMAT_INVALID; +} + static void set_appsrc_caps(SpiceGstEncoder *encoder) { if (encoder->src_caps) { @@ -1561,26 +1611,41 @@ static void spice_gst_encoder_add_frame(SpiceGstEncoder *encoder, static VideoEncodeResults spice_gst_encoder_configure_pipeline(SpiceGstEncoder *encoder, uint32_t width, uint32_t height, - const SpiceBitmap *bitmap, + SpiceBitmapFmt spice_format, + uint32_t drm_format, uint32_t frame_mm_time) { - SpiceBitmapFmt format = bitmap ? (SpiceBitmapFmt) bitmap->format : - SPICE_BITMAP_FMT_32BIT; + const SpiceFormatForGStreamer *format = GSTREAMER_FORMAT_INVALID; + + if (spice_format != SPICE_BITMAP_FMT_INVALID) { + format = map_format(spice_format); + } else if (drm_format != DRM_FORMAT_INVALID) { + format = map_drm_format(drm_format); + } + + if (format == GSTREAMER_FORMAT_INVALID || + format == GSTREAMER_DRM_FORMAT_INVALID) { + spice_warning("unable to map format type %d or %u", + spice_format, drm_format); + encoder->errors = 4; + return VIDEO_ENCODER_FRAME_UNSUPPORTED; + } if (width != encoder->width || height != encoder->height || - encoder->spice_format != format) { - spice_debug("video format change: width %d -> %d, height %d -> %d, format %d -> %d", + encoder->spice_format != spice_format || + encoder->drm_format != drm_format) { + spice_debug("video format change: width %d -> %d, height %d -> %d," + "spice format %d -> %d, drm format %u -> %u", encoder->width, width, encoder->height, height, - encoder->spice_format, format); - encoder->format = map_format(format); - if (encoder->format == GSTREAMER_FORMAT_INVALID) { - spice_warning("unable to map format type %d", format); - encoder->errors = 4; - return VIDEO_ENCODER_FRAME_UNSUPPORTED; - } - encoder->spice_format = format; + encoder->spice_format, spice_format, + encoder->drm_format, drm_format); + + encoder->format = format; + encoder->spice_format = spice_format; + encoder->drm_format = drm_format; encoder->width = width; encoder->height = height; + if (encoder->bit_rate == 0) { encoder->history[0].mm_time = frame_mm_time; encoder->max_bit_rate = get_bit_rate_cap(encoder); @@ -1640,7 +1705,9 @@ spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, uint32_t height = src->bottom - src->top; rc = spice_gst_encoder_configure_pipeline(encoder, width, height, - bitmap, frame_mm_time); + bitmap->format, + DRM_FORMAT_INVALID, + frame_mm_time); if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { return rc; } @@ -1691,7 +1758,9 @@ spice_gst_encoder_encode_dmabuf(VideoEncoder *video_encoder, VideoEncodeResults rc; rc = spice_gst_encoder_configure_pipeline(encoder, dmabuf_data->width, - dmabuf_data->height, NULL, + dmabuf_data->height, + SPICE_BITMAP_FMT_INVALID, + dmabuf_data->drm_fourcc_format, frame_mm_time); if (rc != VIDEO_ENCODER_FRAME_ENCODE_DONE) { return rc; diff --git a/server/video-encoder.h b/server/video-encoder.h index 0261bfca..a4cebe0a 100644 --- a/server/video-encoder.h +++ b/server/video-encoder.h @@ -61,6 +61,7 @@ typedef struct VideoEncoderDmabufData { uint32_t width; uint32_t height; uint32_t stride; + uint32_t drm_fourcc_format; void *dcc; void (*free)(struct VideoEncoderDmabufData*); } VideoEncoderDmabufData; From patchwork Thu Feb 27 11:03:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B7C7C282D1 for ; Fri, 28 Feb 2025 11:42:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2F2C110EC69; Fri, 28 Feb 2025 11:42:28 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=rz.uni-freiburg.de header.i=@rz.uni-freiburg.de header.b="Tl235hVV"; dkim-atps=neutral Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id C11B410EAA1 for ; Thu, 27 Feb 2025 11:13:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received; s=s1; t=1740654236; x=1742468637; bh=mZHf/5mA+it8u9rABQpoXpNp2EGQfobBJtDxxa+otXs=; b=Tl235hVV0Nai VJz55e6rB9iQs76Cq1biPoiJFvVCUq4KfzgOL5TQnWNibYH2lByrqOJJ/Bnmysir 5tGbs+av+J42NoGNlSlDloXo3qhbxt1jH2YINnu8ktyvbX1/AHYylpBb2hVQ/58/ N0UcsBbgoIAjNPEo/slxF7vCPyKsG8U= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 30FC42C013F for ; Thu, 27 Feb 2025 12:03:56 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276626; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle Subject: [PATCH 08/10] Update spice-common submodule Date: Thu, 27 Feb 2025 12:03:30 +0100 Message-ID: <23dcc5422093345fab0b9c8122fc807d5243a41f.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This brings in the following changes: common: Add a udev helper to identify GPU Vendor build: Avoid Meson warning Drop Python 2 from m4/spice-deps.m4 Stop using Python six package codegen: Use context manager when opening files Signed-off-by: Michael Scherle --- subprojects/spice-common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/subprojects/spice-common b/subprojects/spice-common index 58d375e5..8c0319e3 160000 --- a/subprojects/spice-common +++ b/subprojects/spice-common @@ -1 +1 @@ -Subproject commit 58d375e5eadc6fb9e587e99fd81adcb95d01e8d6 +Subproject commit 8c0319e31df967e41c74f4121cbdb3b785fe114e From patchwork Thu Feb 27 11:03:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5222CC19776 for ; Fri, 28 Feb 2025 11:42:36 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2A2A510EA49; Fri, 28 Feb 2025 11:42:28 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=rz.uni-freiburg.de header.i=@rz.uni-freiburg.de header.b="fXW7mjVE"; dkim-atps=neutral Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 64C7F10EAA7 for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received; s=s1; t=1740654236; x=1742468637; bh=JqrDFvKMMAgRlPdNufuNTqNV47pR6HSL7IFjhAPJ/+I=; b=fXW7mjVEDQ4e ght0uH6mNmCHcvRV7exrpsTdmS4BXlg7Vl1YCldcUnlikciBie3Sv0rQHvB1xd97 m/49SS5zAKiIr6eevFyjTsgre2KU5mIhdvUvO+ox82JrjrHwZxNpPD1Iu2/l/YNF ZboxSd46UeSKyxJR35Pvv2PRKJEm7uo= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id AC63F2C013D for ; Thu, 27 Feb 2025 12:03:56 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276627; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle Subject: [PATCH 09/10] gstreamer-encoder: Include dmabuf encoding conditionally for Linux Date: Thu, 27 Feb 2025 12:03:31 +0100 Message-ID: <036fa70a60bc9f657476280de7494791b5cf2c1c.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" add dmabuf encoding if `drm/drm_fourcc.h` is present and gstreamer is at least 1.24 due to `gst_video_dma_drm_fourcc_to_format()`. Signed-off-by: Michael Scherle --- meson.build | 10 +++++++++- server/gstreamer-encoder.c | 34 +++++++++++++++++++++++++++------- 2 files changed, 36 insertions(+), 8 deletions(-) diff --git a/meson.build b/meson.build index d6aea60a..d5591d2a 100644 --- a/meson.build +++ b/meson.build @@ -131,15 +131,23 @@ endforeach spice_server_has_gstreamer = false spice_server_gst_version = get_option('gstreamer') if spice_server_gst_version != 'no' + gstreamer_version = '' gst_deps = ['gstreamer', 'gstreamer-base', 'gstreamer-app', 'gstreamer-video', 'gstreamer-allocators'] foreach dep : gst_deps dep = '@0@-@1@'.format(dep, spice_server_gst_version) - spice_server_deps += dependency(dep) + dep = dependency(dep) + if gstreamer_version == '' or dep.version().version_compare('< ' + gstreamer_version) + gstreamer_version = dep.version() + endif + spice_server_deps += dep endforeach spice_server_deps += dependency('orc-0.4') gst_def = 'HAVE_GSTREAMER_1_0' spice_server_config_data.set(gst_def, '1') + if compiler.has_header('drm/drm_fourcc.h') and gstreamer_version.version_compare('>= 1.24') + spice_server_config_data.set('HAVE_GSTREAMER_DMABUF_ENCODING', '1') + endif spice_server_has_gstreamer = true endif diff --git a/server/gstreamer-encoder.c b/server/gstreamer-encoder.c index 196b27ab..a83cf846 100644 --- a/server/gstreamer-encoder.c +++ b/server/gstreamer-encoder.c @@ -27,11 +27,13 @@ # pragma GCC diagnostic ignored "-Wunused-const-variable" #endif #include -#include #include #include #include +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING #include +#include +#endif #include #if defined(__GNUC__) && (__GNUC__ >= 6) # pragma GCC diagnostic pop @@ -41,7 +43,12 @@ #include "video-encoder.h" #include "utils.h" #include "common/udev.h" +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING #include "drm/drm_fourcc.h" +#endif +#ifndef DRM_FORMAT_INVALID +#define DRM_FORMAT_INVALID 0 +#endif #define SPICE_GST_DEFAULT_FPS 30 @@ -793,6 +800,7 @@ static const SpiceFormatForGStreamer *map_format(SpiceBitmapFmt format) return GSTREAMER_FORMAT_INVALID; } +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING static SpiceFormatForGStreamer drm_format_map[] = { DRM_FMT_DESC(DRM_FORMAT_INVALID, 0), DRM_FMT_DESC(DRM_FORMAT_XRGB8888, 32), @@ -822,7 +830,7 @@ static const SpiceFormatForGStreamer *map_drm_format(uint32_t fourcc) } drm_format_map[i].gst_format = gst_format; - strncpy(drm_format_map[i].format, + g_strlcpy(drm_format_map[i].format, gst_video_format_to_string(gst_format), format_size - 1); drm_format_map[i].format[format_size - 1] = '\0'; @@ -833,6 +841,7 @@ static const SpiceFormatForGStreamer *map_drm_format(uint32_t fourcc) return GSTREAMER_DRM_FORMAT_INVALID; } +#endif static void set_appsrc_caps(SpiceGstEncoder *encoder) { @@ -1579,7 +1588,9 @@ static void spice_gst_encoder_destroy(VideoEncoder *video_encoder) { SpiceGstEncoder *encoder = (SpiceGstEncoder*)video_encoder; +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING gst_object_unref(encoder->allocator); +#endif free_pipeline(encoder); pthread_mutex_destroy(&encoder->outbuf_mutex); pthread_cond_destroy(&encoder->outbuf_cond); @@ -1619,12 +1630,17 @@ spice_gst_encoder_configure_pipeline(SpiceGstEncoder *encoder, if (spice_format != SPICE_BITMAP_FMT_INVALID) { format = map_format(spice_format); - } else if (drm_format != DRM_FORMAT_INVALID) { + } +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING + else if (drm_format != DRM_FORMAT_INVALID) { format = map_drm_format(drm_format); } - - if (format == GSTREAMER_FORMAT_INVALID || - format == GSTREAMER_DRM_FORMAT_INVALID) { +#endif + if (format == GSTREAMER_FORMAT_INVALID +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING + || format == GSTREAMER_DRM_FORMAT_INVALID +#endif + ) { spice_warning("unable to map format type %d or %u", spice_format, drm_format); encoder->errors = 4; @@ -1738,6 +1754,7 @@ spice_gst_encoder_encode_frame(VideoEncoder *video_encoder, return rc; } +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING static void spice_gst_mem_free_cb(VideoEncoderDmabufData *dmabuf_data, GstMiniObject *obj) { @@ -1806,6 +1823,7 @@ spice_gst_encoder_encode_dmabuf(VideoEncoder *video_encoder, spice_gst_encoder_add_frame(encoder, outbuf, start, frame_mm_time); return rc; } +#endif static void spice_gst_encoder_client_stream_report(VideoEncoder *video_encoder, uint32_t num_frames, @@ -1997,7 +2015,10 @@ VideoEncoder *gstreamer_encoder_new(SpiceVideoCodecType codec_type, SpiceGstEncoder *encoder = g_new0(SpiceGstEncoder, 1); encoder->base.destroy = spice_gst_encoder_destroy; encoder->base.encode_frame = spice_gst_encoder_encode_frame; +#ifdef HAVE_GSTREAMER_DMABUF_ENCODING encoder->base.encode_dmabuf = spice_gst_encoder_encode_dmabuf; + encoder->allocator = gst_dmabuf_allocator_new(); +#endif encoder->base.client_stream_report = spice_gst_encoder_client_stream_report; encoder->base.notify_server_frame_drop = spice_gst_encoder_notify_server_frame_drop; encoder->base.get_bit_rate = spice_gst_encoder_get_bit_rate; @@ -2010,7 +2031,6 @@ VideoEncoder *gstreamer_encoder_new(SpiceVideoCodecType codec_type, encoder->bitmap_ref = bitmap_ref; encoder->bitmap_unref = bitmap_unref; encoder->format = GSTREAMER_FORMAT_INVALID; - encoder->allocator = gst_dmabuf_allocator_new(); pthread_mutex_init(&encoder->outbuf_mutex, NULL); pthread_cond_init(&encoder->outbuf_cond, NULL); From patchwork Thu Feb 27 11:03:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Scherle X-Patchwork-Id: 13996288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8122C19776 for ; Fri, 28 Feb 2025 11:42:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6C60C10EC62; Fri, 28 Feb 2025 11:42:27 +0000 (UTC) Received: from c1422.mx.srv.dfn.de (c1422.mx.srv.dfn.de [194.95.239.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5F4CE10EA64 for ; Thu, 27 Feb 2025 11:14:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= rz.uni-freiburg.de; h=content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received; s=s1; t=1740654237; x=1742468638; bh=TVxgBJxmZfiHsWtX8JFHPCWQpRFrk1VLF7e9y8dPpiQ=; b=C04sIOcZN1cd DQHAM9eki5k77SLJlz4yCobjprjhk8naYHlwzNv1L14TcBNizFSTRhWIiYLMSPAi /9yQk68zflD8EiRDbCb9vJhbhR4otMHExeDuh0mPZs+Tx0boONCPKxCEo003sJ+6 r0r0B2O/w9bzpe4lr7qQO8kXLspW2oU= Received: from fe1.uni-freiburg.de (fe1.uni-freiburg.de [132.230.2.221]) by c1422.mx.srv.dfn.de (Postfix) with ESMTP id 56FDE2C0125 for ; Thu, 27 Feb 2025 12:03:57 +0100 (CET) Received: from [2001:7c0:2517:a:4b56:9ec4:d188:b1a0] (account michael.scherle@rz.uni-freiburg.de HELO rz-10-126-20-105.eduroam-rz.privat) by mail.uni-freiburg.de (CommuniGate Pro SMTP 6.3.19) with ESMTPSA id 46276639; Thu, 27 Feb 2025 12:03:50 +0100 From: Michael Scherle To: dri-devel@lists.freedesktop.org Cc: Michael Scherle Subject: [PATCH 10/10] dcc-send: Fix freeze when video stream is stopped during ongoing draw Date: Thu, 27 Feb 2025 12:03:32 +0100 Message-ID: <5245ba7dd46644371b7c2b8495e4650ec987c598.1740651328.git.michael.scherle@rz.uni-freiburg.de> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 28 Feb 2025 11:42:15 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Prevent a freeze that occurs if the video stream is stopped while a gl draw is in progress (e.g., when the client requests a new codec). Ensure proper cleanup of the ongoing gl draw. Signed-off-by: Michael Scherle --- server/dcc-send.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/server/dcc-send.cpp b/server/dcc-send.cpp index 7bc20a22..71873563 100644 --- a/server/dcc-send.cpp +++ b/server/dcc-send.cpp @@ -2383,6 +2383,10 @@ static void marshall_gl_draw(DisplayChannelClient *dcc, spice_marshall_msg_display_gl_draw(m, &p->draw); } else if (DCC_TO_DC(dcc)->priv->gl_draw_stream) { red_marshall_gl_draw_stream(dcc, m); + } else if (dcc->priv->gl_draw_ongoing) { + DisplayChannel *display = DCC_TO_DC(dcc); + dcc->priv->gl_draw_ongoing = false; + display_channel_gl_draw_done(display); } else { spice_warning("nothing to send to the client"); }