[v4,22/28] media: iris: handle streamoff/on from client in dynamic resolution change

Message ID	20241014-qcom-video-iris-v4-v4-22-c5eaa4e9ab9e@quicinc.com (mailing list archive)
State	Superseded
Headers	show Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 541561AB52E; Mon, 14 Oct 2024 09:09:24 +0000 (UTC) From: Dikshita Agarwal <quic_dikshita@quicinc.com> Date: Mon, 14 Oct 2024 14:37:43 +0530 Subject: [PATCH v4 22/28] media: iris: handle streamoff/on from client in dynamic resolution change Precedence: bulk MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-ID: <20241014-qcom-video-iris-v4-v4-22-c5eaa4e9ab9e@quicinc.com> References: <20241014-qcom-video-iris-v4-v4-0-c5eaa4e9ab9e@quicinc.com> In-Reply-To: <20241014-qcom-video-iris-v4-v4-0-c5eaa4e9ab9e@quicinc.com> To: Vikash Garodia <quic_vgarodia@quicinc.com>, Abhinav Kumar <quic_abhinavk@quicinc.com>, Mauro Carvalho Chehab <mchehab@kernel.org>, "Rob Herring" <robh@kernel.org>, Krzysztof Kozlowski <krzk+dt@kernel.org>, "Conor Dooley" <conor+dt@kernel.org>, Philipp Zabel <p.zabel@pengutronix.de> CC: Hans Verkuil <hverkuil@xs4all.nl>, Sebastian Fricke <sebastian.fricke@collabora.com>, <linux-arm-msm@vger.kernel.org>, <linux-media@vger.kernel.org>, <devicetree@vger.kernel.org>, <linux-kernel@vger.kernel.org>, Dikshita Agarwal <quic_dikshita@quicinc.com>
Series	Qualcomm iris video decoder driver \| expand [v4,00/28] Qualcomm iris video decoder driver [v4,01/28] dt-bindings: media: Add video support for QCOM SM8550 SoC [v4,02/28] media: iris: add platform driver for iris video device [v4,03/28] media: iris: implement iris v4l2 file ops [v4,04/28] media: iris: introduce iris core state management with shared queues [v4,05/28] media: iris: implement video firmware load/unload [v4,06/28] media: iris: implement boot sequence of the firmware [v4,07/28] media: iris: introduce host firmware interface with necessary hooks [v4,08/28] media: iris: implement power management [v4,09/28] media: iris: implement reqbuf ioctl with vb2_queue_setup [v4,10/28] media: iris: implement s_fmt, g_fmt and try_fmt ioctls [v4,11/28] media: iris: implement g_selection ioctl [v4,12/28] media: iris: implement enum_fmt and enum_frameintervals ioctls [v4,13/28] media: iris: implement subscribe_event and unsubscribe_event ioctls [v4,14/28] media: iris: implement iris v4l2_ctrl_ops [v4,15/28] media: iris: implement query_cap, query_ctrl and query_menu ioctls [v4,16/28] media: iris: implement vb2 streaming ops [v4,17/28] media: iris: implement set properties to firmware during streamon [v4,18/28] media: iris: subscribe parameters and properties to firmware for hfi_gen2 [v4,19/28] media: iris: allocate, initialize and queue internal buffers [v4,20/28] media: iris: implement vb2 ops for buf_queue and firmware response [v4,21/28] media: iris: add support for dynamic resolution change [v4,22/28] media: iris: handle streamoff/on from client in dynamic resolution change [v4,23/28] media: iris: add support for drain sequence [v4,24/28] media: iris: add check whether the video session is supported or not [v4,25/28] media: iris: implement power scaling for vpu2 and vpu3 [v4,26/28] media: iris: add check to allow sub states transitions [v4,27/28] media: iris: enable video driver probe of SM8250 SoC [v4,28/28] media: MAINTAINERS: add Qualcomm iris video accelerator driver

diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index 8df003000dc8..a002675d15d7 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -404,6 +404,47 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) return 0; } +static int iris_release_internal_buffers(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + struct iris_buffers *buffers = &inst->buffers[buffer_type]; + struct iris_buffer *buffer, *next; + int ret; + + list_for_each_entry_safe(buffer, next, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (!(buffer->attr & BUF_ATTR_QUEUED)) + continue; + ret = hfi_ops->session_release_buf(inst, buffer); + if (ret) + return ret; + buffer->attr |= BUF_ATTR_PENDING_RELEASE; + } + + return 0; +} + +static int iris_release_input_internal_buffers(struct iris_inst *inst) +{ + const struct iris_platform_data *platform_data = inst->core->iris_platform_data; + const u32 *internal_buf_type; + u32 internal_buffer_count, i; + int ret; + + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + + for (i = 0; i < internal_buffer_count; i++) { + ret = iris_release_internal_buffers(inst, internal_buf_type[i]); + if (ret) + return ret; + } + + return 0; +} + int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) { struct iris_buffers *buffers = &inst->buffers[BUF_PERSIST]; @@ -435,6 +476,23 @@ int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) return 0; } +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst) +{ + int ret; + + iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + ret = iris_release_input_internal_buffers(inst); + if (ret) + return ret; + + ret = iris_create_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + return iris_queue_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); +} + int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type) { struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index bd8b74a9fbc4..b4433a629331 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -108,6 +108,7 @@ int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index 22b0cbfcce76..ef76207aa749 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -118,6 +118,8 @@ struct iris_hfi_command_ops { int (*session_start)(struct iris_inst *inst, u32 plane); int (*session_queue_buf)(struct iris_inst *inst, struct iris_buffer *buffer); int (*session_release_buf)(struct iris_inst *inst, struct iris_buffer *buffer); + int (*session_pause)(struct iris_inst *inst, u32 plane); + int (*session_resume_drc)(struct iris_inst *inst, u32 plane); int (*session_stop)(struct iris_inst *inst, u32 plane); int (*session_close)(struct iris_inst *inst); }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 82f2513576bf..8fd4495b42d8 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -205,6 +205,15 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) return ret; } +static int iris_hfi_gen1_session_continue(struct iris_inst *inst, u32 plane) +{ + struct hfi_session_pkt packet; + + iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SESSION_CONTINUE); + + return iris_hfi_queue_cmd_write(inst->core, &packet, packet.shdr.hdr.size); +} + static int iris_hfi_gen1_queue_input_buffer(struct iris_inst *inst, struct iris_buffer *buf) { struct hfi_session_empty_buffer_compressed_pkt ip_pkt; @@ -778,6 +787,7 @@ static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .session_start = iris_hfi_gen1_session_start, .session_queue_buf = iris_hfi_gen1_session_queue_buffer, .session_release_buf = iris_hfi_gen1_session_unset_buffers, + .session_resume_drc = iris_hfi_gen1_session_continue, .session_stop = iris_hfi_gen1_session_stop, .session_close = iris_hfi_gen1_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 41e103744bd5..48d0d8e4dfe5 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -34,6 +34,7 @@ #define HFI_CMD_SESSION_FLUSH 0x211008 #define HFI_CMD_SESSION_RELEASE_BUFFERS 0x21100b #define HFI_CMD_SESSION_RELEASE_RESOURCES 0x21100c +#define HFI_CMD_SESSION_CONTINUE 0x21100d #define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 #define HFI_ERR_SESSION_UNSUPPORTED_STREAM 0x100d diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index cc75231f07f1..8efc6a70a57a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -737,6 +737,43 @@ static int iris_hfi_gen2_session_stop(struct iris_inst *inst, u32 plane) return iris_wait_for_session_response(inst, false); } +static int iris_hfi_gen2_session_pause(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_PAUSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_resume_drc(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 payload = HFI_CMD_SETTINGS_CHANGE; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_RESUME, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_U32, + &payload, + sizeof(u32)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + static u32 iris_hfi_gen2_buf_type_from_driver(enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -860,6 +897,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .session_start = iris_hfi_gen2_session_start, .session_queue_buf = iris_hfi_gen2_session_queue_buffer, .session_release_buf = iris_hfi_gen2_session_release_buffer, + .session_pause = iris_hfi_gen2_session_pause, + .session_resume_drc = iris_hfi_gen2_session_resume_drc, .session_stop = iris_hfi_gen2_session_stop, .session_close = iris_hfi_gen2_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 66ffce02ae57..0aaba5f1ce91 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -17,9 +17,11 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_RESUME 0x01000008 #define HFI_CMD_BUFFER 0x01000009 #define HFI_CMD_SUBSCRIBE_MODE 0x0100000B #define HFI_CMD_SETTINGS_CHANGE 0x0100000C +#define HFI_CMD_PAUSE 0x01000011 #define HFI_CMD_END 0x01FFFFFF #define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index c2e308519fe9..4891a513a297 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -332,6 +332,7 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst, struct v4l2_m2m_buffer *m2m_buffer, *n; struct iris_buffer *buf; bool found = false; + int ret; if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { ret = iris_inst_sub_state_change_drc_last(inst); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 0d904d64884c..33a1fc9b997e 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -376,13 +376,37 @@ int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane) static int iris_vdec_process_streamon_input(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + enum iris_inst_sub_state set_sub_state = 0; int ret; ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret; - return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0); + if (ret) + return ret; + } + + if (inst->sub_state & IRIS_INST_SUB_DRC || + inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) { + if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { + if (hfi_ops->session_pause) { + ret = hfi_ops->session_pause(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + } + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } + } + + ret = iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); } int iris_vdec_streamon_input(struct iris_inst *inst) @@ -417,13 +441,52 @@ int iris_vdec_streamon_input(struct iris_inst *inst) static int iris_vdec_process_streamon_output(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; - int ret; + enum iris_inst_sub_state clear_sub_state = 0; + bool drc_active = false; + int ret = 0; + + drc_active = inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST; + + if (drc_active) + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_alloc_and_queue_input_int_bufs(inst); + if (ret) + return ret; + ret = iris_set_stage(inst, STAGE); + if (ret) + return ret; + ret = iris_set_pipe(inst, PIPE); + if (ret) + return ret; + } + + if (inst->state == IRIS_INST_INPUT_STREAMING && + inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = hfi_ops->session_resume_drc(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + + if (inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) + clear_sub_state |= IRIS_INST_SUB_FIRST_IPSC; ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); if (ret) return ret; - return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + + ret = iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + + return iris_inst_change_sub_state(inst, clear_sub_state, 0); } int iris_vdec_streamon_output(struct iris_inst *inst)

[v4,22/28] media: iris: handle streamoff/on from client in dynamic resolution change

Commit Message

Patch