From patchwork Fri Mar 28 06:48:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ming Qian(OSS)" X-Patchwork-Id: 14031707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D8B52C28B20 for ; Fri, 28 Mar 2025 06:52:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7ljEEqKGIjyiNI+y1MvfBRN/8nfmxQruZGYaWFrDjq8=; b=ThGKpDetj2LqpEjUv7XHunvAmJ lJ3pgnJSGCqB8C+ANQZF+f7KFvOfUGLu0FcrJ6u49Sp5iwj0+MJUlpZ5abPZI9jbjc+Yyddk/JlJB 5xWgbls3G7ziS9Y/nC8nHs1zm3kAAcM6wN1xoTOoyFH5WuKOD5u+5lXgwD0J9lsuqJfPGoRYFvMaG sGQTHywuQjZCRZv5/UuME3NNldRF1VlQ/1AfFiahXi2w+jtgmFrnyvoKNvizR25kTvBi4s42XtMTv 3SUuN0gwpob0zoDIGNsj06WtXpmtlwrriTAmv4ovFNs/TgE6/VVqV8x4chBZSjZey8KCaqGvGFEOF 41sYBjDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1ty3Zs-0000000ClWc-1Ew6; Fri, 28 Mar 2025 06:52:28 +0000 Received: from mail-dbaeur03on2060a.outbound.protection.outlook.com ([2a01:111:f403:260d::60a] helo=EUR03-DBA-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1ty3WX-0000000ClGc-1USW for linux-arm-kernel@lists.infradead.org; Fri, 28 Mar 2025 06:49:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hdIn/bHZVWLF2fTFTo37SSlvihvU4cqTLFGQB/f03WobGLBwmEwgmN6uwZBD4Jm+3HU+IG3/zoOJ0HMFB9SFRQZQSeypXUovnCOtXT3dboIWkAyONq84Ia0Z0iBBEpC+51l9Bb6zAjuXNfFfgm6VAPr0rSQGtmbE+Z0fJfBLbtoEieTOfYHyyFDsLdaYYS+8VUFIx6KCGFfk+5KtxWqhC6yLfiWysyLep2NbqQTWw7YSsk1x4BTlVT/HO1+WLZ+iHKRbDwsCVOA+BcAge3FP20sZGmNnFbuk9S5t/vehN5cG0k+7/oEVGowxu4eFE4X0+qAr4EXFroHiEgXqb3bAOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7ljEEqKGIjyiNI+y1MvfBRN/8nfmxQruZGYaWFrDjq8=; b=JOD7zNSb8z5F3MslqBKIvrVgi8CpuzUVhHE2QdBe8uhjyazlUgljYlPeeZtUi/op1E3rGNHfuBQzGFqYRhuW4ctFEtfXHe6P8l1fH1+Ek/lhZxRJgLQfgctcYGxtL/h2Hv1yGC5kHHOZaTR3BHVKAz43knd8PGzhTGLH2tyzly9mSAOThTirRNkF2CIOHXdJ0kFygI5AkxyF6B7dx5khb8+HACpJyRDaAT+xaXpjo49zVRS+8QYehjK5ZbkrXTeJMec3jR9V94O+8bEHP/2EBtvZinZg1QnKumpOFqj5RJC7ptY4giGAO8pIVdNlzMPfIuRqS9YjPL4vXUYh0cSK8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector1-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7ljEEqKGIjyiNI+y1MvfBRN/8nfmxQruZGYaWFrDjq8=; b=UEgWIsJbf99OTGUk1KuX8FHblXtk8hsQRzApuyzDIzJK7y7CKz9kS2hzxezulXdz+DJEaXUX7nR6Bgn31h34h+Fq4gShW7HtHUL/Wtg9OmeYeXV6XdxdeYz7r7UgWM/HaUR11WfLqMiIkdmYULhzUIDBog3w4LTbXH2l/EpqlRmiKZztq+kQ90Q3PerLCdpDUdk1DSpp0mOfHFr1RnW2g0QuNcMbsA/CLvgt/Eu0n9bGJrLc47LkZryhwoMdoLwBgwnYaQwwuc4RAOd9b1o6Urgjm6fQ1On5jzpcZmpczlv0yPEhfoxLPYMT41y5U6Gp2qMzkZDiaz0lC5QA1tW0Mw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) by AS8PR04MB9048.eurprd04.prod.outlook.com (2603:10a6:20b:443::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Fri, 28 Mar 2025 06:48:58 +0000 Received: from PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87]) by PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87%4]) with mapi id 15.20.8534.043; Fri, 28 Mar 2025 06:48:56 +0000 From: ming.qian@oss.nxp.com To: mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: nicolas@ndufresne.ca, sebastian.fricke@collabora.com, shawnguo@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, eagle.zhou@nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 2/2] media: amphion: Add a frame flush mode for decoder Date: Fri, 28 Mar 2025 14:48:17 +0800 Message-ID: <20250328064819.784-2-ming.qian@oss.nxp.com> X-Mailer: git-send-email 2.48.1.windows.1 In-Reply-To: <20250328064819.784-1-ming.qian@oss.nxp.com> References: <20250328064819.784-1-ming.qian@oss.nxp.com> X-ClientProxiedBy: SI1PR02CA0008.apcprd02.prod.outlook.com (2603:1096:4:1f7::14) To PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8254:EE_|AS8PR04MB9048:EE_ X-MS-Office365-Filtering-Correlation-Id: 17427d67-5c9e-4aa2-4b47-08dd6dc49c8e X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|52116014|1800799024|376014|366016|38350700014; X-Microsoft-Antispam-Message-Info: lgTmzothLyBbNN4/A9coftnwngCVBiIZcLmmNl9qQyiMG2Voy3/TuFCTY0iV+0QJm8e8KnEt9A17HaFQ7BYSpaDZgNtn4LVccbaLeTpc13zP80bB7Q5LgNRalT3kwi8lM4soy0w8G244Gd+d7SoOTusi80hLbLmwfBqFovRMFaRJ/rcGELthRrFu4TdVQzPPyETjvdpXKfCWDU21bcb+RAS/ecN3j/hvFUSB8RBmXzOM94u3pJisKMXZ8xHmTfPu+SG4+NpKmnMCM8VXWdAZCKDiYs0mgLvHecMEZEe/YWGVLLZEwkjv7mEn3z1lJYTKOwXyvlPGw2sxrkTTpWdzgufCynfmD9+KWscYNDZHRbbOrH/CU/jowzpepvsjmN4wzDpyLGEVXNHuqZJ8w099DMvtiT911uRI2SbnolyN0jr8C/Q5ndW5tGewvxINAf32QZcj2cITevQciqetjXzKFnM7XjeoZRbuG5rVe9RtPDw9Tw1vCZAFD5Pb1jUhjHMYhEEA+FMOgtIoYVXfS7ErfwI743ux8zHiK+vwKrRDp0MAOKOG28FjNtZ/wrD1gNZgcYhr7Pzc9fEetTLN5liogEReK8dT1seQIaX4HEvNJ44tq73cQT/+87Hup4DBraugHtxUC59jiLJnlKy+UQKoftltI2H2MQvoyw7xs0mOrZlc/jcVWjzyk00ECgXxbvUY3B+P0CEiFPTTuL2l+sxHZCga5D1VepcohIRQNMl+uszuqk5UvQYR9C7wgQzb/u/DXeXFwg4LoduViML/Ca6yDOnI1p4zm8xrlS3fvJ2ctDL8VYY3xECdCunLPag1ZyzaBwaro+y6jhNSC+sbvf+izR2vOBHkx3AQsBtImOE/MTaC4I2tTfRnDkkL7/FGKdH65moqIX3Tewcu/L7Z/S/9CUjcQ+Qt7XdywjS0vz1c6u41RrKz7+EeIvjKMqez9sm1Y027b0jK2ZOUUXawKwuBUJtiZNF0h4O4DAr26gk/p+a0IinB0zrN7MCfv9t+JqxBhlSiWIRiQCc1OttcEwbR4ZIgLOHtRUS+pISzxnaFNi7RBtaGYj0u/0EsCMwN9xtIF+q21ydtINmfSPU2LgTGZtW8MmgG1KndhEjt5TlrSMmJAKbBLpBsf5g6GOgMLF0oPcZpvuhVOZ4x/8avqMJsSOZ6D5nURHfuaC00g54smwBLX9GRWYRQ7aWFHt7c2phLzR5iZ9TzrBljKnvOnFbr34jjCuvaS2H/3Wl8n55iYdn0739SBBXXplhY4GN4YswEM2vjBbRwVC5ju7ENVTIq88xc5XdEmJZYyFmlPlzM9G2d5fga63Wqk2gvTiOQPGRQ/2Lj2qJsmxo3MJthwUZ39Lbpk3dk09knRMiWq2ASCS/pWiISWEISEgaklWcJ3oGuHXbt/ynfpdmfHlO4ms3mFwFcLoAg8kjlzzmfRmuA7oWmFOosXiHjAYicOQ0pzVGs X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR04MB8254.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(52116014)(1800799024)(376014)(366016)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zKjr7T9qGBTIIgg+ZsnBprmyp63Goi/f7lx4BRWUfcEDxIwr4qikQuQZ+DBa0DKPM/LztwGlMpCYgi+TKNIsGFWe5hmUUUYi9Ojo1w/IrZ7s7eJuxSlClYsS/uqAGQkPE5DMtarpVzm57TNIfoLRvra0XM7fq+KKxVc1t/HrN9eITEz1KAJAU8+CkhgKehZSZefztspNpnVLPG1K89qg8Rt/WJLooEGzuGz79DSdu/J/SMePLPBSLEIGXWnIE9wOHSck+2DylCPNUQJDfHEiAPe4vel3ja9U+/uwdSTNvJWjjxwcX88g9Z4qmgCZS4AcHKrCie/malGMe3JRDfwjK0AdYNzbrvCxi9l4YXs7VuGDd9VHYuRnmqWrrHOXE76UUQJJAgWxacmGy8WQ7KnTszdzZvTz2YMkmDk4YT23zISZwmVP5vTaHKqGvUrQgkbjdZHYcdgjiPwmzGuR6kmKlRRkfP3hDrshPbOmaqKvJjj2kQ0xd61VBAhMrBcpZd5vZNgsnjrden8Gqb0XwoXXs3iEIgd+eF0TJvwqGt9vPwLgXmv2j+La+rhxwrNP7fgYsuwDdUa3sWWsqGI38K3w7rFuYoP/Xm51X5le5TRiC44tq/0PzqCyORSlNEGLPd+9dCBrFL00flb2YcmYzgle/s+qxcBbOG3eAi72JA1bvjbtJGxh9H09mE4Tj3QemL8pR2Lu4L5qUoHn2FjVbbV2B/RlSJyDdml8PM9r68ymzU7sNT/2DqbJXcUIAR9MxgskUr0pNLugIXg1P1sWfVs7tgwlCokxwxx7tUi4nBW+dbMoR7oXR0yIDClpeos0dgZ0keACFVlEQW/4g+8M3t6L3YiZnWTE1mQtCcOHUXqBniNxGEOCW5p8+qeZ0Lx8tQN0rWlrHaGf/za4kLRVGilngNDMOnw9HYBKO8MwCyFOsnkWuhugIRkHYoze1C+T/5HeFtbMkKXJq3+kYwLIzLlGRxgw+wM1K+iBscazYOZo6iScHUgPvgA9dXwFWY8B4gy0tuk6SN3b2e0UKXhRBRs2A2I0q5HH9a3w5WmuLu/BKUx1NFYF5Lcs6Em7gPQNAh+fyvF2xd6yxmon30uLJQT/BNM46FRt6f8zWp+tYH98+IdJVE1f+pgznMKYU5983FWa1U7mHrUKNVx0xXVFOb8gdZrlSngyGHEDGHSM12oWAjeyj8kuAjsINT5vZME4BjLCrkiYqyh3Nk0GE8/fGgSYk7rSwk1natUETprvPqf+oY3OWUMzeEGHYZf4CIqM76w1NZRjDwSTSLWOpCSiWxsIgaG8xKMvXFeRGMJ1Qrs3ZLMDBFVFsKAxP9qjyETAdl+Y7Z5N9nWltC1dqy0wNpAoFB81BwpmqF4UnLsDNJzidJysCzkWqrBO9EEJo5acC9VqQ0Ff4IC4LjfbKVRoxy7S+cvilvot8zwHpuGUJ5N56u4ATCX8d5nTuwkrSWTTWD8Vs816aF8ISWCc79os8ypYJKb7cksM80H/QH5PaEz8O1FAToozBvhaOzoYDJNgDFM3/sBPwesBuke1RqT0ssxKGX0idFrg4jCjiuLd6WabXoaNq7daXeFpgPl9wSOPJrRB X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 17427d67-5c9e-4aa2-4b47-08dd6dc49c8e X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8254.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Mar 2025 06:48:56.6964 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: s7or76MARvzfGCfLo69MgHxqgxt8f3KrtG9VhE7fwyyvB4q8MkaY9fUxNm+utSmwB57zrXCj8UeZuF8Y0Hn6KQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB9048 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250327_234901_396181_BF3B5C0D X-CRM114-Status: GOOD ( 15.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ming Qian By default the amphion decoder will pre-parse 3 frames before starting to decode the first frame. Alternatively, a block of flush padding data can be appended to the frame, which will ensure that the decoder can start decoding immediately after parsing the flush padding data, thus potentially reducing decoding latency. This mode was previously only enabled, when the display delay was set to 0. Allow the user to manually toggle the use of that mode via a module parameter called low_latency, which enables the mode without changing the display order. Signed-off-by: Ming Qian Reviewed-by: Nicolas Dufresne --- v4 - Improve the comment expressing v3 - Improve commit message as recommended - Add some comments to avoid code looks cryptic drivers/media/platform/amphion/vpu_malone.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c index 88f8c16a451e..7f6251f7becb 100644 --- a/drivers/media/platform/amphion/vpu_malone.c +++ b/drivers/media/platform/amphion/vpu_malone.c @@ -25,6 +25,10 @@ #include "vpu_imx8q.h" #include "vpu_malone.h" +static bool low_latency; +module_param(low_latency, bool, 0644); +MODULE_PARM_DESC(low_latency, "Set low latency frame flush mode: 0 (disable) or 1 (enable)"); + #define CMD_SIZE 25600 #define MSG_SIZE 25600 #define CODEC_SIZE 0x1000 @@ -1581,7 +1585,15 @@ static int vpu_malone_input_frame_data(struct vpu_malone_str_buffer __iomem *str vpu_malone_update_wptr(str_buf, wptr); - if (disp_imm && !vpu_vb_is_codecconfig(vbuf)) { + /* + * Enable the low latency flush mode if display delay is set to 0 + * or the low latency frame flush mode if it is set to 1. + * The low latency flush mode requires some padding data to be appended to each frame, + * but there must not be any padding data between the sequence header and the frame. + * This module is currently only supported for the H264 and HEVC formats, + * for other formats, vpu_malone_add_scode() will return 0. + */ + if ((disp_imm || low_latency) && !vpu_vb_is_codecconfig(vbuf)) { ret = vpu_malone_add_scode(inst->core->iface, inst->id, &inst->stream_buffer,