From patchwork Tue Apr 1 07:35:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ming Qian(OSS)" X-Patchwork-Id: 14034424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E18CC36008 for ; Tue, 1 Apr 2025 07:38:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-ID:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=B2BbTQIwrMUtrZ4MqH1dH97kHKOn4PoBBw6vrlLL7nY=; b=HSezQ5Tge9FTr7WUXZIQQYJ9jZ oewnnHVl8BPQwi8NA0gOSbPwMvDOj5ieJ8ijfS83Giru+ocaFBlB1+wqlJCwKLHFQMw4yObjn65wa KHuCsqJDp+XJ532Cd3XLTQ+jItlLPLmPdVzrWYYXEqxEkpD5+pe0oquxJYOyrPz68e2w5e3rdngaP MT47NKX6WJlSwtbVcHAYS72bieUiO6OTFid4LsIt0yjAogNusx6k7yGGFgB3TupnC40xDkl2822lZ SbeCC/mxVYSfYcgpevOBp8wnwZU7IkMKBQgZwOTtzf0tAQfYGDH6/sw+GfIbe7WiEVkqbqfVmsD17 ntu9f7sQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzWC4-00000002Cxn-3q8E; Tue, 01 Apr 2025 07:37:57 +0000 Received: from mail-vi1eur03on20622.outbound.protection.outlook.com ([2a01:111:f403:260c::622] helo=EUR03-VI1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzWAG-00000002Cpe-1oCh for linux-arm-kernel@lists.infradead.org; Tue, 01 Apr 2025 07:36:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Y1qxr76rf0Z+Vep5QCRCZMoheZCAr9mP+ZZfsCZZZJydGhh4M1W5FIZ9xwbs3Iz03t+1Iw+PU9brHV3XTELfGk12cZ8jmo+Gc+Ysh243oi35nu2+J7A8XfcUO2Krki/NCIS9N+qyqmLHK41xJXyFiZPYQw4m1bqmb72GhUwUyIn9vwKhQ50gm4auoksG00AVioJ/bz8Pm7ZHRv4OsfQNc+EMM7IUyu6s8b7E+Tn4haNaVK7KUMVqZSbiDOPyWgcSlgBzIEeft3LWiH7xH/qeuZtufq2xtw6zD1THC0ijXwVrbFYeyBgSnguOlHB9vBkuWDlNCxt2WnpevxGWTkjW1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=B2BbTQIwrMUtrZ4MqH1dH97kHKOn4PoBBw6vrlLL7nY=; b=KLhphCB/Uo/msplNREdVGOs/61yUJGyL0GAF/Gr6azpQZOmtXKlGLDWvmgGR+GrStWB0hffspycms2Ho/mbf2FhD56F3gBCCAay45yG16F3puB7hHU/AHVzo8XNVlShu42gl+dnOZ/7wwXehhy09Ox5ghcP00CVUqn/+68QTH1AuuexBQMn/wmp4322xH02jxGmySCaK1U+aNHKmgrvd6vBB4DkzlYikzbsS8Q7jcRH3A3nrwuMXHDttq9yWNCI4ZGiHyERUhkXVATY5Noa2dqZTa4fkFuFgen11QUdrkNUi4o2iqfLd3AjlBAfzbN0P9119tUXNSASDcTvLZVjv5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector1-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B2BbTQIwrMUtrZ4MqH1dH97kHKOn4PoBBw6vrlLL7nY=; b=kQqG3MzMN6jcTOzc56bfFS2HwpIyKgmLPDGA2gs7U2B8pPiUPlE2Cze6TP1T9EkKEUZdrXrhPZ4H38Poo+EuUi/YA5DzCuNrlg7ej9qcMaMML86mwuXLTLtw+ZF45wn3SYpLFJEAQKwPmAYsHYBdADo5+Id4i1t7yaKbkao5UY7lZlXQgjzgsQM4t3IiSOh0i6jfmep/NtgC1bzW84WnTls5+TmYAZwwa7v5M4UhtsClcCT+jPn4ZLO9FIfl60l5fwzGEsYJmtjWax+xlUMHr37bnNmAuYqyl12nr3drwvHSRZeBItbXgiYZ2iPVEhO+7KV+zt3bb7WsB4LEgshXSw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) by PAXPR04MB8893.eurprd04.prod.outlook.com (2603:10a6:102:20c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.52; Tue, 1 Apr 2025 07:35:58 +0000 Received: from PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87]) by PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87%4]) with mapi id 15.20.8534.052; Tue, 1 Apr 2025 07:35:58 +0000 From: ming.qian@oss.nxp.com To: mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: nicolas@ndufresne.ca, sebastian.fricke@collabora.com, shawnguo@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, eagle.zhou@nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 1/2] media: amphion: Reduce decoding latency for HEVC decoder Date: Tue, 1 Apr 2025 15:35:24 +0800 Message-ID: <20250401073527.1626-1-ming.qian@oss.nxp.com> X-Mailer: git-send-email 2.48.1.windows.1 X-ClientProxiedBy: SG2PR01CA0143.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::23) To PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8254:EE_|PAXPR04MB8893:EE_ X-MS-Office365-Filtering-Correlation-Id: cba32e3c-aca8-4842-d37b-08dd70efd81b X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|52116014|376014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: X25uh0WlUxTw5/TsBl3wWv4K8cCBC8g+KNiIS0lBZ8QTVQs9B77KTfKptd94LkErM2KCbmmrPsLCFHuY1wLLt0DqK4zo2xE9XNzWWnDLSgKoDSw9lmfuK2gIRTWhmuTV/whdcm7PPHxENjtKgjCCsnDiXcfQc8nkrUwHMUNqr6bbUE+XiQyllbqUVYkfIDGkr4wSc1X2yyeAVARlxAcfirLwSC3PCHwW3xECTekNkeUqzp3D6OiY+agQzISLDUUhTIrtgOjrmYAbcCaD8+wKt0MNVbBLYTbFrSAFXepSyKvCQFUHjb5FxHVqyXkTkP+qptOzfqFNyadI2IszpMW130xt6lWIMRiQmAHXBZil3HRGEiVTNTDZ09YM6XejCruqTXowuJ68fZKCJehnSmv8MeX9U77oHODRqNMQ5Np4X8J9g1tmRoy19PFUZ7BJUOs0yvdaz2m1nJhMetBe2iQLNsxGXn93QtrHnOQoltbdgoOM0P4O9ExhlFvWVctLstm3VBAeF4/U2c70EC9HsR/idps1ISXHKThm9HO0OXU5AmN12X9ht6g5GQ8as/Wg/HE8FNjgxltY6ZYwkfPLYbUpRARExCK1yvsEv3+jgh3bLM26qvIDgRU6u8fDngGFBIFl3jiCyYqV5WVkLW1bm2QYmJc+1Q3MQTEydhjUx4oEG61d4pdIYVbcPlCQoqvo5g1ci/xfpFXmC5dOlm8VzpqWiWr6PGIZzSTZ7Y/g6I7+YD9+n+4IJfnPlIqAsVUUYGCGZ2OXMej5FpVU1vDwJrDEQ0Nb9XdIu3P641vyhhxm5y37VDxJsebGWzcUAzx3VZo7d5Kzdkt/yYXcim81sEKEL1ConcCdLKsQzFfWxSJJVdQRrgAs+f7wtsQao+OAJMF79BlHKYvSB8SnL1EZlfErth0mBzNVJ3YXuwNJN59wrzYAlOgs1MsArv9K5yzPQGHtqK9nLdNxpiX/c/jccIhvJcJaBWbFy8+RmigYibE5FUm3JtWV90qO9wCuiKWzYm6dgw9Tkvjw39veFk3DQV8s9oflpuD/i29yzoa4Z09lj0iqrQXBAbWpftXYF/ziVGSdXdG/2Sk3QbgNQyV0KNfSdFwzKdb6zo1F4+NLoLWe9B/LKLpOO69TbdWiKL7nV089dCHdQ9M1dW6bjKwm4+av09di2g2iAIbyW+Jqgls9xyD2LKW1I+zYQDoh1n85Sa+xGI1BX47CfPpMjvGP7Ht4XuIHXAIjE/Hkzxu/dy+dk3v9BkQM9+FX9yFpG5GYTinLD29BAik3mlemuLJ23AZchr0RiBKg2u9nHjlR34tVLwKpAWUgt5ZdI8rPrMMwIcruQeNQ5YMISh9G0ZD/e+jilB7SFhF7afHpc0wZhDDWKDDIXvU6lVCMuhx2DFN683OHPNVad7hpDChmvVAg+uT38WGLy37805hYNBujZWYd8MEHLTFaufmvL02zN6/85thk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR04MB8254.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(52116014)(376014)(1800799024)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: e5xGpdc4aDVdZV93uQPMBCxgLHYRoPZV3KE1iQwWWPEgs6183wsNU/NwJ1FwUvsdhAgprPeGGKJw7uTBcJBwoETaN87iPUGaF1dYNWdgQ+q1eowbSaOCgBqS7OihPuZb0rBNP3Fs6U3MnWv8cZY0TdCEAA7ZDWDJBON1b2PgUPXM/KwK+bd/JwJdw6d+ZZ3D9lMGhPTisg+wghDaRUdRUR0d+Ij5vtTm2ny3Xtm7UZphkkMjPUccx/0zqEGWj8eTX+7/TsSERrfr/MJNbsJifsv6/juLYxj3+6DwZ99ogTfM8tN26kaISwtHMUUP5wQ+AQtGegA/HRA6i1b/EuV8yy84trAfwDs6iEH+3gVvj+ei3DqYbgtHD4rEe/jvbLtOxh9cbxdQmBGcfs3oCA//wTGjaOz1Amm/vselrHROAtBgRz9Nlz454KwrpFyvEKlh12bSu9xOpCFzSnyHTwelb/bogED+z485Q7gAcGZz5XtRwmqKVrW9zbj4rZQi6F6XUpadN0aKHqqzrH32LlHd1EWIOGdLV0En9Vw9Z8MwkDeQIyfBMikgbYx0CNXkRotof60iRUDCM+bIfMhi09gOWITKwX0hpEYf0Ulqc2s9mV15uoQT+DQMhhIElCsRPFmOTSuF8F1urUqtomKXj/vGohdCfKXv7EDpOtk7kX0MFzDbEJsjgnK+tZWo8HTKlvxhXQmVDG0stuLUiAubQTt3xUanDDoGzlhSn2R6aPv/yqgqs98B2Xyq+Kp1z5193qaRdRszCuD/NRGl/Gs4/aqhcnxwzVLOLi/5ELNjtTvoYY2LQLNqF7iVol7jUO/x89EVFz7y/phpbk09W9HbqOo9fG62lttGpd/s31ryJ7ZsS3anZyy9Vs+efDND8UcmGJYMJ6KQSy2t1nW0Atkqq2rQMEzQgWfS0xFs+Moq1dwVt1hkeQfQ503JULk3w1vgItGcDmPfnFuqgOFhu4ofX3m56O9Xq8LS3LoN/w6gxBgtHSdOmjYS4t1iYeDGc6SoKuIZS9sHMo0yDX4X/oOihRLQM4G+0Bswg5Uu/7rlrM/yVFoYZvZd6dBPPigKq/0vVN8nKxwGg4d4nbxK6HHkI/RFzGmGkvrU2ggDrqXOqRKHaXfW7H8618Zo33CL1mkkIr9opeL+EfDrjGMC6DL+NuoRTmloE8YXndHG5ffVN2dWkiQw77Jmjp8ywKt8/LdLOrWrNKEcn+4r/J/MGeAUCxqkpjCrgDEFySCpIRJWcHPzZEHaI6AsGK5BEWE/TfkGF2t3Buc2ZAqFvnx9FlSJnhAy7hZgciMg6rtS3Aytzj0UnKbcuQQGcYF74KM7S2rA8fX8lVIW6PeB9dwBAcx/GZH7MxR5vUexuPDgvklcKsABr+eD75VhAaxUy6ty+vLiI/6lyqgheUav0hxtqUxpQgPBw7oXwg1I/rh1CzVhA2A/t3XXKtKSdw+EnbPB1kwrJybroLEZcemnXzgq8QU8ti0TRZCkfJP425hxH9GVHDgzJX/6Kyh7cdg30k4pV6heb+f4uBSGt5TrcCHar94ZmhZMMHh+2b1P/yr1N5SoJ3QtYGBKpF7Yxqb64UCAJUxqjP/q X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: cba32e3c-aca8-4842-d37b-08dd70efd81b X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8254.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2025 07:35:58.4149 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ox0uIplFf0P8pmXSSZj51CsbV3YD9JuptlUgHJ7/xzfvwdlu9opcKvjiFeDdI+0dsPAjqIVWl3C36OyUF1Qxlw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8893 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250401_003604_481772_1B79AC03 X-CRM114-Status: GOOD ( 15.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ming Qian The amphion decoder firmware supports a low latency flush mode for the HEVC format since v1.9.0. This feature, which is enabled when the display delay is set to 0, can help to reduce the decoding latency by appending some padding data to every frame. Signed-off-by: Ming Qian Reviewed-by: Nicolas Dufresne --- v5 - Apply FIELD_PREP() and FIELD_GET() in CHECK_VERSION v4 - Add CHECK_VERSION macro v3 - Improve commit message as recommended v2 - Improve commit message - Add firmware version check drivers/media/platform/amphion/vpu_malone.c | 24 ++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c index 5c6b2a841b6f..b6e4996c2d91 100644 --- a/drivers/media/platform/amphion/vpu_malone.c +++ b/drivers/media/platform/amphion/vpu_malone.c @@ -68,6 +68,12 @@ #define MALONE_DEC_FMT_RV_MASK BIT(21) +#define MALONE_VERSION_MASK 0xFFFFF +#define MALONE_VERSION(maj, min, inc) \ + (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | FIELD_PREP(0xFF, inc)) +#define CHECK_VERSION(iface, maj, min) \ + (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) + enum vpu_malone_stream_input_mode { INVALID_MODE = 0, FRAME_LVL, @@ -332,6 +338,8 @@ struct vpu_dec_ctrl { u32 buf_addr[VID_API_NUM_STREAMS]; }; +static const struct malone_padding_scode *get_padding_scode(u32 type, u32 fmt); + u32 vpu_malone_get_data_size(void) { return sizeof(struct vpu_dec_ctrl); @@ -654,9 +662,15 @@ static int vpu_malone_set_params(struct vpu_shared_addr *shared, hc->jpg[instance].jpg_mjpeg_interlaced = 0; } - hc->codec_param[instance].disp_imm = params->display_delay_enable ? 1 : 0; - if (malone_format != MALONE_FMT_AVC) + if (params->display_delay_enable && + get_padding_scode(SCODE_PADDING_BUFFLUSH, params->codec_format)) + hc->codec_param[instance].disp_imm = 1; + else hc->codec_param[instance].disp_imm = 0; + + if (params->codec_format == V4L2_PIX_FMT_HEVC && !CHECK_VERSION(iface, 1, 9)) + hc->codec_param[instance].disp_imm = 0; + hc->codec_param[instance].dbglog_enable = 0; iface->dbglog_desc.level = 0; @@ -1024,6 +1038,7 @@ static const struct malone_padding_scode padding_scodes[] = { {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, 0x0}}, + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_HEVC, {0x3e010000, 0x20}}, }; static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; @@ -1058,8 +1073,11 @@ static int vpu_malone_add_padding_scode(struct vpu_buffer *stream_buffer, int ret; ps = get_padding_scode(scode_type, pixelformat); - if (!ps) + if (!ps) { + if (scode_type == SCODE_PADDING_BUFFLUSH) + return 0; return -EINVAL; + } wptr = readl(&str_buf->wptr); if (wptr < stream_buffer->phys || wptr > stream_buffer->phys + stream_buffer->length) From patchwork Tue Apr 1 07:35:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ming Qian(OSS)" X-Patchwork-Id: 14034425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AA73C36008 for ; Tue, 1 Apr 2025 07:39:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/zUFhkjN07lKFNy4wSSfsWq7x4iV6wJvKuoBP4MGc58=; b=QbUT4eI1MncFvzA24yuK3TQpSj U45eqHmYexVvXr/FPOiBxSNVwKGGCbyCm/pKYDj/a1Y/1mOiYkYiRvbc5wskzaq0lRvgPmfZCMA2L 0XjMVs8o7sPubk/joiofOor5K5CHfsgvkAf+3xj2NPpVku2rW06tYSmhRKU3pdLSAk0TfzVwulknN 20HA9RsO2SSWcDg2m63IWa3a1aukkRlP8PVpK5Nme0VozezrfYT0J+HNF9g51Ed28u8k8DEjXSdnc isBzRdAJFXhxcWezelQ+10prWMf1FHszMyNKfX9kUhkNVpbvtWyVkyS4XOWbWOxCYk/EC9Z39h/fB ZfwOSxCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzWDp-00000002D9D-3QWU; Tue, 01 Apr 2025 07:39:45 +0000 Received: from mail-vi1eur03on20622.outbound.protection.outlook.com ([2a01:111:f403:260c::622] helo=EUR03-VI1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzWAK-00000002Cpe-1MhY for linux-arm-kernel@lists.infradead.org; Tue, 01 Apr 2025 07:36:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RxTvaNZhTvfSPhcBJf9dbXxaH1paZUCEVz+/sV1ZE+uTPUFdwiF+dn4Y6v+0Ckk29m+2LgmfYOlK0bUHrVXPjwucRXgAqMyCiaTtNSanW8SSk665PqdCC/B/FTGdOkEVy/dasN5ejcjIm6Q/onLU7ftMYBUhBq+uedM6XXG6BVd/Esndhd0kJZ4xCZgxZXYvMI5FUwJV8vLABjPAztjckLGqzdj0UawSoHmw1pSyhlAt/8LOlt3Ou5n6qTXUn8wusIg1CvLLG3claktbfct5c4wFUAjBGenXDqW23tVnj4vwu4+e1T/7zXzmJl7+wRsqMeNBCahmKMGfBUHFdhxq7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/zUFhkjN07lKFNy4wSSfsWq7x4iV6wJvKuoBP4MGc58=; b=vHIo7uNS8LapY1NOCAM0XqXWi7pA2UPSO+aG7RhVFmLw2k0CXHeYBCmnyrXaVqWNBRGv6LuIIlg4hS4+WhiBHfmGGOh3xI5ytgktB7LZHdgRJwH7UMxdyukI3j+7gP3+Phxw7xZ5+z1rb8iKmH+ZJocijWL8bvve+KuBdDNltV7/QuLqF0kFxhjHaFyPbJH32VCa/w3opuLD2x7vvHC6/oBYhhCgfrsnHjJ5NLHX6D7A/mNK5D0DCJegGdzTCaUXzv7mDnbthMraTjn34mTxIECFczyjcO8EvRrV2vN4lwsklvYeY9t3RAM/Gs1Sk5FarGSgwv91Fceiy8CD5UqamQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector1-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/zUFhkjN07lKFNy4wSSfsWq7x4iV6wJvKuoBP4MGc58=; b=YC30kJDmCbQSt06WhlJ1L9slZDrudsuJB5vi0oKdctdCVzo5tZ6R9412QfXbHl8ww7lEqO4aSvCU4ZpnSfvbwc0BAwybD3/7shJvWZ/1WywW15Lk0qE6LcWmLglDDimUqSVCvhuVb/0NYdz7f3v7o4yYcP4Y60f1raOAYoiM17kQOro1GZZF8WTOuSZH4w5pIU+mru1VUCb/IMH6agLjDsw73HfGWIBph5pi993R3y1LH4E/DiCFD/ey8E66YjO5/2I0TkzTp2Z7Dm/IyEC9PIHuDNcJeGDXJlu2+C091zOxAX9/inMuA4NM29lfHsfnvWUZvyIOpiBt15abcKfo/g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) by PAXPR04MB8893.eurprd04.prod.outlook.com (2603:10a6:102:20c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.52; Tue, 1 Apr 2025 07:36:04 +0000 Received: from PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87]) by PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87%4]) with mapi id 15.20.8534.052; Tue, 1 Apr 2025 07:36:04 +0000 From: ming.qian@oss.nxp.com To: mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: nicolas@ndufresne.ca, sebastian.fricke@collabora.com, shawnguo@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, eagle.zhou@nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 2/2] media: amphion: Add a frame flush mode for decoder Date: Tue, 1 Apr 2025 15:35:25 +0800 Message-ID: <20250401073527.1626-2-ming.qian@oss.nxp.com> X-Mailer: git-send-email 2.48.1.windows.1 In-Reply-To: <20250401073527.1626-1-ming.qian@oss.nxp.com> References: <20250401073527.1626-1-ming.qian@oss.nxp.com> X-ClientProxiedBy: SG2PR01CA0143.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::23) To PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8254:EE_|PAXPR04MB8893:EE_ X-MS-Office365-Filtering-Correlation-Id: d4111d9a-db13-43f9-a380-08dd70efdb88 X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|52116014|376014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: RGRWc35WjRckGxvrqIkVQQo+FRpECqSyOEYpBqYlQQ+7T6hQdE/ItLEIcfSAxJa8h4HnDCWA8gnwIYbxyRZ9sxOYvUvlM+J0phtwiNSQ48zYg6p2+pgDPF3NA0KQRvlRpOArd97Y82FAyXpgbC9Ht7lin716sxN6pVffIwdPiR7ecsOcX/tH1X2T6Ky26S7QOYjyOLGHaEACfvy6u6mWS19dH9qNDMQ24O1Z46ilt6+jRwjqvLvCtJ8g4NVlDqWnQ68afJlB33l07hnl8pXSkLu5jXl2bvsSjxINZildE/+SbT2m8ACejYmafajTDQMYPGfysd2qnsi6tm7CipU6buTy3GDt1hs437jdZ6I0u27skKPYxsp7T7UogZ1DlDX/4PqvtIy/12Dm+cxmccEotgETvVUEVKp0AGhpQLJ6xFjFNd56/r+xTpdQuphvmEzQaA9jDrVstfWv9UT8rJOHuaNY9fAlUP9jDGanei5yqeaZH4OdL5jes2CAiPN9S6cR2qPW/pofC810pu/q1kMAg/OZo1WNtX9ukFJfzOpcecZtnsh3toODJKvnF91lqK3OLgKgV0wsF1I7JmDLsaYVu60aB1ApWxIKfwEpuiQv9mj6jLDbguL975wfXmUfzRYcsPAK/whwB6Y7hzgrnL4zY8ytAVGeybSAQY0lEBoH7xbLvJg2WG8kVi7tP088rDfA0DK6pha/494jNUW2R9QhxkWPnhsML4A/pvBNBwgkyBN/u3t22zY9kjxd9tsZu+9kbCZIWEN9aYSWoaBE7XBeR2c6zg3H7A1u/yCmMGY7OSQjzBPkuLMk9+9UINNICyEfq+ObMscpl2hfppFbf568PqH6MCjUfA7mNB+G/9UN5rTYgCLIrhrgiWf9EZ/5yBJtXXzRMA4+viQ7AD+CP/VZ98SyOjKpH6FH6VlUVLE9dYtOCO153Y8+6XfQtwtBSfLTx0GFWsMV1tNnXEQ34QPX4BI41qsyqPcrinrdXMPEKgFgc0mjImK8M6f8BYJ5KT/Xlmx7ZRsBvkRtplwO5d+IOfRwPXyCOKUhQnyk+MtknFRX9oxYV4x5AXj1dG0+Ob6d99sxWnd+2R1VqhbrPOk/EZA/qJvssmjtDGVjjWQuDT0+5f2eRRZ7gb7Dq2G9r8Fnt7qlgCpm0qSL6Dp8khLk6JjBLQ0RMTm7/0SUsd2XV03ae5hCb0fDBG9rTWaXvfreR2M+FrE72FIv0f26J1s7PvI1jMxsm2+6qiTLc7++2e2/Gb0MxxS34yl6BP2QBUZNb0xLuU+p/RJGx5Wh2kFkFIYHmUvH5HVm7lGfrQAgFulqaR3IgUUo0pfAOzUsPUpe6gvFF/rWgMA4/+pf6+kPrldL3G9Twm96FaMaqtKB8hfODj06SCka2cZmCTacjIznrtu/2BqcTuK+sq0viL3XTEBag6CR1jZPz4erCkEjBSuTbaWSNpVBTK4i+zSZHIxP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR04MB8254.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(52116014)(376014)(1800799024)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OyiEqJWdg7XDwobuDJ6iI03xl4Zwg70lbv/lBlcnJlzrcqLbBRS0fUEjnDwiHaYJWshuW8NJPrCHBLYDEUAjkXqglVGGovmD1pcPNGUQUv/O25lKyuSZGo+POyjwnXRZ12tT3FtWGuw67diqBgEjYGMREHzJm7BqtFzCBD4OZp/jYSGnL15VTf2pG8eHHOiCPseNTZbMMAylt9ZvOOs/a7s9zqYv/2U/3kco9iBiSj3E3YeZzlfEFxrZFLK7e8JVJEycwMF5gQNGqmUrRFtgPA5vi7LVl0R49P9p7DjC1WnCdKsEna9DkWCPFdB1KR+27MCm4P7P4HfGhT2TeMgmJKUv9Xn7yxJ+BPFPnRRWnIdLTh5aGmWxXEIovC7h2mKTXzVyyBfEjd6Whkpv/Fj67zkiGUUCD6lAwgfXAHcXvTggywXWaoTo3l/9J3lopWsAoyo7V5cC7LIWpXAJAEIDqHA0Zr6UsNHXSYiWpNQcPGxONhTcnMbymedsfbSeyOeamQPN2c+qkQhIvatXgNDYa+Qxjt5WLFXj0m+flp0LEG56Grw0AuLTkNiNJZzoUrsWu//RydjpsEcqkVmefyAC75UTI52CqRhC/pLNn84Y95aEzB3hXvP4pe3D8wMxrEtRTPTBJTRwF5Fidg+Ih0eOU58Aiyez/uCa3NsGeaT142gwTdODr5lO1t6wXeaxRrIwSH8ZAGDyESn8wdeXcI0XbXPHNfCKoRgCVaBMaGrv2EG5IiQuSLPIG2VU/WWtQPlwo2sQ/LnvqCtFOSM5k5DuzI839EKjpDAVOhhGDcQ5WH0vLLvk9wi8kHT5ILA5Km4OPfQAmV4WNUwtTuB29YwspkaeZk7gffLCaLv7ouacZog98qmkEM9Aj6HEs06Ipyd4ZaIJvfTruIl+A8Rn6IxFo6llerHCRUHzP1dfJ2i+kIDv1m6duWHppq3T4udWle95rKobiR3fHxQ143/nChOAZo/Onum1bl1X1Mki5Fz0DN+b18hdTYTCGMzajSa1/zHPkbz+BBvHdXoryZV7CpieWqcIvsdlvl7mKfWDzt1ZLun2nYM8S+pakSYlOFS3EHY6/ExzWK+rKqFqmZJ5HIcX2zMgfETRaRE20OxfnsZOQAf/cIEYdVfCUQVfurAHe50gyggbddNGEdBh3cLs8Dx1B+aIftiIi+4g/yRW1GaHaLhzqrLL/aHoKRnm8KxjwwsH7z/m2Lx1UJ+zFnSMnplCQhtPhnKv5l33zIB/zJGnUAqhr/d7ygv4x3P3duZy6Hbn1MQgehfNkqkhL7R7Qb5sgPqoGHy6kxURVGP0ObNBP5rHAGfC+EuDJVDaYkJi+5zQiL5mEnxDpN8BbHPgFSJ5a130zly0eQFjoezUx9itlkzcnPXzBNQ5be5wWiIkinqA6mxpoXS12b5CJj4zNIt3FYealzrfbhp8Ouj5jgzW1VlCpAhvboBEBkOiQ5VkrlrsoF3gBUHZA12ZN5uCjQjnqzCkbcBgpsHgn2/CXB2xbndAu/8YuMqDQAAVA01PGgwqV9LycWP7WCjWmAetuvq1A6a4T2pvNlrqUV9LGqG5N4kCoJH5T2tjc9CicxpbmwHR X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4111d9a-db13-43f9-a380-08dd70efdb88 X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8254.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2025 07:36:04.1878 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mV7Di2xPxyDIzncScAH+nz5G+WufDw6BOSlgSkCY2mmXiunZ3xplkIGqgtzboTVVCwatFr+CSTegKwbCXEiT7A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8893 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250401_003608_367889_77BEC39A X-CRM114-Status: GOOD ( 15.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ming Qian By default the amphion decoder will pre-parse 3 frames before starting to decode the first frame. Alternatively, a block of flush padding data can be appended to the frame, which will ensure that the decoder can start decoding immediately after parsing the flush padding data, thus potentially reducing decoding latency. This mode was previously only enabled, when the display delay was set to 0. Allow the user to manually toggle the use of that mode via a module parameter called low_latency, which enables the mode without changing the display order. Signed-off-by: Ming Qian Reviewed-by: Nicolas Dufresne --- v4 - Improve the comment expressing v3 - Improve commit message as recommended - Add some comments to avoid code looks cryptic drivers/media/platform/amphion/vpu_malone.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c index b6e4996c2d91..89af89087e2b 100644 --- a/drivers/media/platform/amphion/vpu_malone.c +++ b/drivers/media/platform/amphion/vpu_malone.c @@ -25,6 +25,10 @@ #include "vpu_imx8q.h" #include "vpu_malone.h" +static bool low_latency; +module_param(low_latency, bool, 0644); +MODULE_PARM_DESC(low_latency, "Set low latency frame flush mode: 0 (disable) or 1 (enable)"); + #define CMD_SIZE 25600 #define MSG_SIZE 25600 #define CODEC_SIZE 0x1000 @@ -1581,7 +1585,15 @@ static int vpu_malone_input_frame_data(struct vpu_malone_str_buffer __iomem *str vpu_malone_update_wptr(str_buf, wptr); - if (disp_imm && !vpu_vb_is_codecconfig(vbuf)) { + /* + * Enable the low latency flush mode if display delay is set to 0 + * or the low latency frame flush mode if it is set to 1. + * The low latency flush mode requires some padding data to be appended to each frame, + * but there must not be any padding data between the sequence header and the frame. + * This module is currently only supported for the H264 and HEVC formats, + * for other formats, vpu_malone_add_scode() will return 0. + */ + if ((disp_imm || low_latency) && !vpu_vb_is_codecconfig(vbuf)) { ret = vpu_malone_add_scode(inst->core->iface, inst->id, &inst->stream_buffer,