From patchwork Tue Apr 1 09:06:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ming Qian(OSS)" X-Patchwork-Id: 14034563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3400AC36014 for ; Tue, 1 Apr 2025 09:19:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:Content-Type: Content-Transfer-Encoding:Message-ID:Date:Subject:Cc:To:From:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=tjpFCeE8U/vA4Im1GvKDEDAWyr0Faam67XP/kLqCcXc=; b=urHGJEuhwUsq/YuNpBIaJdeoeK fOuo/1LAe47eU8gYDNYQzbPutzSmw5Du7hrdmZV7hEZ+HTRa+me0c8tc+wmhS1OU5xwRX7kEFSScw 0wi1TxtVJ/eqLpnkFxKJYABWB1Yo9Hk0EfbkR+0mfzJUFhNmJD2svi4LXCt+BGgmGC9asVxwyOQqj wfzLlQleuyMepdkvnXCK7Vl8MS2gRDL4VOvPSMZyTcO2lMPtng/YayBLcSFXMV0I2CzsmCm+LHwdI yhwQr/prEUsQheRZaENBTUnENMz86M7OQOSXRT/Ab0PfJN4jf8N7PX7C1byVqtn8waUEbAa/SF3Py nqm6SymA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzXlr-00000002S22-29sr; Tue, 01 Apr 2025 09:18:59 +0000 Received: from mail-francecentralazlp170130007.outbound.protection.outlook.com ([2a01:111:f403:c20a::7] helo=PA4PR04CU001.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1tzXbF-00000002Pt6-0HPG for linux-arm-kernel@lists.infradead.org; Tue, 01 Apr 2025 09:08:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uUUMFeHJdYt90Ir+DED0ZQYebN8UARZnkcLhfRwyJqfgYtGI2lWWoc/V3Cm6OvmUFMacqEKYjG5dy21lF4u0M4R7eFPB574ehI2KHsX7Z2Gm2pVTakgll09OKWUEXC2+4gJ8x2/wXOH6+UOFrYMpavoYmNHX30gSdvP30n9Bl+c0cTuXNUIRm4tuDXwPapf5EM6zmTJizyTxzwY7PjEKOu/Mcr8i+V0h4dxuZbRP/uPUDDoZZj163LBPZcAjUwWP5yirhXlj/f95J+r1TFpPGJwRLPgKtJxxmvgkRaR0m0l8p0AZ3e7ZYN0ZRl+K0dyZuf1enE3gWlz1z51dj9zPWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tjpFCeE8U/vA4Im1GvKDEDAWyr0Faam67XP/kLqCcXc=; b=sVx8FunAm4frbCObWYTHDnVgxK5GQ+KNdBL2KegT8zA4q+0qhneUVB+TZqa205nddmtUYyeqRxvqmd2nosJevvXSeX6n68vQ+p8ILUvBR+HKRMJg7uFHyIwWRqDMNAZ9PvZ9xdQh4QVIeDiImAH+/B+5eD7pV8PBdgwCX6+AjhIsMawSL5rThKkbHGO2mUThOAMckS+B9xmuomqSWoyy+QzvbIK3FGAbgdgkoPpvmofzMr11kLg9eAnxAQkcCRA8tyaZ5k4qEjkFmF8gG7HBbXQdot/OMpgF4wK75xOiaKVaz5WSsjUfB8ES2ZoX2Fnx4aM3V+GLdW1cacJ0TTq48A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector1-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tjpFCeE8U/vA4Im1GvKDEDAWyr0Faam67XP/kLqCcXc=; b=aen0pOH+gGri5uT0Et5U1KcTGRjPh3XrW5kopC/9daxgWL0Aau3Pv4A9SbpOgXxKc800tJ7RBr7G0UYg8B16cnnCvvLI5lNltEKDbaQaLvBRr7JdkryFyXnQ+mnbaJkdQCXUNj3r59zOiR5M+0nw6EdkcUkWp5EygsPOII7W+n0ajS062QBzbBdHpUIxShiL8xZvKwYtyP13nQaHHmOBRiatrGkhFNeDO0fgL0G1Wf1sI9LYpws8YWYQmIJ5UFyxZa5EKmvi0eA86hCQ9RqR8LAxc+OjozMQ/CNG1YVQ1Wm9fnDdmY+vtUHOqoWncXtDDzVcJOJaH3baw5jiMvRkAg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=oss.nxp.com; Received: from PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) by AM9PR04MB7491.eurprd04.prod.outlook.com (2603:10a6:20b:283::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Tue, 1 Apr 2025 09:07:57 +0000 Received: from PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87]) by PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87%4]) with mapi id 15.20.8534.052; Tue, 1 Apr 2025 09:07:57 +0000 From: ming.qian@oss.nxp.com To: mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: nicolas@ndufresne.ca, sebastian.fricke@collabora.com, shawnguo@kernel.org, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, eagle.zhou@nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 1/2] media: amphion: Reduce decoding latency for HEVC decoder Date: Tue, 1 Apr 2025 17:06:56 +0800 Message-ID: <20250401090659.1702-1-ming.qian@oss.nxp.com> X-Mailer: git-send-email 2.48.1.windows.1 X-ClientProxiedBy: SG2PR04CA0153.apcprd04.prod.outlook.com (2603:1096:4::15) To PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8254:EE_|AM9PR04MB7491:EE_ X-MS-Office365-Filtering-Correlation-Id: 781b24b3-7cd5-43a0-dcad-08dd70fcb1af X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|52116014|366016|38350700014; X-Microsoft-Antispam-Message-Info: +MfDk65ardQHGtBdaKKWLDtssQ55eKXUSAlYDpSfV03xTe17KdihSA8LE7E/DSOCb416zXo9QKFSZ20CdosSmNKvOqnWdgI8TTIhficjxiG/ykkDhriDQpO0swKJErGV8ZdrNUKGt4BG+uYk0bFRA9ZIdS/GbYVLNb+dSl0MYp963FoF46xiKUVB0N8+Fzqa8LVX7VW9Ul9LHysOHx7Fz4a407XILJEe7VZqU8LRgi7hYhyrHss4wuM636UGexgr59mLsIteiltT3pJfDRFLUHh5HkZ6BJuZtanX3gZX1swRKq46paabZPdOEuDQOZCqAisTXMlPxkZz2Iv25bfU/OHDVfypfIUdZINy8uxIH6Yhw4PZ9HmcRLDhz4QYpcz2xZYMtB9bIt1qr1W1OvPZoJ0xozSwDq8BSsbOqadPVwDOvtla+aVB5hRpF/tyxaqdGODt3SpzUcc5jqK9uJ4sS4aU+nfx4f/E3O5nsL4CUZho7idw/ITBvCe+QuKQSAZk87bLMrgtQCvaZVyT6GX119zzDvUg5UnBayNoMeYFFTpNtCeAEZCcXkPKiLVTv+vVz+WVvL+WN9o2bhDEpwMyWCg01+RtUQT54EgCXu+ei8bFsxMq7pFm71rMNHzTVvLXgOghbtWCgma1nCrkIAyY4Hf5m9ZT5fdtxt4B38ykiwyQ5ABLluU2Hjob65v1ifD5Twi+xfGsnMqjnp7lEEWU3ueTs3+DCG/b2u2NF805+LNljaBuiLtWREZG4lSLfYtObT3VqsVPccX1XqZ/5An2ljEtr8pznRi5OkHvLrNqN1sFZJ/OmKHhv1OsTNB8mlRnzCGyXMOqU/iXXLHBBP8etYb8s1okrVhlx+mgjDAaIvd3SJ5NYs6Mq9pLw8SIrTjWgJna/PgOTZRZOUbXPPQqUdKcdJxOoPVQHmhuQVhUCdOJjRkB0zT5vIhZ9T62O0IaPy2gEa5ILkuR0QZ+H0wXjmb76iRyo+TM2I8nNIKodxe2e6LOLsQYFvu63lX+SvfBiSVVggn17G5P6dXQOp9r1ax2C/74u6yP6M81KzwueZFx74nO2i9kB4eLgvhx0zFzSzFaOFY3JPmiE3wz4ljZAe4yBRNLamUWNEHWbW59pOtQbjSWNIAX8Ak38Hjsf7spoXOvl35GpHKgaXmxvQPqjPu59iN/A363kwfk8o4IN7XJe1VGIJ2gDNGUpFyQ5PeahN1zl95ubzj5VL/+ajc8YP5TXKUfd/q0ni+jrf2xclbNwHaNjmWVArxlh/Q0Ytet4TQ72xvk5FFJjIc2S22JKbFHLI7DcY+DWaXXI91KsANLGs15ntoJvUbOyH+VTqQvO5msW8z664ZQwDFrs/DJKYtUhjd4B3EamOG3ZSOZEz34Yb/0qP79XjvP0HkRMhjvcmE8CtWxiNCxHtWTEVLh+PtOGcd8V9OWhLbPCD6peckDBtAnKJPe/v5VargzYvJV X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR04MB8254.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(52116014)(366016)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: h5WhZxh74TV7U9TzdA/vFHHwgSseayP1CrgoZ0hoJfhVjn/w3gTwwHEn7JBFwTIgJMdNupxJMgpEw5rIQZsr1vhsDoP21bGWCvtbkFRXbJuiOUeItrvofGJXHGQFxpx39Uv1rQC1yOI7VEtgAC3L45MGvOyqZBXrPNN/YedaQL9yHgBCoswCaJcHfYePidxymHIfIc2vIz7Tfdr4yE6JHkXBdKOMH1d5Nrqm9akISnZpQkjDm0cGkzlDgBUbUzyfc3yKvmgwGG87dFMc1aSjLPbbma1kzcjpMX+ILus3P27xDuQHRYJZ7M7gRtK4jSbakqGA9HvW103fAUKnrh4KPSw+ZToe7JXcJ/CLE4NCe5x7ayZ5d3XnOWscEPZp1IzXjDJKPl7ktmJZfmdrWA3IDsC6yhA+AnpWoHWUt5JQwu7ZFFLPHwvpA2uTwEMDjlHAULJ2UPo7Dag8/BYNWASZAHt8WXfzYWlhOpv8Z0ZnxUxc+jNdkcxpTHAaI/sS+5jljLAS7AcTrGgPuxt2ehU906MCvgs96AkQlp38jR53DsdIxCKOix0uNeIa2t7Cw/RJ3VyXGvN+8ZyAVtiuoxs/59/DFoi5N+wNoVY1ThjZQG3fkMMducKDMJ4T+FRBN68BbKoalZaxTk9uCXn49OIupc1DW7KpEq0Bbx/ljog8IZf5ihOxRONwfjqryZ4BmXEwH2WB+ZUyfyenAoux2GSLtKxiLPkSx2RRWtaMJzCl0zpaS6D9kX5wGgmXxjl6ypK0R8wzMi6cxlU6D/ONVMiXJsEP7o7hogWxRgTAjmDW2mOA7PKHBXCFYB5SG3eWKfd1MT+ya/n0VRI5aekQo/f5ScrcxAHCHX6oJ7Cj5ApNfLJM8oPwLHcAnLNylCUOZj9xasyLUSYsu8Z7rCV1trBbQ19xzyj3SRGrr25w6naylNFI7CFmmmQDva9k4vrJb9BodOWEIPypOSL5HLEZ1aeE+QryohjkR9APTEEAfqsgfrH1d6slOSTIWiFd6afL7HwQxe68WO93YVU7HDfSvENv1i4HwrkjT0VgB8fQf0ml6dlGldS51xOxReFkG11muY0qC1289uLLtr9DbobVrDSZeB6nJK8dgbhl5/FuPaSE8PcLS53hobvRUmLwoLQwpArq+RAR+aFOpjBXxwNFnHeVw1YNpk34yHH2Ya3X9D1uYmSsceUMZv0Rbio2kClFfPckPL8KRUj9A16O/Qfd1PgQdfatpD6mrXZMWlRDvhrEQLjV9AbM43dHw9qpeJESZ37gW53GuBoxPLsyUp3MbKx4CsbIsjlJG2dQ3ZU9RQTI2fD7+9Sy3xxvRvKZTf872MPevRHNrIzFbRDTFuVqsXF2Nav3QWcQ49+eXpIo1+15Hzrbvd84+vHZf3D0ReBSYvXkRzietVoF8kpIDjmXNv1NtETf4Ku/C2aq6qQdggrfw6wnh9jB3VM/7CjzMMbuZvNJGeidazQgoZ2hauI4LZDknd2yyZhu4h/JO+zMCQaREOdawxGpdQhCn4tiSMlDpOnz72oEWFQ7xNm9mNtD3SMOf1WpVGeHzLDvRusQ2ci8SAhx94sX2thOpfGzqW+FkNSa X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 781b24b3-7cd5-43a0-dcad-08dd70fcb1af X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8254.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2025 09:07:57.4592 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pNBPU/nd4GxN6xG+Nh2C6/DP6ROp1rEgOp9sw22gFm3Gbp6sKqG8cTGtqMLi/x5Ljur92akXEym07wbsNmeDdw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB7491 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250401_020801_122849_D9781F62 X-CRM114-Status: GOOD ( 16.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ming Qian The amphion decoder firmware supports a low latency flush mode for the HEVC format since v1.9.0. This feature, which is enabled when the display delay is set to 0, can help to reduce the decoding latency by appending some padding data to every frame. Signed-off-by: Ming Qian Reviewed-by: Nicolas Dufresne --- v6 - include bitfield.h v5 - Apply FIELD_PREP() and FIELD_GET() in CHECK_VERSION v4 - Add CHECK_VERSION macro v3 - Improve commit message as recommended v2 - Improve commit message - Add firmware version check drivers/media/platform/amphion/vpu_malone.c | 25 ++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c index 5c6b2a841b6f..112645b30e15 100644 --- a/drivers/media/platform/amphion/vpu_malone.c +++ b/drivers/media/platform/amphion/vpu_malone.c @@ -3,6 +3,7 @@ * Copyright 2020-2021 NXP */ +#include #include #include #include @@ -68,6 +69,12 @@ #define MALONE_DEC_FMT_RV_MASK BIT(21) +#define MALONE_VERSION_MASK 0xFFFFF +#define MALONE_VERSION(maj, min, inc) \ + (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | FIELD_PREP(0xFF, inc)) +#define CHECK_VERSION(iface, maj, min) \ + (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) + enum vpu_malone_stream_input_mode { INVALID_MODE = 0, FRAME_LVL, @@ -332,6 +339,8 @@ struct vpu_dec_ctrl { u32 buf_addr[VID_API_NUM_STREAMS]; }; +static const struct malone_padding_scode *get_padding_scode(u32 type, u32 fmt); + u32 vpu_malone_get_data_size(void) { return sizeof(struct vpu_dec_ctrl); @@ -654,9 +663,15 @@ static int vpu_malone_set_params(struct vpu_shared_addr *shared, hc->jpg[instance].jpg_mjpeg_interlaced = 0; } - hc->codec_param[instance].disp_imm = params->display_delay_enable ? 1 : 0; - if (malone_format != MALONE_FMT_AVC) + if (params->display_delay_enable && + get_padding_scode(SCODE_PADDING_BUFFLUSH, params->codec_format)) + hc->codec_param[instance].disp_imm = 1; + else hc->codec_param[instance].disp_imm = 0; + + if (params->codec_format == V4L2_PIX_FMT_HEVC && !CHECK_VERSION(iface, 1, 9)) + hc->codec_param[instance].disp_imm = 0; + hc->codec_param[instance].dbglog_enable = 0; iface->dbglog_desc.level = 0; @@ -1024,6 +1039,7 @@ static const struct malone_padding_scode padding_scodes[] = { {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, 0x0}}, + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_HEVC, {0x3e010000, 0x20}}, }; static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; @@ -1058,8 +1074,11 @@ static int vpu_malone_add_padding_scode(struct vpu_buffer *stream_buffer, int ret; ps = get_padding_scode(scode_type, pixelformat); - if (!ps) + if (!ps) { + if (scode_type == SCODE_PADDING_BUFFLUSH) + return 0; return -EINVAL; + } wptr = readl(&str_buf->wptr); if (wptr < stream_buffer->phys || wptr > stream_buffer->phys + stream_buffer->length)