From patchwork Tue Oct 22 08:40:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Qian X-Patchwork-Id: 13845319 Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11012057.outbound.protection.outlook.com [52.101.66.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEF6E27735; Tue, 22 Oct 2024 08:41:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.66.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729586471; cv=fail; b=SMe2YwRsarMqr3L0yhi7OUZ155idxRuVxThvoaIKNQ3e5cnkY7ZeZyK1B8r/HpVOkcBbNum3axfwCwIc1JK0z5yvpwziNuTBVJa8HHzuw6aDCQJxR3ZhPr892clqCInTiWLspbXWBtSbtOrPCrsJVf2DtqCWSTv6KtmrGZK8sG8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729586471; c=relaxed/simple; bh=FOw6CxVHK0jpzdLXOkr9j7EgRq4zlzREECkEyZnKMRA=; h=From:To:Cc:Subject:Date:Message-ID:Content-Type:MIME-Version; b=d1CGXxWxfhHXXxy+QINGUtHwJjWxhEIT32/aEVcYaYSO3yqEztWh02FHeLP9Srq5JZLLwmyZ/iOyXsu994izkqFmnhF2K0446J5gLxoYq0UhEaEyfIIdNRGQwdgMHAwe8N3v6X6XkJKDDEh8ICqhluh62K7ILxeHHyBobYr5tcc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; dkim=pass (2048-bit key) header.d=nxp.com header.i=@nxp.com header.b=KHTc4qQo; arc=fail smtp.client-ip=52.101.66.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=nxp.com header.i=@nxp.com header.b="KHTc4qQo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qa7uPxqLdyw+M8tWe2gCmrh7nODRwCuIqRd+1SZL0FBTGs5wD/lFPG4cl3UyE/uEQZ0DJfilI18lMLIgDwOmp7W3Xb1UIhAQPy9yTjVPMLBgquu1FWSYWf5ToJ1Z6Y8VKq1KhiqkAzbqRbKBxxgaBtPJvVMEQYd/dxqO6uHXd7Rdk5qhvPpamyNDPTegFY1OhMaS35klH1bH1yJ+B0sqJGtbQDRBE8GsiPq4RC3zbhHWHAadzaWYPgtb931liNIP3kas78+mGBWA9jxhdBUYAaI4aNA6wLmCT3SDci6u4i+UqOJwQ5IembSCami8tk79asZ7kjoJuBsWa22FZRk7YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/PzC/SLr00KRRMxUbP7bMhLcwPvcGunnB2TADIWUYAw=; b=PzFDj+S6nKVLVpx5USbucYJMR7DrXQhDkXyrObF12HDdX3Dtm25OqUxx+jnAqsgfARvlNsFoOWElmV/9XpM4SmaWD7sfhkVzYDZCDsqB91xdArzGo2JyjI6UXP0KU1Nt8xV4S3Dq7zMTGUw6xeZLoLPEYiu/MIT1C0njhWW7A71Vde0M/hfXEjVRkIgw/vy1Oi4lcT8rJdyR/VBNNIjh+53q+14xHibXv4zGeZqJzJs/+zIbFhgNDxsC5KLzKyWQqD/W3W5rl9sHTohi5NQNqmRNE0Nu09vBW5BDOI9Y7m2TA67wR8WbbLU3S24Uv6bbYoyD/KdjrQAK750mNgInDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/PzC/SLr00KRRMxUbP7bMhLcwPvcGunnB2TADIWUYAw=; b=KHTc4qQordKiBR76aDiodeOnZHNXfxMNzBtYFEvT3MonDH7QdKFkM5vFYy2V85c7cZ4mxrOLx1+6+9llRN4vJQi5cqGXXT0dsbZZShUQFlxpoN1jmq1RDoMiFmfF7HXCcHqmxw0Nl3M8djs89ny/w9noSeBC5vYWD8KQJW3E68cwrF7dBYI3RzlVPZL+m0MOdCFTJP9OZq3f+Iw3zXaa1fnadRIXqTVa0XxmDwXNn0FOOUFdKCdOFCFlZKKClEmD6jp/AqNcP6zGTeWPVxT9KVLG058EknkYg5j3npka4TUtuh3VyQir9LH5zIH+9fGF9kImjM+QUBFFv5S/9UBX8A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) by AS8PR04MB7831.eurprd04.prod.outlook.com (2603:10a6:20b:2a8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 22 Oct 2024 08:41:05 +0000 Received: from PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87]) by PAXPR04MB8254.eurprd04.prod.outlook.com ([fe80::2755:55ac:5d6f:4f87%3]) with mapi id 15.20.8069.027; Tue, 22 Oct 2024 08:41:05 +0000 From: Ming Qian To: mchehab@kernel.org, hverkuil-cisco@xs4all.nl Cc: yunkec@google.com, nicolas@ndufresne.ca, s.hauer@pengutronix.de, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, xiahong.bao@nxp.com, ming.zhou@nxp.com, eagle.zhou@nxp.com, tao.jiang_2@nxp.com, ming.qian@oss.nxp.com, imx@lists.linux.dev, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v3 0/4] Add video encoder ROI ctrls Date: Tue, 22 Oct 2024 17:40:36 +0900 Message-ID: <20241022084040.3390878-1-ming.qian@nxp.com> X-Mailer: git-send-email 2.43.0-rc1 X-ClientProxiedBy: SG2P153CA0007.APCP153.PROD.OUTLOOK.COM (2603:1096::17) To PAXPR04MB8254.eurprd04.prod.outlook.com (2603:10a6:102:1cd::24) Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PAXPR04MB8254:EE_|AS8PR04MB7831:EE_ X-MS-Office365-Filtering-Correlation-Id: 5314754a-a696-4257-22dc-08dcf2754478 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|7416014|376014|52116014|38350700014; X-Microsoft-Antispam-Message-Info: RKbrb9S5gwdvBATclWWU28iBMrpZfXoP4XDv+bZypqUHLQ+Te31/QIb8dR32tqnqVyHk6tdUjIjWIH9VJJ4ZUQ+mCSXKBiqeXRYbnKK86nB6BhBuxY6bORafzBRNqAwNA5sLqJxvYnDkNQ2MT9N8DAZ4S6udH9qAcro6SaahSYO9ZCyNQdVnvXBDJyFt8MGChNEdXf3dJ7J+Xn/bNBl+NXWo2+K9x7SNasomWAOfTNQlt5EuG6YKG8Z9XCDO6VvHuzpsOnv/VwBfN40F4WVRDCUM48hhmoY2GV621dQ4f7W93U4zMPDQB04yWmN5YFbhfNIir86Oou3jGggEMI1wp0rCRWAeAxNkYUgBYxpEi35aveK5mLDVHrjPJYxE5hWWNZqJgHIrvdPPMnTATZbpo0WG5hDWxyvSJkxBejKNw+BCf6bDeNLKpD3ZZ27/WgLk5AkJ8GI39DjMHchMo4+K4HMoHOEtXYkEakCCyQ27BBcs2JqDX98ANvlU+UK8XOOcHDDIC16B31u0qUBKV/9W8/8rELGsyqtXMglUcApo9a06xmI5WEL4aau11mWPbm71JXJfBLqYseKkgEj8LyPZIPyRCnvmaOIapH/0u6zS5qjD062sqAxaxgwargbzHf9/Z2IjD7e2zr8V5hF34dPUNoH4BpQPvvib5RSb9eJU/jSGA6qxdu0xDCSQH8RFd+HhubeRtJ3wynktKBRl2a3dyIwyEx7nHdTNUGFG+/wgaNuWnYQ/y4RfIh8LSA94w6KLFFFYxBXIHf32lZ9eRVvbyMH/My8pBcMm/0eCrWgxaDugtst5OZ5czsccY+nmjlRqh6J3/p854fcY4NLq+BXG45CjpJUtqbNns84u491xNtTNoteHYfNpWwwTIxOMLFivO8KZYdum2u5CVSLjPSDoptRsiuacQqGil2WMZ7dsSrRlxPFB0MpKpyI1dRLC+Jhwh3uhM0BZnh7fVYDuS5VapaEC1SLu7vXcsCMlm7X57E5k7WC3rt/btMjty9Hkqd7Ci81CKzae+DWnT566e1QW2T1ssulcPx+3rcQTmeFKAkGIgQnPtqSe8IBxpXFhVKWjIYlrW9AUAZAWJYlL2rGUdbt849C+7DDr7VCY9eU8DPHqlsHc6J2Rl5bSkkE1RUtpOQTomzAlKNvShGlIIYAgCQGxupGDJ0Sox00BaSsGRsHOaw5LGMaDPO6pQiHYPZHcrhMfqExe8Cpic58EJ+GcYsNnBccRlv2WU85gqUO6XXTbim8xO3R8txAgZEnQkjVOPJuVyyFzT+6/kr1mmkv/lpwfRh8YMlntak3oPpgZgQdr8FWp92Npb98S6m2P+Aa/+WPyPjtl2zgl0gn69BnzU7J4w4krLAIkURk5tO/OF1UfwdzXxwP4H87nXMrZY/Ce X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAXPR04MB8254.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(7416014)(376014)(52116014)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CfLkZghtHyBanKeiCc7jdz/XRilu1NWMRUuuimLP5lzXdcScAhye35BXa5z5+e4RkLwG41F9Ud4AHP90+ke+mmwzC2tAaVzfCIQfH9RfMybo3EjNtQnfE2R/rdWvjZvk+XZOLgELFJr3TNSjwlRbqEr3sukgK7pPOcJ8H1acYYRgoColrWh1XhkfGgAbYUca9yooQabowTdNPsZW4r9jkwEFGHhZR8Wr6Sf7kl7EUXsMOhhwIXk+rmrWr0cvIH7j40s+akOS+F2m9SsZysfWEJ3G+en9OguBzwgD8qR5ZLz5HzKqpz77SaEUflXKIU8gNN0CULqUiGiKMnUWCx5RTIR+19zuUXPL2sHzFNd7SXDKINQ316isn2fEelH7qiKDZwSX30yqPdBhO2J/46qSLp9+ZJaUDybITlC6coXMbG4/hAjWtRX/QzypxOcpRjV7Cdc3VCCmwqC3bWh+5A6o3RNFm8pPAJleOMMqWP7ZaUuVdZMG4mkPvLOk0FcdwL/ssWgbLTJASnj/JJ1Gt4PQK3vWt81I2SJ0o9eQF5YP4oeaumaW/z00049bqjCJuVZ8ai5MHaDiDWaQZImy5N1d0vy/roZXz35GAtBZ+GT8Oy6FWAW0va1ToZFFJfMyonrs39+PtBsmhl7o0xgHBbl9Hj3EKcXxhnaFaeMxh0u8KA2UlRVjIc+/RDMSMIuo/0J5BSigS30jvCW3t9u1R7zmlbHkg/QspnmH7C27Sx6j4knNZilDMxHBMLd5xT92hMswxf2JVGkX16IwvjYYnrcMljUK8BhPa28zaa9jQw0IoK8F27EWPdvp6J2mdP7bUv/sPL1vamJ2hQYX1WDjxqPacWVM5STUcsSx3XfL5LWFuYHWgiOOLE0Q476Bpa+vHPJrW8jdN+TG/pNC1+r81zOmkL4dc239cbJyhlz0H+egYmw49azz3xPX/r+2guPH0cu9tZr3OE+QS+4rutBNQ0468rBgHqEm7vrmEFIrY3SLeFNqc4PKl1VBMRaDhEEJZRTFAcmYVdwhnxtQNtmqFsIcG+Xf7gr373usD3f1db99+/v7Y4FwoDZoXwFPGea6XHdNwH4Pt9Nb8lufxPYRdAvQcZjUP26wjambCYyUQLZuXafXWTuZbqXqb3nbHftJ8hI3oJUC3FBQfrroF2xfsTRdZsHURCEqOODFwKCYJuSJ3R8kfz4DwVGvqG70v1bDu8VaoecWZOBriCD1b5f8WMzu3hyg/HXJF9ps3gTSv6+t0bGsOQ29Q5Kcnl3X1cydB4UNcPn17K/qh+z/WJN/2zDI3pLPL+yFaYbBIy8LTgBh+gl7YXGXRKYEUUVVMHFuD2Fe2Qk4NYgJXxVGV/zNBVziQK027fi8rl6NiqkA4UIp+GX5UAXFkbFFs7nDRo8Dtx61bQvSur+u/1PrRwkD9Ta0ISp7iNL2CnPyYIhZVJ40drW/GLpw7QmvzHupDhBKVx3TW5tGVqC5jzHlT1GDpbRveHCrA47+rQkSf0AW9anSgoQY74kamGpwcgKil9aBlHmPUH8kIYLqUHtzOm46bd9fv9sQrMqVK7xpFISLdl30hwsW0HaBENKCLwkV6EZCgns+ X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5314754a-a696-4257-22dc-08dcf2754478 X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB8254.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2024 08:41:05.5370 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: veRf+/6lrhIqMODz1i/jFIxgdBKN+/J8vePmyhocQLK45xoP1IMepsyRpOLMiaI6jZTCbt1tX9ID9uCRhw0Rug== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB7831 Hi, This patch set implements region of interest (ROI) ctrls for video encoder. One video encoder IP may support the following two ROI configurations or one of them: 1. configure ROI as a rectangular region, and set a delta QP parameter. 2. configure ROI as a rectangular region, and set a priority parameter. 3. configure ROI as a QP map as an array. Each value represents the delta QP of a block in raster scan order. The block size is determined by the specific IP. 4. configure ROI as a QP map as an array. Each value represents the absolute QP of a block in raster scan order. The block size is determined by the specific IP. To achieve this, I made the following change: 1. I reuse the type V4L2_CTRL_TYPE_RECT that is defined in the UVC ROI patchset 2. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_MODE to choose ROI configuration 3. Define some ctrl to configure ROI 4. Define a ctrl V4L2_CID_MPEG_VIDEO_ROI_BLOCK_SIZE to query block size I referred the patchset "Implement UVC v1.5 ROI" (https://lwn.net/Articles/953532/) and pick some patches from it. changelog: v3 - Drop the type V4L2_CTRL_TYPE_REGION - Split the compound control into 2 ctrls - Define 4 ROI mode v2 - export symbol of v4l2_ctrl_type_op_minimum - export symbol of v4l2_ctrl_type_op_maximum Hans Verkuil (1): media: v4l2-ctrls: add support for V4L2_CTRL_WHICH_MIN/MAX_VAL Ming Qian (1): media: v4l2-ctrls: Add video encoder ROI ctrls Yunke Cao (2): media: v4l2_ctrl: Add V4L2_CTRL_TYPE_RECT media: vivid: Add an rectangle control .../media/v4l/ext-ctrls-codec.rst | 95 ++++++++++ .../media/v4l/vidioc-g-ext-ctrls.rst | 26 ++- .../media/v4l/vidioc-queryctrl.rst | 14 ++ .../media/videodev2.h.rst.exceptions | 4 + drivers/media/i2c/imx214.c | 4 +- .../media/platform/qcom/venus/venc_ctrls.c | 9 +- .../media/test-drivers/vivid/vivid-ctrls.c | 34 ++++ drivers/media/v4l2-core/v4l2-ctrls-api.c | 54 ++++-- drivers/media/v4l2-core/v4l2-ctrls-core.c | 169 +++++++++++++++--- drivers/media/v4l2-core/v4l2-ctrls-defs.c | 46 +++++ drivers/media/v4l2-core/v4l2-ioctl.c | 4 +- include/media/v4l2-ctrls.h | 62 ++++++- include/uapi/linux/v4l2-controls.h | 16 ++ include/uapi/linux/videodev2.h | 5 + 14 files changed, 493 insertions(+), 49 deletions(-)