| Message ID | 20230109143846.1966301-1-mwen@igalia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add DRM CRTC 3D LUT interface | expand |
On 01/09, Melissa Wen wrote: > Hi, > > After collecting comments in different places, here is a second version > of the work on adding DRM CRTC 3D LUT support to the current DRM color > mgmt interface. In comparison to previous proposals [1][2][3], here we > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > that means the following DRM CRTC color correction pipeline: > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > LUT3D_SIZE, that allows userspace to use different supported settings of > 3D LUT, fitting VA-API and new color API better. In this sense, I > adjusted the pre-blending proposal for post-blending usage. > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > extra/optional patch to define a default value for LUT3D_MODE, inspired > by what we do for the plane blend mode property (pre-multiplied). > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > for DCN 301 (easily extendible to other DCN families). Finally, from > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > driver, exposing modes supported by HW and programming user shaper and > 3D LUT accordingly. > > Our target userspace is Gamescope/SteamOS. > > Basic IGT tests were based on [5][6] and are available here (in-progress): > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > [5] https://patchwork.freedesktop.org/series/90165/ > [6] https://patchwork.freedesktop.org/series/109402/ > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > Let me know your thoughts. +Simon Ser, +Pekka Paalanen who might also be interested in this series. Also please let me know if I forgot to address any comments. Melissa > > Thanks, > > Melissa > > Alex Hung (2): > drm: Add 3D LUT mode and its attributes > drm/amd/display: Define 3D LUT struct for HDR planes > > Melissa Wen (16): > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > drm/drm_color_mgmt: add function to create 3D LUT modes supported > drm/drm_color_mgmt: add function to attach 3D LUT props > drm/drm_color_mgmt: set first lut3d mode as default > drm/amd/display: remove unused regamma condition > drm/amd/display: add comments to describe DM crtc color mgmt behavior > drm/amd/display: encapsulate atomic regamma operation > drm/amd/display: update lut3d and shaper lut to stream > drm/amd/display: handle MPC 3D LUT resources for a given context > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > drm/amd/display: expand array of supported 3D LUT modes > drm/amd/display: enable 3D-LUT DRM properties if supported > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > pipeline > drm/amd/display: decouple steps to reuse in shaper LUT support > drm/amd/display: add user shaper LUT support to amdgpu_dm color > pipeline > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > .../amd/display/modules/color/color_gamma.h | 43 ++ > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > drivers/gpu/drm/drm_fb_helper.c | 5 + > drivers/gpu/drm/drm_mode_config.c | 21 + > include/drm/drm_color_mgmt.h | 8 + > include/drm/drm_crtc.h | 32 +- > include/drm/drm_mode_config.h | 25 ++ > include/drm/drm_mode_object.h | 2 +- > include/uapi/drm/drm_mode.h | 17 + > 18 files changed, 757 insertions(+), 39 deletions(-) > > -- > 2.35.1 >
On Mon, 9 Jan 2023 14:38:09 -0100 Melissa Wen <mwen@igalia.com> wrote: > On 01/09, Melissa Wen wrote: > > Hi, > > > > After collecting comments in different places, here is a second version > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > that means the following DRM CRTC color correction pipeline: > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT Hi Melissa, that makes sense to me, for CRTCs. It would be really good to have that as a diagram in the KMS UAPI documentation. If someone wants to add a 3D LUT to KMS planes as well, then I'm not sure if it should be this order or swapped. I will probably have an opinion about that once Weston is fully HDR capable and has been tried in the wild for a while with the HDR color operations fine-tuned based on community feedback. IOW, not for a long time. The YUV to RGB conversion factors in there as well. > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > LUT3D_SIZE, that allows userspace to use different supported settings of > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > adjusted the pre-blending proposal for post-blending usage. > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > by what we do for the plane blend mode property (pre-multiplied). > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > for DCN 301 (easily extendible to other DCN families). Finally, from > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > driver, exposing modes supported by HW and programming user shaper and > > 3D LUT accordingly. > > > > Our target userspace is Gamescope/SteamOS. > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > [5] https://patchwork.freedesktop.org/series/90165/ > > [6] https://patchwork.freedesktop.org/series/109402/ > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > Let me know your thoughts. > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. Unfortunately I don't have the patch emails to reply to, so here's a messy bunch of comments. I'll concentrate on the UAPI design as always. +/* + * struct drm_mode_lut3d_mode - 3D LUT mode information. + * @lut_size: number of valid points on every dimension of 3D LUT. + * @lut_stride: number of points on every dimension of 3D LUT. + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher + * bit_depth the least significant bits will be truncated. + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. + * @flags: flags for hardware-sepcific features + */ +struct drm_mode_lut3d_mode { + __u16 lut_size; + __u16 lut_stride[3]; + __u16 bit_depth; + __u32 color_format; + __u32 flags; +}; Why is lut_stride an array of 3, but lut_size is not? What is the color_mode the comment is referring to? What is "number of bits of RGB"? Input precision? Output precision? Integer or floating point? Flags cannot be hardware specific, because it makes the whole KMS UAPI hardware specific. That won't work. You have to have driver-agnostic definitions for all possible flags. Why is this the whole first patch? There is no documentation for the UAPI on how this struct works, so I cannot review this. Explaining just the individual fields is not enough to understand it. Is this something the kernel fills in and is read-only to userspace? Is userspace filling this in? + * “LUT3D”: + * Blob property to set the 3D LUT mapping pixel data after the color + * transformation matrix and before gamma 1D lut correction. The + * data is interpreted as an array of &struct drm_color_lut elements. + * Hardware might choose not to use the full precision of the LUT + * elements. + * + * Setting this to NULL (blob property value set to 0) means a the output + * color is identical to the input color. This is generally the driver + * boot-up state too. Drivers can access this blob through + * &drm_crtc_state.gamma_lut. + * You need to define how the 1-D array of drm_color_lut elements blob will be interpreted as a 3-D array for the 3D LUT, and how the dimensions match to the R, G and B channels. It's a bit like the question about row-major or column-major storage for matrices, except more complicated and not in those words. + * “LUT3D_MODE”: + * Enum property to give the mode of the 3D lookup table to be set on the + * LUT3D property. A mode specifies size, stride, bit depth and color + * format and depends on the underlying hardware). If drivers support + * multiple 3D LUT modes, they should be declared in a array of + * drm_color_lut3d_mode and they will be advertised as an enum. How does that work exactly? I didn't get it. I could guess, but having to guess on API is bad. + /** + * @lut3d: + * + * 3D Lookup table for converting pixel data. Position where it takes + * place depends on hw design, after @ctm or @gamma_lut. See + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of + * &struct drm_color_lut. + */ + struct drm_property_blob *lut3d; I do not like the wording of "depends on hw design", and it is used in very many places here. The KMS UAPI semantics cannot vary based on hardware. Your cover letter defines the order in the color pipeline, so I don't understand how this here can depend on hw. What can depend on hardware is which KMS UAPI properties are exposed, and how you map a property to a hardware unit (which can even change based on the exact pipeline configuration as long as the results are as the UAPI doc defines). But this comment here is talking about the UAPI properties, not hw elements. I'm happy that the 3D LUT interface is being developed, but as you can see from my questions, the UAPI documentation is practically missing. I would have no idea how to use this as is. Thanks! pq > > Also please let me know if I forgot to address any comments. > > Melissa > > > > > Thanks, > > > > Melissa > > > > Alex Hung (2): > > drm: Add 3D LUT mode and its attributes > > drm/amd/display: Define 3D LUT struct for HDR planes > > > > Melissa Wen (16): > > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > > drm/drm_color_mgmt: add function to create 3D LUT modes supported > > drm/drm_color_mgmt: add function to attach 3D LUT props > > drm/drm_color_mgmt: set first lut3d mode as default > > drm/amd/display: remove unused regamma condition > > drm/amd/display: add comments to describe DM crtc color mgmt behavior > > drm/amd/display: encapsulate atomic regamma operation > > drm/amd/display: update lut3d and shaper lut to stream > > drm/amd/display: handle MPC 3D LUT resources for a given context > > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > > drm/amd/display: expand array of supported 3D LUT modes > > drm/amd/display: enable 3D-LUT DRM properties if supported > > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > > pipeline > > drm/amd/display: decouple steps to reuse in shaper LUT support > > drm/amd/display: add user shaper LUT support to amdgpu_dm color > > pipeline > > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > > .../amd/display/modules/color/color_gamma.h | 43 ++ > > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > > drivers/gpu/drm/drm_fb_helper.c | 5 + > > drivers/gpu/drm/drm_mode_config.c | 21 + > > include/drm/drm_color_mgmt.h | 8 + > > include/drm/drm_crtc.h | 32 +- > > include/drm/drm_mode_config.h | 25 ++ > > include/drm/drm_mode_object.h | 2 +- > > include/uapi/drm/drm_mode.h | 17 + > > 18 files changed, 757 insertions(+), 39 deletions(-) > > > > -- > > 2.35.1 > >
On 01/31, Pekka Paalanen wrote: > On Mon, 9 Jan 2023 14:38:09 -0100 > Melissa Wen <mwen@igalia.com> wrote: > > > On 01/09, Melissa Wen wrote: > > > Hi, > > > > > > After collecting comments in different places, here is a second version > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > that means the following DRM CRTC color correction pipeline: > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > Hi Melissa, > > that makes sense to me, for CRTCs. It would be really good to have that > as a diagram in the KMS UAPI documentation. > Hi Pekka, Thanks for your feedbacks and your time reviewing this proposal. > If someone wants to add a 3D LUT to KMS planes as well, then I'm not > sure if it should be this order or swapped. I will probably have an > opinion about that once Weston is fully HDR capable and has been tried > in the wild for a while with the HDR color operations fine-tuned based > on community feedback. IOW, not for a long time. The YUV to RGB > conversion factors in there as well. > I see, this is also the reason I reuse here Alex Hung's proposal for pre-blending API. I'll work on better documentation. > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > driver, exposing modes supported by HW and programming user shaper and > > > 3D LUT accordingly. > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > Let me know your thoughts. > > > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. > > Unfortunately I don't have the patch emails to reply to, so here's a > messy bunch of comments. I'll concentrate on the UAPI design as always. Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ In the next version, I won't forget cc'ing you at first. > > +/* > + * struct drm_mode_lut3d_mode - 3D LUT mode information. > + * @lut_size: number of valid points on every dimension of 3D LUT. > + * @lut_stride: number of points on every dimension of 3D LUT. > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > + * bit_depth the least significant bits will be truncated. > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > + * @flags: flags for hardware-sepcific features > + */ > +struct drm_mode_lut3d_mode { > + __u16 lut_size; > + __u16 lut_stride[3]; > + __u16 bit_depth; > + __u32 color_format; > + __u32 flags; > +}; > > Why is lut_stride an array of 3, but lut_size is not? It cames from VA-API: https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc In short, the reason is that lut_size is the valid points and is the same for every dimensions, but lut_stride may vary. > > What is the color_mode the comment is referring to? It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT implementation (should I?), it cames from pre-blending use case. Maybe the main issue here is if reusing the pre-blending 3D LUT mode struct is a good approach or better create a specific for post-blending. > > What is "number of bits of RGB"? Input precision? Output precision? > Integer or floating point? It's the bit depth of the 3D LUT values, the same for every channels. In the AMD case, it's supports 10-bit and 12-bit, for example. > > Flags cannot be hardware specific, because it makes the whole KMS UAPI > hardware specific. That won't work. You have to have driver-agnostic > definitions for all possible flags. > > Why is this the whole first patch? There is no documentation for the > UAPI on how this struct works, so I cannot review this. Explaining just > the individual fields is not enough to understand it. Is this something > the kernel fills in and is read-only to userspace? Is userspace filling > this in? I see. I'll work on explaining/documenting it better. > > > + * “LUT3D”: > + * Blob property to set the 3D LUT mapping pixel data after the color > + * transformation matrix and before gamma 1D lut correction. The > + * data is interpreted as an array of &struct drm_color_lut elements. > + * Hardware might choose not to use the full precision of the LUT > + * elements. > + * > + * Setting this to NULL (blob property value set to 0) means a the output > + * color is identical to the input color. This is generally the driver > + * boot-up state too. Drivers can access this blob through > + * &drm_crtc_state.gamma_lut. > + * > > You need to define how the 1-D array of drm_color_lut elements blob > will be interpreted as a 3-D array for the 3D LUT, and how the > dimensions match to the R, G and B channels. It's a bit like the > question about row-major or column-major storage for matrices, except > more complicated and not in those words. ack > > + * “LUT3D_MODE”: > + * Enum property to give the mode of the 3D lookup table to be set on the > + * LUT3D property. A mode specifies size, stride, bit depth and color > + * format and depends on the underlying hardware). If drivers support > + * multiple 3D LUT modes, they should be declared in a array of > + * drm_color_lut3d_mode and they will be advertised as an enum. > > How does that work exactly? I didn't get it. I could guess, but having > to guess on API is bad. The driver advertises all supported modes (each combination of values) in a array as a enum, userspace can check all accepted modes and set the one that fits the user 3D LUT settings. I think it's possible to get the idea from this IGT test: https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 > > > + /** > + * @lut3d: > + * > + * 3D Lookup table for converting pixel data. Position where it takes > + * place depends on hw design, after @ctm or @gamma_lut. See > + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of > + * &struct drm_color_lut. > + */ > + struct drm_property_blob *lut3d; > > I do not like the wording of "depends on hw design", and it is used in > very many places here. The KMS UAPI semantics cannot vary based on > hardware. Your cover letter defines the order in the color pipeline, so > I don't understand how this here can depend on hw. > > What can depend on hardware is which KMS UAPI properties are exposed, > and how you map a property to a hardware unit (which can even change > based on the exact pipeline configuration as long as the results are as > the UAPI doc defines). But this comment here is talking about the UAPI > properties, not hw elements. > You are right! My initial idea was to explain that it's possible for other vendors color pipeline to fit this pipeline internally, if they need a 1D LUT before the 3D LUT, but not the 1D LUT in the end. > > I'm happy that the 3D LUT interface is being developed, but as you can > see from my questions, the UAPI documentation is practically missing. I > would have no idea how to use this as is. Thank you again for your valuable comments. I'll address your comments in a next version by better explaining all these points. Melissa > > > Thanks! > pq > > > > > Also please let me know if I forgot to address any comments. > > > > Melissa > > > > > > > > Thanks, > > > > > > Melissa > > > > > > Alex Hung (2): > > > drm: Add 3D LUT mode and its attributes > > > drm/amd/display: Define 3D LUT struct for HDR planes > > > > > > Melissa Wen (16): > > > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > > > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > > > drm/drm_color_mgmt: add function to create 3D LUT modes supported > > > drm/drm_color_mgmt: add function to attach 3D LUT props > > > drm/drm_color_mgmt: set first lut3d mode as default > > > drm/amd/display: remove unused regamma condition > > > drm/amd/display: add comments to describe DM crtc color mgmt behavior > > > drm/amd/display: encapsulate atomic regamma operation > > > drm/amd/display: update lut3d and shaper lut to stream > > > drm/amd/display: handle MPC 3D LUT resources for a given context > > > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > > > drm/amd/display: expand array of supported 3D LUT modes > > > drm/amd/display: enable 3D-LUT DRM properties if supported > > > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > > > pipeline > > > drm/amd/display: decouple steps to reuse in shaper LUT support > > > drm/amd/display: add user shaper LUT support to amdgpu_dm color > > > pipeline > > > > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > > > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > > > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > > > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > > > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > > > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > > > .../amd/display/modules/color/color_gamma.h | 43 ++ > > > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > > > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > > > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > > > drivers/gpu/drm/drm_fb_helper.c | 5 + > > > drivers/gpu/drm/drm_mode_config.c | 21 + > > > include/drm/drm_color_mgmt.h | 8 + > > > include/drm/drm_crtc.h | 32 +- > > > include/drm/drm_mode_config.h | 25 ++ > > > include/drm/drm_mode_object.h | 2 +- > > > include/uapi/drm/drm_mode.h | 17 + > > > 18 files changed, 757 insertions(+), 39 deletions(-) > > > > > > -- > > > 2.35.1 > > > >
On Thu, 9 Feb 2023 13:27:02 -0100 Melissa Wen <mwen@igalia.com> wrote: > On 01/31, Pekka Paalanen wrote: > > On Mon, 9 Jan 2023 14:38:09 -0100 > > Melissa Wen <mwen@igalia.com> wrote: > > > > > On 01/09, Melissa Wen wrote: > > > > Hi, > > > > > > > > After collecting comments in different places, here is a second version > > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > that means the following DRM CRTC color correction pipeline: > > > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > Hi Melissa, > > > > that makes sense to me, for CRTCs. It would be really good to have that > > as a diagram in the KMS UAPI documentation. > > > > Hi Pekka, > > Thanks for your feedbacks and your time reviewing this proposal. No problem, and sorry it took so long! I'm just finishing the catch-up with everything that happened during winter holidays. > > If someone wants to add a 3D LUT to KMS planes as well, then I'm not > > sure if it should be this order or swapped. I will probably have an > > opinion about that once Weston is fully HDR capable and has been tried > > in the wild for a while with the HDR color operations fine-tuned based > > on community feedback. IOW, not for a long time. The YUV to RGB > > conversion factors in there as well. > > > I see, this is also the reason I reuse here Alex Hung's proposal for > pre-blending API. I'll work on better documentation. > > > > > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > > driver, exposing modes supported by HW and programming user shaper and > > > > 3D LUT accordingly. > > > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > > > Let me know your thoughts. > > > > > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. > > > > Unfortunately I don't have the patch emails to reply to, so here's a > > messy bunch of comments. I'll concentrate on the UAPI design as always. > > Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ > In the next version, I won't forget cc'ing you at first. > > > > +/* > > + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > + * @lut_size: number of valid points on every dimension of 3D LUT. > > + * @lut_stride: number of points on every dimension of 3D LUT. > > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > + * bit_depth the least significant bits will be truncated. > > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > + * @flags: flags for hardware-sepcific features > > + */ > > +struct drm_mode_lut3d_mode { > > + __u16 lut_size; > > + __u16 lut_stride[3]; > > + __u16 bit_depth; > > + __u32 color_format; > > + __u32 flags; > > +}; > > > > Why is lut_stride an array of 3, but lut_size is not? > > It cames from VA-API: > https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc > > In short, the reason is that lut_size is the valid points and is > the same for every dimensions, but lut_stride may vary. Ok, so lut_size is what I would have guessed it to be. Just needs to be doc'd - pretty much all my questions are lack of docs. I see that lut_stride is supposed to be in points, and not bytes. That is surprising. Are you sure it's good? > > > > What is the color_mode the comment is referring to? > > It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT > implementation (should I?), it cames from pre-blending use case. Maybe > the main issue here is if reusing the pre-blending 3D LUT mode struct is > a good approach or better create a specific for post-blending. Sorry, I have no idea what FB color_mode is. I do not recall any such thing in the KMS uAPI. > > > > What is "number of bits of RGB"? Input precision? Output precision? > > Integer or floating point? > > It's the bit depth of the 3D LUT values, the same for every channels. In > the AMD case, it's supports 10-bit and 12-bit, for example. Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any hardware ever? What exactly is the truncation the comment refers to? It sounds like if input has higher precision than the LUT elements, then "truncation" occurs. I can kind of see that, but I also think it is a false characterisation. The LUT input precision affects the precision of LUT indexing and the precision of interpolation between the LUT elements. I would not expect those two precisions to be truncated to the LUT element precision (but they could be truncated to something else hardware specific). Instead, I do expect the interpolation result to be truncated to the LUT output precision, which probably is the same as the LUT element precision, but not necessarily. Maybe the comment about truncation should simply be removed? The result is obvious if we know the LUT input, element, and output precision, and what exactly happens with the indexing and interpolation is probably good enough to be left hardware-specific if it is difficult to describe in generic terms across different hardware. > > > > Flags cannot be hardware specific, because it makes the whole KMS UAPI > > hardware specific. That won't work. You have to have driver-agnostic > > definitions for all possible flags. > > > > Why is this the whole first patch? There is no documentation for the > > UAPI on how this struct works, so I cannot review this. Explaining just > > the individual fields is not enough to understand it. Is this something > > the kernel fills in and is read-only to userspace? Is userspace filling > > this in? > > I see. I'll work on explaining/documenting it better. > > > > > > + * “LUT3D”: > > + * Blob property to set the 3D LUT mapping pixel data after the color > > + * transformation matrix and before gamma 1D lut correction. The > > + * data is interpreted as an array of &struct drm_color_lut elements. > > + * Hardware might choose not to use the full precision of the LUT > > + * elements. > > + * > > + * Setting this to NULL (blob property value set to 0) means a the output > > + * color is identical to the input color. This is generally the driver > > + * boot-up state too. Drivers can access this blob through > > + * &drm_crtc_state.gamma_lut. > > + * > > > > You need to define how the 1-D array of drm_color_lut elements blob > > will be interpreted as a 3-D array for the 3D LUT, and how the > > dimensions match to the R, G and B channels. It's a bit like the > > question about row-major or column-major storage for matrices, except > > more complicated and not in those words. > > ack > > > > + * “LUT3D_MODE”: > > + * Enum property to give the mode of the 3D lookup table to be set on the > > + * LUT3D property. A mode specifies size, stride, bit depth and color > > + * format and depends on the underlying hardware). If drivers support > > + * multiple 3D LUT modes, they should be declared in a array of > > + * drm_color_lut3d_mode and they will be advertised as an enum. > > > > How does that work exactly? I didn't get it. I could guess, but having > > to guess on API is bad. > > The driver advertises all supported modes (each combination of values) > in a array as a enum, userspace can check all accepted modes and set the > one that fits the user 3D LUT settings. I think it's possible to get the > idea from this IGT test: > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 > > You lost me at "an array as an enum". I understand there is a blob containing an array of struct drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE is an enum property. Where does the blob come from, then? What property provides the blob? Am I correct in guessing that the values of LUT3D_MODE enum property are indices into the array in the blob, and that userspace will set it? That sounds good to me, if it's the integer value of the enum. But enum values also need string names, because that is how the values are usually recognized, so what name strings will be used? In that code example, I cannot see any connection between the array of drm_mode_lut3d_mode entries, the 'name' string, and the actual data generated for the LUT. They must all connect somehow, but it just doesn't seem to be in the code. It is just hardcoding mode_lut3d[0], and the data generator never sees even that - it gets the lut size, but none of the other parameters. I cannot see that working across multiple drivers. Why is it setting a linear shaper LUT explicitly instead of unsetting it? I mean this preparation: shaper_linear = generate_table(data->shaper_lut_size, 1.0); "Linear" actually means "identity", right? That's what people usually write when they mean identity. Surely it's not actually a constant curve? That would make the whole test useless for proving the 3D LUT works. > > > > + /** > > + * @lut3d: > > + * > > + * 3D Lookup table for converting pixel data. Position where it takes > > + * place depends on hw design, after @ctm or @gamma_lut. See > > + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of > > + * &struct drm_color_lut. > > + */ > > + struct drm_property_blob *lut3d; > > > > I do not like the wording of "depends on hw design", and it is used in > > very many places here. The KMS UAPI semantics cannot vary based on > > hardware. Your cover letter defines the order in the color pipeline, so > > I don't understand how this here can depend on hw. > > > > What can depend on hardware is which KMS UAPI properties are exposed, > > and how you map a property to a hardware unit (which can even change > > based on the exact pipeline configuration as long as the results are as > > the UAPI doc defines). But this comment here is talking about the UAPI > > properties, not hw elements. > > > > You are right! My initial idea was to explain that it's possible for > other vendors color pipeline to fit this pipeline internally, if they > need a 1D LUT before the 3D LUT, but not the 1D LUT in the end. > > > > > I'm happy that the 3D LUT interface is being developed, but as you can > > see from my questions, the UAPI documentation is practically missing. I > > would have no idea how to use this as is. > > Thank you again for your valuable comments. I'll address your comments > in a next version by better explaining all these points. Thank you for working on this! pq
On 2/10/23 04:28, Pekka Paalanen wrote: > On Thu, 9 Feb 2023 13:27:02 -0100 > Melissa Wen <mwen@igalia.com> wrote: > >> On 01/31, Pekka Paalanen wrote: >>> On Mon, 9 Jan 2023 14:38:09 -0100 >>> Melissa Wen <mwen@igalia.com> wrote: >>> >>>> On 01/09, Melissa Wen wrote: >>>>> Hi, >>>>> >>>>> After collecting comments in different places, here is a second version >>>>> of the work on adding DRM CRTC 3D LUT support to the current DRM color >>>>> mgmt interface. In comparison to previous proposals [1][2][3], here we >>>>> add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, >>>>> that means the following DRM CRTC color correction pipeline: >>>>> >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT >>> >>> Hi Melissa, >>> >>> that makes sense to me, for CRTCs. It would be really good to have that >>> as a diagram in the KMS UAPI documentation. >>> >> >> Hi Pekka, >> >> Thanks for your feedbacks and your time reviewing this proposal. > > No problem, and sorry it took so long! > > I'm just finishing the catch-up with everything that happened during > winter holidays. > >>> If someone wants to add a 3D LUT to KMS planes as well, then I'm not >>> sure if it should be this order or swapped. I will probably have an >>> opinion about that once Weston is fully HDR capable and has been tried >>> in the wild for a while with the HDR color operations fine-tuned based >>> on community feedback. IOW, not for a long time. The YUV to RGB >>> conversion factors in there as well. >>> >> I see, this is also the reason I reuse here Alex Hung's proposal for >> pre-blending API. I'll work on better documentation. >> >>> >>>>> >>>>> and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung >>>>> proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a >>>>> LUT3D_SIZE, that allows userspace to use different supported settings of >>>>> 3D LUT, fitting VA-API and new color API better. In this sense, I >>>>> adjusted the pre-blending proposal for post-blending usage. >>>>> >>>>> Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to >>>>> the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an >>>>> extra/optional patch to define a default value for LUT3D_MODE, inspired >>>>> by what we do for the plane blend mode property (pre-multiplied). >>>>> >>>>> Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage >>>>> on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current >>>>> AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT >>>>> changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context >>>>> for DCN 301 (easily extendible to other DCN families). Finally, from >>>>> 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display >>>>> driver, exposing modes supported by HW and programming user shaper and >>>>> 3D LUT accordingly. >>>>> >>>>> Our target userspace is Gamescope/SteamOS. >>>>> >>>>> Basic IGT tests were based on [5][6] and are available here (in-progress): >>>>> https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api >>>>> >>>>> [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ >>>>> [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 >>>>> [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ >>>>> [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ >>>>> [5] https://patchwork.freedesktop.org/series/90165/ >>>>> [6] https://patchwork.freedesktop.org/series/109402/ >>>>> [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html >>>>> [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 >>>>> >>>>> Let me know your thoughts. >>>> >>>> +Simon Ser, +Pekka Paalanen who might also be interested in this series. >>> >>> Unfortunately I don't have the patch emails to reply to, so here's a >>> messy bunch of comments. I'll concentrate on the UAPI design as always. >> >> Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ >> In the next version, I won't forget cc'ing you at first. >>> >>> +/* >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. >>> + * @lut_size: number of valid points on every dimension of 3D LUT. >>> + * @lut_stride: number of points on every dimension of 3D LUT. >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher >>> + * bit_depth the least significant bits will be truncated. >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. >>> + * @flags: flags for hardware-sepcific features >>> + */ >>> +struct drm_mode_lut3d_mode { >>> + __u16 lut_size; >>> + __u16 lut_stride[3]; >>> + __u16 bit_depth; >>> + __u32 color_format; >>> + __u32 flags; >>> +}; >>> >>> Why is lut_stride an array of 3, but lut_size is not? >> >> It cames from VA-API: >> https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc >> >> In short, the reason is that lut_size is the valid points and is >> the same for every dimensions, but lut_stride may vary. > > Ok, so lut_size is what I would have guessed it to be. Just needs to be > doc'd - pretty much all my questions are lack of docs. > > I see that lut_stride is supposed to be in points, and not bytes. That > is surprising. Are you sure it's good? > >>> >>> What is the color_mode the comment is referring to? >> >> It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT >> implementation (should I?), it cames from pre-blending use case. Maybe >> the main issue here is if reusing the pre-blending 3D LUT mode struct is >> a good approach or better create a specific for post-blending. > > Sorry, I have no idea what FB color_mode is. I do not recall any such > thing in the KMS uAPI. > > >>> >>> What is "number of bits of RGB"? Input precision? Output precision? >>> Integer or floating point? >> >> It's the bit depth of the 3D LUT values, the same for every channels. In >> the AMD case, it's supports 10-bit and 12-bit, for example. > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > hardware ever? > I haven't had a chance to go through all patches yet but if this is modeled after Alex Hung's work this should be covered by color_format. The idea is that color_format takes a FOURCC value and defines the format of the entries in the 3DLUT blob. The bit_depth describes the actual bit depth that the HW supports. E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might only support 12-bit precision. In that case the least significant bits get truncated. One could define the bit_depth per color, but I'm not sure that'll be necessary. > What exactly is the truncation the comment refers to? > > It sounds like if input has higher precision than the LUT elements, > then "truncation" occurs. I can kind of see that, but I also think it > is a false characterisation. The LUT input precision affects the > precision of LUT indexing and the precision of interpolation between > the LUT elements. I would not expect those two precisions to be > truncated to the LUT element precision (but they could be truncated to > something else hardware specific). Instead, I do expect the > interpolation result to be truncated to the LUT output precision, which > probably is the same as the LUT element precision, but not necessarily. > > Maybe the comment about truncation should simply be removed? The result > is obvious if we know the LUT input, element, and output precision, and > what exactly happens with the indexing and interpolation is probably > good enough to be left hardware-specific if it is difficult to describe > in generic terms across different hardware. > Maybe it makes sense to just drop the bit_depth field. Harry >>> >>> Flags cannot be hardware specific, because it makes the whole KMS UAPI >>> hardware specific. That won't work. You have to have driver-agnostic >>> definitions for all possible flags. >>> >>> Why is this the whole first patch? There is no documentation for the >>> UAPI on how this struct works, so I cannot review this. Explaining just >>> the individual fields is not enough to understand it. Is this something >>> the kernel fills in and is read-only to userspace? Is userspace filling >>> this in? >> >> I see. I'll work on explaining/documenting it better. >>> >>> >>> + * “LUT3D”: >>> + * Blob property to set the 3D LUT mapping pixel data after the color >>> + * transformation matrix and before gamma 1D lut correction. The >>> + * data is interpreted as an array of &struct drm_color_lut elements. >>> + * Hardware might choose not to use the full precision of the LUT >>> + * elements. >>> + * >>> + * Setting this to NULL (blob property value set to 0) means a the output >>> + * color is identical to the input color. This is generally the driver >>> + * boot-up state too. Drivers can access this blob through >>> + * &drm_crtc_state.gamma_lut. >>> + * >>> >>> You need to define how the 1-D array of drm_color_lut elements blob >>> will be interpreted as a 3-D array for the 3D LUT, and how the >>> dimensions match to the R, G and B channels. It's a bit like the >>> question about row-major or column-major storage for matrices, except >>> more complicated and not in those words. >> >> ack >>> >>> + * “LUT3D_MODE”: >>> + * Enum property to give the mode of the 3D lookup table to be set on the >>> + * LUT3D property. A mode specifies size, stride, bit depth and color >>> + * format and depends on the underlying hardware). If drivers support >>> + * multiple 3D LUT modes, they should be declared in a array of >>> + * drm_color_lut3d_mode and they will be advertised as an enum. >>> >>> How does that work exactly? I didn't get it. I could guess, but having >>> to guess on API is bad. >> >> The driver advertises all supported modes (each combination of values) >> in a array as a enum, userspace can check all accepted modes and set the >> one that fits the user 3D LUT settings. I think it's possible to get the >> idea from this IGT test: >> https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 >>> > > You lost me at "an array as an enum". > > I understand there is a blob containing an array of struct > drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE > is an enum property. Where does the blob come from, then? What property > provides the blob? > > Am I correct in guessing that the values of LUT3D_MODE enum property > are indices into the array in the blob, and that userspace will set it? > That sounds good to me, if it's the integer value of the enum. But enum > values also need string names, because that is how the values are > usually recognized, so what name strings will be used? > > In that code example, I cannot see any connection between the array of > drm_mode_lut3d_mode entries, the 'name' string, and the actual data > generated for the LUT. They must all connect somehow, but it just > doesn't seem to be in the code. It is just hardcoding mode_lut3d[0], > and the data generator never sees even that - it gets the lut size, but > none of the other parameters. I cannot see that working across multiple > drivers. > > Why is it setting a linear shaper LUT explicitly instead of unsetting > it? I mean this preparation: > > shaper_linear = generate_table(data->shaper_lut_size, 1.0); > > "Linear" actually means "identity", right? That's what people usually > write when they mean identity. > > Surely it's not actually a constant curve? That would make the whole > test useless for proving the 3D LUT works. > > >>> >>> + /** >>> + * @lut3d: >>> + * >>> + * 3D Lookup table for converting pixel data. Position where it takes >>> + * place depends on hw design, after @ctm or @gamma_lut. See >>> + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of >>> + * &struct drm_color_lut. >>> + */ >>> + struct drm_property_blob *lut3d; >>> >>> I do not like the wording of "depends on hw design", and it is used in >>> very many places here. The KMS UAPI semantics cannot vary based on >>> hardware. Your cover letter defines the order in the color pipeline, so >>> I don't understand how this here can depend on hw. >>> >>> What can depend on hardware is which KMS UAPI properties are exposed, >>> and how you map a property to a hardware unit (which can even change >>> based on the exact pipeline configuration as long as the results are as >>> the UAPI doc defines). But this comment here is talking about the UAPI >>> properties, not hw elements. >>> >> >> You are right! My initial idea was to explain that it's possible for >> other vendors color pipeline to fit this pipeline internally, if they >> need a 1D LUT before the 3D LUT, but not the 1D LUT in the end. >> >>> >>> I'm happy that the 3D LUT interface is being developed, but as you can >>> see from my questions, the UAPI documentation is practically missing. I >>> would have no idea how to use this as is. >> >> Thank you again for your valuable comments. I'll address your comments >> in a next version by better explaining all these points. > > Thank you for working on this! > pq
On Fri, 10 Feb 2023 14:47:50 -0500 Harry Wentland <harry.wentland@amd.com> wrote: > On 2/10/23 04:28, Pekka Paalanen wrote: > > On Thu, 9 Feb 2023 13:27:02 -0100 > > Melissa Wen <mwen@igalia.com> wrote: > > > >> On 01/31, Pekka Paalanen wrote: > >>> On Mon, 9 Jan 2023 14:38:09 -0100 > >>> Melissa Wen <mwen@igalia.com> wrote: > >>> > >>>> On 01/09, Melissa Wen wrote: > >>>>> Hi, > >>>>> > >>>>> After collecting comments in different places, here is a second version > >>>>> of the work on adding DRM CRTC 3D LUT support to the current DRM color > >>>>> mgmt interface. In comparison to previous proposals [1][2][3], here we > >>>>> add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > >>>>> that means the following DRM CRTC color correction pipeline: > >>>>> > >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT ... > >>> +/* > >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. > >>> + * @lut_size: number of valid points on every dimension of 3D LUT. > >>> + * @lut_stride: number of points on every dimension of 3D LUT. > >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > >>> + * bit_depth the least significant bits will be truncated. > >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > >>> + * @flags: flags for hardware-sepcific features > >>> + */ > >>> +struct drm_mode_lut3d_mode { > >>> + __u16 lut_size; > >>> + __u16 lut_stride[3]; > >>> + __u16 bit_depth; > >>> + __u32 color_format; > >>> + __u32 flags; > >>> +}; ... > >>> What is "number of bits of RGB"? Input precision? Output precision? > >>> Integer or floating point? > >> > >> It's the bit depth of the 3D LUT values, the same for every channels. In > >> the AMD case, it's supports 10-bit and 12-bit, for example. > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > > hardware ever? > > > > I haven't had a chance to go through all patches yet but if this is > modeled after Alex Hung's work this should be covered by color_format. > The idea is that color_format takes a FOURCC value and defines the > format of the entries in the 3DLUT blob. > > The bit_depth describes the actual bit depth that the HW supports. > E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might only > support 12-bit precision. In that case the least significant bits get > truncated. > > One could define the bit_depth per color, but I'm not sure that'll be > necessary. Exactly. I just have no idea how sure we should be about that. > > What exactly is the truncation the comment refers to? > > > > It sounds like if input has higher precision than the LUT elements, > > then "truncation" occurs. I can kind of see that, but I also think it > > is a false characterisation. The LUT input precision affects the > > precision of LUT indexing and the precision of interpolation between > > the LUT elements. I would not expect those two precisions to be > > truncated to the LUT element precision (but they could be truncated to > > something else hardware specific). Instead, I do expect the > > interpolation result to be truncated to the LUT output precision, which > > probably is the same as the LUT element precision, but not necessarily. > > > > Maybe the comment about truncation should simply be removed? The result > > is obvious if we know the LUT input, element, and output precision, and > > what exactly happens with the indexing and interpolation is probably > > good enough to be left hardware-specific if it is difficult to describe > > in generic terms across different hardware. > > > > Maybe it makes sense to just drop the bit_depth field. Well, it's really interesting information for userspace, but maybe it should have a more holistic design. Precision is a factor, when userspace considers whether it can use KMS hardware for a conversion or not. Unfortunately, none of the existing KMS color pipeline elements have any information on precision IIRC, so there is more to be fixed. The interesting thing is the minimum guaranteed precision of each element and the connections between them. It might be different for pass-through vs. not. Another interesting thing is the usable value range. This is probably a complex problem, so there should be no need to solve it before a 3D LUT interface can land, given old elements already have the issue. Thanks, pq
On Mon, Feb 13, 2023 at 11:01:31AM +0200, Pekka Paalanen wrote: > On Fri, 10 Feb 2023 14:47:50 -0500 > Harry Wentland <harry.wentland@amd.com> wrote: > > > On 2/10/23 04:28, Pekka Paalanen wrote: > > > On Thu, 9 Feb 2023 13:27:02 -0100 > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > >> On 01/31, Pekka Paalanen wrote: > > >>> On Mon, 9 Jan 2023 14:38:09 -0100 > > >>> Melissa Wen <mwen@igalia.com> wrote: > > >>> > > >>>> On 01/09, Melissa Wen wrote: > > >>>>> Hi, > > >>>>> > > >>>>> After collecting comments in different places, here is a second version > > >>>>> of the work on adding DRM CRTC 3D LUT support to the current DRM color > > >>>>> mgmt interface. In comparison to previous proposals [1][2][3], here we > > >>>>> add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > >>>>> that means the following DRM CRTC color correction pipeline: > > >>>>> > > >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > ... > > > >>> +/* > > >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > >>> + * @lut_size: number of valid points on every dimension of 3D LUT. > > >>> + * @lut_stride: number of points on every dimension of 3D LUT. > > >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > >>> + * bit_depth the least significant bits will be truncated. > > >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > >>> + * @flags: flags for hardware-sepcific features > > >>> + */ > > >>> +struct drm_mode_lut3d_mode { > > >>> + __u16 lut_size; > > >>> + __u16 lut_stride[3]; > > >>> + __u16 bit_depth; > > >>> + __u32 color_format; > > >>> + __u32 flags; > > >>> +}; > > ... > > > >>> What is "number of bits of RGB"? Input precision? Output precision? > > >>> Integer or floating point? > > >> > > >> It's the bit depth of the 3D LUT values, the same for every channels. In > > >> the AMD case, it's supports 10-bit and 12-bit, for example. > > > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > > > hardware ever? > > > > > > > I haven't had a chance to go through all patches yet but if this is > > modeled after Alex Hung's work this should be covered by color_format. > > The idea is that color_format takes a FOURCC value and defines the > > format of the entries in the 3DLUT blob. > > > > The bit_depth describes the actual bit depth that the HW supports. > > E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might only > > support 12-bit precision. In that case the least significant bits get > > truncated. > > > > One could define the bit_depth per color, but I'm not sure that'll be > > necessary. > > Exactly. I just have no idea how sure we should be about that. > > > > What exactly is the truncation the comment refers to? > > > > > > It sounds like if input has higher precision than the LUT elements, > > > then "truncation" occurs. I can kind of see that, but I also think it > > > is a false characterisation. The LUT input precision affects the > > > precision of LUT indexing and the precision of interpolation between > > > the LUT elements. I would not expect those two precisions to be > > > truncated to the LUT element precision (but they could be truncated to > > > something else hardware specific). Instead, I do expect the > > > interpolation result to be truncated to the LUT output precision, which > > > probably is the same as the LUT element precision, but not necessarily. > > > > > > Maybe the comment about truncation should simply be removed? The result > > > is obvious if we know the LUT input, element, and output precision, and > > > what exactly happens with the indexing and interpolation is probably > > > good enough to be left hardware-specific if it is difficult to describe > > > in generic terms across different hardware. > > > > > > > Maybe it makes sense to just drop the bit_depth field. > > Well, it's really interesting information for userspace, but maybe it > should have a more holistic design. Precision is a factor, when > userspace considers whether it can use KMS hardware for a conversion or > not. Unfortunately, none of the existing KMS color pipeline elements > have any information on precision IIRC, so there is more to be fixed. > > The interesting thing is the minimum guaranteed precision of each > element and the connections between them. It might be different for > pass-through vs. not. Another interesting thing is the usable value > range. > > This is probably a complex problem, so there should be no need to solve > it before a 3D LUT interface can land, given old elements already have > the issue. Yeah, I think all the precision stuff is all better handled by eg. the proposed GAMMA_MODE property or something similar. It's going to be needed for 1D LUTs as well. 1D LUTs would also need it to expose diffrent LUT sizes with different precision tradeoffs. As for the 3D LUT blob, I don't think the blob needs any strides/etc. either. I had none of that for my i915 version: https://github.com/vsyrjala/linux/commits/3dlut Just the LUT entries + blob size is sufficient. At least for cube shaped LUTs. Dunno if anyone would have a need for something else? The two things the are absolutely needed: - Position of the LUT in the pipeline. We've already seen at least two variants of this IIRC, so we'll likely need to define a unique property for each tap point. - The order the elements are stored in the blob. I didn't check if all the already known hw (amdgpu, i915, rcar-du, were there also others?) would agree on this or not. If yes, maybe just follow the hw order for all those, and if not, then I guess flip a few coins.
On 02/10, Pekka Paalanen wrote: > On Thu, 9 Feb 2023 13:27:02 -0100 > Melissa Wen <mwen@igalia.com> wrote: > > > On 01/31, Pekka Paalanen wrote: > > > On Mon, 9 Jan 2023 14:38:09 -0100 > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > On 01/09, Melissa Wen wrote: > > > > > Hi, > > > > > > > > > > After collecting comments in different places, here is a second version > > > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > > that means the following DRM CRTC color correction pipeline: > > > > > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > Hi Melissa, > > > > > > that makes sense to me, for CRTCs. It would be really good to have that > > > as a diagram in the KMS UAPI documentation. > > > > > > > Hi Pekka, > > > > Thanks for your feedbacks and your time reviewing this proposal. > > No problem, and sorry it took so long! > > I'm just finishing the catch-up with everything that happened during > winter holidays. > > > > If someone wants to add a 3D LUT to KMS planes as well, then I'm not > > > sure if it should be this order or swapped. I will probably have an > > > opinion about that once Weston is fully HDR capable and has been tried > > > in the wild for a while with the HDR color operations fine-tuned based > > > on community feedback. IOW, not for a long time. The YUV to RGB > > > conversion factors in there as well. > > > > > I see, this is also the reason I reuse here Alex Hung's proposal for > > pre-blending API. I'll work on better documentation. > > > > > > > > > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > > > driver, exposing modes supported by HW and programming user shaper and > > > > > 3D LUT accordingly. > > > > > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > > > > > Let me know your thoughts. > > > > > > > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. > > > > > > Unfortunately I don't have the patch emails to reply to, so here's a > > > messy bunch of comments. I'll concentrate on the UAPI design as always. > > > > Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ > > In the next version, I won't forget cc'ing you at first. > > > > > > +/* > > > + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > + * @lut_size: number of valid points on every dimension of 3D LUT. > > > + * @lut_stride: number of points on every dimension of 3D LUT. > > > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > + * bit_depth the least significant bits will be truncated. > > > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > + * @flags: flags for hardware-sepcific features > > > + */ > > > +struct drm_mode_lut3d_mode { > > > + __u16 lut_size; > > > + __u16 lut_stride[3]; > > > + __u16 bit_depth; > > > + __u32 color_format; > > > + __u32 flags; > > > +}; > > > > > > Why is lut_stride an array of 3, but lut_size is not? > > > > It cames from VA-API: > > https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc > > > > In short, the reason is that lut_size is the valid points and is > > the same for every dimensions, but lut_stride may vary. > > Ok, so lut_size is what I would have guessed it to be. Just needs to be > doc'd - pretty much all my questions are lack of docs. > > I see that lut_stride is supposed to be in points, and not bytes. That > is surprising. Are you sure it's good? > > > > > > > What is the color_mode the comment is referring to? > > > > It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT > > implementation (should I?), it cames from pre-blending use case. Maybe > > the main issue here is if reusing the pre-blending 3D LUT mode struct is > > a good approach or better create a specific for post-blending. > > Sorry, I have no idea what FB color_mode is. I do not recall any such > thing in the KMS uAPI. > > > > > > > > What is "number of bits of RGB"? Input precision? Output precision? > > > Integer or floating point? > > > > It's the bit depth of the 3D LUT values, the same for every channels. In > > the AMD case, it's supports 10-bit and 12-bit, for example. > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > hardware ever? > > What exactly is the truncation the comment refers to? > > It sounds like if input has higher precision than the LUT elements, > then "truncation" occurs. I can kind of see that, but I also think it > is a false characterisation. The LUT input precision affects the > precision of LUT indexing and the precision of interpolation between > the LUT elements. I would not expect those two precisions to be > truncated to the LUT element precision (but they could be truncated to > something else hardware specific). Instead, I do expect the > interpolation result to be truncated to the LUT output precision, which > probably is the same as the LUT element precision, but not necessarily. > > Maybe the comment about truncation should simply be removed? The result > is obvious if we know the LUT input, element, and output precision, and > what exactly happens with the indexing and interpolation is probably > good enough to be left hardware-specific if it is difficult to describe > in generic terms across different hardware. > > > > > > > Flags cannot be hardware specific, because it makes the whole KMS UAPI > > > hardware specific. That won't work. You have to have driver-agnostic > > > definitions for all possible flags. > > > > > > Why is this the whole first patch? There is no documentation for the > > > UAPI on how this struct works, so I cannot review this. Explaining just > > > the individual fields is not enough to understand it. Is this something > > > the kernel fills in and is read-only to userspace? Is userspace filling > > > this in? > > > > I see. I'll work on explaining/documenting it better. > > > > > > > > > + * “LUT3D”: > > > + * Blob property to set the 3D LUT mapping pixel data after the color > > > + * transformation matrix and before gamma 1D lut correction. The > > > + * data is interpreted as an array of &struct drm_color_lut elements. > > > + * Hardware might choose not to use the full precision of the LUT > > > + * elements. > > > + * > > > + * Setting this to NULL (blob property value set to 0) means a the output > > > + * color is identical to the input color. This is generally the driver > > > + * boot-up state too. Drivers can access this blob through > > > + * &drm_crtc_state.gamma_lut. > > > + * > > > > > > You need to define how the 1-D array of drm_color_lut elements blob > > > will be interpreted as a 3-D array for the 3D LUT, and how the > > > dimensions match to the R, G and B channels. It's a bit like the > > > question about row-major or column-major storage for matrices, except > > > more complicated and not in those words. > > > > ack > > > > > > + * “LUT3D_MODE”: > > > + * Enum property to give the mode of the 3D lookup table to be set on the > > > + * LUT3D property. A mode specifies size, stride, bit depth and color > > > + * format and depends on the underlying hardware). If drivers support > > > + * multiple 3D LUT modes, they should be declared in a array of > > > + * drm_color_lut3d_mode and they will be advertised as an enum. > > > > > > How does that work exactly? I didn't get it. I could guess, but having > > > to guess on API is bad. > > > > The driver advertises all supported modes (each combination of values) > > in a array as a enum, userspace can check all accepted modes and set the > > one that fits the user 3D LUT settings. I think it's possible to get the > > idea from this IGT test: > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 > > > > > You lost me at "an array as an enum". > > I understand there is a blob containing an array of struct > drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE > is an enum property. Where does the blob come from, then? What property > provides the blob? > > Am I correct in guessing that the values of LUT3D_MODE enum property > are indices into the array in the blob, and that userspace will set it? > That sounds good to me, if it's the integer value of the enum. But enum > values also need string names, because that is how the values are > usually recognized, so what name strings will be used? So, in this proposal, LUT3D_MODE is a list of indexes for a blob that describe a supported 3D LUT mode i.e. a `struct drm_mode_lut3d_mode` with size, stride, bit depth, etc. Strings here follow this pattern `lut3d_{size}_{bit_depth}bit` [1]. When enabling 3D LUT support, the driver should pass an array of `struct drm_mode_lut3d_mode` as supported modes, with at least one element. I think this drm_info output helps to understand the idea. Currently, we have 5 DRM CRTC color mgmt properties. The drm_info output for them: │ ├───"DEGAMMA_LUT": blob = 0 │ ├───"DEGAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 │ ├───"CTM": blob = 0 │ ├───"GAMMA_LUT": blob = 0 │ ├───"GAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 The idea is to add 4 new properties. - The 3D LUT blob (LUT3D) and the indexes of supported modes (LUT3D_MODE), as follows (example with AMD supported modes): │ ├───"LUT3D": blob = 0 │ ├───"LUT3D_MODE": enum {lut3d_17_12bit, lut3d_17_10bit, lut3d_9_12bit, lut3d_9_10bit} = lut3d_17_12bit - And shaper (1D) LUT - following what we already have for degamma/gamma LUT: │ ├───"SHAPER_LUT": blob = 0 │ └───"SHAPER_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 Each LUT3D_MODE element works as a index to get the blob for the corresponding `struct drm_mode_lut3d_mode` ([2] may works as an example). I've also included a patch to set the first element of the list of supported mode as a predefined value [3], but maybe it's not necessary (or even error prone). [1] https://lore.kernel.org/dri-devel/20230109143846.1966301-5-mwen@igalia.com/ [2] https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/a7b11bab9d573e38e26982b5cc5cf62e9dca6a75#ed5dbc960ac210e3fbacd2361fe0270709767aaa_886_886 [3] https://lore.kernel.org/dri-devel/20230109143846.1966301-7-mwen@igalia.com/ > > In that code example, I cannot see any connection between the array of > drm_mode_lut3d_mode entries, the 'name' string, and the actual data > generated for the LUT. They must all connect somehow, but it just > doesn't seem to be in the code. It is just hardcoding mode_lut3d[0], > and the data generator never sees even that - it gets the lut size, but > none of the other parameters. I cannot see that working across multiple > drivers. > > Why is it setting a linear shaper LUT explicitly instead of unsetting > it? I mean this preparation: > > shaper_linear = generate_table(data->shaper_lut_size, 1.0); > > "Linear" actually means "identity", right? That's what people usually > write when they mean identity. > > Surely it's not actually a constant curve? That would make the whole > test useless for proving the 3D LUT works. So, the idea of the test is only set a 3D LUT, without any shaper LUT and see if the 3D LUT maps to max color values. However, from my tests on AMD HW with 3D LUT support, I was not able to program a 3D LUT without a shaper LUT, that means a shaper LUT in BYPASS mode. By comparing CRC results I could see that if I don't set any shaper LUT, the driver sets shaper LUT in BYPASS mode but the 3D LUT is just ignored, so the combo shaper + 3D LUT seems a HW requirement. To overcome this requirement and keep the expected behavior, the test sets a linear shaper LUT to acts as a BYPASS LUT to overcome this requirement. Am I missing something? It's a really simple test case, but I don't see how this approach invalidates the test results. To generalize the test for other devices, I can restrict shaper LUT usage for AMD. Melissa > > > > > > > > + /** > > > + * @lut3d: > > > + * > > > + * 3D Lookup table for converting pixel data. Position where it takes > > > + * place depends on hw design, after @ctm or @gamma_lut. See > > > + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of > > > + * &struct drm_color_lut. > > > + */ > > > + struct drm_property_blob *lut3d; > > > > > > I do not like the wording of "depends on hw design", and it is used in > > > very many places here. The KMS UAPI semantics cannot vary based on > > > hardware. Your cover letter defines the order in the color pipeline, so > > > I don't understand how this here can depend on hw. > > > > > > What can depend on hardware is which KMS UAPI properties are exposed, > > > and how you map a property to a hardware unit (which can even change > > > based on the exact pipeline configuration as long as the results are as > > > the UAPI doc defines). But this comment here is talking about the UAPI > > > properties, not hw elements. > > > > > > > You are right! My initial idea was to explain that it's possible for > > other vendors color pipeline to fit this pipeline internally, if they > > need a 1D LUT before the 3D LUT, but not the 1D LUT in the end. > > > > > > > > I'm happy that the 3D LUT interface is being developed, but as you can > > > see from my questions, the UAPI documentation is practically missing. I > > > would have no idea how to use this as is. > > > > Thank you again for your valuable comments. I'll address your comments > > in a next version by better explaining all these points. > > Thank you for working on this! > pq
On 02/13, Ville Syrjälä wrote: > On Mon, Feb 13, 2023 at 11:01:31AM +0200, Pekka Paalanen wrote: > > On Fri, 10 Feb 2023 14:47:50 -0500 > > Harry Wentland <harry.wentland@amd.com> wrote: > > > > > On 2/10/23 04:28, Pekka Paalanen wrote: > > > > On Thu, 9 Feb 2023 13:27:02 -0100 > > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > >> On 01/31, Pekka Paalanen wrote: > > > >>> On Mon, 9 Jan 2023 14:38:09 -0100 > > > >>> Melissa Wen <mwen@igalia.com> wrote: > > > >>> > > > >>>> On 01/09, Melissa Wen wrote: > > > >>>>> Hi, > > > >>>>> > > > >>>>> After collecting comments in different places, here is a second version > > > >>>>> of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > >>>>> mgmt interface. In comparison to previous proposals [1][2][3], here we > > > >>>>> add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > >>>>> that means the following DRM CRTC color correction pipeline: > > > >>>>> > > > >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > ... > > > > > >>> +/* > > > >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > >>> + * @lut_size: number of valid points on every dimension of 3D LUT. > > > >>> + * @lut_stride: number of points on every dimension of 3D LUT. > > > >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > >>> + * bit_depth the least significant bits will be truncated. > > > >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > >>> + * @flags: flags for hardware-sepcific features > > > >>> + */ > > > >>> +struct drm_mode_lut3d_mode { > > > >>> + __u16 lut_size; > > > >>> + __u16 lut_stride[3]; > > > >>> + __u16 bit_depth; > > > >>> + __u32 color_format; > > > >>> + __u32 flags; > > > >>> +}; > > > > ... > > > > > >>> What is "number of bits of RGB"? Input precision? Output precision? > > > >>> Integer or floating point? > > > >> > > > >> It's the bit depth of the 3D LUT values, the same for every channels. In > > > >> the AMD case, it's supports 10-bit and 12-bit, for example. > > > > > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > > > > hardware ever? > > > > > > > > > > I haven't had a chance to go through all patches yet but if this is > > > modeled after Alex Hung's work this should be covered by color_format. > > > The idea is that color_format takes a FOURCC value and defines the > > > format of the entries in the 3DLUT blob. > > > > > > The bit_depth describes the actual bit depth that the HW supports. > > > E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might only > > > support 12-bit precision. In that case the least significant bits get > > > truncated. > > > > > > One could define the bit_depth per color, but I'm not sure that'll be > > > necessary. > > > > Exactly. I just have no idea how sure we should be about that. > > > > > > What exactly is the truncation the comment refers to? > > > > > > > > It sounds like if input has higher precision than the LUT elements, > > > > then "truncation" occurs. I can kind of see that, but I also think it > > > > is a false characterisation. The LUT input precision affects the > > > > precision of LUT indexing and the precision of interpolation between > > > > the LUT elements. I would not expect those two precisions to be > > > > truncated to the LUT element precision (but they could be truncated to > > > > something else hardware specific). Instead, I do expect the > > > > interpolation result to be truncated to the LUT output precision, which > > > > probably is the same as the LUT element precision, but not necessarily. > > > > > > > > Maybe the comment about truncation should simply be removed? The result > > > > is obvious if we know the LUT input, element, and output precision, and > > > > what exactly happens with the indexing and interpolation is probably > > > > good enough to be left hardware-specific if it is difficult to describe > > > > in generic terms across different hardware. > > > > > > > > > > Maybe it makes sense to just drop the bit_depth field. > > > > Well, it's really interesting information for userspace, but maybe it > > should have a more holistic design. Precision is a factor, when > > userspace considers whether it can use KMS hardware for a conversion or > > not. Unfortunately, none of the existing KMS color pipeline elements > > have any information on precision IIRC, so there is more to be fixed. > > > > The interesting thing is the minimum guaranteed precision of each > > element and the connections between them. It might be different for > > pass-through vs. not. Another interesting thing is the usable value > > range. > > > > This is probably a complex problem, so there should be no need to solve > > it before a 3D LUT interface can land, given old elements already have > > the issue. > > Yeah, I think all the precision stuff is all better handled by > eg. the proposed GAMMA_MODE property or something similar. > It's going to be needed for 1D LUTs as well. 1D LUTs would > also need it to expose diffrent LUT sizes with different > precision tradeoffs. > > As for the 3D LUT blob, I don't think the blob needs any > strides/etc. either. I had none of that for my i915 version: > https://github.com/vsyrjala/linux/commits/3dlut > Just the LUT entries + blob size is sufficient. At least > for cube shaped LUTs. Dunno if anyone would have a need > for something else? I only use lut_size and bit_depth for programming a CRTC 3D LUT in this proposal, so far GAMMA_MODE also would fit. But don't know for pre-blending 3D LUT. > > The two things the are absolutely needed: > - Position of the LUT in the pipeline. We've already > seen at least two variants of this IIRC, so we'll > likely need to define a unique property for each tap > point. IIRC, I'd say three, since in rcar-du the 3D LUT is before the gamma LUT, but there isn't a shaper 1D LUT before 3D LUT. I'd like to know how the gamma LUT pre-3D LUT acts on intel pipeline. If it's, in the end, somehow similar to a shaper LUT. I mean, if we don't name the LUTs after CTM, we could fit something similar in terms of dimensions, as: -> CTM -> 1D LUT -> 3D LUT -> 1D LUT > - The order the elements are stored in the blob. I didn't > check if all the already known hw (amdgpu, i915, rcar-du, > were there also others?) would agree on this or not. > If yes, maybe just follow the hw order for all those, > and if not, then I guess flip a few coins. > > -- > Ville Syrjälä > Intel
On Mon, 13 Feb 2023 18:26:55 -0100 Melissa Wen <mwen@igalia.com> wrote: > On 02/10, Pekka Paalanen wrote: > > On Thu, 9 Feb 2023 13:27:02 -0100 > > Melissa Wen <mwen@igalia.com> wrote: > > > > > On 01/31, Pekka Paalanen wrote: > > > > On Mon, 9 Jan 2023 14:38:09 -0100 > > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > > > On 01/09, Melissa Wen wrote: > > > > > > Hi, > > > > > > > > > > > > After collecting comments in different places, here is a second version > > > > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > > > that means the following DRM CRTC color correction pipeline: > > > > > > > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > > > Hi Melissa, > > > > > > > > that makes sense to me, for CRTCs. It would be really good to have that > > > > as a diagram in the KMS UAPI documentation. > > > > > > > > > > Hi Pekka, > > > > > > Thanks for your feedbacks and your time reviewing this proposal. > > > > No problem, and sorry it took so long! > > > > I'm just finishing the catch-up with everything that happened during > > winter holidays. > > > > > > If someone wants to add a 3D LUT to KMS planes as well, then I'm not > > > > sure if it should be this order or swapped. I will probably have an > > > > opinion about that once Weston is fully HDR capable and has been tried > > > > in the wild for a while with the HDR color operations fine-tuned based > > > > on community feedback. IOW, not for a long time. The YUV to RGB > > > > conversion factors in there as well. > > > > > > > I see, this is also the reason I reuse here Alex Hung's proposal for > > > pre-blending API. I'll work on better documentation. > > > > > > > > > > > > > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > > > > driver, exposing modes supported by HW and programming user shaper and > > > > > > 3D LUT accordingly. > > > > > > > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > > > > > > > Let me know your thoughts. > > > > > > > > > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. > > > > > > > > Unfortunately I don't have the patch emails to reply to, so here's a > > > > messy bunch of comments. I'll concentrate on the UAPI design as always. > > > > > > Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ > > > In the next version, I won't forget cc'ing you at first. > > > > > > > > +/* > > > > + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > > + * @lut_size: number of valid points on every dimension of 3D LUT. > > > > + * @lut_stride: number of points on every dimension of 3D LUT. > > > > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > > + * bit_depth the least significant bits will be truncated. > > > > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > > + * @flags: flags for hardware-sepcific features > > > > + */ > > > > +struct drm_mode_lut3d_mode { > > > > + __u16 lut_size; > > > > + __u16 lut_stride[3]; > > > > + __u16 bit_depth; > > > > + __u32 color_format; > > > > + __u32 flags; > > > > +}; > > > > > > > > Why is lut_stride an array of 3, but lut_size is not? > > > > > > It cames from VA-API: > > > https://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html#a682756be15d09327ba725b74a863cbcc > > > > > > In short, the reason is that lut_size is the valid points and is > > > the same for every dimensions, but lut_stride may vary. > > > > Ok, so lut_size is what I would have guessed it to be. Just needs to be > > doc'd - pretty much all my questions are lack of docs. > > > > I see that lut_stride is supposed to be in points, and not bytes. That > > is surprising. Are you sure it's good? > > > > > > > > > > What is the color_mode the comment is referring to? > > > > > > It refers to FB color_mode/bpp. I'm not using it in post-blending 3D LUT > > > implementation (should I?), it cames from pre-blending use case. Maybe > > > the main issue here is if reusing the pre-blending 3D LUT mode struct is > > > a good approach or better create a specific for post-blending. > > > > Sorry, I have no idea what FB color_mode is. I do not recall any such > > thing in the KMS uAPI. > > > > > > > > > > > > What is "number of bits of RGB"? Input precision? Output precision? > > > > Integer or floating point? > > > > > > It's the bit depth of the 3D LUT values, the same for every channels. In > > > the AMD case, it's supports 10-bit and 12-bit, for example. > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > > hardware ever? > > > > What exactly is the truncation the comment refers to? > > > > It sounds like if input has higher precision than the LUT elements, > > then "truncation" occurs. I can kind of see that, but I also think it > > is a false characterisation. The LUT input precision affects the > > precision of LUT indexing and the precision of interpolation between > > the LUT elements. I would not expect those two precisions to be > > truncated to the LUT element precision (but they could be truncated to > > something else hardware specific). Instead, I do expect the > > interpolation result to be truncated to the LUT output precision, which > > probably is the same as the LUT element precision, but not necessarily. > > > > Maybe the comment about truncation should simply be removed? The result > > is obvious if we know the LUT input, element, and output precision, and > > what exactly happens with the indexing and interpolation is probably > > good enough to be left hardware-specific if it is difficult to describe > > in generic terms across different hardware. > > > > > > > > > > Flags cannot be hardware specific, because it makes the whole KMS UAPI > > > > hardware specific. That won't work. You have to have driver-agnostic > > > > definitions for all possible flags. > > > > > > > > Why is this the whole first patch? There is no documentation for the > > > > UAPI on how this struct works, so I cannot review this. Explaining just > > > > the individual fields is not enough to understand it. Is this something > > > > the kernel fills in and is read-only to userspace? Is userspace filling > > > > this in? > > > > > > I see. I'll work on explaining/documenting it better. > > > > > > > > > > > > + * “LUT3D”: > > > > + * Blob property to set the 3D LUT mapping pixel data after the color > > > > + * transformation matrix and before gamma 1D lut correction. The > > > > + * data is interpreted as an array of &struct drm_color_lut elements. > > > > + * Hardware might choose not to use the full precision of the LUT > > > > + * elements. > > > > + * > > > > + * Setting this to NULL (blob property value set to 0) means a the output > > > > + * color is identical to the input color. This is generally the driver > > > > + * boot-up state too. Drivers can access this blob through > > > > + * &drm_crtc_state.gamma_lut. > > > > + * > > > > > > > > You need to define how the 1-D array of drm_color_lut elements blob > > > > will be interpreted as a 3-D array for the 3D LUT, and how the > > > > dimensions match to the R, G and B channels. It's a bit like the > > > > question about row-major or column-major storage for matrices, except > > > > more complicated and not in those words. > > > > > > ack > > > > > > > > + * “LUT3D_MODE”: > > > > + * Enum property to give the mode of the 3D lookup table to be set on the > > > > + * LUT3D property. A mode specifies size, stride, bit depth and color > > > > + * format and depends on the underlying hardware). If drivers support > > > > + * multiple 3D LUT modes, they should be declared in a array of > > > > + * drm_color_lut3d_mode and they will be advertised as an enum. > > > > > > > > How does that work exactly? I didn't get it. I could guess, but having > > > > to guess on API is bad. > > > > > > The driver advertises all supported modes (each combination of values) > > > in a array as a enum, userspace can check all accepted modes and set the > > > one that fits the user 3D LUT settings. I think it's possible to get the > > > idea from this IGT test: > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 > > > > > > > > You lost me at "an array as an enum". > > > > I understand there is a blob containing an array of struct > > drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE > > is an enum property. Where does the blob come from, then? What property > > provides the blob? > > > > Am I correct in guessing that the values of LUT3D_MODE enum property > > are indices into the array in the blob, and that userspace will set it? > > That sounds good to me, if it's the integer value of the enum. But enum > > values also need string names, because that is how the values are > > usually recognized, so what name strings will be used? > > So, in this proposal, LUT3D_MODE is a list of indexes for a blob that describe a supported > 3D LUT mode Hi Melissa, are you sure? I believe you are looking at and explaining some kernel internal APIs, and not the uAPI which is the important part. Internal APIs can always be changed later, uAPI cannot. After I had sent that email, I might have understood how it was supposed to work instead: there is no array of struct drm_mode_lut3d_mode. Instead, LUT3D_MODE is a KMS property of type enum. Each integer value of the enum is also a blob id. Each blob named by those ids is a single struct drm_mode_lut3d_mode that userspace needs to retrieve individually to understand what that specific enum value means. > i.e. a `struct drm_mode_lut3d_mode` with size, stride, bit depth, etc. Strings here follow this pattern > `lut3d_{size}_{bit_depth}bit` [1]. When enabling 3D LUT support, the > driver should pass an array of `struct drm_mode_lut3d_mode` as supported > modes, with at least one element. To be clear, I do not care about kernel internal interfaces at all. I only care about the uAPI. Therefore talking about kernel internal API will only confuse me and every other userspace developer. Evidently it has even confused some IGT developers, as I point out some IGT code problems below. If the struct drm_mode_lut3d_mode is the authoritative definition of what each enum value means, then I think the string names should not attempt to convey any meaning nor information. The strings should be just "blob id 77" etc. That makes it clear what the value is. Otherwise userspace will be tempted to use the string names only (as is the usual uAPI design) to find the right enum value, and not look into the struct drm_mode_lut3d_mode at all. Therefore, if you don't encode absolutely everything of struct drm_mode_lut3d_mode into the string name, you should encode nothing in the name. You might also have two different descriptions ending up with the same string name under the same enum property instance, and that must not happen. > I think this drm_info output helps to understand the idea. Currently, we > have 5 DRM CRTC color mgmt properties. The drm_info output for them: > > │ ├───"DEGAMMA_LUT": blob = 0 > │ ├───"DEGAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 > │ ├───"CTM": blob = 0 > │ ├───"GAMMA_LUT": blob = 0 > │ ├───"GAMMA_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 > > The idea is to add 4 new properties. > > - The 3D LUT blob (LUT3D) and the indexes of supported modes > (LUT3D_MODE), as follows (example with AMD supported modes): > > │ ├───"LUT3D": blob = 0 > │ ├───"LUT3D_MODE": enum {lut3d_17_12bit, lut3d_17_10bit, lut3d_9_12bit, lut3d_9_10bit} = lut3d_17_12bit Ok, but this does not show the difference between an index into an array of struct drm_mode_lut3d_mode, and each enum value being a blob id containing a single struct drm_mode_lut3d_mode. > > - And shaper (1D) LUT - following what we already have for degamma/gamma LUT: > > │ ├───"SHAPER_LUT": blob = 0 > │ └───"SHAPER_LUT_SIZE" (immutable): range [0, UINT32_MAX] = 4096 > > > Each LUT3D_MODE element works as a index to get the blob for the > corresponding `struct drm_mode_lut3d_mode` ([2] may works as an > example). I've also included a patch to set the first element of the > list of supported mode as a predefined value [3], but maybe it's not > necessary (or even error prone). Are you sure about that terminology in the uAPI? A blob id is not an index. If you do have an index, then which KMS property is going to tell userspace what is the blob id that contains the whole array? I haven't seen any KMS property for that yet. > [1] https://lore.kernel.org/dri-devel/20230109143846.1966301-5-mwen@igalia.com/ Actually, in that patch, I see a loop that creates a new blob from each element in a kernel-private array of struct drm_mode_lut3d_mode. > [2] https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/a7b11bab9d573e38e26982b5cc5cf62e9dca6a75#ed5dbc960ac210e3fbacd2361fe0270709767aaa_886_886 That is userspace code, which does: mode_3dlut = (struct drm_mode_3dlut_mode *) blob->data; count = blob->length / sizeof(mode_3dlut[0]); igt_assert(count); for (i = 0; i < count; i++) { That code is semantically wrong. The blob data is never an array. It is always a single struct drm_mode_lut3d_mode. This code is misleading the reader really hard. > [3] https://lore.kernel.org/dri-devel/20230109143846.1966301-7-mwen@igalia.com/ Well, it's an enum property. A property always has some value. An enum must always enumerate all possible values. Therefore, you cannot ever avoid setting some value as the default. If you set zero as a value, then zero must also be enumerated. I mentioned before that the drm_mode_lut3d_mode values should be named "blob id %u". The zero value is not a blob id, it is a true constant instead, so its name can be e.g. "pass-through" if that is what it means. Or maybe that doesn't make sense, because setting the LUT3D property to zero means "pass-through", in which case zero is not a valid value for LUT3D_MODE ever, not even in the kernel (but sounds like the DRM core does not guard against the driver setting enum values that do not exist). Note, that value zero literally means "no data". It is not the index to the first struct drm_mode_lut3d_mode element. So the userspace code of [2] is actually broken with respect to this: it will attempt to fetch a blob id 0, and crash an assert. If it does not crash, then I think the enum description is lying to userspace: zero is a valid value that is not listed as existing. Or is there some uAPI specification somewhere saying that any enum property can be set to zero as well, regardless of what values are listed as possible? I would hope not. > > > > In that code example, I cannot see any connection between the array of > > drm_mode_lut3d_mode entries, the 'name' string, and the actual data > > generated for the LUT. They must all connect somehow, but it just > > doesn't seem to be in the code. It is just hardcoding mode_lut3d[0], > > and the data generator never sees even that - it gets the lut size, but > > none of the other parameters. I cannot see that working across multiple > > drivers. > > > > Why is it setting a linear shaper LUT explicitly instead of unsetting > > it? I mean this preparation: > > > > shaper_linear = generate_table(data->shaper_lut_size, 1.0); > > > > "Linear" actually means "identity", right? That's what people usually > > write when they mean identity. > > > > Surely it's not actually a constant curve? That would make the whole > > test useless for proving the 3D LUT works. > > So, the idea of the test is only set a 3D LUT, without any shaper LUT > and see if the 3D LUT maps to max color values. However, from my tests > on AMD HW with 3D LUT support, I was not able to program a 3D LUT > without a shaper LUT, that means a shaper LUT in BYPASS mode. By > comparing CRC results I could see that if I don't set any shaper LUT, > the driver sets shaper LUT in BYPASS mode but the 3D LUT is just > ignored, so the combo shaper + 3D LUT seems a HW requirement. To > overcome this requirement and keep the expected behavior, the test sets > a linear shaper LUT to acts as a BYPASS LUT to overcome this > requirement. Am I missing something? Sorry, no, I just had no idea what generate_table(..., 1.0) does. I confused it with generate_table_max() which I have seen used before. However, this exposes a kernel bug: the driver requires both the shaper and the 3D LUT. If userspace sets the shaper to pass-through, you say the 3D LUT gets ignored. That's a bug: the if the driver cannot make 3D LUT work without a shaper, then it must invent an identity shaper when userspace sets it to pass-through. Or fail the atomic commit. Userspace could as well set the 3D LUT to pass-through and the shaper to something else, and expect it to work. If the atomic test commit does not fail, it must do what userspace prescribes. You just cannot expose arbitrary hardware limitations to userspace like that and silently malfunction if they are violated. At the very least fail the atomic commit if you don't want to make it work in the driver. Thanks, pq > > It's a really simple test case, but I don't see how this approach > invalidates the test results. To generalize the test for other devices, > I can restrict shaper LUT usage for AMD. > > Melissa > > > > > > > > > > > > > + /** > > > > + * @lut3d: > > > > + * > > > > + * 3D Lookup table for converting pixel data. Position where it takes > > > > + * place depends on hw design, after @ctm or @gamma_lut. See > > > > + * drm_crtc_enable_color_mgmt(). The blob (if not NULL) is an array of > > > > + * &struct drm_color_lut. > > > > + */ > > > > + struct drm_property_blob *lut3d; > > > > > > > > I do not like the wording of "depends on hw design", and it is used in > > > > very many places here. The KMS UAPI semantics cannot vary based on > > > > hardware. Your cover letter defines the order in the color pipeline, so > > > > I don't understand how this here can depend on hw. > > > > > > > > What can depend on hardware is which KMS UAPI properties are exposed, > > > > and how you map a property to a hardware unit (which can even change > > > > based on the exact pipeline configuration as long as the results are as > > > > the UAPI doc defines). But this comment here is talking about the UAPI > > > > properties, not hw elements. > > > > > > > > > > You are right! My initial idea was to explain that it's possible for > > > other vendors color pipeline to fit this pipeline internally, if they > > > need a 1D LUT before the 3D LUT, but not the 1D LUT in the end. > > > > > > > > > > > I'm happy that the 3D LUT interface is being developed, but as you can > > > > see from my questions, the UAPI documentation is practically missing. I > > > > would have no idea how to use this as is. > > > > > > Thank you again for your valuable comments. I'll address your comments > > > in a next version by better explaining all these points. > > > > Thank you for working on this! > > pq > >
On Mon, 13 Feb 2023 18:45:40 -0100 Melissa Wen <mwen@igalia.com> wrote: > On 02/13, Ville Syrjälä wrote: > > On Mon, Feb 13, 2023 at 11:01:31AM +0200, Pekka Paalanen wrote: > > > On Fri, 10 Feb 2023 14:47:50 -0500 > > > Harry Wentland <harry.wentland@amd.com> wrote: > > > > > > > On 2/10/23 04:28, Pekka Paalanen wrote: > > > > > On Thu, 9 Feb 2023 13:27:02 -0100 > > > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > > > >> On 01/31, Pekka Paalanen wrote: > > > > >>> On Mon, 9 Jan 2023 14:38:09 -0100 > > > > >>> Melissa Wen <mwen@igalia.com> wrote: > > > > >>> > > > > >>>> On 01/09, Melissa Wen wrote: > > > > >>>>> Hi, > > > > >>>>> > > > > >>>>> After collecting comments in different places, here is a second version > > > > >>>>> of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > >>>>> mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > >>>>> add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > >>>>> that means the following DRM CRTC color correction pipeline: > > > > >>>>> > > > > >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > ... > > > > > > > >>> +/* > > > > >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > > >>> + * @lut_size: number of valid points on every dimension of 3D LUT. > > > > >>> + * @lut_stride: number of points on every dimension of 3D LUT. > > > > >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > > >>> + * bit_depth the least significant bits will be truncated. > > > > >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > > >>> + * @flags: flags for hardware-sepcific features > > > > >>> + */ > > > > >>> +struct drm_mode_lut3d_mode { > > > > >>> + __u16 lut_size; > > > > >>> + __u16 lut_stride[3]; > > > > >>> + __u16 bit_depth; > > > > >>> + __u32 color_format; > > > > >>> + __u32 flags; > > > > >>> +}; > > > > > > ... > > > > > > > >>> What is "number of bits of RGB"? Input precision? Output precision? > > > > >>> Integer or floating point? > > > > >> > > > > >> It's the bit depth of the 3D LUT values, the same for every channels. In > > > > >> the AMD case, it's supports 10-bit and 12-bit, for example. > > > > > > > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on any > > > > > hardware ever? > > > > > > > > > > > > > I haven't had a chance to go through all patches yet but if this is > > > > modeled after Alex Hung's work this should be covered by color_format. > > > > The idea is that color_format takes a FOURCC value and defines the > > > > format of the entries in the 3DLUT blob. > > > > > > > > The bit_depth describes the actual bit depth that the HW supports. > > > > E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might only > > > > support 12-bit precision. In that case the least significant bits get > > > > truncated. > > > > > > > > One could define the bit_depth per color, but I'm not sure that'll be > > > > necessary. > > > > > > Exactly. I just have no idea how sure we should be about that. > > > > > > > > What exactly is the truncation the comment refers to? > > > > > > > > > > It sounds like if input has higher precision than the LUT elements, > > > > > then "truncation" occurs. I can kind of see that, but I also think it > > > > > is a false characterisation. The LUT input precision affects the > > > > > precision of LUT indexing and the precision of interpolation between > > > > > the LUT elements. I would not expect those two precisions to be > > > > > truncated to the LUT element precision (but they could be truncated to > > > > > something else hardware specific). Instead, I do expect the > > > > > interpolation result to be truncated to the LUT output precision, which > > > > > probably is the same as the LUT element precision, but not necessarily. > > > > > > > > > > Maybe the comment about truncation should simply be removed? The result > > > > > is obvious if we know the LUT input, element, and output precision, and > > > > > what exactly happens with the indexing and interpolation is probably > > > > > good enough to be left hardware-specific if it is difficult to describe > > > > > in generic terms across different hardware. > > > > > > > > > > > > > Maybe it makes sense to just drop the bit_depth field. > > > > > > Well, it's really interesting information for userspace, but maybe it > > > should have a more holistic design. Precision is a factor, when > > > userspace considers whether it can use KMS hardware for a conversion or > > > not. Unfortunately, none of the existing KMS color pipeline elements > > > have any information on precision IIRC, so there is more to be fixed. > > > > > > The interesting thing is the minimum guaranteed precision of each > > > element and the connections between them. It might be different for > > > pass-through vs. not. Another interesting thing is the usable value > > > range. > > > > > > This is probably a complex problem, so there should be no need to solve > > > it before a 3D LUT interface can land, given old elements already have > > > the issue. > > > > Yeah, I think all the precision stuff is all better handled by > > eg. the proposed GAMMA_MODE property or something similar. > > It's going to be needed for 1D LUTs as well. 1D LUTs would > > also need it to expose diffrent LUT sizes with different > > precision tradeoffs. > > > > As for the 3D LUT blob, I don't think the blob needs any > > strides/etc. either. I had none of that for my i915 version: > > https://github.com/vsyrjala/linux/commits/3dlut > > Just the LUT entries + blob size is sufficient. At least > > for cube shaped LUTs. Dunno if anyone would have a need > > for something else? > > I only use lut_size and bit_depth for programming a CRTC 3D LUT in this > proposal, so far GAMMA_MODE also would fit. But don't know for > pre-blending 3D LUT. You mean in the driver, or in IGT? Surely every field of struct drm_mode_lut3d_mode needs to be used and tested, for values that are possible for the hardware? If the struct has strides, then the driver must set them and userspace must use them, naturally. Userspace like IGT cannot go and assume something without even looking at the struct fields. This is why I was surprised to see the IGT code ignore most of the struct drm_mode_lut3d_mode fields. > > > > The two things the are absolutely needed: > > - Position of the LUT in the pipeline. We've already > > seen at least two variants of this IIRC, so we'll > > likely need to define a unique property for each tap > > point. > > IIRC, I'd say three, since in rcar-du the 3D LUT is before the gamma > LUT, but there isn't a shaper 1D LUT before 3D LUT. I'd like to know > how the gamma LUT pre-3D LUT acts on intel pipeline. If it's, in the > end, somehow similar to a shaper LUT. > > I mean, if we don't name the LUTs after CTM, we could fit something > similar in terms of dimensions, as: > > -> CTM -> 1D LUT -> 3D LUT -> 1D LUT Yes, to me the LUT names have always been irrelevant. The names are only a way to identify an element's position in the abstract KMS color pipeline defined by all defined KMS properties. Thanks, pq > > - The order the elements are stored in the blob. I didn't > > check if all the already known hw (amdgpu, i915, rcar-du, > > were there also others?) would agree on this or not. > > If yes, maybe just follow the hw order for all those, > > and if not, then I guess flip a few coins. > > > > -- > > Ville Syrjälä > > Intel
[AMD Official Use Only - General] + Uday, for awareness. Regards Shashank -----Original Message----- From: Pekka Paalanen <ppaalanen@gmail.com> Sent: 14 February 2023 10:28 To: Melissa Wen <mwen@igalia.com> Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>; dri-devel@lists.freedesktop.org; airlied@gmail.com; laurent.pinchart+renesas@ideasonboard.com; Sharma, Shashank <Shashank.Sharma@amd.com>; Siqueira, Rodrigo <Rodrigo.Siqueira@amd.com>; amd-gfx@lists.freedesktop.org; Hung, Alex <Alex.Hung@amd.com>; Wentland, Harry <Harry.Wentland@amd.com>; tzimmermann@suse.de; Li, Sun peng (Leo) <Sunpeng.Li@amd.com>; maarten.lankhorst@linux.intel.com; mripard@kernel.org; seanpaul@chromium.org; daniel@ffwll.ch; Lakha, Bhawanpreet <Bhawanpreet.Lakha@amd.com>; Kim, Sung joon <Sungjoon.Kim@amd.com>; contact@emersion.fr; Pan, Xinhui <Xinhui.Pan@amd.com>; Koenig, Christian <Christian.Koenig@amd.com>; kernel-dev@igalia.com; Deucher, Alexander <Alexander.Deucher@amd.com>; Kazlauskas, Nicholas <Nicholas.Kazlauskas@amd.com>; Joshua Ashton <joshua@froggi.es> Subject: Re: [RFC PATCH v2 00/18] Add DRM CRTC 3D LUT interface On Mon, 13 Feb 2023 18:45:40 -0100 Melissa Wen <mwen@igalia.com> wrote: > On 02/13, Ville Syrjälä wrote: > > On Mon, Feb 13, 2023 at 11:01:31AM +0200, Pekka Paalanen wrote: > > > On Fri, 10 Feb 2023 14:47:50 -0500 Harry Wentland > > > <harry.wentland@amd.com> wrote: > > > > > > > On 2/10/23 04:28, Pekka Paalanen wrote: > > > > > On Thu, 9 Feb 2023 13:27:02 -0100 Melissa Wen > > > > > <mwen@igalia.com> wrote: > > > > > > > > > >> On 01/31, Pekka Paalanen wrote: > > > > >>> On Mon, 9 Jan 2023 14:38:09 -0100 Melissa Wen > > > > >>> <mwen@igalia.com> wrote: > > > > >>> > > > > >>>> On 01/09, Melissa Wen wrote: > > > > >>>>> Hi, > > > > >>>>> > > > > >>>>> After collecting comments in different places, here is a > > > > >>>>> second version of the work on adding DRM CRTC 3D LUT > > > > >>>>> support to the current DRM color mgmt interface. In > > > > >>>>> comparison to previous proposals [1][2][3], here we add 3D > > > > >>>>> LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, that means the following DRM CRTC color correction pipeline: > > > > >>>>> > > > > >>>>> Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > ... > > > > > > > >>> +/* > > > > >>> + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > > >>> + * @lut_size: number of valid points on every dimension of 3D LUT. > > > > >>> + * @lut_stride: number of points on every dimension of 3D LUT. > > > > >>> + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > > >>> + * bit_depth the least significant bits will be truncated. > > > > >>> + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > > >>> + * @flags: flags for hardware-sepcific features */ struct > > > > >>> +drm_mode_lut3d_mode { > > > > >>> + __u16 lut_size; > > > > >>> + __u16 lut_stride[3]; > > > > >>> + __u16 bit_depth; > > > > >>> + __u32 color_format; > > > > >>> + __u32 flags; > > > > >>> +}; > > > > > > ... > > > > > > > >>> What is "number of bits of RGB"? Input precision? Output precision? > > > > >>> Integer or floating point? > > > > >> > > > > >> It's the bit depth of the 3D LUT values, the same for every channels. In > > > > >> the AMD case, it's supports 10-bit and 12-bit, for example. > > > > > > > > > > Ok. So e.g. r5g6b5 is not a possible 3D LUT element type on > > > > > any hardware ever? > > > > > > > > > > > > > I haven't had a chance to go through all patches yet but if this > > > > is modeled after Alex Hung's work this should be covered by color_format. > > > > The idea is that color_format takes a FOURCC value and defines > > > > the format of the entries in the 3DLUT blob. > > > > > > > > The bit_depth describes the actual bit depth that the HW supports. > > > > E.g., color_format could be DRM_FORMAT_XRGB16161616 but HW might > > > > only support 12-bit precision. In that case the least > > > > significant bits get truncated. > > > > > > > > One could define the bit_depth per color, but I'm not sure > > > > that'll be necessary. > > > > > > Exactly. I just have no idea how sure we should be about that. > > > > > > > > What exactly is the truncation the comment refers to? > > > > > > > > > > It sounds like if input has higher precision than the LUT > > > > > elements, then "truncation" occurs. I can kind of see that, > > > > > but I also think it is a false characterisation. The LUT input > > > > > precision affects the precision of LUT indexing and the > > > > > precision of interpolation between the LUT elements. I would > > > > > not expect those two precisions to be truncated to the LUT > > > > > element precision (but they could be truncated to something > > > > > else hardware specific). Instead, I do expect the > > > > > interpolation result to be truncated to the LUT output precision, which probably is the same as the LUT element precision, but not necessarily. > > > > > > > > > > Maybe the comment about truncation should simply be removed? > > > > > The result is obvious if we know the LUT input, element, and > > > > > output precision, and what exactly happens with the indexing > > > > > and interpolation is probably good enough to be left > > > > > hardware-specific if it is difficult to describe in generic terms across different hardware. > > > > > > > > > > > > > Maybe it makes sense to just drop the bit_depth field. > > > > > > Well, it's really interesting information for userspace, but maybe > > > it should have a more holistic design. Precision is a factor, when > > > userspace considers whether it can use KMS hardware for a > > > conversion or not. Unfortunately, none of the existing KMS color > > > pipeline elements have any information on precision IIRC, so there is more to be fixed. > > > > > > The interesting thing is the minimum guaranteed precision of each > > > element and the connections between them. It might be different > > > for pass-through vs. not. Another interesting thing is the usable > > > value range. > > > > > > This is probably a complex problem, so there should be no need to > > > solve it before a 3D LUT interface can land, given old elements > > > already have the issue. > > > > Yeah, I think all the precision stuff is all better handled by eg. > > the proposed GAMMA_MODE property or something similar. > > It's going to be needed for 1D LUTs as well. 1D LUTs would also need > > it to expose diffrent LUT sizes with different precision tradeoffs. > > > > As for the 3D LUT blob, I don't think the blob needs any > > strides/etc. either. I had none of that for my i915 version: > > https://github.com/vsyrjala/linux/commits/3dlut > > Just the LUT entries + blob size is sufficient. At least for cube > > shaped LUTs. Dunno if anyone would have a need for something else? > > I only use lut_size and bit_depth for programming a CRTC 3D LUT in > this proposal, so far GAMMA_MODE also would fit. But don't know for > pre-blending 3D LUT. You mean in the driver, or in IGT? Surely every field of struct drm_mode_lut3d_mode needs to be used and tested, for values that are possible for the hardware? If the struct has strides, then the driver must set them and userspace must use them, naturally. Userspace like IGT cannot go and assume something without even looking at the struct fields. This is why I was surprised to see the IGT code ignore most of the struct drm_mode_lut3d_mode fields. > > > > The two things the are absolutely needed: > > - Position of the LUT in the pipeline. We've already > > seen at least two variants of this IIRC, so we'll > > likely need to define a unique property for each tap > > point. > > IIRC, I'd say three, since in rcar-du the 3D LUT is before the gamma > LUT, but there isn't a shaper 1D LUT before 3D LUT. I'd like to know > how the gamma LUT pre-3D LUT acts on intel pipeline. If it's, in the > end, somehow similar to a shaper LUT. > > I mean, if we don't name the LUTs after CTM, we could fit something > similar in terms of dimensions, as: > > -> CTM -> 1D LUT -> 3D LUT -> 1D LUT Yes, to me the LUT names have always been irrelevant. The names are only a way to identify an element's position in the abstract KMS color pipeline defined by all defined KMS properties. Thanks, pq > > - The order the elements are stored in the blob. I didn't > > check if all the already known hw (amdgpu, i915, rcar-du, > > were there also others?) would agree on this or not. > > If yes, maybe just follow the hw order for all those, > > and if not, then I guess flip a few coins. > > > > -- > > Ville Syrjälä > > Intel
On Tue, 14 Feb 2023 11:19:47 +0200 Pekka Paalanen <ppaalanen@gmail.com> wrote: > On Mon, 13 Feb 2023 18:26:55 -0100 > Melissa Wen <mwen@igalia.com> wrote: > > > On 02/10, Pekka Paalanen wrote: > > > On Thu, 9 Feb 2023 13:27:02 -0100 > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > On 01/31, Pekka Paalanen wrote: > > > > > On Mon, 9 Jan 2023 14:38:09 -0100 > > > > > Melissa Wen <mwen@igalia.com> wrote: > > > > > > > > > > > On 01/09, Melissa Wen wrote: > > > > > > > Hi, > > > > > > > > > > > > > > After collecting comments in different places, here is a second version > > > > > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > > > > that means the following DRM CRTC color correction pipeline: > > > > > > > > > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > > > > > Hi Melissa, > > > > > > > > > > that makes sense to me, for CRTCs. It would be really good to have that > > > > > as a diagram in the KMS UAPI documentation. > > > > > > > > > > > > > Hi Pekka, > > > > > > > > Thanks for your feedbacks and your time reviewing this proposal. > > > > > > No problem, and sorry it took so long! > > > > > > I'm just finishing the catch-up with everything that happened during > > > winter holidays. > > > > > > > > If someone wants to add a 3D LUT to KMS planes as well, then I'm not > > > > > sure if it should be this order or swapped. I will probably have an > > > > > opinion about that once Weston is fully HDR capable and has been tried > > > > > in the wild for a while with the HDR color operations fine-tuned based > > > > > on community feedback. IOW, not for a long time. The YUV to RGB > > > > > conversion factors in there as well. > > > > > > > > > I see, this is also the reason I reuse here Alex Hung's proposal for > > > > pre-blending API. I'll work on better documentation. > > > > > > > > > > > > > > > > > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > > > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > > > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > > > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > > > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > > > > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > > > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > > > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > > > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > > > > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > > > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > > > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > > > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > > > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > > > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > > > > > driver, exposing modes supported by HW and programming user shaper and > > > > > > > 3D LUT accordingly. > > > > > > > > > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > > > > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > > > > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > > > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > > > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > > > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > > > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > > > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > > > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > > > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > > > > > > > > > Let me know your thoughts. > > > > > > > > > > > > +Simon Ser, +Pekka Paalanen who might also be interested in this series. > > > > > > > > > > Unfortunately I don't have the patch emails to reply to, so here's a > > > > > messy bunch of comments. I'll concentrate on the UAPI design as always. > > > > > > > > Sorry, the patchset is here: https://lore.kernel.org/dri-devel/20230109143846.1966301-1-mwen@igalia.com/ > > > > In the next version, I won't forget cc'ing you at first. > > > > > > > > > > +/* > > > > > + * struct drm_mode_lut3d_mode - 3D LUT mode information. > > > > > + * @lut_size: number of valid points on every dimension of 3D LUT. > > > > > + * @lut_stride: number of points on every dimension of 3D LUT. > > > > > + * @bit_depth: number of bits of RGB. If color_mode defines entries with higher > > > > > + * bit_depth the least significant bits will be truncated. > > > > > + * @color_format: fourcc values, ex. DRM_FORMAT_XRGB16161616 or DRM_FORMAT_XBGR16161616. > > > > > + * @flags: flags for hardware-sepcific features > > > > > + */ > > > > > +struct drm_mode_lut3d_mode { > > > > > + __u16 lut_size; > > > > > + __u16 lut_stride[3]; > > > > > + __u16 bit_depth; > > > > > + __u32 color_format; > > > > > + __u32 flags; > > > > > +}; Btw. there is an odd number of u16 members before a u32 member. That means there is invisible padding in this struct, I believe. I suppose that's not different between 32-bit and 64-bit architectures, since there are no 64-bit members, but it's still a hole. I assume the kernel makes sure the hole cannot contain uninitialized values which would leak kernel data to userspace... For more things you might want to check, I think Daniel's botching up ioctls article is appropriate. This is not an ioctl struct, but it's... well, essentially it is. Just read-only. https://www.kernel.org/doc/Documentation/ioctl/botching-up-ioctls.rst ... > > > > > + * “LUT3D_MODE”: > > > > > + * Enum property to give the mode of the 3D lookup table to be set on the > > > > > + * LUT3D property. A mode specifies size, stride, bit depth and color > > > > > + * format and depends on the underlying hardware). If drivers support > > > > > + * multiple 3D LUT modes, they should be declared in a array of > > > > > + * drm_color_lut3d_mode and they will be advertised as an enum. > > > > > > > > > > How does that work exactly? I didn't get it. I could guess, but having > > > > > to guess on API is bad. > > > > > > > > The driver advertises all supported modes (each combination of values) > > > > in a array as a enum, userspace can check all accepted modes and set the > > > > one that fits the user 3D LUT settings. I think it's possible to get the > > > > idea from this IGT test: > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commit/8771f444c3dcd126d7590d5a9b1b0db9706bbf6e#ed5dbc960ac210e3fbacd2361fe0270709767aaa_205_205 > > > > > > > > > > > You lost me at "an array as an enum". > > > > > > I understand there is a blob containing an array of struct > > > drm_mode_lut3d_mode. What I don't understand is that you say LUT3D_MODE > > > is an enum property. Where does the blob come from, then? What property > > > provides the blob? > > > > > > Am I correct in guessing that the values of LUT3D_MODE enum property > > > are indices into the array in the blob, and that userspace will set it? > > > That sounds good to me, if it's the integer value of the enum. But enum > > > values also need string names, because that is how the values are > > > usually recognized, so what name strings will be used? > > > > So, in this proposal, LUT3D_MODE is a list of indexes for a blob that describe a supported > > 3D LUT mode > > Hi Melissa, > > are you sure? I believe you are looking at and explaining some kernel > internal APIs, and not the uAPI which is the important part. Internal > APIs can always be changed later, uAPI cannot. > > After I had sent that email, I might have understood how it was > supposed to work instead: there is no array of struct > drm_mode_lut3d_mode. > > Instead, LUT3D_MODE is a KMS property of type enum. Each integer value > of the enum is also a blob id. Each blob named by those ids is a single > struct drm_mode_lut3d_mode that userspace needs to retrieve individually > to understand what that specific enum value means. > > > i.e. a `struct drm_mode_lut3d_mode` with size, stride, bit depth, etc. Strings here follow this pattern > > `lut3d_{size}_{bit_depth}bit` [1]. When enabling 3D LUT support, the > > driver should pass an array of `struct drm_mode_lut3d_mode` as supported > > modes, with at least one element. > > To be clear, I do not care about kernel internal interfaces at all. I > only care about the uAPI. Therefore talking about kernel internal API > will only confuse me and every other userspace developer. Evidently it > has even confused some IGT developers, as I point out some IGT code > problems below. > > If the struct drm_mode_lut3d_mode is the authoritative definition of > what each enum value means, then I think the string names should not > attempt to convey any meaning nor information. The strings should be > just "blob id 77" etc. That makes it clear what the value is. One more thing came to mind: extendability. If it turns out that struct drm_mode_lut3d_mode is not good enough for some future hardware, what could we do? Do you have a plan for that? A) Thinking again, I would suggest to make the enum value string names of the format "drm_mode_lut3d_mode %u". That way the name identifies the blob binary structure and the blob id makes it unique. If we hope that userspace checks the name, the kernel could later add a different structure when necessary. Even if userspace fails in that, it would still be an informative name. B) Otherwise, or in addition, the structure needs to be designed to be extendable like ioctl structs. A flags field maybe? But again we trust that userspace checks the flags before trying to interpret any further. C) Yet another option is a new DRM client CAP, which userspace can set to tell the kernel that userspace does understand and expect that some new kind of structure might be present. But this needs also A or B to identify the struct. D) Invent a whole new KMS property for the next generation 3D LUTs. In that case the new and old properties probably need to be mutually exclusive. > Otherwise userspace will be tempted to use the string names only (as is > the usual uAPI design) to find the right enum value, and not look into > the struct drm_mode_lut3d_mode at all. Therefore, if you don't encode > absolutely everything of struct drm_mode_lut3d_mode into the string > name, you should encode nothing in the name. You might also have two > different descriptions ending up with the same string name under the > same enum property instance, and that must not happen. Thanks, pq
Hello I'm completing the support for 3D LUT on R-Car DU peripheral and I have used this series as a base. I'm wondering, since quite some time has passed without any update if this series is still a thing and it makes any sense for me to try to bring it forward. I'm asking as I've noticed: "[PATCH 00/36] drm/amd/display: add AMD driver-specific properties for color mgmt" which seems to supersede this proposal with driver-specific properties. I asked Melissa privately but I wasn't able to get an hold of her, so if anyone has any clue feel free to reply :) Thanks j On Mon, Jan 09, 2023 at 01:38:28PM -0100, Melissa Wen wrote: > Hi, > > After collecting comments in different places, here is a second version > of the work on adding DRM CRTC 3D LUT support to the current DRM color > mgmt interface. In comparison to previous proposals [1][2][3], here we > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > that means the following DRM CRTC color correction pipeline: > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > LUT3D_SIZE, that allows userspace to use different supported settings of > 3D LUT, fitting VA-API and new color API better. In this sense, I > adjusted the pre-blending proposal for post-blending usage. > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > extra/optional patch to define a default value for LUT3D_MODE, inspired > by what we do for the plane blend mode property (pre-multiplied). > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > for DCN 301 (easily extendible to other DCN families). Finally, from > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > driver, exposing modes supported by HW and programming user shaper and > 3D LUT accordingly. > > Our target userspace is Gamescope/SteamOS. > > Basic IGT tests were based on [5][6] and are available here (in-progress): > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > [5] https://patchwork.freedesktop.org/series/90165/ > [6] https://patchwork.freedesktop.org/series/109402/ > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > Let me know your thoughts. > > Thanks, > > Melissa > > Alex Hung (2): > drm: Add 3D LUT mode and its attributes > drm/amd/display: Define 3D LUT struct for HDR planes > > Melissa Wen (16): > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > drm/drm_color_mgmt: add function to create 3D LUT modes supported > drm/drm_color_mgmt: add function to attach 3D LUT props > drm/drm_color_mgmt: set first lut3d mode as default > drm/amd/display: remove unused regamma condition > drm/amd/display: add comments to describe DM crtc color mgmt behavior > drm/amd/display: encapsulate atomic regamma operation > drm/amd/display: update lut3d and shaper lut to stream > drm/amd/display: handle MPC 3D LUT resources for a given context > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > drm/amd/display: expand array of supported 3D LUT modes > drm/amd/display: enable 3D-LUT DRM properties if supported > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > pipeline > drm/amd/display: decouple steps to reuse in shaper LUT support > drm/amd/display: add user shaper LUT support to amdgpu_dm color > pipeline > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > .../amd/display/modules/color/color_gamma.h | 43 ++ > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > drivers/gpu/drm/drm_fb_helper.c | 5 + > drivers/gpu/drm/drm_mode_config.c | 21 + > include/drm/drm_color_mgmt.h | 8 + > include/drm/drm_crtc.h | 32 +- > include/drm/drm_mode_config.h | 25 ++ > include/drm/drm_mode_object.h | 2 +- > include/uapi/drm/drm_mode.h | 17 + > 18 files changed, 757 insertions(+), 39 deletions(-) > > -- > 2.35.1 >
On Tue, 13 Jun 2023 17:43:55 +0200 Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote: > Hello > > I'm completing the support for 3D LUT on R-Car DU peripheral > and I have used this series as a base. > > I'm wondering, since quite some time has passed without any update if > this series is still a thing and it makes any sense for me to try to > bring it forward. > > I'm asking as I've noticed: > "[PATCH 00/36] drm/amd/display: add AMD driver-specific properties for color mgmt" > > which seems to supersede this proposal with driver-specific > properties. > > I asked Melissa privately but I wasn't able to get an hold of her, so > if anyone has any clue feel free to reply :) Hi, since no-one else replied, I'll point you to the thread starting at https://lists.freedesktop.org/archives/dri-devel/2023-May/403173.html and continuing to June. That is the plan of getting a common UAPI for these things. Thanks, pq > > Thanks > j > > On Mon, Jan 09, 2023 at 01:38:28PM -0100, Melissa Wen wrote: > > Hi, > > > > After collecting comments in different places, here is a second version > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > that means the following DRM CRTC color correction pipeline: > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > LUT3D_SIZE, that allows userspace to use different supported settings of > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > adjusted the pre-blending proposal for post-blending usage. > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > by what we do for the plane blend mode property (pre-multiplied). > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > for DCN 301 (easily extendible to other DCN families). Finally, from > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > driver, exposing modes supported by HW and programming user shaper and > > 3D LUT accordingly. > > > > Our target userspace is Gamescope/SteamOS. > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > [5] https://patchwork.freedesktop.org/series/90165/ > > [6] https://patchwork.freedesktop.org/series/109402/ > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > Let me know your thoughts. > > > > Thanks, > > > > Melissa > > > > Alex Hung (2): > > drm: Add 3D LUT mode and its attributes > > drm/amd/display: Define 3D LUT struct for HDR planes > > > > Melissa Wen (16): > > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > > drm/drm_color_mgmt: add function to create 3D LUT modes supported > > drm/drm_color_mgmt: add function to attach 3D LUT props > > drm/drm_color_mgmt: set first lut3d mode as default > > drm/amd/display: remove unused regamma condition > > drm/amd/display: add comments to describe DM crtc color mgmt behavior > > drm/amd/display: encapsulate atomic regamma operation > > drm/amd/display: update lut3d and shaper lut to stream > > drm/amd/display: handle MPC 3D LUT resources for a given context > > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > > drm/amd/display: expand array of supported 3D LUT modes > > drm/amd/display: enable 3D-LUT DRM properties if supported > > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > > pipeline > > drm/amd/display: decouple steps to reuse in shaper LUT support > > drm/amd/display: add user shaper LUT support to amdgpu_dm color > > pipeline > > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > > .../amd/display/modules/color/color_gamma.h | 43 ++ > > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > > drivers/gpu/drm/drm_fb_helper.c | 5 + > > drivers/gpu/drm/drm_mode_config.c | 21 + > > include/drm/drm_color_mgmt.h | 8 + > > include/drm/drm_crtc.h | 32 +- > > include/drm/drm_mode_config.h | 25 ++ > > include/drm/drm_mode_object.h | 2 +- > > include/uapi/drm/drm_mode.h | 17 + > > 18 files changed, 757 insertions(+), 39 deletions(-) > > > > -- > > 2.35.1 > >
Hi Pekka
thanks for the reply
On Thu, Jun 15, 2023 at 10:14:05AM +0300, Pekka Paalanen wrote:
> On Tue, 13 Jun 2023 17:43:55 +0200
> Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote:
>
> > Hello
> >
> > I'm completing the support for 3D LUT on R-Car DU peripheral
> > and I have used this series as a base.
> >
> > I'm wondering, since quite some time has passed without any update if
> > this series is still a thing and it makes any sense for me to try to
> > bring it forward.
> >
> > I'm asking as I've noticed:
> > "[PATCH 00/36] drm/amd/display: add AMD driver-specific properties for color mgmt"
> >
> > which seems to supersede this proposal with driver-specific
> > properties.
> >
> > I asked Melissa privately but I wasn't able to get an hold of her, so
> > if anyone has any clue feel free to reply :)
>
> Hi,
>
> since no-one else replied, I'll point you to the thread starting at
> https://lists.freedesktop.org/archives/dri-devel/2023-May/403173.html
Yes, Melissa pointed me to that series privately yesterday.
However, and here I might be missing something, per-plane properties do
not apply well to the HW pipeline I'm looking at.
The R-Car DU has a 1D LUT and a 3D LUT at the CRTC level (I guess
'post blending' is the right term here) ? A per-plane property
doesn't seem to match how the HW work, but please feel free to correct
me as this is all rather new to me and I might be overlooking
something.
My plan at the moment would have been to base my work on Melissa's RFC
and re-send to prop discussions, unless it is certainly a dead-end and
I have missed how to properly use per-plane properties on our HW.
Thank you!
> and continuing to June. That is the plan of getting a common UAPI for
> these things.
>
>
> Thanks,
> pq
>
>
> >
> > Thanks
> > j
> >
> > On Mon, Jan 09, 2023 at 01:38:28PM -0100, Melissa Wen wrote:
> > > Hi,
> > >
> > > After collecting comments in different places, here is a second version
> > > of the work on adding DRM CRTC 3D LUT support to the current DRM color
> > > mgmt interface. In comparison to previous proposals [1][2][3], here we
> > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT,
> > > that means the following DRM CRTC color correction pipeline:
> > >
> > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT
> > >
> > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung
> > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a
> > > LUT3D_SIZE, that allows userspace to use different supported settings of
> > > 3D LUT, fitting VA-API and new color API better. In this sense, I
> > > adjusted the pre-blending proposal for post-blending usage.
> > >
> > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to
> > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an
> > > extra/optional patch to define a default value for LUT3D_MODE, inspired
> > > by what we do for the plane blend mode property (pre-multiplied).
> > >
> > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage
> > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current
> > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT
> > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context
> > > for DCN 301 (easily extendible to other DCN families). Finally, from
> > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display
> > > driver, exposing modes supported by HW and programming user shaper and
> > > 3D LUT accordingly.
> > >
> > > Our target userspace is Gamescope/SteamOS.
> > >
> > > Basic IGT tests were based on [5][6] and are available here (in-progress):
> > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api
> > >
> > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/
> > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/
> > > [5] https://patchwork.freedesktop.org/series/90165/
> > > [6] https://patchwork.freedesktop.org/series/109402/
> > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11
> > >
> > > Let me know your thoughts.
> > >
> > > Thanks,
> > >
> > > Melissa
> > >
> > > Alex Hung (2):
> > > drm: Add 3D LUT mode and its attributes
> > > drm/amd/display: Define 3D LUT struct for HDR planes
> > >
> > > Melissa Wen (16):
> > > drm/drm_color_mgmt: add shaper LUT to color mgmt properties
> > > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt
> > > drm/drm_color_mgmt: add function to create 3D LUT modes supported
> > > drm/drm_color_mgmt: add function to attach 3D LUT props
> > > drm/drm_color_mgmt: set first lut3d mode as default
> > > drm/amd/display: remove unused regamma condition
> > > drm/amd/display: add comments to describe DM crtc color mgmt behavior
> > > drm/amd/display: encapsulate atomic regamma operation
> > > drm/amd/display: update lut3d and shaper lut to stream
> > > drm/amd/display: handle MPC 3D LUT resources for a given context
> > > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301
> > > drm/amd/display: expand array of supported 3D LUT modes
> > > drm/amd/display: enable 3D-LUT DRM properties if supported
> > > drm/amd/display: add user 3D LUT support to the amdgpu_dm color
> > > pipeline
> > > drm/amd/display: decouple steps to reuse in shaper LUT support
> > > drm/amd/display: add user shaper LUT support to amdgpu_dm color
> > > pipeline
> > >
> > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 +
> > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 +
> > > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++--
> > > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 +
> > > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++-
> > > drivers/gpu/drm/amd/display/dc/dc.h | 8 +
> > > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++-
> > > .../amd/display/modules/color/color_gamma.h | 43 ++
> > > drivers/gpu/drm/drm_atomic_state_helper.c | 7 +
> > > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++
> > > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++
> > > drivers/gpu/drm/drm_fb_helper.c | 5 +
> > > drivers/gpu/drm/drm_mode_config.c | 21 +
> > > include/drm/drm_color_mgmt.h | 8 +
> > > include/drm/drm_crtc.h | 32 +-
> > > include/drm/drm_mode_config.h | 25 ++
> > > include/drm/drm_mode_object.h | 2 +-
> > > include/uapi/drm/drm_mode.h | 17 +
> > > 18 files changed, 757 insertions(+), 39 deletions(-)
> > >
> > > --
> > > 2.35.1
> > >
>
On Thu, 15 Jun 2023 10:07:35 +0200 Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote: > Hi Pekka > thanks for the reply > > On Thu, Jun 15, 2023 at 10:14:05AM +0300, Pekka Paalanen wrote: > > On Tue, 13 Jun 2023 17:43:55 +0200 > > Jacopo Mondi <jacopo.mondi@ideasonboard.com> wrote: > > > > > Hello > > > > > > I'm completing the support for 3D LUT on R-Car DU peripheral > > > and I have used this series as a base. > > > > > > I'm wondering, since quite some time has passed without any update if > > > this series is still a thing and it makes any sense for me to try to > > > bring it forward. > > > > > > I'm asking as I've noticed: > > > "[PATCH 00/36] drm/amd/display: add AMD driver-specific properties for color mgmt" > > > > > > which seems to supersede this proposal with driver-specific > > > properties. > > > > > > I asked Melissa privately but I wasn't able to get an hold of her, so > > > if anyone has any clue feel free to reply :) > > > > Hi, > > > > since no-one else replied, I'll point you to the thread starting at > > https://lists.freedesktop.org/archives/dri-devel/2023-May/403173.html > > Yes, Melissa pointed me to that series privately yesterday. > > However, and here I might be missing something, per-plane properties do > not apply well to the HW pipeline I'm looking at. > > The R-Car DU has a 1D LUT and a 3D LUT at the CRTC level (I guess > 'post blending' is the right term here) ? A per-plane property > doesn't seem to match how the HW work, but please feel free to correct > me as this is all rather new to me and I might be overlooking > something. Post-blending, correct. The long term plan is to replicate the same idea to post-blending as it is planned for pre-blending. > > My plan at the moment would have been to base my work on Melissa's RFC > and re-send to prop discussions, unless it is certainly a dead-end and > I have missed how to properly use per-plane properties on our HW. I'm not a kernel dev nor a maintainer, so I can't comment on what would be acceptable in the mean time before the new pipeline design is implemented. The long term plan is to supersede all existing color transformation related properties with pipelines. Thanks, pq > > Thank you! > > > and continuing to June. That is the plan of getting a common UAPI for > > these things. > > > > > > Thanks, > > pq > > > > > > > > > > Thanks > > > j > > > > > > On Mon, Jan 09, 2023 at 01:38:28PM -0100, Melissa Wen wrote: > > > > Hi, > > > > > > > > After collecting comments in different places, here is a second version > > > > of the work on adding DRM CRTC 3D LUT support to the current DRM color > > > > mgmt interface. In comparison to previous proposals [1][2][3], here we > > > > add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, > > > > that means the following DRM CRTC color correction pipeline: > > > > > > > > Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT > > > > > > > > and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung > > > > proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a > > > > LUT3D_SIZE, that allows userspace to use different supported settings of > > > > 3D LUT, fitting VA-API and new color API better. In this sense, I > > > > adjusted the pre-blending proposal for post-blending usage. > > > > > > > > Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to > > > > the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an > > > > extra/optional patch to define a default value for LUT3D_MODE, inspired > > > > by what we do for the plane blend mode property (pre-multiplied). > > > > > > > > Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage > > > > on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current > > > > AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT > > > > changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context > > > > for DCN 301 (easily extendible to other DCN families). Finally, from > > > > 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display > > > > driver, exposing modes supported by HW and programming user shaper and > > > > 3D LUT accordingly. > > > > > > > > Our target userspace is Gamescope/SteamOS. > > > > > > > > Basic IGT tests were based on [5][6] and are available here (in-progress): > > > > https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api > > > > > > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ > > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 > > > > [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ > > > > [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ > > > > [5] https://patchwork.freedesktop.org/series/90165/ > > > > [6] https://patchwork.freedesktop.org/series/109402/ > > > > [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html > > > > [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 > > > > > > > > Let me know your thoughts. > > > > > > > > Thanks, > > > > > > > > Melissa > > > > > > > > Alex Hung (2): > > > > drm: Add 3D LUT mode and its attributes > > > > drm/amd/display: Define 3D LUT struct for HDR planes > > > > > > > > Melissa Wen (16): > > > > drm/drm_color_mgmt: add shaper LUT to color mgmt properties > > > > drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt > > > > drm/drm_color_mgmt: add function to create 3D LUT modes supported > > > > drm/drm_color_mgmt: add function to attach 3D LUT props > > > > drm/drm_color_mgmt: set first lut3d mode as default > > > > drm/amd/display: remove unused regamma condition > > > > drm/amd/display: add comments to describe DM crtc color mgmt behavior > > > > drm/amd/display: encapsulate atomic regamma operation > > > > drm/amd/display: update lut3d and shaper lut to stream > > > > drm/amd/display: handle MPC 3D LUT resources for a given context > > > > drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 > > > > drm/amd/display: expand array of supported 3D LUT modes > > > > drm/amd/display: enable 3D-LUT DRM properties if supported > > > > drm/amd/display: add user 3D LUT support to the amdgpu_dm color > > > > pipeline > > > > drm/amd/display: decouple steps to reuse in shaper LUT support > > > > drm/amd/display: add user shaper LUT support to amdgpu_dm color > > > > pipeline > > > > > > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + > > > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + > > > > .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- > > > > .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + > > > > drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- > > > > drivers/gpu/drm/amd/display/dc/dc.h | 8 + > > > > .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- > > > > .../amd/display/modules/color/color_gamma.h | 43 ++ > > > > drivers/gpu/drm/drm_atomic_state_helper.c | 7 + > > > > drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ > > > > drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ > > > > drivers/gpu/drm/drm_fb_helper.c | 5 + > > > > drivers/gpu/drm/drm_mode_config.c | 21 + > > > > include/drm/drm_color_mgmt.h | 8 + > > > > include/drm/drm_crtc.h | 32 +- > > > > include/drm/drm_mode_config.h | 25 ++ > > > > include/drm/drm_mode_object.h | 2 +- > > > > include/uapi/drm/drm_mode.h | 17 + > > > > 18 files changed, 757 insertions(+), 39 deletions(-) > > > > > > > > -- > > > > 2.35.1 > > > > > > > >
Hi, After collecting comments in different places, here is a second version of the work on adding DRM CRTC 3D LUT support to the current DRM color mgmt interface. In comparison to previous proposals [1][2][3], here we add 3D LUT before gamma 1D LUT, but also a shaper 1D LUT before 3D LUT, that means the following DRM CRTC color correction pipeline: Blend -> Degamma 1D LUT -> CTM -> Shaper 1D LUT -> 3D LUT -> Gamma 1D LUT and we also add a DRM CRTC LUT3D_MODE property, based on Alex Hung proposal for pre-blending 3D LUT [4] (Thanks!), instead of just a LUT3D_SIZE, that allows userspace to use different supported settings of 3D LUT, fitting VA-API and new color API better. In this sense, I adjusted the pre-blending proposal for post-blending usage. Patches 1-6 targets the addition of shaper LUT and 3D LUT properties to the current DRM CRTC color mgmt pipeline. Patch 6 can be considered an extra/optional patch to define a default value for LUT3D_MODE, inspired by what we do for the plane blend mode property (pre-multiplied). Patches 7-18 targets AMD display code to enable shaper and 3D LUT usage on DCN 301 (our HW case). Patches 7-9 performs code cleanups on current AMD DM colors code, patch 10 updates AMD stream in case of user 3D LUT changes, patch 11/12 rework AMD MPC 3D LUT resource handling by context for DCN 301 (easily extendible to other DCN families). Finally, from 13-18, we wire up SHAPER LUT, LUT3D and LUT3D MODE to AMD display driver, exposing modes supported by HW and programming user shaper and 3D LUT accordingly. Our target userspace is Gamescope/SteamOS. Basic IGT tests were based on [5][6] and are available here (in-progress): https://gitlab.freedesktop.org/mwen/igt-gpu-tools/-/commits/crtc-lut3d-api [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/ [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69 [3] https://lore.kernel.org/amd-gfx/20220619223104.667413-1-mwen@igalia.com/ [4] https://lore.kernel.org/dri-devel/20221004211451.1475215-1-alex.hung@amd.com/ [5] https://patchwork.freedesktop.org/series/90165/ [6] https://patchwork.freedesktop.org/series/109402/ [VA_API] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html [KMS_pipe_API] https://gitlab.freedesktop.org/pq/color-and-hdr/-/issues/11 Let me know your thoughts. Thanks, Melissa Alex Hung (2): drm: Add 3D LUT mode and its attributes drm/amd/display: Define 3D LUT struct for HDR planes Melissa Wen (16): drm/drm_color_mgmt: add shaper LUT to color mgmt properties drm/drm_color_mgmt: add 3D LUT props to DRM color mgmt drm/drm_color_mgmt: add function to create 3D LUT modes supported drm/drm_color_mgmt: add function to attach 3D LUT props drm/drm_color_mgmt: set first lut3d mode as default drm/amd/display: remove unused regamma condition drm/amd/display: add comments to describe DM crtc color mgmt behavior drm/amd/display: encapsulate atomic regamma operation drm/amd/display: update lut3d and shaper lut to stream drm/amd/display: handle MPC 3D LUT resources for a given context drm/amd/display: acquire/release 3D LUT resources for ctx on DCN301 drm/amd/display: expand array of supported 3D LUT modes drm/amd/display: enable 3D-LUT DRM properties if supported drm/amd/display: add user 3D LUT support to the amdgpu_dm color pipeline drm/amd/display: decouple steps to reuse in shaper LUT support drm/amd/display: add user shaper LUT support to amdgpu_dm color pipeline .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 6 + .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h | 3 + .../amd/display/amdgpu_dm/amdgpu_dm_color.c | 370 ++++++++++++++++-- .../amd/display/amdgpu_dm/amdgpu_dm_crtc.c | 2 + drivers/gpu/drm/amd/display/dc/core/dc.c | 49 ++- drivers/gpu/drm/amd/display/dc/dc.h | 8 + .../amd/display/dc/dcn301/dcn301_resource.c | 47 ++- .../amd/display/modules/color/color_gamma.h | 43 ++ drivers/gpu/drm/drm_atomic_state_helper.c | 7 + drivers/gpu/drm/drm_atomic_uapi.c | 24 ++ drivers/gpu/drm/drm_color_mgmt.c | 127 ++++++ drivers/gpu/drm/drm_fb_helper.c | 5 + drivers/gpu/drm/drm_mode_config.c | 21 + include/drm/drm_color_mgmt.h | 8 + include/drm/drm_crtc.h | 32 +- include/drm/drm_mode_config.h | 25 ++ include/drm/drm_mode_object.h | 2 +- include/uapi/drm/drm_mode.h | 17 + 18 files changed, 757 insertions(+), 39 deletions(-)