| Message ID | 20210406192118.12313-3-leandro.ribeiro@collabora.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Document how userspace should use plane format list and IN_FORMATS | expand |
On Tue, Apr 06, 2021 at 04:21:18PM -0300, Leandro Ribeiro wrote: > Emphasize how userspace should use the plane format list > (format_type_ptr) and the IN_FORMATS blob property. > > Formats exposed in the plane format list (format_type_ptr) do not > require modifiers, and formats that are present in the IN_FORMATS blob > property support modifiers. > > Note that these are not disjoint sets. If a format supports modifiers > but the driver can also handle it without a modifier, it should be > present in both the IN_FORMATS blob property and the plane format list. > > Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com> > --- > drivers/gpu/drm/drm_plane.c | 4 ++++ > include/uapi/drm/drm_mode.h | 3 +++ > 2 files changed, 7 insertions(+) > > diff --git a/drivers/gpu/drm/drm_plane.c b/drivers/gpu/drm/drm_plane.c > index 0dd43882fe7c..b48d9bd81a59 100644 > --- a/drivers/gpu/drm/drm_plane.c > +++ b/drivers/gpu/drm/drm_plane.c > @@ -128,6 +128,10 @@ > * pairs supported by this plane. The blob is a struct > * drm_format_modifier_blob. Without this property the plane doesn't > * support buffers with modifiers. Userspace cannot change this property. > + * > + * To find out the list of buffer formats which are supported without a > + * modifier, userspace should not look at this blob property, but at the > + * formats list of the plane: &drm_mode_get_plane.format_type_ptr. > */ > > static unsigned int drm_num_planes(struct drm_device *dev) > diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h > index 96fc9a6da608..4293800ec095 100644 > --- a/include/uapi/drm/drm_mode.h > +++ b/include/uapi/drm/drm_mode.h > @@ -340,6 +340,9 @@ struct drm_mode_get_plane { > /** > * @format_type_ptr: Pointer to ``__u32`` array of formats that are > * supported by the plane. These formats do not require modifiers. > + * > + * To find out the list of formats that support modifiers, userspace > + * must use the plane IN_FORMATS blob property. > */ Addfb2+modifiers predates the IN_FORMATS blob, so this doesn't match reality. > __u64 format_type_ptr; > }; > -- > 2.31.1 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Wednesday, April 7th, 2021 at 3:51 PM, Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > > + * To find out the list of formats that support modifiers, userspace > > + * must use the plane IN_FORMATS blob property. > > */ > > Addfb2+modifiers predates the IN_FORMATS blob, so this doesn't > match reality. TBH, I'm inclined not to care about this edge-case. It's already complicated enough for user-space to figure out what's the right thing to do when supporting both implicit modifiers and explicit modifiers. Using modifiers without IN_FORMATS is risky, since a whole part of the modifier negotiation mechanism is missing. Maybe we can just stick a "since kernel x.y.z" in here to address your concern.
On Thu, 08 Apr 2021 08:39:10 +0000 Simon Ser <contact@emersion.fr> wrote: > On Wednesday, April 7th, 2021 at 3:51 PM, Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > > > > + * To find out the list of formats that support modifiers, userspace > > > + * must use the plane IN_FORMATS blob property. > > > */ > > > > Addfb2+modifiers predates the IN_FORMATS blob, so this doesn't > > match reality. > > TBH, I'm inclined not to care about this edge-case. It's already > complicated enough for user-space to figure out what's the right thing > to do when supporting both implicit modifiers and explicit modifiers. > Using modifiers without IN_FORMATS is risky, since a whole part of the > modifier negotiation mechanism is missing. > > Maybe we can just stick a "since kernel x.y.z" in here to address your > concern. Yeah, or we could word it less "must", e.g. "The list of supported formats with their explicit modifiers is advertised via IN_FORMATS blob." We don't have to require userspace to not use the explicit modifier uAPI if IN_FORMATS does not exist. There might be other ways how userspace determines the explicit modifiers, like a Wayland compositor advertising those format-modifier pairs that EGL can import. Then clients use those, and the compositor can try to import those to KMS as well. Maybe it works, maybe it doesn't (the same even if IN_FORMATS exists). IN_FORMATS just gives better chances of picking something that ends up working. The thing userspace *must not* do is to try to use the no-modifiers uAPI when it has an explicit modifier. But do we then have exceptions for MOD_LINEAR? If a buffer has been allocated with explicit modifier MOD_LINEAR, does it mean it is ok to import to KMS using the no-modifiers uAPI or not? The other things userspace *must not* do is use the explicit modifier uAPI when it does not have an explicit modifier, in essence pulling a modifier out of a hat. It might be tempting to use MOD_LINEAR in that case, sometimes it might work, but it's wrong. Userspace must use the no-modifiers uAPI instead. Right? The point of these documentation patches is to establish the convention that: - drm_mode_get_plane::format_type_ptr is the list of pixel formats that can work via the no-modifiers uAPI, but says nothing about the explicit modifiers uAPI. - IN_FORMATS is a list of format-modifier pairs that can work via the explicit modifiers API, but says nothing about the no-modifiers uAPI. Is that a reasonable expectation? Is it also so that passing MOD_INVALID to the explicit modifier uAPI (ADDFB2) is invalid argument? Do we have that documented? Thanks, pq
On Thu, Apr 08, 2021 at 12:59:19PM +0300, Pekka Paalanen wrote: > On Thu, 08 Apr 2021 08:39:10 +0000 > Simon Ser <contact@emersion.fr> wrote: > > > On Wednesday, April 7th, 2021 at 3:51 PM, Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > > > > > > + * To find out the list of formats that support modifiers, userspace > > > > + * must use the plane IN_FORMATS blob property. > > > > */ > > > > > > Addfb2+modifiers predates the IN_FORMATS blob, so this doesn't > > > match reality. > > > > TBH, I'm inclined not to care about this edge-case. It's already > > complicated enough for user-space to figure out what's the right thing > > to do when supporting both implicit modifiers and explicit modifiers. > > Using modifiers without IN_FORMATS is risky, since a whole part of the > > modifier negotiation mechanism is missing. > > > > Maybe we can just stick a "since kernel x.y.z" in here to address your > > concern. > > Yeah, or we could word it less "must", e.g. "The list of supported > formats with their explicit modifiers is advertised via IN_FORMATS blob." > > We don't have to require userspace to not use the explicit modifier > uAPI if IN_FORMATS does not exist. There might be other ways how > userspace determines the explicit modifiers, like a Wayland compositor > advertising those format-modifier pairs that EGL can import. Then > clients use those, and the compositor can try to import those to KMS as > well. Maybe it works, maybe it doesn't (the same even if IN_FORMATS > exists). > > IN_FORMATS just gives better chances of picking something that ends up > working. Yup. > The thing userspace *must not* do is to try to use the no-modifiers uAPI > when it has an explicit modifier. But do we then have exceptions for > MOD_LINEAR? > > If a buffer has been allocated with explicit modifier MOD_LINEAR, does > it mean it is ok to import to KMS using the no-modifiers uAPI or not? Maybe it work with current userspace, but not guaranteed. I think we should strongly discourage this at least. The case this can go boom is if userspace allocates a big bo, with some implicit metadata. And then suballocates some linear buffer out of that with explicit modifiers. This could happen if userspace does realize modifiers work, and then does some funky optimization to linear e.g. as part of a resolve pass. Not such hw/driver currently exists, but not something I'd guarantee never happens. If you then create a drm_fb with that with no modifier specified, you get the implicit one from the metadata. > > The other things userspace *must not* do is use the explicit modifier > uAPI when it does not have an explicit modifier, in essence pulling a > modifier out of a hat. It might be tempting to use MOD_LINEAR in that > case, sometimes it might work, but it's wrong. Userspace must use the > no-modifiers uAPI instead. Yes. Userspace can't guess the modifier if it doesn't have it. > > Right? > > The point of these documentation patches is to establish the convention > that: > > - drm_mode_get_plane::format_type_ptr is the list of pixel formats that > can work via the no-modifiers uAPI, but says nothing about the > explicit modifiers uAPI. > > - IN_FORMATS is a list of format-modifier pairs that can work via the > explicit modifiers API, but says nothing about the no-modifiers uAPI. > > Is that a reasonable expectation? I'm not sure. I thought they're the same list underneath in the kernel, at least for drivers that do support modifiers. The current wording I think suggests more meaning than is actually there. > Is it also so that passing MOD_INVALID to the explicit modifier uAPI > (ADDFB2) is invalid argument? Do we have that documented? We'd need to check that, currently it's an out-of-band flag in the struct. Atm DRM_FORMAT_MOD_INVALID is entirely an internal sentinel value to denote end-of-array entries. In practice it wont pass because we validate the modifiers against the advertised list. -Daniel > > > Thanks, > pq > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Thu, 8 Apr 2021 13:30:16 +0200 Daniel Vetter <daniel@ffwll.ch> wrote: > On Thu, Apr 08, 2021 at 12:59:19PM +0300, Pekka Paalanen wrote: > > The point of these documentation patches is to establish the convention > > that: > > > > - drm_mode_get_plane::format_type_ptr is the list of pixel formats that > > can work via the no-modifiers uAPI, but says nothing about the > > explicit modifiers uAPI. > > > > - IN_FORMATS is a list of format-modifier pairs that can work via the > > explicit modifiers API, but says nothing about the no-modifiers uAPI. > > > > Is that a reasonable expectation? > > I'm not sure. I thought they're the same list underneath in the kernel, at > least for drivers that do support modifiers. The current wording I think > suggests more meaning than is actually there. They may be the same list in the kernel today, but do you want to force all future drivers and future formats-modifiers to have that too? Or did the boat sail already? The existing uAPI considers these two to be independent lists, no documentation saying otherwise, is there? Should a kernel driver not have a way to say "this format won't work via the no-modifiers uAPI"? The practical consequence in userspace is how should userspace collect the lists of supported format-modifier pairs, when the kernel has two independent format lists, one carries modifiers explicitly and the other does not. The one that carries explicit modifiers cannot denote "no modifier" AFAIU. So the "obvious" interpretation in userspace is that: - the format list that does not carry any modifier information should be used with the no-modifiers uAPI, and - the format list that does carry explicit modifiers should be used with the explicit modifiers uAPI. If you were to say, that if IN_FORMATS exists, use it and ignore the old no-modifiers format list, then the conclusion in userspace when IN_FORMATS exists is that you cannot use the no-modifiers uAPI, because all formats that are listed as supported carry an explicit modifier. I understand that the format or format-modifier lists are not authoritative. Formats outside of the lists *could* work. But why would anyone bother trying something that is not suggested to work? Or, is the intention such, that all formats in IN_FORMATS list imply some support through the no-modifiers uAPI too, iff buffer allocation does not give you an explicit modifier? Or, should there be an i-g-t test to ensure that both the old and IN_FORMATS lists have the exact same pixel formats always, carving that fact into stone and resolving all this ambiguity? > > Is it also so that passing MOD_INVALID to the explicit modifier uAPI > > (ADDFB2) is invalid argument? Do we have that documented? > > We'd need to check that, currently it's an out-of-band flag in the struct. > Atm DRM_FORMAT_MOD_INVALID is entirely an internal sentinel value to > denote end-of-array entries. > > In practice it wont pass because we validate the modifiers against the > advertised list. Right, so while at it, would be good to document that one cannot substitute no-modifiers uAPI with explicit modifier uAPI using MOD_INVALID. This should be documented, because other userspace APIs have tendency to gravitate towards having just one explicit modifiers function allowing MOD_INVALID, meaning "no modifier". Thanks, pq
On Thu, Apr 08, 2021 at 04:57:51PM +0300, Pekka Paalanen wrote: > On Thu, 8 Apr 2021 13:30:16 +0200 > Daniel Vetter <daniel@ffwll.ch> wrote: > > > On Thu, Apr 08, 2021 at 12:59:19PM +0300, Pekka Paalanen wrote: > > > > The point of these documentation patches is to establish the convention > > > that: > > > > > > - drm_mode_get_plane::format_type_ptr is the list of pixel formats that > > > can work via the no-modifiers uAPI, but says nothing about the > > > explicit modifiers uAPI. > > > > > > - IN_FORMATS is a list of format-modifier pairs that can work via the > > > explicit modifiers API, but says nothing about the no-modifiers uAPI. > > > > > > Is that a reasonable expectation? > > > > I'm not sure. I thought they're the same list underneath in the kernel, at > > least for drivers that do support modifiers. The current wording I think > > suggests more meaning than is actually there. > > They may be the same list in the kernel today, but do you want to force > all future drivers and future formats-modifiers to have that too? > Or did the boat sail already? > > The existing uAPI considers these two to be independent lists, no > documentation saying otherwise, is there? > > Should a kernel driver not have a way to say "this format won't work > via the no-modifiers uAPI"? > > The practical consequence in userspace is how should userspace collect > the lists of supported format-modifier pairs, when the kernel has two > independent format lists, one carries modifiers explicitly and the > other does not. The one that carries explicit modifiers cannot denote > "no modifier" AFAIU. > > So the "obvious" interpretation in userspace is that: > - the format list that does not carry any modifier information should > be used with the no-modifiers uAPI, and > - the format list that does carry explicit modifiers should be used > with the explicit modifiers uAPI. > > If you were to say, that if IN_FORMATS exists, use it and ignore the > old no-modifiers format list, then the conclusion in userspace when > IN_FORMATS exists is that you cannot use the no-modifiers uAPI, because > all formats that are listed as supported carry an explicit modifier. > > I understand that the format or format-modifier lists are not > authoritative. Formats outside of the lists *could* work. But why would > anyone bother trying something that is not suggested to work? IMO formats not listed by any plane should just be rejected by addfb2. I tried to put that check in the drm core actually but there was some weird pushback, so for the moment it's handled by each driver. Some drivers (like i915) will reject anything not supported by any plane, other drivers might not (and probably no one knows how badly they might blow up if you pass in some exotic format...). I also had some igt patches to test that addfb2 behaviour but they didn't go in either. > > Or, is the intention such, that all formats in IN_FORMATS list imply > some support through the no-modifiers uAPI too, iff buffer > allocation does not give you an explicit modifier? > > Or, should there be an i-g-t test to ensure that both the old and > IN_FORMATS lists have the exact same pixel formats always, carving that > fact into stone and resolving all this ambiguity? > > > > Is it also so that passing MOD_INVALID to the explicit modifier uAPI > > > (ADDFB2) is invalid argument? Do we have that documented? > > > > We'd need to check that, currently it's an out-of-band flag in the struct. > > Atm DRM_FORMAT_MOD_INVALID is entirely an internal sentinel value to > > denote end-of-array entries. > > > > In practice it wont pass because we validate the modifiers against the > > advertised list. We don't actually. If the driver provides the .format_mod_supported() hook then it's up to the driver to validate the modifier in said hook. This was done so that people can embed metadata inside the modifier while only having the base modifier on the modifier list. How userspace is supposed to figure out which values for this extra metadata are valid I have no idea.
On Thu, 8 Apr 2021 17:39:22 +0300 Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > On Thu, Apr 08, 2021 at 04:57:51PM +0300, Pekka Paalanen wrote: > > On Thu, 8 Apr 2021 13:30:16 +0200 > > Daniel Vetter <daniel@ffwll.ch> wrote: > > > > > > Is it also so that passing MOD_INVALID to the explicit modifier uAPI > > > > (ADDFB2) is invalid argument? Do we have that documented? > > > > > > We'd need to check that, currently it's an out-of-band flag in the struct. > > > Atm DRM_FORMAT_MOD_INVALID is entirely an internal sentinel value to > > > denote end-of-array entries. > > > > > > In practice it wont pass because we validate the modifiers against the > > > advertised list. > > We don't actually. If the driver provides the .format_mod_supported() > hook then it's up to the driver to validate the modifier in said hook. > This was done so that people can embed metadata inside the modifier > while only having the base modifier on the modifier list. How userspace > is supposed to figure out which values for this extra metadata are valid > I have no idea. Maybe it's the difference between generic userspace and userspace drivers? I've been having the feeling that these two have different "rules". Maybe that distinction should be formalised in documentation somewhere? Generic userspace never looks into modifiers, it just relays them and compares them as opaque 64-bit words. Userspace drivers are allowed to look into what a modifier actually means and fiddle with it. Thanks, pq
On Thu, Apr 08, 2021 at 05:39:22PM +0300, Ville Syrjälä wrote: > On Thu, Apr 08, 2021 at 04:57:51PM +0300, Pekka Paalanen wrote: > > On Thu, 8 Apr 2021 13:30:16 +0200 > > Daniel Vetter <daniel@ffwll.ch> wrote: > > > > > On Thu, Apr 08, 2021 at 12:59:19PM +0300, Pekka Paalanen wrote: > > > > > > The point of these documentation patches is to establish the convention > > > > that: > > > > > > > > - drm_mode_get_plane::format_type_ptr is the list of pixel formats that > > > > can work via the no-modifiers uAPI, but says nothing about the > > > > explicit modifiers uAPI. > > > > > > > > - IN_FORMATS is a list of format-modifier pairs that can work via the > > > > explicit modifiers API, but says nothing about the no-modifiers uAPI. > > > > > > > > Is that a reasonable expectation? > > > > > > I'm not sure. I thought they're the same list underneath in the kernel, at > > > least for drivers that do support modifiers. The current wording I think > > > suggests more meaning than is actually there. > > > > They may be the same list in the kernel today, but do you want to force > > all future drivers and future formats-modifiers to have that too? > > Or did the boat sail already? > > > > The existing uAPI considers these two to be independent lists, no > > documentation saying otherwise, is there? > > > > Should a kernel driver not have a way to say "this format won't work > > via the no-modifiers uAPI"? > > > > The practical consequence in userspace is how should userspace collect > > the lists of supported format-modifier pairs, when the kernel has two > > independent format lists, one carries modifiers explicitly and the > > other does not. The one that carries explicit modifiers cannot denote > > "no modifier" AFAIU. > > > > So the "obvious" interpretation in userspace is that: > > - the format list that does not carry any modifier information should > > be used with the no-modifiers uAPI, and > > - the format list that does carry explicit modifiers should be used > > with the explicit modifiers uAPI. Imo the right interpretation is "it's the same list". > > If you were to say, that if IN_FORMATS exists, use it and ignore the > > old no-modifiers format list, then the conclusion in userspace when > > IN_FORMATS exists is that you cannot use the no-modifiers uAPI, because > > all formats that are listed as supported carry an explicit modifier. So formats without modifiers are very funny in their semantics. It means - implied modifier is untiled - except on i915 and radeon/amdgpu, where there's magic hidden information and you might get something else. But it won't work in multi-gpu situations - except bugs (like imo mesa trying to use tiling without modifiers and without magic in-kernel tiling information forwarding is just broken, and surprise, it's broken). We've had some where parts of mesa where assuming modifiers are ok, but that wasn't the case. Now maybe we can expose to userspace which drivers have magic device-local tiling information sharing, but I don't expect this list to ever grow. Anything beyond that sounds like the kernel should maintain a bug list of things which are broken in userspace. > > I understand that the format or format-modifier lists are not > > authoritative. Formats outside of the lists *could* work. But why would > > anyone bother trying something that is not suggested to work? > > IMO formats not listed by any plane should just be rejected by addfb2. > I tried to put that check in the drm core actually but there was some > weird pushback, so for the moment it's handled by each driver. Some > drivers (like i915) will reject anything not supported by any plane, > other drivers might not (and probably no one knows how badly they > might blow up if you pass in some exotic format...). > > I also had some igt patches to test that addfb2 behaviour but > they didn't go in either. > > > Or, is the intention such, that all formats in IN_FORMATS list imply > > some support through the no-modifiers uAPI too, iff buffer > > allocation does not give you an explicit modifier? > > > > Or, should there be an i-g-t test to ensure that both the old and > > IN_FORMATS lists have the exact same pixel formats always, carving that > > fact into stone and resolving all this ambiguity? > > > > > > Is it also so that passing MOD_INVALID to the explicit modifier uAPI > > > > (ADDFB2) is invalid argument? Do we have that documented? > > > > > > We'd need to check that, currently it's an out-of-band flag in the struct. > > > Atm DRM_FORMAT_MOD_INVALID is entirely an internal sentinel value to > > > denote end-of-array entries. > > > > > > In practice it wont pass because we validate the modifiers against the > > > advertised list. > > We don't actually. If the driver provides the .format_mod_supported() > hook then it's up to the driver to validate the modifier in said hook. > This was done so that people can embed metadata inside the modifier > while only having the base modifier on the modifier list. How userspace > is supposed to figure out which values for this extra metadata are valid > I have no idea. It took me a while to figure this out again, but now I remember. It's because of combinatorial explosion of modifiers with stuff like AFBC. So the modifier list is not necessarily an exhaustive list of all combinations, some of the modifiers count more like flags. Which is kinda hilarious since it means userspace can't do anything with modifiers it gets :-/ -Daniel
diff --git a/drivers/gpu/drm/drm_plane.c b/drivers/gpu/drm/drm_plane.c index 0dd43882fe7c..b48d9bd81a59 100644 --- a/drivers/gpu/drm/drm_plane.c +++ b/drivers/gpu/drm/drm_plane.c @@ -128,6 +128,10 @@ * pairs supported by this plane. The blob is a struct * drm_format_modifier_blob. Without this property the plane doesn't * support buffers with modifiers. Userspace cannot change this property. + * + * To find out the list of buffer formats which are supported without a + * modifier, userspace should not look at this blob property, but at the + * formats list of the plane: &drm_mode_get_plane.format_type_ptr. */ static unsigned int drm_num_planes(struct drm_device *dev) diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h index 96fc9a6da608..4293800ec095 100644 --- a/include/uapi/drm/drm_mode.h +++ b/include/uapi/drm/drm_mode.h @@ -340,6 +340,9 @@ struct drm_mode_get_plane { /** * @format_type_ptr: Pointer to ``__u32`` array of formats that are * supported by the plane. These formats do not require modifiers. + * + * To find out the list of formats that support modifiers, userspace + * must use the plane IN_FORMATS blob property. */ __u64 format_type_ptr; };
Emphasize how userspace should use the plane format list (format_type_ptr) and the IN_FORMATS blob property. Formats exposed in the plane format list (format_type_ptr) do not require modifiers, and formats that are present in the IN_FORMATS blob property support modifiers. Note that these are not disjoint sets. If a format supports modifiers but the driver can also handle it without a modifier, it should be present in both the IN_FORMATS blob property and the plane format list. Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com> --- drivers/gpu/drm/drm_plane.c | 4 ++++ include/uapi/drm/drm_mode.h | 3 +++ 2 files changed, 7 insertions(+)