| Message ID | 20231009-caleb-notch-example-v1-1-9e0a43ae233c@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RFC] dt-bindings: display: document display panel occlusions | expand |
Hi Caleb, thanks for putting this out. On Mon, Oct 09, 2023 at 06:32:50PM +0100, Caleb Connolly wrote: > Some display panels found in modern phones and laptops feature > non-standard display shapes with features like rounded corners, notches > (sections of the display that are cut-out from the edge), and cutouts > (such as circular "hole punch" camera cutouts, or wider pill-shaped > "islands"). > > Some discussion has been underway previously on how best to describe > these features [1][2], including a reference userspace implementation > using SVG paths [3]. Using this previous discussion as a jumping off > point, this RFC allows for describing the following display features: > > * Corner radius (on a per-corner basis) > * Circular or pill-shaped cutouts > * Notches with arbitrary shapes The coordinate systems is 0,0 at top,left display corner? > It's easy to make a case for only using rectangles to describe these > missing parts of a display, however this severely limits their utility. > Describing display occlusions as accurately as possible allows for a lot of > useful UX features. For example, displaying a ring around a hole-punch > camera when it's in use, or wrapping UI elements around a notch. These > behaviours are only possible to implement when the dimensions are known > with near pixel-perfect accuracy. > > Cutouts are described as rectangles with a width, height, and corner > radius. > A radius of half the width longest edge will definitionally be an ellipse. > This simple description is suitable for describing hole-punch cameras, > as well > as pill-shaped cutouts. I'm not aware of any devices that can't be > described like this. I wonder if we shouldn't just use a closed SVG path and put this and notches in one bucket that way. We need to indicate the "type" of a notch/cutout anyway as there's e.g. ones which aren't fully opaque but "just" lower resolution areas. We could start out with one type "fully opaque". > Notches are a little more complicated, they usually feature convex and > concave corners as well as straight lines. Here they are described as a > sequence of optionally disjoint arcs, where the space between one arc ending > and another starting is inferred to be a straight line. gmobile is using a subset of a closed SVG path (so does Android afaik). The upside of "just arcs" would be that bounding box is even simpler but as this affects userspace I think getting a better approximation using a path would be worth it and we could treat it the same as cutouts. > Each arc is described with an X and Y pixel coordinate, a radius, and a > start and end point in degrees. These arcs can precisely describe the > shape of a notch, and easily allow for a basic bounding box to be > derived using the min/max coordinates specified in the path. > > Some displays feature partially occluded edges ("waterfall edges") where > the screen bends, it may be useful for user interfaces to avoid placing > certain UI elements like buttons too close to these edges. These edges > are described by a distance from the edge where it begins to be > occluded, and the number of degrees that the display curves (this > directly affects how usable this edge of the screen is). I've not yet seen a device with this on Linux. Would it make sense to leave it out until we have those coming. E.g. are all waterfalls the same or are some more usable for displaying information than others? What is your plan on informing userspace about that? Feed it through DRM via drm properties or would userspace poke at /sys and try to find the right panel for the connector? drm would have the upside of not being bound to device-tree using platforms. Cheers, -- Guido > > [1]: https://lore.kernel.org/dri-devel/f8747f99-0695-5be0-841f-4f72ba5d5da3@connolly.tech/ > [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/87 > [3]: https://gitlab.gnome.org/World/Phosh/gmobile > > Signed-off-by: Caleb Connolly <caleb.connolly@linaro.org> > --- > Some folks have previously suggested that this information belongs in > userspace and not in devicetree. I would like to be clear that > devicetree is for describing hardware, and parts of a display which can > never actually be seen are very much properties of the underlying > hardware. > --- > base-commit: 268c4b354d66908697299063c44c0b553b01d935 > > // Caleb (they/them) > --- > .../bindings/display/panel/panel-common.yaml | 7 + > .../bindings/display/panel/panel-occlusions.yaml | 182 +++++++++++++++++++++ > 2 files changed, 189 insertions(+) > > diff --git a/Documentation/devicetree/bindings/display/panel/panel-common.yaml b/Documentation/devicetree/bindings/display/panel/panel-common.yaml > index 0a57a31f4f3d..6ea52a031872 100644 > --- a/Documentation/devicetree/bindings/display/panel/panel-common.yaml > +++ b/Documentation/devicetree/bindings/display/panel/panel-common.yaml > @@ -150,6 +150,13 @@ properties: > controller, this property contains a phandle that references the > controller. > > + occlusions: > + $ref: panel-occlusions.yaml# > + description: > + Some panels have non-standard shapes due to features like rounded corners, > + notches, cutouts, and "waterfall" edges. The panel-occlusions bindings > + can be used to describe these features. > + > dependencies: > width-mm: [ height-mm ] > height-mm: [ width-mm ] > diff --git a/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml > new file mode 100644 > index 000000000000..0932026bbd8c > --- /dev/null > +++ b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml > @@ -0,0 +1,182 @@ > +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/display/panel/panel-occlusions.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Common Properties for describing display notches, cutouts, and other occlusions > + > +maintainers: > + - Caleb Connolly <caleb.connolly@linaro.org> > + > +description: | > + This document defines devicetree nodes that can be used to describe > + different kind of display occlusions such as notches, camera cutouts, rounded > + corners, and other features that may require special treatment by the display > + subsystem. All pixel values should be given in the displays native resolution. > + > +properties: > + $nodename: > + const: occlusions > + > + corners: > + type: object > + description: | > + An area of the display which is fully obscured by a notch. > + properties: > + radius-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for all corners > + > + radius-top-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for both top corners > + > + radius-bottom-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for both top corners > + > + radius-top-left-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the top left corner in pixels > + > + radius-top-right-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the top right corner in pixels > + > + radius-bottom-left-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the bottom left corner in pixels > + > + radius-bottom-right-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the bottom right corner in pixels > + > + dependencies: > + radius-top-left-px: [ radius-top-right-px ] > + radius-top-right-px: [ radius-top-left-px ] > + radius-bottom-left-px: [ radius-bottom-right-px ] > + radius-bottom-right-px: [ radius-bottom-left-px ] > + > + anyOf: > + - required: > + - radius-px > + - required: > + - radius-top-px > + - radius-bottom-px > + - required: > + - radius-top-px > + - radius-bottom-left-px > + - radius-bottom-right-px > + - required: > + - radius-bottom-px > + - radius-top-left-px > + - radius-top-right-px > + - required: > + - radius-top-left-px > + - radius-top-right-px > + - radius-bottom-left-px > + - radius-bottom-right-px > + > + additionalProperties: false > + > +patternProperties: > + "^cutout(-[0-9])?$": > + type: object > + description: | > + An area of the display which is not directly adjacent to an > + edge and is fully occluded (for example, a camera cutout). > + properties: > + x-position-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The horizontal coordinate of the center of the cutout. > + > + y-position-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The horizontal coordinate of the center of the cutout. > + > + width-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The width of the cutout in pixels. > + > + height-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The width of the cutout in pixels. > + > + corner-radius-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: | > + The radius of the corners in pixels. For a circle this should be half of > + the width/height. > + > + required: > + - x-position-px > + - y-position-px > + - width-px > + - height-px > + - corner-radius-px > + > + additionalProperties: false > + > + "^notch(-[0-9])?$": > + type: object > + description: | > + An area of the display which is fully occluded by a notch. > + properties: > + path: > + $ref: /schemas/types.yaml#/definitions/uint32-matrix > + description: | > + Sequence of values defining the arcs which describe this path. Lines > + are inserted between arcs that don't directly meet. > + maxItems: 8 # Ought to cover most scenarios > + items: > + items: > + - description: X coordinate of center point in pixels > + - description: Y coordinate of center point in pixels > + - description: Circle radius > + - description: Starting angle in degrees > + - description: Ending angle in degrees > + > + required: > + - path > + > + additionalProperties: false > + > + "^(left|right|top|bottom)-edge$": > + type: object > + description: | > + An edge of the screen with reduced visibility due to a waterfall design > + or similar. > + properties: > + size-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The distance from the edge where it begins to obscure visbility > + > + curve-degrees: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The number of degrees that the display curves. > + maximum: 100 > + > + required: > + - size-px > + - curve-degrees > + > +additionalProperties: false > + > +examples: > + - | # A panel with a large-ish notch on the top edge, and different corner radii > + panel { > + occlusions { > + notch { > + path = <360 8 10 0 90>, > + <435 10 20 270 180>, > + <645 10 20 180 90>, > + <645 8 10 90 0>; > + }; > + > + corners { > + radius-top-px = <30>; > + radius-bottom-px = <40>; > + }; > + }; > + }; >
On Mon, Oct 09, 2023 at 06:32:50PM +0100, Caleb Connolly wrote: > Some display panels found in modern phones and laptops feature > non-standard display shapes with features like rounded corners, notches > (sections of the display that are cut-out from the edge), and cutouts > (such as circular "hole punch" camera cutouts, or wider pill-shaped > "islands"). > > Some discussion has been underway previously on how best to describe > these features [1][2], including a reference userspace implementation > using SVG paths [3]. Using this previous discussion as a jumping off > point, this RFC allows for describing the following display features: > > * Corner radius (on a per-corner basis) > * Circular or pill-shaped cutouts > * Notches with arbitrary shapes > > It's easy to make a case for only using rectangles to describe these > missing parts of a display, however this severely limits their utility. > Describing display occlusions as accurately as possible allows for a lot of > useful UX features. For example, displaying a ring around a hole-punch > camera when it's in use, or wrapping UI elements around a notch. These > behaviours are only possible to implement when the dimensions are known > with near pixel-perfect accuracy. > > Cutouts are described as rectangles with a width, height, and corner > radius. > A radius of half the width longest edge will definitionally be an ellipse. > This simple description is suitable for describing hole-punch cameras, > as well > as pill-shaped cutouts. I'm not aware of any devices that can't be > described like this. > > Notches are a little more complicated, they usually feature convex and > concave corners as well as straight lines. Here they are described as a > sequence of optionally disjoint arcs, where the space between one arc ending > and another starting is inferred to be a straight line. > > Each arc is described with an X and Y pixel coordinate, a radius, and a > start and end point in degrees. These arcs can precisely describe the > shape of a notch, and easily allow for a basic bounding box to be > derived using the min/max coordinates specified in the path. > > Some displays feature partially occluded edges ("waterfall edges") where > the screen bends, it may be useful for user interfaces to avoid placing > certain UI elements like buttons too close to these edges. These edges > are described by a distance from the edge where it begins to be > occluded, and the number of degrees that the display curves (this > directly affects how usable this edge of the screen is). > > [1]: https://lore.kernel.org/dri-devel/f8747f99-0695-5be0-841f-4f72ba5d5da3@connolly.tech/ > [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/87 > [3]: https://gitlab.gnome.org/World/Phosh/gmobile > > Signed-off-by: Caleb Connolly <caleb.connolly@linaro.org> > --- > Some folks have previously suggested that this information belongs in > userspace and not in devicetree. I would like to be clear that > devicetree is for describing hardware, and parts of a display which can > never actually be seen are very much properties of the underlying > hardware. Makes sense to me. There's similar work happening for touchscreens/pads here[1]. Seems like there might be some overlap in terms of what the binding needs. > --- > base-commit: 268c4b354d66908697299063c44c0b553b01d935 > > // Caleb (they/them) > --- > .../bindings/display/panel/panel-common.yaml | 7 + > .../bindings/display/panel/panel-occlusions.yaml | 182 +++++++++++++++++++++ > 2 files changed, 189 insertions(+) > > diff --git a/Documentation/devicetree/bindings/display/panel/panel-common.yaml b/Documentation/devicetree/bindings/display/panel/panel-common.yaml > index 0a57a31f4f3d..6ea52a031872 100644 > --- a/Documentation/devicetree/bindings/display/panel/panel-common.yaml > +++ b/Documentation/devicetree/bindings/display/panel/panel-common.yaml > @@ -150,6 +150,13 @@ properties: > controller, this property contains a phandle that references the > controller. > > + occlusions: > + $ref: panel-occlusions.yaml# > + description: > + Some panels have non-standard shapes due to features like rounded corners, > + notches, cutouts, and "waterfall" edges. The panel-occlusions bindings > + can be used to describe these features. > + > dependencies: > width-mm: [ height-mm ] > height-mm: [ width-mm ] > diff --git a/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml > new file mode 100644 > index 000000000000..0932026bbd8c > --- /dev/null > +++ b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml > @@ -0,0 +1,182 @@ > +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause > +%YAML 1.2 > +--- > +$id: http://devicetree.org/schemas/display/panel/panel-occlusions.yaml# > +$schema: http://devicetree.org/meta-schemas/core.yaml# > + > +title: Common Properties for describing display notches, cutouts, and other occlusions > + > +maintainers: > + - Caleb Connolly <caleb.connolly@linaro.org> > + > +description: | You don't need '|' unless there is formatting which is significant. > + This document defines devicetree nodes that can be used to describe > + different kind of display occlusions such as notches, camera cutouts, rounded > + corners, and other features that may require special treatment by the display > + subsystem. All pixel values should be given in the displays native resolution. > + > +properties: > + $nodename: > + const: occlusions > + > + corners: > + type: object > + description: | > + An area of the display which is fully obscured by a notch. notch? > + properties: > + radius-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for all corners > + > + radius-top-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for both top corners > + > + radius-bottom-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: Describes the radius when it's identical for both top corners > + > + radius-top-left-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the top left corner in pixels > + > + radius-top-right-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the top right corner in pixels > + > + radius-bottom-left-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the bottom left corner in pixels > + > + radius-bottom-right-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The radius of the bottom right corner in pixels I think I would define the corner positions the same way as cutouts rather than encoding the position into property names. I'm not sure I'd try to optimize describing all 4 corners other than the all 4 the same case. > + > + dependencies: > + radius-top-left-px: [ radius-top-right-px ] > + radius-top-right-px: [ radius-top-left-px ] > + radius-bottom-left-px: [ radius-bottom-right-px ] > + radius-bottom-right-px: [ radius-bottom-left-px ] > + > + anyOf: > + - required: > + - radius-px > + - required: > + - radius-top-px > + - radius-bottom-px > + - required: > + - radius-top-px > + - radius-bottom-left-px > + - radius-bottom-right-px > + - required: > + - radius-bottom-px > + - radius-top-left-px > + - radius-top-right-px > + - required: > + - radius-top-left-px > + - radius-top-right-px > + - radius-bottom-left-px > + - radius-bottom-right-px > + > + additionalProperties: false > + > +patternProperties: > + "^cutout(-[0-9])?$": > + type: object > + description: | > + An area of the display which is not directly adjacent to an > + edge and is fully occluded (for example, a camera cutout). > + properties: > + x-position-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The horizontal coordinate of the center of the cutout. > + > + y-position-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The horizontal coordinate of the center of the cutout. > + > + width-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The width of the cutout in pixels. > + > + height-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The width of the cutout in pixels. > + > + corner-radius-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: | > + The radius of the corners in pixels. For a circle this should be half of > + the width/height. > + > + required: > + - x-position-px > + - y-position-px > + - width-px > + - height-px > + - corner-radius-px > + > + additionalProperties: false > + > + "^notch(-[0-9])?$": > + type: object > + description: | > + An area of the display which is fully occluded by a notch. > + properties: > + path: Too vague a name. Ideally, a given property name has only 1 type (globally). > + $ref: /schemas/types.yaml#/definitions/uint32-matrix > + description: | > + Sequence of values defining the arcs which describe this path. Lines > + are inserted between arcs that don't directly meet. > + maxItems: 8 # Ought to cover most scenarios > + items: > + items: > + - description: X coordinate of center point in pixels > + - description: Y coordinate of center point in pixels > + - description: Circle radius > + - description: Starting angle in degrees > + - description: Ending angle in degrees > + > + required: > + - path Couldn't you just describe the whole edge of the screen this way? IOW, the corners and edges, too. > + > + additionalProperties: false > + > + "^(left|right|top|bottom)-edge$": > + type: object > + description: | > + An edge of the screen with reduced visibility due to a waterfall design > + or similar. > + properties: > + size-px: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The distance from the edge where it begins to obscure visbility > + > + curve-degrees: > + $ref: /schemas/types.yaml#/definitions/uint32 > + description: The number of degrees that the display curves. > + maximum: 100 > + > + required: > + - size-px > + - curve-degrees > + > +additionalProperties: false [1] https://lore.kernel.org/all/20230510-feature-ts_virtobj_patch-v4-2-5c6c0fc1eed6@wolfvision.net/
On 10/10/2023 17:52, Rob Herring wrote: > On Mon, Oct 09, 2023 at 06:32:50PM +0100, Caleb Connolly wrote: >> Some display panels found in modern phones and laptops feature >> non-standard display shapes with features like rounded corners, notches >> (sections of the display that are cut-out from the edge), and cutouts >> (such as circular "hole punch" camera cutouts, or wider pill-shaped >> "islands"). >> >> Some discussion has been underway previously on how best to describe >> these features [1][2], including a reference userspace implementation >> using SVG paths [3]. Using this previous discussion as a jumping off >> point, this RFC allows for describing the following display features: >> >> * Corner radius (on a per-corner basis) >> * Circular or pill-shaped cutouts >> * Notches with arbitrary shapes >> >> It's easy to make a case for only using rectangles to describe these >> missing parts of a display, however this severely limits their utility. >> Describing display occlusions as accurately as possible allows for a lot of >> useful UX features. For example, displaying a ring around a hole-punch >> camera when it's in use, or wrapping UI elements around a notch. These >> behaviours are only possible to implement when the dimensions are known >> with near pixel-perfect accuracy. >> >> Cutouts are described as rectangles with a width, height, and corner >> radius. >> A radius of half the width longest edge will definitionally be an ellipse. >> This simple description is suitable for describing hole-punch cameras, >> as well >> as pill-shaped cutouts. I'm not aware of any devices that can't be >> described like this. >> >> Notches are a little more complicated, they usually feature convex and >> concave corners as well as straight lines. Here they are described as a >> sequence of optionally disjoint arcs, where the space between one arc ending >> and another starting is inferred to be a straight line. >> >> Each arc is described with an X and Y pixel coordinate, a radius, and a >> start and end point in degrees. These arcs can precisely describe the >> shape of a notch, and easily allow for a basic bounding box to be >> derived using the min/max coordinates specified in the path. >> >> Some displays feature partially occluded edges ("waterfall edges") where >> the screen bends, it may be useful for user interfaces to avoid placing >> certain UI elements like buttons too close to these edges. These edges >> are described by a distance from the edge where it begins to be >> occluded, and the number of degrees that the display curves (this >> directly affects how usable this edge of the screen is). >> >> [1]: https://lore.kernel.org/dri-devel/f8747f99-0695-5be0-841f-4f72ba5d5da3@connolly.tech/ >> [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/87 >> [3]: https://gitlab.gnome.org/World/Phosh/gmobile >> >> Signed-off-by: Caleb Connolly <caleb.connolly@linaro.org> >> --- >> Some folks have previously suggested that this information belongs in >> userspace and not in devicetree. I would like to be clear that >> devicetree is for describing hardware, and parts of a display which can >> never actually be seen are very much properties of the underlying >> hardware. > > Makes sense to me. > > There's similar work happening for touchscreens/pads here[1]. Seems like > there might be some overlap in terms of what the binding needs. hmm, maybe for the cutouts it would make sense to have a common overlay binding, we would just need to add a radius property. > >> --- >> base-commit: 268c4b354d66908697299063c44c0b553b01d935 >> >> // Caleb (they/them) >> --- >> .../bindings/display/panel/panel-common.yaml | 7 + >> .../bindings/display/panel/panel-occlusions.yaml | 182 +++++++++++++++++++++ >> 2 files changed, 189 insertions(+) >> >> diff --git a/Documentation/devicetree/bindings/display/panel/panel-common.yaml b/Documentation/devicetree/bindings/display/panel/panel-common.yaml >> index 0a57a31f4f3d..6ea52a031872 100644 >> --- a/Documentation/devicetree/bindings/display/panel/panel-common.yaml >> +++ b/Documentation/devicetree/bindings/display/panel/panel-common.yaml >> @@ -150,6 +150,13 @@ properties: >> controller, this property contains a phandle that references the >> controller. >> >> + occlusions: >> + $ref: panel-occlusions.yaml# >> + description: >> + Some panels have non-standard shapes due to features like rounded corners, >> + notches, cutouts, and "waterfall" edges. The panel-occlusions bindings >> + can be used to describe these features. >> + >> dependencies: >> width-mm: [ height-mm ] >> height-mm: [ width-mm ] >> diff --git a/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml >> new file mode 100644 >> index 000000000000..0932026bbd8c >> --- /dev/null >> +++ b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml >> @@ -0,0 +1,182 @@ >> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause >> +%YAML 1.2 >> +--- >> +$id: http://devicetree.org/schemas/display/panel/panel-occlusions.yaml# >> +$schema: http://devicetree.org/meta-schemas/core.yaml# >> + >> +title: Common Properties for describing display notches, cutouts, and other occlusions >> + >> +maintainers: >> + - Caleb Connolly <caleb.connolly@linaro.org> >> + >> +description: | > > You don't need '|' unless there is formatting which is significant. Right, thanks > >> + This document defines devicetree nodes that can be used to describe >> + different kind of display occlusions such as notches, camera cutouts, rounded >> + corners, and other features that may require special treatment by the display >> + subsystem. All pixel values should be given in the displays native resolution. >> + >> +properties: >> + $nodename: >> + const: occlusions >> + >> + corners: >> + type: object >> + description: | >> + An area of the display which is fully obscured by a notch. > > notch? Oops... There's a few other cut/paste issues here, I'll fix them all > >> + properties: >> + radius-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: Describes the radius when it's identical for all corners >> + >> + radius-top-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: Describes the radius when it's identical for both top corners >> + >> + radius-bottom-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: Describes the radius when it's identical for both top corners >> + >> + radius-top-left-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The radius of the top left corner in pixels >> + >> + radius-top-right-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The radius of the top right corner in pixels >> + >> + radius-bottom-left-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The radius of the bottom left corner in pixels >> + >> + radius-bottom-right-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The radius of the bottom right corner in pixels > > I think I would define the corner positions the same way as cutouts > rather than encoding the position into property names. Do you by specifying the positions in pixel coordinates? I wanted to try and avoid specifying information that could just be inferred. > > I'm not sure I'd try to optimize describing all 4 corners other than the > all 4 the same case. Yeah, the top/bottom case is a bit much. > >> + >> + dependencies: >> + radius-top-left-px: [ radius-top-right-px ] >> + radius-top-right-px: [ radius-top-left-px ] >> + radius-bottom-left-px: [ radius-bottom-right-px ] >> + radius-bottom-right-px: [ radius-bottom-left-px ] >> + >> + anyOf: >> + - required: >> + - radius-px >> + - required: >> + - radius-top-px >> + - radius-bottom-px >> + - required: >> + - radius-top-px >> + - radius-bottom-left-px >> + - radius-bottom-right-px >> + - required: >> + - radius-bottom-px >> + - radius-top-left-px >> + - radius-top-right-px >> + - required: >> + - radius-top-left-px >> + - radius-top-right-px >> + - radius-bottom-left-px >> + - radius-bottom-right-px >> + >> + additionalProperties: false >> + >> +patternProperties: >> + "^cutout(-[0-9])?$": >> + type: object >> + description: | >> + An area of the display which is not directly adjacent to an >> + edge and is fully occluded (for example, a camera cutout). >> + properties: >> + x-position-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The horizontal coordinate of the center of the cutout. >> + >> + y-position-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The horizontal coordinate of the center of the cutout. >> + >> + width-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The width of the cutout in pixels. >> + >> + height-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The width of the cutout in pixels. >> + >> + corner-radius-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: | >> + The radius of the corners in pixels. For a circle this should be half of >> + the width/height. >> + >> + required: >> + - x-position-px >> + - y-position-px >> + - width-px >> + - height-px >> + - corner-radius-px >> + >> + additionalProperties: false >> + >> + "^notch(-[0-9])?$": >> + type: object >> + description: | >> + An area of the display which is fully occluded by a notch. >> + properties: >> + path: > > Too vague a name. Ideally, a given property name has only 1 type > (globally). Right, and I realise there maybe doesn't need to be a whole subnode for this, it could just be "occlusion,path-0" and "occlusion,path-names". > >> + $ref: /schemas/types.yaml#/definitions/uint32-matrix >> + description: | >> + Sequence of values defining the arcs which describe this path. Lines >> + are inserted between arcs that don't directly meet. >> + maxItems: 8 # Ought to cover most scenarios >> + items: >> + items: >> + - description: X coordinate of center point in pixels >> + - description: Y coordinate of center point in pixels >> + - description: Circle radius >> + - description: Starting angle in degrees >> + - description: Ending angle in degrees >> + >> + required: >> + - path > > Couldn't you just describe the whole edge of the screen this way? IOW, > the corners and edges, too. Yes, I think you could. We could add additional path segment types, at which point we're comparing corners { radius = <30>; }; with path = <PATH_CORNER_RADIUS_ALL 30>; As a third solution, what about dropping the subnode and using a more verbose property name: occlusions,corner-radius = <30>; > >> + >> + additionalProperties: false >> + >> + "^(left|right|top|bottom)-edge$": >> + type: object >> + description: | >> + An edge of the screen with reduced visibility due to a waterfall design >> + or similar. >> + properties: >> + size-px: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The distance from the edge where it begins to obscure visbility >> + >> + curve-degrees: >> + $ref: /schemas/types.yaml#/definitions/uint32 >> + description: The number of degrees that the display curves. >> + maximum: 100 >> + >> + required: >> + - size-px >> + - curve-degrees >> + >> +additionalProperties: false > > > [1] https://lore.kernel.org/all/20230510-feature-ts_virtobj_patch-v4-2-5c6c0fc1eed6@wolfvision.net/
On 10/10/2023 21:01, Piotr Masłowski wrote: > Hi Caleb, > > Thanks for posting this. I've been meaning to chime in on the discussion > about notches and co. for months now, so this makes a perfect opportunity > to finally do so. > > On Mon Oct 9, 2023 at 7:32 PM CEST, Caleb Connolly wrote: > >> Some folks have previously suggested that this information belongs in >> userspace and not in devicetree. I would like to be clear that >> devicetree is for describing hardware, and parts of a display which can >> never actually be seen are very much properties of the underlying >> hardware. > > Yes, thank you! Seeing the kernel log (or Tuxes) partially hidden behind > the notch (or the top-left rounded corner) is just so annoying, so DT is > definitely the way to go here. Exactly! Glad that others care about the _real_ issue here ;P [...] > > > So why am I writing all of this? Well, the problem I see is that any > shape-based approach will likely suffer from both accuracy and > complexity issues. Describing curves is hard and processing them is > not something that should be included in e.g. whatever handles VTs. My proposal here doesn't require processing curves or doing any complicated calculations. If you know that the display has a 30px radius on all corners, you can adjust the VT to not use the top 30px of the screen and it will start exactly where the radius stops. If you know that there is a (potentially very curvy) notch at the top of the screen, you can just iterate over the arcs, add their radius to their Y coordinate and take the maximum. This will always give you at least the height of the notch (you'd probably want to check that their angle is vaguely downward, but again this is trivial fast integer math). [...] > > However, there's a different approach that is both extremely simple and > yet trivially correct – pixel masks! > > Basically, instead of trying to describe what shape a cutout, notch or > other feature has, we just say which pixels belong to it. In short, this: > > - can be easily processed even by 'dumb' programs – for example, you can > derive minimal margins to 'letterbox' the screen As above (and as mentioned in the patch) this is also trivial to derive from the curves. > - gathering the data is very straightforward – just light the relevant > pixels one-by-one and check if you see them > - pixel-perfect accuracy is the default I think it would be fairly straightforward to do this for curve data too. You just bump the radius up/down until it looks right, or "good enough" > > Of course this is not a perfect solution. Here are the drawback that I > can see: > > - low resolution screens mean low resolution data > - 'smart' programs may or may not need a bit more logic than otherwise > - sub-pixel accuracy is impossible or requires extending this scheme > - non-binary (fractional) weights > - partially-occluded pixels on a separate mask (so each feature would > have two adjacent masks – for its interior and for the border) > > > As a futher improvement, besides looking at pixels we could think about > subpixels instead. Still, this can easily be added later and likely even > in a backwards-compatible manner. > > An orthogonal issue is labeling all of those regions. I think we should > start with fully obscured areas and maybe less readable ones like the > waterfall edges. Still, different features should live on different > masks – even if we don't attach meaningfull labels (like 'notch' or > 'camera cutout') to them right away. > > > What do you all think of that? I didn't see this approach considered in > any of the earlier discussions, yet it seems so elegant to me. Am I > missing something? I think the unfortunate truth is that approximating notches and rounded corners exclusively with regular arcs at the cost of pixel accuracy is just such a no-brainer. Pixel masks would be pixel accurate, but there is no benefit compared to a slightly underfit curve. Any program which wanted to make use of the curve information would have to run a whole curve fitting algorithm, whereas there simply aren't any programs which need to know about display occlusions to a pixel-accurate level. For padding a status bar you do a single trig equation, for avoiding the notch in fullscreen you would query the DRM subsystem which presumably would export the dimensions of a letterboxed "usable size". To be clear though, I wouldn't mind being proven wrong on this, but I'm yet to find some information which doesn't line up with this conclusion. > > -- > Cheers, > Piotr Masłowski
On Tue Oct 10, 2023 at 10:36 PM CEST, Caleb Connolly wrote: >> So why am I writing all of this? Well, the problem I see is that any >> shape-based approach will likely suffer from both accuracy and >> complexity issues. Describing curves is hard and processing them is >> not something that should be included in e.g. whatever handles VTs. > > My proposal here doesn't require processing curves or doing any > complicated calculations. If you know that the display has a 30px radius > on all corners, you can adjust the VT to not use the top 30px of the > screen and it will start exactly where the radius stops. Just a nitpick, but on a round smartwatch this approach will give you a $width × 0px famebuffer ;D > > If you know that there is a (potentially very curvy) notch at the top of > the screen, you can just iterate over the arcs, add their radius to > their Y coordinate and take the maximum. This will always give you at > least the height of the notch (you'd probably want to check that their > angle is vaguely downward, but again this is trivial fast integer math). Yeah, I was talking about curves in general. Your proposal is on the low-complexity side thankfully. >> - gathering the data is very straightforward – just light the relevant >> pixels one-by-one and check if you see them >> - pixel-perfect accuracy is the default > > I think it would be fairly straightforward to do this for curve data > too. You just bump the radius up/down until it looks right, or "good enough" Well, different people will have different standards and what might be good enough for some will annoy others. I'm not saying that we need to be perfect at all cost, but if there's a relatively easy way to cut down on inconsistency, it might be worth taking. Additionally, having a very clear-cut quality indicator could remove a whole class of bikeshedding options. (speaking of the devil xD) The problem I have with curves is, the more 'correct' you make them, the more convoluted they become. Like take for example the corners. Rounded corners are circular right? But what if someone makes them 'squircular'? Well, they are usually quite small anyway so maybe it will work out. But then what about a smartwatch with big, squircle-shaped display? Bam, now you need to complicate how corners are handled. But also: are rounded coners typically circular? Just now I've thrown a OnePlus 6 (thank you so much for the great mainline support btw!) onto a flatbed scanner. While a circle fits decently well there it's not really a perfect fit, so maybe they went with a different curve after all. That being said, imperfection isn't my main issue with curves. It's all the non-discreteness they bring to the table. As in, you now need to care about approximations, rounding, imprecise measurements and so on. > I think the unfortunate truth is that approximating notches and rounded > corners exclusively with regular arcs at the cost of pixel accuracy is > just such a no-brainer. Pixel masks would be pixel accurate, but there > is no benefit compared to a slightly underfit curve. Frankly, I don't see any significant cost here. It's very easy to gather and rather easy to process. Let me think about it… The most common operations I see people actually doing with this data would be: * letterboxing – easy with both curves and masks * ensuring padding | margins for icons on the screen – with curves you can use a formula, but won't it be easier to just count pixels anyway? * routing a spline along the border – like if you want to have some periodic pattern drawn there, it's probably a bit easier to do with curves * drawing something at a constant distance from the border – with curves you can again use exact formulas. But isn't that an overkill really? I'd think most people will go for something like a morphological dilation instead > > Any program which wanted to make use of the curve information would have > to run a whole curve fitting algorithm, whereas there simply aren't any > programs which need to know about display occlusions to a pixel-accurate > level. For padding a status bar you do a single trig equation, for > avoiding the notch in fullscreen you would query the DRM subsystem which > presumably would export the dimensions of a letterboxed "usable size". What could the curve information be needed for though? The more I think about it, the less value I see in it really. Like, if you're adjusting the screen contents, pixels _are_ the smallest unit you need to concern yourself with. The only thing that comes up to my mind is if you were to animate the cutout shape somewhere else, zoomed in. And thats already a quite contrived scenario. Is there actually any use case that instead of ending up with pixels either way, would be better served by a (possibly inaccurate) curve? (future me here: that spline-along-the-border from earlier I guess) -- Cheers, Piotr Masłowski
On 10/10/2023 23:53, Piotr Masłowski wrote: > On Tue Oct 10, 2023 at 10:36 PM CEST, Caleb Connolly wrote: > >>> So why am I writing all of this? Well, the problem I see is that any >>> shape-based approach will likely suffer from both accuracy and >>> complexity issues. Describing curves is hard and processing them is >>> not something that should be included in e.g. whatever handles VTs. >> >> My proposal here doesn't require processing curves or doing any >> complicated calculations. If you know that the display has a 30px radius >> on all corners, you can adjust the VT to not use the top 30px of the >> screen and it will start exactly where the radius stops. > > Just a nitpick, but on a round smartwatch this approach will give you > a $width × 0px famebuffer ;D right right, it's a little more complicated, but not more or less with either approach. [...] >>> - gathering the data is very straightforward – just light the relevant >>> pixels one-by-one and check if you see them >>> - pixel-perfect accuracy is the default >> >> I think it would be fairly straightforward to do this for curve data >> too. You just bump the radius up/down until it looks right, or "good enough" > > Well, different people will have different standards and what might be > good enough for some will annoy others. I'm not saying that we need to > be perfect at all cost, but if there's a relatively easy way to cut down > on inconsistency, it might be worth taking. > > Additionally, having a very clear-cut quality indicator could remove a > whole class of bikeshedding options. (speaking of the devil xD) > > > The problem I have with curves is, the more 'correct' you make them, the > more convoluted they become. Like take for example the corners. Rounded > corners are circular right? But what if someone makes them 'squircular'? > Well, they are usually quite small anyway so maybe it will work out. > > But then what about a smartwatch with big, squircle-shaped display? Bam, > now you need to complicate how corners are handled. They don't need to be correct, you don't need to complicate it, you just need a value that plays nice. When it comes down to it you're much more likely to be constrained by UI layout limitations by not being able to model the precise corner curves of your device. The difference between a circular and elliptical arc is negligible in all real world use cases. > > But also: are rounded coners typically circular? Just now I've thrown a > OnePlus 6 (thank you so much for the great mainline support btw!) onto a > flatbed scanner. While a circle fits decently well there it's not really > a perfect fit, so maybe they went with a different curve after all. > > That being said, imperfection isn't my main issue with curves. It's all > the non-discreteness they bring to the table. As in, you now need to care > about approximations, rounding, imprecise measurements and so on. My point is that actually you don't. Other than for animated visual effects (where an undersized curve would absolutely be acceptable) I can't conceive of a situation where plain triangles wouldn't be suitable, again slightly undersized of course. > > >> I think the unfortunate truth is that approximating notches and rounded >> corners exclusively with regular arcs at the cost of pixel accuracy is >> just such a no-brainer. Pixel masks would be pixel accurate, but there >> is no benefit compared to a slightly underfit curve. > > Frankly, I don't see any significant cost here. It's very easy to gather > and rather easy to process. Unless I'm mistaken, it would mean that for "odd" shapes like rounded corners, the number of values you need to record would be directly proportional to the number of rows. You can do some optimisations, but that worst case is really not great. Especially when the arc approximation requires a single value and covers all the same usecases as well or better. > > Let me think about it… The most common operations I see people actually > doing with this data would be: > > * letterboxing – easy with both curves and masks > * ensuring padding | margins for icons on the screen – with curves you > can use a formula, but won't it be easier to just count pixels anyway? You'd probably want your status bar icons to be equi-distant from the top and side, at least that's what my Android phone does. But hey, maybe you don't, we can't bikeshed UX design all day but I'd think a simple radius is gonna be easier to deal with than a pixel mask in most cases (especially considering that most UI frameworks don't work in the pixel space anyway because of scaling). > * routing a spline along the border – like if you want to have some > periodic pattern drawn there, it's probably a bit easier to do with > curves > * drawing something at a constant distance from the border – with curves > you can again use exact formulas. But isn't that an overkill really? > I'd think most people will go for something like a morphological > dilation instead Right right > >> >> Any program which wanted to make use of the curve information would have >> to run a whole curve fitting algorithm, whereas there simply aren't any >> programs which need to know about display occlusions to a pixel-accurate >> level. For padding a status bar you do a single trig equation, for >> avoiding the notch in fullscreen you would query the DRM subsystem which >> presumably would export the dimensions of a letterboxed "usable size". > > What could the curve information be needed for though? > The more I think about it, the less value I see in it really. Like, if > you're adjusting the screen contents, pixels _are_ the smallest unit you > need to concern yourself with. The only thing that comes up to my mind > is if you were to animate the cutout shape somewhere else, zoomed in. > And thats already a quite contrived scenario. GNOME doesn't even use real pixels, macOS doesn't use real pixels. When scaling comes into play the advantage is lost either way. > > Is there actually any use case that instead of ending up with pixels > either way, would be better served by a (possibly inaccurate) curve? > (future me here: that spline-along-the-border from earlier I guess) I feel like the burden of proof still lies with you here. I feel much more comfortable with a handful of easy to reason about and explain values (that can be easily reviewed by maintainers) over a blob of data that grows with the resolution of your display. Thanks, > > -- > Cheers, > Piotr Masłowski
On Wed Oct 11, 2023 at 1:35 AM CEST, Caleb Connolly wrote: > On 10/10/2023 23:53, Piotr Masłowski wrote: > They don't need to be correct, you don't need to complicate it, you just > need a value that plays nice. When it comes down to it you're much more > likely to be constrained by UI layout limitations by not being able to > model the precise corner curves of your device. The difference between a > circular and elliptical arc is negligible in all real world use cases. Well, with a large, weirdly-shaped display the curve will have to pass through more-or-less the right pixels for things like a-couple-px-thick frame to look right. I would expect e.g. a smartwatch UI to sometimes want to draw a constant-width frame along the screen border. Using a significantly different curve there will break that, so you would need to let them be similarly arbitrary to notches. >> That being said, imperfection isn't my main issue with curves. It's all >> the non-discreteness they bring to the table. As in, you now need to care >> about approximations, rounding, imprecise measurements and so on. > > My point is that actually you don't. Other than for animated visual > effects (where an undersized curve would absolutely be acceptable) I > can't conceive of a situation where plain triangles wouldn't be > suitable, again slightly undersized of course. Well, what I meant is not that you "have to care or it won't look good", but rather "that's an unnecessary degree of freedom" and I'm sure pepole will spend non-insignificant amounts of time caring about it. Both while gathering the data and while processing it. But you're right that it won't matter for typical corners – they are simply too small to make any difference here. >> Frankly, I don't see any significant cost here. It's very easy to gather >> and rather easy to process. > > Unless I'm mistaken, it would mean that for "odd" shapes like rounded > corners, the number of values you need to record would be directly > proportional to the number of rows. You can do some optimisations, but > that worst case is really not great. Especially when the arc > approximation requires a single value and covers all the same usecases > as well or better. You mean the devicetrees getting bloated? The blobs are already dozens of KiB in size. As for the sources, I don't have any real experience with them so I'm not sure what would be the best way of representing masks. Worst-case, they can be treated as raw bitmaps. But since the features will consist of typically one (or just a few) contiguous blobs they can be efficiently represented using for example chain codes. > You'd probably want your status bar icons to be equi-distant from the > top and side, at least that's what my Android phone does. But hey, maybe > you don't, we can't bikeshed UX design all day but I'd think a simple > radius is gonna be easier to deal with than a pixel mask in most cases > (especially considering that most UI frameworks don't work in the pixel > space anyway because of scaling). Good point! (Though don't they need to still align stuff to the pixel grid to avoid needless antialiasing?) Hmm, but known radius only helps you with the corners. For the notch, you would need a complex (and shape-shifting) equation, so you might as well figure out the constant-distance based on a mask I guess. > GNOME doesn't even use real pixels, macOS doesn't use real pixels. When > scaling comes into play the advantage is lost either way. Fractional scaling is a worthwile cause, but I don't think my argument really changes much when you're working with scaled pixels instead of real ones. >> >> Is there actually any use case that instead of ending up with pixels >> either way, would be better served by a (possibly inaccurate) curve? >> (future me here: that spline-along-the-border from earlier I guess) > > I feel like the burden of proof still lies with you here. I feel much > more comfortable with a handful of easy to reason about and explain > values (that can be easily reviewed by maintainers) over a blob of data > that grows with the resolution of your display. How are these reviewable without testing them on a device though? True that with corners a maintainer can make sure the values make some sense. But for notches they can only try visualizing the data to see if it more or less looks right. And that's equally true for pixel masks. Drawing the shape by hand would be less tedious for a curve I guess. Thanks for going over these things with me. It has definitely clarified many points and gave me a better understanding overall. I guess we won't convince each other that easily, so maybe let's see what others think. (but if you want to discuss it further, I'm all ears) And now that I think about it, maybe it's chain codes that could be the best of both worlds? They operate on pixels and so remain exact, but also are basically curves – just very simple ones. I'll have to think this through. -- Cheers, Piotr Masłowski
Hi! > > An orthogonal issue is labeling all of those regions. I think we should > > start with fully obscured areas and maybe less readable ones like the > > waterfall edges. Still, different features should live on different > > masks – even if we don't attach meaningfull labels (like 'notch' or > > 'camera cutout') to them right away. > > > > > > What do you all think of that? I didn't see this approach considered in > > any of the earlier discussions, yet it seems so elegant to me. Am I > > missing something? > > I think the unfortunate truth is that approximating notches and rounded > corners exclusively with regular arcs at the cost of pixel accuracy is > just such a no-brainer. Pixel masks would be pixel accurate, but there > is no benefit compared to a slightly underfit curve. Pixel accuracy may be important for low-resolution displays. And... I have certain low-resolution displays in mind. There are keyboards with individual backlights below each key. Maybe they could/should be treated similarily to displays with occlusions? But we'll really need to be pixel-perfect for that. Best regards, Pavel
diff --git a/Documentation/devicetree/bindings/display/panel/panel-common.yaml b/Documentation/devicetree/bindings/display/panel/panel-common.yaml index 0a57a31f4f3d..6ea52a031872 100644 --- a/Documentation/devicetree/bindings/display/panel/panel-common.yaml +++ b/Documentation/devicetree/bindings/display/panel/panel-common.yaml @@ -150,6 +150,13 @@ properties: controller, this property contains a phandle that references the controller. + occlusions: + $ref: panel-occlusions.yaml# + description: + Some panels have non-standard shapes due to features like rounded corners, + notches, cutouts, and "waterfall" edges. The panel-occlusions bindings + can be used to describe these features. + dependencies: width-mm: [ height-mm ] height-mm: [ width-mm ] diff --git a/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml new file mode 100644 index 000000000000..0932026bbd8c --- /dev/null +++ b/Documentation/devicetree/bindings/display/panel/panel-occlusions.yaml @@ -0,0 +1,182 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/display/panel/panel-occlusions.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Common Properties for describing display notches, cutouts, and other occlusions + +maintainers: + - Caleb Connolly <caleb.connolly@linaro.org> + +description: | + This document defines devicetree nodes that can be used to describe + different kind of display occlusions such as notches, camera cutouts, rounded + corners, and other features that may require special treatment by the display + subsystem. All pixel values should be given in the displays native resolution. + +properties: + $nodename: + const: occlusions + + corners: + type: object + description: | + An area of the display which is fully obscured by a notch. + properties: + radius-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Describes the radius when it's identical for all corners + + radius-top-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Describes the radius when it's identical for both top corners + + radius-bottom-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: Describes the radius when it's identical for both top corners + + radius-top-left-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The radius of the top left corner in pixels + + radius-top-right-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The radius of the top right corner in pixels + + radius-bottom-left-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The radius of the bottom left corner in pixels + + radius-bottom-right-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The radius of the bottom right corner in pixels + + dependencies: + radius-top-left-px: [ radius-top-right-px ] + radius-top-right-px: [ radius-top-left-px ] + radius-bottom-left-px: [ radius-bottom-right-px ] + radius-bottom-right-px: [ radius-bottom-left-px ] + + anyOf: + - required: + - radius-px + - required: + - radius-top-px + - radius-bottom-px + - required: + - radius-top-px + - radius-bottom-left-px + - radius-bottom-right-px + - required: + - radius-bottom-px + - radius-top-left-px + - radius-top-right-px + - required: + - radius-top-left-px + - radius-top-right-px + - radius-bottom-left-px + - radius-bottom-right-px + + additionalProperties: false + +patternProperties: + "^cutout(-[0-9])?$": + type: object + description: | + An area of the display which is not directly adjacent to an + edge and is fully occluded (for example, a camera cutout). + properties: + x-position-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The horizontal coordinate of the center of the cutout. + + y-position-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The horizontal coordinate of the center of the cutout. + + width-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The width of the cutout in pixels. + + height-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The width of the cutout in pixels. + + corner-radius-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + The radius of the corners in pixels. For a circle this should be half of + the width/height. + + required: + - x-position-px + - y-position-px + - width-px + - height-px + - corner-radius-px + + additionalProperties: false + + "^notch(-[0-9])?$": + type: object + description: | + An area of the display which is fully occluded by a notch. + properties: + path: + $ref: /schemas/types.yaml#/definitions/uint32-matrix + description: | + Sequence of values defining the arcs which describe this path. Lines + are inserted between arcs that don't directly meet. + maxItems: 8 # Ought to cover most scenarios + items: + items: + - description: X coordinate of center point in pixels + - description: Y coordinate of center point in pixels + - description: Circle radius + - description: Starting angle in degrees + - description: Ending angle in degrees + + required: + - path + + additionalProperties: false + + "^(left|right|top|bottom)-edge$": + type: object + description: | + An edge of the screen with reduced visibility due to a waterfall design + or similar. + properties: + size-px: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The distance from the edge where it begins to obscure visbility + + curve-degrees: + $ref: /schemas/types.yaml#/definitions/uint32 + description: The number of degrees that the display curves. + maximum: 100 + + required: + - size-px + - curve-degrees + +additionalProperties: false + +examples: + - | # A panel with a large-ish notch on the top edge, and different corner radii + panel { + occlusions { + notch { + path = <360 8 10 0 90>, + <435 10 20 270 180>, + <645 10 20 180 90>, + <645 8 10 90 0>; + }; + + corners { + radius-top-px = <30>; + radius-bottom-px = <40>; + }; + }; + };
Some display panels found in modern phones and laptops feature non-standard display shapes with features like rounded corners, notches (sections of the display that are cut-out from the edge), and cutouts (such as circular "hole punch" camera cutouts, or wider pill-shaped "islands"). Some discussion has been underway previously on how best to describe these features [1][2], including a reference userspace implementation using SVG paths [3]. Using this previous discussion as a jumping off point, this RFC allows for describing the following display features: * Corner radius (on a per-corner basis) * Circular or pill-shaped cutouts * Notches with arbitrary shapes It's easy to make a case for only using rectangles to describe these missing parts of a display, however this severely limits their utility. Describing display occlusions as accurately as possible allows for a lot of useful UX features. For example, displaying a ring around a hole-punch camera when it's in use, or wrapping UI elements around a notch. These behaviours are only possible to implement when the dimensions are known with near pixel-perfect accuracy. Cutouts are described as rectangles with a width, height, and corner radius. A radius of half the width longest edge will definitionally be an ellipse. This simple description is suitable for describing hole-punch cameras, as well as pill-shaped cutouts. I'm not aware of any devices that can't be described like this. Notches are a little more complicated, they usually feature convex and concave corners as well as straight lines. Here they are described as a sequence of optionally disjoint arcs, where the space between one arc ending and another starting is inferred to be a straight line. Each arc is described with an X and Y pixel coordinate, a radius, and a start and end point in degrees. These arcs can precisely describe the shape of a notch, and easily allow for a basic bounding box to be derived using the min/max coordinates specified in the path. Some displays feature partially occluded edges ("waterfall edges") where the screen bends, it may be useful for user interfaces to avoid placing certain UI elements like buttons too close to these edges. These edges are described by a distance from the edge where it begins to be occluded, and the number of degrees that the display curves (this directly affects how usable this edge of the screen is). [1]: https://lore.kernel.org/dri-devel/f8747f99-0695-5be0-841f-4f72ba5d5da3@connolly.tech/ [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/87 [3]: https://gitlab.gnome.org/World/Phosh/gmobile Signed-off-by: Caleb Connolly <caleb.connolly@linaro.org> --- Some folks have previously suggested that this information belongs in userspace and not in devicetree. I would like to be clear that devicetree is for describing hardware, and parts of a display which can never actually be seen are very much properties of the underlying hardware. --- base-commit: 268c4b354d66908697299063c44c0b553b01d935 // Caleb (they/them) --- .../bindings/display/panel/panel-common.yaml | 7 + .../bindings/display/panel/panel-occlusions.yaml | 182 +++++++++++++++++++++ 2 files changed, 189 insertions(+)