| Message ID | 20240409120211.321153-3-angelogioacchino.delregno@collabora.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | drm/mediatek: Add support for OF graphs | expand |
On Tue, Apr 09, 2024 at 02:02:10PM +0200, AngeloGioacchino Del Regno wrote: > Document OF graph on MMSYS/VDOSYS: this supports up to three DDP paths > per HW instance (so potentially up to six displays for multi-vdo SoCs). > > The MMSYS or VDOSYS is always the first component in the DDP pipeline, > so it only supports an output port with multiple endpoints - where each > endpoint defines the starting point for one of the (currently three) > possible hardware paths. > > Signed-off-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> > --- > .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 +++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > index b3c6888c1457..4e9acd966aa5 100644 > --- a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > +++ b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > @@ -93,6 +93,29 @@ properties: > '#reset-cells': > const: 1 > > + port: > + $ref: /schemas/graph.yaml#/properties/port > + description: > + Output port node. This port connects the MMSYS/VDOSYS output to > + the first component of one display pipeline, for example one of > + the available OVL or RDMA blocks. > + Some MediaTek SoCs support up to three display outputs per MMSYS. I'm have a hard time understanding this, but is it 3 outputs simultaneously or connect mmsys to 1 of 3. Generally it's multiple ports for the former and endpoints for the latter. > + properties: > + endpoint@0: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the primary display pipeline > + > + endpoint@1: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the secondary display pipeline > + > + endpoint@2: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the tertiary display pipeline > + > + required: > + - endpoint@0 > + > required: > - compatible > - reg > -- > 2.44.0 >
Il 10/04/24 21:15, Rob Herring ha scritto: > On Tue, Apr 09, 2024 at 02:02:10PM +0200, AngeloGioacchino Del Regno wrote: >> Document OF graph on MMSYS/VDOSYS: this supports up to three DDP paths >> per HW instance (so potentially up to six displays for multi-vdo SoCs). >> >> The MMSYS or VDOSYS is always the first component in the DDP pipeline, >> so it only supports an output port with multiple endpoints - where each >> endpoint defines the starting point for one of the (currently three) >> possible hardware paths. >> >> Signed-off-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> >> --- >> .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 +++++++++++++++++++ >> 1 file changed, 23 insertions(+) >> >> diff --git a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> index b3c6888c1457..4e9acd966aa5 100644 >> --- a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> +++ b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> @@ -93,6 +93,29 @@ properties: >> '#reset-cells': >> const: 1 >> >> + port: >> + $ref: /schemas/graph.yaml#/properties/port >> + description: >> + Output port node. This port connects the MMSYS/VDOSYS output to >> + the first component of one display pipeline, for example one of >> + the available OVL or RDMA blocks. >> + Some MediaTek SoCs support up to three display outputs per MMSYS. > > I'm have a hard time understanding this, but is it 3 outputs > simultaneously or connect mmsys to 1 of 3. Generally it's multiple ports > for the former and endpoints for the latter. > Yes I feel you, MediaTek SoCs are a bit strange, but I do have a reason to use one port and multiple endpoints, instead of multiple ports and one endpoint. On MediaTek SoCs, there are multiple ports: those multiple ports are represented by multiple MMSYS or multiple VDOSYS (depending on the SoC), which do then have multiple endpoints. However, the multiple ports, at least for now, are represented by multiple MMSYS and/or multiple VDOSYS nodes instead of one MM/VDO node with multiple iostart for the multiple blocks in `reg`. The multiple iostart "thing" was the initial design by MediaTek, but there was no way to get them really connected the right way unless adding an iostart restriction in the driver itself (so that the mmsys driver would check an iostart to probe the mediatek-drm components for the right IP number), so, after quite many reviews and many series versions, they had to resort to use multiple nodes for each VDO. I think that, after this series, we could also clean that mess up (sorry for the strong words) and make it right - assigning the MMIO for all VDOSYS blocks to one node, and adding the multiple ports - however, that will require a bit of work that is simply too much for this series alone. Summarizing, so that you don't have to carefully proof-read all this wall of text: - MediaTek SoCs have got multiple `port` for MMSYS and VDOSYS - Currently the driver implementation doesn't allow that - MediaTek had to work around no OF graph support! - Multiple ports are the multiple MMSYS/VDOSYS - One MMSYS / One VDOSYS have multiple `endpoints` That's how the HW is. Hope that's clear now? Cheers, Angelo >> + properties: >> + endpoint@0: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the primary display pipeline >> + >> + endpoint@1: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the secondary display pipeline >> + >> + endpoint@2: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the tertiary display pipeline >> + >> + required: >> + - endpoint@0 >> + >> required: >> - compatible >> - reg >> -- >> 2.44.0 >>
Hi, Angelo: On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno wrote: > Document OF graph on MMSYS/VDOSYS: this supports up to three DDP > paths > per HW instance (so potentially up to six displays for multi-vdo > SoCs). > > The MMSYS or VDOSYS is always the first component in the DDP > pipeline, > so it only supports an output port with multiple endpoints - where > each > endpoint defines the starting point for one of the (currently three) > possible hardware paths. > > Signed-off-by: AngeloGioacchino Del Regno < > angelogioacchino.delregno@collabora.com> > --- > .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 > +++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > index b3c6888c1457..4e9acd966aa5 100644 > --- > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > +++ > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml > @@ -93,6 +93,29 @@ properties: > '#reset-cells': > const: 1 > > + port: > + $ref: /schemas/graph.yaml#/properties/port > + description: > + Output port node. This port connects the MMSYS/VDOSYS output > to > + the first component of one display pipeline, for example one > of > + the available OVL or RDMA blocks. > + Some MediaTek SoCs support up to three display outputs per > MMSYS. > + properties: > + endpoint@0: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the primary display pipeline > + > + endpoint@1: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the secondary display pipeline > + > + endpoint@2: > + $ref: /schemas/graph.yaml#/properties/endpoint > + description: Output to the tertiary display pipeline > + > + required: > + - endpoint@0 > + mmsys/vdosys does not output data to the first component of display pipeline, so this connection looks 'virtual'. Shall we add something virtual in device tree? You add this in order to decide which pipeline is 1st, 2nd, 3rd, but for device it don't care which one is first. In computer, software could change which display is the primary display. I'm not sure it's good to decide display order in device tree? Regards, CK > required: > - compatible > - reg
Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > Hi, Angelo: > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno wrote: >> Document OF graph on MMSYS/VDOSYS: this supports up to three DDP >> paths >> per HW instance (so potentially up to six displays for multi-vdo >> SoCs). >> >> The MMSYS or VDOSYS is always the first component in the DDP >> pipeline, >> so it only supports an output port with multiple endpoints - where >> each >> endpoint defines the starting point for one of the (currently three) >> possible hardware paths. >> >> Signed-off-by: AngeloGioacchino Del Regno < >> angelogioacchino.delregno@collabora.com> >> --- >> .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 >> +++++++++++++++++++ >> 1 file changed, 23 insertions(+) >> >> diff --git >> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> index b3c6888c1457..4e9acd966aa5 100644 >> --- >> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> +++ >> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml >> @@ -93,6 +93,29 @@ properties: >> '#reset-cells': >> const: 1 >> >> + port: >> + $ref: /schemas/graph.yaml#/properties/port >> + description: >> + Output port node. This port connects the MMSYS/VDOSYS output >> to >> + the first component of one display pipeline, for example one >> of >> + the available OVL or RDMA blocks. >> + Some MediaTek SoCs support up to three display outputs per >> MMSYS. >> + properties: >> + endpoint@0: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the primary display pipeline >> + >> + endpoint@1: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the secondary display pipeline >> + >> + endpoint@2: >> + $ref: /schemas/graph.yaml#/properties/endpoint >> + description: Output to the tertiary display pipeline >> + >> + required: >> + - endpoint@0 >> + > > mmsys/vdosys does not output data to the first component of display > pipeline, so this connection looks 'virtual'. Shall we add something > virtual in device tree? You add this in order to decide which pipeline > is 1st, 2nd, 3rd, but for device it don't care which one is first. In > computer, software could change which display is the primary display. > I'm not sure it's good to decide display order in device tree? > Devicetree describes hardware, so nothing virtual can be present - and in any case, the primary/secondary/tertiary pipeline is in relation to MM/VDO SYS, not referred to software. Better explaining, the primary pipeline is not necessarily the primary display in DRM terms: that's a concept that is completely detached from the scope of this series and this graph - and it's something that shall be managed solely by the driver (mediatek-drm in this case). Coming back to the connection looking, but *not* being virtual: the sense here is that the MM/VDOSYS blocks are used in the display pipeline to "stitch" together the various display pipeline hardware blocks, or, said differently, setting up the routing between all of those (P.S.: mmsys_mtxxxx_routing_table!) through the VDO Input Selection (VDOx_SEL_IN) or Output Selection (VDOx_SEL_OUT) and with the assistance of the VDO Multiple Output Mask (VDOx_MOUT) for the multiple outputs usecase, both of which, are described by this graph. This means that the VDOSYS is really the "master" of the display pipeline since everything gets enabled, mixed and matched from there - and that's in the sense of hardware operation, so we are *really* (and not virtually!) flipping switches. Cheers, Angelo > Regards, > CK > > >> required: >> - compatible >> - reg
On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno wrote: > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > Hi, Angelo: > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno > > wrote: > > > Document OF graph on MMSYS/VDOSYS: this supports up to three DDP > > > paths > > > per HW instance (so potentially up to six displays for multi-vdo > > > SoCs). > > > > > > The MMSYS or VDOSYS is always the first component in the DDP > > > pipeline, > > > so it only supports an output port with multiple endpoints - > > > where > > > each > > > endpoint defines the starting point for one of the (currently > > > three) > > > possible hardware paths. > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > angelogioacchino.delregno@collabora.com> > > > --- > > > .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 > > > +++++++++++++++++++ > > > 1 file changed, 23 insertions(+) > > > > > > diff --git > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y > > > aml > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y > > > aml > > > index b3c6888c1457..4e9acd966aa5 100644 > > > --- > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y > > > aml > > > +++ > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y > > > aml > > > @@ -93,6 +93,29 @@ properties: > > > '#reset-cells': > > > const: 1 > > > > > > + port: > > > + $ref: /schemas/graph.yaml#/properties/port > > > + description: > > > + Output port node. This port connects the MMSYS/VDOSYS > > > output > > > to > > > + the first component of one display pipeline, for example > > > one > > > of > > > + the available OVL or RDMA blocks. > > > + Some MediaTek SoCs support up to three display outputs per > > > MMSYS. > > > + properties: > > > + endpoint@0: > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > + description: Output to the primary display pipeline > > > + > > > + endpoint@1: > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > + description: Output to the secondary display pipeline > > > + > > > + endpoint@2: > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > + description: Output to the tertiary display pipeline > > > + > > > + required: > > > + - endpoint@0 > > > + > > > > mmsys/vdosys does not output data to the first component of display > > pipeline, so this connection looks 'virtual'. Shall we add > > something > > virtual in device tree? You add this in order to decide which > > pipeline > > is 1st, 2nd, 3rd, but for device it don't care which one is first. > > In > > computer, software could change which display is the primary > > display. > > I'm not sure it's good to decide display order in device tree? > > > > Devicetree describes hardware, so nothing virtual can be present - > and in any case, > the primary/secondary/tertiary pipeline is in relation to MM/VDO SYS, > not referred > to software. > > Better explaining, the primary pipeline is not necessarily the > primary display in > DRM terms: that's a concept that is completely detached from the > scope of this > series and this graph - and it's something that shall be managed > solely by the > driver (mediatek-drm in this case). > > Coming back to the connection looking, but *not* being virtual: the > sense here is > that the MM/VDOSYS blocks are used in the display pipeline to > "stitch" together > the various display pipeline hardware blocks, or, said differently, > setting up the > routing between all of those (P.S.: mmsys_mtxxxx_routing_table!) > through the VDO > Input Selection (VDOx_SEL_IN) or Output Selection (VDOx_SEL_OUT) and > with the > assistance of the VDO Multiple Output Mask (VDOx_MOUT) for the > multiple outputs > usecase, both of which, are described by this graph. I agree this part, but this is related to display device OF graph. These display device would output video data from one device and input to another video device. These video device would not input or output video data to mmsys/vdosys. > > This means that the VDOSYS is really the "master" of the display > pipeline since > everything gets enabled, mixed and matched from there - and that's in > the sense > of hardware operation, so we are *really* (and not virtually!) > flipping switches. I agree mmsys/vdosys is master of video pipeline, so let's define what the port in mmsys/vdosys is. If the port means the master relationship, mmsys/vdosys should output port to every display device. Or use a simply way to show the master relation ship mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, &color1, ...>; Another problem is how to group display device? If two pipeline could be route to the same display interface, such as rdma0 -> dsi rdma1 -> dsi Would this be single group? mmsys-subdev = <&rdma0, &rdma1, &dsi>; Or two group? mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; I think we should clearly define this. Regards, CK > > > Cheers, > Angelo > > > Regards, > > CK > > > > > > > required: > > > - compatible > > > - reg > >
Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno wrote: >> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>> Hi, Angelo: >>> >>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Document OF graph on MMSYS/VDOSYS: this supports up to three DDP >>>> paths >>>> per HW instance (so potentially up to six displays for multi-vdo >>>> SoCs). >>>> >>>> The MMSYS or VDOSYS is always the first component in the DDP >>>> pipeline, >>>> so it only supports an output port with multiple endpoints - >>>> where >>>> each >>>> endpoint defines the starting point for one of the (currently >>>> three) >>>> possible hardware paths. >>>> >>>> Signed-off-by: AngeloGioacchino Del Regno < >>>> angelogioacchino.delregno@collabora.com> >>>> --- >>>> .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 >>>> +++++++++++++++++++ >>>> 1 file changed, 23 insertions(+) >>>> >>>> diff --git >>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y >>>> aml >>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y >>>> aml >>>> index b3c6888c1457..4e9acd966aa5 100644 >>>> --- >>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y >>>> aml >>>> +++ >>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.y >>>> aml >>>> @@ -93,6 +93,29 @@ properties: >>>> '#reset-cells': >>>> const: 1 >>>> >>>> + port: >>>> + $ref: /schemas/graph.yaml#/properties/port >>>> + description: >>>> + Output port node. This port connects the MMSYS/VDOSYS >>>> output >>>> to >>>> + the first component of one display pipeline, for example >>>> one >>>> of >>>> + the available OVL or RDMA blocks. >>>> + Some MediaTek SoCs support up to three display outputs per >>>> MMSYS. >>>> + properties: >>>> + endpoint@0: >>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>> + description: Output to the primary display pipeline >>>> + >>>> + endpoint@1: >>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>> + description: Output to the secondary display pipeline >>>> + >>>> + endpoint@2: >>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>> + description: Output to the tertiary display pipeline >>>> + >>>> + required: >>>> + - endpoint@0 >>>> + >>> >>> mmsys/vdosys does not output data to the first component of display >>> pipeline, so this connection looks 'virtual'. Shall we add >>> something >>> virtual in device tree? You add this in order to decide which >>> pipeline >>> is 1st, 2nd, 3rd, but for device it don't care which one is first. >>> In >>> computer, software could change which display is the primary >>> display. >>> I'm not sure it's good to decide display order in device tree? >>> >> >> Devicetree describes hardware, so nothing virtual can be present - >> and in any case, >> the primary/secondary/tertiary pipeline is in relation to MM/VDO SYS, >> not referred >> to software. >> >> Better explaining, the primary pipeline is not necessarily the >> primary display in >> DRM terms: that's a concept that is completely detached from the >> scope of this >> series and this graph - and it's something that shall be managed >> solely by the >> driver (mediatek-drm in this case). >> >> Coming back to the connection looking, but *not* being virtual: the >> sense here is >> that the MM/VDOSYS blocks are used in the display pipeline to >> "stitch" together >> the various display pipeline hardware blocks, or, said differently, >> setting up the >> routing between all of those (P.S.: mmsys_mtxxxx_routing_table!) >> through the VDO >> Input Selection (VDOx_SEL_IN) or Output Selection (VDOx_SEL_OUT) and >> with the >> assistance of the VDO Multiple Output Mask (VDOx_MOUT) for the >> multiple outputs >> usecase, both of which, are described by this graph. > > I agree this part, but this is related to display device OF graph. > These display device would output video data from one device and input > to another video device. These video device would not input or output > video data to mmsys/vdosys. > >> >> This means that the VDOSYS is really the "master" of the display >> pipeline since >> everything gets enabled, mixed and matched from there - and that's in >> the sense >> of hardware operation, so we are *really* (and not virtually!) >> flipping switches. > > I agree mmsys/vdosys is master of video pipeline, so let's define what > the port in mmsys/vdosys is. If the port means the master relationship, > mmsys/vdosys should output port to every display device. Or use a > simply way to show the master relation ship > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, &color1, > ...>; > There's no need to list all of the VDO0/VDO1/mmsys devices in one big array property, because the actual possible devices can be defined: 1. In the bindings; and 2. In the actual OF graph that we write for each SoC+board combination. A graph cannot contain a connection to a device that cannot be connected to the previous, so, your "mmsys-subdev" list can be retrieved by looking at the graph: - Start from VDO0/1 or MMSYS - Walk through (visually, even) OUTPUT ports - VDO0 (read output ep) -> ovl0 (read output ep) -> rdma0 (read output ep) -> color0 (...) -> etc - Nothing more - it's all defined there. > > Another problem is how to group display device? If two pipeline could > be route to the same display interface, such as > > rdma0 -> dsi > rdma1 -> dsi > > Would this be single group? There are multiple ways of doing this, but one that comes to my mind right now and that looks clean as well is the following: ovl0@ef01 { ..... ports { port@0 { reg = <0>; ovl0_in: endpoint { remote-endpoint = <&vdosys0_out>; }; }; port@1 { reg = <1>; ovl0_out0: endpoint@0 { remote-endpoint = <&rdma0_in>; }; ovl0_out1: endpoint@1 { remote-endpoint = <&rdma1_in>; }; }; }; }; rdma0@1234 { ..... ports { port@0 { reg = <0>; rdma0_in: endpoint { remote-endpoint = <&ovl0_out0>; /* assuming ovl0 outputs to rdma0...*/ }; }; port@1 { reg = <1>; rdma0_out: endpoint@1 { remote-endpoint = <&dsi_dual_intf0_in>; }; }; }; }; rdma1@5678 { ..... ports { port@0 { reg = <0>; rdma1_in: endpoint { /* assuming ovl0 outputs to rdma1 as well... can be something else. */ remote-endpoint = <&ovl0_out1>; }; }; port@1 { reg = <1>; rdma1_out: endpoint { remote-endpoint = <&dsi_dual_intf1_in>; }; }; }; }; dsi@9abcd { ..... ports { port@0 { reg = <0>; /* Where endpoint@0 could be always DSI LEFT CTRL */ dsi_dual_intf0_in: endpoint@0 { remote-endpoint = <&rdma0_out>; }; /* ...and @1 could be always DSI RIGHT CTRL */ dsi_dual_intf1_in: endpoint@1 { remote-endpoint = <&rdma1_out>; }; }; port@1 { reg = <1>; dsi0_out: endpoint { remote-endpoint = <&dsi_panel_in>; }; }; }; }; ...for a dual-dsi panel, it'd be a similar graph. Cheers, Angelo > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > Or two group? > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > I think we should clearly define this. > > Regards, > CK > >> >> >> Cheers, >> Angelo >> >>> Regards, >>> CK >>> >>> >>>> required: >>>> - compatible >>>> - reg >> >>
On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno wrote: > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno > > wrote: > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > Hi, Angelo: > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Document OF graph on MMSYS/VDOSYS: this supports up to three > > > > > DDP > > > > > paths > > > > > per HW instance (so potentially up to six displays for multi- > > > > > vdo > > > > > SoCs). > > > > > > > > > > The MMSYS or VDOSYS is always the first component in the DDP > > > > > pipeline, > > > > > so it only supports an output port with multiple endpoints - > > > > > where > > > > > each > > > > > endpoint defines the starting point for one of the (currently > > > > > three) > > > > > possible hardware paths. > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > angelogioacchino.delregno@collabora.com> > > > > > --- > > > > > .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 > > > > > +++++++++++++++++++ > > > > > 1 file changed, 23 insertions(+) > > > > > > > > > > diff --git > > > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms > > > > > ys.y > > > > > aml > > > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms > > > > > ys.y > > > > > aml > > > > > index b3c6888c1457..4e9acd966aa5 100644 > > > > > --- > > > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms > > > > > ys.y > > > > > aml > > > > > +++ > > > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms > > > > > ys.y > > > > > aml > > > > > @@ -93,6 +93,29 @@ properties: > > > > > '#reset-cells': > > > > > const: 1 > > > > > > > > > > + port: > > > > > + $ref: /schemas/graph.yaml#/properties/port > > > > > + description: > > > > > + Output port node. This port connects the MMSYS/VDOSYS > > > > > output > > > > > to > > > > > + the first component of one display pipeline, for > > > > > example > > > > > one > > > > > of > > > > > + the available OVL or RDMA blocks. > > > > > + Some MediaTek SoCs support up to three display outputs > > > > > per > > > > > MMSYS. > > > > > + properties: > > > > > + endpoint@0: > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > + description: Output to the primary display pipeline > > > > > + > > > > > + endpoint@1: > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > + description: Output to the secondary display > > > > > pipeline > > > > > + > > > > > + endpoint@2: > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > + description: Output to the tertiary display pipeline > > > > > + > > > > > + required: > > > > > + - endpoint@0 > > > > > + > > > > > > > > mmsys/vdosys does not output data to the first component of > > > > display > > > > pipeline, so this connection looks 'virtual'. Shall we add > > > > something > > > > virtual in device tree? You add this in order to decide which > > > > pipeline > > > > is 1st, 2nd, 3rd, but for device it don't care which one is > > > > first. > > > > In > > > > computer, software could change which display is the primary > > > > display. > > > > I'm not sure it's good to decide display order in device tree? > > > > > > > > > > Devicetree describes hardware, so nothing virtual can be present > > > - > > > and in any case, > > > the primary/secondary/tertiary pipeline is in relation to MM/VDO > > > SYS, > > > not referred > > > to software. > > > > > > Better explaining, the primary pipeline is not necessarily the > > > primary display in > > > DRM terms: that's a concept that is completely detached from the > > > scope of this > > > series and this graph - and it's something that shall be managed > > > solely by the > > > driver (mediatek-drm in this case). > > > > > > Coming back to the connection looking, but *not* being virtual: > > > the > > > sense here is > > > that the MM/VDOSYS blocks are used in the display pipeline to > > > "stitch" together > > > the various display pipeline hardware blocks, or, said > > > differently, > > > setting up the > > > routing between all of those (P.S.: mmsys_mtxxxx_routing_table!) > > > through the VDO > > > Input Selection (VDOx_SEL_IN) or Output Selection (VDOx_SEL_OUT) > > > and > > > with the > > > assistance of the VDO Multiple Output Mask (VDOx_MOUT) for the > > > multiple outputs > > > usecase, both of which, are described by this graph. > > > > I agree this part, but this is related to display device OF graph. > > These display device would output video data from one device and > > input > > to another video device. These video device would not input or > > output > > video data to mmsys/vdosys. > > > > > > > > This means that the VDOSYS is really the "master" of the display > > > pipeline since > > > everything gets enabled, mixed and matched from there - and > > > that's in > > > the sense > > > of hardware operation, so we are *really* (and not virtually!) > > > flipping switches. > > > > I agree mmsys/vdosys is master of video pipeline, so let's define > > what > > the port in mmsys/vdosys is. If the port means the master > > relationship, > > mmsys/vdosys should output port to every display device. Or use a > > simply way to show the master relation ship > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, > > &color1, > > ...>; > > > > There's no need to list all of the VDO0/VDO1/mmsys devices in one big > array > property, because the actual possible devices can be defined: > 1. In the bindings; and > 2. In the actual OF graph that we write for each SoC+board > combination. > > A graph cannot contain a connection to a device that cannot be > connected to > the previous, so, your "mmsys-subdev" list can be retrieved by > looking at the > graph: > - Start from VDO0/1 or MMSYS > - Walk through (visually, even) OUTPUT ports > - VDO0 (read output ep) -> ovl0 (read output ep) -> rdma0 (read > output ep) -> > color0 (...) -> etc > - Nothing more - it's all defined there. > > > > > Another problem is how to group display device? If two pipeline > > could > > be route to the same display interface, such as > > > > rdma0 -> dsi > > rdma1 -> dsi > > > > Would this be single group? > > There are multiple ways of doing this, but one that comes to my mind > right now and > that looks clean as well is the following: > > ovl0@ef01 { > ..... > ports { > port@0 { > reg = <0>; > ovl0_in: endpoint { > remote-endpoint = <&vdosys0_out>; > }; > }; I'm not sure how do you define this port from OVL to vdosys. If this port means 'master relationship', others could add port in COLOR to point to vdosys because COLOR and vdosys has the 'master relationship' and I could not reject this. So we need more specific definition of this port. Only the 'first' device in pipeline could have this port? In mt8173, one pipeline is ovl -> color -> aal -> od -> rdma -> ufo -> dsi But rdma has an option to read data from od or directly from DRAM. If from DRAM, the pipeline would be changed to rdma -> ufo -> dsi So it's confused which one is 'first'. I don't know how to decide which device could point to mmsys/vdosys. So please give a specific definition. Regards, CK > > port@1 { > reg = <1>; > ovl0_out0: endpoint@0 { > remote-endpoint = <&rdma0_in>; > }; > ovl0_out1: endpoint@1 { > remote-endpoint = <&rdma1_in>; > }; > }; > }; > }; > > rdma0@1234 { > ..... > ports { > port@0 { > reg = <0>; > rdma0_in: endpoint { > remote-endpoint = <&ovl0_out0>; /* assuming ovl0 outputs to > rdma0...*/ > }; > }; > port@1 { > reg = <1>; > rdma0_out: endpoint@1 { > remote-endpoint = <&dsi_dual_intf0_in>; > }; > }; > }; > }; > > > rdma1@5678 { > ..... > ports { > port@0 { > reg = <0>; > rdma1_in: endpoint { > /* assuming ovl0 outputs to rdma1 as well... can be > something else. */ > remote-endpoint = <&ovl0_out1>; > }; > }; > port@1 { > reg = <1>; > rdma1_out: endpoint { > remote-endpoint = <&dsi_dual_intf1_in>; > }; > }; > }; > }; > > > dsi@9abcd { > ..... > ports { > port@0 { > reg = <0>; > /* Where endpoint@0 could be always DSI LEFT CTRL */ > dsi_dual_intf0_in: endpoint@0 { > remote-endpoint = <&rdma0_out>; > }; > /* ...and @1 could be always DSI RIGHT CTRL */ > dsi_dual_intf1_in: endpoint@1 { > remote-endpoint = <&rdma1_out>; > }; > }; > > port@1 { > reg = <1>; > dsi0_out: endpoint { > remote-endpoint = <&dsi_panel_in>; > }; > }; > }; > }; > > ...for a dual-dsi panel, it'd be a similar graph. > > Cheers, > Angelo > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > > > Or two group? > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > > > I think we should clearly define this. > > > > Regards, > > CK > > > > > > > > > > > Cheers, > > > Angelo > > > > > > > Regards, > > > > CK > > > > > > > > > > > > > required: > > > > > - compatible > > > > > - reg > > > > > > > > >
Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno wrote: >> Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: >>> On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>>>> Hi, Angelo: >>>>> >>>>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del Regno >>>>> wrote: >>>>>> Document OF graph on MMSYS/VDOSYS: this supports up to three >>>>>> DDP >>>>>> paths >>>>>> per HW instance (so potentially up to six displays for multi- >>>>>> vdo >>>>>> SoCs). >>>>>> >>>>>> The MMSYS or VDOSYS is always the first component in the DDP >>>>>> pipeline, >>>>>> so it only supports an output port with multiple endpoints - >>>>>> where >>>>>> each >>>>>> endpoint defines the starting point for one of the (currently >>>>>> three) >>>>>> possible hardware paths. >>>>>> >>>>>> Signed-off-by: AngeloGioacchino Del Regno < >>>>>> angelogioacchino.delregno@collabora.com> >>>>>> --- >>>>>> .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 >>>>>> +++++++++++++++++++ >>>>>> 1 file changed, 23 insertions(+) >>>>>> >>>>>> diff --git >>>>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms >>>>>> ys.y >>>>>> aml >>>>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms >>>>>> ys.y >>>>>> aml >>>>>> index b3c6888c1457..4e9acd966aa5 100644 >>>>>> --- >>>>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms >>>>>> ys.y >>>>>> aml >>>>>> +++ >>>>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mms >>>>>> ys.y >>>>>> aml >>>>>> @@ -93,6 +93,29 @@ properties: >>>>>> '#reset-cells': >>>>>> const: 1 >>>>>> >>>>>> + port: >>>>>> + $ref: /schemas/graph.yaml#/properties/port >>>>>> + description: >>>>>> + Output port node. This port connects the MMSYS/VDOSYS >>>>>> output >>>>>> to >>>>>> + the first component of one display pipeline, for >>>>>> example >>>>>> one >>>>>> of >>>>>> + the available OVL or RDMA blocks. >>>>>> + Some MediaTek SoCs support up to three display outputs >>>>>> per >>>>>> MMSYS. >>>>>> + properties: >>>>>> + endpoint@0: >>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>> + description: Output to the primary display pipeline >>>>>> + >>>>>> + endpoint@1: >>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>> + description: Output to the secondary display >>>>>> pipeline >>>>>> + >>>>>> + endpoint@2: >>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>> + description: Output to the tertiary display pipeline >>>>>> + >>>>>> + required: >>>>>> + - endpoint@0 >>>>>> + >>>>> >>>>> mmsys/vdosys does not output data to the first component of >>>>> display >>>>> pipeline, so this connection looks 'virtual'. Shall we add >>>>> something >>>>> virtual in device tree? You add this in order to decide which >>>>> pipeline >>>>> is 1st, 2nd, 3rd, but for device it don't care which one is >>>>> first. >>>>> In >>>>> computer, software could change which display is the primary >>>>> display. >>>>> I'm not sure it's good to decide display order in device tree? >>>>> >>>> >>>> Devicetree describes hardware, so nothing virtual can be present >>>> - >>>> and in any case, >>>> the primary/secondary/tertiary pipeline is in relation to MM/VDO >>>> SYS, >>>> not referred >>>> to software. >>>> >>>> Better explaining, the primary pipeline is not necessarily the >>>> primary display in >>>> DRM terms: that's a concept that is completely detached from the >>>> scope of this >>>> series and this graph - and it's something that shall be managed >>>> solely by the >>>> driver (mediatek-drm in this case). >>>> >>>> Coming back to the connection looking, but *not* being virtual: >>>> the >>>> sense here is >>>> that the MM/VDOSYS blocks are used in the display pipeline to >>>> "stitch" together >>>> the various display pipeline hardware blocks, or, said >>>> differently, >>>> setting up the >>>> routing between all of those (P.S.: mmsys_mtxxxx_routing_table!) >>>> through the VDO >>>> Input Selection (VDOx_SEL_IN) or Output Selection (VDOx_SEL_OUT) >>>> and >>>> with the >>>> assistance of the VDO Multiple Output Mask (VDOx_MOUT) for the >>>> multiple outputs >>>> usecase, both of which, are described by this graph. >>> >>> I agree this part, but this is related to display device OF graph. >>> These display device would output video data from one device and >>> input >>> to another video device. These video device would not input or >>> output >>> video data to mmsys/vdosys. >>> >>>> >>>> This means that the VDOSYS is really the "master" of the display >>>> pipeline since >>>> everything gets enabled, mixed and matched from there - and >>>> that's in >>>> the sense >>>> of hardware operation, so we are *really* (and not virtually!) >>>> flipping switches. >>> >>> I agree mmsys/vdosys is master of video pipeline, so let's define >>> what >>> the port in mmsys/vdosys is. If the port means the master >>> relationship, >>> mmsys/vdosys should output port to every display device. Or use a >>> simply way to show the master relation ship >>> >>> mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, >>> &color1, >>> ...>; >>> >> >> There's no need to list all of the VDO0/VDO1/mmsys devices in one big >> array >> property, because the actual possible devices can be defined: >> 1. In the bindings; and >> 2. In the actual OF graph that we write for each SoC+board >> combination. >> >> A graph cannot contain a connection to a device that cannot be >> connected to >> the previous, so, your "mmsys-subdev" list can be retrieved by >> looking at the >> graph: >> - Start from VDO0/1 or MMSYS >> - Walk through (visually, even) OUTPUT ports >> - VDO0 (read output ep) -> ovl0 (read output ep) -> rdma0 (read >> output ep) -> >> color0 (...) -> etc >> - Nothing more - it's all defined there. >> >>> >>> Another problem is how to group display device? If two pipeline >>> could >>> be route to the same display interface, such as >>> >>> rdma0 -> dsi >>> rdma1 -> dsi >>> >>> Would this be single group? >> >> There are multiple ways of doing this, but one that comes to my mind >> right now and >> that looks clean as well is the following: >> >> ovl0@ef01 { >> ..... >> ports { >> port@0 { >> reg = <0>; >> ovl0_in: endpoint { >> remote-endpoint = <&vdosys0_out>; >> }; >> }; > > I'm not sure how do you define this port from OVL to vdosys. If this > port means 'master relationship', others could add port in COLOR to > point to vdosys because COLOR and vdosys has the 'master relationship' > and I could not reject this. So we need more specific definition of > this port. > Only the 'first' device in pipeline could have this port? Correct. Only the first device in a pipeline - and this is actually a restriction that the generic binding definition of port already gives, in a way. > In mt8173, one pipeline is > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi > > But rdma has an option to read data from od or directly from DRAM. If > from DRAM, the pipeline would be changed to > > rdma -> ufo -> dsi > > > So it's confused which one is 'first'. That's why the pipeline is *board-specific* and not soc-generic! And what you described is *exactly* the reason why I'm adding support for the OF graphs in mediatek-drm: specifying the correct pipeline for each board as per what each board wants to use (said differently: for each board's *capabilities*). So, if on a certain board you want to skip OD, you can hook RDMA up directly to MMSYS/VDOSYS. In MT8173, one pipeline for one board uses endpoints IN/OUT like this: MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI and for another board, endpoints will be like MMSYS -> RDMA -> UFO -> DSI ...which is the exact same as you described, and I think that your confusion comes from the fact that you didn't put MMSYS at the beginning of the pipeline :-) In case you need any *temporary override* on any board that defines a pipeline like MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI so that the pipeline *temporarily* becomes (for power management, or for any other reason) RDMA -> UFO -> DSI .... that's not a concern: the graph is present, and it is used to tell to the driver what is the regular pipeline to use. Eventual temporary overrides can be managed transparently inside of the driver with C code and no changes to the devicetree are required. > I don't know how to decide which device could point to mmsys/vdosys. So > please give a specific definition. > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a device. So, mmsys/vdosys must point to the *first device in the pipeline*. Any other doubt? Cheers, Angelo > Regards, > CK > >> >> port@1 { >> reg = <1>; >> ovl0_out0: endpoint@0 { >> remote-endpoint = <&rdma0_in>; >> }; >> ovl0_out1: endpoint@1 { >> remote-endpoint = <&rdma1_in>; >> }; >> }; >> }; >> }; >> >> rdma0@1234 { >> ..... >> ports { >> port@0 { >> reg = <0>; >> rdma0_in: endpoint { >> remote-endpoint = <&ovl0_out0>; /* assuming ovl0 outputs to >> rdma0...*/ >> }; >> }; >> port@1 { >> reg = <1>; >> rdma0_out: endpoint@1 { >> remote-endpoint = <&dsi_dual_intf0_in>; >> }; >> }; >> }; >> }; >> >> >> rdma1@5678 { >> ..... >> ports { >> port@0 { >> reg = <0>; >> rdma1_in: endpoint { >> /* assuming ovl0 outputs to rdma1 as well... can be >> something else. */ >> remote-endpoint = <&ovl0_out1>; >> }; >> }; >> port@1 { >> reg = <1>; >> rdma1_out: endpoint { >> remote-endpoint = <&dsi_dual_intf1_in>; >> }; >> }; >> }; >> }; >> >> >> dsi@9abcd { >> ..... >> ports { >> port@0 { >> reg = <0>; >> /* Where endpoint@0 could be always DSI LEFT CTRL */ >> dsi_dual_intf0_in: endpoint@0 { >> remote-endpoint = <&rdma0_out>; >> }; >> /* ...and @1 could be always DSI RIGHT CTRL */ >> dsi_dual_intf1_in: endpoint@1 { >> remote-endpoint = <&rdma1_out>; >> }; >> }; >> >> port@1 { >> reg = <1>; >> dsi0_out: endpoint { >> remote-endpoint = <&dsi_panel_in>; >> }; >> }; >> }; >> }; >> >> ...for a dual-dsi panel, it'd be a similar graph. >> >> Cheers, >> Angelo >> >>> >>> mmsys-subdev = <&rdma0, &rdma1, &dsi>; >>> >>> Or two group? >>> >>> mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; >>> >>> I think we should clearly define this. >>> >>> Regards, >>> CK >>> >>>> >>>> >>>> Cheers, >>>> Angelo >>>> >>>>> Regards, >>>>> CK >>>>> >>>>> >>>>>> required: >>>>>> - compatible >>>>>> - reg >>>> >>>> >> >> >>
On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno wrote: > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno > > wrote: > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > > > Hi, Angelo: > > > > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del > > > > > > Regno > > > > > > wrote: > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports up to > > > > > > > three > > > > > > > DDP > > > > > > > paths > > > > > > > per HW instance (so potentially up to six displays for > > > > > > > multi- > > > > > > > vdo > > > > > > > SoCs). > > > > > > > > > > > > > > The MMSYS or VDOSYS is always the first component in the > > > > > > > DDP > > > > > > > pipeline, > > > > > > > so it only supports an output port with multiple > > > > > > > endpoints - > > > > > > > where > > > > > > > each > > > > > > > endpoint defines the starting point for one of the > > > > > > > (currently > > > > > > > three) > > > > > > > possible hardware paths. > > > > > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > > > angelogioacchino.delregno@collabora.com> > > > > > > > --- > > > > > > > .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 > > > > > > > +++++++++++++++++++ > > > > > > > 1 file changed, 23 insertions(+) > > > > > > > > > > > > > > diff --git > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek > > > > > > > ,mms > > > > > > > ys.y > > > > > > > aml > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek > > > > > > > ,mms > > > > > > > ys.y > > > > > > > aml > > > > > > > index b3c6888c1457..4e9acd966aa5 100644 > > > > > > > --- > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/mediatek > > > > > > > ,mms > > > > > > > ys.y > > > > > > > aml > > > > > > > +++ > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/mediatek > > > > > > > ,mms > > > > > > > ys.y > > > > > > > aml > > > > > > > @@ -93,6 +93,29 @@ properties: > > > > > > > '#reset-cells': > > > > > > > const: 1 > > > > > > > > > > > > > > + port: > > > > > > > + $ref: /schemas/graph.yaml#/properties/port > > > > > > > + description: > > > > > > > + Output port node. This port connects the > > > > > > > MMSYS/VDOSYS > > > > > > > output > > > > > > > to > > > > > > > + the first component of one display pipeline, for > > > > > > > example > > > > > > > one > > > > > > > of > > > > > > > + the available OVL or RDMA blocks. > > > > > > > + Some MediaTek SoCs support up to three display > > > > > > > outputs > > > > > > > per > > > > > > > MMSYS. > > > > > > > + properties: > > > > > > > + endpoint@0: > > > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > > > + description: Output to the primary display > > > > > > > pipeline > > > > > > > + > > > > > > > + endpoint@1: > > > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > > > + description: Output to the secondary display > > > > > > > pipeline > > > > > > > + > > > > > > > + endpoint@2: > > > > > > > + $ref: /schemas/graph.yaml#/properties/endpoint > > > > > > > + description: Output to the tertiary display > > > > > > > pipeline > > > > > > > + > > > > > > > + required: > > > > > > > + - endpoint@0 > > > > > > > + > > > > > > > > > > > > mmsys/vdosys does not output data to the first component of > > > > > > display > > > > > > pipeline, so this connection looks 'virtual'. Shall we add > > > > > > something > > > > > > virtual in device tree? You add this in order to decide > > > > > > which > > > > > > pipeline > > > > > > is 1st, 2nd, 3rd, but for device it don't care which one is > > > > > > first. > > > > > > In > > > > > > computer, software could change which display is the > > > > > > primary > > > > > > display. > > > > > > I'm not sure it's good to decide display order in device > > > > > > tree? > > > > > > > > > > > > > > > > Devicetree describes hardware, so nothing virtual can be > > > > > present > > > > > - > > > > > and in any case, > > > > > the primary/secondary/tertiary pipeline is in relation to > > > > > MM/VDO > > > > > SYS, > > > > > not referred > > > > > to software. > > > > > > > > > > Better explaining, the primary pipeline is not necessarily > > > > > the > > > > > primary display in > > > > > DRM terms: that's a concept that is completely detached from > > > > > the > > > > > scope of this > > > > > series and this graph - and it's something that shall be > > > > > managed > > > > > solely by the > > > > > driver (mediatek-drm in this case). > > > > > > > > > > Coming back to the connection looking, but *not* being > > > > > virtual: > > > > > the > > > > > sense here is > > > > > that the MM/VDOSYS blocks are used in the display pipeline to > > > > > "stitch" together > > > > > the various display pipeline hardware blocks, or, said > > > > > differently, > > > > > setting up the > > > > > routing between all of those (P.S.: > > > > > mmsys_mtxxxx_routing_table!) > > > > > through the VDO > > > > > Input Selection (VDOx_SEL_IN) or Output Selection > > > > > (VDOx_SEL_OUT) > > > > > and > > > > > with the > > > > > assistance of the VDO Multiple Output Mask (VDOx_MOUT) for > > > > > the > > > > > multiple outputs > > > > > usecase, both of which, are described by this graph. > > > > > > > > I agree this part, but this is related to display device OF > > > > graph. > > > > These display device would output video data from one device > > > > and > > > > input > > > > to another video device. These video device would not input or > > > > output > > > > video data to mmsys/vdosys. > > > > > > > > > > > > > > This means that the VDOSYS is really the "master" of the > > > > > display > > > > > pipeline since > > > > > everything gets enabled, mixed and matched from there - and > > > > > that's in > > > > > the sense > > > > > of hardware operation, so we are *really* (and not > > > > > virtually!) > > > > > flipping switches. > > > > > > > > I agree mmsys/vdosys is master of video pipeline, so let's > > > > define > > > > what > > > > the port in mmsys/vdosys is. If the port means the master > > > > relationship, > > > > mmsys/vdosys should output port to every display device. Or use > > > > a > > > > simply way to show the master relation ship > > > > > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, > > > > &color1, > > > > ...>; > > > > > > > > > > There's no need to list all of the VDO0/VDO1/mmsys devices in one > > > big > > > array > > > property, because the actual possible devices can be defined: > > > 1. In the bindings; and > > > 2. In the actual OF graph that we write for each SoC+board > > > combination. > > > > > > A graph cannot contain a connection to a device that cannot be > > > connected to > > > the previous, so, your "mmsys-subdev" list can be retrieved by > > > looking at the > > > graph: > > > - Start from VDO0/1 or MMSYS > > > - Walk through (visually, even) OUTPUT ports > > > - VDO0 (read output ep) -> ovl0 (read output ep) -> rdma0 > > > (read > > > output ep) -> > > > color0 (...) -> etc > > > - Nothing more - it's all defined there. > > > > > > > > > > > Another problem is how to group display device? If two pipeline > > > > could > > > > be route to the same display interface, such as > > > > > > > > rdma0 -> dsi > > > > rdma1 -> dsi > > > > > > > > Would this be single group? > > > > > > There are multiple ways of doing this, but one that comes to my > > > mind > > > right now and > > > that looks clean as well is the following: > > > > > > ovl0@ef01 { > > > ..... > > > ports { > > > port@0 { > > > reg = <0>; > > > ovl0_in: endpoint { > > > remote-endpoint = <&vdosys0_out>; > > > }; > > > }; > > > > I'm not sure how do you define this port from OVL to vdosys. If > > this > > port means 'master relationship', others could add port in COLOR to > > point to vdosys because COLOR and vdosys has the 'master > > relationship' > > and I could not reject this. So we need more specific definition of > > this port. > > > > Only the 'first' device in pipeline could have this port? > > Correct. Only the first device in a pipeline - and this is actually a > restriction > that the generic binding definition of port already gives, in a way. > > > > In mt8173, one pipeline is > > > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi > > > > But rdma has an option to read data from od or directly from DRAM. > > If > > from DRAM, the pipeline would be changed to > > > > rdma -> ufo -> dsi > > > > > > So it's confused which one is 'first'. > > That's why the pipeline is *board-specific* and not soc-generic! > > And what you described is *exactly* the reason why I'm adding support > for the > OF graphs in mediatek-drm: specifying the correct pipeline for each > board as per > what each board wants to use (said differently: for each board's > *capabilities*). > > So, if on a certain board you want to skip OD, you can hook RDMA up > directly to > MMSYS/VDOSYS. > > In MT8173, one pipeline for one board uses endpoints IN/OUT like > this: > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > and for another board, endpoints will be like > > MMSYS -> RDMA -> UFO -> DSI > > ...which is the exact same as you described, and I think that your > confusion comes > from the fact that you didn't put MMSYS at the beginning of the > pipeline :-) In one board, both OVL and RDMA could switch dynamically. Because each one could be the first in one board, mmsys point to both ovl and rdma? Regards, CK > > > > > In case you need any *temporary override* on any board that defines a > pipeline like > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > so that the pipeline *temporarily* becomes (for power management, or > for any other > reason) RDMA -> UFO -> DSI .... that's not a concern: the graph is > present, and it > is used to tell to the driver what is the regular pipeline to use. > Eventual temporary overrides can be managed transparently inside of > the driver with > C code and no changes to the devicetree are required. > > > > I don't know how to decide which device could point to > > mmsys/vdosys. So > > please give a specific definition. > > > > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a > device. > > So, mmsys/vdosys must point to the *first device in the pipeline*. > > Any other doubt? > > Cheers, > Angelo > > > Regards, > > CK > > > > > > > > port@1 { > > > reg = <1>; > > > ovl0_out0: endpoint@0 { > > > remote-endpoint = <&rdma0_in>; > > > }; > > > ovl0_out1: endpoint@1 { > > > remote-endpoint = <&rdma1_in>; > > > }; > > > }; > > > }; > > > }; > > > > > > rdma0@1234 { > > > ..... > > > ports { > > > port@0 { > > > reg = <0>; > > > rdma0_in: endpoint { > > > remote-endpoint = <&ovl0_out0>; /* assuming ovl0 > > > outputs to > > > rdma0...*/ > > > }; > > > }; > > > port@1 { > > > reg = <1>; > > > rdma0_out: endpoint@1 { > > > remote-endpoint = <&dsi_dual_intf0_in>; > > > }; > > > }; > > > }; > > > }; > > > > > > > > > rdma1@5678 { > > > ..... > > > ports { > > > port@0 { > > > reg = <0>; > > > rdma1_in: endpoint { > > > /* assuming ovl0 outputs to rdma1 as well... can be > > > something else. */ > > > remote-endpoint = <&ovl0_out1>; > > > }; > > > }; > > > port@1 { > > > reg = <1>; > > > rdma1_out: endpoint { > > > remote-endpoint = <&dsi_dual_intf1_in>; > > > }; > > > }; > > > }; > > > }; > > > > > > > > > dsi@9abcd { > > > ..... > > > ports { > > > port@0 { > > > reg = <0>; > > > /* Where endpoint@0 could be always DSI LEFT CTRL */ > > > dsi_dual_intf0_in: endpoint@0 { > > > remote-endpoint = <&rdma0_out>; > > > }; > > > /* ...and @1 could be always DSI RIGHT CTRL */ > > > dsi_dual_intf1_in: endpoint@1 { > > > remote-endpoint = <&rdma1_out>; > > > }; > > > }; > > > > > > port@1 { > > > reg = <1>; > > > dsi0_out: endpoint { > > > remote-endpoint = <&dsi_panel_in>; > > > }; > > > }; > > > }; > > > }; > > > > > > ...for a dual-dsi panel, it'd be a similar graph. > > > > > > Cheers, > > > Angelo > > > > > > > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > > > > > > > Or two group? > > > > > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > > > > > > > I think we should clearly define this. > > > > > > > > Regards, > > > > CK > > > > > > > > > > > > > > > > > > > Cheers, > > > > > Angelo > > > > > > > > > > > Regards, > > > > > > CK > > > > > > > > > > > > > > > > > > > required: > > > > > > > - compatible > > > > > > > - reg > > > > > > > > > > > > > > > > > > > > >
Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno wrote: >> Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: >>> On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: >>>>> On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del Regno >>>>> wrote: >>>>>> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>>>>>> Hi, Angelo: >>>>>>> >>>>>>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del >>>>>>> Regno >>>>>>> wrote: >>>>>>>> Document OF graph on MMSYS/VDOSYS: this supports up to >>>>>>>> three >>>>>>>> DDP >>>>>>>> paths >>>>>>>> per HW instance (so potentially up to six displays for >>>>>>>> multi- >>>>>>>> vdo >>>>>>>> SoCs). >>>>>>>> >>>>>>>> The MMSYS or VDOSYS is always the first component in the >>>>>>>> DDP >>>>>>>> pipeline, >>>>>>>> so it only supports an output port with multiple >>>>>>>> endpoints - >>>>>>>> where >>>>>>>> each >>>>>>>> endpoint defines the starting point for one of the >>>>>>>> (currently >>>>>>>> three) >>>>>>>> possible hardware paths. >>>>>>>> >>>>>>>> Signed-off-by: AngeloGioacchino Del Regno < >>>>>>>> angelogioacchino.delregno@collabora.com> >>>>>>>> --- >>>>>>>> .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 >>>>>>>> +++++++++++++++++++ >>>>>>>> 1 file changed, 23 insertions(+) >>>>>>>> >>>>>>>> diff --git >>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek >>>>>>>> ,mms >>>>>>>> ys.y >>>>>>>> aml >>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek >>>>>>>> ,mms >>>>>>>> ys.y >>>>>>>> aml >>>>>>>> index b3c6888c1457..4e9acd966aa5 100644 >>>>>>>> --- >>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/mediatek >>>>>>>> ,mms >>>>>>>> ys.y >>>>>>>> aml >>>>>>>> +++ >>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/mediatek >>>>>>>> ,mms >>>>>>>> ys.y >>>>>>>> aml >>>>>>>> @@ -93,6 +93,29 @@ properties: >>>>>>>> '#reset-cells': >>>>>>>> const: 1 >>>>>>>> >>>>>>>> + port: >>>>>>>> + $ref: /schemas/graph.yaml#/properties/port >>>>>>>> + description: >>>>>>>> + Output port node. This port connects the >>>>>>>> MMSYS/VDOSYS >>>>>>>> output >>>>>>>> to >>>>>>>> + the first component of one display pipeline, for >>>>>>>> example >>>>>>>> one >>>>>>>> of >>>>>>>> + the available OVL or RDMA blocks. >>>>>>>> + Some MediaTek SoCs support up to three display >>>>>>>> outputs >>>>>>>> per >>>>>>>> MMSYS. >>>>>>>> + properties: >>>>>>>> + endpoint@0: >>>>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>>>> + description: Output to the primary display >>>>>>>> pipeline >>>>>>>> + >>>>>>>> + endpoint@1: >>>>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>>>> + description: Output to the secondary display >>>>>>>> pipeline >>>>>>>> + >>>>>>>> + endpoint@2: >>>>>>>> + $ref: /schemas/graph.yaml#/properties/endpoint >>>>>>>> + description: Output to the tertiary display >>>>>>>> pipeline >>>>>>>> + >>>>>>>> + required: >>>>>>>> + - endpoint@0 >>>>>>>> + >>>>>>> >>>>>>> mmsys/vdosys does not output data to the first component of >>>>>>> display >>>>>>> pipeline, so this connection looks 'virtual'. Shall we add >>>>>>> something >>>>>>> virtual in device tree? You add this in order to decide >>>>>>> which >>>>>>> pipeline >>>>>>> is 1st, 2nd, 3rd, but for device it don't care which one is >>>>>>> first. >>>>>>> In >>>>>>> computer, software could change which display is the >>>>>>> primary >>>>>>> display. >>>>>>> I'm not sure it's good to decide display order in device >>>>>>> tree? >>>>>>> >>>>>> >>>>>> Devicetree describes hardware, so nothing virtual can be >>>>>> present >>>>>> - >>>>>> and in any case, >>>>>> the primary/secondary/tertiary pipeline is in relation to >>>>>> MM/VDO >>>>>> SYS, >>>>>> not referred >>>>>> to software. >>>>>> >>>>>> Better explaining, the primary pipeline is not necessarily >>>>>> the >>>>>> primary display in >>>>>> DRM terms: that's a concept that is completely detached from >>>>>> the >>>>>> scope of this >>>>>> series and this graph - and it's something that shall be >>>>>> managed >>>>>> solely by the >>>>>> driver (mediatek-drm in this case). >>>>>> >>>>>> Coming back to the connection looking, but *not* being >>>>>> virtual: >>>>>> the >>>>>> sense here is >>>>>> that the MM/VDOSYS blocks are used in the display pipeline to >>>>>> "stitch" together >>>>>> the various display pipeline hardware blocks, or, said >>>>>> differently, >>>>>> setting up the >>>>>> routing between all of those (P.S.: >>>>>> mmsys_mtxxxx_routing_table!) >>>>>> through the VDO >>>>>> Input Selection (VDOx_SEL_IN) or Output Selection >>>>>> (VDOx_SEL_OUT) >>>>>> and >>>>>> with the >>>>>> assistance of the VDO Multiple Output Mask (VDOx_MOUT) for >>>>>> the >>>>>> multiple outputs >>>>>> usecase, both of which, are described by this graph. >>>>> >>>>> I agree this part, but this is related to display device OF >>>>> graph. >>>>> These display device would output video data from one device >>>>> and >>>>> input >>>>> to another video device. These video device would not input or >>>>> output >>>>> video data to mmsys/vdosys. >>>>> >>>>>> >>>>>> This means that the VDOSYS is really the "master" of the >>>>>> display >>>>>> pipeline since >>>>>> everything gets enabled, mixed and matched from there - and >>>>>> that's in >>>>>> the sense >>>>>> of hardware operation, so we are *really* (and not >>>>>> virtually!) >>>>>> flipping switches. >>>>> >>>>> I agree mmsys/vdosys is master of video pipeline, so let's >>>>> define >>>>> what >>>>> the port in mmsys/vdosys is. If the port means the master >>>>> relationship, >>>>> mmsys/vdosys should output port to every display device. Or use >>>>> a >>>>> simply way to show the master relation ship >>>>> >>>>> mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, &rdma1, >>>>> &color1, >>>>> ...>; >>>>> >>>> >>>> There's no need to list all of the VDO0/VDO1/mmsys devices in one >>>> big >>>> array >>>> property, because the actual possible devices can be defined: >>>> 1. In the bindings; and >>>> 2. In the actual OF graph that we write for each SoC+board >>>> combination. >>>> >>>> A graph cannot contain a connection to a device that cannot be >>>> connected to >>>> the previous, so, your "mmsys-subdev" list can be retrieved by >>>> looking at the >>>> graph: >>>> - Start from VDO0/1 or MMSYS >>>> - Walk through (visually, even) OUTPUT ports >>>> - VDO0 (read output ep) -> ovl0 (read output ep) -> rdma0 >>>> (read >>>> output ep) -> >>>> color0 (...) -> etc >>>> - Nothing more - it's all defined there. >>>> >>>>> >>>>> Another problem is how to group display device? If two pipeline >>>>> could >>>>> be route to the same display interface, such as >>>>> >>>>> rdma0 -> dsi >>>>> rdma1 -> dsi >>>>> >>>>> Would this be single group? >>>> >>>> There are multiple ways of doing this, but one that comes to my >>>> mind >>>> right now and >>>> that looks clean as well is the following: >>>> >>>> ovl0@ef01 { >>>> ..... >>>> ports { >>>> port@0 { >>>> reg = <0>; >>>> ovl0_in: endpoint { >>>> remote-endpoint = <&vdosys0_out>; >>>> }; >>>> }; >>> >>> I'm not sure how do you define this port from OVL to vdosys. If >>> this >>> port means 'master relationship', others could add port in COLOR to >>> point to vdosys because COLOR and vdosys has the 'master >>> relationship' >>> and I could not reject this. So we need more specific definition of >>> this port. >> >> >>> Only the 'first' device in pipeline could have this port? >> >> Correct. Only the first device in a pipeline - and this is actually a >> restriction >> that the generic binding definition of port already gives, in a way. >> >> >>> In mt8173, one pipeline is >>> >>> ovl -> color -> aal -> od -> rdma -> ufo -> dsi >>> >>> But rdma has an option to read data from od or directly from DRAM. >>> If >>> from DRAM, the pipeline would be changed to >>> >>> rdma -> ufo -> dsi >>> >>> >>> So it's confused which one is 'first'. >> >> That's why the pipeline is *board-specific* and not soc-generic! >> >> And what you described is *exactly* the reason why I'm adding support >> for the >> OF graphs in mediatek-drm: specifying the correct pipeline for each >> board as per >> what each board wants to use (said differently: for each board's >> *capabilities*). >> >> So, if on a certain board you want to skip OD, you can hook RDMA up >> directly to >> MMSYS/VDOSYS. >> >> In MT8173, one pipeline for one board uses endpoints IN/OUT like >> this: >> >> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >> >> and for another board, endpoints will be like >> >> MMSYS -> RDMA -> UFO -> DSI >> >> ...which is the exact same as you described, and I think that your >> confusion comes >> from the fact that you didn't put MMSYS at the beginning of the >> pipeline :-) > > In one board, both OVL and RDMA could switch dynamically. Because each > one could be the first in one board, mmsys point to both ovl and rdma? > No. MMSYS would still point ONLY to OVL, because OVL is the "earliest point" of the pipeline that this one board does support. In that case, RDMA being present at a later point in the pipeline does not matter and does not prevent us from *temporarily* skipping OVL-COLOR-AAL-OD and going MMSYS->RDMA *directly*. Switching dynamically is a driver duty and will be 100% possible (as much as it is right now) to dynamically switch OVL and RDMA as long as both are present in the pipeline. With this exact pipeline: MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA and MMSYS->RDMA as the driver itself *is allowed to* temporarily ignore part of the pipeline. Please note that, in case it is needed (trust me on this: it's not needed) a custom property in the endpoint node can always be introduced later, so that you can declare a node like endpoint@0 { remote-endpoint = <&ovl0_in>; mediatek,short-path = <&rdma0_in>; }; ...but again, that's never going to be needed, as the driver already does have knowledge of the fact that RDMA is in the pipeline, so it is possible to simply do a temporary override in the driver. What the OF Graph support does is to build the same arrays, that we currently have hardcoded in mediatek-drm, by reading from device tree. Nothing else and nothing more - for now. Having the OF Graph support makes us able to also add new dual-path support with more complicated connections than the current ones, without any problem and, in many cases, without even modifying the bindings from what I provided in this series. Cheers, Angelo > Regards, > CK > >> >> >> >> >> In case you need any *temporary override* on any board that defines a >> pipeline like >> >> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >> >> so that the pipeline *temporarily* becomes (for power management, or >> for any other >> reason) RDMA -> UFO -> DSI .... that's not a concern: the graph is >> present, and it >> is used to tell to the driver what is the regular pipeline to use. >> Eventual temporary overrides can be managed transparently inside of >> the driver with >> C code and no changes to the devicetree are required. >> >> >>> I don't know how to decide which device could point to >>> mmsys/vdosys. So >>> please give a specific definition. >>> >> >> Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a >> device. >> >> So, mmsys/vdosys must point to the *first device in the pipeline*. >> >> Any other doubt? >> >> Cheers, >> Angelo >> >>> Regards, >>> CK >>> >>>> >>>> port@1 { >>>> reg = <1>; >>>> ovl0_out0: endpoint@0 { >>>> remote-endpoint = <&rdma0_in>; >>>> }; >>>> ovl0_out1: endpoint@1 { >>>> remote-endpoint = <&rdma1_in>; >>>> }; >>>> }; >>>> }; >>>> }; >>>> >>>> rdma0@1234 { >>>> ..... >>>> ports { >>>> port@0 { >>>> reg = <0>; >>>> rdma0_in: endpoint { >>>> remote-endpoint = <&ovl0_out0>; /* assuming ovl0 >>>> outputs to >>>> rdma0...*/ >>>> }; >>>> }; >>>> port@1 { >>>> reg = <1>; >>>> rdma0_out: endpoint@1 { >>>> remote-endpoint = <&dsi_dual_intf0_in>; >>>> }; >>>> }; >>>> }; >>>> }; >>>> >>>> >>>> rdma1@5678 { >>>> ..... >>>> ports { >>>> port@0 { >>>> reg = <0>; >>>> rdma1_in: endpoint { >>>> /* assuming ovl0 outputs to rdma1 as well... can be >>>> something else. */ >>>> remote-endpoint = <&ovl0_out1>; >>>> }; >>>> }; >>>> port@1 { >>>> reg = <1>; >>>> rdma1_out: endpoint { >>>> remote-endpoint = <&dsi_dual_intf1_in>; >>>> }; >>>> }; >>>> }; >>>> }; >>>> >>>> >>>> dsi@9abcd { >>>> ..... >>>> ports { >>>> port@0 { >>>> reg = <0>; >>>> /* Where endpoint@0 could be always DSI LEFT CTRL */ >>>> dsi_dual_intf0_in: endpoint@0 { >>>> remote-endpoint = <&rdma0_out>; >>>> }; >>>> /* ...and @1 could be always DSI RIGHT CTRL */ >>>> dsi_dual_intf1_in: endpoint@1 { >>>> remote-endpoint = <&rdma1_out>; >>>> }; >>>> }; >>>> >>>> port@1 { >>>> reg = <1>; >>>> dsi0_out: endpoint { >>>> remote-endpoint = <&dsi_panel_in>; >>>> }; >>>> }; >>>> }; >>>> }; >>>> >>>> ...for a dual-dsi panel, it'd be a similar graph. >>>> >>>> Cheers, >>>> Angelo >>>> >>>>> >>>>> mmsys-subdev = <&rdma0, &rdma1, &dsi>; >>>>> >>>>> Or two group? >>>>> >>>>> mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; >>>>> >>>>> I think we should clearly define this. >>>>> >>>>> Regards, >>>>> CK >>>>> >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Angelo >>>>>> >>>>>>> Regards, >>>>>>> CK >>>>>>> >>>>>>> >>>>>>>> required: >>>>>>>> - compatible >>>>>>>> - reg >>>>>> >>>>>> >>>> >>>> >>>> >> >>
On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno wrote: > Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: > > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno > > wrote: > > > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > > > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del > > > > > > Regno > > > > > > wrote: > > > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > > > > > Hi, Angelo: > > > > > > > > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del > > > > > > > > Regno > > > > > > > > wrote: > > > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports up > > > > > > > > > to > > > > > > > > > three > > > > > > > > > DDP > > > > > > > > > paths > > > > > > > > > per HW instance (so potentially up to six displays > > > > > > > > > for > > > > > > > > > multi- > > > > > > > > > vdo > > > > > > > > > SoCs). > > > > > > > > > > > > > > > > > > The MMSYS or VDOSYS is always the first component in > > > > > > > > > the > > > > > > > > > DDP > > > > > > > > > pipeline, > > > > > > > > > so it only supports an output port with multiple > > > > > > > > > endpoints - > > > > > > > > > where > > > > > > > > > each > > > > > > > > > endpoint defines the starting point for one of the > > > > > > > > > (currently > > > > > > > > > three) > > > > > > > > > possible hardware paths. > > > > > > > > > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > > > > > angelogioacchino.delregno@collabora.com> > > > > > > > > > --- > > > > > > > > > .../bindings/arm/mediatek/mediatek,mmsys.yaml | > > > > > > > > > 23 > > > > > > > > > +++++++++++++++++++ > > > > > > > > > 1 file changed, 23 insertions(+) > > > > > > > > > > > > > > > > > > diff --git > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > index b3c6888c1457..4e9acd966aa5 100644 > > > > > > > > > --- > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > +++ > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/medi > > > > > > > > > atek > > > > > > > > > ,mms > > > > > > > > > ys.y > > > > > > > > > aml > > > > > > > > > @@ -93,6 +93,29 @@ properties: > > > > > > > > > '#reset-cells': > > > > > > > > > const: 1 > > > > > > > > > > > > > > > > > > + port: > > > > > > > > > + $ref: /schemas/graph.yaml#/properties/port > > > > > > > > > + description: > > > > > > > > > + Output port node. This port connects the > > > > > > > > > MMSYS/VDOSYS > > > > > > > > > output > > > > > > > > > to > > > > > > > > > + the first component of one display pipeline, > > > > > > > > > for > > > > > > > > > example > > > > > > > > > one > > > > > > > > > of > > > > > > > > > + the available OVL or RDMA blocks. > > > > > > > > > + Some MediaTek SoCs support up to three display > > > > > > > > > outputs > > > > > > > > > per > > > > > > > > > MMSYS. > > > > > > > > > + properties: > > > > > > > > > + endpoint@0: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the primary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + endpoint@1: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the secondary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + endpoint@2: > > > > > > > > > + $ref: > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > + description: Output to the tertiary display > > > > > > > > > pipeline > > > > > > > > > + > > > > > > > > > + required: > > > > > > > > > + - endpoint@0 > > > > > > > > > + > > > > > > > > > > > > > > > > mmsys/vdosys does not output data to the first > > > > > > > > component of > > > > > > > > display > > > > > > > > pipeline, so this connection looks 'virtual'. Shall we > > > > > > > > add > > > > > > > > something > > > > > > > > virtual in device tree? You add this in order to decide > > > > > > > > which > > > > > > > > pipeline > > > > > > > > is 1st, 2nd, 3rd, but for device it don't care which > > > > > > > > one is > > > > > > > > first. > > > > > > > > In > > > > > > > > computer, software could change which display is the > > > > > > > > primary > > > > > > > > display. > > > > > > > > I'm not sure it's good to decide display order in > > > > > > > > device > > > > > > > > tree? > > > > > > > > > > > > > > > > > > > > > > Devicetree describes hardware, so nothing virtual can be > > > > > > > present > > > > > > > - > > > > > > > and in any case, > > > > > > > the primary/secondary/tertiary pipeline is in relation to > > > > > > > MM/VDO > > > > > > > SYS, > > > > > > > not referred > > > > > > > to software. > > > > > > > > > > > > > > Better explaining, the primary pipeline is not > > > > > > > necessarily > > > > > > > the > > > > > > > primary display in > > > > > > > DRM terms: that's a concept that is completely detached > > > > > > > from > > > > > > > the > > > > > > > scope of this > > > > > > > series and this graph - and it's something that shall be > > > > > > > managed > > > > > > > solely by the > > > > > > > driver (mediatek-drm in this case). > > > > > > > > > > > > > > Coming back to the connection looking, but *not* being > > > > > > > virtual: > > > > > > > the > > > > > > > sense here is > > > > > > > that the MM/VDOSYS blocks are used in the display > > > > > > > pipeline to > > > > > > > "stitch" together > > > > > > > the various display pipeline hardware blocks, or, said > > > > > > > differently, > > > > > > > setting up the > > > > > > > routing between all of those (P.S.: > > > > > > > mmsys_mtxxxx_routing_table!) > > > > > > > through the VDO > > > > > > > Input Selection (VDOx_SEL_IN) or Output Selection > > > > > > > (VDOx_SEL_OUT) > > > > > > > and > > > > > > > with the > > > > > > > assistance of the VDO Multiple Output Mask (VDOx_MOUT) > > > > > > > for > > > > > > > the > > > > > > > multiple outputs > > > > > > > usecase, both of which, are described by this graph. > > > > > > > > > > > > I agree this part, but this is related to display device OF > > > > > > graph. > > > > > > These display device would output video data from one > > > > > > device > > > > > > and > > > > > > input > > > > > > to another video device. These video device would not input > > > > > > or > > > > > > output > > > > > > video data to mmsys/vdosys. > > > > > > > > > > > > > > > > > > > > This means that the VDOSYS is really the "master" of the > > > > > > > display > > > > > > > pipeline since > > > > > > > everything gets enabled, mixed and matched from there - > > > > > > > and > > > > > > > that's in > > > > > > > the sense > > > > > > > of hardware operation, so we are *really* (and not > > > > > > > virtually!) > > > > > > > flipping switches. > > > > > > > > > > > > I agree mmsys/vdosys is master of video pipeline, so let's > > > > > > define > > > > > > what > > > > > > the port in mmsys/vdosys is. If the port means the master > > > > > > relationship, > > > > > > mmsys/vdosys should output port to every display device. Or > > > > > > use > > > > > > a > > > > > > simply way to show the master relation ship > > > > > > > > > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, > > > > > > &rdma1, > > > > > > &color1, > > > > > > ...>; > > > > > > > > > > > > > > > > There's no need to list all of the VDO0/VDO1/mmsys devices in > > > > > one > > > > > big > > > > > array > > > > > property, because the actual possible devices can be defined: > > > > > 1. In the bindings; and > > > > > 2. In the actual OF graph that we write for each > > > > > SoC+board > > > > > combination. > > > > > > > > > > A graph cannot contain a connection to a device that cannot > > > > > be > > > > > connected to > > > > > the previous, so, your "mmsys-subdev" list can be retrieved > > > > > by > > > > > looking at the > > > > > graph: > > > > > - Start from VDO0/1 or MMSYS > > > > > - Walk through (visually, even) OUTPUT ports > > > > > - VDO0 (read output ep) -> ovl0 (read output ep) -> > > > > > rdma0 > > > > > (read > > > > > output ep) -> > > > > > color0 (...) -> etc > > > > > - Nothing more - it's all defined there. > > > > > > > > > > > > > > > > > Another problem is how to group display device? If two > > > > > > pipeline > > > > > > could > > > > > > be route to the same display interface, such as > > > > > > > > > > > > rdma0 -> dsi > > > > > > rdma1 -> dsi > > > > > > > > > > > > Would this be single group? > > > > > > > > > > There are multiple ways of doing this, but one that comes to > > > > > my > > > > > mind > > > > > right now and > > > > > that looks clean as well is the following: > > > > > > > > > > ovl0@ef01 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > ovl0_in: endpoint { > > > > > remote-endpoint = <&vdosys0_out>; > > > > > }; > > > > > }; > > > > > > > > I'm not sure how do you define this port from OVL to vdosys. If > > > > this > > > > port means 'master relationship', others could add port in > > > > COLOR to > > > > point to vdosys because COLOR and vdosys has the 'master > > > > relationship' > > > > and I could not reject this. So we need more specific > > > > definition of > > > > this port. > > > > > > > > > > Only the 'first' device in pipeline could have this port? > > > > > > Correct. Only the first device in a pipeline - and this is > > > actually a > > > restriction > > > that the generic binding definition of port already gives, in a > > > way. > > > > > > > > > > In mt8173, one pipeline is > > > > > > > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi > > > > > > > > But rdma has an option to read data from od or directly from > > > > DRAM. > > > > If > > > > from DRAM, the pipeline would be changed to > > > > > > > > rdma -> ufo -> dsi > > > > > > > > > > > > So it's confused which one is 'first'. > > > > > > That's why the pipeline is *board-specific* and not soc-generic! > > > > > > And what you described is *exactly* the reason why I'm adding > > > support > > > for the > > > OF graphs in mediatek-drm: specifying the correct pipeline for > > > each > > > board as per > > > what each board wants to use (said differently: for each board's > > > *capabilities*). > > > > > > So, if on a certain board you want to skip OD, you can hook RDMA > > > up > > > directly to > > > MMSYS/VDOSYS. > > > > > > In MT8173, one pipeline for one board uses endpoints IN/OUT like > > > this: > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > and for another board, endpoints will be like > > > > > > MMSYS -> RDMA -> UFO -> DSI > > > > > > ...which is the exact same as you described, and I think that > > > your > > > confusion comes > > > from the fact that you didn't put MMSYS at the beginning of the > > > pipeline :-) > > > > In one board, both OVL and RDMA could switch dynamically. Because > > each > > one could be the first in one board, mmsys point to both ovl and > > rdma? > > > > No. > > MMSYS would still point ONLY to OVL, because OVL is the "earliest > point" > of the pipeline that this one board does support. > > In that case, RDMA being present at a later point in the pipeline > does not > matter and does not prevent us from *temporarily* skipping OVL-COLOR- > AAL-OD > and going MMSYS->RDMA *directly*. > > Switching dynamically is a driver duty and will be 100% possible (as > much > as it is right now) to dynamically switch OVL and RDMA as long as > both are > present in the pipeline. > > With this exact pipeline: > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA and > MMSYS->RDMA as the driver itself *is allowed to* temporarily ignore > part > of the pipeline. > > Please note that, in case it is needed (trust me on this: it's not > needed) > a custom property in the endpoint node can always be introduced > later, so > that you can declare a node like > > endpoint@0 { > remote-endpoint = <&ovl0_in>; > mediatek,short-path = <&rdma0_in>; > }; > > ...but again, that's never going to be needed, as the driver already > does > have knowledge of the fact that RDMA is in the pipeline, so it is > possible > to simply do a temporary override in the driver. > > What the OF Graph support does is to build the same arrays, that we > currently > have hardcoded in mediatek-drm, by reading from device tree. > > Nothing else and nothing more - for now. > > Having the OF Graph support makes us able to also add new dual-path > support > with more complicated connections than the current ones, without any > problem > and, in many cases, without even modifying the bindings from what I > provided > in this series. OK, please add the information we discussed into binding document to prevent anyone misusing this binding. Regards, CK > > Cheers, > Angelo > > > Regards, > > CK > > > > > > > > > > > > > > > > > In case you need any *temporary override* on any board that > > > defines a > > > pipeline like > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > so that the pipeline *temporarily* becomes (for power management, > > > or > > > for any other > > > reason) RDMA -> UFO -> DSI .... that's not a concern: the graph > > > is > > > present, and it > > > is used to tell to the driver what is the regular pipeline to > > > use. > > > Eventual temporary overrides can be managed transparently inside > > > of > > > the driver with > > > C code and no changes to the devicetree are required. > > > > > > > > > > I don't know how to decide which device could point to > > > > mmsys/vdosys. So > > > > please give a specific definition. > > > > > > > > > > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a > > > device. > > > > > > So, mmsys/vdosys must point to the *first device in the > > > pipeline*. > > > > > > Any other doubt? > > > > > > Cheers, > > > Angelo > > > > > > > Regards, > > > > CK > > > > > > > > > > > > > > port@1 { > > > > > reg = <1>; > > > > > ovl0_out0: endpoint@0 { > > > > > remote-endpoint = <&rdma0_in>; > > > > > }; > > > > > ovl0_out1: endpoint@1 { > > > > > remote-endpoint = <&rdma1_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > rdma0@1234 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > rdma0_in: endpoint { > > > > > remote-endpoint = <&ovl0_out0>; /* assuming ovl0 > > > > > outputs to > > > > > rdma0...*/ > > > > > }; > > > > > }; > > > > > port@1 { > > > > > reg = <1>; > > > > > rdma0_out: endpoint@1 { > > > > > remote-endpoint = <&dsi_dual_intf0_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > > > > > > rdma1@5678 { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > rdma1_in: endpoint { > > > > > /* assuming ovl0 outputs to rdma1 as well... can > > > > > be > > > > > something else. */ > > > > > remote-endpoint = <&ovl0_out1>; > > > > > }; > > > > > }; > > > > > port@1 { > > > > > reg = <1>; > > > > > rdma1_out: endpoint { > > > > > remote-endpoint = <&dsi_dual_intf1_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > > > > > > dsi@9abcd { > > > > > ..... > > > > > ports { > > > > > port@0 { > > > > > reg = <0>; > > > > > /* Where endpoint@0 could be always DSI LEFT CTRL */ > > > > > dsi_dual_intf0_in: endpoint@0 { > > > > > remote-endpoint = <&rdma0_out>; > > > > > }; > > > > > /* ...and @1 could be always DSI RIGHT CTRL */ > > > > > dsi_dual_intf1_in: endpoint@1 { > > > > > remote-endpoint = <&rdma1_out>; > > > > > }; > > > > > }; > > > > > > > > > > port@1 { > > > > > reg = <1>; > > > > > dsi0_out: endpoint { > > > > > remote-endpoint = <&dsi_panel_in>; > > > > > }; > > > > > }; > > > > > }; > > > > > }; > > > > > > > > > > ...for a dual-dsi panel, it'd be a similar graph. > > > > > > > > > > Cheers, > > > > > Angelo > > > > > > > > > > > > > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > > > > > > > > > > > Or two group? > > > > > > > > > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > > > > > > > > > > > I think we should clearly define this. > > > > > > > > > > > > Regards, > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > Angelo > > > > > > > > > > > > > > > Regards, > > > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > required: > > > > > > > > > - compatible > > > > > > > > > - reg > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto: > On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno wrote: >> Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: >>> On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: >>>>> On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del Regno >>>>> wrote: >>>>>> Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: >>>>>>> On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del >>>>>>> Regno >>>>>>> wrote: >>>>>>>> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>>>>>>>> Hi, Angelo: >>>>>>>>> >>>>>>>>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino Del >>>>>>>>> Regno >>>>>>>>> wrote: >>>>>>>>>> Document OF graph on MMSYS/VDOSYS: this supports up >>>>>>>>>> to >>>>>>>>>> three >>>>>>>>>> DDP >>>>>>>>>> paths >>>>>>>>>> per HW instance (so potentially up to six displays >>>>>>>>>> for >>>>>>>>>> multi- >>>>>>>>>> vdo >>>>>>>>>> SoCs). >>>>>>>>>> >>>>>>>>>> The MMSYS or VDOSYS is always the first component in >>>>>>>>>> the >>>>>>>>>> DDP >>>>>>>>>> pipeline, >>>>>>>>>> so it only supports an output port with multiple >>>>>>>>>> endpoints - >>>>>>>>>> where >>>>>>>>>> each >>>>>>>>>> endpoint defines the starting point for one of the >>>>>>>>>> (currently >>>>>>>>>> three) >>>>>>>>>> possible hardware paths. >>>>>>>>>> >>>>>>>>>> Signed-off-by: AngeloGioacchino Del Regno < >>>>>>>>>> angelogioacchino.delregno@collabora.com> >>>>>>>>>> --- >>>>>>>>>> .../bindings/arm/mediatek/mediatek,mmsys.yaml | >>>>>>>>>> 23 >>>>>>>>>> +++++++++++++++++++ >>>>>>>>>> 1 file changed, 23 insertions(+) >>>>>>>>>> >>>>>>>>>> diff --git >>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/medi >>>>>>>>>> atek >>>>>>>>>> ,mms >>>>>>>>>> ys.y >>>>>>>>>> aml >>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/medi >>>>>>>>>> atek >>>>>>>>>> ,mms >>>>>>>>>> ys.y >>>>>>>>>> aml >>>>>>>>>> index b3c6888c1457..4e9acd966aa5 100644 >>>>>>>>>> --- >>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/medi >>>>>>>>>> atek >>>>>>>>>> ,mms >>>>>>>>>> ys.y >>>>>>>>>> aml >>>>>>>>>> +++ >>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/medi >>>>>>>>>> atek >>>>>>>>>> ,mms >>>>>>>>>> ys.y >>>>>>>>>> aml >>>>>>>>>> @@ -93,6 +93,29 @@ properties: >>>>>>>>>> '#reset-cells': >>>>>>>>>> const: 1 >>>>>>>>>> >>>>>>>>>> + port: >>>>>>>>>> + $ref: /schemas/graph.yaml#/properties/port >>>>>>>>>> + description: >>>>>>>>>> + Output port node. This port connects the >>>>>>>>>> MMSYS/VDOSYS >>>>>>>>>> output >>>>>>>>>> to >>>>>>>>>> + the first component of one display pipeline, >>>>>>>>>> for >>>>>>>>>> example >>>>>>>>>> one >>>>>>>>>> of >>>>>>>>>> + the available OVL or RDMA blocks. >>>>>>>>>> + Some MediaTek SoCs support up to three display >>>>>>>>>> outputs >>>>>>>>>> per >>>>>>>>>> MMSYS. >>>>>>>>>> + properties: >>>>>>>>>> + endpoint@0: >>>>>>>>>> + $ref: >>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>> + description: Output to the primary display >>>>>>>>>> pipeline >>>>>>>>>> + >>>>>>>>>> + endpoint@1: >>>>>>>>>> + $ref: >>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>> + description: Output to the secondary display >>>>>>>>>> pipeline >>>>>>>>>> + >>>>>>>>>> + endpoint@2: >>>>>>>>>> + $ref: >>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>> + description: Output to the tertiary display >>>>>>>>>> pipeline >>>>>>>>>> + >>>>>>>>>> + required: >>>>>>>>>> + - endpoint@0 >>>>>>>>>> + >>>>>>>>> >>>>>>>>> mmsys/vdosys does not output data to the first >>>>>>>>> component of >>>>>>>>> display >>>>>>>>> pipeline, so this connection looks 'virtual'. Shall we >>>>>>>>> add >>>>>>>>> something >>>>>>>>> virtual in device tree? You add this in order to decide >>>>>>>>> which >>>>>>>>> pipeline >>>>>>>>> is 1st, 2nd, 3rd, but for device it don't care which >>>>>>>>> one is >>>>>>>>> first. >>>>>>>>> In >>>>>>>>> computer, software could change which display is the >>>>>>>>> primary >>>>>>>>> display. >>>>>>>>> I'm not sure it's good to decide display order in >>>>>>>>> device >>>>>>>>> tree? >>>>>>>>> >>>>>>>> >>>>>>>> Devicetree describes hardware, so nothing virtual can be >>>>>>>> present >>>>>>>> - >>>>>>>> and in any case, >>>>>>>> the primary/secondary/tertiary pipeline is in relation to >>>>>>>> MM/VDO >>>>>>>> SYS, >>>>>>>> not referred >>>>>>>> to software. >>>>>>>> >>>>>>>> Better explaining, the primary pipeline is not >>>>>>>> necessarily >>>>>>>> the >>>>>>>> primary display in >>>>>>>> DRM terms: that's a concept that is completely detached >>>>>>>> from >>>>>>>> the >>>>>>>> scope of this >>>>>>>> series and this graph - and it's something that shall be >>>>>>>> managed >>>>>>>> solely by the >>>>>>>> driver (mediatek-drm in this case). >>>>>>>> >>>>>>>> Coming back to the connection looking, but *not* being >>>>>>>> virtual: >>>>>>>> the >>>>>>>> sense here is >>>>>>>> that the MM/VDOSYS blocks are used in the display >>>>>>>> pipeline to >>>>>>>> "stitch" together >>>>>>>> the various display pipeline hardware blocks, or, said >>>>>>>> differently, >>>>>>>> setting up the >>>>>>>> routing between all of those (P.S.: >>>>>>>> mmsys_mtxxxx_routing_table!) >>>>>>>> through the VDO >>>>>>>> Input Selection (VDOx_SEL_IN) or Output Selection >>>>>>>> (VDOx_SEL_OUT) >>>>>>>> and >>>>>>>> with the >>>>>>>> assistance of the VDO Multiple Output Mask (VDOx_MOUT) >>>>>>>> for >>>>>>>> the >>>>>>>> multiple outputs >>>>>>>> usecase, both of which, are described by this graph. >>>>>>> >>>>>>> I agree this part, but this is related to display device OF >>>>>>> graph. >>>>>>> These display device would output video data from one >>>>>>> device >>>>>>> and >>>>>>> input >>>>>>> to another video device. These video device would not input >>>>>>> or >>>>>>> output >>>>>>> video data to mmsys/vdosys. >>>>>>> >>>>>>>> >>>>>>>> This means that the VDOSYS is really the "master" of the >>>>>>>> display >>>>>>>> pipeline since >>>>>>>> everything gets enabled, mixed and matched from there - >>>>>>>> and >>>>>>>> that's in >>>>>>>> the sense >>>>>>>> of hardware operation, so we are *really* (and not >>>>>>>> virtually!) >>>>>>>> flipping switches. >>>>>>> >>>>>>> I agree mmsys/vdosys is master of video pipeline, so let's >>>>>>> define >>>>>>> what >>>>>>> the port in mmsys/vdosys is. If the port means the master >>>>>>> relationship, >>>>>>> mmsys/vdosys should output port to every display device. Or >>>>>>> use >>>>>>> a >>>>>>> simply way to show the master relation ship >>>>>>> >>>>>>> mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, >>>>>>> &rdma1, >>>>>>> &color1, >>>>>>> ...>; >>>>>>> >>>>>> >>>>>> There's no need to list all of the VDO0/VDO1/mmsys devices in >>>>>> one >>>>>> big >>>>>> array >>>>>> property, because the actual possible devices can be defined: >>>>>> 1. In the bindings; and >>>>>> 2. In the actual OF graph that we write for each >>>>>> SoC+board >>>>>> combination. >>>>>> >>>>>> A graph cannot contain a connection to a device that cannot >>>>>> be >>>>>> connected to >>>>>> the previous, so, your "mmsys-subdev" list can be retrieved >>>>>> by >>>>>> looking at the >>>>>> graph: >>>>>> - Start from VDO0/1 or MMSYS >>>>>> - Walk through (visually, even) OUTPUT ports >>>>>> - VDO0 (read output ep) -> ovl0 (read output ep) -> >>>>>> rdma0 >>>>>> (read >>>>>> output ep) -> >>>>>> color0 (...) -> etc >>>>>> - Nothing more - it's all defined there. >>>>>> >>>>>>> >>>>>>> Another problem is how to group display device? If two >>>>>>> pipeline >>>>>>> could >>>>>>> be route to the same display interface, such as >>>>>>> >>>>>>> rdma0 -> dsi >>>>>>> rdma1 -> dsi >>>>>>> >>>>>>> Would this be single group? >>>>>> >>>>>> There are multiple ways of doing this, but one that comes to >>>>>> my >>>>>> mind >>>>>> right now and >>>>>> that looks clean as well is the following: >>>>>> >>>>>> ovl0@ef01 { >>>>>> ..... >>>>>> ports { >>>>>> port@0 { >>>>>> reg = <0>; >>>>>> ovl0_in: endpoint { >>>>>> remote-endpoint = <&vdosys0_out>; >>>>>> }; >>>>>> }; >>>>> >>>>> I'm not sure how do you define this port from OVL to vdosys. If >>>>> this >>>>> port means 'master relationship', others could add port in >>>>> COLOR to >>>>> point to vdosys because COLOR and vdosys has the 'master >>>>> relationship' >>>>> and I could not reject this. So we need more specific >>>>> definition of >>>>> this port. >>>> >>>> >>>>> Only the 'first' device in pipeline could have this port? >>>> >>>> Correct. Only the first device in a pipeline - and this is >>>> actually a >>>> restriction >>>> that the generic binding definition of port already gives, in a >>>> way. >>>> >>>> >>>>> In mt8173, one pipeline is >>>>> >>>>> ovl -> color -> aal -> od -> rdma -> ufo -> dsi >>>>> >>>>> But rdma has an option to read data from od or directly from >>>>> DRAM. >>>>> If >>>>> from DRAM, the pipeline would be changed to >>>>> >>>>> rdma -> ufo -> dsi >>>>> >>>>> >>>>> So it's confused which one is 'first'. >>>> >>>> That's why the pipeline is *board-specific* and not soc-generic! >>>> >>>> And what you described is *exactly* the reason why I'm adding >>>> support >>>> for the >>>> OF graphs in mediatek-drm: specifying the correct pipeline for >>>> each >>>> board as per >>>> what each board wants to use (said differently: for each board's >>>> *capabilities*). >>>> >>>> So, if on a certain board you want to skip OD, you can hook RDMA >>>> up >>>> directly to >>>> MMSYS/VDOSYS. >>>> >>>> In MT8173, one pipeline for one board uses endpoints IN/OUT like >>>> this: >>>> >>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>> >>>> and for another board, endpoints will be like >>>> >>>> MMSYS -> RDMA -> UFO -> DSI >>>> >>>> ...which is the exact same as you described, and I think that >>>> your >>>> confusion comes >>>> from the fact that you didn't put MMSYS at the beginning of the >>>> pipeline :-) >>> >>> In one board, both OVL and RDMA could switch dynamically. Because >>> each >>> one could be the first in one board, mmsys point to both ovl and >>> rdma? >>> >> >> No. >> >> MMSYS would still point ONLY to OVL, because OVL is the "earliest >> point" >> of the pipeline that this one board does support. >> >> In that case, RDMA being present at a later point in the pipeline >> does not >> matter and does not prevent us from *temporarily* skipping OVL-COLOR- >> AAL-OD >> and going MMSYS->RDMA *directly*. >> >> Switching dynamically is a driver duty and will be 100% possible (as >> much >> as it is right now) to dynamically switch OVL and RDMA as long as >> both are >> present in the pipeline. >> >> With this exact pipeline: >> >> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >> >> the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA and >> MMSYS->RDMA as the driver itself *is allowed to* temporarily ignore >> part >> of the pipeline. >> >> Please note that, in case it is needed (trust me on this: it's not >> needed) >> a custom property in the endpoint node can always be introduced >> later, so >> that you can declare a node like >> >> endpoint@0 { >> remote-endpoint = <&ovl0_in>; >> mediatek,short-path = <&rdma0_in>; >> }; >> >> ...but again, that's never going to be needed, as the driver already >> does >> have knowledge of the fact that RDMA is in the pipeline, so it is >> possible >> to simply do a temporary override in the driver. >> >> What the OF Graph support does is to build the same arrays, that we >> currently >> have hardcoded in mediatek-drm, by reading from device tree. >> >> Nothing else and nothing more - for now. >> >> Having the OF Graph support makes us able to also add new dual-path >> support >> with more complicated connections than the current ones, without any >> problem >> and, in many cases, without even modifying the bindings from what I >> provided >> in this series. > > OK, please add the information we discussed into binding document to > prevent anyone misusing this binding. > Sorry CK, but the binding *does* already have this information inside - not in terms of "phrases", but in terms of restrictions of the binding... If you want, though, I can add this information in the description of the commit that is adding this binding, is that okay for you? Thanks, Angelo > Regards, > CK > >> >> Cheers, >> Angelo >> >>> Regards, >>> CK >>> >>>> >>>> >>>> >>>> >>>> In case you need any *temporary override* on any board that >>>> defines a >>>> pipeline like >>>> >>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>> >>>> so that the pipeline *temporarily* becomes (for power management, >>>> or >>>> for any other >>>> reason) RDMA -> UFO -> DSI .... that's not a concern: the graph >>>> is >>>> present, and it >>>> is used to tell to the driver what is the regular pipeline to >>>> use. >>>> Eventual temporary overrides can be managed transparently inside >>>> of >>>> the driver with >>>> C code and no changes to the devicetree are required. >>>> >>>> >>>>> I don't know how to decide which device could point to >>>>> mmsys/vdosys. So >>>>> please give a specific definition. >>>>> >>>> >>>> Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing to a >>>> device. >>>> >>>> So, mmsys/vdosys must point to the *first device in the >>>> pipeline*. >>>> >>>> Any other doubt? >>>> >>>> Cheers, >>>> Angelo >>>> >>>>> Regards, >>>>> CK >>>>> >>>>>> >>>>>> port@1 { >>>>>> reg = <1>; >>>>>> ovl0_out0: endpoint@0 { >>>>>> remote-endpoint = <&rdma0_in>; >>>>>> }; >>>>>> ovl0_out1: endpoint@1 { >>>>>> remote-endpoint = <&rdma1_in>; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> >>>>>> rdma0@1234 { >>>>>> ..... >>>>>> ports { >>>>>> port@0 { >>>>>> reg = <0>; >>>>>> rdma0_in: endpoint { >>>>>> remote-endpoint = <&ovl0_out0>; /* assuming ovl0 >>>>>> outputs to >>>>>> rdma0...*/ >>>>>> }; >>>>>> }; >>>>>> port@1 { >>>>>> reg = <1>; >>>>>> rdma0_out: endpoint@1 { >>>>>> remote-endpoint = <&dsi_dual_intf0_in>; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> >>>>>> >>>>>> rdma1@5678 { >>>>>> ..... >>>>>> ports { >>>>>> port@0 { >>>>>> reg = <0>; >>>>>> rdma1_in: endpoint { >>>>>> /* assuming ovl0 outputs to rdma1 as well... can >>>>>> be >>>>>> something else. */ >>>>>> remote-endpoint = <&ovl0_out1>; >>>>>> }; >>>>>> }; >>>>>> port@1 { >>>>>> reg = <1>; >>>>>> rdma1_out: endpoint { >>>>>> remote-endpoint = <&dsi_dual_intf1_in>; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> >>>>>> >>>>>> dsi@9abcd { >>>>>> ..... >>>>>> ports { >>>>>> port@0 { >>>>>> reg = <0>; >>>>>> /* Where endpoint@0 could be always DSI LEFT CTRL */ >>>>>> dsi_dual_intf0_in: endpoint@0 { >>>>>> remote-endpoint = <&rdma0_out>; >>>>>> }; >>>>>> /* ...and @1 could be always DSI RIGHT CTRL */ >>>>>> dsi_dual_intf1_in: endpoint@1 { >>>>>> remote-endpoint = <&rdma1_out>; >>>>>> }; >>>>>> }; >>>>>> >>>>>> port@1 { >>>>>> reg = <1>; >>>>>> dsi0_out: endpoint { >>>>>> remote-endpoint = <&dsi_panel_in>; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> }; >>>>>> >>>>>> ...for a dual-dsi panel, it'd be a similar graph. >>>>>> >>>>>> Cheers, >>>>>> Angelo >>>>>> >>>>>>> >>>>>>> mmsys-subdev = <&rdma0, &rdma1, &dsi>; >>>>>>> >>>>>>> Or two group? >>>>>>> >>>>>>> mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; >>>>>>> >>>>>>> I think we should clearly define this. >>>>>>> >>>>>>> Regards, >>>>>>> CK >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Angelo >>>>>>>> >>>>>>>>> Regards, >>>>>>>>> CK >>>>>>>>> >>>>>>>>> >>>>>>>>>> required: >>>>>>>>>> - compatible >>>>>>>>>> - reg >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >> >> >>
On Fri, 2024-05-10 at 12:14 +0200, AngeloGioacchino Del Regno wrote: > Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto: > > On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno > > wrote: > > > Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: > > > > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > > > > > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del > > > > > > Regno > > > > > > wrote: > > > > > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > > > > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del > > > > > > > > Regno > > > > > > > > wrote: > > > > > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > > > > > > > Hi, Angelo: > > > > > > > > > > > > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino > > > > > > > > > > Del > > > > > > > > > > Regno > > > > > > > > > > wrote: > > > > > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports > > > > > > > > > > > up > > > > > > > > > > > to > > > > > > > > > > > three > > > > > > > > > > > DDP > > > > > > > > > > > paths > > > > > > > > > > > per HW instance (so potentially up to six > > > > > > > > > > > displays > > > > > > > > > > > for > > > > > > > > > > > multi- > > > > > > > > > > > vdo > > > > > > > > > > > SoCs). > > > > > > > > > > > > > > > > > > > > > > The MMSYS or VDOSYS is always the first component > > > > > > > > > > > in > > > > > > > > > > > the > > > > > > > > > > > DDP > > > > > > > > > > > pipeline, > > > > > > > > > > > so it only supports an output port with multiple > > > > > > > > > > > endpoints - > > > > > > > > > > > where > > > > > > > > > > > each > > > > > > > > > > > endpoint defines the starting point for one of > > > > > > > > > > > the > > > > > > > > > > > (currently > > > > > > > > > > > three) > > > > > > > > > > > possible hardware paths. > > > > > > > > > > > > > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > > > > > > > angelogioacchino.delregno@collabora.com> > > > > > > > > > > > --- > > > > > > > > > > > .../bindings/arm/mediatek/mediatek,mmsys.ya > > > > > > > > > > > ml | > > > > > > > > > > > 23 > > > > > > > > > > > +++++++++++++++++++ > > > > > > > > > > > 1 file changed, 23 insertions(+) > > > > > > > > > > > > > > > > > > > > > > diff --git > > > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/ > > > > > > > > > > > medi > > > > > > > > > > > atek > > > > > > > > > > > ,mms > > > > > > > > > > > ys.y > > > > > > > > > > > aml > > > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/ > > > > > > > > > > > medi > > > > > > > > > > > atek > > > > > > > > > > > ,mms > > > > > > > > > > > ys.y > > > > > > > > > > > aml > > > > > > > > > > > index b3c6888c1457..4e9acd966aa5 100644 > > > > > > > > > > > --- > > > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/ > > > > > > > > > > > medi > > > > > > > > > > > atek > > > > > > > > > > > ,mms > > > > > > > > > > > ys.y > > > > > > > > > > > aml > > > > > > > > > > > +++ > > > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/ > > > > > > > > > > > medi > > > > > > > > > > > atek > > > > > > > > > > > ,mms > > > > > > > > > > > ys.y > > > > > > > > > > > aml > > > > > > > > > > > @@ -93,6 +93,29 @@ properties: > > > > > > > > > > > '#reset-cells': > > > > > > > > > > > const: 1 > > > > > > > > > > > > > > > > > > > > > > + port: > > > > > > > > > > > + $ref: /schemas/graph.yaml#/properties/port > > > > > > > > > > > + description: > > > > > > > > > > > + Output port node. This port connects the > > > > > > > > > > > MMSYS/VDOSYS > > > > > > > > > > > output > > > > > > > > > > > to > > > > > > > > > > > + the first component of one display > > > > > > > > > > > pipeline, > > > > > > > > > > > for > > > > > > > > > > > example > > > > > > > > > > > one > > > > > > > > > > > of > > > > > > > > > > > + the available OVL or RDMA blocks. > > > > > > > > > > > + Some MediaTek SoCs support up to three > > > > > > > > > > > display > > > > > > > > > > > outputs > > > > > > > > > > > per > > > > > > > > > > > MMSYS. > > > > > > > > > > > + properties: > > > > > > > > > > > + endpoint@0: > > > > > > > > > > > + $ref: > > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > > > + description: Output to the primary > > > > > > > > > > > display > > > > > > > > > > > pipeline > > > > > > > > > > > + > > > > > > > > > > > + endpoint@1: > > > > > > > > > > > + $ref: > > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > > > + description: Output to the secondary > > > > > > > > > > > display > > > > > > > > > > > pipeline > > > > > > > > > > > + > > > > > > > > > > > + endpoint@2: > > > > > > > > > > > + $ref: > > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint > > > > > > > > > > > + description: Output to the tertiary > > > > > > > > > > > display > > > > > > > > > > > pipeline > > > > > > > > > > > + > > > > > > > > > > > + required: > > > > > > > > > > > + - endpoint@0 > > > > > > > > > > > + > > > > > > > > > > > > > > > > > > > > mmsys/vdosys does not output data to the first > > > > > > > > > > component of > > > > > > > > > > display > > > > > > > > > > pipeline, so this connection looks 'virtual'. Shall > > > > > > > > > > we > > > > > > > > > > add > > > > > > > > > > something > > > > > > > > > > virtual in device tree? You add this in order to > > > > > > > > > > decide > > > > > > > > > > which > > > > > > > > > > pipeline > > > > > > > > > > is 1st, 2nd, 3rd, but for device it don't care > > > > > > > > > > which > > > > > > > > > > one is > > > > > > > > > > first. > > > > > > > > > > In > > > > > > > > > > computer, software could change which display is > > > > > > > > > > the > > > > > > > > > > primary > > > > > > > > > > display. > > > > > > > > > > I'm not sure it's good to decide display order in > > > > > > > > > > device > > > > > > > > > > tree? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Devicetree describes hardware, so nothing virtual can > > > > > > > > > be > > > > > > > > > present > > > > > > > > > - > > > > > > > > > and in any case, > > > > > > > > > the primary/secondary/tertiary pipeline is in > > > > > > > > > relation to > > > > > > > > > MM/VDO > > > > > > > > > SYS, > > > > > > > > > not referred > > > > > > > > > to software. > > > > > > > > > > > > > > > > > > Better explaining, the primary pipeline is not > > > > > > > > > necessarily > > > > > > > > > the > > > > > > > > > primary display in > > > > > > > > > DRM terms: that's a concept that is completely > > > > > > > > > detached > > > > > > > > > from > > > > > > > > > the > > > > > > > > > scope of this > > > > > > > > > series and this graph - and it's something that shall > > > > > > > > > be > > > > > > > > > managed > > > > > > > > > solely by the > > > > > > > > > driver (mediatek-drm in this case). > > > > > > > > > > > > > > > > > > Coming back to the connection looking, but *not* > > > > > > > > > being > > > > > > > > > virtual: > > > > > > > > > the > > > > > > > > > sense here is > > > > > > > > > that the MM/VDOSYS blocks are used in the display > > > > > > > > > pipeline to > > > > > > > > > "stitch" together > > > > > > > > > the various display pipeline hardware blocks, or, > > > > > > > > > said > > > > > > > > > differently, > > > > > > > > > setting up the > > > > > > > > > routing between all of those (P.S.: > > > > > > > > > mmsys_mtxxxx_routing_table!) > > > > > > > > > through the VDO > > > > > > > > > Input Selection (VDOx_SEL_IN) or Output Selection > > > > > > > > > (VDOx_SEL_OUT) > > > > > > > > > and > > > > > > > > > with the > > > > > > > > > assistance of the VDO Multiple Output Mask > > > > > > > > > (VDOx_MOUT) > > > > > > > > > for > > > > > > > > > the > > > > > > > > > multiple outputs > > > > > > > > > usecase, both of which, are described by this graph. > > > > > > > > > > > > > > > > I agree this part, but this is related to display > > > > > > > > device OF > > > > > > > > graph. > > > > > > > > These display device would output video data from one > > > > > > > > device > > > > > > > > and > > > > > > > > input > > > > > > > > to another video device. These video device would not > > > > > > > > input > > > > > > > > or > > > > > > > > output > > > > > > > > video data to mmsys/vdosys. > > > > > > > > > > > > > > > > > > > > > > > > > > This means that the VDOSYS is really the "master" of > > > > > > > > > the > > > > > > > > > display > > > > > > > > > pipeline since > > > > > > > > > everything gets enabled, mixed and matched from there > > > > > > > > > - > > > > > > > > > and > > > > > > > > > that's in > > > > > > > > > the sense > > > > > > > > > of hardware operation, so we are *really* (and not > > > > > > > > > virtually!) > > > > > > > > > flipping switches. > > > > > > > > > > > > > > > > I agree mmsys/vdosys is master of video pipeline, so > > > > > > > > let's > > > > > > > > define > > > > > > > > what > > > > > > > > the port in mmsys/vdosys is. If the port means the > > > > > > > > master > > > > > > > > relationship, > > > > > > > > mmsys/vdosys should output port to every display > > > > > > > > device. Or > > > > > > > > use > > > > > > > > a > > > > > > > > simply way to show the master relation ship > > > > > > > > > > > > > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, > > > > > > > > &rdma1, > > > > > > > > &color1, > > > > > > > > ...>; > > > > > > > > > > > > > > > > > > > > > > There's no need to list all of the VDO0/VDO1/mmsys > > > > > > > devices in > > > > > > > one > > > > > > > big > > > > > > > array > > > > > > > property, because the actual possible devices can be > > > > > > > defined: > > > > > > > 1. In the bindings; and > > > > > > > 2. In the actual OF graph that we write for each > > > > > > > SoC+board > > > > > > > combination. > > > > > > > > > > > > > > A graph cannot contain a connection to a device that > > > > > > > cannot > > > > > > > be > > > > > > > connected to > > > > > > > the previous, so, your "mmsys-subdev" list can be > > > > > > > retrieved > > > > > > > by > > > > > > > looking at the > > > > > > > graph: > > > > > > > - Start from VDO0/1 or MMSYS > > > > > > > - Walk through (visually, even) OUTPUT ports > > > > > > > - VDO0 (read output ep) -> ovl0 (read output ep) > > > > > > > -> > > > > > > > rdma0 > > > > > > > (read > > > > > > > output ep) -> > > > > > > > color0 (...) -> etc > > > > > > > - Nothing more - it's all defined there. > > > > > > > > > > > > > > > > > > > > > > > Another problem is how to group display device? If two > > > > > > > > pipeline > > > > > > > > could > > > > > > > > be route to the same display interface, such as > > > > > > > > > > > > > > > > rdma0 -> dsi > > > > > > > > rdma1 -> dsi > > > > > > > > > > > > > > > > Would this be single group? > > > > > > > > > > > > > > There are multiple ways of doing this, but one that comes > > > > > > > to > > > > > > > my > > > > > > > mind > > > > > > > right now and > > > > > > > that looks clean as well is the following: > > > > > > > > > > > > > > ovl0@ef01 { > > > > > > > ..... > > > > > > > ports { > > > > > > > port@0 { > > > > > > > reg = <0>; > > > > > > > ovl0_in: endpoint { > > > > > > > remote-endpoint = <&vdosys0_out>; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > I'm not sure how do you define this port from OVL to > > > > > > vdosys. If > > > > > > this > > > > > > port means 'master relationship', others could add port in > > > > > > COLOR to > > > > > > point to vdosys because COLOR and vdosys has the 'master > > > > > > relationship' > > > > > > and I could not reject this. So we need more specific > > > > > > definition of > > > > > > this port. > > > > > > > > > > > > > > > > Only the 'first' device in pipeline could have this port? > > > > > > > > > > Correct. Only the first device in a pipeline - and this is > > > > > actually a > > > > > restriction > > > > > that the generic binding definition of port already gives, in > > > > > a > > > > > way. > > > > > > > > > > > > > > > > In mt8173, one pipeline is > > > > > > > > > > > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi > > > > > > > > > > > > But rdma has an option to read data from od or directly > > > > > > from > > > > > > DRAM. > > > > > > If > > > > > > from DRAM, the pipeline would be changed to > > > > > > > > > > > > rdma -> ufo -> dsi > > > > > > > > > > > > > > > > > > So it's confused which one is 'first'. > > > > > > > > > > That's why the pipeline is *board-specific* and not soc- > > > > > generic! > > > > > > > > > > And what you described is *exactly* the reason why I'm adding > > > > > support > > > > > for the > > > > > OF graphs in mediatek-drm: specifying the correct pipeline > > > > > for > > > > > each > > > > > board as per > > > > > what each board wants to use (said differently: for each > > > > > board's > > > > > *capabilities*). > > > > > > > > > > So, if on a certain board you want to skip OD, you can hook > > > > > RDMA > > > > > up > > > > > directly to > > > > > MMSYS/VDOSYS. > > > > > > > > > > In MT8173, one pipeline for one board uses endpoints IN/OUT > > > > > like > > > > > this: > > > > > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > > > > > and for another board, endpoints will be like > > > > > > > > > > MMSYS -> RDMA -> UFO -> DSI > > > > > > > > > > ...which is the exact same as you described, and I think that > > > > > your > > > > > confusion comes > > > > > from the fact that you didn't put MMSYS at the beginning of > > > > > the > > > > > pipeline :-) > > > > > > > > In one board, both OVL and RDMA could switch dynamically. > > > > Because > > > > each > > > > one could be the first in one board, mmsys point to both ovl > > > > and > > > > rdma? > > > > > > > > > > No. > > > > > > MMSYS would still point ONLY to OVL, because OVL is the "earliest > > > point" > > > of the pipeline that this one board does support. > > > > > > In that case, RDMA being present at a later point in the pipeline > > > does not > > > matter and does not prevent us from *temporarily* skipping OVL- > > > COLOR- > > > AAL-OD > > > and going MMSYS->RDMA *directly*. > > > > > > Switching dynamically is a driver duty and will be 100% possible > > > (as > > > much > > > as it is right now) to dynamically switch OVL and RDMA as long as > > > both are > > > present in the pipeline. > > > > > > With this exact pipeline: > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA > > > and > > > MMSYS->RDMA as the driver itself *is allowed to* temporarily > > > ignore > > > part > > > of the pipeline. > > > > > > Please note that, in case it is needed (trust me on this: it's > > > not > > > needed) > > > a custom property in the endpoint node can always be introduced > > > later, so > > > that you can declare a node like > > > > > > endpoint@0 { > > > remote-endpoint = <&ovl0_in>; > > > mediatek,short-path = <&rdma0_in>; > > > }; > > > > > > ...but again, that's never going to be needed, as the driver > > > already > > > does > > > have knowledge of the fact that RDMA is in the pipeline, so it is > > > possible > > > to simply do a temporary override in the driver. > > > > > > What the OF Graph support does is to build the same arrays, that > > > we > > > currently > > > have hardcoded in mediatek-drm, by reading from device tree. > > > > > > Nothing else and nothing more - for now. > > > > > > Having the OF Graph support makes us able to also add new dual- > > > path > > > support > > > with more complicated connections than the current ones, without > > > any > > > problem > > > and, in many cases, without even modifying the bindings from what > > > I > > > provided > > > in this series. > > > > OK, please add the information we discussed into binding document > > to > > prevent anyone misusing this binding. > > > > Sorry CK, but the binding *does* already have this information inside > - not > in terms of "phrases", but in terms of restrictions of the binding... > > If you want, though, I can add this information in the description of > the > commit that is adding this binding, is that okay for you? I think what we discuss could be translated to restrictions. This is a restriction description: If one component has option to be first component or middle component of one pipeline, it's treated as middle component not first component. In mt8195, there are 8 vdo1_rdma. So each one is the first component of display pipeline. So the maximum display pipe number is not 3, maybe 8 or more. Regards, CK > > Thanks, > Angelo > > > Regards, > > CK > > > > > > > > Cheers, > > > Angelo > > > > > > > Regards, > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In case you need any *temporary override* on any board that > > > > > defines a > > > > > pipeline like > > > > > > > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI > > > > > > > > > > so that the pipeline *temporarily* becomes (for power > > > > > management, > > > > > or > > > > > for any other > > > > > reason) RDMA -> UFO -> DSI .... that's not a concern: the > > > > > graph > > > > > is > > > > > present, and it > > > > > is used to tell to the driver what is the regular pipeline to > > > > > use. > > > > > Eventual temporary overrides can be managed transparently > > > > > inside > > > > > of > > > > > the driver with > > > > > C code and no changes to the devicetree are required. > > > > > > > > > > > > > > > > I don't know how to decide which device could point to > > > > > > mmsys/vdosys. So > > > > > > please give a specific definition. > > > > > > > > > > > > > > > > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing > > > > > to a > > > > > device. > > > > > > > > > > So, mmsys/vdosys must point to the *first device in the > > > > > pipeline*. > > > > > > > > > > Any other doubt? > > > > > > > > > > Cheers, > > > > > Angelo > > > > > > > > > > > Regards, > > > > > > CK > > > > > > > > > > > > > > > > > > > > port@1 { > > > > > > > reg = <1>; > > > > > > > ovl0_out0: endpoint@0 { > > > > > > > remote-endpoint = <&rdma0_in>; > > > > > > > }; > > > > > > > ovl0_out1: endpoint@1 { > > > > > > > remote-endpoint = <&rdma1_in>; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > rdma0@1234 { > > > > > > > ..... > > > > > > > ports { > > > > > > > port@0 { > > > > > > > reg = <0>; > > > > > > > rdma0_in: endpoint { > > > > > > > remote-endpoint = <&ovl0_out0>; /* assuming > > > > > > > ovl0 > > > > > > > outputs to > > > > > > > rdma0...*/ > > > > > > > }; > > > > > > > }; > > > > > > > port@1 { > > > > > > > reg = <1>; > > > > > > > rdma0_out: endpoint@1 { > > > > > > > remote-endpoint = <&dsi_dual_intf0_in>; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > > > > > > > > rdma1@5678 { > > > > > > > ..... > > > > > > > ports { > > > > > > > port@0 { > > > > > > > reg = <0>; > > > > > > > rdma1_in: endpoint { > > > > > > > /* assuming ovl0 outputs to rdma1 as well... > > > > > > > can > > > > > > > be > > > > > > > something else. */ > > > > > > > remote-endpoint = <&ovl0_out1>; > > > > > > > }; > > > > > > > }; > > > > > > > port@1 { > > > > > > > reg = <1>; > > > > > > > rdma1_out: endpoint { > > > > > > > remote-endpoint = <&dsi_dual_intf1_in>; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > > > > > > > > dsi@9abcd { > > > > > > > ..... > > > > > > > ports { > > > > > > > port@0 { > > > > > > > reg = <0>; > > > > > > > /* Where endpoint@0 could be always DSI LEFT > > > > > > > CTRL */ > > > > > > > dsi_dual_intf0_in: endpoint@0 { > > > > > > > remote-endpoint = <&rdma0_out>; > > > > > > > }; > > > > > > > /* ...and @1 could be always DSI RIGHT CTRL */ > > > > > > > dsi_dual_intf1_in: endpoint@1 { > > > > > > > remote-endpoint = <&rdma1_out>; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > port@1 { > > > > > > > reg = <1>; > > > > > > > dsi0_out: endpoint { > > > > > > > remote-endpoint = <&dsi_panel_in>; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > }; > > > > > > > > > > > > > > ...for a dual-dsi panel, it'd be a similar graph. > > > > > > > > > > > > > > Cheers, > > > > > > > Angelo > > > > > > > > > > > > > > > > > > > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>; > > > > > > > > > > > > > > > > Or two group? > > > > > > > > > > > > > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; > > > > > > > > > > > > > > > > I think we should clearly define this. > > > > > > > > > > > > > > > > Regards, > > > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Angelo > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > CK > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > required: > > > > > > > > > > > - compatible > > > > > > > > > > > - reg > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Il 13/05/24 08:15, CK Hu (胡俊光) ha scritto: > On Fri, 2024-05-10 at 12:14 +0200, AngeloGioacchino Del Regno wrote: >> Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto: >>> On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: >>>>> On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno >>>>> wrote: >>>>>> Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: >>>>>>> On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del >>>>>>> Regno >>>>>>> wrote: >>>>>>>> Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: >>>>>>>>> On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del >>>>>>>>> Regno >>>>>>>>> wrote: >>>>>>>>>> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>>>>>>>>>> Hi, Angelo: >>>>>>>>>>> >>>>>>>>>>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino >>>>>>>>>>> Del >>>>>>>>>>> Regno >>>>>>>>>>> wrote: >>>>>>>>>>>> Document OF graph on MMSYS/VDOSYS: this supports >>>>>>>>>>>> up >>>>>>>>>>>> to >>>>>>>>>>>> three >>>>>>>>>>>> DDP >>>>>>>>>>>> paths >>>>>>>>>>>> per HW instance (so potentially up to six >>>>>>>>>>>> displays >>>>>>>>>>>> for >>>>>>>>>>>> multi- >>>>>>>>>>>> vdo >>>>>>>>>>>> SoCs). >>>>>>>>>>>> >>>>>>>>>>>> The MMSYS or VDOSYS is always the first component >>>>>>>>>>>> in >>>>>>>>>>>> the >>>>>>>>>>>> DDP >>>>>>>>>>>> pipeline, >>>>>>>>>>>> so it only supports an output port with multiple >>>>>>>>>>>> endpoints - >>>>>>>>>>>> where >>>>>>>>>>>> each >>>>>>>>>>>> endpoint defines the starting point for one of >>>>>>>>>>>> the >>>>>>>>>>>> (currently >>>>>>>>>>>> three) >>>>>>>>>>>> possible hardware paths. >>>>>>>>>>>> >>>>>>>>>>>> Signed-off-by: AngeloGioacchino Del Regno < >>>>>>>>>>>> angelogioacchino.delregno@collabora.com> >>>>>>>>>>>> --- >>>>>>>>>>>> .../bindings/arm/mediatek/mediatek,mmsys.ya >>>>>>>>>>>> ml | >>>>>>>>>>>> 23 >>>>>>>>>>>> +++++++++++++++++++ >>>>>>>>>>>> 1 file changed, 23 insertions(+) >>>>>>>>>>>> >>>>>>>>>>>> diff --git >>>>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> index b3c6888c1457..4e9acd966aa5 100644 >>>>>>>>>>>> --- >>>>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> +++ >>>>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> @@ -93,6 +93,29 @@ properties: >>>>>>>>>>>> '#reset-cells': >>>>>>>>>>>> const: 1 >>>>>>>>>>>> >>>>>>>>>>>> + port: >>>>>>>>>>>> + $ref: /schemas/graph.yaml#/properties/port >>>>>>>>>>>> + description: >>>>>>>>>>>> + Output port node. This port connects the >>>>>>>>>>>> MMSYS/VDOSYS >>>>>>>>>>>> output >>>>>>>>>>>> to >>>>>>>>>>>> + the first component of one display >>>>>>>>>>>> pipeline, >>>>>>>>>>>> for >>>>>>>>>>>> example >>>>>>>>>>>> one >>>>>>>>>>>> of >>>>>>>>>>>> + the available OVL or RDMA blocks. >>>>>>>>>>>> + Some MediaTek SoCs support up to three >>>>>>>>>>>> display >>>>>>>>>>>> outputs >>>>>>>>>>>> per >>>>>>>>>>>> MMSYS. >>>>>>>>>>>> + properties: >>>>>>>>>>>> + endpoint@0: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the primary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + endpoint@1: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the secondary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + endpoint@2: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the tertiary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + required: >>>>>>>>>>>> + - endpoint@0 >>>>>>>>>>>> + >>>>>>>>>>> >>>>>>>>>>> mmsys/vdosys does not output data to the first >>>>>>>>>>> component of >>>>>>>>>>> display >>>>>>>>>>> pipeline, so this connection looks 'virtual'. Shall >>>>>>>>>>> we >>>>>>>>>>> add >>>>>>>>>>> something >>>>>>>>>>> virtual in device tree? You add this in order to >>>>>>>>>>> decide >>>>>>>>>>> which >>>>>>>>>>> pipeline >>>>>>>>>>> is 1st, 2nd, 3rd, but for device it don't care >>>>>>>>>>> which >>>>>>>>>>> one is >>>>>>>>>>> first. >>>>>>>>>>> In >>>>>>>>>>> computer, software could change which display is >>>>>>>>>>> the >>>>>>>>>>> primary >>>>>>>>>>> display. >>>>>>>>>>> I'm not sure it's good to decide display order in >>>>>>>>>>> device >>>>>>>>>>> tree? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Devicetree describes hardware, so nothing virtual can >>>>>>>>>> be >>>>>>>>>> present >>>>>>>>>> - >>>>>>>>>> and in any case, >>>>>>>>>> the primary/secondary/tertiary pipeline is in >>>>>>>>>> relation to >>>>>>>>>> MM/VDO >>>>>>>>>> SYS, >>>>>>>>>> not referred >>>>>>>>>> to software. >>>>>>>>>> >>>>>>>>>> Better explaining, the primary pipeline is not >>>>>>>>>> necessarily >>>>>>>>>> the >>>>>>>>>> primary display in >>>>>>>>>> DRM terms: that's a concept that is completely >>>>>>>>>> detached >>>>>>>>>> from >>>>>>>>>> the >>>>>>>>>> scope of this >>>>>>>>>> series and this graph - and it's something that shall >>>>>>>>>> be >>>>>>>>>> managed >>>>>>>>>> solely by the >>>>>>>>>> driver (mediatek-drm in this case). >>>>>>>>>> >>>>>>>>>> Coming back to the connection looking, but *not* >>>>>>>>>> being >>>>>>>>>> virtual: >>>>>>>>>> the >>>>>>>>>> sense here is >>>>>>>>>> that the MM/VDOSYS blocks are used in the display >>>>>>>>>> pipeline to >>>>>>>>>> "stitch" together >>>>>>>>>> the various display pipeline hardware blocks, or, >>>>>>>>>> said >>>>>>>>>> differently, >>>>>>>>>> setting up the >>>>>>>>>> routing between all of those (P.S.: >>>>>>>>>> mmsys_mtxxxx_routing_table!) >>>>>>>>>> through the VDO >>>>>>>>>> Input Selection (VDOx_SEL_IN) or Output Selection >>>>>>>>>> (VDOx_SEL_OUT) >>>>>>>>>> and >>>>>>>>>> with the >>>>>>>>>> assistance of the VDO Multiple Output Mask >>>>>>>>>> (VDOx_MOUT) >>>>>>>>>> for >>>>>>>>>> the >>>>>>>>>> multiple outputs >>>>>>>>>> usecase, both of which, are described by this graph. >>>>>>>>> >>>>>>>>> I agree this part, but this is related to display >>>>>>>>> device OF >>>>>>>>> graph. >>>>>>>>> These display device would output video data from one >>>>>>>>> device >>>>>>>>> and >>>>>>>>> input >>>>>>>>> to another video device. These video device would not >>>>>>>>> input >>>>>>>>> or >>>>>>>>> output >>>>>>>>> video data to mmsys/vdosys. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> This means that the VDOSYS is really the "master" of >>>>>>>>>> the >>>>>>>>>> display >>>>>>>>>> pipeline since >>>>>>>>>> everything gets enabled, mixed and matched from there >>>>>>>>>> - >>>>>>>>>> and >>>>>>>>>> that's in >>>>>>>>>> the sense >>>>>>>>>> of hardware operation, so we are *really* (and not >>>>>>>>>> virtually!) >>>>>>>>>> flipping switches. >>>>>>>>> >>>>>>>>> I agree mmsys/vdosys is master of video pipeline, so >>>>>>>>> let's >>>>>>>>> define >>>>>>>>> what >>>>>>>>> the port in mmsys/vdosys is. If the port means the >>>>>>>>> master >>>>>>>>> relationship, >>>>>>>>> mmsys/vdosys should output port to every display >>>>>>>>> device. Or >>>>>>>>> use >>>>>>>>> a >>>>>>>>> simply way to show the master relation ship >>>>>>>>> >>>>>>>>> mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, >>>>>>>>> &rdma1, >>>>>>>>> &color1, >>>>>>>>> ...>; >>>>>>>>> >>>>>>>> >>>>>>>> There's no need to list all of the VDO0/VDO1/mmsys >>>>>>>> devices in >>>>>>>> one >>>>>>>> big >>>>>>>> array >>>>>>>> property, because the actual possible devices can be >>>>>>>> defined: >>>>>>>> 1. In the bindings; and >>>>>>>> 2. In the actual OF graph that we write for each >>>>>>>> SoC+board >>>>>>>> combination. >>>>>>>> >>>>>>>> A graph cannot contain a connection to a device that >>>>>>>> cannot >>>>>>>> be >>>>>>>> connected to >>>>>>>> the previous, so, your "mmsys-subdev" list can be >>>>>>>> retrieved >>>>>>>> by >>>>>>>> looking at the >>>>>>>> graph: >>>>>>>> - Start from VDO0/1 or MMSYS >>>>>>>> - Walk through (visually, even) OUTPUT ports >>>>>>>> - VDO0 (read output ep) -> ovl0 (read output ep) >>>>>>>> -> >>>>>>>> rdma0 >>>>>>>> (read >>>>>>>> output ep) -> >>>>>>>> color0 (...) -> etc >>>>>>>> - Nothing more - it's all defined there. >>>>>>>> >>>>>>>>> >>>>>>>>> Another problem is how to group display device? If two >>>>>>>>> pipeline >>>>>>>>> could >>>>>>>>> be route to the same display interface, such as >>>>>>>>> >>>>>>>>> rdma0 -> dsi >>>>>>>>> rdma1 -> dsi >>>>>>>>> >>>>>>>>> Would this be single group? >>>>>>>> >>>>>>>> There are multiple ways of doing this, but one that comes >>>>>>>> to >>>>>>>> my >>>>>>>> mind >>>>>>>> right now and >>>>>>>> that looks clean as well is the following: >>>>>>>> >>>>>>>> ovl0@ef01 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> ovl0_in: endpoint { >>>>>>>> remote-endpoint = <&vdosys0_out>; >>>>>>>> }; >>>>>>>> }; >>>>>>> >>>>>>> I'm not sure how do you define this port from OVL to >>>>>>> vdosys. If >>>>>>> this >>>>>>> port means 'master relationship', others could add port in >>>>>>> COLOR to >>>>>>> point to vdosys because COLOR and vdosys has the 'master >>>>>>> relationship' >>>>>>> and I could not reject this. So we need more specific >>>>>>> definition of >>>>>>> this port. >>>>>> >>>>>> >>>>>>> Only the 'first' device in pipeline could have this port? >>>>>> >>>>>> Correct. Only the first device in a pipeline - and this is >>>>>> actually a >>>>>> restriction >>>>>> that the generic binding definition of port already gives, in >>>>>> a >>>>>> way. >>>>>> >>>>>> >>>>>>> In mt8173, one pipeline is >>>>>>> >>>>>>> ovl -> color -> aal -> od -> rdma -> ufo -> dsi >>>>>>> >>>>>>> But rdma has an option to read data from od or directly >>>>>>> from >>>>>>> DRAM. >>>>>>> If >>>>>>> from DRAM, the pipeline would be changed to >>>>>>> >>>>>>> rdma -> ufo -> dsi >>>>>>> >>>>>>> >>>>>>> So it's confused which one is 'first'. >>>>>> >>>>>> That's why the pipeline is *board-specific* and not soc- >>>>>> generic! >>>>>> >>>>>> And what you described is *exactly* the reason why I'm adding >>>>>> support >>>>>> for the >>>>>> OF graphs in mediatek-drm: specifying the correct pipeline >>>>>> for >>>>>> each >>>>>> board as per >>>>>> what each board wants to use (said differently: for each >>>>>> board's >>>>>> *capabilities*). >>>>>> >>>>>> So, if on a certain board you want to skip OD, you can hook >>>>>> RDMA >>>>>> up >>>>>> directly to >>>>>> MMSYS/VDOSYS. >>>>>> >>>>>> In MT8173, one pipeline for one board uses endpoints IN/OUT >>>>>> like >>>>>> this: >>>>>> >>>>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>>>> >>>>>> and for another board, endpoints will be like >>>>>> >>>>>> MMSYS -> RDMA -> UFO -> DSI >>>>>> >>>>>> ...which is the exact same as you described, and I think that >>>>>> your >>>>>> confusion comes >>>>>> from the fact that you didn't put MMSYS at the beginning of >>>>>> the >>>>>> pipeline :-) >>>>> >>>>> In one board, both OVL and RDMA could switch dynamically. >>>>> Because >>>>> each >>>>> one could be the first in one board, mmsys point to both ovl >>>>> and >>>>> rdma? >>>>> >>>> >>>> No. >>>> >>>> MMSYS would still point ONLY to OVL, because OVL is the "earliest >>>> point" >>>> of the pipeline that this one board does support. >>>> >>>> In that case, RDMA being present at a later point in the pipeline >>>> does not >>>> matter and does not prevent us from *temporarily* skipping OVL- >>>> COLOR- >>>> AAL-OD >>>> and going MMSYS->RDMA *directly*. >>>> >>>> Switching dynamically is a driver duty and will be 100% possible >>>> (as >>>> much >>>> as it is right now) to dynamically switch OVL and RDMA as long as >>>> both are >>>> present in the pipeline. >>>> >>>> With this exact pipeline: >>>> >>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>> >>>> the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA >>>> and >>>> MMSYS->RDMA as the driver itself *is allowed to* temporarily >>>> ignore >>>> part >>>> of the pipeline. >>>> >>>> Please note that, in case it is needed (trust me on this: it's >>>> not >>>> needed) >>>> a custom property in the endpoint node can always be introduced >>>> later, so >>>> that you can declare a node like >>>> >>>> endpoint@0 { >>>> remote-endpoint = <&ovl0_in>; >>>> mediatek,short-path = <&rdma0_in>; >>>> }; >>>> >>>> ...but again, that's never going to be needed, as the driver >>>> already >>>> does >>>> have knowledge of the fact that RDMA is in the pipeline, so it is >>>> possible >>>> to simply do a temporary override in the driver. >>>> >>>> What the OF Graph support does is to build the same arrays, that >>>> we >>>> currently >>>> have hardcoded in mediatek-drm, by reading from device tree. >>>> >>>> Nothing else and nothing more - for now. >>>> >>>> Having the OF Graph support makes us able to also add new dual- >>>> path >>>> support >>>> with more complicated connections than the current ones, without >>>> any >>>> problem >>>> and, in many cases, without even modifying the bindings from what >>>> I >>>> provided >>>> in this series. >>> >>> OK, please add the information we discussed into binding document >>> to >>> prevent anyone misusing this binding. >>> >> >> Sorry CK, but the binding *does* already have this information inside >> - not >> in terms of "phrases", but in terms of restrictions of the binding... >> >> If you want, though, I can add this information in the description of >> the >> commit that is adding this binding, is that okay for you? > > I think what we discuss could be translated to restrictions. This is a > restriction description: > > If one component has option to be first component or middle component > of one pipeline, it's treated as middle component not first component. > > In mt8195, there are 8 vdo1_rdma. So each one is the first component of > display pipeline. So the maximum display pipe number is not 3, maybe 8 > or more. > I will add the information in the commit description for the next version of this series. Thanks, Angelo > Regards, > CK > > >> >> Thanks, >> Angelo >> >>> Regards, >>> CK >>> >>>> >>>> Cheers, >>>> Angelo >>>> >>>>> Regards, >>>>> CK >>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> In case you need any *temporary override* on any board that >>>>>> defines a >>>>>> pipeline like >>>>>> >>>>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>>>> >>>>>> so that the pipeline *temporarily* becomes (for power >>>>>> management, >>>>>> or >>>>>> for any other >>>>>> reason) RDMA -> UFO -> DSI .... that's not a concern: the >>>>>> graph >>>>>> is >>>>>> present, and it >>>>>> is used to tell to the driver what is the regular pipeline to >>>>>> use. >>>>>> Eventual temporary overrides can be managed transparently >>>>>> inside >>>>>> of >>>>>> the driver with >>>>>> C code and no changes to the devicetree are required. >>>>>> >>>>>> >>>>>>> I don't know how to decide which device could point to >>>>>>> mmsys/vdosys. So >>>>>>> please give a specific definition. >>>>>>> >>>>>> >>>>>> Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing >>>>>> to a >>>>>> device. >>>>>> >>>>>> So, mmsys/vdosys must point to the *first device in the >>>>>> pipeline*. >>>>>> >>>>>> Any other doubt? >>>>>> >>>>>> Cheers, >>>>>> Angelo >>>>>> >>>>>>> Regards, >>>>>>> CK >>>>>>> >>>>>>>> >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> ovl0_out0: endpoint@0 { >>>>>>>> remote-endpoint = <&rdma0_in>; >>>>>>>> }; >>>>>>>> ovl0_out1: endpoint@1 { >>>>>>>> remote-endpoint = <&rdma1_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> rdma0@1234 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> rdma0_in: endpoint { >>>>>>>> remote-endpoint = <&ovl0_out0>; /* assuming >>>>>>>> ovl0 >>>>>>>> outputs to >>>>>>>> rdma0...*/ >>>>>>>> }; >>>>>>>> }; >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> rdma0_out: endpoint@1 { >>>>>>>> remote-endpoint = <&dsi_dual_intf0_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> >>>>>>>> rdma1@5678 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> rdma1_in: endpoint { >>>>>>>> /* assuming ovl0 outputs to rdma1 as well... >>>>>>>> can >>>>>>>> be >>>>>>>> something else. */ >>>>>>>> remote-endpoint = <&ovl0_out1>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> rdma1_out: endpoint { >>>>>>>> remote-endpoint = <&dsi_dual_intf1_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> >>>>>>>> dsi@9abcd { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> /* Where endpoint@0 could be always DSI LEFT >>>>>>>> CTRL */ >>>>>>>> dsi_dual_intf0_in: endpoint@0 { >>>>>>>> remote-endpoint = <&rdma0_out>; >>>>>>>> }; >>>>>>>> /* ...and @1 could be always DSI RIGHT CTRL */ >>>>>>>> dsi_dual_intf1_in: endpoint@1 { >>>>>>>> remote-endpoint = <&rdma1_out>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> dsi0_out: endpoint { >>>>>>>> remote-endpoint = <&dsi_panel_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> ...for a dual-dsi panel, it'd be a similar graph. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Angelo >>>>>>>> >>>>>>>>> >>>>>>>>> mmsys-subdev = <&rdma0, &rdma1, &dsi>; >>>>>>>>> >>>>>>>>> Or two group? >>>>>>>>> >>>>>>>>> mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; >>>>>>>>> >>>>>>>>> I think we should clearly define this. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> CK >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Angelo >>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> CK >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> required: >>>>>>>>>>>> - compatible >>>>>>>>>>>> - reg >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >>
On Mon, May 13, 2024 at 03:44:00PM +0200, AngeloGioacchino Del Regno wrote: > Il 13/05/24 08:15, CK Hu (胡俊光) ha scritto: > > On Fri, 2024-05-10 at 12:14 +0200, AngeloGioacchino Del Regno wrote: > > > Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto: > > > > On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno > > > > wrote: > > > > > Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: > > > > > > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno > > > > > > wrote: > > > > > > > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: > > > > > > > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del > > > > > > > > Regno > > > > > > > > wrote: > > > > > > > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: > > > > > > > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del > > > > > > > > > > Regno > > > > > > > > > > wrote: > > > > > > > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: > > > > > > > > > > > > Hi, Angelo: > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino > > > > > > > > > > > > Del > > > > > > > > > > > > Regno > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports > > > > > > > > > > > > > up > > > > > > > > > > > > > to > > > > > > > > > > > > > three > > > > > > > > > > > > > DDP > > > > > > > > > > > > > paths > > > > > > > > > > > > > per HW instance (so potentially up to six > > > > > > > > > > > > > displays > > > > > > > > > > > > > for > > > > > > > > > > > > > multi- > > > > > > > > > > > > > vdo > > > > > > > > > > > > > SoCs). > > > > > > > > > > > > > > > > > > > > > > > > > > The MMSYS or VDOSYS is always the first component > > > > > > > > > > > > > in > > > > > > > > > > > > > the > > > > > > > > > > > > > DDP > > > > > > > > > > > > > pipeline, > > > > > > > > > > > > > so it only supports an output port with multiple > > > > > > > > > > > > > endpoints - > > > > > > > > > > > > > where > > > > > > > > > > > > > each > > > > > > > > > > > > > endpoint defines the starting point for one of > > > > > > > > > > > > > the > > > > > > > > > > > > > (currently > > > > > > > > > > > > > three) > > > > > > > > > > > > > possible hardware paths. > > > > > > > > > > > > > > > > > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno < > > > > > > > > > > > > > angelogioacchino.delregno@collabora.com> One of you guys, probably 胡俊光, has a mail client that is completely mangling these quotes. Can you try to fix that please? It makes reading the thread unfun :/
Il 13/05/24 08:15, CK Hu (胡俊光) ha scritto: > On Fri, 2024-05-10 at 12:14 +0200, AngeloGioacchino Del Regno wrote: >> Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto: >>> On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno >>> wrote: >>>> Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto: >>>>> On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno >>>>> wrote: >>>>>> Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto: >>>>>>> On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del >>>>>>> Regno >>>>>>> wrote: >>>>>>>> Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto: >>>>>>>>> On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del >>>>>>>>> Regno >>>>>>>>> wrote: >>>>>>>>>> Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto: >>>>>>>>>>> Hi, Angelo: >>>>>>>>>>> >>>>>>>>>>> On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino >>>>>>>>>>> Del >>>>>>>>>>> Regno >>>>>>>>>>> wrote: >>>>>>>>>>>> Document OF graph on MMSYS/VDOSYS: this supports >>>>>>>>>>>> up >>>>>>>>>>>> to >>>>>>>>>>>> three >>>>>>>>>>>> DDP >>>>>>>>>>>> paths >>>>>>>>>>>> per HW instance (so potentially up to six >>>>>>>>>>>> displays >>>>>>>>>>>> for >>>>>>>>>>>> multi- >>>>>>>>>>>> vdo >>>>>>>>>>>> SoCs). >>>>>>>>>>>> >>>>>>>>>>>> The MMSYS or VDOSYS is always the first component >>>>>>>>>>>> in >>>>>>>>>>>> the >>>>>>>>>>>> DDP >>>>>>>>>>>> pipeline, >>>>>>>>>>>> so it only supports an output port with multiple >>>>>>>>>>>> endpoints - >>>>>>>>>>>> where >>>>>>>>>>>> each >>>>>>>>>>>> endpoint defines the starting point for one of >>>>>>>>>>>> the >>>>>>>>>>>> (currently >>>>>>>>>>>> three) >>>>>>>>>>>> possible hardware paths. >>>>>>>>>>>> >>>>>>>>>>>> Signed-off-by: AngeloGioacchino Del Regno < >>>>>>>>>>>> angelogioacchino.delregno@collabora.com> >>>>>>>>>>>> --- >>>>>>>>>>>> .../bindings/arm/mediatek/mediatek,mmsys.ya >>>>>>>>>>>> ml | >>>>>>>>>>>> 23 >>>>>>>>>>>> +++++++++++++++++++ >>>>>>>>>>>> 1 file changed, 23 insertions(+) >>>>>>>>>>>> >>>>>>>>>>>> diff --git >>>>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> index b3c6888c1457..4e9acd966aa5 100644 >>>>>>>>>>>> --- >>>>>>>>>>>> a/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> +++ >>>>>>>>>>>> b/Documentation/devicetree/bindings/arm/mediatek/ >>>>>>>>>>>> medi >>>>>>>>>>>> atek >>>>>>>>>>>> ,mms >>>>>>>>>>>> ys.y >>>>>>>>>>>> aml >>>>>>>>>>>> @@ -93,6 +93,29 @@ properties: >>>>>>>>>>>> '#reset-cells': >>>>>>>>>>>> const: 1 >>>>>>>>>>>> >>>>>>>>>>>> + port: >>>>>>>>>>>> + $ref: /schemas/graph.yaml#/properties/port >>>>>>>>>>>> + description: >>>>>>>>>>>> + Output port node. This port connects the >>>>>>>>>>>> MMSYS/VDOSYS >>>>>>>>>>>> output >>>>>>>>>>>> to >>>>>>>>>>>> + the first component of one display >>>>>>>>>>>> pipeline, >>>>>>>>>>>> for >>>>>>>>>>>> example >>>>>>>>>>>> one >>>>>>>>>>>> of >>>>>>>>>>>> + the available OVL or RDMA blocks. >>>>>>>>>>>> + Some MediaTek SoCs support up to three >>>>>>>>>>>> display >>>>>>>>>>>> outputs >>>>>>>>>>>> per >>>>>>>>>>>> MMSYS. >>>>>>>>>>>> + properties: >>>>>>>>>>>> + endpoint@0: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the primary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + endpoint@1: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the secondary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + endpoint@2: >>>>>>>>>>>> + $ref: >>>>>>>>>>>> /schemas/graph.yaml#/properties/endpoint >>>>>>>>>>>> + description: Output to the tertiary >>>>>>>>>>>> display >>>>>>>>>>>> pipeline >>>>>>>>>>>> + >>>>>>>>>>>> + required: >>>>>>>>>>>> + - endpoint@0 >>>>>>>>>>>> + >>>>>>>>>>> >>>>>>>>>>> mmsys/vdosys does not output data to the first >>>>>>>>>>> component of >>>>>>>>>>> display >>>>>>>>>>> pipeline, so this connection looks 'virtual'. Shall >>>>>>>>>>> we >>>>>>>>>>> add >>>>>>>>>>> something >>>>>>>>>>> virtual in device tree? You add this in order to >>>>>>>>>>> decide >>>>>>>>>>> which >>>>>>>>>>> pipeline >>>>>>>>>>> is 1st, 2nd, 3rd, but for device it don't care >>>>>>>>>>> which >>>>>>>>>>> one is >>>>>>>>>>> first. >>>>>>>>>>> In >>>>>>>>>>> computer, software could change which display is >>>>>>>>>>> the >>>>>>>>>>> primary >>>>>>>>>>> display. >>>>>>>>>>> I'm not sure it's good to decide display order in >>>>>>>>>>> device >>>>>>>>>>> tree? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Devicetree describes hardware, so nothing virtual can >>>>>>>>>> be >>>>>>>>>> present >>>>>>>>>> - >>>>>>>>>> and in any case, >>>>>>>>>> the primary/secondary/tertiary pipeline is in >>>>>>>>>> relation to >>>>>>>>>> MM/VDO >>>>>>>>>> SYS, >>>>>>>>>> not referred >>>>>>>>>> to software. >>>>>>>>>> >>>>>>>>>> Better explaining, the primary pipeline is not >>>>>>>>>> necessarily >>>>>>>>>> the >>>>>>>>>> primary display in >>>>>>>>>> DRM terms: that's a concept that is completely >>>>>>>>>> detached >>>>>>>>>> from >>>>>>>>>> the >>>>>>>>>> scope of this >>>>>>>>>> series and this graph - and it's something that shall >>>>>>>>>> be >>>>>>>>>> managed >>>>>>>>>> solely by the >>>>>>>>>> driver (mediatek-drm in this case). >>>>>>>>>> >>>>>>>>>> Coming back to the connection looking, but *not* >>>>>>>>>> being >>>>>>>>>> virtual: >>>>>>>>>> the >>>>>>>>>> sense here is >>>>>>>>>> that the MM/VDOSYS blocks are used in the display >>>>>>>>>> pipeline to >>>>>>>>>> "stitch" together >>>>>>>>>> the various display pipeline hardware blocks, or, >>>>>>>>>> said >>>>>>>>>> differently, >>>>>>>>>> setting up the >>>>>>>>>> routing between all of those (P.S.: >>>>>>>>>> mmsys_mtxxxx_routing_table!) >>>>>>>>>> through the VDO >>>>>>>>>> Input Selection (VDOx_SEL_IN) or Output Selection >>>>>>>>>> (VDOx_SEL_OUT) >>>>>>>>>> and >>>>>>>>>> with the >>>>>>>>>> assistance of the VDO Multiple Output Mask >>>>>>>>>> (VDOx_MOUT) >>>>>>>>>> for >>>>>>>>>> the >>>>>>>>>> multiple outputs >>>>>>>>>> usecase, both of which, are described by this graph. >>>>>>>>> >>>>>>>>> I agree this part, but this is related to display >>>>>>>>> device OF >>>>>>>>> graph. >>>>>>>>> These display device would output video data from one >>>>>>>>> device >>>>>>>>> and >>>>>>>>> input >>>>>>>>> to another video device. These video device would not >>>>>>>>> input >>>>>>>>> or >>>>>>>>> output >>>>>>>>> video data to mmsys/vdosys. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> This means that the VDOSYS is really the "master" of >>>>>>>>>> the >>>>>>>>>> display >>>>>>>>>> pipeline since >>>>>>>>>> everything gets enabled, mixed and matched from there >>>>>>>>>> - >>>>>>>>>> and >>>>>>>>>> that's in >>>>>>>>>> the sense >>>>>>>>>> of hardware operation, so we are *really* (and not >>>>>>>>>> virtually!) >>>>>>>>>> flipping switches. >>>>>>>>> >>>>>>>>> I agree mmsys/vdosys is master of video pipeline, so >>>>>>>>> let's >>>>>>>>> define >>>>>>>>> what >>>>>>>>> the port in mmsys/vdosys is. If the port means the >>>>>>>>> master >>>>>>>>> relationship, >>>>>>>>> mmsys/vdosys should output port to every display >>>>>>>>> device. Or >>>>>>>>> use >>>>>>>>> a >>>>>>>>> simply way to show the master relation ship >>>>>>>>> >>>>>>>>> mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1, >>>>>>>>> &rdma1, >>>>>>>>> &color1, >>>>>>>>> ...>; >>>>>>>>> >>>>>>>> >>>>>>>> There's no need to list all of the VDO0/VDO1/mmsys >>>>>>>> devices in >>>>>>>> one >>>>>>>> big >>>>>>>> array >>>>>>>> property, because the actual possible devices can be >>>>>>>> defined: >>>>>>>> 1. In the bindings; and >>>>>>>> 2. In the actual OF graph that we write for each >>>>>>>> SoC+board >>>>>>>> combination. >>>>>>>> >>>>>>>> A graph cannot contain a connection to a device that >>>>>>>> cannot >>>>>>>> be >>>>>>>> connected to >>>>>>>> the previous, so, your "mmsys-subdev" list can be >>>>>>>> retrieved >>>>>>>> by >>>>>>>> looking at the >>>>>>>> graph: >>>>>>>> - Start from VDO0/1 or MMSYS >>>>>>>> - Walk through (visually, even) OUTPUT ports >>>>>>>> - VDO0 (read output ep) -> ovl0 (read output ep) >>>>>>>> -> >>>>>>>> rdma0 >>>>>>>> (read >>>>>>>> output ep) -> >>>>>>>> color0 (...) -> etc >>>>>>>> - Nothing more - it's all defined there. >>>>>>>> >>>>>>>>> >>>>>>>>> Another problem is how to group display device? If two >>>>>>>>> pipeline >>>>>>>>> could >>>>>>>>> be route to the same display interface, such as >>>>>>>>> >>>>>>>>> rdma0 -> dsi >>>>>>>>> rdma1 -> dsi >>>>>>>>> >>>>>>>>> Would this be single group? >>>>>>>> >>>>>>>> There are multiple ways of doing this, but one that comes >>>>>>>> to >>>>>>>> my >>>>>>>> mind >>>>>>>> right now and >>>>>>>> that looks clean as well is the following: >>>>>>>> >>>>>>>> ovl0@ef01 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> ovl0_in: endpoint { >>>>>>>> remote-endpoint = <&vdosys0_out>; >>>>>>>> }; >>>>>>>> }; >>>>>>> >>>>>>> I'm not sure how do you define this port from OVL to >>>>>>> vdosys. If >>>>>>> this >>>>>>> port means 'master relationship', others could add port in >>>>>>> COLOR to >>>>>>> point to vdosys because COLOR and vdosys has the 'master >>>>>>> relationship' >>>>>>> and I could not reject this. So we need more specific >>>>>>> definition of >>>>>>> this port. >>>>>> >>>>>> >>>>>>> Only the 'first' device in pipeline could have this port? >>>>>> >>>>>> Correct. Only the first device in a pipeline - and this is >>>>>> actually a >>>>>> restriction >>>>>> that the generic binding definition of port already gives, in >>>>>> a >>>>>> way. >>>>>> >>>>>> >>>>>>> In mt8173, one pipeline is >>>>>>> >>>>>>> ovl -> color -> aal -> od -> rdma -> ufo -> dsi >>>>>>> >>>>>>> But rdma has an option to read data from od or directly >>>>>>> from >>>>>>> DRAM. >>>>>>> If >>>>>>> from DRAM, the pipeline would be changed to >>>>>>> >>>>>>> rdma -> ufo -> dsi >>>>>>> >>>>>>> >>>>>>> So it's confused which one is 'first'. >>>>>> >>>>>> That's why the pipeline is *board-specific* and not soc- >>>>>> generic! >>>>>> >>>>>> And what you described is *exactly* the reason why I'm adding >>>>>> support >>>>>> for the >>>>>> OF graphs in mediatek-drm: specifying the correct pipeline >>>>>> for >>>>>> each >>>>>> board as per >>>>>> what each board wants to use (said differently: for each >>>>>> board's >>>>>> *capabilities*). >>>>>> >>>>>> So, if on a certain board you want to skip OD, you can hook >>>>>> RDMA >>>>>> up >>>>>> directly to >>>>>> MMSYS/VDOSYS. >>>>>> >>>>>> In MT8173, one pipeline for one board uses endpoints IN/OUT >>>>>> like >>>>>> this: >>>>>> >>>>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>>>> >>>>>> and for another board, endpoints will be like >>>>>> >>>>>> MMSYS -> RDMA -> UFO -> DSI >>>>>> >>>>>> ...which is the exact same as you described, and I think that >>>>>> your >>>>>> confusion comes >>>>>> from the fact that you didn't put MMSYS at the beginning of >>>>>> the >>>>>> pipeline :-) >>>>> >>>>> In one board, both OVL and RDMA could switch dynamically. >>>>> Because >>>>> each >>>>> one could be the first in one board, mmsys point to both ovl >>>>> and >>>>> rdma? >>>>> >>>> >>>> No. >>>> >>>> MMSYS would still point ONLY to OVL, because OVL is the "earliest >>>> point" >>>> of the pipeline that this one board does support. >>>> >>>> In that case, RDMA being present at a later point in the pipeline >>>> does not >>>> matter and does not prevent us from *temporarily* skipping OVL- >>>> COLOR- >>>> AAL-OD >>>> and going MMSYS->RDMA *directly*. >>>> >>>> Switching dynamically is a driver duty and will be 100% possible >>>> (as >>>> much >>>> as it is right now) to dynamically switch OVL and RDMA as long as >>>> both are >>>> present in the pipeline. >>>> >>>> With this exact pipeline: >>>> >>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>> >>>> the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA >>>> and >>>> MMSYS->RDMA as the driver itself *is allowed to* temporarily >>>> ignore >>>> part >>>> of the pipeline. >>>> >>>> Please note that, in case it is needed (trust me on this: it's >>>> not >>>> needed) >>>> a custom property in the endpoint node can always be introduced >>>> later, so >>>> that you can declare a node like >>>> >>>> endpoint@0 { >>>> remote-endpoint = <&ovl0_in>; >>>> mediatek,short-path = <&rdma0_in>; >>>> }; >>>> >>>> ...but again, that's never going to be needed, as the driver >>>> already >>>> does >>>> have knowledge of the fact that RDMA is in the pipeline, so it is >>>> possible >>>> to simply do a temporary override in the driver. >>>> >>>> What the OF Graph support does is to build the same arrays, that >>>> we >>>> currently >>>> have hardcoded in mediatek-drm, by reading from device tree. >>>> >>>> Nothing else and nothing more - for now. >>>> >>>> Having the OF Graph support makes us able to also add new dual- >>>> path >>>> support >>>> with more complicated connections than the current ones, without >>>> any >>>> problem >>>> and, in many cases, without even modifying the bindings from what >>>> I >>>> provided >>>> in this series. >>> >>> OK, please add the information we discussed into binding document >>> to >>> prevent anyone misusing this binding. >>> >> >> Sorry CK, but the binding *does* already have this information inside >> - not >> in terms of "phrases", but in terms of restrictions of the binding... >> >> If you want, though, I can add this information in the description of >> the >> commit that is adding this binding, is that okay for you? > > I think what we discuss could be translated to restrictions. This is a > restriction description: > > If one component has option to be first component or middle component > of one pipeline, it's treated as middle component not first component. > This cannot be added to the binding description, as the binding describes hardware, and what we're talking about here is a *driver* behavior detail, not suitable for describing a binding by itself. Also, that's not true: if a component has option to be first or middle, it is going to be treated as per what you describe in the graph - if you place it as first, it's going to be first (unless temporary overrides are used in the driver which are, again, transparent to all this) otherwise, if you place it as middle, it's going to normally be treated as middle. Besides - to address your concern about misusing the graph... that's not possible because of multiple reasons: 1. The bindings won't pass validation - dtbs_check will give errors and/or warnings upon misuse of bindings in device trees 2. The driver simply won't work, as it's going to refuse probing if it detects an invalid graph (which corresponds to any binding misuse). > In mt8195, there are 8 vdo1_rdma. So each one is the first component of > display pipeline. So the maximum display pipe number is not 3, maybe 8 > or more. Yes but we currently only support three paths - adding more to the bindings later is trivial anyway, so I prefer to describe what is currently supported and what can currently be tested on the real (and commonly availlable) hardware, and not what might be, one day, maybe supported. Whenever any commonly available hardware supporting more than three paths will appear, I'll change the binding to do that, and it's going to be a 10 minutes job. Besides, I'm about to send a v4 of this series, containing some fixes for the multi-path support on all SoCs. Regards, Angelo > > Regards, > CK > > >> >> Thanks, >> Angelo >> >>> Regards, >>> CK >>> >>>> >>>> Cheers, >>>> Angelo >>>> >>>>> Regards, >>>>> CK >>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> In case you need any *temporary override* on any board that >>>>>> defines a >>>>>> pipeline like >>>>>> >>>>>> MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI >>>>>> >>>>>> so that the pipeline *temporarily* becomes (for power >>>>>> management, >>>>>> or >>>>>> for any other >>>>>> reason) RDMA -> UFO -> DSI .... that's not a concern: the >>>>>> graph >>>>>> is >>>>>> present, and it >>>>>> is used to tell to the driver what is the regular pipeline to >>>>>> use. >>>>>> Eventual temporary overrides can be managed transparently >>>>>> inside >>>>>> of >>>>>> the driver with >>>>>> C code and no changes to the devicetree are required. >>>>>> >>>>>> >>>>>>> I don't know how to decide which device could point to >>>>>>> mmsys/vdosys. So >>>>>>> please give a specific definition. >>>>>>> >>>>>> >>>>>> Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing >>>>>> to a >>>>>> device. >>>>>> >>>>>> So, mmsys/vdosys must point to the *first device in the >>>>>> pipeline*. >>>>>> >>>>>> Any other doubt? >>>>>> >>>>>> Cheers, >>>>>> Angelo >>>>>> >>>>>>> Regards, >>>>>>> CK >>>>>>> >>>>>>>> >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> ovl0_out0: endpoint@0 { >>>>>>>> remote-endpoint = <&rdma0_in>; >>>>>>>> }; >>>>>>>> ovl0_out1: endpoint@1 { >>>>>>>> remote-endpoint = <&rdma1_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> rdma0@1234 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> rdma0_in: endpoint { >>>>>>>> remote-endpoint = <&ovl0_out0>; /* assuming >>>>>>>> ovl0 >>>>>>>> outputs to >>>>>>>> rdma0...*/ >>>>>>>> }; >>>>>>>> }; >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> rdma0_out: endpoint@1 { >>>>>>>> remote-endpoint = <&dsi_dual_intf0_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> >>>>>>>> rdma1@5678 { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> rdma1_in: endpoint { >>>>>>>> /* assuming ovl0 outputs to rdma1 as well... >>>>>>>> can >>>>>>>> be >>>>>>>> something else. */ >>>>>>>> remote-endpoint = <&ovl0_out1>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> rdma1_out: endpoint { >>>>>>>> remote-endpoint = <&dsi_dual_intf1_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> >>>>>>>> dsi@9abcd { >>>>>>>> ..... >>>>>>>> ports { >>>>>>>> port@0 { >>>>>>>> reg = <0>; >>>>>>>> /* Where endpoint@0 could be always DSI LEFT >>>>>>>> CTRL */ >>>>>>>> dsi_dual_intf0_in: endpoint@0 { >>>>>>>> remote-endpoint = <&rdma0_out>; >>>>>>>> }; >>>>>>>> /* ...and @1 could be always DSI RIGHT CTRL */ >>>>>>>> dsi_dual_intf1_in: endpoint@1 { >>>>>>>> remote-endpoint = <&rdma1_out>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> port@1 { >>>>>>>> reg = <1>; >>>>>>>> dsi0_out: endpoint { >>>>>>>> remote-endpoint = <&dsi_panel_in>; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> }; >>>>>>>> >>>>>>>> ...for a dual-dsi panel, it'd be a similar graph. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Angelo >>>>>>>> >>>>>>>>> >>>>>>>>> mmsys-subdev = <&rdma0, &rdma1, &dsi>; >>>>>>>>> >>>>>>>>> Or two group? >>>>>>>>> >>>>>>>>> mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>; >>>>>>>>> >>>>>>>>> I think we should clearly define this. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> CK >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Angelo >>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> CK >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> required: >>>>>>>>>>>> - compatible >>>>>>>>>>>> - reg >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >>
diff --git a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml index b3c6888c1457..4e9acd966aa5 100644 --- a/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml +++ b/Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml @@ -93,6 +93,29 @@ properties: '#reset-cells': const: 1 + port: + $ref: /schemas/graph.yaml#/properties/port + description: + Output port node. This port connects the MMSYS/VDOSYS output to + the first component of one display pipeline, for example one of + the available OVL or RDMA blocks. + Some MediaTek SoCs support up to three display outputs per MMSYS. + properties: + endpoint@0: + $ref: /schemas/graph.yaml#/properties/endpoint + description: Output to the primary display pipeline + + endpoint@1: + $ref: /schemas/graph.yaml#/properties/endpoint + description: Output to the secondary display pipeline + + endpoint@2: + $ref: /schemas/graph.yaml#/properties/endpoint + description: Output to the tertiary display pipeline + + required: + - endpoint@0 + required: - compatible - reg
Document OF graph on MMSYS/VDOSYS: this supports up to three DDP paths per HW instance (so potentially up to six displays for multi-vdo SoCs). The MMSYS or VDOSYS is always the first component in the DDP pipeline, so it only supports an output port with multiple endpoints - where each endpoint defines the starting point for one of the (currently three) possible hardware paths. Signed-off-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> --- .../bindings/arm/mediatek/mediatek,mmsys.yaml | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+)