| Message ID | 20240112234222.913138-1-anatoliy.klymenko@amd.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Fixing live video input in ZynqMP DPSUB | expand |
On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > Patches 1/4,2/4,3/4 are minor fixes. > > DPSUB requires input live video format to be configured. > Patch 4/4: The DP Subsystem requires the input live video format to be > configured. In this patch we are assuming that the CRTC's bus format is fixed > and comes from the device tree. This is a proposed solution, as there are no api > to query CRTC output bus format. > > Is this a good approach to go with? I guess you would need to expand a bit on what "live video input" is? Is it some kind of mechanism to bypass memory and take your pixels straight from a FIFO from another device, or something else? Maxime
On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > Patches 1/4,2/4,3/4 are minor fixes. > > > > DPSUB requires input live video format to be configured. > > Patch 4/4: The DP Subsystem requires the input live video format to be > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > and comes from the device tree. This is a proposed solution, as there are no api > > to query CRTC output bus format. > > > > Is this a good approach to go with? > > I guess you would need to expand a bit on what "live video input" is? Is > it some kind of mechanism to bypass memory and take your pixels straight > from a FIFO from another device, or something else? Yes and no. The DPSUB integrates DMA engines, a blending engine (two planes), and a DP encoder. The dpsub driver supports all of this, and creates a DRM device. The DP encoder hardware always takes its input data from the output of the blending engine. The blending engine can optionally take input data from a bus connected to the FPGA fabric, instead of taking it from the DPSUB internal DMA engines. When operating in that mode, the dpsub driver exposes the DP encoder as a bridge, and internally programs the blending engine to disable blending. Typically, the FPGA fabric will then contain a CRTC of some sort, with a driver that will acquire the DP encoder bridge as usually done. In this mode of operation, it is typical for the IP cores in FPGA fabric to be synthesized with a fixed format (as that saves resources), while the DPSUB supports multiple input formats. Bridge drivers in the upstream kernel work the other way around, with the bridge hardware supporting a limited set of formats, and the CRTC then being programmed with whatever the bridges chain needs. Here, the negotiation needs to go the other way around, as the CRTC is the limiting factor, not the bridge. Is this explanation clear ?
On Wed, Jan 17, 2024 at 04:23:43PM +0200, Laurent Pinchart wrote: > On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > > Patches 1/4,2/4,3/4 are minor fixes. > > > > > > DPSUB requires input live video format to be configured. > > > Patch 4/4: The DP Subsystem requires the input live video format to be > > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > > and comes from the device tree. This is a proposed solution, as there are no api > > > to query CRTC output bus format. > > > > > > Is this a good approach to go with? > > > > I guess you would need to expand a bit on what "live video input" is? Is > > it some kind of mechanism to bypass memory and take your pixels straight > > from a FIFO from another device, or something else? > > Yes and no. > > The DPSUB integrates DMA engines, a blending engine (two planes), and a > DP encoder. The dpsub driver supports all of this, and creates a DRM > device. The DP encoder hardware always takes its input data from the > output of the blending engine. > > The blending engine can optionally take input data from a bus connected > to the FPGA fabric, instead of taking it from the DPSUB internal DMA > engines. When operating in that mode, the dpsub driver exposes the DP > encoder as a bridge, and internally programs the blending engine to > disable blending. Typically, the FPGA fabric will then contain a CRTC of > some sort, with a driver that will acquire the DP encoder bridge as > usually done. > > In this mode of operation, it is typical for the IP cores in FPGA fabric > to be synthesized with a fixed format (as that saves resources), while > the DPSUB supports multiple input formats. Where is that CRTC driver? It's not clear to me why the format would need to be in the device tree at all. Format negociation between the CRTC and whatever comes next is already done in a number of drivers so it would be useful to have that kind of API outside of the bridge support. > Bridge drivers in the upstream kernel work the other way around, with > the bridge hardware supporting a limited set of formats, and the CRTC > then being programmed with whatever the bridges chain needs. Here, the > negotiation needs to go the other way around, as the CRTC is the > limiting factor, not the bridge. Sounds like there's something to rework in the API then? Maxime
> -----Original Message----- > From: Maxime Ripard <mripard@kernel.org> > Sent: Friday, January 26, 2024 4:26 AM > To: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > Cc: Klymenko, Anatoliy <Anatoliy.Klymenko@amd.com>; > maarten.lankhorst@linux.intel.com; tzimmermann@suse.de; airlied@gmail.com; > daniel@ffwll.ch; Simek, Michal <michal.simek@amd.com>; dri- > devel@lists.freedesktop.org; linux-arm-kernel@lists.infradead.org; linux- > kernel@vger.kernel.org > Subject: Re: Re: [PATCH 0/4] Fixing live video input in ZynqMP DPSUB > > On Wed, Jan 17, 2024 at 04:23:43PM +0200, Laurent Pinchart wrote: > > On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > > > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > > > Patches 1/4,2/4,3/4 are minor fixes. > > > > > > > > DPSUB requires input live video format to be configured. > > > > Patch 4/4: The DP Subsystem requires the input live video format to be > > > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > > > and comes from the device tree. This is a proposed solution, as there are no > api > > > > to query CRTC output bus format. > > > > > > > > Is this a good approach to go with? > > > > > > I guess you would need to expand a bit on what "live video input" is? Is > > > it some kind of mechanism to bypass memory and take your pixels straight > > > from a FIFO from another device, or something else? > > > > Yes and no. > > > > The DPSUB integrates DMA engines, a blending engine (two planes), and a > > DP encoder. The dpsub driver supports all of this, and creates a DRM > > device. The DP encoder hardware always takes its input data from the > > output of the blending engine. > > > > The blending engine can optionally take input data from a bus connected > > to the FPGA fabric, instead of taking it from the DPSUB internal DMA > > engines. When operating in that mode, the dpsub driver exposes the DP > > encoder as a bridge, and internally programs the blending engine to > > disable blending. Typically, the FPGA fabric will then contain a CRTC of > > some sort, with a driver that will acquire the DP encoder bridge as > > usually done. > > > > In this mode of operation, it is typical for the IP cores in FPGA fabric > > to be synthesized with a fixed format (as that saves resources), while > > the DPSUB supports multiple input formats. > > Where is that CRTC driver? It's not clear to me why the format would > need to be in the device tree at all. Format negociation between the > CRTC and whatever comes next is already done in a number of drivers so > it would be useful to have that kind of API outside of the bridge > support. > One example of such CRTC driver: https://github.com/Xilinx/linux-xlnx/blob/master/drivers/gpu/drm/xlnx/xlnx_mixer.c It's not upstreamed yet. Bus format negotiations here are handled by utilizing Xilinx-specific bridge framework. Ideally, it would be nice to rework this to comply with the upstream DRM bridge framework. > > Bridge drivers in the upstream kernel work the other way around, with > > the bridge hardware supporting a limited set of formats, and the CRTC > > then being programmed with whatever the bridges chain needs. Here, the > > negotiation needs to go the other way around, as the CRTC is the > > limiting factor, not the bridge. > > Sounds like there's something to rework in the API then? > Adding an optional CRTC callback imposing CRTC specific bus format restrictions, which may be called from here https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/drm_bridge.c#L935 would solve the problem. > Maxime -- Regards, Anatoliy
On Fri, Jan 26, 2024 at 11:18:30PM +0000, Klymenko, Anatoliy wrote: > > > > -----Original Message----- > > From: Maxime Ripard <mripard@kernel.org> > > Sent: Friday, January 26, 2024 4:26 AM > > To: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > > Cc: Klymenko, Anatoliy <Anatoliy.Klymenko@amd.com>; > > maarten.lankhorst@linux.intel.com; tzimmermann@suse.de; airlied@gmail.com; > > daniel@ffwll.ch; Simek, Michal <michal.simek@amd.com>; dri- > > devel@lists.freedesktop.org; linux-arm-kernel@lists.infradead.org; linux- > > kernel@vger.kernel.org > > Subject: Re: Re: [PATCH 0/4] Fixing live video input in ZynqMP DPSUB > > > > On Wed, Jan 17, 2024 at 04:23:43PM +0200, Laurent Pinchart wrote: > > > On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > > > > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > > > > Patches 1/4,2/4,3/4 are minor fixes. > > > > > > > > > > DPSUB requires input live video format to be configured. > > > > > Patch 4/4: The DP Subsystem requires the input live video format to be > > > > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > > > > and comes from the device tree. This is a proposed solution, as there are no > > api > > > > > to query CRTC output bus format. > > > > > > > > > > Is this a good approach to go with? > > > > > > > > I guess you would need to expand a bit on what "live video input" is? Is > > > > it some kind of mechanism to bypass memory and take your pixels straight > > > > from a FIFO from another device, or something else? > > > > > > Yes and no. > > > > > > The DPSUB integrates DMA engines, a blending engine (two planes), and a > > > DP encoder. The dpsub driver supports all of this, and creates a DRM > > > device. The DP encoder hardware always takes its input data from the > > > output of the blending engine. > > > > > > The blending engine can optionally take input data from a bus connected > > > to the FPGA fabric, instead of taking it from the DPSUB internal DMA > > > engines. When operating in that mode, the dpsub driver exposes the DP > > > encoder as a bridge, and internally programs the blending engine to > > > disable blending. Typically, the FPGA fabric will then contain a CRTC of > > > some sort, with a driver that will acquire the DP encoder bridge as > > > usually done. > > > > > > In this mode of operation, it is typical for the IP cores in FPGA fabric > > > to be synthesized with a fixed format (as that saves resources), while > > > the DPSUB supports multiple input formats. > > > > Where is that CRTC driver? It's not clear to me why the format would > > need to be in the device tree at all. Format negociation between the > > CRTC and whatever comes next is already done in a number of drivers so > > it would be useful to have that kind of API outside of the bridge > > support. > > One example of such CRTC driver: > https://github.com/Xilinx/linux-xlnx/blob/master/drivers/gpu/drm/xlnx/xlnx_mixer.c It's not > upstreamed yet. Bus format negotiations here are handled by utilizing Xilinx-specific bridge > framework. Ideally, it would be nice to rework this to comply with the upstream DRM bridge > framework. > > > > Bridge drivers in the upstream kernel work the other way around, with > > > the bridge hardware supporting a limited set of formats, and the CRTC > > > then being programmed with whatever the bridges chain needs. Here, the > > > negotiation needs to go the other way around, as the CRTC is the > > > limiting factor, not the bridge. > > > > Sounds like there's something to rework in the API then? > > > Adding an optional CRTC callback imposing CRTC specific bus format restrictions, which may be > called from here https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/drm_bridge.c#L935 > would solve the problem. CRTCs and bridges are orthogonal. If anything, I'd expect that callback to be set at the CRTC, encoder and connector levels and filled by the drm_bridge code if relevant. Maxime
On Thu, Feb 01, 2024 at 06:01:01PM +0100, Maxime Ripard wrote: > On Fri, Jan 26, 2024 at 11:18:30PM +0000, Klymenko, Anatoliy wrote: > > On Friday, January 26, 2024 4:26 AM, Maxime Ripard wrote: > > > On Wed, Jan 17, 2024 at 04:23:43PM +0200, Laurent Pinchart wrote: > > > > On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > > > > > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > > > > > Patches 1/4,2/4,3/4 are minor fixes. > > > > > > > > > > > > DPSUB requires input live video format to be configured. > > > > > > Patch 4/4: The DP Subsystem requires the input live video format to be > > > > > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > > > > > and comes from the device tree. This is a proposed solution, as there are no api > > > > > > to query CRTC output bus format. > > > > > > > > > > > > Is this a good approach to go with? > > > > > > > > > > I guess you would need to expand a bit on what "live video input" is? Is > > > > > it some kind of mechanism to bypass memory and take your pixels straight > > > > > from a FIFO from another device, or something else? > > > > > > > > Yes and no. > > > > > > > > The DPSUB integrates DMA engines, a blending engine (two planes), and a > > > > DP encoder. The dpsub driver supports all of this, and creates a DRM > > > > device. The DP encoder hardware always takes its input data from the > > > > output of the blending engine. > > > > > > > > The blending engine can optionally take input data from a bus connected > > > > to the FPGA fabric, instead of taking it from the DPSUB internal DMA > > > > engines. When operating in that mode, the dpsub driver exposes the DP > > > > encoder as a bridge, and internally programs the blending engine to > > > > disable blending. Typically, the FPGA fabric will then contain a CRTC of > > > > some sort, with a driver that will acquire the DP encoder bridge as > > > > usually done. > > > > > > > > In this mode of operation, it is typical for the IP cores in FPGA fabric > > > > to be synthesized with a fixed format (as that saves resources), while > > > > the DPSUB supports multiple input formats. > > > > > > Where is that CRTC driver? It's not clear to me why the format would > > > need to be in the device tree at all. Format negociation between the > > > CRTC and whatever comes next is already done in a number of drivers so > > > it would be useful to have that kind of API outside of the bridge > > > support. > > > > One example of such CRTC driver: > > https://github.com/Xilinx/linux-xlnx/blob/master/drivers/gpu/drm/xlnx/xlnx_mixer.c It's not > > upstreamed yet. Bus format negotiations here are handled by utilizing Xilinx-specific bridge > > framework. Ideally, it would be nice to rework this to comply with the upstream DRM bridge > > framework. > > > > > > Bridge drivers in the upstream kernel work the other way around, with > > > > the bridge hardware supporting a limited set of formats, and the CRTC > > > > then being programmed with whatever the bridges chain needs. Here, the > > > > negotiation needs to go the other way around, as the CRTC is the > > > > limiting factor, not the bridge. > > > > > > Sounds like there's something to rework in the API then? > > > > > Adding an optional CRTC callback imposing CRTC specific bus format restrictions, which may be > > called from here https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/drm_bridge.c#L935 > > would solve the problem. > > CRTCs and bridges are orthogonal. If anything, I'd expect that callback > to be set at the CRTC, encoder and connector levels and filled by the > drm_bridge code if relevant. I'm thinking about a new CRTC operation that would be called by the bridge chain format negotiation helper drm_atomic_bridge_chain_select_bus_fmts() (or one of the functions it calls), to filter the list of formats supported by the chain based on what the CRTC supports, or possibly to pick a format in that list. This needs to be prototyped
On Sun, Feb 04, 2024 at 11:56:18AM +0200, Laurent Pinchart wrote: > On Thu, Feb 01, 2024 at 06:01:01PM +0100, Maxime Ripard wrote: > > On Fri, Jan 26, 2024 at 11:18:30PM +0000, Klymenko, Anatoliy wrote: > > > On Friday, January 26, 2024 4:26 AM, Maxime Ripard wrote: > > > > On Wed, Jan 17, 2024 at 04:23:43PM +0200, Laurent Pinchart wrote: > > > > > On Mon, Jan 15, 2024 at 09:28:39AM +0100, Maxime Ripard wrote: > > > > > > On Fri, Jan 12, 2024 at 03:42:18PM -0800, Anatoliy Klymenko wrote: > > > > > > > Patches 1/4,2/4,3/4 are minor fixes. > > > > > > > > > > > > > > DPSUB requires input live video format to be configured. > > > > > > > Patch 4/4: The DP Subsystem requires the input live video format to be > > > > > > > configured. In this patch we are assuming that the CRTC's bus format is fixed > > > > > > > and comes from the device tree. This is a proposed solution, as there are no api > > > > > > > to query CRTC output bus format. > > > > > > > > > > > > > > Is this a good approach to go with? > > > > > > > > > > > > I guess you would need to expand a bit on what "live video input" is? Is > > > > > > it some kind of mechanism to bypass memory and take your pixels straight > > > > > > from a FIFO from another device, or something else? > > > > > > > > > > Yes and no. > > > > > > > > > > The DPSUB integrates DMA engines, a blending engine (two planes), and a > > > > > DP encoder. The dpsub driver supports all of this, and creates a DRM > > > > > device. The DP encoder hardware always takes its input data from the > > > > > output of the blending engine. > > > > > > > > > > The blending engine can optionally take input data from a bus connected > > > > > to the FPGA fabric, instead of taking it from the DPSUB internal DMA > > > > > engines. When operating in that mode, the dpsub driver exposes the DP > > > > > encoder as a bridge, and internally programs the blending engine to > > > > > disable blending. Typically, the FPGA fabric will then contain a CRTC of > > > > > some sort, with a driver that will acquire the DP encoder bridge as > > > > > usually done. > > > > > > > > > > In this mode of operation, it is typical for the IP cores in FPGA fabric > > > > > to be synthesized with a fixed format (as that saves resources), while > > > > > the DPSUB supports multiple input formats. > > > > > > > > Where is that CRTC driver? It's not clear to me why the format would > > > > need to be in the device tree at all. Format negociation between the > > > > CRTC and whatever comes next is already done in a number of drivers so > > > > it would be useful to have that kind of API outside of the bridge > > > > support. > > > > > > One example of such CRTC driver: > > > https://github.com/Xilinx/linux-xlnx/blob/master/drivers/gpu/drm/xlnx/xlnx_mixer.c It's not > > > upstreamed yet. Bus format negotiations here are handled by utilizing Xilinx-specific bridge > > > framework. Ideally, it would be nice to rework this to comply with the upstream DRM bridge > > > framework. > > > > > > > > Bridge drivers in the upstream kernel work the other way around, with > > > > > the bridge hardware supporting a limited set of formats, and the CRTC > > > > > then being programmed with whatever the bridges chain needs. Here, the > > > > > negotiation needs to go the other way around, as the CRTC is the > > > > > limiting factor, not the bridge. > > > > > > > > Sounds like there's something to rework in the API then? > > > > > > > Adding an optional CRTC callback imposing CRTC specific bus format restrictions, which may be > > > called from here https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/drm_bridge.c#L935 > > > would solve the problem. > > > > CRTCs and bridges are orthogonal. If anything, I'd expect that callback > > to be set at the CRTC, encoder and connector levels and filled by the > > drm_bridge code if relevant. > > I'm thinking about a new CRTC operation that would be called by the > bridge chain format negotiation helper > drm_atomic_bridge_chain_select_bus_fmts() (or one of the functions it > calls), to filter the list of formats supported by the chain based on > what the CRTC supports, or possibly to pick a format in that list. This > needs to be prototyped As long as we come up with something that works for regular encoders, I'm fine with that. Maxime