| Message ID | 20190910115037.23539-2-peter.ujfalusi@ti.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | dmaengine: Support for DMA domain controllers | expand |
On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote: > On systems where multiple DMA controllers available, non Slave (for example > memcpy operation) users can not be described in DT as there is no device > involved from the DMA controller's point of view, DMA binding is not usable. > However in these systems still a peripheral might need to be serviced by or > it is better to serviced by specific DMA controller. > When a memcpy is used to/from a memory mapped region for example a DMA in the > same domain can perform better. > For generic software modules doing mem 2 mem operations it also matter that > they will get a channel from a controller which is faster in DDR to DDR mode > rather then from the first controller happen to be loaded. > > This property is inherited, so it may be specified in a device node or in any > of its parent nodes. If a device needs mem2mem dma, I think we should just use the existing dma binding. The provider will need a way to define cell values which mean mem2mem. For generic s/w, it should be able to query the dma speed or get a preferred one IMO. It's not a DT problem. We measure memcpy speeds at boot time to select the fastest implementation for a chip, why not do that for mem2mem DMA? > > Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> > --- > .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ > 1 file changed, 88 insertions(+) > create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml Note that you have several errors in your schema. Run 'make dt_bindings_check'. Rob
On 13/09/2019 17.36, Rob Herring wrote: > On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote: >> On systems where multiple DMA controllers available, non Slave (for example >> memcpy operation) users can not be described in DT as there is no device >> involved from the DMA controller's point of view, DMA binding is not usable. >> However in these systems still a peripheral might need to be serviced by or >> it is better to serviced by specific DMA controller. >> When a memcpy is used to/from a memory mapped region for example a DMA in the >> same domain can perform better. >> For generic software modules doing mem 2 mem operations it also matter that >> they will get a channel from a controller which is faster in DDR to DDR mode >> rather then from the first controller happen to be loaded. >> >> This property is inherited, so it may be specified in a device node or in any >> of its parent nodes. > > If a device needs mem2mem dma, I think we should just use the existing > dma binding. The provider will need a way to define cell values which > mean mem2mem. But isn't it going to be an abuse of the binding? Each DMA controller would hack this in different ways, probably using out of range DMA request/trigger number or if they have direction in the binding or some other parameter would be set to something invalid... > For generic s/w, it should be able to query the dma speed or get a > preferred one IMO. It's not a DT problem. > > We measure memcpy speeds at boot time to select the fastest > implementation for a chip, why not do that for mem2mem DMA? It would make an impact on boot time since the tests would need to be done with a large enough copy to be able to see clearly which one is faster. Also we should be able to handle different probing orders: client1 should have mem2mem channel from dma2. - dma1 probes - client1 probes and asks for a mem2mem channel - dma2 probes Here client1 should deffer until dma2 is probed. Probably the property should be dma-mem2mem-domain to be more precise on it's purpose and avoid confusion? > >> >> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> >> --- >> .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ >> 1 file changed, 88 insertions(+) >> create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml > > Note that you have several errors in your schema. Run 'make dt_bindings_check'. That does not do anything on my system, but git dt-doc-validate running via https://github.com/robherring/yaml-bindings.git. - Péter Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
On 16/09/2019 14.21, Peter Ujfalusi wrote: > > > On 13/09/2019 17.36, Rob Herring wrote: >> On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote: >>> On systems where multiple DMA controllers available, non Slave (for example >>> memcpy operation) users can not be described in DT as there is no device >>> involved from the DMA controller's point of view, DMA binding is not usable. >>> However in these systems still a peripheral might need to be serviced by or >>> it is better to serviced by specific DMA controller. >>> When a memcpy is used to/from a memory mapped region for example a DMA in the >>> same domain can perform better. >>> For generic software modules doing mem 2 mem operations it also matter that >>> they will get a channel from a controller which is faster in DDR to DDR mode >>> rather then from the first controller happen to be loaded. >>> >>> This property is inherited, so it may be specified in a device node or in any >>> of its parent nodes. >> >> If a device needs mem2mem dma, I think we should just use the existing >> dma binding. The provider will need a way to define cell values which >> mean mem2mem. > > But isn't it going to be an abuse of the binding? Each DMA controller > would hack this in different ways, probably using out of range DMA > request/trigger number or if they have direction in the binding or some > other parameter would be set to something invalid... > >> For generic s/w, it should be able to query the dma speed or get a >> preferred one IMO. It's not a DT problem. >> >> We measure memcpy speeds at boot time to select the fastest >> implementation for a chip, why not do that for mem2mem DMA? > > It would make an impact on boot time since the tests would need to be > done with a large enough copy to be able to see clearly which one is faster. > > Also we should be able to handle different probing orders: > client1 should have mem2mem channel from dma2. > > - dma1 probes > - client1 probes and asks for a mem2mem channel > - dma2 probes > > Here client1 should deffer until dma2 is probed. > > Probably the property should be dma-mem2mem-domain to be more precise on > it's purpose and avoid confusion? Is it OK if I go with dma-mem2mem-domain or dma-mem2mem-controller for v2, but keeping the logic and approach intact? Regards, - Péter > >> >>> >>> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> >>> --- >>> .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ >>> 1 file changed, 88 insertions(+) >>> create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml >> >> Note that you have several errors in your schema. Run 'make dt_bindings_check'. > > That does not do anything on my system, but git dt-doc-validate running > via https://github.com/robherring/yaml-bindings.git. > > - Péter > > Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. > Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki > Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
On Mon, Sep 16, 2019 at 6:21 AM Peter Ujfalusi <peter.ujfalusi@ti.com> wrote: > > > > On 13/09/2019 17.36, Rob Herring wrote: > > On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote: > >> On systems where multiple DMA controllers available, non Slave (for example > >> memcpy operation) users can not be described in DT as there is no device > >> involved from the DMA controller's point of view, DMA binding is not usable. > >> However in these systems still a peripheral might need to be serviced by or > >> it is better to serviced by specific DMA controller. > >> When a memcpy is used to/from a memory mapped region for example a DMA in the > >> same domain can perform better. > >> For generic software modules doing mem 2 mem operations it also matter that > >> they will get a channel from a controller which is faster in DDR to DDR mode > >> rather then from the first controller happen to be loaded. > >> > >> This property is inherited, so it may be specified in a device node or in any > >> of its parent nodes. > > > > If a device needs mem2mem dma, I think we should just use the existing > > dma binding. The provider will need a way to define cell values which > > mean mem2mem. > > But isn't it going to be an abuse of the binding? Each DMA controller > would hack this in different ways, probably using out of range DMA > request/trigger number or if they have direction in the binding or some > other parameter would be set to something invalid... What's in the cells is defined by the provider which can define whatever they want. We do standardize that in some cases. I think we have some cases where we set the channel priority in the cells. What if someone wants to do that for mem2mem as well? > > For generic s/w, it should be able to query the dma speed or get a > > preferred one IMO. It's not a DT problem. > > > > We measure memcpy speeds at boot time to select the fastest > > implementation for a chip, why not do that for mem2mem DMA? > > It would make an impact on boot time since the tests would need to be > done with a large enough copy to be able to see clearly which one is faster. Have you measured it? It could be done in parallel and may have little to no impact. > Also we should be able to handle different probing orders: > client1 should have mem2mem channel from dma2. > > - dma1 probes > - client1 probes and asks for a mem2mem channel > - dma2 probes > > Here client1 should deffer until dma2 is probed. Depending on the driver, don't make the decision in probe, but when you start using the driver. For example, serial drivers decide on DMA or no DMA in open(). > Probably the property should be dma-mem2mem-domain to be more precise on > it's purpose and avoid confusion? > > > > >> > >> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> > >> --- > >> .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ > >> 1 file changed, 88 insertions(+) > >> create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml > > > > Note that you have several errors in your schema. Run 'make dt_bindings_check'. > > That does not do anything on my system, but git dt-doc-validate running > via https://github.com/robherring/yaml-bindings.git. It should do *something*... Do you have libyaml-dev installed? BTW, while I still mirror to that repo, use https://github.com/devicetree-org/dt-schema instead. Rob
On 9/24/19 5:44 PM, Rob Herring wrote: > On Mon, Sep 16, 2019 at 6:21 AM Peter Ujfalusi <peter.ujfalusi@ti.com> wrote: >> >> >> >> On 13/09/2019 17.36, Rob Herring wrote: >>> On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote: >>>> On systems where multiple DMA controllers available, non Slave (for example >>>> memcpy operation) users can not be described in DT as there is no device >>>> involved from the DMA controller's point of view, DMA binding is not usable. >>>> However in these systems still a peripheral might need to be serviced by or >>>> it is better to serviced by specific DMA controller. >>>> When a memcpy is used to/from a memory mapped region for example a DMA in the >>>> same domain can perform better. >>>> For generic software modules doing mem 2 mem operations it also matter that >>>> they will get a channel from a controller which is faster in DDR to DDR mode >>>> rather then from the first controller happen to be loaded. >>>> >>>> This property is inherited, so it may be specified in a device node or in any >>>> of its parent nodes. >>> >>> If a device needs mem2mem dma, I think we should just use the existing >>> dma binding. The provider will need a way to define cell values which >>> mean mem2mem. >> >> But isn't it going to be an abuse of the binding? Each DMA controller >> would hack this in different ways, probably using out of range DMA >> request/trigger number or if they have direction in the binding or some >> other parameter would be set to something invalid... > > What's in the cells is defined by the provider which can define > whatever they want. We do standardize that in some cases. There is a substantiation difference how HW synchronized (slave) channels and SW triggered (non slave) channels can be described, handled. For slave channels we have bindings as we can describe the DMA request line to be used for the DMA channel. In some cases the requests are locked to DMA channels (TI's EDMA for example), while in other cases any channel can handle any operation (TI's sDMA, UDMAP). For EDMA the DMA request == EDMA channel number For sDMA we give the DMA request number and we pick any free lchan to handle it. Non slave channels on the other hand have no identification. They are channels which can execute SW triggered task. For EDMA for example we need to mark channels which is not in use and tell the system that they can be used for non slave mode. sDMA on the other hand can just pick any free lchan. Passing 0xffffffff as channel number or DMA request number surely be something which most likely invalid, but every driver needs to somehow figure out their invalid parameter and all of them needs to be modified, generic match helpers can not be used anymore as they will fail in random ways. > I think we have some cases where we set the channel priority in the > cells. What if someone wants to do that for mem2mem as well? That's a valid point, but even with bindings for non slave channels clients from modules will need to be handled in a different way as they don't have presence in DT.. But I would argue that from system point of view the non slave channels might be better to have the same priority as a group on the given controller. > >>> For generic s/w, it should be able to query the dma speed or get a >>> preferred one IMO. It's not a DT problem. >>> >>> We measure memcpy speeds at boot time to select the fastest >>> implementation for a chip, why not do that for mem2mem DMA? >> >> It would make an impact on boot time since the tests would need to be >> done with a large enough copy to be able to see clearly which one is faster. > > Have you measured it? It could be done in parallel and may have little > to no impact. Depends on the DMA controller, but for UDMAP I can say it is around 1M buffer which clearly tells apart the two DMAs. Still, we should delay all non slave user's probe until we have all information on all of them and we don't know how many DMAs the HW have. How can you know that after the testing the first DMA's speed you will not have another one probing? Or after the second DMA controller there could be a third coming which can be the fastest? >> Also we should be able to handle different probing orders: >> client1 should have mem2mem channel from dma2. >> >> - dma1 probes >> - client1 probes and asks for a mem2mem channel >> - dma2 probes >> >> Here client1 should deffer until dma2 is probed. > > Depending on the driver, don't make the decision in probe, but when > you start using the driver. For example, serial drivers decide on DMA > or no DMA in open(). Which is causing a lot of pain as serial is really broken for deferred probing against DMA. Also note that dma channel request/release is not 'free' it might be something which can take time, thus making DMA use for non slave transfers less tempting. With adding dma-nonslave-domain (third or fourth name for this?) none of the DMA drivers have to be modified, the change in core is relatively small and backwards compatible and it describes the HW. > >> Probably the property should be dma-mem2mem-domain to be more precise on >> it's purpose and avoid confusion? >> >>> >>>> >>>> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> >>>> --- >>>> .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ >>>> 1 file changed, 88 insertions(+) >>>> create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml >>> >>> Note that you have several errors in your schema. Run 'make dt_bindings_check'. >> >> That does not do anything on my system, but git dt-doc-validate running >> via https://github.com/robherring/yaml-bindings.git. > > It should do *something*... Do you have libyaml-dev installed? > > BTW, while I still mirror to that repo, use > https://github.com/devicetree-org/dt-schema instead. Thanks a lot, got it working now correctly, I belive. - Peter Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
diff --git a/Documentation/devicetree/bindings/dma/dma-domain.yaml b/Documentation/devicetree/bindings/dma/dma-domain.yaml new file mode 100644 index 000000000000..da59bb129c58 --- /dev/null +++ b/Documentation/devicetree/bindings/dma/dma-domain.yaml @@ -0,0 +1,88 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/dma/dma-controller.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: DMA Domain Controller Definition + +maintainers: + - Vinod Koul <vkoul@kernel.org> + +allOf: + - $ref: "dma-controller.yaml#" + +description: + On systems where multiple DMA controllers available, none Slave (for example + memcpy operation) users can not be described in DT as there is no device + involved from the DMA controller's point of view, DMA binding is not usable. + However in these systems still a peripheral might need to be serviced by or + it is better to serviced by specific DMA controller. + When a memcpy is used to/from a memory mapped region for example a DMA in the + same domain can perform better. + For generic software modules doing mem 2 mem operations it also matter that + they will get a channel from a controller which is faster in DDR to DDR mode + rather then from the first controller happen to be loaded. + + This property is inherited, so it may be specified in a device node or in any + of its parent nodes. + +properties: + $dma-domain-controller: + $ref: /schemas/types.yaml#definitions/phandle + description: + phande to the DMA controller node which should be used for the device or + domain. + +examples: + - | + / { + model = "Texas Instruments K3 AM654 SoC"; + compatible = "ti,am654"; + interrupt-parent = <&gic500>; + /* For modules without device, DDR to DDR is faster on main UDMAP */ + dma-domain-controller = <&main_udmap>; + #address-cells = <2>; + #size-cells = <2>; + ... + }; + + &cbass_main { + /* For modules within MAIN domain, use main UDMAP */ + dma-domain-controller = <&main_udmap>; + + cbass_main_navss: interconnect0 { + ... + main_udmap: dma-controller@31150000 { + compatible = "ti,am654-navss-main-udmap"; + ... + }; + }; + }; + + &cbass_mcu { + /* For modules within MCU domain, use mcu UDMAP */ + dma-domain-controller = <&mcu_udmap>; + + cbass_mcu_navss: interconnect1 { + ... + mcu_udmap: dma-controller@285c0000 { + compatible = "ti,am654-navss-mcu-udmap"; + ... + }; + }; + + fss: fss@47000000 { + compatible = "simple-bus"; + #address-cells = <2>; + #size-cells = <2>; + ranges; + + ospi0: spi@47040000 { + compatible = "ti,am654-ospi", "cdns,qspi-nor"; + ... + /* memcpy channel will be request from mcu_udmap */ + }; + }; + }; +...
On systems where multiple DMA controllers available, non Slave (for example memcpy operation) users can not be described in DT as there is no device involved from the DMA controller's point of view, DMA binding is not usable. However in these systems still a peripheral might need to be serviced by or it is better to serviced by specific DMA controller. When a memcpy is used to/from a memory mapped region for example a DMA in the same domain can perform better. For generic software modules doing mem 2 mem operations it also matter that they will get a channel from a controller which is faster in DDR to DDR mode rather then from the first controller happen to be loaded. This property is inherited, so it may be specified in a device node or in any of its parent nodes. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com> --- .../devicetree/bindings/dma/dma-domain.yaml | 88 +++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml