[v5,00/11] dmaengine: dw: Take Baikal-T1 SoC DW DMAC peculiarities into account

Message

Serge Semin May 29, 2020, 2:40 p.m. UTC

Baikal-T1 SoC has an DW DMAC on-board to provide a Mem-to-Mem, low-speed
peripherals Dev-to-Mem and Mem-to-Dev functionality. Mostly it's compatible
with currently implemented in the kernel DW DMAC driver, but there are some
peculiarities which must be taken into account in order to have the device
fully supported.

First of all traditionally we replaced the legacy plain text-based dt-binding
file with yaml-based one. Secondly Baikal-T1 DW DMA Controller provides eight
channels, which alas have different max burst length configuration.
In particular first two channels may burst up to 128 bits (16 bytes) at a time
while the rest of them just up to 32 bits. We must make sure that the DMA
subsystem doesn't set values exceeding these limitations otherwise the
controller will hang up. In third currently we discovered the problem in using
the DW APB SPI driver together with DW DMAC. The problem happens if there is no
natively implemented multi-block LLP transfers support and the SPI-transfer
length exceeds the max lock size. In this case due to asynchronous handling of
Tx- and Rx- SPI transfers interrupt we might end up with Dw APB SSI Rx FIFO
overflow. So if DW APB SSI (or any other DMAC service consumer) intends to use
the DMAC to asynchronously execute the transfers we'd have to at least warn
the user of the possible errors. In forth it's worth to set the DMA device max
segment size with max block size config specific to the DW DMA controller. It
shall help the DMA clients to create size-optimized SG-list items for the
controller. This in turn will cause less dw_desc allocations, less LLP
reinitializations, better DMA device performance.

Finally there is a bug in the algorithm of the nollp flag detection.
In particular even if DW DMAC parameters state the multi-block transfers
support there is still HC_LLP (hardcode LLP) flag, which if set makes expected
by the driver true multi-block LLP functionality unusable. This happens cause'
if HC_LLP flag is set the LLP registers will be hardcoded to zero so the
contiguous multi-block transfers will be only supported. We must take the
flag into account when detecting the LLP support otherwise the driver just
won't work correctly.

This patchset is rebased and tested on the mainline Linux kernel 5.7-rc4:
0e698dfa2822 ("Linux 5.7-rc4")
tag: v5.7-rc4

Changelog v2:
- Rearrange SoBs.
- Move $ref to the root level of the properties. So do do with the
  constraints in the DT binding.
- Replace "additionalProperties: false" with "unevaluatedProperties: false"
  property in the DT binding file.
- Discard default settings defined out of property enum constraint.
- Set default max-burst-len to 256 TR-WIDTH words in the DT binding.
- Discard noLLP and block_size accessors.
- Set max segment size of the DMA device structure with the DW DMA block size
  config.
- Print warning if noLLP flag is set.
- Discard max burst length accessor.
- Add comment about why hardware accelerated LLP list support depends
  on both MBLK_EN and HC_LLP configs setting.
- Use explicit bits state comparison operator in noLLP flag setting.

Link: https://lore.kernel.org/dmaengine/[e-mail]/
Changelog v3:
- Use the block_size found for the very first channel instead of looking for
  the maximum of maximum block sizes.
- Don't define device-specific device_dma_parameters object, since it has
  already been defined by the platform device core.
- Add more details into the property description about what limitations
  snps,max-burst-len defines.
- Move commit fb7e3bbfc830 ("dmaengine: dw: Take HC_LLP flag into account for
  noLLP auto-config") to the head of the series.
- Add a new patch "dmaengine: Introduce min burst length capability" as a
  result of the discussion with Vinod and Andy regarding the burst length
  capability.
- Add a new patch "dmaengine: Introduce max SG list entries capability"
  suggested by Andy.
- Add a new patch "dmaengine: Introduce DMA-device device_caps callback" as
  a result of the discussion with Vinud and Andy in the framework of DW DMA
  burst and LLP capabilities.
- Add a new patch "dmaengine: dw: Add dummy device_caps callback" as a
  preparation commit before setting the max_burst and max_sg_nents
  DW DMA capabilities.
- Override the slave channel max_burst capability instead of calculating
  the minimum value of max burst lengths and setting the DMA-device
  generic capability.
- Add a new patch "dmaengine: dw: Initialize max_sg_nents with nollp flag".
  This is required to fix the DW APB SSI issue of the Tx and Rx DMA
  channels de-synchronization.

Link: https://lore.kernel.org/dmaengine/[e-mail]/
Changelog v4:
- Use explicit if-else statement when assigning the max_sg_nents field.
- Clamp the dst and src burst lengths in the generic dwc_config() method
  instead of doing that in the encode_maxburst() callback.
- Define max_burst with u32 type in struct dw_dma_platform_data.
- Perform of_property_read_u32_array() with the platform data
  max_burst member passed directly.
- Add a new patch "dmaengine: dw: Initialize min_burst capability",
  which initializes the min_burst capability with 1.
- Fix of->if typo. It should be definitely "of" in the max_sg_list
  capability description.

Link: https://lore.kernel.org/dmaengine/[e-mail]
Changelog v5:
- Introduce macro with extreme min and max burst lengths supported by the
  DW DMA controller. Define them in the patch with default min and max burst
  length iintializations.
- Initialize max_burst length capability with extreme burst length supported
  by the DW DMAC IP-core.
- Move DW_DMA_MAX_BURST macro definition to the patch "dmaengine: dw:
  Initialize min and max burst DMA device capability".
- Add in-line comment at the point of the device_caps callback invocation.
- Add doc-comment for the device_caps member of struct dma_device

Signed-off-by: Serge Semin <[e-mail]>
Cc: Alexey Malahov <[e-mail]>
Cc: Maxim Kaurkin <[e-mail]>
Cc: Pavel Parkhomenko <[e-mail]>
Cc: Ramil Zaripov <[e-mail]>
Cc: Ekaterina Skachko <[e-mail]>
Cc: Vadim Vlasov <[e-mail]>
Cc: Alexey Kolotnikov <[e-mail]>
Cc: Thomas Bogendoerfer <[e-mail]>
Cc: Arnd Bergmann <[e-mail]>
Cc: Andy Shevchenko <[e-mail]>
Cc: Rob Herring <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]

Serge Semin (11):
  dt-bindings: dma: dw: Convert DW DMAC to DT binding
  dt-bindings: dma: dw: Add max burst transaction length property
  dmaengine: Introduce min burst length capability
  dmaengine: Introduce max SG list entries capability
  dmaengine: Introduce DMA-device device_caps callback
  dmaengine: dw: Take HC_LLP flag into account for noLLP auto-config
  dmaengine: dw: Set DMA device max segment size parameter
  dmaengine: dw: Add dummy device_caps callback
  dmaengine: dw: Initialize min and max burst DMA device capability
  dmaengine: dw: Introduce max burst length hw config
  dmaengine: dw: Initialize max_sg_nents capability

 .../bindings/dma/snps,dma-spear1340.yaml      | 176 ++++++++++++++++++
 .../devicetree/bindings/dma/snps-dma.txt      |  69 -------
 drivers/dma/dmaengine.c                       |  12 ++
 drivers/dma/dw/core.c                         |  48 ++++-
 drivers/dma/dw/of.c                           |   5 +
 drivers/dma/dw/regs.h                         |   3 +
 include/linux/dmaengine.h                     |  16 ++
 include/linux/platform_data/dma-dw.h          |   5 +
 8 files changed, 264 insertions(+), 70 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/dma/snps,dma-spear1340.yaml
 delete mode 100644 Documentation/devicetree/bindings/dma/snps-dma.txt

Comments

Serge Semin June 2, 2020, 9:27 a.m. UTC | #1

Vinod, Viresh

Andy's finished his review. So all the patches of the series (except one rather
decorative, which we have different opinion of) are tagged by him. Since merge
window is about to be opened please consider to merge the series in. I'll really
need it to be in the kernel to provide the noLLP-problem fix for the Dw APB SSI
in 5.8.

-Sergey

On Fri, May 29, 2020 at 05:40:43PM +0300, Serge Semin wrote:
> Baikal-T1 SoC has an DW DMAC on-board to provide a Mem-to-Mem, low-speed
> peripherals Dev-to-Mem and Mem-to-Dev functionality. Mostly it's compatible
> with currently implemented in the kernel DW DMAC driver, but there are some
> peculiarities which must be taken into account in order to have the device
> fully supported.
> 
> First of all traditionally we replaced the legacy plain text-based dt-binding
> file with yaml-based one. Secondly Baikal-T1 DW DMA Controller provides eight
> channels, which alas have different max burst length configuration.
> In particular first two channels may burst up to 128 bits (16 bytes) at a time
> while the rest of them just up to 32 bits. We must make sure that the DMA
> subsystem doesn't set values exceeding these limitations otherwise the
> controller will hang up. In third currently we discovered the problem in using
> the DW APB SPI driver together with DW DMAC. The problem happens if there is no
> natively implemented multi-block LLP transfers support and the SPI-transfer
> length exceeds the max lock size. In this case due to asynchronous handling of
> Tx- and Rx- SPI transfers interrupt we might end up with Dw APB SSI Rx FIFO
> overflow. So if DW APB SSI (or any other DMAC service consumer) intends to use
> the DMAC to asynchronously execute the transfers we'd have to at least warn
> the user of the possible errors. In forth it's worth to set the DMA device max
> segment size with max block size config specific to the DW DMA controller. It
> shall help the DMA clients to create size-optimized SG-list items for the
> controller. This in turn will cause less dw_desc allocations, less LLP
> reinitializations, better DMA device performance.
> 
> Finally there is a bug in the algorithm of the nollp flag detection.
> In particular even if DW DMAC parameters state the multi-block transfers
> support there is still HC_LLP (hardcode LLP) flag, which if set makes expected
> by the driver true multi-block LLP functionality unusable. This happens cause'
> if HC_LLP flag is set the LLP registers will be hardcoded to zero so the
> contiguous multi-block transfers will be only supported. We must take the
> flag into account when detecting the LLP support otherwise the driver just
> won't work correctly.
> 
> This patchset is rebased and tested on the mainline Linux kernel 5.7-rc4:
> 0e698dfa2822 ("Linux 5.7-rc4")
> tag: v5.7-rc4
> 
> Changelog v2:
> - Rearrange SoBs.
> - Move $ref to the root level of the properties. So do do with the
>   constraints in the DT binding.
> - Replace "additionalProperties: false" with "unevaluatedProperties: false"
>   property in the DT binding file.
> - Discard default settings defined out of property enum constraint.
> - Set default max-burst-len to 256 TR-WIDTH words in the DT binding.
> - Discard noLLP and block_size accessors.
> - Set max segment size of the DMA device structure with the DW DMA block size
>   config.
> - Print warning if noLLP flag is set.
> - Discard max burst length accessor.
> - Add comment about why hardware accelerated LLP list support depends
>   on both MBLK_EN and HC_LLP configs setting.
> - Use explicit bits state comparison operator in noLLP flag setting.
> 
> Link: https://lore.kernel.org/dmaengine/[e-mail]/
> Changelog v3:
> - Use the block_size found for the very first channel instead of looking for
>   the maximum of maximum block sizes.
> - Don't define device-specific device_dma_parameters object, since it has
>   already been defined by the platform device core.
> - Add more details into the property description about what limitations
>   snps,max-burst-len defines.
> - Move commit fb7e3bbfc830 ("dmaengine: dw: Take HC_LLP flag into account for
>   noLLP auto-config") to the head of the series.
> - Add a new patch "dmaengine: Introduce min burst length capability" as a
>   result of the discussion with Vinod and Andy regarding the burst length
>   capability.
> - Add a new patch "dmaengine: Introduce max SG list entries capability"
>   suggested by Andy.
> - Add a new patch "dmaengine: Introduce DMA-device device_caps callback" as
>   a result of the discussion with Vinud and Andy in the framework of DW DMA
>   burst and LLP capabilities.
> - Add a new patch "dmaengine: dw: Add dummy device_caps callback" as a
>   preparation commit before setting the max_burst and max_sg_nents
>   DW DMA capabilities.
> - Override the slave channel max_burst capability instead of calculating
>   the minimum value of max burst lengths and setting the DMA-device
>   generic capability.
> - Add a new patch "dmaengine: dw: Initialize max_sg_nents with nollp flag".
>   This is required to fix the DW APB SSI issue of the Tx and Rx DMA
>   channels de-synchronization.
> 
> Link: https://lore.kernel.org/dmaengine/[e-mail]/
> Changelog v4:
> - Use explicit if-else statement when assigning the max_sg_nents field.
> - Clamp the dst and src burst lengths in the generic dwc_config() method
>   instead of doing that in the encode_maxburst() callback.
> - Define max_burst with u32 type in struct dw_dma_platform_data.
> - Perform of_property_read_u32_array() with the platform data
>   max_burst member passed directly.
> - Add a new patch "dmaengine: dw: Initialize min_burst capability",
>   which initializes the min_burst capability with 1.
> - Fix of->if typo. It should be definitely "of" in the max_sg_list
>   capability description.
> 
> Link: https://lore.kernel.org/dmaengine/[e-mail]
> Changelog v5:
> - Introduce macro with extreme min and max burst lengths supported by the
>   DW DMA controller. Define them in the patch with default min and max burst
>   length iintializations.
> - Initialize max_burst length capability with extreme burst length supported
>   by the DW DMAC IP-core.
> - Move DW_DMA_MAX_BURST macro definition to the patch "dmaengine: dw:
>   Initialize min and max burst DMA device capability".
> - Add in-line comment at the point of the device_caps callback invocation.
> - Add doc-comment for the device_caps member of struct dma_device
> 
> Signed-off-by: Serge Semin <[e-mail]>
> Cc: Alexey Malahov <[e-mail]>
> Cc: Maxim Kaurkin <[e-mail]>
> Cc: Pavel Parkhomenko <[e-mail]>
> Cc: Ramil Zaripov <[e-mail]>
> Cc: Ekaterina Skachko <[e-mail]>
> Cc: Vadim Vlasov <[e-mail]>
> Cc: Alexey Kolotnikov <[e-mail]>
> Cc: Thomas Bogendoerfer <[e-mail]>
> Cc: Arnd Bergmann <[e-mail]>
> Cc: Andy Shevchenko <[e-mail]>
> Cc: Rob Herring <[e-mail]>
> Cc: [e-mail]
> Cc: [e-mail]
> Cc: [e-mail]
> Cc: [e-mail]
> 
> Serge Semin (11):
>   dt-bindings: dma: dw: Convert DW DMAC to DT binding
>   dt-bindings: dma: dw: Add max burst transaction length property
>   dmaengine: Introduce min burst length capability
>   dmaengine: Introduce max SG list entries capability
>   dmaengine: Introduce DMA-device device_caps callback
>   dmaengine: dw: Take HC_LLP flag into account for noLLP auto-config
>   dmaengine: dw: Set DMA device max segment size parameter
>   dmaengine: dw: Add dummy device_caps callback
>   dmaengine: dw: Initialize min and max burst DMA device capability
>   dmaengine: dw: Introduce max burst length hw config
>   dmaengine: dw: Initialize max_sg_nents capability
> 
>  .../bindings/dma/snps,dma-spear1340.yaml      | 176 ++++++++++++++++++
>  .../devicetree/bindings/dma/snps-dma.txt      |  69 -------
>  drivers/dma/dmaengine.c                       |  12 ++
>  drivers/dma/dw/core.c                         |  48 ++++-
>  drivers/dma/dw/of.c                           |   5 +
>  drivers/dma/dw/regs.h                         |   3 +
>  include/linux/dmaengine.h                     |  16 ++
>  include/linux/platform_data/dma-dw.h          |   5 +
>  8 files changed, 264 insertions(+), 70 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/dma/snps,dma-spear1340.yaml
>  delete mode 100644 Documentation/devicetree/bindings/dma/snps-dma.txt
> 
> -- 
> 2.26.2
>

Vinod Koul June 16, 2020, 4:32 p.m. UTC | #2

Hi Serge,

On 02-06-20, 12:27, Serge Semin wrote:
> Vinod, Viresh
> 
> Andy's finished his review. So all the patches of the series (except one rather
> decorative, which we have different opinion of) are tagged by him. Since merge
> window is about to be opened please consider to merge the series in. I'll really
> need it to be in the kernel to provide the noLLP-problem fix for the Dw APB SSI
> in 5.8.

Sorry it was too late for 5.8.. merge window is closed now, i will
review it shortly

Serge Semin June 16, 2020, 7:07 p.m. UTC | #3

On Tue, Jun 16, 2020 at 10:02:42PM +0530, Vinod Koul wrote:
> Hi Serge,
> 
> On 02-06-20, 12:27, Serge Semin wrote:
> > Vinod, Viresh
> > 
> > Andy's finished his review. So all the patches of the series (except one rather
> > decorative, which we have different opinion of) are tagged by him. Since merge
> > window is about to be opened please consider to merge the series in. I'll really
> > need it to be in the kernel to provide the noLLP-problem fix for the Dw APB SSI
> > in 5.8.
> 
> Sorry it was too late for 5.8.. merge window is closed now, i will
> review it shortly

Yeah, alas It's too late now. On the other hand this is probably for better since
I've discovered that the noLLP-related problem is more complicated than I thought
in the first place. Yes, the interrupt handling latency will cause the problem
with Rx FIFO overflow, but the overflow might still happen due to several other
reasons (mostly due to our APB bus being too slow, but it can still be partly
fixed, and I am looking for a convenient solution at the moment). So could you
please merge the series in without the next patches:
[PATCH v5 04/11] dmaengine: Introduce max SG list entries capability
[PATCH v5 11/11] dmaengine: dw: Initialize max_sg_nents capability 
If I manage to fix the problem in general then I'll send another series with
those two patches being part of it. Otherwise they won't be much useful for my
platform. Sorry for inconvenience and thanks for understanding.

-Sergey

> 
> -- 
> ~Vinod