| Message ID | 20190506223334.1834-3-nicoleotsuka@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Optimize dma_*_from_contiguous calls | expand |
On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > The addresses within a single page are always contiguous, so it's > not so necessary to always allocate one single page from CMA area. > Since the CMA area has a limited predefined size of space, it may > run out of space in heavy use cases, where there might be quite a > lot CMA pages being allocated for single pages. > > However, there is also a concern that a device might care where a > page comes from -- it might expect the page from CMA area and act > differently if the page doesn't. > > This patch tries to use the fallback alloc_pages path, instead of > one-page size allocations from the global CMA area in case that a > device does not have its own CMA area. This'd save resources from > the CMA global area for more CMA allocations, and also reduce CMA > fragmentations resulted from trivial allocations. > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) broke the DMA for my MMC driver in the following way: [ 1.876755] mmc0: ADMA error [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 1.985271] mmc0: sdhci: ============================================ [ 1.991758] mmc0: error -5 whilst initialising MMC card [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, base_baud = 768000) is a 16550A [ 2.011011] hctosys: unable to open rtc device (rtc0) [ 2.017694] Freeing unused kernel memory: 2368K [ 2.027131] Run /init as init process Starting syslogd: OK Starting klogd: OK Initializing random number generator... [ 2.074399] random: dd: uninitialized urandom read (512 bytes read) done. Starting network: OK [ 2.109593] mmc0: ADMA error [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.214379] mmc0: sdhci: ============================================ [ 2.220881] mmc0: error -5 whilst initialising MMC card Welcome to Buildroot buildroot login: [ 2.332786] mmc0: ADMA error [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.437557] mmc0: sdhci: ============================================ [ 2.444031] mmc0: error -5 whilst initialising MMC card [ 2.572203] mmc0: ADMA error [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 [ 2.676979] mmc0: sdhci: ============================================ [ 2.683450] mmc0: error -5 whilst initialising MMC card CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline | ttyUSB0 Reverting this commit fixed the problem.
+ linux-mmc, Ulf Hansson, Adrian Hunter, ADMA of SDHCI is not working since bd2e75633c8012fc8a7431c82fda66237133bf7e Did anybody see the same problem? Masahiro On Fri, Aug 23, 2019 at 9:49 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > > > The addresses within a single page are always contiguous, so it's > > not so necessary to always allocate one single page from CMA area. > > Since the CMA area has a limited predefined size of space, it may > > run out of space in heavy use cases, where there might be quite a > > lot CMA pages being allocated for single pages. > > > > However, there is also a concern that a device might care where a > > page comes from -- it might expect the page from CMA area and act > > differently if the page doesn't. > > > > This patch tries to use the fallback alloc_pages path, instead of > > one-page size allocations from the global CMA area in case that a > > device does not have its own CMA area. This'd save resources from > > the CMA global area for more CMA allocations, and also reduce CMA > > fragmentations resulted from trivial allocations. > > > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > > > This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) > broke the DMA for my MMC driver in the following way: > > > > > [ 1.876755] mmc0: ADMA error > [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 > [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 > [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 1.985271] mmc0: sdhci: ============================================ > [ 1.991758] mmc0: error -5 whilst initialising MMC card > [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, > base_baud = 768000) is a 16550A > [ 2.011011] hctosys: unable to open rtc device (rtc0) > [ 2.017694] Freeing unused kernel memory: 2368K > [ 2.027131] Run /init as init process > Starting syslogd: OK > Starting klogd: OK > Initializing random number generator... [ 2.074399] random: dd: > uninitialized urandom read (512 bytes read) > done. > Starting network: OK > [ 2.109593] mmc0: ADMA error > [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 > [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.214379] mmc0: sdhci: ============================================ > > [ 2.220881] mmc0: error -5 whilst initialising MMC card > Welcome to Buildroot > buildroot login: [ 2.332786] mmc0: ADMA error > [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 > [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.437557] mmc0: sdhci: ============================================ > [ 2.444031] mmc0: error -5 whilst initialising MMC card > [ 2.572203] mmc0: ADMA error > [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 > [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 > [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.676979] mmc0: sdhci: ============================================ > [ 2.683450] mmc0: error -5 whilst initialising MMC card > > CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline > | ttyUSB0 > > Reverting this commit fixed the problem. > > > > -- > Best Regards > Masahiro Yamada
On Fri, Aug 23, 2019 at 09:49:46PM +0900, Masahiro Yamada wrote: > On Tue, May 7, 2019 at 7:36 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > > > The addresses within a single page are always contiguous, so it's > > not so necessary to always allocate one single page from CMA area. > > Since the CMA area has a limited predefined size of space, it may > > run out of space in heavy use cases, where there might be quite a > > lot CMA pages being allocated for single pages. > > > > However, there is also a concern that a device might care where a > > page comes from -- it might expect the page from CMA area and act > > differently if the page doesn't. > > > > This patch tries to use the fallback alloc_pages path, instead of > > one-page size allocations from the global CMA area in case that a > > device does not have its own CMA area. This'd save resources from > > the CMA global area for more CMA allocations, and also reduce CMA > > fragmentations resulted from trivial allocations. > > > > Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> > > > This commit (bd2e75633c8012fc8a7431c82fda66237133bf7e) > broke the DMA for my MMC driver in the following way: > > > > > [ 1.876755] mmc0: ADMA error > [ 1.883385] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 1.889834] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 1.896284] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 1.902733] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 1.909182] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 1.915631] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 1.922081] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000fa07 > [ 1.928530] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 1.934981] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 1.941429] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 1.947880] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 1.954329] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 1.960778] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 1.967229] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 1.973678] mmc0: sdhci: Host ctl2: 0x00000000 > [ 1.978125] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 1.985271] mmc0: sdhci: ============================================ > [ 1.991758] mmc0: error -5 whilst initialising MMC card > [ 1.991913] 43fb0000.uart: ttyS1 at MMIO 0x43fb0000 (irq = 0, > base_baud = 768000) is a 16550A > [ 2.011011] hctosys: unable to open rtc device (rtc0) > [ 2.017694] Freeing unused kernel memory: 2368K > [ 2.027131] Run /init as init process > Starting syslogd: OK > Starting klogd: OK > Initializing random number generator... [ 2.074399] random: dd: > uninitialized urandom read (512 bytes read) > done. > Starting network: OK > [ 2.109593] mmc0: ADMA error > [ 2.112488] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.118941] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.125389] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.131840] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.138289] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.144738] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.151188] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00004e47 > [ 2.157637] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.164087] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.170536] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.176987] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.183435] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.189886] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.196335] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.202784] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.207232] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.214379] mmc0: sdhci: ============================================ > > [ 2.220881] mmc0: error -5 whilst initialising MMC card > Welcome to Buildroot > buildroot login: [ 2.332786] mmc0: ADMA error > [ 2.335668] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.342119] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.348568] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.355018] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.361468] mmc0: sdhci: Present: 0x01ff02f6 | Host ctl: 0x00000019 > [ 2.367917] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.374367] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000f447 > [ 2.380816] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.387267] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.393716] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.400166] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.406615] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.413065] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.419515] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.425963] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.430412] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.437557] mmc0: sdhci: ============================================ > [ 2.444031] mmc0: error -5 whilst initialising MMC card > [ 2.572203] mmc0: ADMA error > [ 2.575089] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== > [ 2.581540] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00000002 > [ 2.587989] mmc0: sdhci: Blk size: 0x00007200 | Blk cnt: 0x00000001 > [ 2.594439] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000013 > [ 2.600889] mmc0: sdhci: Present: 0x01ef02f6 | Host ctl: 0x00000019 > [ 2.607339] mmc0: sdhci: Power: 0x0000000b | Blk gap: 0x00000000 > [ 2.613788] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x0000e8c7 > [ 2.620237] mmc0: sdhci: Timeout: 0x0000000b | Int stat: 0x00000001 > [ 2.626686] mmc0: sdhci: Int enab: 0x03ff008b | Sig enab: 0x03ff008b > [ 2.633137] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 > [ 2.639586] mmc0: sdhci: Caps: 0x546ec800 | Caps_1: 0x00000000 > [ 2.646036] mmc0: sdhci: Cmd: 0x0000083a | Max curr: 0x00000000 > [ 2.652485] mmc0: sdhci: Resp[0]: 0x00000900 | Resp[1]: 0xffffffff > [ 2.658936] mmc0: sdhci: Resp[2]: 0x320f5903 | Resp[3]: 0x3fd05e00 > [ 2.665384] mmc0: sdhci: Host ctl2: 0x00000000 > [ 2.669832] mmc0: sdhci: ADMA Err: 0x00000001 | ADMA Ptr: 0x000000013965b200 > [ 2.676979] mmc0: sdhci: ============================================ > [ 2.683450] mmc0: error -5 whilst initialising MMC card > > CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7.1 | VT102 | Offline > | ttyUSB0 > > Reverting this commit fixed the problem. We are having another problem with the new API and Christoph submitted a patch at: https://lkml.org/lkml/2019/8/20/86 Would it be possible for you to test to see if it can fix? We can revert my fallback change after all, if Christoph's patch doesn't work for you either. Thanks Nicolin
On Fri, Aug 23, 2019 at 09:56:52PM +0900, Masahiro Yamada wrote: > + linux-mmc, Ulf Hansson, Adrian Hunter, > > > ADMA of SDHCI is not working > since bd2e75633c8012fc8a7431c82fda66237133bf7e Does it work for you with this commit: http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/90ae409f9eb3bcaf38688f9ec22375816053a08e
Hi Nicolin, On Sat, Aug 24, 2019 at 7:10 AM Nicolin Chen <nicoleotsuka@gmail.com> wrote: > > On Fri, Aug 23, 2019 at 09:49:46PM +0900, Masahiro Yamada wrote: > > > > Reverting this commit fixed the problem. > > We are having another problem with the new API and Christoph > submitted a patch at: https://lkml.org/lkml/2019/8/20/86 > > Would it be possible for you to test to see if it can fix? It is included in 5.3-rc6 I tested 5.3-rc6 in on my board, but I still see the same DMA fauilure. Masahiro > We can revert my fallback change after all, if Christoph's > patch doesn't work for you either. > > Thanks > Nicolin
Christoph, On Sun, Aug 25, 2019 at 10:10 AM Christoph Hellwig <hch@lst.de> wrote: > > On Fri, Aug 23, 2019 at 09:56:52PM +0900, Masahiro Yamada wrote: > > + linux-mmc, Ulf Hansson, Adrian Hunter, > > > > > > ADMA of SDHCI is not working > > since bd2e75633c8012fc8a7431c82fda66237133bf7e > > Does it work for you with this commit: > > http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/90ae409f9eb3bcaf38688f9ec22375816053a08e This is included in v5.3-rc6 so I tested it. No, it did not fix the problem. -- Best Regards Masahiro Yamada
On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > This is included in v5.3-rc6 > so I tested it. So there is no allocation failure, but you get I/O errors later? Does the device use a device-private CMA area? Does it work with Linux 5.2 if CONFIG_DMA_CMA is disabled?
On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > This is included in v5.3-rc6 > > so I tested it. > > So there is no allocation failure, but you get I/O errors later? Right. > > Does the device use a device-private CMA area? Not sure. My driver is drivers/mmc/host/sdhci-cadence.c It reuses routines in drivers/mmc/host/sdhci.c > Does it work with Linux > 5.2 if CONFIG_DMA_CMA is disabled? No. 5.2 + disable CONFIG_DMA_CMA failed in the same way.
On Tue, Aug 27, 2019 at 04:45:20PM +0900, Masahiro Yamada wrote: > On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > > This is included in v5.3-rc6 > > > so I tested it. > > > > So there is no allocation failure, but you get I/O errors later? > > Right. > > > > > Does the device use a device-private CMA area? > > Not sure. > My driver is drivers/mmc/host/sdhci-cadence.c > It reuses routines in drivers/mmc/host/sdhci.c > > > > > Does it work with Linux > > 5.2 if CONFIG_DMA_CMA is disabled? > > No. > 5.2 + disable CONFIG_DMA_CMA > failed in the same way. So it seems like the device wants CMA memory. I guess the patch below will fix it, but that isn't the solution. Can you try it to confirm? In the end it probably assumes a dma mask it doesn't set that the CMA memory satisfies or something similar. diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 69cfb4345388..bd2f24aa7f19 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -236,7 +236,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) if (dev && dev->cma_area) cma = dev->cma_area; - else if (count > 1) + else cma = dma_contiguous_default_area; /* CMA can be used only in the context which permits sleeping */
On Tue, Aug 27, 2019 at 4:50 PM Christoph Hellwig <hch@lst.de> wrote: > > On Tue, Aug 27, 2019 at 04:45:20PM +0900, Masahiro Yamada wrote: > > On Mon, Aug 26, 2019 at 4:33 PM Christoph Hellwig <hch@lst.de> wrote: > > > > > > On Mon, Aug 26, 2019 at 11:05:00AM +0900, Masahiro Yamada wrote: > > > > This is included in v5.3-rc6 > > > > so I tested it. > > > > > > So there is no allocation failure, but you get I/O errors later? > > > > Right. > > > > > > > > Does the device use a device-private CMA area? > > > > Not sure. > > My driver is drivers/mmc/host/sdhci-cadence.c > > It reuses routines in drivers/mmc/host/sdhci.c > > > > > > > > > Does it work with Linux > > > 5.2 if CONFIG_DMA_CMA is disabled? > > > > No. > > 5.2 + disable CONFIG_DMA_CMA > > failed in the same way. > > So it seems like the device wants CMA memory. I guess the patch > below will fix it, but that isn't the solution. Can you try it > to confirm? In the end it probably assumes a dma mask it doesn't > set that the CMA memory satisfies or something similar. Thanks for the pointer. > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index 69cfb4345388..bd2f24aa7f19 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -236,7 +236,7 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) > > if (dev && dev->cma_area) > cma = dev->cma_area; > - else if (count > 1) > + else > cma = dma_contiguous_default_area; > > /* CMA can be used only in the context which permits sleeping */ Yes, this makes my driver working again when CONFIG_DMA_CMA=y. If I apply the following, my driver gets back working irrespective of CONFIG_DMA_CMA. diff --git a/drivers/mmc/host/sdhci-cadence.c b/drivers/mmc/host/sdhci-cadence.c index 163d1cf4367e..504459395c39 100644 --- a/drivers/mmc/host/sdhci-cadence.c +++ b/drivers/mmc/host/sdhci-cadence.c @@ -237,6 +237,7 @@ static const struct sdhci_ops sdhci_cdns_ops = { static const struct sdhci_pltfm_data sdhci_cdns_pltfm_data = { .ops = &sdhci_cdns_ops, + .quirks2 = SDHCI_QUIRK2_BROKEN_64_BIT_DMA, }; static int sdhci_cdns_set_tune_val(struct sdhci_host *host, unsigned int val)
On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > Yes, this makes my driver working again > when CONFIG_DMA_CMA=y. > > > If I apply the following, my driver gets back working > irrespective of CONFIG_DMA_CMA. That sounds a lot like the device simply isn't 64-bit DMA capable, and previously always got CMA allocations under the limit it actually supported. I suggest that you submit this quirk to the mmc maintainers.
Hi Christoph, On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > Yes, this makes my driver working again > > when CONFIG_DMA_CMA=y. > > > > > > If I apply the following, my driver gets back working > > irrespective of CONFIG_DMA_CMA. > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > previously always got CMA allocations under the limit it actually > supported. I suggest that you submit this quirk to the mmc maintainers. I tested v5.2 and my MMC host controller works with dma_address that exceeds 32-bit physical address. So, I believe my MMC device is 64-bit DMA capable. I am still looking into the code to find out what was changed. -- Best Regards Masahiro Yamada
On Wed, Aug 28, 2019 at 7:53 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > Hi Christoph, > > On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > > Yes, this makes my driver working again > > > when CONFIG_DMA_CMA=y. > > > > > > > > > If I apply the following, my driver gets back working > > > irrespective of CONFIG_DMA_CMA. > > > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > > previously always got CMA allocations under the limit it actually > > supported. I suggest that you submit this quirk to the mmc maintainers. > > > I tested v5.2 and my MMC host controller works with > dma_address that exceeds 32-bit physical address. > > So, I believe my MMC device is 64-bit DMA capable. > > I am still looking into the code > to find out what was changed. I retract this comment. Prior to bd2e75633c8012fc8a7431c82fda66237133bf7e, the descriptor table for ADMA is placed within the 32-bit phys address range, not exceeds the 32-bit limit. Probably, my device is not 64-bit capable. I will talk to the hardware engineer, and check the hardware spec just in case. Thanks.
On Wed, Aug 28, 2019 at 9:23 PM Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > On Wed, Aug 28, 2019 at 7:53 PM Masahiro Yamada > <yamada.masahiro@socionext.com> wrote: > > > > Hi Christoph, > > > > On Tue, Aug 27, 2019 at 8:55 PM Christoph Hellwig <hch@lst.de> wrote: > > > > > > On Tue, Aug 27, 2019 at 06:03:14PM +0900, Masahiro Yamada wrote: > > > > Yes, this makes my driver working again > > > > when CONFIG_DMA_CMA=y. > > > > > > > > > > > > If I apply the following, my driver gets back working > > > > irrespective of CONFIG_DMA_CMA. > > > > > > That sounds a lot like the device simply isn't 64-bit DMA capable, and > > > previously always got CMA allocations under the limit it actually > > > supported. I suggest that you submit this quirk to the mmc maintainers. > > > > > > I tested v5.2 and my MMC host controller works with > > dma_address that exceeds 32-bit physical address. > > > > So, I believe my MMC device is 64-bit DMA capable. > > > > I am still looking into the code > > to find out what was changed. > > > I retract this comment. > > Prior to bd2e75633c8012fc8a7431c82fda66237133bf7e, > the descriptor table for ADMA is placed within the > 32-bit phys address range, not exceeds the 32-bit limit. > > Probably, my device is not 64-bit capable. > > I will talk to the hardware engineer, > and check the hardware spec just in case. > After looking more into my hardware, I found out how to fix my driver: https://lore.kernel.org/patchwork/patch/1121600/
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 21f39a6cb04f..6914b92d5c88 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -223,14 +223,23 @@ bool dma_release_from_contiguous(struct device *dev, struct page *pages, * This function allocates contiguous memory buffer for specified device. It * first tries to use device specific contiguous memory area if available or * the default global one, then tries a fallback allocation of normal pages. + * + * Note that it byapss one-page size of allocations from the global area as + * the addresses within one page are always contiguous, so there is no need + * to waste CMA pages for that kind; it also helps reduce fragmentations. */ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp) { int node = dev ? dev_to_node(dev) : NUMA_NO_NODE; size_t count = PAGE_ALIGN(size) >> PAGE_SHIFT; size_t align = get_order(PAGE_ALIGN(size)); - struct cma *cma = dev_get_cma_area(dev); struct page *page = NULL; + struct cma *cma = NULL; + + if (dev && dev->cma_area) + cma = dev->cma_area; + else if (count > 1) + cma = dma_contiguous_default_area; /* CMA can be used only in the context which permits sleeping */ if (cma && gfpflags_allow_blocking(gfp)) {
The addresses within a single page are always contiguous, so it's not so necessary to always allocate one single page from CMA area. Since the CMA area has a limited predefined size of space, it may run out of space in heavy use cases, where there might be quite a lot CMA pages being allocated for single pages. However, there is also a concern that a device might care where a page comes from -- it might expect the page from CMA area and act differently if the page doesn't. This patch tries to use the fallback alloc_pages path, instead of one-page size allocations from the global CMA area in case that a device does not have its own CMA area. This'd save resources from the CMA global area for more CMA allocations, and also reduce CMA fragmentations resulted from trivial allocations. Signed-off-by: Nicolin Chen <nicoleotsuka@gmail.com> --- kernel/dma/contiguous.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)