| Message ID | 1506793068-27445-4-git-send-email-alagusankar@silex-india.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
silexcommon@gmail.com writes: > From: Alagu Sankar <alagusankar@silex-india.com> > > Some SD host controllers still need bounce buffers for SDIO data > transfers. While the transfers worked fine on x86 platforms, > this is found to be required for i.MX6 based systems. > > Changes are similar to and derived from the ath6kl sdio driver. > > Signed-off-by: Alagu Sankar <alagusankar@silex-india.com> Why is the bounce buffer needed exactly, what are the symptoms etc? To me this sounds like an ugly workaround for a SDIO controller driver bug.
On 2017-12-22 21:38, Kalle Valo wrote: > silexcommon@gmail.com writes: > >> From: Alagu Sankar <alagusankar@silex-india.com> >> >> Some SD host controllers still need bounce buffers for SDIO data >> transfers. While the transfers worked fine on x86 platforms, >> this is found to be required for i.MX6 based systems. >> >> Changes are similar to and derived from the ath6kl sdio driver. >> >> Signed-off-by: Alagu Sankar <alagusankar@silex-india.com> > > Why is the bounce buffer needed exactly, what are the symptoms etc? To > me this sounds like an ugly workaround for a SDIO controller driver > bug. We faced problems with i.MX6. The authentication frame sent by the driver never reached the air. The host driver accepted the buffer, but did not send out the packet to the sdio module. No errors reported anywhere, but the buffer is not accepted due to alignment. The same driver however works fine without bounce buffer on x86 platform with stdhci drivers. To make it compliant with all host controllers, we introduced the bounce buffers, similar to what was done in ath6kl_sdio drivers.
Hi, I think Kalle is pointing out that maybe it's the SDHCI driver responsibility to do the bounce buffering? -adrian
On 12/25/2017 1:26 PM, Alagu Sankar wrote: > On 2017-12-22 21:38, Kalle Valo wrote: >> silexcommon@gmail.com writes: >> >>> From: Alagu Sankar <alagusankar@silex-india.com> >>> >>> Some SD host controllers still need bounce buffers for SDIO data >>> transfers. While the transfers worked fine on x86 platforms, >>> this is found to be required for i.MX6 based systems. >>> >>> Changes are similar to and derived from the ath6kl sdio driver. >>> >>> Signed-off-by: Alagu Sankar <alagusankar@silex-india.com> >> >> Why is the bounce buffer needed exactly, what are the symptoms etc? To >> me this sounds like an ugly workaround for a SDIO controller driver bug. > > We faced problems with i.MX6. The authentication frame sent by the > driver never reached the air. The host driver accepted the buffer, but > did not send out the packet to the sdio module. No errors reported > anywhere, but the buffer is not accepted due to alignment. The same > driver however works fine without bounce buffer on x86 platform with > stdhci drivers. To make it compliant with all host controllers, we > introduced the bounce buffers, similar to what was done in ath6kl_sdio > drivers. As mentioned by Adrian the comment from Kalle is that you are solving an issue caused by the sdio host controller. Although strictly speaking it may not be a driver bug, but a requirement of the host controller hardware. Either way it seems the obvious place to solve this is in the sdio host controller driver to which the issue applies. Or make it a generic quirk which can be enabled for sdio host controller drivers that need it. However, there may reasons to do it in the networking driver. For instance, the buffer you want to transfer might be the data buffer of an sk_buff you got from the networking stack and you want to have a zero-copy solution towards the wireless device. Your solution checks for 4-byte alignment which is a requirement for ADMA as per SDIO spec. However, I have come across host controllers which have different alignment requirements. Also when CONFIG_ARCH_DMA_ADDR_T_64BIT is enabled the alignment changes from 4 to 8 bytes. So it seems you are solving a specific case you have come across, but you may want to design for more flexibility. Hope this helps. Regards, Arend
[top post for emphasis] Arend is right. You won't be the only driver which has issues with a controller that doesn't handle non-aligned data payloads. Please push it into the stack or the controller side, but not in the driver side. That'll be a forever game of whack-a-mole. -adrian (I'm living this dream right now and it's unfun) On 27 December 2017 at 10:49, Arend van Spriel <arend.vanspriel@broadcom.com> wrote: > On 12/25/2017 1:26 PM, Alagu Sankar wrote: >> >> On 2017-12-22 21:38, Kalle Valo wrote: >>> >>> silexcommon@gmail.com writes: >>> >>>> From: Alagu Sankar <alagusankar@silex-india.com> >>>> >>>> Some SD host controllers still need bounce buffers for SDIO data >>>> transfers. While the transfers worked fine on x86 platforms, >>>> this is found to be required for i.MX6 based systems. >>>> >>>> Changes are similar to and derived from the ath6kl sdio driver. >>>> >>>> Signed-off-by: Alagu Sankar <alagusankar@silex-india.com> >>> >>> >>> Why is the bounce buffer needed exactly, what are the symptoms etc? To >>> me this sounds like an ugly workaround for a SDIO controller driver bug. >> >> >> We faced problems with i.MX6. The authentication frame sent by the >> driver never reached the air. The host driver accepted the buffer, but >> did not send out the packet to the sdio module. No errors reported >> anywhere, but the buffer is not accepted due to alignment. The same >> driver however works fine without bounce buffer on x86 platform with >> stdhci drivers. To make it compliant with all host controllers, we >> introduced the bounce buffers, similar to what was done in ath6kl_sdio >> drivers. > > > As mentioned by Adrian the comment from Kalle is that you are solving an > issue caused by the sdio host controller. Although strictly speaking it may > not be a driver bug, but a requirement of the host controller hardware. > Either way it seems the obvious place to solve this is in the sdio host > controller driver to which the issue applies. Or make it a generic quirk > which can be enabled for sdio host controller drivers that need it. However, > there may reasons to do it in the networking driver. For instance, the > buffer you want to transfer might be the data buffer of an sk_buff you got > from the networking stack and you want to have a zero-copy solution towards > the wireless device. > > Your solution checks for 4-byte alignment which is a requirement for ADMA as > per SDIO spec. However, I have come across host controllers which have > different alignment requirements. Also when CONFIG_ARCH_DMA_ADDR_T_64BIT is > enabled the alignment changes from 4 to 8 bytes. So it seems you are solving > a specific case you have come across, but you may want to design for more > flexibility. > > Hope this helps. > > Regards, > Arend
Alagu Sankar <alagusankar@silex-india.com> writes: > On 2017-12-22 21:38, Kalle Valo wrote: >> silexcommon@gmail.com writes: >> >>> From: Alagu Sankar <alagusankar@silex-india.com> >>> >>> Some SD host controllers still need bounce buffers for SDIO data >>> transfers. While the transfers worked fine on x86 platforms, >>> this is found to be required for i.MX6 based systems. >>> >>> Changes are similar to and derived from the ath6kl sdio driver. >>> >>> Signed-off-by: Alagu Sankar <alagusankar@silex-india.com> >> >> Why is the bounce buffer needed exactly, what are the symptoms etc? To >> me this sounds like an ugly workaround for a SDIO controller driver >> bug. > > We faced problems with i.MX6. The authentication frame sent by the > driver never reached the air. The host driver accepted the buffer, but > did not send out the packet to the sdio module. No errors reported > anywhere, but the buffer is not accepted due to alignment. So what kind of alignment works with i.MX6 and how are the packets aligned by ath10k? Of course, this might be still a bug in ath10k but most likely it's elsewhere and should be properly investigated. There must be a much better approach to handle this problem. > The same driver however works fine without bounce buffer on x86 > platform with stdhci drivers. To make it compliant with all host > controllers, we introduced the bounce buffers, similar to what was > done in ath6kl_sdio drivers. That bounce buffer was horrible in ath6kl and with the bounce buffer you are forcing all working platforms to suffer from a copy of every packet. And ath10k is getting so complex that we really need to keep the code as simple as possible to keep it maintainable.
diff --git a/drivers/net/wireless/ath/ath10k/sdio.c b/drivers/net/wireless/ath/ath10k/sdio.c index 03a69e5..77d4fa4 100644 --- a/drivers/net/wireless/ath/ath10k/sdio.c +++ b/drivers/net/wireless/ath/ath10k/sdio.c @@ -34,8 +34,20 @@ #include "trace.h" #include "sdio.h" +#define ATH10K_SDIO_DMA_BUF_SIZE (32 * 1024) + /* inlined helper functions */ +/* Macro to check if DMA buffer is WORD-aligned and DMA-able. + * Most host controllers assume the buffer is DMA'able and will + * bug-check otherwise (i.e. buffers on the stack). virt_addr_valid + * check fails on stack memory. + */ +static inline bool buf_needs_bounce(const u8 *buf) +{ + return ((unsigned long)buf & 0x3) || !virt_addr_valid(buf); +} + static inline int ath10k_sdio_calc_txrx_padded_len(struct ath10k_sdio *ar_sdio, size_t len) { @@ -303,15 +315,29 @@ static int ath10k_sdio_read(struct ath10k *ar, u32 addr, void *buf, size_t len) { struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar); struct sdio_func *func = ar_sdio->func; + bool bounced = false; + u8 *tbuf = NULL; int ret; + if (buf_needs_bounce(buf)) { + if (!ar_sdio->dma_buffer) + return -ENOMEM; + mutex_lock(&ar_sdio->dma_buffer_mutex); + tbuf = ar_sdio->dma_buffer; + bounced = true; + } else { + tbuf = buf; + } + sdio_claim_host(func); - ret = sdio_memcpy_fromio(func, buf, addr, len); + ret = sdio_memcpy_fromio(func, tbuf, addr, len); if (ret) { ath10k_warn(ar, "failed to read from address 0x%x: %d\n", addr, ret); goto out; + } else if (bounced) { + memcpy(buf, tbuf, len); } ath10k_dbg(ar, ATH10K_DBG_SDIO, "sdio read addr 0x%x buf 0x%p len %zu\n", @@ -320,6 +346,8 @@ static int ath10k_sdio_read(struct ath10k *ar, u32 addr, void *buf, size_t len) out: sdio_release_host(func); + if (bounced) + mutex_unlock(&ar_sdio->dma_buffer_mutex); return ret; } @@ -328,14 +356,27 @@ static int ath10k_sdio_write(struct ath10k *ar, u32 addr, const void *buf, size_ { struct ath10k_sdio *ar_sdio = ath10k_sdio_priv(ar); struct sdio_func *func = ar_sdio->func; + bool bounced = false; + u8 *tbuf = NULL; int ret; + if (buf_needs_bounce(buf)) { + if (!ar_sdio->dma_buffer) + return -ENOMEM; + mutex_lock(&ar_sdio->dma_buffer_mutex); + tbuf = ar_sdio->dma_buffer; + memcpy(tbuf, buf, len); + bounced = true; + } else { + tbuf = (u8 *)buf; + } + sdio_claim_host(func); /* For some reason toio() doesn't have const for the buffer, need * an ugly hack to workaround that. */ - ret = sdio_memcpy_toio(func, addr, (void *)buf, len); + ret = sdio_memcpy_toio(func, addr, (void *)tbuf, len); if (ret) { ath10k_warn(ar, "failed to write to address 0x%x: %d\n", addr, ret); @@ -348,6 +389,8 @@ static int ath10k_sdio_write(struct ath10k *ar, u32 addr, const void *buf, size_ out: sdio_release_host(func); + if (bounced) + mutex_unlock(&ar_sdio->dma_buffer_mutex); return ret; } @@ -1978,6 +2021,12 @@ static int ath10k_sdio_probe(struct sdio_func *func, goto err_free_en_reg; } + ar_sdio->dma_buffer = kzalloc(ATH10K_SDIO_DMA_BUF_SIZE, GFP_KERNEL); + if (!ar_sdio->dma_buffer) { + ret = -ENOMEM; + goto err_free_bmi_buf; + } + ar_sdio->func = func; sdio_set_drvdata(func, ar_sdio); @@ -1987,6 +2036,7 @@ static int ath10k_sdio_probe(struct sdio_func *func, spin_lock_init(&ar_sdio->lock); spin_lock_init(&ar_sdio->wr_async_lock); mutex_init(&ar_sdio->irq_data.mtx); + mutex_init(&ar_sdio->dma_buffer_mutex); INIT_LIST_HEAD(&ar_sdio->bus_req_freeq); INIT_LIST_HEAD(&ar_sdio->wr_asyncq); @@ -1995,7 +2045,7 @@ static int ath10k_sdio_probe(struct sdio_func *func, ar_sdio->workqueue = create_singlethread_workqueue("ath10k_sdio_wq"); if (!ar_sdio->workqueue) { ret = -ENOMEM; - goto err_free_bmi_buf; + goto err_dma; } for (i = 0; i < ATH10K_SDIO_BUS_REQUEST_MAX_NUM; i++) @@ -2040,6 +2090,8 @@ static int ath10k_sdio_probe(struct sdio_func *func, err_free_wq: destroy_workqueue(ar_sdio->workqueue); +err_dma: + kfree(ar_sdio->dma_buffer); err_free_bmi_buf: kfree(ar_sdio->bmi_buf); err_free_en_reg: @@ -2065,6 +2117,7 @@ static void ath10k_sdio_remove(struct sdio_func *func) cancel_work_sync(&ar_sdio->wr_async_work); ath10k_core_unregister(ar); ath10k_core_destroy(ar); + kfree(ar_sdio->dma_buffer); } static const struct sdio_device_id ath10k_sdio_devices[] = { diff --git a/drivers/net/wireless/ath/ath10k/sdio.h b/drivers/net/wireless/ath/ath10k/sdio.h index 4ff7b54..718b8b7 100644 --- a/drivers/net/wireless/ath/ath10k/sdio.h +++ b/drivers/net/wireless/ath/ath10k/sdio.h @@ -207,6 +207,11 @@ struct ath10k_sdio { struct ath10k *ar; struct ath10k_sdio_irq_data irq_data; + u8 *dma_buffer; + + /* protects access to dma_buffer */ + struct mutex dma_buffer_mutex; + /* temporary buffer for BMI requests */ u8 *bmi_buf;