| Message ID | 1458252137-24497-1-git-send-email-okaya@codeaurora.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Mar 17, 2016 at 06:02:15PM -0400, Sinan Kaya wrote: > Getting ready to remove dma_to_phys API. Drivers should not be > using this API for DMA operations. Instead, they should go > through the dma_map or dma_alloc APIs. > > Signed-off-by: Sinan Kaya <okaya@codeaurora.org> > --- > drivers/crypto/marvell/cesa.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/crypto/marvell/cesa.c b/drivers/crypto/marvell/cesa.c > index c0656e7..52ddfa4 100644 > --- a/drivers/crypto/marvell/cesa.c > +++ b/drivers/crypto/marvell/cesa.c > @@ -350,8 +350,8 @@ static int mv_cesa_get_sram(struct platform_device *pdev, int idx) > if (IS_ERR(engine->sram)) > return PTR_ERR(engine->sram); > > - engine->sram_dma = phys_to_dma(cesa->dev, > - (phys_addr_t)res->start); > + engine->sram_dma = dma_map_single(cesa->dev, engine->sram, > + DMA_TO_DEVICE); Documentation/DMA-API.txt dma_addr_t dma_map_single(struct device *dev, void *cpu_addr, size_t size, enum dma_data_direction direction) Notes: Not all memory regions in a machine can be mapped by this API. Further, contiguous kernel virtual space may not be contiguous as physical memory. Since this API does not provide any scatter/gather capability, it will fail if the user tries to map a non-physically contiguous piece of memory. For this reason, memory to be mapped by this API should be obtained from sources which guarantee it to be physically contiguous (like kmalloc). Specifically, ioremapped memory will *not* work as you expect with dma_map_single().
On 2016-03-17 18:54, Russell King - ARM Linux wrote: > On Thu, Mar 17, 2016 at 06:02:15PM -0400, Sinan Kaya wrote: >> Getting ready to remove dma_to_phys API. Drivers should not be >> using this API for DMA operations. Instead, they should go >> through the dma_map or dma_alloc APIs. >> >> Signed-off-by: Sinan Kaya <okaya@codeaurora.org> >> --- >> drivers/crypto/marvell/cesa.c | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/crypto/marvell/cesa.c >> b/drivers/crypto/marvell/cesa.c >> index c0656e7..52ddfa4 100644 >> --- a/drivers/crypto/marvell/cesa.c >> +++ b/drivers/crypto/marvell/cesa.c >> @@ -350,8 +350,8 @@ static int mv_cesa_get_sram(struct platform_device >> *pdev, int idx) >> if (IS_ERR(engine->sram)) >> return PTR_ERR(engine->sram); >> >> - engine->sram_dma = phys_to_dma(cesa->dev, >> - (phys_addr_t)res->start); >> + engine->sram_dma = dma_map_single(cesa->dev, engine->sram, >> + DMA_TO_DEVICE); > > Documentation/DMA-API.txt > > dma_addr_t > dma_map_single(struct device *dev, void *cpu_addr, size_t size, > enum dma_data_direction direction) > > Notes: Not all memory regions in a machine can be mapped by this API. > Further, contiguous kernel virtual space may not be contiguous as > physical memory. Since this API does not provide any scatter/gather > capability, it will fail if the user tries to map a non-physically > contiguous piece of memory. For this reason, memory to be mapped by > this API should be obtained from sources which guarantee it to be > physically contiguous (like kmalloc). > > Specifically, ioremapped memory will *not* work as you expect with > dma_map_single(). What is the correct way? I don't want to write engine->sram_dma = sram
On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
> What is the correct way? I don't want to write engine->sram_dma = sram
Well, what the driver _is_ wanting to do is to go from a CPU physical
address to a device DMA address. phys_to_dma() looks like the correct
thing there to me, but I guess that's just an offset and doesn't take
account of any IOMMU that may be in the way.
If you have an IOMMU, then the whole phys_to_dma() thing is a total
failure as it only does a linear translation, and there are no
interfaces in the kernel to take account of an IOMMU in the way. So,
it needs something designed for the job, implemented and discussed by
the normal methods of proposing a new cross-arch interface for drivers
to use.
What I'm certain of, though, is that the change proposed in this patch
will break current users of this driver: virt_to_page() on an address
returned by ioremap() is completely undefined, and will result in
either a kernel oops, or if not poking at memory which isn't a struct
page, ultimately resulting in something that isn't SRAM being pointed
to by "engine->sram_dma".
On Thu, 17 Mar 2016 23:50:20 +0000 Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: > On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: > > What is the correct way? I don't want to write engine->sram_dma = sram > > Well, what the driver _is_ wanting to do is to go from a CPU physical > address to a device DMA address. phys_to_dma() looks like the correct > thing there to me, but I guess that's just an offset and doesn't take > account of any IOMMU that may be in the way. > > If you have an IOMMU, then the whole phys_to_dma() thing is a total > failure as it only does a linear translation, and there are no > interfaces in the kernel to take account of an IOMMU in the way. So, > it needs something designed for the job, implemented and discussed by > the normal methods of proposing a new cross-arch interface for drivers > to use. > > What I'm certain of, though, is that the change proposed in this patch > will break current users of this driver: virt_to_page() on an address > returned by ioremap() is completely undefined, and will result in > either a kernel oops, or if not poking at memory which isn't a struct > page, ultimately resulting in something that isn't SRAM being pointed > to by "engine->sram_dma". > Or we could just do engine->sram_dma = res->start; which is pretty much what the SRAM/genalloc code is doing already.
On 18/03/16 09:30, Boris Brezillon wrote: > On Thu, 17 Mar 2016 23:50:20 +0000 > Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: > >> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: >>> What is the correct way? I don't want to write engine->sram_dma = sram >> >> Well, what the driver _is_ wanting to do is to go from a CPU physical >> address to a device DMA address. phys_to_dma() looks like the correct >> thing there to me, but I guess that's just an offset and doesn't take >> account of any IOMMU that may be in the way. >> >> If you have an IOMMU, then the whole phys_to_dma() thing is a total >> failure as it only does a linear translation, and there are no >> interfaces in the kernel to take account of an IOMMU in the way. So, >> it needs something designed for the job, implemented and discussed by >> the normal methods of proposing a new cross-arch interface for drivers >> to use. >> >> What I'm certain of, though, is that the change proposed in this patch >> will break current users of this driver: virt_to_page() on an address >> returned by ioremap() is completely undefined, and will result in >> either a kernel oops, or if not poking at memory which isn't a struct >> page, ultimately resulting in something that isn't SRAM being pointed >> to by "engine->sram_dma". >> > > Or we could just do > > engine->sram_dma = res->start; > > which is pretty much what the SRAM/genalloc code is doing already. As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be. Robin. [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
On Fri, 18 Mar 2016 11:25:48 +0000 Robin Murphy <robin.murphy@arm.com> wrote: > On 18/03/16 09:30, Boris Brezillon wrote: > > On Thu, 17 Mar 2016 23:50:20 +0000 > > Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: > > > >> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: > >>> What is the correct way? I don't want to write engine->sram_dma = sram > >> > >> Well, what the driver _is_ wanting to do is to go from a CPU physical > >> address to a device DMA address. phys_to_dma() looks like the correct > >> thing there to me, but I guess that's just an offset and doesn't take > >> account of any IOMMU that may be in the way. > >> > >> If you have an IOMMU, then the whole phys_to_dma() thing is a total > >> failure as it only does a linear translation, and there are no > >> interfaces in the kernel to take account of an IOMMU in the way. So, > >> it needs something designed for the job, implemented and discussed by > >> the normal methods of proposing a new cross-arch interface for drivers > >> to use. > >> > >> What I'm certain of, though, is that the change proposed in this patch > >> will break current users of this driver: virt_to_page() on an address > >> returned by ioremap() is completely undefined, and will result in > >> either a kernel oops, or if not poking at memory which isn't a struct > >> page, ultimately resulting in something that isn't SRAM being pointed > >> to by "engine->sram_dma". > >> > > > > Or we could just do > > > > engine->sram_dma = res->start; > > > > which is pretty much what the SRAM/genalloc code is doing already. > > As Russell points out this is yet another type of "set up a DMA master > to access something other than kernel RAM" - there's already discussion > in progress over how to handle this for dmaengine slaves[1], so > gathering more use-cases might help distil exactly what the design of > not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else > needs to be. > > Robin. > > [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html > Hm, interesting, thanks for the pointer.
On 3/18/2016 7:25 AM, Robin Murphy wrote: > On 18/03/16 09:30, Boris Brezillon wrote: >> On Thu, 17 Mar 2016 23:50:20 +0000 >> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: >> >>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: >>>> What is the correct way? I don't want to write engine->sram_dma = sram >>> >>> Well, what the driver _is_ wanting to do is to go from a CPU physical >>> address to a device DMA address. phys_to_dma() looks like the correct >>> thing there to me, but I guess that's just an offset and doesn't take >>> account of any IOMMU that may be in the way. >>> >>> If you have an IOMMU, then the whole phys_to_dma() thing is a total >>> failure as it only does a linear translation, and there are no >>> interfaces in the kernel to take account of an IOMMU in the way. So, >>> it needs something designed for the job, implemented and discussed by >>> the normal methods of proposing a new cross-arch interface for drivers >>> to use. >>> >>> What I'm certain of, though, is that the change proposed in this patch >>> will break current users of this driver: virt_to_page() on an address >>> returned by ioremap() is completely undefined, and will result in >>> either a kernel oops, or if not poking at memory which isn't a struct >>> page, ultimately resulting in something that isn't SRAM being pointed >>> to by "engine->sram_dma". >>> >> >> Or we could just do >> >> engine->sram_dma = res->start; >> >> which is pretty much what the SRAM/genalloc code is doing already. > > As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be. > > Robin. > > [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html > Thanks for the link. dma_map_resource looks like to be the correct way of doing things. Just from the purist point of view, a driver is not supposed to know the physical address of a DMA address. That kills the intent of using DMA API. When programming descriptors, the DMA addresses should be programmed not physical addresses so that the same driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA address to a bus address that is not physical address. All of this operation needs to be isolated from the device driver. I don't know the architecture or the driver enough to write this. This is not ideally right but I can do this if Boris you are OK with this. engine->sram_dma = res->start; Another option is I can write engine->sram_dma = swiotlb_dma_to_phys(res->start) I agree that dma_map_single is not the right thing.
On 3/18/2016 9:51 AM, Sinan Kaya wrote: > Another option is I can write > > engine->sram_dma = swiotlb_dma_to_phys(res->start) I realized that I made a mistake in the commit message and the code above. The code is trying to find DMA address from physical address. Not the other way around. I'll fix it on the next version. The correct suggestion above would be engine->sram_dma = swiotlb_phys_to_dmares->start)
On Fri, 18 Mar 2016 09:51:37 -0400 Sinan Kaya <okaya@codeaurora.org> wrote: > On 3/18/2016 7:25 AM, Robin Murphy wrote: > > On 18/03/16 09:30, Boris Brezillon wrote: > >> On Thu, 17 Mar 2016 23:50:20 +0000 > >> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: > >> > >>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: > >>>> What is the correct way? I don't want to write engine->sram_dma = sram > >>> > >>> Well, what the driver _is_ wanting to do is to go from a CPU physical > >>> address to a device DMA address. phys_to_dma() looks like the correct > >>> thing there to me, but I guess that's just an offset and doesn't take > >>> account of any IOMMU that may be in the way. > >>> > >>> If you have an IOMMU, then the whole phys_to_dma() thing is a total > >>> failure as it only does a linear translation, and there are no > >>> interfaces in the kernel to take account of an IOMMU in the way. So, > >>> it needs something designed for the job, implemented and discussed by > >>> the normal methods of proposing a new cross-arch interface for drivers > >>> to use. > >>> > >>> What I'm certain of, though, is that the change proposed in this patch > >>> will break current users of this driver: virt_to_page() on an address > >>> returned by ioremap() is completely undefined, and will result in > >>> either a kernel oops, or if not poking at memory which isn't a struct > >>> page, ultimately resulting in something that isn't SRAM being pointed > >>> to by "engine->sram_dma". > >>> > >> > >> Or we could just do > >> > >> engine->sram_dma = res->start; > >> > >> which is pretty much what the SRAM/genalloc code is doing already. > > > > As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be. > > > > Robin. > > > > [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html > > > > Thanks for the link. > > dma_map_resource looks like to be the correct way of doing things. Just from > the purist point of view, a driver is not supposed to know the physical address > of a DMA address. That kills the intent of using DMA API. When programming descriptors, > the DMA addresses should be programmed not physical addresses so that the same > driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA > address to a bus address that is not physical address. All of this operation needs > to be isolated from the device driver. > > > I don't know the architecture or the driver enough to write this. This is not ideally > right but I can do this if Boris you are OK with this. > > engine->sram_dma = res->start; I don't know. How about waiting for the 'dma_{map,unmap}_resource' discussion to settle down before removing phy_to_dma()/dma_to_phys() APIs (as suggested by Robin and Russell)?
On 3/18/2016 10:20 AM, Boris Brezillon wrote: > On Fri, 18 Mar 2016 09:51:37 -0400 > Sinan Kaya <okaya@codeaurora.org> wrote: > >> On 3/18/2016 7:25 AM, Robin Murphy wrote: >>> On 18/03/16 09:30, Boris Brezillon wrote: >>>> On Thu, 17 Mar 2016 23:50:20 +0000 >>>> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote: >>>> >>>>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote: >>>>>> What is the correct way? I don't want to write engine->sram_dma = sram >>>>> >>>>> Well, what the driver _is_ wanting to do is to go from a CPU physical >>>>> address to a device DMA address. phys_to_dma() looks like the correct >>>>> thing there to me, but I guess that's just an offset and doesn't take >>>>> account of any IOMMU that may be in the way. >>>>> >>>>> If you have an IOMMU, then the whole phys_to_dma() thing is a total >>>>> failure as it only does a linear translation, and there are no >>>>> interfaces in the kernel to take account of an IOMMU in the way. So, >>>>> it needs something designed for the job, implemented and discussed by >>>>> the normal methods of proposing a new cross-arch interface for drivers >>>>> to use. >>>>> >>>>> What I'm certain of, though, is that the change proposed in this patch >>>>> will break current users of this driver: virt_to_page() on an address >>>>> returned by ioremap() is completely undefined, and will result in >>>>> either a kernel oops, or if not poking at memory which isn't a struct >>>>> page, ultimately resulting in something that isn't SRAM being pointed >>>>> to by "engine->sram_dma". >>>>> >>>> >>>> Or we could just do >>>> >>>> engine->sram_dma = res->start; >>>> >>>> which is pretty much what the SRAM/genalloc code is doing already. >>> >>> As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be. >>> >>> Robin. >>> >>> [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html >>> >> >> Thanks for the link. >> >> dma_map_resource looks like to be the correct way of doing things. Just from >> the purist point of view, a driver is not supposed to know the physical address >> of a DMA address. That kills the intent of using DMA API. When programming descriptors, >> the DMA addresses should be programmed not physical addresses so that the same >> driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA >> address to a bus address that is not physical address. All of this operation needs >> to be isolated from the device driver. >> >> >> I don't know the architecture or the driver enough to write this. This is not ideally >> right but I can do this if Boris you are OK with this. >> >> engine->sram_dma = res->start; > > I don't know. > > How about waiting for the 'dma_{map,unmap}_resource' discussion to > settle down before removing phy_to_dma()/dma_to_phys() APIs (as > suggested by Robin and Russell)? > > Sure, that's fine for me.
Hi Sinan,
[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on v4.5 next-20160318]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]
url: https://github.com/0day-ci/linux/commits/Sinan-Kaya/crypto-marvell-cesa-replace-dma_to_phys-with-dma_map_single/20160318-060640
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/core
config: arm-allmodconfig (attached as .config)
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm
All errors (new ones prefixed by >>):
drivers/crypto/marvell/cesa.c: In function 'mv_cesa_get_sram':
>> drivers/crypto/marvell/cesa.c:354:21: error: macro "dma_map_single" requires 4 arguments, but only 3 given
DMA_TO_DEVICE);
^
>> drivers/crypto/marvell/cesa.c:353:21: error: 'dma_map_single' undeclared (first use in this function)
engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
^
drivers/crypto/marvell/cesa.c:353:21: note: each undeclared identifier is reported only once for each function it appears in
vim +/dma_map_single +354 drivers/crypto/marvell/cesa.c
347 return -EINVAL;
348
349 engine->sram = devm_ioremap_resource(cesa->dev, res);
350 if (IS_ERR(engine->sram))
351 return PTR_ERR(engine->sram);
352
> 353 engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
> 354 DMA_TO_DEVICE);
355
356 return 0;
357 }
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
diff --git a/drivers/crypto/marvell/cesa.c b/drivers/crypto/marvell/cesa.c index c0656e7..52ddfa4 100644 --- a/drivers/crypto/marvell/cesa.c +++ b/drivers/crypto/marvell/cesa.c @@ -350,8 +350,8 @@ static int mv_cesa_get_sram(struct platform_device *pdev, int idx) if (IS_ERR(engine->sram)) return PTR_ERR(engine->sram); - engine->sram_dma = phys_to_dma(cesa->dev, - (phys_addr_t)res->start); + engine->sram_dma = dma_map_single(cesa->dev, engine->sram, + DMA_TO_DEVICE); return 0; }
Getting ready to remove dma_to_phys API. Drivers should not be using this API for DMA operations. Instead, they should go through the dma_map or dma_alloc APIs. Signed-off-by: Sinan Kaya <okaya@codeaurora.org> --- drivers/crypto/marvell/cesa.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)