| Message ID | 20190111180523.27862-14-afd@ti.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Misc ION cleanups and adding unmapped heap | expand |
On Fri, 11 Jan 2019, Andrew F. Davis wrote: > Buffers may not be mapped from the CPU so skip cache maintenance here. > Accesses from the CPU to a cached heap should be bracketed with > {begin,end}_cpu_access calls so maintenance should not be needed anyway. > > Signed-off-by: Andrew F. Davis <afd@ti.com> > --- > drivers/staging/android/ion/ion.c | 7 ++++--- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > index 14e48f6eb734..09cb5a8e2b09 100644 > --- a/drivers/staging/android/ion/ion.c > +++ b/drivers/staging/android/ion/ion.c > @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > > table = a->table; > > - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > - direction)) > + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > + direction, DMA_ATTR_SKIP_CPU_SYNC)) Unfortunately I don't think you can do this for a couple reasons. You can't rely on {begin,end}_cpu_access calls to do cache maintenance. If the calls to {begin,end}_cpu_access were made before the call to dma_buf_attach then there won't have been a device attached so the calls to {begin,end}_cpu_access won't have done any cache maintenance. Also ION no longer provides DMA ready memory, so if you are not doing CPU access then there is no requirement (that I am aware of) for you to call {begin,end}_cpu_access before passing the buffer to the device and if this buffer is cached and your device is not IO-coherent then the cache maintenance in ion_map_dma_buf and ion_unmap_dma_buf is required. > return ERR_PTR(-ENOMEM); > > return table; > @@ -272,7 +272,8 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment, > struct sg_table *table, > enum dma_data_direction direction) > { > - dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction); > + dma_unmap_sg_attrs(attachment->dev, table->sgl, table->nents, > + direction, DMA_ATTR_SKIP_CPU_SYNC); > } > > static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) > -- > 2.19.1 > > _______________________________________________ > devel mailing list > devel@linuxdriverproject.org > http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On 1/14/19 11:13 AM, Liam Mark wrote: > On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >> Buffers may not be mapped from the CPU so skip cache maintenance here. >> Accesses from the CPU to a cached heap should be bracketed with >> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >> >> Signed-off-by: Andrew F. Davis <afd@ti.com> >> --- >> drivers/staging/android/ion/ion.c | 7 ++++--- >> 1 file changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >> index 14e48f6eb734..09cb5a8e2b09 100644 >> --- a/drivers/staging/android/ion/ion.c >> +++ b/drivers/staging/android/ion/ion.c >> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >> >> table = a->table; >> >> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >> - direction)) >> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > > Unfortunately I don't think you can do this for a couple reasons. > You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > If the calls to {begin,end}_cpu_access were made before the call to > dma_buf_attach then there won't have been a device attached so the calls > to {begin,end}_cpu_access won't have done any cache maintenance. > That should be okay though, if you have no attachments (or all attachments are IO-coherent) then there is no need for cache maintenance. Unless you mean a sequence where a non-io-coherent device is attached later after data has already been written. Does that sequence need supporting? DMA-BUF doesn't have to allocate the backing memory until map_dma_buf() time, and that should only happen after all the devices have attached so it can know where to put the buffer. So we shouldn't expect any CPU access to buffers before all the devices are attached and mapped, right? > Also ION no longer provides DMA ready memory, so if you are not doing CPU > access then there is no requirement (that I am aware of) for you to call > {begin,end}_cpu_access before passing the buffer to the device and if this > buffer is cached and your device is not IO-coherent then the cache maintenance > in ion_map_dma_buf and ion_unmap_dma_buf is required. > If I am not doing any CPU access then why do I need CPU cache maintenance on the buffer? Andrew >> return ERR_PTR(-ENOMEM); >> >> return table; >> @@ -272,7 +272,8 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment, >> struct sg_table *table, >> enum dma_data_direction direction) >> { >> - dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction); >> + dma_unmap_sg_attrs(attachment->dev, table->sgl, table->nents, >> + direction, DMA_ATTR_SKIP_CPU_SYNC); >> } >> >> static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) >> -- >> 2.19.1 >> >> _______________________________________________ >> devel mailing list >> devel@linuxdriverproject.org >> http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On Tue, 15 Jan 2019, Andrew F. Davis wrote: > On 1/14/19 11:13 AM, Liam Mark wrote: > > On Fri, 11 Jan 2019, Andrew F. Davis wrote: > > > >> Buffers may not be mapped from the CPU so skip cache maintenance here. > >> Accesses from the CPU to a cached heap should be bracketed with > >> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >> > >> Signed-off-by: Andrew F. Davis <afd@ti.com> > >> --- > >> drivers/staging/android/ion/ion.c | 7 ++++--- > >> 1 file changed, 4 insertions(+), 3 deletions(-) > >> > >> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >> index 14e48f6eb734..09cb5a8e2b09 100644 > >> --- a/drivers/staging/android/ion/ion.c > >> +++ b/drivers/staging/android/ion/ion.c > >> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >> > >> table = a->table; > >> > >> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >> - direction)) > >> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > > > > Unfortunately I don't think you can do this for a couple reasons. > > You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > > If the calls to {begin,end}_cpu_access were made before the call to > > dma_buf_attach then there won't have been a device attached so the calls > > to {begin,end}_cpu_access won't have done any cache maintenance. > > > > That should be okay though, if you have no attachments (or all > attachments are IO-coherent) then there is no need for cache > maintenance. Unless you mean a sequence where a non-io-coherent device > is attached later after data has already been written. Does that > sequence need supporting? Yes, but also I think there are cases where CPU access can happen before in Android, but I will focus on later for now. > DMA-BUF doesn't have to allocate the backing > memory until map_dma_buf() time, and that should only happen after all > the devices have attached so it can know where to put the buffer. So we > shouldn't expect any CPU access to buffers before all the devices are > attached and mapped, right? > Here is an example where CPU access can happen later in Android. Camera device records video -> software post processing -> video device (who does compression of raw data) and writes to a file In this example assume the buffer is cached and the devices are not IO-coherent (quite common). ION buffer is allocated. //Camera device records video dma_buf_attach dma_map_attachment (buffer needs to be cleaned) [camera device writes to buffer] dma_buf_unmap_attachment (buffer needs to be invalidated) dma_buf_detach (device cannot stay attached because it is being sent down the pipeline and Camera doesn't know the end of the use case) //buffer is send down the pipeline // Usersapce software post processing occurs mmap buffer DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no devices attached to buffer [CPU reads/writes to the buffer] DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no devices attached to buffer munmap buffer //buffer is send down the pipeline // Buffer is send to video device (who does compression of raw data) and writes to a file dma_buf_attach dma_map_attachment (buffer needs to be cleaned) [video device writes to buffer] dma_buf_unmap_attachment dma_buf_detach (device cannot stay attached because it is being sent down the pipeline and Video doesn't know the end of the use case) > > Also ION no longer provides DMA ready memory, so if you are not doing CPU > > access then there is no requirement (that I am aware of) for you to call > > {begin,end}_cpu_access before passing the buffer to the device and if this > > buffer is cached and your device is not IO-coherent then the cache maintenance > > in ion_map_dma_buf and ion_unmap_dma_buf is required. > > > > If I am not doing any CPU access then why do I need CPU cache > maintenance on the buffer? > Because ION no longer provides DMA ready memory. Take the above example. ION allocates memory from buddy allocator and requests zeroing. Zeros are written to the cache. You pass the buffer to the camera device which is not IO-coherent. The camera devices writes directly to the buffer in DDR. Since you didn't clean the buffer a dirty cache line (one of the zeros) is evicted from the cache, this zero overwrites data the camera device has written which corrupts your data. Liam Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On 1/15/19 11:45 AM, Liam Mark wrote: > On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >> On 1/14/19 11:13 AM, Liam Mark wrote: >>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>> >>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>> Accesses from the CPU to a cached heap should be bracketed with >>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>> >>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>> --- >>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>> --- a/drivers/staging/android/ion/ion.c >>>> +++ b/drivers/staging/android/ion/ion.c >>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>> >>>> table = a->table; >>>> >>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>> - direction)) >>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>> >>> Unfortunately I don't think you can do this for a couple reasons. >>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>> If the calls to {begin,end}_cpu_access were made before the call to >>> dma_buf_attach then there won't have been a device attached so the calls >>> to {begin,end}_cpu_access won't have done any cache maintenance. >>> >> >> That should be okay though, if you have no attachments (or all >> attachments are IO-coherent) then there is no need for cache >> maintenance. Unless you mean a sequence where a non-io-coherent device >> is attached later after data has already been written. Does that >> sequence need supporting? > > Yes, but also I think there are cases where CPU access can happen before > in Android, but I will focus on later for now. > >> DMA-BUF doesn't have to allocate the backing >> memory until map_dma_buf() time, and that should only happen after all >> the devices have attached so it can know where to put the buffer. So we >> shouldn't expect any CPU access to buffers before all the devices are >> attached and mapped, right? >> > > Here is an example where CPU access can happen later in Android. > > Camera device records video -> software post processing -> video device > (who does compression of raw data) and writes to a file > > In this example assume the buffer is cached and the devices are not > IO-coherent (quite common). > This is the start of the problem, having cached mappings of memory that is also being accessed non-coherently is going to cause issues one way or another. On top of the speculative cache fills that have to be constantly fought back against with CMOs like below; some coherent interconnects behave badly when you mix coherent and non-coherent access (snoop filters get messed up). The solution is to either always have the addresses marked non-coherent (like device memory, no-map carveouts), or if you really want to use regular system memory allocated at runtime, then all cached mappings of it need to be dropped, even the kernel logical address (area as painful as that would be). > ION buffer is allocated. > > //Camera device records video > dma_buf_attach > dma_map_attachment (buffer needs to be cleaned) Why does the buffer need to be cleaned here? I just got through reading the thread linked by Laura in the other reply. I do like +Brian's suggestion of tracking if the buffer has had CPU access since the last time and only flushing the cache if it has. As unmapped heaps never get CPU mapped this would never be the case for unmapped heaps, it solves my problem. > [camera device writes to buffer] > dma_buf_unmap_attachment (buffer needs to be invalidated) It doesn't know there will be any further CPU access, it could get freed after this for all we know, the invalidate can be saved until the CPU requests access again. > dma_buf_detach (device cannot stay attached because it is being sent down > the pipeline and Camera doesn't know the end of the use case) > This seems like a broken use-case, I understand the desire to keep everything as modular as possible and separate the steps, but at this point no one owns this buffers backing memory, not the CPU or any device. I would go as far as to say DMA-BUF should be free now to de-allocate the backing storage if it wants, that way it could get ready for the next attachment, which may change the required backing memory completely. All devices should attach before the first mapping, and only let go after the task is complete, otherwise this buffers data needs copied off to a different location or the CPU needs to take ownership in-between. > //buffer is send down the pipeline > > // Usersapce software post processing occurs > mmap buffer Perhaps the invalidate should happen here in mmap. > DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > devices attached to buffer And that should be okay, mmap does the sync, and if no devices are attached nothing could have changed the underlying memory in the mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > [CPU reads/writes to the buffer] > DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > devices attached to buffer > munmap buffer > > //buffer is send down the pipeline > // Buffer is send to video device (who does compression of raw data) and > writes to a file > dma_buf_attach > dma_map_attachment (buffer needs to be cleaned) > [video device writes to buffer] > dma_buf_unmap_attachment > dma_buf_detach (device cannot stay attached because it is being sent down > the pipeline and Video doesn't know the end of the use case) > > > >>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>> access then there is no requirement (that I am aware of) for you to call >>> {begin,end}_cpu_access before passing the buffer to the device and if this >>> buffer is cached and your device is not IO-coherent then the cache maintenance >>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>> >> >> If I am not doing any CPU access then why do I need CPU cache >> maintenance on the buffer? >> > > Because ION no longer provides DMA ready memory. > Take the above example. > > ION allocates memory from buddy allocator and requests zeroing. > Zeros are written to the cache. > > You pass the buffer to the camera device which is not IO-coherent. > The camera devices writes directly to the buffer in DDR. > Since you didn't clean the buffer a dirty cache line (one of the zeros) is > evicted from the cache, this zero overwrites data the camera device has > written which corrupts your data. > The zeroing *is* a CPU access, therefor it should handle the needed CMO for CPU access at the time of zeroing. Andrew > Liam > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On 1/15/19 12:38 PM, Andrew F. Davis wrote: > On 1/15/19 11:45 AM, Liam Mark wrote: >> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >> >>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>> >>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>> >>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>> --- >>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>> >>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>> --- a/drivers/staging/android/ion/ion.c >>>>> +++ b/drivers/staging/android/ion/ion.c >>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>> >>>>> table = a->table; >>>>> >>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>> - direction)) >>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>> >>>> Unfortunately I don't think you can do this for a couple reasons. >>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>> If the calls to {begin,end}_cpu_access were made before the call to >>>> dma_buf_attach then there won't have been a device attached so the calls >>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>> >>> >>> That should be okay though, if you have no attachments (or all >>> attachments are IO-coherent) then there is no need for cache >>> maintenance. Unless you mean a sequence where a non-io-coherent device >>> is attached later after data has already been written. Does that >>> sequence need supporting? >> >> Yes, but also I think there are cases where CPU access can happen before >> in Android, but I will focus on later for now. >> >>> DMA-BUF doesn't have to allocate the backing >>> memory until map_dma_buf() time, and that should only happen after all >>> the devices have attached so it can know where to put the buffer. So we >>> shouldn't expect any CPU access to buffers before all the devices are >>> attached and mapped, right? >>> >> >> Here is an example where CPU access can happen later in Android. >> >> Camera device records video -> software post processing -> video device >> (who does compression of raw data) and writes to a file >> >> In this example assume the buffer is cached and the devices are not >> IO-coherent (quite common). >> > > This is the start of the problem, having cached mappings of memory that > is also being accessed non-coherently is going to cause issues one way > or another. On top of the speculative cache fills that have to be > constantly fought back against with CMOs like below; some coherent > interconnects behave badly when you mix coherent and non-coherent access > (snoop filters get messed up). > > The solution is to either always have the addresses marked non-coherent > (like device memory, no-map carveouts), or if you really want to use > regular system memory allocated at runtime, then all cached mappings of > it need to be dropped, even the kernel logical address (area as painful > as that would be). > >> ION buffer is allocated. >> >> //Camera device records video >> dma_buf_attach >> dma_map_attachment (buffer needs to be cleaned) > > Why does the buffer need to be cleaned here? I just got through reading > the thread linked by Laura in the other reply. I do like +Brian's Actually +Brian this time :) > suggestion of tracking if the buffer has had CPU access since the last > time and only flushing the cache if it has. As unmapped heaps never get > CPU mapped this would never be the case for unmapped heaps, it solves my > problem. > >> [camera device writes to buffer] >> dma_buf_unmap_attachment (buffer needs to be invalidated) > > It doesn't know there will be any further CPU access, it could get freed > after this for all we know, the invalidate can be saved until the CPU > requests access again. > >> dma_buf_detach (device cannot stay attached because it is being sent down >> the pipeline and Camera doesn't know the end of the use case) >> > > This seems like a broken use-case, I understand the desire to keep > everything as modular as possible and separate the steps, but at this > point no one owns this buffers backing memory, not the CPU or any > device. I would go as far as to say DMA-BUF should be free now to > de-allocate the backing storage if it wants, that way it could get ready > for the next attachment, which may change the required backing memory > completely. > > All devices should attach before the first mapping, and only let go > after the task is complete, otherwise this buffers data needs copied off > to a different location or the CPU needs to take ownership in-between. > >> //buffer is send down the pipeline >> >> // Usersapce software post processing occurs >> mmap buffer > > Perhaps the invalidate should happen here in mmap. > >> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >> devices attached to buffer > > And that should be okay, mmap does the sync, and if no devices are > attached nothing could have changed the underlying memory in the > mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >> [CPU reads/writes to the buffer] >> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >> devices attached to buffer >> munmap buffer >> >> //buffer is send down the pipeline >> // Buffer is send to video device (who does compression of raw data) and >> writes to a file >> dma_buf_attach >> dma_map_attachment (buffer needs to be cleaned) >> [video device writes to buffer] >> dma_buf_unmap_attachment >> dma_buf_detach (device cannot stay attached because it is being sent down >> the pipeline and Video doesn't know the end of the use case) >> >> >> >>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>> access then there is no requirement (that I am aware of) for you to call >>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>> >>> >>> If I am not doing any CPU access then why do I need CPU cache >>> maintenance on the buffer? >>> >> >> Because ION no longer provides DMA ready memory. >> Take the above example. >> >> ION allocates memory from buddy allocator and requests zeroing. >> Zeros are written to the cache. >> >> You pass the buffer to the camera device which is not IO-coherent. >> The camera devices writes directly to the buffer in DDR. >> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >> evicted from the cache, this zero overwrites data the camera device has >> written which corrupts your data. >> > > The zeroing *is* a CPU access, therefor it should handle the needed CMO > for CPU access at the time of zeroing. > > Andrew > >> Liam >> >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >> a Linux Foundation Collaborative Project >>
On 1/15/19 10:38 AM, Andrew F. Davis wrote: > On 1/15/19 11:45 AM, Liam Mark wrote: >> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >> >>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>> >>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>> >>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>> --- >>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>> >>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>> --- a/drivers/staging/android/ion/ion.c >>>>> +++ b/drivers/staging/android/ion/ion.c >>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>> >>>>> table = a->table; >>>>> >>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>> - direction)) >>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>> >>>> Unfortunately I don't think you can do this for a couple reasons. >>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>> If the calls to {begin,end}_cpu_access were made before the call to >>>> dma_buf_attach then there won't have been a device attached so the calls >>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>> >>> >>> That should be okay though, if you have no attachments (or all >>> attachments are IO-coherent) then there is no need for cache >>> maintenance. Unless you mean a sequence where a non-io-coherent device >>> is attached later after data has already been written. Does that >>> sequence need supporting? >> >> Yes, but also I think there are cases where CPU access can happen before >> in Android, but I will focus on later for now. >> >>> DMA-BUF doesn't have to allocate the backing >>> memory until map_dma_buf() time, and that should only happen after all >>> the devices have attached so it can know where to put the buffer. So we >>> shouldn't expect any CPU access to buffers before all the devices are >>> attached and mapped, right? >>> >> >> Here is an example where CPU access can happen later in Android. >> >> Camera device records video -> software post processing -> video device >> (who does compression of raw data) and writes to a file >> >> In this example assume the buffer is cached and the devices are not >> IO-coherent (quite common). >> > > This is the start of the problem, having cached mappings of memory that > is also being accessed non-coherently is going to cause issues one way > or another. On top of the speculative cache fills that have to be > constantly fought back against with CMOs like below; some coherent > interconnects behave badly when you mix coherent and non-coherent access > (snoop filters get messed up). > > The solution is to either always have the addresses marked non-coherent > (like device memory, no-map carveouts), or if you really want to use > regular system memory allocated at runtime, then all cached mappings of > it need to be dropped, even the kernel logical address (area as painful > as that would be). > I agree it's broken, hence my desire to remove it :) The other problem is that uncached buffers are being used for performance reason so anything that would involve getting rid of the logical address would probably negate any performance benefit. >> ION buffer is allocated. >> >> //Camera device records video >> dma_buf_attach >> dma_map_attachment (buffer needs to be cleaned) > > Why does the buffer need to be cleaned here? I just got through reading > the thread linked by Laura in the other reply. I do like +Brian's > suggestion of tracking if the buffer has had CPU access since the last > time and only flushing the cache if it has. As unmapped heaps never get > CPU mapped this would never be the case for unmapped heaps, it solves my > problem. > >> [camera device writes to buffer] >> dma_buf_unmap_attachment (buffer needs to be invalidated) > > It doesn't know there will be any further CPU access, it could get freed > after this for all we know, the invalidate can be saved until the CPU > requests access again. > >> dma_buf_detach (device cannot stay attached because it is being sent down >> the pipeline and Camera doesn't know the end of the use case) >> > > This seems like a broken use-case, I understand the desire to keep > everything as modular as possible and separate the steps, but at this > point no one owns this buffers backing memory, not the CPU or any > device. I would go as far as to say DMA-BUF should be free now to > de-allocate the backing storage if it wants, that way it could get ready > for the next attachment, which may change the required backing memory > completely. > > All devices should attach before the first mapping, and only let go > after the task is complete, otherwise this buffers data needs copied off > to a different location or the CPU needs to take ownership in-between. > Maybe it's broken but it's the status quo and we spent a good amount of time at plumbers concluding there isn't a great way to fix it :/ >> //buffer is send down the pipeline >> >> // Usersapce software post processing occurs >> mmap buffer > > Perhaps the invalidate should happen here in mmap. > >> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >> devices attached to buffer > > And that should be okay, mmap does the sync, and if no devices are > attached nothing could have changed the underlying memory in the > mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >> [CPU reads/writes to the buffer] >> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >> devices attached to buffer >> munmap buffer >> >> //buffer is send down the pipeline >> // Buffer is send to video device (who does compression of raw data) and >> writes to a file >> dma_buf_attach >> dma_map_attachment (buffer needs to be cleaned) >> [video device writes to buffer] >> dma_buf_unmap_attachment >> dma_buf_detach (device cannot stay attached because it is being sent down >> the pipeline and Video doesn't know the end of the use case) >> >> >> >>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>> access then there is no requirement (that I am aware of) for you to call >>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>> >>> >>> If I am not doing any CPU access then why do I need CPU cache >>> maintenance on the buffer? >>> >> >> Because ION no longer provides DMA ready memory. >> Take the above example. >> >> ION allocates memory from buddy allocator and requests zeroing. >> Zeros are written to the cache. >> >> You pass the buffer to the camera device which is not IO-coherent. >> The camera devices writes directly to the buffer in DDR. >> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >> evicted from the cache, this zero overwrites data the camera device has >> written which corrupts your data. >> > > The zeroing *is* a CPU access, therefor it should handle the needed CMO > for CPU access at the time of zeroing. > > Andrew > >> Liam >> >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >> a Linux Foundation Collaborative Project >>
Hi :-) On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > On 1/15/19 12:38 PM, Andrew F. Davis wrote: > > On 1/15/19 11:45 AM, Liam Mark wrote: > >> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >> > >>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>> > >>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > >>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >>>>> > >>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>> --- > >>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>> > >>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>> --- a/drivers/staging/android/ion/ion.c > >>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >>>>> > >>>>> table = a->table; > >>>>> > >>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>> - direction)) > >>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>> > >>>> Unfortunately I don't think you can do this for a couple reasons. > >>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > >>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>> dma_buf_attach then there won't have been a device attached so the calls > >>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>> > >>> > >>> That should be okay though, if you have no attachments (or all > >>> attachments are IO-coherent) then there is no need for cache > >>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>> is attached later after data has already been written. Does that > >>> sequence need supporting? > >> > >> Yes, but also I think there are cases where CPU access can happen before > >> in Android, but I will focus on later for now. > >> > >>> DMA-BUF doesn't have to allocate the backing > >>> memory until map_dma_buf() time, and that should only happen after all > >>> the devices have attached so it can know where to put the buffer. So we > >>> shouldn't expect any CPU access to buffers before all the devices are > >>> attached and mapped, right? > >>> > >> > >> Here is an example where CPU access can happen later in Android. > >> > >> Camera device records video -> software post processing -> video device > >> (who does compression of raw data) and writes to a file > >> > >> In this example assume the buffer is cached and the devices are not > >> IO-coherent (quite common). > >> > > > > This is the start of the problem, having cached mappings of memory that > > is also being accessed non-coherently is going to cause issues one way > > or another. On top of the speculative cache fills that have to be > > constantly fought back against with CMOs like below; some coherent > > interconnects behave badly when you mix coherent and non-coherent access > > (snoop filters get messed up). > > > > The solution is to either always have the addresses marked non-coherent > > (like device memory, no-map carveouts), or if you really want to use > > regular system memory allocated at runtime, then all cached mappings of > > it need to be dropped, even the kernel logical address (area as painful > > as that would be). Ouch :-( I wasn't aware about these potential interconnect issues. How "real" is that? It seems that we aren't really hitting that today on real devices. > > > >> ION buffer is allocated. > >> > >> //Camera device records video > >> dma_buf_attach > >> dma_map_attachment (buffer needs to be cleaned) > > > > Why does the buffer need to be cleaned here? I just got through reading > > the thread linked by Laura in the other reply. I do like +Brian's > > Actually +Brian this time :) > > > suggestion of tracking if the buffer has had CPU access since the last > > time and only flushing the cache if it has. As unmapped heaps never get > > CPU mapped this would never be the case for unmapped heaps, it solves my > > problem. > > > >> [camera device writes to buffer] > >> dma_buf_unmap_attachment (buffer needs to be invalidated) > > > > It doesn't know there will be any further CPU access, it could get freed > > after this for all we know, the invalidate can be saved until the CPU > > requests access again. We don't have any API to allow the invalidate to happen on CPU access if all devices already detached. We need a struct device pointer to give to the DMA API, otherwise on arm64 there'll be no invalidate. I had a chat with a few people internally after the previous discussion with Liam. One suggestion was to use DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least one other device attached (guarantees that we can do an invalidate in the future if begin_cpu_access is called). If the last device detaches, do a sync then. Conversely, in map_dma_buf, we would track if there was any CPU access and use/skip the sync appropriately. I did start poking the code to check out how that would look, but then Christmas happened and I'm still catching back up. > > > >> dma_buf_detach (device cannot stay attached because it is being sent down > >> the pipeline and Camera doesn't know the end of the use case) > >> > > > > This seems like a broken use-case, I understand the desire to keep > > everything as modular as possible and separate the steps, but at this > > point no one owns this buffers backing memory, not the CPU or any > > device. I would go as far as to say DMA-BUF should be free now to > > de-allocate the backing storage if it wants, that way it could get ready > > for the next attachment, which may change the required backing memory > > completely. > > > > All devices should attach before the first mapping, and only let go > > after the task is complete, otherwise this buffers data needs copied off > > to a different location or the CPU needs to take ownership in-between. > > Yeah.. that's certainly the theory. Are there any DMA-BUF implementations which actually do that? I hear it quoted a lot, because that's what the docs say - but if the reality doesn't match it, maybe we should change the docs. > >> //buffer is send down the pipeline > >> > >> // Usersapce software post processing occurs > >> mmap buffer > > > > Perhaps the invalidate should happen here in mmap. > > > >> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >> devices attached to buffer > > > > And that should be okay, mmap does the sync, and if no devices are > > attached nothing could have changed the underlying memory in the > > mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. Liam was saying that it's too painful for them to do that every time a device unmaps - when in many cases (device->device, no CPU) it's not needed. > > > >> [CPU reads/writes to the buffer] > >> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >> devices attached to buffer > >> munmap buffer > >> > >> //buffer is send down the pipeline > >> // Buffer is send to video device (who does compression of raw data) and > >> writes to a file > >> dma_buf_attach > >> dma_map_attachment (buffer needs to be cleaned) > >> [video device writes to buffer] > >> dma_buf_unmap_attachment > >> dma_buf_detach (device cannot stay attached because it is being sent down > >> the pipeline and Video doesn't know the end of the use case) > >> > >> > >> > >>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > >>>> access then there is no requirement (that I am aware of) for you to call > >>>> {begin,end}_cpu_access before passing the buffer to the device and if this > >>>> buffer is cached and your device is not IO-coherent then the cache maintenance > >>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>> > >>> > >>> If I am not doing any CPU access then why do I need CPU cache > >>> maintenance on the buffer? > >>> > >> > >> Because ION no longer provides DMA ready memory. > >> Take the above example. > >> > >> ION allocates memory from buddy allocator and requests zeroing. > >> Zeros are written to the cache. > >> > >> You pass the buffer to the camera device which is not IO-coherent. > >> The camera devices writes directly to the buffer in DDR. > >> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > >> evicted from the cache, this zero overwrites data the camera device has > >> written which corrupts your data. > >> > > > > The zeroing *is* a CPU access, therefor it should handle the needed CMO > > for CPU access at the time of zeroing. > > Actually that should be at the point of the first non-coherent device mapping the buffer right? No point in doing CMO if the future accesses are coherent. Cheers, -Brian > > Andrew > > > >> Liam > >> > >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >> a Linux Foundation Collaborative Project > >>
On 1/15/19 1:05 PM, Laura Abbott wrote: > On 1/15/19 10:38 AM, Andrew F. Davis wrote: >> On 1/15/19 11:45 AM, Liam Mark wrote: >>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>> >>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>> here. >>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>> anyway. >>>>>> >>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>> --- >>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>> >>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>> b/drivers/staging/android/ion/ion.c >>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct >>>>>> dma_buf_attachment *attachment, >>>>>> table = a->table; >>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>> - direction)) >>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>> >>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>> maintenance. >>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>> dma_buf_attach then there won't have been a device attached so the >>>>> calls >>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>> >>>> >>>> That should be okay though, if you have no attachments (or all >>>> attachments are IO-coherent) then there is no need for cache >>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>> is attached later after data has already been written. Does that >>>> sequence need supporting? >>> >>> Yes, but also I think there are cases where CPU access can happen before >>> in Android, but I will focus on later for now. >>> >>>> DMA-BUF doesn't have to allocate the backing >>>> memory until map_dma_buf() time, and that should only happen after all >>>> the devices have attached so it can know where to put the buffer. So we >>>> shouldn't expect any CPU access to buffers before all the devices are >>>> attached and mapped, right? >>>> >>> >>> Here is an example where CPU access can happen later in Android. >>> >>> Camera device records video -> software post processing -> video device >>> (who does compression of raw data) and writes to a file >>> >>> In this example assume the buffer is cached and the devices are not >>> IO-coherent (quite common). >>> >> >> This is the start of the problem, having cached mappings of memory that >> is also being accessed non-coherently is going to cause issues one way >> or another. On top of the speculative cache fills that have to be >> constantly fought back against with CMOs like below; some coherent >> interconnects behave badly when you mix coherent and non-coherent access >> (snoop filters get messed up). >> >> The solution is to either always have the addresses marked non-coherent >> (like device memory, no-map carveouts), or if you really want to use >> regular system memory allocated at runtime, then all cached mappings of >> it need to be dropped, even the kernel logical address (area as painful >> as that would be). >> > > I agree it's broken, hence my desire to remove it :) > > The other problem is that uncached buffers are being used for > performance reason so anything that would involve getting > rid of the logical address would probably negate any performance > benefit. > I wouldn't go as far as to remove them just yet.. Liam seems pretty adamant that they have valid uses. I'm just not sure performance is one of them, maybe in the case of software locks between devices or something where there needs to be a lot of back and forth interleaved access on small amounts of data? >>> ION buffer is allocated. >>> >>> //Camera device records video >>> dma_buf_attach >>> dma_map_attachment (buffer needs to be cleaned) >> >> Why does the buffer need to be cleaned here? I just got through reading >> the thread linked by Laura in the other reply. I do like +Brian's >> suggestion of tracking if the buffer has had CPU access since the last >> time and only flushing the cache if it has. As unmapped heaps never get >> CPU mapped this would never be the case for unmapped heaps, it solves my >> problem. >> >>> [camera device writes to buffer] >>> dma_buf_unmap_attachment (buffer needs to be invalidated) >> >> It doesn't know there will be any further CPU access, it could get freed >> after this for all we know, the invalidate can be saved until the CPU >> requests access again. >> >>> dma_buf_detach (device cannot stay attached because it is being sent >>> down >>> the pipeline and Camera doesn't know the end of the use case) >>> >> >> This seems like a broken use-case, I understand the desire to keep >> everything as modular as possible and separate the steps, but at this >> point no one owns this buffers backing memory, not the CPU or any >> device. I would go as far as to say DMA-BUF should be free now to >> de-allocate the backing storage if it wants, that way it could get ready >> for the next attachment, which may change the required backing memory >> completely. >> >> All devices should attach before the first mapping, and only let go >> after the task is complete, otherwise this buffers data needs copied off >> to a different location or the CPU needs to take ownership in-between. >> > > Maybe it's broken but it's the status quo and we spent a good > amount of time at plumbers concluding there isn't a great way > to fix it :/ > Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ Perhaps just stronger rules on sequencing of operations? I'm not saying I have a good solution either, I just don't see any way forward without some use-case getting broken, so better to fix now over later. >>> //buffer is send down the pipeline >>> >>> // Usersapce software post processing occurs >>> mmap buffer >> >> Perhaps the invalidate should happen here in mmap. >> >>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>> devices attached to buffer >> >> And that should be okay, mmap does the sync, and if no devices are >> attached nothing could have changed the underlying memory in the >> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >> >>> [CPU reads/writes to the buffer] >>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>> devices attached to buffer >>> munmap buffer >>> >>> //buffer is send down the pipeline >>> // Buffer is send to video device (who does compression of raw data) and >>> writes to a file >>> dma_buf_attach >>> dma_map_attachment (buffer needs to be cleaned) >>> [video device writes to buffer] >>> dma_buf_unmap_attachment >>> dma_buf_detach (device cannot stay attached because it is being sent >>> down >>> the pipeline and Video doesn't know the end of the use case) >>> >>> >>> >>>>> Also ION no longer provides DMA ready memory, so if you are not >>>>> doing CPU >>>>> access then there is no requirement (that I am aware of) for you to >>>>> call >>>>> {begin,end}_cpu_access before passing the buffer to the device and >>>>> if this >>>>> buffer is cached and your device is not IO-coherent then the cache >>>>> maintenance >>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>> >>>> >>>> If I am not doing any CPU access then why do I need CPU cache >>>> maintenance on the buffer? >>>> >>> >>> Because ION no longer provides DMA ready memory. >>> Take the above example. >>> >>> ION allocates memory from buddy allocator and requests zeroing. >>> Zeros are written to the cache. >>> >>> You pass the buffer to the camera device which is not IO-coherent. >>> The camera devices writes directly to the buffer in DDR. >>> Since you didn't clean the buffer a dirty cache line (one of the >>> zeros) is >>> evicted from the cache, this zero overwrites data the camera device has >>> written which corrupts your data. >>> >> >> The zeroing *is* a CPU access, therefor it should handle the needed CMO >> for CPU access at the time of zeroing. >> >> Andrew >> >>> Liam >>> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>> a Linux Foundation Collaborative Project >>> >
On 1/16/19 9:19 AM, Brian Starkey wrote: > Hi :-) > > On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: >> On 1/15/19 12:38 PM, Andrew F. Davis wrote: >>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>> >>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>> >>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>>>> >>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>> --- >>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>>>> >>>>>>> table = a->table; >>>>>>> >>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>> - direction)) >>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>> >>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>> dma_buf_attach then there won't have been a device attached so the calls >>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>> >>>>> >>>>> That should be okay though, if you have no attachments (or all >>>>> attachments are IO-coherent) then there is no need for cache >>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>> is attached later after data has already been written. Does that >>>>> sequence need supporting? >>>> >>>> Yes, but also I think there are cases where CPU access can happen before >>>> in Android, but I will focus on later for now. >>>> >>>>> DMA-BUF doesn't have to allocate the backing >>>>> memory until map_dma_buf() time, and that should only happen after all >>>>> the devices have attached so it can know where to put the buffer. So we >>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>> attached and mapped, right? >>>>> >>>> >>>> Here is an example where CPU access can happen later in Android. >>>> >>>> Camera device records video -> software post processing -> video device >>>> (who does compression of raw data) and writes to a file >>>> >>>> In this example assume the buffer is cached and the devices are not >>>> IO-coherent (quite common). >>>> >>> >>> This is the start of the problem, having cached mappings of memory that >>> is also being accessed non-coherently is going to cause issues one way >>> or another. On top of the speculative cache fills that have to be >>> constantly fought back against with CMOs like below; some coherent >>> interconnects behave badly when you mix coherent and non-coherent access >>> (snoop filters get messed up). >>> >>> The solution is to either always have the addresses marked non-coherent >>> (like device memory, no-map carveouts), or if you really want to use >>> regular system memory allocated at runtime, then all cached mappings of >>> it need to be dropped, even the kernel logical address (area as painful >>> as that would be). > > Ouch :-( I wasn't aware about these potential interconnect issues. How > "real" is that? It seems that we aren't really hitting that today on > real devices. > Sadly there is at least one real device like this now (TI AM654). We spent some time working with the ARM interconnect spec designers to see if this was allowed behavior, final conclusion was mixing coherent and non-coherent accesses is never a good idea.. So we have been working to try to minimize any cases of mixed attributes [0], if a region is coherent then everyone in the system needs to treat it as such and vice-versa, even clever CMO that work on other systems wont save you here. :( [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 >>> >>>> ION buffer is allocated. >>>> >>>> //Camera device records video >>>> dma_buf_attach >>>> dma_map_attachment (buffer needs to be cleaned) >>> >>> Why does the buffer need to be cleaned here? I just got through reading >>> the thread linked by Laura in the other reply. I do like +Brian's >> >> Actually +Brian this time :) >> >>> suggestion of tracking if the buffer has had CPU access since the last >>> time and only flushing the cache if it has. As unmapped heaps never get >>> CPU mapped this would never be the case for unmapped heaps, it solves my >>> problem. >>> >>>> [camera device writes to buffer] >>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>> >>> It doesn't know there will be any further CPU access, it could get freed >>> after this for all we know, the invalidate can be saved until the CPU >>> requests access again. > > We don't have any API to allow the invalidate to happen on CPU access > if all devices already detached. We need a struct device pointer to > give to the DMA API, otherwise on arm64 there'll be no invalidate. > > I had a chat with a few people internally after the previous > discussion with Liam. One suggestion was to use > DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > one other device attached (guarantees that we can do an invalidate in > the future if begin_cpu_access is called). If the last device > detaches, do a sync then. > > Conversely, in map_dma_buf, we would track if there was any CPU access > and use/skip the sync appropriately. > Now that I think this all through I agree this patch is probably wrong. The real fix needs to be better handling in the dma_map_sg() to deal with the case of the memory not being mapped (what I'm dealing with for unmapped heaps), and for cases when the memory in question is not cached (Liam's issue I think). For both these cases the dma_map_sg() does the wrong thing. > I did start poking the code to check out how that would look, but then > Christmas happened and I'm still catching back up. > >>> >>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>> the pipeline and Camera doesn't know the end of the use case) >>>> >>> >>> This seems like a broken use-case, I understand the desire to keep >>> everything as modular as possible and separate the steps, but at this >>> point no one owns this buffers backing memory, not the CPU or any >>> device. I would go as far as to say DMA-BUF should be free now to >>> de-allocate the backing storage if it wants, that way it could get ready >>> for the next attachment, which may change the required backing memory >>> completely. >>> >>> All devices should attach before the first mapping, and only let go >>> after the task is complete, otherwise this buffers data needs copied off >>> to a different location or the CPU needs to take ownership in-between. >>> > > Yeah.. that's certainly the theory. Are there any DMA-BUF > implementations which actually do that? I hear it quoted a lot, > because that's what the docs say - but if the reality doesn't match > it, maybe we should change the docs. > Do you mean on the userspace side? I'm not sure, seems like Android might be doing this wrong from what I can gather. From kernel side if you mean the "de-allocate the backing storage", we will have some cases like this soon, so I want to make sure userspace is not abusing DMA-BUF in ways not specified in the documentation. Changing the docs to force the backing memory to always be allocated breaks the central goal in having attach/map in DMA-BUF separate. >>>> //buffer is send down the pipeline >>>> >>>> // Usersapce software post processing occurs >>>> mmap buffer >>> >>> Perhaps the invalidate should happen here in mmap. >>> >>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>> devices attached to buffer >>> >>> And that should be okay, mmap does the sync, and if no devices are >>> attached nothing could have changed the underlying memory in the >>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > > Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > Liam was saying that it's too painful for them to do that every time a > device unmaps - when in many cases (device->device, no CPU) it's not > needed. Invalidates are painless, at least compared to a real cache flush, just set the invalid bit vs actually writing out lines. I thought the issue was on the map side. > >>> >>>> [CPU reads/writes to the buffer] >>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>> devices attached to buffer >>>> munmap buffer >>>> >>>> //buffer is send down the pipeline >>>> // Buffer is send to video device (who does compression of raw data) and >>>> writes to a file >>>> dma_buf_attach >>>> dma_map_attachment (buffer needs to be cleaned) >>>> [video device writes to buffer] >>>> dma_buf_unmap_attachment >>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>> the pipeline and Video doesn't know the end of the use case) >>>> >>>> >>>> >>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>>>> access then there is no requirement (that I am aware of) for you to call >>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>> >>>>> >>>>> If I am not doing any CPU access then why do I need CPU cache >>>>> maintenance on the buffer? >>>>> >>>> >>>> Because ION no longer provides DMA ready memory. >>>> Take the above example. >>>> >>>> ION allocates memory from buddy allocator and requests zeroing. >>>> Zeros are written to the cache. >>>> >>>> You pass the buffer to the camera device which is not IO-coherent. >>>> The camera devices writes directly to the buffer in DDR. >>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >>>> evicted from the cache, this zero overwrites data the camera device has >>>> written which corrupts your data. >>>> >>> >>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>> for CPU access at the time of zeroing. >>> > > Actually that should be at the point of the first non-coherent device > mapping the buffer right? No point in doing CMO if the future accesses > are coherent. I see your point, as long as the zeroing is guaranteed to be the first access to this buffer then it should be safe. Andrew > > Cheers, > -Brian > >>> Andrew >>> >>>> Liam >>>> >>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>> a Linux Foundation Collaborative Project >>>>
On Wed, 16 Jan 2019, Andrew F. Davis wrote: > On 1/15/19 1:05 PM, Laura Abbott wrote: > > On 1/15/19 10:38 AM, Andrew F. Davis wrote: > >> On 1/15/19 11:45 AM, Liam Mark wrote: > >>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>> > >>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>> > >>>>>> Buffers may not be mapped from the CPU so skip cache maintenance > >>>>>> here. > >>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>> {begin,end}_cpu_access calls so maintenance should not be needed > >>>>>> anyway. > >>>>>> > >>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>> --- > >>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>> > >>>>>> diff --git a/drivers/staging/android/ion/ion.c > >>>>>> b/drivers/staging/android/ion/ion.c > >>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct > >>>>>> dma_buf_attachment *attachment, > >>>>>> table = a->table; > >>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>> - direction)) > >>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>> > >>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>> You can't rely on {begin,end}_cpu_access calls to do cache > >>>>> maintenance. > >>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>> dma_buf_attach then there won't have been a device attached so the > >>>>> calls > >>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>> > >>>> > >>>> That should be okay though, if you have no attachments (or all > >>>> attachments are IO-coherent) then there is no need for cache > >>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>> is attached later after data has already been written. Does that > >>>> sequence need supporting? > >>> > >>> Yes, but also I think there are cases where CPU access can happen before > >>> in Android, but I will focus on later for now. > >>> > >>>> DMA-BUF doesn't have to allocate the backing > >>>> memory until map_dma_buf() time, and that should only happen after all > >>>> the devices have attached so it can know where to put the buffer. So we > >>>> shouldn't expect any CPU access to buffers before all the devices are > >>>> attached and mapped, right? > >>>> > >>> > >>> Here is an example where CPU access can happen later in Android. > >>> > >>> Camera device records video -> software post processing -> video device > >>> (who does compression of raw data) and writes to a file > >>> > >>> In this example assume the buffer is cached and the devices are not > >>> IO-coherent (quite common). > >>> > >> > >> This is the start of the problem, having cached mappings of memory that > >> is also being accessed non-coherently is going to cause issues one way > >> or another. On top of the speculative cache fills that have to be > >> constantly fought back against with CMOs like below; some coherent > >> interconnects behave badly when you mix coherent and non-coherent access > >> (snoop filters get messed up). > >> > >> The solution is to either always have the addresses marked non-coherent > >> (like device memory, no-map carveouts), or if you really want to use > >> regular system memory allocated at runtime, then all cached mappings of > >> it need to be dropped, even the kernel logical address (area as painful > >> as that would be). > >> > > > > I agree it's broken, hence my desire to remove it :) > > > > The other problem is that uncached buffers are being used for > > performance reason so anything that would involve getting > > rid of the logical address would probably negate any performance > > benefit. > > > > I wouldn't go as far as to remove them just yet.. Liam seems pretty > adamant that they have valid uses. I'm just not sure performance is one > of them, maybe in the case of software locks between devices or > something where there needs to be a lot of back and forth interleaved > access on small amounts of data? > I wasn't aware that ARM considered this not supported, I thought it was supported but they advised against it because of the potential performance impact. This is after all supported in the DMA APIs and up until now devices have been successfully commercializing with this configurations, and I think they will continue to commercialize with these configurations for quite a while. It would be really unfortunate if support was removed as I think that would drive clients away from using upstream ION. > >>> ION buffer is allocated. > >>> > >>> //Camera device records video > >>> dma_buf_attach > >>> dma_map_attachment (buffer needs to be cleaned) > >> > >> Why does the buffer need to be cleaned here? I just got through reading > >> the thread linked by Laura in the other reply. I do like +Brian's > >> suggestion of tracking if the buffer has had CPU access since the last > >> time and only flushing the cache if it has. As unmapped heaps never get > >> CPU mapped this would never be the case for unmapped heaps, it solves my > >> problem. > >> > >>> [camera device writes to buffer] > >>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >> > >> It doesn't know there will be any further CPU access, it could get freed > >> after this for all we know, the invalidate can be saved until the CPU > >> requests access again. > >> > >>> dma_buf_detach (device cannot stay attached because it is being sent > >>> down > >>> the pipeline and Camera doesn't know the end of the use case) > >>> > >> > >> This seems like a broken use-case, I understand the desire to keep > >> everything as modular as possible and separate the steps, but at this > >> point no one owns this buffers backing memory, not the CPU or any > >> device. I would go as far as to say DMA-BUF should be free now to > >> de-allocate the backing storage if it wants, that way it could get ready > >> for the next attachment, which may change the required backing memory > >> completely. > >> > >> All devices should attach before the first mapping, and only let go > >> after the task is complete, otherwise this buffers data needs copied off > >> to a different location or the CPU needs to take ownership in-between. > >> > > > > Maybe it's broken but it's the status quo and we spent a good > > amount of time at plumbers concluding there isn't a great way > > to fix it :/ > > > > Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ > > Perhaps just stronger rules on sequencing of operations? I'm not saying > I have a good solution either, I just don't see any way forward without > some use-case getting broken, so better to fix now over later. > I can see the benefits of Android doing things the way they do, I would request that changes we make continue to support Android, or we find a way to convice them to change, as they are the main ION client and I assume other ION clients in the future will want to do this as well. I am concerned that if you go with a solution which enforces what you mention above, and bring ION out of staging that way, it will make it that much harder to solve this for Android and therefore harder to get Android clients to move to the upstream ION (and get everybody off their vendor modified Android versions). > >>> //buffer is send down the pipeline > >>> > >>> // Usersapce software post processing occurs > >>> mmap buffer > >> > >> Perhaps the invalidate should happen here in mmap. > >> > >>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>> devices attached to buffer > >> > >> And that should be okay, mmap does the sync, and if no devices are > >> attached nothing could have changed the underlying memory in the > >> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >> > >>> [CPU reads/writes to the buffer] > >>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>> devices attached to buffer > >>> munmap buffer > >>> > >>> //buffer is send down the pipeline > >>> // Buffer is send to video device (who does compression of raw data) and > >>> writes to a file > >>> dma_buf_attach > >>> dma_map_attachment (buffer needs to be cleaned) > >>> [video device writes to buffer] > >>> dma_buf_unmap_attachment > >>> dma_buf_detach (device cannot stay attached because it is being sent > >>> down > >>> the pipeline and Video doesn't know the end of the use case) > >>> > >>> > >>> > >>>>> Also ION no longer provides DMA ready memory, so if you are not > >>>>> doing CPU > >>>>> access then there is no requirement (that I am aware of) for you to > >>>>> call > >>>>> {begin,end}_cpu_access before passing the buffer to the device and > >>>>> if this > >>>>> buffer is cached and your device is not IO-coherent then the cache > >>>>> maintenance > >>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>> > >>>> > >>>> If I am not doing any CPU access then why do I need CPU cache > >>>> maintenance on the buffer? > >>>> > >>> > >>> Because ION no longer provides DMA ready memory. > >>> Take the above example. > >>> > >>> ION allocates memory from buddy allocator and requests zeroing. > >>> Zeros are written to the cache. > >>> > >>> You pass the buffer to the camera device which is not IO-coherent. > >>> The camera devices writes directly to the buffer in DDR. > >>> Since you didn't clean the buffer a dirty cache line (one of the > >>> zeros) is > >>> evicted from the cache, this zero overwrites data the camera device has > >>> written which corrupts your data. > >>> > >> > >> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >> for CPU access at the time of zeroing. > >> > >> Andrew > >> > >>> Liam > >>> > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>> a Linux Foundation Collaborative Project > >>> > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On Wed, 16 Jan 2019, Andrew F. Davis wrote: > On 1/16/19 9:19 AM, Brian Starkey wrote: > > Hi :-) > > > > On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > >> On 1/15/19 12:38 PM, Andrew F. Davis wrote: > >>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>> > >>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>> > >>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > >>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >>>>>>> > >>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>> --- > >>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>> > >>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >>>>>>> > >>>>>>> table = a->table; > >>>>>>> > >>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>> - direction)) > >>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>> > >>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > >>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>> dma_buf_attach then there won't have been a device attached so the calls > >>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>> > >>>>> > >>>>> That should be okay though, if you have no attachments (or all > >>>>> attachments are IO-coherent) then there is no need for cache > >>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>> is attached later after data has already been written. Does that > >>>>> sequence need supporting? > >>>> > >>>> Yes, but also I think there are cases where CPU access can happen before > >>>> in Android, but I will focus on later for now. > >>>> > >>>>> DMA-BUF doesn't have to allocate the backing > >>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>> the devices have attached so it can know where to put the buffer. So we > >>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>> attached and mapped, right? > >>>>> > >>>> > >>>> Here is an example where CPU access can happen later in Android. > >>>> > >>>> Camera device records video -> software post processing -> video device > >>>> (who does compression of raw data) and writes to a file > >>>> > >>>> In this example assume the buffer is cached and the devices are not > >>>> IO-coherent (quite common). > >>>> > >>> > >>> This is the start of the problem, having cached mappings of memory that > >>> is also being accessed non-coherently is going to cause issues one way > >>> or another. On top of the speculative cache fills that have to be > >>> constantly fought back against with CMOs like below; some coherent > >>> interconnects behave badly when you mix coherent and non-coherent access > >>> (snoop filters get messed up). > >>> > >>> The solution is to either always have the addresses marked non-coherent > >>> (like device memory, no-map carveouts), or if you really want to use > >>> regular system memory allocated at runtime, then all cached mappings of > >>> it need to be dropped, even the kernel logical address (area as painful > >>> as that would be). > > > > Ouch :-( I wasn't aware about these potential interconnect issues. How > > "real" is that? It seems that we aren't really hitting that today on > > real devices. > > > > Sadly there is at least one real device like this now (TI AM654). We > spent some time working with the ARM interconnect spec designers to see > if this was allowed behavior, final conclusion was mixing coherent and > non-coherent accesses is never a good idea.. So we have been working to > try to minimize any cases of mixed attributes [0], if a region is > coherent then everyone in the system needs to treat it as such and > vice-versa, even clever CMO that work on other systems wont save you > here. :( > > [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 > > > >>> > >>>> ION buffer is allocated. > >>>> > >>>> //Camera device records video > >>>> dma_buf_attach > >>>> dma_map_attachment (buffer needs to be cleaned) > >>> > >>> Why does the buffer need to be cleaned here? I just got through reading > >>> the thread linked by Laura in the other reply. I do like +Brian's > >> > >> Actually +Brian this time :) > >> > >>> suggestion of tracking if the buffer has had CPU access since the last > >>> time and only flushing the cache if it has. As unmapped heaps never get > >>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>> problem. > >>> > >>>> [camera device writes to buffer] > >>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>> > >>> It doesn't know there will be any further CPU access, it could get freed > >>> after this for all we know, the invalidate can be saved until the CPU > >>> requests access again. > > > > We don't have any API to allow the invalidate to happen on CPU access > > if all devices already detached. We need a struct device pointer to > > give to the DMA API, otherwise on arm64 there'll be no invalidate. > > > > I had a chat with a few people internally after the previous > > discussion with Liam. One suggestion was to use > > DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > > one other device attached (guarantees that we can do an invalidate in > > the future if begin_cpu_access is called). If the last device > > detaches, do a sync then. > > > > Conversely, in map_dma_buf, we would track if there was any CPU access > > and use/skip the sync appropriately. > > > > Now that I think this all through I agree this patch is probably wrong. > The real fix needs to be better handling in the dma_map_sg() to deal > with the case of the memory not being mapped (what I'm dealing with for > unmapped heaps), and for cases when the memory in question is not cached > (Liam's issue I think). For both these cases the dma_map_sg() does the > wrong thing. > > > I did start poking the code to check out how that would look, but then > > Christmas happened and I'm still catching back up. > > > >>> > >>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>> the pipeline and Camera doesn't know the end of the use case) > >>>> > >>> > >>> This seems like a broken use-case, I understand the desire to keep > >>> everything as modular as possible and separate the steps, but at this > >>> point no one owns this buffers backing memory, not the CPU or any > >>> device. I would go as far as to say DMA-BUF should be free now to > >>> de-allocate the backing storage if it wants, that way it could get ready > >>> for the next attachment, which may change the required backing memory > >>> completely. > >>> > >>> All devices should attach before the first mapping, and only let go > >>> after the task is complete, otherwise this buffers data needs copied off > >>> to a different location or the CPU needs to take ownership in-between. > >>> > > > > Yeah.. that's certainly the theory. Are there any DMA-BUF > > implementations which actually do that? I hear it quoted a lot, > > because that's what the docs say - but if the reality doesn't match > > it, maybe we should change the docs. > > > > Do you mean on the userspace side? I'm not sure, seems like Android > might be doing this wrong from what I can gather. From kernel side if > you mean the "de-allocate the backing storage", we will have some cases > like this soon, so I want to make sure userspace is not abusing DMA-BUF > in ways not specified in the documentation. Changing the docs to force > the backing memory to always be allocated breaks the central goal in > having attach/map in DMA-BUF separate. > > >>>> //buffer is send down the pipeline > >>>> > >>>> // Usersapce software post processing occurs > >>>> mmap buffer > >>> > >>> Perhaps the invalidate should happen here in mmap. > >>> > >>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>> devices attached to buffer > >>> > >>> And that should be okay, mmap does the sync, and if no devices are > >>> attached nothing could have changed the underlying memory in the > >>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > > > > Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > > Liam was saying that it's too painful for them to do that every time a > > device unmaps - when in many cases (device->device, no CPU) it's not > > needed. > > Invalidates are painless, at least compared to a real cache flush, just > set the invalid bit vs actually writing out lines. I thought the issue > was on the map side. > Invalidates aren't painless for us because we have a coherent system cache so clean lines get written out. And these invalidates can occur on fairly large buffers. That is why we haven't went with using cached ION memory and "tracking CPU access" because it only solves half the problem, ie there isn't a way to safely skip the invalidate (because we can't read the future). Our solution was to go with uncached ION memory (when possible), but as you can see in other discussions upstream support for uncached memory has its own issues. > > > >>> > >>>> [CPU reads/writes to the buffer] > >>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>> devices attached to buffer > >>>> munmap buffer > >>>> > >>>> //buffer is send down the pipeline > >>>> // Buffer is send to video device (who does compression of raw data) and > >>>> writes to a file > >>>> dma_buf_attach > >>>> dma_map_attachment (buffer needs to be cleaned) > >>>> [video device writes to buffer] > >>>> dma_buf_unmap_attachment > >>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>> the pipeline and Video doesn't know the end of the use case) > >>>> > >>>> > >>>> > >>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > >>>>>> access then there is no requirement (that I am aware of) for you to call > >>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this > >>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance > >>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>> > >>>>> > >>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>> maintenance on the buffer? > >>>>> > >>>> > >>>> Because ION no longer provides DMA ready memory. > >>>> Take the above example. > >>>> > >>>> ION allocates memory from buddy allocator and requests zeroing. > >>>> Zeros are written to the cache. > >>>> > >>>> You pass the buffer to the camera device which is not IO-coherent. > >>>> The camera devices writes directly to the buffer in DDR. > >>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > >>>> evicted from the cache, this zero overwrites data the camera device has > >>>> written which corrupts your data. > >>>> > >>> > >>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>> for CPU access at the time of zeroing. > >>> > > > > Actually that should be at the point of the first non-coherent device > > mapping the buffer right? No point in doing CMO if the future accesses > > are coherent. > > I see your point, as long as the zeroing is guaranteed to be the first > access to this buffer then it should be safe. > > Andrew > > > > > Cheers, > > -Brian > > > >>> Andrew > >>> > >>>> Liam > >>>> > >>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>> a Linux Foundation Collaborative Project > >>>> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On 1/16/19 4:48 PM, Liam Mark wrote: > On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >> On 1/15/19 1:05 PM, Laura Abbott wrote: >>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>> >>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>> >>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>> here. >>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>> anyway. >>>>>>>> >>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>> --- >>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>> >>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct >>>>>>>> dma_buf_attachment *attachment, >>>>>>>> table = a->table; >>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>> - direction)) >>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>> >>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>> maintenance. >>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>> calls >>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>> >>>>>> >>>>>> That should be okay though, if you have no attachments (or all >>>>>> attachments are IO-coherent) then there is no need for cache >>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>> is attached later after data has already been written. Does that >>>>>> sequence need supporting? >>>>> >>>>> Yes, but also I think there are cases where CPU access can happen before >>>>> in Android, but I will focus on later for now. >>>>> >>>>>> DMA-BUF doesn't have to allocate the backing >>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>> attached and mapped, right? >>>>>> >>>>> >>>>> Here is an example where CPU access can happen later in Android. >>>>> >>>>> Camera device records video -> software post processing -> video device >>>>> (who does compression of raw data) and writes to a file >>>>> >>>>> In this example assume the buffer is cached and the devices are not >>>>> IO-coherent (quite common). >>>>> >>>> >>>> This is the start of the problem, having cached mappings of memory that >>>> is also being accessed non-coherently is going to cause issues one way >>>> or another. On top of the speculative cache fills that have to be >>>> constantly fought back against with CMOs like below; some coherent >>>> interconnects behave badly when you mix coherent and non-coherent access >>>> (snoop filters get messed up). >>>> >>>> The solution is to either always have the addresses marked non-coherent >>>> (like device memory, no-map carveouts), or if you really want to use >>>> regular system memory allocated at runtime, then all cached mappings of >>>> it need to be dropped, even the kernel logical address (area as painful >>>> as that would be). >>>> >>> >>> I agree it's broken, hence my desire to remove it :) >>> >>> The other problem is that uncached buffers are being used for >>> performance reason so anything that would involve getting >>> rid of the logical address would probably negate any performance >>> benefit. >>> >> >> I wouldn't go as far as to remove them just yet.. Liam seems pretty >> adamant that they have valid uses. I'm just not sure performance is one >> of them, maybe in the case of software locks between devices or >> something where there needs to be a lot of back and forth interleaved >> access on small amounts of data? >> > > I wasn't aware that ARM considered this not supported, I thought it was > supported but they advised against it because of the potential performance > impact. > Not sure what you mean by "this" being not supported, do you mean mixed attribute mappings? If so, it will certainly cause problems, and the problems will change from platform to platform, avoid at all costs is my understanding of ARM's position. > This is after all supported in the DMA APIs and up until now devices have > been successfully commercializing with this configurations, and I think > they will continue to commercialize with these configurations for quite a > while. > Use of uncached memory mappings are almost always wrong in my experience and are used to work around some bug or because the user doesn't want to implement proper CMOs. Counter examples welcome. > It would be really unfortunate if support was removed as I think that > would drive clients away from using upstream ION. > I'm not petitioning to remove support, but at very least lets reverse the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by default, to get uncached you should need to add a flag to your allocation command pointing out you know what you are doing. >>>>> ION buffer is allocated. >>>>> >>>>> //Camera device records video >>>>> dma_buf_attach >>>>> dma_map_attachment (buffer needs to be cleaned) >>>> >>>> Why does the buffer need to be cleaned here? I just got through reading >>>> the thread linked by Laura in the other reply. I do like +Brian's >>>> suggestion of tracking if the buffer has had CPU access since the last >>>> time and only flushing the cache if it has. As unmapped heaps never get >>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>> problem. >>>> >>>>> [camera device writes to buffer] >>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>> >>>> It doesn't know there will be any further CPU access, it could get freed >>>> after this for all we know, the invalidate can be saved until the CPU >>>> requests access again. >>>> >>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>> down >>>>> the pipeline and Camera doesn't know the end of the use case) >>>>> >>>> >>>> This seems like a broken use-case, I understand the desire to keep >>>> everything as modular as possible and separate the steps, but at this >>>> point no one owns this buffers backing memory, not the CPU or any >>>> device. I would go as far as to say DMA-BUF should be free now to >>>> de-allocate the backing storage if it wants, that way it could get ready >>>> for the next attachment, which may change the required backing memory >>>> completely. >>>> >>>> All devices should attach before the first mapping, and only let go >>>> after the task is complete, otherwise this buffers data needs copied off >>>> to a different location or the CPU needs to take ownership in-between. >>>> >>> >>> Maybe it's broken but it's the status quo and we spent a good >>> amount of time at plumbers concluding there isn't a great way >>> to fix it :/ >>> >> >> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ >> >> Perhaps just stronger rules on sequencing of operations? I'm not saying >> I have a good solution either, I just don't see any way forward without >> some use-case getting broken, so better to fix now over later. >> > > I can see the benefits of Android doing things the way they do, I would > request that changes we make continue to support Android, or we find a way > to convice them to change, as they are the main ION client and I assume > other ION clients in the future will want to do this as well. > Android may be the biggest user today (makes sense, Ion come out of the Android project), but that can change, and getting changes into Android will be easier that the upstream kernel once Ion is out of staging. Unlike some other big ARM vendors, we (TI) do not primarily build mobile chips targeting Android, our core offerings target more traditional Linux userspaces, and I'm guessing others will start to do the same as ARM tries to push more into desktop, server, and other spaces again. > I am concerned that if you go with a solution which enforces what you > mention above, and bring ION out of staging that way, it will make it that > much harder to solve this for Android and therefore harder to get > Android clients to move to the upstream ION (and get everybody off their > vendor modified Android versions). > That would be an Android problem, reducing functionality in upstream to match what some evil vendor trees do to support Android is not the way forward on this. At least for us we are going to try to make all our software offerings follow proper buffer ownership (including our Android offering). >>>>> //buffer is send down the pipeline >>>>> >>>>> // Usersapce software post processing occurs >>>>> mmap buffer >>>> >>>> Perhaps the invalidate should happen here in mmap. >>>> >>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>> devices attached to buffer >>>> >>>> And that should be okay, mmap does the sync, and if no devices are >>>> attached nothing could have changed the underlying memory in the >>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>>> >>>>> [CPU reads/writes to the buffer] >>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>> devices attached to buffer >>>>> munmap buffer >>>>> >>>>> //buffer is send down the pipeline >>>>> // Buffer is send to video device (who does compression of raw data) and >>>>> writes to a file >>>>> dma_buf_attach >>>>> dma_map_attachment (buffer needs to be cleaned) >>>>> [video device writes to buffer] >>>>> dma_buf_unmap_attachment >>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>> down >>>>> the pipeline and Video doesn't know the end of the use case) >>>>> >>>>> >>>>> >>>>>>> Also ION no longer provides DMA ready memory, so if you are not >>>>>>> doing CPU >>>>>>> access then there is no requirement (that I am aware of) for you to >>>>>>> call >>>>>>> {begin,end}_cpu_access before passing the buffer to the device and >>>>>>> if this >>>>>>> buffer is cached and your device is not IO-coherent then the cache >>>>>>> maintenance >>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>> >>>>>> >>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>> maintenance on the buffer? >>>>>> >>>>> >>>>> Because ION no longer provides DMA ready memory. >>>>> Take the above example. >>>>> >>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>> Zeros are written to the cache. >>>>> >>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>> The camera devices writes directly to the buffer in DDR. >>>>> Since you didn't clean the buffer a dirty cache line (one of the >>>>> zeros) is >>>>> evicted from the cache, this zero overwrites data the camera device has >>>>> written which corrupts your data. >>>>> >>>> >>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>> for CPU access at the time of zeroing. >>>> >>>> Andrew >>>> >>>>> Liam >>>>> >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>> a Linux Foundation Collaborative Project >>>>> >>> >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On 1/16/19 4:54 PM, Liam Mark wrote: > On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >> On 1/16/19 9:19 AM, Brian Starkey wrote: >>> Hi :-) >>> >>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: >>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: >>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>> >>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>> >>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>>>>>> >>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>> --- >>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>> >>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>>>>>> >>>>>>>>> table = a->table; >>>>>>>>> >>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>> - direction)) >>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>> >>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>> dma_buf_attach then there won't have been a device attached so the calls >>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>> >>>>>>> >>>>>>> That should be okay though, if you have no attachments (or all >>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>> is attached later after data has already been written. Does that >>>>>>> sequence need supporting? >>>>>> >>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>> in Android, but I will focus on later for now. >>>>>> >>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>> attached and mapped, right? >>>>>>> >>>>>> >>>>>> Here is an example where CPU access can happen later in Android. >>>>>> >>>>>> Camera device records video -> software post processing -> video device >>>>>> (who does compression of raw data) and writes to a file >>>>>> >>>>>> In this example assume the buffer is cached and the devices are not >>>>>> IO-coherent (quite common). >>>>>> >>>>> >>>>> This is the start of the problem, having cached mappings of memory that >>>>> is also being accessed non-coherently is going to cause issues one way >>>>> or another. On top of the speculative cache fills that have to be >>>>> constantly fought back against with CMOs like below; some coherent >>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>> (snoop filters get messed up). >>>>> >>>>> The solution is to either always have the addresses marked non-coherent >>>>> (like device memory, no-map carveouts), or if you really want to use >>>>> regular system memory allocated at runtime, then all cached mappings of >>>>> it need to be dropped, even the kernel logical address (area as painful >>>>> as that would be). >>> >>> Ouch :-( I wasn't aware about these potential interconnect issues. How >>> "real" is that? It seems that we aren't really hitting that today on >>> real devices. >>> >> >> Sadly there is at least one real device like this now (TI AM654). We >> spent some time working with the ARM interconnect spec designers to see >> if this was allowed behavior, final conclusion was mixing coherent and >> non-coherent accesses is never a good idea.. So we have been working to >> try to minimize any cases of mixed attributes [0], if a region is >> coherent then everyone in the system needs to treat it as such and >> vice-versa, even clever CMO that work on other systems wont save you >> here. :( >> >> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 >> >> >>>>> >>>>>> ION buffer is allocated. >>>>>> >>>>>> //Camera device records video >>>>>> dma_buf_attach >>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>> >>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>> >>>> Actually +Brian this time :) >>>> >>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>> problem. >>>>> >>>>>> [camera device writes to buffer] >>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>> >>>>> It doesn't know there will be any further CPU access, it could get freed >>>>> after this for all we know, the invalidate can be saved until the CPU >>>>> requests access again. >>> >>> We don't have any API to allow the invalidate to happen on CPU access >>> if all devices already detached. We need a struct device pointer to >>> give to the DMA API, otherwise on arm64 there'll be no invalidate. >>> >>> I had a chat with a few people internally after the previous >>> discussion with Liam. One suggestion was to use >>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least >>> one other device attached (guarantees that we can do an invalidate in >>> the future if begin_cpu_access is called). If the last device >>> detaches, do a sync then. >>> >>> Conversely, in map_dma_buf, we would track if there was any CPU access >>> and use/skip the sync appropriately. >>> >> >> Now that I think this all through I agree this patch is probably wrong. >> The real fix needs to be better handling in the dma_map_sg() to deal >> with the case of the memory not being mapped (what I'm dealing with for >> unmapped heaps), and for cases when the memory in question is not cached >> (Liam's issue I think). For both these cases the dma_map_sg() does the >> wrong thing. >> >>> I did start poking the code to check out how that would look, but then >>> Christmas happened and I'm still catching back up. >>> >>>>> >>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>> >>>>> >>>>> This seems like a broken use-case, I understand the desire to keep >>>>> everything as modular as possible and separate the steps, but at this >>>>> point no one owns this buffers backing memory, not the CPU or any >>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>> for the next attachment, which may change the required backing memory >>>>> completely. >>>>> >>>>> All devices should attach before the first mapping, and only let go >>>>> after the task is complete, otherwise this buffers data needs copied off >>>>> to a different location or the CPU needs to take ownership in-between. >>>>> >>> >>> Yeah.. that's certainly the theory. Are there any DMA-BUF >>> implementations which actually do that? I hear it quoted a lot, >>> because that's what the docs say - but if the reality doesn't match >>> it, maybe we should change the docs. >>> >> >> Do you mean on the userspace side? I'm not sure, seems like Android >> might be doing this wrong from what I can gather. From kernel side if >> you mean the "de-allocate the backing storage", we will have some cases >> like this soon, so I want to make sure userspace is not abusing DMA-BUF >> in ways not specified in the documentation. Changing the docs to force >> the backing memory to always be allocated breaks the central goal in >> having attach/map in DMA-BUF separate. >> >>>>>> //buffer is send down the pipeline >>>>>> >>>>>> // Usersapce software post processing occurs >>>>>> mmap buffer >>>>> >>>>> Perhaps the invalidate should happen here in mmap. >>>>> >>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>>> devices attached to buffer >>>>> >>>>> And that should be okay, mmap does the sync, and if no devices are >>>>> attached nothing could have changed the underlying memory in the >>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>> >>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. >>> Liam was saying that it's too painful for them to do that every time a >>> device unmaps - when in many cases (device->device, no CPU) it's not >>> needed. >> >> Invalidates are painless, at least compared to a real cache flush, just >> set the invalid bit vs actually writing out lines. I thought the issue >> was on the map side. >> > > Invalidates aren't painless for us because we have a coherent system cache > so clean lines get written out. That seems very broken, why would clean lines ever need to be written out, that defeats the whole point of having the invalidate separate from clean. How do you deal with stale cache lines? I guess in your case this is what forces you to have to use uncached memory for DMA-able memory. > And these invalidates can occur on fairly large buffers. > > That is why we haven't went with using cached ION memory and "tracking CPU > access" because it only solves half the problem, ie there isn't a way to > safely skip the invalidate (because we can't read the future). > Our solution was to go with uncached ION memory (when possible), but as > you can see in other discussions upstream support for uncached memory has > its own issues. > Sounds like you need to fix upstream support then, finding a way to drop all cacheable mappings of memory you want to make uncached mappings for seems to be the only solution. >>> >>>>> >>>>>> [CPU reads/writes to the buffer] >>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>>> devices attached to buffer >>>>>> munmap buffer >>>>>> >>>>>> //buffer is send down the pipeline >>>>>> // Buffer is send to video device (who does compression of raw data) and >>>>>> writes to a file >>>>>> dma_buf_attach >>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>> [video device writes to buffer] >>>>>> dma_buf_unmap_attachment >>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>> the pipeline and Video doesn't know the end of the use case) >>>>>> >>>>>> >>>>>> >>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>>>>>> access then there is no requirement (that I am aware of) for you to call >>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>>> >>>>>>> >>>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>>> maintenance on the buffer? >>>>>>> >>>>>> >>>>>> Because ION no longer provides DMA ready memory. >>>>>> Take the above example. >>>>>> >>>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>>> Zeros are written to the cache. >>>>>> >>>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>>> The camera devices writes directly to the buffer in DDR. >>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >>>>>> evicted from the cache, this zero overwrites data the camera device has >>>>>> written which corrupts your data. >>>>>> >>>>> >>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>>> for CPU access at the time of zeroing. >>>>> >>> >>> Actually that should be at the point of the first non-coherent device >>> mapping the buffer right? No point in doing CMO if the future accesses >>> are coherent. >> >> I see your point, as long as the zeroing is guaranteed to be the first >> access to this buffer then it should be safe. >> >> Andrew >> >>> >>> Cheers, >>> -Brian >>> >>>>> Andrew >>>>> >>>>>> Liam >>>>>> >>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>>> a Linux Foundation Collaborative Project >>>>>> >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On Thu, 17 Jan 2019, Andrew F. Davis wrote: > On 1/16/19 4:48 PM, Liam Mark wrote: > > On Wed, 16 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/15/19 1:05 PM, Laura Abbott wrote: > >>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: > >>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>> > >>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>> > >>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance > >>>>>>>> here. > >>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed > >>>>>>>> anyway. > >>>>>>>> > >>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>> --- > >>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>> > >>>>>>>> diff --git a/drivers/staging/android/ion/ion.c > >>>>>>>> b/drivers/staging/android/ion/ion.c > >>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct > >>>>>>>> dma_buf_attachment *attachment, > >>>>>>>> table = a->table; > >>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>> - direction)) > >>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>> > >>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache > >>>>>>> maintenance. > >>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>> dma_buf_attach then there won't have been a device attached so the > >>>>>>> calls > >>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>> > >>>>>> > >>>>>> That should be okay though, if you have no attachments (or all > >>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>> is attached later after data has already been written. Does that > >>>>>> sequence need supporting? > >>>>> > >>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>> in Android, but I will focus on later for now. > >>>>> > >>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>> attached and mapped, right? > >>>>>> > >>>>> > >>>>> Here is an example where CPU access can happen later in Android. > >>>>> > >>>>> Camera device records video -> software post processing -> video device > >>>>> (who does compression of raw data) and writes to a file > >>>>> > >>>>> In this example assume the buffer is cached and the devices are not > >>>>> IO-coherent (quite common). > >>>>> > >>>> > >>>> This is the start of the problem, having cached mappings of memory that > >>>> is also being accessed non-coherently is going to cause issues one way > >>>> or another. On top of the speculative cache fills that have to be > >>>> constantly fought back against with CMOs like below; some coherent > >>>> interconnects behave badly when you mix coherent and non-coherent access > >>>> (snoop filters get messed up). > >>>> > >>>> The solution is to either always have the addresses marked non-coherent > >>>> (like device memory, no-map carveouts), or if you really want to use > >>>> regular system memory allocated at runtime, then all cached mappings of > >>>> it need to be dropped, even the kernel logical address (area as painful > >>>> as that would be). > >>>> > >>> > >>> I agree it's broken, hence my desire to remove it :) > >>> > >>> The other problem is that uncached buffers are being used for > >>> performance reason so anything that would involve getting > >>> rid of the logical address would probably negate any performance > >>> benefit. > >>> > >> > >> I wouldn't go as far as to remove them just yet.. Liam seems pretty > >> adamant that they have valid uses. I'm just not sure performance is one > >> of them, maybe in the case of software locks between devices or > >> something where there needs to be a lot of back and forth interleaved > >> access on small amounts of data? > >> > > > > I wasn't aware that ARM considered this not supported, I thought it was > > supported but they advised against it because of the potential performance > > impact. > > > > Not sure what you mean by "this" being not supported, do you mean mixed > attribute mappings? If so, it will certainly cause problems, and the > problems will change from platform to platform, avoid at all costs is my > understanding of ARM's position. > > > This is after all supported in the DMA APIs and up until now devices have > > been successfully commercializing with this configurations, and I think > > they will continue to commercialize with these configurations for quite a > > while. > > > > Use of uncached memory mappings are almost always wrong in my experience > and are used to work around some bug or because the user doesn't want to > implement proper CMOs. Counter examples welcome. > Okay, let me first try to clarify what I am referring to, as perhaps I am misunderstanding the conversation. In this discussion I was originally referring to a use case with cached memory being accessed by a non io-cohernet device. "In this example assume the buffer is cached and the devices are not IO-coherent (quite common)." to which you did not think was supported: "This is the start of the problem, having cached mappings of memory that is also being accessed non-coherently is going to cause issues one way or another. " And I interpreted Laura's comment below as saying she wanted to remove support in ION for cached memory being accessed by non io-cohernet devices: "I agree it's broken, hence my desire to remove it :)" So assuming my understanding above is correct (and you are not talking about something separate such as removing uncached ION allocation support). Then I guess I am not clear why current uses which use cached memory with non IO-coherent devices are considered to be working around some bug or are not implementing proper CMOs. They use CPU cached mappings because that is the most effective way to access the memory from the CPU side and the devices have an uncached IOMMU mapping because they don't support IO-coherency, and currenlty in the CPU they do cache mainteance at the time of dma map and dma umap so to me they are implementing correct CMOs. > > It would be really unfortunate if support was removed as I think that > > would drive clients away from using upstream ION. > > > > I'm not petitioning to remove support, but at very least lets reverse > the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by > default, to get uncached you should need to add a flag to your > allocation command pointing out you know what you are doing. > You may not be petitioning to remove support for using cached memory with non io-coherent devices but I interpreted Laura's comment as wanting to do so, and I had concerns about that. > >>>>> ION buffer is allocated. > >>>>> > >>>>> //Camera device records video > >>>>> dma_buf_attach > >>>>> dma_map_attachment (buffer needs to be cleaned) > >>>> > >>>> Why does the buffer need to be cleaned here? I just got through reading > >>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>> suggestion of tracking if the buffer has had CPU access since the last > >>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>> problem. > >>>> > >>>>> [camera device writes to buffer] > >>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>> > >>>> It doesn't know there will be any further CPU access, it could get freed > >>>> after this for all we know, the invalidate can be saved until the CPU > >>>> requests access again. > >>>> > >>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>> down > >>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>> > >>>> > >>>> This seems like a broken use-case, I understand the desire to keep > >>>> everything as modular as possible and separate the steps, but at this > >>>> point no one owns this buffers backing memory, not the CPU or any > >>>> device. I would go as far as to say DMA-BUF should be free now to > >>>> de-allocate the backing storage if it wants, that way it could get ready > >>>> for the next attachment, which may change the required backing memory > >>>> completely. > >>>> > >>>> All devices should attach before the first mapping, and only let go > >>>> after the task is complete, otherwise this buffers data needs copied off > >>>> to a different location or the CPU needs to take ownership in-between. > >>>> > >>> > >>> Maybe it's broken but it's the status quo and we spent a good > >>> amount of time at plumbers concluding there isn't a great way > >>> to fix it :/ > >>> > >> > >> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ > >> > >> Perhaps just stronger rules on sequencing of operations? I'm not saying > >> I have a good solution either, I just don't see any way forward without > >> some use-case getting broken, so better to fix now over later. > >> > > > > I can see the benefits of Android doing things the way they do, I would > > request that changes we make continue to support Android, or we find a way > > to convice them to change, as they are the main ION client and I assume > > other ION clients in the future will want to do this as well. > > > > Android may be the biggest user today (makes sense, Ion come out of the > Android project), but that can change, and getting changes into Android > will be easier that the upstream kernel once Ion is out of staging. > > Unlike some other big ARM vendors, we (TI) do not primarily build mobile > chips targeting Android, our core offerings target more traditional > Linux userspaces, and I'm guessing others will start to do the same as > ARM tries to push more into desktop, server, and other spaces again. > > > I am concerned that if you go with a solution which enforces what you > > mention above, and bring ION out of staging that way, it will make it that > > much harder to solve this for Android and therefore harder to get > > Android clients to move to the upstream ION (and get everybody off their > > vendor modified Android versions). > > > > That would be an Android problem, reducing functionality in upstream to > match what some evil vendor trees do to support Android is not the way > forward on this. At least for us we are going to try to make all our > software offerings follow proper buffer ownership (including our Android > offering). > > >>>>> //buffer is send down the pipeline > >>>>> > >>>>> // Usersapce software post processing occurs > >>>>> mmap buffer > >>>> > >>>> Perhaps the invalidate should happen here in mmap. > >>>> > >>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>> devices attached to buffer > >>>> > >>>> And that should be okay, mmap does the sync, and if no devices are > >>>> attached nothing could have changed the underlying memory in the > >>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>>> > >>>>> [CPU reads/writes to the buffer] > >>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>> devices attached to buffer > >>>>> munmap buffer > >>>>> > >>>>> //buffer is send down the pipeline > >>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>> writes to a file > >>>>> dma_buf_attach > >>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>> [video device writes to buffer] > >>>>> dma_buf_unmap_attachment > >>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>> down > >>>>> the pipeline and Video doesn't know the end of the use case) > >>>>> > >>>>> > >>>>> > >>>>>>> Also ION no longer provides DMA ready memory, so if you are not > >>>>>>> doing CPU > >>>>>>> access then there is no requirement (that I am aware of) for you to > >>>>>>> call > >>>>>>> {begin,end}_cpu_access before passing the buffer to the device and > >>>>>>> if this > >>>>>>> buffer is cached and your device is not IO-coherent then the cache > >>>>>>> maintenance > >>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>> > >>>>>> > >>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>> maintenance on the buffer? > >>>>>> > >>>>> > >>>>> Because ION no longer provides DMA ready memory. > >>>>> Take the above example. > >>>>> > >>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>> Zeros are written to the cache. > >>>>> > >>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>> The camera devices writes directly to the buffer in DDR. > >>>>> Since you didn't clean the buffer a dirty cache line (one of the > >>>>> zeros) is > >>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>> written which corrupts your data. > >>>>> > >>>> > >>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>> for CPU access at the time of zeroing. > >>>> > >>>> Andrew > >>>> > >>>>> Liam > >>>>> > >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>> a Linux Foundation Collaborative Project > >>>>> > >>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On Thu, 17 Jan 2019, Andrew F. Davis wrote: > On 1/16/19 4:54 PM, Liam Mark wrote: > > On Wed, 16 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/16/19 9:19 AM, Brian Starkey wrote: > >>> Hi :-) > >>> > >>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > >>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: > >>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>>> > >>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>>> > >>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > >>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >>>>>>>>> > >>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>>> --- > >>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>>> > >>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >>>>>>>>> > >>>>>>>>> table = a->table; > >>>>>>>>> > >>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>>> - direction)) > >>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>>> > >>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > >>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>>> dma_buf_attach then there won't have been a device attached so the calls > >>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>>> > >>>>>>> > >>>>>>> That should be okay though, if you have no attachments (or all > >>>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>>> is attached later after data has already been written. Does that > >>>>>>> sequence need supporting? > >>>>>> > >>>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>>> in Android, but I will focus on later for now. > >>>>>> > >>>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>>> attached and mapped, right? > >>>>>>> > >>>>>> > >>>>>> Here is an example where CPU access can happen later in Android. > >>>>>> > >>>>>> Camera device records video -> software post processing -> video device > >>>>>> (who does compression of raw data) and writes to a file > >>>>>> > >>>>>> In this example assume the buffer is cached and the devices are not > >>>>>> IO-coherent (quite common). > >>>>>> > >>>>> > >>>>> This is the start of the problem, having cached mappings of memory that > >>>>> is also being accessed non-coherently is going to cause issues one way > >>>>> or another. On top of the speculative cache fills that have to be > >>>>> constantly fought back against with CMOs like below; some coherent > >>>>> interconnects behave badly when you mix coherent and non-coherent access > >>>>> (snoop filters get messed up). > >>>>> > >>>>> The solution is to either always have the addresses marked non-coherent > >>>>> (like device memory, no-map carveouts), or if you really want to use > >>>>> regular system memory allocated at runtime, then all cached mappings of > >>>>> it need to be dropped, even the kernel logical address (area as painful > >>>>> as that would be). > >>> > >>> Ouch :-( I wasn't aware about these potential interconnect issues. How > >>> "real" is that? It seems that we aren't really hitting that today on > >>> real devices. > >>> > >> > >> Sadly there is at least one real device like this now (TI AM654). We > >> spent some time working with the ARM interconnect spec designers to see > >> if this was allowed behavior, final conclusion was mixing coherent and > >> non-coherent accesses is never a good idea.. So we have been working to > >> try to minimize any cases of mixed attributes [0], if a region is > >> coherent then everyone in the system needs to treat it as such and > >> vice-versa, even clever CMO that work on other systems wont save you > >> here. :( > >> > >> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 > >> > >> > >>>>> > >>>>>> ION buffer is allocated. > >>>>>> > >>>>>> //Camera device records video > >>>>>> dma_buf_attach > >>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>> > >>>>> Why does the buffer need to be cleaned here? I just got through reading > >>>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>> > >>>> Actually +Brian this time :) > >>>> > >>>>> suggestion of tracking if the buffer has had CPU access since the last > >>>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>>> problem. > >>>>> > >>>>>> [camera device writes to buffer] > >>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>>> > >>>>> It doesn't know there will be any further CPU access, it could get freed > >>>>> after this for all we know, the invalidate can be saved until the CPU > >>>>> requests access again. > >>> > >>> We don't have any API to allow the invalidate to happen on CPU access > >>> if all devices already detached. We need a struct device pointer to > >>> give to the DMA API, otherwise on arm64 there'll be no invalidate. > >>> > >>> I had a chat with a few people internally after the previous > >>> discussion with Liam. One suggestion was to use > >>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > >>> one other device attached (guarantees that we can do an invalidate in > >>> the future if begin_cpu_access is called). If the last device > >>> detaches, do a sync then. > >>> > >>> Conversely, in map_dma_buf, we would track if there was any CPU access > >>> and use/skip the sync appropriately. > >>> > >> > >> Now that I think this all through I agree this patch is probably wrong. > >> The real fix needs to be better handling in the dma_map_sg() to deal > >> with the case of the memory not being mapped (what I'm dealing with for > >> unmapped heaps), and for cases when the memory in question is not cached > >> (Liam's issue I think). For both these cases the dma_map_sg() does the > >> wrong thing. > >> > >>> I did start poking the code to check out how that would look, but then > >>> Christmas happened and I'm still catching back up. > >>> > >>>>> > >>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>>> > >>>>> > >>>>> This seems like a broken use-case, I understand the desire to keep > >>>>> everything as modular as possible and separate the steps, but at this > >>>>> point no one owns this buffers backing memory, not the CPU or any > >>>>> device. I would go as far as to say DMA-BUF should be free now to > >>>>> de-allocate the backing storage if it wants, that way it could get ready > >>>>> for the next attachment, which may change the required backing memory > >>>>> completely. > >>>>> > >>>>> All devices should attach before the first mapping, and only let go > >>>>> after the task is complete, otherwise this buffers data needs copied off > >>>>> to a different location or the CPU needs to take ownership in-between. > >>>>> > >>> > >>> Yeah.. that's certainly the theory. Are there any DMA-BUF > >>> implementations which actually do that? I hear it quoted a lot, > >>> because that's what the docs say - but if the reality doesn't match > >>> it, maybe we should change the docs. > >>> > >> > >> Do you mean on the userspace side? I'm not sure, seems like Android > >> might be doing this wrong from what I can gather. From kernel side if > >> you mean the "de-allocate the backing storage", we will have some cases > >> like this soon, so I want to make sure userspace is not abusing DMA-BUF > >> in ways not specified in the documentation. Changing the docs to force > >> the backing memory to always be allocated breaks the central goal in > >> having attach/map in DMA-BUF separate. > >> > >>>>>> //buffer is send down the pipeline > >>>>>> > >>>>>> // Usersapce software post processing occurs > >>>>>> mmap buffer > >>>>> > >>>>> Perhaps the invalidate should happen here in mmap. > >>>>> > >>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>>> devices attached to buffer > >>>>> > >>>>> And that should be okay, mmap does the sync, and if no devices are > >>>>> attached nothing could have changed the underlying memory in the > >>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>> > >>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > >>> Liam was saying that it's too painful for them to do that every time a > >>> device unmaps - when in many cases (device->device, no CPU) it's not > >>> needed. > >> > >> Invalidates are painless, at least compared to a real cache flush, just > >> set the invalid bit vs actually writing out lines. I thought the issue > >> was on the map side. > >> > > > > Invalidates aren't painless for us because we have a coherent system cache > > so clean lines get written out. > > That seems very broken, why would clean lines ever need to be written > out, that defeats the whole point of having the invalidate separate from > clean. How do you deal with stale cache lines? I guess in your case this > is what forces you to have to use uncached memory for DMA-able memory. > My understanding is that our ARM invalidate is a clean + invalidate, I had concerns about the clean lines being written to the the system cache as part of the 'clean', but the following 'invalidate' would take care of actually invalidating the lines (so nothign broken). But i am probably wrong on this and it is probably smart enough not to the writing of the clean lines. But regardless, targets supporting a coherent system cache is a legitamate configuration and an invalidate on this configuration does have to go to the bus to invalidate the system cache (which isn't free) so I dont' think you can make the assumption that invalidates are cheap so that it is okay to do them (even if they are not needed) on every dma unmap. > > And these invalidates can occur on fairly large buffers. > > > > That is why we haven't went with using cached ION memory and "tracking CPU > > access" because it only solves half the problem, ie there isn't a way to > > safely skip the invalidate (because we can't read the future). > > Our solution was to go with uncached ION memory (when possible), but as > > you can see in other discussions upstream support for uncached memory has > > its own issues. > > > > Sounds like you need to fix upstream support then, finding a way to drop > all cacheable mappings of memory you want to make uncached mappings for > seems to be the only solution. > I think we can probably agree that there woudln't be a good way to remove cached mappings without causing an unacceptable performance degradation since it would fragment all the nice 1GB kernel mappings we have. So I am trying to find an alternative solution. > >>> > >>>>> > >>>>>> [CPU reads/writes to the buffer] > >>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>>> devices attached to buffer > >>>>>> munmap buffer > >>>>>> > >>>>>> //buffer is send down the pipeline > >>>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>>> writes to a file > >>>>>> dma_buf_attach > >>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>> [video device writes to buffer] > >>>>>> dma_buf_unmap_attachment > >>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>> the pipeline and Video doesn't know the end of the use case) > >>>>>> > >>>>>> > >>>>>> > >>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > >>>>>>>> access then there is no requirement (that I am aware of) for you to call > >>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this > >>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance > >>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>>> > >>>>>>> > >>>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>>> maintenance on the buffer? > >>>>>>> > >>>>>> > >>>>>> Because ION no longer provides DMA ready memory. > >>>>>> Take the above example. > >>>>>> > >>>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>>> Zeros are written to the cache. > >>>>>> > >>>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>>> The camera devices writes directly to the buffer in DDR. > >>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > >>>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>>> written which corrupts your data. > >>>>>> > >>>>> > >>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>>> for CPU access at the time of zeroing. > >>>>> > >>> > >>> Actually that should be at the point of the first non-coherent device > >>> mapping the buffer right? No point in doing CMO if the future accesses > >>> are coherent. > >> > >> I see your point, as long as the zeroing is guaranteed to be the first > >> access to this buffer then it should be safe. > >> > >> Andrew > >> > >>> > >>> Cheers, > >>> -Brian > >>> > >>>>> Andrew > >>>>> > >>>>>> Liam > >>>>>> > >>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>>> a Linux Foundation Collaborative Project > >>>>>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On 1/17/19 7:04 PM, Liam Mark wrote: > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > >> On 1/16/19 4:48 PM, Liam Mark wrote: >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/15/19 1:05 PM, Laura Abbott wrote: >>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>> >>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>> >>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>>>> here. >>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>>>> anyway. >>>>>>>>>> >>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>> --- >>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>> >>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct >>>>>>>>>> dma_buf_attachment *attachment, >>>>>>>>>> table = a->table; >>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>>> - direction)) >>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>> >>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>>>> maintenance. >>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>>>> calls >>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>> >>>>>>>> >>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>>> is attached later after data has already been written. Does that >>>>>>>> sequence need supporting? >>>>>>> >>>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>>> in Android, but I will focus on later for now. >>>>>>> >>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>>> attached and mapped, right? >>>>>>>> >>>>>>> >>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>> >>>>>>> Camera device records video -> software post processing -> video device >>>>>>> (who does compression of raw data) and writes to a file >>>>>>> >>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>> IO-coherent (quite common). >>>>>>> >>>>>> >>>>>> This is the start of the problem, having cached mappings of memory that >>>>>> is also being accessed non-coherently is going to cause issues one way >>>>>> or another. On top of the speculative cache fills that have to be >>>>>> constantly fought back against with CMOs like below; some coherent >>>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>>> (snoop filters get messed up). >>>>>> >>>>>> The solution is to either always have the addresses marked non-coherent >>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>> regular system memory allocated at runtime, then all cached mappings of >>>>>> it need to be dropped, even the kernel logical address (area as painful >>>>>> as that would be). >>>>>> >>>>> >>>>> I agree it's broken, hence my desire to remove it :) >>>>> >>>>> The other problem is that uncached buffers are being used for >>>>> performance reason so anything that would involve getting >>>>> rid of the logical address would probably negate any performance >>>>> benefit. >>>>> >>>> >>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty >>>> adamant that they have valid uses. I'm just not sure performance is one >>>> of them, maybe in the case of software locks between devices or >>>> something where there needs to be a lot of back and forth interleaved >>>> access on small amounts of data? >>>> >>> >>> I wasn't aware that ARM considered this not supported, I thought it was >>> supported but they advised against it because of the potential performance >>> impact. >>> >> >> Not sure what you mean by "this" being not supported, do you mean mixed >> attribute mappings? If so, it will certainly cause problems, and the >> problems will change from platform to platform, avoid at all costs is my >> understanding of ARM's position. >> >>> This is after all supported in the DMA APIs and up until now devices have >>> been successfully commercializing with this configurations, and I think >>> they will continue to commercialize with these configurations for quite a >>> while. >>> >> >> Use of uncached memory mappings are almost always wrong in my experience >> and are used to work around some bug or because the user doesn't want to >> implement proper CMOs. Counter examples welcome. >> > > Okay, let me first try to clarify what I am referring to, as perhaps I am > misunderstanding the conversation. > > In this discussion I was originally referring to a use case with cached > memory being accessed by a non io-cohernet device. > > "In this example assume the buffer is cached and the devices are not > IO-coherent (quite common)." > > to which you did not think was supported: > > "This is the start of the problem, having cached mappings of memory > that is also being accessed non-coherently is going to cause issues > one way or another. > " > > And I interpreted Laura's comment below as saying she wanted to remove > support in ION for cached memory being accessed by non io-cohernet > devices: > "I agree it's broken, hence my desire to remove it :)" > > So assuming my understanding above is correct (and you are not talking > about something separate such as removing uncached ION allocation > support). > Ah, I think here is where we diverged, I'm assuming Laura's comment to be referencing my issue with uncached mappings being handed out without first removing all cached mappings of the same memory. Therefore it is uncached heaps that are broken. > Then I guess I am not clear why current uses which use cached memory with > non IO-coherent devices are considered to be working around some bug or > are not implementing proper CMOs. > > They use CPU cached mappings because that is the most effective way to > access the memory from the CPU side and the devices have an uncached > IOMMU mapping because they don't support IO-coherency, and currenlty in > the CPU they do cache mainteance at the time of dma map and dma umap so > to me they are implementing correct CMOs. > Fully agree here, using cached mappings and performing CMOs when needed is the way to go when dealing with memory. IMHO the *only* time when uncached mappings are appropriate is for memory mapped I/O (although it looks like video memory was often treated as uncached (wc)). >>> It would be really unfortunate if support was removed as I think that >>> would drive clients away from using upstream ION. >>> >> >> I'm not petitioning to remove support, but at very least lets reverse >> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by >> default, to get uncached you should need to add a flag to your >> allocation command pointing out you know what you are doing. >> > > You may not be petitioning to remove support for using cached memory with > non io-coherent devices but I interpreted Laura's comment as wanting to do > so, and I had concerns about that. > What I would like is for the default memory handed out by Ion to be normal cacheable memory, just like is always handed out to users-space. DMA-BUF already provides the means to deal with the CMOs required to work with non-io-coherent devices so all should be good here. If you want Ion to give out uncached memory then I think you should need to explicitly state so with an allocation flag. And right now the uncached memory you will get back may have other cached mappings (kernel lowmem mappings) meaning you will have hard to predict results (on ARM at least). I just don't see much use for them (uncached mappings of regular memory) right now. >>>>>>> ION buffer is allocated. >>>>>>> >>>>>>> //Camera device records video >>>>>>> dma_buf_attach >>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>> >>>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>>> problem. >>>>>> >>>>>>> [camera device writes to buffer] >>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>> >>>>>> It doesn't know there will be any further CPU access, it could get freed >>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>> requests access again. >>>>>> >>>>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>>>> down >>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>> >>>>>> >>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>> everything as modular as possible and separate the steps, but at this >>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>>> for the next attachment, which may change the required backing memory >>>>>> completely. >>>>>> >>>>>> All devices should attach before the first mapping, and only let go >>>>>> after the task is complete, otherwise this buffers data needs copied off >>>>>> to a different location or the CPU needs to take ownership in-between. >>>>>> >>>>> >>>>> Maybe it's broken but it's the status quo and we spent a good >>>>> amount of time at plumbers concluding there isn't a great way >>>>> to fix it :/ >>>>> >>>> >>>> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ >>>> >>>> Perhaps just stronger rules on sequencing of operations? I'm not saying >>>> I have a good solution either, I just don't see any way forward without >>>> some use-case getting broken, so better to fix now over later. >>>> >>> >>> I can see the benefits of Android doing things the way they do, I would >>> request that changes we make continue to support Android, or we find a way >>> to convice them to change, as they are the main ION client and I assume >>> other ION clients in the future will want to do this as well. >>> >> >> Android may be the biggest user today (makes sense, Ion come out of the >> Android project), but that can change, and getting changes into Android >> will be easier that the upstream kernel once Ion is out of staging. >> >> Unlike some other big ARM vendors, we (TI) do not primarily build mobile >> chips targeting Android, our core offerings target more traditional >> Linux userspaces, and I'm guessing others will start to do the same as >> ARM tries to push more into desktop, server, and other spaces again. >> >>> I am concerned that if you go with a solution which enforces what you >>> mention above, and bring ION out of staging that way, it will make it that >>> much harder to solve this for Android and therefore harder to get >>> Android clients to move to the upstream ION (and get everybody off their >>> vendor modified Android versions). >>> >> >> That would be an Android problem, reducing functionality in upstream to >> match what some evil vendor trees do to support Android is not the way >> forward on this. At least for us we are going to try to make all our >> software offerings follow proper buffer ownership (including our Android >> offering). >> >>>>>>> //buffer is send down the pipeline >>>>>>> >>>>>>> // Usersapce software post processing occurs >>>>>>> mmap buffer >>>>>> >>>>>> Perhaps the invalidate should happen here in mmap. >>>>>> >>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>>>> devices attached to buffer >>>>>> >>>>>> And that should be okay, mmap does the sync, and if no devices are >>>>>> attached nothing could have changed the underlying memory in the >>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>>>>> >>>>>>> [CPU reads/writes to the buffer] >>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>>>> devices attached to buffer >>>>>>> munmap buffer >>>>>>> >>>>>>> //buffer is send down the pipeline >>>>>>> // Buffer is send to video device (who does compression of raw data) and >>>>>>> writes to a file >>>>>>> dma_buf_attach >>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>> [video device writes to buffer] >>>>>>> dma_buf_unmap_attachment >>>>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>>>> down >>>>>>> the pipeline and Video doesn't know the end of the use case) >>>>>>> >>>>>>> >>>>>>> >>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not >>>>>>>>> doing CPU >>>>>>>>> access then there is no requirement (that I am aware of) for you to >>>>>>>>> call >>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and >>>>>>>>> if this >>>>>>>>> buffer is cached and your device is not IO-coherent then the cache >>>>>>>>> maintenance >>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>>>> >>>>>>>> >>>>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>>>> maintenance on the buffer? >>>>>>>> >>>>>>> >>>>>>> Because ION no longer provides DMA ready memory. >>>>>>> Take the above example. >>>>>>> >>>>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>>>> Zeros are written to the cache. >>>>>>> >>>>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>>>> The camera devices writes directly to the buffer in DDR. >>>>>>> Since you didn't clean the buffer a dirty cache line (one of the >>>>>>> zeros) is >>>>>>> evicted from the cache, this zero overwrites data the camera device has >>>>>>> written which corrupts your data. >>>>>>> >>>>>> >>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>>>> for CPU access at the time of zeroing. >>>>>> >>>>>> Andrew >>>>>> >>>>>>> Liam >>>>>>> >>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>>>> a Linux Foundation Collaborative Project >>>>>>> >>>>> >>>> >>> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>> a Linux Foundation Collaborative Project >>> >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On 1/17/19 7:11 PM, Liam Mark wrote: > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > >> On 1/16/19 4:54 PM, Liam Mark wrote: >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/16/19 9:19 AM, Brian Starkey wrote: >>>>> Hi :-) >>>>> >>>>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: >>>>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: >>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>>> >>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>>> >>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>>>>>>>> >>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>>> --- >>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>>> >>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>>>>>>>> >>>>>>>>>>> table = a->table; >>>>>>>>>>> >>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>>>> - direction)) >>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>>> >>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>>>> dma_buf_attach then there won't have been a device attached so the calls >>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>>> >>>>>>>>> >>>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>>>> is attached later after data has already been written. Does that >>>>>>>>> sequence need supporting? >>>>>>>> >>>>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>>>> in Android, but I will focus on later for now. >>>>>>>> >>>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>>>> attached and mapped, right? >>>>>>>>> >>>>>>>> >>>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>>> >>>>>>>> Camera device records video -> software post processing -> video device >>>>>>>> (who does compression of raw data) and writes to a file >>>>>>>> >>>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>>> IO-coherent (quite common). >>>>>>>> >>>>>>> >>>>>>> This is the start of the problem, having cached mappings of memory that >>>>>>> is also being accessed non-coherently is going to cause issues one way >>>>>>> or another. On top of the speculative cache fills that have to be >>>>>>> constantly fought back against with CMOs like below; some coherent >>>>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>>>> (snoop filters get messed up). >>>>>>> >>>>>>> The solution is to either always have the addresses marked non-coherent >>>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>>> regular system memory allocated at runtime, then all cached mappings of >>>>>>> it need to be dropped, even the kernel logical address (area as painful >>>>>>> as that would be). >>>>> >>>>> Ouch :-( I wasn't aware about these potential interconnect issues. How >>>>> "real" is that? It seems that we aren't really hitting that today on >>>>> real devices. >>>>> >>>> >>>> Sadly there is at least one real device like this now (TI AM654). We >>>> spent some time working with the ARM interconnect spec designers to see >>>> if this was allowed behavior, final conclusion was mixing coherent and >>>> non-coherent accesses is never a good idea.. So we have been working to >>>> try to minimize any cases of mixed attributes [0], if a region is >>>> coherent then everyone in the system needs to treat it as such and >>>> vice-versa, even clever CMO that work on other systems wont save you >>>> here. :( >>>> >>>> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 >>>> >>>> >>>>>>> >>>>>>>> ION buffer is allocated. >>>>>>>> >>>>>>>> //Camera device records video >>>>>>>> dma_buf_attach >>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>> >>>>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>> >>>>>> Actually +Brian this time :) >>>>>> >>>>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>>>> problem. >>>>>>> >>>>>>>> [camera device writes to buffer] >>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>>> >>>>>>> It doesn't know there will be any further CPU access, it could get freed >>>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>>> requests access again. >>>>> >>>>> We don't have any API to allow the invalidate to happen on CPU access >>>>> if all devices already detached. We need a struct device pointer to >>>>> give to the DMA API, otherwise on arm64 there'll be no invalidate. >>>>> >>>>> I had a chat with a few people internally after the previous >>>>> discussion with Liam. One suggestion was to use >>>>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least >>>>> one other device attached (guarantees that we can do an invalidate in >>>>> the future if begin_cpu_access is called). If the last device >>>>> detaches, do a sync then. >>>>> >>>>> Conversely, in map_dma_buf, we would track if there was any CPU access >>>>> and use/skip the sync appropriately. >>>>> >>>> >>>> Now that I think this all through I agree this patch is probably wrong. >>>> The real fix needs to be better handling in the dma_map_sg() to deal >>>> with the case of the memory not being mapped (what I'm dealing with for >>>> unmapped heaps), and for cases when the memory in question is not cached >>>> (Liam's issue I think). For both these cases the dma_map_sg() does the >>>> wrong thing. >>>> >>>>> I did start poking the code to check out how that would look, but then >>>>> Christmas happened and I'm still catching back up. >>>>> >>>>>>> >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>>> >>>>>>> >>>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>>> everything as modular as possible and separate the steps, but at this >>>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>>>> for the next attachment, which may change the required backing memory >>>>>>> completely. >>>>>>> >>>>>>> All devices should attach before the first mapping, and only let go >>>>>>> after the task is complete, otherwise this buffers data needs copied off >>>>>>> to a different location or the CPU needs to take ownership in-between. >>>>>>> >>>>> >>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF >>>>> implementations which actually do that? I hear it quoted a lot, >>>>> because that's what the docs say - but if the reality doesn't match >>>>> it, maybe we should change the docs. >>>>> >>>> >>>> Do you mean on the userspace side? I'm not sure, seems like Android >>>> might be doing this wrong from what I can gather. From kernel side if >>>> you mean the "de-allocate the backing storage", we will have some cases >>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF >>>> in ways not specified in the documentation. Changing the docs to force >>>> the backing memory to always be allocated breaks the central goal in >>>> having attach/map in DMA-BUF separate. >>>> >>>>>>>> //buffer is send down the pipeline >>>>>>>> >>>>>>>> // Usersapce software post processing occurs >>>>>>>> mmap buffer >>>>>>> >>>>>>> Perhaps the invalidate should happen here in mmap. >>>>>>> >>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>>>>> devices attached to buffer >>>>>>> >>>>>>> And that should be okay, mmap does the sync, and if no devices are >>>>>>> attached nothing could have changed the underlying memory in the >>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>>>> >>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. >>>>> Liam was saying that it's too painful for them to do that every time a >>>>> device unmaps - when in many cases (device->device, no CPU) it's not >>>>> needed. >>>> >>>> Invalidates are painless, at least compared to a real cache flush, just >>>> set the invalid bit vs actually writing out lines. I thought the issue >>>> was on the map side. >>>> >>> >>> Invalidates aren't painless for us because we have a coherent system cache >>> so clean lines get written out. >> >> That seems very broken, why would clean lines ever need to be written >> out, that defeats the whole point of having the invalidate separate from >> clean. How do you deal with stale cache lines? I guess in your case this >> is what forces you to have to use uncached memory for DMA-able memory. >> > > My understanding is that our ARM invalidate is a clean + invalidate, I had > concerns about the clean lines being written to the the system cache as > part of the 'clean', but the following 'invalidate' would take care of > actually invalidating the lines (so nothign broken). > But i am probably wrong on this and it is probably smart enough not to the > writing of the clean lines. > You are correct that for a lot of ARM cores "invalidate" is always a "clean + invalidate". At first I thought this was kinda silly as there is now no way to mark a dirty line invalid without it getting written out first, but if you think about it any dirty cache-line can be written out (cleaned) at anytime anyway, so this doesn't actually change system behavior. You should just not write to memory (make the line dirty) anything you don't want eventually written out. Point two, it's not just smart enough to not write-out clean lines, it is guaranteed not to write them out by the spec. Otherwise since cache-lines can be randomly filled if those same clean lines got written out on invalidate operations there would be no way to maintain coherency and things would be written over top each other all over the place. > But regardless, targets supporting a coherent system cache is a legitamate > configuration and an invalidate on this configuration does have to go to > the bus to invalidate the system cache (which isn't free) so I dont' think > you can make the assumption that invalidates are cheap so that it is okay > to do them (even if they are not needed) on every dma unmap. > Very true, CMOs need to be broadcast to other coherent masters on a coherent interconnect (and the interconnect itself if it has a cache as well (L3)), so not 100% free, but almost, just the infinitesimal cost of the cache tag check in hardware. If there are no non-coherent devices attached then the CMOs are no-ops, if there are then the data needs to be written out either way, doing it every access like is done with uncached memory (- any write combining) will blow away any saving made from the one less CMO. Either way you lose with uncached mappings of memory. If I'm wrong I would love to know. >>> And these invalidates can occur on fairly large buffers. >>> >>> That is why we haven't went with using cached ION memory and "tracking CPU >>> access" because it only solves half the problem, ie there isn't a way to >>> safely skip the invalidate (because we can't read the future). >>> Our solution was to go with uncached ION memory (when possible), but as >>> you can see in other discussions upstream support for uncached memory has >>> its own issues. >>> >> >> Sounds like you need to fix upstream support then, finding a way to drop >> all cacheable mappings of memory you want to make uncached mappings for >> seems to be the only solution. >> > > I think we can probably agree that there woudln't be a good way to remove > cached mappings without causing an unacceptable performance degradation > since it would fragment all the nice 1GB kernel mappings we have. > > So I am trying to find an alternative solution. > I'm not sure there is a better solution. How hard is this solution to implement anyway? The kernel already has to make gaps and cut up that nice 1GB mapping when you make a reserved memory space in the lowmem area, so all the logic is probably already implemented. Just need to allow it to be hooked into from Ion when doing doing the uncached mappings. >>>>> >>>>>>> >>>>>>>> [CPU reads/writes to the buffer] >>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>>>>> devices attached to buffer >>>>>>>> munmap buffer >>>>>>>> >>>>>>>> //buffer is send down the pipeline >>>>>>>> // Buffer is send to video device (who does compression of raw data) and >>>>>>>> writes to a file >>>>>>>> dma_buf_attach >>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>>> [video device writes to buffer] >>>>>>>> dma_buf_unmap_attachment >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>>>> the pipeline and Video doesn't know the end of the use case) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>>>>>>>> access then there is no requirement (that I am aware of) for you to call >>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>>>>> >>>>>>>>> >>>>>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>>>>> maintenance on the buffer? >>>>>>>>> >>>>>>>> >>>>>>>> Because ION no longer provides DMA ready memory. >>>>>>>> Take the above example. >>>>>>>> >>>>>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>>>>> Zeros are written to the cache. >>>>>>>> >>>>>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>>>>> The camera devices writes directly to the buffer in DDR. >>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >>>>>>>> evicted from the cache, this zero overwrites data the camera device has >>>>>>>> written which corrupts your data. >>>>>>>> >>>>>>> >>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>>>>> for CPU access at the time of zeroing. >>>>>>> >>>>> >>>>> Actually that should be at the point of the first non-coherent device >>>>> mapping the buffer right? No point in doing CMO if the future accesses >>>>> are coherent. >>>> >>>> I see your point, as long as the zeroing is guaranteed to be the first >>>> access to this buffer then it should be safe. >>>> >>>> Andrew >>>> >>>>> >>>>> Cheers, >>>>> -Brian >>>>> >>>>>>> Andrew >>>>>>> >>>>>>>> Liam >>>>>>>> >>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>>>>> a Linux Foundation Collaborative Project >>>>>>>> >>>> >>> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>> a Linux Foundation Collaborative Project >>> >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On 1/17/19 8:13 AM, Andrew F. Davis wrote: > On 1/16/19 4:48 PM, Liam Mark wrote: >> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >> >>> On 1/15/19 1:05 PM, Laura Abbott wrote: >>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>> >>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>> >>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>>> here. >>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>>> anyway. >>>>>>>>> >>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>> --- >>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>> >>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct >>>>>>>>> dma_buf_attachment *attachment, >>>>>>>>> table = a->table; >>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>> - direction)) >>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>> >>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>>> maintenance. >>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>>> calls >>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>> >>>>>>> >>>>>>> That should be okay though, if you have no attachments (or all >>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>> is attached later after data has already been written. Does that >>>>>>> sequence need supporting? >>>>>> >>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>> in Android, but I will focus on later for now. >>>>>> >>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>> attached and mapped, right? >>>>>>> >>>>>> >>>>>> Here is an example where CPU access can happen later in Android. >>>>>> >>>>>> Camera device records video -> software post processing -> video device >>>>>> (who does compression of raw data) and writes to a file >>>>>> >>>>>> In this example assume the buffer is cached and the devices are not >>>>>> IO-coherent (quite common). >>>>>> >>>>> >>>>> This is the start of the problem, having cached mappings of memory that >>>>> is also being accessed non-coherently is going to cause issues one way >>>>> or another. On top of the speculative cache fills that have to be >>>>> constantly fought back against with CMOs like below; some coherent >>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>> (snoop filters get messed up). >>>>> >>>>> The solution is to either always have the addresses marked non-coherent >>>>> (like device memory, no-map carveouts), or if you really want to use >>>>> regular system memory allocated at runtime, then all cached mappings of >>>>> it need to be dropped, even the kernel logical address (area as painful >>>>> as that would be). >>>>> >>>> >>>> I agree it's broken, hence my desire to remove it :) >>>> >>>> The other problem is that uncached buffers are being used for >>>> performance reason so anything that would involve getting >>>> rid of the logical address would probably negate any performance >>>> benefit. >>>> >>> >>> I wouldn't go as far as to remove them just yet.. Liam seems pretty >>> adamant that they have valid uses. I'm just not sure performance is one >>> of them, maybe in the case of software locks between devices or >>> something where there needs to be a lot of back and forth interleaved >>> access on small amounts of data? >>> >> >> I wasn't aware that ARM considered this not supported, I thought it was >> supported but they advised against it because of the potential performance >> impact. >> > > Not sure what you mean by "this" being not supported, do you mean mixed > attribute mappings? If so, it will certainly cause problems, and the > problems will change from platform to platform, avoid at all costs is my > understanding of ARM's position. > >> This is after all supported in the DMA APIs and up until now devices have >> been successfully commercializing with this configurations, and I think >> they will continue to commercialize with these configurations for quite a >> while. >> > > Use of uncached memory mappings are almost always wrong in my experience > and are used to work around some bug or because the user doesn't want to > implement proper CMOs. Counter examples welcome. > >> It would be really unfortunate if support was removed as I think that >> would drive clients away from using upstream ION. >> > > I'm not petitioning to remove support, but at very least lets reverse > the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by > default, to get uncached you should need to add a flag to your > allocation command pointing out you know what you are doing. > I thought about doing that, the problem is it becomes an ABI break for existing users which I really didn't want to do again. If it ends up being the last thing we do before moving out of staging, I'd consider doing it. >>>>>> ION buffer is allocated. >>>>>> >>>>>> //Camera device records video >>>>>> dma_buf_attach >>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>> >>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>> problem. >>>>> >>>>>> [camera device writes to buffer] >>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>> >>>>> It doesn't know there will be any further CPU access, it could get freed >>>>> after this for all we know, the invalidate can be saved until the CPU >>>>> requests access again. >>>>> >>>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>>> down >>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>> >>>>> >>>>> This seems like a broken use-case, I understand the desire to keep >>>>> everything as modular as possible and separate the steps, but at this >>>>> point no one owns this buffers backing memory, not the CPU or any >>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>> for the next attachment, which may change the required backing memory >>>>> completely. >>>>> >>>>> All devices should attach before the first mapping, and only let go >>>>> after the task is complete, otherwise this buffers data needs copied off >>>>> to a different location or the CPU needs to take ownership in-between. >>>>> >>>> >>>> Maybe it's broken but it's the status quo and we spent a good >>>> amount of time at plumbers concluding there isn't a great way >>>> to fix it :/ >>>> >>> >>> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ >>> >>> Perhaps just stronger rules on sequencing of operations? I'm not saying >>> I have a good solution either, I just don't see any way forward without >>> some use-case getting broken, so better to fix now over later. >>> >> >> I can see the benefits of Android doing things the way they do, I would >> request that changes we make continue to support Android, or we find a way >> to convice them to change, as they are the main ION client and I assume >> other ION clients in the future will want to do this as well. >> > > Android may be the biggest user today (makes sense, Ion come out of the > Android project), but that can change, and getting changes into Android > will be easier that the upstream kernel once Ion is out of staging. > > Unlike some other big ARM vendors, we (TI) do not primarily build mobile > chips targeting Android, our core offerings target more traditional > Linux userspaces, and I'm guessing others will start to do the same as > ARM tries to push more into desktop, server, and other spaces again. > >> I am concerned that if you go with a solution which enforces what you >> mention above, and bring ION out of staging that way, it will make it that >> much harder to solve this for Android and therefore harder to get >> Android clients to move to the upstream ION (and get everybody off their >> vendor modified Android versions). >> > > That would be an Android problem, reducing functionality in upstream to > match what some evil vendor trees do to support Android is not the way > forward on this. At least for us we are going to try to make all our > software offerings follow proper buffer ownership (including our Android > offering). > I don't think this is reducing functionality, it's about not breaking what already works. There is some level of Android testing on a mainline tree (hikey boards). I would say if we can come to an agreement on a correct API, we could always merge the 'correct' version out of staging and keep a legacy driver around for some time as a transition. Thanks, Laura
On 1/18/19 2:31 PM, Laura Abbott wrote: > On 1/17/19 8:13 AM, Andrew F. Davis wrote: >> On 1/16/19 4:48 PM, Liam Mark wrote: >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/15/19 1:05 PM, Laura Abbott wrote: >>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>> >>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>> >>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>>>> here. >>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>>>> anyway. >>>>>>>>>> >>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>> --- >>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>> >>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table >>>>>>>>>> *ion_map_dma_buf(struct >>>>>>>>>> dma_buf_attachment *attachment, >>>>>>>>>> table = a->table; >>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, >>>>>>>>>> table->nents, >>>>>>>>>> - direction)) >>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, >>>>>>>>>> table->nents, >>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>> >>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>>>> maintenance. >>>>>>>>> If the calls to {begin,end}_cpu_access were made before the >>>>>>>>> call to >>>>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>>>> calls >>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>> >>>>>>>> >>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent >>>>>>>> device >>>>>>>> is attached later after data has already been written. Does that >>>>>>>> sequence need supporting? >>>>>>> >>>>>>> Yes, but also I think there are cases where CPU access can happen >>>>>>> before >>>>>>> in Android, but I will focus on later for now. >>>>>>> >>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>> memory until map_dma_buf() time, and that should only happen >>>>>>>> after all >>>>>>>> the devices have attached so it can know where to put the >>>>>>>> buffer. So we >>>>>>>> shouldn't expect any CPU access to buffers before all the >>>>>>>> devices are >>>>>>>> attached and mapped, right? >>>>>>>> >>>>>>> >>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>> >>>>>>> Camera device records video -> software post processing -> video >>>>>>> device >>>>>>> (who does compression of raw data) and writes to a file >>>>>>> >>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>> IO-coherent (quite common). >>>>>>> >>>>>> >>>>>> This is the start of the problem, having cached mappings of memory >>>>>> that >>>>>> is also being accessed non-coherently is going to cause issues one >>>>>> way >>>>>> or another. On top of the speculative cache fills that have to be >>>>>> constantly fought back against with CMOs like below; some coherent >>>>>> interconnects behave badly when you mix coherent and non-coherent >>>>>> access >>>>>> (snoop filters get messed up). >>>>>> >>>>>> The solution is to either always have the addresses marked >>>>>> non-coherent >>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>> regular system memory allocated at runtime, then all cached >>>>>> mappings of >>>>>> it need to be dropped, even the kernel logical address (area as >>>>>> painful >>>>>> as that would be). >>>>>> >>>>> >>>>> I agree it's broken, hence my desire to remove it :) >>>>> >>>>> The other problem is that uncached buffers are being used for >>>>> performance reason so anything that would involve getting >>>>> rid of the logical address would probably negate any performance >>>>> benefit. >>>>> >>>> >>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty >>>> adamant that they have valid uses. I'm just not sure performance is one >>>> of them, maybe in the case of software locks between devices or >>>> something where there needs to be a lot of back and forth interleaved >>>> access on small amounts of data? >>>> >>> >>> I wasn't aware that ARM considered this not supported, I thought it was >>> supported but they advised against it because of the potential >>> performance >>> impact. >>> >> >> Not sure what you mean by "this" being not supported, do you mean mixed >> attribute mappings? If so, it will certainly cause problems, and the >> problems will change from platform to platform, avoid at all costs is my >> understanding of ARM's position. >> >>> This is after all supported in the DMA APIs and up until now devices >>> have >>> been successfully commercializing with this configurations, and I think >>> they will continue to commercialize with these configurations for >>> quite a >>> while. >>> >> >> Use of uncached memory mappings are almost always wrong in my experience >> and are used to work around some bug or because the user doesn't want to >> implement proper CMOs. Counter examples welcome. >> >>> It would be really unfortunate if support was removed as I think that >>> would drive clients away from using upstream ION. >>> >> >> I'm not petitioning to remove support, but at very least lets reverse >> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by >> default, to get uncached you should need to add a flag to your >> allocation command pointing out you know what you are doing. >> > > I thought about doing that, the problem is it becomes an ABI break for > existing users which I really didn't want to do again. If it > ends up being the last thing we do before moving out of staging, > I'd consider doing it. > >>>>>>> ION buffer is allocated. >>>>>>> >>>>>>> //Camera device records video >>>>>>> dma_buf_attach >>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>> >>>>>> Why does the buffer need to be cleaned here? I just got through >>>>>> reading >>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>> suggestion of tracking if the buffer has had CPU access since the >>>>>> last >>>>>> time and only flushing the cache if it has. As unmapped heaps >>>>>> never get >>>>>> CPU mapped this would never be the case for unmapped heaps, it >>>>>> solves my >>>>>> problem. >>>>>> >>>>>>> [camera device writes to buffer] >>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>> >>>>>> It doesn't know there will be any further CPU access, it could get >>>>>> freed >>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>> requests access again. >>>>>> >>>>>>> dma_buf_detach (device cannot stay attached because it is being >>>>>>> sent >>>>>>> down >>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>> >>>>>> >>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>> everything as modular as possible and separate the steps, but at this >>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>> de-allocate the backing storage if it wants, that way it could get >>>>>> ready >>>>>> for the next attachment, which may change the required backing memory >>>>>> completely. >>>>>> >>>>>> All devices should attach before the first mapping, and only let go >>>>>> after the task is complete, otherwise this buffers data needs >>>>>> copied off >>>>>> to a different location or the CPU needs to take ownership >>>>>> in-between. >>>>>> >>>>> >>>>> Maybe it's broken but it's the status quo and we spent a good >>>>> amount of time at plumbers concluding there isn't a great way >>>>> to fix it :/ >>>>> >>>> >>>> Hmm, guess that doesn't prove there is not a great way to fix it >>>> either.. :/ >>>> >>>> Perhaps just stronger rules on sequencing of operations? I'm not saying >>>> I have a good solution either, I just don't see any way forward without >>>> some use-case getting broken, so better to fix now over later. >>>> >>> >>> I can see the benefits of Android doing things the way they do, I would >>> request that changes we make continue to support Android, or we find >>> a way >>> to convice them to change, as they are the main ION client and I assume >>> other ION clients in the future will want to do this as well. >>> >> >> Android may be the biggest user today (makes sense, Ion come out of the >> Android project), but that can change, and getting changes into Android >> will be easier that the upstream kernel once Ion is out of staging. >> >> Unlike some other big ARM vendors, we (TI) do not primarily build mobile >> chips targeting Android, our core offerings target more traditional >> Linux userspaces, and I'm guessing others will start to do the same as >> ARM tries to push more into desktop, server, and other spaces again. >> >>> I am concerned that if you go with a solution which enforces what you >>> mention above, and bring ION out of staging that way, it will make it >>> that >>> much harder to solve this for Android and therefore harder to get >>> Android clients to move to the upstream ION (and get everybody off their >>> vendor modified Android versions). >>> >> >> That would be an Android problem, reducing functionality in upstream to >> match what some evil vendor trees do to support Android is not the way >> forward on this. At least for us we are going to try to make all our >> software offerings follow proper buffer ownership (including our Android >> offering). >> > > I don't think this is reducing functionality, it's about not breaking > what already works. There is some level of Android testing on a mainline > tree (hikey boards). I would say if we can come to an agreement on > a correct API, we could always merge the 'correct' version out of > staging and keep a legacy driver around for some time as a transition. > I'm not sure that is what staging should be for, but I can certainly see why you would want that (I help maintain our Android offering and every kernel migration I get to go fixup libion and all its users..). I'm sure we all know the API will get broken to get this out of staging, so maybe we need to start a list (or update the TODO) with all the things we agree need changed during the last step before destaging. Sounds like you agree about the ION_FLAG_CACHED reversal for starters. I think direct heap managed dma_buf_ops will be needed. What's left, do we have any current proposals for the heap query floating around that can go up for review? Thanks, Andrew > Thanks, > Laura
On 1/18/19 12:43 PM, Andrew F. Davis wrote: > On 1/18/19 2:31 PM, Laura Abbott wrote: >> On 1/17/19 8:13 AM, Andrew F. Davis wrote: >>> On 1/16/19 4:48 PM, Liam Mark wrote: >>>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>>> >>>>> On 1/15/19 1:05 PM, Laura Abbott wrote: >>>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>>> >>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>>> >>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>>>>> here. >>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>>>>> anyway. >>>>>>>>>>> >>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>>> --- >>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>>> >>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table >>>>>>>>>>> *ion_map_dma_buf(struct >>>>>>>>>>> dma_buf_attachment *attachment, >>>>>>>>>>> table = a->table; >>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, >>>>>>>>>>> table->nents, >>>>>>>>>>> - direction)) >>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, >>>>>>>>>>> table->nents, >>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>>> >>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>>>>> maintenance. >>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the >>>>>>>>>> call to >>>>>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>>>>> calls >>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>>> >>>>>>>>> >>>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent >>>>>>>>> device >>>>>>>>> is attached later after data has already been written. Does that >>>>>>>>> sequence need supporting? >>>>>>>> >>>>>>>> Yes, but also I think there are cases where CPU access can happen >>>>>>>> before >>>>>>>> in Android, but I will focus on later for now. >>>>>>>> >>>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>>> memory until map_dma_buf() time, and that should only happen >>>>>>>>> after all >>>>>>>>> the devices have attached so it can know where to put the >>>>>>>>> buffer. So we >>>>>>>>> shouldn't expect any CPU access to buffers before all the >>>>>>>>> devices are >>>>>>>>> attached and mapped, right? >>>>>>>>> >>>>>>>> >>>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>>> >>>>>>>> Camera device records video -> software post processing -> video >>>>>>>> device >>>>>>>> (who does compression of raw data) and writes to a file >>>>>>>> >>>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>>> IO-coherent (quite common). >>>>>>>> >>>>>>> >>>>>>> This is the start of the problem, having cached mappings of memory >>>>>>> that >>>>>>> is also being accessed non-coherently is going to cause issues one >>>>>>> way >>>>>>> or another. On top of the speculative cache fills that have to be >>>>>>> constantly fought back against with CMOs like below; some coherent >>>>>>> interconnects behave badly when you mix coherent and non-coherent >>>>>>> access >>>>>>> (snoop filters get messed up). >>>>>>> >>>>>>> The solution is to either always have the addresses marked >>>>>>> non-coherent >>>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>>> regular system memory allocated at runtime, then all cached >>>>>>> mappings of >>>>>>> it need to be dropped, even the kernel logical address (area as >>>>>>> painful >>>>>>> as that would be). >>>>>>> >>>>>> >>>>>> I agree it's broken, hence my desire to remove it :) >>>>>> >>>>>> The other problem is that uncached buffers are being used for >>>>>> performance reason so anything that would involve getting >>>>>> rid of the logical address would probably negate any performance >>>>>> benefit. >>>>>> >>>>> >>>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty >>>>> adamant that they have valid uses. I'm just not sure performance is one >>>>> of them, maybe in the case of software locks between devices or >>>>> something where there needs to be a lot of back and forth interleaved >>>>> access on small amounts of data? >>>>> >>>> >>>> I wasn't aware that ARM considered this not supported, I thought it was >>>> supported but they advised against it because of the potential >>>> performance >>>> impact. >>>> >>> >>> Not sure what you mean by "this" being not supported, do you mean mixed >>> attribute mappings? If so, it will certainly cause problems, and the >>> problems will change from platform to platform, avoid at all costs is my >>> understanding of ARM's position. >>> >>>> This is after all supported in the DMA APIs and up until now devices >>>> have >>>> been successfully commercializing with this configurations, and I think >>>> they will continue to commercialize with these configurations for >>>> quite a >>>> while. >>>> >>> >>> Use of uncached memory mappings are almost always wrong in my experience >>> and are used to work around some bug or because the user doesn't want to >>> implement proper CMOs. Counter examples welcome. >>> >>>> It would be really unfortunate if support was removed as I think that >>>> would drive clients away from using upstream ION. >>>> >>> >>> I'm not petitioning to remove support, but at very least lets reverse >>> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by >>> default, to get uncached you should need to add a flag to your >>> allocation command pointing out you know what you are doing. >>> >> >> I thought about doing that, the problem is it becomes an ABI break for >> existing users which I really didn't want to do again. If it >> ends up being the last thing we do before moving out of staging, >> I'd consider doing it. >> >>>>>>>> ION buffer is allocated. >>>>>>>> >>>>>>>> //Camera device records video >>>>>>>> dma_buf_attach >>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>> >>>>>>> Why does the buffer need to be cleaned here? I just got through >>>>>>> reading >>>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>>> suggestion of tracking if the buffer has had CPU access since the >>>>>>> last >>>>>>> time and only flushing the cache if it has. As unmapped heaps >>>>>>> never get >>>>>>> CPU mapped this would never be the case for unmapped heaps, it >>>>>>> solves my >>>>>>> problem. >>>>>>> >>>>>>>> [camera device writes to buffer] >>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>>> >>>>>>> It doesn't know there will be any further CPU access, it could get >>>>>>> freed >>>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>>> requests access again. >>>>>>> >>>>>>>> dma_buf_detach (device cannot stay attached because it is being >>>>>>>> sent >>>>>>>> down >>>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>>> >>>>>>> >>>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>>> everything as modular as possible and separate the steps, but at this >>>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>>> de-allocate the backing storage if it wants, that way it could get >>>>>>> ready >>>>>>> for the next attachment, which may change the required backing memory >>>>>>> completely. >>>>>>> >>>>>>> All devices should attach before the first mapping, and only let go >>>>>>> after the task is complete, otherwise this buffers data needs >>>>>>> copied off >>>>>>> to a different location or the CPU needs to take ownership >>>>>>> in-between. >>>>>>> >>>>>> >>>>>> Maybe it's broken but it's the status quo and we spent a good >>>>>> amount of time at plumbers concluding there isn't a great way >>>>>> to fix it :/ >>>>>> >>>>> >>>>> Hmm, guess that doesn't prove there is not a great way to fix it >>>>> either.. :/ >>>>> >>>>> Perhaps just stronger rules on sequencing of operations? I'm not saying >>>>> I have a good solution either, I just don't see any way forward without >>>>> some use-case getting broken, so better to fix now over later. >>>>> >>>> >>>> I can see the benefits of Android doing things the way they do, I would >>>> request that changes we make continue to support Android, or we find >>>> a way >>>> to convice them to change, as they are the main ION client and I assume >>>> other ION clients in the future will want to do this as well. >>>> >>> >>> Android may be the biggest user today (makes sense, Ion come out of the >>> Android project), but that can change, and getting changes into Android >>> will be easier that the upstream kernel once Ion is out of staging. >>> >>> Unlike some other big ARM vendors, we (TI) do not primarily build mobile >>> chips targeting Android, our core offerings target more traditional >>> Linux userspaces, and I'm guessing others will start to do the same as >>> ARM tries to push more into desktop, server, and other spaces again. >>> >>>> I am concerned that if you go with a solution which enforces what you >>>> mention above, and bring ION out of staging that way, it will make it >>>> that >>>> much harder to solve this for Android and therefore harder to get >>>> Android clients to move to the upstream ION (and get everybody off their >>>> vendor modified Android versions). >>>> >>> >>> That would be an Android problem, reducing functionality in upstream to >>> match what some evil vendor trees do to support Android is not the way >>> forward on this. At least for us we are going to try to make all our >>> software offerings follow proper buffer ownership (including our Android >>> offering). >>> >> >> I don't think this is reducing functionality, it's about not breaking >> what already works. There is some level of Android testing on a mainline >> tree (hikey boards). I would say if we can come to an agreement on >> a correct API, we could always merge the 'correct' version out of >> staging and keep a legacy driver around for some time as a transition. >> > > I'm not sure that is what staging should be for, but I can certainly see > why you would want that (I help maintain our Android offering and every > kernel migration I get to go fixup libion and all its users..). > > I'm sure we all know the API will get broken to get this out of staging, > so maybe we need to start a list (or update the TODO) with all the > things we agree need changed during the last step before destaging. > Sounds like you agree about the ION_FLAG_CACHED reversal for starters. I > think direct heap managed dma_buf_ops will be needed. > > What's left, do we have any current proposals for the heap query > floating around that can go up for review? > I was hoping the last time I broke the API would be the last time we would need to break it again. The query ioctl is already merged and I haven't seen any other counter proposals around for discussion. The TODO list could probably use some updating though. > Thanks, > Andrew > >> Thanks, >> Laura
On Fri, 18 Jan 2019, Andrew F. Davis wrote: > On 1/17/19 7:04 PM, Liam Mark wrote: > > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/16/19 4:48 PM, Liam Mark wrote: > >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >>> > >>>> On 1/15/19 1:05 PM, Laura Abbott wrote: > >>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: > >>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>>>> > >>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>>>> > >>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance > >>>>>>>>>> here. > >>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed > >>>>>>>>>> anyway. > >>>>>>>>>> > >>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>>>> --- > >>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>>>> > >>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c > >>>>>>>>>> b/drivers/staging/android/ion/ion.c > >>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct > >>>>>>>>>> dma_buf_attachment *attachment, > >>>>>>>>>> table = a->table; > >>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>>>> - direction)) > >>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>>>> > >>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache > >>>>>>>>> maintenance. > >>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>>>> dma_buf_attach then there won't have been a device attached so the > >>>>>>>>> calls > >>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>>>> > >>>>>>>> > >>>>>>>> That should be okay though, if you have no attachments (or all > >>>>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>>>> is attached later after data has already been written. Does that > >>>>>>>> sequence need supporting? > >>>>>>> > >>>>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>>>> in Android, but I will focus on later for now. > >>>>>>> > >>>>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>>>> attached and mapped, right? > >>>>>>>> > >>>>>>> > >>>>>>> Here is an example where CPU access can happen later in Android. > >>>>>>> > >>>>>>> Camera device records video -> software post processing -> video device > >>>>>>> (who does compression of raw data) and writes to a file > >>>>>>> > >>>>>>> In this example assume the buffer is cached and the devices are not > >>>>>>> IO-coherent (quite common). > >>>>>>> > >>>>>> > >>>>>> This is the start of the problem, having cached mappings of memory that > >>>>>> is also being accessed non-coherently is going to cause issues one way > >>>>>> or another. On top of the speculative cache fills that have to be > >>>>>> constantly fought back against with CMOs like below; some coherent > >>>>>> interconnects behave badly when you mix coherent and non-coherent access > >>>>>> (snoop filters get messed up). > >>>>>> > >>>>>> The solution is to either always have the addresses marked non-coherent > >>>>>> (like device memory, no-map carveouts), or if you really want to use > >>>>>> regular system memory allocated at runtime, then all cached mappings of > >>>>>> it need to be dropped, even the kernel logical address (area as painful > >>>>>> as that would be). > >>>>>> > >>>>> > >>>>> I agree it's broken, hence my desire to remove it :) > >>>>> > >>>>> The other problem is that uncached buffers are being used for > >>>>> performance reason so anything that would involve getting > >>>>> rid of the logical address would probably negate any performance > >>>>> benefit. > >>>>> > >>>> > >>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty > >>>> adamant that they have valid uses. I'm just not sure performance is one > >>>> of them, maybe in the case of software locks between devices or > >>>> something where there needs to be a lot of back and forth interleaved > >>>> access on small amounts of data? > >>>> > >>> > >>> I wasn't aware that ARM considered this not supported, I thought it was > >>> supported but they advised against it because of the potential performance > >>> impact. > >>> > >> > >> Not sure what you mean by "this" being not supported, do you mean mixed > >> attribute mappings? If so, it will certainly cause problems, and the > >> problems will change from platform to platform, avoid at all costs is my > >> understanding of ARM's position. > >> > >>> This is after all supported in the DMA APIs and up until now devices have > >>> been successfully commercializing with this configurations, and I think > >>> they will continue to commercialize with these configurations for quite a > >>> while. > >>> > >> > >> Use of uncached memory mappings are almost always wrong in my experience > >> and are used to work around some bug or because the user doesn't want to > >> implement proper CMOs. Counter examples welcome. > >> > > > > Okay, let me first try to clarify what I am referring to, as perhaps I am > > misunderstanding the conversation. > > > > In this discussion I was originally referring to a use case with cached > > memory being accessed by a non io-cohernet device. > > > > "In this example assume the buffer is cached and the devices are not > > IO-coherent (quite common)." > > > > to which you did not think was supported: > > > > "This is the start of the problem, having cached mappings of memory > > that is also being accessed non-coherently is going to cause issues > > one way or another. > > " > > > > And I interpreted Laura's comment below as saying she wanted to remove > > support in ION for cached memory being accessed by non io-cohernet > > devices: > > "I agree it's broken, hence my desire to remove it :)" > > > > So assuming my understanding above is correct (and you are not talking > > about something separate such as removing uncached ION allocation > > support). > > > > Ah, I think here is where we diverged, I'm assuming Laura's comment to > be referencing my issue with uncached mappings being handed out without > first removing all cached mappings of the same memory. Therefore it is > uncached heaps that are broken. > I am glad that is clarified, but can you then clarify for me your following statement, I am stil not clear what the problem is then. In response to: "In this example assume the buffer is cached and the devices are not IO-coherent (quite common)." You said: "This is the start of the problem, having cached mappings of memory that is also being accessed non-coherently is going to cause issues one way or another. " > > Then I guess I am not clear why current uses which use cached memory with > > non IO-coherent devices are considered to be working around some bug or > > are not implementing proper CMOs. > > > > They use CPU cached mappings because that is the most effective way to > > access the memory from the CPU side and the devices have an uncached > > IOMMU mapping because they don't support IO-coherency, and currenlty in > > the CPU they do cache mainteance at the time of dma map and dma umap so > > to me they are implementing correct CMOs. > > > > Fully agree here, using cached mappings and performing CMOs when needed > is the way to go when dealing with memory. IMHO the *only* time when > uncached mappings are appropriate is for memory mapped I/O (although it > looks like video memory was often treated as uncached (wc)). > > >>> It would be really unfortunate if support was removed as I think that > >>> would drive clients away from using upstream ION. > >>> > >> > >> I'm not petitioning to remove support, but at very least lets reverse > >> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by > >> default, to get uncached you should need to add a flag to your > >> allocation command pointing out you know what you are doing. > >> > > > > You may not be petitioning to remove support for using cached memory with > > non io-coherent devices but I interpreted Laura's comment as wanting to do > > so, and I had concerns about that. > > > > What I would like is for the default memory handed out by Ion to be > normal cacheable memory, just like is always handed out to users-space. > DMA-BUF already provides the means to deal with the CMOs required to > work with non-io-coherent devices so all should be good here. > > If you want Ion to give out uncached memory then I think you should need > to explicitly state so with an allocation flag. And right now the > uncached memory you will get back may have other cached mappings (kernel > lowmem mappings) meaning you will have hard to predict results (on ARM > at least). Yes, I can understand why it would make sense to default to cached memory. > I just don't see much use for them (uncached mappings of > regular memory) right now. > I can understand why people don't like ION providing uncached support, but I have been pushing to keep un-cached support in order to keep the performance of Android ION clients on par with the previous version of ION until a better solution can be found (either by changing ION or Android). Basically most ION use cases don't involve any (or very little) CPU access so for now by using uncached ION allocations we can avoid the uncessary cache maitenance and we are safe if some CPU access is required. Of course for use cases involving a lot of CPU access the clients switch over to using cached buffers. > >>>>>>> ION buffer is allocated. > >>>>>>> > >>>>>>> //Camera device records video > >>>>>>> dma_buf_attach > >>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>> > >>>>>> Why does the buffer need to be cleaned here? I just got through reading > >>>>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>>>> suggestion of tracking if the buffer has had CPU access since the last > >>>>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>>>> problem. > >>>>>> > >>>>>>> [camera device writes to buffer] > >>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>>>> > >>>>>> It doesn't know there will be any further CPU access, it could get freed > >>>>>> after this for all we know, the invalidate can be saved until the CPU > >>>>>> requests access again. > >>>>>> > >>>>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>>>> down > >>>>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>>>> > >>>>>> > >>>>>> This seems like a broken use-case, I understand the desire to keep > >>>>>> everything as modular as possible and separate the steps, but at this > >>>>>> point no one owns this buffers backing memory, not the CPU or any > >>>>>> device. I would go as far as to say DMA-BUF should be free now to > >>>>>> de-allocate the backing storage if it wants, that way it could get ready > >>>>>> for the next attachment, which may change the required backing memory > >>>>>> completely. > >>>>>> > >>>>>> All devices should attach before the first mapping, and only let go > >>>>>> after the task is complete, otherwise this buffers data needs copied off > >>>>>> to a different location or the CPU needs to take ownership in-between. > >>>>>> > >>>>> > >>>>> Maybe it's broken but it's the status quo and we spent a good > >>>>> amount of time at plumbers concluding there isn't a great way > >>>>> to fix it :/ > >>>>> > >>>> > >>>> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ > >>>> > >>>> Perhaps just stronger rules on sequencing of operations? I'm not saying > >>>> I have a good solution either, I just don't see any way forward without > >>>> some use-case getting broken, so better to fix now over later. > >>>> > >>> > >>> I can see the benefits of Android doing things the way they do, I would > >>> request that changes we make continue to support Android, or we find a way > >>> to convice them to change, as they are the main ION client and I assume > >>> other ION clients in the future will want to do this as well. > >>> > >> > >> Android may be the biggest user today (makes sense, Ion come out of the > >> Android project), but that can change, and getting changes into Android > >> will be easier that the upstream kernel once Ion is out of staging. > >> > >> Unlike some other big ARM vendors, we (TI) do not primarily build mobile > >> chips targeting Android, our core offerings target more traditional > >> Linux userspaces, and I'm guessing others will start to do the same as > >> ARM tries to push more into desktop, server, and other spaces again. > >> > >>> I am concerned that if you go with a solution which enforces what you > >>> mention above, and bring ION out of staging that way, it will make it that > >>> much harder to solve this for Android and therefore harder to get > >>> Android clients to move to the upstream ION (and get everybody off their > >>> vendor modified Android versions). > >>> > >> > >> That would be an Android problem, reducing functionality in upstream to > >> match what some evil vendor trees do to support Android is not the way > >> forward on this. At least for us we are going to try to make all our > >> software offerings follow proper buffer ownership (including our Android > >> offering). > >> > >>>>>>> //buffer is send down the pipeline > >>>>>>> > >>>>>>> // Usersapce software post processing occurs > >>>>>>> mmap buffer > >>>>>> > >>>>>> Perhaps the invalidate should happen here in mmap. > >>>>>> > >>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>>>> devices attached to buffer > >>>>>> > >>>>>> And that should be okay, mmap does the sync, and if no devices are > >>>>>> attached nothing could have changed the underlying memory in the > >>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>>>>> > >>>>>>> [CPU reads/writes to the buffer] > >>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>>>> devices attached to buffer > >>>>>>> munmap buffer > >>>>>>> > >>>>>>> //buffer is send down the pipeline > >>>>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>>>> writes to a file > >>>>>>> dma_buf_attach > >>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>> [video device writes to buffer] > >>>>>>> dma_buf_unmap_attachment > >>>>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>>>> down > >>>>>>> the pipeline and Video doesn't know the end of the use case) > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not > >>>>>>>>> doing CPU > >>>>>>>>> access then there is no requirement (that I am aware of) for you to > >>>>>>>>> call > >>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and > >>>>>>>>> if this > >>>>>>>>> buffer is cached and your device is not IO-coherent then the cache > >>>>>>>>> maintenance > >>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>>>> > >>>>>>>> > >>>>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>>>> maintenance on the buffer? > >>>>>>>> > >>>>>>> > >>>>>>> Because ION no longer provides DMA ready memory. > >>>>>>> Take the above example. > >>>>>>> > >>>>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>>>> Zeros are written to the cache. > >>>>>>> > >>>>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>>>> The camera devices writes directly to the buffer in DDR. > >>>>>>> Since you didn't clean the buffer a dirty cache line (one of the > >>>>>>> zeros) is > >>>>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>>>> written which corrupts your data. > >>>>>>> > >>>>>> > >>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>>>> for CPU access at the time of zeroing. > >>>>>> > >>>>>> Andrew > >>>>>> > >>>>>>> Liam > >>>>>>> > >>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>>>> a Linux Foundation Collaborative Project > >>>>>>> > >>>>> > >>>> > >>> > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>> a Linux Foundation Collaborative Project > >>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Hi, Sorry for being a bit sporadic on this. I was out travelling last week with little time for email. On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote: > On 1/17/19 7:11 PM, Liam Mark wrote: > > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/16/19 4:54 PM, Liam Mark wrote: > >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >>> > >>>> On 1/16/19 9:19 AM, Brian Starkey wrote: > >>>>> Hi :-) > >>>>> > >>>>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > >>>>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: > >>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>>>>> > >>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>>>>> > >>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > >>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >>>>>>>>>>> > >>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>>>>> --- > >>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>>>>> > >>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >>>>>>>>>>> > >>>>>>>>>>> table = a->table; > >>>>>>>>>>> > >>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>> - direction)) > >>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>>>>> > >>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > >>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>>>>> dma_buf_attach then there won't have been a device attached so the calls > >>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> That should be okay though, if you have no attachments (or all > >>>>>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>>>>> is attached later after data has already been written. Does that > >>>>>>>>> sequence need supporting? > >>>>>>>> > >>>>>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>>>>> in Android, but I will focus on later for now. > >>>>>>>> > >>>>>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>>>>> attached and mapped, right? > >>>>>>>>> > >>>>>>>> > >>>>>>>> Here is an example where CPU access can happen later in Android. > >>>>>>>> > >>>>>>>> Camera device records video -> software post processing -> video device > >>>>>>>> (who does compression of raw data) and writes to a file > >>>>>>>> > >>>>>>>> In this example assume the buffer is cached and the devices are not > >>>>>>>> IO-coherent (quite common). > >>>>>>>> > >>>>>>> > >>>>>>> This is the start of the problem, having cached mappings of memory that > >>>>>>> is also being accessed non-coherently is going to cause issues one way > >>>>>>> or another. On top of the speculative cache fills that have to be > >>>>>>> constantly fought back against with CMOs like below; some coherent > >>>>>>> interconnects behave badly when you mix coherent and non-coherent access > >>>>>>> (snoop filters get messed up). > >>>>>>> > >>>>>>> The solution is to either always have the addresses marked non-coherent > >>>>>>> (like device memory, no-map carveouts), or if you really want to use > >>>>>>> regular system memory allocated at runtime, then all cached mappings of > >>>>>>> it need to be dropped, even the kernel logical address (area as painful > >>>>>>> as that would be). > >>>>> > >>>>> Ouch :-( I wasn't aware about these potential interconnect issues. How > >>>>> "real" is that? It seems that we aren't really hitting that today on > >>>>> real devices. > >>>>> > >>>> > >>>> Sadly there is at least one real device like this now (TI AM654). We > >>>> spent some time working with the ARM interconnect spec designers to see > >>>> if this was allowed behavior, final conclusion was mixing coherent and > >>>> non-coherent accesses is never a good idea.. So we have been working to > >>>> try to minimize any cases of mixed attributes [0], if a region is > >>>> coherent then everyone in the system needs to treat it as such and > >>>> vice-versa, even clever CMO that work on other systems wont save you > >>>> here. :( > >>>> > >>>> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 > >>>> "Never a good idea" - but I think it should still be well defined by the ARMv8 ARM (Section B2.8). Does this apply to your system? "If the mismatched attributes for a memory location all assign the same shareability attribute to a Location that has a cacheable attribute, any loss of uniprocessor semantics, ordering, or coherency within a shareability domain can be avoided by use of software cache management" https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile If the cache is invalidated when switching between access types, shouldn't the snoop filters get un-messed-up? > >>>> > >>>>>>> > >>>>>>>> ION buffer is allocated. > >>>>>>>> > >>>>>>>> //Camera device records video > >>>>>>>> dma_buf_attach > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>> > >>>>>>> Why does the buffer need to be cleaned here? I just got through reading > >>>>>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>>>> > >>>>>> Actually +Brian this time :) > >>>>>> > >>>>>>> suggestion of tracking if the buffer has had CPU access since the last > >>>>>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>>>>> problem. > >>>>>>> > >>>>>>>> [camera device writes to buffer] > >>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>>>>> > >>>>>>> It doesn't know there will be any further CPU access, it could get freed > >>>>>>> after this for all we know, the invalidate can be saved until the CPU > >>>>>>> requests access again. > >>>>> > >>>>> We don't have any API to allow the invalidate to happen on CPU access > >>>>> if all devices already detached. We need a struct device pointer to > >>>>> give to the DMA API, otherwise on arm64 there'll be no invalidate. > >>>>> > >>>>> I had a chat with a few people internally after the previous > >>>>> discussion with Liam. One suggestion was to use > >>>>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > >>>>> one other device attached (guarantees that we can do an invalidate in > >>>>> the future if begin_cpu_access is called). If the last device > >>>>> detaches, do a sync then. > >>>>> > >>>>> Conversely, in map_dma_buf, we would track if there was any CPU access > >>>>> and use/skip the sync appropriately. > >>>>> > >>>> > >>>> Now that I think this all through I agree this patch is probably wrong. > >>>> The real fix needs to be better handling in the dma_map_sg() to deal > >>>> with the case of the memory not being mapped (what I'm dealing with for > >>>> unmapped heaps), and for cases when the memory in question is not cached > >>>> (Liam's issue I think). For both these cases the dma_map_sg() does the > >>>> wrong thing. > >>>> > >>>>> I did start poking the code to check out how that would look, but then > >>>>> Christmas happened and I'm still catching back up. > >>>>> > >>>>>>> > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>>>>> > >>>>>>> > >>>>>>> This seems like a broken use-case, I understand the desire to keep > >>>>>>> everything as modular as possible and separate the steps, but at this > >>>>>>> point no one owns this buffers backing memory, not the CPU or any > >>>>>>> device. I would go as far as to say DMA-BUF should be free now to > >>>>>>> de-allocate the backing storage if it wants, that way it could get ready > >>>>>>> for the next attachment, which may change the required backing memory > >>>>>>> completely. > >>>>>>> > >>>>>>> All devices should attach before the first mapping, and only let go > >>>>>>> after the task is complete, otherwise this buffers data needs copied off > >>>>>>> to a different location or the CPU needs to take ownership in-between. > >>>>>>> > >>>>> > >>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF > >>>>> implementations which actually do that? I hear it quoted a lot, > >>>>> because that's what the docs say - but if the reality doesn't match > >>>>> it, maybe we should change the docs. > >>>>> > >>>> > >>>> Do you mean on the userspace side? I'm not sure, seems like Android > >>>> might be doing this wrong from what I can gather. From kernel side if > >>>> you mean the "de-allocate the backing storage", we will have some cases > >>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF > >>>> in ways not specified in the documentation. Changing the docs to force > >>>> the backing memory to always be allocated breaks the central goal in > >>>> having attach/map in DMA-BUF separate. Actually I meant in the kernel, in exporters. I haven't seen anyone using the API as it was intended (defer allocation until first map, migrate between different attachments, etc.). Mostly, backing storage seems to get allocated at the point of export, and device mappings are often held persistently (e.g. the DRM prime code maps the buffer at import time, and keeps it mapped: drm_gem_prime_import_dev). I wasn't aware that CPU access before first device access was considered an abuse of the API - it seems like a valid thing to want to do. > >>>> > >>>>>>>> //buffer is send down the pipeline > >>>>>>>> > >>>>>>>> // Usersapce software post processing occurs > >>>>>>>> mmap buffer > >>>>>>> > >>>>>>> Perhaps the invalidate should happen here in mmap. > >>>>>>> > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>>>>> devices attached to buffer > >>>>>>> > >>>>>>> And that should be okay, mmap does the sync, and if no devices are > >>>>>>> attached nothing could have changed the underlying memory in the > >>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>>>> > >>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > >>>>> Liam was saying that it's too painful for them to do that every time a > >>>>> device unmaps - when in many cases (device->device, no CPU) it's not > >>>>> needed. > >>>> > >>>> Invalidates are painless, at least compared to a real cache flush, just > >>>> set the invalid bit vs actually writing out lines. I thought the issue > >>>> was on the map side. > >>>> > >>> > >>> Invalidates aren't painless for us because we have a coherent system cache > >>> so clean lines get written out. > >> > >> That seems very broken, why would clean lines ever need to be written > >> out, that defeats the whole point of having the invalidate separate from > >> clean. How do you deal with stale cache lines? I guess in your case this > >> is what forces you to have to use uncached memory for DMA-able memory. > >> > > > > My understanding is that our ARM invalidate is a clean + invalidate, I had > > concerns about the clean lines being written to the the system cache as > > part of the 'clean', but the following 'invalidate' would take care of > > actually invalidating the lines (so nothign broken). > > But i am probably wrong on this and it is probably smart enough not to the > > writing of the clean lines. > > > > You are correct that for a lot of ARM cores "invalidate" is always a > "clean + invalidate". At first I thought this was kinda silly as there > is now no way to mark a dirty line invalid without it getting written > out first, but if you think about it any dirty cache-line can be written > out (cleaned) at anytime anyway, so this doesn't actually change system > behavior. You should just not write to memory (make the line dirty) > anything you don't want eventually written out. > > Point two, it's not just smart enough to not write-out clean lines, it > is guaranteed not to write them out by the spec. Otherwise since > cache-lines can be randomly filled if those same clean lines got written > out on invalidate operations there would be no way to maintain coherency > and things would be written over top each other all over the place. > > > But regardless, targets supporting a coherent system cache is a legitamate > > configuration and an invalidate on this configuration does have to go to > > the bus to invalidate the system cache (which isn't free) so I dont' think > > you can make the assumption that invalidates are cheap so that it is okay > > to do them (even if they are not needed) on every dma unmap. > > > > Very true, CMOs need to be broadcast to other coherent masters on a > coherent interconnect (and the interconnect itself if it has a cache as > well (L3)), so not 100% free, but almost, just the infinitesimal cost of > the cache tag check in hardware. If there are no non-coherent devices > attached then the CMOs are no-ops, if there are then the data needs to > be written out either way, doing it every access like is done with > uncached memory (- any write combining) will blow away any saving made > from the one less CMO. Either way you lose with uncached mappings of > memory. If I'm wrong I would love to know. > From what I understand, the current DMA APIs are not equipped to handle having coherent and non-coherent devices attached at the same time. The buffer is either in "CPU land" or "Device land", there's no smaller granule of "Coherent Device land" or "Non-Coherent Device land". I think if there's devices which are making coherent accesses, and devices which are making non-coherent accesses, then we can't support them being attached at the same time without some enhancements to the APIs. > >>> And these invalidates can occur on fairly large buffers. > >>> > >>> That is why we haven't went with using cached ION memory and "tracking CPU > >>> access" because it only solves half the problem, ie there isn't a way to > >>> safely skip the invalidate (because we can't read the future). > >>> Our solution was to go with uncached ION memory (when possible), but as > >>> you can see in other discussions upstream support for uncached memory has > >>> its own issues. > >>> @Liam, in your problematic use-cases, are both devices detached when the buffer moves between them? 1) dev 1 map, access, unmap 2) dev 1 detach 3) (maybe) CPU access 4) dev 2 attach 5) dev 2 map, access I still think a pretty pragmatic solution is to use DMA_ATTR_SKIP_CPU_SYNC until the last device detaches. That won't work if your access sequence looks like above... ...however, if your sequence looks like above, then you probably need to keep at least one of the devices attached anyway. Otherwise, per the API, the buffer could get migrated after 2)/before 5). That will surely hurt a lot more than an invalidate. > >> > >> Sounds like you need to fix upstream support then, finding a way to drop > >> all cacheable mappings of memory you want to make uncached mappings for > >> seems to be the only solution. > >> > > > > I think we can probably agree that there woudln't be a good way to remove > > cached mappings without causing an unacceptable performance degradation > > since it would fragment all the nice 1GB kernel mappings we have. > > > > So I am trying to find an alternative solution. > > > > I'm not sure there is a better solution. How hard is this solution to > implement anyway? The kernel already has to make gaps and cut up that > nice 1GB mapping when you make a reserved memory space in the lowmem > area, so all the logic is probably already implemented. Just need to > allow it to be hooked into from Ion when doing doing the uncached mappings. > I haven't looked recently, but I'm not sure the early memblock code can be reused as-is at runtime. I seem to remember it makes a bunch of assumptions about the fact that it's running "early". If CPU uncached mappings of normal system memory is really the way forward, I could envisage a heap which maintains a pool of chunks of memory which it removed from the kernel mapping. The pool could grow (remove more pages from the kernel mapping)/shrink (add them back to the kernel mapping) as needed. John Reitan implemented a compound-page heap, which used compaction to get a pool of 2MB contiguous pages. Something like that would at least prevent needing full 4kB granularity when removing things from the kernel mapping. Even better, could it somehow be restricted to a region which is already fragmented? (e.g. the one which was used for the default CMA heap) Thanks, -Brian > >>>>> > >>>>>>> > >>>>>>>> [CPU reads/writes to the buffer] > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>>>>> devices attached to buffer > >>>>>>>> munmap buffer > >>>>>>>> > >>>>>>>> //buffer is send down the pipeline > >>>>>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>>>>> writes to a file > >>>>>>>> dma_buf_attach > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>>> [video device writes to buffer] > >>>>>>>> dma_buf_unmap_attachment > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>>>> the pipeline and Video doesn't know the end of the use case) > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > >>>>>>>>>> access then there is no requirement (that I am aware of) for you to call > >>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this > >>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance > >>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>>>>> maintenance on the buffer? > >>>>>>>>> > >>>>>>>> > >>>>>>>> Because ION no longer provides DMA ready memory. > >>>>>>>> Take the above example. > >>>>>>>> > >>>>>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>>>>> Zeros are written to the cache. > >>>>>>>> > >>>>>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>>>>> The camera devices writes directly to the buffer in DDR. > >>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > >>>>>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>>>>> written which corrupts your data. > >>>>>>>> > >>>>>>> > >>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>>>>> for CPU access at the time of zeroing. > >>>>>>> > >>>>> > >>>>> Actually that should be at the point of the first non-coherent device > >>>>> mapping the buffer right? No point in doing CMO if the future accesses > >>>>> are coherent. > >>>> > >>>> I see your point, as long as the zeroing is guaranteed to be the first > >>>> access to this buffer then it should be safe. > >>>> > >>>> Andrew > >>>> > >>>>> > >>>>> Cheers, > >>>>> -Brian > >>>>> > >>>>>>> Andrew > >>>>>>> > >>>>>>>> Liam > >>>>>>>> > >>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>>>>> a Linux Foundation Collaborative Project > >>>>>>>> > >>>> > >>> > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>> a Linux Foundation Collaborative Project > >>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > >
On 1/18/19 3:43 PM, Liam Mark wrote: > On Fri, 18 Jan 2019, Andrew F. Davis wrote: > >> On 1/17/19 7:04 PM, Liam Mark wrote: >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/16/19 4:48 PM, Liam Mark wrote: >>>>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>>>> >>>>>> On 1/15/19 1:05 PM, Laura Abbott wrote: >>>>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: >>>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>>>> >>>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>>>> >>>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance >>>>>>>>>>>> here. >>>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed >>>>>>>>>>>> anyway. >>>>>>>>>>>> >>>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>>>> --- >>>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>>>> >>>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c >>>>>>>>>>>> b/drivers/staging/android/ion/ion.c >>>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct >>>>>>>>>>>> dma_buf_attachment *attachment, >>>>>>>>>>>> table = a->table; >>>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>>>>> - direction)) >>>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>>>> >>>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache >>>>>>>>>>> maintenance. >>>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>>>>> dma_buf_attach then there won't have been a device attached so the >>>>>>>>>>> calls >>>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>>>>> is attached later after data has already been written. Does that >>>>>>>>>> sequence need supporting? >>>>>>>>> >>>>>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>>>>> in Android, but I will focus on later for now. >>>>>>>>> >>>>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>>>>> attached and mapped, right? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>>>> >>>>>>>>> Camera device records video -> software post processing -> video device >>>>>>>>> (who does compression of raw data) and writes to a file >>>>>>>>> >>>>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>>>> IO-coherent (quite common). >>>>>>>>> >>>>>>>> >>>>>>>> This is the start of the problem, having cached mappings of memory that >>>>>>>> is also being accessed non-coherently is going to cause issues one way >>>>>>>> or another. On top of the speculative cache fills that have to be >>>>>>>> constantly fought back against with CMOs like below; some coherent >>>>>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>>>>> (snoop filters get messed up). >>>>>>>> >>>>>>>> The solution is to either always have the addresses marked non-coherent >>>>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>>>> regular system memory allocated at runtime, then all cached mappings of >>>>>>>> it need to be dropped, even the kernel logical address (area as painful >>>>>>>> as that would be). >>>>>>>> >>>>>>> >>>>>>> I agree it's broken, hence my desire to remove it :) >>>>>>> >>>>>>> The other problem is that uncached buffers are being used for >>>>>>> performance reason so anything that would involve getting >>>>>>> rid of the logical address would probably negate any performance >>>>>>> benefit. >>>>>>> >>>>>> >>>>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty >>>>>> adamant that they have valid uses. I'm just not sure performance is one >>>>>> of them, maybe in the case of software locks between devices or >>>>>> something where there needs to be a lot of back and forth interleaved >>>>>> access on small amounts of data? >>>>>> >>>>> >>>>> I wasn't aware that ARM considered this not supported, I thought it was >>>>> supported but they advised against it because of the potential performance >>>>> impact. >>>>> >>>> >>>> Not sure what you mean by "this" being not supported, do you mean mixed >>>> attribute mappings? If so, it will certainly cause problems, and the >>>> problems will change from platform to platform, avoid at all costs is my >>>> understanding of ARM's position. >>>> >>>>> This is after all supported in the DMA APIs and up until now devices have >>>>> been successfully commercializing with this configurations, and I think >>>>> they will continue to commercialize with these configurations for quite a >>>>> while. >>>>> >>>> >>>> Use of uncached memory mappings are almost always wrong in my experience >>>> and are used to work around some bug or because the user doesn't want to >>>> implement proper CMOs. Counter examples welcome. >>>> >>> >>> Okay, let me first try to clarify what I am referring to, as perhaps I am >>> misunderstanding the conversation. >>> >>> In this discussion I was originally referring to a use case with cached >>> memory being accessed by a non io-cohernet device. >>> >>> "In this example assume the buffer is cached and the devices are not >>> IO-coherent (quite common)." >>> >>> to which you did not think was supported: >>> >>> "This is the start of the problem, having cached mappings of memory >>> that is also being accessed non-coherently is going to cause issues >>> one way or another. >>> " >>> >>> And I interpreted Laura's comment below as saying she wanted to remove >>> support in ION for cached memory being accessed by non io-cohernet >>> devices: >>> "I agree it's broken, hence my desire to remove it :)" >>> >>> So assuming my understanding above is correct (and you are not talking >>> about something separate such as removing uncached ION allocation >>> support). >>> >> >> Ah, I think here is where we diverged, I'm assuming Laura's comment to >> be referencing my issue with uncached mappings being handed out without >> first removing all cached mappings of the same memory. Therefore it is >> uncached heaps that are broken. >> > > I am glad that is clarified, but can you then clarify for me your > following statement, I am stil not clear what the problem is then. > > In response to: > "In this example assume the buffer is cached and the devices are > not IO-coherent (quite common)." > > You said: > > "This is the start of the problem, having cached mappings of memory that > is also being accessed non-coherently is going to cause issues one way or > another. " > This was a parallel thought, completely my fault for any confusion here. I was wanting to point out that there will inherently be some issues with both situations. Not that it is a blocker, was just on my mind. >>> Then I guess I am not clear why current uses which use cached memory with >>> non IO-coherent devices are considered to be working around some bug or >>> are not implementing proper CMOs. >>> >>> They use CPU cached mappings because that is the most effective way to >>> access the memory from the CPU side and the devices have an uncached >>> IOMMU mapping because they don't support IO-coherency, and currenlty in >>> the CPU they do cache mainteance at the time of dma map and dma umap so >>> to me they are implementing correct CMOs. >>> >> >> Fully agree here, using cached mappings and performing CMOs when needed >> is the way to go when dealing with memory. IMHO the *only* time when >> uncached mappings are appropriate is for memory mapped I/O (although it >> looks like video memory was often treated as uncached (wc)). >> >>>>> It would be really unfortunate if support was removed as I think that >>>>> would drive clients away from using upstream ION. >>>>> >>>> >>>> I'm not petitioning to remove support, but at very least lets reverse >>>> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by >>>> default, to get uncached you should need to add a flag to your >>>> allocation command pointing out you know what you are doing. >>>> >>> >>> You may not be petitioning to remove support for using cached memory with >>> non io-coherent devices but I interpreted Laura's comment as wanting to do >>> so, and I had concerns about that. >>> >> >> What I would like is for the default memory handed out by Ion to be >> normal cacheable memory, just like is always handed out to users-space. >> DMA-BUF already provides the means to deal with the CMOs required to >> work with non-io-coherent devices so all should be good here. >> >> If you want Ion to give out uncached memory then I think you should need >> to explicitly state so with an allocation flag. And right now the >> uncached memory you will get back may have other cached mappings (kernel >> lowmem mappings) meaning you will have hard to predict results (on ARM >> at least). > > Yes, I can understand why it would make sense to default to cached memory. > >> I just don't see much use for them (uncached mappings of >> regular memory) right now. >> > > I can understand why people don't like ION providing uncached support, but > I have been pushing to keep un-cached support in order to keep the > performance of Android ION clients on par with the previous version of ION > until a better solution can be found (either by changing ION or Android). > > Basically most ION use cases don't involve any (or very little) CPU access > so for now by using uncached ION allocations we can avoid the uncessary > cache maitenance and we are safe if some CPU access is required. Of course > for use cases involving a lot of CPU access the clients switch over to > using cached buffers. > If you have very few CPU accesses then there should be very few CMOs I agree. It seems the problem is you are constantly detaching and re-attaching devices each time the buffer is passed around between each device. To me that is broken usage, the devices should all attach, use the buffer, then all detach. Otherwise after detach there is no clean way to know what is the right thing to do with the buffer (CMO or not). > >>>>>>>>> ION buffer is allocated. >>>>>>>>> >>>>>>>>> //Camera device records video >>>>>>>>> dma_buf_attach >>>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>>> >>>>>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>>>>> problem. >>>>>>>> >>>>>>>>> [camera device writes to buffer] >>>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>>>> >>>>>>>> It doesn't know there will be any further CPU access, it could get freed >>>>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>>>> requests access again. >>>>>>>> >>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>>>>>> down >>>>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>>>> >>>>>>>> >>>>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>>>> everything as modular as possible and separate the steps, but at this >>>>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>>>>> for the next attachment, which may change the required backing memory >>>>>>>> completely. >>>>>>>> >>>>>>>> All devices should attach before the first mapping, and only let go >>>>>>>> after the task is complete, otherwise this buffers data needs copied off >>>>>>>> to a different location or the CPU needs to take ownership in-between. >>>>>>>> >>>>>>> >>>>>>> Maybe it's broken but it's the status quo and we spent a good >>>>>>> amount of time at plumbers concluding there isn't a great way >>>>>>> to fix it :/ >>>>>>> >>>>>> >>>>>> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ >>>>>> >>>>>> Perhaps just stronger rules on sequencing of operations? I'm not saying >>>>>> I have a good solution either, I just don't see any way forward without >>>>>> some use-case getting broken, so better to fix now over later. >>>>>> >>>>> >>>>> I can see the benefits of Android doing things the way they do, I would >>>>> request that changes we make continue to support Android, or we find a way >>>>> to convice them to change, as they are the main ION client and I assume >>>>> other ION clients in the future will want to do this as well. >>>>> >>>> >>>> Android may be the biggest user today (makes sense, Ion come out of the >>>> Android project), but that can change, and getting changes into Android >>>> will be easier that the upstream kernel once Ion is out of staging. >>>> >>>> Unlike some other big ARM vendors, we (TI) do not primarily build mobile >>>> chips targeting Android, our core offerings target more traditional >>>> Linux userspaces, and I'm guessing others will start to do the same as >>>> ARM tries to push more into desktop, server, and other spaces again. >>>> >>>>> I am concerned that if you go with a solution which enforces what you >>>>> mention above, and bring ION out of staging that way, it will make it that >>>>> much harder to solve this for Android and therefore harder to get >>>>> Android clients to move to the upstream ION (and get everybody off their >>>>> vendor modified Android versions). >>>>> >>>> >>>> That would be an Android problem, reducing functionality in upstream to >>>> match what some evil vendor trees do to support Android is not the way >>>> forward on this. At least for us we are going to try to make all our >>>> software offerings follow proper buffer ownership (including our Android >>>> offering). >>>> >>>>>>>>> //buffer is send down the pipeline >>>>>>>>> >>>>>>>>> // Usersapce software post processing occurs >>>>>>>>> mmap buffer >>>>>>>> >>>>>>>> Perhaps the invalidate should happen here in mmap. >>>>>>>> >>>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>>>>>> devices attached to buffer >>>>>>>> >>>>>>>> And that should be okay, mmap does the sync, and if no devices are >>>>>>>> attached nothing could have changed the underlying memory in the >>>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>>>>>>> >>>>>>>>> [CPU reads/writes to the buffer] >>>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>>>>>> devices attached to buffer >>>>>>>>> munmap buffer >>>>>>>>> >>>>>>>>> //buffer is send down the pipeline >>>>>>>>> // Buffer is send to video device (who does compression of raw data) and >>>>>>>>> writes to a file >>>>>>>>> dma_buf_attach >>>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>>>> [video device writes to buffer] >>>>>>>>> dma_buf_unmap_attachment >>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent >>>>>>>>> down >>>>>>>>> the pipeline and Video doesn't know the end of the use case) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not >>>>>>>>>>> doing CPU >>>>>>>>>>> access then there is no requirement (that I am aware of) for you to >>>>>>>>>>> call >>>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and >>>>>>>>>>> if this >>>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache >>>>>>>>>>> maintenance >>>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>>>>>> maintenance on the buffer? >>>>>>>>>> >>>>>>>>> >>>>>>>>> Because ION no longer provides DMA ready memory. >>>>>>>>> Take the above example. >>>>>>>>> >>>>>>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>>>>>> Zeros are written to the cache. >>>>>>>>> >>>>>>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>>>>>> The camera devices writes directly to the buffer in DDR. >>>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the >>>>>>>>> zeros) is >>>>>>>>> evicted from the cache, this zero overwrites data the camera device has >>>>>>>>> written which corrupts your data. >>>>>>>>> >>>>>>>> >>>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>>>>>> for CPU access at the time of zeroing. >>>>>>>> >>>>>>>> Andrew >>>>>>>> >>>>>>>>> Liam >>>>>>>>> >>>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>>>>>> a Linux Foundation Collaborative Project >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>> a Linux Foundation Collaborative Project >>>>> >>>> >>> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>> a Linux Foundation Collaborative Project >>> >> > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > a Linux Foundation Collaborative Project >
On Mon, 21 Jan 2019, Andrew F. Davis wrote: > On 1/18/19 3:43 PM, Liam Mark wrote: > > On Fri, 18 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/17/19 7:04 PM, Liam Mark wrote: > >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote: > >>> > >>>> On 1/16/19 4:48 PM, Liam Mark wrote: > >>>>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >>>>> > >>>>>> On 1/15/19 1:05 PM, Laura Abbott wrote: > >>>>>>> On 1/15/19 10:38 AM, Andrew F. Davis wrote: > >>>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>>>>>> > >>>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance > >>>>>>>>>>>> here. > >>>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed > >>>>>>>>>>>> anyway. > >>>>>>>>>>>> > >>>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>>>>>> --- > >>>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>>>>>> > >>>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c > >>>>>>>>>>>> b/drivers/staging/android/ion/ion.c > >>>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct > >>>>>>>>>>>> dma_buf_attachment *attachment, > >>>>>>>>>>>> table = a->table; > >>>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>>> - direction)) > >>>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>>>>>> > >>>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache > >>>>>>>>>>> maintenance. > >>>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>>>>>> dma_buf_attach then there won't have been a device attached so the > >>>>>>>>>>> calls > >>>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> That should be okay though, if you have no attachments (or all > >>>>>>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>>>>>> is attached later after data has already been written. Does that > >>>>>>>>>> sequence need supporting? > >>>>>>>>> > >>>>>>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>>>>>> in Android, but I will focus on later for now. > >>>>>>>>> > >>>>>>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>>>>>> attached and mapped, right? > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> Here is an example where CPU access can happen later in Android. > >>>>>>>>> > >>>>>>>>> Camera device records video -> software post processing -> video device > >>>>>>>>> (who does compression of raw data) and writes to a file > >>>>>>>>> > >>>>>>>>> In this example assume the buffer is cached and the devices are not > >>>>>>>>> IO-coherent (quite common). > >>>>>>>>> > >>>>>>>> > >>>>>>>> This is the start of the problem, having cached mappings of memory that > >>>>>>>> is also being accessed non-coherently is going to cause issues one way > >>>>>>>> or another. On top of the speculative cache fills that have to be > >>>>>>>> constantly fought back against with CMOs like below; some coherent > >>>>>>>> interconnects behave badly when you mix coherent and non-coherent access > >>>>>>>> (snoop filters get messed up). > >>>>>>>> > >>>>>>>> The solution is to either always have the addresses marked non-coherent > >>>>>>>> (like device memory, no-map carveouts), or if you really want to use > >>>>>>>> regular system memory allocated at runtime, then all cached mappings of > >>>>>>>> it need to be dropped, even the kernel logical address (area as painful > >>>>>>>> as that would be). > >>>>>>>> > >>>>>>> > >>>>>>> I agree it's broken, hence my desire to remove it :) > >>>>>>> > >>>>>>> The other problem is that uncached buffers are being used for > >>>>>>> performance reason so anything that would involve getting > >>>>>>> rid of the logical address would probably negate any performance > >>>>>>> benefit. > >>>>>>> > >>>>>> > >>>>>> I wouldn't go as far as to remove them just yet.. Liam seems pretty > >>>>>> adamant that they have valid uses. I'm just not sure performance is one > >>>>>> of them, maybe in the case of software locks between devices or > >>>>>> something where there needs to be a lot of back and forth interleaved > >>>>>> access on small amounts of data? > >>>>>> > >>>>> > >>>>> I wasn't aware that ARM considered this not supported, I thought it was > >>>>> supported but they advised against it because of the potential performance > >>>>> impact. > >>>>> > >>>> > >>>> Not sure what you mean by "this" being not supported, do you mean mixed > >>>> attribute mappings? If so, it will certainly cause problems, and the > >>>> problems will change from platform to platform, avoid at all costs is my > >>>> understanding of ARM's position. > >>>> > >>>>> This is after all supported in the DMA APIs and up until now devices have > >>>>> been successfully commercializing with this configurations, and I think > >>>>> they will continue to commercialize with these configurations for quite a > >>>>> while. > >>>>> > >>>> > >>>> Use of uncached memory mappings are almost always wrong in my experience > >>>> and are used to work around some bug or because the user doesn't want to > >>>> implement proper CMOs. Counter examples welcome. > >>>> > >>> > >>> Okay, let me first try to clarify what I am referring to, as perhaps I am > >>> misunderstanding the conversation. > >>> > >>> In this discussion I was originally referring to a use case with cached > >>> memory being accessed by a non io-cohernet device. > >>> > >>> "In this example assume the buffer is cached and the devices are not > >>> IO-coherent (quite common)." > >>> > >>> to which you did not think was supported: > >>> > >>> "This is the start of the problem, having cached mappings of memory > >>> that is also being accessed non-coherently is going to cause issues > >>> one way or another. > >>> " > >>> > >>> And I interpreted Laura's comment below as saying she wanted to remove > >>> support in ION for cached memory being accessed by non io-cohernet > >>> devices: > >>> "I agree it's broken, hence my desire to remove it :)" > >>> > >>> So assuming my understanding above is correct (and you are not talking > >>> about something separate such as removing uncached ION allocation > >>> support). > >>> > >> > >> Ah, I think here is where we diverged, I'm assuming Laura's comment to > >> be referencing my issue with uncached mappings being handed out without > >> first removing all cached mappings of the same memory. Therefore it is > >> uncached heaps that are broken. > >> > > > > I am glad that is clarified, but can you then clarify for me your > > following statement, I am stil not clear what the problem is then. > > > > In response to: > > "In this example assume the buffer is cached and the devices are > > not IO-coherent (quite common)." > > > > You said: > > > > "This is the start of the problem, having cached mappings of memory that > > is also being accessed non-coherently is going to cause issues one way or > > another. " > > > > This was a parallel thought, completely my fault for any confusion here. > I was wanting to point out that there will inherently be some issues > with both situations. Not that it is a blocker, was just on my mind. > > >>> Then I guess I am not clear why current uses which use cached memory with > >>> non IO-coherent devices are considered to be working around some bug or > >>> are not implementing proper CMOs. > >>> > >>> They use CPU cached mappings because that is the most effective way to > >>> access the memory from the CPU side and the devices have an uncached > >>> IOMMU mapping because they don't support IO-coherency, and currenlty in > >>> the CPU they do cache mainteance at the time of dma map and dma umap so > >>> to me they are implementing correct CMOs. > >>> > >> > >> Fully agree here, using cached mappings and performing CMOs when needed > >> is the way to go when dealing with memory. IMHO the *only* time when > >> uncached mappings are appropriate is for memory mapped I/O (although it > >> looks like video memory was often treated as uncached (wc)). > >> > >>>>> It would be really unfortunate if support was removed as I think that > >>>>> would drive clients away from using upstream ION. > >>>>> > >>>> > >>>> I'm not petitioning to remove support, but at very least lets reverse > >>>> the ION_FLAG_CACHED flag. Ion should hand out cached normal memory by > >>>> default, to get uncached you should need to add a flag to your > >>>> allocation command pointing out you know what you are doing. > >>>> > >>> > >>> You may not be petitioning to remove support for using cached memory with > >>> non io-coherent devices but I interpreted Laura's comment as wanting to do > >>> so, and I had concerns about that. > >>> > >> > >> What I would like is for the default memory handed out by Ion to be > >> normal cacheable memory, just like is always handed out to users-space. > >> DMA-BUF already provides the means to deal with the CMOs required to > >> work with non-io-coherent devices so all should be good here. > >> > >> If you want Ion to give out uncached memory then I think you should need > >> to explicitly state so with an allocation flag. And right now the > >> uncached memory you will get back may have other cached mappings (kernel > >> lowmem mappings) meaning you will have hard to predict results (on ARM > >> at least). > > > > Yes, I can understand why it would make sense to default to cached memory. > > > >> I just don't see much use for them (uncached mappings of > >> regular memory) right now. > >> > > > > I can understand why people don't like ION providing uncached support, but > > I have been pushing to keep un-cached support in order to keep the > > performance of Android ION clients on par with the previous version of ION > > until a better solution can be found (either by changing ION or Android). > > > > Basically most ION use cases don't involve any (or very little) CPU access > > so for now by using uncached ION allocations we can avoid the uncessary > > cache maitenance and we are safe if some CPU access is required. Of course > > for use cases involving a lot of CPU access the clients switch over to > > using cached buffers. > > > > If you have very few CPU accesses then there should be very few CMOs I > agree. It seems the problem is you are constantly detaching and > re-attaching devices each time the buffer is passed around between each > device. To me that is broken usage, the devices should all attach, use > the buffer, then all detach. Otherwise after detach there is no clean > way to know what is the right thing to do with the buffer (CMO or not). > Yes it would be be great if all the devices could attach beforehand and be kept attached. Unfortunately it would be difficult in a buffer "pipelining" use case such as Android's to get all the devices to attach beforehand. We have spoken to the Google Android team about having some kind of destructor support added so that devices could stay attached and then when the use case ends be notified of the end of use case (through the destructor) so that they could know when to detach. Unfortunately the Google Android team said this was too difficult to add as they don't track all the buffers in the system. And please note that there are other complexities to having all the devices in the pipeline attached at once, for example the begin_cpu_access/end_cpu_access calls currently do cache maintenance on the buffer for each of the attached devices, with lots of attached devices this would result in a lot of duplicated cache maintenance on the buffer. You would need some way to optimally apply cache maintenance that satisfies all the devices. Also you would need to not only keep the devices attached, you need to the buffers dma-mapped (see thread "dma-buf: add support for mapping with dma mapping attributes") for the cache maintenance to be applied correctly. > > > >>>>>>>>> ION buffer is allocated. > >>>>>>>>> > >>>>>>>>> //Camera device records video > >>>>>>>>> dma_buf_attach > >>>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>>> > >>>>>>>> Why does the buffer need to be cleaned here? I just got through reading > >>>>>>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>>>>>> suggestion of tracking if the buffer has had CPU access since the last > >>>>>>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>>>>>> problem. > >>>>>>>> > >>>>>>>>> [camera device writes to buffer] > >>>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>>>>>> > >>>>>>>> It doesn't know there will be any further CPU access, it could get freed > >>>>>>>> after this for all we know, the invalidate can be saved until the CPU > >>>>>>>> requests access again. > >>>>>>>> > >>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>>>>>> down > >>>>>>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>>>>>> > >>>>>>>> > >>>>>>>> This seems like a broken use-case, I understand the desire to keep > >>>>>>>> everything as modular as possible and separate the steps, but at this > >>>>>>>> point no one owns this buffers backing memory, not the CPU or any > >>>>>>>> device. I would go as far as to say DMA-BUF should be free now to > >>>>>>>> de-allocate the backing storage if it wants, that way it could get ready > >>>>>>>> for the next attachment, which may change the required backing memory > >>>>>>>> completely. > >>>>>>>> > >>>>>>>> All devices should attach before the first mapping, and only let go > >>>>>>>> after the task is complete, otherwise this buffers data needs copied off > >>>>>>>> to a different location or the CPU needs to take ownership in-between. > >>>>>>>> > >>>>>>> > >>>>>>> Maybe it's broken but it's the status quo and we spent a good > >>>>>>> amount of time at plumbers concluding there isn't a great way > >>>>>>> to fix it :/ > >>>>>>> > >>>>>> > >>>>>> Hmm, guess that doesn't prove there is not a great way to fix it either.. :/ > >>>>>> > >>>>>> Perhaps just stronger rules on sequencing of operations? I'm not saying > >>>>>> I have a good solution either, I just don't see any way forward without > >>>>>> some use-case getting broken, so better to fix now over later. > >>>>>> > >>>>> > >>>>> I can see the benefits of Android doing things the way they do, I would > >>>>> request that changes we make continue to support Android, or we find a way > >>>>> to convice them to change, as they are the main ION client and I assume > >>>>> other ION clients in the future will want to do this as well. > >>>>> > >>>> > >>>> Android may be the biggest user today (makes sense, Ion come out of the > >>>> Android project), but that can change, and getting changes into Android > >>>> will be easier that the upstream kernel once Ion is out of staging. > >>>> > >>>> Unlike some other big ARM vendors, we (TI) do not primarily build mobile > >>>> chips targeting Android, our core offerings target more traditional > >>>> Linux userspaces, and I'm guessing others will start to do the same as > >>>> ARM tries to push more into desktop, server, and other spaces again. > >>>> > >>>>> I am concerned that if you go with a solution which enforces what you > >>>>> mention above, and bring ION out of staging that way, it will make it that > >>>>> much harder to solve this for Android and therefore harder to get > >>>>> Android clients to move to the upstream ION (and get everybody off their > >>>>> vendor modified Android versions). > >>>>> > >>>> > >>>> That would be an Android problem, reducing functionality in upstream to > >>>> match what some evil vendor trees do to support Android is not the way > >>>> forward on this. At least for us we are going to try to make all our > >>>> software offerings follow proper buffer ownership (including our Android > >>>> offering). > >>>> > >>>>>>>>> //buffer is send down the pipeline > >>>>>>>>> > >>>>>>>>> // Usersapce software post processing occurs > >>>>>>>>> mmap buffer > >>>>>>>> > >>>>>>>> Perhaps the invalidate should happen here in mmap. > >>>>>>>> > >>>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>>>>>> devices attached to buffer > >>>>>>>> > >>>>>>>> And that should be okay, mmap does the sync, and if no devices are > >>>>>>>> attached nothing could have changed the underlying memory in the > >>>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>>>>>>> > >>>>>>>>> [CPU reads/writes to the buffer] > >>>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>>>>>> devices attached to buffer > >>>>>>>>> munmap buffer > >>>>>>>>> > >>>>>>>>> //buffer is send down the pipeline > >>>>>>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>>>>>> writes to a file > >>>>>>>>> dma_buf_attach > >>>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>>>> [video device writes to buffer] > >>>>>>>>> dma_buf_unmap_attachment > >>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent > >>>>>>>>> down > >>>>>>>>> the pipeline and Video doesn't know the end of the use case) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not > >>>>>>>>>>> doing CPU > >>>>>>>>>>> access then there is no requirement (that I am aware of) for you to > >>>>>>>>>>> call > >>>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and > >>>>>>>>>>> if this > >>>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache > >>>>>>>>>>> maintenance > >>>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>>>>>> maintenance on the buffer? > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> Because ION no longer provides DMA ready memory. > >>>>>>>>> Take the above example. > >>>>>>>>> > >>>>>>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>>>>>> Zeros are written to the cache. > >>>>>>>>> > >>>>>>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>>>>>> The camera devices writes directly to the buffer in DDR. > >>>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the > >>>>>>>>> zeros) is > >>>>>>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>>>>>> written which corrupts your data. > >>>>>>>>> > >>>>>>>> > >>>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>>>>>> for CPU access at the time of zeroing. > >>>>>>>> > >>>>>>>> Andrew > >>>>>>>> > >>>>>>>>> Liam > >>>>>>>>> > >>>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>>>>>> a Linux Foundation Collaborative Project > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>> a Linux Foundation Collaborative Project > >>>>> > >>>> > >>> > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>> a Linux Foundation Collaborative Project > >>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On Fri, 18 Jan 2019, Andrew F. Davis wrote: > On 1/17/19 7:11 PM, Liam Mark wrote: > > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > > > >> On 1/16/19 4:54 PM, Liam Mark wrote: > >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: > >>> > >>>> On 1/16/19 9:19 AM, Brian Starkey wrote: > >>>>> Hi :-) > >>>>> > >>>>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > >>>>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: > >>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > >>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > >>>>>>>> > >>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > >>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > >>>>>>>>>> > >>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > >>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > >>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > >>>>>>>>>>> > >>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > >>>>>>>>>>> --- > >>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > >>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > >>>>>>>>>>> > >>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > >>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > >>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c > >>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > >>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > >>>>>>>>>>> > >>>>>>>>>>> table = a->table; > >>>>>>>>>>> > >>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>> - direction)) > >>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > >>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > >>>>>>>>>> > >>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > >>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > >>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > >>>>>>>>>> dma_buf_attach then there won't have been a device attached so the calls > >>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> That should be okay though, if you have no attachments (or all > >>>>>>>>> attachments are IO-coherent) then there is no need for cache > >>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > >>>>>>>>> is attached later after data has already been written. Does that > >>>>>>>>> sequence need supporting? > >>>>>>>> > >>>>>>>> Yes, but also I think there are cases where CPU access can happen before > >>>>>>>> in Android, but I will focus on later for now. > >>>>>>>> > >>>>>>>>> DMA-BUF doesn't have to allocate the backing > >>>>>>>>> memory until map_dma_buf() time, and that should only happen after all > >>>>>>>>> the devices have attached so it can know where to put the buffer. So we > >>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are > >>>>>>>>> attached and mapped, right? > >>>>>>>>> > >>>>>>>> > >>>>>>>> Here is an example where CPU access can happen later in Android. > >>>>>>>> > >>>>>>>> Camera device records video -> software post processing -> video device > >>>>>>>> (who does compression of raw data) and writes to a file > >>>>>>>> > >>>>>>>> In this example assume the buffer is cached and the devices are not > >>>>>>>> IO-coherent (quite common). > >>>>>>>> > >>>>>>> > >>>>>>> This is the start of the problem, having cached mappings of memory that > >>>>>>> is also being accessed non-coherently is going to cause issues one way > >>>>>>> or another. On top of the speculative cache fills that have to be > >>>>>>> constantly fought back against with CMOs like below; some coherent > >>>>>>> interconnects behave badly when you mix coherent and non-coherent access > >>>>>>> (snoop filters get messed up). > >>>>>>> > >>>>>>> The solution is to either always have the addresses marked non-coherent > >>>>>>> (like device memory, no-map carveouts), or if you really want to use > >>>>>>> regular system memory allocated at runtime, then all cached mappings of > >>>>>>> it need to be dropped, even the kernel logical address (area as painful > >>>>>>> as that would be). > >>>>> > >>>>> Ouch :-( I wasn't aware about these potential interconnect issues. How > >>>>> "real" is that? It seems that we aren't really hitting that today on > >>>>> real devices. > >>>>> > >>>> > >>>> Sadly there is at least one real device like this now (TI AM654). We > >>>> spent some time working with the ARM interconnect spec designers to see > >>>> if this was allowed behavior, final conclusion was mixing coherent and > >>>> non-coherent accesses is never a good idea.. So we have been working to > >>>> try to minimize any cases of mixed attributes [0], if a region is > >>>> coherent then everyone in the system needs to treat it as such and > >>>> vice-versa, even clever CMO that work on other systems wont save you > >>>> here. :( > >>>> > >>>> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 > >>>> > >>>> > >>>>>>> > >>>>>>>> ION buffer is allocated. > >>>>>>>> > >>>>>>>> //Camera device records video > >>>>>>>> dma_buf_attach > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>> > >>>>>>> Why does the buffer need to be cleaned here? I just got through reading > >>>>>>> the thread linked by Laura in the other reply. I do like +Brian's > >>>>>> > >>>>>> Actually +Brian this time :) > >>>>>> > >>>>>>> suggestion of tracking if the buffer has had CPU access since the last > >>>>>>> time and only flushing the cache if it has. As unmapped heaps never get > >>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > >>>>>>> problem. > >>>>>>> > >>>>>>>> [camera device writes to buffer] > >>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > >>>>>>> > >>>>>>> It doesn't know there will be any further CPU access, it could get freed > >>>>>>> after this for all we know, the invalidate can be saved until the CPU > >>>>>>> requests access again. > >>>>> > >>>>> We don't have any API to allow the invalidate to happen on CPU access > >>>>> if all devices already detached. We need a struct device pointer to > >>>>> give to the DMA API, otherwise on arm64 there'll be no invalidate. > >>>>> > >>>>> I had a chat with a few people internally after the previous > >>>>> discussion with Liam. One suggestion was to use > >>>>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > >>>>> one other device attached (guarantees that we can do an invalidate in > >>>>> the future if begin_cpu_access is called). If the last device > >>>>> detaches, do a sync then. > >>>>> > >>>>> Conversely, in map_dma_buf, we would track if there was any CPU access > >>>>> and use/skip the sync appropriately. > >>>>> > >>>> > >>>> Now that I think this all through I agree this patch is probably wrong. > >>>> The real fix needs to be better handling in the dma_map_sg() to deal > >>>> with the case of the memory not being mapped (what I'm dealing with for > >>>> unmapped heaps), and for cases when the memory in question is not cached > >>>> (Liam's issue I think). For both these cases the dma_map_sg() does the > >>>> wrong thing. > >>>> > >>>>> I did start poking the code to check out how that would look, but then > >>>>> Christmas happened and I'm still catching back up. > >>>>> > >>>>>>> > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>>>> the pipeline and Camera doesn't know the end of the use case) > >>>>>>>> > >>>>>>> > >>>>>>> This seems like a broken use-case, I understand the desire to keep > >>>>>>> everything as modular as possible and separate the steps, but at this > >>>>>>> point no one owns this buffers backing memory, not the CPU or any > >>>>>>> device. I would go as far as to say DMA-BUF should be free now to > >>>>>>> de-allocate the backing storage if it wants, that way it could get ready > >>>>>>> for the next attachment, which may change the required backing memory > >>>>>>> completely. > >>>>>>> > >>>>>>> All devices should attach before the first mapping, and only let go > >>>>>>> after the task is complete, otherwise this buffers data needs copied off > >>>>>>> to a different location or the CPU needs to take ownership in-between. > >>>>>>> > >>>>> > >>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF > >>>>> implementations which actually do that? I hear it quoted a lot, > >>>>> because that's what the docs say - but if the reality doesn't match > >>>>> it, maybe we should change the docs. > >>>>> > >>>> > >>>> Do you mean on the userspace side? I'm not sure, seems like Android > >>>> might be doing this wrong from what I can gather. From kernel side if > >>>> you mean the "de-allocate the backing storage", we will have some cases > >>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF > >>>> in ways not specified in the documentation. Changing the docs to force > >>>> the backing memory to always be allocated breaks the central goal in > >>>> having attach/map in DMA-BUF separate. > >>>> > >>>>>>>> //buffer is send down the pipeline > >>>>>>>> > >>>>>>>> // Usersapce software post processing occurs > >>>>>>>> mmap buffer > >>>>>>> > >>>>>>> Perhaps the invalidate should happen here in mmap. > >>>>>>> > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > >>>>>>>> devices attached to buffer > >>>>>>> > >>>>>>> And that should be okay, mmap does the sync, and if no devices are > >>>>>>> attached nothing could have changed the underlying memory in the > >>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > >>>>> > >>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > >>>>> Liam was saying that it's too painful for them to do that every time a > >>>>> device unmaps - when in many cases (device->device, no CPU) it's not > >>>>> needed. > >>>> > >>>> Invalidates are painless, at least compared to a real cache flush, just > >>>> set the invalid bit vs actually writing out lines. I thought the issue > >>>> was on the map side. > >>>> > >>> > >>> Invalidates aren't painless for us because we have a coherent system cache > >>> so clean lines get written out. > >> > >> That seems very broken, why would clean lines ever need to be written > >> out, that defeats the whole point of having the invalidate separate from > >> clean. How do you deal with stale cache lines? I guess in your case this > >> is what forces you to have to use uncached memory for DMA-able memory. > >> > > > > My understanding is that our ARM invalidate is a clean + invalidate, I had > > concerns about the clean lines being written to the the system cache as > > part of the 'clean', but the following 'invalidate' would take care of > > actually invalidating the lines (so nothign broken). > > But i am probably wrong on this and it is probably smart enough not to the > > writing of the clean lines. > > > > You are correct that for a lot of ARM cores "invalidate" is always a > "clean + invalidate". At first I thought this was kinda silly as there > is now no way to mark a dirty line invalid without it getting written > out first, but if you think about it any dirty cache-line can be written > out (cleaned) at anytime anyway, so this doesn't actually change system > behavior. You should just not write to memory (make the line dirty) > anything you don't want eventually written out. > > Point two, it's not just smart enough to not write-out clean lines, it > is guaranteed not to write them out by the spec. Otherwise since > cache-lines can be randomly filled if those same clean lines got written > out on invalidate operations there would be no way to maintain coherency > and things would be written over top each other all over the place. > > > But regardless, targets supporting a coherent system cache is a legitamate > > configuration and an invalidate on this configuration does have to go to > > the bus to invalidate the system cache (which isn't free) so I dont' think > > you can make the assumption that invalidates are cheap so that it is okay > > to do them (even if they are not needed) on every dma unmap. > > > > Very true, CMOs need to be broadcast to other coherent masters on a > coherent interconnect (and the interconnect itself if it has a cache as > well (L3)), so not 100% free, but almost, just the infinitesimal cost of > the cache tag check in hardware. If there are no non-coherent devices > attached then the CMOs are no-ops, if there are then the data needs to > be written out either way, doing it every access like is done with > uncached memory (- any write combining) will blow away any saving made > from the one less CMO. Either way you lose with uncached mappings of > memory. If I'm wrong I would love to know. > I would need to think about this more before replying. > >>> And these invalidates can occur on fairly large buffers. > >>> > >>> That is why we haven't went with using cached ION memory and "tracking CPU > >>> access" because it only solves half the problem, ie there isn't a way to > >>> safely skip the invalidate (because we can't read the future). > >>> Our solution was to go with uncached ION memory (when possible), but as > >>> you can see in other discussions upstream support for uncached memory has > >>> its own issues. > >>> > >> > >> Sounds like you need to fix upstream support then, finding a way to drop > >> all cacheable mappings of memory you want to make uncached mappings for > >> seems to be the only solution. > >> > > > > I think we can probably agree that there woudln't be a good way to remove > > cached mappings without causing an unacceptable performance degradation > > since it would fragment all the nice 1GB kernel mappings we have. > > > > So I am trying to find an alternative solution. > > > > I'm not sure there is a better solution. How hard is this solution to > implement anyway? The kernel already has to make gaps and cut up that > nice 1GB mapping when you make a reserved memory space in the lowmem > area, so all the logic is probably already implemented. Just need to > allow it to be hooked into from Ion when doing doing the uncached mappings. > Even before attempting to implement it may be best to first run an experiment where block mappings are disabled for the kernel mappings in order to simulate a system where a lot of memory is allocated with uncached mappings (such as could happen when using ION with Android). Then try running any important benchmarks, my expectation is that the performance impact of losing these kernel mappings will be unacceptable. It was a couple years ago, but I remember when I played around with the kernel block mappings it was impacting some of our benchmarks. > >>>>> > >>>>>>> > >>>>>>>> [CPU reads/writes to the buffer] > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > >>>>>>>> devices attached to buffer > >>>>>>>> munmap buffer > >>>>>>>> > >>>>>>>> //buffer is send down the pipeline > >>>>>>>> // Buffer is send to video device (who does compression of raw data) and > >>>>>>>> writes to a file > >>>>>>>> dma_buf_attach > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > >>>>>>>> [video device writes to buffer] > >>>>>>>> dma_buf_unmap_attachment > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > >>>>>>>> the pipeline and Video doesn't know the end of the use case) > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > >>>>>>>>>> access then there is no requirement (that I am aware of) for you to call > >>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this > >>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance > >>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> If I am not doing any CPU access then why do I need CPU cache > >>>>>>>>> maintenance on the buffer? > >>>>>>>>> > >>>>>>>> > >>>>>>>> Because ION no longer provides DMA ready memory. > >>>>>>>> Take the above example. > >>>>>>>> > >>>>>>>> ION allocates memory from buddy allocator and requests zeroing. > >>>>>>>> Zeros are written to the cache. > >>>>>>>> > >>>>>>>> You pass the buffer to the camera device which is not IO-coherent. > >>>>>>>> The camera devices writes directly to the buffer in DDR. > >>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > >>>>>>>> evicted from the cache, this zero overwrites data the camera device has > >>>>>>>> written which corrupts your data. > >>>>>>>> > >>>>>>> > >>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > >>>>>>> for CPU access at the time of zeroing. > >>>>>>> > >>>>> > >>>>> Actually that should be at the point of the first non-coherent device > >>>>> mapping the buffer right? No point in doing CMO if the future accesses > >>>>> are coherent. > >>>> > >>>> I see your point, as long as the zeroing is guaranteed to be the first > >>>> access to this buffer then it should be safe. > >>>> > >>>> Andrew > >>>> > >>>>> > >>>>> Cheers, > >>>>> -Brian > >>>>> > >>>>>>> Andrew > >>>>>>> > >>>>>>>> Liam > >>>>>>>> > >>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>>>>>>> a Linux Foundation Collaborative Project > >>>>>>>> > >>>> > >>> > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > >>> a Linux Foundation Collaborative Project > >>> > >> > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > a Linux Foundation Collaborative Project > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
On 1/21/19 5:22 AM, Brian Starkey wrote: > Hi, > > Sorry for being a bit sporadic on this. I was out travelling last week > with little time for email. > > On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote: >> On 1/17/19 7:11 PM, Liam Mark wrote: >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote: >>> >>>> On 1/16/19 4:54 PM, Liam Mark wrote: >>>>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: >>>>> >>>>>> On 1/16/19 9:19 AM, Brian Starkey wrote: >>>>>>> Hi :-) >>>>>>> >>>>>>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: >>>>>>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: >>>>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: >>>>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: >>>>>>>>>> >>>>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: >>>>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. >>>>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with >>>>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. >>>>>>>>>>>>> >>>>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> >>>>>>>>>>>>> --- >>>>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- >>>>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) >>>>>>>>>>>>> >>>>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c >>>>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 >>>>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c >>>>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c >>>>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, >>>>>>>>>>>>> >>>>>>>>>>>>> table = a->table; >>>>>>>>>>>>> >>>>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, >>>>>>>>>>>>> - direction)) >>>>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, >>>>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. >>>>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. >>>>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to >>>>>>>>>>>> dma_buf_attach then there won't have been a device attached so the calls >>>>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> That should be okay though, if you have no attachments (or all >>>>>>>>>>> attachments are IO-coherent) then there is no need for cache >>>>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device >>>>>>>>>>> is attached later after data has already been written. Does that >>>>>>>>>>> sequence need supporting? >>>>>>>>>> >>>>>>>>>> Yes, but also I think there are cases where CPU access can happen before >>>>>>>>>> in Android, but I will focus on later for now. >>>>>>>>>> >>>>>>>>>>> DMA-BUF doesn't have to allocate the backing >>>>>>>>>>> memory until map_dma_buf() time, and that should only happen after all >>>>>>>>>>> the devices have attached so it can know where to put the buffer. So we >>>>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are >>>>>>>>>>> attached and mapped, right? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Here is an example where CPU access can happen later in Android. >>>>>>>>>> >>>>>>>>>> Camera device records video -> software post processing -> video device >>>>>>>>>> (who does compression of raw data) and writes to a file >>>>>>>>>> >>>>>>>>>> In this example assume the buffer is cached and the devices are not >>>>>>>>>> IO-coherent (quite common). >>>>>>>>>> >>>>>>>>> >>>>>>>>> This is the start of the problem, having cached mappings of memory that >>>>>>>>> is also being accessed non-coherently is going to cause issues one way >>>>>>>>> or another. On top of the speculative cache fills that have to be >>>>>>>>> constantly fought back against with CMOs like below; some coherent >>>>>>>>> interconnects behave badly when you mix coherent and non-coherent access >>>>>>>>> (snoop filters get messed up). >>>>>>>>> >>>>>>>>> The solution is to either always have the addresses marked non-coherent >>>>>>>>> (like device memory, no-map carveouts), or if you really want to use >>>>>>>>> regular system memory allocated at runtime, then all cached mappings of >>>>>>>>> it need to be dropped, even the kernel logical address (area as painful >>>>>>>>> as that would be). >>>>>>> >>>>>>> Ouch :-( I wasn't aware about these potential interconnect issues. How >>>>>>> "real" is that? It seems that we aren't really hitting that today on >>>>>>> real devices. >>>>>>> >>>>>> >>>>>> Sadly there is at least one real device like this now (TI AM654). We >>>>>> spent some time working with the ARM interconnect spec designers to see >>>>>> if this was allowed behavior, final conclusion was mixing coherent and >>>>>> non-coherent accesses is never a good idea.. So we have been working to >>>>>> try to minimize any cases of mixed attributes [0], if a region is >>>>>> coherent then everyone in the system needs to treat it as such and >>>>>> vice-versa, even clever CMO that work on other systems wont save you >>>>>> here. :( >>>>>> >>>>>> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 >>>>>> > > "Never a good idea" - but I think it should still be well defined by > the ARMv8 ARM (Section B2.8). Does this apply to your system? > > "If the mismatched attributes for a memory location all assign the > same shareability attribute to a Location that has a cacheable > attribute, any loss of uniprocessor semantics, ordering, or coherency > within a shareability domain can be avoided by use of software cache > management" > > https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile > > If the cache is invalidated when switching between access types, > shouldn't the snoop filters get un-messed-up? > The details of the issue are this, our coherent interconnect (MSMC) has a snoop filter (for those following along at home it's a list of which cache lines are currently inside each connected master so snoop requests can be filtered for masters that wont care). When a "NoSnoop"(non-cached or non-shareable) transaction is received for a location from any master it assumes that location cannot be in the cache of *any* master (as the correct cache-line state transition a given core will take for that line is not defined by ARM spec), so it drops all records of that line. The only way to recover from this is for every master to invalidate the line and pick it back up again so the snoop filer can re-learn who really has it again. Invalidate on one core also doesn't propagate to the different cores as those are requests are also blocked by the now confused snoop filter, so each and every core must manually do it.. It behaves much more like later in ARMv8 ARM (Section B2.8): "If the mismatched attributes for a Location mean that multiple cacheable accesses to the Location might be made with different shareability attributes, then uniprocessor semantics, ordering, and coherency are guaranteed only if: • Each PE that accesses the Location with a cacheable attribute performs a clean and invalidate of the Location before and after accessing that Location. • A DMB barrier with scope that covers the full shareability of the accesses is placed between any accesses to the same memory Location that use different attributes." >>>>>> >>>>>>>>> >>>>>>>>>> ION buffer is allocated. >>>>>>>>>> >>>>>>>>>> //Camera device records video >>>>>>>>>> dma_buf_attach >>>>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>>>> >>>>>>>>> Why does the buffer need to be cleaned here? I just got through reading >>>>>>>>> the thread linked by Laura in the other reply. I do like +Brian's >>>>>>>> >>>>>>>> Actually +Brian this time :) >>>>>>>> >>>>>>>>> suggestion of tracking if the buffer has had CPU access since the last >>>>>>>>> time and only flushing the cache if it has. As unmapped heaps never get >>>>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my >>>>>>>>> problem. >>>>>>>>> >>>>>>>>>> [camera device writes to buffer] >>>>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) >>>>>>>>> >>>>>>>>> It doesn't know there will be any further CPU access, it could get freed >>>>>>>>> after this for all we know, the invalidate can be saved until the CPU >>>>>>>>> requests access again. >>>>>>> >>>>>>> We don't have any API to allow the invalidate to happen on CPU access >>>>>>> if all devices already detached. We need a struct device pointer to >>>>>>> give to the DMA API, otherwise on arm64 there'll be no invalidate. >>>>>>> >>>>>>> I had a chat with a few people internally after the previous >>>>>>> discussion with Liam. One suggestion was to use >>>>>>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least >>>>>>> one other device attached (guarantees that we can do an invalidate in >>>>>>> the future if begin_cpu_access is called). If the last device >>>>>>> detaches, do a sync then. >>>>>>> >>>>>>> Conversely, in map_dma_buf, we would track if there was any CPU access >>>>>>> and use/skip the sync appropriately. >>>>>>> >>>>>> >>>>>> Now that I think this all through I agree this patch is probably wrong. >>>>>> The real fix needs to be better handling in the dma_map_sg() to deal >>>>>> with the case of the memory not being mapped (what I'm dealing with for >>>>>> unmapped heaps), and for cases when the memory in question is not cached >>>>>> (Liam's issue I think). For both these cases the dma_map_sg() does the >>>>>> wrong thing. >>>>>> >>>>>>> I did start poking the code to check out how that would look, but then >>>>>>> Christmas happened and I'm still catching back up. >>>>>>> >>>>>>>>> >>>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>>>>>> the pipeline and Camera doesn't know the end of the use case) >>>>>>>>>> >>>>>>>>> >>>>>>>>> This seems like a broken use-case, I understand the desire to keep >>>>>>>>> everything as modular as possible and separate the steps, but at this >>>>>>>>> point no one owns this buffers backing memory, not the CPU or any >>>>>>>>> device. I would go as far as to say DMA-BUF should be free now to >>>>>>>>> de-allocate the backing storage if it wants, that way it could get ready >>>>>>>>> for the next attachment, which may change the required backing memory >>>>>>>>> completely. >>>>>>>>> >>>>>>>>> All devices should attach before the first mapping, and only let go >>>>>>>>> after the task is complete, otherwise this buffers data needs copied off >>>>>>>>> to a different location or the CPU needs to take ownership in-between. >>>>>>>>> >>>>>>> >>>>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF >>>>>>> implementations which actually do that? I hear it quoted a lot, >>>>>>> because that's what the docs say - but if the reality doesn't match >>>>>>> it, maybe we should change the docs. >>>>>>> >>>>>> >>>>>> Do you mean on the userspace side? I'm not sure, seems like Android >>>>>> might be doing this wrong from what I can gather. From kernel side if >>>>>> you mean the "de-allocate the backing storage", we will have some cases >>>>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF >>>>>> in ways not specified in the documentation. Changing the docs to force >>>>>> the backing memory to always be allocated breaks the central goal in >>>>>> having attach/map in DMA-BUF separate. > > Actually I meant in the kernel, in exporters. I haven't seen anyone > using the API as it was intended (defer allocation until first map, > migrate between different attachments, etc.). Mostly, backing storage > seems to get allocated at the point of export, and device mappings are > often held persistently (e.g. the DRM prime code maps the buffer at > import time, and keeps it mapped: drm_gem_prime_import_dev). > I haven't either, which is a shame as it allows for some really useful management strategies for shared memory resources. I'm working on one such case right now, maybe I'll get to be the first to upstream one :) > I wasn't aware that CPU access before first device access was > considered an abuse of the API - it seems like a valid thing to want > to do. > That's just it, I don't know if it is an abuse of API, I'm trying to get some clarity on that. If we do want to allow early CPU access then that seems to be in contrast to the idea of deferred allocation until first device map, what is supposed to backing the buffer if no devices have attached or mapped yet? Just some system memory followed by migration on the first attach to the proper backing? Seems too time wasteful to be have a valid use. Maybe it should be up to the exporter if early CPU access is allowed? I'm hoping someone with authority over the DMA-BUF framework can clarify original intentions here. >>>>>> >>>>>>>>>> //buffer is send down the pipeline >>>>>>>>>> >>>>>>>>>> // Usersapce software post processing occurs >>>>>>>>>> mmap buffer >>>>>>>>> >>>>>>>>> Perhaps the invalidate should happen here in mmap. >>>>>>>>> >>>>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no >>>>>>>>>> devices attached to buffer >>>>>>>>> >>>>>>>>> And that should be okay, mmap does the sync, and if no devices are >>>>>>>>> attached nothing could have changed the underlying memory in the >>>>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. >>>>>>> >>>>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. >>>>>>> Liam was saying that it's too painful for them to do that every time a >>>>>>> device unmaps - when in many cases (device->device, no CPU) it's not >>>>>>> needed. >>>>>> >>>>>> Invalidates are painless, at least compared to a real cache flush, just >>>>>> set the invalid bit vs actually writing out lines. I thought the issue >>>>>> was on the map side. >>>>>> >>>>> >>>>> Invalidates aren't painless for us because we have a coherent system cache >>>>> so clean lines get written out. >>>> >>>> That seems very broken, why would clean lines ever need to be written >>>> out, that defeats the whole point of having the invalidate separate from >>>> clean. How do you deal with stale cache lines? I guess in your case this >>>> is what forces you to have to use uncached memory for DMA-able memory. >>>> >>> >>> My understanding is that our ARM invalidate is a clean + invalidate, I had >>> concerns about the clean lines being written to the the system cache as >>> part of the 'clean', but the following 'invalidate' would take care of >>> actually invalidating the lines (so nothign broken). >>> But i am probably wrong on this and it is probably smart enough not to the >>> writing of the clean lines. >>> >> >> You are correct that for a lot of ARM cores "invalidate" is always a >> "clean + invalidate". At first I thought this was kinda silly as there >> is now no way to mark a dirty line invalid without it getting written >> out first, but if you think about it any dirty cache-line can be written >> out (cleaned) at anytime anyway, so this doesn't actually change system >> behavior. You should just not write to memory (make the line dirty) >> anything you don't want eventually written out. >> >> Point two, it's not just smart enough to not write-out clean lines, it >> is guaranteed not to write them out by the spec. Otherwise since >> cache-lines can be randomly filled if those same clean lines got written >> out on invalidate operations there would be no way to maintain coherency >> and things would be written over top each other all over the place. >> >>> But regardless, targets supporting a coherent system cache is a legitamate >>> configuration and an invalidate on this configuration does have to go to >>> the bus to invalidate the system cache (which isn't free) so I dont' think >>> you can make the assumption that invalidates are cheap so that it is okay >>> to do them (even if they are not needed) on every dma unmap. >>> >> >> Very true, CMOs need to be broadcast to other coherent masters on a >> coherent interconnect (and the interconnect itself if it has a cache as >> well (L3)), so not 100% free, but almost, just the infinitesimal cost of >> the cache tag check in hardware. If there are no non-coherent devices >> attached then the CMOs are no-ops, if there are then the data needs to >> be written out either way, doing it every access like is done with >> uncached memory (- any write combining) will blow away any saving made >> from the one less CMO. Either way you lose with uncached mappings of >> memory. If I'm wrong I would love to know. >> > > From what I understand, the current DMA APIs are not equipped to > handle having coherent and non-coherent devices attached at the same > time. The buffer is either in "CPU land" or "Device land", there's no > smaller granule of "Coherent Device land" or "Non-Coherent Device > land". > > I think if there's devices which are making coherent accesses, and > devices which are making non-coherent accesses, then we can't support > them being attached at the same time without some enhancements to the > APIs. > I think you are right, we only handle sync to/from the CPU out to "Device land". To sync from device to device I'm not sure there is anything right now, they all have to be able to talk to each other without any maintenance from the host CPU. This will probably lead to some interesting cases like in OpenVX where a key selling point is keeping the host out of the loop and let the remote devices do all the sharing between themselves. >>>>> And these invalidates can occur on fairly large buffers. >>>>> >>>>> That is why we haven't went with using cached ION memory and "tracking CPU >>>>> access" because it only solves half the problem, ie there isn't a way to >>>>> safely skip the invalidate (because we can't read the future). >>>>> Our solution was to go with uncached ION memory (when possible), but as >>>>> you can see in other discussions upstream support for uncached memory has >>>>> its own issues. >>>>> > > @Liam, in your problematic use-cases, are both devices detached when > the buffer moves between them? > > 1) dev 1 map, access, unmap > 2) dev 1 detach > 3) (maybe) CPU access > 4) dev 2 attach > 5) dev 2 map, access > > I still think a pretty pragmatic solution is to use > DMA_ATTR_SKIP_CPU_SYNC until the last device detaches. That won't work > if your access sequence looks like above... > > ...however, if your sequence looks like above, then you probably need > to keep at least one of the devices attached anyway. Otherwise, per > the API, the buffer could get migrated after 2)/before 5). That will > surely hurt a lot more than an invalidate. > >>>> >>>> Sounds like you need to fix upstream support then, finding a way to drop >>>> all cacheable mappings of memory you want to make uncached mappings for >>>> seems to be the only solution. >>>> >>> >>> I think we can probably agree that there woudln't be a good way to remove >>> cached mappings without causing an unacceptable performance degradation >>> since it would fragment all the nice 1GB kernel mappings we have. >>> >>> So I am trying to find an alternative solution. >>> >> >> I'm not sure there is a better solution. How hard is this solution to >> implement anyway? The kernel already has to make gaps and cut up that >> nice 1GB mapping when you make a reserved memory space in the lowmem >> area, so all the logic is probably already implemented. Just need to >> allow it to be hooked into from Ion when doing doing the uncached mappings. >> > > I haven't looked recently, but I'm not sure the early memblock code > can be reused as-is at runtime. I seem to remember it makes a bunch of > assumptions about the fact that it's running "early". > > If CPU uncached mappings of normal system memory is really the way > forward, I could envisage a heap which maintains a pool of chunks of > memory which it removed from the kernel mapping. The pool could grow > (remove more pages from the kernel mapping)/shrink (add them back to > the kernel mapping) as needed. > > John Reitan implemented a compound-page heap, which used compaction to > get a pool of 2MB contiguous pages. Something like that would at least > prevent needing full 4kB granularity when removing things from the > kernel mapping. > > Even better, could it somehow be restricted to a region which is > already fragmented? (e.g. the one which was used for the default CMA > heap) > > Thanks, > -Brian > >>>>>>> >>>>>>>>> >>>>>>>>>> [CPU reads/writes to the buffer] >>>>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no >>>>>>>>>> devices attached to buffer >>>>>>>>>> munmap buffer >>>>>>>>>> >>>>>>>>>> //buffer is send down the pipeline >>>>>>>>>> // Buffer is send to video device (who does compression of raw data) and >>>>>>>>>> writes to a file >>>>>>>>>> dma_buf_attach >>>>>>>>>> dma_map_attachment (buffer needs to be cleaned) >>>>>>>>>> [video device writes to buffer] >>>>>>>>>> dma_buf_unmap_attachment >>>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down >>>>>>>>>> the pipeline and Video doesn't know the end of the use case) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU >>>>>>>>>>>> access then there is no requirement (that I am aware of) for you to call >>>>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this >>>>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance >>>>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If I am not doing any CPU access then why do I need CPU cache >>>>>>>>>>> maintenance on the buffer? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Because ION no longer provides DMA ready memory. >>>>>>>>>> Take the above example. >>>>>>>>>> >>>>>>>>>> ION allocates memory from buddy allocator and requests zeroing. >>>>>>>>>> Zeros are written to the cache. >>>>>>>>>> >>>>>>>>>> You pass the buffer to the camera device which is not IO-coherent. >>>>>>>>>> The camera devices writes directly to the buffer in DDR. >>>>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is >>>>>>>>>> evicted from the cache, this zero overwrites data the camera device has >>>>>>>>>> written which corrupts your data. >>>>>>>>>> >>>>>>>>> >>>>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO >>>>>>>>> for CPU access at the time of zeroing. >>>>>>>>> >>>>>>> >>>>>>> Actually that should be at the point of the first non-coherent device >>>>>>> mapping the buffer right? No point in doing CMO if the future accesses >>>>>>> are coherent. >>>>>> >>>>>> I see your point, as long as the zeroing is guaranteed to be the first >>>>>> access to this buffer then it should be safe. >>>>>> >>>>>> Andrew >>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> -Brian >>>>>>> >>>>>>>>> Andrew >>>>>>>>> >>>>>>>>>> Liam >>>>>>>>>> >>>>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>>>>>>> a Linux Foundation Collaborative Project >>>>>>>>>> >>>>>> >>>>> >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>>>> a Linux Foundation Collaborative Project >>>>> >>>> >>> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, >>> a Linux Foundation Collaborative Project >>>
Hello everyone,
Sincere apologies for chiming in a bit late here, but was off due to
some health issues.
Also, adding Daniel Vetter to the mix, since he has been one of the
core guys who shaped up dma-buf as it is today.
On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <afd@ti.com> wrote:
>
> On 1/21/19 5:22 AM, Brian Starkey wrote:
> > Hi,
> >
> > Sorry for being a bit sporadic on this. I was out travelling last week
> > with little time for email.
> >
> > On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote:
> >> On 1/17/19 7:11 PM, Liam Mark wrote:
> >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote:
> >>>
> >>>> On 1/16/19 4:54 PM, Liam Mark wrote:
> >>>>> On Wed, 16 Jan 2019, Andrew F. Davis wrote:
> >>>>>
> >>>>>> On 1/16/19 9:19 AM, Brian Starkey wrote:
> >>>>>>> Hi :-)
> >>>>>>>
> >>>>>>> On Tue, Jan 15, 2019 at 12:40:16PM
-0600, Andrew F. Davis wrote:
> >>>>>>>> On 1/15/19 12:38 PM, Andrew F.
Davis wrote:
> >>>>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote:
> >>>>>>>>>> On Tue, 15 Jan 2019,
Andrew F. Davis wrote:
> >>>>>>>>>>
> >>>>>>>>>>> On 1/14/19 11:13 AM,
Liam Mark wrote:
> >>>>>>>>>>>> On Fri, 11 Jan
2019, Andrew F. Davis wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Buffers may
not be mapped from the CPU so skip cache maintenance here.
> >>>>>>>>>>>>> Accesses
from the CPU to a cached heap should be bracketed with
> >>>>>>>>>>>>>
{begin,end}_cpu_access calls so maintenance should not be needed
anyway.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
Signed-off-by: Andrew F. Davis <afd@ti.com>
> >>>>>>>>>>>>> ---
> >>>>>>>>>>>>>
drivers/staging/android/ion/ion.c | 7 ++++---
> >>>>>>>>>>>>> 1 file
changed, 4 insertions(+), 3 deletions(-)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> diff --git
a/drivers/staging/android/ion/ion.c
b/drivers/staging/android/ion/ion.c
> >>>>>>>>>>>>> index
14e48f6eb734..09cb5a8e2b09 100644
> >>>>>>>>>>>>> ---
a/drivers/staging/android/ion/ion.c
> >>>>>>>>>>>>> +++
b/drivers/staging/android/ion/ion.c
> >>>>>>>>>>>>> @@ -261,8
+261,8 @@ static struct sg_table *ion_map_dma_buf(struct
dma_buf_attachment *attachment,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> table = a->table;
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - if
(!dma_map_sg(attachment->dev, table->sgl, table->nents,
> >>>>>>>>>>>>> -
direction))
> >>>>>>>>>>>>> + if
(!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents,
> >>>>>>>>>>>>> +
direction, DMA_ATTR_SKIP_CPU_SYNC))
> >>>>>>>>>>>>
> >>>>>>>>>>>> Unfortunately I
don't think you can do this for a couple reasons.
> >>>>>>>>>>>> You can't rely
on {begin,end}_cpu_access calls to do cache maintenance.
> >>>>>>>>>>>> If the calls to
{begin,end}_cpu_access were made before the call to
> >>>>>>>>>>>> dma_buf_attach
then there won't have been a device attached so the calls
> >>>>>>>>>>>> to
{begin,end}_cpu_access won't have done any cache maintenance.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> That should be okay
though, if you have no attachments (or all
> >>>>>>>>>>> attachments are
IO-coherent) then there is no need for cache
> >>>>>>>>>>> maintenance. Unless
you mean a sequence where a non-io-coherent device
> >>>>>>>>>>> is attached later
after data has already been written. Does that
> >>>>>>>>>>> sequence need supporting?
> >>>>>>>>>>
> >>>>>>>>>> Yes, but also I think
there are cases where CPU access can happen before
> >>>>>>>>>> in Android, but I will
focus on later for now.
> >>>>>>>>>>
> >>>>>>>>>>> DMA-BUF doesn't have
to allocate the backing
> >>>>>>>>>>> memory until
map_dma_buf() time, and that should only happen after all
> >>>>>>>>>>> the devices have
attached so it can know where to put the buffer. So we
> >>>>>>>>>>> shouldn't expect any
CPU access to buffers before all the devices are
> >>>>>>>>>>> attached and mapped, right?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Here is an example where
CPU access can happen later in Android.
> >>>>>>>>>>
> >>>>>>>>>> Camera device records
video -> software post processing -> video device
> >>>>>>>>>> (who does compression of
raw data) and writes to a file
> >>>>>>>>>>
> >>>>>>>>>> In this example assume
the buffer is cached and the devices are not
> >>>>>>>>>> IO-coherent (quite common).
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> This is the start of the
problem, having cached mappings of memory that
> >>>>>>>>> is also being accessed
non-coherently is going to cause issues one way
> >>>>>>>>> or another. On top of the
speculative cache fills that have to be
> >>>>>>>>> constantly fought back
against with CMOs like below; some coherent
> >>>>>>>>> interconnects behave badly
when you mix coherent and non-coherent access
> >>>>>>>>> (snoop filters get messed up).
> >>>>>>>>>
> >>>>>>>>> The solution is to either
always have the addresses marked non-coherent
> >>>>>>>>> (like device memory, no-map
carveouts), or if you really want to use
> >>>>>>>>> regular system memory
allocated at runtime, then all cached mappings of
> >>>>>>>>> it need to be dropped, even
the kernel logical address (area as painful
> >>>>>>>>> as that would be).
> >>>>>>>
> >>>>>>> Ouch :-( I wasn't aware about these
potential interconnect issues. How
> >>>>>>> "real" is that? It seems that we
aren't really hitting that today on
> >>>>>>> real devices.
> >>>>>>>
> >>>>>>
> >>>>>> Sadly there is at least one real device
like this now (TI AM654). We
> >>>>>> spent some time working with the ARM
interconnect spec designers to see
> >>>>>> if this was allowed behavior, final
conclusion was mixing coherent and
> >>>>>> non-coherent accesses is never a good
idea.. So we have been working to
> >>>>>> try to minimize any cases of mixed
attributes [0], if a region is
> >>>>>> coherent then everyone in the system
needs to treat it as such and
> >>>>>> vice-versa, even clever CMO that work on
other systems wont save you
> >>>>>> here. :(
> >>>>>>
> >>>>>> [0]
https://github.com/ARM-software/arm-trusted-firmware/pull/1553
> >>>>>>
> >
> > "Never a good idea" - but I think it should still be well defined by
> > the ARMv8 ARM (Section B2.8). Does this apply to your system?
> >
> > "If the mismatched attributes for a memory location all assign the
> > same shareability attribute to a Location that has a cacheable
> > attribute, any loss of uniprocessor semantics, ordering, or coherency
> > within a shareability domain can be avoided by use of software cache
> > management"
> >
> > https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile
> >
> > If the cache is invalidated when switching between access types,
> > shouldn't the snoop filters get un-messed-up?
> >
>
> The details of the issue are this, our coherent interconnect (MSMC) has
> a snoop filter (for those following along at home it's a list of which
> cache lines are currently inside each connected master so snoop requests
> can be filtered for masters that wont care). When a "NoSnoop"(non-cached
> or non-shareable) transaction is received for a location from any master
> it assumes that location cannot be in the cache of *any* master (as the
> correct cache-line state transition a given core will take for that line
> is not defined by ARM spec), so it drops all records of that line. The
> only way to recover from this is for every master to invalidate the line
> and pick it back up again so the snoop filer can re-learn who really has
> it again. Invalidate on one core also doesn't propagate to the different
> cores as those are requests are also blocked by the now confused snoop
> filter, so each and every core must manually do it..
>
> It behaves much more like later in ARMv8 ARM (Section B2.8):
>
> "If the mismatched attributes for a Location mean that multiple
> cacheable accesses to the Location might be made with different
> shareability attributes, then uniprocessor semantics, ordering, and
> coherency are guaranteed only if:
> • Each PE that accesses the Location with a cacheable attribute performs
> a clean and invalidate of the Location before and after accessing that
> Location.
> • A DMB barrier with scope that covers the full shareability of the
> accesses is placed between any accesses to the same memory Location that
> use different attributes."
>
> >>>>>>
> >>>>>>>>>
> >>>>>>>>>> ION buffer is allocated.
> >>>>>>>>>>
> >>>>>>>>>> //Camera device records video
> >>>>>>>>>> dma_buf_attach
> >>>>>>>>>> dma_map_attachment
(buffer needs to be cleaned)
> >>>>>>>>>
> >>>>>>>>> Why does the buffer need to
be cleaned here? I just got through reading
> >>>>>>>>> the thread linked by Laura
in the other reply. I do like +Brian's
> >>>>>>>>
> >>>>>>>> Actually +Brian this time :)
> >>>>>>>>
> >>>>>>>>> suggestion of tracking if
the buffer has had CPU access since the last
> >>>>>>>>> time and only flushing the
cache if it has. As unmapped heaps never get
> >>>>>>>>> CPU mapped this would never
be the case for unmapped heaps, it solves my
> >>>>>>>>> problem.
> >>>>>>>>>
> >>>>>>>>>> [camera device writes to buffer]
> >>>>>>>>>> dma_buf_unmap_attachment
(buffer needs to be invalidated)
> >>>>>>>>>
> >>>>>>>>> It doesn't know there will
be any further CPU access, it could get freed
> >>>>>>>>> after this for all we know,
the invalidate can be saved until the CPU
> >>>>>>>>> requests access again.
> >>>>>>>
> >>>>>>> We don't have any API to allow the
invalidate to happen on CPU access
> >>>>>>> if all devices already detached. We
need a struct device pointer to
> >>>>>>> give to the DMA API, otherwise on
arm64 there'll be no invalidate.
> >>>>>>>
> >>>>>>> I had a chat with a few people
internally after the previous
> >>>>>>> discussion with Liam. One suggestion
was to use
> >>>>>>> DMA_ATTR_SKIP_CPU_SYNC in
unmap_dma_buf, but only if there's at least
> >>>>>>> one other device attached
(guarantees that we can do an invalidate in
> >>>>>>> the future if begin_cpu_access is
called). If the last device
> >>>>>>> detaches, do a sync then.
> >>>>>>>
> >>>>>>> Conversely, in map_dma_buf, we would
track if there was any CPU access
> >>>>>>> and use/skip the sync appropriately.
> >>>>>>>
> >>>>>>
> >>>>>> Now that I think this all through I
agree this patch is probably wrong.
> >>>>>> The real fix needs to be better handling
in the dma_map_sg() to deal
> >>>>>> with the case of the memory not being
mapped (what I'm dealing with for
> >>>>>> unmapped heaps), and for cases when the
memory in question is not cached
> >>>>>> (Liam's issue I think). For both these
cases the dma_map_sg() does the
> >>>>>> wrong thing.
> >>>>>>
> >>>>>>> I did start poking the code to check
out how that would look, but then
> >>>>>>> Christmas happened and I'm still
catching back up.
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>>> dma_buf_detach (device
cannot stay attached because it is being sent down
> >>>>>>>>>> the pipeline and Camera
doesn't know the end of the use case)
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> This seems like a broken
use-case, I understand the desire to keep
> >>>>>>>>> everything as modular as
possible and separate the steps, but at this
> >>>>>>>>> point no one owns this
buffers backing memory, not the CPU or any
> >>>>>>>>> device. I would go as far as
to say DMA-BUF should be free now to
> >>>>>>>>> de-allocate the backing
storage if it wants, that way it could get ready
> >>>>>>>>> for the next attachment,
which may change the required backing memory
> >>>>>>>>> completely.
> >>>>>>>>>
> >>>>>>>>> All devices should attach
before the first mapping, and only let go
> >>>>>>>>> after the task is complete,
otherwise this buffers data needs copied off
> >>>>>>>>> to a different location or
the CPU needs to take ownership in-between.
> >>>>>>>>>
> >>>>>>>
> >>>>>>> Yeah.. that's certainly the theory.
Are there any DMA-BUF
> >>>>>>> implementations which actually do
that? I hear it quoted a lot,
> >>>>>>> because that's what the docs say -
but if the reality doesn't match
> >>>>>>> it, maybe we should change the docs.
> >>>>>>>
> >>>>>>
> >>>>>> Do you mean on the userspace side? I'm
not sure, seems like Android
> >>>>>> might be doing this wrong from what I
can gather. From kernel side if
> >>>>>> you mean the "de-allocate the backing
storage", we will have some cases
> >>>>>> like this soon, so I want to make sure
userspace is not abusing DMA-BUF
> >>>>>> in ways not specified in the
documentation. Changing the docs to force
> >>>>>> the backing memory to always be
allocated breaks the central goal in
> >>>>>> having attach/map in DMA-BUF separate.
> >
> > Actually I meant in the kernel, in exporters. I haven't seen anyone
> > using the API as it was intended (defer allocation until first map,
> > migrate between different attachments, etc.). Mostly, backing storage
> > seems to get allocated at the point of export, and device mappings are
> > often held persistently (e.g. the DRM prime code maps the buffer at
> > import time, and keeps it mapped: drm_gem_prime_import_dev).
> >
>
So I suppose some clarification on the 'intended use' part of dma-buf
about deferred allocation is due, so here it is: (Daniel, please feel
free to chime in with your opinion here)
- dma-buf was of course designed as a framework to help intelligent
exporters to defer allocation until first map, and be able to migrate
backing storage if required etc. At the same time, it is not a
_requirement_ from any exporter, so exporters so far have just used it
as a convenient mechanism for zero-copy.
- ION is one of the few dma-buf exporters in kernel, which satisfies a
certain set of expectations from its users.
> I haven't either, which is a shame as it allows for some really useful
> management strategies for shared memory resources. I'm working on one
> such case right now, maybe I'll get to be the first to upstream one :)
>
That will be a really good thing! Though perhaps we ought to think if
for what you're trying to do, is ION the right place, or should you
have a device-specific exporter, available to users via dma-buf apis?
> > I wasn't aware that CPU access before first device access was
> > considered an abuse of the API - it seems like a valid thing to want
> > to do.
> >
>
> That's just it, I don't know if it is an abuse of API, I'm trying to get
> some clarity on that. If we do want to allow early CPU access then that
> seems to be in contrast to the idea of deferred allocation until first
> device map, what is supposed to backing the buffer if no devices have
> attached or mapped yet? Just some system memory followed by migration on
> the first attach to the proper backing? Seems too time wasteful to be
> have a valid use.
>
> Maybe it should be up to the exporter if early CPU access is allowed?
>
> I'm hoping someone with authority over the DMA-BUF framework can clarify
> original intentions here.
I don't think dma-buf as a framework stops early CPU access, and the
exporter can definitely decide on that by implementing
begin_cpu_access / end_cpu_access operations to not allow early CPU
access, if it so desires.
</afd@ti.com></afd@ti.com>
>
> >>>>>>
> >>>>>>>>>> //buffer is send down the pipeline
> >>>>>>>>>>
> >>>>>>>>>> // Usersapce software post processing occurs
> >>>>>>>>>> mmap buffer
> >>>>>>>>>
> >>>>>>>>> Perhaps the invalidate should happen here in mmap.
> >>>>>>>>>
> >>>>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no
> >>>>>>>>>> devices attached to buffer
> >>>>>>>>>
> >>>>>>>>> And that should be okay, mmap does the sync, and if no devices are
> >>>>>>>>> attached nothing could have changed the underlying memory in the
> >>>>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are.
> >>>>>>>
> >>>>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf.
> >>>>>>> Liam was saying that it's too painful for them to do that every time a
> >>>>>>> device unmaps - when in many cases (device->device, no CPU) it's not
> >>>>>>> needed.
> >>>>>>
> >>>>>> Invalidates are painless, at least compared to a real cache flush, just
> >>>>>> set the invalid bit vs actually writing out lines. I thought the issue
> >>>>>> was on the map side.
> >>>>>>
> >>>>>
> >>>>> Invalidates aren't painless for us because we have a coherent system cache
> >>>>> so clean lines get written out.
> >>>>
> >>>> That seems very broken, why would clean lines ever need to be written
> >>>> out, that defeats the whole point of having the invalidate separate from
> >>>> clean. How do you deal with stale cache lines? I guess in your case this
> >>>> is what forces you to have to use uncached memory for DMA-able memory.
> >>>>
> >>>
> >>> My understanding is that our ARM invalidate is a clean + invalidate, I had
> >>> concerns about the clean lines being written to the the system cache as
> >>> part of the 'clean', but the following 'invalidate' would take care of
> >>> actually invalidating the lines (so nothign broken).
> >>> But i am probably wrong on this and it is probably smart enough not to the
> >>> writing of the clean lines.
> >>>
> >>
> >> You are correct that for a lot of ARM cores "invalidate" is always a
> >> "clean + invalidate". At first I thought this was kinda silly as there
> >> is now no way to mark a dirty line invalid without it getting written
> >> out first, but if you think about it any dirty cache-line can be written
> >> out (cleaned) at anytime anyway, so this doesn't actually change system
> >> behavior. You should just not write to memory (make the line dirty)
> >> anything you don't want eventually written out.
> >>
> >> Point two, it's not just smart enough to not write-out clean lines, it
> >> is guaranteed not to write them out by the spec. Otherwise since
> >> cache-lines can be randomly filled if those same clean lines got written
> >> out on invalidate operations there would be no way to maintain coherency
> >> and things would be written over top each other all over the place.
> >>
> >>> But regardless, targets supporting a coherent system cache is a legitamate
> >>> configuration and an invalidate on this configuration does have to go to
> >>> the bus to invalidate the system cache (which isn't free) so I dont' think
> >>> you can make the assumption that invalidates are cheap so that it is okay
> >>> to do them (even if they are not needed) on every dma unmap.
> >>>
> >>
> >> Very true, CMOs need to be broadcast to other coherent masters on a
> >> coherent interconnect (and the interconnect itself if it has a cache as
> >> well (L3)), so not 100% free, but almost, just the infinitesimal cost of
> >> the cache tag check in hardware. If there are no non-coherent devices
> >> attached then the CMOs are no-ops, if there are then the data needs to
> >> be written out either way, doing it every access like is done with
> >> uncached memory (- any write combining) will blow away any saving made
> >> from the one less CMO. Either way you lose with uncached mappings of
> >> memory. If I'm wrong I would love to know.
> >>
> >
> > From what I understand, the current DMA APIs are not equipped to
> > handle having coherent and non-coherent devices attached at the same
> > time. The buffer is either in "CPU land" or "Device land", there's no
> > smaller granule of "Coherent Device land" or "Non-Coherent Device
> > land".
> >
> > I think if there's devices which are making coherent accesses, and
> > devices which are making non-coherent accesses, then we can't support
> > them being attached at the same time without some enhancements to the
> > APIs.
> >
>
> I think you are right, we only handle sync to/from the CPU out to
> "Device land". To sync from device to device I'm not sure there is
> anything right now, they all have to be able to talk to each other
> without any maintenance from the host CPU.
>
> This will probably lead to some interesting cases like in OpenVX where a
> key selling point is keeping the host out of the loop and let the remote
> devices do all the sharing between themselves.
>
> >>>>> And these invalidates can occur on fairly large buffers.
> >>>>>
> >>>>> That is why we haven't went with using cached ION memory and "tracking CPU
> >>>>> access" because it only solves half the problem, ie there isn't a way to
> >>>>> safely skip the invalidate (because we can't read the future).
> >>>>> Our solution was to go with uncached ION memory (when possible), but as
> >>>>> you can see in other discussions upstream support for uncached memory has
> >>>>> its own issues.
> >>>>>
> >
> > @Liam, in your problematic use-cases, are both devices detached when
> > the buffer moves between them?
> >
> > 1) dev 1 map, access, unmap
> > 2) dev 1 detach
> > 3) (maybe) CPU access
> > 4) dev 2 attach
> > 5) dev 2 map, access
> >
> > I still think a pretty pragmatic solution is to use
> > DMA_ATTR_SKIP_CPU_SYNC until the last device detaches. That won't work
> > if your access sequence looks like above...
> >
> > ...however, if your sequence looks like above, then you probably need
> > to keep at least one of the devices attached anyway. Otherwise, per
> > the API, the buffer could get migrated after 2)/before 5). That will
> > surely hurt a lot more than an invalidate.
> >
> >>>>
> >>>> Sounds like you need to fix upstream support then, finding a way to drop
> >>>> all cacheable mappings of memory you want to make uncached mappings for
> >>>> seems to be the only solution.
> >>>>
> >>>
> >>> I think we can probably agree that there woudln't be a good way to remove
> >>> cached mappings without causing an unacceptable performance degradation
> >>> since it would fragment all the nice 1GB kernel mappings we have.
> >>>
> >>> So I am trying to find an alternative solution.
> >>>
> >>
> >> I'm not sure there is a better solution. How hard is this solution to
> >> implement anyway? The kernel already has to make gaps and cut up that
> >> nice 1GB mapping when you make a reserved memory space in the lowmem
> >> area, so all the logic is probably already implemented. Just need to
> >> allow it to be hooked into from Ion when doing doing the uncached mappings.
> >>
> >
> > I haven't looked recently, but I'm not sure the early memblock code
> > can be reused as-is at runtime. I seem to remember it makes a bunch of
> > assumptions about the fact that it's running "early".
> >
> > If CPU uncached mappings of normal system memory is really the way
> > forward, I could envisage a heap which maintains a pool of chunks of
> > memory which it removed from the kernel mapping. The pool could grow
> > (remove more pages from the kernel mapping)/shrink (add them back to
> > the kernel mapping) as needed.
> >
> > John Reitan implemented a compound-page heap, which used compaction to
> > get a pool of 2MB contiguous pages. Something like that would at least
> > prevent needing full 4kB granularity when removing things from the
> > kernel mapping.
> >
> > Even better, could it somehow be restricted to a region which is
> > already fragmented? (e.g. the one which was used for the default CMA
> > heap)
> >
> > Thanks,
> > -Brian
> >
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>>> [CPU reads/writes to the buffer]
> >>>>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no
> >>>>>>>>>> devices attached to buffer
> >>>>>>>>>> munmap buffer
> >>>>>>>>>>
> >>>>>>>>>> //buffer is send down the pipeline
> >>>>>>>>>> // Buffer is send to video device (who does compression of raw data) and
> >>>>>>>>>> writes to a file
> >>>>>>>>>> dma_buf_attach
> >>>>>>>>>> dma_map_attachment (buffer needs to be cleaned)
> >>>>>>>>>> [video device writes to buffer]
> >>>>>>>>>> dma_buf_unmap_attachment
> >>>>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down
> >>>>>>>>>> the pipeline and Video doesn't know the end of the use case)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU
> >>>>>>>>>>>> access then there is no requirement (that I am aware of) for you to call
> >>>>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this
> >>>>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance
> >>>>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> If I am not doing any CPU access then why do I need CPU cache
> >>>>>>>>>>> maintenance on the buffer?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Because ION no longer provides DMA ready memory.
> >>>>>>>>>> Take the above example.
> >>>>>>>>>>
> >>>>>>>>>> ION allocates memory from buddy allocator and requests zeroing.
> >>>>>>>>>> Zeros are written to the cache.
> >>>>>>>>>>
> >>>>>>>>>> You pass the buffer to the camera device which is not IO-coherent.
> >>>>>>>>>> The camera devices writes directly to the buffer in DDR.
> >>>>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is
> >>>>>>>>>> evicted from the cache, this zero overwrites data the camera device has
> >>>>>>>>>> written which corrupts your data.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO
> >>>>>>>>> for CPU access at the time of zeroing.
> >>>>>>>>>
> >>>>>>>
> >>>>>>> Actually that should be at the point of the first non-coherent device
> >>>>>>> mapping the buffer right? No point in doing CMO if the future accesses
> >>>>>>> are coherent.
> >>>>>>
> >>>>>> I see your point, as long as the zeroing is guaranteed to be the first
> >>>>>> access to this buffer then it should be safe.
> >>>>>>
> >>>>>> Andrew
> >>>>>>
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> -Brian
> >>>>>>>
> >>>>>>>>> Andrew
> >>>>>>>>>
> >>>>>>>>>> Liam
> >>>>>>>>>>
> >>>>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> >>>>>>>>>> a Linux Foundation Collaborative Project
> >>>>>>>>>>
> >>>>>>
> >>>>>
> >>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> >>>>> a Linux Foundation Collaborative Project
> >>>>>
> >>>>
> >>>
> >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
> >>> a Linux Foundation Collaborative Project
> >>>
Best,
Sumit.
On Mon, 21 Jan 2019, Brian Starkey wrote: > Hi, > > Sorry for being a bit sporadic on this. I was out travelling last week > with little time for email. > > On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote: > > On 1/17/19 7:11 PM, Liam Mark wrote: > > > On Thu, 17 Jan 2019, Andrew F. Davis wrote: > > > > > >> On 1/16/19 4:54 PM, Liam Mark wrote: > > >>> On Wed, 16 Jan 2019, Andrew F. Davis wrote: > > >>> > > >>>> On 1/16/19 9:19 AM, Brian Starkey wrote: > > >>>>> Hi :-) > > >>>>> > > >>>>> On Tue, Jan 15, 2019 at 12:40:16PM -0600, Andrew F. Davis wrote: > > >>>>>> On 1/15/19 12:38 PM, Andrew F. Davis wrote: > > >>>>>>> On 1/15/19 11:45 AM, Liam Mark wrote: > > >>>>>>>> On Tue, 15 Jan 2019, Andrew F. Davis wrote: > > >>>>>>>> > > >>>>>>>>> On 1/14/19 11:13 AM, Liam Mark wrote: > > >>>>>>>>>> On Fri, 11 Jan 2019, Andrew F. Davis wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Buffers may not be mapped from the CPU so skip cache maintenance here. > > >>>>>>>>>>> Accesses from the CPU to a cached heap should be bracketed with > > >>>>>>>>>>> {begin,end}_cpu_access calls so maintenance should not be needed anyway. > > >>>>>>>>>>> > > >>>>>>>>>>> Signed-off-by: Andrew F. Davis <afd@ti.com> > > >>>>>>>>>>> --- > > >>>>>>>>>>> drivers/staging/android/ion/ion.c | 7 ++++--- > > >>>>>>>>>>> 1 file changed, 4 insertions(+), 3 deletions(-) > > >>>>>>>>>>> > > >>>>>>>>>>> diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c > > >>>>>>>>>>> index 14e48f6eb734..09cb5a8e2b09 100644 > > >>>>>>>>>>> --- a/drivers/staging/android/ion/ion.c > > >>>>>>>>>>> +++ b/drivers/staging/android/ion/ion.c > > >>>>>>>>>>> @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, > > >>>>>>>>>>> > > >>>>>>>>>>> table = a->table; > > >>>>>>>>>>> > > >>>>>>>>>>> - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, > > >>>>>>>>>>> - direction)) > > >>>>>>>>>>> + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, > > >>>>>>>>>>> + direction, DMA_ATTR_SKIP_CPU_SYNC)) > > >>>>>>>>>> > > >>>>>>>>>> Unfortunately I don't think you can do this for a couple reasons. > > >>>>>>>>>> You can't rely on {begin,end}_cpu_access calls to do cache maintenance. > > >>>>>>>>>> If the calls to {begin,end}_cpu_access were made before the call to > > >>>>>>>>>> dma_buf_attach then there won't have been a device attached so the calls > > >>>>>>>>>> to {begin,end}_cpu_access won't have done any cache maintenance. > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> That should be okay though, if you have no attachments (or all > > >>>>>>>>> attachments are IO-coherent) then there is no need for cache > > >>>>>>>>> maintenance. Unless you mean a sequence where a non-io-coherent device > > >>>>>>>>> is attached later after data has already been written. Does that > > >>>>>>>>> sequence need supporting? > > >>>>>>>> > > >>>>>>>> Yes, but also I think there are cases where CPU access can happen before > > >>>>>>>> in Android, but I will focus on later for now. > > >>>>>>>> > > >>>>>>>>> DMA-BUF doesn't have to allocate the backing > > >>>>>>>>> memory until map_dma_buf() time, and that should only happen after all > > >>>>>>>>> the devices have attached so it can know where to put the buffer. So we > > >>>>>>>>> shouldn't expect any CPU access to buffers before all the devices are > > >>>>>>>>> attached and mapped, right? > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> Here is an example where CPU access can happen later in Android. > > >>>>>>>> > > >>>>>>>> Camera device records video -> software post processing -> video device > > >>>>>>>> (who does compression of raw data) and writes to a file > > >>>>>>>> > > >>>>>>>> In this example assume the buffer is cached and the devices are not > > >>>>>>>> IO-coherent (quite common). > > >>>>>>>> > > >>>>>>> > > >>>>>>> This is the start of the problem, having cached mappings of memory that > > >>>>>>> is also being accessed non-coherently is going to cause issues one way > > >>>>>>> or another. On top of the speculative cache fills that have to be > > >>>>>>> constantly fought back against with CMOs like below; some coherent > > >>>>>>> interconnects behave badly when you mix coherent and non-coherent access > > >>>>>>> (snoop filters get messed up). > > >>>>>>> > > >>>>>>> The solution is to either always have the addresses marked non-coherent > > >>>>>>> (like device memory, no-map carveouts), or if you really want to use > > >>>>>>> regular system memory allocated at runtime, then all cached mappings of > > >>>>>>> it need to be dropped, even the kernel logical address (area as painful > > >>>>>>> as that would be). > > >>>>> > > >>>>> Ouch :-( I wasn't aware about these potential interconnect issues. How > > >>>>> "real" is that? It seems that we aren't really hitting that today on > > >>>>> real devices. > > >>>>> > > >>>> > > >>>> Sadly there is at least one real device like this now (TI AM654). We > > >>>> spent some time working with the ARM interconnect spec designers to see > > >>>> if this was allowed behavior, final conclusion was mixing coherent and > > >>>> non-coherent accesses is never a good idea.. So we have been working to > > >>>> try to minimize any cases of mixed attributes [0], if a region is > > >>>> coherent then everyone in the system needs to treat it as such and > > >>>> vice-versa, even clever CMO that work on other systems wont save you > > >>>> here. :( > > >>>> > > >>>> [0] https://github.com/ARM-software/arm-trusted-firmware/pull/1553 > > >>>> > > "Never a good idea" - but I think it should still be well defined by > the ARMv8 ARM (Section B2.8). Does this apply to your system? > > "If the mismatched attributes for a memory location all assign the > same shareability attribute to a Location that has a cacheable > attribute, any loss of uniprocessor semantics, ordering, or coherency > within a shareability domain can be avoided by use of software cache > management" > > https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile > > If the cache is invalidated when switching between access types, > shouldn't the snoop filters get un-messed-up? > > > >>>> > > >>>>>>> > > >>>>>>>> ION buffer is allocated. > > >>>>>>>> > > >>>>>>>> //Camera device records video > > >>>>>>>> dma_buf_attach > > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > > >>>>>>> > > >>>>>>> Why does the buffer need to be cleaned here? I just got through reading > > >>>>>>> the thread linked by Laura in the other reply. I do like +Brian's > > >>>>>> > > >>>>>> Actually +Brian this time :) > > >>>>>> > > >>>>>>> suggestion of tracking if the buffer has had CPU access since the last > > >>>>>>> time and only flushing the cache if it has. As unmapped heaps never get > > >>>>>>> CPU mapped this would never be the case for unmapped heaps, it solves my > > >>>>>>> problem. > > >>>>>>> > > >>>>>>>> [camera device writes to buffer] > > >>>>>>>> dma_buf_unmap_attachment (buffer needs to be invalidated) > > >>>>>>> > > >>>>>>> It doesn't know there will be any further CPU access, it could get freed > > >>>>>>> after this for all we know, the invalidate can be saved until the CPU > > >>>>>>> requests access again. > > >>>>> > > >>>>> We don't have any API to allow the invalidate to happen on CPU access > > >>>>> if all devices already detached. We need a struct device pointer to > > >>>>> give to the DMA API, otherwise on arm64 there'll be no invalidate. > > >>>>> > > >>>>> I had a chat with a few people internally after the previous > > >>>>> discussion with Liam. One suggestion was to use > > >>>>> DMA_ATTR_SKIP_CPU_SYNC in unmap_dma_buf, but only if there's at least > > >>>>> one other device attached (guarantees that we can do an invalidate in > > >>>>> the future if begin_cpu_access is called). If the last device > > >>>>> detaches, do a sync then. > > >>>>> > > >>>>> Conversely, in map_dma_buf, we would track if there was any CPU access > > >>>>> and use/skip the sync appropriately. > > >>>>> > > >>>> > > >>>> Now that I think this all through I agree this patch is probably wrong. > > >>>> The real fix needs to be better handling in the dma_map_sg() to deal > > >>>> with the case of the memory not being mapped (what I'm dealing with for > > >>>> unmapped heaps), and for cases when the memory in question is not cached > > >>>> (Liam's issue I think). For both these cases the dma_map_sg() does the > > >>>> wrong thing. > > >>>> > > >>>>> I did start poking the code to check out how that would look, but then > > >>>>> Christmas happened and I'm still catching back up. > > >>>>> > > >>>>>>> > > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > > >>>>>>>> the pipeline and Camera doesn't know the end of the use case) > > >>>>>>>> > > >>>>>>> > > >>>>>>> This seems like a broken use-case, I understand the desire to keep > > >>>>>>> everything as modular as possible and separate the steps, but at this > > >>>>>>> point no one owns this buffers backing memory, not the CPU or any > > >>>>>>> device. I would go as far as to say DMA-BUF should be free now to > > >>>>>>> de-allocate the backing storage if it wants, that way it could get ready > > >>>>>>> for the next attachment, which may change the required backing memory > > >>>>>>> completely. > > >>>>>>> > > >>>>>>> All devices should attach before the first mapping, and only let go > > >>>>>>> after the task is complete, otherwise this buffers data needs copied off > > >>>>>>> to a different location or the CPU needs to take ownership in-between. > > >>>>>>> > > >>>>> > > >>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF > > >>>>> implementations which actually do that? I hear it quoted a lot, > > >>>>> because that's what the docs say - but if the reality doesn't match > > >>>>> it, maybe we should change the docs. > > >>>>> > > >>>> > > >>>> Do you mean on the userspace side? I'm not sure, seems like Android > > >>>> might be doing this wrong from what I can gather. From kernel side if > > >>>> you mean the "de-allocate the backing storage", we will have some cases > > >>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF > > >>>> in ways not specified in the documentation. Changing the docs to force > > >>>> the backing memory to always be allocated breaks the central goal in > > >>>> having attach/map in DMA-BUF separate. > > Actually I meant in the kernel, in exporters. I haven't seen anyone > using the API as it was intended (defer allocation until first map, > migrate between different attachments, etc.). Mostly, backing storage > seems to get allocated at the point of export, and device mappings are > often held persistently (e.g. the DRM prime code maps the buffer at > import time, and keeps it mapped: drm_gem_prime_import_dev). > > I wasn't aware that CPU access before first device access was > considered an abuse of the API - it seems like a valid thing to want > to do. > > > >>>> > > >>>>>>>> //buffer is send down the pipeline > > >>>>>>>> > > >>>>>>>> // Usersapce software post processing occurs > > >>>>>>>> mmap buffer > > >>>>>>> > > >>>>>>> Perhaps the invalidate should happen here in mmap. > > >>>>>>> > > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCT with flags DMA_BUF_SYNC_START // No CMO since no > > >>>>>>>> devices attached to buffer > > >>>>>>> > > >>>>>>> And that should be okay, mmap does the sync, and if no devices are > > >>>>>>> attached nothing could have changed the underlying memory in the > > >>>>>>> mean-time, DMA_BUF_SYNC_START can safely be a no-op as they are. > > >>>>> > > >>>>> Yeah, that's true - so long as you did an invalidate in unmap_dma_buf. > > >>>>> Liam was saying that it's too painful for them to do that every time a > > >>>>> device unmaps - when in many cases (device->device, no CPU) it's not > > >>>>> needed. > > >>>> > > >>>> Invalidates are painless, at least compared to a real cache flush, just > > >>>> set the invalid bit vs actually writing out lines. I thought the issue > > >>>> was on the map side. > > >>>> > > >>> > > >>> Invalidates aren't painless for us because we have a coherent system cache > > >>> so clean lines get written out. > > >> > > >> That seems very broken, why would clean lines ever need to be written > > >> out, that defeats the whole point of having the invalidate separate from > > >> clean. How do you deal with stale cache lines? I guess in your case this > > >> is what forces you to have to use uncached memory for DMA-able memory. > > >> > > > > > > My understanding is that our ARM invalidate is a clean + invalidate, I had > > > concerns about the clean lines being written to the the system cache as > > > part of the 'clean', but the following 'invalidate' would take care of > > > actually invalidating the lines (so nothign broken). > > > But i am probably wrong on this and it is probably smart enough not to the > > > writing of the clean lines. > > > > > > > You are correct that for a lot of ARM cores "invalidate" is always a > > "clean + invalidate". At first I thought this was kinda silly as there > > is now no way to mark a dirty line invalid without it getting written > > out first, but if you think about it any dirty cache-line can be written > > out (cleaned) at anytime anyway, so this doesn't actually change system > > behavior. You should just not write to memory (make the line dirty) > > anything you don't want eventually written out. > > > > Point two, it's not just smart enough to not write-out clean lines, it > > is guaranteed not to write them out by the spec. Otherwise since > > cache-lines can be randomly filled if those same clean lines got written > > out on invalidate operations there would be no way to maintain coherency > > and things would be written over top each other all over the place. > > > > > But regardless, targets supporting a coherent system cache is a legitamate > > > configuration and an invalidate on this configuration does have to go to > > > the bus to invalidate the system cache (which isn't free) so I dont' think > > > you can make the assumption that invalidates are cheap so that it is okay > > > to do them (even if they are not needed) on every dma unmap. > > > > > > > Very true, CMOs need to be broadcast to other coherent masters on a > > coherent interconnect (and the interconnect itself if it has a cache as > > well (L3)), so not 100% free, but almost, just the infinitesimal cost of > > the cache tag check in hardware. If there are no non-coherent devices > > attached then the CMOs are no-ops, if there are then the data needs to > > be written out either way, doing it every access like is done with > > uncached memory (- any write combining) will blow away any saving made > > from the one less CMO. Either way you lose with uncached mappings of > > memory. If I'm wrong I would love to know. > > > > From what I understand, the current DMA APIs are not equipped to > handle having coherent and non-coherent devices attached at the same > time. The buffer is either in "CPU land" or "Device land", there's no > smaller granule of "Coherent Device land" or "Non-Coherent Device > land". > > I think if there's devices which are making coherent accesses, and > devices which are making non-coherent accesses, then we can't support > them being attached at the same time without some enhancements to the > APIs. > > > >>> And these invalidates can occur on fairly large buffers. > > >>> > > >>> That is why we haven't went with using cached ION memory and "tracking CPU > > >>> access" because it only solves half the problem, ie there isn't a way to > > >>> safely skip the invalidate (because we can't read the future). > > >>> Our solution was to go with uncached ION memory (when possible), but as > > >>> you can see in other discussions upstream support for uncached memory has > > >>> its own issues. > > >>> > > @Liam, in your problematic use-cases, are both devices detached when > the buffer moves between them? > > 1) dev 1 map, access, unmap > 2) dev 1 detach > 3) (maybe) CPU access > 4) dev 2 attach > 5) dev 2 map, access > > I still think a pretty pragmatic solution is to use > DMA_ATTR_SKIP_CPU_SYNC until the last device detaches. That won't work > if your access sequence looks like above... > Yes the piplelining case in Android looks like the above. > ...however, if your sequence looks like above, then you probably need > to keep at least one of the devices attached anyway. It would be nice if a device could be kept attached, but that doesn't always work very well for an pipelining case as there isn't necessarily a good place for someone to know when to detach it. > Otherwise, per > the API, the buffer could get migrated after 2)/before 5). That will > surely hurt a lot more than an invalidate. > You are right, in theory it could but in practice it isn't getting migrated. I understand the theoretical case, but it doesn't sounds clean as well to force clients to stay attached when there isn't a very nice way of knowing when to have them detach. > > >> > > >> Sounds like you need to fix upstream support then, finding a way to drop > > >> all cacheable mappings of memory you want to make uncached mappings for > > >> seems to be the only solution. > > >> > > > > > > I think we can probably agree that there woudln't be a good way to remove > > > cached mappings without causing an unacceptable performance degradation > > > since it would fragment all the nice 1GB kernel mappings we have. > > > > > > So I am trying to find an alternative solution. > > > > > > > I'm not sure there is a better solution. How hard is this solution to > > implement anyway? The kernel already has to make gaps and cut up that > > nice 1GB mapping when you make a reserved memory space in the lowmem > > area, so all the logic is probably already implemented. Just need to > > allow it to be hooked into from Ion when doing doing the uncached mappings. > > > > I haven't looked recently, but I'm not sure the early memblock code > can be reused as-is at runtime. I seem to remember it makes a bunch of > assumptions about the fact that it's running "early". > > If CPU uncached mappings of normal system memory is really the way > forward, I could envisage a heap which maintains a pool of chunks of > memory which it removed from the kernel mapping. The pool could grow > (remove more pages from the kernel mapping)/shrink (add them back to > the kernel mapping) as needed. > > John Reitan implemented a compound-page heap, which used compaction to > get a pool of 2MB contiguous pages. Something like that would at least > prevent needing full 4kB granularity when removing things from the > kernel mapping. > > Even better, could it somehow be restricted to a region which is > already fragmented? (e.g. the one which was used for the default CMA > heap) > > Thanks, > -Brian > > > >>>>> > > >>>>>>> > > >>>>>>>> [CPU reads/writes to the buffer] > > >>>>>>>> DMA_BUF_IOCTL_SYNC IOCTL with flags DMA_BUF_SYNC_END // No CMO since no > > >>>>>>>> devices attached to buffer > > >>>>>>>> munmap buffer > > >>>>>>>> > > >>>>>>>> //buffer is send down the pipeline > > >>>>>>>> // Buffer is send to video device (who does compression of raw data) and > > >>>>>>>> writes to a file > > >>>>>>>> dma_buf_attach > > >>>>>>>> dma_map_attachment (buffer needs to be cleaned) > > >>>>>>>> [video device writes to buffer] > > >>>>>>>> dma_buf_unmap_attachment > > >>>>>>>> dma_buf_detach (device cannot stay attached because it is being sent down > > >>>>>>>> the pipeline and Video doesn't know the end of the use case) > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>>> Also ION no longer provides DMA ready memory, so if you are not doing CPU > > >>>>>>>>>> access then there is no requirement (that I am aware of) for you to call > > >>>>>>>>>> {begin,end}_cpu_access before passing the buffer to the device and if this > > >>>>>>>>>> buffer is cached and your device is not IO-coherent then the cache maintenance > > >>>>>>>>>> in ion_map_dma_buf and ion_unmap_dma_buf is required. > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> If I am not doing any CPU access then why do I need CPU cache > > >>>>>>>>> maintenance on the buffer? > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> Because ION no longer provides DMA ready memory. > > >>>>>>>> Take the above example. > > >>>>>>>> > > >>>>>>>> ION allocates memory from buddy allocator and requests zeroing. > > >>>>>>>> Zeros are written to the cache. > > >>>>>>>> > > >>>>>>>> You pass the buffer to the camera device which is not IO-coherent. > > >>>>>>>> The camera devices writes directly to the buffer in DDR. > > >>>>>>>> Since you didn't clean the buffer a dirty cache line (one of the zeros) is > > >>>>>>>> evicted from the cache, this zero overwrites data the camera device has > > >>>>>>>> written which corrupts your data. > > >>>>>>>> > > >>>>>>> > > >>>>>>> The zeroing *is* a CPU access, therefor it should handle the needed CMO > > >>>>>>> for CPU access at the time of zeroing. > > >>>>>>> > > >>>>> > > >>>>> Actually that should be at the point of the first non-coherent device > > >>>>> mapping the buffer right? No point in doing CMO if the future accesses > > >>>>> are coherent. > > >>>> > > >>>> I see your point, as long as the zeroing is guaranteed to be the first > > >>>> access to this buffer then it should be safe. > > >>>> > > >>>> Andrew > > >>>> > > >>>>> > > >>>>> Cheers, > > >>>>> -Brian > > >>>>> > > >>>>>>> Andrew > > >>>>>>> > > >>>>>>>> Liam > > >>>>>>>> > > >>>>>>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > >>>>>>>> a Linux Foundation Collaborative Project > > >>>>>>>> > > >>>> > > >>> > > >>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > >>> a Linux Foundation Collaborative Project > > >>> > > >> > > > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, > > > a Linux Foundation Collaborative Project > > > > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Hello everyone, (Thanks to Dan for letting me know my last email got corrupted :/ - resending it here) Sincere apologies for chiming in a bit late here, but was off due to some health issues. Also, adding Daniel Vetter to the mix, since he has been one of the core guys who shaped up dma-buf as it is today. On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <afd@ti.com> wrote: > > On 1/21/19 5:22 AM, Brian Starkey wrote: > > Hi, > > > > Sorry for being a bit sporadic on this. I was out travelling last week > > with little time for email. > > > > On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote: > >> On 1/17/19 7:11 PM, Liam Mark wrote: > >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote: > >>> <snip> > >>>>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF > >>>>>>> implementations which actually do that? I hear it quoted a lot, > >>>>>>> because that's what the docs say - but if the reality doesn't match > >>>>>>> it, maybe we should change the docs. > >>>>>>> > >>>>>> > >>>>>> Do you mean on the userspace side? I'm not sure, seems like Android > >>>>>> might be doing this wrong from what I can gather. From kernel side if > >>>>>> you mean the "de-allocate the backing storage", we will have some cases > >>>>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF > >>>>>> in ways not specified in the documentation. Changing the docs to force > >>>>>> the backing memory to always be allocated breaks the central goal in > >>>>>> having attach/map in DMA-BUF separate. > > > > Actually I meant in the kernel, in exporters. I haven't seen anyone > > using the API as it was intended (defer allocation until first map, > > migrate between different attachments, etc.). Mostly, backing storage > > seems to get allocated at the point of export, and device mappings are > > often held persistently (e.g. the DRM prime code maps the buffer at > > import time, and keeps it mapped: drm_gem_prime_import_dev). > > > I suppose some clarification on the 'intended use' part of dma-buf about deferred allocation is due, so here it is: ( Daniel, please feel free to chime in with your opinion here) - dma-buf was of course designed as a framework to allow intelligent exporters to defer allocation until first map, and be able to migrate backing storage if required etc. At the same time, it is not a _requirement_ from any exporter, so exporters so far have just used it as a convenient mechanism for zero-copy. - from dma-buf PoV, ION is an exporter of dma-buf buffers, for its users that have specific requirements. > I haven't either, which is a shame as it allows for some really useful > management strategies for shared memory resources. I'm working on one > such case right now, maybe I'll get to be the first to upstream one :) > Yes, it would, and great that you're looking to be the first one to do it :) > > I wasn't aware that CPU access before first device access was > > considered an abuse of the API - it seems like a valid thing to want > > to do. > > > > That's just it, I don't know if it is an abuse of API, I'm trying to get > some clarity on that. If we do want to allow early CPU access then that > seems to be in contrast to the idea of deferred allocation until first > device map, what is supposed to backing the buffer if no devices have > attached or mapped yet? Just some system memory followed by migration on > the first attach to the proper backing? Seems too time wasteful to be > have a valid use. > > Maybe it should be up to the exporter if early CPU access is allowed? > > I'm hoping someone with authority over the DMA-BUF framework can clarify > original intentions here. > I suppose dma-buf as a framework can't know or decide what the exporter wants or can do - whether the exporter wants to use it for 'only zero-copy', or do some intelligent things behind the scene, I think should be best left to the exporter. Hope this helps, Sumit. <div dir="ltr">Hello everyone,<br><br>(Thanks to Dan for letting me know my last email got corrupted :/ - resending it here)<div><br>Sincere apologies for chiming in a bit late here, but was off due to some health issues.<br><br>Also, adding Daniel Vetter to the mix, since he has been one of the core guys who shaped up dma-buf as it is today.<br><br>On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <<a href="mailto:afd@ti.com">afd@ti.com</a>> wrote:<br>><br>> On 1/21/19 5:22 AM, Brian Starkey wrote:<br>> > Hi,<br>> ><div>> > Sorry for being a bit sporadic on this. I was out travelling last week<br>> > with little time for email.<br>> ><br>> > On Fri, Jan 18, 2019 at 11:16:31AM -0600, Andrew F. Davis wrote:<br>> >> On 1/17/19 7:11 PM, Liam Mark wrote:<br>> >>> On Thu, 17 Jan 2019, Andrew F. Davis wrote:<br>> >>><br><snip><br>> >>>>>>> Yeah.. that's certainly the theory. Are there any DMA-BUF<br>> >>>>>>> implementations which actually do that? I hear it quoted a lot,<br>> >>>>>>> because that's what the docs say - but if the reality doesn't match<br>> >>>>>>> it, maybe we should change the docs.<br>> >>>>>>><br>> >>>>>><br>> >>>>>> Do you mean on the userspace side? I'm not sure, seems like Android<br>> >>>>>> might be doing this wrong from what I can gather. From kernel side if<br>> >>>>>> you mean the "de-allocate the backing storage", we will have some cases<br>> >>>>>> like this soon, so I want to make sure userspace is not abusing DMA-BUF<br>> >>>>>> in ways not specified in the documentation. Changing the docs to force<br>> >>>>>> the backing memory to always be allocated breaks the central goal in<br>> >>>>>> having attach/map in DMA-BUF separate.<br>> ><br>> > Actually I meant in the kernel, in exporters. I haven't seen anyone<br>> > using the API as it was intended (defer allocation until first map,<br>> > migrate between different attachments, etc.). Mostly, backing storage<br>> > seems to get allocated at the point of export, and device mappings are<br>> > often held persistently (e.g. the DRM prime code maps the buffer at<br>> > import time, and keeps it mapped: drm_gem_prime_import_dev).<br>> ><br>><br>I suppose some clarification on the 'intended use' part of dma-buf about deferred allocation is due, so here it is: ( Daniel, please feel free to chime in with your opinion here)<br> - dma-buf was of course designed as a framework to allow intelligent exporters to defer allocation until first map, and be able to migrate backing storage if required etc. At the same time, it is not a _requirement_ from any exporter, so exporters so far have just used it as a convenient mechanism for zero-copy.<br>- from dma-buf PoV, ION is an exporter of dma-buf buffers, for its users that have specific requirements.<br><br>> I haven't either, which is a shame as it allows for some really useful<br>> management strategies for shared memory resources. I'm working on one<div>> such case right now, maybe I'll get to be the first to upstream one :)<br>><br>Yes, it would, and great that you're looking to be the first one to do it :)<br><br>> > I wasn't aware that CPU access before first device access was<br>> > considered an abuse of the API - it seems like a valid thing to want<br>> > to do.<br>> ><br>><br>> That's just it, I don't know if it is an abuse of API, I'm trying to get<br>> some clarity on that. If we do want to allow early CPU access then that<br>> seems to be in contrast to the idea of deferred allocation until first<br>> device map, what is supposed to backing the buffer if no devices have<br>> attached or mapped yet? Just some system memory followed by migration on<br>> the first attach to the proper backing? Seems too time wasteful to be<br>> have a valid use.<br>><br>> Maybe it should be up to the exporter if early CPU access is allowed?<br>><br>> I'm hoping someone with authority over the DMA-BUF framework can clarify<br>> original intentions here.<br>><br><br>I suppose dma-buf as a framework can't know or decide what the exporter wants or can do - whether the exporter wants to use it for 'only zero-copy', or do some intelligent things behind the scene, I think should be best left to the exporter.</div><div><br></div><div>Hope this helps,</div><div>Sumit.</div><div><br></div></div></div></div>
On 1/22/19 11:33 AM, Sumit Semwal wrote: > Hello everyone, > > Sincere apologies for chiming in a bit late here, but was off due to > some health issues. > Hope you are feeling better friend :) Looks like this email was a bit broken and you replied again, the responses are a little different in each email, so I'd like to respond to bits of both, I'll fix up the formatting. > Also, adding Daniel Vetter to the mix, since he has been one of the > core guys who shaped up dma-buf as it is today. > > On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <afd@ti.com> wrote: >> >> On 1/21/19 5:22 AM, Brian Starkey wrote: [snip] >>> >>> Actually I meant in the kernel, in exporters. I haven't seen anyone >>> using the API as it was intended (defer allocation until first map, >>> migrate between different attachments, etc.). Mostly, backing storage >>> seems to get allocated at the point of export, and device mappings are >>> often held persistently (e.g. the DRM prime code maps the buffer at >>> import time, and keeps it mapped: drm_gem_prime_import_dev). >>> >> > > So I suppose some clarification on the 'intended use' part of dma-buf > about deferred allocation is due, so here it is: (Daniel, please feel > free to chime in with your opinion here) > > - dma-buf was of course designed as a framework to help intelligent > exporters to defer allocation until first map, and be able to migrate > backing storage if required etc. At the same time, it is not a > _requirement_ from any exporter, so exporters so far have just used it > as a convenient mechanism for zero-copy. > - ION is one of the few dma-buf exporters in kernel, which satisfies a > certain set of expectations from its users. > The issue here is that Ion is blocking the ability to late allocate, it expects its heaps to have the memory ready at allocation time. My point being if the DMA-BUFs intended design was to allow this then Ion should respect that and also allow the same from its heap exporters. >> I haven't either, which is a shame as it allows for some really useful >> management strategies for shared memory resources. I'm working on one >> such case right now, maybe I'll get to be the first to upstream one :) >> > That will be a really good thing! Though perhaps we ought to think if > for what you're trying to do, is ION the right place, or should you > have a device-specific exporter, available to users via dma-buf apis? > I'm starting to question if Ion is the right place myself.. At a conceptual level I don't believe userspace should be picking the backing memory type. This is because the right type of backing memory for a task will change from system to system. The kernel should abstract away these hardware differences from userspace as much as it can to allow portable code. For instance a device may need a contiguous buffer on one system but the same device on another may have some IOMMU. So which type of memory do we allocate? Same issue for cacheability and other properties. What we need is a central allocator with full system knowledge to do the choosing for us. It seems many agree with the above and I take inspiration from your cenalloc patchset. The thing I'm not sure about is letting the device drivers set their constraints, because they also don't have the full system integration details. For cases where devices are behind an IOMMU it is easy enough for the device to know, but what about when we have external MMUs out on the bus for anyone to use (I'm guessing you remember TILER..). I would propose the central allocator keep per-system knowledge (or fetch it from DT, or if this is considered policy then userspace) which it can use to directly check the attached devices and pick the right memory. Anyway the central system allocator could handle 90% of cases I can think of, and this is where Ion comes back in, the other cases would still require the program to manually pick the right memory (maybe for performance reasons, etc.). So my vision is to have Ion as the the main front-end for DMA-BUF allocations, and expose the central allocator through it (maybe as a default heap type that can be populated on a per-system basis), but also have other individual heap types exported for the edge cases where manual selection is needed like we do now. Hence why Ion should allow direct control of the dma_buf_ops from the heaps, so we can build central allocators as Ion heaps. If I'm off into the weeds here and you have some other ideas I'm all ears. Andrew >>> I wasn't aware that CPU access before first device access was >>> considered an abuse of the API - it seems like a valid thing to want >>> to do. >>> >> >> That's just it, I don't know if it is an abuse of API, I'm trying to get >> some clarity on that. If we do want to allow early CPU access then that >> seems to be in contrast to the idea of deferred allocation until first >> device map, what is supposed to backing the buffer if no devices have >> attached or mapped yet? Just some system memory followed by migration on >> the first attach to the proper backing? Seems too time wasteful to be >> have a valid use. >> >> Maybe it should be up to the exporter if early CPU access is allowed? >> >> I'm hoping someone with authority over the DMA-BUF framework can clarify >> original intentions here. > > I don't think dma-buf as a framework stops early CPU access, and the > exporter can definitely decide on that by implementing > begin_cpu_access / end_cpu_access operations to not allow early CPU > access, if it so desires. >
On 1/22/19 9:23 PM, Sumit Semwal wrote: > Hello everyone, > > (Thanks to Dan for letting me know my last email got corrupted :/ - > resending it here) > Hmm, this one seems a bit messed up also (Thunderbird doesn't seem to like it at least). [snip] > - from dma-buf PoV, ION is an exporter of dma-buf buffers, for its users > that have specific requirements. > This is what I'm hoping to change up a little bit, Ion shouldn't be the exporter, its heaps should be the exporters (manage the dma_buf_ops), Ion would only do advertising of available heaps and allow allocating DMA-BUFs from them. IMO that would clear up the other discussions going on right now about how Ion should handle different dma-buf syncing tasks, it simply wouldn't :). Plus Ion core gets slimmed down, maybe even enough for destaging.. >> I haven't either, which is a shame as it allows for some really useful >> management strategies for shared memory resources. I'm working on one >> such case right now, maybe I'll get to be the first to upstream one :) >> > Yes, it would, and great that you're looking to be the first one to do it :) > >> > I wasn't aware that CPU access before first device access was >> > considered an abuse of the API - it seems like a valid thing to want >> > to do. >> > >> >> That's just it, I don't know if it is an abuse of API, I'm trying to get >> some clarity on that. If we do want to allow early CPU access then that >> seems to be in contrast to the idea of deferred allocation until first >> device map, what is supposed to backing the buffer if no devices have >> attached or mapped yet? Just some system memory followed by migration on >> the first attach to the proper backing? Seems too time wasteful to be >> have a valid use. >> >> Maybe it should be up to the exporter if early CPU access is allowed? >> >> I'm hoping someone with authority over the DMA-BUF framework can clarify >> original intentions here. >> > > I suppose dma-buf as a framework can't know or decide what the exporter > wants or can do - whether the exporter wants to use it for 'only > zero-copy', or do some intelligent things behind the scene, I think > should be best left to the exporter. > > Hope this helps, Yes, these inputs are very helpful, thanks, Andrew > Sumit. >
Hi Andrew, On Wed, Jan 23, 2019 at 10:51:24AM -0600, Andrew F. Davis wrote: > On 1/22/19 11:33 AM, Sumit Semwal wrote: > > Hello everyone, > > > > Sincere apologies for chiming in a bit late here, but was off due to > > some health issues. > > > > Hope you are feeling better friend :) > > Looks like this email was a bit broken and you replied again, the > responses are a little different in each email, so I'd like to respond > to bits of both, I'll fix up the formatting. > > > Also, adding Daniel Vetter to the mix, since he has been one of the > > core guys who shaped up dma-buf as it is today. > > > > On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <afd@ti.com> wrote: > >> > >> On 1/21/19 5:22 AM, Brian Starkey wrote: > > [snip] > > >>> > >>> Actually I meant in the kernel, in exporters. I haven't seen anyone > >>> using the API as it was intended (defer allocation until first map, > >>> migrate between different attachments, etc.). Mostly, backing storage > >>> seems to get allocated at the point of export, and device mappings are > >>> often held persistently (e.g. the DRM prime code maps the buffer at > >>> import time, and keeps it mapped: drm_gem_prime_import_dev). > >>> > >> > > > > So I suppose some clarification on the 'intended use' part of dma-buf > > about deferred allocation is due, so here it is: (Daniel, please feel > > free to chime in with your opinion here) > > > > - dma-buf was of course designed as a framework to help intelligent > > exporters to defer allocation until first map, and be able to migrate > > backing storage if required etc. At the same time, it is not a > > _requirement_ from any exporter, so exporters so far have just used it > > as a convenient mechanism for zero-copy. > > - ION is one of the few dma-buf exporters in kernel, which satisfies a > > certain set of expectations from its users. > > > > The issue here is that Ion is blocking the ability to late allocate, it > expects its heaps to have the memory ready at allocation time. My point > being if the DMA-BUFs intended design was to allow this then Ion should > respect that and also allow the same from its heap exporters. > > >> I haven't either, which is a shame as it allows for some really useful > >> management strategies for shared memory resources. I'm working on one > >> such case right now, maybe I'll get to be the first to upstream one :) > >> > > That will be a really good thing! Though perhaps we ought to think if > > for what you're trying to do, is ION the right place, or should you > > have a device-specific exporter, available to users via dma-buf apis? > > > > I'm starting to question if Ion is the right place myself.. > > At a conceptual level I don't believe userspace should be picking the > backing memory type. This is because the right type of backing memory > for a task will change from system to system. The kernel should abstract > away these hardware differences from userspace as much as it can to > allow portable code. > > For instance a device may need a contiguous buffer on one system but the > same device on another may have some IOMMU. So which type of memory do > we allocate? Same issue for cacheability and other properties. > > What we need is a central allocator with full system knowledge to do the > choosing for us. It seems many agree with the above and I take > inspiration from your cenalloc patchset. The thing I'm not sure about is > letting the device drivers set their constraints, because they also > don't have the full system integration details. For cases where devices > are behind an IOMMU it is easy enough for the device to know, but what > about when we have external MMUs out on the bus for anyone to use (I'm > guessing you remember TILER..). > > I would propose the central allocator keep per-system knowledge (or > fetch it from DT, or if this is considered policy then userspace) which > it can use to directly check the attached devices and pick the right memory. > > Anyway the central system allocator could handle 90% of cases I can > think of, and this is where Ion comes back in, the other cases would > still require the program to manually pick the right memory (maybe for > performance reasons, etc.). > > So my vision is to have Ion as the the main front-end for DMA-BUF > allocations, and expose the central allocator through it (maybe as a > default heap type that can be populated on a per-system basis), but also > have other individual heap types exported for the edge cases where > manual selection is needed like we do now. > > Hence why Ion should allow direct control of the dma_buf_ops from the > heaps, so we can build central allocators as Ion heaps. > > If I'm off into the weeds here and you have some other ideas I'm all ears. > This is a topic I've gone around a few times. The crux of it is, as you know, a central allocator is Really Hard. I don't know what you've seen/done so far in this area, so please forgive me if this is old hat to you. Android's platform-specific Gralloc module actually does a pretty good job, and because it's platform-specific, it can be simple. That does have a certain appeal to it over something generic but complex. It seems that generic gets insanely complicated really quickly - picking out "compatible" is hard enough, but improving that to pick out "optimal" is ... well, I've not seen any good proposals for that. In case you didn't come across it already, the effort which seems to have gained the most "air-time" recently is https://github.com/cubanismo/allocator, which is still a userspace module (perhaps some concepts from there could go into the kernel?), but makes some attempts at generic constraint solving. It's also not really moving anywhere at the moment. Cheers, -Brian > Andrew > > >>> I wasn't aware that CPU access before first device access was > >>> considered an abuse of the API - it seems like a valid thing to want > >>> to do. > >>> > >> > >> That's just it, I don't know if it is an abuse of API, I'm trying to get > >> some clarity on that. If we do want to allow early CPU access then that > >> seems to be in contrast to the idea of deferred allocation until first > >> device map, what is supposed to backing the buffer if no devices have > >> attached or mapped yet? Just some system memory followed by migration on > >> the first attach to the proper backing? Seems too time wasteful to be > >> have a valid use. > >> > >> Maybe it should be up to the exporter if early CPU access is allowed? > >> > >> I'm hoping someone with authority over the DMA-BUF framework can clarify > >> original intentions here. > > > > I don't think dma-buf as a framework stops early CPU access, and the > > exporter can definitely decide on that by implementing > > begin_cpu_access / end_cpu_access operations to not allow early CPU > > access, if it so desires. > >
On 1/23/19 11:11 AM, Brian Starkey wrote: > Hi Andrew, > > On Wed, Jan 23, 2019 at 10:51:24AM -0600, Andrew F. Davis wrote: >> On 1/22/19 11:33 AM, Sumit Semwal wrote: >>> Hello everyone, >>> >>> Sincere apologies for chiming in a bit late here, but was off due to >>> some health issues. >>> >> >> Hope you are feeling better friend :) >> >> Looks like this email was a bit broken and you replied again, the >> responses are a little different in each email, so I'd like to respond >> to bits of both, I'll fix up the formatting. >> >>> Also, adding Daniel Vetter to the mix, since he has been one of the >>> core guys who shaped up dma-buf as it is today. >>> >>> On Tue, 22 Jan 2019 at 02:51, Andrew F. Davis <afd@ti.com> wrote: >>>> >>>> On 1/21/19 5:22 AM, Brian Starkey wrote: >> >> [snip] >> >>>>> >>>>> Actually I meant in the kernel, in exporters. I haven't seen anyone >>>>> using the API as it was intended (defer allocation until first map, >>>>> migrate between different attachments, etc.). Mostly, backing storage >>>>> seems to get allocated at the point of export, and device mappings are >>>>> often held persistently (e.g. the DRM prime code maps the buffer at >>>>> import time, and keeps it mapped: drm_gem_prime_import_dev). >>>>> >>>> >>> >>> So I suppose some clarification on the 'intended use' part of dma-buf >>> about deferred allocation is due, so here it is: (Daniel, please feel >>> free to chime in with your opinion here) >>> >>> - dma-buf was of course designed as a framework to help intelligent >>> exporters to defer allocation until first map, and be able to migrate >>> backing storage if required etc. At the same time, it is not a >>> _requirement_ from any exporter, so exporters so far have just used it >>> as a convenient mechanism for zero-copy. >>> - ION is one of the few dma-buf exporters in kernel, which satisfies a >>> certain set of expectations from its users. >>> >> >> The issue here is that Ion is blocking the ability to late allocate, it >> expects its heaps to have the memory ready at allocation time. My point >> being if the DMA-BUFs intended design was to allow this then Ion should >> respect that and also allow the same from its heap exporters. >> >>>> I haven't either, which is a shame as it allows for some really useful >>>> management strategies for shared memory resources. I'm working on one >>>> such case right now, maybe I'll get to be the first to upstream one :) >>>> >>> That will be a really good thing! Though perhaps we ought to think if >>> for what you're trying to do, is ION the right place, or should you >>> have a device-specific exporter, available to users via dma-buf apis? >>> >> >> I'm starting to question if Ion is the right place myself.. >> >> At a conceptual level I don't believe userspace should be picking the >> backing memory type. This is because the right type of backing memory >> for a task will change from system to system. The kernel should abstract >> away these hardware differences from userspace as much as it can to >> allow portable code. >> >> For instance a device may need a contiguous buffer on one system but the >> same device on another may have some IOMMU. So which type of memory do >> we allocate? Same issue for cacheability and other properties. >> >> What we need is a central allocator with full system knowledge to do the >> choosing for us. It seems many agree with the above and I take >> inspiration from your cenalloc patchset. The thing I'm not sure about is >> letting the device drivers set their constraints, because they also >> don't have the full system integration details. For cases where devices >> are behind an IOMMU it is easy enough for the device to know, but what >> about when we have external MMUs out on the bus for anyone to use (I'm >> guessing you remember TILER..). >> >> I would propose the central allocator keep per-system knowledge (or >> fetch it from DT, or if this is considered policy then userspace) which >> it can use to directly check the attached devices and pick the right memory. >> >> Anyway the central system allocator could handle 90% of cases I can >> think of, and this is where Ion comes back in, the other cases would >> still require the program to manually pick the right memory (maybe for >> performance reasons, etc.). >> >> So my vision is to have Ion as the the main front-end for DMA-BUF >> allocations, and expose the central allocator through it (maybe as a >> default heap type that can be populated on a per-system basis), but also >> have other individual heap types exported for the edge cases where >> manual selection is needed like we do now. >> >> Hence why Ion should allow direct control of the dma_buf_ops from the >> heaps, so we can build central allocators as Ion heaps. >> >> If I'm off into the weeds here and you have some other ideas I'm all ears. >> > > This is a topic I've gone around a few times. The crux of it is, as > you know, a central allocator is Really Hard. I don't know what you've > seen/done so far in this area, so please forgive me if this is old hat > to you. > I'm very new to all this, so any pointers to history in this area are appreciated. > Android's platform-specific Gralloc module actually does a pretty good > job, and because it's platform-specific, it can be simple. That does > have a certain appeal to it over something generic but complex. > > It seems that generic gets insanely complicated really quickly - > picking out "compatible" is hard enough, but improving that to pick > out "optimal" is ... well, I've not seen any good proposals for that. > I quickly found out the extent of how complicated this can get. I originally wanted a generic central allocator, but to be complete it would have to handle every one of everyones uses, so just not going to happen. So I refocused on simply a central allocator, it didn't need to be generic and can be made specifically for each given system of ours. Even that quickly ran into objections when proposed internally, there will always be edge use-cases that require manual selection of allocation type (due to performance and resource partitioning). Now I'm at an optional central allocator plus the standard allocator heaps provided by Ion, the system specific central allocator being also exported by Ion for simplicity. So all I want is to find is "compatible" when needed, leave "optimal" selection up to the applications that need it. > In case you didn't come across it already, the effort which seems to > have gained the most "air-time" recently is > https://github.com/cubanismo/allocator, which is still a userspace > module (perhaps some concepts from there could go into the kernel?), > but makes some attempts at generic constraint solving. It's also not > really moving anywhere at the moment. > Very interesting, I'm going to have to stare at this for a bit. Thanks, Andrew > Cheers, > -Brian > >> Andrew >> >>>>> I wasn't aware that CPU access before first device access was >>>>> considered an abuse of the API - it seems like a valid thing to want >>>>> to do. >>>>> >>>> >>>> That's just it, I don't know if it is an abuse of API, I'm trying to get >>>> some clarity on that. If we do want to allow early CPU access then that >>>> seems to be in contrast to the idea of deferred allocation until first >>>> device map, what is supposed to backing the buffer if no devices have >>>> attached or mapped yet? Just some system memory followed by migration on >>>> the first attach to the proper backing? Seems too time wasteful to be >>>> have a valid use. >>>> >>>> Maybe it should be up to the exporter if early CPU access is allowed? >>>> >>>> I'm hoping someone with authority over the DMA-BUF framework can clarify >>>> original intentions here. >>> >>> I don't think dma-buf as a framework stops early CPU access, and the >>> exporter can definitely decide on that by implementing >>> begin_cpu_access / end_cpu_access operations to not allow early CPU >>> access, if it so desires. >>>
On Thu, Jan 24, 2019 at 10:04:46AM -0600, Andrew F. Davis wrote: > On 1/23/19 11:11 AM, Brian Starkey wrote: [snip] > > I'm very new to all this, so any pointers to history in this area are > appreciated. > [snip] > > > In case you didn't come across it already, the effort which seems to > > have gained the most "air-time" recently is > > https://github.com/cubanismo/allocator, which is still a userspace > > module (perhaps some concepts from there could go into the kernel?), > > but makes some attempts at generic constraint solving. It's also not > > really moving anywhere at the moment. > > > > Very interesting, I'm going to have to stare at this for a bit. In which case, some reading material that might be of interest :-) https://www.x.org/wiki/Events/XDC2016/Program/Unix_Device_Memory_Allocation.pdf https://www.x.org/wiki/Events/XDC2017/jones_allocator.pdf https://lists.freedesktop.org/archives/mesa-dev/2017-November/177632.html -Brian
On 1/24/19 8:44 AM, Brian Starkey wrote: > On Thu, Jan 24, 2019 at 10:04:46AM -0600, Andrew F. Davis wrote: >> On 1/23/19 11:11 AM, Brian Starkey wrote: > > [snip] > >> >> I'm very new to all this, so any pointers to history in this area are >> appreciated. >> > > [snip] > >> >>> In case you didn't come across it already, the effort which seems to >>> have gained the most "air-time" recently is >>> https://github.com/cubanismo/allocator, which is still a userspace >>> module (perhaps some concepts from there could go into the kernel?), >>> but makes some attempts at generic constraint solving. It's also not >>> really moving anywhere at the moment. >>> >> >> Very interesting, I'm going to have to stare at this for a bit. > > In which case, some reading material that might be of interest :-) > > https://www.x.org/wiki/Events/XDC2016/Program/Unix_Device_Memory_Allocation.pdf > https://www.x.org/wiki/Events/XDC2017/jones_allocator.pdf > https://lists.freedesktop.org/archives/mesa-dev/2017-November/177632.html > > -Brian > In some respects this is more a question of "what is the purpose of Ion". Once upon a time, Ion was going to do constraint solving but that never really happened and I figured Ion would get deprecated. People keep coming out of the woodwork with new uses for Ion so its stuck around. This is why I've primarily focused on Ion as a framework for exposing available memory types to userspace and leave the constraint solving to someone else, since that's what most users seem to want out of Ion ("I know I want memory type X please give it to me"). That's not to say that this was a perfect or even the correct approach, just what made the most sense based on users. Thanks, Laura
diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c index 14e48f6eb734..09cb5a8e2b09 100644 --- a/drivers/staging/android/ion/ion.c +++ b/drivers/staging/android/ion/ion.c @@ -261,8 +261,8 @@ static struct sg_table *ion_map_dma_buf(struct dma_buf_attachment *attachment, table = a->table; - if (!dma_map_sg(attachment->dev, table->sgl, table->nents, - direction)) + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC)) return ERR_PTR(-ENOMEM); return table; @@ -272,7 +272,8 @@ static void ion_unmap_dma_buf(struct dma_buf_attachment *attachment, struct sg_table *table, enum dma_data_direction direction) { - dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction); + dma_unmap_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC); } static int ion_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
Buffers may not be mapped from the CPU so skip cache maintenance here. Accesses from the CPU to a cached heap should be bracketed with {begin,end}_cpu_access calls so maintenance should not be needed anyway. Signed-off-by: Andrew F. Davis <afd@ti.com> --- drivers/staging/android/ion/ion.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)