| Message ID | 1592719388-13819-4-git-send-email-andrey.grodzovsky@amd.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | RFC Support hot device unplug in amdgpu | expand |
On Sun, Jun 21, 2020 at 02:03:03AM -0400, Andrey Grodzovsky wrote: > Helper function to be used to invalidate all BOs CPU mappings > once device is removed. > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> This seems to be missing the code to invalidate all the dma-buf mmaps? Probably needs more testcases if you're not yet catching this. Or am I missing something, and we're exchanging the the address space also for dma-buf? -Daniel > --- > drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- > include/drm/ttm/ttm_bo_driver.h | 7 +++++++ > 2 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index c5b516f..926a365 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) > ttm_bo_unmap_virtual_locked(bo); > ttm_mem_io_unlock(man); > } > - > - > EXPORT_SYMBOL(ttm_bo_unmap_virtual); > > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) > +{ > + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); > +} > +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); > + > int ttm_bo_wait(struct ttm_buffer_object *bo, > bool interruptible, bool no_wait) > { > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > index c9e0fd0..39ea44f 100644 > --- a/include/drm/ttm/ttm_bo_driver.h > +++ b/include/drm/ttm/ttm_bo_driver.h > @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, > void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); > > /** > + * ttm_bo_unmap_virtual_address_space > + * > + * @bdev: tear down all the virtual mappings for this device > + */ > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); > + > +/** > * ttm_bo_unmap_virtual > * > * @bo: tear down the virtual mappings for this BO > -- > 2.7.4 >
Am 21.06.20 um 08:03 schrieb Andrey Grodzovsky: > Helper function to be used to invalidate all BOs CPU mappings > once device is removed. > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> > --- > drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- > include/drm/ttm/ttm_bo_driver.h | 7 +++++++ > 2 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index c5b516f..926a365 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) > ttm_bo_unmap_virtual_locked(bo); > ttm_mem_io_unlock(man); > } > - > - > EXPORT_SYMBOL(ttm_bo_unmap_virtual); > > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) > +{ > + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); > +} > +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); > + > int ttm_bo_wait(struct ttm_buffer_object *bo, > bool interruptible, bool no_wait) > { > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > index c9e0fd0..39ea44f 100644 > --- a/include/drm/ttm/ttm_bo_driver.h > +++ b/include/drm/ttm/ttm_bo_driver.h > @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, > void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); > > /** > + * ttm_bo_unmap_virtual_address_space > + * > + * @bdev: tear down all the virtual mappings for this device > + */ > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); > + > +/** > * ttm_bo_unmap_virtual > * > * @bo: tear down the virtual mappings for this BO
On Sun, Jun 21, 2020 at 2:05 AM Andrey Grodzovsky <andrey.grodzovsky@amd.com> wrote: > > Helper function to be used to invalidate all BOs CPU mappings > once device is removed. > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> Typo in the subject: unampping -> unmapping Alex > --- > drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- > include/drm/ttm/ttm_bo_driver.h | 7 +++++++ > 2 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index c5b516f..926a365 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) > ttm_bo_unmap_virtual_locked(bo); > ttm_mem_io_unlock(man); > } > - > - > EXPORT_SYMBOL(ttm_bo_unmap_virtual); > > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) > +{ > + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); > +} > +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); > + > int ttm_bo_wait(struct ttm_buffer_object *bo, > bool interruptible, bool no_wait) > { > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > index c9e0fd0..39ea44f 100644 > --- a/include/drm/ttm/ttm_bo_driver.h > +++ b/include/drm/ttm/ttm_bo_driver.h > @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, > void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); > > /** > + * ttm_bo_unmap_virtual_address_space > + * > + * @bdev: tear down all the virtual mappings for this device > + */ > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); > + > +/** > * ttm_bo_unmap_virtual > * > * @bo: tear down the virtual mappings for this BO > -- > 2.7.4 >
On 6/22/20 5:45 AM, Daniel Vetter wrote: > On Sun, Jun 21, 2020 at 02:03:03AM -0400, Andrey Grodzovsky wrote: >> Helper function to be used to invalidate all BOs CPU mappings >> once device is removed. >> >> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> > This seems to be missing the code to invalidate all the dma-buf mmaps? > > Probably needs more testcases if you're not yet catching this. Or am I > missing something, and we're exchanging the the address space also for > dma-buf? > -Daniel IMHO the device address space includes all user clients having a CPU view of the BO either from direct mapping though drm file or by mapping through imported BO's FD. Andrey > >> --- >> drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- >> include/drm/ttm/ttm_bo_driver.h | 7 +++++++ >> 2 files changed, 13 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c >> index c5b516f..926a365 100644 >> --- a/drivers/gpu/drm/ttm/ttm_bo.c >> +++ b/drivers/gpu/drm/ttm/ttm_bo.c >> @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) >> ttm_bo_unmap_virtual_locked(bo); >> ttm_mem_io_unlock(man); >> } >> - >> - >> EXPORT_SYMBOL(ttm_bo_unmap_virtual); >> >> +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) >> +{ >> + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); >> +} >> +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); >> + >> int ttm_bo_wait(struct ttm_buffer_object *bo, >> bool interruptible, bool no_wait) >> { >> diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h >> index c9e0fd0..39ea44f 100644 >> --- a/include/drm/ttm/ttm_bo_driver.h >> +++ b/include/drm/ttm/ttm_bo_driver.h >> @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, >> void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); >> >> /** >> + * ttm_bo_unmap_virtual_address_space >> + * >> + * @bdev: tear down all the virtual mappings for this device >> + */ >> +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); >> + >> +/** >> * ttm_bo_unmap_virtual >> * >> * @bo: tear down the virtual mappings for this BO >> -- >> 2.7.4 >>
On Tue, Jun 23, 2020 at 01:00:02AM -0400, Andrey Grodzovsky wrote: > > On 6/22/20 5:45 AM, Daniel Vetter wrote: > > On Sun, Jun 21, 2020 at 02:03:03AM -0400, Andrey Grodzovsky wrote: > > > Helper function to be used to invalidate all BOs CPU mappings > > > once device is removed. > > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> > > This seems to be missing the code to invalidate all the dma-buf mmaps? > > > > Probably needs more testcases if you're not yet catching this. Or am I > > missing something, and we're exchanging the the address space also for > > dma-buf? > > -Daniel > > > IMHO the device address space includes all user clients having a CPU view of > the BO either from direct mapping though drm file or by mapping through > imported BO's FD. Uh this is all very confusing and very much midlayer-y thanks to ttm. I think a much better solution would be to have a core gem helper for this (well not even gem really, this is core drm), which directly uses drm_device->anon_inode->i_mapping. Then a) it clearly matches what drm_prime.c does on export b) can be reused across all drivers, not just ttm So much better. What's more, we could then very easily make the generic drm_dev_unplug_and_unmap helper I've talked about for the amdgpu patch, which I think would be really neat&pretty. Thoughts? -Daniel > > Andrey > > > > > > > --- > > > drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- > > > include/drm/ttm/ttm_bo_driver.h | 7 +++++++ > > > 2 files changed, 13 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > > > index c5b516f..926a365 100644 > > > --- a/drivers/gpu/drm/ttm/ttm_bo.c > > > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > > > @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) > > > ttm_bo_unmap_virtual_locked(bo); > > > ttm_mem_io_unlock(man); > > > } > > > - > > > - > > > EXPORT_SYMBOL(ttm_bo_unmap_virtual); > > > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) > > > +{ > > > + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); > > > +} > > > +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); > > > + > > > int ttm_bo_wait(struct ttm_buffer_object *bo, > > > bool interruptible, bool no_wait) > > > { > > > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > > > index c9e0fd0..39ea44f 100644 > > > --- a/include/drm/ttm/ttm_bo_driver.h > > > +++ b/include/drm/ttm/ttm_bo_driver.h > > > @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, > > > void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); > > > /** > > > + * ttm_bo_unmap_virtual_address_space > > > + * > > > + * @bdev: tear down all the virtual mappings for this device > > > + */ > > > +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); > > > + > > > +/** > > > * ttm_bo_unmap_virtual > > > * > > > * @bo: tear down the virtual mappings for this BO > > > -- > > > 2.7.4 > > >
Am 23.06.20 um 12:25 schrieb Daniel Vetter: > On Tue, Jun 23, 2020 at 01:00:02AM -0400, Andrey Grodzovsky wrote: >> On 6/22/20 5:45 AM, Daniel Vetter wrote: >>> On Sun, Jun 21, 2020 at 02:03:03AM -0400, Andrey Grodzovsky wrote: >>>> Helper function to be used to invalidate all BOs CPU mappings >>>> once device is removed. >>>> >>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> >>> This seems to be missing the code to invalidate all the dma-buf mmaps? >>> >>> Probably needs more testcases if you're not yet catching this. Or am I >>> missing something, and we're exchanging the the address space also for >>> dma-buf? >>> -Daniel >> >> IMHO the device address space includes all user clients having a CPU view of >> the BO either from direct mapping though drm file or by mapping through >> imported BO's FD. > Uh this is all very confusing and very much midlayer-y thanks to ttm. > > I think a much better solution would be to have a core gem helper for > this (well not even gem really, this is core drm), which directly uses > drm_device->anon_inode->i_mapping. > > Then > a) it clearly matches what drm_prime.c does on export > b) can be reused across all drivers, not just ttm > > So much better. > > What's more, we could then very easily make the generic > drm_dev_unplug_and_unmap helper I've talked about for the amdgpu patch, > which I think would be really neat&pretty. Good point, that is indeed a rather nice idea. Christian. > > Thoughts? > -Daniel > >> Andrey >> >> >>>> --- >>>> drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- >>>> include/drm/ttm/ttm_bo_driver.h | 7 +++++++ >>>> 2 files changed, 13 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c >>>> index c5b516f..926a365 100644 >>>> --- a/drivers/gpu/drm/ttm/ttm_bo.c >>>> +++ b/drivers/gpu/drm/ttm/ttm_bo.c >>>> @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) >>>> ttm_bo_unmap_virtual_locked(bo); >>>> ttm_mem_io_unlock(man); >>>> } >>>> - >>>> - >>>> EXPORT_SYMBOL(ttm_bo_unmap_virtual); >>>> +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) >>>> +{ >>>> + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); >>>> +} >>>> +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); >>>> + >>>> int ttm_bo_wait(struct ttm_buffer_object *bo, >>>> bool interruptible, bool no_wait) >>>> { >>>> diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h >>>> index c9e0fd0..39ea44f 100644 >>>> --- a/include/drm/ttm/ttm_bo_driver.h >>>> +++ b/include/drm/ttm/ttm_bo_driver.h >>>> @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, >>>> void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); >>>> /** >>>> + * ttm_bo_unmap_virtual_address_space >>>> + * >>>> + * @bdev: tear down all the virtual mappings for this device >>>> + */ >>>> +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); >>>> + >>>> +/** >>>> * ttm_bo_unmap_virtual >>>> * >>>> * @bo: tear down the virtual mappings for this BO >>>> -- >>>> 2.7.4 >>>>
diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index c5b516f..926a365 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) ttm_bo_unmap_virtual_locked(bo); ttm_mem_io_unlock(man); } - - EXPORT_SYMBOL(ttm_bo_unmap_virtual); +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) +{ + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); +} +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); + int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait) { diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index c9e0fd0..39ea44f 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); /** + * ttm_bo_unmap_virtual_address_space + * + * @bdev: tear down all the virtual mappings for this device + */ +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); + +/** * ttm_bo_unmap_virtual * * @bo: tear down the virtual mappings for this BO
Helper function to be used to invalidate all BOs CPU mappings once device is removed. Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com> --- drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- include/drm/ttm/ttm_bo_driver.h | 7 +++++++ 2 files changed, 13 insertions(+), 2 deletions(-)