| Message ID | 20210414064621.29273-9-Felix.Kuehling@amd.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Implement multi-GPU DMA mappings for KFD | expand |
On Wed, Apr 14, 2021 at 02:46:20AM -0400, Felix Kuehling wrote: > Pages in SG BOs were not allocated by TTM. So don't count them against > TTM's pages limit. > > Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> This sounds like papering over the lack of shrinker in ttm. Do we guarantee that someone else has already accounted these buffers against the ttm memory hard-limit anywhere? If not I think this needs to wait for the shrinker work to get solid, since fixing the double-accounting for this is probably not worth it. -Daniel > --- > drivers/gpu/drm/ttm/ttm_tt.c | 27 ++++++++++++++++++--------- > 1 file changed, 18 insertions(+), 9 deletions(-) > > diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c > index 5d8820725b75..e8b8c3257392 100644 > --- a/drivers/gpu/drm/ttm/ttm_tt.c > +++ b/drivers/gpu/drm/ttm/ttm_tt.c > @@ -317,9 +317,12 @@ int ttm_tt_populate(struct ttm_device *bdev, > if (ttm_tt_is_populated(ttm)) > return 0; > > - atomic_long_add(ttm->num_pages, &ttm_pages_allocated); > - if (bdev->pool.use_dma32) > - atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); > + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { > + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); > + if (bdev->pool.use_dma32) > + atomic_long_add(ttm->num_pages, > + &ttm_dma32_pages_allocated); > + } > > while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || > atomic_long_read(&ttm_dma32_pages_allocated) > > @@ -350,9 +353,12 @@ int ttm_tt_populate(struct ttm_device *bdev, > return 0; > > error: > - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); > - if (bdev->pool.use_dma32) > - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); > + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { > + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); > + if (bdev->pool.use_dma32) > + atomic_long_sub(ttm->num_pages, > + &ttm_dma32_pages_allocated); > + } > return ret; > } > EXPORT_SYMBOL(ttm_tt_populate); > @@ -382,9 +388,12 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) > else > ttm_pool_free(&bdev->pool, ttm); > > - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); > - if (bdev->pool.use_dma32) > - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); > + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { > + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); > + if (bdev->pool.use_dma32) > + atomic_long_sub(ttm->num_pages, > + &ttm_dma32_pages_allocated); > + } > > ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; > } > -- > 2.31.1 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 5d8820725b75..e8b8c3257392 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -317,9 +317,12 @@ int ttm_tt_populate(struct ttm_device *bdev, if (ttm_tt_is_populated(ttm)) return 0; - atomic_long_add(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_add(ttm->num_pages, + &ttm_dma32_pages_allocated); + } while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || atomic_long_read(&ttm_dma32_pages_allocated) > @@ -350,9 +353,12 @@ int ttm_tt_populate(struct ttm_device *bdev, return 0; error: - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } return ret; } EXPORT_SYMBOL(ttm_tt_populate); @@ -382,9 +388,12 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) else ttm_pool_free(&bdev->pool, ttm); - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; }
Pages in SG BOs were not allocated by TTM. So don't count them against TTM's pages limit. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> --- drivers/gpu/drm/ttm/ttm_tt.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-)