| Message ID | 20201109005432.861936-2-airlied@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/4] drm/ttm: add multihop infrastrucutre (v2) | expand |
On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > Am 09.11.20 um 01:54 schrieb Dave Airlie: > > @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > > if (bo->mem.mem_type != TTM_PL_SYSTEM) { > > struct ttm_operation_ctx ctx = { false, false }; > > struct ttm_resource evict_mem; > > + struct ttm_place hop = {}; > > Please always use memset() if you want to zero initialize something in > the kernel, we had enough trouble with that. What trouble is that? I've not heard of anything, and we use ={} quite extensively in drm land.
Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: >> Am 09.11.20 um 01:54 schrieb Dave Airlie: >>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) >>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { >>> struct ttm_operation_ctx ctx = { false, false }; >>> struct ttm_resource evict_mem; >>> + struct ttm_place hop = {}; >> Please always use memset() if you want to zero initialize something in >> the kernel, we had enough trouble with that. > What trouble is that? I've not heard of anything, and we use > ={} quite extensively in drm land. ={} initializes only named fields, not padding. The result is that for example when doing a hash or CRC of a structure you can come up with different results depending on the architecture and/or structure layout. Another problem are information leaks from the kernel to userspace because of this. Because of this Mesa for example strongly discourages using ={} for zeroing a structure. Regards, Christian.
On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > > On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > >> Am 09.11.20 um 01:54 schrieb Dave Airlie: > >>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > >>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > >>> struct ttm_operation_ctx ctx = { false, false }; > >>> struct ttm_resource evict_mem; > >>> + struct ttm_place hop = {}; > >> Please always use memset() if you want to zero initialize something in > >> the kernel, we had enough trouble with that. > > What trouble is that? I've not heard of anything, and we use > > ={} quite extensively in drm land. > > ={} initializes only named fields, not padding. Has that actually happened? > > The result is that for example when doing a hash or CRC of a structure > you can come up with different results depending on the architecture > and/or structure layout. > > Another problem are information leaks from the kernel to userspace > because of this. > > Because of this Mesa for example strongly discourages using ={} for > zeroing a structure. > > Regards, > Christian.
Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: >> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: >>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: >>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: >>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) >>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { >>>>> struct ttm_operation_ctx ctx = { false, false }; >>>>> struct ttm_resource evict_mem; >>>>> + struct ttm_place hop = {}; >>>> Please always use memset() if you want to zero initialize something in >>>> the kernel, we had enough trouble with that. >>> What trouble is that? I've not heard of anything, and we use >>> ={} quite extensively in drm land. >> ={} initializes only named fields, not padding. > Has that actually happened? YES! Numerous times! And the first few times it took us weeks to figure out what was actually happening. Christian. > >> The result is that for example when doing a hash or CRC of a structure >> you can come up with different results depending on the architecture >> and/or structure layout. >> >> Another problem are information leaks from the kernel to userspace >> because of this. >> >> Because of this Mesa for example strongly discourages using ={} for >> zeroing a structure. >> >> Regards, >> Christian.
On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: > Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > > On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > >> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > >>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > >>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: > >>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > >>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > >>>>> struct ttm_operation_ctx ctx = { false, false }; > >>>>> struct ttm_resource evict_mem; > >>>>> + struct ttm_place hop = {}; > >>>> Please always use memset() if you want to zero initialize something in > >>>> the kernel, we had enough trouble with that. > >>> What trouble is that? I've not heard of anything, and we use > >>> ={} quite extensively in drm land. > >> ={} initializes only named fields, not padding. > > Has that actually happened? > > YES! Numerous times! You wouldn't happen to have links/etc. to the discussion? I'd like to check them out.
Am 09.11.20 um 17:43 schrieb Ville Syrjälä: > On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: >> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: >>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: >>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: >>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: >>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: >>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) >>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { >>>>>>> struct ttm_operation_ctx ctx = { false, false }; >>>>>>> struct ttm_resource evict_mem; >>>>>>> + struct ttm_place hop = {}; >>>>>> Please always use memset() if you want to zero initialize something in >>>>>> the kernel, we had enough trouble with that. >>>>> What trouble is that? I've not heard of anything, and we use >>>>> ={} quite extensively in drm land. >>>> ={} initializes only named fields, not padding. >>> Has that actually happened? >> YES! Numerous times! > You wouldn't happen to have links/etc. to the discussion? > I'd like to check them out. Uff, that was years ago. Just google for something like "mesa memset hash", there was recently (~ the last year) another discussion because somebody ran into exactly that problem once more. Christian.
On Mon, Nov 09, 2020 at 09:48:04PM +0100, Christian König wrote: > Am 09.11.20 um 17:43 schrieb Ville Syrjälä: > > On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: > >> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > >>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > >>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > >>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > >>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: > >>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > >>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > >>>>>>> struct ttm_operation_ctx ctx = { false, false }; > >>>>>>> struct ttm_resource evict_mem; > >>>>>>> + struct ttm_place hop = {}; > >>>>>> Please always use memset() if you want to zero initialize something in > >>>>>> the kernel, we had enough trouble with that. > >>>>> What trouble is that? I've not heard of anything, and we use > >>>>> ={} quite extensively in drm land. > >>>> ={} initializes only named fields, not padding. > >>> Has that actually happened? > >> YES! Numerous times! > > You wouldn't happen to have links/etc. to the discussion? > > I'd like to check them out. > > Uff, that was years ago. Just google for something like "mesa memset > hash", there was recently (~ the last year) another discussion because > somebody ran into exactly that problem once more. Found this: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2071 which does suprise me a bit. Though I suspect ={} might behave differently since the compiler might treat it more like a memset(). Also makes me wonder about padding in general, because IIRC the standard allows padding to be clobbered even after initialization whenever any member is getting written. So I think technically there is no guaranteed way to clear the padding unless you never store anything into the struct.
On Tue, 10 Nov 2020 at 07:27, Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > > On Mon, Nov 09, 2020 at 09:48:04PM +0100, Christian König wrote: > > Am 09.11.20 um 17:43 schrieb Ville Syrjälä: > > > On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: > > >> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > > >>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > > >>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > > >>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > > >>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: > > >>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > > >>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > > >>>>>>> struct ttm_operation_ctx ctx = { false, false }; > > >>>>>>> struct ttm_resource evict_mem; > > >>>>>>> + struct ttm_place hop = {}; > > >>>>>> Please always use memset() if you want to zero initialize something in > > >>>>>> the kernel, we had enough trouble with that. > > >>>>> What trouble is that? I've not heard of anything, and we use > > >>>>> ={} quite extensively in drm land. > > >>>> ={} initializes only named fields, not padding. > > >>> Has that actually happened? > > >> YES! Numerous times! > > > You wouldn't happen to have links/etc. to the discussion? > > > I'd like to check them out. > > > > Uff, that was years ago. Just google for something like "mesa memset > > hash", there was recently (~ the last year) another discussion because > > somebody ran into exactly that problem once more. > > Found this: > https://gitlab.freedesktop.org/mesa/mesa/-/issues/2071 > which does suprise me a bit. Though I suspect ={} might > behave differently since the compiler might treat it > more like a memset(). It doesn't there's a bit of info out there on what happens, it really only matters for structs we are passing to userspace, but it's unlikely to matter here, but I've changed this to memset in my local tree, because why not. Dave
On Mon, Nov 09, 2020 at 10:54:29AM +1000, Dave Airlie wrote: > From: Dave Airlie <airlied@redhat.com> > > Currently drivers get called to move a buffer, but if they have to > move it temporarily through another space (SYSTEM->VRAM via TT) > then they can end up with a lot of ttm->driver->ttm call stacks, > if the temprorary space moves requires eviction. > > Instead of letting the driver do all the placement/space for the > temporary, allow it to report back (-EMULTIHOP) and a placement (hop) > to the move code, which will then do the temporary move, and the > correct placement move afterwards. > > This removes a lot of code from drivers, at the expense of > adding some midlayering. I've some further ideas on how to turn > it inside out, but I think this is a good solution to the call > stack problems. > > v2: separate out the driver patches, add WARN for getting > MULTHOP in paths we shouldn't (Daniel) > > Signed-off-by: Dave Airlie <airlied@redhat.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +- > drivers/gpu/drm/drm_gem_vram_helper.c | 3 +- > drivers/gpu/drm/nouveau/nouveau_bo.c | 3 +- > drivers/gpu/drm/qxl/qxl_ttm.c | 3 +- > drivers/gpu/drm/radeon/radeon_ttm.c | 3 +- > drivers/gpu/drm/ttm/ttm_bo.c | 68 +++++++++++++++++++--- > drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c | 3 +- > include/drm/ttm/ttm_bo_driver.h | 7 ++- > 8 files changed, 77 insertions(+), 16 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > index c01c060e4ac5..ce0d82802333 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > @@ -656,7 +656,8 @@ static bool amdgpu_mem_visible(struct amdgpu_device *adev, > */ > static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct amdgpu_device *adev; > struct amdgpu_bo *abo; > diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c > index 16d68c04ea5d..2cec7b1482b8 100644 > --- a/drivers/gpu/drm/drm_gem_vram_helper.c > +++ b/drivers/gpu/drm/drm_gem_vram_helper.c > @@ -964,7 +964,8 @@ static void bo_driver_delete_mem_notify(struct ttm_buffer_object *bo) > static int bo_driver_move(struct ttm_buffer_object *bo, > bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct drm_gem_vram_object *gbo; > > diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c > index 8133377d865d..fee07b9d19ed 100644 > --- a/drivers/gpu/drm/nouveau/nouveau_bo.c > +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c > @@ -1023,7 +1023,8 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo, > static int > nouveau_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_reg) > + struct ttm_resource *new_reg, > + struct ttm_place *hop) > { > struct nouveau_drm *drm = nouveau_bdev(bo->bdev); > struct nouveau_bo *nvbo = nouveau_bo(bo); > diff --git a/drivers/gpu/drm/qxl/qxl_ttm.c b/drivers/gpu/drm/qxl/qxl_ttm.c > index a80d59634143..128c38c8a837 100644 > --- a/drivers/gpu/drm/qxl/qxl_ttm.c > +++ b/drivers/gpu/drm/qxl/qxl_ttm.c > @@ -140,7 +140,8 @@ static void qxl_bo_move_notify(struct ttm_buffer_object *bo, > > static int qxl_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct ttm_resource *old_mem = &bo->mem; > int ret; > diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c > index 95038ac3382e..29062dbea299 100644 > --- a/drivers/gpu/drm/radeon/radeon_ttm.c > +++ b/drivers/gpu/drm/radeon/radeon_ttm.c > @@ -303,7 +303,8 @@ static int radeon_move_ram_vram(struct ttm_buffer_object *bo, > > static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct radeon_device *rdev; > struct radeon_bo *rbo; > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index e2a124b3affb..9f840f2a7836 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -231,7 +231,8 @@ EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); > > static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, > struct ttm_resource *mem, bool evict, > - struct ttm_operation_ctx *ctx) > + struct ttm_operation_ctx *ctx, > + struct ttm_place *hop) > { > struct ttm_bo_device *bdev = bo->bdev; > struct ttm_resource_manager *old_man = ttm_manager_type(bdev, bo->mem.mem_type); > @@ -259,9 +260,12 @@ static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, > } > } > > - ret = bdev->driver->move(bo, evict, ctx, mem); > - if (ret) > + ret = bdev->driver->move(bo, evict, ctx, mem, hop); > + if (ret) { > + if (ret == -EMULTIHOP) > + return ret; > goto out_err; > + } > > ctx->bytes_moved += bo->num_pages << PAGE_SHIFT; > return 0; > @@ -566,6 +570,7 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, > struct ttm_bo_device *bdev = bo->bdev; > struct ttm_resource evict_mem; > struct ttm_placement placement; > + struct ttm_place hop = {}; > int ret = 0; > > dma_resv_assert_held(bo->base.resv); > @@ -596,8 +601,9 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, > goto out; > } > > - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx); > + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx, &hop); > if (unlikely(ret)) { > + WARN(ret == -EMULTIHOP, "Unexpected multihop in eviction - likely driver bug\n"); > if (ret != -ERESTARTSYS) > pr_err("Buffer eviction failed\n"); > ttm_resource_free(bo, &evict_mem); > @@ -936,11 +942,39 @@ int ttm_bo_mem_space(struct ttm_buffer_object *bo, > } > EXPORT_SYMBOL(ttm_bo_mem_space); > > +static int ttm_bo_bounce_temp_buffer(struct ttm_buffer_object *bo, > + struct ttm_resource *mem, > + struct ttm_operation_ctx *ctx, > + struct ttm_place *hop) > +{ > + struct ttm_placement hop_placement; > + int ret; > + struct ttm_resource hop_mem = *mem; > + > + hop_mem.mm_node = NULL; > + hop_mem.mem_type = TTM_PL_SYSTEM; > + hop_mem.placement = 0; > + > + hop_placement.num_placement = hop_placement.num_busy_placement = 1; > + hop_placement.placement = hop_placement.busy_placement = hop; > + > + /* find space in the bounce domain */ > + ret = ttm_bo_mem_space(bo, &hop_placement, &hop_mem, ctx); > + if (ret) > + return ret; > + /* move to the bounce domain */ > + ret = ttm_bo_handle_move_mem(bo, &hop_mem, false, ctx, NULL); > + if (ret) > + return ret; > + return 0; > +} > + > static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, > struct ttm_placement *placement, > struct ttm_operation_ctx *ctx) > { > int ret = 0; > + struct ttm_place hop = {}; > struct ttm_resource mem; > > dma_resv_assert_held(bo->base.resv); > @@ -954,12 +988,25 @@ static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, > > /* > * Determine where to move the buffer. > + * > + * If driver determines move is going to need > + * an extra step then it will return -EMULTIHOP > + * and the buffer will be moved to the temporary > + * stop and the driver will be called to make > + * the second hop. > */ > +bounce: > ret = ttm_bo_mem_space(bo, placement, &mem, ctx); > if (ret) > - goto out_unlock; > - ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx); > -out_unlock: > + return ret; > + ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx, &hop); > + if (ret == -EMULTIHOP) { > + ret = ttm_bo_bounce_temp_buffer(bo, &mem, ctx, &hop); > + if (ret) > + return ret; > + /* try and move to final place now. */ > + goto bounce; > + } > if (ret) > ttm_resource_free(bo, &mem); > return ret; > @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > if (bo->mem.mem_type != TTM_PL_SYSTEM) { > struct ttm_operation_ctx ctx = { false, false }; > struct ttm_resource evict_mem; > + struct ttm_place hop = {}; > > evict_mem = bo->mem; > evict_mem.mm_node = NULL; > evict_mem.placement = 0; > evict_mem.mem_type = TTM_PL_SYSTEM; > > - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx); > - if (unlikely(ret != 0)) > + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx, &hop); > + if (unlikely(ret != 0)) { > + WARN(ret == -EMULTIHOP, "Unexpected multihop in swaput - likely driver bug.\n"); > goto out; > + } > } > > /** > diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > index 51f70bea41cc..6a04261ce760 100644 > --- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > @@ -695,7 +695,8 @@ static void vmw_swap_notify(struct ttm_buffer_object *bo) > static int vmw_move(struct ttm_buffer_object *bo, > bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct ttm_resource_manager *old_man = ttm_manager_type(bo->bdev, bo->mem.mem_type); > struct ttm_resource_manager *new_man = ttm_manager_type(bo->bdev, new_mem->mem_type); > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > index da8208f43378..f02f7cf9ae90 100644 > --- a/include/drm/ttm/ttm_bo_driver.h > +++ b/include/drm/ttm/ttm_bo_driver.h > @@ -121,6 +121,8 @@ struct ttm_bo_driver { > * Return the bo flags for a buffer which is not mapped to the hardware. > * These will be placed in proposed_flags so that when the move is > * finished, they'll end up in bo->mem.flags > + * This should not cause multihop evictions, and the core will warn > + * if one is proposed. > */ > > void (*evict_flags)(struct ttm_buffer_object *bo, > @@ -134,12 +136,15 @@ struct ttm_bo_driver { > * the graphics address space > * @ctx: context for this move with parameters > * @new_mem: the new memory region receiving the buffer > + @ @hop: placement for driver directed intermediate hop > * > * Move a buffer between two memory regions. > + * Returns errno -EMULTIHOP if driver requests a hop > */ > int (*move)(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem); > + struct ttm_resource *new_mem, > + struct ttm_place *hop); I think the warnings look all good, and everything else too. Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> On the = {} bikeshed I frankly don't care, since for anything where this matters vs memset you really should have a pad-less structure (by padding the structure properly). So imo = {} is perfectly fine. -Daniel > > /** > * struct ttm_bo_driver_member verify_access > -- > 2.27.0 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
On Tue, Nov 10, 2020 at 03:24:32PM +1000, Dave Airlie wrote: > On Tue, 10 Nov 2020 at 07:27, Ville Syrjälä > <ville.syrjala@linux.intel.com> wrote: > > > > On Mon, Nov 09, 2020 at 09:48:04PM +0100, Christian König wrote: > > > Am 09.11.20 um 17:43 schrieb Ville Syrjälä: > > > > On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: > > > >> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > > > >>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > > > >>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > > > >>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > > > >>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: > > > >>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > > > >>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > > > >>>>>>> struct ttm_operation_ctx ctx = { false, false }; > > > >>>>>>> struct ttm_resource evict_mem; > > > >>>>>>> + struct ttm_place hop = {}; > > > >>>>>> Please always use memset() if you want to zero initialize something in > > > >>>>>> the kernel, we had enough trouble with that. > > > >>>>> What trouble is that? I've not heard of anything, and we use > > > >>>>> ={} quite extensively in drm land. > > > >>>> ={} initializes only named fields, not padding. > > > >>> Has that actually happened? > > > >> YES! Numerous times! > > > > You wouldn't happen to have links/etc. to the discussion? > > > > I'd like to check them out. > > > > > > Uff, that was years ago. Just google for something like "mesa memset > > > hash", there was recently (~ the last year) another discussion because > > > somebody ran into exactly that problem once more. > > > > Found this: > > https://gitlab.freedesktop.org/mesa/mesa/-/issues/2071 > > which does suprise me a bit. Though I suspect ={} might > > behave differently since the compiler might treat it > > more like a memset(). > > It doesn't there's a bit of info out there on what happens, it really > only matters for structs we are passing to userspace, but it's > unlikely to matter here, > but I've changed this to memset in my local tree, because why not. Structs passed to userspace should really have no implicit padding. One of those how to botch your ioctl things... The main problems with memset() are: - it's ugly - people often get the sizeof wrong I guess the latter could be alleviated with a cpp macro that does the sizeof correctly for you.
On Tue, Nov 10, 2020 at 4:48 PM Ville Syrjälä <ville.syrjala@linux.intel.com> wrote: > > On Tue, Nov 10, 2020 at 03:24:32PM +1000, Dave Airlie wrote: > > On Tue, 10 Nov 2020 at 07:27, Ville Syrjälä > > <ville.syrjala@linux.intel.com> wrote: > > > > > > On Mon, Nov 09, 2020 at 09:48:04PM +0100, Christian König wrote: > > > > Am 09.11.20 um 17:43 schrieb Ville Syrjälä: > > > > > On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: > > > > >> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: > > > > >>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: > > > > >>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: > > > > >>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: > > > > >>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: > > > > >>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > > > > >>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { > > > > >>>>>>> struct ttm_operation_ctx ctx = { false, false }; > > > > >>>>>>> struct ttm_resource evict_mem; > > > > >>>>>>> + struct ttm_place hop = {}; > > > > >>>>>> Please always use memset() if you want to zero initialize something in > > > > >>>>>> the kernel, we had enough trouble with that. > > > > >>>>> What trouble is that? I've not heard of anything, and we use > > > > >>>>> ={} quite extensively in drm land. > > > > >>>> ={} initializes only named fields, not padding. > > > > >>> Has that actually happened? > > > > >> YES! Numerous times! > > > > > You wouldn't happen to have links/etc. to the discussion? > > > > > I'd like to check them out. > > > > > > > > Uff, that was years ago. Just google for something like "mesa memset > > > > hash", there was recently (~ the last year) another discussion because > > > > somebody ran into exactly that problem once more. > > > > > > Found this: > > > https://gitlab.freedesktop.org/mesa/mesa/-/issues/2071 > > > which does suprise me a bit. Though I suspect ={} might > > > behave differently since the compiler might treat it > > > more like a memset(). > > > > It doesn't there's a bit of info out there on what happens, it really > > only matters for structs we are passing to userspace, but it's > > unlikely to matter here, > > but I've changed this to memset in my local tree, because why not. > > Structs passed to userspace should really have no implicit padding. > One of those how to botch your ioctl things... > > The main problems with memset() are: > - it's ugly > - people often get the sizeof wrong > > I guess the latter could be alleviated with a cpp macro that > does the sizeof correctly for you. Yeah imo not using = {} and instead insisting on memset is really bad w/a for not padding your api structs properly. memset is only one place where this goes horribly wrong. I think for the gallium state tracker hasing issue memset is probably ok ot use for these specifically, with a big comment explaining why. And some code assurance that the memset is the same length you're passing to the crc function and all that. But anywhere were it's more (like anything related to api, which the gallium CSO hashing isnt) you really want to have no implicit holes at all. -Daniel
Am 09.11.20 um 01:54 schrieb Dave Airlie: > From: Dave Airlie <airlied@redhat.com> > > Currently drivers get called to move a buffer, but if they have to > move it temporarily through another space (SYSTEM->VRAM via TT) > then they can end up with a lot of ttm->driver->ttm call stacks, > if the temprorary space moves requires eviction. > > Instead of letting the driver do all the placement/space for the > temporary, allow it to report back (-EMULTIHOP) and a placement (hop) > to the move code, which will then do the temporary move, and the > correct placement move afterwards. > > This removes a lot of code from drivers, at the expense of > adding some midlayering. I've some further ideas on how to turn > it inside out, but I think this is a good solution to the call > stack problems. > > v2: separate out the driver patches, add WARN for getting > MULTHOP in paths we shouldn't (Daniel) > > Signed-off-by: Dave Airlie <airlied@redhat.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +- > drivers/gpu/drm/drm_gem_vram_helper.c | 3 +- > drivers/gpu/drm/nouveau/nouveau_bo.c | 3 +- > drivers/gpu/drm/qxl/qxl_ttm.c | 3 +- > drivers/gpu/drm/radeon/radeon_ttm.c | 3 +- > drivers/gpu/drm/ttm/ttm_bo.c | 68 +++++++++++++++++++--- > drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c | 3 +- > include/drm/ttm/ttm_bo_driver.h | 7 ++- > 8 files changed, 77 insertions(+), 16 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > index c01c060e4ac5..ce0d82802333 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > @@ -656,7 +656,8 @@ static bool amdgpu_mem_visible(struct amdgpu_device *adev, > */ > static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct amdgpu_device *adev; > struct amdgpu_bo *abo; > diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c > index 16d68c04ea5d..2cec7b1482b8 100644 > --- a/drivers/gpu/drm/drm_gem_vram_helper.c > +++ b/drivers/gpu/drm/drm_gem_vram_helper.c > @@ -964,7 +964,8 @@ static void bo_driver_delete_mem_notify(struct ttm_buffer_object *bo) > static int bo_driver_move(struct ttm_buffer_object *bo, > bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct drm_gem_vram_object *gbo; > > diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c > index 8133377d865d..fee07b9d19ed 100644 > --- a/drivers/gpu/drm/nouveau/nouveau_bo.c > +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c > @@ -1023,7 +1023,8 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo, > static int > nouveau_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_reg) > + struct ttm_resource *new_reg, > + struct ttm_place *hop) > { > struct nouveau_drm *drm = nouveau_bdev(bo->bdev); > struct nouveau_bo *nvbo = nouveau_bo(bo); > diff --git a/drivers/gpu/drm/qxl/qxl_ttm.c b/drivers/gpu/drm/qxl/qxl_ttm.c > index a80d59634143..128c38c8a837 100644 > --- a/drivers/gpu/drm/qxl/qxl_ttm.c > +++ b/drivers/gpu/drm/qxl/qxl_ttm.c > @@ -140,7 +140,8 @@ static void qxl_bo_move_notify(struct ttm_buffer_object *bo, > > static int qxl_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct ttm_resource *old_mem = &bo->mem; > int ret; > diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c > index 95038ac3382e..29062dbea299 100644 > --- a/drivers/gpu/drm/radeon/radeon_ttm.c > +++ b/drivers/gpu/drm/radeon/radeon_ttm.c > @@ -303,7 +303,8 @@ static int radeon_move_ram_vram(struct ttm_buffer_object *bo, > > static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct radeon_device *rdev; > struct radeon_bo *rbo; > diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c > index e2a124b3affb..9f840f2a7836 100644 > --- a/drivers/gpu/drm/ttm/ttm_bo.c > +++ b/drivers/gpu/drm/ttm/ttm_bo.c > @@ -231,7 +231,8 @@ EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); > > static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, > struct ttm_resource *mem, bool evict, > - struct ttm_operation_ctx *ctx) > + struct ttm_operation_ctx *ctx, > + struct ttm_place *hop) > { > struct ttm_bo_device *bdev = bo->bdev; > struct ttm_resource_manager *old_man = ttm_manager_type(bdev, bo->mem.mem_type); > @@ -259,9 +260,12 @@ static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, > } > } > > - ret = bdev->driver->move(bo, evict, ctx, mem); > - if (ret) > + ret = bdev->driver->move(bo, evict, ctx, mem, hop); > + if (ret) { > + if (ret == -EMULTIHOP) > + return ret; > goto out_err; > + } > > ctx->bytes_moved += bo->num_pages << PAGE_SHIFT; > return 0; > @@ -566,6 +570,7 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, > struct ttm_bo_device *bdev = bo->bdev; > struct ttm_resource evict_mem; > struct ttm_placement placement; > + struct ttm_place hop = {}; > int ret = 0; > > dma_resv_assert_held(bo->base.resv); > @@ -596,8 +601,9 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, > goto out; > } > > - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx); > + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx, &hop); > if (unlikely(ret)) { > + WARN(ret == -EMULTIHOP, "Unexpected multihop in eviction - likely driver bug\n"); > if (ret != -ERESTARTSYS) > pr_err("Buffer eviction failed\n"); > ttm_resource_free(bo, &evict_mem); > @@ -936,11 +942,39 @@ int ttm_bo_mem_space(struct ttm_buffer_object *bo, > } > EXPORT_SYMBOL(ttm_bo_mem_space); > > +static int ttm_bo_bounce_temp_buffer(struct ttm_buffer_object *bo, > + struct ttm_resource *mem, > + struct ttm_operation_ctx *ctx, > + struct ttm_place *hop) > +{ > + struct ttm_placement hop_placement; > + int ret; > + struct ttm_resource hop_mem = *mem; > + > + hop_mem.mm_node = NULL; > + hop_mem.mem_type = TTM_PL_SYSTEM; > + hop_mem.placement = 0; > + > + hop_placement.num_placement = hop_placement.num_busy_placement = 1; > + hop_placement.placement = hop_placement.busy_placement = hop; > + > + /* find space in the bounce domain */ > + ret = ttm_bo_mem_space(bo, &hop_placement, &hop_mem, ctx); > + if (ret) > + return ret; > + /* move to the bounce domain */ > + ret = ttm_bo_handle_move_mem(bo, &hop_mem, false, ctx, NULL); > + if (ret) > + return ret; > + return 0; > +} > + > static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, > struct ttm_placement *placement, > struct ttm_operation_ctx *ctx) > { > int ret = 0; > + struct ttm_place hop = {}; > struct ttm_resource mem; > > dma_resv_assert_held(bo->base.resv); > @@ -954,12 +988,25 @@ static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, > > /* > * Determine where to move the buffer. > + * > + * If driver determines move is going to need > + * an extra step then it will return -EMULTIHOP > + * and the buffer will be moved to the temporary > + * stop and the driver will be called to make > + * the second hop. > */ > +bounce: > ret = ttm_bo_mem_space(bo, placement, &mem, ctx); > if (ret) > - goto out_unlock; > - ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx); > -out_unlock: > + return ret; > + ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx, &hop); > + if (ret == -EMULTIHOP) { > + ret = ttm_bo_bounce_temp_buffer(bo, &mem, ctx, &hop); > + if (ret) > + return ret; > + /* try and move to final place now. */ > + goto bounce; > + } > if (ret) > ttm_resource_free(bo, &mem); > return ret; > @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) > if (bo->mem.mem_type != TTM_PL_SYSTEM) { > struct ttm_operation_ctx ctx = { false, false }; > struct ttm_resource evict_mem; > + struct ttm_place hop = {}; Please always use memset() if you want to zero initialize something in the kernel, we had enough trouble with that. Apart from that looks good to me, Christian. > > evict_mem = bo->mem; > evict_mem.mm_node = NULL; > evict_mem.placement = 0; > evict_mem.mem_type = TTM_PL_SYSTEM; > > - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx); > - if (unlikely(ret != 0)) > + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx, &hop); > + if (unlikely(ret != 0)) { > + WARN(ret == -EMULTIHOP, "Unexpected multihop in swaput - likely driver bug.\n"); > goto out; > + } > } > > /** > diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > index 51f70bea41cc..6a04261ce760 100644 > --- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c > @@ -695,7 +695,8 @@ static void vmw_swap_notify(struct ttm_buffer_object *bo) > static int vmw_move(struct ttm_buffer_object *bo, > bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem) > + struct ttm_resource *new_mem, > + struct ttm_place *hop) > { > struct ttm_resource_manager *old_man = ttm_manager_type(bo->bdev, bo->mem.mem_type); > struct ttm_resource_manager *new_man = ttm_manager_type(bo->bdev, new_mem->mem_type); > diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h > index da8208f43378..f02f7cf9ae90 100644 > --- a/include/drm/ttm/ttm_bo_driver.h > +++ b/include/drm/ttm/ttm_bo_driver.h > @@ -121,6 +121,8 @@ struct ttm_bo_driver { > * Return the bo flags for a buffer which is not mapped to the hardware. > * These will be placed in proposed_flags so that when the move is > * finished, they'll end up in bo->mem.flags > + * This should not cause multihop evictions, and the core will warn > + * if one is proposed. > */ > > void (*evict_flags)(struct ttm_buffer_object *bo, > @@ -134,12 +136,15 @@ struct ttm_bo_driver { > * the graphics address space > * @ctx: context for this move with parameters > * @new_mem: the new memory region receiving the buffer > + @ @hop: placement for driver directed intermediate hop > * > * Move a buffer between two memory regions. > + * Returns errno -EMULTIHOP if driver requests a hop > */ > int (*move)(struct ttm_buffer_object *bo, bool evict, > struct ttm_operation_ctx *ctx, > - struct ttm_resource *new_mem); > + struct ttm_resource *new_mem, > + struct ttm_place *hop); > > /** > * struct ttm_bo_driver_member verify_access
Am 10.11.20 um 18:11 schrieb Daniel Vetter: > On Tue, Nov 10, 2020 at 4:48 PM Ville Syrjälä > <ville.syrjala@linux.intel.com> wrote: >> On Tue, Nov 10, 2020 at 03:24:32PM +1000, Dave Airlie wrote: >>> On Tue, 10 Nov 2020 at 07:27, Ville Syrjälä >>> <ville.syrjala@linux.intel.com> wrote: >>>> On Mon, Nov 09, 2020 at 09:48:04PM +0100, Christian König wrote: >>>>> Am 09.11.20 um 17:43 schrieb Ville Syrjälä: >>>>>> On Mon, Nov 09, 2020 at 05:20:17PM +0100, Christian König wrote: >>>>>>> Am 09.11.20 um 17:18 schrieb Ville Syrjälä: >>>>>>>> On Mon, Nov 09, 2020 at 04:57:29PM +0100, Christian König wrote: >>>>>>>>> Am 09.11.20 um 16:16 schrieb Ville Syrjälä: >>>>>>>>>> On Wed, Nov 11, 2020 at 06:13:02PM +0100, Christian König wrote: >>>>>>>>>>> Am 09.11.20 um 01:54 schrieb Dave Airlie: >>>>>>>>>>>> @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) >>>>>>>>>>>> if (bo->mem.mem_type != TTM_PL_SYSTEM) { >>>>>>>>>>>> struct ttm_operation_ctx ctx = { false, false }; >>>>>>>>>>>> struct ttm_resource evict_mem; >>>>>>>>>>>> + struct ttm_place hop = {}; >>>>>>>>>>> Please always use memset() if you want to zero initialize something in >>>>>>>>>>> the kernel, we had enough trouble with that. >>>>>>>>>> What trouble is that? I've not heard of anything, and we use >>>>>>>>>> ={} quite extensively in drm land. >>>>>>>>> ={} initializes only named fields, not padding. >>>>>>>> Has that actually happened? >>>>>>> YES! Numerous times! >>>>>> You wouldn't happen to have links/etc. to the discussion? >>>>>> I'd like to check them out. >>>>> Uff, that was years ago. Just google for something like "mesa memset >>>>> hash", there was recently (~ the last year) another discussion because >>>>> somebody ran into exactly that problem once more. >>>> Found this: >>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F2071&data=04%7C01%7Cchristian.koenig%40amd.com%7C66be3d367f2b401b2e2808d8859ba4ee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637406250838085232%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=X2z5zxsBvhXQvmd2oJzuN2YzDMGZnYUff6qxuJL%2BjLE%3D&reserved=0 >>>> which does suprise me a bit. Though I suspect ={} might >>>> behave differently since the compiler might treat it >>>> more like a memset(). >>> It doesn't there's a bit of info out there on what happens, it really >>> only matters for structs we are passing to userspace, but it's >>> unlikely to matter here, >>> but I've changed this to memset in my local tree, because why not. >> Structs passed to userspace should really have no implicit padding. >> One of those how to botch your ioctl things... >> >> The main problems with memset() are: >> - it's ugly >> - people often get the sizeof wrong >> >> I guess the latter could be alleviated with a cpp macro that >> does the sizeof correctly for you. > Yeah imo not using = {} and instead insisting on memset is really bad > w/a for not padding your api structs properly. memset is only one > place where this goes horribly wrong. I'm not a fan of memset either, but there are more problems than just the padding in the UAPI. I've seen at least the following in the wild: 1. UAPI information leak. 2. Hash and fingerprinting functions returning unstable results even when all meaningful members of a structure are initialized. 3. Valgrind/KASAN/Coverity randomly pointing this out as problem. 4. There is the discussion if ={} (not ISO C conform) or ={0] or even ={{0}} should be used. My preference would be to just teach compilers that not initializing padding in the kernel is a undesired behavior which should be avoided. Regards Christian. > > I think for the gallium state tracker hasing issue memset is probably > ok ot use for these specifically, with a big comment explaining why. > And some code assurance that the memset is the same length you're > passing to the crc function and all that. But anywhere were it's more > (like anything related to api, which the gallium CSO hashing isnt) you > really want to have no implicit holes at all. > -Daniel
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index c01c060e4ac5..ce0d82802333 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -656,7 +656,8 @@ static bool amdgpu_mem_visible(struct amdgpu_device *adev, */ static int amdgpu_bo_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *new_mem, + struct ttm_place *hop) { struct amdgpu_device *adev; struct amdgpu_bo *abo; diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c index 16d68c04ea5d..2cec7b1482b8 100644 --- a/drivers/gpu/drm/drm_gem_vram_helper.c +++ b/drivers/gpu/drm/drm_gem_vram_helper.c @@ -964,7 +964,8 @@ static void bo_driver_delete_mem_notify(struct ttm_buffer_object *bo) static int bo_driver_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *new_mem, + struct ttm_place *hop) { struct drm_gem_vram_object *gbo; diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c index 8133377d865d..fee07b9d19ed 100644 --- a/drivers/gpu/drm/nouveau/nouveau_bo.c +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c @@ -1023,7 +1023,8 @@ nouveau_bo_vm_cleanup(struct ttm_buffer_object *bo, static int nouveau_bo_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_reg) + struct ttm_resource *new_reg, + struct ttm_place *hop) { struct nouveau_drm *drm = nouveau_bdev(bo->bdev); struct nouveau_bo *nvbo = nouveau_bo(bo); diff --git a/drivers/gpu/drm/qxl/qxl_ttm.c b/drivers/gpu/drm/qxl/qxl_ttm.c index a80d59634143..128c38c8a837 100644 --- a/drivers/gpu/drm/qxl/qxl_ttm.c +++ b/drivers/gpu/drm/qxl/qxl_ttm.c @@ -140,7 +140,8 @@ static void qxl_bo_move_notify(struct ttm_buffer_object *bo, static int qxl_bo_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *new_mem, + struct ttm_place *hop) { struct ttm_resource *old_mem = &bo->mem; int ret; diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c index 95038ac3382e..29062dbea299 100644 --- a/drivers/gpu/drm/radeon/radeon_ttm.c +++ b/drivers/gpu/drm/radeon/radeon_ttm.c @@ -303,7 +303,8 @@ static int radeon_move_ram_vram(struct ttm_buffer_object *bo, static int radeon_bo_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *new_mem, + struct ttm_place *hop) { struct radeon_device *rdev; struct radeon_bo *rbo; diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index e2a124b3affb..9f840f2a7836 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -231,7 +231,8 @@ EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_resource *mem, bool evict, - struct ttm_operation_ctx *ctx) + struct ttm_operation_ctx *ctx, + struct ttm_place *hop) { struct ttm_bo_device *bdev = bo->bdev; struct ttm_resource_manager *old_man = ttm_manager_type(bdev, bo->mem.mem_type); @@ -259,9 +260,12 @@ static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, } } - ret = bdev->driver->move(bo, evict, ctx, mem); - if (ret) + ret = bdev->driver->move(bo, evict, ctx, mem, hop); + if (ret) { + if (ret == -EMULTIHOP) + return ret; goto out_err; + } ctx->bytes_moved += bo->num_pages << PAGE_SHIFT; return 0; @@ -566,6 +570,7 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, struct ttm_bo_device *bdev = bo->bdev; struct ttm_resource evict_mem; struct ttm_placement placement; + struct ttm_place hop = {}; int ret = 0; dma_resv_assert_held(bo->base.resv); @@ -596,8 +601,9 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, goto out; } - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx); + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, ctx, &hop); if (unlikely(ret)) { + WARN(ret == -EMULTIHOP, "Unexpected multihop in eviction - likely driver bug\n"); if (ret != -ERESTARTSYS) pr_err("Buffer eviction failed\n"); ttm_resource_free(bo, &evict_mem); @@ -936,11 +942,39 @@ int ttm_bo_mem_space(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_mem_space); +static int ttm_bo_bounce_temp_buffer(struct ttm_buffer_object *bo, + struct ttm_resource *mem, + struct ttm_operation_ctx *ctx, + struct ttm_place *hop) +{ + struct ttm_placement hop_placement; + int ret; + struct ttm_resource hop_mem = *mem; + + hop_mem.mm_node = NULL; + hop_mem.mem_type = TTM_PL_SYSTEM; + hop_mem.placement = 0; + + hop_placement.num_placement = hop_placement.num_busy_placement = 1; + hop_placement.placement = hop_placement.busy_placement = hop; + + /* find space in the bounce domain */ + ret = ttm_bo_mem_space(bo, &hop_placement, &hop_mem, ctx); + if (ret) + return ret; + /* move to the bounce domain */ + ret = ttm_bo_handle_move_mem(bo, &hop_mem, false, ctx, NULL); + if (ret) + return ret; + return 0; +} + static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, struct ttm_placement *placement, struct ttm_operation_ctx *ctx) { int ret = 0; + struct ttm_place hop = {}; struct ttm_resource mem; dma_resv_assert_held(bo->base.resv); @@ -954,12 +988,25 @@ static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, /* * Determine where to move the buffer. + * + * If driver determines move is going to need + * an extra step then it will return -EMULTIHOP + * and the buffer will be moved to the temporary + * stop and the driver will be called to make + * the second hop. */ +bounce: ret = ttm_bo_mem_space(bo, placement, &mem, ctx); if (ret) - goto out_unlock; - ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx); -out_unlock: + return ret; + ret = ttm_bo_handle_move_mem(bo, &mem, false, ctx, &hop); + if (ret == -EMULTIHOP) { + ret = ttm_bo_bounce_temp_buffer(bo, &mem, ctx, &hop); + if (ret) + return ret; + /* try and move to final place now. */ + goto bounce; + } if (ret) ttm_resource_free(bo, &mem); return ret; @@ -1432,15 +1479,18 @@ int ttm_bo_swapout(struct ttm_operation_ctx *ctx) if (bo->mem.mem_type != TTM_PL_SYSTEM) { struct ttm_operation_ctx ctx = { false, false }; struct ttm_resource evict_mem; + struct ttm_place hop = {}; evict_mem = bo->mem; evict_mem.mm_node = NULL; evict_mem.placement = 0; evict_mem.mem_type = TTM_PL_SYSTEM; - ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx); - if (unlikely(ret != 0)) + ret = ttm_bo_handle_move_mem(bo, &evict_mem, true, &ctx, &hop); + if (unlikely(ret != 0)) { + WARN(ret == -EMULTIHOP, "Unexpected multihop in swaput - likely driver bug.\n"); goto out; + } } /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c index 51f70bea41cc..6a04261ce760 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c @@ -695,7 +695,8 @@ static void vmw_swap_notify(struct ttm_buffer_object *bo) static int vmw_move(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem) + struct ttm_resource *new_mem, + struct ttm_place *hop) { struct ttm_resource_manager *old_man = ttm_manager_type(bo->bdev, bo->mem.mem_type); struct ttm_resource_manager *new_man = ttm_manager_type(bo->bdev, new_mem->mem_type); diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index da8208f43378..f02f7cf9ae90 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -121,6 +121,8 @@ struct ttm_bo_driver { * Return the bo flags for a buffer which is not mapped to the hardware. * These will be placed in proposed_flags so that when the move is * finished, they'll end up in bo->mem.flags + * This should not cause multihop evictions, and the core will warn + * if one is proposed. */ void (*evict_flags)(struct ttm_buffer_object *bo, @@ -134,12 +136,15 @@ struct ttm_bo_driver { * the graphics address space * @ctx: context for this move with parameters * @new_mem: the new memory region receiving the buffer + @ @hop: placement for driver directed intermediate hop * * Move a buffer between two memory regions. + * Returns errno -EMULTIHOP if driver requests a hop */ int (*move)(struct ttm_buffer_object *bo, bool evict, struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem); + struct ttm_resource *new_mem, + struct ttm_place *hop); /** * struct ttm_bo_driver_member verify_access