mbox series

[v2,0/8] per-process address spaces for MMUv2

Message ID 20190417135023.26977-1-l.stach@pengutronix.de (mailing list archive)
Headers show
Series per-process address spaces for MMUv2 | expand

Message

Lucas Stach April 17, 2019, 1:50 p.m. UTC
Hi all,

v1 cover letter:

the following patches finally implement one of the longstanding TODO
items in the etnaviv driver: per-process address spaces. They are only
enabled for MMUv2, as switching the MMU context on MMUv1 would require
a full stop of the FE, which I deemed too expensive to do on potentially
every submitted commandstream.

For now this only provides better isolation between GPU clients, but it
is also a big step in the direction of supporting softpin. Softpin will
later be used by GC7000 userspace drivers to deal with texture descriptors
without the need to add even more relocation interfaces to the etnaviv
UAPI.

The changes are big and pretty disruptive, so I acknowledge that they
aren't prime targets for a quick review, but I would appreciate a
second pairs of eyes on them.

Changes since v1:
- fixed an issue where a debugsfs read could run into a kernel NULL
  ptr dereference due to no current MMU context
- fixed an issue where the current MMU context could be destroyed
  due to the DRM client going away, while it is still in use by
  an active GPU job
- more extensive testing on GC880, GC2000, GC3000 and GC7000

Regards,
Lucas

Lucas Stach (8):
  drm/etnaviv: split out cmdbuf mapping into address space
  drm/etnaviv: share a single cmdbuf suballoc region across all GPUs
  drm/etnaviv: replace MMU flush marker with flush sequence
  drm/etnaviv: rework MMU handling
  drm/etnaviv: split out starting of FE idle loop
  drm/etnaviv: provide MMU context to etnaviv_gem_mapping_get
  drm/etnaviv: implement per-process address spaces on MMUv2
  drm/etnaviv: dump only failing submit

 drivers/gpu/drm/etnaviv/etnaviv_buffer.c     |  83 +++--
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c     |  56 ++--
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h     |  16 +-
 drivers/gpu/drm/etnaviv/etnaviv_drv.c        |  50 ++-
 drivers/gpu/drm/etnaviv/etnaviv_drv.h        |  12 +-
 drivers/gpu/drm/etnaviv/etnaviv_dump.c       |  64 ++--
 drivers/gpu/drm/etnaviv/etnaviv_dump.h       |   4 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem.c        |  27 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem.h        |   5 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c |  12 +-
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c        | 134 ++++-----
 drivers/gpu/drm/etnaviv/etnaviv_gpu.h        |  12 +-
 drivers/gpu/drm/etnaviv/etnaviv_iommu.c      | 160 +++++-----
 drivers/gpu/drm/etnaviv/etnaviv_iommu.h      |  20 --
 drivers/gpu/drm/etnaviv/etnaviv_iommu_v2.c   | 279 ++++++++---------
 drivers/gpu/drm/etnaviv/etnaviv_mmu.c        | 301 ++++++++++++-------
 drivers/gpu/drm/etnaviv/etnaviv_mmu.h        | 109 +++++--
 drivers/gpu/drm/etnaviv/etnaviv_sched.c      |   2 +-
 18 files changed, 740 insertions(+), 606 deletions(-)
 delete mode 100644 drivers/gpu/drm/etnaviv/etnaviv_iommu.h

Comments

Guido Günther May 3, 2019, 11:10 a.m. UTC | #1
Hi Lucas,
On Wed, Apr 17, 2019 at 03:50:15PM +0200, Lucas Stach wrote:
> 
> Hi all,
> 
> v1 cover letter:
> 
> the following patches finally implement one of the longstanding TODO
> items in the etnaviv driver: per-process address spaces. They are only
> enabled for MMUv2, as switching the MMU context on MMUv1 would require
> a full stop of the FE, which I deemed too expensive to do on potentially
> every submitted commandstream.
> 
> For now this only provides better isolation between GPU clients, but it
> is also a big step in the direction of supporting softpin. Softpin will
> later be used by GC7000 userspace drivers to deal with texture descriptors
> without the need to add even more relocation interfaces to the etnaviv
> UAPI.
> 
> The changes are big and pretty disruptive, so I acknowledge that they
> aren't prime targets for a quick review, but I would appreciate a
> second pairs of eyes on them.
> 
> Changes since v1:
> - fixed an issue where a debugsfs read could run into a kernel NULL
>   ptr dereference due to no current MMU context
> - fixed an issue where the current MMU context could be destroyed
>   due to the DRM client going away, while it is still in use by
>   an active GPU job
> - more extensive testing on GC880, GC2000, GC3000 and GC7000

I gave this series a test on GC7000 and it looks good. I'll do some more
testing over the next week.
Cheers,
 -- Guido
Guido Günther May 10, 2019, 3:07 p.m. UTC | #2
Hi Lucas,
On Fri, May 03, 2019 at 01:10:26PM +0200, Guido Günther wrote:
> Hi Lucas,
> On Wed, Apr 17, 2019 at 03:50:15PM +0200, Lucas Stach wrote:
> > 
> > Hi all,
> > 
> > v1 cover letter:
> > 
> > the following patches finally implement one of the longstanding TODO
> > items in the etnaviv driver: per-process address spaces. They are only
> > enabled for MMUv2, as switching the MMU context on MMUv1 would require
> > a full stop of the FE, which I deemed too expensive to do on potentially
> > every submitted commandstream.
> > 
> > For now this only provides better isolation between GPU clients, but it
> > is also a big step in the direction of supporting softpin. Softpin will
> > later be used by GC7000 userspace drivers to deal with texture descriptors
> > without the need to add even more relocation interfaces to the etnaviv
> > UAPI.
> > 
> > The changes are big and pretty disruptive, so I acknowledge that they
> > aren't prime targets for a quick review, but I would appreciate a
> > second pairs of eyes on them.
> > 
> > Changes since v1:
> > - fixed an issue where a debugsfs read could run into a kernel NULL
> >   ptr dereference due to no current MMU context
> > - fixed an issue where the current MMU context could be destroyed
> >   due to the DRM client going away, while it is still in use by
> >   an active GPU job
> > - more extensive testing on GC880, GC2000, GC3000 and GC7000
> 
> I gave this series a test on GC7000 and it looks good. I'll do some more
> testing over the next week.

I gave this some more testing and it works nicely. One thing i noticed
though are occasional GPU hangs on compositor startup like:

[   58.929906] etnaviv-gpu 38000000.gpu: MMU fault status 0x00000001
[   58.936021] etnaviv-gpu 38000000.gpu: MMU 0 fault addr 0x00484b80
[   59.972655] etnaviv-gpu 38000000.gpu: recover hung GPU!

but these only happen with GALLIUM_DDEBUG=always. I can't reproduce them
when either

- dropping this patch series
- disabling GALLIUM_DDEBUG

I see if I can figure out some more details.
Cheers,
 -- Guido