[RFC,v4,00/12] Waitboost drm syncobj waits

Message

Tvrtko Ursulin March 7, 2023, 10:48 a.m. UTC

From: Tvrtko Ursulin <[e-mail]>

In i915 we have this concept of "wait boosting" where we give a priority boost
for instance to fences which are actively waited upon from userspace. This has
it's pros and cons and can certainly be discussed at lenght. However fact is
some workloads really like it.

Problem is that with the arrival of drm syncobj and a new userspace waiting
entry point it added, the waitboost mechanism was bypassed. AFAIU this mostly
happens with all Vulkan based userspaces. Hence I cooked up this mini series to
see if discussion about restoring the waitboost can be had.

The series adds a concept of "wait count" to dma fence which is intended to
represent explicit userspace waits. It is therefore incremented for every
explicit dma_fence_enable_sw_signaling and dma_fence_add_wait_callback (like
dma_fence_add_callback but from explicit/userspace wait paths). Individual
drivers can then inspect this via dma_fence_wait_count() and decide to wait
boost the waits on such fences.

Patch has been slightly tested for performance impact by Google using some clvk
workloads and shows a good improvement (frame time improved from 16ms to 13ms).

It is also important to mention that although benefits of waitboosting may not
only be with workloads related to frame presentation time, but also with
serialized computations which constantly move between the CPU and GPU, there are
also workloads which do not improve on the performance but degrade on
efficiency. Therefore extending this approach needs to be carefully evaluated.

*)
https://gitlab.freedesktop.org/drm/intel/-/issues/8014

v2:
 * Small fixups based on CI feedback:
    * Handle decrement correctly for already signalled case while adding callback.
    * Remove i915 assert which was making sure struct i915_request does not grow.
 * Split out the i915 patch into three separate functional changes.

v3:
 * Handle drivers which open-code callback additions.

v4:
 * Handle dma-fence-array and dma-fence-chain.

Tvrtko Ursulin (12):
  dma-fence: Move i915 helpers into common
  dma-fence: Add callback initialization helper
  drm/i915: Use fence callback initialization helper
  drm/vmwgfx: Use fence callback initialization helper
  dma-fence: Track explicit waiters
  dma-fence: Export __dma_fence_add_callback
  dma-fence-array: Propagate wait status to contained fences
  dma-fence-chain: Propagate wait status to contained fences
  drm/syncobj: Mark syncobj waits as external waiters
  drm/i915: Waitboost external waits
  drm/i915: Mark waits as explicit
  drm/i915: Wait boost requests waited upon by others

 drivers/dma-buf/dma-fence-array.c           |   5 +-
 drivers/dma-buf/dma-fence-chain.c           |  22 +++-
 drivers/dma-buf/dma-fence.c                 | 138 ++++++++++++++------
 drivers/gpu/drm/drm_syncobj.c               |   6 +-
 drivers/gpu/drm/i915/gt/intel_breadcrumbs.c |  22 ----
 drivers/gpu/drm/i915/gt/intel_engine_pm.c   |   1 -
 drivers/gpu/drm/i915/i915_active.c          |   2 +-
 drivers/gpu/drm/i915/i915_active.h          |   2 +-
 drivers/gpu/drm/i915/i915_request.c         |  13 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_fence.c       |   2 +-
 include/linux/dma-fence-chain.h             |   1 +
 include/linux/dma-fence.h                   |  29 ++++
 12 files changed, 165 insertions(+), 78 deletions(-)