diff mbox series

[1/2] drm/i915/icl: Default to Thread Group preemption for compute workloads

Message ID 20190305124827.23446-1-michal.winiarski@intel.com (mailing list archive)
State New, archived
Headers show
Series [1/2] drm/i915/icl: Default to Thread Group preemption for compute workloads | expand

Commit Message

Michał Winiarski March 5, 2019, 12:48 p.m. UTC 
We assumed that the default preemption granularity is fine for ICL.
Unfortunately, it turns out that some drivers don't support mid-thread
preemption for compute workloads.
If a workload that doesn't support mid-thread preemption gets mid-thread
preempted, we're going to observe a GPU hang.
While I'm here, let's also update the "workaround" naming.

Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
Cc: Anuj Phogat <anuj.phogat@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Cc: Rafael Antognolli <rafael.antognolli@intel.com>
Tested-by: Anuj Phogat <anuj.phogat@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/i915/intel_workarounds.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Comments

Rafael Antognolli March 5, 2019, 5:30 p.m. UTC | #1 
On Tue, Mar 05, 2019 at 01:48:26PM +0100, Michał Winiarski wrote:
> We assumed that the default preemption granularity is fine for ICL.
> Unfortunately, it turns out that some drivers don't support mid-thread
> preemption for compute workloads.
> If a workload that doesn't support mid-thread preemption gets mid-thread
> preempted, we're going to observe a GPU hang.
> While I'm here, let's also update the "workaround" naming.

Yeah, in Mesa we are not implementing the SIP, so we can't do
thread-level preemption yet and need the granularity to be no higher
than thread group level.

Acked-by: Rafael Antognolli <rafael.antognolli@intel.com>

> Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
> Cc: Anuj Phogat <anuj.phogat@intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Matt Roper <matthew.d.roper@intel.com>
> Cc: Rafael Antognolli <rafael.antognolli@intel.com>
> Tested-by: Anuj Phogat <anuj.phogat@intel.com>
> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/i915/intel_workarounds.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/intel_workarounds.c b/drivers/gpu/drm/i915/intel_workarounds.c
> index 89b4007d5200..2fba33509f4e 100644
> --- a/drivers/gpu/drm/i915/intel_workarounds.c
> +++ b/drivers/gpu/drm/i915/intel_workarounds.c
> @@ -555,6 +555,11 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine)
>  			   GEN10_CACHE_MODE_SS,
>  			   0, /* write-only, so skip validation */
>  			   _MASKED_BIT_ENABLE(FLOAT_BLEND_OPTIMIZATION_ENABLE));
> +
> +	/* WaDisableGPGPUMidThreadPreemption:icl */
> +	WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1,
> +			    GEN9_PREEMPT_GPGPU_LEVEL_MASK,
> +			    GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL);
>  }
>  
>  void intel_engine_init_ctx_wa(struct intel_engine_cs *engine)
> @@ -1162,8 +1167,8 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
>  				    GEN7_DISABLE_SAMPLER_PREFETCH);
>  	}
>  
> -	if (IS_GEN(i915, 9) || IS_CANNONLAKE(i915)) {
> -		/* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,cnl */
> +	if (IS_GEN_RANGE(i915, 9, 11)) {
> +		/* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl */
>  		wa_masked_en(wal,
>  			     GEN7_FF_SLICE_CS_CHICKEN1,
>  			     GEN9_FFSC_PERCTX_PREEMPT_CTRL);
> -- 
> 2.20.1
>
Anuj Phogat March 5, 2019, 5:37 p.m. UTC | #2 
Fixes multiple gpu hangs in piglit and vulkancts.
Both patches are:
Tested-by: Anuj Phogat <anuj.phogat@intel.com>

On Tue, Mar 5, 2019 at 4:48 AM Michał Winiarski
<michal.winiarski@intel.com> wrote:
>
> We assumed that the default preemption granularity is fine for ICL.
> Unfortunately, it turns out that some drivers don't support mid-thread
> preemption for compute workloads.
> If a workload that doesn't support mid-thread preemption gets mid-thread
> preempted, we're going to observe a GPU hang.
> While I'm here, let's also update the "workaround" naming.
>
> Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
> Cc: Anuj Phogat <anuj.phogat@intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Cc: Matt Roper <matthew.d.roper@intel.com>
> Cc: Rafael Antognolli <rafael.antognolli@intel.com>
> Tested-by: Anuj Phogat <anuj.phogat@intel.com>
> Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/i915/intel_workarounds.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/intel_workarounds.c b/drivers/gpu/drm/i915/intel_workarounds.c
> index 89b4007d5200..2fba33509f4e 100644
> --- a/drivers/gpu/drm/i915/intel_workarounds.c
> +++ b/drivers/gpu/drm/i915/intel_workarounds.c
> @@ -555,6 +555,11 @@ static void icl_ctx_workarounds_init(struct intel_engine_cs *engine)
>                            GEN10_CACHE_MODE_SS,
>                            0, /* write-only, so skip validation */
>                            _MASKED_BIT_ENABLE(FLOAT_BLEND_OPTIMIZATION_ENABLE));
> +
> +       /* WaDisableGPGPUMidThreadPreemption:icl */
> +       WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1,
> +                           GEN9_PREEMPT_GPGPU_LEVEL_MASK,
> +                           GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL);
>  }
>
>  void intel_engine_init_ctx_wa(struct intel_engine_cs *engine)
> @@ -1162,8 +1167,8 @@ rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
>                                     GEN7_DISABLE_SAMPLER_PREFETCH);
>         }
>
> -       if (IS_GEN(i915, 9) || IS_CANNONLAKE(i915)) {
> -               /* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,cnl */
> +       if (IS_GEN_RANGE(i915, 9, 11)) {
> +               /* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl */
>                 wa_masked_en(wal,
>                              GEN7_FF_SLICE_CS_CHICKEN1,
>                              GEN9_FFSC_PERCTX_PREEMPT_CTRL);
> --
> 2.20.1
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Chris Wilson March 5, 2019, 7:10 p.m. UTC | #3 
Quoting Rafael Antognolli (2019-03-05 17:30:00)
> On Tue, Mar 05, 2019 at 01:48:26PM +0100, Michał Winiarski wrote:
> > We assumed that the default preemption granularity is fine for ICL.
> > Unfortunately, it turns out that some drivers don't support mid-thread
> > preemption for compute workloads.
> > If a workload that doesn't support mid-thread preemption gets mid-thread
> > preempted, we're going to observe a GPU hang.
> > While I'm here, let's also update the "workaround" naming.
> 
> Yeah, in Mesa we are not implementing the SIP, so we can't do
> thread-level preemption yet and need the granularity to be no higher
> than thread group level.
> 
> Acked-by: Rafael Antognolli <rafael.antognolli@intel.com>
> 
> > Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
> > Cc: Anuj Phogat <anuj.phogat@intel.com>
> > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > Cc: Matt Roper <matthew.d.roper@intel.com>
> > Cc: Rafael Antognolli <rafael.antognolli@intel.com>
> > Tested-by: Anuj Phogat <anuj.phogat@intel.com>
> > Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

And pushed, thanks everyone for the testing and reviewed. I've held off
on pushing the second patch as we just want to double check that the
whitelisting is required.
-Chris
Joonas Lahtinen March 6, 2019, 11:20 a.m. UTC | #4 
Quoting Chris Wilson (2019-03-05 21:10:42)
> Quoting Rafael Antognolli (2019-03-05 17:30:00)
> > On Tue, Mar 05, 2019 at 01:48:26PM +0100, Michał Winiarski wrote:
> > > We assumed that the default preemption granularity is fine for ICL.
> > > Unfortunately, it turns out that some drivers don't support mid-thread
> > > preemption for compute workloads.
> > > If a workload that doesn't support mid-thread preemption gets mid-thread
> > > preempted, we're going to observe a GPU hang.
> > > While I'm here, let's also update the "workaround" naming.
> > 
> > Yeah, in Mesa we are not implementing the SIP, so we can't do
> > thread-level preemption yet and need the granularity to be no higher
> > than thread group level.
> > 
> > Acked-by: Rafael Antognolli <rafael.antognolli@intel.com>
> > 
> > > Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
> > > Cc: Anuj Phogat <anuj.phogat@intel.com>
> > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > Cc: Matt Roper <matthew.d.roper@intel.com>
> > > Cc: Rafael Antognolli <rafael.antognolli@intel.com>
> > > Tested-by: Anuj Phogat <anuj.phogat@intel.com>
> > > Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> 
> And pushed, thanks everyone for the testing and reviewed. I've held off
> on pushing the second patch as we just want to double check that the
> whitelisting is required.

Yeah, we should only need to push it once there is an actual consumer
that will enable it.

Regards, Joonas

> -Chris
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Jason Ekstrand March 3, 2020, 6:31 p.m. UTC | #5 
FYI: For compute shaders, we have a bit in INTERFACE_DESCRIPTOR_DATA
for this which we can set from userspace without whitelisting a
register.  If drivers can't handle mid-thread, they should just set
that bit.  Unless we can mid-thread preempt media or 3D which don't
have such a bit in which case maybe we need to do something in the
kernel.

--Jason

On Wed, Mar 6, 2019 at 5:20 AM Joonas Lahtinen
<joonas.lahtinen@linux.intel.com> wrote:
>
> Quoting Chris Wilson (2019-03-05 21:10:42)
> > Quoting Rafael Antognolli (2019-03-05 17:30:00)
> > > On Tue, Mar 05, 2019 at 01:48:26PM +0100, Michał Winiarski wrote:
> > > > We assumed that the default preemption granularity is fine for ICL.
> > > > Unfortunately, it turns out that some drivers don't support mid-thread
> > > > preemption for compute workloads.
> > > > If a workload that doesn't support mid-thread preemption gets mid-thread
> > > > preempted, we're going to observe a GPU hang.
> > > > While I'm here, let's also update the "workaround" naming.
> > >
> > > Yeah, in Mesa we are not implementing the SIP, so we can't do
> > > thread-level preemption yet and need the granularity to be no higher
> > > than thread group level.
> > >
> > > Acked-by: Rafael Antognolli <rafael.antognolli@intel.com>
> > >
> > > > Signed-off-by: Michał Winiarski <michal.winiarski@intel.com>
> > > > Cc: Anuj Phogat <anuj.phogat@intel.com>
> > > > Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> > > > Cc: Matt Roper <matthew.d.roper@intel.com>
> > > > Cc: Rafael Antognolli <rafael.antognolli@intel.com>
> > > > Tested-by: Anuj Phogat <anuj.phogat@intel.com>
> > > > Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> >
> > And pushed, thanks everyone for the testing and reviewed. I've held off
> > on pushing the second patch as we just want to double check that the
> > whitelisting is required.
>
> Yeah, we should only need to push it once there is an actual consumer
> that will enable it.
>
> Regards, Joonas
>
> > -Chris
> > _______________________________________________
> > Intel-gfx mailing list
> > Intel-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/intel-gfx
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
diff mbox series

Patch

diff --git a/drivers/gpu/drm/i915/intel_workarounds.c b/drivers/gpu/drm/i915/intel_workarounds.c
index 89b4007d5200..2fba33509f4e 100644
--- a/drivers/gpu/drm/i915/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/intel_workarounds.c
@@ -555,6 +555,11 @@  static void icl_ctx_workarounds_init(struct intel_engine_cs *engine)
 			   GEN10_CACHE_MODE_SS,
 			   0, /* write-only, so skip validation */
 			   _MASKED_BIT_ENABLE(FLOAT_BLEND_OPTIMIZATION_ENABLE));
+
+	/* WaDisableGPGPUMidThreadPreemption:icl */
+	WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1,
+			    GEN9_PREEMPT_GPGPU_LEVEL_MASK,
+			    GEN9_PREEMPT_GPGPU_THREAD_GROUP_LEVEL);
 }
 
 void intel_engine_init_ctx_wa(struct intel_engine_cs *engine)
@@ -1162,8 +1167,8 @@  rcs_engine_wa_init(struct intel_engine_cs *engine, struct i915_wa_list *wal)
 				    GEN7_DISABLE_SAMPLER_PREFETCH);
 	}
 
-	if (IS_GEN(i915, 9) || IS_CANNONLAKE(i915)) {
-		/* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,cnl */
+	if (IS_GEN_RANGE(i915, 9, 11)) {
+		/* FtrPerCtxtPreemptionGranularityControl:skl,bxt,kbl,cfl,cnl,icl */
 		wa_masked_en(wal,
 			     GEN7_FF_SLICE_CS_CHICKEN1,
 			     GEN9_FFSC_PERCTX_PREEMPT_CTRL);