From patchwork Tue Dec 18 13:24:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mika Kuoppala X-Patchwork-Id: 10735651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA46513B5 for ; Tue, 18 Dec 2018 13:25:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA08A286FF for ; Tue, 18 Dec 2018 13:25:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE67328EBA; Tue, 18 Dec 2018 13:25:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5A32A286FF for ; Tue, 18 Dec 2018 13:25:24 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 009DE6EBC8; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 81D916E926 for ; Tue, 18 Dec 2018 13:25:20 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 05:25:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,368,1539673200"; d="scan'208";a="126987205" Received: from rosetta.fi.intel.com ([10.237.72.186]) by fmsmga002.fm.intel.com with ESMTP; 18 Dec 2018 05:25:07 -0800 Received: by rosetta.fi.intel.com (Postfix, from userid 1000) id E948B8404F7; Tue, 18 Dec 2018 15:24:26 +0200 (EET) From: Mika Kuoppala To: intel-gfx@lists.freedesktop.org Date: Tue, 18 Dec 2018 15:24:23 +0200 Message-Id: <20181218132425.11189-1-mika.kuoppala@linux.intel.com> X-Mailer: git-send-email 2.17.1 Subject: [Intel-gfx] [PATCH 1/3] drm/i915: Prepare for larger CSB status FIFO size X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Make csb entry count variable in preparation for larger CSB status FIFO size found on gen11+ hardware. v2: adapt to hwsp access only (Chris) non continuous mmio (Daniele) Cc: Daniele Ceraolo Spurio Cc: Chris Wilson Cc: Joonas Lahtinen Signed-off-by: Mika Kuoppala --- drivers/gpu/drm/i915/intel_engine_cs.c | 23 +++++++++++++---------- drivers/gpu/drm/i915/intel_lrc.c | 10 +++++++--- drivers/gpu/drm/i915/intel_lrc.h | 19 ++++++++++++------- drivers/gpu/drm/i915/intel_ringbuffer.h | 5 +++++ 4 files changed, 37 insertions(+), 20 deletions(-) diff --git a/drivers/gpu/drm/i915/intel_engine_cs.c b/drivers/gpu/drm/i915/intel_engine_cs.c index 6f165f9ad2bf..f5960b3dc5d7 100644 --- a/drivers/gpu/drm/i915/intel_engine_cs.c +++ b/drivers/gpu/drm/i915/intel_engine_cs.c @@ -1305,37 +1305,40 @@ static void intel_engine_print_registers(const struct intel_engine_cs *engine, } if (HAS_EXECLISTS(dev_priv)) { - const u32 *hws = &engine->status_page.page_addr[I915_HWS_CSB_BUF0_INDEX]; + const u32 *hws = execlists->csb_status; + const u8 entries = execlists->csb_size; unsigned int idx; u8 read, write; - drm_printf(m, "\tExeclist status: 0x%08x %08x\n", + drm_printf(m, "\tExeclist status: 0x%08x %08x, entries %u\n", I915_READ(RING_EXECLIST_STATUS_LO(engine)), - I915_READ(RING_EXECLIST_STATUS_HI(engine))); + I915_READ(RING_EXECLIST_STATUS_HI(engine)), + entries); read = execlists->csb_head; write = READ_ONCE(*execlists->csb_write); drm_printf(m, "\tExeclist CSB read %d, write %d [mmio:%d], tasklet queued? %s (%s)\n", read, write, - GEN8_CSB_WRITE_PTR(I915_READ(RING_CONTEXT_STATUS_PTR(engine))), + I915_READ(RING_CONTEXT_STATUS_PTR(engine)) & + GEN11_CSB_WRITE_PTR_MASK, yesno(test_bit(TASKLET_STATE_SCHED, &engine->execlists.tasklet.state)), enableddisabled(!atomic_read(&engine->execlists.tasklet.count))); - if (read >= GEN8_CSB_ENTRIES) + if (read >= entries) read = 0; - if (write >= GEN8_CSB_ENTRIES) + if (write >= entries) write = 0; if (read > write) - write += GEN8_CSB_ENTRIES; + write += entries; while (read < write) { - idx = ++read % GEN8_CSB_ENTRIES; + idx = ++read % entries; drm_printf(m, "\tExeclist CSB[%d]: 0x%08x [mmio:0x%08x], context: %d [mmio:%d]\n", idx, hws[idx * 2], - I915_READ(RING_CONTEXT_STATUS_BUF_LO(engine, idx)), + I915_READ(RING_CSB_LO(engine, idx)), hws[idx * 2 + 1], - I915_READ(RING_CONTEXT_STATUS_BUF_HI(engine, idx))); + I915_READ(RING_CSB_HI(engine, idx))); } rcu_read_lock(); diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 1c9748f391fe..bd35ac3b7b77 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -760,7 +760,7 @@ invalidate_csb_entries(const u32 *first, const u32 *last) static void reset_csb_pointers(struct intel_engine_execlists *execlists) { - const unsigned int reset_value = GEN8_CSB_ENTRIES - 1; + const unsigned int reset_value = execlists->csb_size - 1; /* * After a reset, the HW starts writing into CSB entry [0]. We @@ -867,7 +867,7 @@ static void process_csb(struct intel_engine_cs *engine) struct intel_engine_execlists * const execlists = &engine->execlists; struct execlist_port *port = execlists->port; const u32 * const buf = execlists->csb_status; - u8 head, tail; + u8 head, tail, entries; /* * Note that csb_write, csb_status may be either in HWSP or mmio. @@ -885,6 +885,8 @@ static void process_csb(struct intel_engine_cs *engine) if (unlikely(head == tail)) return; + entries = execlists->csb_size; + /* * Hopefully paired with a wmb() in HW! * @@ -900,7 +902,7 @@ static void process_csb(struct intel_engine_cs *engine) unsigned int status; unsigned int count; - if (++head == GEN8_CSB_ENTRIES) + if (++head == entries) head = 0; /* @@ -2252,6 +2254,8 @@ static int logical_ring_init(struct intel_engine_cs *engine) execlists->csb_write = &engine->status_page.page_addr[intel_hws_csb_write_index(i915)]; + execlists->csb_size = GEN8_CSB_ENTRIES; + reset_csb_pointers(execlists); return 0; diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h index f5a5502ecf70..206b0da6fbcc 100644 --- a/drivers/gpu/drm/i915/intel_lrc.h +++ b/drivers/gpu/drm/i915/intel_lrc.h @@ -36,9 +36,13 @@ #define CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT (1 << 0) #define CTX_CTRL_RS_CTX_ENABLE (1 << 1) #define CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT (1 << 2) -#define RING_CONTEXT_STATUS_BUF_BASE(engine) _MMIO((engine)->mmio_base + 0x370) -#define RING_CONTEXT_STATUS_BUF_LO(engine, i) _MMIO((engine)->mmio_base + 0x370 + (i) * 8) -#define RING_CONTEXT_STATUS_BUF_HI(engine, i) _MMIO((engine)->mmio_base + 0x370 + (i) * 8 + 4) + +#define _RING_CSB_OFFSET(i) ((i) < 6 ? \ + (0x370 + (i) * 8) : \ + (0x3c0 + ((i) - 6) * 8)) +#define RING_CSB_OFFSET(engine, i) ((engine)->mmio_base + _RING_CSB_OFFSET(i)) +#define RING_CSB_LO(engine, i) _MMIO(RING_CSB_OFFSET(engine, i)) +#define RING_CSB_HI(engine, i) _MMIO(RING_CSB_OFFSET(engine, i) + 4) #define RING_CONTEXT_STATUS_PTR(engine) _MMIO((engine)->mmio_base + 0x3a0) #define RING_EXECLIST_SQ_CONTENTS(engine) _MMIO((engine)->mmio_base + 0x510) #define RING_EXECLIST_CONTROL(engine) _MMIO((engine)->mmio_base + 0x550) @@ -55,10 +59,11 @@ #define GEN8_CSB_PTR_MASK 0x7 #define GEN8_CSB_READ_PTR_MASK (GEN8_CSB_PTR_MASK << 8) #define GEN8_CSB_WRITE_PTR_MASK (GEN8_CSB_PTR_MASK << 0) -#define GEN8_CSB_WRITE_PTR(csb_status) \ - (((csb_status) & GEN8_CSB_WRITE_PTR_MASK) >> 0) -#define GEN8_CSB_READ_PTR(csb_status) \ - (((csb_status) & GEN8_CSB_READ_PTR_MASK) >> 8) + +#define GEN11_CSB_ENTRIES 12 +#define GEN11_CSB_PTR_MASK 0xf +#define GEN11_CSB_READ_PTR_MASK (GEN11_CSB_PTR_MASK << 8) +#define GEN11_CSB_WRITE_PTR_MASK (GEN11_CSB_PTR_MASK << 0) enum { INTEL_CONTEXT_SCHEDULE_IN = 0, diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h index 1ae74e579386..64a54d8f9b2b 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.h +++ b/drivers/gpu/drm/i915/intel_ringbuffer.h @@ -332,6 +332,11 @@ struct intel_engine_execlists { */ u32 preempt_complete_status; + /** + * @csb_size: context status buffer FIFO size + */ + u8 csb_size; + /** * @csb_head: context status buffer head */ From patchwork Tue Dec 18 13:24:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mika Kuoppala X-Patchwork-Id: 10735647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1DFD11399 for ; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0147B286FF for ; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E259D296DA; Tue, 18 Dec 2018 13:25:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 57FA1286FF for ; Tue, 18 Dec 2018 13:25:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C11DA6E926; Tue, 18 Dec 2018 13:25:21 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9F6466E927 for ; Tue, 18 Dec 2018 13:25:20 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 05:25:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,368,1539673200"; d="scan'208";a="284529013" Received: from rosetta.fi.intel.com ([10.237.72.186]) by orsmga005.jf.intel.com with ESMTP; 18 Dec 2018 05:25:07 -0800 Received: by rosetta.fi.intel.com (Postfix, from userid 1000) id EBDFF840143; Tue, 18 Dec 2018 15:24:26 +0200 (EET) From: Mika Kuoppala To: intel-gfx@lists.freedesktop.org Date: Tue, 18 Dec 2018 15:24:24 +0200 Message-Id: <20181218132425.11189-2-mika.kuoppala@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181218132425.11189-1-mika.kuoppala@linux.intel.com> References: <20181218132425.11189-1-mika.kuoppala@linux.intel.com> Subject: [Intel-gfx] [PATCH 2/3] drm/i915/icl: Switch to using 12 deep CSB status FIFO X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paulo Zanoni MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Now when we can support variable csb fifo sizes, disable legacy mode. By disabling legacy we hope to get better hw testing coverage by assuming everyone else have switched over. v2: rebase Cc: Chris Wilson Cc: Joonas Lahtinen Cc: Daniele Ceraolo Spurio Cc: Kelvin Gardiner Cc: Paulo Zanoni Signed-off-by: Mika Kuoppala --- drivers/gpu/drm/i915/intel_lrc.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index bd35ac3b7b77..039e2a54b1fc 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -1659,17 +1659,9 @@ static void enable_execlists(struct intel_engine_cs *engine) I915_WRITE(RING_HWSTAM(engine->mmio_base), 0xffffffff); - /* - * Make sure we're not enabling the new 12-deep CSB - * FIFO as that requires a slightly updated handling - * in the ctx switch irq. Since we're currently only - * using only 2 elements of the enhanced execlists the - * deeper FIFO it's not needed and it's not worth adding - * more statements to the irq handler to support it. - */ if (INTEL_GEN(dev_priv) >= 11) I915_WRITE(RING_MODE_GEN7(engine), - _MASKED_BIT_DISABLE(GEN11_GFX_DISABLE_LEGACY_MODE)); + _MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE)); else I915_WRITE(RING_MODE_GEN7(engine), _MASKED_BIT_ENABLE(GFX_RUN_LIST_ENABLE)); @@ -2254,7 +2246,10 @@ static int logical_ring_init(struct intel_engine_cs *engine) execlists->csb_write = &engine->status_page.page_addr[intel_hws_csb_write_index(i915)]; - execlists->csb_size = GEN8_CSB_ENTRIES; + if (INTEL_GEN(engine->i915) < 11) + execlists->csb_size = GEN8_CSB_ENTRIES; + else + execlists->csb_size = GEN11_CSB_ENTRIES; reset_csb_pointers(execlists); From patchwork Tue Dec 18 13:24:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mika Kuoppala X-Patchwork-Id: 10735649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 80DBA14DE for ; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7044B286FF for ; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5B6FA28760; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 059FE29F43 for ; Tue, 18 Dec 2018 13:25:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1ECC46E927; Tue, 18 Dec 2018 13:25:22 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9D4B26E926 for ; Tue, 18 Dec 2018 13:25:20 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Dec 2018 05:25:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,368,1539673200"; d="scan'208";a="110369202" Received: from rosetta.fi.intel.com ([10.237.72.186]) by fmsmga008.fm.intel.com with ESMTP; 18 Dec 2018 05:25:07 -0800 Received: by rosetta.fi.intel.com (Postfix, from userid 1000) id ED278840733; Tue, 18 Dec 2018 15:24:26 +0200 (EET) From: Mika Kuoppala To: intel-gfx@lists.freedesktop.org Date: Tue, 18 Dec 2018 15:24:25 +0200 Message-Id: <20181218132425.11189-3-mika.kuoppala@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181218132425.11189-1-mika.kuoppala@linux.intel.com> References: <20181218132425.11189-1-mika.kuoppala@linux.intel.com> Subject: [Intel-gfx] [PATCH 3/3] drm/i915/icl: Introduce gen11 flush/invalidate X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP There are more pipe control levers to pull with icl. Take them into use both for flushing and invalidating to ensure completeness. Doing so, avoid overloading the gen8 flush/invalidate further and make a gen11 specific callback. Cc: Chris Wilson Cc: Tvrtko Ursulin Signed-off-by: Mika Kuoppala --- drivers/gpu/drm/i915/intel_gpu_commands.h | 2 + drivers/gpu/drm/i915/intel_lrc.c | 61 ++++++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/intel_gpu_commands.h b/drivers/gpu/drm/i915/intel_gpu_commands.h index 105e2a9e874a..62112e9da610 100644 --- a/drivers/gpu/drm/i915/intel_gpu_commands.h +++ b/drivers/gpu/drm/i915/intel_gpu_commands.h @@ -195,6 +195,8 @@ #define DISPLAY_PLANE_A (0<<20) #define DISPLAY_PLANE_B (1<<20) #define GFX_OP_PIPE_CONTROL(len) ((0x3<<29)|(0x3<<27)|(0x2<<24)|((len)-2)) +#define PIPE_CONTROL_COMMAND_CACHE_INVALIDATE (1<<29) /* gen11+ */ +#define PIPE_CONTROL_TILE_CACHE_FLUSH (1<<28) /* gen11+ */ #define PIPE_CONTROL_FLUSH_L3 (1<<27) #define PIPE_CONTROL_GLOBAL_GTT_IVB (1<<24) /* gen7+ */ #define PIPE_CONTROL_MMIO_WRITE (1<<23) diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c index 039e2a54b1fc..d4c7dab9cdb4 100644 --- a/drivers/gpu/drm/i915/intel_lrc.c +++ b/drivers/gpu/drm/i915/intel_lrc.c @@ -2021,6 +2021,60 @@ static int gen8_emit_flush_render(struct i915_request *request, return 0; } +static int gen11_emit_flush_render(struct i915_request *request, + u32 mode) +{ + struct intel_engine_cs *engine = request->engine; + const u32 scratch_addr = + i915_scratch_offset(engine->i915) + 2 * CACHELINE_BYTES; + u32 *cs; + + if (mode & EMIT_FLUSH) { + u32 flags = PIPE_CONTROL_CS_STALL; + + flags |= PIPE_CONTROL_TILE_CACHE_FLUSH; + flags |= PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH; + flags |= PIPE_CONTROL_STALL_AT_SCOREBOARD; + flags |= PIPE_CONTROL_DEPTH_CACHE_FLUSH; + flags |= PIPE_CONTROL_DC_FLUSH_ENABLE; + flags |= PIPE_CONTROL_FLUSH_ENABLE; + flags |= PIPE_CONTROL_QW_WRITE; + flags |= PIPE_CONTROL_GLOBAL_GTT_IVB; + + cs = intel_ring_begin(request, 6); + if (IS_ERR(cs)) + return PTR_ERR(cs); + + cs = gen8_emit_pipe_control(cs, flags, scratch_addr); + + intel_ring_advance(request, cs); + } + + if (mode & EMIT_INVALIDATE) { + u32 flags = PIPE_CONTROL_CS_STALL; + + flags |= PIPE_CONTROL_COMMAND_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_TLB_INVALIDATE; + flags |= PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_VF_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_CONST_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_STATE_CACHE_INVALIDATE; + flags |= PIPE_CONTROL_QW_WRITE; + flags |= PIPE_CONTROL_GLOBAL_GTT_IVB; + + cs = intel_ring_begin(request, 6); + if (IS_ERR(cs)) + return PTR_ERR(cs); + + cs = gen8_emit_pipe_control(cs, flags, scratch_addr); + + intel_ring_advance(request, cs); + } + + return 0; +} + /* * Reserve space for 2 NOOPs at the end of each request to be * used as a workaround for not being allowed to do lite @@ -2268,7 +2322,12 @@ int logical_render_ring_init(struct intel_engine_cs *engine) /* Override some for render ring. */ engine->init_context = gen8_init_rcs_context; - engine->emit_flush = gen8_emit_flush_render; + + if (INTEL_GEN(dev_priv) >= 11) + engine->emit_flush = gen11_emit_flush_render; + else + engine->emit_flush = gen8_emit_flush_render; + engine->emit_breadcrumb = gen8_emit_breadcrumb_rcs; engine->emit_breadcrumb_sz = gen8_emit_breadcrumb_rcs_sz;