From patchwork Fri Jul 5 14:29:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mika Kuoppala X-Patchwork-Id: 11032747 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9B91138D for ; Fri, 5 Jul 2019 14:29:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 999452896F for ; Fri, 5 Jul 2019 14:29:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DEBB28A81; Fri, 5 Jul 2019 14:29:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 16F2B2896F for ; Fri, 5 Jul 2019 14:29:30 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 146EA6E4D6; Fri, 5 Jul 2019 14:29:30 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by gabe.freedesktop.org (Postfix) with ESMTPS id ADCB66E4CE for ; Fri, 5 Jul 2019 14:29:26 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Jul 2019 07:29:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,455,1557212400"; d="scan'208";a="363191491" Received: from rosetta.fi.intel.com ([10.237.72.194]) by fmsmga005.fm.intel.com with ESMTP; 05 Jul 2019 07:29:25 -0700 Received: by rosetta.fi.intel.com (Postfix, from userid 1000) id 79F5B840501; Fri, 5 Jul 2019 17:29:09 +0300 (EEST) From: Mika Kuoppala To: intel-gfx@lists.freedesktop.org Date: Fri, 5 Jul 2019 17:29:08 +0300 Message-Id: <20190705142908.4921-4-mika.kuoppala@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190705142908.4921-1-mika.kuoppala@linux.intel.com> References: <20190705142908.4921-1-mika.kuoppala@linux.intel.com> Subject: [Intel-gfx] [PATCH 4/4] drm/i915/gtt: Introduce release_pd_entry X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP By encapsulating the locking upper level and used check for entry into a helper function, we can use it in all callsites. v2: get rid of atomic_reads on lower level clears (Chris) Cc: Chris Wilson Signed-off-by: Mika Kuoppala --- drivers/gpu/drm/i915/i915_gem_gtt.c | 125 +++++++++++++--------------- 1 file changed, 60 insertions(+), 65 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index 0a55b0932c86..3d9612c776dc 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -813,6 +813,42 @@ __clear_pd_entry(struct i915_page_directory * const pd, #define clear_pd_entry(pd, pde, to) \ __clear_pd_entry((pd), (pde), px_base(to), gen8_pde_encode) +static bool +release_pt_entry(struct i915_page_directory * const pd, + const unsigned short pde, + struct i915_page_table * const pt, + struct i915_page_table * const scratch) +{ + bool free = false; + + spin_lock(&pd->lock); + if (atomic_dec_and_test(&pt->used)) { + clear_pd_entry(pd, pde, scratch); + free = true; + } + spin_unlock(&pd->lock); + + return free; +} + +static bool +release_pd_entry(struct i915_page_directory * const pd, + const unsigned short pde, + struct i915_page_directory * const entry, + struct i915_page_directory * const scratch) +{ + bool free = false; + + spin_lock(&pd->lock); + if (atomic_dec_and_test(&entry->used)) { + clear_pd_entry(pd, pde, scratch); + free = true; + } + spin_unlock(&pd->lock); + + return free; +} + /* * PDE TLBs are a pain to invalidate on GEN8+. When we modify * the page table structures, we mark them dirty so that @@ -827,11 +863,11 @@ static void mark_tlbs_dirty(struct i915_ppgtt *ppgtt) /* Removes entries from a single page table, releasing it if it's empty. * Caller can use the return value to update higher-level entries. */ -static bool gen8_ppgtt_clear_pt(const struct i915_address_space *vm, +static void gen8_ppgtt_clear_pt(const struct i915_address_space *vm, struct i915_page_table *pt, u64 start, u64 length) { - unsigned int num_entries = gen8_pte_count(start, length); + const unsigned int num_entries = gen8_pte_count(start, length); gen8_pte_t *vaddr; vaddr = kmap_atomic_px(pt); @@ -839,10 +875,11 @@ static bool gen8_ppgtt_clear_pt(const struct i915_address_space *vm, kunmap_atomic(vaddr); GEM_BUG_ON(num_entries > atomic_read(&pt->used)); - return !atomic_sub_return(num_entries, &pt->used); + + atomic_sub(num_entries, &pt->used); } -static bool gen8_ppgtt_clear_pd(struct i915_address_space *vm, +static void gen8_ppgtt_clear_pd(struct i915_address_space *vm, struct i915_page_directory *pd, u64 start, u64 length) { @@ -850,30 +887,21 @@ static bool gen8_ppgtt_clear_pd(struct i915_address_space *vm, u32 pde; gen8_for_each_pde(pt, pd, start, length, pde) { - bool free = false; - GEM_BUG_ON(pt == vm->scratch_pt); - if (!gen8_ppgtt_clear_pt(vm, pt, start, length)) - continue; + atomic_inc(&pt->used); - spin_lock(&pd->lock); - if (!atomic_read(&pt->used)) { - clear_pd_entry(pd, pde, vm->scratch_pt); - free = true; - } - spin_unlock(&pd->lock); - if (free) + gen8_ppgtt_clear_pt(vm, pt, start, length); + + if (release_pt_entry(pd, pde, pt, vm->scratch_pt)) free_pt(vm, pt); } - - return !atomic_read(&pd->used); } /* Removes entries from a single page dir pointer, releasing it if it's empty. * Caller can use the return value to update higher-level entries */ -static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, +static void gen8_ppgtt_clear_pdp(struct i915_address_space *vm, struct i915_page_directory * const pdp, u64 start, u64 length) { @@ -881,24 +909,15 @@ static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, unsigned int pdpe; gen8_for_each_pdpe(pd, pdp, start, length, pdpe) { - bool free = false; - GEM_BUG_ON(pd == vm->scratch_pd); - if (!gen8_ppgtt_clear_pd(vm, pd, start, length)) - continue; + atomic_inc(&pd->used); - spin_lock(&pdp->lock); - if (!atomic_read(&pd->used)) { - clear_pd_entry(pdp, pdpe, vm->scratch_pd); - free = true; - } - spin_unlock(&pdp->lock); - if (free) + gen8_ppgtt_clear_pd(vm, pd, start, length); + + if (release_pd_entry(pdp, pdpe, pd, vm->scratch_pd)) free_pd(vm, pd); } - - return !atomic_read(&pdp->used); } static void gen8_ppgtt_clear_3lvl(struct i915_address_space *vm, @@ -922,19 +941,13 @@ static void gen8_ppgtt_clear_4lvl(struct i915_address_space *vm, GEM_BUG_ON(!i915_vm_is_4lvl(vm)); gen8_for_each_pml4e(pdp, pml4, start, length, pml4e) { - bool free = false; GEM_BUG_ON(pdp == vm->scratch_pdp); - if (!gen8_ppgtt_clear_pdp(vm, pdp, start, length)) - continue; + atomic_inc(&pdp->used); - spin_lock(&pml4->lock); - if (!atomic_read(&pdp->used)) { - clear_pd_entry(pml4, pml4e, vm->scratch_pdp); - free = true; - } - spin_unlock(&pml4->lock); - if (free) + gen8_ppgtt_clear_pdp(vm, pdp, start, length); + + if (release_pd_entry(pml4, pml4e, pdp, vm->scratch_pdp)) free_pd(vm, pdp); } } @@ -1457,17 +1470,8 @@ static int gen8_ppgtt_alloc_pdp(struct i915_address_space *vm, goto out; unwind_pd: - if (alloc) { - free_pd(vm, alloc); - alloc = NULL; - } - spin_lock(&pdp->lock); - if (atomic_dec_and_test(&pd->used)) { - GEM_BUG_ON(alloc); - alloc = pd; /* defer the free to after the lock */ - clear_pd_entry(pdp, pdpe, vm->scratch_pd); - } - spin_unlock(&pdp->lock); + if (release_pd_entry(pdp, pdpe, pd, vm->scratch_pd)) + free_pd(vm, pd); unwind: gen8_ppgtt_clear_pdp(vm, pdp, from, start - from); out: @@ -1530,17 +1534,8 @@ static int gen8_ppgtt_alloc_4lvl(struct i915_address_space *vm, goto out; unwind_pdp: - if (alloc) { - free_pd(vm, alloc); - alloc = NULL; - } - spin_lock(&pml4->lock); - if (atomic_dec_and_test(&pdp->used)) { - GEM_BUG_ON(alloc); - alloc = pdp; /* defer the free until after the lock */ - clear_pd_entry(pml4, pml4e, vm->scratch_pdp); - } - spin_unlock(&pml4->lock); + if (release_pd_entry(pml4, pml4e, pdp, vm->scratch_pdp)) + free_pd(vm, pdp); unwind: gen8_ppgtt_clear_4lvl(vm, from, start - from); out: @@ -1572,8 +1567,8 @@ static int gen8_preallocate_top_level_pdp(struct i915_ppgtt *ppgtt) unwind: start -= from; gen8_for_each_pdpe(pd, pdp, from, start, pdpe) { - clear_pd_entry(pdp, pdpe, vm->scratch_pd); - free_pd(vm, pd); + if (release_pd_entry(pdp, pdpe, pd, vm->scratch_pd)) + free_pd(vm, pd); } atomic_set(&pdp->used, 0); return -ENOMEM;