From patchwork Fri Nov 27 12:06:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Auld X-Patchwork-Id: 11935951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45CD7C63777 for ; Fri, 27 Nov 2020 12:11:56 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D8E5A20665 for ; Fri, 27 Nov 2020 12:11:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8E5A20665 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0A03B6ECA8; Fri, 27 Nov 2020 12:10:14 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by gabe.freedesktop.org (Postfix) with ESMTPS id E7A626ECAC; Fri, 27 Nov 2020 12:10:07 +0000 (UTC) IronPort-SDR: Sxn2oYOZjCFpAkMTjJXPASeDf1i6apz6PZBGlJKWE1PI+KV7MzjmzAmKTr7j586PZOfC7aWysV c5mV7GLBSBDQ== X-IronPort-AV: E=McAfee;i="6000,8403,9817"; a="172540768" X-IronPort-AV: E=Sophos;i="5.78,374,1599548400"; d="scan'208";a="172540768" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Nov 2020 04:10:07 -0800 IronPort-SDR: 5U426cH1mLdXPfw8zgn0MGOf9xye9wWx1lLiugMZFNFuHnnxZTLX/hi8nIeAqdpmCpnS1++kh8 4qjjyFT6PnMA== X-IronPort-AV: E=Sophos;i="5.78,374,1599548400"; d="scan'208";a="548029236" Received: from mjgleeso-mobl.ger.corp.intel.com (HELO mwauld-desk1.ger.corp.intel.com) ([10.251.85.2]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Nov 2020 04:10:06 -0800 From: Matthew Auld To: intel-gfx@lists.freedesktop.org Subject: [RFC PATCH 087/162] drm/i915: Delay publishing objects on the eviction lists Date: Fri, 27 Nov 2020 12:06:03 +0000 Message-Id: <20201127120718.454037-88-matthew.auld@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201127120718.454037-1-matthew.auld@intel.com> References: <20201127120718.454037-1-matthew.auld@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Thomas Hellström When an object is published on an eviction list, it's considered for eviction and can be locked by other threads. This is strictly not necessary until the object has pages. To limit eviction lookups that need to discard the object and facilitate a longer period during which we can lock the object isolated (trylock or ww lock without chance of deadlock or interruption), delay eviction list publishing until pages are set. Also take the object off the eviction lists when pages are unset. Finally make sure that an object is either locked or isolated when eviction list manipulation happens. Signed-off-by: Thomas Hellström Signed-off-by: Maarten Lankhorst Cc: Matthew Auld --- drivers/gpu/drm/i915/gem/i915_gem_object.c | 2 ++ drivers/gpu/drm/i915/gem/i915_gem_pages.c | 22 +++++++++++++++++++++- drivers/gpu/drm/i915/gem/i915_gem_region.c | 18 ++---------------- 3 files changed, 25 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c index 08d806bbf48e..5326b4b5a9f7 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c @@ -66,6 +66,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, INIT_LIST_HEAD(&obj->vma.list); INIT_LIST_HEAD(&obj->mm.link); + INIT_LIST_HEAD(&obj->mm.region_link); INIT_LIST_HEAD(&obj->lut_list); spin_lock_init(&obj->lut_lock); @@ -79,6 +80,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, GEM_BUG_ON(flags & ~I915_BO_ALLOC_FLAGS); obj->flags = flags; + obj->mm.region = NULL; obj->mm.madv = I915_MADV_WILLNEED; INIT_RADIX_TREE(&obj->mm.get_page.radix, GFP_KERNEL | __GFP_NOWARN); mutex_init(&obj->mm.get_page.lock); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pages.c b/drivers/gpu/drm/i915/gem/i915_gem_pages.c index 4a8be759832b..eacad971b955 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -16,6 +16,8 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, { struct drm_i915_private *i915 = to_i915(obj->base.dev); unsigned long supported = INTEL_INFO(i915)->page_sizes; + struct intel_memory_region *mem; + struct list_head *list; int i; assert_object_held_shared(obj); @@ -64,7 +66,6 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, GEM_BUG_ON(!HAS_PAGE_SIZES(i915, obj->mm.page_sizes.sg)); if (i915_gem_object_is_shrinkable(obj)) { - struct list_head *list; unsigned long flags; assert_object_held(obj); @@ -82,6 +83,18 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, atomic_set(&obj->mm.shrink_pin, 0); spin_unlock_irqrestore(&i915->mm.obj_lock, flags); } + + mem = obj->mm.region; + if (mem) { + mutex_lock(&mem->objects.lock); + GEM_WARN_ON(!list_empty(&obj->mm.region_link)); + if (obj->mm.madv != I915_MADV_WILLNEED) + list = &mem->objects.purgeable; + else + list = &mem->objects.list; + list_move_tail(&obj->mm.region_link, list); + mutex_unlock(&mem->objects.lock); + } } int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj) @@ -192,6 +205,7 @@ static void unmap_object(struct drm_i915_gem_object *obj, void *ptr) struct sg_table * __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj) { + struct intel_memory_region *mem = obj->mm.region; struct sg_table *pages; assert_object_held_shared(obj); @@ -205,6 +219,12 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj) i915_gem_object_make_unshrinkable(obj); + if (mem) { + mutex_lock(&mem->objects.lock); + list_del_init(&obj->mm.region_link); + mutex_unlock(&mem->objects.lock); + } + if (obj->mm.mapping) { unmap_object(obj, page_mask_bits(obj->mm.mapping)); obj->mm.mapping = NULL; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_region.c b/drivers/gpu/drm/i915/gem/i915_gem_region.c index 6a96741253b3..58bf5f9e3199 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_region.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_region.c @@ -105,30 +105,16 @@ void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj, struct intel_memory_region *mem) { INIT_LIST_HEAD(&obj->mm.blocks); + WARN_ON(i915_gem_object_has_pages(obj)); obj->mm.region = intel_memory_region_get(mem); if (obj->base.size <= mem->min_page_size) obj->flags |= I915_BO_ALLOC_CONTIGUOUS; - - mutex_lock(&mem->objects.lock); - - if (obj->flags & I915_BO_ALLOC_VOLATILE) - list_add(&obj->mm.region_link, &mem->objects.purgeable); - else - list_add(&obj->mm.region_link, &mem->objects.list); - - mutex_unlock(&mem->objects.lock); } void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj) { - struct intel_memory_region *mem = obj->mm.region; - - mutex_lock(&mem->objects.lock); - list_del(&obj->mm.region_link); - mutex_unlock(&mem->objects.lock); - - intel_memory_region_put(mem); + intel_memory_region_put(obj->mm.region); } struct drm_i915_gem_object *