From patchwork Tue Nov 29 07:26:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niranjana Vishwanathapura X-Patchwork-Id: 13058285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C387C4332F for ; Tue, 29 Nov 2022 07:29:37 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D865A10E37D; Tue, 29 Nov 2022 07:29:25 +0000 (UTC) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id D335010E36F; Tue, 29 Nov 2022 07:26:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669706807; x=1701242807; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yxjx6MJXbvZYmUz0wA4oTprLYByDO2vJsSd3h6Ewty8=; b=OaS4d7X9p6O+if7n4MI2nlpsVOhhSPafD/6gD/2tVy3Ctx6OUBEOgcCL VHspsbE2L9cdYutQyvwDHn07UtrKyccIyCOWN/eQQGSGTT6rKPlFeWe6o P+OXNyqYDbM7R6JXpEx1GfJHiDL7sR4T46s3xa07hAywyehF5rQ6VjYhV ZQ8/DeXgZMdwCz0OifvPiGGaiBx6rlhk88WIemuFc1naIPM0SOVvTb3YN nuHHcSk5kN5GVOXOie8l3Uzqstedk73AbT5n7RGIiWhHDDc/jJKR8II7l fDV06XO5GPzGcAcMytra7CL3s/K0qsxZyl9+oS6mqL6zwo8Z9NyBj+jlS g==; X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="295418344" X-IronPort-AV: E=Sophos;i="5.96,202,1665471600"; d="scan'208";a="295418344" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 23:26:44 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="674525572" X-IronPort-AV: E=Sophos;i="5.96,202,1665471600"; d="scan'208";a="674525572" Received: from nvishwa1-desk.sc.intel.com ([172.25.29.76]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 23:26:43 -0800 From: Niranjana Vishwanathapura To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v8 07/22] drm/i915/vm_bind: Add support to handle object evictions Date: Mon, 28 Nov 2022 23:26:20 -0800 Message-Id: <20221129072635.847-8-niranjana.vishwanathapura@intel.com> X-Mailer: git-send-email 2.21.0.rc0.32.g243a4c7e27 In-Reply-To: <20221129072635.847-1-niranjana.vishwanathapura@intel.com> References: <20221129072635.847-1-niranjana.vishwanathapura@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: matthew.brost@intel.com, paulo.r.zanoni@intel.com, tvrtko.ursulin@intel.com, jani.nikula@intel.com, lionel.g.landwerlin@intel.com, thomas.hellstrom@intel.com, matthew.auld@intel.com, jason@jlekstrand.net, andi.shyti@linux.intel.com, daniel.vetter@intel.com, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Support eviction by maintaining a list of evicted persistent vmas for rebinding during next submission. Ensure the list do not include persistent vmas that are being purged. v2: Remove unused I915_VMA_PURGED definition. v3: Properly handle __i915_vma_unbind_async() case. Reviewed-by: Matthew Auld Signed-off-by: Niranjana Vishwanathapura Signed-off-by: Andi Shyti --- .../drm/i915/gem/i915_gem_vm_bind_object.c | 6 ++++ drivers/gpu/drm/i915/gt/intel_gtt.c | 2 ++ drivers/gpu/drm/i915/gt/intel_gtt.h | 4 +++ drivers/gpu/drm/i915/i915_vma.c | 31 +++++++++++++++++-- drivers/gpu/drm/i915/i915_vma.h | 10 ++++++ drivers/gpu/drm/i915/i915_vma_types.h | 8 +++++ 6 files changed, 59 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_vm_bind_object.c b/drivers/gpu/drm/i915/gem/i915_gem_vm_bind_object.c index 4f9df4b756d2..dc738677466b 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_vm_bind_object.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_vm_bind_object.c @@ -86,6 +86,12 @@ static void i915_gem_vm_bind_remove(struct i915_vma *vma, bool release_obj) { lockdep_assert_held(&vma->vm->vm_bind_lock); + spin_lock(&vma->vm->vm_rebind_lock); + if (!list_empty(&vma->vm_rebind_link)) + list_del_init(&vma->vm_rebind_link); + i915_vma_set_purged(vma); + spin_unlock(&vma->vm->vm_rebind_lock); + list_del_init(&vma->vm_bind_link); list_del_init(&vma->non_priv_vm_bind_link); i915_vm_bind_it_remove(vma, &vma->vm->va); diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.c b/drivers/gpu/drm/i915/gt/intel_gtt.c index 542c0f85bf6f..401075776a83 100644 --- a/drivers/gpu/drm/i915/gt/intel_gtt.c +++ b/drivers/gpu/drm/i915/gt/intel_gtt.c @@ -291,6 +291,8 @@ void i915_address_space_init(struct i915_address_space *vm, int subclass) INIT_LIST_HEAD(&vm->vm_bound_list); mutex_init(&vm->vm_bind_lock); INIT_LIST_HEAD(&vm->non_priv_vm_bind_list); + INIT_LIST_HEAD(&vm->vm_rebind_list); + spin_lock_init(&vm->vm_rebind_lock); } void *__px_vaddr(struct drm_i915_gem_object *p) diff --git a/drivers/gpu/drm/i915/gt/intel_gtt.h b/drivers/gpu/drm/i915/gt/intel_gtt.h index 8c1b81d2a56c..fc1b2622c66f 100644 --- a/drivers/gpu/drm/i915/gt/intel_gtt.h +++ b/drivers/gpu/drm/i915/gt/intel_gtt.h @@ -266,6 +266,10 @@ struct i915_address_space { struct list_head vm_bind_list; /** @vm_bound_list: List of vm_binding completed */ struct list_head vm_bound_list; + /** @vm_rebind_list: list of vmas to be rebinded */ + struct list_head vm_rebind_list; + /** @vm_rebind_lock: protects vm_rebound_list */ + spinlock_t vm_rebind_lock; /** @va: tree of persistent vmas */ struct rb_root_cached va; /** @non_priv_vm_bind_list: list of non-private object mappings */ diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c index 92dea99cc735..8de9f7a5b306 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -243,6 +243,7 @@ vma_create(struct drm_i915_gem_object *obj, INIT_LIST_HEAD(&vma->vm_bind_link); INIT_LIST_HEAD(&vma->non_priv_vm_bind_link); + INIT_LIST_HEAD(&vma->vm_rebind_link); return vma; err_unlock: @@ -1686,6 +1687,14 @@ static void force_unbind(struct i915_vma *vma) if (!drm_mm_node_allocated(&vma->node)) return; + /* + * Persistent vma should have been purged by now. + * If not, issue a warning and purge it. + */ + if (GEM_WARN_ON(i915_vma_is_persistent(vma) && + !i915_vma_is_purged(vma))) + i915_vma_set_purged(vma); + atomic_and(~I915_VMA_PIN_MASK, &vma->flags); WARN_ON(__i915_vma_unbind(vma)); GEM_BUG_ON(drm_mm_node_allocated(&vma->node)); @@ -2052,6 +2061,16 @@ int __i915_vma_unbind(struct i915_vma *vma) __i915_vma_evict(vma, false); drm_mm_remove_node(&vma->node); /* pairs with i915_vma_release() */ + + if (i915_vma_is_persistent(vma)) { + spin_lock(&vma->vm->vm_rebind_lock); + if (list_empty(&vma->vm_rebind_link) && + !i915_vma_is_purged(vma)) + list_add_tail(&vma->vm_rebind_link, + &vma->vm->vm_rebind_list); + spin_unlock(&vma->vm->vm_rebind_lock); + } + return 0; } @@ -2064,8 +2083,7 @@ static struct dma_fence *__i915_vma_unbind_async(struct i915_vma *vma) if (!drm_mm_node_allocated(&vma->node)) return NULL; - if (i915_vma_is_pinned(vma) || - &vma->obj->mm.rsgt->table != vma->resource->bi.pages) + if (i915_vma_is_pinned(vma)) return ERR_PTR(-EAGAIN); /* @@ -2087,6 +2105,15 @@ static struct dma_fence *__i915_vma_unbind_async(struct i915_vma *vma) drm_mm_remove_node(&vma->node); /* pairs with i915_vma_release() */ + if (i915_vma_is_persistent(vma)) { + spin_lock(&vma->vm->vm_rebind_lock); + if (list_empty(&vma->vm_rebind_link) && + !i915_vma_is_purged(vma)) + list_add_tail(&vma->vm_rebind_link, + &vma->vm->vm_rebind_list); + spin_unlock(&vma->vm->vm_rebind_lock); + } + return fence; } diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h index 0a4662fbe6c3..e5a1a5fc36a2 100644 --- a/drivers/gpu/drm/i915/i915_vma.h +++ b/drivers/gpu/drm/i915/i915_vma.h @@ -153,6 +153,16 @@ static inline void i915_vma_set_persistent(struct i915_vma *vma) set_bit(I915_VMA_PERSISTENT_BIT, __i915_vma_flags(vma)); } +static inline bool i915_vma_is_purged(const struct i915_vma *vma) +{ + return test_bit(I915_VMA_PURGED_BIT, __i915_vma_flags(vma)); +} + +static inline void i915_vma_set_purged(struct i915_vma *vma) +{ + set_bit(I915_VMA_PURGED_BIT, __i915_vma_flags(vma)); +} + static inline struct i915_vma *i915_vma_get(struct i915_vma *vma) { i915_gem_object_get(vma->obj); diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h index 9cd055738997..61d0ec1a4e18 100644 --- a/drivers/gpu/drm/i915/i915_vma_types.h +++ b/drivers/gpu/drm/i915/i915_vma_types.h @@ -267,8 +267,14 @@ struct i915_vma { /** * I915_VMA_PERSISTENT_BIT: * The vma is persistent (created with VM_BIND call). + * + * I915_VMA_PURGED_BIT: + * The persistent vma is force unbound either due to VM_UNBIND call + * from UMD or VM is released. Do not check/wait for VM activeness + * in i915_vma_is_active() and i915_vma_sync() calls. */ #define I915_VMA_PERSISTENT_BIT 19 +#define I915_VMA_PURGED_BIT 20 struct i915_active active; @@ -299,6 +305,8 @@ struct i915_vma { struct list_head vm_bind_link; /** @non_priv_vm_bind_link: Link in non-private persistent VMA list */ struct list_head non_priv_vm_bind_link; + /** @vm_rebind_link: link to vm_rebind_list and protected by vm_rebind_lock */ + struct list_head vm_rebind_link; /* Link in vm_rebind_list */ /** Interval tree structures for persistent vma */