From patchwork Thu Feb 21 18:42:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 10825437 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB2E2139A for ; Fri, 22 Feb 2019 08:21:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC66C31B01 for ; Fri, 22 Feb 2019 08:21:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A0BC331B10; Fri, 22 Feb 2019 08:21:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2589431B01 for ; Fri, 22 Feb 2019 08:21:18 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C7D288931F; Fri, 22 Feb 2019 08:20:54 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4C63789115 for ; Thu, 21 Feb 2019 18:42:35 +0000 (UTC) Received: from willy by bombadil.infradead.org with local (Exim 4.90_1 #2 (Red Hat Linux)) id 1gwtIo-0000gS-Sk; Thu, 21 Feb 2019 18:42:34 +0000 From: Matthew Wilcox To: dri-devel@lists.freedesktop.org Subject: [PATCH 21/34] drm/i915: Convert get_page to XArray Date: Thu, 21 Feb 2019 10:42:01 -0800 Message-Id: <20190221184226.2149-42-willy@infradead.org> X-Mailer: git-send-email 2.14.5 In-Reply-To: <20190221184226.2149-1-willy@infradead.org> References: <20190221184226.2149-1-willy@infradead.org> X-Mailman-Approved-At: Fri, 22 Feb 2019 08:20:52 +0000 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=dZJ/Rx06sz3zxwEbCNCGgkT/gZEz1Maq87z6ECdd1nc=; b=IehvfB8GZZWVST3H5hfYETHSa W/2HHhj2Yr0/QBvzrr6JRtuPKlTOOHX5ucnP1tPEi1FXAt9od/K9+wXMeA46UXDADrfH+BqkZ6Xwf RJ09SZ1+iQWnQPWGuu4vxz0iwqx6i19zTTFzYzgOYiyKmA0JlB6h+fHcUpsf4cXuStSHPGYFZPdU4 CstfBj/EO0icBzxcOPz1zBkIMZqJ6x/1lEGFcjREEDRES/a4oey6dBe6S99rUpItuaf+An2/TNLFE aeFD8W7ZTBu9yrga2OdpN8loQl3vlwMH8po6iVtJ3n0eIZiwgqrG7SWsdVfCGFTYMpUiUWaeuz0fa 5QRfnsWvQ==; X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Matthew Wilcox MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This initially seemed like an ideal use-case for the store_range functionality, but there's no easy way to determine where we were relative to the base of the entry. So this is a straightforward conversion which doesn't even touch the locking. Signed-off-by: Matthew Wilcox --- drivers/gpu/drm/i915/i915_gem.c | 38 ++++++++++--------------- drivers/gpu/drm/i915/i915_gem_context.h | 2 +- drivers/gpu/drm/i915/i915_gem_object.h | 4 +-- 3 files changed, 18 insertions(+), 26 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 0cdccc886587..f8c36c2d32c2 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -2428,13 +2428,7 @@ i915_gem_object_put_pages_gtt(struct drm_i915_gem_object *obj, static void __i915_gem_object_reset_page_iter(struct drm_i915_gem_object *obj) { - struct radix_tree_iter iter; - void __rcu **slot; - - rcu_read_lock(); - radix_tree_for_each_slot(slot, &obj->mm.get_page.radix, &iter, 0) - radix_tree_delete(&obj->mm.get_page.radix, iter.index); - rcu_read_unlock(); + xa_destroy(&obj->mm.get_page.xa); } static struct sg_table * @@ -4716,7 +4710,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, init_request_active(&obj->frontbuffer_write, frontbuffer_retire); obj->mm.madv = I915_MADV_WILLNEED; - INIT_RADIX_TREE(&obj->mm.get_page.radix, GFP_KERNEL | __GFP_NOWARN); + xa_init(&obj->mm.get_page.xa); mutex_init(&obj->mm.get_page.lock); i915_gem_info_add_obj(to_i915(obj->base.dev), obj->base.size); @@ -6073,8 +6067,8 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, GEM_BUG_ON(n >= obj->base.size >> PAGE_SHIFT); GEM_BUG_ON(!i915_gem_object_has_pinned_pages(obj)); - /* As we iterate forward through the sg, we record each entry in a - * radixtree for quick repeated (backwards) lookups. If we have seen + /* As we iterate forward through the sg, we record each entry in an + * xarray for quick repeated (backwards) lookups. If we have seen * this index previously, we will have an entry for it. * * Initial lookup is O(N), but this is amortized to O(1) for @@ -6089,7 +6083,7 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, /* We prefer to reuse the last sg so that repeated lookup of this * (or the subsequent) sg are fast - comparing against the last - * sg is faster than going through the radixtree. + * sg is faster than going through the xarray. */ sg = iter->sg_pos; @@ -6099,24 +6093,22 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, while (idx + count <= n) { void *entry; unsigned long i; - int ret; /* If we cannot allocate and insert this entry, or the * individual pages from this range, cancel updating the * sg_idx so that on this lookup we are forced to linearly * scan onwards, but on future lookups we will try the - * insertion again (in which case we need to be careful of - * the error return reporting that we have already inserted - * this index). + * insertion again. */ - ret = radix_tree_insert(&iter->radix, idx, sg); - if (ret && ret != -EEXIST) + if (xa_store(&iter->xa, idx, sg, GFP_KERNEL | __GFP_NOWARN) + == XA_ERROR(-ENOMEM)) goto scan; entry = xa_mk_value(idx); for (i = 1; i < count; i++) { - ret = radix_tree_insert(&iter->radix, idx + i, entry); - if (ret && ret != -EEXIST) + if (xa_store(&iter->xa, idx + i, entry, + GFP_KERNEL | __GFP_NOWARN) + == XA_ERROR(-ENOMEM)) goto scan; } @@ -6134,7 +6126,7 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, if (unlikely(n < idx)) /* insertion completed by another thread */ goto lookup; - /* In case we failed to insert the entry into the radixtree, we need + /* In case we failed to insert the entry into the xarray, we need * to look beyond the current sg. */ while (idx + count <= n) { @@ -6149,11 +6141,11 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, lookup: rcu_read_lock(); - sg = radix_tree_lookup(&iter->radix, n); + sg = xa_load(&iter->xa, n); GEM_BUG_ON(!sg); /* If this index is in the middle of multi-page sg entry, - * the radix tree will contain a value entry that points + * the xarray will contain a value entry that points * to the start of that range. We will return the pointer to * the base page and the offset of this page within the * sg entry's range. @@ -6162,7 +6154,7 @@ i915_gem_object_get_sg(struct drm_i915_gem_object *obj, if (unlikely(xa_is_value(sg))) { unsigned long base = xa_to_value(sg); - sg = radix_tree_lookup(&iter->radix, base); + sg = xa_load(&iter->xa, base); GEM_BUG_ON(!sg); *offset = n - base; diff --git a/drivers/gpu/drm/i915/i915_gem_context.h b/drivers/gpu/drm/i915/i915_gem_context.h index ec22de370a22..052ad66a5bae 100644 --- a/drivers/gpu/drm/i915/i915_gem_context.h +++ b/drivers/gpu/drm/i915/i915_gem_context.h @@ -27,7 +27,7 @@ #include #include -#include +#include #include "i915_gem.h" #include "i915_scheduler.h" diff --git a/drivers/gpu/drm/i915/i915_gem_object.h b/drivers/gpu/drm/i915/i915_gem_object.h index 7f6493229f50..9febe70b4da9 100644 --- a/drivers/gpu/drm/i915/i915_gem_object.h +++ b/drivers/gpu/drm/i915/i915_gem_object.h @@ -40,7 +40,7 @@ struct drm_i915_gem_object; /* * struct i915_lut_handle tracks the fast lookups from handle to vma used - * for execbuf. Although we use a radixtree for that mapping, in order to + * for execbuf. Although we use an xarray for that mapping, in order to * remove them as the object or context is closed, we need a secondary list * and a translation entry (i915_lut_handle). */ @@ -218,7 +218,7 @@ struct drm_i915_gem_object { struct scatterlist *sg_pos; unsigned int sg_idx; /* in pages, but 32bit eek! */ - struct radix_tree_root radix; + struct xarray xa; struct mutex lock; /* protects this cache */ } get_page;