From patchwork Tue Jan 30 09:32:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 10191477 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E101B601A0 for ; Tue, 30 Jan 2018 09:32:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0FA228372 for ; Tue, 30 Jan 2018 09:32:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5F1128984; Tue, 30 Jan 2018 09:32:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 685C428372 for ; Tue, 30 Jan 2018 09:32:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 893D76E71A; Tue, 30 Jan 2018 09:32:54 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E676E6E70F; Tue, 30 Jan 2018 09:32:51 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 10498769-1500050 for multiple; Tue, 30 Jan 2018 09:32:35 +0000 Received: by haswell.alporthouse.com (sSMTP sendmail emulation); Tue, 30 Jan 2018 09:32:36 +0000 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH] dma-buf: Refanctor reservation_object_add_shared_fence() Date: Tue, 30 Jan 2018 09:32:35 +0000 Message-Id: <20180130093235.2859-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.15.1 MIME-Version: 1.0 X-Originating-IP: 78.156.65.138 X-Country: code=GB country="United Kingdom" ip=78.156.65.138 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , intel-gfx@lists.freedesktop.org, =?UTF-8?q?Christian=20K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Adding a shared fence to a reservation_object is currently split into two handlers, one to insert the fence into the existing array and the other to replace the existing array with a new larger array. The first step in both of these routines involves scanning the existing array to decide if it can prune any of the existing fences. As both routines perform essentially the same loop, we can combine them into one routine. During the first scan over the existing array, we search for a slot with which we can reuse for the new fence (discarding the previous). If we find no available slot to reuse, and the array is already at its max size, then we know that we need to switch to the larger array, and that no existing fence needs to be discard, they can all be copied over. add/remove: 0/0 grow/shrink: 0/3 up/down: 0/-627 (-627) Function old new delta __warned 2352 2350 -2 reservation_object_reserve_shared 196 185 -11 reservation_object_add_shared_fence 1272 658 -614 Signed-off-by: Chris Wilson Cc: Christian König Cc: Alex Deucher --- drivers/dma-buf/reservation.c | 178 ++++++++++++++---------------------------- 1 file changed, 60 insertions(+), 118 deletions(-) diff --git a/drivers/dma-buf/reservation.c b/drivers/dma-buf/reservation.c index 314eb1071cce..de7b6e709a68 100644 --- a/drivers/dma-buf/reservation.c +++ b/drivers/dma-buf/reservation.c @@ -76,8 +76,6 @@ int reservation_object_reserve_shared(struct reservation_object *obj) if (old && old->shared_max) { if (old->shared_count < old->shared_max) { /* perform an in-place update */ - kfree(obj->staged); - obj->staged = NULL; return 0; } else max = old->shared_max * 2; @@ -95,146 +93,90 @@ int reservation_object_reserve_shared(struct reservation_object *obj) obj->staged = fobj; fobj->shared_max = max; + fobj->shared_count = 0; return 0; } EXPORT_SYMBOL(reservation_object_reserve_shared); -static void -reservation_object_add_shared_inplace(struct reservation_object *obj, - struct reservation_object_list *fobj, - struct dma_fence *fence) +/** + * reservation_object_add_shared_fence - Add a fence to a shared slot + * @obj: the reservation object + * @fence: the shared fence to add + * + * Add a fence to a shared slot, obj->lock must be held, and + * reservation_object_reserve_shared() has been called. + */ +void reservation_object_add_shared_fence(struct reservation_object *obj, + struct dma_fence *fence) { - struct dma_fence *signaled = NULL; - u32 i, signaled_idx; + struct reservation_object_list *old, *fobj; + struct dma_fence *replace = NULL; + u32 max = 0, i = 0; dma_fence_get(fence); - preempt_disable(); - write_seqcount_begin(&obj->seq); - - for (i = 0; i < fobj->shared_count; ++i) { - struct dma_fence *old_fence; - - old_fence = rcu_dereference_protected(fobj->shared[i], - reservation_object_held(obj)); - - if (old_fence->context == fence->context) { - /* memory barrier is added by write_seqcount_begin */ - RCU_INIT_POINTER(fobj->shared[i], fence); - write_seqcount_end(&obj->seq); - preempt_enable(); - - dma_fence_put(old_fence); - return; - } - - if (!signaled && dma_fence_is_signaled(old_fence)) { - signaled = old_fence; - signaled_idx = i; - } - } - /* - * memory barrier is added by write_seqcount_begin, - * fobj->shared_count is protected by this lock too + * Find an existing slot we can overwrite with the new fence. + * + * It is assumed that fences are added sequentially, that a second + * fence from the same context as an existing fence is strictly later + * than the existing fence. Therefore any new fence on an existing + * context can simply replace that existing fence. + * + * Likewise, if a fence is already signaled it is no longer required + * as part of the reservation_object and can be replaced by the new + * fence. This allows us to gradually prune the shared fence array + * and prevent its excessive growth. */ - if (signaled) { - RCU_INIT_POINTER(fobj->shared[signaled_idx], fence); - } else { - RCU_INIT_POINTER(fobj->shared[fobj->shared_count], fence); - fobj->shared_count++; - } - - write_seqcount_end(&obj->seq); - preempt_enable(); + old = reservation_object_get_list(obj); + if (old) { + u32 ctx = fence->context; - dma_fence_put(signaled); -} + for (; i < old->shared_count; ++i) { + struct dma_fence *check; -static void -reservation_object_add_shared_replace(struct reservation_object *obj, - struct reservation_object_list *old, - struct reservation_object_list *fobj, - struct dma_fence *fence) -{ - unsigned i, j, k; + check = rcu_dereference_protected(old->shared[i], + reservation_object_held(obj)); - dma_fence_get(fence); + if (check->context == ctx || + dma_fence_is_signaled(check)) { + replace = check; + break; + } + } - if (!old) { - RCU_INIT_POINTER(fobj->shared[0], fence); - fobj->shared_count = 1; - goto done; + fobj = old; + max = old->shared_max; } - /* - * no need to bump fence refcounts, rcu_read access - * requires the use of kref_get_unless_zero, and the - * references from the old struct are carried over to - * the new. - */ - for (i = 0, j = 0, k = fobj->shared_max; i < old->shared_count; ++i) { - struct dma_fence *check; - - check = rcu_dereference_protected(old->shared[i], - reservation_object_held(obj)); + if (i >= max) { + fobj = obj->staged; + obj->staged = NULL; - if (check->context == fence->context || - dma_fence_is_signaled(check)) - RCU_INIT_POINTER(fobj->shared[--k], check); - else - RCU_INIT_POINTER(fobj->shared[j++], check); + if (old) { + /* Borrow the old fence references */ + memcpy(fobj->shared, old->shared, + old->shared_count * sizeof(*old->shared)); + fobj->shared_count = old->shared_count; + } } - fobj->shared_count = j; - RCU_INIT_POINTER(fobj->shared[fobj->shared_count], fence); - fobj->shared_count++; -done: preempt_disable(); write_seqcount_begin(&obj->seq); - /* - * RCU_INIT_POINTER can be used here, - * seqcount provides the necessary barriers - */ - RCU_INIT_POINTER(obj->fence, fobj); - write_seqcount_end(&obj->seq); - preempt_enable(); - - if (!old) - return; - - /* Drop the references to the signaled fences */ - for (i = k; i < fobj->shared_max; ++i) { - struct dma_fence *f; - f = rcu_dereference_protected(fobj->shared[i], - reservation_object_held(obj)); - dma_fence_put(f); - } - kfree_rcu(old, rcu); -} - -/** - * reservation_object_add_shared_fence - Add a fence to a shared slot - * @obj: the reservation object - * @fence: the shared fence to add - * - * Add a fence to a shared slot, obj->lock must be held, and - * reservation_object_reserve_shared() has been called. - */ -void reservation_object_add_shared_fence(struct reservation_object *obj, - struct dma_fence *fence) -{ - struct reservation_object_list *old, *fobj = obj->staged; + RCU_INIT_POINTER(fobj->shared[i], fence); + if (!replace) + fobj->shared_count++; + if (fobj != old) + RCU_INIT_POINTER(obj->fence, fobj); - old = reservation_object_get_list(obj); - obj->staged = NULL; + /* write_seqcount_end() provides the necessary mb for the RCU writes */ + write_seqcount_end(&obj->seq); + preempt_enable(); - if (!fobj) { - BUG_ON(old->shared_count >= old->shared_max); - reservation_object_add_shared_inplace(obj, old, fence); - } else - reservation_object_add_shared_replace(obj, old, fobj, fence); + dma_fence_put(replace); + if (old && fobj != old) + kfree_rcu(old, rcu); } EXPORT_SYMBOL(reservation_object_add_shared_fence);