From patchwork Tue Aug 6 15:01:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 11085815 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E994912 for ; Fri, 9 Aug 2019 09:16:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CCBE28B38 for ; Fri, 9 Aug 2019 09:16:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 50DC828C3E; Fri, 9 Aug 2019 09:16:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AF72F28B38 for ; Fri, 9 Aug 2019 09:16:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 697B86EDA5; Fri, 9 Aug 2019 09:15:54 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by gabe.freedesktop.org (Postfix) with ESMTPS id BA24A6E3BB; Tue, 6 Aug 2019 15:01:43 +0000 (UTC) Received: by mail-wr1-x441.google.com with SMTP id c2so85015224wrm.8; Tue, 06 Aug 2019 08:01:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zalB8gq1beSq4ta9XAvlbVYYxDTmwakxB9DRHWgejUg=; b=bKiHkqzBzPoVTOW9oEHqhYcZKI7EDvRC6qasEhxWo6ZcG6+mSt1HtYKi85gu2+znlZ CJeUH7KH0wdsuMfmeEgpGV2jCTQe3T0ZE6YfhM0xuGRR/KCN77YQHD5FNTF2skKt+S2D 4LzXs/FygbEqKAoZum+qPDmvyaqLrjkfZkvL75oo9/uJuJqATNYNbiIYSsWVab7UIC7Z QdydSn6cXBfnxU4KrEATTFUUOxQeWOB0qAYu7+FMVKGidnQroZXRifioE/OruF6C3AOk cP++eqjnlZdlNpcwxmnJWgnRvv3elfDM5ctbetvWUMIavx1sUMAXnm4oCBpJ19fqrAnJ hknA== X-Gm-Message-State: APjAAAXcbV9bZ1MPO/vPycEbD4ZSDdLmFvGLgfTNm6PRY4h7X04nccet mLDRtDfZwAbJViva0YABRi+wvtap X-Google-Smtp-Source: APXvYqxNjBUNUdlpAUZmAps4Mc7pazvmhMfBuAZ95n4cdBgcJvyHIK/6o9C2G5roYiFhLj2ePURWiA== X-Received: by 2002:adf:ea4c:: with SMTP id j12mr5533215wrn.75.1565103701601; Tue, 06 Aug 2019 08:01:41 -0700 (PDT) Received: from abel.fritz.box ([2a02:908:1252:fb60:d1e0:fffc:b0e3:c66d]) by smtp.gmail.com with ESMTPSA id g2sm78766828wmh.0.2019.08.06.08.01.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Aug 2019 08:01:41 -0700 (PDT) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: intel-gfx@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk Date: Tue, 6 Aug 2019 17:01:33 +0200 Message-Id: <20190806150134.104222-7-christian.koenig@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190806150134.104222-1-christian.koenig@amd.com> References: <20190806150134.104222-1-christian.koenig@amd.com> MIME-Version: 1.0 X-Mailman-Approved-At: Fri, 09 Aug 2019 09:15:44 +0000 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zalB8gq1beSq4ta9XAvlbVYYxDTmwakxB9DRHWgejUg=; b=ZXS6v3F4t9G3Rblrcns1QtgET1Hg2p7hBG07F6V3LP5C7NQeSz04+/Zgp+SZezBKJr DSnzCvd+qTjpOw9SwkOYGHjdsrGR88Qh3B/VYjQMkS7J5U6NYw3/rYoGvNc3CpLMOmUf SSNGZHu0+l62OEJfyfA3kHm4F5ECdEeNs50LZQUhr16xSVnT03Q+Yp6dsaBe4WYVKCs5 MTHXxBQSwfJqP00LCwbD05ZQoqsqM+pp1cf+BsFrRAfOrDm6+SUgszfyW6WOyG3yunoz aSVuH6dKhRE57qQR0VABlNK/wn+7/7tKHSMCrDeXr/aH1vtBc6AqfJLvp/Dj04RaztiU 0JWw== Subject: [Intel-gfx] [PATCH 7/8] dma-buf: add reservation_object_fences helper X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Add a new helper to get a consistent set of pointers from the reservation object. While at it group all access helpers together in the header file. Signed-off-by: Christian König --- drivers/dma-buf/dma-buf.c | 27 ++------ drivers/dma-buf/reservation.c | 60 ++++++------------ include/linux/reservation.h | 113 ++++++++++++++++++++-------------- 3 files changed, 94 insertions(+), 106 deletions(-) diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f45bfb29ef96..0d11b3cc961e 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -199,7 +199,7 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll) struct reservation_object_list *fobj; struct dma_fence *fence_excl; __poll_t events; - unsigned shared_count, seq; + unsigned shared_count; dmabuf = file->private_data; if (!dmabuf || !dmabuf->resv) @@ -213,20 +213,12 @@ static __poll_t dma_buf_poll(struct file *file, poll_table *poll) if (!events) return 0; -retry: - seq = read_seqcount_begin(&resv->seq); rcu_read_lock(); - - fobj = rcu_dereference(resv->fence); + reservation_object_fences(resv, &fence_excl, &fobj); if (fobj) shared_count = fobj->shared_count; else shared_count = 0; - fence_excl = rcu_dereference(resv->fence_excl); - if (read_seqcount_retry(&resv->seq, seq)) { - rcu_read_unlock(); - goto retry; - } if (fence_excl && (!(events & EPOLLOUT) || shared_count == 0)) { struct dma_buf_poll_cb_t *dcb = &dmabuf->cb_excl; @@ -1157,7 +1149,6 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) struct reservation_object *robj; struct reservation_object_list *fobj; struct dma_fence *fence; - unsigned seq; int count = 0, attach_count, shared_count, i; size_t size = 0; @@ -1188,16 +1179,10 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) buf_obj->name ?: ""); robj = buf_obj->resv; - while (true) { - seq = read_seqcount_begin(&robj->seq); - rcu_read_lock(); - fobj = rcu_dereference(robj->fence); - shared_count = fobj ? fobj->shared_count : 0; - fence = rcu_dereference(robj->fence_excl); - if (!read_seqcount_retry(&robj->seq, seq)) - break; - rcu_read_unlock(); - } + rcu_read_lock(); + reservation_object_fences(robj, &fence, &fobj); + shared_count = fobj ? fobj->shared_count : 0; + rcu_read_unlock(); if (fence) seq_printf(s, "\tExclusive fence: %s %s %ssignalled\n", diff --git a/drivers/dma-buf/reservation.c b/drivers/dma-buf/reservation.c index 7505eb289023..839d72af7ad8 100644 --- a/drivers/dma-buf/reservation.c +++ b/drivers/dma-buf/reservation.c @@ -316,9 +316,9 @@ int reservation_object_copy_fences(struct reservation_object *dst, reservation_object_assert_held(dst); rcu_read_lock(); - src_list = rcu_dereference(src->fence); retry: + reservation_object_fences(src, &new, &src_list); if (src_list) { unsigned shared_count = src_list->shared_count; @@ -329,7 +329,7 @@ int reservation_object_copy_fences(struct reservation_object *dst, return -ENOMEM; rcu_read_lock(); - src_list = rcu_dereference(src->fence); + reservation_object_fences(src, &new, &src_list); if (!src_list || src_list->shared_count > shared_count) { kfree(dst_list); goto retry; @@ -346,7 +346,6 @@ int reservation_object_copy_fences(struct reservation_object *dst, if (!dma_fence_get_rcu(fence)) { reservation_object_list_free(dst_list); - src_list = rcu_dereference(src->fence); goto retry; } @@ -361,7 +360,10 @@ int reservation_object_copy_fences(struct reservation_object *dst, dst_list = NULL; } - new = dma_fence_get_rcu_safe(&src->fence_excl); + if (new && !dma_fence_get_rcu(new)) { + reservation_object_list_free(dst_list); + goto retry; + } rcu_read_unlock(); src_list = reservation_object_get_list(dst); @@ -407,19 +409,17 @@ int reservation_object_get_fences_rcu(struct reservation_object *obj, do { struct reservation_object_list *fobj; - unsigned int i, seq; + unsigned int i; size_t sz = 0; shared_count = i = 0; rcu_read_lock(); - seq = read_seqcount_begin(&obj->seq); + reservation_object_fences(obj, &fence_excl, &fobj); - fence_excl = rcu_dereference(obj->fence_excl); if (fence_excl && !dma_fence_get_rcu(fence_excl)) goto unlock; - fobj = rcu_dereference(obj->fence); if (fobj) sz += sizeof(*shared) * fobj->shared_max; @@ -455,7 +455,7 @@ int reservation_object_get_fences_rcu(struct reservation_object *obj, } } - if (i != shared_count || read_seqcount_retry(&obj->seq, seq)) { + if (i != shared_count) { while (i--) dma_fence_put(shared[i]); dma_fence_put(fence_excl); @@ -499,18 +499,18 @@ long reservation_object_wait_timeout_rcu(struct reservation_object *obj, bool wait_all, bool intr, unsigned long timeout) { + struct reservation_object_list *fobj; struct dma_fence *fence; - unsigned seq, shared_count; + unsigned shared_count; long ret = timeout ? timeout : 1; int i; retry: shared_count = 0; - seq = read_seqcount_begin(&obj->seq); rcu_read_lock(); i = -1; - fence = rcu_dereference(obj->fence_excl); + reservation_object_fences(obj, &fence, &fobj); if (fence && !test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) { if (!dma_fence_get_rcu(fence)) goto unlock_retry; @@ -525,9 +525,6 @@ long reservation_object_wait_timeout_rcu(struct reservation_object *obj, } if (wait_all) { - struct reservation_object_list *fobj = - rcu_dereference(obj->fence); - if (fobj) shared_count = fobj->shared_count; @@ -553,11 +550,6 @@ long reservation_object_wait_timeout_rcu(struct reservation_object *obj, rcu_read_unlock(); if (fence) { - if (read_seqcount_retry(&obj->seq, seq)) { - dma_fence_put(fence); - goto retry; - } - ret = dma_fence_wait_timeout(fence, intr, ret); dma_fence_put(fence); if (ret > 0 && wait_all && (i + 1 < shared_count)) @@ -602,21 +594,20 @@ reservation_object_test_signaled_single(struct dma_fence *passed_fence) bool reservation_object_test_signaled_rcu(struct reservation_object *obj, bool test_all) { - unsigned seq, shared_count; + struct reservation_object_list *fobj; + struct dma_fence *fence_excl; + unsigned shared_count; int ret; rcu_read_lock(); retry: ret = true; shared_count = 0; - seq = read_seqcount_begin(&obj->seq); + reservation_object_fences(obj, &fence_excl, &fobj); if (test_all) { unsigned i; - struct reservation_object_list *fobj = - rcu_dereference(obj->fence); - if (fobj) shared_count = fobj->shared_count; @@ -629,23 +620,12 @@ bool reservation_object_test_signaled_rcu(struct reservation_object *obj, else if (!ret) break; } - - if (read_seqcount_retry(&obj->seq, seq)) - goto retry; } - if (!shared_count) { - struct dma_fence *fence_excl = rcu_dereference(obj->fence_excl); - - if (fence_excl) { - ret = reservation_object_test_signaled_single( - fence_excl); - if (ret < 0) - goto retry; - - if (read_seqcount_retry(&obj->seq, seq)) - goto retry; - } + if (!shared_count && fence_excl) { + ret = reservation_object_test_signaled_single(fence_excl); + if (ret < 0) + goto retry; } rcu_read_unlock(); diff --git a/include/linux/reservation.h b/include/linux/reservation.h index 56b782fec49b..b8b8273eef00 100644 --- a/include/linux/reservation.h +++ b/include/linux/reservation.h @@ -81,6 +81,51 @@ struct reservation_object { #define reservation_object_assert_held(obj) \ lockdep_assert_held(&(obj)->lock.base) +/** + * reservation_object_get_excl - get the reservation object's + * exclusive fence, with update-side lock held + * @obj: the reservation object + * + * Returns the exclusive fence (if any). Does NOT take a + * reference. Writers must hold obj->lock, readers may only + * hold a RCU read side lock. + * + * RETURNS + * The exclusive fence or NULL + */ +static inline struct dma_fence * +reservation_object_get_excl(struct reservation_object *obj) +{ + return rcu_dereference_protected(obj->fence_excl, + reservation_object_held(obj)); +} + +/** + * reservation_object_get_excl_rcu - get the reservation object's + * exclusive fence, without lock held. + * @obj: the reservation object + * + * If there is an exclusive fence, this atomically increments it's + * reference count and returns it. + * + * RETURNS + * The exclusive fence or NULL if none + */ +static inline struct dma_fence * +reservation_object_get_excl_rcu(struct reservation_object *obj) +{ + struct dma_fence *fence; + + if (!rcu_access_pointer(obj->fence_excl)) + return NULL; + + rcu_read_lock(); + fence = dma_fence_get_rcu_safe(&obj->fence_excl); + rcu_read_unlock(); + + return fence; +} + /** * reservation_object_get_list - get the reservation object's * shared fence list, with update-side lock held @@ -96,6 +141,29 @@ reservation_object_get_list(struct reservation_object *obj) reservation_object_held(obj)); } +/** + * reservation_object_fences - read consistent fence pointers + * @obj: reservation object where we get the fences from + * @excl: pointer for the exclusive fence + * @list: pointer for the shared fence list + * + * Make sure we have a consisten exclusive fence and shared fence list. + * Must be called with rcu read side lock held. + */ +static inline void +reservation_object_fences(struct reservation_object *obj, + struct dma_fence **excl, + struct reservation_object_list **list) +{ + unsigned int seq; + + do { + seq = read_seqcount_begin(&obj->seq); + *excl = rcu_dereference(obj->fence_excl); + *list = rcu_dereference(obj->fence); + } while (read_seqcount_retry(&obj->seq, seq)); +} + /** * reservation_object_lock - lock the reservation object * @obj: the reservation object @@ -239,51 +307,6 @@ reservation_object_unlock(struct reservation_object *obj) ww_mutex_unlock(&obj->lock); } -/** - * reservation_object_get_excl - get the reservation object's - * exclusive fence, with update-side lock held - * @obj: the reservation object - * - * Returns the exclusive fence (if any). Does NOT take a - * reference. Writers must hold obj->lock, readers may only - * hold a RCU read side lock. - * - * RETURNS - * The exclusive fence or NULL - */ -static inline struct dma_fence * -reservation_object_get_excl(struct reservation_object *obj) -{ - return rcu_dereference_protected(obj->fence_excl, - reservation_object_held(obj)); -} - -/** - * reservation_object_get_excl_rcu - get the reservation object's - * exclusive fence, without lock held. - * @obj: the reservation object - * - * If there is an exclusive fence, this atomically increments it's - * reference count and returns it. - * - * RETURNS - * The exclusive fence or NULL if none - */ -static inline struct dma_fence * -reservation_object_get_excl_rcu(struct reservation_object *obj) -{ - struct dma_fence *fence; - - if (!rcu_access_pointer(obj->fence_excl)) - return NULL; - - rcu_read_lock(); - fence = dma_fence_get_rcu_safe(&obj->fence_excl); - rcu_read_unlock(); - - return fence; -} - void reservation_object_init(struct reservation_object *obj); void reservation_object_fini(struct reservation_object *obj); int reservation_object_reserve_shared(struct reservation_object *obj,