From patchwork Sat Aug 17 14:47:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 180E714DB for ; Sat, 17 Aug 2019 14:48:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0756A27EE2 for ; Sat, 17 Aug 2019 14:48:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F0101284BD; Sat, 17 Aug 2019 14:48:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 64C4827EE2 for ; Sat, 17 Aug 2019 14:48:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2AAA26E514; Sat, 17 Aug 2019 14:48:01 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E5D0C6E501; Sat, 17 Aug 2019 14:47:49 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173475-1500050 for multiple; Sat, 17 Aug 2019 15:47:38 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 1/6] dma-buf: Introduce selftesting framework Date: Sat, 17 Aug 2019 15:47:31 +0100 Message-Id: <20190817144736.7826-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP In light of recent review slip ups, the absence of a suite of tests for dma-buf became apparent. Given the current plethora of testing frameworks, opt for one already in use by Intel's CI and so allow easy hook up into igt. We introduce a new module that when loaded will execute the list of selftests and their subtest. The names of the selftests are put into the modinfo as parameters so that igt can identify each, and run them independently, principally for ease of error reporting. Signed-off-by: Chris Wilson Cc: Daniel Vetter --- drivers/dma-buf/Kconfig | 5 ++ drivers/dma-buf/Makefile | 3 + drivers/dma-buf/selftest.c | 167 ++++++++++++++++++++++++++++++++++++ drivers/dma-buf/selftest.h | 30 +++++++ drivers/dma-buf/selftests.h | 12 +++ 5 files changed, 217 insertions(+) create mode 100644 drivers/dma-buf/selftest.c create mode 100644 drivers/dma-buf/selftest.h create mode 100644 drivers/dma-buf/selftests.h diff --git a/drivers/dma-buf/Kconfig b/drivers/dma-buf/Kconfig index b6a9c2f1bc41..a23b6752d11a 100644 --- a/drivers/dma-buf/Kconfig +++ b/drivers/dma-buf/Kconfig @@ -39,4 +39,9 @@ config UDMABUF A driver to let userspace turn memfd regions into dma-bufs. Qemu can use this to create host dmabufs for guest framebuffers. +config DMABUF_SELFTESTS + tristate "Selftests for the dma-buf interfaces" + default n + depends on DMA_SHARED_BUFFER + endmenu diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index dcfb01e7c6f4..b5ae122a9349 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -4,3 +4,6 @@ obj-y := dma-buf.o dma-fence.o dma-fence-array.o dma-fence-chain.o \ obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o + +dmabuf_selftests-y := selftest.o +obj-$(CONFIG_DMABUF_SELFTESTS) += dmabuf_selftests.o diff --git a/drivers/dma-buf/selftest.c b/drivers/dma-buf/selftest.c new file mode 100644 index 000000000000..c60b6944b4bd --- /dev/null +++ b/drivers/dma-buf/selftest.c @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: MIT */ + +/* + * Copyright © 2019 Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include "selftest.h" + +enum { +#define selftest(n, func) __idx_##n, +#include "selftests.h" +#undef selftest +}; + +#define selftest(n, f) [__idx_##n] = { .name = #n, .func = f }, +static struct selftest { + bool enabled; + const char *name; + int (*func)(void); +} selftests[] = { +#include "selftests.h" +}; +#undef selftest + +/* Embed the line number into the parameter name so that we can order tests */ +#define param(n) __PASTE(igt__, __PASTE(__PASTE(__LINE__, __), n)) +#define selftest_0(n, func, id) \ +module_param_named(id, selftests[__idx_##n].enabled, bool, 0400); +#define selftest(n, func) selftest_0(n, func, param(n)) +#include "selftests.h" +#undef selftest + +int __sanitycheck__(void) +{ + pr_debug("Hello World!\n"); + return 0; +} + +static char *__st_filter; + +static bool apply_subtest_filter(const char *caller, const char *name) +{ + char *filter, *sep, *tok; + bool result = true; + + filter = kstrdup(__st_filter, GFP_KERNEL); + for (sep = filter; (tok = strsep(&sep, ","));) { + bool allow = true; + char *sl; + + if (*tok == '!') { + allow = false; + tok++; + } + + if (*tok == '\0') + continue; + + sl = strchr(tok, '/'); + if (sl) { + *sl++ = '\0'; + if (strcmp(tok, caller)) { + if (allow) + result = false; + continue; + } + tok = sl; + } + + if (strcmp(tok, name)) { + if (allow) + result = false; + continue; + } + + result = allow; + break; + } + kfree(filter); + + return result; +} + +int +__subtests(const char *caller, const struct subtest *st, int count, void *data) +{ + int err; + + for (; count--; st++) { + cond_resched(); + if (signal_pending(current)) + return -EINTR; + + if (!apply_subtest_filter(caller, st->name)) + continue; + + pr_info("dma-buf: Running %s/%s\n", caller, st->name); + + err = st->func(data); + if (err && err != -EINTR) { + pr_err("dma-buf/%s: %s failed with error %d\n", + caller, st->name, err); + return err; + } + } + + return 0; +} + +static void set_default_test_all(struct selftest *st, unsigned long count) +{ + unsigned long i; + + for (i = 0; i < count; i++) + if (st[i].enabled) + return; + + for (i = 0; i < count; i++) + st[i].enabled = true; +} + +static int run_selftests(struct selftest *st, unsigned long count) +{ + int err = 0; + + set_default_test_all(st, count); + + /* Tests are listed in natural order in selftests.h */ + for (; count--; st++) { + if (!st->enabled) + continue; + + pr_info("dma-buf: Running %s\n", st->name); + err = st->func(); + if (err) + break; + } + + if (WARN(err > 0 || err == -ENOTTY, + "%s returned %d, conflicting with selftest's magic values!\n", + st->name, err)) + err = -1; + + return err; +} + +static int __init st_init(void) +{ + return run_selftests(selftests, ARRAY_SIZE(selftests)); +} + +static void __exit st_exit(void) +{ +} + +module_param_named(st_filter, __st_filter, charp, 0400); +module_init(st_init); +module_exit(st_exit); + +MODULE_DESCRIPTION("Self-test harness for dma-buf"); +MODULE_LICENSE("GPL and additional rights"); diff --git a/drivers/dma-buf/selftest.h b/drivers/dma-buf/selftest.h new file mode 100644 index 000000000000..45793aff6142 --- /dev/null +++ b/drivers/dma-buf/selftest.h @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: MIT + +/* + * Copyright © 2019 Intel Corporation + */ + +#ifndef __SELFTEST_H__ +#define __SELFTEST_H__ + +#include + +#define selftest(name, func) int func(void); +#include "selftests.h" +#undef selftest + +struct subtest { + int (*func)(void *data); + const char *name; +}; + +int __subtests(const char *caller, + const struct subtest *st, + int count, + void *data); +#define subtests(T, data) \ + __subtests(__func__, T, ARRAY_SIZE(T), data) + +#define SUBTEST(x) { x, #x } + +#endif /* __SELFTEST_H__ */ diff --git a/drivers/dma-buf/selftests.h b/drivers/dma-buf/selftests.h new file mode 100644 index 000000000000..44b44390d23a --- /dev/null +++ b/drivers/dma-buf/selftests.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: MIT */ +/* List each unit test as selftest(name, function) + * + * The name is used as both an enum and expanded as subtest__name to create + * a module parameter. It must be unique and legal for a C identifier. + * + * The function should be of type int function(void). It may be conditionally + * compiled using #if IS_ENABLED(DRM_I915_SELFTEST). + * + * Tests are executed in order by igt/dmabuf_selftest + */ +selftest(sanitycheck, __sanitycheck__) /* keep first (igt selfcheck) */ From patchwork Sat Aug 17 14:47:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099109 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9866313A0 for ; Sat, 17 Aug 2019 14:48:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8801F27EE2 for ; Sat, 17 Aug 2019 14:48:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7BE6228488; Sat, 17 Aug 2019 14:48:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B0C5527EE2 for ; Sat, 17 Aug 2019 14:48:02 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8940D6E501; Sat, 17 Aug 2019 14:47:52 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id D07746E4F1; Sat, 17 Aug 2019 14:47:50 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173476-1500050 for multiple; Sat, 17 Aug 2019 15:47:38 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 2/6] dma-buf: Add selftests for dma-fence Date: Sat, 17 Aug 2019 15:47:32 +0100 Message-Id: <20190817144736.7826-2-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190817144736.7826-1-chris@chris-wilson.co.uk> References: <20190817144736.7826-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Exercise the dma-fence API exported to drivers. Signed-off-by: Chris Wilson Cc: Daniel Vetter --- drivers/dma-buf/Makefile | 5 +- drivers/dma-buf/selftests.h | 1 + drivers/dma-buf/st-dma-fence.c | 571 +++++++++++++++++++++++++++++++++ 3 files changed, 576 insertions(+), 1 deletion(-) create mode 100644 drivers/dma-buf/st-dma-fence.c diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index b5ae122a9349..03479da06422 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -5,5 +5,8 @@ obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o -dmabuf_selftests-y := selftest.o +dmabuf_selftests-y := \ + selftest.o \ + st-dma-fence.o + obj-$(CONFIG_DMABUF_SELFTESTS) += dmabuf_selftests.o diff --git a/drivers/dma-buf/selftests.h b/drivers/dma-buf/selftests.h index 44b44390d23a..5320386f02e5 100644 --- a/drivers/dma-buf/selftests.h +++ b/drivers/dma-buf/selftests.h @@ -10,3 +10,4 @@ * Tests are executed in order by igt/dmabuf_selftest */ selftest(sanitycheck, __sanitycheck__) /* keep first (igt selfcheck) */ +selftest(dma_fence, dma_fence) diff --git a/drivers/dma-buf/st-dma-fence.c b/drivers/dma-buf/st-dma-fence.c new file mode 100644 index 000000000000..d052bea4fb9f --- /dev/null +++ b/drivers/dma-buf/st-dma-fence.c @@ -0,0 +1,571 @@ +/* SPDX-License-Identifier: MIT */ + +/* + * Copyright © 2019 Intel Corporation + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "selftest.h" + +static struct kmem_cache *slab_fences; + +static struct mock_fence { + struct dma_fence base; + struct spinlock lock; +} *to_mock_fence(struct dma_fence *f) { + return container_of(f, struct mock_fence, base); +} + +static const char *mock_name(struct dma_fence *f) +{ + return "mock"; +} + +static void mock_fence_release(struct dma_fence *f) +{ + kmem_cache_free(slab_fences, to_mock_fence(f)); +} + +struct wait_cb { + struct dma_fence_cb cb; + struct task_struct *task; +}; + +static void mock_wakeup(struct dma_fence *f, struct dma_fence_cb *cb) +{ + wake_up_process(container_of(cb, struct wait_cb, cb)->task); +} + +static long mock_wait(struct dma_fence *f, bool intr, long timeout) +{ + const int state = intr ? TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE; + struct wait_cb cb = { .task = current }; + + if (dma_fence_add_callback(f, &cb.cb, mock_wakeup)) + return timeout; + + while (timeout) { + set_current_state(state); + + if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &f->flags)) + break; + + if (signal_pending_state(state, current)) + break; + + timeout = schedule_timeout(timeout); + } + __set_current_state(TASK_RUNNING); + + if (!dma_fence_remove_callback(f, &cb.cb)) + return timeout; + + if (signal_pending_state(state, current)) + return -ERESTARTSYS; + + return -ETIME; +} + +static const struct dma_fence_ops mock_ops = { + .get_driver_name = mock_name, + .get_timeline_name = mock_name, + .wait = mock_wait, + .release = mock_fence_release, +}; + +static struct dma_fence *mock_fence(void) +{ + struct mock_fence *f; + + f = kmem_cache_alloc(slab_fences, GFP_KERNEL); + if (!f) + return NULL; + + spin_lock_init(&f->lock); + dma_fence_init(&f->base, &mock_ops, &f->lock, 0, 0); + + return &f->base; +} + +static int sanitycheck(void *arg) +{ + struct dma_fence *f; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + dma_fence_signal(f); + dma_fence_put(f); + + return 0; +} + +static int test_signaling(void *arg) +{ + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_is_signaled(f)) { + pr_err("Fence unexpectedly signaled on creation\n"); + goto err_free; + } + + if (dma_fence_signal(f)) { + pr_err("Fence reported being already signaled\n"); + goto err_free; + } + + if (!dma_fence_is_signaled(f)) { + pr_err("Fence not reporting signaled\n"); + goto err_free; + } + + if (!dma_fence_signal(f)) { + pr_err("Fence reported not being already signaled\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +struct simple_cb { + struct dma_fence_cb cb; + bool seen; +}; + +static void simple_callback(struct dma_fence *f, struct dma_fence_cb *cb) +{ + smp_store_mb(container_of(cb, struct simple_cb, cb)->seen, true); +} + +static int test_add_callback(void *arg) +{ + struct simple_cb cb = {}; + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_add_callback(f, &cb.cb, simple_callback)) { + pr_err("Failed to add callback, fence already signaled!\n"); + goto err_free; + } + + dma_fence_signal(f); + if (!cb.seen) { + pr_err("Callback failed!\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_late_add_callback(void *arg) +{ + struct simple_cb cb = {}; + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + dma_fence_signal(f); + + if (!dma_fence_add_callback(f, &cb.cb, simple_callback)) { + pr_err("Added callback, but fence was already signaled!\n"); + goto err_free; + } + + dma_fence_signal(f); + if (cb.seen) { + pr_err("Callback called after failed attachment !\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_rm_callback(void *arg) +{ + struct simple_cb cb = {}; + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_add_callback(f, &cb.cb, simple_callback)) { + pr_err("Failed to add callback, fence already signaled!\n"); + goto err_free; + } + + if (!dma_fence_remove_callback(f, &cb.cb)) { + pr_err("Failed to remove callback!\n"); + goto err_free; + } + + dma_fence_signal(f); + if (cb.seen) { + pr_err("Callback still signaled after removal!\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_late_rm_callback(void *arg) +{ + struct simple_cb cb = {}; + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_add_callback(f, &cb.cb, simple_callback)) { + pr_err("Failed to add callback, fence already signaled!\n"); + goto err_free; + } + + dma_fence_signal(f); + if (!cb.seen) { + pr_err("Callback failed!\n"); + goto err_free; + } + + if (dma_fence_remove_callback(f, &cb.cb)) { + pr_err("Callback removal succeed after being executed!\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_status(void *arg) +{ + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_get_status(f)) { + pr_err("Fence unexpectedly has signaled status on creation\n"); + goto err_free; + } + + dma_fence_signal(f); + if (!dma_fence_get_status(f)) { + pr_err("Fence not reporting signaled status\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_error(void *arg) +{ + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + dma_fence_set_error(f, -EIO); + + if (dma_fence_get_status(f)) { + pr_err("Fence unexpectedly has error status before signal\n"); + goto err_free; + } + + dma_fence_signal(f); + if (dma_fence_get_status(f) != -EIO) { + pr_err("Fence not reporting error status, got %d\n", + dma_fence_get_status(f)); + goto err_free; + } + + err = 0; +err_free: + dma_fence_put(f); + return err; +} + +static int test_wait(void *arg) +{ + struct dma_fence *f; + int err = -EINVAL; + + f = mock_fence(); + if (!f) + return -ENOMEM; + + if (dma_fence_wait_timeout(f, false, 0) != -ETIME) { + pr_err("Wait reported complete before being signaled\n"); + goto err_free; + } + + dma_fence_signal(f); + + if (dma_fence_wait_timeout(f, false, 0) != 0) { + pr_err("Wait reported incomplete after being signaled\n"); + goto err_free; + } + + err = 0; +err_free: + dma_fence_signal(f); + dma_fence_put(f); + return err; +} + +struct wait_timer { + struct timer_list timer; + struct dma_fence *f; +}; + +static void wait_timer(struct timer_list *timer) +{ + struct wait_timer *wt = from_timer(wt, timer, timer); + + dma_fence_signal(wt->f); +} + +static int test_wait_timeout(void *arg) +{ + struct wait_timer wt; + int err = -EINVAL; + + timer_setup(&wt.timer, wait_timer, 0); + + wt.f = mock_fence(); + if (!wt.f) + return -ENOMEM; + + if (dma_fence_wait_timeout(wt.f, false, 1) != -ETIME) { + pr_err("Wait reported complete before being signaled\n"); + goto err_free; + } + + mod_timer(&wt.timer, 1); + + if (dma_fence_wait_timeout(wt.f, false, 2) == -ETIME) { + if (timer_pending(&wt.timer)) { + pr_notice("Timer did not fire within the jiffie!\n"); + err = 0; /* not our fault! */ + } else { + pr_err("Wait reported incomplete after timeout\n"); + } + goto err_free; + } + + err = 0; +err_free: + del_timer_sync(&wt.timer); + dma_fence_signal(wt.f); + dma_fence_put(wt.f); + return err; +} + +static int test_stub(void *arg) +{ + struct dma_fence *f[64]; + int err = -EINVAL; + int i; + + for (i = 0; i < ARRAY_SIZE(f); i++) { + f[i] = dma_fence_get_stub(); + if (!dma_fence_is_signaled(f[i])) { + pr_err("Obtained unsignaled stub fence!\n"); + goto err; + } + } + + err = 0; +err: + while (i--) + dma_fence_put(f[i]); + return err; +} + +/* Now off to the races! */ + +struct race_thread { + struct dma_fence __rcu **fences; + struct task_struct *task; + bool before; + int id; +}; + +static void __wait_for_callbacks(struct dma_fence *f) +{ + spin_lock_irq(f->lock); + spin_unlock_irq(f->lock); +} + +static int thread_signal_callback(void *arg) +{ + const struct race_thread *t = arg; + unsigned long pass = 0; + unsigned long miss = 0; + int err = 0; + + while (!err && !kthread_should_stop()) { + struct dma_fence *f1, *f2; + struct simple_cb cb; + + f1 = mock_fence(); + if (!f1) { + err = -ENOMEM; + break; + } + + rcu_assign_pointer(t->fences[t->id], f1); + smp_wmb(); + + rcu_read_lock(); + do { + f2 = dma_fence_get_rcu_safe(&t->fences[!t->id]); + } while (!f2 && !kthread_should_stop()); + rcu_read_unlock(); + + if (t->before) + dma_fence_signal(f1); + + smp_store_mb(cb.seen, false); + if (!f2 || dma_fence_add_callback(f2, &cb.cb, simple_callback)) + miss++, cb.seen = true; + + if (!t->before) + dma_fence_signal(f1); + + if (!cb.seen) { + dma_fence_wait(f2, false); + __wait_for_callbacks(f2); + } + + if (!READ_ONCE(cb.seen)) { + pr_err("Callback not seen on thread %d, pass %lu (%lu misses), signaling %s add_callback; fence signaled? %s\n", + t->id, pass, miss, + t->before ? "before" : "after", + dma_fence_is_signaled(f2) ? "yes" : "no"); + err = -EINVAL; + } + + dma_fence_put(f2); + + rcu_assign_pointer(t->fences[t->id], NULL); + smp_wmb(); + + dma_fence_put(f1); + + pass++; + } + + pr_info("%s[%d] completed %lu passes, %lu misses\n", + __func__, t->id, pass, miss); + return err; +} + +static int race_signal_callback(void *arg) +{ + struct dma_fence __rcu *f[2] = {}; + int ret = 0; + int pass; + + for (pass = 0; !ret && pass <= 1; pass++) { + struct race_thread t[2]; + int i; + + for (i = 0; i < ARRAY_SIZE(t); i++) { + t[i].fences = f; + t[i].id = i; + t[i].before = pass; + t[i].task = kthread_run(thread_signal_callback, &t[i], + "dma-fence:%d", i); + get_task_struct(t[i].task); + } + + msleep(50); + + for (i = 0; i < ARRAY_SIZE(t); i++) { + int err; + + err = kthread_stop(t[i].task); + if (err && !ret) + ret = err; + + put_task_struct(t[i].task); + } + } + + return ret; +} + +int dma_fence(void) +{ + static const struct subtest tests[] = { + SUBTEST(sanitycheck), + SUBTEST(test_signaling), + SUBTEST(test_add_callback), + SUBTEST(test_late_add_callback), + SUBTEST(test_rm_callback), + SUBTEST(test_late_rm_callback), + SUBTEST(test_status), + SUBTEST(test_error), + SUBTEST(test_wait), + SUBTEST(test_wait_timeout), + SUBTEST(test_stub), + SUBTEST(race_signal_callback), + }; + int ret; + + pr_info("sizeof(dma_fence)=%lu\n", sizeof(struct dma_fence)); + + slab_fences = KMEM_CACHE(mock_fence, + SLAB_TYPESAFE_BY_RCU | + SLAB_HWCACHE_ALIGN); + + ret = subtests(tests, NULL); + + kmem_cache_destroy(slab_fences); + + return ret; +} From patchwork Sat Aug 17 14:47:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099105 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48F7114DB for ; Sat, 17 Aug 2019 14:47:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39E2D27EE2 for ; Sat, 17 Aug 2019 14:47:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2DFDB28488; Sat, 17 Aug 2019 14:47:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E31C227EE2 for ; Sat, 17 Aug 2019 14:47:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B50D96E503; Sat, 17 Aug 2019 14:47:51 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E16FC6E4FF; Sat, 17 Aug 2019 14:47:49 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173477-1500050 for multiple; Sat, 17 Aug 2019 15:47:39 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 3/6] dma-fence: Shrink size of struct dma_fence Date: Sat, 17 Aug 2019 15:47:33 +0100 Message-Id: <20190817144736.7826-3-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190817144736.7826-1-chris@chris-wilson.co.uk> References: <20190817144736.7826-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Rearrange the couple of 32-bit atomics hidden amongst the field of pointers that unnecessarily caused the compiler to insert some padding, shrinks the size of the base struct dma_fence from 80 to 72 bytes on x86-64. Signed-off-by: Chris Wilson Cc: Christian König Reviewed-by: Christian König --- include/linux/dma-fence.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index 404aa748eda6..2ce4d877d33e 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -63,7 +63,7 @@ struct dma_fence_cb; * been completed, or never called at all. */ struct dma_fence { - struct kref refcount; + spinlock_t *lock; const struct dma_fence_ops *ops; /* We clear the callback list on kref_put so that by the time we * release the fence it is unused. No one should be adding to the cb_list @@ -73,11 +73,11 @@ struct dma_fence { struct rcu_head rcu; struct list_head cb_list; }; - spinlock_t *lock; u64 context; u64 seqno; - unsigned long flags; ktime_t timestamp; + unsigned long flags; + struct kref refcount; int error; }; From patchwork Sat Aug 17 14:47:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B09D13A0 for ; Sat, 17 Aug 2019 14:47:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C9D927EE2 for ; Sat, 17 Aug 2019 14:47:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3B33B28488; Sat, 17 Aug 2019 14:47:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2724227EE2 for ; Sat, 17 Aug 2019 14:47:55 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 055D36E4FF; Sat, 17 Aug 2019 14:47:51 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id E3AD56E500; Sat, 17 Aug 2019 14:47:49 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173479-1500050 for multiple; Sat, 17 Aug 2019 15:47:39 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 4/6] dma-fence: Avoid list_del during fence->cb_list iteration Date: Sat, 17 Aug 2019 15:47:34 +0100 Message-Id: <20190817144736.7826-4-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190817144736.7826-1-chris@chris-wilson.co.uk> References: <20190817144736.7826-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Before we notify the fence signal callback, we remove the cb from the list. However, since we are processing the entire list from underneath the spinlock, we do not need to individual delete each element, but can simply reset the link and the entire list. Signed-off-by: Chris Wilson Cc: Daniel Vetter Cc: Christian König Reviewed-by: Christian König --- drivers/dma-buf/dma-fence.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 8025a891d3e9..ff0cd6eae766 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -149,9 +149,12 @@ int dma_fence_signal_locked(struct dma_fence *fence) trace_dma_fence_signaled(fence); } - list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { - list_del_init(&cur->node); - cur->func(fence, cur); + if (!list_empty(&fence->cb_list)) { + list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { + INIT_LIST_HEAD(&cur->node); + cur->func(fence, cur); + } + INIT_LIST_HEAD(&fence->cb_list); } return ret; } From patchwork Sat Aug 17 14:47:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099123 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBE2B13A0 for ; Sat, 17 Aug 2019 14:48:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACDE527EE2 for ; Sat, 17 Aug 2019 14:48:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A15A3284BD; Sat, 17 Aug 2019 14:48:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4E8C327EE2 for ; Sat, 17 Aug 2019 14:48:10 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 159906E517; Sat, 17 Aug 2019 14:48:02 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id CEA056E506; Sat, 17 Aug 2019 14:47:51 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173481-1500050 for multiple; Sat, 17 Aug 2019 15:47:39 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 5/6] dma-fence: Simply wrap dma_fence_signal_locked with dma_fence_signal Date: Sat, 17 Aug 2019 15:47:35 +0100 Message-Id: <20190817144736.7826-5-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190817144736.7826-1-chris@chris-wilson.co.uk> References: <20190817144736.7826-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Currently dma_fence_signal() tries to avoid the spinlock and only takes it if absolutely required to walk the callback list. However, to allow for some users to surreptitiously insert lazy signal callbacks that do not depend on enabling the signaling mechanism around every fence, we always need to notify the callbacks on signaling. As such, we will always need to take the spinlock and dma_fence_signal() effectively becomes a clone of dma_fence_signal_locked(). v2: Update the test_and_set_bit() before entering the spinlock. Signed-off-by: Chris Wilson Cc: Christian König Cc: Daniel Vetter --- drivers/dma-buf/dma-fence.c | 43 +++++++++++-------------------------- 1 file changed, 13 insertions(+), 30 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index ff0cd6eae766..89d96e3e6df6 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -129,25 +129,16 @@ EXPORT_SYMBOL(dma_fence_context_alloc); int dma_fence_signal_locked(struct dma_fence *fence) { struct dma_fence_cb *cur, *tmp; - int ret = 0; lockdep_assert_held(fence->lock); - if (WARN_ON(!fence)) + if (unlikely(test_and_set_bit(DMA_FENCE_FLAG_SIGNALED_BIT, + &fence->flags))) return -EINVAL; - if (test_and_set_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) { - ret = -EINVAL; - - /* - * we might have raced with the unlocked dma_fence_signal, - * still run through all callbacks - */ - } else { - fence->timestamp = ktime_get(); - set_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags); - trace_dma_fence_signaled(fence); - } + fence->timestamp = ktime_get(); + set_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags); + trace_dma_fence_signaled(fence); if (!list_empty(&fence->cb_list)) { list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { @@ -156,7 +147,8 @@ int dma_fence_signal_locked(struct dma_fence *fence) } INIT_LIST_HEAD(&fence->cb_list); } - return ret; + + return 0; } EXPORT_SYMBOL(dma_fence_signal_locked); @@ -176,28 +168,19 @@ EXPORT_SYMBOL(dma_fence_signal_locked); int dma_fence_signal(struct dma_fence *fence) { unsigned long flags; + int ret; if (!fence) return -EINVAL; - if (test_and_set_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) + if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) return -EINVAL; - fence->timestamp = ktime_get(); - set_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags); - trace_dma_fence_signaled(fence); - - if (test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT, &fence->flags)) { - struct dma_fence_cb *cur, *tmp; + spin_lock_irqsave(fence->lock, flags); + ret = dma_fence_signal_locked(fence); + spin_unlock_irqrestore(fence->lock, flags); - spin_lock_irqsave(fence->lock, flags); - list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { - list_del_init(&cur->node); - cur->func(fence, cur); - } - spin_unlock_irqrestore(fence->lock, flags); - } - return 0; + return ret; } EXPORT_SYMBOL(dma_fence_signal); From patchwork Sat Aug 17 14:47:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chris Wilson X-Patchwork-Id: 11099107 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4986314DB for ; Sat, 17 Aug 2019 14:48:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 39DD527EE2 for ; Sat, 17 Aug 2019 14:48:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E65928488; Sat, 17 Aug 2019 14:48:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BE1E727EE2 for ; Sat, 17 Aug 2019 14:48:00 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1A1F06E4F1; Sat, 17 Aug 2019 14:47:52 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from fireflyinternet.com (mail.fireflyinternet.com [109.228.58.192]) by gabe.freedesktop.org (Postfix) with ESMTPS id DEE356E4F1; Sat, 17 Aug 2019 14:47:49 +0000 (UTC) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from haswell.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 18173482-1500050 for multiple; Sat, 17 Aug 2019 15:47:39 +0100 From: Chris Wilson To: dri-devel@lists.freedesktop.org Subject: [PATCH 6/6] dma-fence: Store the timestamp in the same union as the cb_list Date: Sat, 17 Aug 2019 15:47:36 +0100 Message-Id: <20190817144736.7826-6-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.23.0.rc1 In-Reply-To: <20190817144736.7826-1-chris@chris-wilson.co.uk> References: <20190817144736.7826-1-chris@chris-wilson.co.uk> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP The timestamp and the cb_list are mutually exclusive, the cb_list can only be added to prior to being signaled (and once signaled we drain), while the timestamp is only valid upon being signaled. Both the timestamp and the cb_list are only valid while the fence is alive, and as soon as no references are held can be replaced by the rcu_head. By reusing the union for the timestamp, we squeeze the base dma_fence struct to 64 bytes on x86-64. Suggested-by: Christian König Signed-off-by: Chris Wilson Cc: Christian König --- drivers/dma-buf/dma-fence.c | 16 +++++++++------- drivers/gpu/drm/i915/gt/intel_breadcrumbs.c | 13 +++++++------ drivers/gpu/drm/vmwgfx/vmwgfx_fence.c | 3 +++ include/linux/dma-fence.h | 17 +++++++++++++---- 4 files changed, 32 insertions(+), 17 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 89d96e3e6df6..2c21115b1a37 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -129,6 +129,7 @@ EXPORT_SYMBOL(dma_fence_context_alloc); int dma_fence_signal_locked(struct dma_fence *fence) { struct dma_fence_cb *cur, *tmp; + struct list_head cb_list; lockdep_assert_held(fence->lock); @@ -136,16 +137,16 @@ int dma_fence_signal_locked(struct dma_fence *fence) &fence->flags))) return -EINVAL; + /* Stash the cb_list before replacing it with the timestamp */ + list_replace(&fence->cb_list, &cb_list); + fence->timestamp = ktime_get(); set_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags); trace_dma_fence_signaled(fence); - if (!list_empty(&fence->cb_list)) { - list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { - INIT_LIST_HEAD(&cur->node); - cur->func(fence, cur); - } - INIT_LIST_HEAD(&fence->cb_list); + list_for_each_entry_safe(cur, tmp, &cb_list, node) { + INIT_LIST_HEAD(&cur->node); + cur->func(fence, cur); } return 0; @@ -234,7 +235,8 @@ void dma_fence_release(struct kref *kref) trace_dma_fence_destroy(fence); - if (WARN(!list_empty(&fence->cb_list), + if (WARN(!list_empty(&fence->cb_list) && + !test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags), "Fence %s:%s:%llx:%llx released with pending signals!\n", fence->ops->get_driver_name(fence), fence->ops->get_timeline_name(fence), diff --git a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c index 2bc9c460e78d..09c68dda2098 100644 --- a/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c +++ b/drivers/gpu/drm/i915/gt/intel_breadcrumbs.c @@ -114,18 +114,18 @@ __dma_fence_signal__timestamp(struct dma_fence *fence, ktime_t timestamp) } static void -__dma_fence_signal__notify(struct dma_fence *fence) +__dma_fence_signal__notify(struct dma_fence *fence, + const struct list_head *list) { struct dma_fence_cb *cur, *tmp; lockdep_assert_held(fence->lock); lockdep_assert_irqs_disabled(); - list_for_each_entry_safe(cur, tmp, &fence->cb_list, node) { + list_for_each_entry_safe(cur, tmp, list, node) { INIT_LIST_HEAD(&cur->node); cur->func(fence, cur); } - INIT_LIST_HEAD(&fence->cb_list); } void intel_engine_breadcrumbs_irq(struct intel_engine_cs *engine) @@ -187,11 +187,12 @@ void intel_engine_breadcrumbs_irq(struct intel_engine_cs *engine) list_for_each_safe(pos, next, &signal) { struct i915_request *rq = list_entry(pos, typeof(*rq), signal_link); - - __dma_fence_signal__timestamp(&rq->fence, timestamp); + struct list_head cb_list; spin_lock(&rq->lock); - __dma_fence_signal__notify(&rq->fence); + list_replace(&rq->fence.cb_list, &cb_list); + __dma_fence_signal__timestamp(&rq->fence, timestamp); + __dma_fence_signal__notify(&rq->fence, &cb_list); spin_unlock(&rq->lock); i915_request_put(rq); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c index 434dfadb0e52..178a6cd1a06f 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c @@ -185,6 +185,9 @@ static long vmw_fence_wait(struct dma_fence *f, bool intr, signed long timeout) spin_lock(f->lock); + if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &f->flags)) + goto out; + if (intr && signal_pending(current)) { ret = -ERESTARTSYS; goto out; diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index 2ce4d877d33e..6c36502a0562 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -65,17 +65,26 @@ struct dma_fence_cb; struct dma_fence { spinlock_t *lock; const struct dma_fence_ops *ops; - /* We clear the callback list on kref_put so that by the time we - * release the fence it is unused. No one should be adding to the cb_list - * that they don't themselves hold a reference for. + /* + * We clear the callback list on kref_put so that by the time we + * release the fence it is unused. No one should be adding to the + * cb_list that they don't themselves hold a reference for. + * + * The lifetime of the timestamp is similarly tied to both the + * rcu freelist and the cb_list. The timestamp is only set upon + * signaling while simultaneously notifying the cb_list. Ergo, we + * only use either the cb_list of timestamp. Upon destruction, + * neither are accessible, and so we can use the rcu. This means + * that the cb_list is *only* valid until the signal bit is set, + * and to read either you *must* hold a reference to the fence. */ union { struct rcu_head rcu; struct list_head cb_list; + ktime_t timestamp; }; u64 context; u64 seqno; - ktime_t timestamp; unsigned long flags; struct kref refcount; int error;