From patchwork Thu Oct 18 15:28:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 10647523 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F4F915E2 for ; Thu, 18 Oct 2018 15:29:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F336828C76 for ; Thu, 18 Oct 2018 15:29:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E7A3628D97; Thu, 18 Oct 2018 15:29:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 618FC28C76 for ; Thu, 18 Oct 2018 15:29:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 57F6F6E0D0; Thu, 18 Oct 2018 15:28:49 +0000 (UTC) X-Original-To: Intel-gfx@lists.freedesktop.org Delivered-To: Intel-gfx@lists.freedesktop.org Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by gabe.freedesktop.org (Postfix) with ESMTPS id D85A56E046 for ; Thu, 18 Oct 2018 15:28:38 +0000 (UTC) Received: by mail-wm1-x344.google.com with SMTP id 143-v6so705875wmf.1 for ; Thu, 18 Oct 2018 08:28:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zF9IPHD4gKP/6Zix86oPEwrf5vlPGi7/+W7j+mg4BS0=; b=GrabtT9Y9hCb6WSC9vQB9e7Fw5VTkkGcM4HH5GFtuTPH7g88vOA6csJYlREa3feVQg au0/Fbs2BPqR+OKhO2bvl0yRj+v9AzGz+ONAM8O1aotrtnYvsUfgPpWP12leN2VjrHkt Ss3zZ88jGiTvD1GOqDE+2eI3XheTmkkdEmdaUKBbzDUmWN/y/O001BSh16l8F4VtI9h9 P0h8YnBCS3cacGq6fP2o5TQl50L9Vf0pio2KQqdPThFJi6VXSYHEN/iNZzj1i490M+MU linKZ8LcKP/eftdYo6Ecsl+pakb2ROuxakXv+DjccDw5Olf63xYzHfT7eMa7f2uAsJxI +8aQ== X-Gm-Message-State: ABuFfoiN4XZjJzhQNSEhTikfwGV26JDILzFRI1lqVOuWzLS+NH/PfOm9 YGR+IoAEErxuMQYkdMS36jLhUyiDHaw= X-Google-Smtp-Source: ACcGV63+ECv/DNiiQin+M3J6biv3xGoF1cpDVAJDOmDLFLVdhr3Zi9Tmq0tttzAdg6FiHoJqUXsmDg== X-Received: by 2002:a1c:f514:: with SMTP id t20-v6mr799497wmh.129.1539876517182; Thu, 18 Oct 2018 08:28:37 -0700 (PDT) Received: from localhost.localdomain ([91.110.193.16]) by smtp.gmail.com with ESMTPSA id i6-v6sm19530387wrq.4.2018.10.18.08.28.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 18 Oct 2018 08:28:36 -0700 (PDT) From: Tvrtko Ursulin X-Google-Original-From: Tvrtko Ursulin To: igt-dev@lists.freedesktop.org Date: Thu, 18 Oct 2018 16:28:13 +0100 Message-Id: <20181018152815.31816-16-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181018152815.31816-1-tvrtko.ursulin@linux.intel.com> References: <20181018152815.31816-1-tvrtko.ursulin@linux.intel.com> Subject: [Intel-gfx] [PATCH i-g-t 15/17] gem_wsim: Engine bond command X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel-gfx@lists.freedesktop.org MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Tvrtko Ursulin Engine bonds are an i915 uAPI applicable to load balanced contexts with engine map. They allow expression rules of engine selection between two contexts when submissions are also tied with submit fences. Please refer to the README for a more detailed description. Signed-off-by: Tvrtko Ursulin --- benchmarks/gem_wsim.c | 100 ++++++++++++++++++++++++++++++++++++++--- benchmarks/wsim/README | 50 +++++++++++++++++++++ 2 files changed, 143 insertions(+), 7 deletions(-) diff --git a/benchmarks/gem_wsim.c b/benchmarks/gem_wsim.c index a772e2c588b5..b5ade7b33883 100644 --- a/benchmarks/gem_wsim.c +++ b/benchmarks/gem_wsim.c @@ -84,6 +84,7 @@ enum w_type PREEMPTION, ENGINE_MAP, LOAD_BALANCE, + BOND, }; struct deps @@ -99,6 +100,11 @@ struct w_arg { int prio; }; +struct bond { + uint64_t mask; + enum intel_engine_id master; +}; + struct w_step { /* Workload step metadata */ @@ -122,6 +128,10 @@ struct w_step enum intel_engine_id *engine_map; }; bool load_balance; + struct { + uint64_t bond_mask; + enum intel_engine_id bond_master; + }; }; /* Implementation details */ @@ -153,6 +163,8 @@ struct ctx { int priority; unsigned int engine_map_count; enum intel_engine_id *engine_map; + unsigned int bond_count; + struct bond *bonds; bool targets_instance; bool wants_balance; unsigned int static_vcs; @@ -523,6 +535,40 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w) step.type = LOAD_BALANCE; goto add_step; + } else if (!strcmp(field, "b")) { + unsigned int nr = 0; + while ((field = strtok_r(fstart, ".", &fctx))) { + tmp = atoi(field); + check_arg(nr == 0 && tmp <= 0, + "Invalid context at step %u!\n", + nr_steps); + check_arg(nr == 1 && + (tmp < -1 || tmp == 0), + "Invalid siblings mask at step %u!\n", + nr_steps); + check_arg(nr > 2, + "Invalid bond format at step %u!\n", + nr_steps); + + if (nr == 0) { + step.context = tmp; + } else if (nr == 1) { + step.bond_mask = tmp; + } else if (nr == 2) { + tmp = str_to_engine(field); + check_arg(tmp <= 0 || + tmp == VCS || + tmp == DEFAULT, + "Invalid master engine at step %u!\n", + nr_steps); + step.bond_master = tmp; + } + + nr++; + } + + step.type = BOND; + goto add_step; } tmp = atoi(field); @@ -1044,6 +1090,8 @@ prepare_workload(unsigned int id, struct workload *wrk, unsigned int flags) * Transfer over engine map configuration from the workload step. */ for (j = 0; j < wrk->nr_ctxs; j += 2) { + struct ctx *ctx = &wrk->ctx_list[j]; + bool targets = false; bool balance = false; @@ -1057,16 +1105,28 @@ prepare_workload(unsigned int id, struct workload *wrk, unsigned int flags) else targets = true; } else if (w->type == ENGINE_MAP) { - wrk->ctx_list[j].engine_map = w->engine_map; - wrk->ctx_list[j].engine_map_count = - w->engine_map_count; + ctx->engine_map = w->engine_map; + ctx->engine_map_count = w->engine_map_count; } else if (w->type == LOAD_BALANCE) { - if (!wrk->ctx_list[j].engine_map) { + if (!ctx->engine_map) { wsim_err("Load balancing needs an engine map!\n"); return 1; } - wrk->ctx_list[j].wants_balance = - w->load_balance; + ctx->wants_balance = w->load_balance; + } else if (w->type == BOND) { + if (!ctx->wants_balance) { + wsim_err("Engine bonds need load balancing engine map!\n"); + return 1; + } + ctx->bond_count++; + ctx->bonds = realloc(ctx->bonds, + ctx->bond_count * + sizeof(struct bond)); + igt_assert(ctx->bonds); + ctx->bonds[ctx->bond_count - 1].mask = + w->bond_mask; + ctx->bonds[ctx->bond_count - 1].master = + w->bond_master; } } @@ -1196,6 +1256,7 @@ prepare_workload(unsigned int id, struct workload *wrk, unsigned int flags) { .base.name = I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE, .engines_mask = -1, }; + struct i915_context_engines_bond *bonds = NULL; if (ctx->wants_balance) { set_engines.extensions = @@ -1211,7 +1272,31 @@ prepare_workload(unsigned int id, struct workload *wrk, unsigned int flags) ctx->engine_map[j] - VCS1; /* FIXME */ } + if (ctx->bond_count) { + bonds = calloc(ctx->bond_count, sizeof(*bonds)); + load_balance.base.next_extension = + to_user_pointer(&bonds[0]); + } + + for (j = 0; j < ctx->bond_count; j++) { + struct i915_context_engines_bond *bond = + &bonds[j]; + + if (j < (ctx->bond_count - 1)) + bond->base.next_extension = + to_user_pointer(bond + 1); + + bond->base.name = I915_CONTEXT_ENGINES_EXT_BOND; + bond->master_class = I915_ENGINE_CLASS_VIDEO; + bond->master_instance = + ctx->bonds[j].master - VCS1; + bond->sibling_mask = ctx->bonds[j].mask; + } + gem_context_set_param(fd, ¶m); + + if (bonds) + free(bonds); } else if (ctx->wants_balance) { struct i915_context_engines_load_balance load_balance = { .base.name = I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE, @@ -2182,7 +2267,8 @@ static void *run_workload(void *data) continue; } else if (w->type == PREEMPTION || w->type == ENGINE_MAP || - w->type == LOAD_BALANCE) { + w->type == LOAD_BALANCE || + w->type == BOND) { continue; } diff --git a/benchmarks/wsim/README b/benchmarks/wsim/README index 58dada675357..f2974992ab68 100644 --- a/benchmarks/wsim/README +++ b/benchmarks/wsim/README @@ -7,6 +7,7 @@ B. M..[|]... P|X.. d|p|s|t|q|a.,... +b... f For duration a range can be given from which a random value will be picked @@ -26,6 +27,7 @@ Additional workload steps are also supported: 'f' - Create a sync fence. 'a' - Advance the previously created sync fence. 'B' - Turn on context load balancing. + 'b' - Set up engine bonds. 'M' - Set up engine map. 'P' - Context priority. 'X' - Context preemption control. @@ -194,3 +196,51 @@ This enables load balancing for context number one. Submissions to load balanced contexts are only allowed to use the DEFAULT engine specifier. + +Engine bonds +------------ + +Engine bonds are extensions on load balanced contexts. They allow expressing +rules of engine selection between two co-operating contexts tied with submit +fences. In other words, the rule expression is telling the driver: "If you pick +this engine for context one, then you have to pick that engine for context two". + +Syntax is: + b... + +Engine mask is a bitmask representing engines in the engine map configured for +the same context. + +There can be multiple bonds tied to the same context. + +Example: + + M.1.RCS|VECS + B.1 + M.2.VCS1|VCS2 + B.2 + b.2.1.RCS + b.2.2.VECS + +This tells the driver that if it picked RCS for context one, it has to pick VCS1 +for context two. And if it picked VECS for context one, it has to pick VCS1 for +context two. + +If we extend the above example with more workload directives: + + 1.DEFAULT.1000.0.0 + 2.DEFAULT.1000.s-1.0 + +We get to a fully functional example where two batch buffers are submitted in a +load balanced fashion, telling the driver they should run simultaneously and +that valid engine pairs are either RCS + VCS1 (for two contexts respectively), +or VECS + VCS2. + +This can also be extended using sync fences to improve chances of the first +submission not getting on the hardware after the second one. Second block would +then look like: + + f + 1.DEFAULT.1000.f-1.0 + 2.DEFAULT.1000.s-1.0 + a.-3