From patchwork Thu Jul 10 21:50:17 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 4528271 Return-Path: X-Original-To: patchwork-dri-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 5AAA59F37C for ; Thu, 10 Jul 2014 23:33:39 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6162D201D5 for ; Thu, 10 Jul 2014 23:33:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id 559FE201B4 for ; Thu, 10 Jul 2014 23:33:37 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 739176E796; Thu, 10 Jul 2014 16:33:15 -0700 (PDT) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-we0-f171.google.com (mail-we0-f171.google.com [74.125.82.171]) by gabe.freedesktop.org (Postfix) with ESMTP id 5ED276E3C0 for ; Thu, 10 Jul 2014 14:52:00 -0700 (PDT) Received: by mail-we0-f171.google.com with SMTP id q58so199470wes.30 for ; Thu, 10 Jul 2014 14:51:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cfP39jQfkxW709leRLXUrsAah5e5glomDXxsm2NfP00=; b=NJplhqhTwHvDChoqlKN6lw+FogdOfIGw7u/pHInrx9hdJiYDJZpEtV7AWT69VK2iFH i/Sf0wwZ4wEgvlfySfeKgynd8Pg2bzumJ4jsAgYawThaPoZo0IfAiflVeXbOhfUbA3aj qZk3llz9TK4sLFVXLJvmoLHhjY4YDZRm4wxEOMH0JRLnuVdRJAwzXyii5GkOabkfqDXS aC0ynLd3x4E8Q6H6q3VJnR6fUdCA2X6q0Zg2v1nb5OR1ogW8tFyenXaYvUwTmAcr9inZ +cwDVcnn150f4UXg7ON9Qo8ClZIN6H8IIKShwas+rzg4XqRDEBsXoP3X4XUQBr2CBLfQ E7OA== X-Received: by 10.194.234.1 with SMTP id ua1mr942176wjc.42.1405029119087; Thu, 10 Jul 2014 14:51:59 -0700 (PDT) Received: from localhost.localdomain ([77.127.59.49]) by mx.google.com with ESMTPSA id n2sm805353wjf.40.2014.07.10.14.51.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 10 Jul 2014 14:51:58 -0700 (PDT) From: Oded Gabbay X-Google-Original-From: Oded Gabbay To: David Airlie , Alex Deucher , Jerome Glisse Subject: [PATCH 17/83] hsa/radeon: Handle deactivation of queues using interrupts Date: Fri, 11 Jul 2014 00:50:17 +0300 Message-Id: <1405029027-6085-16-git-send-email-oded.gabbay@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1405029027-6085-1-git-send-email-oded.gabbay@amd.com> References: <1405029027-6085-1-git-send-email-oded.gabbay@amd.com> X-Mailman-Approved-At: Thu, 10 Jul 2014 16:33:12 -0700 Cc: Andrew Lewycky , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_WEB, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch modifies the scheduler code to use interrupts to handle the deactivation of queues. We prefer to use interrupts because the deactivation could take a long time since we need to wait for the wavefront to finish executing before deactivating the queue. There is an array of waitqueues, each cell is represents queues for a specific pipe. When a queue should be deactivated, it is inserted to the wait queue. The event that triggers the waitqueue is a dequeue-complete interrupt that arrives through the isr function of the scheduler. Signed-off-by: Oded Gabbay --- drivers/gpu/hsa/radeon/cik_regs.h | 1 + drivers/gpu/hsa/radeon/kfd_sched_cik_static.c | 45 +++++++++++++++++++++------ 2 files changed, 37 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/hsa/radeon/cik_regs.h b/drivers/gpu/hsa/radeon/cik_regs.h index ef1d7ab..9c3ce97 100644 --- a/drivers/gpu/hsa/radeon/cik_regs.h +++ b/drivers/gpu/hsa/radeon/cik_regs.h @@ -166,6 +166,7 @@ #define CP_HQD_DEQUEUE_REQUEST 0xC974 #define DEQUEUE_REQUEST_DRAIN 1 +#define DEQUEUE_INT (1U << 8) #define CP_HQD_SEMA_CMD 0xC97Cu #define CP_HQD_MSG_TYPE 0xC980u diff --git a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c index f86f958..5d42e88 100644 --- a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c +++ b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c @@ -139,6 +139,13 @@ struct cik_static_private { /* Queue q on pipe p is at bit QUEUES_PER_PIPE * p + q. */ unsigned long free_queues[DIV_ROUND_UP(CIK_MAX_PIPES * CIK_QUEUES_PER_PIPE, BITS_PER_LONG)]; + /* + * Dequeue waits for waves to finish so it could take a long time. We + * defer through an interrupt. dequeue_wait is woken when a dequeue- + * complete interrupt comes for that pipe. + */ + wait_queue_head_t dequeue_wait[CIK_MAX_PIPES]; + kfd_mem_obj hpd_mem; /* Single allocation for HPDs for all KFD pipes. */ kfd_mem_obj mqd_mem; /* Single allocation for all MQDs for all KFD * pipes. This is actually struct cik_mqd_padded. */ @@ -411,6 +418,9 @@ static int cik_static_create(struct kfd_dev *dev, struct kfd_scheduler **schedul priv->free_vmid_mask = dev->shared_resources.compute_vmid_bitmap; + for (i = 0; i < priv->num_pipes; i++) + init_waitqueue_head(&priv->dequeue_wait[i]); + /* * Allocate memory for the HPDs. This is hardware-owned per-pipe data. * The driver never accesses this memory after zeroing it. It doesn't even have @@ -712,15 +722,18 @@ static void activate_queue(struct cik_static_private *priv, struct cik_static_qu unlock_srbm_index(priv); } -static void drain_hqd(struct cik_static_private *priv) +static bool queue_inactive(struct cik_static_private *priv, struct cik_static_queue *queue) { - WRITE_REG(priv->dev, CP_HQD_DEQUEUE_REQUEST, DEQUEUE_REQUEST_DRAIN); -} + bool inactive; -static void wait_hqd_inactive(struct cik_static_private *priv) -{ - while (READ_REG(priv->dev, CP_HQD_ACTIVE) != 0) - cpu_relax(); + lock_srbm_index(priv); + queue_select(priv, queue->queue); + + inactive = (READ_REG(priv->dev, CP_HQD_ACTIVE) == 0); + + unlock_srbm_index(priv); + + return inactive; } static void deactivate_queue(struct cik_static_private *priv, struct cik_static_queue *queue) @@ -728,10 +741,12 @@ static void deactivate_queue(struct cik_static_private *priv, struct cik_static_ lock_srbm_index(priv); queue_select(priv, queue->queue); - drain_hqd(priv); - wait_hqd_inactive(priv); + WRITE_REG(priv->dev, CP_HQD_DEQUEUE_REQUEST, DEQUEUE_REQUEST_DRAIN | DEQUEUE_INT); unlock_srbm_index(priv); + + wait_event(priv->dequeue_wait[queue->queue/CIK_QUEUES_PER_PIPE], + queue_inactive(priv, queue)); } #define BIT_MASK_64(high, low) (((1ULL << (high)) - 1) & ~((1ULL << (low)) - 1)) @@ -791,6 +806,14 @@ cik_static_destroy_queue(struct kfd_scheduler *scheduler, struct kfd_scheduler_q release_hqd(priv, hwq->queue); } +static void +dequeue_int_received(struct cik_static_private *priv, uint32_t pipe_id) +{ + /* The waiting threads will check CP_HQD_ACTIVE to see whether their + * queue completed. */ + wake_up_all(&priv->dequeue_wait[pipe_id]); +} + /* Figure out the KFD compute pipe ID for an interrupt ring entry. * Returns true if it's a KFD compute pipe, false otherwise. */ static bool int_compute_pipe(const struct cik_static_private *priv, @@ -829,6 +852,10 @@ cik_static_interrupt_isr(struct kfd_scheduler *scheduler, const void *ih_ring_en ihre->source_id, ihre->data, pipe_id, ihre->vmid, ihre->pasid); switch (source_id) { + case CIK_INTSRC_DEQUEUE_COMPLETE: + dequeue_int_received(priv, pipe_id); + return false; /* Already handled. */ + default: return false; /* Not interested. */ }