From patchwork Sat Jun 6 18:30:41 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 6559881 Return-Path: X-Original-To: patchwork-dri-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id D6157C0020 for ; Sat, 6 Jun 2015 18:30:54 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B208620624 for ; Sat, 6 Jun 2015 18:30:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id 91A2420623 for ; Sat, 6 Jun 2015 18:30:52 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 30DE689230; Sat, 6 Jun 2015 11:30:50 -0700 (PDT) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wg0-f51.google.com (mail-wg0-f51.google.com [74.125.82.51]) by gabe.freedesktop.org (Postfix) with ESMTP id 4711C89230 for ; Sat, 6 Jun 2015 11:30:49 -0700 (PDT) Received: by wgez8 with SMTP id z8so76590713wge.0 for ; Sat, 06 Jun 2015 11:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=dQsf9+z97UqWdVTbmTHPDUAePJdy8pr+wy+D5kMKlq8=; b=tnlr9FCN4cGjf0NhymD8ugDgG+y4IQ1mSnDtZ+TDxFgWdw9rRscCBnpYTwCa+qM8Vs Hy53jjMOClcGVrwUWb1rATw+R97cbPBzGvH/vaMoBQTzy/lDFN4/gzj1U/9PMqIwz2Xs IeiNr/nFFiYR6Sez04ReVcO//3ZiOqNpNIdoGPe1e8KZFpVewQtbbpuL4jFghTZZZCvB N87vW0tULeU+mn5WHOgmt3CvCE1zG/1ZhmrD+x3oII3Gp8usaRGiRAIaZoelF2RFAZcP /BSPVTSUljf8Jnqr55qk4bud95zlOHyDe+35UflWU/UVAWZkTjs/RyxJSVO+jSx7oA8I m3qg== X-Received: by 10.180.73.175 with SMTP id m15mr7496174wiv.38.1433615448187; Sat, 06 Jun 2015 11:30:48 -0700 (PDT) Received: from odedg-home.localdomain ([77.125.85.14]) by mx.google.com with ESMTPSA id h1sm3575380wiz.6.2015.06.06.11.30.46 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 06 Jun 2015 11:30:47 -0700 (PDT) From: Oded Gabbay To: dri-devel@lists.freedesktop.org Subject: [PATCH] drm/amdkfd: do reset wavefronts per process per device Date: Sat, 6 Jun 2015 21:30:41 +0300 Message-Id: <1433615441-16223-1-git-send-email-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.4.2 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ben Goz This commit moves the reset wavefront flag to per process per device data structure, so we can support multiple devices. Signed-off-by: Ben Goz Signed-off-by: Oded Gabbay --- .../gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 7 ++++--- drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 10 +++++----- drivers/gpu/drm/amd/amdkfd/kfd_process.c | 21 ++++++++++++++++----- 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c index 547b0a5..4bb7f42 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c @@ -946,7 +946,7 @@ static int destroy_queues_cpsch(struct device_queue_manager *dqm, { int retval; enum kfd_preempt_type_filter preempt_type; - struct kfd_process *p; + struct kfd_process_device *pdd; BUG_ON(!dqm); @@ -981,8 +981,9 @@ static int destroy_queues_cpsch(struct device_queue_manager *dqm, retval = amdkfd_fence_wait_timeout(dqm->fence_addr, KFD_FENCE_COMPLETED, QUEUE_PREEMPT_DEFAULT_TIMEOUT_MS); if (retval != 0) { - p = kfd_get_process(current); - p->reset_wavefronts = true; + pdd = kfd_get_process_device_data(dqm->dev, + kfd_get_process(current)); + pdd->reset_wavefronts = true; goto out; } pm_release_ib(&dqm->packets); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index cb79046..d0d5f4b 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -463,6 +463,11 @@ struct kfd_process_device { /* Is this process/pasid bound to this device? (amd_iommu_bind_pasid) */ bool bound; + + /* This flag tells if we should reset all + * wavefronts on process termination + */ + bool reset_wavefronts; }; #define qpd_to_pdd(x) container_of(x, struct kfd_process_device, qpd) @@ -519,11 +524,6 @@ struct kfd_process { event_pages */ u32 next_nonsignal_event_id; size_t signal_event_count; - /* - * This flag tells if we should reset all wavefronts on - * process termination - */ - bool reset_wavefronts; }; /** diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c index 56b904f..8a1f999 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c @@ -173,7 +173,7 @@ static void kfd_process_wq_release(struct work_struct *work) pr_debug("Releasing pdd (topology id %d) for process (pasid %d) in workqueue\n", pdd->dev->id, p->pasid); - if (p->reset_wavefronts) + if (pdd->reset_wavefronts) dbgdev_wave_reset_wavefronts(pdd->dev, p); amd_iommu_unbind_pasid(pdd->dev->pdev, p->pasid); @@ -222,6 +222,7 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn, struct mm_struct *mm) { struct kfd_process *p; + struct kfd_process_device *pdd = NULL; /* * The kfd_process structure can not be free because the @@ -240,6 +241,15 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn, /* In case our notifier is called before IOMMU notifier */ pqm_uninit(&p->pqm); + /* Iterate over all process device data structure and check + * if we should reset all wavefronts */ + list_for_each_entry(pdd, &p->per_device_data, per_device_list) + if (pdd->reset_wavefronts) { + pr_warn("amdkfd: Resetting all wave fronts\n"); + dbgdev_wave_reset_wavefronts(pdd->dev, p); + pdd->reset_wavefronts = false; + } + mutex_unlock(&p->mutex); /* @@ -305,8 +315,6 @@ static struct kfd_process *create_process(const struct task_struct *thread) if (kfd_init_apertures(process) != 0) goto err_init_apretures; - process->reset_wavefronts = false; - return process; err_init_apretures: @@ -348,6 +356,7 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev, INIT_LIST_HEAD(&pdd->qpd.queues_list); INIT_LIST_HEAD(&pdd->qpd.priv_queue_list); pdd->qpd.dqm = dev->dqm; + pdd->reset_wavefronts = false; list_add(&pdd->per_device_list, &p->per_device_data); } @@ -409,10 +418,12 @@ void kfd_unbind_process_from_device(struct kfd_dev *dev, unsigned int pasid) kfd_dbgmgr_destroy(dev->dbgmgr); pqm_uninit(&p->pqm); - if (p->reset_wavefronts) - dbgdev_wave_reset_wavefronts(dev, p); pdd = kfd_get_process_device_data(dev, p); + if (pdd->reset_wavefronts) { + dbgdev_wave_reset_wavefronts(pdd->dev, p); + pdd->reset_wavefronts = false; + } /* * Just mark pdd as unbound, because we still need it to call