From patchwork Thu Jul 10 21:54:03 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 4528541 Return-Path: X-Original-To: patchwork-dri-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D17FE9F37C for ; Thu, 10 Jul 2014 23:34:07 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CEC8E201D5 for ; Thu, 10 Jul 2014 23:34:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id 7999D201F2 for ; Thu, 10 Jul 2014 23:34:05 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 529396E7A7; Thu, 10 Jul 2014 16:33:24 -0700 (PDT) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wg0-f41.google.com (mail-wg0-f41.google.com [74.125.82.41]) by gabe.freedesktop.org (Postfix) with ESMTP id 031506E163 for ; Thu, 10 Jul 2014 14:55:50 -0700 (PDT) Received: by mail-wg0-f41.google.com with SMTP id z12so194530wgg.0 for ; Thu, 10 Jul 2014 14:55:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nAm2kugHyMqSfWmiFdj+ICm5l3AS/3LN4+a+vI3gZ4Q=; b=U5l+nhQmH41A7rHhf2MS48CtZiFbD28/qbiHrmtDM3A7Geop4mZWhNFOjozt0waU2c o0dB6/JIepEOXd+cImrfA1dM5DBdw0QG6+7npGPTQVbKpM0nn8F9Mys8VbxS7aujOUmU tHcrxqGBZM9Fdz0myJoFJRnOoi2IXf0EEl4f9dPsp6o/LEF/5Ny9vfdIO9AVxhN4SMS9 +FaX2OQigdYKVhA/BuVHpufVDgT0PH7rB9Quw3NvS+2/HXS0f7SveLVjdSgScaVAaxIo qyNzCNraKX1eMwve7KSHLl4MdUvUuyCNHICb0FtbL2IWZb+FXbEHCs3p57ixl57/9mpR tXpQ== X-Received: by 10.180.211.19 with SMTP id my19mr429658wic.2.1405029349399; Thu, 10 Jul 2014 14:55:49 -0700 (PDT) Received: from localhost.localdomain ([77.127.59.49]) by mx.google.com with ESMTPSA id pq9sm831097wjc.35.2014.07.10.14.55.47 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 10 Jul 2014 14:55:48 -0700 (PDT) From: Oded Gabbay X-Google-Original-From: Oded Gabbay To: David Airlie , Alex Deucher , Jerome Glisse Subject: [PATCH 47/83] hsa/radeon: Add support allocating kernel doorbells Date: Fri, 11 Jul 2014 00:54:03 +0300 Message-Id: <1405029279-6894-19-git-send-email-oded.gabbay@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com> References: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com> X-Mailman-Approved-At: Thu, 10 Jul 2014 16:33:12 -0700 Cc: Andrew Lewycky , Ben Goz , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_WEB, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ben Goz This patch adds infrastructure to allocate doorbells which are not exposed to user space. Signed-off-by: Ben Goz Signed-off-by: Oded Gabbay --- drivers/gpu/hsa/radeon/kfd_doorbell.c | 76 ++++++++++++++++++++++++++++++++++- drivers/gpu/hsa/radeon/kfd_priv.h | 5 +++ 2 files changed, 80 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/hsa/radeon/kfd_doorbell.c b/drivers/gpu/hsa/radeon/kfd_doorbell.c index 3de8a02..abf4cb0 100644 --- a/drivers/gpu/hsa/radeon/kfd_doorbell.c +++ b/drivers/gpu/hsa/radeon/kfd_doorbell.c @@ -23,6 +23,16 @@ #include "kfd_priv.h" #include #include +#include + +/* + * This extension supports a kernel level doorbells management for the kernel queues + * basically the last doorbells page is devoted to kernel queues and that's assures + * that any user process won't get access to the kernel doorbells page + */ +static DEFINE_MUTEX(doorbell_mutex); +static unsigned long doorbell_available_index[DIV_ROUND_UP(MAX_PROCESS_QUEUES, BITS_PER_LONG)] = { 0 }; +#define KERNEL_DOORBELL_PASID 1 /* * Each device exposes a doorbell aperture, a PCI MMIO aperture that @@ -67,7 +77,22 @@ void radeon_kfd_doorbell_init(struct kfd_dev *kfd) kfd->doorbell_base = kfd->shared_resources.doorbell_physical_address + doorbell_start_offset; kfd->doorbell_id_offset = doorbell_start_offset / sizeof(doorbell_t); - kfd->doorbell_process_limit = doorbell_process_limit; + kfd->doorbell_process_limit = doorbell_process_limit - 1; + + kfd->doorbell_kernel_ptr = ioremap(kfd->doorbell_base, doorbell_process_allocation()); + BUG_ON(!kfd->doorbell_kernel_ptr); + + pr_debug("kfd: doorbell initialization\n" + " doorbell base == 0x%08lX\n" + " doorbell_id_offset == 0x%08lu\n" + " doorbell_process_limit == 0x%08lu\n" + " doorbell_kernel_offset == 0x%08lX\n" + " doorbell aperture size == 0x%08lX\n" + " doorbell kernel address == 0x%08lX\n", + (uintptr_t)kfd->doorbell_base, kfd->doorbell_id_offset, doorbell_process_limit, + (uintptr_t)kfd->doorbell_base, kfd->shared_resources.doorbell_aperture_size, + (uintptr_t)kfd->doorbell_kernel_ptr); + } /* This is the /dev/kfd mmap (for doorbell) implementation. We intend that this is only called through map_doorbells, @@ -136,6 +161,53 @@ map_doorbells(struct file *devkfd, struct kfd_process *process, struct kfd_dev * return 0; } +/* get kernel iomem pointer for a doorbell */ +u32 __iomem *radeon_kfd_get_kernel_doorbell(struct kfd_dev *kfd, unsigned int *doorbell_off) +{ + u32 inx; + + BUG_ON(!kfd || !doorbell_off); + + mutex_lock(&doorbell_mutex); + inx = find_first_zero_bit(doorbell_available_index, MAX_PROCESS_QUEUES); + __set_bit(inx, doorbell_available_index); + mutex_unlock(&doorbell_mutex); + + if (inx >= MAX_PROCESS_QUEUES) + return NULL; + + /* caluculating the kernel doorbell offset using "faked" kernel pasid that allocated for kernel queues only */ + *doorbell_off = KERNEL_DOORBELL_PASID * (doorbell_process_allocation()/sizeof(doorbell_t)) + inx; + + pr_debug("kfd: get kernel queue doorbell\n" + " doorbell offset == 0x%08d\n" + " kernel address == 0x%08lX\n", + *doorbell_off, (uintptr_t)(kfd->doorbell_kernel_ptr + inx)); + + return kfd->doorbell_kernel_ptr + inx; +} + +void radeon_kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr) +{ + unsigned int inx; + + BUG_ON(!kfd || !db_addr); + + inx = (unsigned int)(db_addr - kfd->doorbell_kernel_ptr); + + mutex_lock(&doorbell_mutex); + __clear_bit(inx, doorbell_available_index); + mutex_unlock(&doorbell_mutex); +} + +inline void write_kernel_doorbell(u32 __iomem *db, u32 value) +{ + if (db) { + writel(value, db); + pr_debug("writing %d to doorbell address 0x%p\n", value, db); + } +} + /* Get the user-mode address of a doorbell. Assumes that the process mutex is being held. */ doorbell_t __user *radeon_kfd_get_doorbell(struct file *devkfd, struct kfd_process *process, struct kfd_dev *dev, unsigned int doorbell_index) @@ -152,6 +224,8 @@ doorbell_t __user *radeon_kfd_get_doorbell(struct file *devkfd, struct kfd_proce pdd = radeon_kfd_get_process_device_data(dev, process); BUG_ON(pdd == NULL); /* map_doorbells would have failed otherwise */ + pr_debug("doorbell value on creation 0x%x\n", pdd->doorbell_mapping[doorbell_index]); + return &pdd->doorbell_mapping[doorbell_index]; } diff --git a/drivers/gpu/hsa/radeon/kfd_priv.h b/drivers/gpu/hsa/radeon/kfd_priv.h index 14a3f9b..df17387 100644 --- a/drivers/gpu/hsa/radeon/kfd_priv.h +++ b/drivers/gpu/hsa/radeon/kfd_priv.h @@ -95,6 +95,7 @@ struct kfd_dev { * at the start) */ size_t doorbell_process_limit; /* Number of processes we have doorbell space for. */ + u32 __iomem *doorbell_kernel_ptr; /* this is a pointer for a doorbells page used by kernel queue */ struct kgd2kfd_shared_resources shared_resources; @@ -288,6 +289,10 @@ void radeon_kfd_doorbell_init(struct kfd_dev *kfd); int radeon_kfd_doorbell_mmap(struct kfd_process *process, struct vm_area_struct *vma); doorbell_t __user *radeon_kfd_get_doorbell(struct file *devkfd, struct kfd_process *process, struct kfd_dev *dev, unsigned int doorbell_index); +u32 __iomem *radeon_kfd_get_kernel_doorbell(struct kfd_dev *kfd, unsigned int *doorbell_off); +void radeon_kfd_release_kernel_doorbell(struct kfd_dev *kfd, u32 __iomem *db_addr); +u32 read_kernel_doorbell(u32 __iomem *db); +void write_kernel_doorbell(u32 __iomem *db, u32 value); unsigned int radeon_kfd_queue_id_to_doorbell(struct kfd_dev *kfd, struct kfd_process *process, unsigned int queue_id); extern struct device *kfd_device;