From patchwork Sat Oct 19 19:20:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 517F1D3C92E for ; Sat, 19 Oct 2024 19:20:15 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4C37A10E3B4; Sat, 19 Oct 2024 19:20:07 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="PJIejuHU"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id 374A910E3AA; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365605; x=1760901605; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T+3+mUn65i0LFR+EqTxgGf6IBD8UueUBauc02U88mmo=; b=PJIejuHUL0kj3d2Qmmx0G7HXkXIAfUy6gJT6SKw0hje4fQ2xQmuLuKpe lKxf4i5nUxvZnoUB6VJ2pzHsbhiDpoapP5uvEBr9H91xpA3wkWNiaZ2pN ooraWdZC/NiPOFTP+4uj2g3GDjZWzNSaFkiWH/jVX36dbliJzI7oZt/Lf qbCLZbUZLv/GN5HA9gb3EzQEySISObKYGoENHx8jP1f/dEXZtWXPmHpqo XA/qAcCdYBrSEShXGoLLMBLEEJMpEYxXHnI/07jBmctZLrWR1SEPNIIwq qEvnXQJdeo/JdVG1i2iLrREkb2ClnX7fhWkZ5BOs4BBqpjg+TECezFpDP Q==; X-CSE-ConnectionGUID: Y4nUAeCOTwmQpcDfxKBm2w== X-CSE-MsgGUID: YLh5rc8YQ/2+9FtPsF9KMw== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978584" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978584" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: Uw9zXQwJRrCbITFLq/lkCg== X-CSE-MsgGUID: XM5x6YgQQXGmvdzD5tPLNA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212396" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 1/6] drm/ttm: Add ttm_bo_access Date: Sat, 19 Oct 2024 12:20:25 -0700 Message-Id: <20241019192030.1505020-2-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add ttm_bo_access, which is similar to ttm_bo_vm_access, to access such memory. Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost --- drivers/gpu/drm/ttm/ttm_bo_util.c | 86 +++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_bo_vm.c | 65 +---------------------- include/drm/ttm/ttm_bo.h | 2 + 3 files changed, 89 insertions(+), 64 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index d939925efa81..084d2e9b9123 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -919,3 +919,89 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, return progress; } + +static int ttm_bo_access_kmap(struct ttm_buffer_object *bo, + unsigned long offset, + uint8_t *buf, int len, int write) +{ + unsigned long page = offset >> PAGE_SHIFT; + unsigned long bytes_left = len; + int ret; + + /* Copy a page at a time, that way no extra virtual address + * mapping is needed + */ + offset -= page << PAGE_SHIFT; + do { + unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); + struct ttm_bo_kmap_obj map; + void *ptr; + bool is_iomem; + + ret = ttm_bo_kmap(bo, page, 1, &map); + if (ret) + return ret; + + ptr = (uint8_t *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; + WARN_ON_ONCE(is_iomem); + if (write) + memcpy(ptr, buf, bytes); + else + memcpy(buf, ptr, bytes); + ttm_bo_kunmap(&map); + + page++; + buf += bytes; + bytes_left -= bytes; + offset = 0; + } while (bytes_left); + + return len; +} + +/** + * ttm_bo_access - Helper to access a buffer object + * + * @bo: ttm buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Utility function to access a buffer object. Useful when buffer object cannot + * be easily mapped (non-contiguous, non-visible, etc...). + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write) +{ + int ret; + + if (len < 1 || (offset + len) > bo->base.size) + return -EIO; + + ret = ttm_bo_reserve(bo, true, false, NULL); + if (ret) + return ret; + + switch (bo->resource->mem_type) { + case TTM_PL_SYSTEM: + fallthrough; + case TTM_PL_TT: + ret = ttm_bo_access_kmap(bo, offset, buf, len, write); + break; + default: + if (bo->bdev->funcs->access_memory) + ret = bo->bdev->funcs->access_memory( + bo, offset, buf, len, write); + else + ret = -EIO; + } + + ttm_bo_unreserve(bo); + + return ret; +} +EXPORT_SYMBOL(ttm_bo_access); diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 2c699ed1963a..20b1e5f78684 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -366,45 +366,6 @@ void ttm_bo_vm_close(struct vm_area_struct *vma) } EXPORT_SYMBOL(ttm_bo_vm_close); -static int ttm_bo_vm_access_kmap(struct ttm_buffer_object *bo, - unsigned long offset, - uint8_t *buf, int len, int write) -{ - unsigned long page = offset >> PAGE_SHIFT; - unsigned long bytes_left = len; - int ret; - - /* Copy a page at a time, that way no extra virtual address - * mapping is needed - */ - offset -= page << PAGE_SHIFT; - do { - unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); - struct ttm_bo_kmap_obj map; - void *ptr; - bool is_iomem; - - ret = ttm_bo_kmap(bo, page, 1, &map); - if (ret) - return ret; - - ptr = (uint8_t *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; - WARN_ON_ONCE(is_iomem); - if (write) - memcpy(ptr, buf, bytes); - else - memcpy(buf, ptr, bytes); - ttm_bo_kunmap(&map); - - page++; - buf += bytes; - bytes_left -= bytes; - offset = 0; - } while (bytes_left); - - return len; -} - int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write) { @@ -412,32 +373,8 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, unsigned long offset = (addr) - vma->vm_start + ((vma->vm_pgoff - drm_vma_node_start(&bo->base.vma_node)) << PAGE_SHIFT); - int ret; - - if (len < 1 || (offset + len) > bo->base.size) - return -EIO; - ret = ttm_bo_reserve(bo, true, false, NULL); - if (ret) - return ret; - - switch (bo->resource->mem_type) { - case TTM_PL_SYSTEM: - fallthrough; - case TTM_PL_TT: - ret = ttm_bo_vm_access_kmap(bo, offset, buf, len, write); - break; - default: - if (bo->bdev->funcs->access_memory) - ret = bo->bdev->funcs->access_memory( - bo, offset, buf, len, write); - else - ret = -EIO; - } - - ttm_bo_unreserve(bo); - - return ret; + return ttm_bo_access(bo, offset, buf, len, write); } EXPORT_SYMBOL(ttm_bo_vm_access); diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h index 5804408815be..8ea11cd8df39 100644 --- a/include/drm/ttm/ttm_bo.h +++ b/include/drm/ttm/ttm_bo.h @@ -421,6 +421,8 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo); int ttm_bo_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, struct ttm_operation_ctx *ctx); +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write); vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, struct vm_fault *vmf); vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, From patchwork Sat Oct 19 19:20:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1ECEAD3C92F for ; Sat, 19 Oct 2024 19:20:16 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E97EB10E3B8; Sat, 19 Oct 2024 19:20:07 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="Rk4fmGm6"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6EF7310E3AD; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365605; x=1760901605; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A+Hjs8IdGMGIVSvHMXH+Km4QNx9MIv6KAIxXzDhxuKI=; b=Rk4fmGm6AcC8JlZqJaGq1AU2PRudmWyUWxqLbHd3rj35HAIYDtJTP9c7 Ln81Oq8uNZpndABLydu/dCCcJUkmp/EPv+VvbIZEKfEpD3p6iCgZsY1gi OA5lfQUqkKP2he8vlwza1/YAoyj2FJt3zI/Vo0+c7AMhYnT3QLb76lPCG 5qJ6b8rvfzIch1rqIozmnaDCptx6O9XK1XLISyKcLUq0B2gUFUIPvbUv0 tvbWlqqb0+bYSiYr7lelIvUg+JKsuc8LYDpQzjfm/TFud5gzZss49hgvT C9RyXEM1r+/cYgzNKNbE/i2ezhNqA7e8Oj17PN33K2KDRn4xa779UUJhn g==; X-CSE-ConnectionGUID: BRYJ6YIvTRmiOlmj9cIEjQ== X-CSE-MsgGUID: o7T4Yu/DQVaJLZ/AVSWu7A== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978585" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978585" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: 5TV14+4nTOGyaXsN+efGwA== X-CSE-MsgGUID: zCVCXmAtSUGzF8SoDFzQDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212399" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 2/6] drm/xe: Add xe_ttm_access_memory Date: Sat, 19 Oct 2024 12:20:26 -0700 Message-Id: <20241019192030.1505020-3-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add xe_ttm_access_memory which hooks into ttm_bo_access to access such memory. Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 62 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 5b232f2951b1..99557af16120 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -442,6 +442,14 @@ static void xe_ttm_tt_destroy(struct ttm_device *ttm_dev, struct ttm_tt *tt) kfree(tt); } +static bool xe_ttm_resource_visible(struct ttm_resource *mem) +{ + struct xe_ttm_vram_mgr_resource *vres = + to_xe_ttm_vram_mgr_resource(mem); + + return vres->used_visible_size == mem->size; +} + static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem) { @@ -453,11 +461,9 @@ static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, return 0; case XE_PL_VRAM0: case XE_PL_VRAM1: { - struct xe_ttm_vram_mgr_resource *vres = - to_xe_ttm_vram_mgr_resource(mem); struct xe_mem_region *vram = res_to_mem_region(mem); - if (vres->used_visible_size < mem->size) + if (!xe_ttm_resource_visible(mem)) return -EINVAL; mem->bus.offset = mem->start << PAGE_SHIFT; @@ -1111,6 +1117,55 @@ static void xe_ttm_bo_swap_notify(struct ttm_buffer_object *ttm_bo) } } +static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_bo *bo = ttm_to_xe_bo(ttm_bo); + struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev); + struct iosys_map vmap; + struct xe_res_cursor cursor; + struct xe_mem_region *vram; + int bytes_left = len; + + xe_bo_assert_held(bo); + + if (!mem_type_is_vram(ttm_bo->resource->mem_type)) + return -EIO; + + /* FIXME: Use GPU for non-visible VRAM */ + if (!xe_ttm_resource_visible(ttm_bo->resource)) + return -EIO; + + if (!xe_pm_runtime_get_if_in_use(xe)) + return -EIO; + + vram = res_to_mem_region(ttm_bo->resource); + xe_res_first(ttm_bo->resource, offset & PAGE_MASK, bo->size, &cursor); + + do { + unsigned long page_offset = (offset & ~PAGE_MASK); + int byte_count = min((int)(PAGE_SIZE - page_offset), bytes_left); + + iosys_map_set_vaddr_iomem(&vmap, (u8 __iomem *)vram->mapping + + cursor.start); + if (write) + xe_map_memcpy_to(xe, &vmap, page_offset, buf, byte_count); + else + xe_map_memcpy_from(xe, buf, &vmap, page_offset, byte_count); + + offset += byte_count; + buf += byte_count; + bytes_left -= byte_count; + if (bytes_left) + xe_res_next(&cursor, PAGE_SIZE); + } while (bytes_left); + + xe_pm_runtime_put(xe); + + return len; +} + const struct ttm_device_funcs xe_ttm_funcs = { .ttm_tt_create = xe_ttm_tt_create, .ttm_tt_populate = xe_ttm_tt_populate, @@ -1120,6 +1175,7 @@ const struct ttm_device_funcs xe_ttm_funcs = { .move = xe_bo_move, .io_mem_reserve = xe_ttm_io_mem_reserve, .io_mem_pfn = xe_ttm_io_mem_pfn, + .access_memory = xe_ttm_access_memory, .release_notify = xe_ttm_bo_release_notify, .eviction_valuable = ttm_bo_eviction_valuable, .delete_mem_notify = xe_ttm_bo_delete_mem_notify, From patchwork Sat Oct 19 19:20:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03E54D3C92D for ; Sat, 19 Oct 2024 19:20:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5B85310E3AC; Sat, 19 Oct 2024 19:20:06 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="LQCpDhA5"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7E53610E3AE; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365605; x=1760901605; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gbqPgb+EokkiVFLqP3w67S0+9aTdCVnbXtN+UEPgAJE=; b=LQCpDhA5/VxNM4IYI/SbP0ygY0218R1LvXdCDgJJiPT1qDW3RU1Qn7mC Xa/lyDatcN/zqTzgai+IcslipDefj6DJeJLnLikU7gWtEfLZMjg+eXye8 L+1Q6vQiIBBR06LRKzLICbCiQpW1JCiRV7TUftq5boG8RuXsTvUiAHYrq n0M8lZurwF1CophVkgxOq3wv58vaBVBUxJReT9LaKc7xkaKQH3mICTqoA scqcI42nWfh8R5mN+ZvEXtkX3M5ME+7eWVYVaAPaXgUJ2NotMtN8RZc+s MK5c2SHcvqKtNbliZjpFWC09vHfY9QLsIpqA+HFqq4mw0WV9C0vI0WsVR g==; X-CSE-ConnectionGUID: FiAmBt1NQnu4owxJSF/I6A== X-CSE-MsgGUID: oPZsPRd8SgOcGvV01oit8A== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978586" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978586" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: 9J7VkPYFRZWur+aHGlz3PQ== X-CSE-MsgGUID: thYmr1U2RZq2BONoG9rySQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212402" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 3/6] drm/xe: Update xe_ttm_access_memory to use GPU for non-visible access Date: Sat, 19 Oct 2024 12:20:27 -0700 Message-Id: <20241019192030.1505020-4-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add migrate layer functions to access VRAM and update xe_ttm_access_memory to use for non-visible access. Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 18 ++- drivers/gpu/drm/xe/xe_migrate.c | 267 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_migrate.h | 4 + 3 files changed, 285 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 99557af16120..0a7b91df69c2 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1133,13 +1133,22 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, if (!mem_type_is_vram(ttm_bo->resource->mem_type)) return -EIO; - /* FIXME: Use GPU for non-visible VRAM */ - if (!xe_ttm_resource_visible(ttm_bo->resource)) - return -EIO; - if (!xe_pm_runtime_get_if_in_use(xe)) return -EIO; + if (!xe_ttm_resource_visible(ttm_bo->resource) || len >= SZ_16K) { + struct xe_migrate *migrate = + mem_type_to_migrate(xe, ttm_bo->resource->mem_type); + int err; + + err = xe_migrate_access_memory(migrate, bo, offset, buf, len, + write); + if (err) + return err; + + goto out; + } + vram = res_to_mem_region(ttm_bo->resource); xe_res_first(ttm_bo->resource, offset & PAGE_MASK, bo->size, &cursor); @@ -1161,6 +1170,7 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, xe_res_next(&cursor, PAGE_SIZE); } while (bytes_left); +out: xe_pm_runtime_put(xe); return len; diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index cfd31ae49cc1..ccdca1f79bb2 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -1542,6 +1542,273 @@ void xe_migrate_wait(struct xe_migrate *m) dma_fence_wait(m->fence, false); } +static u32 pte_update_cmd_size(u64 size) +{ + u32 dword; + u64 entries = DIV_ROUND_UP(size, XE_PAGE_SIZE); + + XE_WARN_ON(size > MAX_PREEMPTDISABLE_TRANSFER); + /* + * MI_STORE_DATA_IMM command is used to update page table. Each + * instruction can update maximumly 0x1ff pte entries. To update + * n (n <= 0x1ff) pte entries, we need: + * 1 dword for the MI_STORE_DATA_IMM command header (opcode etc) + * 2 dword for the page table's physical location + * 2*n dword for value of pte to fill (each pte entry is 2 dwords) + */ + dword = (1 + 2) * DIV_ROUND_UP(entries, 0x1ff); + dword += entries * 2; + + return dword; +} + +static void build_pt_update_batch_sram(struct xe_migrate *m, + struct xe_bb *bb, u32 pt_offset, + dma_addr_t *sram_addr, u32 size) +{ + u16 pat_index = tile_to_xe(m->tile)->pat.idx[XE_CACHE_WB]; + u32 ptes; + int i = 0; + + ptes = DIV_ROUND_UP(size, XE_PAGE_SIZE); + while (ptes) { + u32 chunk = min(0x1ffU, ptes); + + bb->cs[bb->len++] = MI_STORE_DATA_IMM | MI_SDI_NUM_QW(chunk); + bb->cs[bb->len++] = pt_offset; + bb->cs[bb->len++] = 0; + + pt_offset += chunk * 8; + ptes -= chunk; + + while (chunk--) { + u64 addr = sram_addr[i++] & PAGE_MASK; + + xe_tile_assert(m->tile, addr); + addr = m->q->vm->pt_ops->pte_encode_addr(m->tile->xe, + addr, pat_index, + 0, false, 0); + bb->cs[bb->len++] = lower_32_bits(addr); + bb->cs[bb->len++] = upper_32_bits(addr); + } + } +} + +enum xe_migrate_copy_dir { + XE_MIGRATE_COPY_TO_VRAM, + XE_MIGRATE_COPY_TO_SRAM, +}; + +static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, + unsigned long len, + unsigned long sram_offset, + dma_addr_t *sram_addr, u64 vram_addr, + const enum xe_migrate_copy_dir dir) +{ + struct xe_gt *gt = m->tile->primary_gt; + struct xe_device *xe = gt_to_xe(gt); + struct dma_fence *fence = NULL; + u32 batch_size = 2; + u64 src_L0_ofs, dst_L0_ofs; + struct xe_sched_job *job; + struct xe_bb *bb; + u32 update_idx, pt_slot = 0; + unsigned long npages = DIV_ROUND_UP(len + sram_offset, PAGE_SIZE); + int err; + + xe_assert(xe, npages * PAGE_SIZE <= MAX_PREEMPTDISABLE_TRANSFER); + + batch_size += pte_update_cmd_size(len); + batch_size += EMIT_COPY_DW; + + bb = xe_bb_new(gt, batch_size, true); + if (IS_ERR(bb)) { + err = PTR_ERR(bb); + return ERR_PTR(err); + } + + build_pt_update_batch_sram(m, bb, pt_slot * XE_PAGE_SIZE, + sram_addr, len + sram_offset); + + if (dir == XE_MIGRATE_COPY_TO_VRAM) { + src_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + dst_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + + } else { + src_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + dst_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + } + + bb->cs[bb->len++] = MI_BATCH_BUFFER_END; + update_idx = bb->len; + + emit_copy(gt, bb, src_L0_ofs, dst_L0_ofs, len, XE_PAGE_SIZE); + + job = xe_bb_create_migration_job(m->q, bb, + xe_migrate_batch_base(m, true), + update_idx); + if (IS_ERR(job)) { + err = PTR_ERR(job); + goto err; + } + + xe_sched_job_add_migrate_flush(job, 0); + + mutex_lock(&m->job_mutex); + xe_sched_job_arm(job); + fence = dma_fence_get(&job->drm.s_fence->finished); + xe_sched_job_push(job); + + dma_fence_put(m->fence); + m->fence = dma_fence_get(fence); + mutex_unlock(&m->job_mutex); + + xe_bb_free(bb, fence); + + return fence; + +err: + xe_bb_free(bb, NULL); + + return ERR_PTR(err); +} + +static void xe_migrate_dma_unmap(struct xe_device *xe, dma_addr_t *dma_addr, + int len, int write) +{ + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + for (i = 0; i < npages; ++i) { + if (!dma_addr[i]) + continue; + + dma_unmap_page(xe->drm.dev, dma_addr[i], PAGE_SIZE, + write ? DMA_TO_DEVICE : DMA_FROM_DEVICE); + } + kfree(dma_addr); +} + +static dma_addr_t *xe_migrate_dma_map(struct xe_device *xe, + void *buf, int len, int write) +{ + dma_addr_t *dma_addr; + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + dma_addr = kcalloc(npages, sizeof(*dma_addr), GFP_KERNEL); + if (!dma_addr) + return ERR_PTR(-ENOMEM); + + for (i = 0; i < npages; ++i) { + dma_addr_t addr; + struct page *page; + + if (is_vmalloc_addr(buf)) + page = vmalloc_to_page(buf); + else + page = virt_to_page(buf); + + addr = dma_map_page(xe->drm.dev, + page, 0, PAGE_SIZE, + write ? DMA_TO_DEVICE : + DMA_FROM_DEVICE); + if (dma_mapping_error(xe->drm.dev, addr)) + goto err_fault; + + dma_addr[i] = addr; + buf += PAGE_SIZE; + } + + return dma_addr; + +err_fault: + xe_migrate_dma_unmap(xe, dma_addr, len, write); + return ERR_PTR(-EFAULT); +} + +/** + * xe_migrate_access_memory - Access memory of a BO via GPU + * + * @m: The migration context. + * @bo: buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Access memory of a BO via GPU either reading in or writing from a passed in + * pointer. Pointer is dma mapped for GPU access and GPU commands are issued to + * read to or write from pointer. + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_tile *tile = m->tile; + struct xe_device *xe = tile_to_xe(tile); + struct xe_res_cursor cursor; + struct dma_fence *fence = NULL; + dma_addr_t *dma_addr; + unsigned long page_offset = (unsigned long)buf & ~PAGE_MASK; + int bytes_left = len, current_page = 0; + + xe_bo_assert_held(bo); + + dma_addr = xe_migrate_dma_map(xe, buf, len + page_offset, write); + if (IS_ERR(dma_addr)) + return PTR_ERR(dma_addr); + + xe_res_first(bo->ttm.resource, offset, bo->size, &cursor); + + do { + struct dma_fence *__fence; + u64 vram_addr = vram_region_gpu_offset(bo->ttm.resource) + + cursor.start; + int current_bytes; + + if (cursor.size > MAX_PREEMPTDISABLE_TRANSFER) + current_bytes = min_t(int, bytes_left, + MAX_PREEMPTDISABLE_TRANSFER); + else + current_bytes = min_t(int, bytes_left, cursor.size); + + if (fence) + dma_fence_put(fence); + + __fence = xe_migrate_vram(m, current_bytes, + (unsigned long)buf & ~PAGE_MASK, + dma_addr + current_page, + vram_addr, write ? + XE_MIGRATE_COPY_TO_VRAM : + XE_MIGRATE_COPY_TO_SRAM); + if (IS_ERR(__fence)) { + if (fence) + dma_fence_wait(fence, false); + fence = __fence; + goto out_err; + } + fence = __fence; + + buf += current_bytes; + offset += current_bytes; + current_page += DIV_ROUND_UP(current_bytes, PAGE_SIZE); + bytes_left -= current_bytes; + if (bytes_left) + xe_res_next(&cursor, current_bytes); + } while (bytes_left); + + dma_fence_wait(fence, false); + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + + return 0; + +out_err: + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + return PTR_ERR(fence); +} + #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) #include "tests/xe_migrate.c" #endif diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h index 0109866e398a..94197d262178 100644 --- a/drivers/gpu/drm/xe/xe_migrate.h +++ b/drivers/gpu/drm/xe/xe_migrate.h @@ -102,6 +102,10 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, struct ttm_resource *dst, bool copy_only_ccs); +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write); + #define XE_MIGRATE_CLEAR_FLAG_BO_DATA BIT(0) #define XE_MIGRATE_CLEAR_FLAG_CCS_DATA BIT(1) #define XE_MIGRATE_CLEAR_FLAG_FULL (XE_MIGRATE_CLEAR_FLAG_BO_DATA | \ From patchwork Sat Oct 19 19:20:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D08A4D3C92C for ; Sat, 19 Oct 2024 19:20:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E514D10E3B2; Sat, 19 Oct 2024 19:20:06 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="kStHKP0l"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9AAF410E3AA; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365605; x=1760901605; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rgdWjLCAsWDICAk8WcCYB+yo/wtl3ra7r8m7MBpFXLE=; b=kStHKP0lo4tjjhhLqEaa7YX3JJVxft9UlgQRFgzMAonnO8nZde1Hjk39 UFngqtZT5JH4uEB9kAsSxBxSH6IidSmy1P1DLYlZZHUClk8BFz7Dr/ovN OTdyw4aoImabbYeenrTn7OoQC2YeyXBFzCiZOYIjQBTAsUxlUWClOBnCy aba68r+bBDWnKO66uGLIs3pIlgnIVU22R+78C9LViMjeNz7sdmzR+dcaE lXVjxwawH/EsF1a1jzoMALnip7/hDxBL3McPhBmrHPn6988d1aair5dzL hZ1DVaxIjwOcnGaJtpi1VM42F41wRuICr7rUgKecinnm6LxyAQEqC9L4A A==; X-CSE-ConnectionGUID: 6MYElfkCTxmFzNZukYBetw== X-CSE-MsgGUID: 5874iQkJTraU+WZCF2D4Jw== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978587" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978587" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: lwzLkfViTEG/snsVS7navQ== X-CSE-MsgGUID: w0fpAjuZQ3KZ+D1DuFEffw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212406" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:06 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 4/6] drm/xe: Use ttm_bo_access in xe_vm_snapshot_capture_delayed Date: Sat, 19 Oct 2024 12:20:28 -0700 Message-Id: <20241019192030.1505020-5-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous mapping of BO in VRAM doesn't work, use ttm_bo_access instead. v2: - Fix error handling Fixes: 0eb2a18a8fad ("drm/xe: Implement VM snapshot support for BO's and userptr") Suggested-by: Matthew Auld Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_vm.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index c99380271de6..c8782da3a5c3 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -3303,7 +3303,6 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) for (int i = 0; i < snap->num_snaps; i++) { struct xe_bo *bo = snap->snap[i].bo; - struct iosys_map src; int err; if (IS_ERR(snap->snap[i].data)) @@ -3316,16 +3315,12 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) } if (bo) { - xe_bo_lock(bo, false); - err = ttm_bo_vmap(&bo->ttm, &src); - if (!err) { - xe_map_memcpy_from(xe_bo_device(bo), - snap->snap[i].data, - &src, snap->snap[i].bo_ofs, - snap->snap[i].len); - ttm_bo_vunmap(&bo->ttm, &src); - } - xe_bo_unlock(bo); + err = ttm_bo_access(&bo->ttm, snap->snap[i].bo_ofs, + snap->snap[i].data, snap->snap[i].len, 0); + if (!(err < 0) && err != snap->snap[i].len) + err = -EIO; + else if (!(err < 0)) + err = 0; } else { void __user *userptr = (void __user *)(size_t)snap->snap[i].bo_ofs; From patchwork Sat Oct 19 19:20:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BA21D3C92D for ; Sat, 19 Oct 2024 19:20:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3E13F10E3A8; Sat, 19 Oct 2024 19:20:07 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="aDQdpNvk"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id B645610E3AC; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365605; x=1760901605; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hSbQ9RTe5ZXX2fn3WclbmFPhn+u7StIBRgeeFTsYYZ4=; b=aDQdpNvk+gBuRqIPi5KSDlsTWSFehu3abO/JtY607zadI7UVPVjbWhEP 1Yj1vRa4HsK95ZcH+0t5EPlqy4kl+Gmpb/FRRsKMsJ7YPgX9kdm95tfnT S6+ZTsCqeGCJax7GnOzCRFRXxmGV18Ha73g1HSNvpgwZGWS1Ig0t/05DS 22YwhPi8XsiQ4h89bcPCIerRqS+ZIweFSXz3H5eZM6kh+8D2DLF/P1mF1 64/WNmZ+n0O1gEWZMbszNEg8Dq+R6aCT58WFSFtsZHqzjGfFXzelSmNWD C2c756+woH4mos3gN/R8lf78kj/PwTHncG2uGPZT7oUcP5yfaAXS7jvo8 Q==; X-CSE-ConnectionGUID: lqCyDG5iTM6PJxycZv89/Q== X-CSE-MsgGUID: imxNUCnISp+ybEJWjaMYfw== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978588" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978588" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: h6hN4qu9S7eUl1SbQ63rYQ== X-CSE-MsgGUID: s6NmQkOlSPaNU4mt+z2r3w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212408" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:06 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 5/6] drm/xe: Set XE_BO_FLAG_PINNED in migrate selftest BOs Date: Sat, 19 Oct 2024 12:20:29 -0700 Message-Id: <20241019192030.1505020-6-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We only allow continguous BOs to be vmapped, set XE_BO_FLAG_PINNED on BOs in migrate selftest as this forces continguous BOs and selftest uses vmaps. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/tests/xe_migrate.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/tests/xe_migrate.c b/drivers/gpu/drm/xe/tests/xe_migrate.c index 1a192a2a941b..4cef3b20bd17 100644 --- a/drivers/gpu/drm/xe/tests/xe_migrate.c +++ b/drivers/gpu/drm/xe/tests/xe_migrate.c @@ -83,7 +83,8 @@ static void test_copy(struct xe_migrate *m, struct xe_bo *bo, bo->size, ttm_bo_type_kernel, region | - XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(remote)) { KUNIT_FAIL(test, "Failed to allocate remote bo for %s: %pe\n", str, remote); @@ -642,7 +643,9 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til sys_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_SYSTEM | + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(sys_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -666,7 +669,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til ccs_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(ccs_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -690,7 +694,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til vram_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(vram_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", PTR_ERR(vram_bo)); From patchwork Sat Oct 19 19:20:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13842868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3757AD3C92D for ; Sat, 19 Oct 2024 19:20:10 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BEC4710E3B0; Sat, 19 Oct 2024 19:20:06 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="MI/DWn7y"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) by gabe.freedesktop.org (Postfix) with ESMTPS id CCEEB10E3AA; Sat, 19 Oct 2024 19:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729365606; x=1760901606; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LKihkz9CqYFI8n3HKEXZ5Df5dsNqpssKiNVE+2hzODI=; b=MI/DWn7yOmzrxD1g0OqM0g1ehPMnoXkpdsxWuIpcco+5/2fnLY+O+7Gr 4P+FhnZtURoR9PkzoUp+rVgiU2+riE8zZrzrIfYywJqfSbBM9wXSXnhw5 2jOjwr/cR1rzgDsFHyQvdUmGzLUd53bUXkQvRKWsMAmBGd2iiFEnArDxN bzV1mGxhABjr3KyUA6pcg9RyLfmEnNBVaNVZ4T8go5JdqA4UkFEs6nLtI JhgEbnzoEhrD2bP7kpw3w628mar8VgTDBKGy6h2dOa9c6Xg5BG9Uduu+2 9VdDTyoh0RlXqknDqfaFYM1QuLIlaXo8I9Bhs/ZfPB86GOl9jL0iONJqJ g==; X-CSE-ConnectionGUID: MJjq3yr3Rd+t7Rko1q0CuQ== X-CSE-MsgGUID: WOPMhPvdQR+1qk4Ns+f65g== X-IronPort-AV: E=McAfee;i="6700,10204,11230"; a="28978589" X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="28978589" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:05 -0700 X-CSE-ConnectionGUID: PKB6ffHaS/2dls73HvxlXA== X-CSE-MsgGUID: BMxxILcZQWOm1yiUkpkiTg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,217,1725346800"; d="scan'208";a="84212411" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2024 12:20:06 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v3 6/6] drm/xe: Only allow contiguous BOs to use xe_bo_vmap Date: Sat, 19 Oct 2024 12:20:30 -0700 Message-Id: <20241019192030.1505020-7-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241019192030.1505020-1-matthew.brost@intel.com> References: <20241019192030.1505020-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" xe_bo_vmap only works on contiguous BOs, disallow xe_bo_vmap on BO unless we are certain the BO is contiguous. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_bo.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 0a7b91df69c2..46c640f8db9e 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -162,6 +162,15 @@ static void try_add_system(struct xe_device *xe, struct xe_bo *bo, } } +static bool force_contiguous(u32 bo_flags) +{ + /* + * For eviction / restore on suspend / resume objects pinned in VRAM + * must be contiguous, also only contiguous BOs support xe_bo_vmap. + */ + return bo_flags & (XE_BO_FLAG_PINNED | XE_BO_FLAG_GGTT); +} + static void add_vram(struct xe_device *xe, struct xe_bo *bo, struct ttm_place *places, u32 bo_flags, u32 mem_type, u32 *c) { @@ -175,12 +184,7 @@ static void add_vram(struct xe_device *xe, struct xe_bo *bo, xe_assert(xe, vram && vram->usable_size); io_size = vram->io_size; - /* - * For eviction / restore on suspend / resume objects - * pinned in VRAM must be contiguous - */ - if (bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT)) + if (force_contiguous(bo_flags)) place.flags |= TTM_PL_FLAG_CONTIGUOUS; if (io_size < vram->usable_size) { @@ -212,8 +216,7 @@ static void try_add_stolen(struct xe_device *xe, struct xe_bo *bo, bo->placements[*c] = (struct ttm_place) { .mem_type = XE_PL_STOLEN, - .flags = bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT) ? + .flags = force_contiguous(bo_flags) ? TTM_PL_FLAG_CONTIGUOUS : 0, }; *c += 1; @@ -2024,13 +2027,15 @@ dma_addr_t xe_bo_addr(struct xe_bo *bo, u64 offset, size_t page_size) int xe_bo_vmap(struct xe_bo *bo) { + struct xe_device *xe = ttm_to_xe_device(bo->ttm.bdev); void *virtual; bool is_iomem; int ret; xe_bo_assert_held(bo); - if (!(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS)) + if (drm_WARN_ON(&xe->drm, !(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS) || + !force_contiguous(bo->flags))) return -EINVAL; if (!iosys_map_is_null(&bo->vmap))