From patchwork Mon Oct 21 20:15:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 169F9D17133 for ; Mon, 21 Oct 2024 20:14:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 507C610E59B; Mon, 21 Oct 2024 20:14:47 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="jqPOSecI"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2E9C010E037; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541684; x=1761077684; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IF3szAzYLAmUMg1AdI9+r6nlyZGZlqpJGjOyeoigxZo=; b=jqPOSecI43d1+ByOkO+D9LNDzEsFzaDjIphE59gLg5bgkGxpRAiY97MQ lUMZNRrspLByHxxM7bKEMMlW+kPbTIOoVwfY6r/CldFWmMeYTZziVwIUW NrQCQrWOKtlJg2tf31cSeLMxmLcuGVF/US8ubRNmcPFpiuMUwaLY/ZC0S OjF62uHF5MTpbnuZJ5zW5Kl0XREJjm+ZGSqPpxWPogOkRaJYP4KF6c6sj DcCxbMwVpCM4/2k0JDOlKNqcuw4pUMSJu5CojC7o4vNwlMAN8aQ2trKUn kWEf12YUiWTO3yLAgONws3Ytove0A+aRvvAP2EYHrSbgbExUKkNLxMIBf w==; X-CSE-ConnectionGUID: tF8IsQvcQoKGNf4sjhNSyw== X-CSE-MsgGUID: EDOLCQMnR4i7GRTo2m8qyw== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167478" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167478" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: sbEwa0sHR8esi8OC+KWngA== X-CSE-MsgGUID: jGuFNRYrTJeGZ/ZcdRqGCg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472038" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:43 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 1/9] drm/xe: Add xe_bo_vm_access Date: Mon, 21 Oct 2024 13:15:01 -0700 Message-Id: <20241021201509.1668074-2-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add xe_bo_vm_access which is wrapper around ttm_bo_vm_access which takes rpm refs for device access. Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 5b232f2951b1..0261a8b29351 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1236,11 +1236,26 @@ static vm_fault_t xe_gem_fault(struct vm_fault *vmf) return ret; } +static int xe_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, + void *buf, int len, int write) +{ + struct ttm_buffer_object *ttm_bo = vma->vm_private_data; + struct xe_bo *bo = ttm_to_xe_bo(ttm_bo); + struct xe_device *xe = xe_bo_device(bo); + int ret; + + xe_pm_runtime_get(xe); + ret = ttm_bo_vm_access(vma, addr, buf, len, write); + xe_pm_runtime_put(xe); + + return ret; +} + static const struct vm_operations_struct xe_gem_vm_ops = { .fault = xe_gem_fault, .open = ttm_bo_vm_open, .close = ttm_bo_vm_close, - .access = ttm_bo_vm_access + .access = xe_bo_vm_access, }; static const struct drm_gem_object_funcs xe_gem_object_funcs = { From patchwork Mon Oct 21 20:15:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1621D17132 for ; Mon, 21 Oct 2024 20:14:45 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0139210E592; Mon, 21 Oct 2024 20:14:45 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="B7di5Wp8"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id 54FF810E58F; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541684; x=1761077684; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3AU1zRIG9DZRiU/Zn6ZOVBUtqJM6HJATPXMDbkynCIc=; b=B7di5Wp8v6gtJvp9oDUXCjCr7PWeAR3wIRWn3pXu29J0Oe3KP/iA0kzo 6jwUTdISZnE0z0ufB1qkdRRpjaC6IsP2HIKBYifXIo40aL3VoFOPGEl7F 2X2DsV2Y7Wu1JZhYhvS8LVNhYLQZHMMIaeGnH3yFJOvrCAWevMiOAcJYf j9kIi6jdEUJFpsqBkqbnEju5Zfq74kDIbzuTF4FPMYbpfJ3SEEvQQ3bpQ xZHzWuA1ODx+Powc2Ih8nifpR7aJ7hxGyGixHbi1fct4p6ZFJFHE6qgR3 NY4vTuYemk2ntEg6zLsS52+PBw3w4SwAxCtS1jU7Qfhv7xfuakyFhC43e A==; X-CSE-ConnectionGUID: TRzDIhxGSD6YvdUSUnUCVQ== X-CSE-MsgGUID: UsBeXssDTwCSRiDdSvDbaw== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167479" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167479" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: aj6exD3cSuOtZdG1wU8GSA== X-CSE-MsgGUID: DRbzlwBFQHa0+17d+NWU5w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472041" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 2/9] drm/ttm: Add ttm_bo_access Date: Mon, 21 Oct 2024 13:15:02 -0700 Message-Id: <20241021201509.1668074-3-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add ttm_bo_access, which is similar to ttm_bo_vm_access, to access such memory. v4: - Fix checkpatch warnings (CI) Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost --- drivers/gpu/drm/ttm/ttm_bo_util.c | 86 +++++++++++++++++++++++++++++++ drivers/gpu/drm/ttm/ttm_bo_vm.c | 65 +---------------------- include/drm/ttm/ttm_bo.h | 2 + 3 files changed, 89 insertions(+), 64 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index d939925efa81..290bf1ee0202 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -919,3 +919,89 @@ s64 ttm_lru_walk_for_evict(struct ttm_lru_walk *walk, struct ttm_device *bdev, return progress; } + +static int ttm_bo_access_kmap(struct ttm_buffer_object *bo, + unsigned long offset, + void *buf, int len, int write) +{ + unsigned long page = offset >> PAGE_SHIFT; + unsigned long bytes_left = len; + int ret; + + /* Copy a page at a time, that way no extra virtual address + * mapping is needed + */ + offset -= page << PAGE_SHIFT; + do { + unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); + struct ttm_bo_kmap_obj map; + void *ptr; + bool is_iomem; + + ret = ttm_bo_kmap(bo, page, 1, &map); + if (ret) + return ret; + + ptr = (void *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; + WARN_ON_ONCE(is_iomem); + if (write) + memcpy(ptr, buf, bytes); + else + memcpy(buf, ptr, bytes); + ttm_bo_kunmap(&map); + + page++; + buf += bytes; + bytes_left -= bytes; + offset = 0; + } while (bytes_left); + + return len; +} + +/** + * ttm_bo_access - Helper to access a buffer object + * + * @bo: ttm buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Utility function to access a buffer object. Useful when buffer object cannot + * be easily mapped (non-contiguous, non-visible, etc...). + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write) +{ + int ret; + + if (len < 1 || (offset + len) > bo->base.size) + return -EIO; + + ret = ttm_bo_reserve(bo, true, false, NULL); + if (ret) + return ret; + + switch (bo->resource->mem_type) { + case TTM_PL_SYSTEM: + fallthrough; + case TTM_PL_TT: + ret = ttm_bo_access_kmap(bo, offset, buf, len, write); + break; + default: + if (bo->bdev->funcs->access_memory) + ret = bo->bdev->funcs->access_memory (bo, offset, buf, + len, write); + else + ret = -EIO; + } + + ttm_bo_unreserve(bo); + + return ret; +} +EXPORT_SYMBOL(ttm_bo_access); diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 2c699ed1963a..20b1e5f78684 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -366,45 +366,6 @@ void ttm_bo_vm_close(struct vm_area_struct *vma) } EXPORT_SYMBOL(ttm_bo_vm_close); -static int ttm_bo_vm_access_kmap(struct ttm_buffer_object *bo, - unsigned long offset, - uint8_t *buf, int len, int write) -{ - unsigned long page = offset >> PAGE_SHIFT; - unsigned long bytes_left = len; - int ret; - - /* Copy a page at a time, that way no extra virtual address - * mapping is needed - */ - offset -= page << PAGE_SHIFT; - do { - unsigned long bytes = min(bytes_left, PAGE_SIZE - offset); - struct ttm_bo_kmap_obj map; - void *ptr; - bool is_iomem; - - ret = ttm_bo_kmap(bo, page, 1, &map); - if (ret) - return ret; - - ptr = (uint8_t *)ttm_kmap_obj_virtual(&map, &is_iomem) + offset; - WARN_ON_ONCE(is_iomem); - if (write) - memcpy(ptr, buf, bytes); - else - memcpy(buf, ptr, bytes); - ttm_bo_kunmap(&map); - - page++; - buf += bytes; - bytes_left -= bytes; - offset = 0; - } while (bytes_left); - - return len; -} - int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write) { @@ -412,32 +373,8 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, unsigned long offset = (addr) - vma->vm_start + ((vma->vm_pgoff - drm_vma_node_start(&bo->base.vma_node)) << PAGE_SHIFT); - int ret; - - if (len < 1 || (offset + len) > bo->base.size) - return -EIO; - ret = ttm_bo_reserve(bo, true, false, NULL); - if (ret) - return ret; - - switch (bo->resource->mem_type) { - case TTM_PL_SYSTEM: - fallthrough; - case TTM_PL_TT: - ret = ttm_bo_vm_access_kmap(bo, offset, buf, len, write); - break; - default: - if (bo->bdev->funcs->access_memory) - ret = bo->bdev->funcs->access_memory( - bo, offset, buf, len, write); - else - ret = -EIO; - } - - ttm_bo_unreserve(bo); - - return ret; + return ttm_bo_access(bo, offset, buf, len, write); } EXPORT_SYMBOL(ttm_bo_vm_access); diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h index 5804408815be..8ea11cd8df39 100644 --- a/include/drm/ttm/ttm_bo.h +++ b/include/drm/ttm/ttm_bo.h @@ -421,6 +421,8 @@ void ttm_bo_unpin(struct ttm_buffer_object *bo); int ttm_bo_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, struct ttm_operation_ctx *ctx); +int ttm_bo_access(struct ttm_buffer_object *bo, unsigned long offset, + void *buf, int len, int write); vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, struct vm_fault *vmf); vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, From patchwork Mon Oct 21 20:15:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 113C7D17130 for ; Mon, 21 Oct 2024 20:14:49 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 66BE710E595; Mon, 21 Oct 2024 20:14:45 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="WjH94JHR"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6684810E037; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541684; x=1761077684; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a5HCOwVKHgAmUvo9H7JPdG4YV6V4zmZgKNtprKhXjJ4=; b=WjH94JHRdTXKwLYxPS1IP0K56YFCZ/xC+bKwN/xjvV8Z/LG0ODg6oiyA kDGhgWUCcxBuAYjpjNfD+FafAZDIWVzWYPoHqK+M6nxDARbs4bMYORWP4 mkndnweMSxV/llxtKWb7rhEYphi1ZBfLipG1TQrLfBQFtgNRrILP0QlkM 9gQkp2DYqJ9XOxbFWmIkDcqDsLB4wFHj4rZL1sq7QyjPhVbeVYJjlTFfd +bzMCnTaJye8uAvUTCfbdkibuhmxzoJ2Jl6XpyzSlsmrx6jw5oPg2pvSi ++2yp37PDiBJQ//trHHpD/H388bmXYHQ0h1bYKbu/tVsmU1O69bujuxzH g==; X-CSE-ConnectionGUID: cfmXxZoCSsKBdiwvkEj+JQ== X-CSE-MsgGUID: FXz1sWWnRsCXUZTbvK4FFw== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167480" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167480" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: ZVFlUh43S0u2Bt51fBhPlg== X-CSE-MsgGUID: wrUV9By7QiO+TMI/esfCyg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472044" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 3/9] drm/xe: Add xe_ttm_access_memory Date: Mon, 21 Oct 2024 13:15:03 -0700 Message-Id: <20241021201509.1668074-4-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous VRAM cannot easily be mapped in TTM nor can non-visible VRAM easily be accessed. Add xe_ttm_access_memory which hooks into ttm_bo_access to access such memory. v4: - Assert memory access rather than taking RPM ref (Thomas / Auld) - Fix warning on xe_res_cursor.h for non-zero offset (Mika) Reported-by: Christoph Manszewski Suggested-by: Thomas Hellström Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 59 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 56 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 0261a8b29351..04bc042f3bc7 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -442,6 +442,14 @@ static void xe_ttm_tt_destroy(struct ttm_device *ttm_dev, struct ttm_tt *tt) kfree(tt); } +static bool xe_ttm_resource_visible(struct ttm_resource *mem) +{ + struct xe_ttm_vram_mgr_resource *vres = + to_xe_ttm_vram_mgr_resource(mem); + + return vres->used_visible_size == mem->size; +} + static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem) { @@ -453,11 +461,9 @@ static int xe_ttm_io_mem_reserve(struct ttm_device *bdev, return 0; case XE_PL_VRAM0: case XE_PL_VRAM1: { - struct xe_ttm_vram_mgr_resource *vres = - to_xe_ttm_vram_mgr_resource(mem); struct xe_mem_region *vram = res_to_mem_region(mem); - if (vres->used_visible_size < mem->size) + if (!xe_ttm_resource_visible(mem)) return -EINVAL; mem->bus.offset = mem->start << PAGE_SHIFT; @@ -1111,6 +1117,52 @@ static void xe_ttm_bo_swap_notify(struct ttm_buffer_object *ttm_bo) } } +static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_bo *bo = ttm_to_xe_bo(ttm_bo); + struct xe_device *xe = ttm_to_xe_device(ttm_bo->bdev); + struct iosys_map vmap; + struct xe_res_cursor cursor; + struct xe_mem_region *vram; + int bytes_left = len; + + xe_bo_assert_held(bo); + xe_device_assert_mem_access(xe); + + if (!mem_type_is_vram(ttm_bo->resource->mem_type)) + return -EIO; + + /* FIXME: Use GPU for non-visible VRAM */ + if (!xe_ttm_resource_visible(ttm_bo->resource)) + return -EIO; + + vram = res_to_mem_region(ttm_bo->resource); + xe_res_first(ttm_bo->resource, offset & PAGE_MASK, + bo->size - (offset & PAGE_MASK), &cursor); + + do { + unsigned long page_offset = (offset & ~PAGE_MASK); + int byte_count = min((int)(PAGE_SIZE - page_offset), bytes_left); + + iosys_map_set_vaddr_iomem(&vmap, (u8 __iomem *)vram->mapping + + cursor.start); + if (write) + xe_map_memcpy_to(xe, &vmap, page_offset, buf, byte_count); + else + xe_map_memcpy_from(xe, buf, &vmap, page_offset, byte_count); + + offset += byte_count; + buf += byte_count; + bytes_left -= byte_count; + if (bytes_left) + xe_res_next(&cursor, PAGE_SIZE); + } while (bytes_left); + + return len; +} + const struct ttm_device_funcs xe_ttm_funcs = { .ttm_tt_create = xe_ttm_tt_create, .ttm_tt_populate = xe_ttm_tt_populate, @@ -1120,6 +1172,7 @@ const struct ttm_device_funcs xe_ttm_funcs = { .move = xe_bo_move, .io_mem_reserve = xe_ttm_io_mem_reserve, .io_mem_pfn = xe_ttm_io_mem_pfn, + .access_memory = xe_ttm_access_memory, .release_notify = xe_ttm_bo_release_notify, .eviction_valuable = ttm_bo_eviction_valuable, .delete_mem_notify = xe_ttm_bo_delete_mem_notify, From patchwork Mon Oct 21 20:15:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E453AD1712E for ; Mon, 21 Oct 2024 20:14:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BE1A710E596; Mon, 21 Oct 2024 20:14:45 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="kb5nXyq2"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id 78FDD10E3CD; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541684; x=1761077684; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VFeTNJiJ7lL3Wyx7JIq1pnfndPJQjcZDjNTMoynIz8U=; b=kb5nXyq2e6ZRqYNLKGcNFyeDh6VHxOIP8SBHUDknIBiz/onJ26BdYNeC gKlztKiyks3lcx3SkjiVysdBirb6C2hTgpyo0P01X6BOCmoQB9i2kNtH8 VDLBTRAFI37WxryHkNhJHEhWBCKmwwSQNPJuTaYZlSE3C5CYSe2Y693/c cdUNGc5QS0KVGXR1hWNm3hQBg5FAkk4futCmYWYTykIxT2a5ird1SIsQ8 nMWivIH/ZjXGXYyQSs/jwLwQfoRGvyGeZsmOA25XZHVY9QDdtyp7ailYN UeH2xR4l2MhMdAsbEnvdBE0HsOnX5AWXBqEtRh+EiQpeZTALkn9Wc847A g==; X-CSE-ConnectionGUID: qDX1KYBMTvKLXhvWELSMrA== X-CSE-MsgGUID: tsnCozmpQ/SVpRsxpeBwtg== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167481" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167481" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: fSOrjTTzRWmE8exN17OYHg== X-CSE-MsgGUID: W1Ji1s4QTFWkDvoXuUb/tg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472047" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 4/9] drm/xe: Take PM ref in delayed snapshot capture worker Date: Mon, 21 Oct 2024 13:15:04 -0700 Message-Id: <20241021201509.1668074-5-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The delayed snapshot capture worker can access the GPU or VRAM both of which require a PM reference. Take a reference in this worker. Cc: Rodrigo Vivi Cc: Maarten Lankhorst Fixes: 4f04d07c0a94 ("drm/xe: Faster devcoredump") Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_devcoredump.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c index 8b0ea77661b2..956bf087d0db 100644 --- a/drivers/gpu/drm/xe/xe_devcoredump.c +++ b/drivers/gpu/drm/xe/xe_devcoredump.c @@ -23,6 +23,7 @@ #include "xe_guc_submit.h" #include "xe_hw_engine.h" #include "xe_module.h" +#include "xe_pm.h" #include "xe_sched_job.h" #include "xe_vm.h" @@ -158,8 +159,11 @@ static void xe_devcoredump_deferred_snap_work(struct work_struct *work) { struct xe_devcoredump_snapshot *ss = container_of(work, typeof(*ss), work); struct xe_devcoredump *coredump = container_of(ss, typeof(*coredump), snapshot); + struct xe_device *xe = coredump_to_xe(coredump); unsigned int fw_ref; + xe_pm_runtime_get(xe); + /* keep going if fw fails as we still want to save the memory and SW data */ fw_ref = xe_force_wake_get(gt_to_fw(ss->gt), XE_FORCEWAKE_ALL); if (!xe_force_wake_ref_has_domain(fw_ref, XE_FORCEWAKE_ALL)) @@ -168,6 +172,8 @@ static void xe_devcoredump_deferred_snap_work(struct work_struct *work) xe_guc_exec_queue_snapshot_capture_delayed(ss->ge); xe_force_wake_put(gt_to_fw(ss->gt), fw_ref); + xe_pm_runtime_put(xe); + /* Calculate devcoredump size */ ss->read.size = __xe_devcoredump_read(NULL, INT_MAX, coredump); From patchwork Mon Oct 21 20:15:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C794D1712E for ; Mon, 21 Oct 2024 20:14:55 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A52EB10E59D; Mon, 21 Oct 2024 20:14:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="SXyt/DjX"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9539010E037; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541685; x=1761077685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=msmMJ8OcRbbjHqFZudpik2ixZFJL2gw0otG39NZEU/o=; b=SXyt/DjXU85eEyiJJzco5FTNcQIdaeoIWgEjbOUU/wmYFOyBJL/I6I1F SIjRYQQfH90NIt+ejefiN42ZxP/EFkDQYPrMkxNtRZdxa9VxgIEq0UuDr q5I/43xxFmEyEoGMSPQT3ZuzJ1MgnRgECeQrKX70Ld7SN9+aNlyTzuPi3 DcdRzI3rTydUchR6g3l/IaqdqM6iZFlZRg0mwHYa4CC+8CGOysD0eCSOT Wdr3h/+UzO2o8MhpOPqYuWA0k+ulRISmpxiUiiDXXzobkfSQ0GrpfLi8B ujjKcFvyH054h1PYfdM9tsHfzFu/AN9VzhfCblQ5/TlrV30urwyF4TJ/c Q==; X-CSE-ConnectionGUID: I949QCMDQGSiShn0aiLsDQ== X-CSE-MsgGUID: YvJ8Wb5PRt6DEkFQni/75Q== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167482" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167482" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: b3aNPMESQFiau5skC0GtCw== X-CSE-MsgGUID: D6NGJ2o3ScOLqM+TSofgAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472050" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 5/9] drm/xe: Update xe_ttm_access_memory to use GPU for non-visible access Date: Mon, 21 Oct 2024 13:15:05 -0700 Message-Id: <20241021201509.1668074-6-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add migrate layer functions to access VRAM and update xe_ttm_access_memory to use for non-visible access and large (more tahn 16k) BO access. v4: - Fix non-page aligned accesses - Add support for small / unaligned access - Update commit message indicating migrate used for large accesses (Auld) - Fix warning in xe_res_cursor for non-zero offset Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_bo.c | 15 +- drivers/gpu/drm/xe/xe_migrate.c | 325 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_migrate.h | 4 + 3 files changed, 340 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 04bc042f3bc7..4e1f54aad5bc 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1127,6 +1127,7 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, struct xe_res_cursor cursor; struct xe_mem_region *vram; int bytes_left = len; + int err = 0; xe_bo_assert_held(bo); xe_device_assert_mem_access(xe); @@ -1134,9 +1135,14 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, if (!mem_type_is_vram(ttm_bo->resource->mem_type)) return -EIO; - /* FIXME: Use GPU for non-visible VRAM */ - if (!xe_ttm_resource_visible(ttm_bo->resource)) - return -EIO; + if (!xe_ttm_resource_visible(ttm_bo->resource) || len >= SZ_16K) { + struct xe_migrate *migrate = + mem_type_to_migrate(xe, ttm_bo->resource->mem_type); + + err = xe_migrate_access_memory(migrate, bo, offset, buf, len, + write); + goto out; + } vram = res_to_mem_region(ttm_bo->resource); xe_res_first(ttm_bo->resource, offset & PAGE_MASK, @@ -1160,7 +1166,8 @@ static int xe_ttm_access_memory(struct ttm_buffer_object *ttm_bo, xe_res_next(&cursor, PAGE_SIZE); } while (bytes_left); - return len; +out: + return err ?: len; } const struct ttm_device_funcs xe_ttm_funcs = { diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c index cfd31ae49cc1..c658b94a35af 100644 --- a/drivers/gpu/drm/xe/xe_migrate.c +++ b/drivers/gpu/drm/xe/xe_migrate.c @@ -669,6 +669,7 @@ static void emit_copy(struct xe_gt *gt, struct xe_bb *bb, u32 mocs = 0; u32 tile_y = 0; + xe_gt_assert(gt, !(pitch & 3)); xe_gt_assert(gt, size / pitch <= S16_MAX); xe_gt_assert(gt, pitch / 4 <= S16_MAX); xe_gt_assert(gt, pitch <= U16_MAX); @@ -1542,6 +1543,330 @@ void xe_migrate_wait(struct xe_migrate *m) dma_fence_wait(m->fence, false); } +static u32 pte_update_cmd_size(u64 size) +{ + u32 dword; + u64 entries = DIV_ROUND_UP(size, XE_PAGE_SIZE); + + XE_WARN_ON(size > MAX_PREEMPTDISABLE_TRANSFER); + /* + * MI_STORE_DATA_IMM command is used to update page table. Each + * instruction can update maximumly 0x1ff pte entries. To update + * n (n <= 0x1ff) pte entries, we need: + * 1 dword for the MI_STORE_DATA_IMM command header (opcode etc) + * 2 dword for the page table's physical location + * 2*n dword for value of pte to fill (each pte entry is 2 dwords) + */ + dword = (1 + 2) * DIV_ROUND_UP(entries, 0x1ff); + dword += entries * 2; + + return dword; +} + +static void build_pt_update_batch_sram(struct xe_migrate *m, + struct xe_bb *bb, u32 pt_offset, + dma_addr_t *sram_addr, u32 size) +{ + u16 pat_index = tile_to_xe(m->tile)->pat.idx[XE_CACHE_WB]; + u32 ptes; + int i = 0; + + ptes = DIV_ROUND_UP(size, XE_PAGE_SIZE); + while (ptes) { + u32 chunk = min(0x1ffU, ptes); + + bb->cs[bb->len++] = MI_STORE_DATA_IMM | MI_SDI_NUM_QW(chunk); + bb->cs[bb->len++] = pt_offset; + bb->cs[bb->len++] = 0; + + pt_offset += chunk * 8; + ptes -= chunk; + + while (chunk--) { + u64 addr = sram_addr[i++] & PAGE_MASK; + + xe_tile_assert(m->tile, addr); + addr = m->q->vm->pt_ops->pte_encode_addr(m->tile->xe, + addr, pat_index, + 0, false, 0); + bb->cs[bb->len++] = lower_32_bits(addr); + bb->cs[bb->len++] = upper_32_bits(addr); + } + } +} + +enum xe_migrate_copy_dir { + XE_MIGRATE_COPY_TO_VRAM, + XE_MIGRATE_COPY_TO_SRAM, +}; + +#define CACHELINE_BYTES 64ull +#define CACHELINE_MASK (CACHELINE_BYTES - 1) + +static struct dma_fence *xe_migrate_vram(struct xe_migrate *m, + unsigned long len, + unsigned long sram_offset, + dma_addr_t *sram_addr, u64 vram_addr, + const enum xe_migrate_copy_dir dir) +{ + struct xe_gt *gt = m->tile->primary_gt; + struct xe_device *xe = gt_to_xe(gt); + struct dma_fence *fence = NULL; + u32 batch_size = 2; + u64 src_L0_ofs, dst_L0_ofs; + struct xe_sched_job *job; + struct xe_bb *bb; + u32 update_idx, pt_slot = 0; + unsigned long npages = DIV_ROUND_UP(len + sram_offset, PAGE_SIZE); + unsigned int pitch = len >= PAGE_SIZE && !(len & ~PAGE_MASK) ? + PAGE_SIZE : 4; + int err; + + if (drm_WARN_ON(&xe->drm, (len & CACHELINE_MASK) || + (sram_offset | vram_addr) & CACHELINE_MASK)) + return ERR_PTR(-EOPNOTSUPP); + + xe_assert(xe, npages * PAGE_SIZE <= MAX_PREEMPTDISABLE_TRANSFER); + + batch_size += pte_update_cmd_size(len); + batch_size += EMIT_COPY_DW; + + bb = xe_bb_new(gt, batch_size, true); + if (IS_ERR(bb)) { + err = PTR_ERR(bb); + return ERR_PTR(err); + } + + build_pt_update_batch_sram(m, bb, pt_slot * XE_PAGE_SIZE, + sram_addr, len + sram_offset); + + if (dir == XE_MIGRATE_COPY_TO_VRAM) { + src_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + dst_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + + } else { + src_L0_ofs = xe_migrate_vram_ofs(xe, vram_addr, false); + dst_L0_ofs = xe_migrate_vm_addr(pt_slot, 0) + sram_offset; + } + + bb->cs[bb->len++] = MI_BATCH_BUFFER_END; + update_idx = bb->len; + + emit_copy(gt, bb, src_L0_ofs, dst_L0_ofs, len, pitch); + + job = xe_bb_create_migration_job(m->q, bb, + xe_migrate_batch_base(m, true), + update_idx); + if (IS_ERR(job)) { + err = PTR_ERR(job); + goto err; + } + + xe_sched_job_add_migrate_flush(job, 0); + + mutex_lock(&m->job_mutex); + xe_sched_job_arm(job); + fence = dma_fence_get(&job->drm.s_fence->finished); + xe_sched_job_push(job); + + dma_fence_put(m->fence); + m->fence = dma_fence_get(fence); + mutex_unlock(&m->job_mutex); + + xe_bb_free(bb, fence); + + return fence; + +err: + xe_bb_free(bb, NULL); + + return ERR_PTR(err); +} + +static void xe_migrate_dma_unmap(struct xe_device *xe, dma_addr_t *dma_addr, + int len, int write) +{ + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + for (i = 0; i < npages; ++i) { + if (!dma_addr[i]) + continue; + + dma_unmap_page(xe->drm.dev, dma_addr[i], PAGE_SIZE, + write ? DMA_TO_DEVICE : DMA_FROM_DEVICE); + } + kfree(dma_addr); +} + +static dma_addr_t *xe_migrate_dma_map(struct xe_device *xe, + void *buf, int len, int write) +{ + dma_addr_t *dma_addr; + unsigned long i, npages = DIV_ROUND_UP(len, PAGE_SIZE); + + dma_addr = kcalloc(npages, sizeof(*dma_addr), GFP_KERNEL); + if (!dma_addr) + return ERR_PTR(-ENOMEM); + + for (i = 0; i < npages; ++i) { + dma_addr_t addr; + struct page *page; + + if (is_vmalloc_addr(buf)) + page = vmalloc_to_page(buf); + else + page = virt_to_page(buf); + + addr = dma_map_page(xe->drm.dev, + page, 0, PAGE_SIZE, + write ? DMA_TO_DEVICE : + DMA_FROM_DEVICE); + if (dma_mapping_error(xe->drm.dev, addr)) + goto err_fault; + + dma_addr[i] = addr; + buf += PAGE_SIZE; + } + + return dma_addr; + +err_fault: + xe_migrate_dma_unmap(xe, dma_addr, len, write); + return ERR_PTR(-EFAULT); +} + +/** + * xe_migrate_access_memory - Access memory of a BO via GPU + * + * @m: The migration context. + * @bo: buffer object + * @offset: access offset into buffer object + * @buf: pointer to caller memory to read into or write from + * @len: length of access + * @write: write access + * + * Access memory of a BO via GPU either reading in or writing from a passed in + * pointer. Pointer is dma mapped for GPU access and GPU commands are issued to + * read to or write from pointer. + * + * Returns: + * 0 if successful, negative error code on failure. + */ +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write) +{ + struct xe_tile *tile = m->tile; + struct xe_device *xe = tile_to_xe(tile); + struct xe_res_cursor cursor; + struct dma_fence *fence = NULL; + dma_addr_t *dma_addr; + unsigned long page_offset = (unsigned long)buf & ~PAGE_MASK; + int bytes_left = len, current_page = 0; + void *orig_buf = buf; + + xe_bo_assert_held(bo); + + /* Use bounce buffer for small access and unaligned access */ + if (len & CACHELINE_MASK || ((u64)buf | offset) & CACHELINE_MASK) { + int buf_offset = 0; + + /* + * Less than ideal for large unaligned access but this should be + * fairly rare, can fixup if this becomes common. + */ + do { + u8 bounce[CACHELINE_BYTES]; + void *ptr = (void *)bounce; + int err; + int copy_bytes = min_t(int, bytes_left, + CACHELINE_BYTES - + (offset & CACHELINE_MASK)); + int ptr_offset = offset & CACHELINE_MASK; + + err = xe_migrate_access_memory(m, bo, + offset & ~CACHELINE_MASK, + (void *)ptr, + sizeof(bounce), 0); + if (err) + return err; + + if (!write) { + memcpy(buf + buf_offset, ptr + ptr_offset, + copy_bytes); + goto next; + } + + memcpy(ptr + ptr_offset, buf + buf_offset, copy_bytes); + err = xe_migrate_access_memory(m, bo, + offset & ~CACHELINE_MASK, + (void *)ptr, + sizeof(bounce), 0); + if (err) + return err; + +next: + bytes_left -= copy_bytes; + buf_offset += copy_bytes; + offset += copy_bytes; + } while (bytes_left); + + return 0; + } + + dma_addr = xe_migrate_dma_map(xe, buf, len + page_offset, write); + if (IS_ERR(dma_addr)) + return PTR_ERR(dma_addr); + + xe_res_first(bo->ttm.resource, offset, bo->size - offset, &cursor); + + do { + struct dma_fence *__fence; + u64 vram_addr = vram_region_gpu_offset(bo->ttm.resource) + + cursor.start; + int current_bytes; + + if (cursor.size > MAX_PREEMPTDISABLE_TRANSFER) + current_bytes = min_t(int, bytes_left, + MAX_PREEMPTDISABLE_TRANSFER); + else + current_bytes = min_t(int, bytes_left, cursor.size); + + if (fence) + dma_fence_put(fence); + + __fence = xe_migrate_vram(m, current_bytes, + (unsigned long)buf & ~PAGE_MASK, + dma_addr + current_page, + vram_addr, write ? + XE_MIGRATE_COPY_TO_VRAM : + XE_MIGRATE_COPY_TO_SRAM); + if (IS_ERR(__fence)) { + if (fence) + dma_fence_wait(fence, false); + fence = __fence; + goto out_err; + } + fence = __fence; + + buf += current_bytes; + offset += current_bytes; + current_page = (int)(buf - orig_buf) / PAGE_SIZE; + bytes_left -= current_bytes; + if (bytes_left) + xe_res_next(&cursor, current_bytes); + } while (bytes_left); + + dma_fence_wait(fence, false); + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + + return 0; + +out_err: + xe_migrate_dma_unmap(xe, dma_addr, len + page_offset, write); + return PTR_ERR(fence); +} + #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) #include "tests/xe_migrate.c" #endif diff --git a/drivers/gpu/drm/xe/xe_migrate.h b/drivers/gpu/drm/xe/xe_migrate.h index 0109866e398a..94197d262178 100644 --- a/drivers/gpu/drm/xe/xe_migrate.h +++ b/drivers/gpu/drm/xe/xe_migrate.h @@ -102,6 +102,10 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, struct ttm_resource *dst, bool copy_only_ccs); +int xe_migrate_access_memory(struct xe_migrate *m, struct xe_bo *bo, + unsigned long offset, void *buf, int len, + int write); + #define XE_MIGRATE_CLEAR_FLAG_BO_DATA BIT(0) #define XE_MIGRATE_CLEAR_FLAG_CCS_DATA BIT(1) #define XE_MIGRATE_CLEAR_FLAG_FULL (XE_MIGRATE_CLEAR_FLAG_BO_DATA | \ From patchwork Mon Oct 21 20:15:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AFB6D17134 for ; Mon, 21 Oct 2024 20:14:53 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 10F7710E599; Mon, 21 Oct 2024 20:14:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="KUfZJ+b4"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id A6FCE10E58F; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541685; x=1761077685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5uXjQe9E+NBXZdyGgi2OooU8JncrVZKopc7IxW16Bl0=; b=KUfZJ+b436QqnaScXPKUZL5i6seemd6zF36Gp49AOAC/djoWYsSIoyvG /w8cRC3nCutLxJwoD/pUAw3QqQus3HyHtRjHTxBCQFkLvSO8a/5kwxQHA g4x8wm6p13dPLZegQprgxxX2aSP8XBj9YQWJCzdSHNeEbXMQ2lqMSIeZR KXbJmSP7DIEI561vw3ewnTTZ9xuFQXrDHiywTdOY6VFQVhn304eDkwgeg lsiGbtYUu/t0f2cpYVSEPZDA4CTk8muZqYiAEj2HKIBN1Hr2NK543wUzt gja/HjpBuoHYEbjVLH77huRirBtcgg0HZqKE1DrPHwhUY+9hXRuLi6iCy w==; X-CSE-ConnectionGUID: +c+hmsJOS76TevxHwpYbyw== X-CSE-MsgGUID: 3XArb0CWT7efu2r8qdYDMw== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167483" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167483" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: xaWshuiRRP+Xxo0YB+j7TQ== X-CSE-MsgGUID: MGe8nWCMQ6uBHyj+nkNlRQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472053" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 6/9] drm/xe/display: Update intel_bo_read_from_page to use ttm_bo_access Date: Mon, 21 Oct 2024 13:15:06 -0700 Message-Id: <20241021201509.1668074-7-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Don't open code vmap of a BO, use ttm_bo_access helper which is safe for non-contiguous BOs and non-visible BOs. Suggested-by: Matthew Auld Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/display/intel_bo.c | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/drivers/gpu/drm/xe/display/intel_bo.c b/drivers/gpu/drm/xe/display/intel_bo.c index 9f54fad0f1c0..43141964f6f2 100644 --- a/drivers/gpu/drm/xe/display/intel_bo.c +++ b/drivers/gpu/drm/xe/display/intel_bo.c @@ -40,31 +40,8 @@ int intel_bo_fb_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) int intel_bo_read_from_page(struct drm_gem_object *obj, u64 offset, void *dst, int size) { struct xe_bo *bo = gem_to_xe_bo(obj); - struct ttm_bo_kmap_obj map; - void *src; - bool is_iomem; - int ret; - ret = xe_bo_lock(bo, true); - if (ret) - return ret; - - ret = ttm_bo_kmap(&bo->ttm, offset >> PAGE_SHIFT, 1, &map); - if (ret) - goto out_unlock; - - offset &= ~PAGE_MASK; - src = ttm_kmap_obj_virtual(&map, &is_iomem); - src += offset; - if (is_iomem) - memcpy_fromio(dst, (void __iomem *)src, size); - else - memcpy(dst, src, size); - - ttm_bo_kunmap(&map); -out_unlock: - xe_bo_unlock(bo); - return ret; + return ttm_bo_access(&bo->ttm, offset, dst, size, 0); } struct intel_frontbuffer *intel_bo_get_frontbuffer(struct drm_gem_object *obj) From patchwork Mon Oct 21 20:15:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DA177D17131 for ; Mon, 21 Oct 2024 20:14:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BA9D110E59F; Mon, 21 Oct 2024 20:14:47 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="AflrBuqC"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id B7CB510E037; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541685; x=1761077685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rgdWjLCAsWDICAk8WcCYB+yo/wtl3ra7r8m7MBpFXLE=; b=AflrBuqCWNb+JdrCf5XamhPA3hJDMO3z5E2lbxyBP97Dw7VTkW22/jI+ IcqiT/9tITsHtz2A6nu6ZXfGuBbcYMHkbBA+CmYo1wAOTdDsFbe7YJtcW SOvPc22qqgA/ANIhaRkYBJow4w/R48iV72+YlijBqUs2xrY/xjyiEykLP Phn7qqw8HkiEx3AWE/5tOj0lu3JyMtqA8suCX/aMl2px0c7wrKhPQiNyH wy4GaIsBvpLTFVV+afBwJwHsoMtQkPKCWzXoxh3PZf5jUg5kUoqxFX/GC Sb+pK5UowDVeFnrTxQ30g0o8ndvt4qSSckeNzwL5FKFzs+VQfm5EJIDIH A==; X-CSE-ConnectionGUID: gI5hXUVMRf+kbt1htW8d5g== X-CSE-MsgGUID: tZFFTDehTBK7hOi97iTxkg== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167484" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167484" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: hZbDjI/1RquHM0n4cn4k+Q== X-CSE-MsgGUID: a1QaPWfBSAug5L7A04FRfg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472056" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 7/9] drm/xe: Use ttm_bo_access in xe_vm_snapshot_capture_delayed Date: Mon, 21 Oct 2024 13:15:07 -0700 Message-Id: <20241021201509.1668074-8-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Non-contiguous mapping of BO in VRAM doesn't work, use ttm_bo_access instead. v2: - Fix error handling Fixes: 0eb2a18a8fad ("drm/xe: Implement VM snapshot support for BO's and userptr") Suggested-by: Matthew Auld Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_vm.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index c99380271de6..c8782da3a5c3 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -3303,7 +3303,6 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) for (int i = 0; i < snap->num_snaps; i++) { struct xe_bo *bo = snap->snap[i].bo; - struct iosys_map src; int err; if (IS_ERR(snap->snap[i].data)) @@ -3316,16 +3315,12 @@ void xe_vm_snapshot_capture_delayed(struct xe_vm_snapshot *snap) } if (bo) { - xe_bo_lock(bo, false); - err = ttm_bo_vmap(&bo->ttm, &src); - if (!err) { - xe_map_memcpy_from(xe_bo_device(bo), - snap->snap[i].data, - &src, snap->snap[i].bo_ofs, - snap->snap[i].len); - ttm_bo_vunmap(&bo->ttm, &src); - } - xe_bo_unlock(bo); + err = ttm_bo_access(&bo->ttm, snap->snap[i].bo_ofs, + snap->snap[i].data, snap->snap[i].len, 0); + if (!(err < 0) && err != snap->snap[i].len) + err = -EIO; + else if (!(err < 0)) + err = 0; } else { void __user *userptr = (void __user *)(size_t)snap->snap[i].bo_ofs; From patchwork Mon Oct 21 20:15:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86594D17132 for ; Mon, 21 Oct 2024 20:14:54 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8E47410E59C; Mon, 21 Oct 2024 20:14:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="ATTQEa0n"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id D3E0210E3CD; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541685; x=1761077685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nZMJwEdjNoBvlGVgeI7A/wYeq0nHCWPF1Z/N58o2JT8=; b=ATTQEa0nIDEFyr6XEypqWTpYrL2c3aAqCyf5n62I9ymw+lAEh8rFrC+i 0XtWiKhBVrarOHZgRbz5wxAXxoF6GgA5ADUOHeORMWgW2Akmf4XBwtlMm BoaE8i6gBTkZhre5fu5sOuV+z9nZ3b/dP6owTTdo21NaCb/QYoT8bRxU8 a87IQZgyuiJZ2KVphgZ5EODBwKBWTmE+eoxJuwCqCgicjcfVNX5qDLWcp xRF9K/V1AR2nPKriiTPFaLFYA0c6thzNH0uBAccpRtvjVVREy3DfJKy1i O7eJ+aAWi0AnXrbW3uovhlmCYkLFGsQcQkuz56PG3mwY/QKIKZSGrg7Y6 g==; X-CSE-ConnectionGUID: XF0KBVmIRaaZDvuAlDAy5Q== X-CSE-MsgGUID: Yo9EAU3TQCavBTgJLyju/Q== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167485" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167485" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: +L02YhuATkGLj3Ww6QjEFw== X-CSE-MsgGUID: XuaM94TuQrKddakG0m+PfQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472059" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 8/9] drm/xe: Set XE_BO_FLAG_PINNED in migrate selftest BOs Date: Mon, 21 Oct 2024 13:15:08 -0700 Message-Id: <20241021201509.1668074-9-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We only allow continguous BOs to be vmapped, set XE_BO_FLAG_PINNED on BOs in migrate selftest as this forces continguous BOs and selftest uses vmaps. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/tests/xe_migrate.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/xe/tests/xe_migrate.c b/drivers/gpu/drm/xe/tests/xe_migrate.c index 1a192a2a941b..4cef3b20bd17 100644 --- a/drivers/gpu/drm/xe/tests/xe_migrate.c +++ b/drivers/gpu/drm/xe/tests/xe_migrate.c @@ -83,7 +83,8 @@ static void test_copy(struct xe_migrate *m, struct xe_bo *bo, bo->size, ttm_bo_type_kernel, region | - XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(remote)) { KUNIT_FAIL(test, "Failed to allocate remote bo for %s: %pe\n", str, remote); @@ -642,7 +643,9 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til sys_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - XE_BO_FLAG_SYSTEM | XE_BO_FLAG_NEEDS_CPU_ACCESS); + XE_BO_FLAG_SYSTEM | + XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(sys_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -666,7 +669,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til ccs_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(ccs_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", @@ -690,7 +694,8 @@ static void validate_ccs_test_run_tile(struct xe_device *xe, struct xe_tile *til vram_bo = xe_bo_create_user(xe, NULL, NULL, SZ_4M, DRM_XE_GEM_CPU_CACHING_WC, - bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS); + bo_flags | XE_BO_FLAG_NEEDS_CPU_ACCESS | + XE_BO_FLAG_PINNED); if (IS_ERR(vram_bo)) { KUNIT_FAIL(test, "xe_bo_create() failed with err=%ld\n", PTR_ERR(vram_bo)); From patchwork Mon Oct 21 20:15:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Brost X-Patchwork-Id: 13844632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7CF91D17134 for ; Mon, 21 Oct 2024 20:14:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 29C0A10E5A0; Mon, 21 Oct 2024 20:14:48 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="FXb6zt4u"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) by gabe.freedesktop.org (Postfix) with ESMTPS id E84EC10E58F; Mon, 21 Oct 2024 20:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729541685; x=1761077685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eSG5vY32/ZdefK/ah9eMHe8q26+p/YDG8IpBok17Zzc=; b=FXb6zt4umQrg3yY1KnzVDf0MWCRc/KfmUNESIJWBu+PjAtwYFpKI7eHz xyRtSNL6oZ+2VYP8wZOdQQmenXkGofP9jxPR0gWZfgsGM5kpvhWEf3BTv ON7Z3NQaw7uyPogN5NxxB2t4Li3M7NIDZ7k8+IeJQtFyM/9xBH/cHn5Xq 8cQgJYQk+deplL3ptQgXuZHX+3aW08b4VWX7ZYLSCKTAT1rp+tJaraU0T Btcl5XxegSF+d9lb7HUd2RjtlO8nmqzZLE7b9TNoPkKTU8nAifXnnNhku JooykaGS7fBqpFUoBIaiLpAfGmW3moPO6Fv6aN6f8WSMU6kmk7K3r4aOu A==; X-CSE-ConnectionGUID: YQTzoKbYTjKjWeM5n/D9Jw== X-CSE-MsgGUID: RXGTKtZjSxW6Cq3LJQ1BLg== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="40167486" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="40167486" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 X-CSE-ConnectionGUID: QiGYrWmdQFevIOqUbcOoOA== X-CSE-MsgGUID: zFact2ktQl2VA8pj/Zu5oA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,221,1725346800"; d="scan'208";a="110472062" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2024 13:14:44 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: mika.kuoppala@intel.com, matthew.auld@intel.com, thomas.hellstrom@linux.intel.com Subject: [PATCH v4 9/9] drm/xe: Only allow contiguous BOs to use xe_bo_vmap Date: Mon, 21 Oct 2024 13:15:09 -0700 Message-Id: <20241021201509.1668074-10-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241021201509.1668074-1-matthew.brost@intel.com> References: <20241021201509.1668074-1-matthew.brost@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" xe_bo_vmap only works on contiguous BOs, disallow xe_bo_vmap on BO unless we are certain the BO is contiguous. Signed-off-by: Matthew Brost Reviewed-by: Matthew Auld --- drivers/gpu/drm/xe/xe_bo.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 4e1f54aad5bc..59766369a15b 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -162,6 +162,15 @@ static void try_add_system(struct xe_device *xe, struct xe_bo *bo, } } +static bool force_contiguous(u32 bo_flags) +{ + /* + * For eviction / restore on suspend / resume objects pinned in VRAM + * must be contiguous, also only contiguous BOs support xe_bo_vmap. + */ + return bo_flags & (XE_BO_FLAG_PINNED | XE_BO_FLAG_GGTT); +} + static void add_vram(struct xe_device *xe, struct xe_bo *bo, struct ttm_place *places, u32 bo_flags, u32 mem_type, u32 *c) { @@ -175,12 +184,7 @@ static void add_vram(struct xe_device *xe, struct xe_bo *bo, xe_assert(xe, vram && vram->usable_size); io_size = vram->io_size; - /* - * For eviction / restore on suspend / resume objects - * pinned in VRAM must be contiguous - */ - if (bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT)) + if (force_contiguous(bo_flags)) place.flags |= TTM_PL_FLAG_CONTIGUOUS; if (io_size < vram->usable_size) { @@ -212,8 +216,7 @@ static void try_add_stolen(struct xe_device *xe, struct xe_bo *bo, bo->placements[*c] = (struct ttm_place) { .mem_type = XE_PL_STOLEN, - .flags = bo_flags & (XE_BO_FLAG_PINNED | - XE_BO_FLAG_GGTT) ? + .flags = force_contiguous(bo_flags) ? TTM_PL_FLAG_CONTIGUOUS : 0, }; *c += 1; @@ -2033,13 +2036,15 @@ dma_addr_t xe_bo_addr(struct xe_bo *bo, u64 offset, size_t page_size) int xe_bo_vmap(struct xe_bo *bo) { + struct xe_device *xe = ttm_to_xe_device(bo->ttm.bdev); void *virtual; bool is_iomem; int ret; xe_bo_assert_held(bo); - if (!(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS)) + if (drm_WARN_ON(&xe->drm, !(bo->flags & XE_BO_FLAG_NEEDS_CPU_ACCESS) || + !force_contiguous(bo->flags))) return -EINVAL; if (!iosys_map_is_null(&bo->vmap))