From patchwork Wed Mar 12 21:04:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m?= X-Patchwork-Id: 14013909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1B9BDC28B28 for ; Wed, 12 Mar 2025 21:05:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7490110E7C1; Wed, 12 Mar 2025 21:05:55 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="krnNTW9G"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4691D10E7C1; Wed, 12 Mar 2025 21:05:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1741813555; x=1773349555; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yhyIDKB+utiH3DNrMcbHUt4V9laIkxtlEiwdW3pYV3k=; b=krnNTW9Gv1qe7qHTvr7Ez3ApEIDqAD4CQIOi3R4GvaMHVtRvDrONcyPl ArGQ8QgxyEYyojHhvbsV4Qndc7WGJCbgfRZ//MzCCGdZYkw4n3ANrndJ6 pRNVhgGrQNAeqH2WkCWxMOElD2xS2ul91WPKyawdoxDlbITHmS0VRhkot Z5x/pZYh2LSQ264fz7tDHjsN3z01yMjvXvMKFAsygf4qcxxM1MUXZ3BCl X4jZLFteYKWUf0zGAyP+P+yKgZTvCHNbX+Cxz+2WL5JJkHBClqXUTSC20 RnjnL+TzRiWGOGXu23WhqTnXqnJYwWOPKQDfXhlCZWedwa87gDprEa+Ek Q==; X-CSE-ConnectionGUID: PR0ZDp+RTMm5k+MQGQeLiA== X-CSE-MsgGUID: Jdgqn/LQT7CGRKQbFt8Zaw== X-IronPort-AV: E=McAfee;i="6700,10204,11371"; a="46562298" X-IronPort-AV: E=Sophos;i="6.14,242,1736841600"; d="scan'208";a="46562298" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2025 14:05:55 -0700 X-CSE-ConnectionGUID: bGAC534XS4WMhSgoebhumw== X-CSE-MsgGUID: ftjd9YU0Rxym2XCejNsV2A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.14,242,1736841600"; d="scan'208";a="120791287" Received: from klitkey1-mobl1.ger.corp.intel.com (HELO fedora..) ([10.245.246.73]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2025 14:05:51 -0700 From: =?utf-8?q?Thomas_Hellstr=C3=B6m?= To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m?= , himal.prasad.ghimiray@intel.com, apopple@nvidia.com, airlied@gmail.com, Simona Vetter , felix.kuehling@amd.com, Matthew Brost , =?utf-8?q?Christian_K=C3=B6nig?= , dakr@kernel.org, "Mrozek, Michal" , Joonas Lahtinen Subject: [RFC PATCH 19/19] drm/xe: HAX: Use pcie p2p dma to test fast interconnect Date: Wed, 12 Mar 2025 22:04:16 +0100 Message-ID: <20250312210416.3120-20-thomas.hellstrom@linux.intel.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250312210416.3120-1-thomas.hellstrom@linux.intel.com> References: <20250312210416.3120-1-thomas.hellstrom@linux.intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Knowing that this is not the correct way to support pcie_p2p over hmm + the dma api, pretend that pcie_p2p is a driver-private fast interconnect to demonstrate how multi-device SVM can be done. This has been used to test SVM on a BMG client with a pagemap created on a DG1 GPU over pcie p2p. Signed-off-by: Thomas Hellström --- drivers/gpu/drm/xe/xe_svm.c | 50 ++++++++++++++++++++++++++++++++++--- drivers/gpu/drm/xe/xe_svm.h | 1 + 2 files changed, 48 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_svm.c b/drivers/gpu/drm/xe/xe_svm.c index 56c2c731be27..0b562b411fa4 100644 --- a/drivers/gpu/drm/xe/xe_svm.c +++ b/drivers/gpu/drm/xe/xe_svm.c @@ -3,6 +3,8 @@ * Copyright © 2024 Intel Corporation */ +#include + #include #include #include @@ -379,6 +381,25 @@ static u64 xe_page_to_dpa(struct page *page) return dpa; } +static u64 xe_page_to_pcie(struct page *page) +{ + struct xe_pagemap *xpagemap = xe_page_to_pagemap(page); + struct xe_vram_region *vr = xe_pagemap_to_vr(xpagemap); + u64 hpa_base = xpagemap->hpa_base; + u64 ioaddr; + struct xe_tile *tile = xe_vr_to_tile(vr); + u64 pfn = page_to_pfn(page); + u64 offset; + + xe_tile_assert(tile, is_device_private_page(page)); + xe_tile_assert(tile, (pfn << PAGE_SHIFT) >= hpa_base); + + offset = (pfn << PAGE_SHIFT) - hpa_base; + ioaddr = vr->io_start + offset; + + return ioaddr; +} + enum xe_svm_copy_dir { XE_SVM_COPY_TO_VRAM, XE_SVM_COPY_TO_SRAM, @@ -940,13 +961,27 @@ xe_drm_pagemap_device_map(struct drm_pagemap *dpagemap, addr = xe_page_to_dpa(page); prot = XE_INTERCONNECT_VRAM; } else { - addr = DMA_MAPPING_ERROR; - prot = 0; + addr = dma_map_resource(dev, + xe_page_to_pcie(page), + PAGE_SIZE << order, dir, + DMA_ATTR_SKIP_CPU_SYNC); + prot = XE_INTERCONNECT_P2P; } return drm_pagemap_device_addr_encode(addr, prot, order, dir); } +static void xe_drm_pagemap_device_unmap(struct drm_pagemap *dpagemap, + struct device *dev, + struct drm_pagemap_device_addr addr) +{ + if (addr.proto != XE_INTERCONNECT_P2P) + return; + + dma_unmap_resource(dev, addr.addr, PAGE_SIZE << addr.order, + addr.dir, DMA_ATTR_SKIP_CPU_SYNC); +} + static void xe_pagemap_fini(struct xe_pagemap *xpagemap) { struct dev_pagemap *pagemap = &xpagemap->pagemap; @@ -1004,13 +1039,22 @@ static bool xe_has_interconnect(struct drm_pagemap_peer *peer1, struct device *dev1 = xpagemap1->dpagemap.drm->dev; struct device *dev2 = xpagemap2->dpagemap.drm->dev; - return dev1 == dev2; + if (dev1 == dev2) + return true; + + /* Define this if your system can correctly identify pci_p2p capability */ +#ifdef XE_P2P_CAPABLE + return pci_p2pdma_distance(to_pci_dev(dev1), dev2, true) >= 0; +#else + return true; +#endif } static DRM_PAGEMAP_OWNER_LIST_DEFINE(xe_owner_list); static const struct drm_pagemap_ops xe_drm_pagemap_ops = { .device_map = xe_drm_pagemap_device_map, + .device_unmap = xe_drm_pagemap_device_unmap, .populate_mm = xe_drm_pagemap_populate_mm, .destroy = xe_pagemap_destroy, }; diff --git a/drivers/gpu/drm/xe/xe_svm.h b/drivers/gpu/drm/xe/xe_svm.h index 7c076c36c1c5..59b7a46f2bd9 100644 --- a/drivers/gpu/drm/xe/xe_svm.h +++ b/drivers/gpu/drm/xe/xe_svm.h @@ -13,6 +13,7 @@ #include #define XE_INTERCONNECT_VRAM DRM_INTERCONNECT_DRIVER +#define XE_INTERCONNECT_P2P (XE_INTERCONNECT_VRAM + 1) struct drm_device; struct drm_file;