From patchwork Fri Mar 28 18:11:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Lis X-Patchwork-Id: 14032357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2AECC28B20 for ; Fri, 28 Mar 2025 18:12:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6265410EA8B; Fri, 28 Mar 2025 18:12:09 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="diKJ/8SA"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id A760610EA8B for ; Fri, 28 Mar 2025 18:12:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743185525; x=1774721525; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T53QCUqkXcX9lHjwShX9lvvpE3mUBzCddMQKa6FhvsI=; b=diKJ/8SAOr0Rm0aMRdVFn4UsBF34ywdRANIdS/l7vZWUQuH+R7WqeA10 X7h4oHB0l5flNKOphjan3Kdo5S5ptQPTyULUlFLHLk5nUHz8O/9sOig+Q i+F5pVk9TXT7TMmK3EKfOibQFIXuZ5u6DPKD4t6STUYfc3VhUmFuNV9dH FY7A+XxX+RifcJdQVLfl1Y20IDXuA/lkaXNgcZwYVD+H6I1qEU0X0NxW6 fJK8u78XmXcQwDYWYMZeGyutBDlCBNIH3h3M1ZmkT//7EohbRKr8eKmq7 lnOMwXv0+8ahDvT29mXujDx8w6p6v9ZPajmmNVV8Roe0OKA/GNPQ7L3VI w==; X-CSE-ConnectionGUID: uQH/7wZ2RW2xFDSIq6OkUQ== X-CSE-MsgGUID: 6vb0Dg+/SuaPd0CTXGKVdQ== X-IronPort-AV: E=McAfee;i="6700,10204,11387"; a="48338806" X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="48338806" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2025 11:12:05 -0700 X-CSE-ConnectionGUID: QOQbcTgKQw62id9ZW8Yr3Q== X-CSE-MsgGUID: uy5ziifzTbm+0JRH8SpK1A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="130616373" Received: from gkczarna.igk.intel.com ([10.211.131.163]) by fmviesa004.fm.intel.com with ESMTP; 28 Mar 2025 11:12:03 -0700 From: Tomasz Lis To: dri-devel@lists.freedesktop.org Cc: Matthew Brost , Lucas De Marchi , =?utf-8?b?VmlsbGUgU3lyasOkbMOk?= , =?utf-8?q?Micha=C5=82_Winiarski?= , =?utf-8?q?M?= =?utf-8?q?icha=C5=82_Wajdeczko?= , =?utf-8?q?Pi?= =?utf-8?q?otr_Pi=C3=B3rkowski?= Subject: [PATCH v5 1/4] drm/xe/vf: Divide ballooning into allocation and insertion Date: Fri, 28 Mar 2025 19:11:57 +0100 Message-Id: <20250328181200.1011623-2-tomasz.lis@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250328181200.1011623-1-tomasz.lis@intel.com> References: <20250328181200.1011623-1-tomasz.lis@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The balloon nodes used to fill areas of GGTT inaccessible for a specific VF, were allocaten and inserted into GGTT within one function. This disallowed re-using the insertion part during VF migration recovery. This patch separates allocation (init/fini functs) from the insertion of balloons (balloon/deballoon functs). Locks are also moved to ensure calls from post-migration recovery worker will not cause a deadlock. Signed-off-by: Tomasz Lis --- drivers/gpu/drm/xe/xe_ggtt.c | 9 +--- drivers/gpu/drm/xe/xe_ggtt.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 64 ++++++++++++++++++++++------- 3 files changed, 52 insertions(+), 23 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c index 5fcb2b4c2c13..2d7456e37ef4 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.c +++ b/drivers/gpu/drm/xe/xe_ggtt.c @@ -452,9 +452,7 @@ int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end) node->base.start = start; node->base.size = end - start; - mutex_lock(&ggtt->lock); err = drm_mm_reserve_node(&ggtt->mm, &node->base); - mutex_unlock(&ggtt->lock); if (xe_gt_WARN(ggtt->tile->primary_gt, err, "Failed to balloon GGTT %#llx-%#llx (%pe)\n", @@ -477,16 +475,11 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) return; if (!drm_mm_node_allocated(&node->base)) - goto free_node; + return; xe_ggtt_dump_node(node->ggtt, &node->base, "remove-balloon"); - mutex_lock(&node->ggtt->lock); drm_mm_remove_node(&node->base); - mutex_unlock(&node->ggtt->lock); - -free_node: - xe_ggtt_node_fini(node); } /** diff --git a/drivers/gpu/drm/xe/xe_ggtt.h b/drivers/gpu/drm/xe/xe_ggtt.h index 27e7d67de004..fddb68f309d9 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.h +++ b/drivers/gpu/drm/xe/xe_ggtt.h @@ -18,6 +18,8 @@ void xe_ggtt_node_fini(struct xe_ggtt_node *node); int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 size); void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node); +int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt); +void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt); int xe_ggtt_node_insert(struct xe_ggtt_node *node, u32 size, u32 align); int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node, diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c index a439261bf4d7..9edbe34f45f4 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c @@ -560,26 +560,33 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) return gt->sriov.vf.self_config.lmem_size; } -static struct xe_ggtt_node * -vf_balloon_ggtt_node(struct xe_ggtt *ggtt, u64 start, u64 end) +static int +vf_balloon_ggtt_node(struct xe_ggtt *ggtt, struct xe_ggtt_node *node, + u64 start, u64 end) { - struct xe_ggtt_node *node; int err; - node = xe_ggtt_node_init(ggtt); if (IS_ERR(node)) - return node; + return PTR_ERR(node); err = xe_ggtt_node_insert_balloon(node, start, end); if (err) { - xe_ggtt_node_fini(node); - return ERR_PTR(err); + return err; } - return node; + return 0; } -static int vf_balloon_ggtt(struct xe_gt *gt) +static void xe_gt_sriov_vf_balloon_init(struct xe_gt *gt) +{ + struct xe_tile *tile = gt_to_tile(gt); + struct xe_ggtt *ggtt = tile->mem.ggtt; + + tile->sriov.vf.ggtt_balloon[0] = xe_ggtt_node_init(ggtt); + tile->sriov.vf.ggtt_balloon[1] = xe_ggtt_node_init(ggtt); +} + +int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt) { struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; struct xe_tile *tile = gt_to_tile(gt); @@ -589,6 +596,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt) xe_gt_assert(gt, IS_SRIOV_VF(xe)); xe_gt_assert(gt, !xe_gt_is_media_type(gt)); + lockdep_assert_held(&tile->mem.ggtt->lock); if (!config->ggtt_size) return -ENODATA; @@ -611,7 +619,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt) start = xe_wopcm_size(xe); end = config->ggtt_base; if (end != start) { - tile->sriov.vf.ggtt_balloon[0] = vf_balloon_ggtt_node(ggtt, start, end); + vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[0], start, end); if (IS_ERR(tile->sriov.vf.ggtt_balloon[0])) return PTR_ERR(tile->sriov.vf.ggtt_balloon[0]); } @@ -619,7 +627,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt) start = config->ggtt_base + config->ggtt_size; end = GUC_GGTT_TOP; if (end != start) { - tile->sriov.vf.ggtt_balloon[1] = vf_balloon_ggtt_node(ggtt, start, end); + vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[1], start, end); if (IS_ERR(tile->sriov.vf.ggtt_balloon[1])) { xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]); return PTR_ERR(tile->sriov.vf.ggtt_balloon[1]); @@ -629,15 +637,34 @@ static int vf_balloon_ggtt(struct xe_gt *gt) return 0; } -static void deballoon_ggtt(struct drm_device *drm, void *arg) +void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt) { - struct xe_tile *tile = arg; + struct xe_tile *tile = gt_to_tile(gt); xe_tile_assert(tile, IS_SRIOV_VF(tile_to_xe(tile))); + lockdep_assert_held(&tile->mem.ggtt->lock); xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[1]); xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]); } +static void xe_gt_sriov_vf_balloon_fini(struct xe_gt *gt) +{ + struct xe_tile *tile = gt_to_tile(gt); + + xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[1]); + xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[0]); +} + +static void deballoon_ggtt(struct drm_device *drm, void *arg) +{ + struct xe_tile *tile = arg; + + mutex_lock(&tile->mem.ggtt->lock); + xe_gt_sriov_vf_deballoon_ggtt(tile->primary_gt); + mutex_unlock(&tile->mem.ggtt->lock); + xe_gt_sriov_vf_balloon_fini(tile->primary_gt); +} + /** * xe_gt_sriov_vf_prepare_ggtt - Prepare a VF's GGTT configuration. * @gt: the &xe_gt @@ -650,14 +677,21 @@ int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt) { struct xe_tile *tile = gt_to_tile(gt); struct xe_device *xe = tile_to_xe(tile); + struct xe_ggtt *ggtt = tile->mem.ggtt; int err; if (xe_gt_is_media_type(gt)) return 0; - err = vf_balloon_ggtt(gt); - if (err) + xe_gt_sriov_vf_balloon_init(gt); + + mutex_lock(&ggtt->lock); + err = xe_gt_sriov_vf_balloon_ggtt(gt); + mutex_unlock(&ggtt->lock); + if (err) { + xe_gt_sriov_vf_balloon_fini(gt); return err; + } return drmm_add_action_or_reset(&xe->drm, deballoon_ggtt, tile); } From patchwork Fri Mar 28 18:11:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Lis X-Patchwork-Id: 14032359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A83D7C36010 for ; Fri, 28 Mar 2025 18:12:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E342110EA92; Fri, 28 Mar 2025 18:12:12 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="a8fE1XXh"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id E102510EA8B for ; Fri, 28 Mar 2025 18:12:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743185527; x=1774721527; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B0ZYK86iSHrdxJAxg5jbQ7rkd+AYOD/CzKyo2JahHCk=; b=a8fE1XXhUk3Yh+gSxqhfM3H/VSYCPWA3VZVQJaqRzVNR0rhM5tZ5bQ6T 7Bi782UrTufFCmMKsiYpx2zh2bfRBnpEApvV5DdGOqqPNwQ8pIpcwP+F3 ZCbIzB2h2yaFur+xu6D5fg5bauq0jZCc1nyYdzUNj72oZ9D3kRjbNLfJV tLdvN0wTHzetf2wEwVfSHwOtafAk9Ekr5HlLMj6isULEz4CEc+hqCTqlK skybsySJNlMr4znH7CSsdFUt0iA3jtbFHvalQMDeY3Y9Y93giYMw15nzr kXN9jQw+uTnejk/kLDd30KdkuciWq6hj4uA1nmaNPyS8Mb0i+AdF+FWpG g==; X-CSE-ConnectionGUID: pnCil1zcQceLglgXMokJOQ== X-CSE-MsgGUID: iret7uYRTlGmQpkqLIHMJA== X-IronPort-AV: E=McAfee;i="6700,10204,11387"; a="48338816" X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="48338816" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2025 11:12:07 -0700 X-CSE-ConnectionGUID: 3w4BQVq+QyizLhBsk8vMqw== X-CSE-MsgGUID: iBzGSgYnRFuj39NG/dxvOw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="130616382" Received: from gkczarna.igk.intel.com ([10.211.131.163]) by fmviesa004.fm.intel.com with ESMTP; 28 Mar 2025 11:12:05 -0700 From: Tomasz Lis To: dri-devel@lists.freedesktop.org Cc: Matthew Brost , Lucas De Marchi , =?utf-8?b?VmlsbGUgU3lyasOkbMOk?= , =?utf-8?q?Micha=C5=82_Winiarski?= , =?utf-8?q?M?= =?utf-8?q?icha=C5=82_Wajdeczko?= , =?utf-8?q?Pi?= =?utf-8?q?otr_Pi=C3=B3rkowski?= Subject: [PATCH v5 2/4] drm/xe/sriov: Shifting GGTT area post migration Date: Fri, 28 Mar 2025 19:11:58 +0100 Message-Id: <20250328181200.1011623-3-tomasz.lis@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250328181200.1011623-1-tomasz.lis@intel.com> References: <20250328181200.1011623-1-tomasz.lis@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We have only one GGTT for all IOV functions, with each VF having assigned a range of addresses for its use. After migration, a VF can receive a different range of addresses than it had initially. This implements shifting GGTT addresses within drm_mm nodes, so that VMAs stay valid after migration. This will make the driver use new addresses when accessing GGTT from the moment the shifting ends. By taking the ggtt->lock for the period of VMA fixups, this change also adds constraint on that mutex. Any locks used during the recovery cannot ever wait for hardware response - because after migration, the hardware will not do anything until fixups are finished. v2: Moved some functs to xe_ggtt.c; moved shift computation to just after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles (Michal) v3: Updated kerneldocs, removed unused funct, properly allocate balloning nodes if non existent v4: Re-used ballooning functions from VF init, used bool in place of standard error codes Signed-off-by: Tomasz Lis --- drivers/gpu/drm/xe/xe_ggtt.c | 33 +++++++++ drivers/gpu/drm/xe/xe_ggtt.h | 1 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 84 +++++++++++++++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 1 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 2 + drivers/gpu/drm/xe/xe_sriov_vf.c | 17 +++++ 6 files changed, 138 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c index 2d7456e37ef4..e9e6088104d9 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.c +++ b/drivers/gpu/drm/xe/xe_ggtt.c @@ -482,6 +482,39 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) drm_mm_remove_node(&node->base); } +/** + * xe_ggtt_node_shift_nodes - Shift GGTT nodes to adjust for a change in usable address range. + * @ggtt: the &xe_ggtt struct instance + * @shift: change to the location of area provisioned for current VF + */ +void xe_ggtt_node_shift_nodes(struct xe_ggtt *ggtt, s64 shift) +{ + struct drm_mm_node *node, *tmpn; + LIST_HEAD(temp_list_head); + int err; + + lockdep_assert_held(&ggtt->lock); + + /* + * Move nodes, from range previously assigned to this VF, to a temp list. + */ + drm_mm_for_each_node_safe(node, tmpn, &ggtt->mm) { + drm_mm_remove_node(node); + list_add(&node->node_list, &temp_list_head); + } + + /* + * Now the GGTT VM contains no nodes. We can re-add all VF nodes with + * shifted offsets. + */ + list_for_each_entry_safe(node, tmpn, &temp_list_head, node_list) { + list_del(&node->node_list); + node->start += shift; + err = drm_mm_reserve_node(&ggtt->mm, node); + xe_tile_assert(ggtt->tile, !err); + } +} + /** * xe_ggtt_node_insert_locked - Locked version to insert a &xe_ggtt_node into the GGTT * @node: the &xe_ggtt_node to be inserted diff --git a/drivers/gpu/drm/xe/xe_ggtt.h b/drivers/gpu/drm/xe/xe_ggtt.h index fddb68f309d9..557906413761 100644 --- a/drivers/gpu/drm/xe/xe_ggtt.h +++ b/drivers/gpu/drm/xe/xe_ggtt.h @@ -18,6 +18,7 @@ void xe_ggtt_node_fini(struct xe_ggtt_node *node); int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 size); void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node); +void xe_ggtt_node_shift_nodes(struct xe_ggtt *ggtt, s64 shift); int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt); void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt); diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c index 9edbe34f45f4..859294796aec 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf.c +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf.c @@ -415,6 +415,7 @@ static int vf_get_ggtt_info(struct xe_gt *gt) xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n", start, start + size - 1, size / SZ_1K); + config->ggtt_shift = start - (s64)config->ggtt_base; config->ggtt_base = start; config->ggtt_size = size; @@ -972,6 +973,89 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) return err; } +/** + * xe_gt_sriov_vf_fixup_ggtt_nodes - Shift GGTT allocations to match assigned range. + * @gt: the &xe_gt struct instance + * Return: True if fixups are necessary + * + * Since Global GTT is not virtualized, each VF has an assigned range + * within the global space. This range might have changed during migration, + * which requires all memory addresses pointing to GGTT to be shifted. + */ +bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt) +{ + struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; + struct xe_tile *tile = gt_to_tile(gt); + struct xe_ggtt *ggtt = tile->mem.ggtt; + s64 ggtt_shift; + + mutex_lock(&ggtt->lock); + ggtt_shift = config->ggtt_shift; + /* + * Move nodes, including balloons, from range previously assigned to this VF, + * into newly provisioned area. + * + * The balloon nodes are there to eliminate unavailable ranges from use: one + * reserves the GGTT area below the range for current VF, and another one + * reserves area above. + * + * Below is a GGTT layout of example VF, with a certain address range assigned to + * said VF, and inaccessible areas above and below: + * + * 0 4GiB + * |<--------------------------- Total GGTT size ----------------------------->| + * WOPCM GUC_TOP + * |<-------------- Area mappable by xe_ggtt instance ---------------->| + * + * +---+---------------------------------+----------+----------------------+---+ + * |\\\|/////////////////////////////////| VF mem |//////////////////////|\\\| + * +---+---------------------------------+----------+----------------------+---+ + * + * Hardware enforced access rules before migration: + * + * |<------- inaccessible for VF ------->||<-- inaccessible for VF ->| + * + * drm_mm nodes used for tracking allocations: + * + * |<----------- balloon ------------>|<- nodes->|<----- balloon ------>| + * + * After the migration, GGTT area assigned to the VF might have shifted, either + * to lower or to higher address. But we expect the total size and extra areas to + * be identical, as migration can only happen between matching platforms. + * Below is an example of GGTT layout of the VF after migration. Content of the + * GGTT for VF has been moved to a new area, and we receive its address from GuC: + * + * +---+----------------------+----------+---------------------------------+---+ + * |\\\|//////////////////////| VF mem |/////////////////////////////////|\\\| + * +---+----------------------+----------+---------------------------------+---+ + * + * Hardware enforced access rules after migration: + * + * |<- inaccessible for VF -->||<------- inaccessible for VF ------->| + * + * So the VF has a new slice of GGTT assigned, and during migration process, the + * memory content was copied to that new area. But the drm_mm nodes within xe kmd + * are still tracking allocations using the old addresses. The nodes within VF + * owned area have to be shifted, and balloon nodes need to be resized to + * properly mask out areas not owned by the VF. + * + * Fixed drm_mm nodes used for tracking allocations: + * + * |<------ balloon ------>|<- nodes->|<----------- balloon ----------->| + * + * Due to use of GPU profiles, we do not expect the old and new GGTT ares to + * overlap; but our node shifting will fix addresses properly regardless. + * + */ + if (ggtt_shift) { + xe_gt_sriov_vf_deballoon_ggtt(gt); + xe_ggtt_node_shift_nodes(ggtt, ggtt_shift); + xe_gt_sriov_vf_balloon_ggtt(gt); + } + mutex_unlock(&ggtt->lock); + return ggtt_shift != 0; +} + static int vf_runtime_reg_cmp(const void *a, const void *b) { const struct vf_runtime_reg *ra = a; diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf.h b/drivers/gpu/drm/xe/xe_gt_sriov_vf.h index ba6c5d74e326..26d3140bba91 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf.h +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf.h @@ -18,6 +18,7 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt); int xe_gt_sriov_vf_connect(struct xe_gt *gt); int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt); int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt); +bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt); int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt); void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt); diff --git a/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h b/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h index a57f13b5afcd..5ccbdf8d08b6 100644 --- a/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +++ b/drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h @@ -40,6 +40,8 @@ struct xe_gt_sriov_vf_selfconfig { u64 ggtt_base; /** @ggtt_size: assigned size of the GGTT region. */ u64 ggtt_size; + /** @ggtt_shift: difference in ggtt_base on last migration */ + s64 ggtt_shift; /** @lmem_size: assigned size of the LMEM. */ u64 lmem_size; /** @num_ctxs: assigned number of GuC submission context IDs. */ diff --git a/drivers/gpu/drm/xe/xe_sriov_vf.c b/drivers/gpu/drm/xe/xe_sriov_vf.c index c1275e64aa9c..2eb6b8d8a217 100644 --- a/drivers/gpu/drm/xe/xe_sriov_vf.c +++ b/drivers/gpu/drm/xe/xe_sriov_vf.c @@ -7,6 +7,7 @@ #include "xe_assert.h" #include "xe_device.h" +#include "xe_gt.h" #include "xe_gt_sriov_printk.h" #include "xe_gt_sriov_vf.h" #include "xe_pm.h" @@ -170,6 +171,20 @@ static bool vf_post_migration_imminent(struct xe_device *xe) work_pending(&xe->sriov.vf.migration.worker); } +static bool vf_post_migration_fixup_ggtt_nodes(struct xe_device *xe) +{ + struct xe_tile *tile; + unsigned int id; + bool need_fixups = false; + + for_each_tile(tile, xe, id) { + struct xe_gt *gt = tile->primary_gt; + + need_fixups |= xe_gt_sriov_vf_fixup_ggtt_nodes(gt); + } + return need_fixups; +} + /* * Notify all GuCs about resource fixups apply finished. */ @@ -191,6 +206,7 @@ static void vf_post_migration_notify_resfix_done(struct xe_device *xe) static void vf_post_migration_recovery(struct xe_device *xe) { + bool need_fixups; int err; drm_dbg(&xe->drm, "migration recovery in progress\n"); @@ -201,6 +217,7 @@ static void vf_post_migration_recovery(struct xe_device *xe) if (unlikely(err)) goto fail; + need_fixups = vf_post_migration_fixup_ggtt_nodes(xe); /* FIXME: add the recovery steps */ vf_post_migration_notify_resfix_done(xe); xe_pm_runtime_put(xe); From patchwork Fri Mar 28 18:11:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Lis X-Patchwork-Id: 14032360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 537FBC36013 for ; Fri, 28 Mar 2025 18:12:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 76EA810EA99; Fri, 28 Mar 2025 18:12:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="YmXMnE6s"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id A3EF710EA92 for ; Fri, 28 Mar 2025 18:12:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743185529; x=1774721529; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5x1XIpgIJ46XyaNxiDv5NuPUYSUhdlj95ULLyabr2S4=; b=YmXMnE6sUjHBxOCSqxag8Dvpk4tvVc0uQ3BQraKKI0MpA1YPE650LvCg 3FIjvZ9fMjOZr5A2+TrsMfvwtlOlgzRAiEl6w0/iiIMsTidfbV7AyPPST U9ldkgUGDfQJt0P5RxBUcUzXa6rdf3NxhJyBPesUhhb26aHgKsDsMrZ/2 hlhGpSTsi7rjQ3CQLghLyky4/y0aDuTZZcp9i+/Rk/J+pMJTssGllDr/c 9ZPuX46EaZBogX5526lkfnQ5RpJ3lqrBH5lqkO31PpHlnH5pycEFhvS+G ZAibMdAQr9Kjz+XY5ScnSOcIzLsDneEirVKlC0/dDLc/qT47p4yKOoUA9 g==; X-CSE-ConnectionGUID: OhmCV77DRku+Ubq8NSPUjw== X-CSE-MsgGUID: u2ouWkdQT52utHln+PeMbg== X-IronPort-AV: E=McAfee;i="6700,10204,11387"; a="48338818" X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="48338818" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2025 11:12:09 -0700 X-CSE-ConnectionGUID: WfE+BXRiTQKAdubv53RmCQ== X-CSE-MsgGUID: ui1cipAySDSplbfdXKeP/A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="130616386" Received: from gkczarna.igk.intel.com ([10.211.131.163]) by fmviesa004.fm.intel.com with ESMTP; 28 Mar 2025 11:12:08 -0700 From: Tomasz Lis To: dri-devel@lists.freedesktop.org Cc: Matthew Brost , Lucas De Marchi , =?utf-8?b?VmlsbGUgU3lyasOkbMOk?= , =?utf-8?q?Micha=C5=82_Winiarski?= , =?utf-8?q?M?= =?utf-8?q?icha=C5=82_Wajdeczko?= , =?utf-8?q?Pi?= =?utf-8?q?otr_Pi=C3=B3rkowski?= Subject: [PATCH v5 3/4] drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G Date: Fri, 28 Mar 2025 19:11:59 +0100 Message-Id: <20250328181200.1011623-4-tomasz.lis@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250328181200.1011623-1-tomasz.lis@intel.com> References: <20250328181200.1011623-1-tomasz.lis@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some GuC messages are constructed with incrementing dword counter rather than referencing specific DWORDs, as described in GuC interface specification. This change introduces the definitions of DWORD numbers for parameters which will need to be referenced in a CTB parser to be added in a following patch. To ensure correctness of these DWORDs, verification in form of asserts was added to the message construction code. Signed-off-by: Tomasz Lis --- drivers/gpu/drm/xe/abi/guc_actions_abi.h | 7 +++++++ drivers/gpu/drm/xe/xe_guc_submit.c | 4 ++++ 2 files changed, 11 insertions(+) diff --git a/drivers/gpu/drm/xe/abi/guc_actions_abi.h b/drivers/gpu/drm/xe/abi/guc_actions_abi.h index 448afb86e05c..86bd4b092c7c 100644 --- a/drivers/gpu/drm/xe/abi/guc_actions_abi.h +++ b/drivers/gpu/drm/xe/abi/guc_actions_abi.h @@ -161,6 +161,13 @@ enum xe_guc_preempt_options { XE_GUC_PREEMPT_OPTION_DROP_SUBMIT_Q = 0x8, }; +enum xe_guc_register_context_multi_lrc_param_offsets { + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC = 5, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE = 7, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN = 10, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA = 11 +}; + enum xe_guc_report_status { XE_GUC_REPORT_STATUS_UNKNOWN = 0x0, XE_GUC_REPORT_STATUS_ACKED = 0x1, diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c index 31bc2022bfc2..3c8574e2c1f3 100644 --- a/drivers/gpu/drm/xe/xe_guc_submit.c +++ b/drivers/gpu/drm/xe/xe_guc_submit.c @@ -469,12 +469,16 @@ static void __register_mlrc_exec_queue(struct xe_guc *guc, action[len++] = info->context_idx; action[len++] = info->engine_class; action[len++] = info->engine_submit_mask; + xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC); action[len++] = info->wq_desc_lo; action[len++] = info->wq_desc_hi; + xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE); action[len++] = info->wq_base_lo; action[len++] = info->wq_base_hi; action[len++] = info->wq_size; + xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN); action[len++] = q->width; + xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA); action[len++] = info->hwlrca_lo; action[len++] = info->hwlrca_hi; From patchwork Fri Mar 28 18:12:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomasz Lis X-Patchwork-Id: 14032358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65E89C28B20 for ; Fri, 28 Mar 2025 18:12:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C881E10EA8E; Fri, 28 Mar 2025 18:12:12 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="Ln0r9JB+"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9F1AC10EA8E for ; Fri, 28 Mar 2025 18:12:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743185531; x=1774721531; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lO5sWc/yHfoUVr0xVOV/MpPdVAaIA0kdRoiUvtqgZCY=; b=Ln0r9JB+a2iD9Hb7VUrHJij9VTo07drOhGcLHmqfY40xkyOj3elo9zBV gh8XLcOGl2x3efnJgomGaqcEZ8h9s1Q3K9MWjSC6Qj2ZN+aL8xFWTNrHz SBdlb62qRAH4U0H1vChQPiatFruBQCdIu6tI02uq8p4VfqxLoLZyCVBlr 5f9wuiRCAO6XMyN+ekippJVv0XaKWbhhq7FYUfX/I8w/hDEO2LhqueG/Q X5XDeb1vlW+zn2Y2sAlh9OxCjHPD7BBSAFhJ57KUredm4bne5jwwKbgwg RjHqYinJYufFgPIDfFiqCSgzr2bvoF3TG6L1fWExGF84vVeoOcMjl++qS g==; X-CSE-ConnectionGUID: 26yQ1RUyT2ikWlgPqVj6Gg== X-CSE-MsgGUID: P5t3rkrkTGiLi5HB20PmjQ== X-IronPort-AV: E=McAfee;i="6700,10204,11387"; a="48338820" X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="48338820" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2025 11:12:11 -0700 X-CSE-ConnectionGUID: 4BZwDfA7RBi+6NuMweTvgg== X-CSE-MsgGUID: Rlq2cLBNQb2uFFHKf6UdMw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.14,284,1736841600"; d="scan'208";a="130616395" Received: from gkczarna.igk.intel.com ([10.211.131.163]) by fmviesa004.fm.intel.com with ESMTP; 28 Mar 2025 11:12:09 -0700 From: Tomasz Lis To: dri-devel@lists.freedesktop.org Cc: Matthew Brost , Lucas De Marchi , =?utf-8?b?VmlsbGUgU3lyasOkbMOk?= , =?utf-8?q?Micha=C5=82_Winiarski?= , =?utf-8?q?M?= =?utf-8?q?icha=C5=82_Wajdeczko?= , =?utf-8?q?Pi?= =?utf-8?q?otr_Pi=C3=B3rkowski?= Subject: [PATCH v5 4/4] drm/xe/vf: Fixup CTB send buffer messages after migration Date: Fri, 28 Mar 2025 19:12:00 +0100 Message-Id: <20250328181200.1011623-5-tomasz.lis@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20250328181200.1011623-1-tomasz.lis@intel.com> References: <20250328181200.1011623-1-tomasz.lis@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" During post-migration recovery of a VF, it is necessary to update GGTT references included in messages which are going to be sent to GuC. GuC will start consuming messages after VF KMD will inform it about fixups being done; before that, the VF KMD is expected to update any H2G messages which are already in send buffer but were not consumed by GuC. Only a small subset of messages allowed for VFs have GGTT references in them. This patch adds the functionality to parse the CTB send ring buffer and shift addresses contained within. While fixing the CTB content, ct->lock is not taken. This means the only barrier taken remains GGTT address lock - which is ok, because only requests with GGTT addresses matter, but it also means tail changes can happen during the CTB fixups execution (which may be ignored as any new messages will not have anything to fix). The GGTT address locking will be introduced in a future series. v2: removed storing shift as that's now done in VMA nodes patch; macros to inlines; warns to asserts; log messages fixes (Michal) v3: removed inline keywords, enums for offsets in CTB messages, less error messages, if return unused then made functs void (Michal) v4: update the cached head before starting fixups v5: removed/updated comments, wrapped lines, converted assert into error, enums for offsets to separate patch, reused xe_map_rd Signed-off-by: Tomasz Lis --- drivers/gpu/drm/xe/xe_guc_ct.c | 142 +++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_ct.h | 2 + drivers/gpu/drm/xe/xe_sriov_vf.c | 25 ++++++ 3 files changed, 169 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c index 686fe664c20d..6c80106e5e8b 100644 --- a/drivers/gpu/drm/xe/xe_guc_ct.c +++ b/drivers/gpu/drm/xe/xe_guc_ct.c @@ -84,6 +84,8 @@ struct g2h_fence { bool done; }; +#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo))) + static void g2h_fence_init(struct g2h_fence *g2h_fence, u32 *response_buffer) { g2h_fence->response_buffer = response_buffer; @@ -1622,6 +1624,146 @@ static void g2h_worker_func(struct work_struct *w) receive_g2h(ct); } +static u32 xe_map_rd_u32(struct xe_device *xe, struct iosys_map *cmds, + u32 head, u32 pos) +{ + return xe_map_rd(xe, cmds, (head + pos) * sizeof(u32), u32); +} + +static void xe_map_fixup_u64(struct xe_device *xe, struct iosys_map *cmds, + u32 head, u32 pos, s64 shift) +{ + u32 msg[2]; + u64 offset; + + xe_map_memcpy_from(xe, msg, cmds, (head + pos) * sizeof(u32), + 2 * sizeof(u32)); + offset = make_u64(msg[1], msg[0]); + offset += shift; + msg[0] = lower_32_bits(offset); + msg[1] = upper_32_bits(offset); + xe_map_memcpy_to(xe, cmds, (head + pos) * sizeof(u32), msg, 2 * sizeof(u32)); +} + +/* + * Shift any GGTT addresses within a single message left within CTB from + * before post-migration recovery. + * @ct: pointer to CT struct of the target GuC + * @cmds: iomap buffer containing CT messages + * @head: start of the target message within the buffer + * @len: length of the target message + * @size: size of the commands buffer + * @shift: the address shift to be added to each GGTT reference + */ +static void ct_update_addresses_in_message(struct xe_guc_ct *ct, + struct iosys_map *cmds, u32 head, + u32 len, u32 size, s64 shift) +{ + struct xe_device *xe = ct_to_xe(ct); + u32 action, i, n; + u32 msg[1]; + + xe_map_memcpy_from(xe, msg, cmds, head * sizeof(u32), + 1 * sizeof(u32)); + action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]); + switch (action) { + case XE_GUC_ACTION_REGISTER_CONTEXT: + case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC: + xe_map_fixup_u64(xe, cmds, head, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC, shift); + xe_map_fixup_u64(xe, cmds, head, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE, shift); + if (action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC) { + n = xe_map_rd_u32(xe, cmds, head, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN); + for (i = 0; i < n; i++) + xe_map_fixup_u64(xe, cmds, head, + XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA + + 2 * i, shift); + } else { + xe_map_fixup_u64(xe, cmds, head, 10, shift); + } + break; + default: + break; + } +} + +static int ct_update_addresses_in_buffer(struct xe_guc_ct *ct, + struct guc_ctb *h2g, + s64 shift, u32 *mhead, s32 avail) +{ + struct xe_device *xe = ct_to_xe(ct); + u32 head = *mhead; + u32 size = h2g->info.size; + u32 msg[1]; + u32 len; + + /* Read header */ + xe_map_memcpy_from(xe, msg, &h2g->cmds, sizeof(u32) * head, + sizeof(u32)); + len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN; + + if (unlikely(len > (u32)avail)) { + xe_gt_err(ct_to_gt(ct), "H2G channel broken on read, avail=%d, len=%d, fixups skipped\n", + avail, len); + return 0; + } + + head = (head + 1) % size; + ct_update_addresses_in_message(ct, &h2g->cmds, head, len - 1, size, shift); + *mhead = (head + len - 1) % size; + + return avail - len; +} + +/** + * xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages by updating + * GGTT offsets in their payloads. + * @ct: pointer to CT struct of the target GuC + * @ggtt_shift: shift to be added to all GGTT addresses within the CTB + */ +void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift) +{ + struct xe_guc *guc = ct_to_guc(ct); + struct xe_gt *gt = guc_to_gt(guc); + struct guc_ctb *h2g = &ct->ctbs.h2g; + u32 head, tail, size; + s32 avail; + + if (unlikely(h2g->info.broken)) + return; + + h2g->info.head = desc_read(ct_to_xe(ct), h2g, head); + head = h2g->info.head; + tail = READ_ONCE(h2g->info.tail); + size = h2g->info.size; + + if (unlikely(head > size)) + goto corrupted; + + if (unlikely(tail >= size)) + goto corrupted; + + avail = tail - head; + + /* beware of buffer wrap case */ + if (unlikely(avail < 0)) + avail += size; + xe_gt_dbg(gt, "available %d (%u:%u:%u)\n", avail, head, tail, size); + xe_gt_assert(gt, avail >= 0); + + while (avail > 0) + avail = ct_update_addresses_in_buffer(ct, h2g, ggtt_shift, &head, avail); + + return; + +corrupted: + xe_gt_err(gt, "Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n", + head, tail, size); + h2g->info.broken = true; +} + static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc(struct xe_guc_ct *ct, bool atomic, bool want_ctb) { diff --git a/drivers/gpu/drm/xe/xe_guc_ct.h b/drivers/gpu/drm/xe/xe_guc_ct.h index 82c4ae458dda..5649bda82823 100644 --- a/drivers/gpu/drm/xe/xe_guc_ct.h +++ b/drivers/gpu/drm/xe/xe_guc_ct.h @@ -22,6 +22,8 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_pr void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot); void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p, bool want_ctb); +void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift); + static inline bool xe_guc_ct_enabled(struct xe_guc_ct *ct) { return ct->state == XE_GUC_CT_STATE_ENABLED; diff --git a/drivers/gpu/drm/xe/xe_sriov_vf.c b/drivers/gpu/drm/xe/xe_sriov_vf.c index 2eb6b8d8a217..9a1b578fdd03 100644 --- a/drivers/gpu/drm/xe/xe_sriov_vf.c +++ b/drivers/gpu/drm/xe/xe_sriov_vf.c @@ -10,6 +10,7 @@ #include "xe_gt.h" #include "xe_gt_sriov_printk.h" #include "xe_gt_sriov_vf.h" +#include "xe_guc_ct.h" #include "xe_pm.h" #include "xe_sriov.h" #include "xe_sriov_printk.h" @@ -158,6 +159,27 @@ static int vf_post_migration_requery_guc(struct xe_device *xe) return ret; } +static s32 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt) +{ + struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; + + return config->ggtt_shift; +} + +static void vf_post_migration_fixup_ctb(struct xe_device *xe) +{ + struct xe_gt *gt; + unsigned int id; + + xe_assert(xe, IS_SRIOV_VF(xe)); + + for_each_gt(gt, xe, id) { + s32 shift = xe_gt_sriov_vf_ggtt_shift(gt); + + xe_guc_ct_fixup_messages_with_ggtt(>->uc.guc.ct, shift); + } +} + /* * vf_post_migration_imminent - Check if post-restore recovery is coming. * @xe: the &xe_device struct instance @@ -219,6 +241,9 @@ static void vf_post_migration_recovery(struct xe_device *xe) need_fixups = vf_post_migration_fixup_ggtt_nodes(xe); /* FIXME: add the recovery steps */ + if (need_fixups) + vf_post_migration_fixup_ctb(xe); + vf_post_migration_notify_resfix_done(xe); xe_pm_runtime_put(xe); drm_notice(&xe->drm, "migration recovery ended\n");