From patchwork Mon Mar 20 20:23:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andi Shyti X-Patchwork-Id: 13181825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD980C6FD1D for ; Mon, 20 Mar 2023 20:24:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 40A0310E65E; Mon, 20 Mar 2023 20:24:06 +0000 (UTC) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0341D10E65E; Mon, 20 Mar 2023 20:24:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679343844; x=1710879844; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=h5p7kbd5yUNPfOeWM80IMyGXZBKhemAYLileuX/U6yg=; b=mdpmikIXdD1euISw0VODNHGWa1ZwZgD8prq9SG0v9iGTYkPWUTJgE/Ou HJv6BuuwtvaL6EIvRN/Fp7dCCu7iG/jJULxcPketWG36Qd/G2o6cZvJ0h K5q1JzoniVLaYTIBNfH+xu+Jd/KqGnZSmZlo8IcCrCgfm0O8hNmFiLPXF BVGICWRlUvUyOdoZ2k+Cf7A0DRp0I4MqOUxSGLoz3QwN7HjnGz9nY3Fbq QX82E5OuwpoG4+E3fJsD+vhw8VWIKjDJNqUXWrNd6FiwiltEU0YECQpSl odkFL37VIsfo9bgc5eew/YPgEpq4MZCdmEdK00iqDvghgKsl3OFVwlYlr A==; X-IronPort-AV: E=McAfee;i="6600,9927,10655"; a="401343212" X-IronPort-AV: E=Sophos;i="5.98,276,1673942400"; d="scan'208";a="401343212" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Mar 2023 13:24:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10655"; a="805025615" X-IronPort-AV: E=Sophos;i="5.98,276,1673942400"; d="scan'208";a="805025615" Received: from ivolodix-mobl.ccr.corp.intel.com (HELO intel.com) ([10.252.62.92]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Mar 2023 13:24:01 -0700 From: Andi Shyti To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Matt Roper Date: Mon, 20 Mar 2023 21:23:25 +0100 Message-Id: <20230320202326.296498-2-andi.shyti@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320202326.296498-1-andi.shyti@linux.intel.com> References: <20230320202326.296498-1-andi.shyti@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/2] drm/i915: Sanitycheck MMIO access early in driver load X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Matt Roper We occasionally see the PCI device in a non-accessible state at the point the driver is loaded. When this happens, all BAR accesses will read back as 0xFFFFFFFF. Rather than reading registers and misinterpreting their (invalid) values, let's specifically check for 0xFFFFFFFF in a register that cannot have that value to see if the device is accessible. Signed-off-by: Matt Roper Cc: Mika Kuoppala Signed-off-by: Andi Shyti --- drivers/gpu/drm/i915/intel_uncore.c | 34 +++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c index e1e1f34490c8e..14ec45e6facfa 100644 --- a/drivers/gpu/drm/i915/intel_uncore.c +++ b/drivers/gpu/drm/i915/intel_uncore.c @@ -2602,11 +2602,45 @@ static int uncore_forcewake_init(struct intel_uncore *uncore) return 0; } +static int sanity_check_mmio_access(struct intel_uncore *uncore) +{ + struct drm_i915_private *i915 = uncore->i915; + + if (GRAPHICS_VER(i915) < 8) + return 0; + + /* + * Sanitycheck that MMIO access to the device is working properly. If + * the CPU is unable to communcate with a PCI device, BAR reads will + * return 0xFFFFFFFF. Let's make sure the device isn't in this state + * before we start trying to access registers. + * + * We use the primary GT's forcewake register as our guinea pig since + * it's been around since HSW and it's a masked register so the upper + * 16 bits can never read back as 1's if device access is operating + * properly. + * + * If MMIO isn't working, we'll wait up to 2 seconds to see if it + * recovers, then give up. + */ +#define COND (__raw_uncore_read32(uncore, FORCEWAKE_MT) != ~0) + if (wait_for(COND, 2000) == -ETIMEDOUT) { + drm_err(&i915->drm, "Device is non-operational; MMIO access returns 0xFFFFFFFF!\n"); + return -EIO; + } + + return 0; +} + int intel_uncore_init_mmio(struct intel_uncore *uncore) { struct drm_i915_private *i915 = uncore->i915; int ret; + ret = sanity_check_mmio_access(uncore); + if (ret) + return ret; + /* * The boot firmware initializes local memory and assesses its health. * If memory training fails, the punit will have been instructed to From patchwork Mon Mar 20 20:23:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andi Shyti X-Patchwork-Id: 13181826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 76B8AC6FD1C for ; Mon, 20 Mar 2023 20:24:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0C58910E658; Mon, 20 Mar 2023 20:24:11 +0000 (UTC) Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9E2F810E64E; Mon, 20 Mar 2023 20:24:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679343848; x=1710879848; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m3dCqcFyFG/fACxCjySSjXusWYgaQZyRy/40ttC+ldw=; b=gHsyW9VTAenZQ83a48NtsVflvfsw8rQzrZLdFDRMW8XXK6JK3K557hU1 cDQn4CDyfdWOfaAI/MaMtDFP4exdkNqC5fxzJl4u4z8yX67NFpXaJQzhT ID1hire1PpUaT6LjByBqeqZeJaRz/DX+aWSPDkwTXHM4J3qkU66UUyiQV Pa4fi8kmPjeFXVw5JVMD9lO+0rCEFasko/uLLEh+jXu2moyGzi59OLyp6 W75dEMsRPbzORtarXBmLy9EV4Q+dtENRorYSMdTT4XporSx+oD5pO056l aNLmXgolxvsFCevhU+0AyaYMLUgEPDulaIxFze1ma0gA3IgokiWhL9+Al w==; X-IronPort-AV: E=McAfee;i="6600,9927,10655"; a="401343263" X-IronPort-AV: E=Sophos;i="5.98,276,1673942400"; d="scan'208";a="401343263" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Mar 2023 13:24:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10655"; a="805025626" X-IronPort-AV: E=Sophos;i="5.98,276,1673942400"; d="scan'208";a="805025626" Received: from ivolodix-mobl.ccr.corp.intel.com (HELO intel.com) ([10.252.62.92]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Mar 2023 13:24:06 -0700 From: Andi Shyti To: intel-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Matt Roper Date: Mon, 20 Mar 2023 21:23:26 +0100 Message-Id: <20230320202326.296498-3-andi.shyti@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320202326.296498-1-andi.shyti@linux.intel.com> References: <20230320202326.296498-1-andi.shyti@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/2] drm/i915: Check for unreliable MMIO during forcewake X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andi Shyti Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Matt Roper Although we now sanitycheck MMIO access during driver load to make sure the MMIO BAR isn't returning all 0xFFFFFFFF, there have been a few cases where (temporarily?) unreliable MMIO access has happened after GPU resets or power events. We'll often notice this on our next GT register access since forcewake handling will fail; let's change our handling slightly so that when this happens we print a more meaningful message clarifying that the problem is the MMIO access, not forcewake specifically. Signed-off-by: Matt Roper Cc: Mika Kuoppala Signed-off-by: Andi Shyti --- drivers/gpu/drm/i915/intel_uncore.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c index 14ec45e6facfa..8b40a9772df5f 100644 --- a/drivers/gpu/drm/i915/intel_uncore.c +++ b/drivers/gpu/drm/i915/intel_uncore.c @@ -178,9 +178,15 @@ static inline void fw_domain_wait_ack_clear(const struct intel_uncore_forcewake_domain *d) { if (wait_ack_clear(d, FORCEWAKE_KERNEL)) { - drm_err(&d->uncore->i915->drm, - "%s: timed out waiting for forcewake ack to clear.\n", - intel_uncore_forcewake_domain_to_str(d->id)); + if (fw_ack(d) == ~0) + drm_err(&d->uncore->i915->drm, + "%s: MMIO unreliable (forcewake register returns 0xFFFFFFFF)!\n", + intel_uncore_forcewake_domain_to_str(d->id)); + else + drm_err(&d->uncore->i915->drm, + "%s: timed out waiting for forcewake ack to clear.\n", + intel_uncore_forcewake_domain_to_str(d->id)); + add_taint_for_CI(d->uncore->i915, TAINT_WARN); /* CI now unreliable */ } }