From patchwork Wed Sep 20 19:56:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Serge Semin X-Patchwork-Id: 13393399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 541F9C04FF8 for ; Wed, 20 Sep 2023 19:59:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mZU5pL8hURTdbZvvZYThRQVceorcpR8Rz40DPYPOKNo=; b=xpNDh2YlzNZRzY qcxMJamE58lL9PhJyQ4Cv4xcU/0ZJDLYK3iKOTDnw0TPlIqJrTglkI6KjgtuDzYV2HCb8S6by+LIa 8POHidpKAqYC4bijE4Y+Chw1Mr8zi5FUaII0x/8sQlo9hEDig7QlvN1W66YSqQnKrj+A3SiQjzM90 Cz3QaMIkkQjN9teYyclAHWZbGFEid5uZ6Esjme6V/b3ytKH0u6AkTkIsqab2NzVKskXtlrxAfUvot 2fArnGmgPNLfweIUykQhuPKNZ6fVucm3IKATYMkRWIXz3Jdreakgtt+UbT6FeC8G/Nc6/DosWITU8 0KLHx/nsIoZ04NzFNGOg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qj3LN-00452p-1U; Wed, 20 Sep 2023 19:58:41 +0000 Received: from mail-lj1-x22e.google.com ([2a00:1450:4864:20::22e]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qj3L7-0044nf-1V for linux-arm-kernel@lists.infradead.org; Wed, 20 Sep 2023 19:58:29 +0000 Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-2bfc1d8f2d2so1846441fa.0 for ; Wed, 20 Sep 2023 12:58:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695239901; x=1695844701; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f2m14abObIwVOTnzzN0qiVIdW0RP4getGDnrkVkhG8M=; b=Cctb3F7WRvkO5RLb+vq0a2rXin3c9Um7Sr1nUnM5WwRY/6xA0Zjk4HDa4pru0D1BMe qI4ChM9Z+FHxNN2/TL8bxGQe6Oc7prNWrBWf8K0AkSVPxGF6i3Hol0cCpCHY1dBgTlUp 6z8eislG6e8Gy8f8wF3VAQM4vlPQH5SOzSFN/YxVPogdBMJfxSlMRZTMUen9i+dXVFPd Aqdq2yvsPkONqxA6hKCEeoXDnkDIge3LAKlkWqPwr8rJXCwSDd5Z5EAr3wJ2YdP+8vFU v3ybostBnV6K0ZqqN+/vUsuvIECswvSfW0G9ejvhvbsqN7alShon52EbvkHu9p+EZcXm mHwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695239901; x=1695844701; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f2m14abObIwVOTnzzN0qiVIdW0RP4getGDnrkVkhG8M=; b=BzazvPIvdxVLt9hZXwicDCRDgHi25xhHehUxNv4IH8WQy1DtL7hGa2VbxwxwWAe4qt gsnF/dQBkVDPFUs7C4JGBJFfJZLgfU78oO0C6A2bfvlActtu9uFgHHe+fTwSlQcvRRZI LJIsOGhEiVf2BUaHAHqQLygBkCwBAIjoP0WhDujTbL/2K/9I9kh9y7356JdphecAUVEx SAdzSxuHHtG0XiaWM/K0om55WmOV68JOsBRwaF09VaEwDJFr/Fu5X+HuZYqyj6ul/Sce 5R8MAMsMzqn4vZrI72y0NOyqEUuxXqw4ZZPILcS89xd9a24F281X/5gwBf0w1yNoeiCF foAg== X-Gm-Message-State: AOJu0YykIbWrzJkaBDImrz6Ry0kJHZv8hdo7EKAqKNy2cBfGSIU6l1UH BXfA1FEEx3BHN0e0rmMUnmI= X-Google-Smtp-Source: AGHT+IHCrMsfMt5QfQ0loPlFFm/IKiGzFXuv6BbiZlH2IIqx+sLJ9I//Hq2+6d3lWNiVT6f0EMBp6w== X-Received: by 2002:a2e:3513:0:b0:2bd:1615:f9f9 with SMTP id z19-20020a2e3513000000b002bd1615f9f9mr1389758ljz.0.1695239901337; Wed, 20 Sep 2023 12:58:21 -0700 (PDT) Received: from localhost ([85.140.0.70]) by smtp.gmail.com with ESMTPSA id v8-20020a2ea448000000b002c128e45245sm157523ljn.23.2023.09.20.12.58.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Sep 2023 12:58:20 -0700 (PDT) From: Serge Semin To: Michal Simek , Alexander Stein , Borislav Petkov , Tony Luck , James Morse , Mauro Carvalho Chehab , Robert Richter , Rob Herring , Krzysztof Kozlowski Cc: Serge Semin , Punnaiah Choudary Kalluri , Dinh Nguyen , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 11/13] EDAC/synopsys: Add ECC Scrubber support Date: Wed, 20 Sep 2023 22:56:42 +0300 Message-ID: <20230920195720.32047-12-fancer.lancer@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230920195720.32047-1-fancer.lancer@gmail.com> References: <20230920195720.32047-1-fancer.lancer@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230920_125825_636379_8B9ED306 X-CRM114-Status: GOOD ( 37.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DW uMCTL2 DDR controller IP-core can by synthesized with an embedded Scrubber engine. The ECC Scrubber (SBR) is a block which initiates periodic background burst read commands to the DDRC and further towards the DDR memory in an attempt to trigger Correctable or Uncorrectable errors. If a Correctable error is detected the ECC Scrub feature will execute the Read-Modify-Write (RMW) procedure in order to fix the ECC. In case of the Uncorrectable error it will be just reported as the corresponding IRQ event. So it's definitely very useful feature. Let's add it to the driver then especially seeing the MCI core already has some infrastructure for it. First of all the Scrubber clock needs to be enabled if one is supplied otherwise the engine won't work. Secondly the Scrubber engine support needs to be detected. Alas there is no any special CSR indicating whether the DW uMCTL2 DDRC IP-core has been synthesized with one embedded. Instead implement the detection procedure based on the Scrubber-specific CSRs writability. So if the SBRWDATA0 CSR is writable then the CSR exists, which means the Scrubber is available, otherwise the capability will be considered as absent. Thirdly the MCI core provides two callbacks utilized for the Scrubber tuning: set the Scrubber bandwidth in bytes, which can also be used to disable the periodic scrubbing; get the Scrubber bandwidth (zero if disabled). Both of them can be implemented by using the Scrubber CSRs the controller provides. In particular aside with the back-to-back periodic reads the Scrubber provides a way to delay the next read command for the predefined set of 512's Core/Scrubber clock cycles. It can be used to change the Scrubber bandwidth from the DDR maximal bandwidth (no delay) to up to (0x1FFF * 512) Core/Scrubber clock cycles (see the inline comments for details and utilized formulae). Note the Scrubber clock must be synchronous to the Core clock by the controller design so use the Core clock rate for the calculations. Pleas also note if no Core clock specified the Scrubber will still be supported, but the bandwidth will be used directly to calculate the Scrubber reads interval. The back-to-back reads mode in this case will be indicated by the INT_MAX bandwidth. Fourthly the back-to-back scrubbing most likely will cause the significant system performance drop. The manual says that it has been added to the controller for the initial SDRAM initialization and the fast SDRAM scrubbing after getting out of the low-power state. In anyway it is supposed to be enabled only for a single SDRAM pass. So preserve that semantic here to avoid the system lagging and disable the back-to-back scrubbing in the Scrubber Done IRQ handler after the Scrubbing work is done. Finally the denoted scrub-rate callbacks and the SCRUB_FLAG_HW_PROG and SCRUB_FLAG_HW_TUN flags will be set to the MCI descriptor based on the detected Scrubber capability. So no capability - no flags and no callbacks. Signed-off-by: Serge Semin --- Changelog v4: - Use div_u64() instead of do_div(). - Use FIELD_MAX() instead of open-coding the bitwise shift to find the max field value. --- drivers/edac/synopsys_edac.c | 299 +++++++++++++++++++++++++++++++++++ 1 file changed, 299 insertions(+) diff --git a/drivers/edac/synopsys_edac.c b/drivers/edac/synopsys_edac.c index ab4c7cc2daf5..e589aa9f7876 100644 --- a/drivers/edac/synopsys_edac.c +++ b/drivers/edac/synopsys_edac.c @@ -12,12 +12,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include #include #include @@ -34,6 +36,7 @@ /* DDR capabilities */ #define SNPS_CAP_ECC_SCRUB BIT(0) +#define SNPS_CAP_ECC_SCRUBBER BIT(1) #define SNPS_CAP_ZYNQMP BIT(31) /* Synopsys uMCTL2 DDR controller registers that are relevant to ECC */ @@ -102,6 +105,12 @@ #define DDR_SARBASE0_OFST 0xF04 #define DDR_SARSIZE0_OFST 0xF08 +/* ECC Scrubber Registers */ +#define ECC_SBRCTL_OFST 0xF24 +#define ECC_SBRSTAT_OFST 0xF28 +#define ECC_SBRWDATA0_OFST 0xF2C +#define ECC_SBRWDATA1_OFST 0xF30 + /* ZynqMP DDR QOS Registers */ #define ZYNQMP_DDR_QOS_IRQ_STAT_OFST 0x20200 #define ZYNQMP_DDR_QOS_IRQ_EN_OFST 0x20208 @@ -242,6 +251,18 @@ #define DDR_MAX_NSAR 4 #define DDR_MIN_SARSIZE SZ_256M +/* ECC Scrubber registers definitions */ +#define ECC_SBRCTL_SCRUB_INTERVAL GENMASK(20, 8) +#define ECC_SBRCTL_INTERVAL_STEP 512 +#define ECC_SBRCTL_INTERVAL_MIN 0 +#define ECC_SBRCTL_INTERVAL_SAFE 1 +#define ECC_SBRCTL_INTERVAL_MAX FIELD_MAX(ECC_SBRCTL_SCRUB_INTERVAL) +#define ECC_SBRCTL_SCRUB_BURST GENMASK(6, 4) +#define ECC_SBRCTL_SCRUB_MODE_WR BIT(2) +#define ECC_SBRCTL_SCRUB_EN BIT(0) +#define ECC_SBRSTAT_SCRUB_DONE BIT(1) +#define ECC_SBRSTAT_SCRUB_BUSY BIT(0) + /* ZynqMP DDR QOS Interrupt register definitions */ #define ZYNQMP_DDR_QOS_UE_MASK BIT(2) #define ZYNQMP_DDR_QOS_CE_MASK BIT(1) @@ -912,6 +933,47 @@ static irqreturn_t snps_dfi_irq_handler(int irq, void *dev_id) return IRQ_HANDLED; } +/** + * snps_sbr_irq_handler - Scrubber Done interrupt handler. + * @irq: IRQ number. + * @dev_id: Device ID. + * + * It just checks whether the IRQ has been caused by the Scrubber Done event + * and disables the back-to-back scrubbing by falling back to the smallest + * delay between the Scrubber read commands. + * + * Return: IRQ_NONE, if interrupt not set or IRQ_HANDLED otherwise. + */ +static irqreturn_t snps_sbr_irq_handler(int irq, void *dev_id) +{ + struct mem_ctl_info *mci = dev_id; + struct snps_edac_priv *priv = mci->pvt_info; + unsigned long flags; + u32 regval, en; + + /* Make sure IRQ is caused by the Scrubber Done event */ + regval = readl(priv->baseaddr + ECC_SBRSTAT_OFST); + if (!(regval & ECC_SBRSTAT_SCRUB_DONE)) + return IRQ_NONE; + + spin_lock_irqsave(&priv->reglock, flags); + + regval = readl(priv->baseaddr + ECC_SBRCTL_OFST); + en = regval & ECC_SBRCTL_SCRUB_EN; + writel(regval & ~en, priv->baseaddr + ECC_SBRCTL_OFST); + + regval = FIELD_PREP(ECC_SBRCTL_SCRUB_INTERVAL, ECC_SBRCTL_INTERVAL_SAFE); + writel(regval, priv->baseaddr + ECC_SBRCTL_OFST); + + writel(regval | en, priv->baseaddr + ECC_SBRCTL_OFST); + + spin_unlock_irqrestore(&priv->reglock, flags); + + edac_mc_printk(mci, KERN_WARNING, "Back-to-back scrubbing disabled\n"); + + return IRQ_HANDLED; +} + /** * snps_com_irq_handler - Interrupt IRQ signal handler. * @irq: IRQ number. @@ -921,6 +983,8 @@ static irqreturn_t snps_dfi_irq_handler(int irq, void *dev_id) */ static irqreturn_t snps_com_irq_handler(int irq, void *dev_id) { + struct mem_ctl_info *mci = dev_id; + struct snps_edac_priv *priv = mci->pvt_info; irqreturn_t rc = IRQ_NONE; rc |= snps_ce_irq_handler(irq, dev_id); @@ -929,6 +993,9 @@ static irqreturn_t snps_com_irq_handler(int irq, void *dev_id) rc |= snps_dfi_irq_handler(irq, dev_id); + if (priv->info.caps & SNPS_CAP_ECC_SCRUBBER) + rc |= snps_sbr_irq_handler(irq, dev_id); + return rc; } @@ -983,6 +1050,200 @@ static void snps_disable_irq(struct snps_edac_priv *priv) spin_unlock_irqrestore(&priv->reglock, flags); } +/** + * snps_get_sdram_bw - Get SDRAM bandwidth. + * @priv: DDR memory controller private instance data. + * + * The SDRAM interface bandwidth is calculated based on the DDRC Core clock rate + * and the DW uMCTL2 IP-core parameters like DQ-bus width and mode and + * Core/SDRAM clocks frequency ratio. Note it returns the theoretical bandwidth + * which in reality is hardly possible to reach. + * + * Return: SDRAM bandwidth or zero if no Core clock specified. + */ +static u64 snps_get_sdram_bw(struct snps_edac_priv *priv) +{ + unsigned long rate; + + /* + * Depending on the ratio mode the SDRAM clock either matches the Core + * clock or runs with the twice its frequency. + */ + rate = clk_get_rate(priv->clks[SNPS_CORE_CLK].clk); + rate *= priv->info.freq_ratio; + + /* + * Scale up by 2 since it's DDR (Double Data Rate) and subtract the + * DQ-mode since in non-Full mode only a part of the DQ-bus is utilised + * on each SDRAM clock edge. + */ + return (2U << (priv->info.dq_width - priv->info.dq_mode)) * (u64)rate; +} + +/** + * snps_get_scrub_bw - Get Scrubber bandwidth. + * @priv: DDR memory controller private instance data. + * @interval: Scrub interval. + * + * DW uMCTL2 DDRC Scrubber performs periodical progressive burst reads (RMW if + * ECC CE is detected) commands from the whole memory space. The read commands + * can be delayed by means of the SBRCTL.scrub_interval field. The Scrubber + * cycles look as follows: + * + * |-HIF-burst-read-|-------delay-------|-HIF-burst-read-|------- etc + * + * Tb = Bl*[DQ]/Bw[RAM], Td = 512*interval/Fc - periods of the HIF-burst-read + * and delay stages, where + * Bl - HIF burst length, [DQ] - Full DQ-bus width, Bw[RAM] - SDRAM bandwidth, + * Fc - Core clock frequency (Scrubber and Core clocks are synchronous). + * + * After some simple calculations the expressions above can be used to get the + * next Scrubber bandwidth formulae: + * + * Bw[Sbr] = Bw[RAM] / (1 + F * interval), where + * F = 2 * 512 * Fr * Fc * [DQ]e - interval scale factor with + * Fr - HIF/SDRAM clock frequency ratio (1 or 2), [DQ]e - DQ-bus width mode. + * + * Return: Scrubber bandwidth or zero if no Core clock specified. + */ +static u64 snps_get_scrub_bw(struct snps_edac_priv *priv, u32 interval) +{ + unsigned long fac; + u64 bw_ram; + + fac = (2 * ECC_SBRCTL_INTERVAL_STEP * priv->info.freq_ratio) / + (priv->info.hif_burst_len * (1UL << priv->info.dq_mode)); + + bw_ram = snps_get_sdram_bw(priv); + + return div_u64(bw_ram, 1 + fac * interval); +} + +/** + * snps_get_scrub_interval - Get Scrubber delay interval. + * @priv: DDR memory controller private instance data. + * @bw: Scrubber bandwidth. + * + * Similarly to the Scrubber bandwidth the interval formulae can be inferred + * from the same expressions: + * + * interval = (Bw[RAM] - Bw[Sbr]) / (F * Bw[Sbr]) + * + * Return: Scrubber delay interval or zero if no Core clock specified. + */ +static u32 snps_get_scrub_interval(struct snps_edac_priv *priv, u32 bw) +{ + unsigned long fac; + u64 bw_ram; + + fac = (2 * priv->info.freq_ratio * ECC_SBRCTL_INTERVAL_STEP) / + (priv->info.hif_burst_len * (1UL << priv->info.dq_mode)); + + bw_ram = snps_get_sdram_bw(priv); + + /* Divide twice so not to cause the integer overflow in (fac * bw) */ + return div_u64(div_u64(bw_ram - bw, bw), fac); +} + +/** + * snps_set_sdram_scrub_rate - Set the Scrubber bandwidth. + * @mci: EDAC memory controller instance. + * @bw: Bandwidth. + * + * It calculates the delay between the Scrubber read commands based on the + * specified bandwidth and the Core clock rate. If the Core clock is unavailable + * the passed bandwidth will be directly used as the interval value. + * + * Note the method warns about the back-to-back scrubbing since it may + * significantly degrade the system performance. This mode is supposed to be + * used for a single SDRAM scrubbing pass only. So it will be turned off in the + * Scrubber Done IRQ handler. + * + * Return: Actually set bandwidth (interval-based approximated bandwidth if the + * Core clock is unavailable) or zero if the Scrubber was disabled. + */ +static int snps_set_sdram_scrub_rate(struct mem_ctl_info *mci, u32 bw) +{ + struct snps_edac_priv *priv = mci->pvt_info; + u32 regval, interval; + unsigned long flags; + u64 bw_min, bw_max; + + /* Don't bother with the calculations just disable and return. */ + if (!bw) { + spin_lock_irqsave(&priv->reglock, flags); + + regval = readl(priv->baseaddr + ECC_SBRCTL_OFST); + regval &= ~ECC_SBRCTL_SCRUB_EN; + writel(regval, priv->baseaddr + ECC_SBRCTL_OFST); + + spin_unlock_irqrestore(&priv->reglock, flags); + + return 0; + } + + /* If no Core clock specified fallback to the direct interval setup. */ + bw_max = snps_get_scrub_bw(priv, ECC_SBRCTL_INTERVAL_MIN); + if (bw_max) { + bw_min = snps_get_scrub_bw(priv, ECC_SBRCTL_INTERVAL_MAX); + bw = clamp_t(u64, bw, bw_min, bw_max); + + interval = snps_get_scrub_interval(priv, bw); + } else { + bw = clamp_val(bw, ECC_SBRCTL_INTERVAL_MIN, ECC_SBRCTL_INTERVAL_MAX); + + interval = ECC_SBRCTL_INTERVAL_MAX - bw; + } + + /* + * SBRCTL.scrub_en bitfield must be accessed separately from the other + * CSR bitfields. It means the flag must be set/cleared with no updates + * to the rest of the fields. + */ + spin_lock_irqsave(&priv->reglock, flags); + + regval = FIELD_PREP(ECC_SBRCTL_SCRUB_INTERVAL, interval); + writel(regval, priv->baseaddr + ECC_SBRCTL_OFST); + + writel(regval | ECC_SBRCTL_SCRUB_EN, priv->baseaddr + ECC_SBRCTL_OFST); + + spin_unlock_irqrestore(&priv->reglock, flags); + + if (!interval) + edac_mc_printk(mci, KERN_WARNING, "Back-to-back scrubbing enabled\n"); + + if (!bw_max) + return interval ? bw : INT_MAX; + + return snps_get_scrub_bw(priv, interval); +} + +/** + * snps_get_sdram_scrub_rate - Get the Scrubber bandwidth. + * @mci: EDAC memory controller instance. + * + * Return: Scrubber bandwidth (interval-based approximated bandwidth if the + * Core clock is unavailable) or zero if the Scrubber was disabled. + */ +static int snps_get_sdram_scrub_rate(struct mem_ctl_info *mci) +{ + struct snps_edac_priv *priv = mci->pvt_info; + u32 regval; + u64 bw; + + regval = readl(priv->baseaddr + ECC_SBRCTL_OFST); + if (!(regval & ECC_SBRCTL_SCRUB_EN)) + return 0; + + regval = FIELD_GET(ECC_SBRCTL_SCRUB_INTERVAL, regval); + + bw = snps_get_scrub_bw(priv, regval); + if (!bw) + return regval ? ECC_SBRCTL_INTERVAL_MAX - regval : INT_MAX; + + return bw; +} + /** * snps_create_data - Create private data. * @pdev: platform device. @@ -1049,7 +1310,18 @@ static int snps_get_res(struct snps_edac_priv *priv) return rc; } + rc = clk_prepare_enable(priv->clks[SNPS_SBR_CLK].clk); + if (rc) { + edac_printk(KERN_INFO, EDAC_MC, "Couldn't enable Scrubber clock\n"); + goto err_disable_pclk; + } + return 0; + +err_disable_pclk: + clk_disable_unprepare(priv->clks[SNPS_CSR_CLK].clk); + + return rc; } /** @@ -1058,6 +1330,8 @@ static int snps_get_res(struct snps_edac_priv *priv) */ static void snps_put_res(struct snps_edac_priv *priv) { + clk_disable_unprepare(priv->clks[SNPS_SBR_CLK].clk); + clk_disable_unprepare(priv->clks[SNPS_CSR_CLK].clk); } @@ -1158,6 +1432,14 @@ static int snps_get_ddrc_info(struct snps_edac_priv *priv) if (!(regval & ECC_CFG0_DIS_SCRUB)) priv->info.caps |= SNPS_CAP_ECC_SCRUB; + /* Auto-detect the scrubber by writing to the SBRWDATA0 CSR */ + regval = readl(priv->baseaddr + ECC_SBRWDATA0_OFST); + writel(~regval, priv->baseaddr + ECC_SBRWDATA0_OFST); + if (regval != readl(priv->baseaddr + ECC_SBRWDATA0_OFST)) { + priv->info.caps |= SNPS_CAP_ECC_SCRUBBER; + writel(regval, priv->baseaddr + ECC_SBRWDATA0_OFST); + } + /* Auto-detect the basic HIF/SDRAM bus parameters */ regval = readl(priv->baseaddr + DDR_MSTR_OFST); @@ -1644,6 +1926,12 @@ static struct mem_ctl_info *snps_mc_create(struct snps_edac_priv *priv) mci->scrub_cap = SCRUB_FLAG_SW_SRC; } + if (priv->info.caps & SNPS_CAP_ECC_SCRUBBER) { + mci->scrub_cap |= SCRUB_FLAG_HW_PROG | SCRUB_FLAG_HW_TUN; + mci->set_sdram_scrub_rate = snps_set_sdram_scrub_rate; + mci->get_sdram_scrub_rate = snps_get_sdram_scrub_rate; + } + mci->ctl_name = "snps_umctl2_ddrc"; mci->dev_name = SNPS_EDAC_MOD_STRING; mci->mod_name = SNPS_EDAC_MOD_VER; @@ -1718,6 +2006,15 @@ static int snps_request_ind_irq(struct mem_ctl_info *mci) } } + irq = platform_get_irq_byname_optional(priv->pdev, "ecc_sbr"); + if (irq > 0) { + rc = devm_request_irq(dev, irq, snps_sbr_irq_handler, 0, "ecc_sbr", mci); + if (rc) { + edac_printk(KERN_ERR, EDAC_MC, "Failed to request Sbr IRQ\n"); + return rc; + } + } + return 0; } @@ -1824,6 +2121,8 @@ static int snps_ddrc_info_show(struct seq_file *s, void *data) if (priv->info.caps) { if (priv->info.caps & SNPS_CAP_ECC_SCRUB) seq_puts(s, " +Scrub"); + if (priv->info.caps & SNPS_CAP_ECC_SCRUBBER) + seq_puts(s, " +Scrubber"); if (priv->info.caps & SNPS_CAP_ZYNQMP) seq_puts(s, " +ZynqMP"); } else {