From patchwork Mon Feb 27 16:12:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 9593541 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DCB2160471 for ; Mon, 27 Feb 2017 16:15:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDED12838E for ; Mon, 27 Feb 2017 16:15:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C276C28488; Mon, 27 Feb 2017 16:15:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B3C4F2838E for ; Mon, 27 Feb 2017 16:15:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=hfkOsGPzx/NflDzOY2r+9Wup6DW9aPcnI8esp+95MAw=; b=aYpQZWwdkQ/vALSYwMwyZwjxmj ZKFW89PGXVa7TUUiMtg7zO82Phb5WSvKCDqzUxvoVG1c3ebD/gICiDt/sznM9/V/LCLnaub97mlMj mpGJF4R4V6kRAGvKeVdiRQEWUP5SNxt3iCpOP0vk67d/Sgfpl2q17ANtB9CyNX7w1nY2Iyyz7gnaC CW0l7d5t3rWHVHaITvlDzKGJXKIqXkQfQDm9MkBu1gLVZ/RbnbTg4byP/RemQUbrpqJf9xoOoH3xV +MkA4cVh8548Ycj9N3O4flj3NIJ/lt4bh2pPqV/ronh2NE9nAGvShmZlVKkZJgfeqxlrzYOLnZUFG nNhAKWgQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1ciNx9-0007OP-Eb; Mon, 27 Feb 2017 16:15:11 +0000 Received: from mail.free-electrons.com ([62.4.15.54]) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1ciNv4-0006Iu-Ja for linux-arm-kernel@lists.infradead.org; Mon, 27 Feb 2017 16:13:08 +0000 Received: by mail.free-electrons.com (Postfix, from userid 110) id 65903206ED; Mon, 27 Feb 2017 17:12:40 +0100 (CET) Received: from localhost (LStLambert-657-1-97-87.w90-63.abo.wanadoo.fr [90.63.216.87]) by mail.free-electrons.com (Postfix) with ESMTPSA id 379BA206F9; Mon, 27 Feb 2017 17:12:30 +0100 (CET) From: Thomas Petazzoni To: Thomas Gleixner , Jason Cooper , Marc Zyngier Subject: [PATCH v2 3/4] irqchip: armada-370-xp: re-enable per-CPU interrupts at resume time Date: Mon, 27 Feb 2017 17:12:19 +0100 Message-Id: <1488211940-26489-4-git-send-email-thomas.petazzoni@free-electrons.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1488211940-26489-1-git-send-email-thomas.petazzoni@free-electrons.com> References: <1488211940-26489-1-git-send-email-thomas.petazzoni@free-electrons.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170227_081303_064279_1984C266 X-CRM114-Status: GOOD ( 21.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Petazzoni , Andrew Lunn , Yehuda Yitschak , Hanna Hawa , Nadav Haklai , Gregory Clement , linux-arm-kernel@lists.infradead.org, Sebastian Hesselbarth MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Commit d17cab4451df1 ("irqchip: Kill off set_irq_flags usage") changed the code of armada_370_xp_mpic_irq_map() from using set_irq_flags() to irq_set_probe(). While the commit log seems to imply that there are no functional changes, there are indeed functional changes introduced by this commit: the IRQ_NOAUTOEN flag is no longer cleared. This functional change caused a regression on Armada XP, which no longer works properly after suspend/resume because per-CPU interrupts remain disabled. This regression was temporarly worked around in commit 353d6d6c82e5d ("irqchip/armada-370-xp: Fix regression by clearing IRQ_NOAUTOEN"), but it is not the most satisfying solution. This commit implements the solution that was initially discussed with Thomas Gleixner. Due to how the hardware registers work, the irq-armada-370-xp cannot simply save/restore a bunch of registers at suspend/resume to make sure that the interrupts remain in the same state after resuming. Therefore, it relies on the kernel to say whether the interrupt is disabled or not, using the irqd_irq_disabled() function. This was all working fine while the IRQ_NOAUTOEN flag was cleared. With the change introduced by Rob Herring in d17cab4451df1, the IRQ_NOAUTOEN flag is now set for all interrupts. irqd_irq_disabled() returns false for per-CPU interrupts, and therefore our per-CPU interrupts are no longer re-enabled after resume. This commit fixes that by using irqd_irq_disabled() only for global interrupts, and using the newly introduced irq_percpu_is_enabled() for per-CPU interrupts. Also, it fixes a related problems that per-CPU interrupts were only re-enabled on the boot CPU and not other CPUs. Until now this wasn't a problem since on this platform, only the local timers are using per-CPU interrupts and the local timers of secondary CPUs are turned off/on during CPU hotplug before suspend, after after resume. However, since Linux 4.4, we are also be using per-CPU interrupts for the network controller, so we need to properly restore the per-CPU interrupts on secondary CPUs as well. Signed-off-by: Thomas Petazzoni --- drivers/irqchip/irq-armada-370-xp.c | 46 ++++++++++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 6 deletions(-) diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c index 1f7dea6..1d4de6e 100644 --- a/drivers/irqchip/irq-armada-370-xp.c +++ b/drivers/irqchip/irq-armada-370-xp.c @@ -360,7 +360,6 @@ static int armada_370_xp_mpic_irq_map(struct irq_domain *h, irq_set_percpu_devid(virq); irq_set_chip_and_handler(virq, &armada_370_xp_irq_chip, handle_percpu_devid_irq); - } else { irq_set_chip_and_handler(virq, &armada_370_xp_irq_chip, handle_level_irq); @@ -424,16 +423,40 @@ static void armada_mpic_send_doorbell(const struct cpumask *mask, ARMADA_370_XP_SW_TRIG_INT_OFFS); } +static void armada_xp_mpic_reenable_percpu(void) +{ + unsigned int irq; + + /* Re-enable per-CPU interrupts that were enabled before suspend */ + for (irq = 0; irq < ARMADA_370_XP_MAX_PER_CPU_IRQS; irq++) { + struct irq_data *data; + int virq; + + virq = irq_linear_revmap(armada_370_xp_mpic_domain, irq); + if (virq == 0) + continue; + + data = irq_get_irq_data(virq); + + if (!irq_percpu_is_enabled(virq)) + continue; + + armada_370_xp_irq_unmask(data); + } +} + static int armada_xp_mpic_starting_cpu(unsigned int cpu) { armada_xp_mpic_perf_init(); armada_xp_mpic_smp_cpu_init(); + armada_xp_mpic_reenable_percpu(); return 0; } static int mpic_cascaded_starting_cpu(unsigned int cpu) { armada_xp_mpic_perf_init(); + armada_xp_mpic_reenable_percpu(); enable_percpu_irq(parent_irq, IRQ_TYPE_NONE); return 0; } @@ -581,16 +604,27 @@ static void armada_370_xp_mpic_resume(void) if (virq == 0) continue; - if (!is_percpu_irq(irq)) + data = irq_get_irq_data(virq); + + if (!is_percpu_irq(irq)) { + /* Non per-CPU interrupts */ writel(irq, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS); - else + if (!irqd_irq_disabled(data)) + armada_370_xp_irq_unmask(data); + } else { + /* Per-CPU interrupts */ writel(irq, main_int_base + ARMADA_370_XP_INT_SET_ENABLE_OFFS); - data = irq_get_irq_data(virq); - if (!irqd_irq_disabled(data)) - armada_370_xp_irq_unmask(data); + /* + * Re-enable on the current CPU, + * armada_xp_mpic_reenable_percpu() will take + * care of secondary CPUs when they come up. + */ + if (irq_percpu_is_enabled(virq)) + armada_370_xp_irq_unmask(data); + } } /* Reconfigure doorbells for IPIs and MSIs */