From patchwork Mon Aug 11 14:03:09 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 4707241 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C3A619F375 for ; Mon, 11 Aug 2014 13:44:57 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 8AE7620122 for ; Mon, 11 Aug 2014 13:44:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DD8F720109 for ; Mon, 11 Aug 2014 13:44:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753672AbaHKNow (ORCPT ); Mon, 11 Aug 2014 09:44:52 -0400 Received: from v094114.home.net.pl ([79.96.170.134]:59492 "HELO v094114.home.net.pl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1753547AbaHKNov (ORCPT ); Mon, 11 Aug 2014 09:44:51 -0400 Received: from aayi173.neoplus.adsl.tpnet.pl [83.6.120.173] (HELO vostro.rjw.lan) by serwer1319399.home.pl [79.96.170.134] with SMTP (IdeaSmtpServer v0.80) id d4b7def09d23057c; Mon, 11 Aug 2014 15:44:49 +0200 From: "Rafael J. Wysocki" To: Thomas Gleixner Cc: Peter Zijlstra , Linux PM list , Linux Kernel Mailing List , Linux PCI Subject: [PATCH 6/6 v2] irq / PM: Document rules related to system suspend and interrupts Date: Mon, 11 Aug 2014 16:03:09 +0200 Message-ID: <32757447.HpzYdgnW9A@vostro.rjw.lan> User-Agent: KMail/4.11.5 (Linux/3.16.0-rc5+; KDE/4.11.5; x86_64; ; ) In-Reply-To: <26580319.OZP7jvJnA9@vostro.rjw.lan> References: <26580319.OZP7jvJnA9@vostro.rjw.lan> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Rafael J. Wysocki Add a document describing how IRQs are managed during system suspend and resume, how to set up a wakeup interrupt and what the IRQF_NO_SUSPEND flag is supposed to be used for. Signed-off-by: Rafael J. Wysocki --- Documentation/power/suspend-and-interrupts.txt | 156 +++++++++++++++++++++++++ 1 file changed, 156 insertions(+) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux-pm/Documentation/power/suspend-and-interrupts.txt =================================================================== --- /dev/null +++ linux-pm/Documentation/power/suspend-and-interrupts.txt @@ -0,0 +1,156 @@ +System Suspend and Device Interrupts + +Copyright (C) 2014 Intel Corp. +Author: Rafael J. Wysocki + + +Suspending and Resuming Device IRQs +----------------------------------- + +Device interrupt request lines (IRQs) are generally disabled during system +suspend after the "late" phase of suspending devices (that is, after all of the +->prepare, ->suspend and ->suspend_late callbacks have been executed for all +devices). That is done by suspend_device_irqs(). + +The rationale for doing so is that after the "late" phase of device suspend +there is no legitimate reason why any interrupts from suspended devices should +trigger and if any devices have not been suspended properly yet, it is better to +block interrupts from them anyway. Also, in the past we had problems with +interrupt handlers for shared IRQs that device drivers using them were not +prepared for interrupts triggering after their devices had been suspended. +In some cases they would attempt to access, for example, memory address spaces +of suspended devices and cause unpredictable behavior to ensue as a result. +Unfortunately, such problems are very difficult to debug and the introduction +of suspend_device_irqs(), along with the "noirq" phase of device suspend and +resume, was the only practical way to prevent them from happening. + +Device IRQs are re-enabled during system resume, right before the "early" phase +of resuming devices (that is, before starting to execute ->resume_early +callbacks for devices). The function doing that is resume_device_irqs(). + + +The IRQF_NO_SUSPEND Flag +------------------------ + +There are interrupts that can legitimately trigger during the entire system +suspend-resume cycle, including the "noirq" phases of suspending and resuming +devices as well as during the time when nonboot CPUs are taken offline and +brought back online. That applies to timer interrupts in the first place, +but also to IPIs and to some other special-purpose interrupts, such as the ACPI +SCI that isn't associated with any specific device and is used for signaling +many different types of events. + +The IRQF_NO_SUSPEND flag is used to indicate that to the IRQ subsystem when +requesting a special-purpose interrupt. It causes suspend_device_irqs() to +leave the corresponding IRQ enabled so as to allow the interrupt to work all +the time as expected. + +If that IRQ is shared, it will still be left enabled by suspend_device_irqs(), +but the interrupt handlers that were installed for it without IRQF_NO_SUSPEND +set will not be executed after suspend_device_irqs() has returned. Then, the +IRQ subsystem will only execute interrupt handlers for which IRQF_NO_SUSPEND is +set. In that case, if the final result of handing an interrupt is IRQ_NONE, the +IRQ subsystem will assume that the interrupt came from one of the apparently +suspended devices that share the IRQ with an IRQF_NO_SUSPEND interrupt source +(or more of them). As a result, the IRQ will be disabled and marked as +"suspended" to prevent spurious interrupts from triggering going forward. Next +(1) if a system suspend is in progress, it will be aborted or +(2) if the system is in the "freeze" sleep state (suspend-to-idle), it will be + woken up, +to allow the system to recover from that potentially unstable condition +gracefully. + +The IRQF_NO_SUSPEND flag should not be used when requesting interrupts that +subsequently will be configured for waking up the system from sleep states +via enable_irq_wake() (that is, system wakeup interrupts). + + +System Wakeup Interrupts, enable_irq_wake() and disable_irq_wake() +------------------------------------------------------------------ + +System wakeup interrupts generally need to be configured to wake up the system +from sleep states, especially if they are used for different purposes (e.g. as +I/O interrupts) in the working state. + +That may involve turning on a special signal handling logic within the platform +(such as an SoC) so that signals from a given line are routed in a different way +during system sleep so as to trigger a system wakeup when needed. For example, +the platform may include a dedicated interrupt controller used specifically for +handling system wakeup events. Then, if a given interrupt line is supposed to +wake up the system from sleep sates, the corresponding input of that interrupt +controller needs to be enabled to handle signals from the line in question. +After wakeup, it generally is better to disable that input to prevent the +dedicated controller from triggering interrupts unnecessarily. + +The IRQ subsystem provides two helper functions to be used by device drivers for +those purposes. Namely, enable_irq_wake() turns on the platform's logic for +handling the given IRQ as a system wakeup interrupt line and disable_irq_wake() +turns that logic off. + +Calling enable_irq_wake() doesn't prevent the working-state logic for handling +the given IRQ from being disabled by suspend_device_irqs(), so after the "late" +phase of suspending devices the IRQ can only be expected to work as a system +wakeup interrupt line. The IRQ subsystem checks if there are any pending +interrupts on those lines by calling check_wakeup_irqs() at the last (syscore) +stage of full system suspend. If there are any, it aborts the system suspend +by returning -EBUSY from that function. It does not, however, invoke any +interrupt handlers for system wakeup IRQs after suspend_device_irqs(). + + +System Wakeup Interrupts and Suspend-to-Idle +-------------------------------------------- + +Suspend-to-idle (also known as the "freeze" sleep state) is a relatively new +system sleep state that works by idling all of the processors and waiting for +interrupts right after the "noirq" phase of suspending devices. + +Of course, this means that all of the interrupts with the IRQF_NO_SUSPEND flag +set will bring CPUs out of idle while in that state, but they will not cause the +IRQ subsystem to trigger a system wakeup, unless the final result of handling an +interrupt for a shared IRQ is IRQ_NONE. + +System wakeup interrupts, in turn, are generally expected to trigger wakeup from +system sleep states, including suspend-to-idle. For this reason, all of the +IRQs configured for system wakeup with enable_irq_wake(), previously disabled +by suspend_device_irqs(), are re-enabled right before starting the final "go to +an idle state and wait for an interrupt" loop of suspend-to-idle. However, they +are also reconfigured so that the original handlers associated with them will +not be invoked at that time. Those interrupt handlers are all temporarily +replaced with a special "wakeup mode" interrupt handler that disables the IRQ, +marks it as "suspended" and pending and triggers wakeup from suspend-to-idle. + +The rationale here is to keep the behavior consistent with the previously +existing way of handling wakeup interrupts during full system suspends, which +is that (a) interrupt handlers are not executed for them after +suspend_device_irqs() and (b) full system suspends are aborted if any of them +are received after suspend_device_irqs(). Combined with the requirement that +system wakeup interrupts wake up the system from suspend-to-idle, (a) and (b) +imply that interrupt handlers should not be invoked for wakeup interrupts in +suspend-to-idle, although those interrupts should still wake up the system from +that state. The IRQs that triggered wakeup also need to be marked as pending +to avoid losing wakeup events. + +When wakeup from suspend-to-idle is triggered, the wakeup IRQs are disabled +again and their original behavior is restored. They are subsequently re-enabled +by resume_device_irqs(), as usual. + + +IRQF_NO_SUSPEND and enable_irq_wake() +------------------------------------- + +There are no valid reasons to use both enable_irq_wake() and the IRQF_NO_SUSPEND +flag on the same IRQ. + +First of all, if the IRQ is not shared, the rules for handling IRQF_NO_SUSPEND +interrupts (interrupt handlers are invoked after suspend_device_irqs()) are +directly at odds with the rules for handling system wakeup interrupts (interrupt +handlers are not invoked after suspend_device_irqs()). + +Secondly, enable_irq_wake() applies to entire IRQs and not to individual +interrupt handlers, so sharing an IRQ between a system wakeup interrupt source +and an IRQF_NO_SUSPEND interrupt source does not make sense in principle. If +attempted, it leads to situations in which genuine wakeup interrups are lost, +because they are regarded as interrupts from the IRQF_NO_SUSPEND interrupt +source, or are reported as unhandled interrupts, depending on the implementation +of the interrupt handler for that source, the timing of events and other +factors.