| Message ID | 20220513110027.31015-2-manivannan.sadhasivam@linaro.org (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | PCI: Notify PCI drivers about powerdown during suspend | expand |
On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > On some systems like Chromebooks based on Qcom chipsets, the OS may > powerdown all PCIe devices during system suspend for aggressive > powersaving. In that case, the PCI host controller drivers need to notify > the PCI device drivers that the power will be taken off during system > suspend so that the drivers can prepare the devices accordingly. "The OS may powerdown all PCIe devices ..." makes it sound like this is an OS policy decision. Where exactly (what function) is that? Or if it's not an OS policy decision, but rather some property of the hardware, say that specifically. > One prime example is the PCI NVMe driver. This flag can be used by the > driver to shutdown the NVMe device during suspend and recover it during > resume. > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > --- > include/linux/pci.h | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 60adf42460ab..069caf1fe88d 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -578,6 +578,7 @@ struct pci_host_bridge { > unsigned int preserve_config:1; /* Preserve FW resource setup */ > unsigned int size_windows:1; /* Enable root bus sizing */ > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ > > /* Resource alignment requirements */ > resource_size_t (*align_resource)(struct pci_dev *dev, > -- > 2.25.1 >
On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote: > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > > On some systems like Chromebooks based on Qcom chipsets, the OS may > > powerdown all PCIe devices during system suspend for aggressive > > powersaving. In that case, the PCI host controller drivers need to notify > > the PCI device drivers that the power will be taken off during system > > suspend so that the drivers can prepare the devices accordingly. > > "The OS may powerdown all PCIe devices ..." makes it sound like this > is an OS policy decision. Where exactly (what function) is that? > > Or if it's not an OS policy decision, but rather some property of the > hardware, say that specifically. > On SC7280, it is the Resource Power Manager(RPMh) that's powering the devices down by cutting off the PCIe voltage domain. But the SC7280 RC driver itself may put the PCIe devices into D3cold state during system suspend. https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/ So to cover both cases (one is a hardware independent of SoC and another one is the device driver), and to be generic, I've used the term "OS" after looking at the previous flags. Thanks, Mani > > One prime example is the PCI NVMe driver. This flag can be used by the > > driver to shutdown the NVMe device during suspend and recover it during > > resume. > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > --- > > include/linux/pci.h | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > index 60adf42460ab..069caf1fe88d 100644 > > --- a/include/linux/pci.h > > +++ b/include/linux/pci.h > > @@ -578,6 +578,7 @@ struct pci_host_bridge { > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > unsigned int size_windows:1; /* Enable root bus sizing */ > > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ > > > > /* Resource alignment requirements */ > > resource_size_t (*align_resource)(struct pci_dev *dev, > > -- > > 2.25.1 > >
On Tue, May 17, 2022 at 08:39:08PM +0530, Manivannan Sadhasivam wrote: > On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote: > > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > > > On some systems like Chromebooks based on Qcom chipsets, the OS may > > > powerdown all PCIe devices during system suspend for aggressive > > > powersaving. In that case, the PCI host controller drivers need to notify > > > the PCI device drivers that the power will be taken off during system > > > suspend so that the drivers can prepare the devices accordingly. > > > > "The OS may powerdown all PCIe devices ..." makes it sound like this > > is an OS policy decision. Where exactly (what function) is that? > > > > Or if it's not an OS policy decision, but rather some property of the > > hardware, say that specifically. > > On SC7280, it is the Resource Power Manager(RPMh) that's powering > the devices down by cutting off the PCIe voltage domain. But the > SC7280 RC driver itself may put the PCIe devices into D3cold state > during system suspend. > https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/ > > So to cover both cases (one is a hardware independent of SoC and > another one is the device driver), and to be generic, I've used the > term "OS" after looking at the previous flags. This sort of device-specific behavior definitely needs a pointer to an example. Otherwise it seems like it could be generic PCIe behavior that should be documented in the PCIe base spec. > > > One prime example is the PCI NVMe driver. This flag can be used by the > > > driver to shutdown the NVMe device during suspend and recover it during > > > resume. Apparently nvme is broken, or at least sub-optimal, without this flag. What other drivers will be similarly affected? > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > > --- > > > include/linux/pci.h | 1 + > > > 1 file changed, 1 insertion(+) > > > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > index 60adf42460ab..069caf1fe88d 100644 > > > --- a/include/linux/pci.h > > > +++ b/include/linux/pci.h > > > @@ -578,6 +578,7 @@ struct pci_host_bridge { > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ > > > > > > /* Resource alignment requirements */ > > > resource_size_t (*align_resource)(struct pci_dev *dev,
On Tue, May 17, 2022 at 12:24:23PM -0500, Bjorn Helgaas wrote: > On Tue, May 17, 2022 at 08:39:08PM +0530, Manivannan Sadhasivam wrote: > > On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote: > > > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > > > > On some systems like Chromebooks based on Qcom chipsets, the OS may > > > > powerdown all PCIe devices during system suspend for aggressive > > > > powersaving. In that case, the PCI host controller drivers need to notify > > > > the PCI device drivers that the power will be taken off during system > > > > suspend so that the drivers can prepare the devices accordingly. > > > > > > "The OS may powerdown all PCIe devices ..." makes it sound like this > > > is an OS policy decision. Where exactly (what function) is that? > > > > > > Or if it's not an OS policy decision, but rather some property of the > > > hardware, say that specifically. > > > > On SC7280, it is the Resource Power Manager(RPMh) that's powering > > the devices down by cutting off the PCIe voltage domain. But the > > SC7280 RC driver itself may put the PCIe devices into D3cold state > > during system suspend. > > https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/ > > > > So to cover both cases (one is a hardware independent of SoC and > > another one is the device driver), and to be generic, I've used the > > term "OS" after looking at the previous flags. > > This sort of device-specific behavior definitely needs a pointer to an > example. Otherwise it seems like it could be generic PCIe behavior > that should be documented in the PCIe base spec. > Okay. > > > > One prime example is the PCI NVMe driver. This flag can be used by the > > > > driver to shutdown the NVMe device during suspend and recover it during > > > > resume. > > Apparently nvme is broken, or at least sub-optimal, without this flag. Yes, broken on SC7280 or any other SoCs that turn off power. > What other drivers will be similarly affected? > I don't have a list but the drivers that don't expect the device to be turned off or reset during suspend may experience this issue. Right now, we have only identified the issue with NVMe because that's what used on Chromebooks. But in the coming days, we may need to fix some of the drivers also. Thanks, Mani > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > > > --- > > > > include/linux/pci.h | 1 + > > > > 1 file changed, 1 insertion(+) > > > > > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > > index 60adf42460ab..069caf1fe88d 100644 > > > > --- a/include/linux/pci.h > > > > +++ b/include/linux/pci.h > > > > @@ -578,6 +578,7 @@ struct pci_host_bridge { > > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ > > > > > > > > /* Resource alignment requirements */ > > > > resource_size_t (*align_resource)(struct pci_dev *dev,
On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote: > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > > On some systems like Chromebooks based on Qcom chipsets, the OS may > > powerdown all PCIe devices during system suspend for aggressive > > powersaving. In that case, the PCI host controller drivers need to notify > > the PCI device drivers that the power will be taken off during system > > suspend so that the drivers can prepare the devices accordingly. > > "The OS may powerdown all PCIe devices ..." makes it sound like this > is an OS policy decision. Where exactly (what function) is that? > > Or if it's not an OS policy decision, but rather some property of the > hardware, say that specifically. > > > One prime example is the PCI NVMe driver. This flag can be used by the > > driver to shutdown the NVMe device during suspend and recover it during > > resume. > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > --- > > include/linux/pci.h | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > index 60adf42460ab..069caf1fe88d 100644 > > --- a/include/linux/pci.h > > +++ b/include/linux/pci.h > > @@ -578,6 +578,7 @@ struct pci_host_bridge { > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > unsigned int size_windows:1; /* Enable root bus sizing */ > > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ Why does this apply to the whole host bridge? What if you have multiple devices and some are powered off and others aren't? Rob
diff --git a/include/linux/pci.h b/include/linux/pci.h index 60adf42460ab..069caf1fe88d 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -578,6 +578,7 @@ struct pci_host_bridge { unsigned int preserve_config:1; /* Preserve FW resource setup */ unsigned int size_windows:1; /* Enable root bus sizing */ unsigned int msi_domain:1; /* Bridge wants MSI domain */ + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ /* Resource alignment requirements */ resource_size_t (*align_resource)(struct pci_dev *dev,
On some systems like Chromebooks based on Qcom chipsets, the OS may powerdown all PCIe devices during system suspend for aggressive powersaving. In that case, the PCI host controller drivers need to notify the PCI device drivers that the power will be taken off during system suspend so that the drivers can prepare the devices accordingly. One prime example is the PCI NVMe driver. This flag can be used by the driver to shutdown the NVMe device during suspend and recover it during resume. Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> --- include/linux/pci.h | 1 + 1 file changed, 1 insertion(+)