| Message ID | 20190107130152.83350-1-mika.westerberg@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | [v2] PCI: Block power management of certain ports with slot implemented bit set | expand |
On Mon, Jan 7, 2019 at 2:01 PM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is > connected to Alpine Ridge Thunderbolt controller. This port has slot > implemented bit set in the config space but other than that it is not > hotplug capable in the sense we are expecting in Linux (it has > dev->is_hotplug_bridge set to 0): > > 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 > Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 > Memory behind bridge: 78000000-8fffffff [size=384M] > Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] > ... > Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 > ... > SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- > Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ > SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- > Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- > SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- > Changed: MRL- PresDet+ LinkState+ > > This system is using ACPI based hotplug to notify the OS that it needs > to rescan the PCI bus (ACPI hotplug). > > If there is nothing connected to any of the Thunderbolt ports the root > port will not have any runtime PM active children and is thus > automatically runtime suspended pretty soon after boot by PCI PM core. > Now, when a device is connected the BIOS SMI handler responsible for > enumerating newly added devices is not able to find anything because the > port is in D3. > > For this reason we block power management of PCIe root and downstream > ports that have slot implemented set and have node in ACPI namespace. > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 > Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > --- > Changes from v1: > > - Block PM for all root and downstream ports with slot implemented bit set > and has an ACPI companion. > > The previous version can be found here: > > https://patchwork.kernel.org/patch/10711553/ > > drivers/pci/pci.c | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index c9d8e3c837de..04bdbcf1dfb7 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -2510,10 +2510,14 @@ void pci_config_pm_runtime_put(struct pci_dev *pdev) > */ > bool pci_bridge_d3_possible(struct pci_dev *bridge) > { > + int type; > + > if (!pci_is_pcie(bridge)) > return false; > > - switch (pci_pcie_type(bridge)) { > + type = pci_pcie_type(bridge); > + > + switch (type) { > case PCI_EXP_TYPE_ROOT_PORT: > case PCI_EXP_TYPE_UPSTREAM: > case PCI_EXP_TYPE_DOWNSTREAM: > @@ -2546,6 +2550,18 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge) > if (bridge->is_hotplug_bridge) > return false; > > + /* > + * Some systems such as Gigabyte X299 the root port is > + * not marked hotplug capable but ACPI based hotplug is > + * still used to bring in the Thunderbolt controller. To > + * make sure those ports do not enter D3 and possibly > + * confuse the BIOS SMI handler, block D3 for them. > + */ > + if (has_acpi_companion(&bridge->dev) && > + type != PCI_EXP_TYPE_UPSTREAM && > + pcie_caps_reg(bridge) & PCI_EXP_FLAGS_SLOT) > + return false; > + > /* > * It should be safe to put PCIe ports from 2015 or newer > * to D3. > -- > 2.19.2 >
On Mon, Jan 07, 2019 at 02:13:14PM +0100, Rafael J. Wysocki wrote: > On Mon, Jan 7, 2019 at 2:01 PM Mika Westerberg > <mika.westerberg@linux.intel.com> wrote: > > > > Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is > > connected to Alpine Ridge Thunderbolt controller. This port has slot > > implemented bit set in the config space but other than that it is not > > hotplug capable in the sense we are expecting in Linux (it has > > dev->is_hotplug_bridge set to 0): > > > > 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 > > Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 > > Memory behind bridge: 78000000-8fffffff [size=384M] > > Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] > > ... > > Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 > > ... > > SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- > > Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ > > SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- > > Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- > > SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- > > Changed: MRL- PresDet+ LinkState+ > > > > This system is using ACPI based hotplug to notify the OS that it needs > > to rescan the PCI bus (ACPI hotplug). > > > > If there is nothing connected to any of the Thunderbolt ports the root > > port will not have any runtime PM active children and is thus > > automatically runtime suspended pretty soon after boot by PCI PM core. > > Now, when a device is connected the BIOS SMI handler responsible for > > enumerating newly added devices is not able to find anything because the > > port is in D3. > > > > For this reason we block power management of PCIe root and downstream > > ports that have slot implemented set and have node in ACPI namespace. > > > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 > > Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Thanks! However, I'm having second toughts about this because I remembered that people put a lot of effort getting discrete graphics with power resource attached to the root port powering off properly. If the root port matches the criteria in this patch it will not be able to go into D3 anymore. It might affect others such as M.2 connected NVMe or WiFi chip as well. For that reason I would still prefer blacklist, at least for now.
On Tue, Jan 8, 2019 at 10:35 AM Mika Westerberg <mika.westerberg@linux.intel.com> wrote: > > On Mon, Jan 07, 2019 at 02:13:14PM +0100, Rafael J. Wysocki wrote: > > On Mon, Jan 7, 2019 at 2:01 PM Mika Westerberg > > <mika.westerberg@linux.intel.com> wrote: > > > > > > Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is > > > connected to Alpine Ridge Thunderbolt controller. This port has slot > > > implemented bit set in the config space but other than that it is not > > > hotplug capable in the sense we are expecting in Linux (it has > > > dev->is_hotplug_bridge set to 0): > > > > > > 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 > > > Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 > > > Memory behind bridge: 78000000-8fffffff [size=384M] > > > Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] > > > ... > > > Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 > > > ... > > > SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- > > > Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ > > > SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- > > > Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- > > > SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- > > > Changed: MRL- PresDet+ LinkState+ > > > > > > This system is using ACPI based hotplug to notify the OS that it needs > > > to rescan the PCI bus (ACPI hotplug). > > > > > > If there is nothing connected to any of the Thunderbolt ports the root > > > port will not have any runtime PM active children and is thus > > > automatically runtime suspended pretty soon after boot by PCI PM core. > > > Now, when a device is connected the BIOS SMI handler responsible for > > > enumerating newly added devices is not able to find anything because the > > > port is in D3. > > > > > > For this reason we block power management of PCIe root and downstream > > > ports that have slot implemented set and have node in ACPI namespace. > > > > > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 > > > Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> > > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > Thanks! > > However, I'm having second toughts about this because I remembered that > people put a lot of effort getting discrete graphics with power resource > attached to the root port powering off properly. If the root port > matches the criteria in this patch it will not be able to go into D3 > anymore. It might affect others such as M.2 connected NVMe or WiFi chip > as well. For that reason I would still prefer blacklist, at least for now. Blacklist would be fine by me too.
Hi Peter, On Tue, Jan 08, 2019 at 11:35:07AM +0200, Mika Westerberg wrote: > On Mon, Jan 07, 2019 at 02:13:14PM +0100, Rafael J. Wysocki wrote: > > On Mon, Jan 7, 2019 at 2:01 PM Mika Westerberg > > <mika.westerberg@linux.intel.com> wrote: > > > > > > Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is > > > connected to Alpine Ridge Thunderbolt controller. This port has slot > > > implemented bit set in the config space but other than that it is not > > > hotplug capable in the sense we are expecting in Linux (it has > > > dev->is_hotplug_bridge set to 0): > > > > > > 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 > > > Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 > > > Memory behind bridge: 78000000-8fffffff [size=384M] > > > Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] > > > ... > > > Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 > > > ... > > > SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- > > > Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ > > > SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- > > > Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- > > > SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- > > > Changed: MRL- PresDet+ LinkState+ > > > > > > This system is using ACPI based hotplug to notify the OS that it needs > > > to rescan the PCI bus (ACPI hotplug). > > > > > > If there is nothing connected to any of the Thunderbolt ports the root > > > port will not have any runtime PM active children and is thus > > > automatically runtime suspended pretty soon after boot by PCI PM core. > > > Now, when a device is connected the BIOS SMI handler responsible for > > > enumerating newly added devices is not able to find anything because the > > > port is in D3. > > > > > > For this reason we block power management of PCIe root and downstream > > > ports that have slot implemented set and have node in ACPI namespace. > > > > > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 > > > Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> > > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > Thanks! > > However, I'm having second toughts about this because I remembered that > people put a lot of effort getting discrete graphics with power resource > attached to the root port powering off properly. If the root port > matches the criteria in this patch it will not be able to go into D3 > anymore. It might affect others such as M.2 connected NVMe or WiFi chip > as well. For that reason I would still prefer blacklist, at least for now. Would this patch: https://patchwork.ozlabs.org/patch/1021317/ break runtime D3cold for the discrete GPU on Optimus laptops such as your Clevo P651RA? Specifically, is the Root Port above the GPU marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci output in https://github.com/Lekensteyn/acpi-stuff) Thanks, Lukas
Hi Lukas, On Tue, Jan 08, 2019 at 11:16:00AM +0100, Lukas Wunner wrote: > Hi Peter, > > On Tue, Jan 08, 2019 at 11:35:07AM +0200, Mika Westerberg wrote: > > On Mon, Jan 07, 2019 at 02:13:14PM +0100, Rafael J. Wysocki wrote: > > > On Mon, Jan 7, 2019 at 2:01 PM Mika Westerberg > > > <mika.westerberg@linux.intel.com> wrote: > > > > > > > > Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is > > > > connected to Alpine Ridge Thunderbolt controller. This port has slot > > > > implemented bit set in the config space but other than that it is not > > > > hotplug capable in the sense we are expecting in Linux (it has > > > > dev->is_hotplug_bridge set to 0): > > > > > > > > 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 > > > > Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 > > > > Memory behind bridge: 78000000-8fffffff [size=384M] > > > > Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] > > > > ... > > > > Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 > > > > ... > > > > SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- > > > > Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ > > > > SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- > > > > Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- > > > > SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- > > > > Changed: MRL- PresDet+ LinkState+ > > > > > > > > This system is using ACPI based hotplug to notify the OS that it needs > > > > to rescan the PCI bus (ACPI hotplug). > > > > > > > > If there is nothing connected to any of the Thunderbolt ports the root > > > > port will not have any runtime PM active children and is thus > > > > automatically runtime suspended pretty soon after boot by PCI PM core. > > > > Now, when a device is connected the BIOS SMI handler responsible for > > > > enumerating newly added devices is not able to find anything because the > > > > port is in D3. > > > > > > > > For this reason we block power management of PCIe root and downstream > > > > ports that have slot implemented set and have node in ACPI namespace. > > > > > > > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 > > > > Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> > > > > Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> > > > > > > Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > > > Thanks! > > > > However, I'm having second toughts about this because I remembered that > > people put a lot of effort getting discrete graphics with power resource > > attached to the root port powering off properly. If the root port > > matches the criteria in this patch it will not be able to go into D3 > > anymore. It might affect others such as M.2 connected NVMe or WiFi chip > > as well. For that reason I would still prefer blacklist, at least for now. > > Would this patch: > > https://patchwork.ozlabs.org/patch/1021317/ > > break runtime D3cold for the discrete GPU on Optimus laptops such as > your Clevo P651RA? Specifically, is the Root Port above the GPU > marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci > output in https://github.com/Lekensteyn/acpi-stuff) Thanks for bringing this into my attention. There are a couple of full lspci dumps, for example for the Dell XPS 9560. https://github.com/Lekensteyn/acpi-stuff/blob/master/d3test/XPS9560/lspci-bare-metal.txt This has upstream port 00:01.0 attached to GPU 01:00.0 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05) (prog-if 00 [Normal decode]) ... Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00 and would indeed be negatively affected by this patch. I can observe the same for the Clevo P651RA (for which I can also send the full lspci dump if you need).
On Tue, Jan 08, 2019 at 11:58:24AM +0100, Peter Wu wrote: > > Would this patch: > > > > https://patchwork.ozlabs.org/patch/1021317/ > > > > break runtime D3cold for the discrete GPU on Optimus laptops such as > > your Clevo P651RA? Specifically, is the Root Port above the GPU > > marked "(Slot+)" in lspci -vv? (There doesn't seem to be raw lspci > > output in https://github.com/Lekensteyn/acpi-stuff) > > Thanks for bringing this into my attention. There are a couple of full > lspci dumps, for example for the Dell XPS 9560. > https://github.com/Lekensteyn/acpi-stuff/blob/master/d3test/XPS9560/lspci-bare-metal.txt > This has upstream port 00:01.0 attached to GPU 01:00.0 > > 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05) (prog-if 00 [Normal decode]) > ... > Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00 > > and would indeed be negatively affected by this patch. I can observe the > same for the Clevo P651RA (for which I can also send the full lspci dump > if you need). Thanks both. I guess that leaves us the blacklist. I can re-send it rebased on top of v5.0-rc1 if there are no objections.
On Tue, Jan 08, 2019 at 02:45:59PM +0200, Mika Westerberg wrote: > I guess that leaves us the blacklist. I can re-send it rebased on top of > v5.0-rc1 if there are no objections. Please constrain either the blacklist entry for the Gigabyte mainboard or the blacklist check itself to x86 using IS_ENABLED() or #ifdef to avoid code bloat on other arches. Thanks, Lukas
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index c9d8e3c837de..04bdbcf1dfb7 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2510,10 +2510,14 @@ void pci_config_pm_runtime_put(struct pci_dev *pdev) */ bool pci_bridge_d3_possible(struct pci_dev *bridge) { + int type; + if (!pci_is_pcie(bridge)) return false; - switch (pci_pcie_type(bridge)) { + type = pci_pcie_type(bridge); + + switch (type) { case PCI_EXP_TYPE_ROOT_PORT: case PCI_EXP_TYPE_UPSTREAM: case PCI_EXP_TYPE_DOWNSTREAM: @@ -2546,6 +2550,18 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge) if (bridge->is_hotplug_bridge) return false; + /* + * Some systems such as Gigabyte X299 the root port is + * not marked hotplug capable but ACPI based hotplug is + * still used to bring in the Thunderbolt controller. To + * make sure those ports do not enter D3 and possibly + * confuse the BIOS SMI handler, block D3 for them. + */ + if (has_acpi_companion(&bridge->dev) && + type != PCI_EXP_TYPE_UPSTREAM && + pcie_caps_reg(bridge) & PCI_EXP_FLAGS_SLOT) + return false; + /* * It should be safe to put PCIe ports from 2015 or newer * to D3.
Gigabyte X299 DESIGNARE EX motherboard has one PCIe root port that is connected to Alpine Ridge Thunderbolt controller. This port has slot implemented bit set in the config space but other than that it is not hotplug capable in the sense we are expecting in Linux (it has dev->is_hotplug_bridge set to 0): 00:1c.4 PCI bridge: Intel Corporation 200 Series PCH PCI Express Root Port #5 Bus: primary=00, secondary=05, subordinate=46, sec-latency=0 Memory behind bridge: 78000000-8fffffff [size=384M] Prefetchable memory behind bridge: 00003800f8000000-00003800ffffffff [size=128M] ... Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 ... SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #8, PowerLimit 25.000W; Interlock- NoCompl+ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet+ LinkState+ This system is using ACPI based hotplug to notify the OS that it needs to rescan the PCI bus (ACPI hotplug). If there is nothing connected to any of the Thunderbolt ports the root port will not have any runtime PM active children and is thus automatically runtime suspended pretty soon after boot by PCI PM core. Now, when a device is connected the BIOS SMI handler responsible for enumerating newly added devices is not able to find anything because the port is in D3. For this reason we block power management of PCIe root and downstream ports that have slot implemented set and have node in ACPI namespace. Link: https://bugzilla.kernel.org/show_bug.cgi?id=202031 Reported-by: Kedar A Dongre <kedar.a.dongre@intel.com> Signed-off-by: Mika Westerberg <mika.westerberg@linux.intel.com> --- Changes from v1: - Block PM for all root and downstream ports with slot implemented bit set and has an ACPI companion. The previous version can be found here: https://patchwork.kernel.org/patch/10711553/ drivers/pci/pci.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)