| Message ID | 20240207111854.576402-2-jhp@endlessos.org (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | [v3,1/3] PCI: vmd: Enable PCI PM's L1 substates of remapped PCIe Root Port and NVMe | expand |
On Wed, 7 Feb 2024, Jian-Hong Pan wrote: > The original __pci_enable_link_state() configs the links directly without: > * Check the L1 substates features which are supported, or not > * Calculate & program related parameters for L1.2, such as T_POWER_ON, > Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD > > This leads some supported L1 PM substates of the link between VMD remapped > PCIe Root Port and NVMe get wrong configs when a caller tries to enabled > them. > > Here is a failed example on ASUS B1400CEAE with enabled VMD: > > Capabilities: [900 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ > PortCommonModeRestoreTime=32us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > T_CommonMode=0us LTR1.2_Threshold=0ns > L1SubCtl2: T_PwrOn=10us > > This patch initializes the link's L1 PM substates to get the supported > features and programs relating paramters, if some of them are going to be > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if > the caller intends to enable them and they are supported. > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org> > --- > v2: > - Prepare the PCIe LTR parameters before enable L1 Substates > > v3: > - Only enable supported features for the L1 Substates part > > drivers/pci/pcie/aspm.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > index a39d2ee744cb..c866971cae70 100644 > --- a/drivers/pci/pcie/aspm.c > +++ b/drivers/pci/pcie/aspm.c > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) > link->aspm_default |= ASPM_STATE_L0S; > if (state & PCIE_LINK_STATE_L1) > link->aspm_default |= ASPM_STATE_L1; > - /* L1 PM substates require L1 */ > - if (state & PCIE_LINK_STATE_L1_1) > + if (state & ASPM_STATE_L1_2_MASK) > + aspm_l1ss_init(link); Your commit message didn't explain why you need to add this call here, why isn't the existing call to aspm_l1ss_init() enough to initialize what is needed? > + /* L1 PM substates require L1 and should be in supported list */ > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1) > link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2) > link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_1_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM) > link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM) > link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1; It would be simpler and cleaner to just AND once with link->aspm_support to clear unsupported bits from state rather than doing it on every line like that.
On Wed, 2024-02-07 at 19:18 +0800, Jian-Hong Pan wrote: > The original __pci_enable_link_state() configs the links directly without: > * Check the L1 substates features which are supported, or not > * Calculate & program related parameters for L1.2, such as T_POWER_ON, > Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD > > This leads some supported L1 PM substates of the link between VMD remapped > PCIe Root Port and NVMe get wrong configs when a caller tries to enabled > them. > > Here is a failed example on ASUS B1400CEAE with enabled VMD: > > Capabilities: [900 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > L1_PM_Substates+ > PortCommonModeRestoreTime=32us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > T_CommonMode=0us LTR1.2_Threshold=0ns > L1SubCtl2: T_PwrOn=10us > > This patch initializes the link's L1 PM substates to get the supported > features and programs relating paramters, if some of them are going to be > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if > the caller intends to enable them and they are supported. > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org> > --- > v2: > - Prepare the PCIe LTR parameters before enable L1 Substates > > v3: > - Only enable supported features for the L1 Substates part > > drivers/pci/pcie/aspm.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > index a39d2ee744cb..c866971cae70 100644 > --- a/drivers/pci/pcie/aspm.c > +++ b/drivers/pci/pcie/aspm.c > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev > *pdev, int state, bool locked) > link->aspm_default |= ASPM_STATE_L0S; > if (state & PCIE_LINK_STATE_L1) > link->aspm_default |= ASPM_STATE_L1; > - /* L1 PM substates require L1 */ > - if (state & PCIE_LINK_STATE_L1_1) > + if (state & ASPM_STATE_L1_2_MASK) > + aspm_l1ss_init(link); This mixes ASPM_STATE flags with PCIE_LINK_STATE register mapping. This may work but I don't know if it's intended to. Rather do, if (link->default & ASPM_STATE_L1_2_MASK) after collecting all of the states to be enabled. I understand that you are calling aspm_l1ss_init() to do the L1.2 calculations but it does more than this that you don't need. Maybe it would be more appropriate to call aspm_calc_l12_info() directly through an additional function that finds the parent and determines both ends of the link support L1SS. > + /* L1 PM substates require L1 and should be in supported list */ > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1) > link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2) > link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_1_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM) > link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM) > link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1; > pcie_config_aspm_link(link, policy_to_aspm_state(link)); > I don't think these changes are necessary. pcie_config_aspm_link() already checks link->aspm_capable which was initialized from link->aspm_support. David
On 2/7/24 3:18 AM, Jian-Hong Pan wrote: > The original __pci_enable_link_state() configs the links directly without: Since you are referring to the current version of this function, just start with "Currently, " > * Check the L1 substates features which are supported, or not > * Calculate & program related parameters for L1.2, such as T_POWER_ON, > Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD > > This leads some supported L1 PM substates of the link between VMD remapped > PCIe Root Port and NVMe get wrong configs when a caller tries to enabled > them. > > Here is a failed example on ASUS B1400CEAE with enabled VMD: > > Capabilities: [900 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ > PortCommonModeRestoreTime=32us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > T_CommonMode=0us LTR1.2_Threshold=0ns > L1SubCtl2: T_PwrOn=10us > > This patch initializes the link's L1 PM substates to get the supported > features and programs relating paramters, if some of them are going to be > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if > the caller intends to enable them and they are supported. > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org> > --- > v2: > - Prepare the PCIe LTR parameters before enable L1 Substates > > v3: > - Only enable supported features for the L1 Substates part > > drivers/pci/pcie/aspm.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > index a39d2ee744cb..c866971cae70 100644 > --- a/drivers/pci/pcie/aspm.c > +++ b/drivers/pci/pcie/aspm.c > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) > link->aspm_default |= ASPM_STATE_L0S; > if (state & PCIE_LINK_STATE_L1) > link->aspm_default |= ASPM_STATE_L1; > - /* L1 PM substates require L1 */ > - if (state & PCIE_LINK_STATE_L1_1) > + if (state & ASPM_STATE_L1_2_MASK) > + aspm_l1ss_init(link); This is a new change. Explain why you are doing it? > + /* L1 PM substates require L1 and should be in supported list */ > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1) > link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2) > link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_1_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM) > link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1; > - if (state & PCIE_LINK_STATE_L1_2_PCIPM) > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM) > link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1; > pcie_config_aspm_link(link, policy_to_aspm_state(link)); >
David E. Box <david.e.box@linux.intel.com> 於 2024年2月8日 週四 上午12:02寫道： > > On Wed, 2024-02-07 at 19:18 +0800, Jian-Hong Pan wrote: > > The original __pci_enable_link_state() configs the links directly without: > > * Check the L1 substates features which are supported, or not > > * Calculate & program related parameters for L1.2, such as T_POWER_ON, > > Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD > > > > This leads some supported L1 PM substates of the link between VMD remapped > > PCIe Root Port and NVMe get wrong configs when a caller tries to enabled > > them. > > > > Here is a failed example on ASUS B1400CEAE with enabled VMD: > > > > Capabilities: [900 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > > L1_PM_Substates+ > > PortCommonModeRestoreTime=32us PortTPowerOnTime=10us > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- > > T_CommonMode=0us LTR1.2_Threshold=0ns > > L1SubCtl2: T_PwrOn=10us > > > > This patch initializes the link's L1 PM substates to get the supported > > features and programs relating paramters, if some of them are going to be > > enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if > > the caller intends to enable them and they are supported. > > > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 > > Signed-off-by: Jian-Hong Pan <jhp@endlessos.org> > > --- > > v2: > > - Prepare the PCIe LTR parameters before enable L1 Substates > > > > v3: > > - Only enable supported features for the L1 Substates part > > > > drivers/pci/pcie/aspm.c | 12 +++++++----- > > 1 file changed, 7 insertions(+), 5 deletions(-) > > > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > > index a39d2ee744cb..c866971cae70 100644 > > --- a/drivers/pci/pcie/aspm.c > > +++ b/drivers/pci/pcie/aspm.c > > @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev > > *pdev, int state, bool locked) > > link->aspm_default |= ASPM_STATE_L0S; > > if (state & PCIE_LINK_STATE_L1) > > link->aspm_default |= ASPM_STATE_L1; > > - /* L1 PM substates require L1 */ > > - if (state & PCIE_LINK_STATE_L1_1) > > + if (state & ASPM_STATE_L1_2_MASK) > > + aspm_l1ss_init(link); > > This mixes ASPM_STATE flags with PCIE_LINK_STATE register mapping. This may work > but I don't know if it's intended to. Rather do, > > if (link->default & ASPM_STATE_L1_2_MASK) > > after collecting all of the states to be enabled. > > I understand that you are calling aspm_l1ss_init() to do the L1.2 calculations > but it does more than this that you don't need. Maybe it would be more > appropriate to call aspm_calc_l12_info() directly through an additional function > that finds the parent and determines both ends of the link support L1SS. After think & check twice, focusing on fixing the L1.2 parameters makes sense. I am preparing a new patch. Thanks! Jian-Hong Pan > > + /* L1 PM substates require L1 and should be in supported list */ > > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1) > > link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1; > > - if (state & PCIE_LINK_STATE_L1_2) > > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2) > > link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1; > > - if (state & PCIE_LINK_STATE_L1_1_PCIPM) > > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM) > > link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1; > > - if (state & PCIE_LINK_STATE_L1_2_PCIPM) > > + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM) > > link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1; > > pcie_config_aspm_link(link, policy_to_aspm_state(link)); > > > > I don't think these changes are necessary. pcie_config_aspm_link() already > checks link->aspm_capable which was initialized from link->aspm_support. > > David
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index a39d2ee744cb..c866971cae70 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1389,14 +1389,16 @@ static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) link->aspm_default |= ASPM_STATE_L0S; if (state & PCIE_LINK_STATE_L1) link->aspm_default |= ASPM_STATE_L1; - /* L1 PM substates require L1 */ - if (state & PCIE_LINK_STATE_L1_1) + if (state & ASPM_STATE_L1_2_MASK) + aspm_l1ss_init(link); + /* L1 PM substates require L1 and should be in supported list */ + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1) link->aspm_default |= ASPM_STATE_L1_1 | ASPM_STATE_L1; - if (state & PCIE_LINK_STATE_L1_2) + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2) link->aspm_default |= ASPM_STATE_L1_2 | ASPM_STATE_L1; - if (state & PCIE_LINK_STATE_L1_1_PCIPM) + if (state & link->aspm_support & PCIE_LINK_STATE_L1_1_PCIPM) link->aspm_default |= ASPM_STATE_L1_1_PCIPM | ASPM_STATE_L1; - if (state & PCIE_LINK_STATE_L1_2_PCIPM) + if (state & link->aspm_support & PCIE_LINK_STATE_L1_2_PCIPM) link->aspm_default |= ASPM_STATE_L1_2_PCIPM | ASPM_STATE_L1; pcie_config_aspm_link(link, policy_to_aspm_state(link));
The original __pci_enable_link_state() configs the links directly without: * Check the L1 substates features which are supported, or not * Calculate & program related parameters for L1.2, such as T_POWER_ON, Common_Mode_Restore_Time, and LTR_L1.2_THRESHOLD This leads some supported L1 PM substates of the link between VMD remapped PCIe Root Port and NVMe get wrong configs when a caller tries to enabled them. Here is a failed example on ASUS B1400CEAE with enabled VMD: Capabilities: [900 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+ PortCommonModeRestoreTime=32us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2+ ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us This patch initializes the link's L1 PM substates to get the supported features and programs relating paramters, if some of them are going to be enabled in __pci_enable_link_state(). Then, enables the L1 PM substates if the caller intends to enable them and they are supported. Link: https://bugzilla.kernel.org/show_bug.cgi?id=218394 Signed-off-by: Jian-Hong Pan <jhp@endlessos.org> --- v2: - Prepare the PCIe LTR parameters before enable L1 Substates v3: - Only enable supported features for the L1 Substates part drivers/pci/pcie/aspm.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)