diff mbox series

[RESEND,5/5] PCI: aardvark: Implement workaround for PCIe Completion Timeout

Message ID 20210624222621.4776-6-pali@kernel.org (mailing list archive)
State New, archived
Headers show
Series PCI: aardvark: Initialization fixes | expand

Commit Message

Pali Rohár June 24, 2021, 10:26 p.m. UTC 
Marvell Armada 3700 Functional Errata, Guidelines, and Restrictions
document describes in erratum 3.12 PCIe Completion Timeout (Ref #: 251),
that PCIe IP does not support a strong-ordered model for inbound posted vs.
outbound completion.

As a workaround for this erratum, DIS_ORD_CHK flag in Debug Mux Control
register must be set. It disables the ordering check in the core between
Completions and Posted requests received from the link.

It was reported that enabling this workaround fixes instability issues and
"Unhandled fault" errors when using 60 GHz WiFi 802.11ad card with Qualcomm
QCA6335 chip under significant load which were caused by interrupt status
stuck in the outbound CMPLT queue traced back to this erratum.

This workaround fixes also kernel panic triggered after some minutes of
usage 5 GHz WiFi 802.11ax card with Mediatek MT7915 chip:

    Internal error: synchronous external abort: 96000210 [#1] SMP
    Kernel panic - not syncing: Fatal exception in interrupt

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Pali Rohár <pali@kernel.org>
Cc: stable@vger.kernel.org
---
Patch was originally written by Thomas and is already for a long time part
of Marvell SDK. I have just re-written/re-applied it on top of mainline
kernel and also wrote a new updated commit message.

Please note that this patch is questionable as Bjorn has some objections
and nobody, including Marvell, was not able to explain erratum nor what
is workaround exactly doing. Documentation about this topic is basically
missing.

We just know that it fixes real kernel crashes when using WiFi cards.
---
 drivers/pci/controller/pci-aardvark.c | 7 +++++++
 1 file changed, 7 insertions(+)

Comments

Pali Rohár Aug. 25, 2021, 7:59 p.m. UTC | #1 
On Friday 25 June 2021 00:26:21 Pali Rohár wrote:
> Marvell Armada 3700 Functional Errata, Guidelines, and Restrictions
> document describes in erratum 3.12 PCIe Completion Timeout (Ref #: 251),
> that PCIe IP does not support a strong-ordered model for inbound posted vs.
> outbound completion.
> 
> As a workaround for this erratum, DIS_ORD_CHK flag in Debug Mux Control
> register must be set. It disables the ordering check in the core between
> Completions and Posted requests received from the link.
> 
> It was reported that enabling this workaround fixes instability issues and
> "Unhandled fault" errors when using 60 GHz WiFi 802.11ad card with Qualcomm
> QCA6335 chip under significant load which were caused by interrupt status
> stuck in the outbound CMPLT queue traced back to this erratum.
> 
> This workaround fixes also kernel panic triggered after some minutes of
> usage 5 GHz WiFi 802.11ax card with Mediatek MT7915 chip:
> 
>     Internal error: synchronous external abort: 96000210 [#1] SMP
>     Kernel panic - not syncing: Fatal exception in interrupt
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Signed-off-by: Pali Rohár <pali@kernel.org>
> Cc: stable@vger.kernel.org
> ---
> Patch was originally written by Thomas and is already for a long time part
> of Marvell SDK. I have just re-written/re-applied it on top of mainline
> kernel and also wrote a new updated commit message.
> 
> Please note that this patch is questionable as Bjorn has some objections
> and nobody, including Marvell, was not able to explain erratum nor what
> is workaround exactly doing. Documentation about this topic is basically
> missing.

See also https://lore.kernel.org/linux-pci/20210723221710.wtztsrddudnxeoj3@pali/

> We just know that it fixes real kernel crashes when using WiFi cards.
> ---
>  drivers/pci/controller/pci-aardvark.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> index 9ff68abd8d1e..231f4469d87e 100644
> --- a/drivers/pci/controller/pci-aardvark.c
> +++ b/drivers/pci/controller/pci-aardvark.c
> @@ -167,6 +167,8 @@
>  #define     LTSSM_L0				0x10
>  #define     RC_BAR_CONFIG			0x300
>  #define VENDOR_ID_REG				(LMI_BASE_ADDR + 0x44)
> +#define DEBUG_MUX_CTRL_REG			(LMI_BASE_ADDR + 0x208)
> +#define     DIS_ORD_CHK				BIT(30)
>  
>  /* PCIe core controller registers */
>  #define CTRL_CORE_BASE_ADDR			0x18000
> @@ -450,6 +452,11 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
>  		PCIE_CORE_CTRL2_TD_ENABLE;
>  	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
>  
> +	/* Disable ordering checks, workaround for erratum 3.12 "PCIe completion timeout" */
> +	reg = advk_readl(pcie, DEBUG_MUX_CTRL_REG);
> +	reg |= DIS_ORD_CHK;
> +	advk_writel(pcie, reg, DEBUG_MUX_CTRL_REG);
> +
>  	/* Set lane X1 */
>  	reg = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
>  	reg &= ~LANE_CNT_MSK;
> -- 
> 2.20.1
>
Pali Rohár Sept. 8, 2021, 7:42 p.m. UTC | #2 
On Wednesday 25 August 2021 21:59:53 Pali Rohár wrote:
> On Friday 25 June 2021 00:26:21 Pali Rohár wrote:
> > Marvell Armada 3700 Functional Errata, Guidelines, and Restrictions
> > document describes in erratum 3.12 PCIe Completion Timeout (Ref #: 251),
> > that PCIe IP does not support a strong-ordered model for inbound posted vs.
> > outbound completion.
> > 
> > As a workaround for this erratum, DIS_ORD_CHK flag in Debug Mux Control
> > register must be set. It disables the ordering check in the core between
> > Completions and Posted requests received from the link.
> > 
> > It was reported that enabling this workaround fixes instability issues and
> > "Unhandled fault" errors when using 60 GHz WiFi 802.11ad card with Qualcomm
> > QCA6335 chip under significant load which were caused by interrupt status
> > stuck in the outbound CMPLT queue traced back to this erratum.
> > 
> > This workaround fixes also kernel panic triggered after some minutes of
> > usage 5 GHz WiFi 802.11ax card with Mediatek MT7915 chip:
> > 
> >     Internal error: synchronous external abort: 96000210 [#1] SMP
> >     Kernel panic - not syncing: Fatal exception in interrupt
> > 
> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > Signed-off-by: Pali Rohár <pali@kernel.org>
> > Cc: stable@vger.kernel.org
> > ---
> > Patch was originally written by Thomas and is already for a long time part
> > of Marvell SDK. I have just re-written/re-applied it on top of mainline
> > kernel and also wrote a new updated commit message.
> > 
> > Please note that this patch is questionable as Bjorn has some objections
> > and nobody, including Marvell, was not able to explain erratum nor what
> > is workaround exactly doing. Documentation about this topic is basically
> > missing.
> 
> See also https://lore.kernel.org/linux-pci/20210723221710.wtztsrddudnxeoj3@pali/

Hello Lorenzo. For now let just this one patch (5/5) as is. As we do not
know how to process this issue and there is open (above) question.

I hope that Marvell people would respond to this above issue.

Other remaining patches in this series are fine.

> > We just know that it fixes real kernel crashes when using WiFi cards.
> > ---
> >  drivers/pci/controller/pci-aardvark.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> > index 9ff68abd8d1e..231f4469d87e 100644
> > --- a/drivers/pci/controller/pci-aardvark.c
> > +++ b/drivers/pci/controller/pci-aardvark.c
> > @@ -167,6 +167,8 @@
> >  #define     LTSSM_L0				0x10
> >  #define     RC_BAR_CONFIG			0x300
> >  #define VENDOR_ID_REG				(LMI_BASE_ADDR + 0x44)
> > +#define DEBUG_MUX_CTRL_REG			(LMI_BASE_ADDR + 0x208)
> > +#define     DIS_ORD_CHK				BIT(30)
> >  
> >  /* PCIe core controller registers */
> >  #define CTRL_CORE_BASE_ADDR			0x18000
> > @@ -450,6 +452,11 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie)
> >  		PCIE_CORE_CTRL2_TD_ENABLE;
> >  	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
> >  
> > +	/* Disable ordering checks, workaround for erratum 3.12 "PCIe completion timeout" */
> > +	reg = advk_readl(pcie, DEBUG_MUX_CTRL_REG);
> > +	reg |= DIS_ORD_CHK;
> > +	advk_writel(pcie, reg, DEBUG_MUX_CTRL_REG);
> > +
> >  	/* Set lane X1 */
> >  	reg = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
> >  	reg &= ~LANE_CNT_MSK;
> > -- 
> > 2.20.1
> >
diff mbox series

Patch

diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
index 9ff68abd8d1e..231f4469d87e 100644
--- a/drivers/pci/controller/pci-aardvark.c
+++ b/drivers/pci/controller/pci-aardvark.c
@@ -167,6 +167,8 @@ 
 #define     LTSSM_L0				0x10
 #define     RC_BAR_CONFIG			0x300
 #define VENDOR_ID_REG				(LMI_BASE_ADDR + 0x44)
+#define DEBUG_MUX_CTRL_REG			(LMI_BASE_ADDR + 0x208)
+#define     DIS_ORD_CHK				BIT(30)
 
 /* PCIe core controller registers */
 #define CTRL_CORE_BASE_ADDR			0x18000
@@ -450,6 +452,11 @@  static void advk_pcie_setup_hw(struct advk_pcie *pcie)
 		PCIE_CORE_CTRL2_TD_ENABLE;
 	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
 
+	/* Disable ordering checks, workaround for erratum 3.12 "PCIe completion timeout" */
+	reg = advk_readl(pcie, DEBUG_MUX_CTRL_REG);
+	reg |= DIS_ORD_CHK;
+	advk_writel(pcie, reg, DEBUG_MUX_CTRL_REG);
+
 	/* Set lane X1 */
 	reg = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
 	reg &= ~LANE_CNT_MSK;