| Message ID | 20210511021656.17719-1-ansuelsmth@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm: Enlarge IO_SPACE_LIMIT needed for some SoC | expand |
On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote:
> Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port.
Do you really? I mean, yes, theoretically, I understand it, the
hardware supports 64kB of I/O port space per root port. But I/O
port space is rather deprecated these days. My laptop has precisely
two devices with I/O ports, one with 64 bytes and the other with 32
bytes. Would you really suffer by allocating 16kB of I/O port
space to each root port?
On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > Do you really? I mean, yes, theoretically, I understand it, the > hardware supports 64kB of I/O port space per root port. But I/O > port space is rather deprecated these days. My laptop has precisely > two devices with I/O ports, one with 64 bytes and the other with 32 > bytes. Would you really suffer by allocating 16kB of I/O port > space to each root port? We were talking about this in the other wrong patch. I also think this much space looks wrong. The current ipq806x dts have this space so it's actually broken from a long time. The only reason pci worked before was because the pci driver didn't actually check if the settings were right. New kernel introduced more checks and this problem showed up. (to be more precise, the pci port are commonly used by the ath10k wifi and the second ath10k wifi fails to init because of this problem) If you can give me any hint on how to check if the space can be reduced I would be very happy to investigate it. In the driver I notice that the max buffer is set to 2k, could be this a hint?
On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > Do you really? I mean, yes, theoretically, I understand it, the > > hardware supports 64kB of I/O port space per root port. But I/O > > port space is rather deprecated these days. My laptop has precisely > > two devices with I/O ports, one with 64 bytes and the other with 32 > > bytes. Would you really suffer by allocating 16kB of I/O port > > space to each root port? > > We were talking about this in the other wrong patch. I also think this > much space looks wrong. The current ipq806x dts have this space so it's > actually broken from a long time. The only reason pci worked before was > because the pci driver didn't actually check if the settings were right. > New kernel introduced more checks and this problem showed up. (to be > more precise, the pci port are commonly used by the ath10k wifi and the > second ath10k wifi fails to init because of this problem) > If you can give me any hint on how to check if the space can be reduced > I would be very happy to investigate it. > In the driver I notice that the max buffer is set to 2k, could be this a > hint? > Could you share the output of lspci -vv from such a system? I agree with Matthew that fiddling with the size of the I/O space range probably papers over another problem, and with the odd exception, no PCIe card used on ARM systems actually uses their I/O BARs, even when they have them. (I used to carry a PCIe serial port card to UEFI plugfests because that was the only thing that would stop working if a system configured its I/O resource window incorrectly)
On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > hardware supports 64kB of I/O port space per root port. But I/O > > > port space is rather deprecated these days. My laptop has precisely > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > space to each root port? > > > > We were talking about this in the other wrong patch. I also think this > > much space looks wrong. The current ipq806x dts have this space so it's > > actually broken from a long time. The only reason pci worked before was > > because the pci driver didn't actually check if the settings were right. > > New kernel introduced more checks and this problem showed up. (to be > > more precise, the pci port are commonly used by the ath10k wifi and the > > second ath10k wifi fails to init because of this problem) > > If you can give me any hint on how to check if the space can be reduced > > I would be very happy to investigate it. > > In the driver I notice that the max buffer is set to 2k, could be this a > > hint? > > > > Could you share the output of lspci -vv from such a system? > > I agree with Matthew that fiddling with the size of the I/O space > range probably papers over another problem, and with the odd > exception, no PCIe card used on ARM systems actually uses their I/O > BARs, even when they have them. (I used to carry a PCIe serial port > card to UEFI plugfests because that was the only thing that would stop > working if a system configured its I/O resource window incorrectly) Here is the output of lspci -vv 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 44 Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 I/O behind bridge: [disabled] Memory behind bridge: 08000000-081fffff [size=2M] Prefetchable memory behind bridge: [disabled] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 000000004361bc88 Data: 0000 Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt+ RootCap: CRSVisible- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn+ NFERptEn+ FERptEn+ RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Kernel driver in use: pcieport lspci: Unable to load libkmod resources: error -12 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter Subsystem: Qualcomm Atheros Device cafe Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 53 Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ Address: 000000004361bc88 Data: 0001 Masking: fffffffe Pending: 00000000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s (ok), Width x1 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [160 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1SubCtl2: Kernel driver in use: ath10k_pci 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 46 Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 I/O behind bridge: [disabled] Memory behind bridge: 2e000000-2e1fffff [size=2M] Prefetchable memory behind bridge: [disabled] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000435e8688 Data: 0000 Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- RootCap: CRSVisible- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn+ NFERptEn+ FERptEn+ RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Kernel driver in use: pcieport 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter Subsystem: Qualcomm Atheros Device cafe Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 54 Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ Address: 00000000435e8688 Data: 0001 Masking: fffffffe Pending: 00000000 Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [158 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [160 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1SubCtl2: Kernel driver in use: ath10k_pci
On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > port space is rather deprecated these days. My laptop has precisely > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > space to each root port? > > > > > > We were talking about this in the other wrong patch. I also think this > > > much space looks wrong. The current ipq806x dts have this space so it's > > > actually broken from a long time. The only reason pci worked before was > > > because the pci driver didn't actually check if the settings were right. > > > New kernel introduced more checks and this problem showed up. (to be > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > second ath10k wifi fails to init because of this problem) > > > If you can give me any hint on how to check if the space can be reduced > > > I would be very happy to investigate it. > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > hint? > > > > > > > Could you share the output of lspci -vv from such a system? > > > > I agree with Matthew that fiddling with the size of the I/O space > > range probably papers over another problem, and with the odd > > exception, no PCIe card used on ARM systems actually uses their I/O > > BARs, even when they have them. (I used to carry a PCIe serial port > > card to UEFI plugfests because that was the only thing that would stop > > working if a system configured its I/O resource window incorrectly) > > Here is the output of lspci -vv > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > I/O behind bridge: [disabled] > Memory behind bridge: 08000000-081fffff [size=2M] > Prefetchable memory behind bridge: [disabled] So this a MMIO window to the endpoint ... > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] ... and the endpoint has a single *MMIO* BAR of size 2 MiB. This has *nothing* to do with port I/O, which is what you are modifying with your patch. Did you check that the problem exists without the patch, and that the patch makes it go away? > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > Address: 000000004361bc88 Data: 0001 > Masking: fffffffe Pending: 00000000 > Capabilities: [70] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 5GT/s (ok), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- TPHComp- ExtTPHComp- > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > AtomicOpsCtl: ReqEn- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > Capabilities: [158 v1] Latency Tolerance Reporting > Max snoop latency: 0ns > Max no snoop latency: 0ns > Capabilities: [160 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > L1SubCtl2: > Kernel driver in use: ath10k_pci > > 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 46 > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > I/O behind bridge: [disabled] > Memory behind bridge: 2e000000-2e1fffff [size=2M] > Prefetchable memory behind bridge: [disabled] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ > Address: 00000000435e8688 Data: 0000 > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > DevCap: MaxPayload 128 bytes, PhantFunc 0 > ExtTag- RBE+ > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > RootCap: CRSVisible- > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- > AtomicOpsCtl: ReqEn- EgressBlck- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v1] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > RootCmd: CERptEn+ NFERptEn+ FERptEn+ > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > Kernel driver in use: pcieport > > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 54 > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > Address: 00000000435e8688 Data: 0001 > Masking: fffffffe Pending: 00000000 > Capabilities: [70] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > FRS- TPHComp- ExtTPHComp- > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > AtomicOpsCtl: ReqEn- > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > Capabilities: [158 v1] Latency Tolerance Reporting > Max snoop latency: 0ns > Max no snoop latency: 0ns > Capabilities: [160 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > L1SubCtl2: > Kernel driver in use: ath10k_pci
On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > space to each root port? > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > actually broken from a long time. The only reason pci worked before was > > > > because the pci driver didn't actually check if the settings were right. > > > > New kernel introduced more checks and this problem showed up. (to be > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > second ath10k wifi fails to init because of this problem) > > > > If you can give me any hint on how to check if the space can be reduced > > > > I would be very happy to investigate it. > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > hint? > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > range probably papers over another problem, and with the odd > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > card to UEFI plugfests because that was the only thing that would stop > > > working if a system configured its I/O resource window incorrectly) > > > > Here is the output of lspci -vv > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > I/O behind bridge: [disabled] > > Memory behind bridge: 08000000-081fffff [size=2M] > > Prefetchable memory behind bridge: [disabled] > > So this a MMIO window to the endpoint > > ... > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > This has *nothing* to do with port I/O, which is what you are > modifying with your patch. > > Did you check that the problem exists without the patch, and that the > patch makes it go away? > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to init as it can't access the reg. Only the first pci wifi works but the second one fails to init. By increasing the limit all comes back to normal. What I really can't understand is if the big IO space set in the ipq8064 dtsi was wrong from the start and the ath10k fails to init just because is missconfigured. Any idea how to find the appropriate max IO space for the pci? > > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > > Address: 000000004361bc88 Data: 0001 > > Masking: fffffffe Pending: 00000000 > > Capabilities: [70] Express (v2) Endpoint, MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 5GT/s (ok), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- TPHComp- ExtTPHComp- > > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > > AtomicOpsCtl: ReqEn- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v2] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > > Capabilities: [158 v1] Latency Tolerance Reporting > > Max snoop latency: 0ns > > Max no snoop latency: 0ns > > Capabilities: [160 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > > L1SubCtl2: > > Kernel driver in use: ath10k_pci > > > > 0001:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0 > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 46 > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > I/O behind bridge: [disabled] > > Memory behind bridge: 2e000000-2e1fffff [size=2M] > > Prefetchable memory behind bridge: [disabled] > > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- > > BridgeCtl: Parity+ SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ > > Address: 00000000435e8688 Data: 0000 > > Capabilities: [70] Express (v2) Root Port (Slot-), MSI 00 > > DevCap: MaxPayload 128 bytes, PhantFunc 0 > > ExtTag- RBE+ > > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us > > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded) > > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- > > RootCap: CRSVisible- > > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- > > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > > DevCap2: Completion Timeout: Not Supported, TimeoutDis+ NROPrPrP+ LTR- > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- > > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd- > > AtomicOpsCtl: ReqEn- EgressBlck- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v1] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > RootCmd: CERptEn+ NFERptEn+ FERptEn+ > > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > > FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 > > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > > Kernel driver in use: pcieport > > > > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > Subsystem: Qualcomm Atheros Device cafe > > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 0, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 54 > > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] > > Capabilities: [40] Power Management version 3 > > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Capabilities: [50] MSI: Enable+ Count=1/32 Maskable+ 64bit+ > > Address: 00000000435e8688 Data: 0001 > > Masking: fffffffe Pending: 00000000 > > Capabilities: [70] Express (v2) Endpoint, MSI 00 > > DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W > > DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > > MaxPayload 128 bytes, MaxReadReq 128 bytes > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend- > > LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM not supported > > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > > LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > > LnkSta: Speed 2.5GT/s (downgraded), Width x1 (ok) > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ > > 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix- > > EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- > > FRS- TPHComp- ExtTPHComp- > > AtomicOpsCap: 32bit- 64bit- 128bitCAS- > > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, > > AtomicOpsCtl: ReqEn- > > LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- > > LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- > > Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- > > Compliance De-emphasis: -6dB > > LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- > > EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- > > Retimer- 2Retimers- CrosslinkRes: unsupported > > Capabilities: [100 v2] Advanced Error Reporting > > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- > > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- > > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ > > AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn- > > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > > HeaderLog: 00000000 00000000 00000000 00000000 > > Capabilities: [148 v1] Device Serial Number 00-00-00-00-00-00-00-00 > > Capabilities: [158 v1] Latency Tolerance Reporting > > Max snoop latency: 0ns > > Max no snoop latency: 0ns > > Capabilities: [160 v1] L1 PM Substates > > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- L1_PM_Substates- > > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > > L1SubCtl2: > > Kernel driver in use: ath10k_pci
On Tue, 11 May 2021 at 14:37, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > > space to each root port? > > > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > > actually broken from a long time. The only reason pci worked before was > > > > > because the pci driver didn't actually check if the settings were right. > > > > > New kernel introduced more checks and this problem showed up. (to be > > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > > second ath10k wifi fails to init because of this problem) > > > > > If you can give me any hint on how to check if the space can be reduced > > > > > I would be very happy to investigate it. > > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > > hint? > > > > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > > range probably papers over another problem, and with the odd > > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > > card to UEFI plugfests because that was the only thing that would stop > > > > working if a system configured its I/O resource window incorrectly) > > > > > > Here is the output of lspci -vv > > > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > > I/O behind bridge: [disabled] > > > Memory behind bridge: 08000000-081fffff [size=2M] > > > Prefetchable memory behind bridge: [disabled] > > > > So this a MMIO window to the endpoint > > > > ... > > > > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > > > This has *nothing* to do with port I/O, which is what you are > > modifying with your patch. > > > > Did you check that the problem exists without the patch, and that the > > patch makes it go away? > > > > > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to > init as it can't access the reg. Only the first pci wifi works but the > second one fails to init. By increasing the limit all comes back to > normal. What I really can't understand is if the big IO space set in the > ipq8064 dtsi was wrong from the start and the ath10k fails to init just > because is missconfigured. Any idea how to find the appropriate max IO space > for the pci? > OK, so the entire second host bridge fails to probe, because there is no virtual address space left for the I/O window. Just change the 'downstream I/O' window size in the DT to 64k (0x10000) - I assume the current size (0x100000) is a typo anyway.
On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote: > On Tue, 11 May 2021 at 14:37, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > On Tue, May 11, 2021 at 02:30:36PM +0200, Ard Biesheuvel wrote: > > > On Tue, 11 May 2021 at 14:15, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > On Tue, May 11, 2021 at 06:26:28AM +0200, Ard Biesheuvel wrote: > > > > > On Tue, 11 May 2021 at 04:32, Ansuel Smith <ansuelsmth@gmail.com> wrote: > > > > > > > > > > > > On Tue, May 11, 2021 at 03:24:29AM +0100, Matthew Wilcox wrote: > > > > > > > On Tue, May 11, 2021 at 04:16:54AM +0200, Ansuel Smith wrote: > > > > > > > > Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. > > > > > > > > > > > > > > Do you really? I mean, yes, theoretically, I understand it, the > > > > > > > hardware supports 64kB of I/O port space per root port. But I/O > > > > > > > port space is rather deprecated these days. My laptop has precisely > > > > > > > two devices with I/O ports, one with 64 bytes and the other with 32 > > > > > > > bytes. Would you really suffer by allocating 16kB of I/O port > > > > > > > space to each root port? > > > > > > > > > > > > We were talking about this in the other wrong patch. I also think this > > > > > > much space looks wrong. The current ipq806x dts have this space so it's > > > > > > actually broken from a long time. The only reason pci worked before was > > > > > > because the pci driver didn't actually check if the settings were right. > > > > > > New kernel introduced more checks and this problem showed up. (to be > > > > > > more precise, the pci port are commonly used by the ath10k wifi and the > > > > > > second ath10k wifi fails to init because of this problem) > > > > > > If you can give me any hint on how to check if the space can be reduced > > > > > > I would be very happy to investigate it. > > > > > > In the driver I notice that the max buffer is set to 2k, could be this a > > > > > > hint? > > > > > > > > > > > > > > > > Could you share the output of lspci -vv from such a system? > > > > > > > > > > I agree with Matthew that fiddling with the size of the I/O space > > > > > range probably papers over another problem, and with the odd > > > > > exception, no PCIe card used on ARM systems actually uses their I/O > > > > > BARs, even when they have them. (I used to carry a PCIe serial port > > > > > card to UEFI plugfests because that was the only thing that would stop > > > > > working if a system configured its I/O resource window incorrectly) > > > > > > > > Here is the output of lspci -vv > > > > > > > > 0000:00:00.0 PCI bridge: Qualcomm Device 0101 (prog-if 00 [Normal decode]) > > > > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > > > > I/O behind bridge: [disabled] > > > > Memory behind bridge: 08000000-081fffff [size=2M] > > > > Prefetchable memory behind bridge: [disabled] > > > > > > So this a MMIO window to the endpoint > > > > > > ... > > > > > > > > > > > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > > > > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] > > > > > > ... and the endpoint has a single *MMIO* BAR of size 2 MiB. > > > > > > This has *nothing* to do with port I/O, which is what you are > > > modifying with your patch. > > > > > > Did you check that the problem exists without the patch, and that the > > > patch makes it go away? > > > > > > > > > > Yes without the change to IO_SPACE_LIMIT, the ath10k driver fails to > > init as it can't access the reg. Only the first pci wifi works but the > > second one fails to init. By increasing the limit all comes back to > > normal. What I really can't understand is if the big IO space set in the > > ipq8064 dtsi was wrong from the start and the ath10k fails to init just > > because is missconfigured. Any idea how to find the appropriate max IO space > > for the pci? > > > > OK, so the entire second host bridge fails to probe, because there is > no virtual address space left for the I/O window. > > Just change the 'downstream I/O' window size in the DT to 64k > (0x10000) - I assume the current size (0x100000) is a typo anyway. Ok, so my fear were right... someone just typo the IO when the dtsi was pushed and was wrong from all that times. Much easier and cleaner solution.
On Tue, May 11, 2021 at 02:15:21PM +0200, Ansuel Smith wrote: > 0000:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b500000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 53 > Region 0: Memory at 08000000 (64-bit, non-prefetchable) [size=2M] It seems this Atheros device does not make any use of I/O, so this device should be fine. > 0001:01:00.0 Network controller: Qualcomm Atheros QCA9984 802.11ac Wave 2 Wireless Network Adapter > Subsystem: Qualcomm Atheros Device cafe > Device tree node: /sys/firmware/devicetree/base/soc/pci@1b700000/bridge@0,0/wifi@1,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 54 > Region 0: Memory at 2e000000 (64-bit, non-prefetchable) [size=2M] This device also seems to not make use of any I/O. I don't see any reason probing of these two drivers should fail if there is no or reduced I/O space. Please can you share the kernel messages so we can see exactly what is failing and how?
On Tue, May 11, 2021 at 02:51:15PM +0200, Ansuel Smith wrote: > On Tue, May 11, 2021 at 02:46:49PM +0200, Ard Biesheuvel wrote: > > OK, so the entire second host bridge fails to probe, because there is > > no virtual address space left for the I/O window. > > > > Just change the 'downstream I/O' window size in the DT to 64k > > (0x10000) - I assume the current size (0x100000) is a typo anyway. > > Ok, so my fear were right... someone just typo the IO when the dtsi was > pushed and was wrong from all that times. Much easier and cleaner > solution. Great news.
On Tue, May 11, 2021 at 6:26 AM Ard Biesheuvel <ardb@kernel.org> wrote: > I used to carry a PCIe serial port > card to UEFI plugfests because that was the only thing that would stop > working if a system configured its I/O resource window incorrectly I've been looking for a thing like that for testing, I was actually thinking that it would perhaps be a good idea to add a I/O-resource requiring device to QEMU just to be able to test this kind of thing for completeness. Yours, Linus Walleij
diff --git a/Documentation/arm/memory.rst b/Documentation/arm/memory.rst index 0cb1e2938823..938fe74c9020 100644 --- a/Documentation/arm/memory.rst +++ b/Documentation/arm/memory.rst @@ -53,7 +53,7 @@ ffc00000 ffc7ffff Guard region ff800000 ffbfffff Permanent, fixed read-only mapping of the firmware provided DT blob -fee00000 feffffff Mapping of PCI I/O space. This is a static +fee00000 ff0fffff Mapping of PCI I/O space. This is a static mapping within the vmalloc space. VMALLOC_START VMALLOC_END-1 vmalloc() / ioremap() space. diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h index fc748122f1e0..76f1c668df5e 100644 --- a/arch/arm/include/asm/io.h +++ b/arch/arm/include/asm/io.h @@ -197,7 +197,7 @@ void __iomem *pci_remap_cfgspace(resource_size_t res_cookie, size_t size); #ifdef CONFIG_NEED_MACH_IO_H #include <mach/io.h> #elif defined(CONFIG_PCI) -#define IO_SPACE_LIMIT ((resource_size_t)0xfffff) +#define IO_SPACE_LIMIT ((resource_size_t)0x2fffff) #define __io(a) __typesafe_io(PCI_IO_VIRT_BASE + ((a) & IO_SPACE_LIMIT)) #else #define __io(a) __typesafe_io((a) & IO_SPACE_LIMIT)
Ipq8064 SoC requires larger IO_SPACE_LIMIT on second and third pci port. Each pci line I/O space require 0x100000 and the current limit in the specific case of ipq8064 cause the malfunction of the second and the third line that are commonly used for wifi cards. Current IO space is set to support only 0x100000 space while for ipq806x if all 3 pcie lines are connected, it's required 0x300000 of space. Update the IO_SPACE_LIMIT to permit this larger space and update the documentation for the Mapping PCI I/O space memory end to the new value of ff0fffff. Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com> --- Documentation/arm/memory.rst | 2 +- arch/arm/include/asm/io.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)