mbox series

[v2,0/2] pci: Add boot interrupt quirk mechanism for Xeon chipsets

Message ID 20200220192930.64820-1-sean.v.kelley@linux.intel.com (mailing list archive)
Headers show
Series pci: Add boot interrupt quirk mechanism for Xeon chipsets | expand

Message

Sean V Kelley Feb. 20, 2020, 7:29 p.m. UTC
Changes since v1 [1]:

- Correct Documentation section title for 6300ESB chipset.
(Jonathan Derrick)

- Use consistent abbreviations in comments for IO-APIC and Core IO.
(Andy Shevchenko)

- Retained Reviewed-by tag due to no technical changes.

[1]: https://lore.kernel.org/lkml/20200214213313.66622-1-sean.v.kelley@linux.intel.com/

Bjorn, I'm open for it to go to stable as well.

--

When IRQ lines on secondary or higher IO-APICs are masked (e.g.,
Real-Time threaded interrupts), many chipsets redirect IRQs on
this line to the legacy PCH and in turn the base IO-APIC in the
system. The unhandled interrupts on the base IO-APIC will be
identified by the Linux kernel as Spurious Interrupts and can
lead to disabled IRQ lines.

Disabling this legacy PCI interrupt routing is chipset-specific and
varies in mechanism between chipset vendors and across generations.
In some cases the mechanism is exposed to BIOS but not all BIOS
vendors chose to pick it up. With the increasing usage of RT as it
marches towards mainline, additional issues have been raised with
more recent Xeon chipsets.

This patchset disables the boot interrupt on these Xeon chipsets where
this is possible with an additional mechanism. In addition, this
patchset includes documentation covering the background of this quirk.


Sean V Kelley (2):
  pci: Add boot interrupt quirk mechanism for Xeon chipsets
  Documentation:PCI: Add background on Boot Interrupts

 Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
 Documentation/PCI/index.rst           |   1 +
 drivers/pci/quirks.c                  |  80 ++++++++++++--
 3 files changed, 227 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/PCI/boot-interrupts.rst

--
2.25.1

Comments

Bjorn Helgaas Feb. 27, 2020, 10:49 p.m. UTC | #1
On Thu, Feb 20, 2020 at 11:29:28AM -0800, Sean V Kelley wrote:
> Changes since v1 [1]:
> 
> - Correct Documentation section title for 6300ESB chipset.
> (Jonathan Derrick)
> 
> - Use consistent abbreviations in comments for IO-APIC and Core IO.
> (Andy Shevchenko)
> 
> - Retained Reviewed-by tag due to no technical changes.
> 
> [1]: https://lore.kernel.org/lkml/20200214213313.66622-1-sean.v.kelley@linux.intel.com/
> 
> Bjorn, I'm open for it to go to stable as well.
> 
> --
> 
> When IRQ lines on secondary or higher IO-APICs are masked (e.g.,
> Real-Time threaded interrupts), many chipsets redirect IRQs on
> this line to the legacy PCH and in turn the base IO-APIC in the
> system. The unhandled interrupts on the base IO-APIC will be
> identified by the Linux kernel as Spurious Interrupts and can
> lead to disabled IRQ lines.
> 
> Disabling this legacy PCI interrupt routing is chipset-specific and
> varies in mechanism between chipset vendors and across generations.
> In some cases the mechanism is exposed to BIOS but not all BIOS
> vendors chose to pick it up. With the increasing usage of RT as it
> marches towards mainline, additional issues have been raised with
> more recent Xeon chipsets.
> 
> This patchset disables the boot interrupt on these Xeon chipsets where
> this is possible with an additional mechanism. In addition, this
> patchset includes documentation covering the background of this quirk.
> 
> 
> Sean V Kelley (2):
>   pci: Add boot interrupt quirk mechanism for Xeon chipsets
>   Documentation:PCI: Add background on Boot Interrupts
> 
>  Documentation/PCI/boot-interrupts.rst | 153 ++++++++++++++++++++++++++
>  Documentation/PCI/index.rst           |   1 +
>  drivers/pci/quirks.c                  |  80 ++++++++++++--
>  3 files changed, 227 insertions(+), 7 deletions(-)
>  create mode 100644 Documentation/PCI/boot-interrupts.rst

Applied to pci/interrupts for v5.7.  I added a stable tag.

Thanks a lot; this is really a nice piece of work!
Thomas Gleixner Nov. 25, 2020, 11:54 a.m. UTC | #2
Stefan,

On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:

sorry for the delay. This fell through the cracks.

> this quirk breaks our serial ports PCIe card (i.e. we don't see any 
> output from the connected devices; no idea whether anything we send 
> reaches them):
>
> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)
> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
> 06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART)
> function 0 (Disabled)

Can you please provide the output of:

 for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done

Thanks,

        tglx
Stefan Bühler Nov. 25, 2020, 1:41 p.m. UTC | #3
Hi tglx,

On 11/25/20 12:54 PM, Thomas Gleixner wrote:
> Stefan,
> 
> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
> 
> sorry for the delay. This fell through the cracks.
> 
>> this quirk breaks our serial ports PCIe card (i.e. we don't see any 
>> output from the connected devices; no idea whether anything we send 
>> reaches them):
>>
>> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)
>> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
>> 06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
>> 06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
>> 06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART)
>> function 0 (Disabled)
> 
> Can you please provide the output of:
> 
>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
> 

See attachment.

Also I boot the affected systems now with "pci=noioapicquirk", which
"solves" the issue too (instead of patching the kernel).

cheers,
Stefan
05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: fb400000-fb4fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>

06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Region 0: I/O ports at e0e0 [size=32]
	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e0c0 [size=32]
	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: serial

06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e0a0 [disabled] [size=32]
	Region 1: Memory at fb405000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e080 [disabled] [size=32]
	Region 3: Memory at fb404000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: <access denied>

06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	NUMA node: 0
	Region 0: I/O ports at e060 [size=32]
	Region 1: Memory at fb403000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e040 [size=32]
	Region 3: Memory at fb402000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: <access denied>
	Kernel driver in use: serial

06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e020 [disabled] [size=32]
	Region 1: Memory at fb401000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e000 [disabled] [size=32]
	Region 3: Memory at fb400000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: <access denied>
Thomas Gleixner Nov. 26, 2020, 11:45 p.m. UTC | #4
Stefan,

On Wed, Nov 25 2020 at 14:41, Stefan Bühler wrote:
> On 11/25/20 12:54 PM, Thomas Gleixner wrote:
>> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
>> Can you please provide the output of:
>> 
>>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
>
> See attachment.
>
> Also I boot the affected systems now with "pci=noioapicquirk", which
> "solves" the issue too (instead of patching the kernel).

Yes, it skips the quirks.

> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> 	Physical Slot: 1
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 32 bytes
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
> 	I/O behind bridge: 0000e000-0000efff
> 	Memory behind bridge: fb400000-fb4fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
> 	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
> 		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> 	Capabilities: <access denied>

Can you please run this as root so the Capabilities are accessible?

Thanks,

        tglx
Stefan Bühler Nov. 27, 2020, 9:17 a.m. UTC | #5
Hi tglx,

On 11/27/20 12:45 AM, Thomas Gleixner wrote:
> Stefan,
> 
> On Wed, Nov 25 2020 at 14:41, Stefan Bühler wrote:
>> On 11/25/20 12:54 PM, Thomas Gleixner wrote:
>>> On Wed, Sep 16 2020 at 12:12, Stefan Bühler wrote:
>>> Can you please provide the output of:
>>>
>>>  for ID in 05:00.0 06:00.0 06:00.1 06:01.0 06:01.1; do lspci -s $ID -vvv; done
>>
>> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>      ...
>> 	Capabilities: <access denied>
> 
> Can you please run this as root so the Capabilities are accessible?

My bad, sorry. I did intend to run it as root, but should have checked
the output.  Again see attached file.


While we're at it: the EEPROM for the PEX8112 is:

00000000  5a 03 3c 00 10 00 00 00  00 00 00 00 b5 10 12 81  |Z.<.............|
00000010  64 00 20 00 00 00 00 01  04 00 01 00 0c 10 00 fe  |d. .............|
00000020  fe 03 20 10 f0 10 00 00  00 10 33 00 00 00 70 00  |.. .......3...p.|
00000030  00 00 11 00 48 00 00 00  00 00 34 00 50 00 00 00  |....H.....4.P...|
00000040  04 00 55 66 77 88                                 |..Ufw.|
00000046

(This is what the firmware tool provided to me writes, although I think 
the cards usually came pre-flashed with it.  They gave me the tool 
because on some cards the second function on OX16PCI954 was sometimes 
uninitialized, came up with device id 0x9511 "8-bit bus" 
(PCI_DEVICE_ID_OXSEMI_16PCI95N) and the kernel tries to treat it as UART 
too.)

I think some time ago I found a PDF to decode this here:
https://www.broadcom.com/products/pcie-switches-bridges/pcie-bridges/pex8112#documentation

But the broadcom site is completely broken right now (at least for me; 
there own search for "PEX 8112" links it, but then it says "not found").

Anyway, back then I decoded this to:

- `0x5A 0x03`: Magic Header, contains register and shared memory settings
- `0x003C` = 60 bytes for configs (10 registers):
  - `@0x0010`: `0x00000000` -- BAR0: Locate anywhere in 32-bit
  - `@0x0000`: `0x811210B5` -- Vendor `10B5`, Device `8112` (default)
  - `@0x0064`: `0x00000020` -- Device Capability: Enable "Support 8-bit Tag" field
  - `@0x0100`: `0x00010004` -- Power Budget Enhanced Capability Header (default)
  - `@0x100C`: `0x03FEFE00` -- PCI Control:
    - PCI-To-PCI Express Retry Count set to 0xFE (default: `0x80`)
    - PCI Express-to-PCI Retry Count set to 0xFE (default: `0x00`)
  - `@0x1020`: `0x000010F0` -- GPIO Control
    - GPIO[1-3] Output enable (GPIO[0] is Output enabled by default)
    - GPIO Diagnostic Select: `10b` (default: `01b`)
  - `@0x1000`: `0x00000033` -- Device Initialization (default)
  - `@0x0070`: `0x00110000` -- Link control: default
  - `@0x0048`: `0x00000000` -- Device-Specific Control (default 0)
  - `@0x0034`: `0x00000050` -- PCI Capability pointer `0x50` (default: `0x40`)
    - Skips (disables) Power Management Capability
    - Remaining: MSI and PCI Express
- `0x0004` bytes for shared memory:
  - `0x55`, `0x66`, `0x77`, `0x88`


TLDR: the most notable part probably being "disabling Power Management 
Capability" by the EEPROM.

cheers,
Stefan
05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: fb400000-fb4fffff
	Prefetchable memory behind bridge: fff00000-000fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [60] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- SlotPowerLimit 26.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <16us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Power Budgeting <?>

06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	NUMA node: 0
	Region 0: I/O ports at e0e0 [size=32]
	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e0c0 [size=32]
	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: serial

06:00.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e0a0 [disabled] [size=32]
	Region 1: Memory at fb405000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e080 [disabled] [size=32]
	Region 3: Memory at fb404000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

06:01.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 17
	NUMA node: 0
	Region 0: I/O ports at e060 [size=32]
	Region 1: Memory at fb403000 (32-bit, non-prefetchable) [size=4K]
	Region 2: I/O ports at e040 [size=32]
	Region 3: Memory at fb402000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: serial

06:01.1 Bridge: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Disabled)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	NUMA node: 0
	Region 0: I/O ports at e020 [disabled] [size=32]
	Region 1: Memory at fb401000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Region 2: I/O ports at e000 [disabled] [size=32]
	Region 3: Memory at fb400000 (32-bit, non-prefetchable) [disabled] [size=4K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Thomas Gleixner Nov. 30, 2020, 10:48 a.m. UTC | #6
Stefan,

On Fri, Nov 27 2020 at 10:17, Stefan Bühler wrote:
> On 11/27/20 12:45 AM, Thomas Gleixner wrote:
>> Can you please run this as root so the Capabilities are accessible?
>
> My bad, sorry. I did intend to run it as root, but should have checked
> the output.  Again see attached file.

No problem.

> 05:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> 	Physical Slot: 1
> 	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 0, Cache Line Size: 32 bytes
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Bus: primary=05, secondary=06, subordinate=06, sec-latency=64
> 	I/O behind bridge: 0000e000-0000efff
> 	Memory behind bridge: fb400000-fb4fffff
> 	Prefetchable memory behind bridge: fff00000-000fffff
> 	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
> 	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
> 		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> 	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> 		Address: 0000000000000000  Data: 0000

So the bridge would support MSI, but obviously the devices on the PCI
side do not.

> 06:00.0 Serial controller: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart) (prog-if 06 [16950])
> 	Subsystem: Oxford Semiconductor Ltd OX16PCI954 (Quad 16950 UART) function 0 (Uart)
> 	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Interrupt: pin A routed to IRQ 16
> 	NUMA node: 0
> 	Region 0: I/O ports at e0e0 [size=32]
> 	Region 1: Memory at fb407000 (32-bit, non-prefetchable) [size=4K]
> 	Region 2: I/O ports at e0c0 [size=32]
> 	Region 3: Memory at fb406000 (32-bit, non-prefetchable) [size=4K]
> 	Capabilities: [40] Power Management version 1
> 		Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1-,D2+,D3hot+,D3cold-)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

But that should still work because the boot interrupt quirk should not
affect interrupts which are routed through the IOAPIC.

Sean, any idea what's going on here?

Thanks,

        tglx