From patchwork Wed Sep 6 03:36:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dongdong Liu X-Patchwork-Id: 9939899 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CB2986038C for ; Wed, 6 Sep 2017 03:37:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF6002894F for ; Wed, 6 Sep 2017 03:37:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A31D128A28; Wed, 6 Sep 2017 03:37:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6723E2894F for ; Wed, 6 Sep 2017 03:37:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753188AbdIFDhL (ORCPT ); Tue, 5 Sep 2017 23:37:11 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:5546 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752221AbdIFDhJ (ORCPT ); Tue, 5 Sep 2017 23:37:09 -0400 Received: from 172.30.72.59 (EHLO DGGEMS411-HUB.china.huawei.com) ([172.30.72.59]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DGR38901; Wed, 06 Sep 2017 11:37:06 +0800 (CST) Received: from [127.0.0.1] (10.63.141.25) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.301.0; Wed, 6 Sep 2017 11:36:56 +0800 Subject: Re: [PATCH V2] PCI/portdrv: Fix MSI/MSI-X bug for PCIe port service drivers To: Bjorn Helgaas References: <1504091375-116149-1-git-send-email-liudongdong3@huawei.com> <20170906000427.GD14367@bhelgaas-glaptop.roam.corp.google.com> CC: , , , , From: Dongdong Liu Message-ID: <55c0f3e3-879e-054a-6c09-ddf75f0dee8b@huawei.com> Date: Wed, 6 Sep 2017 11:36:55 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20170906000427.GD14367@bhelgaas-glaptop.roam.corp.google.com> X-Originating-IP: [10.63.141.25] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020205.59AF6D62.00CB, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: b414d1c3a930ebbfc72898d705d996c1 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Bjorn Many thanks for your review. 在 2017/9/6 8:04, Bjorn Helgaas 写道: > On Wed, Aug 30, 2017 at 07:09:35PM +0800, Dongdong Liu wrote: >> Current code is broken as calling pci_free_irq_vectors() >> invalidates the IRQ numbers returned before by pci_irq_vectors(); >> so we need to move all the assignment of the Linux IRQ numbers at >> the bottom of the function. >> >> After removing and adding back the PCI root port device, >> we see the PCIe port service drivers request irq failed. > > What exactly is the connection between the root port removal/addition > and the request IRQ failure? The current code does: > > pci_alloc_irq_vectors(dev, ...) > irqs[x] = pci_irq_vector(dev, entry) > pci_free_irq_vectors(dev) > pci_alloc_irq_vectors(dev, ...) > > So I guess the problem is that the second pci_alloc_irq_vectors() call > gets different vectors than the first one? Yes, you are right. [ 477.055527] pcieport 0000:00:00.0: irq [33-64] for MSI [ 477.122631] pcieport 0000:00:00.0: irq [23-25] for MSI [ 477.149352] pcie_pme: probe of 0000:00:00.0:pcie001 failed with error -22 [ 477.193697] pcieport 0000:00:00.0: request AER IRQ 34 failed [ 477.211684] aer: probe of 0000:00:00.0:pcie002 failed with error -22 > How is this related to removal/addition? -[0000:00]-+-00.0-[01-02]--+-00.0 Device 8086:10fb | \-00.1 Device 8086:10fb After we boot the os. ixgbe 0000:01:00.0: irq 23-24. ixgbe 0000:01:00.1: irq 25-26. pcieport 0000:00:00.0: irq [27-29]. After we remove the rp(echo 1 > /sys/devices/pci0000:00/0000:00:00.0/remove). irq 23-29 are free, but the number of irqs does not have 32 (23-29 ) . After we rescan the rp (echo 1 > /sys/bus/pci/rescan) [ 477.055527] pcieport 0000:00:00.0: irq [33-64] for MSI [ 477.122631] pcieport 0000:00:00.0: irq [23-25] for MSI I debug this, the detail information, please see the attchment boot.log. I have not found this. > >> pcie_pme: probe of 0000:00:00.0:pcie001 failed with error -22 >> aer: probe of 0000:00:00.0:pcie002 failed with error -22 >> pciehp 0000:00:00.0:pcie004: Slot #0 AttnBtn- PwrCtrl- MRL- AttnInd- >> PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+ >> pciehp 0000:00:00.0:pcie004: Cannot get irq 20 for the hotplug controller >> pciehp 0000:00:00.0:pcie004: Notification initialization failed (-1) >> dpc 0000:00:00.0:pcie010: request IRQ22 failed: -22 >> dpc: probe of 0000:00:00.0:pcie010 failed with error -22 >> >> Cc: >> Fixes: 3674cc4 ("PCI/portdrv: Use pci_irq_alloc_vectors()") > > Please use a 12-character SHA1. Ok, I will change it as below. Fixes: 3674cc49da9a ("PCI/portdrv: Use pci_irq_alloc_vectors()") > >> Signed-off-by: Dongdong Liu >> Signed-off-by: Gabriele Paoloni >> --- >> v1->v2: >> - Fix comments on PATCH v1. >> - Simplify implementation. >> --- >> drivers/pci/pcie/portdrv_core.c | 17 ++++++++++++----- >> 1 file changed, 12 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c >> index 313a21d..89f4cf5 100644 >> --- a/drivers/pci/pcie/portdrv_core.c >> +++ b/drivers/pci/pcie/portdrv_core.c >> @@ -55,7 +55,8 @@ static void release_pcie_device(struct device *dev) >> static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) >> { >> int nr_entries, entry, nvec = 0; >> - >> + int i; >> + int idx[PCIE_PORT_DEVICE_MAXSERVICES]; >> /* >> * Allocate as many entries as the port wants, so that we can check >> * which of them will be useful. Moreover, if nr_entries is correctly > > This is not so much a question about your patch, but about the whole > interrupt vector allocation strategy in pcie_port_enable_irq_vec(). > > The current code basically does this: > > - allocate 32 vectors > - figure out vector used by PME and hotplug > - figure out vector used by AER > - figure out vector used by DPC > - free the 32 vectors we allocated > - allocate only as many vectors as we need Correct, that is the pcie_port_enable_irq_vec() do now. > > What's the point of allocating those 32 vectors up front? Why can't > we first look up the interrupt message numbers used by each service > (PME, hotplut, AER, DPC), figure out how many vectors we need, > allocate them, and then fill in the slots in the irqs[] table? Agree. > > For example, something like this: > > if (mask & (PCIE_PORT_SERVICE_PME | PCIE_PORT_SERVICE_HP)) { > pcie_capability_read_word(dev, PCI_EXP_FLAGS, ®16); > pme_msg = (reg16 & PCI_EXP_FLAGS_IRQ) >> 9; > } > > if (mask & PCIE_PORT_SERVICE_AER) { > pci_read_config_dword(dev, dev->aer_cap + PCI_ERR_ROOT_STATUS, ®32); > aer_msg = reg32 >> 27; > } > > if (mask & PCIE_PORT_SERVICE_DPC) { > pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC); > pci_read_config_word(dev, pos + PCI_EXP_DPC_CAP, ®16); > dpc_msg = reg16 & 0x1f; > } > > nvec = max(pme_msg, max(aer_msg, dpc_msg)) + 1; > pci_alloc_irq_vectors(dev, nvec, nvec, PCI_IRQ_MSIX | PCI_IRQ_MSI); > > if (mask & PCIE_PORT_SERVICE_PME) > irqs[PCIE_PORT_SERVICE_PME_SHIFT] = pci_irq_vector(dev, pme_msg); > if (mask & PCIE_PORT_SERVICE_HP) > irqs[PCIE_PORT_SERVICE_HP_SHIFT] = pci_irq_vector(dev, pme_msg); > if (mask & PCIE_PORT_SERVICE_AER) > irqs[PCIE_PORT_SERVICE_AER_SHIFT] = pci_irq_vector(dev, aer_msg); > if (mask & PCIE_PORT_SERVICE_DPC) > irqs[PCIE_PORT_SERVICE_DPC_SHIFT] = pci_irq_vector(dev, dpc_msg); > > We currently only look at each service once, and what I'm proposing > would require looking at each service twice (once to read the message > number from the hardware and again (after allocating the vectors) to > fill in the irqs[] table). But I think it would be simpler overall. It looks good to me. Thanks, Dongdong [ 0.000000] Booting Linux on physical CPU 0x30000 [ 0.000000] Linux version 4.13.0-rc7-dirty (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316)) #49 SMP PREEMPT Wed Sep 6 10:29:35 CST 2017 [ 0.000000] Boot CPU: AArch64 Processor [480fd010] [ 0.000000] earlycon: pl11 at MMIO32 0x0000000094080000 (options '') [ 0.000000] bootconsole [pl11] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] efi: EFI v2.60 by EDK II [ 0.000000] efi: ACPI 2.0=0x39ce0000 MEMATTR=0x3e8c4018 [ 0.000000] cma: Reserved 16 MiB at 0x000000003b400000 [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x0000000039CE0000 000024 (v02 HISI ) [ 0.000000] ACPI: XSDT 0x0000000039CD0000 00007C (v01 HISI HIP08 00000000 01000013) [ 0.000000] ACPI: FACP 0x0000000039C10000 000114 (v06 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: DSDT 0x0000000039BB0000 000971 (v02 HISI HIP08 00000000 INTL 20170119) [ 0.000000] ACPI: GTDT 0x0000000039C00000 000098 (v02 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: DBG2 0x0000000039BF0000 00005A (v00 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: MCFG 0x0000000039BE0000 00003C (v01 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: SPCR 0x0000000039BD0000 000050 (v02 HISI HIP07 00000000 INTL 20151124) [ 0.000000] ACPI: APIC 0x0000000039BC0000 0000C0 (v04 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: IORT 0x0000000039BA0000 000120 (v00 HISI D08 00000000 INTL 20170119) [ 0.000000] ACPI: HEST 0x0000000039B80000 00013C (v01 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: EINJ 0x0000000039B60000 000170 (v01 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: ERST 0x0000000031A90000 000230 (v01 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: BERT 0x0000000031A80000 008030 (v01 HISI HIP08 00000000 INTL 20151124) [ 0.000000] ACPI: SPCR: console: pl011,mmio32,0x94080000,115200 [ 0.000000] ACPI: NUMA: Failed to initialise from firmware [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000003fbfffff] [ 0.000000] NUMA: NODE_DATA [mem 0x3fbee500-0x3fbeffff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fbfffff] [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000000000-0x0000000031a5ffff] [ 0.000000] node 0: [mem 0x0000000031a60000-0x0000000031a9ffff] [ 0.000000] node 0: [mem 0x0000000031aa0000-0x0000000039b5ffff] [ 0.000000] node 0: [mem 0x0000000039b60000-0x0000000039b6ffff] [ 0.000000] node 0: [mem 0x0000000039b70000-0x0000000039b79fff] [ 0.000000] node 0: [mem 0x0000000039b7a000-0x0000000039b8ffff] [ 0.000000] node 0: [mem 0x0000000039b90000-0x0000000039b91fff] [ 0.000000] node 0: [mem 0x0000000039b92000-0x000000003a0cffff] [ 0.000000] node 0: [mem 0x000000003a0d0000-0x000000003a0d1fff] [ 0.000000] node 0: [mem 0x000000003a0d2000-0x000000003a0dbfff] [ 0.000000] node 0: [mem 0x000000003a0dc000-0x000000003f0bffff] [ 0.000000] node 0: [mem 0x000000003f0c0000-0x000000003f0effff] [ 0.000000] node 0: [mem 0x000000003f0f0000-0x000000003fbfffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000003fbfffff] [ 0.000000] On node 0 totalpages: 261120 [ 0.000000] DMA zone: 4080 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 261120 pages, LIFO batch:31 [ 0.000000] psci: probing for conduit method from ACPI. NOTICE: [psci_smc_handler]:[397L] PSCI_VERSION CALL [ 0.000000] psci: PSCIv1.0 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: MIGRATE_INFO_TYPE not supported. [ 0.000000] percpu: Embedded 24 pages/cpu @ffff80003fbd3000 s59264 r8192 d30848 u98304 [ 0.000000] pcpu-alloc: s59264 r8192 d30848 u98304 alloc=24*4096 [ 0.000000] pcpu-alloc: [0] 0 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 257040 [ 0.000000] Policy zone: DMA [ 0.000000] Kernel command line: earlycon=pl011,mmio32,0x94080000 initrd=filesystem.cpio.gz acpi=force loglevel=8 [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Memory: 959336K/1044480K available (9148K kernel code, 1108K rwdata, 4136K rodata, 1088K init, 398K bss, 68760K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB) [ 0.000000] .text : 0xffff000008080000 - 0xffff000008970000 ( 9152 KB) [ 0.000000] .rodata : 0xffff000008970000 - 0xffff000008d80000 ( 4160 KB) [ 0.000000] .init : 0xffff000008d80000 - 0xffff000008e90000 ( 1088 KB) [ 0.000000] .data : 0xffff000008e90000 - 0xffff000008fa5200 ( 1109 KB) [ 0.000000] .bss : 0xffff000008fa5200 - 0xffff000009008c70 ( 399 KB) [ 0.000000] fixed : 0xffff7dfffe7fd000 - 0xffff7dfffec00000 ( 4108 KB) [ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB) [ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum) [ 0.000000] 0xffff7e0000000000 - 0xffff7e0000ff0000 ( 15 MB actual) [ 0.000000] memory : 0xffff800000000000 - 0xffff80003fc00000 ( 1020 MB) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1. [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] ACPI: SRAT not present [ 0.000000] ITS [mem 0xa8100000-0xa811ffff] [ 0.000000] ITS@0x00000000a8100000: allocated 8192 Devices @3a100000 (indirect, esz 8, psz 16K, shr 1) [ 0.000000] ITS@0x00000000a8100000: allocated 1024 Virtual CPUs @3a0e4000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] ITS@0x00000000a8100000: allocated 256 Interrupt Collections @3a0e8000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] GIC: using LPI property table @0x000000003a110000 [ 0.000000] ITS: Allocated 1792 chunks for LPIs [ 0.000000] GICv3: CPU0: found redistributor 30000 region 0:0x00000000aa100000 [ 0.000000] CPU0: using LPI pending table @0x000000003a120000 [ 0.000000] arch_timer: cp15 timer(s) running at 6.25MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171024e6b, max_idle_ns: 440795202301 ns [ 0.000329] sched_clock: 56 bits at 6MHz, resolution 160ns, wraps every 4398046511040ns [ 0.022603] Console: colour dummy device 80x25 [ 0.034500] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.50 BogoMIPS (lpj=25000) [ 0.053277] pid_max: default: 32768 minimum: 301 [ 0.062980] ACPI: Core revision 20170531 [ 0.119428] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.136157] Security Framework initialized [ 0.190025] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) [ 0.225984] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.240352] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.253407] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes) [ 0.362723] ASID allocator initialised with 65536 entries [ 0.379364] Hierarchical SRCU implementation. [ 0.409156] PCI/MSI: ITS@0xa8100000 domain created [ 0.420450] Platform MSI: ITS@0xa8100000 domain created [ 0.437286] Remapping and enabling EFI services. [ 0.448258] EFI remap 0x0000000031a60000 => 0000000020000000 [ 0.459979] EFI remap 0x0000000039c30000 => 0000000020020000 [ 0.471578] EFI remap 0x0000000039c80000 => 0000000020070000 [ 0.483099] EFI remap 0x0000000039cf0000 => 00000000200c0000 [ 0.494579] EFI remap 0x0000000039d40000 => 0000000020110000 [ 0.506102] EFI remap 0x0000000039d90000 => 0000000020160000 [ 0.517611] EFI remap 0x0000000039de0000 => 00000000201b0000 [ 0.529248] EFI remap 0x0000000039e30000 => 0000000020200000 [ 0.540737] EFI remap 0x0000000039e80000 => 0000000020250000 [ 0.552216] EFI remap 0x0000000039ed0000 => 00000000202a0000 [ 0.563700] EFI remap 0x0000000039f20000 => 00000000202f0000 [ 0.575540] EFI remap 0x0000000039f70000 => 0000000020340000 [ 0.587015] EFI remap 0x0000000039fe0000 => 00000000203b0000 [ 0.598629] EFI remap 0x000000003a030000 => 0000000020400000 [ 0.610118] EFI remap 0x000000003a080000 => 0000000020450000 [ 0.621288] EFI remap 0x000000003f0c0000 => 00000000204a0000 [ 0.650692] smp: Bringing up secondary CPUs ... [ 0.659943] smp: Brought up 1 node, 1 CPU [ 0.667468] SMP: Total of 1 processors activated. [ 0.676346] CPU features: detected feature: GIC system register CPU interface [ 0.689110] CPU features: detected feature: Privileged Access Never [ 0.700384] CPU features: detected feature: User Access Override [ 0.711175] CPU features: detected feature: Virtualization Host Extensions [ 0.731364] CPU: All CPU(s) started at EL2 [ 0.739396] alternatives: patching kernel code [ 0.920360] devtmpfs: initialized [ 0.990978] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 1.031879] futex hash table entries: 256 (order: 3, 32768 bytes) [ 1.100210] pinctrl core: initialized pinctrl subsystem [ 1.231764] random: fast init done [ 1.441138] DMI not present or invalid. [ 1.531527] NET: Registered protocol family 16 [ 1.954694] cpuidle: using governor menu [ 1.997861] vdso: 2 pages (1 code @ ffff000008976000, 1 data @ ffff000008e95000) [ 2.014553] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers. [ 2.086616] DMA: preallocated 256 KiB pool for atomic allocations [ 2.169828] ACPI: bus type PCI registered [ 2.269948] Serial: AMBA PL011 UART driver [ 6.266490] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 6.442895] ACPI: Added _OSI(Module Device) [ 6.452727] ACPI: Added _OSI(Processor Device) [ 6.461662] ACPI: Added _OSI(3.0 _SCP Extensions) [ 6.470751] ACPI: Added _OSI(Processor Aggregator Device) [ 6.567598] ACPI: Interpreter enabled [ 6.575160] ACPI: Using GIC for interrupt routing [ 6.595078] ACPI: MCFG table detected, 1 entries [ 7.427562] ARMH0011:00: ttyAMA0 at MMIO 0x94080000 (irq = 5, base_baud = 0) is a SBSA [ 7.446220] console [ttyAMA0] enabled [ 7.446220] console [ttyAMA0] enabled [ 7.458472] bootconsole [pl11] disabled [ 7.458472] bootconsole [pl11] disabled [ 7.536276] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3f]) [ 7.544988] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 7.584851] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] [ 7.617867] acpi PNP0A08:00: [Firmware Bug]: ECAM area [mem 0xd0000000-0xd3ffffff] not reserved in ACPI namespace [ 7.630773] acpi PNP0A08:00: ECAM at [mem 0xd0000000-0xd3ffffff] for [bus 00-3f] [ 7.645534] Remapped I/O 0x00000000e0000000 to [io 0x0000-0xffff window] [ 7.669434] PCI host bridge to bus 0000:00 [ 7.682031] pci_bus 0000:00: root bus resource [mem 0x80000000000-0x83fffffffff pref window] [ 7.700166] pci_bus 0000:00: root bus resource [mem 0xe0010000-0xe7ffffff window] [ 7.709762] pci_bus 0000:00: root bus resource [io 0x0000-0xffff window] [ 7.718134] pci_bus 0000:00: root bus resource [bus 00-3f] [ 7.725526] pci_bus 0000:00: scanning bus [ 7.732374] pci 0000:00:00.0: [19e5:a120] type 01 class 0x060400 [ 7.748480] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 7.756070] pci 0000:00:00.0: PME# disabled [ 7.781356] pci 0000:00:08.0: [19e5:a120] type 01 class 0x060400 [ 7.809322] pci 0000:00:08.0: PME# supported from D0 D1 D2 D3hot D3cold [ 7.817775] pci 0000:00:08.0: PME# disabled [ 7.842221] pci_bus 0000:00: fixups for bus [ 7.857795] pci 0000:00:00.0: scanning [bus 01-02] behind bridge, pass 0 [ 7.881935] pci_bus 0000:01: scanning bus [ 7.894711] pci 0000:01:00.0: [8086:10fb] type 00 class 0x020000 [ 7.905129] pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x84 [ 7.914365] pci 0000:01:00.0: reg 0x10: [mem 0x80000c00000-0x80000ffffff 64bit pref] [ 7.923845] pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] [ 7.931977] pci 0000:01:00.0: reg 0x20: [mem 0x80001404000-0x80001407fff 64bit pref] [ 7.941331] pci 0000:01:00.0: reg 0x30: [mem 0xffc00000-0xffffffff pref] [ 7.954316] pci 0000:01:00.0: PME# supported from D0 D3hot [ 7.960786] pci 0000:01:00.0: PME# disabled [ 7.967206] pci 0000:01:00.0: reg 0x184: [mem 0x80001304000-0x80001307fff 64bit pref] [ 7.975904] pci 0000:01:00.0: VF(n) BAR0 space: [mem 0x80001304000-0x80001403fff 64bit pref] (contains BAR0 for 64 VFs) [ 7.988004] pci 0000:01:00.0: reg 0x190: [mem 0x80001204000-0x80001207fff 64bit pref] [ 7.996493] pci 0000:01:00.0: VF(n) BAR3 space: [mem 0x80001204000-0x80001303fff 64bit pref] (contains BAR3 for 64 VFs) [ 8.036499] pci 0000:01:00.1: [8086:10fb] type 00 class 0x020000 [ 8.055046] pci 0000:01:00.1: calling quirk_f0_vpd_link+0x0/0x84 [ 8.064331] pci 0000:01:00.1: reg 0x10: [mem 0x80000800000-0x80000bfffff 64bit pref] [ 8.074492] pci 0000:01:00.1: reg 0x18: [io 0x0000-0x001f] [ 8.082047] pci 0000:01:00.1: reg 0x20: [mem 0x80001200000-0x80001203fff 64bit pref] [ 8.091130] pci 0000:01:00.1: reg 0x30: [mem 0xffc00000-0xffffffff pref] [ 8.103548] pci 0000:01:00.1: PME# supported from D0 D3hot [ 8.109878] pci 0000:01:00.1: PME# disabled [ 8.115950] pci 0000:01:00.1: reg 0x184: [mem 0x80001100000-0x80001103fff 64bit pref] [ 8.124516] pci 0000:01:00.1: VF(n) BAR0 space: [mem 0x80001100000-0x800011fffff 64bit pref] (contains BAR0 for 64 VFs) [ 8.136598] pci 0000:01:00.1: reg 0x190: [mem 0x80001000000-0x80001003fff 64bit pref] [ 8.145071] pci 0000:01:00.1: VF(n) BAR3 space: [mem 0x80001000000-0x800010fffff 64bit pref] (contains BAR3 for 64 VFs) [ 8.186151] pci_bus 0000:01: fixups for bus [ 8.200134] pci_bus 0000:01: bus scan returning with max=01 [ 8.209477] pci 0000:00:08.0: scanning [bus 03-04] behind bridge, pass 0 [ 8.231444] pci_bus 0000:03: scanning bus [ 8.244266] pci 0000:03:00.0: [8086:1521] type 00 class 0x020000 [ 8.255004] pci 0000:03:00.0: calling quirk_f0_vpd_link+0x0/0x84 [ 8.264702] pci 0000:03:00.0: reg 0x10: [mem 0xe0300000-0xe03fffff] [ 8.273645] pci 0000:03:00.0: reg 0x1c: [mem 0xe040c000-0xe040ffff] [ 8.282075] pci 0000:03:00.0: reg 0x30: [mem 0xfff00000-0xffffffff pref] [ 8.295226] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold [ 8.302214] pci 0000:03:00.0: PME# disabled [ 8.308536] pci 0000:03:00.0: reg 0x184: [mem 0x800015e0000-0x800015e3fff 64bit pref] [ 8.317179] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x800015e0000-0x800015fffff 64bit pref] (contains BAR0 for 8 VFs) [ 8.329199] pci 0000:03:00.0: reg 0x190: [mem 0x800015c0000-0x800015c3fff 64bit pref] [ 8.337636] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x800015c0000-0x800015dffff 64bit pref] (contains BAR3 for 8 VFs) [ 8.368526] pci 0000:03:00.1: [8086:1521] type 00 class 0x020000 [ 8.386885] pci 0000:03:00.1: calling quirk_f0_vpd_link+0x0/0x84 [ 8.395857] pci 0000:03:00.1: reg 0x10: [mem 0xe0200000-0xe02fffff] [ 8.404993] pci 0000:03:00.1: reg 0x1c: [mem 0xe0408000-0xe040bfff] [ 8.413251] pci 0000:03:00.1: reg 0x30: [mem 0xfff00000-0xffffffff pref] [ 8.426293] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold [ 8.433273] pci 0000:03:00.1: PME# disabled [ 8.439321] pci 0000:03:00.1: reg 0x184: [mem 0x800015a0000-0x800015a3fff 64bit pref] [ 8.447914] pci 0000:03:00.1: VF(n) BAR0 space: [mem 0x800015a0000-0x800015bffff 64bit pref] (contains BAR0 for 8 VFs) [ 8.459963] pci 0000:03:00.1: reg 0x190: [mem 0x80001580000-0x80001583fff 64bit pref] [ 8.468410] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x80001580000-0x8000159ffff 64bit pref] (contains BAR3 for 8 VFs) [ 8.499856] pci 0000:03:00.2: [8086:1521] type 00 class 0x020000 [ 8.518734] pci 0000:03:00.2: calling quirk_f0_vpd_link+0x0/0x84 [ 8.527539] pci 0000:03:00.2: reg 0x10: [mem 0xe0100000-0xe01fffff] [ 8.536650] pci 0000:03:00.2: reg 0x1c: [mem 0xe0404000-0xe0407fff] [ 8.550703] pci 0000:03:00.2: PME# supported from D0 D3hot D3cold [ 8.557839] pci 0000:03:00.2: PME# disabled [ 8.563852] pci 0000:03:00.2: reg 0x184: [mem 0x80001560000-0x80001563fff 64bit pref] [ 8.572450] pci 0000:03:00.2: VF(n) BAR0 space: [mem 0x80001560000-0x8000157ffff 64bit pref] (contains BAR0 for 8 VFs) [ 8.584466] pci 0000:03:00.2: reg 0x190: [mem 0x80001540000-0x80001543fff 64bit pref] [ 8.592948] pci 0000:03:00.2: VF(n) BAR3 space: [mem 0x80001540000-0x8000155ffff 64bit pref] (contains BAR3 for 8 VFs) [ 8.623880] pci 0000:03:00.3: [8086:1521] type 00 class 0x020000 [ 8.643212] pci 0000:03:00.3: calling quirk_f0_vpd_link+0x0/0x84 [ 8.652565] pci 0000:03:00.3: reg 0x10: [mem 0xe0000000-0xe00fffff] [ 8.661184] pci 0000:03:00.3: reg 0x1c: [mem 0xe0400000-0xe0403fff] [ 8.675795] pci 0000:03:00.3: PME# supported from D0 D3hot D3cold [ 8.683456] pci 0000:03:00.3: PME# disabled [ 8.689539] pci 0000:03:00.3: reg 0x184: [mem 0x80001520000-0x80001523fff 64bit pref] [ 8.698171] pci 0000:03:00.3: VF(n) BAR0 space: [mem 0x80001520000-0x8000153ffff 64bit pref] (contains BAR0 for 8 VFs) [ 8.710194] pci 0000:03:00.3: reg 0x190: [mem 0x80001500000-0x80001503fff 64bit pref] [ 8.718661] pci 0000:03:00.3: VF(n) BAR3 space: [mem 0x80001500000-0x8000151ffff 64bit pref] (contains BAR3 for 8 VFs) [ 8.751770] pci_bus 0000:03: fixups for bus [ 8.767326] pci_bus 0000:03: bus scan returning with max=03 [ 8.775771] pci 0000:00:00.0: scanning [bus 01-02] behind bridge, pass 1 [ 8.784088] pci 0000:00:08.0: scanning [bus 03-04] behind bridge, pass 1 [ 8.792775] pci_bus 0000:00: bus scan returning with max=04 [ 8.802531] pci 0000:00:00.0: BAR 14: assigned [mem 0xe0400000-0xe0bfffff] [ 8.811578] pci 0000:00:00.0: BAR 15: assigned [mem 0x80000000000-0x80000dfffff 64bit pref] [ 8.820739] pci 0000:00:08.0: BAR 14: assigned [mem 0xe0c00000-0xe12fffff] [ 8.828338] pci 0000:00:08.0: BAR 15: assigned [mem 0x80000e00000-0x80000ffffff 64bit pref] [ 8.837354] pci 0000:00:00.0: BAR 13: assigned [io 0x1000-0x1fff] [ 8.844176] pci 0000:00:08.0: BAR 13: assigned [io 0x2000-0x2fff] [ 8.851420] pci 0000:01:00.0: BAR 0: assigned [mem 0x80000000000-0x800003fffff 64bit pref] [ 8.860790] pci 0000:01:00.0: BAR 6: assigned [mem 0xe0400000-0xe07fffff pref] [ 8.868711] pci 0000:01:00.1: BAR 0: assigned [mem 0x80000400000-0x800007fffff 64bit pref] [ 8.878024] pci 0000:01:00.1: BAR 6: assigned [mem 0xe0800000-0xe0bfffff pref] [ 8.885861] pci 0000:01:00.0: BAR 4: assigned [mem 0x80000800000-0x80000803fff 64bit pref] [ 8.895108] pci 0000:01:00.0: BAR 7: assigned [mem 0x80000804000-0x80000903fff 64bit pref] [ 8.904170] pci 0000:01:00.0: BAR 10: assigned [mem 0x80000904000-0x80000a03fff 64bit pref] [ 8.913312] pci 0000:01:00.1: BAR 4: assigned [mem 0x80000a04000-0x80000a07fff 64bit pref] [ 8.922554] pci 0000:01:00.1: BAR 7: assigned [mem 0x80000a08000-0x80000b07fff 64bit pref] [ 8.931594] pci 0000:01:00.1: BAR 10: assigned [mem 0x80000b08000-0x80000c07fff 64bit pref] [ 8.940710] pci 0000:01:00.0: BAR 2: assigned [io 0x1000-0x101f] [ 8.947512] pci 0000:01:00.1: BAR 2: assigned [io 0x1020-0x103f] [ 8.956549] pci 0000:00:00.0: PCI bridge to [bus 01-02] [ 8.962641] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 8.969541] pci 0000:00:00.0: bridge window [mem 0xe0400000-0xe0bfffff] [ 8.977069] pci 0000:00:00.0: bridge window [mem 0x80000000000-0x80000dfffff 64bit pref] [ 8.986830] pci 0000:03:00.0: BAR 0: assigned [mem 0xe0c00000-0xe0cfffff] [ 8.994486] pci 0000:03:00.0: BAR 6: assigned [mem 0xe0d00000-0xe0dfffff pref] [ 9.002391] pci 0000:03:00.1: BAR 0: assigned [mem 0xe0e00000-0xe0efffff] [ 9.009901] pci 0000:03:00.1: BAR 6: assigned [mem 0xe0f00000-0xe0ffffff pref] [ 9.017770] pci 0000:03:00.2: BAR 0: assigned [mem 0xe1000000-0xe10fffff] [ 9.025268] pci 0000:03:00.3: BAR 0: assigned [mem 0xe1100000-0xe11fffff] [ 9.032763] pci 0000:03:00.0: BAR 3: assigned [mem 0xe1200000-0xe1203fff] [ 9.040281] pci 0000:03:00.0: BAR 7: assigned [mem 0x80000e00000-0x80000e1ffff 64bit pref] [ 9.049319] pci 0000:03:00.0: BAR 10: assigned [mem 0x80000e20000-0x80000e3ffff 64bit pref] [ 9.058434] pci 0000:03:00.1: BAR 3: assigned [mem 0xe1204000-0xe1207fff] [ 9.065936] pci 0000:03:00.1: BAR 7: assigned [mem 0x80000e40000-0x80000e5ffff 64bit pref] [ 9.074953] pci 0000:03:00.1: BAR 10: assigned [mem 0x80000e60000-0x80000e7ffff 64bit pref] [ 9.084074] pci 0000:03:00.2: BAR 3: assigned [mem 0xe1208000-0xe120bfff] [ 9.091562] pci 0000:03:00.2: BAR 7: assigned [mem 0x80000e80000-0x80000e9ffff 64bit pref] [ 9.100587] pci 0000:03:00.2: BAR 10: assigned [mem 0x80000ea0000-0x80000ebffff 64bit pref] [ 9.109690] pci 0000:03:00.3: BAR 3: assigned [mem 0xe120c000-0xe120ffff] [ 9.117187] pci 0000:03:00.3: BAR 7: assigned [mem 0x80000ec0000-0x80000edffff 64bit pref] [ 9.126233] pci 0000:03:00.3: BAR 10: assigned [mem 0x80000ee0000-0x80000efffff 64bit pref] [ 9.135511] pci 0000:00:08.0: PCI bridge to [bus 03-04] [ 9.141357] pci 0000:00:08.0: bridge window [io 0x2000-0x2fff] [ 9.148217] pci 0000:00:08.0: bridge window [mem 0xe0c00000-0xe12fffff] [ 9.155707] pci 0000:00:08.0: bridge window [mem 0x80000e00000-0x80000ffffff 64bit pref] [ 9.560015] vgaarb: loaded [ 9.640424] SCSI subsystem initialized [ 9.698093] libata version 3.00 loaded. [ 9.780441] ACPI: bus type USB registered [ 9.826557] usbcore: registered new interface driver usbfs [ 9.854891] usbcore: registered new interface driver hub [ 9.878112] usbcore: registered new device driver usb [ 10.024058] pps_core: LinuxPPS API ver. 1 registered [ 10.040988] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 10.058492] PTP clock support registered [ 10.136679] dmi: Firmware registration failed. [ 10.150629] Registered efivars operations [ 10.218533] Advanced Linux Sound Architecture Driver Initialized. [ 10.372769] clocksource: Switched to clocksource arch_sys_counter [ 10.396969] VFS: Disk quotas dquot_6.6.0 [ 10.406240] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 10.454282] pnp: PnP ACPI init [ 10.496852] pnp: PnP ACPI: found 0 devices [ 12.594530] NET: Registered protocol family 2 [ 12.641412] TCP established hash table entries: 8192 (order: 4, 65536 bytes) [ 12.653888] TCP bind hash table entries: 8192 (order: 5, 131072 bytes) [ 12.671318] TCP: Hash tables configured (established 8192 bind 8192) [ 12.687407] UDP hash table entries: 512 (order: 2, 16384 bytes) [ 12.696177] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes) [ 12.713559] NET: Registered protocol family 1 [ 12.751919] RPC: Registered named UNIX socket transport module. [ 12.760732] RPC: Registered udp transport module. [ 12.766108] RPC: Registered tcp transport module. [ 12.772643] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 12.783117] pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.791253] pci 0000:01:00.1: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.798827] pci 0000:03:00.0: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.805985] PCI: CLS mismatch (64 != 128), using 128 bytes [ 12.812477] pci 0000:03:00.1: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.819852] pci 0000:03:00.2: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.827500] pci 0000:03:00.3: calling quirk_e100_interrupt+0x0/0x1c0 [ 12.853503] Unpacking initramfs... [ 93.115023] Initramfs unpacking failed: junk in compressed archive [ 94.520143] Freeing initrd memory: 28172K [ 94.541265] hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 13 counters available [ 94.551107] kvm [1]: 16-bit VMID [ 94.555180] kvm [1]: VHE mode initialized successfully [ 94.561474] kvm [1]: GICv3: no GICV resource entry [ 94.566985] kvm [1]: disabling GICv2 emulation [ 94.573285] kvm [1]: GIC system register CPU interface enabled [ 94.583554] kvm [1]: vgic interrupt IRQ1 [ 94.589094] kvm [1]: virtual timer IRQ3 [ 94.980154] audit: initializing netlink subsys (disabled) [ 95.009820] audit: type=2000 audit(92.583:1): state=initialized audit_enabled=0 res=1 [ 95.127062] workingset: timestamp_bits=44 max_order=18 bucket_order=0 [ 96.798116] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 96.969554] NFS: Registering the id_resolver key type [ 96.979188] Key type id_resolver registered [ 96.985193] Key type id_legacy registered [ 96.991028] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 97.018200] 9p: Installing v9fs 9p2000 file system support [ 97.330580] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 97.351086] io scheduler noop registered [ 97.403202] io scheduler cfq registered (default) [ 97.418177] io scheduler mq-deadline registered [ 97.426968] io scheduler kyber registered [ 98.764128] acpi-ged ACPI0013:00: GED listening GSI 129 @ IRQ 6 [ 100.898280] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 101.310451] SuperH (H)SCI(F) driver initialized [ 101.381953] msm_serial: driver initialized [ 101.533722] arm-smmu-v3 arm-smmu-v3.0.auto: option mask 0x0 [ 101.545745] arm-smmu-v3 arm-smmu-v3.0.auto: ias 48-bit, oas 48-bit (features 0x00000fef) [ 101.578260] arm-smmu-v3 arm-smmu-v3.0.auto: failed to allocate MSIs [ 101.622191] cacheinfo: Unable to detect cache hierarchy for CPU 0 [ 102.507569] loop: module loaded [ 102.734046] hisi_sas: driver version v1.6 [ 103.361973] libphy: Fixed MDIO Bus: probed [ 103.582183] tun: Universal TUN/TAP device driver, 1.6 [ 103.811192] libphy: Hisilicon MII Bus: probed [ 103.886133] hns-nic HISI00C2:00: not find ae-handle [ 103.902399] hns-nic: probe of HISI00C2:00 failed with error -22 [ 103.920648] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k [ 103.937988] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. [ 103.961915] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k [ 103.981971] igb: Copyright (c) 2007-2014 Intel Corporation. [ 104.026744] iommu: Adding device 0000:03:00.0 to group 0 [ 104.039941] igb 0000:03:00.0: runtime IRQ mapping not provided by arch [ 104.060977] igb 0000:03:00.0: enabling device (0000 -> 0002) [ 104.069357] igb 0000:03:00.0: enabling bus mastering [ 104.107285] igb 0000:03:00.0: irq 9 for MSI [ 104.137704] igb 0000:03:00.0: irq 9 for MSI [ 104.143757] igb 0000:03:00.0: irq 10 for MSI [ 104.149258] igb 0000:03:00.0: irq 11 for MSI [ 104.762718] igb 0000:03:00.0: added PHC on eth0 [ 104.778186] igb 0000:03:00.0: Intel(R) Gigabit Ethernet Network Connection [ 104.790609] igb 0000:03:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 60:08:10:b9:d8:61 [ 104.800924] igb 0000:03:00.0: eth0: PBA No: 106300-000 [ 104.807883] igb 0000:03:00.0: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s) [ 104.839294] iommu: Adding device 0000:03:00.1 to group 1 [ 104.853148] igb 0000:03:00.1: runtime IRQ mapping not provided by arch [ 104.873162] igb 0000:03:00.1: enabling device (0000 -> 0002) [ 104.881690] igb 0000:03:00.1: enabling bus mastering [ 104.919958] igb 0000:03:00.1: irq 13 for MSI [ 104.947522] igb 0000:03:00.1: irq 13 for MSI [ 104.954060] igb 0000:03:00.1: irq 14 for MSI [ 104.959320] igb 0000:03:00.1: irq 15 for MSI [ 105.493974] igb 0000:03:00.1: added PHC on eth1 [ 105.507684] igb 0000:03:00.1: Intel(R) Gigabit Ethernet Network Connection [ 105.520897] igb 0000:03:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 60:08:10:b9:d8:62 [ 105.530370] igb 0000:03:00.1: eth1: PBA No: 106300-000 [ 105.536968] igb 0000:03:00.1: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s) [ 105.567646] iommu: Adding device 0000:03:00.2 to group 2 [ 105.581190] igb 0000:03:00.2: runtime IRQ mapping not provided by arch [ 105.601115] igb 0000:03:00.2: enabling device (0000 -> 0002) [ 105.609543] igb 0000:03:00.2: enabling bus mastering [ 105.646074] igb 0000:03:00.2: irq 17 for MSI [ 105.674381] igb 0000:03:00.2: irq 17 for MSI [ 105.683278] igb 0000:03:00.2: irq 18 for MSI [ 105.688778] igb 0000:03:00.2: irq 19 for MSI [ 106.217634] igb 0000:03:00.2: added PHC on eth2 [ 106.233860] igb 0000:03:00.2: Intel(R) Gigabit Ethernet Network Connection [ 106.243657] igb 0000:03:00.2: eth2: (PCIe:2.5Gb/s:Width x4) 60:08:10:b9:d8:63 [ 106.254372] igb 0000:03:00.2: eth2: PBA No: 106300-000 [ 106.263316] igb 0000:03:00.2: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s) [ 106.295470] iommu: Adding device 0000:03:00.3 to group 3 [ 106.308924] igb 0000:03:00.3: runtime IRQ mapping not provided by arch [ 106.321877] pci 0000:00:08.0: can't derive routing for PCI INT D [ 106.328793] igb 0000:03:00.3: PCI INT D: no GSI [ 106.334925] igb 0000:03:00.3: enabling device (0000 -> 0002) [ 106.342987] igb 0000:03:00.3: enabling bus mastering [ 106.381164] igb 0000:03:00.3: irq 20 for MSI [ 106.395108] pci 0000:00:08.0: can't derive routing for PCI INT D [ 106.402247] igb 0000:03:00.3: PCI INT D: no GSI [ 106.423980] igb 0000:03:00.3: irq 20 for MSI [ 106.430404] igb 0000:03:00.3: irq 21 for MSI [ 106.435604] igb 0000:03:00.3: irq 22 for MSI [ 106.969195] igb 0000:03:00.3: added PHC on eth3 [ 106.986599] igb 0000:03:00.3: Intel(R) Gigabit Ethernet Network Connection [ 106.997192] igb 0000:03:00.3: eth3: (PCIe:2.5Gb/s:Width x4) 60:08:10:b9:d8:64 [ 107.006052] igb 0000:03:00.3: eth3: PBA No: 106300-000 [ 107.014340] igb 0000:03:00.3: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s) [ 107.034774] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k [ 107.051759] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. [ 107.073731] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 5.1.0-k [ 107.093439] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 107.147616] iommu: Adding device 0000:01:00.0 to group 4 [ 107.161275] ixgbe 0000:01:00.0: runtime IRQ mapping not provided by arch [ 107.176976] ixgbe 0000:01:00.0: enabling device (0000 -> 0002) [ 107.185395] ixgbe 0000:01:00.0: enabling bus mastering [ 108.634988] ixgbe 0000:01:00.0: irq 23 for MSI [ 108.643436] ixgbe 0000:01:00.0: irq 24 for MSI [ 108.651323] ixgbe 0000:01:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1 XDP Queue count = 0 [ 108.663349] ixgbe 0000:01:00.0: PCI Express bandwidth of 8GT/s available [ 108.670828] ixgbe 0000:01:00.0: (Speed:2.5GT/s, Width: x4, Encoding Loss:20%) [ 108.678675] ixgbe 0000:01:00.0: This is not sufficient for optimal performance of this card. [ 108.687764] ixgbe 0000:01:00.0: For optimal performance, at least 20GT/s of bandwidth is required. [ 108.697626] ixgbe 0000:01:00.0: A slot with more lanes and/or higher speed is suggested. [ 108.706705] ixgbe 0000:01:00.0: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF [ 108.713609] ixgbe 0000:01:00.0: 9c:37:f4:90:cd:23 [ 110.602795] ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection [ 110.637473] iommu: Adding device 0000:01:00.1 to group 5 [ 110.650676] ixgbe 0000:01:00.1: runtime IRQ mapping not provided by arch [ 110.665725] ixgbe 0000:01:00.1: enabling device (0000 -> 0002) [ 110.674161] ixgbe 0000:01:00.1: enabling bus mastering [ 111.150789] ixgbe 0000:01:00.1: irq 25 for MSI [ 111.159276] ixgbe 0000:01:00.1: irq 26 for MSI [ 111.167407] ixgbe 0000:01:00.1: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1 XDP Queue count = 0 [ 111.179704] ixgbe 0000:01:00.1: PCI Express bandwidth of 8GT/s available [ 111.187549] ixgbe 0000:01:00.1: (Speed:2.5GT/s, Width: x4, Encoding Loss:20%) [ 111.195360] ixgbe 0000:01:00.1: This is not sufficient for optimal performance of this card. [ 111.204463] ixgbe 0000:01:00.1: For optimal performance, at least 20GT/s of bandwidth is required. [ 111.214037] ixgbe 0000:01:00.1: A slot with more lanes and/or higher speed is suggested. [ 111.223129] ixgbe 0000:01:00.1: MAC: 2, PHY: 17, SFP+: 10, PBA No: FFFFFF-0FF [ 111.230947] ixgbe 0000:01:00.1: 9c:37:f4:90:cd:24 [ 113.207022] ixgbe 0000:01:00.1: Intel(R) 10 Gigabit Network Connection [ 113.322096] sky2: driver version 1.30 [ 113.462071] VFIO - User Level meta-driver version: 0.3 [ 113.771122] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 113.787380] ehci-pci: EHCI PCI platform driver [ 113.806857] ehci-platform: EHCI generic platform driver [ 113.839730] ehci-orion: EHCI orion driver [ 113.870892] ehci-exynos: EHCI EXYNOS driver [ 113.901359] ehci-msm: Qualcomm On-Chip EHCI Host Controller [ 113.935225] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 113.954224] ohci-pci: OHCI PCI platform driver [ 113.975549] ohci-platform: OHCI generic platform driver [ 114.010294] ohci-exynos: OHCI EXYNOS driver [ 114.141756] usbcore: registered new interface driver usb-storage [ 114.773837] i2c /dev entries driver [ 115.641863] sdhci: Secure Digital Host Controller Interface driver [ 115.663068] sdhci: Copyright(c) Pierre Ossman [ 115.729786] Synopsys Designware Multimedia Card Interface Driver [ 115.918274] sdhci-pltfm: SDHCI platform and OF driver helper [ 116.189809] ledtrig-cpu: registered to indicate activity on CPUs [ 116.554506] usbcore: registered new interface driver usbhid [ 116.569432] usbhid: USB HID core driver [ 116.995307] NET: Registered protocol family 17 [ 117.008752] 9pnet: Installing 9P2000 support [ 117.025356] Key type dns_resolver registered [ 117.170735] registered taskstats version 1 [ 117.231555] iommu: Adding device 0000:00:00.0 to group 6 [ 117.245350] pcieport 0000:00:00.0: runtime IRQ mapping not provided by arch [ 117.297232] pcieport 0000:00:00.0: irq [27-58] for MSI [ 117.361464] pcieport 0000:00:00.0: irq [27-29] for MSI [ 117.389455] pcieport 0000:00:00.0: Signaling PME with IRQ 27 [ 117.446120] pcieport 0000:00:00.0: AER enabled with IRQ 28 [ 117.475969] pciehp 0000:00:00.0:pcie004: Slot #0 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+ [ 117.506102] pci_bus 0000:01: dev 00, created physical slot 0 [ 117.592065] dpc 0000:00:00.0:pcie010: DPC error containment capabilities: Int Msg #2, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+ [ 117.632624] iommu: Adding device 0000:00:08.0 to group 7 [ 117.645307] pcieport 0000:00:08.0: runtime IRQ mapping not provided by arch [ 117.693898] pcieport 0000:00:08.0: irq [30-61] for MSI [ 117.757333] pcieport 0000:00:08.0: irq [30-32] for MSI [ 117.785514] pcieport 0000:00:08.0: Signaling PME with IRQ 30 [ 117.833459] pcieport 0000:00:08.0: AER enabled with IRQ 31 [ 117.870840] pciehp 0000:00:08.0:pcie004: Slot #0 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+ [ 117.898814] pci_bus 0000:03: dev 00, created physical slot 0-1 [ 117.966720] dpc 0000:00:08.0:pcie010: DPC error containment capabilities: Int Msg #2, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+ [ 118.017679] hctosys: unable to open rtc device (rtc0) [ 118.609748] ALSA device list: [ 118.614432] No soundcards found. [ 118.691983] Freeing unused kernel memory: 1088K root@(none)$ root@(none)$ cd /home/ root@(none)$ ./lspci -tv -[0000:00]-+-00.0-[01-02]--+-00.0 Device 8086:10fb | \-00.1 Device 8086:10fb \-08.0-[03-04]--+-00.0 Device 8086:1521 +-00.1 Device 8086:1521 +-00.2 Device 8086:1521 \-00.3 Device 8086:1521 root@(none)$ echo1 1 > /sys/devices/pci0000:00/0000:00:00.0/remove [ 447.630066] ixgbe 0000:01:00.1: PME# disabled [ 449.503490] ixgbe 0000:01:00.1: complete [ 449.515380] ixgbe 0000:01:00.0: PME# disabled [ 451.323235] ixgbe 0000:01:00.0: complete [ 451.335069] pcieport 0000:00:00.0: PME# disabled [ 451.398016] pci_bus 0000:01: dev 00, dec refcount to 0 [ 451.414247] pci_bus 0000:01: dev 00, released physical slot 0 [ 451.523677] iommu: Removing device 0000:01:00.0 from group 4 [ 451.563037] iommu: Removing device 0000:01:00.1 from group 5 [ 451.596134] pci_bus 0000:01: busn_res: [bus 01-02] is released [ 451.643249] iommu: Removing device 0000:00:00.0 from group 6 root@(none)$ echo 1 > /sys/bus/pci/rescan [ 476.261084] pci_bus 0000:00: scanning bus [ 476.268738] pci 0000:00:00.0: [19e5:a120] type 01 class 0x060400 [ 476.285212] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 476.292934] pci 0000:00:00.0: PME# disabled [ 476.323146] pcieport 0000:00:08.0: scanning [bus 03-04] behind bridge, pass 0 [ 476.342380] pci_bus 0000:03: scanning bus [ 476.355514] pci_bus 0000:03: bus scan returning with max=03 [ 476.363649] pci 0000:00:00.0: scanning [bus 01-02] behind bridge, pass 0 [ 476.389654] pci_bus 0000:01: scanning bus [ 476.403577] pci 0000:01:00.0: [8086:10fb] type 00 class 0x020000 [ 476.418389] pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x84 [ 476.427450] pci 0000:01:00.0: reg 0x10: [mem 0x80000000000-0x800003fffff 64bit pref] [ 476.438299] pci 0000:01:00.0: reg 0x18: [io 0x1000-0x101f] [ 476.446411] pci 0000:01:00.0: reg 0x20: [mem 0x80000800000-0x80000803fff 64bit pref] [ 476.456342] pci 0000:01:00.0: reg 0x30: [mem 0xffc00000-0xffffffff pref] [ 476.469753] pci 0000:01:00.0: PME# supported from D0 D3hot [ 476.476429] pci 0000:01:00.0: PME# disabled [ 476.482955] pci 0000:01:00.0: reg 0x184: [mem 0x80000804000-0x80000807fff 64bit pref] [ 476.491789] pci 0000:01:00.0: VF(n) BAR0 space: [mem 0x80000804000-0x80000903fff 64bit pref] (contains BAR0 for 64 VFs) [ 476.504795] pci 0000:01:00.0: reg 0x190: [mem 0x80000904000-0x80000907fff 64bit pref] [ 476.513379] pci 0000:01:00.0: VF(n) BAR3 space: [mem 0x80000904000-0x80000a03fff 64bit pref] (contains BAR3 for 64 VFs) [ 476.555511] pci 0000:01:00.1: [8086:10fb] type 00 class 0x020000 [ 476.578766] pci 0000:01:00.1: calling quirk_f0_vpd_link+0x0/0x84 [ 476.589784] pci 0000:01:00.1: reg 0x10: [mem 0x80000400000-0x800007fffff 64bit pref] [ 476.599717] pci 0000:01:00.1: reg 0x18: [io 0x1020-0x103f] [ 476.608373] pci 0000:01:00.1: reg 0x20: [mem 0x80000a04000-0x80000a07fff 64bit pref] [ 476.617225] pci 0000:01:00.1: reg 0x30: [mem 0xffc00000-0xffffffff pref] [ 476.629753] pci 0000:01:00.1: PME# supported from D0 D3hot [ 476.636368] pci 0000:01:00.1: PME# disabled [ 476.642766] pci 0000:01:00.1: reg 0x184: [mem 0x80000a08000-0x80000a0bfff 64bit pref] [ 476.651533] pci 0000:01:00.1: VF(n) BAR0 space: [mem 0x80000a08000-0x80000b07fff 64bit pref] (contains BAR0 for 64 VFs) [ 476.664255] pci 0000:01:00.1: reg 0x190: [mem 0x80000b08000-0x80000b0bfff 64bit pref] [ 476.672824] pci 0000:01:00.1: VF(n) BAR3 space: [mem 0x80000b08000-0x80000c07fff 64bit pref] (contains BAR3 for 64 VFs) [ 476.717233] pci_bus 0000:01: fixups for bus [ 476.734843] pci_bus 0000:01: bus scan returning with max=01 [ 476.743552] pcieport 0000:00:08.0: scanning [bus 03-04] behind bridge, pass 1 [ 476.753981] pci 0000:00:00.0: scanning [bus 01-02] behind bridge, pass 1 [ 476.762484] pci_bus 0000:00: bus scan returning with max=04 [ 476.771676] pci 0000:00:00.0: bridge window [mem 0x00200000-0x00bfffff 64bit pref] to [bus 01-02] add_size 400000 add_align 200000 [ 476.786395] pci 0000:00:00.0: bridge window [mem 0x00400000-0x00bfffff] to [bus 01-02] add_size 400000 add_align 400000 [ 476.798969] pci 0000:00:00.0: BAR 14: assigned [mem 0xe1400000-0xe1ffffff] [ 476.806837] pci 0000:00:00.0: BAR 15: assigned [mem 0x80000000000-0x80000dfffff 64bit pref] [ 476.816253] pci 0000:00:00.0: BAR 13: assigned [io 0x1000-0x1fff] [ 476.825090] pci 0000:01:00.0: BAR 0: assigned [mem 0x80000000000-0x800003fffff 64bit pref] [ 476.834626] pci 0000:01:00.0: BAR 6: assigned [mem 0xe1400000-0xe17fffff pref] [ 476.842698] pci 0000:01:00.1: BAR 0: assigned [mem 0x80000400000-0x800007fffff 64bit pref] [ 476.852353] pci 0000:01:00.1: BAR 6: assigned [mem 0xe1800000-0xe1bfffff pref] [ 476.860311] pci 0000:01:00.0: BAR 4: assigned [mem 0x80000800000-0x80000803fff 64bit pref] [ 476.869637] pci 0000:01:00.0: BAR 7: assigned [mem 0x80000804000-0x80000903fff 64bit pref] [ 476.878800] pci 0000:01:00.0: BAR 10: assigned [mem 0x80000904000-0x80000a03fff 64bit pref] [ 476.888354] pci 0000:01:00.1: BAR 4: assigned [mem 0x80000a04000-0x80000a07fff 64bit pref] [ 476.897688] pci 0000:01:00.1: BAR 7: assigned [mem 0x80000a08000-0x80000b07fff 64bit pref] [ 476.906856] pci 0000:01:00.1: BAR 10: assigned [mem 0x80000b08000-0x80000c07fff 64bit pref] [ 476.916342] pci 0000:01:00.0: BAR 2: assigned [io 0x1000-0x101f] [ 476.923231] pci 0000:01:00.1: BAR 2: assigned [io 0x1020-0x103f] [ 476.930426] pci 0000:00:00.0: PCI bridge to [bus 01-02] [ 476.936429] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 476.943383] pci 0000:00:00.0: bridge window [mem 0xe1400000-0xe1ffffff] [ 476.950994] pci 0000:00:00.0: bridge window [mem 0x80000000000-0x80000dfffff 64bit pref] [ 476.989468] iommu: Adding device 0000:00:00.0 to group 4 [ 477.001974] pcieport 0000:00:00.0: runtime IRQ mapping not provided by arch [ 477.055527] pcieport 0000:00:00.0: irq [33-64] for MSI [ 477.122631] pcieport 0000:00:00.0: irq [23-25] for MSI [ 477.149352] pcie_pme: probe of 0000:00:00.0:pcie001 failed with error -22 [ 477.193697] pcieport 0000:00:00.0: request AER IRQ 34 failed [ 477.211684] aer: probe of 0000:00:00.0:pcie002 failed with error -22 [ 477.237964] pciehp 0000:00:00.0:pcie004: Slot #0 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+ [ 477.267030] pci_bus 0000:01: dev 00, created physical slot 0 [ 477.286902] pciehp 0000:00:00.0:pcie004: Cannot get irq 33 for the hotplug controller [ 477.305707] pciehp 0000:00:00.0:pcie004: Notification initialization failed (-1) [ 477.317644] pci_bus 0000:01: dev 00, dec refcount to 0 [ 477.332681] pci_bus 0000:01: dev 00, released physical slot 0 [ 477.392148] dpc 0000:00:00.0:pcie010: request IRQ35 failed: -22 [ 477.410632] dpc: probe of 0000:00:00.0:pcie010 failed with error -22 [ 477.430064] pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x1c0 [ 477.464772] iommu: Adding device 0000:01:00.0 to group 5 [ 477.477650] ixgbe 0000:01:00.0: runtime IRQ mapping not provided by arch [ 477.494175] ixgbe 0000:01:00.0: enabling bus mastering [ 478.942614] ixgbe 0000:01:00.0: irq 26 for MSI [ 478.952806] ixgbe 0000:01:00.0: irq 27 for MSI [ 478.961336] ixgbe 0000:01:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1 XDP Queue count = 0 [ 478.973341] ixgbe 0000:01:00.0: PCI Express bandwidth of 8GT/s available [ 478.980921] ixgbe 0000:01:00.0: (Speed:2.5GT/s, Width: x4, Encoding Loss:20%) [ 478.988765] ixgbe 0000:01:00.0: This is not sufficient for optimal performance of this card. [ 478.997873] ixgbe 0000:01:00.0: For optimal performance, at least 20GT/s of bandwidth is required. [ 479.007462] ixgbe 0000:01:00.0: A slot with more lanes and/or higher speed is suggested. [ 479.016667] ixgbe 0000:01:00.0: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF [ 479.023628] ixgbe 0000:01:00.0: 9c:37:f4:90:cd:23 [ 481.014736] ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection [ 481.030916] pci 0000:01:00.1: calling quirk_e100_interrupt+0x0/0x1c0 [ 481.066757] iommu: Adding device 0000:01:00.1 to group 6 [ 481.079297] ixgbe 0000:01:00.1: runtime IRQ mapping not provided by arch [ 481.095780] ixgbe 0000:01:00.1: enabling bus mastering [ 481.574664] ixgbe 0000:01:00.1: irq 28 for MSI [ 481.584598] ixgbe 0000:01:00.1: irq 29 for MSI [ 481.593121] ixgbe 0000:01:00.1: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1 XDP Queue count = 0 [ 481.605182] ixgbe 0000:01:00.1: PCI Express bandwidth of 8GT/s available [ 481.612810] ixgbe 0000:01:00.1: (Speed:2.5GT/s, Width: x4, Encoding Loss:20%) [ 481.620683] ixgbe 0000:01:00.1: This is not sufficient for optimal performance of this card. [ 481.629823] ixgbe 0000:01:00.1: For optimal performance, at least 20GT/s of bandwidth is required. [ 481.639444] ixgbe 0000:01:00.1: A slot with more lanes and/or higher speed is suggested. [ 481.648550] ixgbe 0000:01:00.1: MAC: 2, PHY: 17, SFP+: 10, PBA No: FFFFFF-0FF [ 481.656423] ixgbe 0000:01:00.1: 9c:37:f4:90:cd:24 [ 483.786457] ixgbe 0000:01:00.1: Intel(R) 10 Gigabit Network Connection diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 48eadf4..888230d 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -382,9 +382,9 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, for_each_msi_entry(desc, dev) { virq = desc->irq; if (desc->nvec_used == 1) - dev_dbg(dev, "irq %d for MSI\n", virq); + dev_info(dev, "irq %d for MSI\n", virq); else - dev_dbg(dev, "irq [%d-%d] for MSI\n", + dev_info(dev, "irq [%d-%d] for MSI\n", virq, virq + desc->nvec_used - 1); > Is there some cleanup we're missing during the removal?