From patchwork Fri Aug 25 09:50:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongdong Liu X-Patchwork-Id: 9921511 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D6D03600C5 for ; Fri, 25 Aug 2017 09:23:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C809526E47 for ; Fri, 25 Aug 2017 09:23:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7C2A27E5A; Fri, 25 Aug 2017 09:23:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E37BC26E47 for ; Fri, 25 Aug 2017 09:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754877AbdHYJXQ (ORCPT ); Fri, 25 Aug 2017 05:23:16 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:4572 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754859AbdHYJXP (ORCPT ); Fri, 25 Aug 2017 05:23:15 -0400 Received: from 172.30.72.59 (EHLO DGGEMS411-HUB.china.huawei.com) ([172.30.72.59]) by dggrg05-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id DFX98451; Fri, 25 Aug 2017 17:23:10 +0800 (CST) Received: from linux-ioko.site (10.71.200.31) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.301.0; Fri, 25 Aug 2017 17:23:01 +0800 From: Dongdong Liu To: CC: , , , , Dongdong Liu Subject: [PATCH] PCI/portdrv: Fix MSI/MSI-X bug for PCIe port service drivers Date: Fri, 25 Aug 2017 17:50:42 +0800 Message-ID: <1503654642-59977-1-git-send-email-liudongdong3@huawei.com> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 X-Originating-IP: [10.71.200.31] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020206.599FEC7E.004D, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 7a9864070fdcf63a55c4c5655d2f5135 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Current code is broken as calling pci_free_irq_vectors() invalidates the IRQ numbers returned before by pci_irq_vectors(); so we need to move all the assignment of the Linux IRQ numbers at the bottom of the function. After removing and adding back the PCI root port device, we see the PCIe port service drivers request irq failed. root@(none)$ lspci -tv -[0000:00]-+-00.0-[01]----00.0 Device 19e5:0123 \-08.0-[02-03]--+-00.0 Device 8086:10fb \-00.1 Device 8086:10fb root@(none)$ echo 1 > /sys/devices/pci0000\:00/0000\:00\:00.0/remove iommu: Removing device 0000:00:00.0 from group 2 root@(none)$ echo 1 > /sys/devices/pci0000\:00/pci_bus/0000\:00/rescan pci 0000:01:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force' pci 0000:00:00.0: BAR 14: assigned [mem 0xe0100000-0xe03fffff] pci 0000:00:00.0: BAR 15: assigned [mem 0x80000e00000-0x80000ffffff 64bit pref] pci 0000:00:00.0: BAR 13: assigned [io 0x1000-0x1fff] pci 0000:01:00.0: BAR 0: assigned [mem 0xe0100000-0xe013ffff 64bit] pci 0000:01:00.0: BAR 6: assigned [mem 0xe0140000-0xe015ffff pref] pci 0000:00:00.0: PCI bridge to [bus 01] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] pci 0000:00:00.0: bridge window [mem 0xe0100000-0xe03fffff] pci 0000:00:00.0: bridge window [mem 0x80000e00000-0x80000ffffff 64bit pref] iommu: Adding device 0000:00:00.0 to group 2 pcie_pme: probe of 0000:00:00.0:pcie001 failed with error -22 aer: probe of 0000:00:00.0:pcie002 failed with error -22 pciehp 0000:00:00.0:pcie004: Slot #0 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl- LLActRep+ pciehp 0000:00:00.0:pcie004: Cannot get irq 20 for the hotplug controller pciehp 0000:00:00.0:pcie004: Notification initialization failed (-1) dpc 0000:00:00.0:pcie010: request IRQ22 failed: -22 dpc: probe of 0000:00:00.0:pcie010 failed with error -22 Signed-off-by: Dongdong Liu Signed-off-by: Gabriele Paoloni --- drivers/pci/pcie/portdrv_core.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c index 313a21d..4cac558 100644 --- a/drivers/pci/pcie/portdrv_core.c +++ b/drivers/pci/pcie/portdrv_core.c @@ -54,7 +54,10 @@ static void release_pcie_device(struct device *dev) */ static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) { - int nr_entries, entry, nvec = 0; + int nr_entries, nvec = 0; + int entry_hp = 0; + int entry_aer = 0; + int entry_dpc = 0; /* * Allocate as many entries as the port wants, so that we can check @@ -86,14 +89,11 @@ static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) * interrupt message." */ pcie_capability_read_word(dev, PCI_EXP_FLAGS, ®16); - entry = (reg16 & PCI_EXP_FLAGS_IRQ) >> 9; - if (entry >= nr_entries) + entry_hp = (reg16 & PCI_EXP_FLAGS_IRQ) >> 9; + if (entry_hp >= nr_entries) goto out_free_irqs; - irqs[PCIE_PORT_SERVICE_PME_SHIFT] = pci_irq_vector(dev, entry); - irqs[PCIE_PORT_SERVICE_HP_SHIFT] = pci_irq_vector(dev, entry); - - nvec = max(nvec, entry + 1); + nvec = max(nvec, entry_hp + 1); } if (mask & PCIE_PORT_SERVICE_AER) { @@ -114,13 +114,11 @@ static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) */ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ERR); pci_read_config_dword(dev, pos + PCI_ERR_ROOT_STATUS, ®32); - entry = reg32 >> 27; - if (entry >= nr_entries) + entry_aer = reg32 >> 27; + if (entry_aer >= nr_entries) goto out_free_irqs; - irqs[PCIE_PORT_SERVICE_AER_SHIFT] = pci_irq_vector(dev, entry); - - nvec = max(nvec, entry + 1); + nvec = max(nvec, entry_aer + 1); } if (mask & PCIE_PORT_SERVICE_DPC) { @@ -141,13 +139,11 @@ static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) */ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DPC); pci_read_config_word(dev, pos + PCI_EXP_DPC_CAP, ®16); - entry = reg16 & 0x1f; - if (entry >= nr_entries) + entry_dpc = reg16 & 0x1f; + if (entry_dpc >= nr_entries) goto out_free_irqs; - irqs[PCIE_PORT_SERVICE_DPC_SHIFT] = pci_irq_vector(dev, entry); - - nvec = max(nvec, entry + 1); + nvec = max(nvec, entry_dpc + 1); } /* @@ -166,6 +162,17 @@ static int pcie_port_enable_irq_vec(struct pci_dev *dev, int *irqs, int mask) return nr_entries; } + if (mask & (PCIE_PORT_SERVICE_PME | PCIE_PORT_SERVICE_HP)) { + irqs[PCIE_PORT_SERVICE_PME_SHIFT] = pci_irq_vector(dev, entry_hp); + irqs[PCIE_PORT_SERVICE_HP_SHIFT] = pci_irq_vector(dev, entry_hp); + } + + if (mask & PCIE_PORT_SERVICE_AER) + irqs[PCIE_PORT_SERVICE_AER_SHIFT] = pci_irq_vector(dev, entry_aer); + + if (mask & PCIE_PORT_SERVICE_DPC) + irqs[PCIE_PORT_SERVICE_DPC_SHIFT] = pci_irq_vector(dev, entry_dpc); + return 0; out_free_irqs: