From patchwork Mon Feb 24 03:44:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13987406 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F17781AAA0D; Mon, 24 Feb 2025 03:45:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368715; cv=none; b=n3f4lIzVBZA0lzh2Q69HvrXi4kRtCbdsmJmwn6pQTdldqOw3Jx5hEFxKmQZoU4HpJgjKqhamdpOpl8PlrIEBBCV75FCr5cmSirHsvWX0BpkxXv8bss93Jb+Y8+guZBhuTscrDg7BITe8EKERnKzFDQEPkuFJMLl+mvsyhNTOZpk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368715; c=relaxed/simple; bh=Z63BjGA1BVkMRgurztK/iK1uMx3eBg2BCSpAiKRwz+E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Hfe40SePLB2Ng6WH9DLS1k9yFxDKzhFiTTT0C/FicN5UT1JbVsIe/XKUuM8E69uqDLO4dFY0vE0I6xb7sZ4mktow0HnQzcq6BIzVjzpVTdHms5AtdGC5dWVdC963TzI749iqrXiP0FmKDVQdmUhSGH9/GKczujLIJLSL6l/yisY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=sLqoVtAq; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="sLqoVtAq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1740368704; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=V6qMCrYnpDuf4CL4/21Ay3Wnk4Z6RjxNTwozxvs3+uY=; b=sLqoVtAqYpYda84oDfO9dpcnUGUfaBmqNJuZGCP7Owb01P+VaJwUl7F29PjNIFVA7YSNXyGGoG9UjJMAylIGmJyg/1f9HR+D3uI06ZUVNVLK96PkDktQJsSg0lQPS31IZdGyhSteKZ6vnF2xmL4sUw1Z0xrpSsonbJ334KnoHWs= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WQ1rZfk_1740368702 cluster:ay36) by smtp.aliyun-inc.com; Mon, 24 Feb 2025 11:45:03 +0800 From: Feng Tang To: Bjorn Helgaas , Lukas Wunner , Sathyanarayanan Kuppuswamy , Liguang Zhang , Guanghui Feng , rafael@kernel.org Cc: Markus Elfring , lkp@intel.com, Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 1/4] PCI: portdrv: pciehp: Move PCIe hotplug command waiting logic to port driver Date: Mon, 24 Feb 2025 11:44:57 +0800 Message-Id: <20250224034500.23024-2-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250224034500.23024-1-feng.tang@linux.alibaba.com> References: <20250224034500.23024-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 According to PCIe spec, after sending a hotplug command, software should wait some time for the command completion. Currently the waiting logic is implemented in pciehp driver, as the same logic will be reused by PCIe port driver, move it to port driver, which complies with the logic of CONFIG_HOTPLUG_PCI_PCIE depending on CONFIG_PCIEPORTBUS. Also convert the loop wait logic to helper read_poll_timeout() as suggested by Sathyanarayanan Kuppuswamy. Signed-off-by: Feng Tang --- drivers/pci/hotplug/pciehp_hpc.c | 38 ++++++++------------------------ drivers/pci/pci.h | 5 +++++ drivers/pci/pcie/portdrv.c | 25 +++++++++++++++++++++ 3 files changed, 39 insertions(+), 29 deletions(-) diff --git a/drivers/pci/hotplug/pciehp_hpc.c b/drivers/pci/hotplug/pciehp_hpc.c index bb5a8d9f03ad..24e346f558db 100644 --- a/drivers/pci/hotplug/pciehp_hpc.c +++ b/drivers/pci/hotplug/pciehp_hpc.c @@ -83,32 +83,6 @@ static inline void pciehp_free_irq(struct controller *ctrl) free_irq(ctrl->pcie->irq, ctrl); } -static int pcie_poll_cmd(struct controller *ctrl, int timeout) -{ - struct pci_dev *pdev = ctrl_dev(ctrl); - u16 slot_status; - - do { - pcie_capability_read_word(pdev, PCI_EXP_SLTSTA, &slot_status); - if (PCI_POSSIBLE_ERROR(slot_status)) { - ctrl_info(ctrl, "%s: no response from device\n", - __func__); - return 0; - } - - if (slot_status & PCI_EXP_SLTSTA_CC) { - pcie_capability_write_word(pdev, PCI_EXP_SLTSTA, - PCI_EXP_SLTSTA_CC); - ctrl->cmd_busy = 0; - smp_mb(); - return 1; - } - msleep(10); - timeout -= 10; - } while (timeout >= 0); - return 0; /* timeout */ -} - static void pcie_wait_cmd(struct controller *ctrl) { unsigned int msecs = pciehp_poll_mode ? 2500 : 1000; @@ -138,10 +112,16 @@ static void pcie_wait_cmd(struct controller *ctrl) timeout = cmd_timeout - now; if (ctrl->slot_ctrl & PCI_EXP_SLTCTL_HPIE && - ctrl->slot_ctrl & PCI_EXP_SLTCTL_CCIE) + ctrl->slot_ctrl & PCI_EXP_SLTCTL_CCIE) { rc = wait_event_timeout(ctrl->queue, !ctrl->cmd_busy, timeout); - else - rc = pcie_poll_cmd(ctrl, jiffies_to_msecs(timeout)); + } else { + rc = pcie_poll_sltctl_cmd(ctrl_dev(ctrl), jiffies_to_msecs(timeout)); + if (!rc) { + ctrl->cmd_busy = 0; + smp_mb(); + rc = 1; + } + } if (!rc) ctrl_info(ctrl, "Timeout on hotplug command %#06x (issued %u msec ago)\n", diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 01e51db8d285..4c94a589de4a 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -759,12 +759,17 @@ static inline void pcie_ecrc_get_policy(char *str) { } #ifdef CONFIG_PCIEPORTBUS void pcie_reset_lbms_count(struct pci_dev *port); int pcie_lbms_count(struct pci_dev *port, unsigned long *val); +int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms); #else static inline void pcie_reset_lbms_count(struct pci_dev *port) {} static inline int pcie_lbms_count(struct pci_dev *port, unsigned long *val) { return -EOPNOTSUPP; } +static inline int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms) +{ + return 0; +} #endif struct pci_dev_reset_methods { diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c index 02e73099bad0..bb00ba45ee51 100644 --- a/drivers/pci/pcie/portdrv.c +++ b/drivers/pci/pcie/portdrv.c @@ -18,6 +18,7 @@ #include #include #include +#include #include "../pci.h" #include "portdrv.h" @@ -205,6 +206,30 @@ static int pcie_init_service_irqs(struct pci_dev *dev, int *irqs, int mask) return 0; } +/* Return 0 on command completed on time, otherwise return -ETIMEOUT */ +int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms) +{ + u16 slot_status = 0; + u32 slot_cap; + int ret = 0; + int __maybe_unused ret1; + + /* Don't wait if the command complete event is not well supported */ + pcie_capability_read_dword(dev, PCI_EXP_SLTCAP, &slot_cap); + if (!(slot_cap & PCI_EXP_SLTCAP_HPC) || slot_cap & PCI_EXP_SLTCAP_NCCS) + return ret; + + ret = read_poll_timeout(pcie_capability_read_word, ret1, + (slot_status & PCI_EXP_SLTSTA_CC), 10000, + timeout_ms * 1000, true, dev, PCI_EXP_SLTSTA, + &slot_status); + if (!ret) + pcie_capability_write_word(dev, PCI_EXP_SLTSTA, + PCI_EXP_SLTSTA_CC); + + return ret; +} + /** * get_port_device_capability - discover capabilities of a PCI Express port * @dev: PCI Express port to examine From patchwork Mon Feb 24 03:44:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13987405 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0148B1AAA1A; Mon, 24 Feb 2025 03:45:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.132 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368715; cv=none; b=tYIn3TgPMjbkrGtGQNFe1EsmeZrNENOZ4MUUaetb81tvWwgQOxdNzK9XhBmPLsl2hziqYjNlOKkYu+aQQNJH9MZjq7NPJ2nE8aO/yneR1U3mVXF1lwNS7+xp3gRzY1qJ+3T9anlAmjqRMrqvNGf1jhbTcQLaaGYPSMo1dpxYZxY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368715; c=relaxed/simple; bh=zQblHyhvwohGfU+6u4WN9LwOs0v+WagpCNoY/EchxTI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Za5TJ+BIJzwdFss5m1VtUi4y7VZtQAsGvzOZv8eeMMaZNS9BTGTy2i3C8dSRod0ggFAuAEcN6UiPs44f11/NoPR2EwAheMeOHPoI0Fs3yH7bfmTPtAwWiKm/ZoMfEAlySpfsr64yO9il3Q4n7FBF6n239Tv/UGi1OzdVFYSpDMs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=fKiCx8CW; arc=none smtp.client-ip=115.124.30.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="fKiCx8CW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1740368704; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=4fmmqZE5L8pnbvnXPansBrSd9fl8lXiV+nvcr/YF6Og=; b=fKiCx8CWgN8NoEKHpnZiMiodH1NkY78SmCOtgOz9vFTEHAuMdeJu28p+4JiD4btjaeaiXbg90awTL2lzLdEZnxVmSmJ+LifBx41X8jvOn1A1xhx/8TW0X2uuOLnOrM4pjPe2Kl/2fyCmsbvybcJe1mL67Eif22BNqpHs8t75/4w= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WQ1wP9u_1740368703 cluster:ay36) by smtp.aliyun-inc.com; Mon, 24 Feb 2025 11:45:03 +0800 From: Feng Tang To: Bjorn Helgaas , Lukas Wunner , Sathyanarayanan Kuppuswamy , Liguang Zhang , Guanghui Feng , rafael@kernel.org Cc: Markus Elfring , lkp@intel.com, Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 2/4] PCI/portdrv: Add necessary wait for disabling hotplug events Date: Mon, 24 Feb 2025 11:44:58 +0800 Message-Id: <20250224034500.23024-3-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250224034500.23024-1-feng.tang@linux.alibaba.com> References: <20250224034500.23024-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There was problem reported by firmware developers that they received two PCIe hotplug commands in very short intervals on an ARM server, which doesn't comply with PCIe spec, and broke their state machine and work flow. According to PCIe 6.1 spec, section 6.7.3.2, software needs to wait at least 1 second for the command-complete event, before resending the command or sending a new command. In the failure case, the first PCIe hotplug command firmware received is from get_port_device_capability(), which sends command to disable PCIe hotplug interrupts without waiting for its completion, and the second command comes from pcie_enable_notification() of pciehp driver, which enables hotplug interrupts again. Fix it by adding the necessary wait to comply with PCIe spec. Fixes: 2bd50dd800b5 ("PCI: PCIe: Disable PCIe port services during port initialization") Originally-by: Liguang Zhang Signed-off-by: Feng Tang --- drivers/pci/pci.h | 2 ++ drivers/pci/pcie/portdrv.c | 19 +++++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 4c94a589de4a..a1138ebc2689 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -760,6 +760,7 @@ static inline void pcie_ecrc_get_policy(char *str) { } void pcie_reset_lbms_count(struct pci_dev *port); int pcie_lbms_count(struct pci_dev *port, unsigned long *val); int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms); +void pcie_disable_hp_interrupts_early(struct pci_dev *dev); #else static inline void pcie_reset_lbms_count(struct pci_dev *port) {} static inline int pcie_lbms_count(struct pci_dev *port, unsigned long *val) @@ -770,6 +771,7 @@ static inline int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms) { return 0; } +static inline void pcie_disable_hp_interrupts_early(struct pci_dev *dev) {} #endif struct pci_dev_reset_methods { diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c index bb00ba45ee51..ca4f21dff486 100644 --- a/drivers/pci/pcie/portdrv.c +++ b/drivers/pci/pcie/portdrv.c @@ -230,6 +230,22 @@ int pcie_poll_sltctl_cmd(struct pci_dev *dev, int timeout_ms) return ret; } +void pcie_disable_hp_interrupts_early(struct pci_dev *dev) +{ + u16 slot_ctrl = 0; + + pcie_capability_read_word(dev, PCI_EXP_SLTCTL, &slot_ctrl); + /* Bail out early if it is already disabled */ + if (!(slot_ctrl & (PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE))) + return; + + pcie_capability_clear_word(dev, PCI_EXP_SLTCTL, + PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE); + + if (pcie_poll_sltctl_cmd(dev, 1000)) + pci_info(dev, "Timeout on disabling PCIe hot-plug interrupt\n"); +} + /** * get_port_device_capability - discover capabilities of a PCI Express port * @dev: PCI Express port to examine @@ -255,8 +271,7 @@ static int get_port_device_capability(struct pci_dev *dev) * Disable hot-plug interrupts in case they have been enabled * by the BIOS and the hot-plug service driver is not loaded. */ - pcie_capability_clear_word(dev, PCI_EXP_SLTCTL, - PCI_EXP_SLTCTL_CCIE | PCI_EXP_SLTCTL_HPIE); + pcie_disable_hp_interrupts_early(dev); } #ifdef CONFIG_PCIEAER From patchwork Mon Feb 24 03:44:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13987407 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 130731B21BF; Mon, 24 Feb 2025 03:45:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368716; cv=none; b=n6jQQQs7Xw2m3TN27z9ErT64COc8Qqqq6lEhpg7SL6lff+2p4k7L5VqCd53nGRHsuvCPv2v91F4XoEd58bOuzdH10lUQZGVAHaEBt/Ek7ZKdXeFoj3P4tKhVDuwfzLcDevUPDWlHie0kOpypC+WqamHrijhsgKxJdpT5yxG+DkQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740368716; c=relaxed/simple; bh=PjVIVURsiTWXN7S/uHQMS2YiaBnO/xXEB3uw33ifyKo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iekrsflH80Eil6N+/sYm2jJ0MLN+b9mUnnDddwzPQIKBpOoahmszdhOqFOQeryujIwAikuxeyIYZRtYytqopXKeXj5Kd5AuZrqeiLia/WEYX7MHdI6IDqodfgOMYNulAd8e5vt5yS2wIDI5GicJiLDNiJNjccxW2GTu0C0B/O/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=nBl6tJVf; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="nBl6tJVf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1740368706; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=MtQeeh/3PwVb5Y4qvoWpaZCTj/9V1FHfpbzFSWYE6SE=; b=nBl6tJVfK+Qm/zcxnYKXmbzcYjU8Wtu3rYj85gPTMkUbUo2b/umDf4Y/cHBHHq2NcOB0y8ax/kYKy6h+gq6nKXHwjFYUPele31V5oucM9E+w1wcl11iRmDHr5IqIy0pbsVs3SKD6s1z8bPBCDBOg9argdpd93y4Yfg5YzoH3v/g= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WQ1rZgH_1740368704 cluster:ay36) by smtp.aliyun-inc.com; Mon, 24 Feb 2025 11:45:04 +0800 From: Feng Tang To: Bjorn Helgaas , Lukas Wunner , Sathyanarayanan Kuppuswamy , Liguang Zhang , Guanghui Feng , rafael@kernel.org Cc: Markus Elfring , lkp@intel.com, Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 3/4] PCI/portdrv: Loose the condition check for disabling hotplug interrupts Date: Mon, 24 Feb 2025 11:44:59 +0800 Message-Id: <20250224034500.23024-4-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250224034500.23024-1-feng.tang@linux.alibaba.com> References: <20250224034500.23024-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Currently when 'pcie_ports_native' and host's 'native_pcie_hotplug' are both false, kernel will not disable PCIe hotplug interrupts. But as those could be affected by software setup like kernel cmdline parameter, remove the depency over them. Suggested-by: Lukas Wunner Signed-off-by: Feng Tang --- drivers/pci/pcie/portdrv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pcie/portdrv.c b/drivers/pci/pcie/portdrv.c index ca4f21dff486..619d06b1b3e8 100644 --- a/drivers/pci/pcie/portdrv.c +++ b/drivers/pci/pcie/portdrv.c @@ -263,9 +263,9 @@ static int get_port_device_capability(struct pci_dev *dev) if (dev->is_hotplug_bridge && (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT || - pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) && - (pcie_ports_native || host->native_pcie_hotplug)) { - services |= PCIE_PORT_SERVICE_HP; + pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM)) { + if (pcie_ports_native || host->native_pcie_hotplug) + services |= PCIE_PORT_SERVICE_HP; /* * Disable hot-plug interrupts in case they have been enabled From patchwork Mon Feb 24 03:45:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13987408 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D12B21A0739; Mon, 24 Feb 2025 03:50:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740369030; cv=none; b=fqrBBSxQkpkAmv7KiQ0K4nbNwIf5DxYK6Q9U+Q1FOmKlTYNnKJ1bqtp2dzxji+uXKBpHe8X748vVEUIhmM16OlHKy5D5853gshEc+v0UUiqHJwI1knhzitA7JTGr84UdLRxIxFrIo7SD3p2ptZY343EOBRFMaRq2HtqW6Ee27/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740369030; c=relaxed/simple; bh=LgkDryLjvEZr6bXpKPqMRGIdv3/PyW9iQTas3G5G60M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Q3TBIEf7oRmNCAD1lOjzyQPHOoUdo5WTA59nPyIQg+ct9R6nAWcmXMlu6TDu31/hvio4WlLlIx2hacVKv4cqLczPku/JnF9sf3jYYlPTBBlerJrUnNDMSzN5uT6AXjQkCcIZK5kGdoy9Vbiw+5SwVrq20mkqiwjigLJWCFeTF9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=dlStPqtj; arc=none smtp.client-ip=115.124.30.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="dlStPqtj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1740369024; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=JWdHXTM9C/dfJCHXa70ZhO8Ss2I8+1cp0VWtCNbkVn0=; b=dlStPqtjlvWvD9ScvLHz6ay6b/s4XS90EovacqcAvpu/+dHmC+m0lerTOKm9Bo1bTUtJKGXrdrTipBkrh/QUQHLi2OGRobtU/h+FULfU1YtyfaSKC3CiaYAKay7iRN80ns6Lptmy2AYrUhFYq7fy0s6SZ5zc+6v5RT09/6Qe6mI= Received: from localhost(mailfrom:feng.tang@linux.alibaba.com fp:SMTPD_---0WQ1rZgc_1740368704 cluster:ay36) by smtp.aliyun-inc.com; Mon, 24 Feb 2025 11:45:05 +0800 From: Feng Tang To: Bjorn Helgaas , Lukas Wunner , Sathyanarayanan Kuppuswamy , Liguang Zhang , Guanghui Feng , rafael@kernel.org Cc: Markus Elfring , lkp@intel.com, Jonathan Cameron , ilpo.jarvinen@linux.intel.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v3 4/4] PCI: Disable PCIe hotplug interrupts early when msi is disabled Date: Mon, 24 Feb 2025 11:45:00 +0800 Message-Id: <20250224034500.23024-5-feng.tang@linux.alibaba.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250224034500.23024-1-feng.tang@linux.alibaba.com> References: <20250224034500.23024-1-feng.tang@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 There was an irq storm bug when testing "pci=nomsi" case, and the root cause is: 'nomsi' will disable MSI and let devices and root ports use legacy INTX interrupt, and likely make several devices/ports share one interrupt. In the failure case, BIOS doesn't disable the PCIe hotplug interrupts, and the command-complete interrupt was actually asserted. So the timeline is: 1. pciehp's CCIE/HPIE enabled and command-complete interrupts asserted 2. the interrupt is shared by PCIe root port and nvme/nic device 3. nvme/nic driver's probe function enables the interrupt line 4. pciehp driver is loaded later or never loaded And the "nobody cared irq storm" happens between 3 and 4. This is not an issue for normal MSI case, as each interrupt is controlled by its own driver. When the driver is not loaded, the interrupt won't get fired to kernel even if it is physically asserted. So disable the PCIe hotplug CCIE/HPIE interrupt in early boot phase when MSI is not enabled. Signed-off-by: Feng Tang --- drivers/pci/probe.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 246744d8d268..ffea7851366a 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1665,6 +1665,15 @@ void set_pcie_hotplug_bridge(struct pci_dev *pdev) pcie_capability_read_dword(pdev, PCI_EXP_SLTCAP, ®32); if (reg32 & PCI_EXP_SLTCAP_HPC) pdev->is_hotplug_bridge = 1; + + /* + * When MSI is disabled, root port will use legacy INTX, and likely + * share INTX interrupt line with other devices like NIC/NVME. There + * was real world issue that the CCIE IRQ is asserted afer boot, but + * will not be handled well and cause IRQ storm. So disable it early. + */ + if (!pci_msi_enabled()) + pcie_disable_hp_interrupts_early(pdev); } static void set_pcie_thunderbolt(struct pci_dev *dev)