From patchwork Wed May 23 02:44:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sinan Kaya X-Patchwork-Id: 10420071 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D0F9C6016C for ; Wed, 23 May 2018 02:46:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF3E028EA8 for ; Wed, 23 May 2018 02:46:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B3C2028F99; Wed, 23 May 2018 02:46:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 251C528EA8 for ; Wed, 23 May 2018 02:46:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=m359RzPMrP+BRpQH4FHfPQORJQ9KeHnVqgWyOa/ntKI=; b=tLQCxE0fWwmoAK fSdCKn1bHYUedY4kc0Opq/zG0slYJ0dblUmXfPmm+JrqPY3gSqqO6nB0skuUm6/rYmUzKOPEKpCGc n97Is1ScICWU0kmJEeazSkx6HxEf4LqubR0RndKRK8MOow36I4FflTF05AeDqQk0d/msUg2SMQNGV 3mFgVDr93846X3RzSxzQqLLWR4vDlYXCBQ6d9nak6+fs5r8QlgKZ4eY+W+ixHapaka1XWXUazyBaN 2MsNMWGEIRSj0ut97CelyyVJxzAndgKHVw9r9/hQY8T4D21NrVdYySIw47r0l6mBrBefpgxIQYEJS PNpd3SBwy6D4v8wQn5vA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fLJnF-00039c-OR; Wed, 23 May 2018 02:46:25 +0000 Received: from smtp.codeaurora.org ([198.145.29.96]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fLJm0-0001ZD-KJ for linux-arm-kernel@lists.infradead.org; Wed, 23 May 2018 02:45:29 +0000 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 48A24604D4; Wed, 23 May 2018 02:44:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1527043498; bh=M+Hwn7zIENN3Y5OTKi9Eu4vPgY0AtD0xwUYkU64wiXY=; h=From:To:Cc:Subject:Date:From; b=H4l/80dw25DCsHZzPXVaUoHJTvJSAN4HkRk/8m2Wyvj+hkNOBQGo+7CF4clrysI4f tMrCj+sSxR4GD5YQ9WbdpJfXV6Bc+BOjlHxVUnwziKrxjw3BMvxm3fxK7fZGo42OIE lwT5NQKCeLExb9xVlsQwlFz1uw6Z3g3q127BOUUY= Received: from drakthul.qualcomm.com (global_nat1_iad_fw.qualcomm.com [129.46.232.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: okaya@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id A36BA604D4; Wed, 23 May 2018 02:44:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1527043496; bh=M+Hwn7zIENN3Y5OTKi9Eu4vPgY0AtD0xwUYkU64wiXY=; h=From:To:Cc:Subject:Date:From; b=fR9ccv6UDCqe2xd3t8JAkvp4xAnVi92/dS0jbaIDw5RlRczHAIJXx72oPeOJq8Kvx Gl++7qtIlB7HD4KXzix27G6xHQdxj6U6zSrvEjdiEJrd7QvmkInQFus8sLfE/Ij5H2 Oe8PxvwV1Wjmeh6PpdA5Inhp2lCOFyUL7BN5/z/U= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org A36BA604D4 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=okaya@codeaurora.org From: Sinan Kaya To: linux-pci@vger.kernel.org, timur@codeaurora.org Subject: [PATCH V2] PCI/portdrv: do not disable device on reboot/shutdown Date: Tue, 22 May 2018 22:44:46 -0400 Message-Id: <1527043490-17268-1-git-send-email-okaya@codeaurora.org> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180522_194508_751465_BAC274AD X-CRM114-Status: GOOD ( 15.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ryan@finnie.org, Kate Stewart , Frederick Lawler , Gabriele Paoloni , linux-arm-msm@vger.kernel.org, "Rafael J. Wysocki" , open list , stable@vger.kernel.org, Sinan Kaya , Dongdong Liu , Greg Kroah-Hartman , Bjorn Helgaas , Thomas Gleixner , Mika Westerberg , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP 'Commit cc27b735ad3a ("PCI/portdrv: Turn off PCIe services during shutdown")' has been added to kernel to shutdown pending PCIe port service interrupts during reboot so that a newly started kexec kernel wouldn't observe pending interrupts. pcie_port_device_remove() is disabling the root port and switches by calling pci_disable_device() after all PCIe service drivers are shutdown. pci_disable_device() has a much wider impact then port service itself and it prevents all inbound transactions to reach to the system and impacts the entire PCI traffic behind the bridge. Issue is that pcie_port_device_remove() doesn't maintain any coordination with the rest of the PCI device drivers in the system before clearing the bus master bit. This has been found to cause crashes on HP DL360 Gen9 machines during reboot. Besides, kexec is already clearing the bus master bit in pci_device_shutdown() after all PCI drivers are removed. Just remove the extra clear in shutdown path by seperating the remove and shutdown APIs in the PORTDRV. Signed-off-by: Sinan Kaya Link: https://bugzilla.kernel.org/show_bug.cgi?id=199779 Fixes: cc27b735ad3a ("PCI/portdrv: Turn off PCIe services during shutdown") Cc: stable@vger.kernel.org Reported-by: Ryan Finnie --- drivers/pci/pcie/portdrv.h | 2 +- drivers/pci/pcie/portdrv_core.c | 5 +++-- drivers/pci/pcie/portdrv_pci.c | 16 +++++++++++++--- 3 files changed, 17 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pcie/portdrv.h b/drivers/pci/pcie/portdrv.h index d0c6783..f6e88fe 100644 --- a/drivers/pci/pcie/portdrv.h +++ b/drivers/pci/pcie/portdrv.h @@ -86,7 +86,7 @@ int pcie_port_device_register(struct pci_dev *dev); int pcie_port_device_suspend(struct device *dev); int pcie_port_device_resume(struct device *dev); #endif -void pcie_port_device_remove(struct pci_dev *dev); +void pcie_port_device_remove(struct pci_dev *dev, bool disable); int __must_check pcie_port_bus_register(void); void pcie_port_bus_unregister(void); diff --git a/drivers/pci/pcie/portdrv_core.c b/drivers/pci/pcie/portdrv_core.c index c9c0663..f35341e 100644 --- a/drivers/pci/pcie/portdrv_core.c +++ b/drivers/pci/pcie/portdrv_core.c @@ -405,11 +405,12 @@ static int remove_iter(struct device *dev, void *data) * Remove PCI Express port service devices associated with given port and * disable MSI-X or MSI for the port. */ -void pcie_port_device_remove(struct pci_dev *dev) +void pcie_port_device_remove(struct pci_dev *dev, bool disable) { device_for_each_child(&dev->dev, NULL, remove_iter); pci_free_irq_vectors(dev); - pci_disable_device(dev); + if (disable) + pci_disable_device(dev); } /** diff --git a/drivers/pci/pcie/portdrv_pci.c b/drivers/pci/pcie/portdrv_pci.c index 973f1b8..29afaf3 100644 --- a/drivers/pci/pcie/portdrv_pci.c +++ b/drivers/pci/pcie/portdrv_pci.c @@ -137,7 +137,7 @@ static int pcie_portdrv_probe(struct pci_dev *dev, return 0; } -static void pcie_portdrv_remove(struct pci_dev *dev) +static void __pcie_portdrv_remove(struct pci_dev *dev, bool disable) { if (pci_bridge_d3_possible(dev)) { pm_runtime_forbid(&dev->dev); @@ -145,7 +145,17 @@ static void pcie_portdrv_remove(struct pci_dev *dev) pm_runtime_dont_use_autosuspend(&dev->dev); } - pcie_port_device_remove(dev); + pcie_port_device_remove(dev, disable); +} + +static void pcie_portdrv_remove(struct pci_dev *dev) +{ + return __pcie_portdrv_remove(dev, true); +} + +static void pcie_portdrv_shutdown(struct pci_dev *dev) +{ + return __pcie_portdrv_remove(dev, false); } static pci_ers_result_t pcie_portdrv_error_detected(struct pci_dev *dev, @@ -218,7 +228,7 @@ static struct pci_driver pcie_portdriver = { .probe = pcie_portdrv_probe, .remove = pcie_portdrv_remove, - .shutdown = pcie_portdrv_remove, + .shutdown = pcie_portdrv_shutdown, .err_handler = &pcie_portdrv_err_handler,