From patchwork Thu May 31 15:05:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Alex G." X-Patchwork-Id: 10441283 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D4F97602BF for ; Thu, 31 May 2018 15:05:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B82C31FF60 for ; Thu, 31 May 2018 15:05:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8AD91FF66; Thu, 31 May 2018 15:05:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 357EA1FF60 for ; Thu, 31 May 2018 15:05:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755359AbeEaPFv (ORCPT ); Thu, 31 May 2018 11:05:51 -0400 Received: from mail-oi0-f66.google.com ([209.85.218.66]:41165 "EHLO mail-oi0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755036AbeEaPFu (ORCPT ); Thu, 31 May 2018 11:05:50 -0400 Received: by mail-oi0-f66.google.com with SMTP id 11-v6so19767634ois.8; Thu, 31 May 2018 08:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=0DwkhwAFaQQ7X/Yh+2crGW77jeywIXkGzJOBmI/cXBo=; b=ods9YZzjfvIvbd3bSZv8HqCpS4g3bte8WGvuwemY4MnIPRs96v615L5e9k3xm+wFOs GmwWDjZVvTsvuzb7zTbZkQ3VtqmQ3OSHNPzlLChHsBg6UUp3KcsRliugmHqgB97dO8Gd sTRAG6wdQBhox8HXoWvMNkBVRfAAFos9t8FMmV2oCEE7XOVa269oBcuhiifOUkryMchr fpRWFRAMy8J+AP3RgsLBDwLW4DxNBPJic2XHAXxRX0Y2jM86zxui3PTK9T503zFswgiY aQOywC53AzEnlR6mLQ6JpjObKYj2kSJMnvLSiaLUaMGUhSJEMvJzYJX9v/I2HFf/CYm1 mT9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=0DwkhwAFaQQ7X/Yh+2crGW77jeywIXkGzJOBmI/cXBo=; b=Guv7/6D5/mAAraBKz9u86wcTHW0NgpcEVmXusYfoxlCnAsdLWUA7D5y37ERaOohAXb 6fSHHmLn4evn/P0TjGH2yZ7aH3bMQYKuFbb9a/PrbrI6WmTd8PLgCD5P32d0ENFbYpJ7 tHmAE3YOIY1898ok21ZnXU2dbdmBqeaUhrJw3QNsnPlW+vTPgklMDDEK8CnpTnUl61cC 1pYwzZS748MHYNjTPugnfV7vIuUc6cxZDB/hiPhh57DhIC0DM/DC0IU/bNOyUVhkolJg VpqRaDRYnZF+0iLqwkR+puZwmI1bqVn6zI7bri+9LJCDGSPdh3z0h8N26e3kdPWM0w+b yxjw== X-Gm-Message-State: APt69E1KnrucmPuTbveY4QMvxhSOwqTUkiorsp5IqRgY5xGZ24WBcl2u fFxNDChLQpkcLD/N2zHRx3U= X-Google-Smtp-Source: ADUXVKISVEHmLI7Vx8KqUkbA0/HWSle0UUdSRea1KvfnPlvzl7DJwh2OyBSRI3j6UuuXO/MAXSz76A== X-Received: by 2002:aca:4943:: with SMTP id w64-v6mr4283280oia.197.1527779149269; Thu, 31 May 2018 08:05:49 -0700 (PDT) Received: from nuclearis2_1.lan (c-98-201-114-184.hsd1.tx.comcast.net. [98.201.114.184]) by smtp.gmail.com with ESMTPSA id k13-v6sm19346743oiw.28.2018.05.31.08.05.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 31 May 2018 08:05:48 -0700 (PDT) From: Alexandru Gagniuc To: bhelgaas@google.com Cc: alex_gagniuc@dellteam.com, austin_bolen@dell.com, shyam_iyer@dell.com, keith.busch@intel.com, Alexandru Gagniuc , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH] PCI: Check for PCIe downtraining conditions Date: Thu, 31 May 2018 10:05:33 -0500 Message-Id: <20180531150535.9684-1-mr.nuke.me@gmail.com> X-Mailer: git-send-email 2.14.3 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP PCIe downtraining happens when both the device and PCIe port are capable of a larger bus width or higher speed than negotiated. Downtraining might be indicative of other problems in the system, and identifying this from userspace is neither intuitive, nor straigh forward. Instead, check for such conditions on device probe, and print an appropriate message. Signed-off-by: Alexandru Gagniuc --- drivers/pci/probe.c | 78 +++++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/pci_regs.h | 1 + 2 files changed, 79 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index ac91b6fd0bcd..b58c5de70540 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2146,6 +2146,82 @@ static struct pci_dev *pci_scan_device(struct pci_bus *bus, int devfn) return dev; } +static void pcie_max_link_cap(struct pci_dev *dev, enum pci_bus_speed *speed, + enum pcie_link_width *width) +{ + uint32_t lnkcap; + + pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &lnkcap); + + *speed = pcie_link_speed[lnkcap & PCI_EXP_LNKCAP_SLS]; + *width = (lnkcap & PCI_EXP_LNKCAP_MLW) >> PCI_EXP_LNKCAP_MLW_SHIFT; +} + +static void pcie_cur_link_sta(struct pci_dev *dev, enum pci_bus_speed *speed, + enum pcie_link_width *width) +{ + uint16_t lnksta; + + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); + *speed = pcie_link_speed[lnksta & PCI_EXP_LNKSTA_CLS]; + *width = (lnksta & PCI_EXP_LNKSTA_NLW) >> PCI_EXP_LNKSTA_NLW_SHIFT; +} + +static const char *pcie_bus_speed_name(enum pci_bus_speed speed) +{ + switch (speed) { + case PCIE_SPEED_2_5GT: + return "2.5 GT/s"; + case PCIE_SPEED_5_0GT: + return "5.0 GT/s"; + case PCIE_SPEED_8_0GT: + return "8.0 GT/s"; + default: + return "unknown"; + } +} + +static void pcie_check_downtrain_errors(struct pci_dev *dev) +{ + enum pci_bus_speed dev_max_speed, dev_cur_speed; + enum pci_bus_speed max_link_speed, bus_max_speed; + enum pcie_link_width dev_cur_width, dev_max_width; + enum pcie_link_width bus_max_width, max_link_width; + struct pci_dev *uport = pci_upstream_bridge(dev); + + if (!pci_is_pcie(dev) || !uport) + return; + + /* Look from the device up to avoid downstream ports with no devices. */ + if ((pci_pcie_type(dev) != PCI_EXP_TYPE_ENDPOINT) && + (pci_pcie_type(dev) != PCI_EXP_TYPE_LEG_END) && + (pci_pcie_type(dev) != PCI_EXP_TYPE_UPSTREAM)) + return; + + /* Multi-function PCIe share the same link/status. */ + if (PCI_FUNC(dev->devfn) != 0) + return; + + pcie_cur_link_sta(dev, &dev_cur_speed, &dev_cur_width); + pcie_max_link_cap(dev, &dev_max_speed, &dev_max_width); + pcie_max_link_cap(uport, &bus_max_speed, &bus_max_width); + + max_link_speed = min(bus_max_speed, dev_max_speed); + max_link_width = min(bus_max_width, dev_max_width); + + + if (dev_cur_speed < max_link_speed) + pci_warn(dev, "PCIe downtrain: link speed is %s (%s capable)", + pcie_bus_speed_name(dev_cur_speed), + pcie_bus_speed_name(max_link_speed)); + + if (dev_cur_width < max_link_width) { + /* Lanes might not be routed, so use info instead of warn. */ + pci_info(dev, "PCIe downtrain: Port and device capable of x%d, but link running at x%d", + max_link_width, dev_cur_width); + } +} + static void pci_init_capabilities(struct pci_dev *dev) { /* Enhanced Allocation */ @@ -2181,6 +2257,8 @@ static void pci_init_capabilities(struct pci_dev *dev) /* Advanced Error Reporting */ pci_aer_init(dev); + pcie_check_downtrain_errors(dev); + if (pci_probe_reset_function(dev) == 0) dev->reset_fn = 1; } diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index 103ba797a8f3..5557e6dfd05a 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -522,6 +522,7 @@ #define PCI_EXP_LNKCAP_SLS_8_0GB 0x00000003 /* LNKCAP2 SLS Vector bit 2 */ #define PCI_EXP_LNKCAP_SLS_16_0GB 0x00000004 /* LNKCAP2 SLS Vector bit 3 */ #define PCI_EXP_LNKCAP_MLW 0x000003f0 /* Maximum Link Width */ +#define PCI_EXP_LNKCAP_MLW_SHIFT 4 /* start of MLW mask in link status */ #define PCI_EXP_LNKCAP_ASPMS 0x00000c00 /* ASPM Support */ #define PCI_EXP_LNKCAP_L0SEL 0x00007000 /* L0s Exit Latency */ #define PCI_EXP_LNKCAP_L1EL 0x00038000 /* L1 Exit Latency */