From patchwork Wed Jul 18 18:51:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Myron Stowe X-Patchwork-Id: 10533091 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4A09F6020A for ; Wed, 18 Jul 2018 18:52:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3874429C0E for ; Wed, 18 Jul 2018 18:52:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 36B6629C48; Wed, 18 Jul 2018 18:52:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF39129C0E for ; Wed, 18 Jul 2018 18:52:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727131AbeGRTbM (ORCPT ); Wed, 18 Jul 2018 15:31:12 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:58584 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726909AbeGRTbM (ORCPT ); Wed, 18 Jul 2018 15:31:12 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 349CF407564D; Wed, 18 Jul 2018 18:51:59 +0000 (UTC) Received: from tak.stowe (ovpn-120-77.rdu2.redhat.com [10.10.120.77]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7861F2156893; Wed, 18 Jul 2018 18:51:58 +0000 (UTC) Subject: [PATCH] PCI: Match Root Port's MPS to endpoint's MPSS when necessary From: Myron Stowe To: bhelgaas@google.com, linux-pci@vger.kernel.org Cc: keith.busch@intel.com, jdmason@kudzu.us Date: Wed, 18 Jul 2018 12:51:58 -0600 Message-ID: <20180718185158.149373.77902.stgit@tak.stowe> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 18 Jul 2018 18:51:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Wed, 18 Jul 2018 18:51:59 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'myron.stowe@redhat.com' RCPT:'' Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In commit 27d868b5e6cf ("PCI: Set MPS to match upstream bridge"), we made sure every device's MPS setting matches its upstream bridge, making it more likely that a hot-added device will work in a system with an optimized MPS configuration. Recently I've started encountering systems where the endpoint device's MPSS capability is less than its root port's current MPS value, thus the endpoint is not capable of matching its upstream bridge's MPS setting (see: bugzilla via "Link:" below). This leaves the system vunerable - the upstream root port could respond with larger sized TLPs than the endpoint can handle, and the endpoint will consider them to be 'Malformed'. One could use the "pci=pcie_bus_safe" kernel parameter to resolve the issue, but, it both forces a user to have to supply a kernel parameter to get the system to function reliable, and may end up limiting MPS settings of other, non-related, sub-topologies which could benefit from maintaining their larger values. This patch augments Keith's approach to include tuning down a root port's MPS setting when its hot-added endpoint device is not capable of matching it. The tuning down, so that both the root port and endpoint match, is limited to root ports with downstream endpoint device sub-topologies. Link: https://bugzilla.kernel.org/show_bug.cgi?id=200527 Cc: Keith Busch Cc: Jon Mason Cc: Sinan Kaya Signed-off-by: Myron Stowe Acked-by: Jon Mason --- drivers/pci/probe.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index ac91b6f..2987bd9 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1670,7 +1670,7 @@ int pci_setup_device(struct pci_dev *dev) static void pci_configure_mps(struct pci_dev *dev) { struct pci_dev *bridge = pci_upstream_bridge(dev); - int mps, p_mps, rc; + int mps, mpss, p_mps, rc; if (!pci_is_pcie(dev) || !bridge || !pci_is_pcie(bridge)) return; @@ -1694,6 +1694,14 @@ static void pci_configure_mps(struct pci_dev *dev) if (pcie_bus_config != PCIE_BUS_DEFAULT) return; + mpss = 128 << dev->pcie_mpss; + if (mpss < p_mps && pci_pcie_type(bridge) == PCI_EXP_TYPE_ROOT_PORT) { + pcie_set_mps(bridge, mpss); + pci_info(dev, "Upstream bridge's Max Payload Size set to %d (was %d, max %d)\n", + mpss, p_mps, 128 << bridge->pcie_mpss); + p_mps = pcie_get_mps(bridge); + } + rc = pcie_set_mps(dev, p_mps); if (rc) { pci_warn(dev, "can't set Max Payload Size to %d; if necessary, use \"pci=pcie_bus_safe\" and report a bug\n", @@ -1702,7 +1710,7 @@ static void pci_configure_mps(struct pci_dev *dev) } pci_info(dev, "Max Payload Size set to %d (was %d, max %d)\n", - p_mps, mps, 128 << dev->pcie_mpss); + p_mps, mps, mpss); } static struct hpp_type0 pci_default_type0 = {