From patchwork Thu Aug 22 03:24:47 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yijing Wang X-Patchwork-Id: 2848030 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 59AFA9F271 for ; Thu, 22 Aug 2013 03:25:35 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2D3A0201FF for ; Thu, 22 Aug 2013 03:25:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2F44F201F7 for ; Thu, 22 Aug 2013 03:25:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753196Ab3HVDZc (ORCPT ); Wed, 21 Aug 2013 23:25:32 -0400 Received: from szxga01-in.huawei.com ([119.145.14.64]:44913 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752806Ab3HVDZc (ORCPT ); Wed, 21 Aug 2013 23:25:32 -0400 Received: from 172.24.2.119 (EHLO szxeml212-edg.china.huawei.com) ([172.24.2.119]) by szxrg01-dlp.huawei.com (MOS 4.3.4-GA FastPath queued) with ESMTP id BHA76618; Thu, 22 Aug 2013 11:25:11 +0800 (CST) Received: from SZXEML423-HUB.china.huawei.com (10.82.67.162) by szxeml212-edg.china.huawei.com (172.24.2.181) with Microsoft SMTP Server (TLS) id 14.1.323.7; Thu, 22 Aug 2013 11:25:10 +0800 Received: from localhost (10.135.76.69) by szxeml423-hub.china.huawei.com (10.82.67.162) with Microsoft SMTP Server id 14.1.323.7; Thu, 22 Aug 2013 11:25:05 +0800 From: Yijing Wang To: Bjorn Helgaas CC: Jon Mason , , Hanjun Guo , , Yijing Wang Subject: [PATCH v8 5/6] PCI: Don't restrict MPS for slots below Root Ports Date: Thu, 22 Aug 2013 11:24:47 +0800 Message-ID: <1377141888-7000-6-git-send-email-wangyijing@huawei.com> X-Mailer: git-send-email 1.7.11.msysgit.1 In-Reply-To: <1377141888-7000-1-git-send-email-wangyijing@huawei.com> References: <1377141888-7000-1-git-send-email-wangyijing@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.135.76.69] X-CFilter-Loop: Reflected Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-9.7 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When booting with "pci=pcie_bus_safe", we previously limited the fabric MPS to 128 when we found: (1) A hotplug-capable Downstream Port ("dev->is_hotplug_bridge && pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT"), or (2) A hotplug-capable Root Port with a slot that was either empty or contained a multi-function device ("dev->is_hotplug_bridge && !list_is_singular(&dev->bus->devices)") Part (1) is valid, but part (2) is not. After a hot-add in the slot below a Root Port, we can reconfigure all MPS values in the fabric below the Root Port because the new device is the only thing below the Root Port and there are no active drivers. Therefore, there's no reason to limit the MPS for Root Ports, no matter what's in the slot. Test info: -+-[0000:40]-+-07.0-[0000:46]--+-00.0 Intel 82576 NIC \-00.1 Intel 82576 NIC 0000:40:07.0 Root Port bridge to [bus 46] (MPS supported=256) 0000:46:00.0 Endpoint (MPS supported=512) 0000:46:00.1 Endpoint (MPS supported=512) # echo 0 > /sys/bus/pci/slots/7/power # echo 1 > /sys/bus/pci/slots/7/power # dmesg ... pcieport 0000:40:07.0: PCI-E Max Payload Size set to 256/ 256 (was 256) pci 0000:46:00.0: PCI-E Max Payload Size set to 256/ 512 (was 128) pci 0000:46:00.1: PCI-E Max Payload Size set to 256/ 512 (was 128) Before this change, we set MPS to 128 for the Root Port and both NICs because the slot contained a multi-function device and dev->is_hotplug_bridge && !list_is_singular(&dev->bus->devices) was true. After this change, we set it to 256. [bhelgaas: changelog, comments, split out upstream bridge check] Signed-off-by: Yijing Wang Signed-off-by: Bjorn Helgaas Cc: Jon Mason --- drivers/pci/probe.c | 32 ++++++++++++++++---------------- 1 files changed, 16 insertions(+), 16 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 87be31b..4afd158 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1491,23 +1491,23 @@ static int pcie_find_smpss(struct pci_dev *dev, void *data) if (!pci_is_pcie(dev)) return 0; - /* For PCIE hotplug enabled slots not connected directly to a - * PCI-E root port, there can be problems when hotplugging - * devices. This is due to the possibility of hotplugging a - * device into the fabric with a smaller MPS that the devices - * currently running have configured. Modifying the MPS on the - * running devices could cause a fatal bus error due to an - * incoming frame being larger than the newly configured MPS. - * To work around this, the MPS for the entire fabric must be - * set to the minimum size. Any devices hotplugged into this - * fabric will have the minimum MPS set. If the PCI hotplug - * slot is directly connected to the root port and there are not - * other devices on the fabric (which seems to be the most - * common case), then this is not an issue and MPS discovery - * will occur as normal. + /* + * We don't have a way to change MPS settings on devices that have + * drivers attached. A hot-added device might support only the minimum + * MPS setting (MPS=128). Therefore, if the fabric contains a bridge + * where devices may be hot-added, we limit the fabric MPS to 128 so + * hot-added devices will work correctly. + * + * However, if we hot-add a device to a slot directly below a Root + * Port, it's impossible for there to be other existing devices below + * the port. We don't limit the MPS in this case because we can + * reconfigure MPS on both the Root Port and the hot-added device, + * and there are no other devices involved. + * + * Note that this PCIE_BUS_SAFE path assumes no peer-to-peer DMA. */ - if (dev->is_hotplug_bridge && (!list_is_singular(&dev->bus->devices) || - pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT)) + if (dev->is_hotplug_bridge && + pci_pcie_type(dev) != PCI_EXP_TYPE_ROOT_PORT) *smpss = 0; if (*smpss > dev->pcie_mpss)