From patchwork Fri Sep 4 20:51:00 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duyck, Alexander H" X-Patchwork-Id: 45788 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n84Kp39t002578 for ; Fri, 4 Sep 2009 20:51:03 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932751AbZIDUu7 (ORCPT ); Fri, 4 Sep 2009 16:50:59 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S933855AbZIDUu7 (ORCPT ); Fri, 4 Sep 2009 16:50:59 -0400 Received: from mga09.intel.com ([134.134.136.24]:49043 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932751AbZIDUu6 (ORCPT ); Fri, 4 Sep 2009 16:50:58 -0400 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 04 Sep 2009 13:36:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.44,334,1249282800"; d="scan'208";a="547924942" Received: from ahduyck-desk.amr.corp.intel.com (HELO [134.134.3.129]) ([134.134.3.129]) by orsmga001.jf.intel.com with ESMTP; 04 Sep 2009 13:50:22 -0700 Message-ID: <4AA17DB4.8040704@intel.com> Date: Fri, 04 Sep 2009 13:51:00 -0700 From: Alexander Duyck User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: linux-pci@vger.kernel.org Subject: pci=assign-busses w/ SR-IOV capable devices can cause bridge downstream ports to disappear Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Hi, I think I found an issue with the pci=assign-busses option when trying to work around a system that has SR-IOV disabled. I believe I have narrowed down the issue to a problem of bridge routing but I am not certain how best to go about resolving the issue. Below is a snippet from an lspci -t when the devices appear as they are supposed to. With the patch included at the end of the email applied it changes to the broken layout. I originally found the issue while testing SR-IOV on a system with 7 82576 quad port adapters. The issue appears to occur when the upstream secondary bus matches one of the downstream secondary buses. When it occurs the system is unable to detect some or all of the downstream ports. I generated the patch below as a debugging step so that I could target specific buses when reproducing the issue. In the example below I saw the issue when the secondary bus on the upstream port was 3 or 4, but once it went to 5 the devices appeared again without issue. I was also able to work around my SR-IOV issue by increasing the buses sufficiently that there was no upstream/downstream overlap. The issue doesn't appear to be bridge specific as I can recreate it on Intel Corporation 6311ESB/6321ESB PCI Express Upstream Port, Integrated Device Technology, Inc. PES12N3A PCI Express Switch, and a Integrated Device Technology, Inc. PES24N3A PCI Express Switch. Thanks, Alex What it should looks like without the change: -[0000:00]-+-00.0 +-01.0-[0000:01-04]----00.0-[0000:02-04]--+-02.0-[0000:03]--+-00.0 | | \-00.1 | \-04.0-[0000:04]--+-00.0 | \-00.1 What it looks like when broken: -[0000:00]-+-00.0 +-01.0-[0000:02-03]----00.0-[0000:03-03]-- | | | --- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 40e75f6..62c0c17 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1094,6 +1094,10 @@ unsigned int __devinit pci_scan_child_bus(struct pci_bus *bus) /* Reserve buses for SR-IOV capability. */ max += pci_iov_bus_range(bus); + /* Shift all bus numbers by 1 in the hopes of creating a conflict */ + if (bus->number == 0) + max += 1; + /* * After performing arch-dependent fixup of the bus, look behind * all PCI-to-PCI bridges on this bus.