From patchwork Mon Aug 13 20:33:00 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 1315831 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 77229DF25A for ; Mon, 13 Aug 2012 20:33:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753723Ab2HMUdr (ORCPT ); Mon, 13 Aug 2012 16:33:47 -0400 Received: from moutng.kundenserver.de ([212.227.126.186]:56136 "EHLO moutng.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754697Ab2HMUdX (ORCPT ); Mon, 13 Aug 2012 16:33:23 -0400 Received: from mailbox.adnet.avionic-design.de (mailbox.avionic-design.de [109.75.18.3]) by mrelayeu.kundenserver.de (node=mrbap1) with ESMTP (Nemesis) id 0MPZs9-1T5ABt0NZx-005UOr; Mon, 13 Aug 2012 22:33:04 +0200 Received: from localhost (localhost [127.0.0.1]) by mailbox.adnet.avionic-design.de (Postfix) with ESMTP id 56C3A2A282FE; Mon, 13 Aug 2012 22:33:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at avionic-design.de Received: from mailbox.adnet.avionic-design.de ([127.0.0.1]) by localhost (mailbox.avionic-design.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9PzNdqXFhKGf; Mon, 13 Aug 2012 22:33:01 +0200 (CEST) Received: from localhost (avionic-0098.adnet.avionic-design.de [172.20.31.233]) (Authenticated sender: thierry.reding) by mailbox.adnet.avionic-design.de (Postfix) with ESMTPA id 728B52A282B2; Mon, 13 Aug 2012 22:33:01 +0200 (CEST) Date: Mon, 13 Aug 2012 22:33:00 +0200 From: Thierry Reding To: Stephen Warren Cc: Russell King , linux-tegra@vger.kernel.org, Bjorn Helgaas , linux-pci@vger.kernel.org, Grant Likely , Rob Herring , devicetree-discuss@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, Colin Cross , Olof Johansson , Mitch Bradley , Arnd Bergmann Subject: Re: [PATCH v3 00/10] ARM: tegra: Add PCIe device tree support Message-ID: <20120813203300.GA8891@avionic-0098.mockup.avionic-design.de> References: <1343332512-28762-1-git-send-email-thierry.reding@avionic-design.de> <50201E1D.5060200@wwwdotorg.org> <20120813174003.GA2527@avionic-0098.mockup.avionic-design.de> <50294BCA.1070807@wwwdotorg.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <50294BCA.1070807@wwwdotorg.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Provags-ID: V02:K0:4694K46brXuomv6mYPwEmSQCvJGFXFpDYHWtTZyz6cy RwbRpbqCexLzdP33wfATOPtyhMporvSLX6pQtOuESJD2J0JZ+u irYZg9VUBm5fqj8Mgn7M+lQRLvHq+Z1OLiYXkkBzoBty0ENRxH JtkkGI8wGCAVeY628ntbxe+c73LLX+kiZaFrkdjpRiU0R1wNSV i2Ldvc2XmYHrOU6LBYdRxViRXbsloEUS2SuMkC6iFAjELn2zKO dMPwKdoFYLvh8fk1DxSvYXfAZrNhv1ldh9zhurGVZmmb7VeI+B i/08Ijgb2MU8OWrXBfaY+CVVRKeL6ygEs+FqyEHQCWdB50x8ZY WVrwU60WnW/kbq8l8bZUJlHj7OkHZP2JhHdtrV/Z0ptTCvQ5iD VrKzCfDeu1ZUGUj1VWw9RccyDDSrY2rQJ2ab7NBqzG53rGfVdR onF8d Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Mon, Aug 13, 2012 at 12:47:38PM -0600, Stephen Warren wrote: > On 08/13/2012 11:40 AM, Thierry Reding wrote: > > On Mon, Aug 06, 2012 at 01:42:21PM -0600, Stephen Warren wrote: > >> On 07/26/2012 01:55 PM, Thierry Reding wrote: > >>> This patch series adds support for device tree based probing of > >>> the PCIe controller found on Tegra SoCs. > >> > >> Thierry, I just tested all Tegra boards in v3.6-rc1, and noticed > >> that PCIe doesn't work on TrimSlice when booting use device tree. > >> I think I found the cause, and I can't see why the same problem > >> doesn't affect this series. Perhaps you can enlighten me? > ... > >> PCI: Device 0000:01:00.0 not available because of resource > >> collisions > ... > > I've looked into this a bit, and it seems like ARM is using an > > open- coded version of the pci_enable_resources() function here, > > with the only difference being the unconditional enabling of both > > I/O and memory- mapped access for bridges. On Tegra there is > > already a PCI fixup to do this, so pci_enable_resources() can be > > used as-is. I came up with the attached patch but haven't been able > > to test it yet. > > Thanks very much for looking into this. > > The patch did alter the behavior a little for TrimSlice, but didn't > solve the problem. The old error messages: > > > [ 2.173971] PCI: Device 0000:01:00.0 not available because of resource collisions > > [ 2.181453] r8169 0000:01:00.0: (unregistered net_device): enable failure > > [ 2.188254] r8169: probe of 0000:01:00.0 failed with error -22 > > Were replaced with the following with your patch: > > > [ 2.174010] r8169 0000:01:00.0: device not available (can't reserve [io 0x0000-0x00ff]) > > [ 2.182098] r8169 0000:01:00.0: (unregistered net_device): enable failure > > [ 2.188900] r8169: probe of 0000:01:00.0 failed with error -22 > > This message appears from drivers/pci/setup-res.c pci_enable_resources() > due to: > > > if (!r->parent) { > > dev_err(&dev->dev, "device not available " > > "(can't reserve %pR)\n", r); > > return -EINVAL; > > } Looking at the code some more, this may be caused by the pci_remap_io() patch series, so you might want to revert that patch and see if it fixes the I/O resources. > That check doesn't appear in ARM's custom pcibios_enable_device(). > Disabling that check yields: > > > [ 2.174192] r8169 0000:01:00.0: enabling device (0140 -> 0143) > > [ 2.180041] r8169 0000:01:00.0: BAR 2: can't reserve [mem 0x00000000-0x00000fff 64bit pref] > > [ 2.188386] r8169 0000:01:00.0: (unregistered net_device): could not request regions > > [ 2.196140] r8169: probe of 0000:01:00.0 failed with error -16 > > I think that's because the pci_dev's resources are initially assigned > PCI-aperture-relative addresses, and then these are later patched up to > take account of where the aperture is mapped into the CPU's address space. > > Boot log using board files: > > > [ 1.146145] pci 0000:01:00.0: reg 10: [io 0x0000-0x00ff] > > [ 1.151745] pci 0000:01:00.0: reg 18: [mem 0x00000000-0x00000fff 64bit pref] > > [ 1.159007] pci 0000:01:00.0: reg 20: [mem 0x00000000-0x00003fff 64bit pref] > > [ 1.166270] pci 0000:01:00.0: reg 30: [mem 0x00000000-0x0001ffff pref] > ... > > [ 1.217829] pci 0000:01:00.0: BAR 6: assigned [mem 0xa0000000-0xa001ffff pref] > > [ 1.225264] pci 0000:01:00.0: BAR 4: assigned [mem 0xa0020000-0xa0023fff 64bit pref] > > [ 1.233236] pci 0000:01:00.0: BAR 2: assigned [mem 0xa0024000-0xa0024fff 64bit pref] > > [ 1.241206] pci 0000:01:00.0: BAR 0: assigned [io 0x1000-0x10ff] > ... (I added some extra printks:) > > [ 1.488007] r8169 0000:01:00.0: BAR 0: requesting [io 0x1000-0x10ff] > > [ 1.501483] r8169 0000:01:00.0: BAR 2: requesting [mem 0xa0024000-0xa0024fff 64bit pref] > > [ 1.516611] r8169 0000:01:00.0: BAR 4: requesting [mem 0xa0020000-0xa0023fff 64bit pref] > > whereas for a device tree boot: > > (same): > > [ 2.112217] pci 0000:01:00.0: reg 10: [io 0x0000-0x00ff] > > [ 2.117635] pci 0000:01:00.0: reg 18: [mem 0x00000000-0x00000fff 64bit pref] > > [ 2.124690] pci 0000:01:00.0: reg 20: [mem 0x00000000-0x00003fff 64bit pref] > > [ 2.131731] pci 0000:01:00.0: reg 30: [mem 0x00000000-0x0001ffff pref] > ... (request region happens early) > > [ 2.179838] r8169 0000:01:00.0: BAR 0: requesting [io 0x0000-0x00ff] > > [ 2.193312] r8169 0000:01:00.0: BAR 2: requesting [mem 0x00000000-0x00000fff 64bit pref] > > [ 2.201397] r8169 0000:01:00.0: BAR 2: can't reserve [mem 0x00000000-0x00000fff 64bit pref] > > [ 2.209742] r8169 0000:01:00.0: (unregistered net_device): could not request regions > ... (same, just happens too late) > > [ 2.236818] pci 0000:01:00.0: BAR 6: assigned [mem 0xa0000000-0xa001ffff pref] > > [ 2.244027] pci 0000:01:00.0: BAR 4: assigned [mem 0xa0020000-0xa0023fff 64bit pref] > > [ 2.251794] pci 0000:01:00.0: BAR 2: assigned [mem 0xa0024000-0xa0024fff 64bit pref] > > [ 2.259542] pci 0000:01:00.0: BAR 0: assigned [io 0x1000-0x10ff] > > I suspect this is all still related to the PCI devices themselves being > probed much earlier in the overall PCI initialization sequence when the > PCI controller is probed later in the boot sequence, whereas PCI device > probe is deferred until the overall PCI initialization sequence is > complete if the PCI controller is probed very early in the boot sequence. > > Does anyone know where/what that "probe now" vs. "probe later" decision > point is? I'll try and track it down if nobody beats me to it. There's the io_offset and mem_offset fields that I've completely ignored up to now. Can you try the patch below to see if it changes anything? I'm sorry but I can't test any of this myself right now. Thierry diff --git a/arch/arm/mach-tegra/pcie.c b/arch/arm/mach-tegra/pcie.c index 3463fb5..9b9b3e0 100644 --- a/arch/arm/mach-tegra/pcie.c +++ b/arch/arm/mach-tegra/pcie.c @@ -395,7 +395,7 @@ static int tegra_pcie_setup(int nr, struct pci_sys_data *sys) pp->res[0].flags = IORESOURCE_MEM; if (request_resource(&iomem_resource, &pp->res[0])) panic("Request PCIe Memory resource failed\n"); - pci_add_resource_offset(&sys->resources, &pp->res[0], sys->mem_offset); + pci_add_resource_offset(&sys->resources, &pp->res[0], pp->res[0].start); /* * IORESOURCE_MEM | IORESOURCE_PREFETCH @@ -414,7 +414,7 @@ static int tegra_pcie_setup(int nr, struct pci_sys_data *sys) pp->res[1].flags = IORESOURCE_MEM | IORESOURCE_PREFETCH; if (request_resource(&iomem_resource, &pp->res[1])) panic("Request PCIe Prefetch Memory resource failed\n"); - pci_add_resource_offset(&sys->resources, &pp->res[1], sys->mem_offset); + pci_add_resource_offset(&sys->resources, &pp->res[1], pp->res[1].start); return 1; }