From patchwork Fri Mar 28 01:03:55 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Neil Greatorex X-Patchwork-Id: 3900501 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 9AC9C9F2B6 for ; Fri, 28 Mar 2014 01:04:58 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7E6B6202BE for ; Fri, 28 Mar 2014 01:04:57 +0000 (UTC) Received: from casper.infradead.org (casper.infradead.org [85.118.1.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C001202B4 for ; Fri, 28 Mar 2014 01:04:55 +0000 (UTC) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WTLDq-0008Ko-4H; Fri, 28 Mar 2014 01:04:38 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WTLDn-0006Fv-O4; Fri, 28 Mar 2014 01:04:35 +0000 Received: from mail-wi0-x22a.google.com ([2a00:1450:400c:c05::22a]) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WTLDk-0006Fa-92 for linux-arm-kernel@lists.infradead.org; Fri, 28 Mar 2014 01:04:33 +0000 Received: by mail-wi0-f170.google.com with SMTP id bs8so236874wib.1 for ; Thu, 27 Mar 2014 18:04:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fatboyfat.co.uk; s=google; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version:content-type; bh=8i6YXp5F9CA4NtZXIpQyEC9hIbwNSVw0iA+wTT/4ywY=; b=Cy8plbuikUkEFkHpFt69ux5DMryXd7auNRI9pC0bbU5kJLQM96/EDtKwLY/WsgN2Dq hh2b6HOPRrvkM6D5dObiV3FNuLPt2LR8AnK7MS4AgBAFOFRCV+Qtnmh73Aa7GqFAG9IB SgmXjPqNXKifPFEYjVBRtZuFqwl7NMTc+IjWg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type; bh=8i6YXp5F9CA4NtZXIpQyEC9hIbwNSVw0iA+wTT/4ywY=; b=MG7yUvWyEFrfiHgJQUtBFjtxB5r3ZRwynkAC+ZT4lkDuIORXvp23+GNzVduLsuYSyD nuCZ2cVwHauhDljRHrZjspGVgI+MSOPryJ2oY1/TdJbw6DbtS1ch0DkGu4L3jQCLKTxc xw/CW77e/FPnD9h2JcQ6RENmNZe1bCoY+Z5P9DKKxlev3iprH8nlgcMSzQTo8dcp3BMT /yjkuFoRJQxjvKeRrN0onJJ+1d21p419FTO00zUuTZmRUnOJX7C1Q8qrHzOB4PgBTMuC Yn2c27Or4b24B5Otf0EVtPK08EYuXliLUBZmSBxBlrSod6h8m60MQkC8Wh0fwFa9rXZP JOBw== X-Gm-Message-State: ALoCoQl/beWgIxEph2qQH8Um95CCi5VC4UhPIq5AwuhczWsxfKb/nfdc4lSPmSsegEcJJBYBgsME X-Received: by 10.180.107.136 with SMTP id hc8mr43795447wib.11.1395968645323; Thu, 27 Mar 2014 18:04:05 -0700 (PDT) Received: from vroombuntu.lan (97e4ff9e.skybroadband.com. [151.228.255.158]) by mx.google.com with ESMTPSA id fs8sm1842348wib.8.2014.03.27.18.04.03 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 18:04:04 -0700 (PDT) Date: Fri, 28 Mar 2014 01:03:55 +0000 (GMT) From: Neil Greatorex X-X-Sender: neil@vroombuntu To: Jason Gunthorpe , Thomas Petazzoni Subject: Re: Intel I350 mini-PCIe card (igb) on Mirabox (mvebu / Armada 370) In-Reply-To: <20140327044054.GA22681@obsidianresearch.com> Message-ID: References: <20140325202249.GA10378@obsidianresearch.com> <20140325213638.5aba54b6@skate> <20140325222404.GC14718@obsidianresearch.com> <20140325223510.GD14718@obsidianresearch.com> <20140326201243.GA1536@obsidianresearch.com> <20140326214259.GA12330@obsidianresearch.com> <20140327044054.GA22681@obsidianresearch.com> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140327_210432_652285_8ACFCFB6 X-CRM114-Status: GOOD ( 20.06 ) X-Spam-Score: -2.0 (--) Cc: Bjorn Helgaas , Jason Cooper , linux-arm-kernel , Willy Tarreau X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Jason, Thomas, With the help you've given, I think I've identified what the problem is. It is basically what Thomas suggested the problem was (that he'd seen with the Armada 38x). The call to mvebu_pcie_set_local_dev_nr causes the PCIe link to reset. In my tests with this Intel card, it takes ~25ms for the link to go down, and then another ~13ms for it to come back up. I have put together a patch below that I would be interested to get your opinions on and also for you to test against any cards you may have. I only have the one mini PCIe card so it's not a great test! With my card, it is now detected correctly, and there are no issues when rescanning the PCI bus. Cheers, Neil From a0ca1552e737c18b5651fe1f27ade76f512ca191 Mon Sep 17 00:00:00 2001 From: Neil Greatorex Date: Fri, 28 Mar 2014 00:35:04 +0000 Subject: [PATCH] PCI: mvebu: Wait for PCIe link to reset after setting device number After setting the local device nr, the PCIe link may go down briefly. If we don't wait for the link to come back up we may miss that device when we scan the bus (assuming it is not present). This patch waits up to 40ms for the link to go down, and if it did go down, waits up to 100ms for it to come back up. If the link was down to begin with, it will not wait at all. Signed-off-by: Neil Greatorex --- drivers/pci/host/pci-mvebu.c | 57 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) continue; @@ -975,8 +976,64 @@ static int mvebu_pcie_probe(struct platform_device *pdev) continue; } + /* After setting the local dev number, the PCI-E link may go + * down for a period of time. If we don't wait for it to come + * back up, then we risk missing enumerating that device when + * we scan the bus. However, if the link was down to begin with, + * there's no point waiting, so check now. + */ + link_present = mvebu_pcie_link_up(port); + mvebu_pcie_set_local_dev_nr(port, 1); + if (link_present) { + /* In testing, the link took ~25ms to go down, and + * another ~15ms to come back up, so wait ~40ms for the + * link to go down, and then up to ~100ms for it to come + * back up. If the link doesn't go down after ~40ms, then + * it probably won't go down at all, so carry on. + */ + if (mvebu_pcie_link_up(port)) { + while (retry_count < 40) { + if (!mvebu_pcie_link_up(port)) + break; + retry_count++; + mdelay(1); + } + + if (retry_count == 40) { + dev_dbg(&pdev->dev, + "PCIe%d.%d: after setting dev nr, link stayed up\n", + port->port, port->lane); + } else { + dev_info(&pdev->dev, + "PCIe%d.%d: after setting dev nr, link went down after %d polls\n", + port->port, port->lane, retry_count); + } + } + + retry_count = 0; + + if (!mvebu_pcie_link_up(port)) { + while (retry_count < 100) { + if (mvebu_pcie_link_up(port)) + break; + retry_count++; + mdelay(1); + } + + if (retry_count == 100) { + dev_info(&pdev->dev, + "PCIe%d.%d: after setting dev nr, link failed to come back up (timeout)\n", + port->port, port->lane); + } else { + dev_info(&pdev->dev, + "PCIe%d.%d: after setting dev nr, link came back up after %d polls ", + port->port, port->lane, retry_count); + } + } + } + port->dn = child; spin_lock_init(&port->conf_lock); mvebu_sw_pci_bridge_init(port); diff --git a/drivers/pci/host/pci-mvebu.c b/drivers/pci/host/pci-mvebu.c index 0e79665..afd0dce 100644 --- a/drivers/pci/host/pci-mvebu.c +++ b/drivers/pci/host/pci-mvebu.c @@ -891,6 +891,7 @@ static int mvebu_pcie_probe(struct platform_device *pdev) for_each_child_of_node(pdev->dev.of_node, child) { struct mvebu_pcie_port *port = &pcie->ports[i]; enum of_gpio_flags flags; + u32 link_present, retry_count = 0; if (!of_device_is_available(child))