From patchwork Fri Oct 6 23:21:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 9990711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A2CE16020F for ; Fri, 6 Oct 2017 23:22:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9155728E35 for ; Fri, 6 Oct 2017 23:22:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84A6828E38; Fri, 6 Oct 2017 23:22:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EC2DF28E35 for ; Fri, 6 Oct 2017 23:22:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ks7ZmKg53TWdN/tcFH2eBdopUxodYdpMn1HEF7Kzdz4=; b=jYoJQ0IGxx8yaC eVTCNtXx8S7gIa8WJncFWCSkyTUHv1rpripHRGo87uItTpjjeniZZ8HAge3/B28FCkG9/PZpSWzGc b/3OZGKs75Belh7li9jn1v9eOkNS74g1VpQBZuJ5Vn17LMdkXpbMqDsaohm471UjbI9czRLKIV79X Kp/E2hX/Z1mQZPiyZ3weT0Lxb/EZAVmhsR1kGTnPlAI5ybVZS9Rjt/BLcCT8uIkQ+NxJG1EPa3v2g 8GuC47ICOKvGkXu5QfzxqeLKccVJfMMynnOJKYogJcG+go7MGroFcAm+eYboqt7Q0uABB2rLeAZ+q wUepn79vVTfyUez6hzJA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e0bwn-0002ZO-G6; Fri, 06 Oct 2017 23:22:25 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e0bwj-0002SD-QW for linux-arm-kernel@lists.infradead.org; Fri, 06 Oct 2017 23:22:23 +0000 Received: from localhost (unknown [69.71.4.159]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A3D7E21881; Fri, 6 Oct 2017 23:22:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3D7E21881 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=helgaas@kernel.org Date: Fri, 6 Oct 2017 18:21:59 -0500 From: Bjorn Helgaas To: Ard Biesheuvel Subject: Re: [PATCH v4 1/2] PCI: pci-host-generic: add support for Synopsys DesignWare RC in ECAM mode Message-ID: <20171006232158.GI25517@bhelgaas-glaptop.roam.corp.google.com> References: <20171006163919.14898-1-ard.biesheuvel@linaro.org> <20171006163919.14898-2-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20171006163919.14898-2-ard.biesheuvel@linaro.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171006_162221_924592_3F35E800 X-CRM114-Status: GOOD ( 28.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Herring , Graeme Gregory , linux-pci@vger.kernel.org, Will Deacon , linux-kernel@vger.kernel.org, Leif Lindholm , Bjorn Helgaas , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, Oct 06, 2017 at 05:39:18PM +0100, Ard Biesheuvel wrote: > Some implementations of the Synopsys DesignWare PCIe controller implement > a so-called ECAM shift mode, which allows a static memory window to be > configured that covers the configuration space of the entire bus range. > > Usually, when the firmware performs all the low level configuration that > is required to expose this controller in a fully ECAM compatible manner, > we can simply describe it as "pci-host-ecam-generic" and be done with it. > However, in some cases (e.g., the Marvell Armada 80x0 as well as the > Socionext SynQuacer Soc), the IP was synthesized with an ATU window > granularity that does not allow the first bus to be mapped in a way that > prevents the device on the downstream port from appearing more than once, > and so we still need special handling in software to drive this static > almost-ECAM configuration. > > So extend the pci-host-generic driver so it can support these controllers > as well, by adding special config space accessors that take the above > quirk into account. > > Note that, unlike most drivers for this IP, this driver does not expose > a fake bridge device at B/D/F 00:00.0. There is no point in doing so, > given that this is not a true bridge, and does not require any windows > to be configured in order for the downstream device to operate correctly. > Omitting it also prevents the PCI resource allocation routines from > handing out BAR space to it unnecessarily. > > Signed-off-by: Ard Biesheuvel > --- > drivers/pci/host/pci-host-generic.c | 46 ++++++++++++++++++++ > 1 file changed, 46 insertions(+) > > diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c > index 7d709a7e0aa8..01e81a30e303 100644 > --- a/drivers/pci/host/pci-host-generic.c > +++ b/drivers/pci/host/pci-host-generic.c > @@ -35,6 +35,43 @@ static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = { > } > }; > > +static int pci_dw_ecam_config_read(struct pci_bus *bus, u32 devfn, int where, > + int size, u32 *val) > +{ > + struct pci_config_window *cfg = bus->sysdata; > + > + /* > + * The Synopsys DesignWare PCIe controller in ECAM mode will not filter > + * type 0 config TLPs sent to devices 1 and up on its downstream port, > + * resulting in devices appearing multiple times on bus 0 unless we > + * filter out those accesses here. > + */ > + if (bus->number == cfg->busr.start && PCI_SLOT(devfn) > 0) > + return PCIBIOS_DEVICE_NOT_FOUND; I think we should return 0xffffffff data here, as we do in other similar accessors (dw_pcie_rd_conf(), altera_pcie_cfg_read(), rockchip_pcie_rd_conf()). Actually, xilinx-nwl has a nice trick: it implements nwl_pcie_map_bus(), which returns NULL for invalid devices. Then it can use the generic accessors, and pci_generic_config_read() already fills in ~0 for this case. What do you think of the following? I put it on my pci/host-generic branch for now (pending your response and an ack from Will). commit 5aec78ea05fe23c7c17f8e6c757bc64fb9142728 Author: Ard Biesheuvel Date: Fri Oct 6 17:39:18 2017 +0100 PCI: generic: Add support for Synopsys DesignWare RC in ECAM mode Some implementations of the Synopsys DesignWare PCIe controller implement a so-called ECAM shift mode, which allows a static memory window to be configured that covers the configuration space of the entire bus range. Usually, when the firmware performs all the low level configuration that is required to expose this controller in a fully ECAM compatible manner, we can simply describe it as "pci-host-ecam-generic" and be done with it. However, in some cases (e.g., the Marvell Armada 80x0 as well as the Socionext SynQuacer Soc), the IP was synthesized with an ATU window granularity that does not allow the first bus to be mapped in a way that prevents the device on the downstream port from appearing more than once, and so we still need special handling in software to drive this static almost-ECAM configuration. So extend the pci-host-generic driver so it can support these controllers as well, by adding special config space accessors that take the above quirk into account. Note that, unlike most drivers for this IP, this driver does not expose a fake bridge device at B/D/F 00:00.0. There is no point in doing so, given that this is not a true bridge, and does not require any windows to be configured in order for the downstream device to operate correctly. Omitting it also prevents the PCI resource allocation routines from handing out BAR space to it unnecessarily. Signed-off-by: Ard Biesheuvel [bhelgaas: factor out pci_dw_valid_device(), add pci_dw_ecam_map_bus() and use generic read/write functions] Signed-off-by: Bjorn Helgaas Acked-by: Will Deacon diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c index 7d709a7e0aa8..2f05511ce718 100644 --- a/drivers/pci/host/pci-host-generic.c +++ b/drivers/pci/host/pci-host-generic.c @@ -35,6 +35,40 @@ static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = { } }; +static bool pci_dw_valid_device(struct pci_bus *bus, unsigned int devfn) +{ + struct pci_config_window *cfg = bus->sysdata; + + /* + * The Synopsys DesignWare PCIe controller in ECAM mode will not filter + * type 0 config TLPs sent to devices 1 and up on its downstream port, + * resulting in devices appearing multiple times on bus 0 unless we + * filter out those accesses here. + */ + if (bus->number == cfg->busr.start && PCI_SLOT(devfn) > 0) + return false; + + return true; +} + +static void __iomem *pci_dw_ecam_map_bus(struct pci_bus *bus, + unsigned int devfn, int where) +{ + if (!pci_dw_valid_device(bus, devfn)) + return NULL; + + return pci_ecam_map_bus(bus, devfn, where); +} + +static struct pci_ecam_ops pci_dw_ecam_bus_ops = { + .bus_shift = 20, + .pci_ops = { + .map_bus = pci_dw_ecam_map_bus, + .read = pci_generic_config_read, + .write = pci_generic_config_write, + } +}; + static const struct of_device_id gen_pci_of_match[] = { { .compatible = "pci-host-cam-generic", .data = &gen_pci_cfg_cam_bus_ops }, @@ -42,6 +76,15 @@ static const struct of_device_id gen_pci_of_match[] = { { .compatible = "pci-host-ecam-generic", .data = &pci_generic_ecam_ops }, + { .compatible = "marvell,armada8k-pcie-ecam", + .data = &pci_dw_ecam_bus_ops }, + + { .compatible = "socionext,synquacer-pcie-ecam", + .data = &pci_dw_ecam_bus_ops }, + + { .compatible = "snps,dw-pcie-ecam", + .data = &pci_dw_ecam_bus_ops }, + { }, };