From patchwork Thu Dec 14 13:56:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 10112231 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 02A7B60327 for ; Thu, 14 Dec 2017 13:56:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8F7428D9B for ; Thu, 14 Dec 2017 13:56:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9DA3C28DAD; Thu, 14 Dec 2017 13:56:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E519828D9B for ; Thu, 14 Dec 2017 13:56:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751447AbdLNN4Z (ORCPT ); Thu, 14 Dec 2017 08:56:25 -0500 Received: from mail-wm0-f65.google.com ([74.125.82.65]:44424 "EHLO mail-wm0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751084AbdLNN4Y (ORCPT ); Thu, 14 Dec 2017 08:56:24 -0500 Received: by mail-wm0-f65.google.com with SMTP id t8so11448611wmc.3; Thu, 14 Dec 2017 05:56:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Cj63sDcnUtI5cHnzSd0dFyKiFhc0s+SKldkS9ZUzQ9o=; b=pa2NHaf7SlAAUrSGgDpBDq0hF6umGRxDyK04v83U6ON6ucgCxMQZma5yqztEBRaLCb nHC3ZdXTKydSRL/rEWaMQTbiQ5fRqrKUyPU+BDCbZi6iER5Ja55XoGBEK5WeU+YG4l09 iJNDz+TCGIkfNuZ2kzDlElaw2u4N1nSEwd0tH+piHYeyXhYXJf/BOJ9MKjza0zOrUljZ HZdTSohbV8NZKdt2HugxJgMiakmIs4XertNyf6AXDu3ZU1P8KxRu9cgAxiwsa4FNZlAV TrZxYRCyY8Q0yMZ1sw7QcPjIyFQ6skeR32qczv+gF3/6VPEB33sSmhNtqthqWoSJGPvA 4Fnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Cj63sDcnUtI5cHnzSd0dFyKiFhc0s+SKldkS9ZUzQ9o=; b=SD/aCwNWB5yPljvuu4IOWQm2chQGh7hGTifyVxLlRlmo+q1OlYgZWHc7gdI1hcvLIF bRjid2RH2TBYWmMtdeGFKiHg3D6XjiSUVDARr8V0NOjkIKVZZTj5IltTmRdyjoQk8vHA TFGPM5N4GEnJspa1Rg/FCWsgVheQmqH+sspIAh+Cm7+8qVcvujr14PGwOg9V3/4sGwQa ppOz6mRYfFGuKWi1lgviHONkF640Y4FZvIUj67MDsvvFKzMv/fu/5xvpZ8aX9jXTdQp9 CmcDRozXyROJ43Asxe9vzdG2thEfTHwK7Yoy5deMeqqmGXWkGEtx32QqSNO1iYbBCCEq N+fg== X-Gm-Message-State: AKGB3mJV9f1xuytITN58NAgtxk5t/hNJrpqkgfT15zOfyUomhxEgxGmV M6zjTBzNXE4t1sli5xHJ2Lc= X-Google-Smtp-Source: ACJfBovSS+CA5v28M+v2t0OAz3+6lE55nr58r/ZVghLua3a3Ja+BJOD+PkFREMUNapTQTZWUS5MtcQ== X-Received: by 10.28.203.197 with SMTP id b188mr2101782wmg.105.1513259782688; Thu, 14 Dec 2017 05:56:22 -0800 (PST) Received: from localhost (p200300E41F200F003F65F430A8AE2E44.dip0.t-ipconnect.de. [2003:e4:1f20:f00:3f65:f430:a8ae:2e44]) by smtp.gmail.com with ESMTPSA id k135sm5404831wmd.45.2017.12.14.05.56.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Dec 2017 05:56:22 -0800 (PST) From: Thierry Reding To: Lorenzo Pieralisi , Bjorn Helgaas Cc: Jonathan Hunter , Vidya Saga , Manikanta Maddireddy , linux-pci@vger.kernel.org, linux-tegra@vger.kernel.org Subject: [PATCH v2] PCI: tegra: Do not allocate MSI target memory Date: Thu, 14 Dec 2017 14:56:20 +0100 Message-Id: <20171214135620.13564-1-thierry.reding@gmail.com> X-Mailer: git-send-email 2.15.1 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Thierry Reding The PCI host bridge found on Tegra SoCs doesn't require the MSI target address to be backed by physical system memory. Writes are intercepted within the controller and never make it to the memory pointed to. Since no actual system memory is required, remove the allocation of a single page and hardcode the MSI target address with a special address on a per-SoC basis. Ideally this would be an address to an MMIO memory region (such as where the controller's register are located). However, those addresses don't work reliably across all Tegra generations. The only set of addresses that work consistently are those that point to external memory. This is not ideal, since those addresses could technically be used for DMA and hence be confusing. However, the first page of external memory is unlikely to be used and special enough to avoid confusion. Signed-off-by: Thierry Reding --- Changes in v2: - use physical base address of system memory as MSI target address Vidya, Manikanta, I've tested this on all of Tegra20, Tegra30, Tegra124, Tegra210 and Tegra186, so I'm confident that this is finally a good set of addresses. However, it'd be good to get confirmation from you since we had conflicting test results the last time around. Thanks, Thierry drivers/pci/host/pci-tegra.c | 41 +++++++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/drivers/pci/host/pci-tegra.c b/drivers/pci/host/pci-tegra.c index ee193767f77b..f2214ca1ad17 100644 --- a/drivers/pci/host/pci-tegra.c +++ b/drivers/pci/host/pci-tegra.c @@ -236,7 +236,6 @@ struct tegra_msi { struct msi_controller chip; DECLARE_BITMAP(used, INT_PCI_MSI_NR); struct irq_domain *domain; - unsigned long pages; struct mutex lock; u64 phys; int irq; @@ -246,6 +245,7 @@ struct tegra_msi { struct tegra_pcie_soc { unsigned int num_ports; unsigned int msi_base_shift; + u64 msi_target; u32 pads_pll_ctl; u32 tx_ref_sel; u32 pads_refclk_cfg0; @@ -1576,9 +1576,35 @@ static int tegra_pcie_enable_msi(struct tegra_pcie *pcie) goto err; } - /* setup AFI/FPCI range */ - msi->pages = __get_free_pages(GFP_KERNEL, 0); - msi->phys = virt_to_phys((void *)msi->pages); + /* + * The PCI host bridge on Tegra contains some logic that intercepts + * MSI writes, which means that the MSI target address doesn't have + * to point to actual physical memory. Rather than allocating one 4 + * KiB page of system memory that's never used, we can simply pick + * an arbitrary address within a reserved area in the FPCI address + * map. + * + * Note that MSI writes are never committed to memory, so it doesn't + * really matter which address we pick. However, since the address + * may show up at some point and potentially confuse users, the best + * solution would be to pick an address that is obviously not within + * system memory. The FPCI address map contains a range that cannot + * be accessed by any Tegra CPU, but unfortunately that excludes the + * usage for 32-bit MSI capable devices. + * + * A good choice that supports both 32-bit and 64-bit MSI would be to + * pick a memory address within the PCI MMIO region that is currently + * not used for configuration space, downstream I/O, prefetchable or + * non prefetchable memory. However, it seems like not all chips will + * be able to intercept MSI writes to those addresses. + * + * The only reliable addresses that seem to work point to external + * memory. Pick the physical base address of system memory as the MSI + * target address. This is slightly less than ideal because it could + * technically, though unlikely, be used for DMA. However, since the + * MSI writes will be intercepted, this should be of little concern. + */ + msi->phys = soc->msi_target; afi_writel(pcie, msi->phys >> soc->msi_base_shift, AFI_MSI_FPCI_BAR_ST); afi_writel(pcie, msi->phys, AFI_MSI_AXI_BAR_ST); @@ -1630,8 +1656,6 @@ static int tegra_pcie_disable_msi(struct tegra_pcie *pcie) afi_writel(pcie, 0, AFI_MSI_EN_VEC6); afi_writel(pcie, 0, AFI_MSI_EN_VEC7); - free_pages(msi->pages, 0); - if (msi->irq > 0) free_irq(msi->irq, pcie); @@ -2140,6 +2164,7 @@ static void tegra_pcie_enable_ports(struct tegra_pcie *pcie) static const struct tegra_pcie_soc tegra20_pcie = { .num_ports = 2, .msi_base_shift = 0, + .msi_target = 0x00000000, .pads_pll_ctl = PADS_PLL_CTL_TEGRA20, .tx_ref_sel = PADS_PLL_CTL_TXCLKREF_DIV10, .pads_refclk_cfg0 = 0xfa5cfa5c, @@ -2155,6 +2180,7 @@ static const struct tegra_pcie_soc tegra20_pcie = { static const struct tegra_pcie_soc tegra30_pcie = { .num_ports = 3, .msi_base_shift = 8, + .msi_target = 0x80000000, .pads_pll_ctl = PADS_PLL_CTL_TEGRA30, .tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN, .pads_refclk_cfg0 = 0xfa5cfa5c, @@ -2171,6 +2197,7 @@ static const struct tegra_pcie_soc tegra30_pcie = { static const struct tegra_pcie_soc tegra124_pcie = { .num_ports = 2, .msi_base_shift = 8, + .msi_target = 0x80000000, .pads_pll_ctl = PADS_PLL_CTL_TEGRA30, .tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN, .pads_refclk_cfg0 = 0x44ac44ac, @@ -2186,6 +2213,7 @@ static const struct tegra_pcie_soc tegra124_pcie = { static const struct tegra_pcie_soc tegra210_pcie = { .num_ports = 2, .msi_base_shift = 8, + .msi_target = 0x80000000, .pads_pll_ctl = PADS_PLL_CTL_TEGRA30, .tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN, .pads_refclk_cfg0 = 0x90b890b8, @@ -2201,6 +2229,7 @@ static const struct tegra_pcie_soc tegra210_pcie = { static const struct tegra_pcie_soc tegra186_pcie = { .num_ports = 3, .msi_base_shift = 8, + .msi_target = 0x80000000, .pads_pll_ctl = PADS_PLL_CTL_TEGRA30, .tx_ref_sel = PADS_PLL_CTL_TXCLKREF_BUF_EN, .pads_refclk_cfg0 = 0x80b880b8,