From patchwork Fri Sep 11 17:52:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 11771291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F20959D for ; Fri, 11 Sep 2020 17:54:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8B8D21D79 for ; Fri, 11 Sep 2020 17:54:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MdaESNqF"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="V02wtmeO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8B8D21D79 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type:MIME-Version: Cc:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: References:In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YjFU0RHIpm5/l8BNvoNye1p/LGOS1RBoObtREPl53Sc=; b=MdaESNqFA0C5LaHgWmuGaAoWr OsZvubBrYjfL8ef8ToAPFHxu91R36fMU7BN05z7lt0Zw6nCB0e2awKrh50fkOr50GP95Sa4wDkPRm YW/HfZT6Fox0lRutLg/QA6DSPDK9bx1nBehDwl+arrWYdANLZ7Eiq1FADwADbM8mPiQIs2bFlgW7M cBePtmc5uwQ9JWyFLfVDQUFkAXdDKOG2TbFO8MS6lsi6wjNtrbKTN4htgUzLc994nCaR5kg3gqchk fqrjTlTDEo+EJLiM3WrP5p1i0x+Ue64oWw7axDs+F4lVcHxSsMBD+PMCZ10q8UP0ksGMSN3szBGA3 LyUhFTAYQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kGnFM-0006hk-5L; Fri, 11 Sep 2020 17:54:04 +0000 Received: from mail-pj1-x1044.google.com ([2607:f8b0:4864:20::1044]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kGnEJ-0006KH-VQ for linux-arm-kernel@lists.infradead.org; Fri, 11 Sep 2020 17:53:11 +0000 Received: by mail-pj1-x1044.google.com with SMTP id fa1so2075657pjb.0 for ; Fri, 11 Sep 2020 10:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oqinev3I/sBLLUQvoEFdn+sdtM0AP9V2wgt9sEN8sYg=; b=V02wtmeOAT4YbSAFOymwJNxm5A+i02VptRHUDYx3PEY0NgxB3V/IokyWf8W5kNjrq/ yhCu66G0Q9i648m2TsQ0Yx+KUDvyYzF2uwFnf5veVvqmRvY33T6ucdyyREylbPHnA5cw 2aeS90soKMcyCAkmYkkwtSb2heQiI9owqsgNw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oqinev3I/sBLLUQvoEFdn+sdtM0AP9V2wgt9sEN8sYg=; b=YicAn696v6sVk1EP9jqT5XPhMl+PRnIJQxdaNE0/mX0fiu+aJwmxqrbHjMZP4xSnut YP94vAN6KCE/cdsi3VCjw8svSVPPaMxny6zywx5O1aBDk459762GlSP0EsGr0bOZMEho 4R+/2osx7T7LgjCaHUWzAqKylAP+mQiBpE7C5J/R18UawbqRg88GR4wbd94YMXLVSHdR zIVCyknLpRPG+mL+6AsKqD0SHCmsIZ8FRc4RHwzNTjW9Das/7n1AoW1raZJmkV2qXiut 1Wyciereq4LDz/h+f93zj8gJK79cDOrprJZpvlX/2L2c23F7u3rsu8cSKl3DgyL8b/wd loWw== X-Gm-Message-State: AOAM533cv8FpmJwSJaUFshzQsbc6RD96ilerglpaIWbZ+J+y4Gz7C1+X LDyYhUfRUsyM0PkFWra7tzNVxw== X-Google-Smtp-Source: ABdhPJyHOXUq+eIxWdPk/FmdERUDomyyriLd4N9G41eP73dbkeiuIM38nD0ROjLBrxqretI3wsGfNw== X-Received: by 2002:a17:90a:df91:: with SMTP id p17mr3287426pjv.29.1599846777460; Fri, 11 Sep 2020 10:52:57 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.231.250]) by smtp.gmail.com with ESMTPSA id d77sm2871963pfd.121.2020.09.11.10.52.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2020 10:52:56 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Christoph Hellwig , Robin Murphy , bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com Subject: [PATCH v12 07/10] PCI: brcmstb: Set additional internal memory DMA viewport sizes Date: Fri, 11 Sep 2020 13:52:27 -0400 Message-Id: <20200911175232.19016-8-james.quinlan@broadcom.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200911175232.19016-1-james.quinlan@broadcom.com> References: <20200911175232.19016-1-james.quinlan@broadcom.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200911_135300_180661_C1721472 X-CRM114-Status: GOOD ( 28.40 ) X-Spam-Score: -0.1 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-0.1 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1044 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 T_TVD_MIME_NO_HEADERS BODY: No description available. -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 MIME_HEADER_CTYPE_ONLY 'Content-Type' found without required MIME headers -0.0 DKIMWL_WL_HIGH DKIMwl.org - Whitelisted High sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Herring , Lorenzo Pieralisi , open list , Florian Fainelli , "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" , Bjorn Helgaas , "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org The Raspberry Pi (RPI) is currently the only chip using this driver (pcie-brcmstb.c). There, only one memory controller is used, without an extension region, and the SCB0 viewport size is set to the size of the first and only dma-range region. Other BrcmSTB SOCs have more complicated memory configurations that require setting additional viewport sizes. BrcmSTB PCIe controllers are intimately connected to the memory controller(s) on the SOC. The SOC may have one to three memory controllers; they are indicated by the term SCBi. Each controller has a base region and an optional extension region. In physical memory, the base and extension regions of a controller are not adjacent, but in PCIe-space they are. There is a "viewport" for each memory controller that allows DMA from endpoint devices. Each viewport's size must be set to a power of two, and that size must be equal to or larger than the amount of memory each controller supports which is the sum of base region and its optional extension. Further, the 1-3 viewports are also adjacent in PCIe-space. Unfortunately the viewport sizes cannot be ascertained from the "dma-ranges" property so they have their own property, "brcm,scb-sizes". This is because dma-range information does not indicate what memory controller it is associated. For example, consider the following case where the size of one dma-range is 2GB and the second dma-range is 1GB: /* Case 1: SCB0 size set to 4GB */ dma-range0: 2GB (from memc0-base) dma-range1: 1GB (from memc0-extension) /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */ dma-range0: 2GB (from memc0-base) dma-range1: 1GB (from memc0-extension) By just looking at the dma-ranges information, one cannot tell which situation applies. That is why an additional property is needed. Its length indicates the number of memory controllers being used and each value indicates the viewport size. Note that the RPI DT does not have a "brcm,scb-sizes" property value, as it is assumed that it only requires one memory controller and no extension. So the optional use of "brcm,scb-sizes" will be backwards compatible. One last layer of complexity exists: all of the viewports sizes must be added and rounded up to a power of two to determine what the "BAR" size is. Further, an offset must be given that indicates the base PCIe address of this "BAR". The use of the term BAR is typically associated with endpoint devices, and the term is used here because the PCIe HW may be used as an RC or an EP. In the former case, all of the system memory appears in a single "BAR" region in PCIe memory. As it turns out, BrcmSTB PCIe HW is rarely used in the EP role and its system of mapping memory is an artifact that requires multiple dma-ranges regions. Signed-off-by: Jim Quinlan Acked-by: Florian Fainelli Reviewed-by: Rob Herring --- drivers/pci/controller/pcie-brcmstb.c | 67 ++++++++++++++++++++------- 1 file changed, 49 insertions(+), 18 deletions(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index 8a84811972bd..330f68c5b579 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -57,6 +57,9 @@ #define PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK 0x300000 #define PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128 0x0 #define PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK 0xf8000000 +#define PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK 0x07c00000 +#define PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK 0x0000001f +#define SCB_SIZE_MASK(x) PCIE_MISC_MISC_CTRL_SCB ## x ## _SIZE_MASK #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO 0x400c #define PCIE_MEM_WIN0_LO(win) \ @@ -159,6 +162,7 @@ #define SSC_STATUS_OFFSET 0x1 #define SSC_STATUS_SSC_MASK 0x400 #define SSC_STATUS_PLL_LOCK_MASK 0x800 +#define PCIE_BRCM_MAX_MEMC 3 #define IDX_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_INDEX]) #define DATA_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_DATA]) @@ -737,22 +741,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie, u64 *rc_bar2_offset) { struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie); - struct device *dev = pcie->dev; struct resource_entry *entry; + struct device *dev = pcie->dev; + u64 lowest_pcie_addr = ~(u64)0; + int ret, i = 0; + u64 size = 0; - entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM); - if (!entry) - return -ENODEV; + resource_list_for_each_entry(entry, &bridge->dma_ranges) { + u64 pcie_beg = entry->res->start - entry->offset; + size += entry->res->end - entry->res->start + 1; + if (pcie_beg < lowest_pcie_addr) + lowest_pcie_addr = pcie_beg; + } - /* - * The controller expects the inbound window offset to be calculated as - * the difference between PCIe's address space and CPU's. The offset - * provided by the firmware is calculated the opposite way, so we - * negate it. - */ - *rc_bar2_offset = -entry->offset; - *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start); + if (lowest_pcie_addr == ~(u64)0) { + dev_err(dev, "DT node has no dma-ranges\n"); + return -EINVAL; + } + + ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1, + PCIE_BRCM_MAX_MEMC); + + if (ret <= 0) { + /* Make an educated guess */ + pcie->num_memc = 1; + pcie->memc_size[0] = 1ULL << fls64(size - 1); + } else { + pcie->num_memc = ret; + } + + /* Each memc is viewed through a "port" that is a power of 2 */ + for (i = 0, size = 0; i < pcie->num_memc; i++) + size += pcie->memc_size[i]; + + /* System memory starts at this address in PCIe-space */ + *rc_bar2_offset = lowest_pcie_addr; + /* The sum of all memc views must also be a power of 2 */ + *rc_bar2_size = 1ULL << fls64(size - 1); /* * We validate the inbound memory view even though we should trust @@ -804,12 +830,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) void __iomem *base = pcie->base; struct device *dev = pcie->dev; struct resource_entry *entry; - unsigned int scb_size_val; bool ssc_good = false; struct resource *res; int num_out_wins = 0; u16 nlw, cls, lnksta; - int i, ret; + int i, ret, memc; u32 tmp, aspm_support; /* Reset the bridge */ @@ -844,11 +869,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) writel(upper_32_bits(rc_bar2_offset), base + PCIE_MISC_RC_BAR2_CONFIG_HI); - scb_size_val = rc_bar2_size ? - ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */ tmp = readl(base + PCIE_MISC_MISC_CTRL); - u32p_replace_bits(&tmp, scb_size_val, - PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK); + for (memc = 0; memc < pcie->num_memc; memc++) { + u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15; + + if (memc == 0) + u32p_replace_bits(&tmp, scb_size_val, SCB_SIZE_MASK(0)); + else if (memc == 1) + u32p_replace_bits(&tmp, scb_size_val, SCB_SIZE_MASK(1)); + else if (memc == 2) + u32p_replace_bits(&tmp, scb_size_val, SCB_SIZE_MASK(2)); + } writel(tmp, base + PCIE_MISC_MISC_CTRL); /*