From patchwork Mon Nov 20 08:40:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manivannan Sadhasivam X-Patchwork-Id: 13460910 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="GrzGFcsT" Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33C509E for ; Mon, 20 Nov 2023 00:40:37 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6cb7951d713so663722b3a.1 for ; Mon, 20 Nov 2023 00:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1700469636; x=1701074436; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DzhddvXC6cI/X85xHzqd3aV1zDt2NEPQsub4e4mhWPM=; b=GrzGFcsTE7E0OSD0KPfcnsc3hPHxUMa9PwkgiRBYKVEQgQumYz6QmuQw/4vabufGaT oR2iAYz5SRvIZpUlO+fpBhFQMAp+wavzIOoyF3ibbKhm/l0Vf6PJ+eKRwXPz1jB+sWQ0 4PTtuhRJNdX3Ij7pRQc7YY+HVBMxCWDjbEHf1taEkRorAU3Grrri5OTv/92R6FBJkGnt 60Zx+Irnj3HV1uL3MGjPvSXVq6gv+Nob+2fRBIHv3yI9ZBpzkLpVBgMiQgXawhVXLO8R q56/CgnNNOLY9p7auTKI9juLx7YC0IPfcbDNh+gm60aVNDN8s/73javJ/3/Cxdxqg2jM WVIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700469636; x=1701074436; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DzhddvXC6cI/X85xHzqd3aV1zDt2NEPQsub4e4mhWPM=; b=Nc3Yojt8w1E2VV43muQj1UiDmRmMzHMoQkqkDCQRclG8nskXVL4a+gCt1q6A38iRsE i3fiVLtibV4mrq3qbtS+lUjPozSLLzx9th62Gr8rf7x4Zm66s7btvtaTwuY7wtYNBoEx RDPq+tHJ5WzUjEPySr+YdjJY6xD4WSnN4ic4irbd4QnseRA1ZJK1YWLJIFVqM+1Z9wog oaSAOavRrUx9KPg/elcwpaFqIF9n8DkPcRSq9g1t4Gn3HJXr67LRSyzw47Q5umzQM6X5 nwSOe4h/lNjjwauQFmmR4lYYhCeAsAwasg/+ij8CEtx/2bf/sH76myypKA5yL6rnpzrk 9XZg== X-Gm-Message-State: AOJu0Yyi0QVDbJ4Kmi8uL9yajyLhTBBx+sL2dCcLupy0BHzvfsTE2FoB cs83bTwSpa9Bt5Tizc9txILN X-Google-Smtp-Source: AGHT+IEkJYS/cK3xWs4QajqygLgZGgqKY/XJaKKfd2aJxYCNj68mUvW+/8cvZW7Cgnl1LXFr7ygXVQ== X-Received: by 2002:a05:6a00:870d:b0:68f:a92a:8509 with SMTP id hj13-20020a056a00870d00b0068fa92a8509mr14956671pfb.7.1700469636618; Mon, 20 Nov 2023 00:40:36 -0800 (PST) Received: from localhost.localdomain ([117.248.2.5]) by smtp.gmail.com with ESMTPSA id e24-20020a62ee18000000b00694ebe2b0d4sm5567019pfi.191.2023.11.20.00.40.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 00:40:35 -0800 (PST) From: Manivannan Sadhasivam To: jingoohan1@gmail.com, gustavo.pimentel@synopsys.com, lpieralisi@kernel.org, robh@kernel.org, kw@linux.com, bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, quic_bjorande@quicinc.com, fancer.lancer@gmail.com, vidyas@nvidia.com, Manivannan Sadhasivam Subject: [PATCH v7 1/2] PCI: designware-ep: Fix DBI access before core init Date: Mon, 20 Nov 2023 14:10:13 +0530 Message-Id: <20231120084014.108274-2-manivannan.sadhasivam@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231120084014.108274-1-manivannan.sadhasivam@linaro.org> References: <20231120084014.108274-1-manivannan.sadhasivam@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The drivers for platforms requiring reference clock from the PCIe host for initializing their PCIe EP core, make use of the 'core_init_notifier' feature exposed by the DWC common code. On these platforms, access to the hw registers like DBI before completing the core initialization will result in a whole system hang. But the current DWC EP driver tries to access DBI registers during dw_pcie_ep_init() without waiting for core initialization and it results in system hang on platforms making use of 'core_init_notifier' such as Tegra194 and Qcom SM8450. To workaround this issue, users of the above mentioned platforms have to maintain the dependency with the PCIe host by booting the PCIe EP after host boot. But this won't provide a good user experience, since PCIe EP is _one_ of the features of those platforms and it doesn't make sense to delay the whole platform booting due to the PCIe dependency. So to fix this issue, let's move all the DBI access during dw_pcie_ep_init() in the DWC EP driver to the dw_pcie_ep_init_complete() API that gets called only after core initialization on these platforms. This makes sure that the DBI register accesses are skipped during dw_pcie_ep_init() and accessed later once the core initialization happens. For the rest of the platforms, DBI access happens as usual. Co-developed-by: Vidya Sagar Signed-off-by: Vidya Sagar Signed-off-by: Manivannan Sadhasivam --- .../pci/controller/dwc/pcie-designware-ep.c | 139 ++++++++++++------ 1 file changed, 91 insertions(+), 48 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index f6207989fc6a..b1c79cd8e25f 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -662,14 +662,19 @@ static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap) return 0; } -int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) +static int dw_pcie_ep_late_init(struct dw_pcie_ep *ep) { struct dw_pcie *pci = to_dw_pcie_from_ep(ep); + struct dw_pcie_ep_func *ep_func; + struct device *dev = pci->dev; + struct pci_epc *epc = ep->epc; unsigned int offset, ptm_cap_base; unsigned int nbars; u8 hdr_type; + u8 func_no; + int i, ret; + void *addr; u32 reg; - int i; hdr_type = dw_pcie_readb_dbi(pci, PCI_HEADER_TYPE) & PCI_HEADER_TYPE_MASK; @@ -680,6 +685,55 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) return -EIO; } + dw_pcie_version_detect(pci); + + dw_pcie_iatu_detect(pci); + + ret = dw_pcie_edma_detect(pci); + if (ret) + return ret; + + if (!ep->ib_window_map) { + ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, + GFP_KERNEL); + if (!ep->ib_window_map) + goto err_remove_edma; + } + + if (!ep->ob_window_map) { + ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, + GFP_KERNEL); + if (!ep->ob_window_map) + goto err_remove_edma; + } + + if (!ep->outbound_addr) { + addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), + GFP_KERNEL); + if (!addr) + goto err_remove_edma; + ep->outbound_addr = addr; + } + + for (func_no = 0; func_no < epc->max_functions; func_no++) { + + ep_func = dw_pcie_ep_get_func_from_ep(ep, func_no); + if (ep_func) + continue; + + ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); + if (!ep_func) + goto err_remove_edma; + + ep_func->func_no = func_no; + ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSI); + ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSIX); + + list_add_tail(&ep_func->list, &ep->func_list); + } + offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); ptm_cap_base = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_PTM); @@ -714,14 +768,38 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) dw_pcie_dbi_ro_wr_dis(pci); return 0; + +err_remove_edma: + dw_pcie_edma_remove(pci); + + return ret; +} + +int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) +{ + struct pci_epc *epc = ep->epc; + int ret; + + ret = dw_pcie_ep_late_init(ep); + if (ret) + goto err_cleanup; + + return 0; + +err_cleanup: + pci_epc_mem_free_addr(epc, ep->msi_mem_phys, ep->msi_mem, + epc->mem->window.page_size); + pci_epc_mem_exit(epc); + if (ep->ops->deinit) + ep->ops->deinit(ep); + + return ret; } EXPORT_SYMBOL_GPL(dw_pcie_ep_init_complete); int dw_pcie_ep_init(struct dw_pcie_ep *ep) { int ret; - void *addr; - u8 func_no; struct resource *res; struct pci_epc *epc; struct dw_pcie *pci = to_dw_pcie_from_ep(ep); @@ -729,7 +807,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) struct platform_device *pdev = to_platform_device(dev); struct device_node *np = dev->of_node; const struct pci_epc_features *epc_features; - struct dw_pcie_ep_func *ep_func; INIT_LIST_HEAD(&ep->func_list); @@ -747,26 +824,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) if (ep->ops->pre_init) ep->ops->pre_init(ep); - dw_pcie_version_detect(pci); - - dw_pcie_iatu_detect(pci); - - ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, - GFP_KERNEL); - if (!ep->ib_window_map) - return -ENOMEM; - - ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, - GFP_KERNEL); - if (!ep->ob_window_map) - return -ENOMEM; - - addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), - GFP_KERNEL); - if (!addr) - return -ENOMEM; - ep->outbound_addr = addr; - epc = devm_pci_epc_create(dev, &epc_ops); if (IS_ERR(epc)) { dev_err(dev, "Failed to create epc device\n"); @@ -780,20 +837,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) if (ret < 0) epc->max_functions = 1; - for (func_no = 0; func_no < epc->max_functions; func_no++) { - ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); - if (!ep_func) - return -ENOMEM; - - ep_func->func_no = func_no; - ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSI); - ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSIX); - - list_add_tail(&ep_func->list, &ep->func_list); - } - if (ep->ops->ep_init) ep->ops->ep_init(ep); @@ -812,25 +855,25 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) goto err_exit_epc_mem; } - ret = dw_pcie_edma_detect(pci); - if (ret) - goto err_free_epc_mem; - if (ep->ops->get_features) { epc_features = ep->ops->get_features(ep); if (epc_features->core_init_notifier) return 0; } - ret = dw_pcie_ep_init_complete(ep); + /* + * NOTE:- Avoid accessing the hardware (Ex:- DBI space) before this + * step as platforms that implement 'core_init_notifier' feature may + * not have the hardware ready (i.e. core initialized) for access + * (Ex: tegra194). Any hardware access on such platforms result + * in system hang. + */ + ret = dw_pcie_ep_late_init(ep); if (ret) - goto err_remove_edma; + goto err_free_epc_mem; return 0; -err_remove_edma: - dw_pcie_edma_remove(pci); - err_free_epc_mem: pci_epc_mem_free_addr(epc, ep->msi_mem_phys, ep->msi_mem, epc->mem->window.page_size); From patchwork Mon Nov 20 08:40:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manivannan Sadhasivam X-Patchwork-Id: 13460911 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="FXqmwKK2" Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D267126 for ; Mon, 20 Nov 2023 00:40:47 -0800 (PST) Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-5a9bc2ec556so2318166a12.0 for ; Mon, 20 Nov 2023 00:40:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1700469646; x=1701074446; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cA4et5DEe58CvaUPaUoVn2JhyhKCxIJLdzNc+n4DX2I=; b=FXqmwKK2EVv6oFkDad7iQc9A3S53jV1tk7MOfvqm2FUJ+pncZHWTXUPhM6875sYCwT mYMiq4uLb2fqOTVfqlLPFTw3oJXCIj7yrz6Ss1eQP06zzwIS/xTWNonZGUFh508bYnIF cOYxYIm5nh953R7UEfqUsXOxKqeWTKuTUgSnszeI1eYbRYwIHlzaFAO4Z5tGS1ffUrQ7 pOOx7dvJ7fATTAb7gawGXNZ5Koq2lns7rKHwAN5M39USxiGEyIq4CeognX1oHdSgZs/T voy4OXWXw7b0PBE57Tv7kk+3TuH7Ni+OrAfv+XsSOEgcP3nR99rhY67CqVAq9CI/jq+U 5HYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700469646; x=1701074446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cA4et5DEe58CvaUPaUoVn2JhyhKCxIJLdzNc+n4DX2I=; b=A/JY582SPzdgN8Y4DER2HIdwyD2eQiZIZs0TpygA9l8iQZQDrFWAUKULKGbGJPH0Yw HWSPOite6piBiQPKDiYSKyAWD88621KI9Ph4EeVNgWyID993H6e8bHmp2El7FN3CNyGJ Oc89FXy+lLubkDiYY0NViw7OnMEp1tk65N52vFkS2uslg9tiNWqGFftCn/cNfd+1yy6v ROtk7zqH51ux/6SlB20K0mYwMiIDMrIZN15EAkVHezlO6jsgygo4tBJ3dV9DH/skest3 8bo+IiYNKv4VxmAsQ43yMw3m+3SkNsI35y03N5zIOMcz/ZCbJJlrDg+cFX0Ue9r9KGxg FSzw== X-Gm-Message-State: AOJu0Yxd4mO9HehLc6KY57db4fx1sg3vwn6RYiY3pE1KZLaOccJy40UR 0KupAoiJVJ9wPEM+uyRR1xFR X-Google-Smtp-Source: AGHT+IElVewiVppHRvMO+lPS3lN61OStbKOmnNSX89BRCubPBY4/WTTj8C1eQ+g8SFtw8K29OgPTfQ== X-Received: by 2002:a05:6a21:9209:b0:174:7f7:d049 with SMTP id tl9-20020a056a21920900b0017407f7d049mr5987907pzb.9.1700469646494; Mon, 20 Nov 2023 00:40:46 -0800 (PST) Received: from localhost.localdomain ([117.248.2.5]) by smtp.gmail.com with ESMTPSA id e24-20020a62ee18000000b00694ebe2b0d4sm5567019pfi.191.2023.11.20.00.40.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 00:40:43 -0800 (PST) From: Manivannan Sadhasivam To: jingoohan1@gmail.com, gustavo.pimentel@synopsys.com, lpieralisi@kernel.org, robh@kernel.org, kw@linux.com, bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, quic_bjorande@quicinc.com, fancer.lancer@gmail.com, vidyas@nvidia.com, Manivannan Sadhasivam Subject: [PATCH v7 2/2] PCI: designware-ep: Move pci_epc_init_notify() inside dw_pcie_ep_init_complete() Date: Mon, 20 Nov 2023 14:10:14 +0530 Message-Id: <20231120084014.108274-3-manivannan.sadhasivam@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231120084014.108274-1-manivannan.sadhasivam@linaro.org> References: <20231120084014.108274-1-manivannan.sadhasivam@linaro.org> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since pci_epc_init_notify() API is getting called right after the dw_pcie_ep_init_complete() API in the DWC glue drivers, let's move it inside dw_pcie_ep_init_complete() API as there is no compelling reason to call it separately. Co-developed-by: Vidya Sagar Signed-off-by: Vidya Sagar Signed-off-by: Manivannan Sadhasivam --- drivers/pci/controller/dwc/pcie-designware-ep.c | 10 ++-------- drivers/pci/controller/dwc/pcie-designware.h | 5 ----- drivers/pci/controller/dwc/pcie-qcom-ep.c | 2 -- drivers/pci/controller/dwc/pcie-tegra194.c | 2 -- 4 files changed, 2 insertions(+), 17 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index b1c79cd8e25f..63bb99d1c48f 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -22,14 +22,6 @@ void dw_pcie_ep_linkup(struct dw_pcie_ep *ep) } EXPORT_SYMBOL_GPL(dw_pcie_ep_linkup); -void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) -{ - struct pci_epc *epc = ep->epc; - - pci_epc_init_notify(epc); -} -EXPORT_SYMBOL_GPL(dw_pcie_ep_init_notify); - struct dw_pcie_ep_func * dw_pcie_ep_get_func_from_ep(struct dw_pcie_ep *ep, u8 func_no) { @@ -784,6 +776,8 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) if (ret) goto err_cleanup; + pci_epc_init_notify(epc); + return 0; err_cleanup: diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 55ff76e3d384..53bf38989eea 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -578,7 +578,6 @@ static inline void __iomem *dw_pcie_own_conf_map_bus(struct pci_bus *bus, void dw_pcie_ep_linkup(struct dw_pcie_ep *ep); int dw_pcie_ep_init(struct dw_pcie_ep *ep); int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep); -void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep); void dw_pcie_ep_exit(struct dw_pcie_ep *ep); int dw_pcie_ep_raise_legacy_irq(struct dw_pcie_ep *ep, u8 func_no); int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no, @@ -605,10 +604,6 @@ static inline int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) return 0; } -static inline void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) -{ -} - static inline void dw_pcie_ep_exit(struct dw_pcie_ep *ep) { } diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c index 9e58f055199a..4a8119779a29 100644 --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c @@ -482,8 +482,6 @@ static int qcom_pcie_perst_deassert(struct dw_pcie *pci) val &= ~PARF_MSTR_AXI_CLK_EN; writel_relaxed(val, pcie_ep->parf + PARF_MHI_CLOCK_RESET_CTRL); - dw_pcie_ep_init_notify(&pcie_ep->pci.ep); - /* Enable LTSSM */ val = readl_relaxed(pcie_ep->parf + PARF_LTSSM); val |= BIT(8); diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c index 0fe113598ebb..1126d1f5830c 100644 --- a/drivers/pci/controller/dwc/pcie-tegra194.c +++ b/drivers/pci/controller/dwc/pcie-tegra194.c @@ -1901,8 +1901,6 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie) goto fail_init_complete; } - dw_pcie_ep_init_notify(ep); - /* Program the private control to allow sending LTR upstream */ if (pcie->of_data->has_ltr_req_fix) { val = appl_readl(pcie, APPL_LTR_MSG_2);