| Message ID | aca00bd672ee576ad96d279414fc0835ff31f637.1720022580.git.lorenzo@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Add Airoha EN7581 PCIe support | expand |
On Wed, 2024-07-03 at 18:12 +0200, Lorenzo Bianconi wrote: > > External email : Please do not click links or open attachments until > you have verified the sender or the content. > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > PCIe controller driver. > > Reviewed-by: AngeloGioacchino Del Regno < > angelogioacchino.delregno@collabora.com> > Tested-by: Zhengping Zhang <zhengping.zhang@airoha.com> > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> Acked-by: Jianjun Wang <jianjun.wang@mediatek.com>
[+cc Jim, Krishna, Vidya, Shashank] On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > PCIe controller driver. > +++ b/drivers/pci/controller/pcie-mediatek-gen3.c > +#define PCIE_EQ_PRESET_01_REG 0x100 > +#define PCIE_VAL_LN0_DOWNSTREAM GENMASK(6, 0) > +#define PCIE_VAL_LN0_UPSTREAM GENMASK(14, 8) > +#define PCIE_VAL_LN1_DOWNSTREAM GENMASK(22, 16) > +#define PCIE_VAL_LN1_UPSTREAM GENMASK(30, 24) > ... > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > +{ > ... > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); This looks like it might be for the Lane Equalization Control registers (PCIe r6.0, sec 7.7.3.4)? I would expect those values (0x47, 0x41) to be related to the platform design, so maybe not completely determined by the SoC itself? Jim and Krishna have been working on DT schema for the equalization values, which seems like the right place for them: https://lore.kernel.org/linux-pci/20241018182247.41130-2-james.quinlan@broadcom.com/ https://lore.kernel.org/r/77d3a1a9-c22d-0fd3-5942-91b9a3d74a43@quicinc.com Maybe that would be applicable here as well? It would at least be nice to use a common #define for the Lane Equalization Control register offset from the capability base. Although I see that no such #define exists in pci_regs.h, so I guess there's nothing to do here yet. The only users of equalization settings I could find so far are: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-tegra194.c?id=v6.11#n832 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-qcom-common.c?id=v6.12-rc1#n11 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-mediatek-gen3.c?id=v6.12-rc1#n909 Bjorn
On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > PCIe controller driver. > ... > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > +{ > + struct device *dev = pcie->dev; > + int err; > + u32 val; > + > + /* > + * Wait for the time needed to complete the bulk assert in > + * mtk_pcie_setup for EN7581 SoC. > + */ > + mdelay(PCIE_EN7581_RESET_TIME_MS); It looks wrong to me to do the assert and deassert in different places: mtk_pcie_setup reset_control_bulk_assert(pcie->phy_resets) <-- mtk_pcie_en7581_power_up mdelay(PCIE_EN7581_RESET_TIME_MS) reset_control_bulk_deassert(pcie->phy_resets) <-- mdelay(PCIE_EN7581_RESET_TIME_MS) That makes the code hard to understand. > + err = phy_init(pcie->phy); > + if (err) { > + dev_err(dev, "failed to initialize PHY\n"); > + return err; > + } > + > + err = phy_power_on(pcie->phy); > + if (err) { > + dev_err(dev, "failed to power on PHY\n"); > + goto err_phy_on; > + } > + > + err = reset_control_bulk_deassert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); > + if (err) { > + dev_err(dev, "failed to deassert PHYs\n"); > + goto err_phy_deassert; > + } > + > + /* > + * Wait for the time needed to complete the bulk de-assert above. > + * This time is specific for EN7581 SoC. > + */ > + mdelay(PCIE_EN7581_RESET_TIME_MS); > + > + pm_runtime_enable(dev); > + pm_runtime_get_sync(dev); > + > + err = clk_bulk_prepare(pcie->num_clks, pcie->clks); > + if (err) { > + dev_err(dev, "failed to prepare clock\n"); > + goto err_clk_prepare; > + } > + > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); > + > + val = PCIE_K_PHYPARAM_QUERY | PCIE_K_QUERY_TIMEOUT | > + FIELD_PREP(PCIE_K_PRESET_TO_USE_16G, 0x80) | > + FIELD_PREP(PCIE_K_PRESET_TO_USE, 0x2) | > + FIELD_PREP(PCIE_K_FINETUNE_MAX, 0xf); > + writel_relaxed(val, pcie->base + PCIE_PIPE4_PIE8_REG); Why is this equalization stuff in the middle between clk_bulk_prepare() and clk_bulk_enable()? Is the split an actual requirement, or could we use clk_bulk_prepare_enable() here, like we do in mtk_pcie_power_up()? If the split is required, a comment about why would be helpful. > + err = clk_bulk_enable(pcie->num_clks, pcie->clks); > + if (err) { > + dev_err(dev, "failed to prepare clock\n"); > + goto err_clk_enable; > + } Per https://lore.kernel.org/r/ZypgYOn7dcYIoW4i@lore-desk, REG_PCI_CONTROL is asserted/deasserted here by en7581_pci_enable(). Is this where PERST# is asserted? If so, a comment to that effect would be helpful. Where is PERST# deasserted? Where are the required delays before deassert done? Bjorn
> On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > PCIe controller driver. > > ... > > > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > > +{ > > + struct device *dev = pcie->dev; > > + int err; > > + u32 val; > > + > > + /* > > + * Wait for the time needed to complete the bulk assert in > > + * mtk_pcie_setup for EN7581 SoC. > > + */ > > + mdelay(PCIE_EN7581_RESET_TIME_MS); Hi Bjorn, > > It looks wrong to me to do the assert and deassert in different > places: > > mtk_pcie_setup > reset_control_bulk_assert(pcie->phy_resets) <-- > mtk_pcie_en7581_power_up > mdelay(PCIE_EN7581_RESET_TIME_MS) > reset_control_bulk_deassert(pcie->phy_resets) <-- > mdelay(PCIE_EN7581_RESET_TIME_MS) > > That makes the code hard to understand. The phy reset line was already asserted running reset_control_assert() in mtk_pcie_setup() and de-asserted running reset_control_deassert() in mtk_pcie_power_up() before adding EN7581 support. Moreover, EN7581 requires to run phy_init()/phy_power_on() before de-asserting the phy reset lines. I guess I can add a comment to make it more clear. Agree? > > > + err = phy_init(pcie->phy); > > + if (err) { > > + dev_err(dev, "failed to initialize PHY\n"); > > + return err; > > + } > > + > > + err = phy_power_on(pcie->phy); > > + if (err) { > > + dev_err(dev, "failed to power on PHY\n"); > > + goto err_phy_on; > > + } > > + > > + err = reset_control_bulk_deassert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); > > + if (err) { > > + dev_err(dev, "failed to deassert PHYs\n"); > > + goto err_phy_deassert; > > + } > > + > > + /* > > + * Wait for the time needed to complete the bulk de-assert above. > > + * This time is specific for EN7581 SoC. > > + */ > > + mdelay(PCIE_EN7581_RESET_TIME_MS); > > + > > + pm_runtime_enable(dev); > > + pm_runtime_get_sync(dev); > > + > > > + err = clk_bulk_prepare(pcie->num_clks, pcie->clks); > > + if (err) { > > + dev_err(dev, "failed to prepare clock\n"); > > + goto err_clk_prepare; > > + } > > + > > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); > > + > > + val = PCIE_K_PHYPARAM_QUERY | PCIE_K_QUERY_TIMEOUT | > > + FIELD_PREP(PCIE_K_PRESET_TO_USE_16G, 0x80) | > > + FIELD_PREP(PCIE_K_PRESET_TO_USE, 0x2) | > > + FIELD_PREP(PCIE_K_FINETUNE_MAX, 0xf); > > + writel_relaxed(val, pcie->base + PCIE_PIPE4_PIE8_REG); > > Why is this equalization stuff in the middle between > clk_bulk_prepare() and clk_bulk_enable()? Is the split an actual > requirement, or could we use clk_bulk_prepare_enable() here, like we > do in mtk_pcie_power_up()? Nope, we can replace clk_bulk_enable() with clk_bulk_prepare_enable() and remove clk_bulk_prepare() in mtk_pcie_en7581_power_up() since we actually implements just enable callback for EN7581 in clk-en7523.c. > > If the split is required, a comment about why would be helpful. > > > + err = clk_bulk_enable(pcie->num_clks, pcie->clks); > > + if (err) { > > + dev_err(dev, "failed to prepare clock\n"); > > + goto err_clk_enable; > > + } > > Per https://lore.kernel.org/r/ZypgYOn7dcYIoW4i@lore-desk, > REG_PCI_CONTROL is asserted/deasserted here by en7581_pci_enable(). correct > > Is this where PERST# is asserted? If so, a comment to that effect > would be helpful. Where is PERST# deasserted? Where are the required > delays before deassert done? I can add a comment in en7581_pci_enable() describing the PERST issue for EN7581. Please note we have a 250ms delay in en7581_pci_enable() after configuring REG_PCI_CONTROL register. https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 Regards, Lorenzo > > Bjorn
On Tue, Nov 5, 2024 at 4:33 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > [+cc Jim, Krishna, Vidya, Shashank] > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > PCIe controller driver. > > > +++ b/drivers/pci/controller/pcie-mediatek-gen3.c > > > +#define PCIE_EQ_PRESET_01_REG 0x100 > > +#define PCIE_VAL_LN0_DOWNSTREAM GENMASK(6, 0) > > +#define PCIE_VAL_LN0_UPSTREAM GENMASK(14, 8) > > +#define PCIE_VAL_LN1_DOWNSTREAM GENMASK(22, 16) > > +#define PCIE_VAL_LN1_UPSTREAM GENMASK(30, 24) > > ... > > > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > > +{ > > ... > > > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); Not sure it is worth the trouble to define fields. In fact, you are already combining fields (rec+trans) so why not go further and just write each lane as a u16? > > This looks like it might be for the Lane Equalization Control > registers (PCIe r6.0, sec 7.7.3.4)? > > I would expect those values (0x47, 0x41) to be related to the platform > design, so maybe not completely determined by the SoC itself? Jim and > Krishna have been working on DT schema for the equalization values, > which seems like the right place for them: > > https://lore.kernel.org/linux-pci/20241018182247.41130-2-james.quinlan@broadcom.com/ > https://lore.kernel.org/r/77d3a1a9-c22d-0fd3-5942-91b9a3d74a43@quicinc.com > > Maybe that would be applicable here as well? It would at least be > nice to use a common #define for the Lane Equalization Control > register offset from the capability base. FWIW, these registers are HwInit/RO. In our (Broadcom) case we have to write them using an internal register block that is not visible in the config space. In other words, we do not use the cap offset. Regards, Jim Broadcom STB/CM > > Although I see that no such #define exists in pci_regs.h, so I guess > there's nothing to do here yet. > > The only users of equalization settings I could find so far are: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-tegra194.c?id=v6.11#n832 > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-qcom-common.c?id=v6.12-rc1#n11 > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-mediatek-gen3.c?id=v6.12-rc1#n909 > > Bjorn >
On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > PCIe controller driver. > > > ... > > > > > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > > > +{ > > > + struct device *dev = pcie->dev; > > > + int err; > > > + u32 val; > > > + > > > + /* > > > + * Wait for the time needed to complete the bulk assert in > > > + * mtk_pcie_setup for EN7581 SoC. > > > + */ > > > + mdelay(PCIE_EN7581_RESET_TIME_MS); > > > It looks wrong to me to do the assert and deassert in different > > places: > > > > mtk_pcie_setup > > reset_control_bulk_assert(pcie->phy_resets) <-- > > mtk_pcie_en7581_power_up > > mdelay(PCIE_EN7581_RESET_TIME_MS) > > reset_control_bulk_deassert(pcie->phy_resets) <-- > > mdelay(PCIE_EN7581_RESET_TIME_MS) > > > > That makes the code hard to understand. > > The phy reset line was already asserted running reset_control_assert() in > mtk_pcie_setup() and de-asserted running reset_control_deassert() in > mtk_pcie_power_up() before adding EN7581 support. Moreover, EN7581 requires > to run phy_init()/phy_power_on() before de-asserting the phy reset lines. > I guess I can add a comment to make it more clear. Agree? I assume the first deassert(phy_resets) in mtk_pcie_setup() is not paired with anything in this driver. I think it would be better to pair the other assert/deasserts in the same functions like the below. Then it's easy to see the matching. While looking at this, I noticed that we assert(mac_reset) in mtk_pcie_setup(), but it's never deasserted for EN7581. mtk_pcie_setup reset_control_bulk_deassert(phy_resets) mtk_pcie_en7581_power_up reset_control_bulk_assert(phy_resets) # move here reset_control_assert(mac_reset) # move here mdelay(PCIE_EN7581_RESET_TIME_MS) phy_init phy_power_on reset_control_deassert(mac_reset) # add; seems missing? reset_control_bulk_deassert(phy_resets) mdelay(PCIE_EN7581_RESET_TIME_MS) mtk_pcie_setup reset_control_bulk_deassert(phy_resets) mtk_pcie_power_up reset_control_bulk_assert(phy_resets) # move here reset_control_assert(mac_reset) # move here reset_control_bulk_deassert(phy_resets) phy_init phy_power_on reset_control_deassert(mac_reset) > > > + err = phy_init(pcie->phy); > > > + if (err) { > > > + dev_err(dev, "failed to initialize PHY\n"); > > > + return err; > > > + } > > > + > > > + err = phy_power_on(pcie->phy); > > > + if (err) { > > > + dev_err(dev, "failed to power on PHY\n"); > > > + goto err_phy_on; > > > + } > > > + > > > + err = reset_control_bulk_deassert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); > > > + if (err) { > > > + dev_err(dev, "failed to deassert PHYs\n"); > > > + goto err_phy_deassert; > > > + } > > > + > > > + /* > > > + * Wait for the time needed to complete the bulk de-assert above. > > > + * This time is specific for EN7581 SoC. > > > + */ > > > + mdelay(PCIE_EN7581_RESET_TIME_MS); > > > + > > > + pm_runtime_enable(dev); > > > + pm_runtime_get_sync(dev); > > > + > > > > > + err = clk_bulk_prepare(pcie->num_clks, pcie->clks); > > > + if (err) { > > > + dev_err(dev, "failed to prepare clock\n"); > > > + goto err_clk_prepare; > > > + } > > > + > > > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > > > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > > > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > > > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > > > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); > > > + > > > + val = PCIE_K_PHYPARAM_QUERY | PCIE_K_QUERY_TIMEOUT | > > > + FIELD_PREP(PCIE_K_PRESET_TO_USE_16G, 0x80) | > > > + FIELD_PREP(PCIE_K_PRESET_TO_USE, 0x2) | > > > + FIELD_PREP(PCIE_K_FINETUNE_MAX, 0xf); > > > + writel_relaxed(val, pcie->base + PCIE_PIPE4_PIE8_REG); > > > > Why is this equalization stuff in the middle between > > clk_bulk_prepare() and clk_bulk_enable()? Is the split an actual > > requirement, or could we use clk_bulk_prepare_enable() here, like we > > do in mtk_pcie_power_up()? > > Nope, we can replace clk_bulk_enable() with clk_bulk_prepare_enable() and > remove clk_bulk_prepare() in mtk_pcie_en7581_power_up() since we actually > implements just enable callback for EN7581 in clk-en7523.c. > > > If the split is required, a comment about why would be helpful. > > > > > + err = clk_bulk_enable(pcie->num_clks, pcie->clks); > > > + if (err) { > > > + dev_err(dev, "failed to prepare clock\n"); > > > + goto err_clk_enable; > > > + } > > > > Per https://lore.kernel.org/r/ZypgYOn7dcYIoW4i@lore-desk, > > REG_PCI_CONTROL is asserted/deasserted here by en7581_pci_enable(). > > correct > > > Is this where PERST# is asserted? If so, a comment to that effect > > would be helpful. Where is PERST# deasserted? Where are the required > > delays before deassert done? > > I can add a comment in en7581_pci_enable() describing the PERST issue for > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > configuring REG_PCI_CONTROL register. > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 Does that 250ms delay correspond to a PCIe mandatory delay, e.g., something like PCIE_T_PVPERL_MS? I think it would be nice to have the required PCI delays in this driver if possible so it's easy to verify that they are all covered. Bjorn
On Wed, Nov 06, 2024 at 06:00:08PM -0500, Jim Quinlan wrote: > On Tue, Nov 5, 2024 at 4:33 PM Bjorn Helgaas <helgaas@kernel.org> wrote: > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > PCIe controller driver. > > > > > +++ b/drivers/pci/controller/pcie-mediatek-gen3.c > > > > > +#define PCIE_EQ_PRESET_01_REG 0x100 > > > +#define PCIE_VAL_LN0_DOWNSTREAM GENMASK(6, 0) > > > +#define PCIE_VAL_LN0_UPSTREAM GENMASK(14, 8) > > > +#define PCIE_VAL_LN1_DOWNSTREAM GENMASK(22, 16) > > > +#define PCIE_VAL_LN1_UPSTREAM GENMASK(30, 24) > > > ... > > > > > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > > > +{ > > > ... > > > > > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > > > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > > > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > > > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > > > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); > > Not sure it is worth the trouble to define fields. In fact, you are > already combining fields (rec+trans) so why not go further and just > write each lane as a u16? > > > > This looks like it might be for the Lane Equalization Control > > registers (PCIe r6.0, sec 7.7.3.4)? > > > > I would expect those values (0x47, 0x41) to be related to the platform > > design, so maybe not completely determined by the SoC itself? Jim and > > Krishna have been working on DT schema for the equalization values, > > which seems like the right place for them: > > > > https://lore.kernel.org/linux-pci/20241018182247.41130-2-james.quinlan@broadcom.com/ > > https://lore.kernel.org/r/77d3a1a9-c22d-0fd3-5942-91b9a3d74a43@quicinc.com > > > > Maybe that would be applicable here as well? It would at least be > > nice to use a common #define for the Lane Equalization Control > > register offset from the capability base. > > FWIW, these registers are HwInit/RO. In our (Broadcom) case we have > to write them using an internal register block that is not visible in > the config space. In other words, we do not use the cap offset. Good point. It looks like they're a mix of HwInit/RsvdP and Hwinit/RO. RsvdP is for writes, so I guess the config space registers must be write-once and subsequently read-only until reset. In any case, mtk is using an internal register block as well, so a cap offset wouldn't be useful. Maybe it would still be worthwhile to define the fields themselves in pci_regs.h so we can someday have common code to parse the DT properties and assemble them. Although I suppose there's no requirement that the registers in the internal block even be laid out the same as the config space register. > > Although I see that no such #define exists in pci_regs.h, so I guess > > there's nothing to do here yet. > > > > The only users of equalization settings I could find so far are: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-tegra194.c?id=v6.11#n832 > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pcie-qcom-common.c?id=v6.12-rc1#n11 > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-mediatek-gen3.c?id=v6.12-rc1#n909 > > > > Bjorn > >
> On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > > PCIe controller driver. > > > > ... > > > > > > > +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) > > > > +{ > > > > + struct device *dev = pcie->dev; > > > > + int err; > > > > + u32 val; > > > > + > > > > + /* > > > > + * Wait for the time needed to complete the bulk assert in > > > > + * mtk_pcie_setup for EN7581 SoC. > > > > + */ > > > > + mdelay(PCIE_EN7581_RESET_TIME_MS); > > > > > It looks wrong to me to do the assert and deassert in different > > > places: > > > > > > mtk_pcie_setup > > > reset_control_bulk_assert(pcie->phy_resets) <-- > > > mtk_pcie_en7581_power_up > > > mdelay(PCIE_EN7581_RESET_TIME_MS) > > > reset_control_bulk_deassert(pcie->phy_resets) <-- > > > mdelay(PCIE_EN7581_RESET_TIME_MS) > > > > > > That makes the code hard to understand. > > > > The phy reset line was already asserted running reset_control_assert() in > > mtk_pcie_setup() and de-asserted running reset_control_deassert() in > > mtk_pcie_power_up() before adding EN7581 support. Moreover, EN7581 requires > > to run phy_init()/phy_power_on() before de-asserting the phy reset lines. > > I guess I can add a comment to make it more clear. Agree? > > I assume the first deassert(phy_resets) in mtk_pcie_setup() is not > paired with anything in this driver. correct > > I think it would be better to pair the other assert/deasserts in the > same functions like the below. Then it's easy to see the matching. ack, I will post a fix for it > > While looking at this, I noticed that we assert(mac_reset) in > mtk_pcie_setup(), but it's never deasserted for EN7581. ack, I will post a fix for it > > mtk_pcie_setup > reset_control_bulk_deassert(phy_resets) > mtk_pcie_en7581_power_up > reset_control_bulk_assert(phy_resets) # move here > reset_control_assert(mac_reset) # move here > mdelay(PCIE_EN7581_RESET_TIME_MS) > phy_init > phy_power_on > reset_control_deassert(mac_reset) # add; seems missing? > reset_control_bulk_deassert(phy_resets) > mdelay(PCIE_EN7581_RESET_TIME_MS) > > mtk_pcie_setup > reset_control_bulk_deassert(phy_resets) > mtk_pcie_power_up > reset_control_bulk_assert(phy_resets) # move here > reset_control_assert(mac_reset) # move here > reset_control_bulk_deassert(phy_resets) > phy_init > phy_power_on > reset_control_deassert(mac_reset) > > > > > + err = phy_init(pcie->phy); > > > > + if (err) { > > > > + dev_err(dev, "failed to initialize PHY\n"); > > > > + return err; > > > > + } > > > > + > > > > + err = phy_power_on(pcie->phy); > > > > + if (err) { > > > > + dev_err(dev, "failed to power on PHY\n"); > > > > + goto err_phy_on; > > > > + } > > > > + > > > > + err = reset_control_bulk_deassert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); > > > > + if (err) { > > > > + dev_err(dev, "failed to deassert PHYs\n"); > > > > + goto err_phy_deassert; > > > > + } > > > > + > > > > + /* > > > > + * Wait for the time needed to complete the bulk de-assert above. > > > > + * This time is specific for EN7581 SoC. > > > > + */ > > > > + mdelay(PCIE_EN7581_RESET_TIME_MS); > > > > + > > > > + pm_runtime_enable(dev); > > > > + pm_runtime_get_sync(dev); > > > > + > > > > > > > + err = clk_bulk_prepare(pcie->num_clks, pcie->clks); > > > > + if (err) { > > > > + dev_err(dev, "failed to prepare clock\n"); > > > > + goto err_clk_prepare; > > > > + } > > > > + > > > > + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | > > > > + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | > > > > + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | > > > > + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); > > > > + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); > > > > + > > > > + val = PCIE_K_PHYPARAM_QUERY | PCIE_K_QUERY_TIMEOUT | > > > > + FIELD_PREP(PCIE_K_PRESET_TO_USE_16G, 0x80) | > > > > + FIELD_PREP(PCIE_K_PRESET_TO_USE, 0x2) | > > > > + FIELD_PREP(PCIE_K_FINETUNE_MAX, 0xf); > > > > + writel_relaxed(val, pcie->base + PCIE_PIPE4_PIE8_REG); > > > > > > Why is this equalization stuff in the middle between > > > clk_bulk_prepare() and clk_bulk_enable()? Is the split an actual > > > requirement, or could we use clk_bulk_prepare_enable() here, like we > > > do in mtk_pcie_power_up()? > > > > Nope, we can replace clk_bulk_enable() with clk_bulk_prepare_enable() and > > remove clk_bulk_prepare() in mtk_pcie_en7581_power_up() since we actually > > implements just enable callback for EN7581 in clk-en7523.c. > > > > > If the split is required, a comment about why would be helpful. > > > > > > > + err = clk_bulk_enable(pcie->num_clks, pcie->clks); > > > > + if (err) { > > > > + dev_err(dev, "failed to prepare clock\n"); > > > > + goto err_clk_enable; > > > > + } > > > > > > Per https://lore.kernel.org/r/ZypgYOn7dcYIoW4i@lore-desk, > > > REG_PCI_CONTROL is asserted/deasserted here by en7581_pci_enable(). > > > > correct > > > > > Is this where PERST# is asserted? If so, a comment to that effect > > > would be helpful. Where is PERST# deasserted? Where are the required > > > delays before deassert done? > > > > I can add a comment in en7581_pci_enable() describing the PERST issue for > > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > > configuring REG_PCI_CONTROL register. > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 > > Does that 250ms delay correspond to a PCIe mandatory delay, e.g., > something like PCIE_T_PVPERL_MS? I think it would be nice to have the > required PCI delays in this driver if possible so it's easy to verify > that they are all covered. IIRC I just used the delay value used in the vendor sdk. I do not have a strong opinion about it but I guess if we move it in the pcie-mediatek-gen3 driver, we will need to add it in each driver where this clock is used. What do you think? Regards, Lorenzo > > Bjorn
On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > > > PCIe controller driver. > > > > > ... > > > > Is this where PERST# is asserted? If so, a comment to that effect > > > > would be helpful. Where is PERST# deasserted? Where are the required > > > > delays before deassert done? > > > > > > I can add a comment in en7581_pci_enable() describing the PERST issue for > > > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > > > configuring REG_PCI_CONTROL register. > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 > > > > Does that 250ms delay correspond to a PCIe mandatory delay, e.g., > > something like PCIE_T_PVPERL_MS? I think it would be nice to have the > > required PCI delays in this driver if possible so it's easy to verify > > that they are all covered. > > IIRC I just used the delay value used in the vendor sdk. I do not > have a strong opinion about it but I guess if we move it in the > pcie-mediatek-gen3 driver, we will need to add it in each driver > where this clock is used. What do you think? I don't know what the 250ms delay is for. If it is for a required PCI delay, we should use the relevant standard #define for it, and it should be in the PCI controller driver. Otherwise it's impossible to verify that all the drivers are doing the correct delays. I don't know what other drivers are using that clock. Are you suggesting that it may be used in non-PCI situations where the required delay might be different? If another user requires 250ms, but PCI requires only 100ms, I think it would be worth having separate delays in each user so PCI wouldn't have to pay that extra 150ms. Bjorn
On Nov 07, Bjorn Helgaas wrote: > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > > > > PCIe controller driver. > > > > > > ... > > > > > > Is this where PERST# is asserted? If so, a comment to that effect > > > > > would be helpful. Where is PERST# deasserted? Where are the required > > > > > delays before deassert done? > > > > > > > > I can add a comment in en7581_pci_enable() describing the PERST issue for > > > > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > > > > configuring REG_PCI_CONTROL register. > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, e.g., > > > something like PCIE_T_PVPERL_MS? I think it would be nice to have the > > > required PCI delays in this driver if possible so it's easy to verify > > > that they are all covered. > > > > IIRC I just used the delay value used in the vendor sdk. I do not > > have a strong opinion about it but I guess if we move it in the > > pcie-mediatek-gen3 driver, we will need to add it in each driver > > where this clock is used. What do you think? > > I don't know what the 250ms delay is for. If it is for a required PCI > delay, we should use the relevant standard #define for it, and it > should be in the PCI controller driver. Otherwise it's impossible to > verify that all the drivers are doing the correct delays. ack, fine to me. Do you prefer to keep 250ms after clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? I can check if 100ms works properly. Regards, Lorenzo > > I don't know what other drivers are using that clock. Are you > suggesting that it may be used in non-PCI situations where the > required delay might be different? If another user requires 250ms, > but PCI requires only 100ms, I think it would be worth having separate > delays in each user so PCI wouldn't have to pay that extra 150ms. > > Bjorn
On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > On Nov 07, Bjorn Helgaas wrote: > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > > > > > PCIe controller driver. > > > > > > > ... > > > > > > > > Is this where PERST# is asserted? If so, a comment to that effect > > > > > > would be helpful. Where is PERST# deasserted? Where are the required > > > > > > delays before deassert done? > > > > > > > > > > I can add a comment in en7581_pci_enable() describing the PERST issue for > > > > > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > > > > > configuring REG_PCI_CONTROL register. > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, e.g., > > > > something like PCIE_T_PVPERL_MS? I think it would be nice to have the > > > > required PCI delays in this driver if possible so it's easy to verify > > > > that they are all covered. > > > > > > IIRC I just used the delay value used in the vendor sdk. I do not > > > have a strong opinion about it but I guess if we move it in the > > > pcie-mediatek-gen3 driver, we will need to add it in each driver > > > where this clock is used. What do you think? > > > > I don't know what the 250ms delay is for. If it is for a required PCI > > delay, we should use the relevant standard #define for it, and it > > should be in the PCI controller driver. Otherwise it's impossible to > > verify that all the drivers are doing the correct delays. > > ack, fine to me. Do you prefer to keep 250ms after clk_bulk_prepare_enable() > in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > I can check if 100ms works properly. It's not clear to me where the relevant events are for these chips. Do you have access to the PCIe CEM spec? The diagram in r6.0, sec 2.2.1, is helpful. It shows the required timings for Power Stable, REFCLK Stable, PERST# deassert, etc. Per sec 2.11.2, PERST# must be asserted for at least 100us (T_PERST), PERST# must be asserted for at least 100ms after Power Stable (T_PVPERL), and PERST# must be asserted for at least 100us after REFCLK Stable. It would be helpful if we could tell by reading the source where some of these critical events happen, and that the relevant delays are there. For example, if PERST# is asserted/deasserted by "clk_enable()" or similar, it's not at all obvious from the code, so we should have a comment to that effect. Bjorn
> On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > > On Nov 07, Bjorn Helgaas wrote: > > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > > Introduce support for Airoha EN7581 PCIe controller to mediatek-gen3 > > > > > > > > PCIe controller driver. > > > > > > > > ... > > > > > > > > > > Is this where PERST# is asserted? If so, a comment to that effect > > > > > > > would be helpful. Where is PERST# deasserted? Where are the required > > > > > > > delays before deassert done? > > > > > > > > > > > > I can add a comment in en7581_pci_enable() describing the PERST issue for > > > > > > EN7581. Please note we have a 250ms delay in en7581_pci_enable() after > > > > > > configuring REG_PCI_CONTROL register. > > > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/clk-en7523.c#L396 > > > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, e.g., > > > > > something like PCIE_T_PVPERL_MS? I think it would be nice to have the > > > > > required PCI delays in this driver if possible so it's easy to verify > > > > > that they are all covered. > > > > > > > > IIRC I just used the delay value used in the vendor sdk. I do not > > > > have a strong opinion about it but I guess if we move it in the > > > > pcie-mediatek-gen3 driver, we will need to add it in each driver > > > > where this clock is used. What do you think? > > > > > > I don't know what the 250ms delay is for. If it is for a required PCI > > > delay, we should use the relevant standard #define for it, and it > > > should be in the PCI controller driver. Otherwise it's impossible to > > > verify that all the drivers are doing the correct delays. > > > > ack, fine to me. Do you prefer to keep 250ms after clk_bulk_prepare_enable() > > in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > > I can check if 100ms works properly. > > It's not clear to me where the relevant events are for these chips. > > Do you have access to the PCIe CEM spec? The diagram in r6.0, sec > 2.2.1, is helpful. It shows the required timings for Power Stable, > REFCLK Stable, PERST# deassert, etc. > > Per sec 2.11.2, PERST# must be asserted for at least 100us (T_PERST), > PERST# must be asserted for at least 100ms after Power Stable > (T_PVPERL), and PERST# must be asserted for at least 100us after > REFCLK Stable. > > It would be helpful if we could tell by reading the source where some > of these critical events happen, and that the relevant delays are > there. For example, if PERST# is asserted/deasserted by > "clk_enable()" or similar, it's not at all obvious from the code, so > we should have a comment to that effect. I reviewed the vendor sdk and it just do something like in clk_enable(): ... val = readl(0x88); writel(val | BIT(16) | BIT(29) | BIT(26), 0x88); /*wait link up*/ mdelay(1000); ... @Hui.Ma: is it fine use msleep(100) (so PCIE_T_PVPERL_MS) instead of msleep(1000) (so PCIE_LINK_RETRAIN_TIMEOUT_MS)? Regards, Lorenzo > > Bjorn
> On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > > On Nov 07, Bjorn Helgaas wrote: > > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > > Introduce support for Airoha EN7581 PCIe controller to > > > > > > > > mediatek-gen3 PCIe controller driver. > > > > > > > > ... > > > > > > > > > > Is this where PERST# is asserted? If so, a comment to > > > > > > > that effect would be helpful. Where is PERST# deasserted? > > > > > > > Where are the required delays before deassert done? > > > > > > > > > > > > I can add a comment in en7581_pci_enable() describing the > > > > > > PERST issue for EN7581. Please note we have a 250ms delay in > > > > > > en7581_pci_enable() after configuring REG_PCI_CONTROL register. > > > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/cl > > > > > > k-en7523.c#L396 > > > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, > > > > > e.g., something like PCIE_T_PVPERL_MS? I think it would be > > > > > nice to have the required PCI delays in this driver if > > > > > possible so it's easy to verify that they are all covered. > > > > > > > > IIRC I just used the delay value used in the vendor sdk. I do > > > > not have a strong opinion about it but I guess if we move it in > > > > the > > > > pcie-mediatek-gen3 driver, we will need to add it in each driver > > > > where this clock is used. What do you think? > > > > > > I don't know what the 250ms delay is for. If it is for a required > > > PCI delay, we should use the relevant standard #define for it, and > > > it should be in the PCI controller driver. Otherwise it's > > > impossible to verify that all the drivers are doing the correct delays. > > > > ack, fine to me. Do you prefer to keep 250ms after > > clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > > I can check if 100ms works properly. > > It's not clear to me where the relevant events are for these chips. > > Do you have access to the PCIe CEM spec? The diagram in r6.0, sec > 2.2.1, is helpful. It shows the required timings for Power Stable, > REFCLK Stable, PERST# deassert, etc. > > Per sec 2.11.2, PERST# must be asserted for at least 100us (T_PERST), > PERST# must be asserted for at least 100ms after Power Stable > (T_PVPERL), and PERST# must be asserted for at least 100us after > REFCLK Stable. > > It would be helpful if we could tell by reading the source where some > of these critical events happen, and that the relevant delays are > there. For example, if PERST# is asserted/deasserted by > "clk_enable()" or similar, it's not at all obvious from the code, so > we should have a comment to that effect. >I reviewed the vendor sdk and it just do something like in clk_enable(): > > ... > val = readl(0x88); > writel(val | BIT(16) | BIT(29) | BIT(26), 0x88); > /*wait link up*/ > mdelay(1000); > ... > >@Hui.Ma: is it fine use msleep(100) (so PCIE_T_PVPERL_MS) instead of msleep(1000) (so PCIE_LINK_RETRAIN_TIMEOUT_MS)? Hi Lorenzo， I think msleep(1000) will be safer,because some device won't link up with msleep(100). Regards, Hui > >Regards, >Lorenzo > > Bjorn
On Fri, Nov 08, 2024 at 01:23:35AM +0000, Hui Ma (马慧) wrote: > > On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > > > On Nov 07, Bjorn Helgaas wrote: > > > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > > > Introduce support for Airoha EN7581 PCIe controller to > > > > > > > > > mediatek-gen3 PCIe controller driver. > > > > > > > > > ... > > > > > > > > > > > > Is this where PERST# is asserted? If so, a comment to > > > > > > > > that effect would be helpful. Where is PERST# deasserted? > > > > > > > > Where are the required delays before deassert done? > > > > > > > > > > > > > > I can add a comment in en7581_pci_enable() describing the > > > > > > > PERST issue for EN7581. Please note we have a 250ms delay in > > > > > > > en7581_pci_enable() after configuring REG_PCI_CONTROL register. > > > > > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/cl > > > > > > > k-en7523.c#L396 > > > > > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, > > > > > > e.g., something like PCIE_T_PVPERL_MS? I think it would be > > > > > > nice to have the required PCI delays in this driver if > > > > > > possible so it's easy to verify that they are all covered. > > > > > > > > > > IIRC I just used the delay value used in the vendor sdk. I > > > > > do not have a strong opinion about it but I guess if we move > > > > > it in the pcie-mediatek-gen3 driver, we will need to add it > > > > > in each driver where this clock is used. What do you think? > > > > > > > > I don't know what the 250ms delay is for. If it is for a required > > > > PCI delay, we should use the relevant standard #define for it, and > > > > it should be in the PCI controller driver. Otherwise it's > > > > impossible to verify that all the drivers are doing the correct delays. > > > > > > ack, fine to me. Do you prefer to keep 250ms after > > > clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > > > I can check if 100ms works properly. > > > > It's not clear to me where the relevant events are for these chips. > > > > Do you have access to the PCIe CEM spec? The diagram in r6.0, sec > > 2.2.1, is helpful. It shows the required timings for Power Stable, > > REFCLK Stable, PERST# deassert, etc. > > > > Per sec 2.11.2, PERST# must be asserted for at least 100us (T_PERST), > > PERST# must be asserted for at least 100ms after Power Stable > > (T_PVPERL), and PERST# must be asserted for at least 100us after > > REFCLK Stable. > > > > It would be helpful if we could tell by reading the source where some > > of these critical events happen, and that the relevant delays are > > there. For example, if PERST# is asserted/deasserted by > > "clk_enable()" or similar, it's not at all obvious from the code, so > > we should have a comment to that effect. > > >I reviewed the vendor sdk and it just do something like in clk_enable(): > > > > ... > > val = readl(0x88); > > writel(val | BIT(16) | BIT(29) | BIT(26), 0x88); > > /*wait link up*/ > > mdelay(1000); > > ... > > > >@Hui.Ma: is it fine use msleep(100) (so PCIE_T_PVPERL_MS) instead > >of msleep(1000) (so PCIE_LINK_RETRAIN_TIMEOUT_MS)? > > I think msleep(1000) will be safer, because some device won't > link up with msleep(100). Do you have details about this? I guess it only hurts mediatek, but increasing the minimum time to bring up a PCI hierarchy by almost an entire second is a pretty big deal. If this delay corresponds to the required T_PVPERL delay and 100ms isn't enough for some endpoints, those endpoints should fail with many host controllers, not just mediatek, so I would suspect the mediatek controller or a certain platform, not the endpoint itself. If this corresponds to T_PVPERL and mediatek needs longer, I would document that by using "PCIE_T_PVPERL_MS * 10" and adding a comment about why (affected platform/device, hardware erratum, etc). Bottom line, I don't really care what the value is, but I *would* like to be able to read pcie-mediatek-gen3.c and see the point where PCI power is stable, a delay of at least T_PVPERL, and where PERST# is deasserted because that's the main timing requirement on software. Bjorn
> On Fri, Nov 08, 2024 at 01:23:35AM +0000, Hui Ma (马慧) wrote: > > > On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > > > > On Nov 07, Bjorn Helgaas wrote: > > > > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > > > > Introduce support for Airoha EN7581 PCIe controller to > > > > > > > > > > mediatek-gen3 PCIe controller driver. > > > > > > > > > > ... > > > > > > > > > > > > > > Is this where PERST# is asserted? If so, a comment to > > > > > > > > > that effect would be helpful. Where is PERST# deasserted? > > > > > > > > > Where are the required delays before deassert done? > > > > > > > > > > > > > > > > I can add a comment in en7581_pci_enable() describing the > > > > > > > > PERST issue for EN7581. Please note we have a 250ms delay in > > > > > > > > en7581_pci_enable() after configuring REG_PCI_CONTROL register. > > > > > > > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/clk/cl > > > > > > > > k-en7523.c#L396 > > > > > > > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory delay, > > > > > > > e.g., something like PCIE_T_PVPERL_MS? I think it would be > > > > > > > nice to have the required PCI delays in this driver if > > > > > > > possible so it's easy to verify that they are all covered. > > > > > > > > > > > > IIRC I just used the delay value used in the vendor sdk. I > > > > > > do not have a strong opinion about it but I guess if we move > > > > > > it in the pcie-mediatek-gen3 driver, we will need to add it > > > > > > in each driver where this clock is used. What do you think? > > > > > > > > > > I don't know what the 250ms delay is for. If it is for a required > > > > > PCI delay, we should use the relevant standard #define for it, and > > > > > it should be in the PCI controller driver. Otherwise it's > > > > > impossible to verify that all the drivers are doing the correct delays. > > > > > > > > ack, fine to me. Do you prefer to keep 250ms after > > > > clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > > > > I can check if 100ms works properly. > > > > > > It's not clear to me where the relevant events are for these chips. > > > > > > Do you have access to the PCIe CEM spec? The diagram in r6.0, sec > > > 2.2.1, is helpful. It shows the required timings for Power Stable, > > > REFCLK Stable, PERST# deassert, etc. > > > > > > Per sec 2.11.2, PERST# must be asserted for at least 100us (T_PERST), > > > PERST# must be asserted for at least 100ms after Power Stable > > > (T_PVPERL), and PERST# must be asserted for at least 100us after > > > REFCLK Stable. > > > > > > It would be helpful if we could tell by reading the source where some > > > of these critical events happen, and that the relevant delays are > > > there. For example, if PERST# is asserted/deasserted by > > > "clk_enable()" or similar, it's not at all obvious from the code, so > > > we should have a comment to that effect. > > > > >I reviewed the vendor sdk and it just do something like in clk_enable(): > > > > > > ... > > > val = readl(0x88); > > > writel(val | BIT(16) | BIT(29) | BIT(26), 0x88); > > > /*wait link up*/ > > > mdelay(1000); > > > ... > > > > > >@Hui.Ma: is it fine use msleep(100) (so PCIE_T_PVPERL_MS) instead > > >of msleep(1000) (so PCIE_LINK_RETRAIN_TIMEOUT_MS)? > > > > I think msleep(1000) will be safer, because some device won't > > link up with msleep(100). > > Do you have details about this? I guess it only hurts mediatek, but > increasing the minimum time to bring up a PCI hierarchy by almost an > entire second is a pretty big deal. > > If this delay corresponds to the required T_PVPERL delay and 100ms > isn't enough for some endpoints, those endpoints should fail with many > host controllers, not just mediatek, so I would suspect the mediatek > controller or a certain platform, not the endpoint itself. > > If this corresponds to T_PVPERL and mediatek needs longer, I would > document that by using "PCIE_T_PVPERL_MS * 10" and adding a comment > about why (affected platform/device, hardware erratum, etc). > > Bottom line, I don't really care what the value is, but I *would* like > to be able to read pcie-mediatek-gen3.c and see the point where PCI > power is stable, a delay of at least T_PVPERL, and where PERST# is > deasserted because that's the main timing requirement on software. I run some testes using 100ms delay (PCIE_T_PVPERL_MS) after clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() and it works fine for me (I tested with a MT7915 WiFi PCIe nic connected to the PCIe sock). Moreover, we already poll PCIE_LINK_STATUS_REG register to check the link status in mtk_pcie_startup_port(), right? I guess we can proceed with 100ms delay in mtk_pcie_en7581_power_up(). Regards, Lorenzo > > Bjorn
> On Fri, Nov 08, 2024 at 01:23:35AM +0000, Hui Ma (马慧) wrote: > > > On Thu, Nov 07, 2024 at 05:21:45PM +0100, Lorenzo Bianconi wrote: > > > > On Nov 07, Bjorn Helgaas wrote: > > > > > On Thu, Nov 07, 2024 at 08:39:43AM +0100, Lorenzo Bianconi wrote: > > > > > > > On Wed, Nov 06, 2024 at 11:40:28PM +0100, Lorenzo Bianconi wrote: > > > > > > > > > On Wed, Jul 03, 2024 at 06:12:44PM +0200, Lorenzo Bianconi wrote: > > > > > > > > > > Introduce support for Airoha EN7581 PCIe controller > > > > > > > > > > to > > > > > > > > > > mediatek-gen3 PCIe controller driver. > > > > > > > > > > ... > > > > > > > > > > > > > > Is this where PERST# is asserted? If so, a comment to > > > > > > > > > that effect would be helpful. Where is PERST# deasserted? > > > > > > > > > Where are the required delays before deassert done? > > > > > > > > > > > > > > > > I can add a comment in en7581_pci_enable() describing > > > > > > > > the PERST issue for EN7581. Please note we have a 250ms > > > > > > > > delay in > > > > > > > > en7581_pci_enable() after configuring REG_PCI_CONTROL register. > > > > > > > > > > > > > > > > https://github.com/torvalds/linux/blob/master/drivers/cl > > > > > > > > k/cl > > > > > > > > k-en7523.c#L396 > > > > > > > > > > > > > > Does that 250ms delay correspond to a PCIe mandatory > > > > > > > delay, e.g., something like PCIE_T_PVPERL_MS? I think it > > > > > > > would be nice to have the required PCI delays in this > > > > > > > driver if possible so it's easy to verify that they are all covered. > > > > > > > > > > > > IIRC I just used the delay value used in the vendor sdk. I > > > > > > do not have a strong opinion about it but I guess if we move > > > > > > it in the pcie-mediatek-gen3 driver, we will need to add it > > > > > > in each driver where this clock is used. What do you think? > > > > > > > > > > I don't know what the 250ms delay is for. If it is for a > > > > > required PCI delay, we should use the relevant standard > > > > > #define for it, and it should be in the PCI controller driver. > > > > > Otherwise it's impossible to verify that all the drivers are doing the correct delays. > > > > > > > > ack, fine to me. Do you prefer to keep 250ms after > > > > clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() or just use PCIE_T_PVPERL_MS (100)? > > > > I can check if 100ms works properly. > > > > > > It's not clear to me where the relevant events are for these chips. > > > > > > Do you have access to the PCIe CEM spec? The diagram in r6.0, sec > > > 2.2.1, is helpful. It shows the required timings for Power > > > Stable, REFCLK Stable, PERST# deassert, etc. > > > > > > Per sec 2.11.2, PERST# must be asserted for at least 100us > > > (T_PERST), PERST# must be asserted for at least 100ms after Power > > > Stable (T_PVPERL), and PERST# must be asserted for at least 100us > > > after REFCLK Stable. > > > > > > It would be helpful if we could tell by reading the source where > > > some of these critical events happen, and that the relevant delays > > > are there. For example, if PERST# is asserted/deasserted by > > > "clk_enable()" or similar, it's not at all obvious from the code, > > > so we should have a comment to that effect. > > > > >I reviewed the vendor sdk and it just do something like in clk_enable(): > > > > > > ... > > > val = readl(0x88); > > > writel(val | BIT(16) | BIT(29) | BIT(26), 0x88); > > > /*wait link up*/ > > > mdelay(1000); > > > ... > > > > > >@Hui.Ma: is it fine use msleep(100) (so PCIE_T_PVPERL_MS) instead > > >of msleep(1000) (so PCIE_LINK_RETRAIN_TIMEOUT_MS)? > > > > I think msleep(1000) will be safer, because some device won't > > link up with msleep(100). >> >> Do you have details about this? I guess it only hurts mediatek, but >> increasing the minimum time to bring up a PCI hierarchy by almost an >> entire second is a pretty big deal. >> >> If this delay corresponds to the required T_PVPERL delay and 100ms >> isn't enough for some endpoints, those endpoints should fail with many >> host controllers, not just mediatek, so I would suspect the mediatek >> controller or a certain platform, not the endpoint itself. >> >> If this corresponds to T_PVPERL and mediatek needs longer, I would >> document that by using "PCIE_T_PVPERL_MS * 10" and adding a comment >> about why (affected platform/device, hardware erratum, etc). >> >> Bottom line, I don't really care what the value is, but I *would* like >> to be able to read pcie-mediatek-gen3.c and see the point where PCI >> power is stable, a delay of at least T_PVPERL, and where PERST# is >> deasserted because that's the main timing requirement on software. >>I run some testes using 100ms delay (PCIE_T_PVPERL_MS) after >>clk_bulk_prepare_enable() in mtk_pcie_en7581_power_up() and it works fine for me (I tested with a MT7915 WiFi PCIe nic connected to the PCIe sock). >>Moreover, we already poll PCIE_LINK_STATUS_REG register to check the link status in mtk_pcie_startup_port(), right? I guess we can proceed with 100ms delay in mtk_pcie_en7581_power_up(). Yes. Hi Lorenzo/Bjorn， After our internal discussion and tests, we confirmed that a 100ms delay is enough. Regards, Hui >Regards, >Lorenzo > > Bjorn
diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig index e534c02ee34f..3bd6c9430010 100644 --- a/drivers/pci/controller/Kconfig +++ b/drivers/pci/controller/Kconfig @@ -196,7 +196,7 @@ config PCIE_MEDIATEK config PCIE_MEDIATEK_GEN3 tristate "MediaTek Gen3 PCIe controller" - depends on ARCH_MEDIATEK || COMPILE_TEST + depends on ARCH_AIROHA || ARCH_MEDIATEK || COMPILE_TEST depends on PCI_MSI help Adds support for PCIe Gen3 MAC controller for MediaTek SoCs. diff --git a/drivers/pci/controller/pcie-mediatek-gen3.c b/drivers/pci/controller/pcie-mediatek-gen3.c index 438a5222d986..e064f467ced6 100644 --- a/drivers/pci/controller/pcie-mediatek-gen3.c +++ b/drivers/pci/controller/pcie-mediatek-gen3.c @@ -6,7 +6,9 @@ * Author: Jianjun Wang <jianjun.wang@mediatek.com> */ +#include <linux/bitfield.h> #include <linux/clk.h> +#include <linux/clk-provider.h> #include <linux/delay.h> #include <linux/iopoll.h> #include <linux/irq.h> @@ -15,6 +17,8 @@ #include <linux/kernel.h> #include <linux/module.h> #include <linux/msi.h> +#include <linux/of_device.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> @@ -29,6 +33,12 @@ #define PCI_CLASS(class) (class << 8) #define PCIE_RC_MODE BIT(0) +#define PCIE_EQ_PRESET_01_REG 0x100 +#define PCIE_VAL_LN0_DOWNSTREAM GENMASK(6, 0) +#define PCIE_VAL_LN0_UPSTREAM GENMASK(14, 8) +#define PCIE_VAL_LN1_DOWNSTREAM GENMASK(22, 16) +#define PCIE_VAL_LN1_UPSTREAM GENMASK(30, 24) + #define PCIE_CFGNUM_REG 0x140 #define PCIE_CFG_DEVFN(devfn) ((devfn) & GENMASK(7, 0)) #define PCIE_CFG_BUS(bus) (((bus) << 8) & GENMASK(15, 8)) @@ -68,6 +78,14 @@ #define PCIE_MSI_SET_ENABLE_REG 0x190 #define PCIE_MSI_SET_ENABLE GENMASK(PCIE_MSI_SET_NUM - 1, 0) +#define PCIE_PIPE4_PIE8_REG 0x338 +#define PCIE_K_FINETUNE_MAX GENMASK(5, 0) +#define PCIE_K_FINETUNE_ERR GENMASK(7, 6) +#define PCIE_K_PRESET_TO_USE GENMASK(18, 8) +#define PCIE_K_PHYPARAM_QUERY BIT(19) +#define PCIE_K_QUERY_TIMEOUT BIT(20) +#define PCIE_K_PRESET_TO_USE_16G GENMASK(31, 21) + #define PCIE_MSI_SET_BASE_REG 0xc00 #define PCIE_MSI_SET_OFFSET 0x10 #define PCIE_MSI_SET_STATUS_OFFSET 0x04 @@ -100,7 +118,10 @@ #define PCIE_ATR_TLP_TYPE_MEM PCIE_ATR_TLP_TYPE(0) #define PCIE_ATR_TLP_TYPE_IO PCIE_ATR_TLP_TYPE(2) -#define MAX_NUM_PHY_RESETS 1 +#define MAX_NUM_PHY_RESETS 3 + +/* Time in ms needed to complete PCIe reset on EN7581 SoC */ +#define PCIE_EN7581_RESET_TIME_MS 100 struct mtk_gen3_pcie; @@ -847,6 +868,85 @@ static int mtk_pcie_parse_port(struct mtk_gen3_pcie *pcie) return 0; } +static int mtk_pcie_en7581_power_up(struct mtk_gen3_pcie *pcie) +{ + struct device *dev = pcie->dev; + int err; + u32 val; + + /* + * Wait for the time needed to complete the bulk assert in + * mtk_pcie_setup for EN7581 SoC. + */ + mdelay(PCIE_EN7581_RESET_TIME_MS); + + err = phy_init(pcie->phy); + if (err) { + dev_err(dev, "failed to initialize PHY\n"); + return err; + } + + err = phy_power_on(pcie->phy); + if (err) { + dev_err(dev, "failed to power on PHY\n"); + goto err_phy_on; + } + + err = reset_control_bulk_deassert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); + if (err) { + dev_err(dev, "failed to deassert PHYs\n"); + goto err_phy_deassert; + } + + /* + * Wait for the time needed to complete the bulk de-assert above. + * This time is specific for EN7581 SoC. + */ + mdelay(PCIE_EN7581_RESET_TIME_MS); + + pm_runtime_enable(dev); + pm_runtime_get_sync(dev); + + err = clk_bulk_prepare(pcie->num_clks, pcie->clks); + if (err) { + dev_err(dev, "failed to prepare clock\n"); + goto err_clk_prepare; + } + + val = FIELD_PREP(PCIE_VAL_LN0_DOWNSTREAM, 0x47) | + FIELD_PREP(PCIE_VAL_LN1_DOWNSTREAM, 0x47) | + FIELD_PREP(PCIE_VAL_LN0_UPSTREAM, 0x41) | + FIELD_PREP(PCIE_VAL_LN1_UPSTREAM, 0x41); + writel_relaxed(val, pcie->base + PCIE_EQ_PRESET_01_REG); + + val = PCIE_K_PHYPARAM_QUERY | PCIE_K_QUERY_TIMEOUT | + FIELD_PREP(PCIE_K_PRESET_TO_USE_16G, 0x80) | + FIELD_PREP(PCIE_K_PRESET_TO_USE, 0x2) | + FIELD_PREP(PCIE_K_FINETUNE_MAX, 0xf); + writel_relaxed(val, pcie->base + PCIE_PIPE4_PIE8_REG); + + err = clk_bulk_enable(pcie->num_clks, pcie->clks); + if (err) { + dev_err(dev, "failed to prepare clock\n"); + goto err_clk_enable; + } + + return 0; + +err_clk_enable: + clk_bulk_unprepare(pcie->num_clks, pcie->clks); +err_clk_prepare: + pm_runtime_put_sync(dev); + pm_runtime_disable(dev); + reset_control_bulk_assert(pcie->soc->phy_resets.num_resets, pcie->phy_resets); +err_phy_deassert: + phy_power_off(pcie->phy); +err_phy_on: + phy_exit(pcie->phy); + + return err; +} + static int mtk_pcie_power_up(struct mtk_gen3_pcie *pcie) { struct device *dev = pcie->dev; @@ -1113,7 +1213,18 @@ static const struct mtk_gen3_pcie_pdata mtk_pcie_soc_mt8192 = { }, }; +static const struct mtk_gen3_pcie_pdata mtk_pcie_soc_en7581 = { + .power_up = mtk_pcie_en7581_power_up, + .phy_resets = { + .id[0] = "phy-lane0", + .id[1] = "phy-lane1", + .id[2] = "phy-lane2", + .num_resets = 3, + }, +}; + static const struct of_device_id mtk_pcie_of_match[] = { + { .compatible = "airoha,en7581-pcie", .data = &mtk_pcie_soc_en7581 }, { .compatible = "mediatek,mt8192-pcie", .data = &mtk_pcie_soc_mt8192 }, {}, };