| Message ID | 20220710175214.544748-1-r.stratiienko@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RFC] ccu-sun50i-h6: Bump-up DDR0 PLL to 1800MHz | expand |
Hi Roman, Dne nedelja, 10. julij 2022 ob 19:52:14 CEST je Roman Stratiienko napisal(a): > While debugging complex animated composition cases I noticed a glitch > which as it turned-out eventually was caused by lack of memory bandwidth. > > I can't find a DRAMC manual to check what input frequency must be > supplied to avoid exceeding the 800MHz LPDDR3 which is installed on > my orangepi3 board. But the system is running stable so far. this is hard topic. Generally, max. supported DRAM frequency is board specific and is set by SPL (part of U-Boot), so we can't assume every board supports max. frequency for its memory type. In fact, this assumption was proven wrong. Some boards were unstable until DRAM frequency was lowered. So in short, Linux clock driver certainly shouldn't set DRAM frequency. There is, however, Linux MBUS driver for H5 and A64, which can throttle DRAM frequency when system is idle, but max. frequency is still that one which was programmed by SPL. There is no definitive DRAM documentation, because Allwinner usually tries to hide all details. DRAM drivers are, except in rare cases, all reverse engineered by community. Register values (mostly) match those produced by vendor driver. Fortunately, at least H6 DRAM controller matches pretty good with one other memory controller (one in Zynq UltraScale+ MPSoC), which is fully documented (see comments in U-Boot driver). MBUS is also pretty standard and it's not a mystery. Note that it's hard to convince maintainers to either lower or increase DRAM frequency as it was common issue in the past. You have to do a lot of memory intensive tests, both CPU and GPU, running at the same time for extended period of time. Initial board bring ups usually just use same frequency as in BSP based images. If I understand procedure correctly, boards are sent to Allwinner, so their engineers determine max. DRAM frequency and timings, so these values are pretty trustwothy. There is other possibility. Along with DRAM controller and PHY, there is MBUS core, which is memory arbiter and responsible for bandwidth allocation. You can change DE priority and bandwith there (again, in U-Boot DRAM driver). I hope this explains why things are the way they are. Best regards, Jernej > > Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com> > --- > drivers/clk/sunxi-ng/ccu-sun50i-h6.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) > > diff --git a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c index 750368a86b8b6..abdde80307993 > 100644 > --- a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > +++ b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > @@ -1203,6 +1203,17 @@ static int sun50i_h6_ccu_probe(struct platform_device > *pdev) val &= ~GENMASK(3, 0); > writel(val, reg + gpu_clk.common.reg); > > + /* > + * Increase DDR0 PLL from 1488(default) to 1800MHz. > + * (DE3.0 require higher memory bandwidth while displaying > + * complex composition at 1920x1080@60FPS) > + */ > + > + val = readl(reg + SUN50I_H6_PLL_DDR0_REG); > + val &= ~GENMASK(15, 0); > + val |= 74 << 8; > + writel(val, reg + SUN50I_H6_PLL_DDR0_REG); > + > /* Enable the lock bits on all PLLs */ > for (i = 0; i < ARRAY_SIZE(pll_regs); i++) { > val = readl(reg + pll_regs[i]); > -- > 2.34.1
On Sun, 10 Jul 2022 20:47:55 +0200 Jernej Škrabec <jernej.skrabec@gmail.com> wrote: Hi Roman, > Dne nedelja, 10. julij 2022 ob 19:52:14 CEST je Roman Stratiienko napisal(a): > > While debugging complex animated composition cases I noticed a glitch > > which as it turned-out eventually was caused by lack of memory bandwidth. > > > > I can't find a DRAMC manual to check what input frequency must be > > supplied to avoid exceeding the 800MHz LPDDR3 which is installed on > > my orangepi3 board. But the system is running stable so far. Well, most recent Allwinner boards indeed have capable LPDDR3 (800 MHz) or DDR3-1600 (or even higher) chips, but we never got them running at the full clock speed, and lowering the DRAM frequency is a common kludge to improve system stability (as in: actually reaching the login prompt). And we figured that those "more aggressive" timings are quite board specific, so while 800 MHz might work on your particular board, your friend's OPi3 might not be as lucky. > this is hard topic. Generally, max. supported DRAM frequency is board specific > and is set by SPL (part of U-Boot), so we can't assume every board supports > max. frequency for its memory type. Yes, while we run most H6 boards at 744 MHz, the Tanix TX6 requires no more than 648 MHz, for instance, partly because it has DDR3 (not LPDDR3) DRAM. Another H6 TV box I have needs an even lower frequency (at least with the current timing settings). > In fact, this assumption was proven wrong. > Some boards were unstable until DRAM frequency was lowered. So in short, Linux > clock driver certainly shouldn't set DRAM frequency. Yes, and on top of that, also keep in mind that many DRAM controller registers take timing parameters in terms of clock cycles, but the actual DDR chips are specified in terms of pico- or nanoseconds. This means the register values must be calculated and programmed based on the clock frequency, see [1] for examples in all those ns_to_t() calls. And just increasing the PLL frequency might now lead to exceeded specifications, and thus subtle instabilities. [1] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/mach-sunxi/dram_timings/h6_ddr3_1333.c > There is, however, Linux > MBUS driver for H5 and A64, which can throttle DRAM frequency when system is > idle, but max. frequency is still that one which was programmed by SPL. > > There is no definitive DRAM documentation, because Allwinner usually tries to > hide all details. DRAM drivers are, except in rare cases, all reverse > engineered by community. Register values (mostly) match those produced by > vendor driver. Fortunately, at least H6 DRAM controller matches pretty good > with one other memory controller (one in Zynq UltraScale+ MPSoC), which is > fully documented (see comments in U-Boot driver). MBUS is also pretty standard > and it's not a mystery. > > Note that it's hard to convince maintainers to either lower or increase DRAM > frequency as it was common issue in the past. You have to do a lot of memory > intensive tests, both CPU and GPU, running at the same time for extended > period of time. Initial board bring ups usually just use same frequency as in > BSP based images. Yes, this DDR3 controller logic is mostly undocumented rocket science, see [2] in our wiki for just *some* (old!) pieces of information and the complexity it involved in the past to come to a set of timing parameters (including the clock frequency) that works stably across all boards. [2] https://linux-sunxi.org/A10_DRAM_Controller_Calibration > If I understand procedure correctly, boards are sent to > Allwinner, so their engineers determine max. DRAM frequency and timings, so > these values are pretty trustwothy. > There is other possibility. Along with DRAM controller and PHY, there is MBUS > core, which is memory arbiter and responsible for bandwidth allocation. You > can change DE priority and bandwith there (again, in U-Boot DRAM driver). That's indeed an interesting topic to explore, though the DE seems to be already in the top tier in our settings: https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/mach-sunxi/dram_sun50i_h6.c#L135-151 Cheers, Andre > I hope this explains why things are the way they are. > > Best regards, > Jernej > > > > > Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com> > > --- > > drivers/clk/sunxi-ng/ccu-sun50i-h6.c | 11 +++++++++++ > > 1 file changed, 11 insertions(+) > > > > diff --git a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > > b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c index 750368a86b8b6..abdde80307993 > > 100644 > > --- a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > > +++ b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c > > @@ -1203,6 +1203,17 @@ static int sun50i_h6_ccu_probe(struct platform_device > > *pdev) val &= ~GENMASK(3, 0); > > writel(val, reg + gpu_clk.common.reg); > > > > + /* > > + * Increase DDR0 PLL from 1488(default) to 1800MHz. > > + * (DE3.0 require higher memory bandwidth while displaying > > + * complex composition at 1920x1080@60FPS) > > + */ > > + > > + val = readl(reg + SUN50I_H6_PLL_DDR0_REG); > > + val &= ~GENMASK(15, 0); > > + val |= 74 << 8; > > + writel(val, reg + SUN50I_H6_PLL_DDR0_REG); > > + > > /* Enable the lock bits on all PLLs */ > > for (i = 0; i < ARRAY_SIZE(pll_regs); i++) { > > val = readl(reg + pll_regs[i]); > > -- > > 2.34.1 > > >
diff --git a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c index 750368a86b8b6..abdde80307993 100644 --- a/drivers/clk/sunxi-ng/ccu-sun50i-h6.c +++ b/drivers/clk/sunxi-ng/ccu-sun50i-h6.c @@ -1203,6 +1203,17 @@ static int sun50i_h6_ccu_probe(struct platform_device *pdev) val &= ~GENMASK(3, 0); writel(val, reg + gpu_clk.common.reg); + /* + * Increase DDR0 PLL from 1488(default) to 1800MHz. + * (DE3.0 require higher memory bandwidth while displaying + * complex composition at 1920x1080@60FPS) + */ + + val = readl(reg + SUN50I_H6_PLL_DDR0_REG); + val &= ~GENMASK(15, 0); + val |= 74 << 8; + writel(val, reg + SUN50I_H6_PLL_DDR0_REG); + /* Enable the lock bits on all PLLs */ for (i = 0; i < ARRAY_SIZE(pll_regs); i++) { val = readl(reg + pll_regs[i]);
While debugging complex animated composition cases I noticed a glitch which as it turned-out eventually was caused by lack of memory bandwidth. I can't find a DRAMC manual to check what input frequency must be supplied to avoid exceeding the 800MHz LPDDR3 which is installed on my orangepi3 board. But the system is running stable so far. Signed-off-by: Roman Stratiienko <r.stratiienko@gmail.com> --- drivers/clk/sunxi-ng/ccu-sun50i-h6.c | 11 +++++++++++ 1 file changed, 11 insertions(+)