| Message ID | 20201213225517.3838501-1-linus.walleij@linaro.org (mailing list archive) |
|---|---|
| State | Queued, archived |
| Headers | show |
| Series | ARM: dts: ux500: Reserve memory carveouts | expand |
On 13.12.20 23:55, Linus Walleij wrote: > The Ux500 platforms have some memory carveouts set aside for > communicating with the modem and for the initial secure software > (ISSW). These areas are protected by the memory controller > and will result in an external abort if accessed like common > read/write memory. > > On the legacy boot loaders, these were set aside by using > cmdline arguments such as this: > > mem=96M@0 mem_mtrace=15M@96M mem_mshared=1M@111M > mem_modem=16M@112M mali.mali_mem=32M@128M mem=96M@160M > hwmem=127M@256M mem_issw=1M@383M mem_ram_console=1M@384M > mem=638M@385M > > Reserve the relevant areas in the device tree instead. The > "mali", "hwmem", "mem_ram_console" and the trailing 1MB at the > end of the memory reservations in the list are not relevant for > the upstream kernel as these are nowadays replaced with > upstream technologies such as CMA. The modem and ISSW > reservations are necessary. > > This was manifested in a bug that surfaced in response to > commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()") > which changes the behaviour of memory allocations > in such a way that the platform will sooner run into these > dangerous areas, with "Unhandled fault: imprecise external > abort (0xc06) at 0xb6fd83dc" or similar: the real reason > turns out to be that the PTE is pointing right into one of > the reserved memory areas. We were just lucky until now. > > We need to augment the DB8500 and DB8520 SoCs similarly > and also create a new include for the DB9500 used in the > Snowball since this does not have a modem and thus does > not need the modem memory reservation, albeit it needs > the ISSW reservation. > > Cc: stable@vger.kernel.org > Cc: David Hildenbrand <david@redhat.com> > Signed-off-by: Linus Walleij <linus.walleij@linaro.org> > --- > ARM SoC folks: please apply this directly for fixes. Can we come up with a Fixes: tag or has this been broken forever? (assuming modern boot loaders) > > David: just FYI if you run into more of these type of > regressions. Actually the patch is unintentionally good > at smoking out other bugs :D Thanks for CCing - I'm adding some people that ran into similar issues, but not sure if the other bugreports are related (or have similar root causes). Thanks a lot! > --- > arch/arm/boot/dts/ste-db8500.dtsi | 38 ++++++++++++++++++++++++++++++ > arch/arm/boot/dts/ste-db8520.dtsi | 38 ++++++++++++++++++++++++++++++ > arch/arm/boot/dts/ste-db9500.dtsi | 35 +++++++++++++++++++++++++++ > arch/arm/boot/dts/ste-snowball.dts | 2 +- > 4 files changed, 112 insertions(+), 1 deletion(-) > create mode 100644 arch/arm/boot/dts/ste-db9500.dtsi > > diff --git a/arch/arm/boot/dts/ste-db8500.dtsi b/arch/arm/boot/dts/ste-db8500.dtsi > index d309fad32229..344d29853bf7 100644 > --- a/arch/arm/boot/dts/ste-db8500.dtsi > +++ b/arch/arm/boot/dts/ste-db8500.dtsi > @@ -12,4 +12,42 @@ cpu@300 { > 200000 0>; > }; > }; > + > + reserved-memory { > + #address-cells = <1>; > + #size-cells = <1>; > + ranges; > + > + /* Modem trace memory */ > + ram@06000000 { > + reg = <0x06000000 0x00f00000>; > + no-map; > + }; > + > + /* Modem shared memory */ > + ram@06f00000 { > + reg = <0x06f00000 0x00100000>; > + no-map; > + }; > + > + /* Modem private memory */ > + ram@07000000 { > + reg = <0x07000000 0x01000000>; > + no-map; > + }; > + > + /* > + * Initial Secure Software ISSW memory > + * > + * This is probably only used if the kernel tries > + * to actually call into trustzone to run secure > + * applications, which the mainline kernel probably > + * will not do on this old chipset. But you can never > + * be too careful, so reserve this memory anyway. > + */ > + ram@17f00000 { > + reg = <0x17f00000 0x00100000>; > + no-map; > + }; > + }; > }; > diff --git a/arch/arm/boot/dts/ste-db8520.dtsi b/arch/arm/boot/dts/ste-db8520.dtsi > index 48bd8728ae27..287804e9e183 100644 > --- a/arch/arm/boot/dts/ste-db8520.dtsi > +++ b/arch/arm/boot/dts/ste-db8520.dtsi > @@ -12,4 +12,42 @@ cpu@300 { > 200000 0>; > }; > }; > + > + reserved-memory { > + #address-cells = <1>; > + #size-cells = <1>; > + ranges; > + > + /* Modem trace memory */ > + ram@06000000 { > + reg = <0x06000000 0x00f00000>; > + no-map; > + }; > + > + /* Modem shared memory */ > + ram@06f00000 { > + reg = <0x06f00000 0x00100000>; > + no-map; > + }; > + > + /* Modem private memory */ > + ram@07000000 { > + reg = <0x07000000 0x01000000>; > + no-map; > + }; > + > + /* > + * Initial Secure Software ISSW memory > + * > + * This is probably only used if the kernel tries > + * to actually call into trustzone to run secure > + * applications, which the mainline kernel probably > + * will not do on this old chipset. But you can never > + * be too careful, so reserve this memory anyway. > + */ > + ram@17f00000 { > + reg = <0x17f00000 0x00100000>; > + no-map; > + }; > + }; > }; > diff --git a/arch/arm/boot/dts/ste-db9500.dtsi b/arch/arm/boot/dts/ste-db9500.dtsi > new file mode 100644 > index 000000000000..0afff703191c > --- /dev/null > +++ b/arch/arm/boot/dts/ste-db9500.dtsi > @@ -0,0 +1,35 @@ > +// SPDX-License-Identifier: GPL-2.0-or-later > + > +#include "ste-dbx5x0.dtsi" > + > +/ { > + cpus { > + cpu@300 { > + /* cpufreq controls */ > + operating-points = <1152000 0 > + 800000 0 > + 400000 0 > + 200000 0>; > + }; > + }; > + > + reserved-memory { > + #address-cells = <1>; > + #size-cells = <1>; > + ranges; > + > + /* > + * Initial Secure Software ISSW memory > + * > + * This is probably only used if the kernel tries > + * to actually call into trustzone to run secure > + * applications, which the mainline kernel probably > + * will not do on this old chipset. But you can never > + * be too careful, so reserve this memory anyway. > + */ > + ram@17f00000 { > + reg = <0x17f00000 0x00100000>; > + no-map; > + }; > + }; > +}; > diff --git a/arch/arm/boot/dts/ste-snowball.dts b/arch/arm/boot/dts/ste-snowball.dts > index be90e73c923e..27d8a07718a0 100644 > --- a/arch/arm/boot/dts/ste-snowball.dts > +++ b/arch/arm/boot/dts/ste-snowball.dts > @@ -4,7 +4,7 @@ > */ > > /dts-v1/; > -#include "ste-db8500.dtsi" > +#include "ste-db9500.dtsi" > #include "ste-href-ab8500.dtsi" > #include "ste-href-family-pinctrl.dtsi" > >
On Mon, Dec 14, 2020 at 10:21 AM David Hildenbrand <david@redhat.com> wrote: > > ARM SoC folks: please apply this directly for fixes. > > Can we come up with a Fixes: tag or has this been broken forever? > (assuming modern boot loaders) It's been broken forever :/ > > David: just FYI if you run into more of these type of > > regressions. Actually the patch is unintentionally good > > at smoking out other bugs :D > > Thanks for CCing - I'm adding some people that ran into similar issues, > but not sure if the other bugreports are related (or have similar root > causes). Yeah we first were convinced there was something wrong with the patch you made but I read it over and over again and there is nothing wrong with it at all. It just alters the behaviour pattern of memory management in some apparently drastic ways. After a lot of silent crashes I finally got an external abort with a reasonable backtrace showing the PTE pointing to this modem memory and then we figured it out. Yours, Linus Walleij
Linus Walleij <linus.walleij@linaro.org> writes: > On Mon, Dec 14, 2020 at 10:21 AM David Hildenbrand <david@redhat.com> wrote: > >> > ARM SoC folks: please apply this directly for fixes. >> >> Can we come up with a Fixes: tag or has this been broken forever? >> (assuming modern boot loaders) > > It's been broken forever :/ > >> > David: just FYI if you run into more of these type of >> > regressions. Actually the patch is unintentionally good >> > at smoking out other bugs :D >> >> Thanks for CCing - I'm adding some people that ran into similar issues, >> but not sure if the other bugreports are related (or have similar root >> causes). > > Yeah we first were convinced there was something wrong with > the patch you made but I read it over and over again and there > is nothing wrong with it at all. It just alters the behaviour pattern of > memory management in some apparently drastic ways. > > After a lot of silent crashes I finally got an external abort with > a reasonable backtrace showing the PTE pointing to this > modem memory and then we figured it out. We had similar experiences with ath11k (Wi-Fi 6) and QCA6390 firmware. So indeed commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()") is a great way to catch odd firmware or hardware problems, which most likely would have gone unnoticed otherwise and users would have end up experiencing random crashes.
From: Arnd Bergmann <arnd@arndb.de> On Sun, 13 Dec 2020 23:55:17 +0100, Linus Walleij wrote: > The Ux500 platforms have some memory carveouts set aside for > communicating with the modem and for the initial secure software > (ISSW). These areas are protected by the memory controller > and will result in an external abort if accessed like common > read/write memory. > > On the legacy boot loaders, these were set aside by using > cmdline arguments such as this: > > [...] I had mistakenly not applied this last year before the merge window, as I misread the description as being not very urgent. On second look, this seems important enough for the v5.11 fixes branch, so I applied it now. [1/1] ARM: dts: ux500: Reserve memory carveouts commit: 8a996b2d8a03beae3cb6adfc12673778c192085d Arnd
On Fri, Jan 15, 2021 at 6:22 PM Arnd Bergmann <arnd@kernel.org> wrote: > I had mistakenly not applied this last year before the merge > window, as I misread the description as being not very urgent. > > On second look, this seems important enough for the v5.11 > fixes branch, so I applied it now. I was just thinking about it while cooking dinner! - Hey what with that critical patch now again... Thanks a lot! Linus
diff --git a/arch/arm/boot/dts/ste-db8500.dtsi b/arch/arm/boot/dts/ste-db8500.dtsi index d309fad32229..344d29853bf7 100644 --- a/arch/arm/boot/dts/ste-db8500.dtsi +++ b/arch/arm/boot/dts/ste-db8500.dtsi @@ -12,4 +12,42 @@ cpu@300 { 200000 0>; }; }; + + reserved-memory { + #address-cells = <1>; + #size-cells = <1>; + ranges; + + /* Modem trace memory */ + ram@06000000 { + reg = <0x06000000 0x00f00000>; + no-map; + }; + + /* Modem shared memory */ + ram@06f00000 { + reg = <0x06f00000 0x00100000>; + no-map; + }; + + /* Modem private memory */ + ram@07000000 { + reg = <0x07000000 0x01000000>; + no-map; + }; + + /* + * Initial Secure Software ISSW memory + * + * This is probably only used if the kernel tries + * to actually call into trustzone to run secure + * applications, which the mainline kernel probably + * will not do on this old chipset. But you can never + * be too careful, so reserve this memory anyway. + */ + ram@17f00000 { + reg = <0x17f00000 0x00100000>; + no-map; + }; + }; }; diff --git a/arch/arm/boot/dts/ste-db8520.dtsi b/arch/arm/boot/dts/ste-db8520.dtsi index 48bd8728ae27..287804e9e183 100644 --- a/arch/arm/boot/dts/ste-db8520.dtsi +++ b/arch/arm/boot/dts/ste-db8520.dtsi @@ -12,4 +12,42 @@ cpu@300 { 200000 0>; }; }; + + reserved-memory { + #address-cells = <1>; + #size-cells = <1>; + ranges; + + /* Modem trace memory */ + ram@06000000 { + reg = <0x06000000 0x00f00000>; + no-map; + }; + + /* Modem shared memory */ + ram@06f00000 { + reg = <0x06f00000 0x00100000>; + no-map; + }; + + /* Modem private memory */ + ram@07000000 { + reg = <0x07000000 0x01000000>; + no-map; + }; + + /* + * Initial Secure Software ISSW memory + * + * This is probably only used if the kernel tries + * to actually call into trustzone to run secure + * applications, which the mainline kernel probably + * will not do on this old chipset. But you can never + * be too careful, so reserve this memory anyway. + */ + ram@17f00000 { + reg = <0x17f00000 0x00100000>; + no-map; + }; + }; }; diff --git a/arch/arm/boot/dts/ste-db9500.dtsi b/arch/arm/boot/dts/ste-db9500.dtsi new file mode 100644 index 000000000000..0afff703191c --- /dev/null +++ b/arch/arm/boot/dts/ste-db9500.dtsi @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#include "ste-dbx5x0.dtsi" + +/ { + cpus { + cpu@300 { + /* cpufreq controls */ + operating-points = <1152000 0 + 800000 0 + 400000 0 + 200000 0>; + }; + }; + + reserved-memory { + #address-cells = <1>; + #size-cells = <1>; + ranges; + + /* + * Initial Secure Software ISSW memory + * + * This is probably only used if the kernel tries + * to actually call into trustzone to run secure + * applications, which the mainline kernel probably + * will not do on this old chipset. But you can never + * be too careful, so reserve this memory anyway. + */ + ram@17f00000 { + reg = <0x17f00000 0x00100000>; + no-map; + }; + }; +}; diff --git a/arch/arm/boot/dts/ste-snowball.dts b/arch/arm/boot/dts/ste-snowball.dts index be90e73c923e..27d8a07718a0 100644 --- a/arch/arm/boot/dts/ste-snowball.dts +++ b/arch/arm/boot/dts/ste-snowball.dts @@ -4,7 +4,7 @@ */ /dts-v1/; -#include "ste-db8500.dtsi" +#include "ste-db9500.dtsi" #include "ste-href-ab8500.dtsi" #include "ste-href-family-pinctrl.dtsi"
The Ux500 platforms have some memory carveouts set aside for communicating with the modem and for the initial secure software (ISSW). These areas are protected by the memory controller and will result in an external abort if accessed like common read/write memory. On the legacy boot loaders, these were set aside by using cmdline arguments such as this: mem=96M@0 mem_mtrace=15M@96M mem_mshared=1M@111M mem_modem=16M@112M mali.mali_mem=32M@128M mem=96M@160M hwmem=127M@256M mem_issw=1M@383M mem_ram_console=1M@384M mem=638M@385M Reserve the relevant areas in the device tree instead. The "mali", "hwmem", "mem_ram_console" and the trailing 1MB at the end of the memory reservations in the list are not relevant for the upstream kernel as these are nowadays replaced with upstream technologies such as CMA. The modem and ISSW reservations are necessary. This was manifested in a bug that surfaced in response to commit 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()") which changes the behaviour of memory allocations in such a way that the platform will sooner run into these dangerous areas, with "Unhandled fault: imprecise external abort (0xc06) at 0xb6fd83dc" or similar: the real reason turns out to be that the PTE is pointing right into one of the reserved memory areas. We were just lucky until now. We need to augment the DB8500 and DB8520 SoCs similarly and also create a new include for the DB9500 used in the Snowball since this does not have a modem and thus does not need the modem memory reservation, albeit it needs the ISSW reservation. Cc: stable@vger.kernel.org Cc: David Hildenbrand <david@redhat.com> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> --- ARM SoC folks: please apply this directly for fixes. David: just FYI if you run into more of these type of regressions. Actually the patch is unintentionally good at smoking out other bugs :D --- arch/arm/boot/dts/ste-db8500.dtsi | 38 ++++++++++++++++++++++++++++++ arch/arm/boot/dts/ste-db8520.dtsi | 38 ++++++++++++++++++++++++++++++ arch/arm/boot/dts/ste-db9500.dtsi | 35 +++++++++++++++++++++++++++ arch/arm/boot/dts/ste-snowball.dts | 2 +- 4 files changed, 112 insertions(+), 1 deletion(-) create mode 100644 arch/arm/boot/dts/ste-db9500.dtsi