| Message ID | 20230222115020.55867-1-avromanov@sberdevices.ru (mailing list archive) |
|---|---|
| Headers | show |
| Series | Meson A1 32-bit support | expand |
On 22/02/2023 12:50, Alexey Romanov wrote: > Hello! > > This patchset adds support for 32-bit Meson A1 board. Hi Alexev, I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes of memory, but those processors are ARMv8 and the arm64 arch code has been designed for those CPUs. So far I didn't find a single good reason to add 32bit support for ARMv8 Amlogic based SoCs, if you have a solid reason please share. And as Krzysztof stated, the support is incomplete and cannot work without a dts file. Neil > We describe device tree with following components: > CPU, GIC, IRQ, Timer, UART, PIN controller. > It's capable of booting up into the serial console. > > We have tested this DTS and used drivers at our > 32-bit Meson A1 board and it works correctly. > > Alexey Romanov (3): > meson: pinctrl: use CONFIG_PINCTRL_A1 with CONFIG_ARM > firmware: meson: use CONFIG_MESON_SM with CONFIG_ARM > arch/arm: dts: introduce meson-a1 device tree > > arch/arm/boot/dts/meson-a1.dtsi | 151 ++++++++++++++++++++++++++++++++ > drivers/firmware/meson/Kconfig | 2 +- > drivers/pinctrl/meson/Kconfig | 2 +- > 3 files changed, 153 insertions(+), 2 deletions(-) > create mode 100644 arch/arm/boot/dts/meson-a1.dtsi >
Hello Neil! On Mon, Feb 27, 2023 at 09:15:04AM +0100, neil.armstrong@linaro.org wrote: [...] > I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes > of memory, but those processors are ARMv8 and the arm64 arch code > has been designed for those CPUs. > > So far I didn't find a single good reason to add 32bit support for > ARMv8 Amlogic based SoCs, if you have a solid reason please share. I totally agree with you, but I suppose it's fully related to 'big' Amlogic SoC like S905_ or A311_ series. A113L (aka 'a1') is a cost-efficient dual-core SoC which is used for small, cheap solutions with cheap components. Every cent is important during BoM development. That's why usually ODMs install small ROM and RAM capacity, and each megabyte is important for RAM/ROM kernel and rootfs footprints. Why am I talking about rootfs? For such small projects a good choice is buildroot rootfs assembling framework. Unfortunatelly, buildroot doesn't support 'compat' mode when kernel and userspace have a different bitness. In the internal project, we save several percents of ROM/RAM free space using 32-bit configuration (mostly rootfs ROM space, to be honest). Therefore, for such 'little' cost-efficient SoCs we can make an exception and support 32-bit configuration, from my point of view. What do you think about that? > > And as Krzysztof stated, the support is incomplete and cannot work > without a dts file. Agreed, we shouldn't merge dead code. But there are several question to discuss there. Please check my reply to Krzysztof message. [...]
On 27/02/2023 15:28, Dmitry Rokosov wrote: > Hello Neil! > > On Mon, Feb 27, 2023 at 09:15:04AM +0100, neil.armstrong@linaro.org wrote: > > [...] > >> I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes >> of memory, but those processors are ARMv8 and the arm64 arch code >> has been designed for those CPUs. >> >> So far I didn't find a single good reason to add 32bit support for >> ARMv8 Amlogic based SoCs, if you have a solid reason please share. > > I totally agree with you, but I suppose it's fully related to 'big' > Amlogic SoC like S905_ or A311_ series. A113L (aka 'a1') is > a cost-efficient dual-core SoC which is used for small, cheap solutions > with cheap components. Every cent is important during BoM development. > That's why usually ODMs install small ROM and RAM capacity, and each > megabyte is important for RAM/ROM kernel and rootfs footprints. Do you have figures ? is 32bit ARM kernel really lighter when ARM64 one is correctly configured ? > Why am I talking about rootfs? For such small projects a good > choice is buildroot rootfs assembling framework. Unfortunatelly, > buildroot doesn't support 'compat' mode when kernel and userspace have > a different bitness. well this is a buildroot problem... the kernel itself is perfectly capable of running an AArch32 userspace. > In the internal project, we save several > percents of ROM/RAM free space using 32-bit configuration (mostly rootfs > ROM space, to be honest). Therefore, for such 'little' cost-efficient > SoCs we can make an exception and support 32-bit configuration, from my > point of view. 32bit ARM is now "legacy", I would need to have an advice from the ARM SoC maintainers, but AFAIK new ARMv8 SoCs should stay in arm64 arch. Arnd ? Olof ? do you have an opinion on this ? > > What do you think about that? >> >> And as Krzysztof stated, the support is incomplete and cannot work >> without a dts file. > > Agreed, we shouldn't merge dead code. But there are several question to > discuss there. Please check my reply to Krzysztof message. > > [...] >
On Mon, Feb 27, 2023, at 15:28, Dmitry Rokosov wrote: > Hello Neil! > > On Mon, Feb 27, 2023 at 09:15:04AM +0100, neil.armstrong@linaro.org wrote: > > [...] > >> I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes >> of memory, but those processors are ARMv8 and the arm64 arch code >> has been designed for those CPUs. >> >> So far I didn't find a single good reason to add 32bit support for >> ARMv8 Amlogic based SoCs, if you have a solid reason please share. > > I totally agree with you, but I suppose it's fully related to 'big' > Amlogic SoC like S905_ or A311_ series. A113L (aka 'a1') is > a cost-efficient dual-core SoC which is used for small, cheap solutions > with cheap components. Every cent is important during BoM development. > That's why usually ODMs install small ROM and RAM capacity, and each > megabyte is important for RAM/ROM kernel and rootfs footprints. > Why am I talking about rootfs? For such small projects a good > choice is buildroot rootfs assembling framework. Unfortunatelly, > buildroot doesn't support 'compat' mode when kernel and userspace have > a different bitness. In the internal project, we save several > percents of ROM/RAM free space using 32-bit configuration (mostly rootfs > ROM space, to be honest). Therefore, for such 'little' cost-efficient > SoCs we can make an exception and support 32-bit configuration, from my > point of view. > > What do you think about that? I would argue that is a problem with buildroot, and using a 32-bit kernel is not something we should encourage over fixing buildroot to do it right, or building the kernel separately from the rootfs. We do allow building support for a couple of ARMv8 SoCs in 32-bit mode, but that is usually because they ship with a 32-bit bootrom and cannot actually run a 64-bit kernel. The overhead of running a 64-bit kernel is usually a few megabytes compared to a 32-bit kernel, to store the larger kernel .text/.data segments, per-thread stack and page tables as well as 'page', and 'inode' structures. I see that A1 only supports DDR3 and DDR4 memory, so I assume that there are always at least 128MB of total RAM available, or 512MB for the most cost-effective size with a single memory chip. My feeling is that for the 256MB configuration, it is very hard to argue for a 32-bit kernel because of the countless downsides, and even for the 128MB configuration, I would still try to avoid it out of principle. Arnd
Hello Arnd, On Mon, Feb 27, 2023 at 03:58:50PM +0100, Arnd Bergmann wrote: > On Mon, Feb 27, 2023, at 15:28, Dmitry Rokosov wrote: > > Hello Neil! > > > > On Mon, Feb 27, 2023 at 09:15:04AM +0100, neil.armstrong@linaro.org wrote: > > > > [...] > > > >> I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes > >> of memory, but those processors are ARMv8 and the arm64 arch code > >> has been designed for those CPUs. > >> > >> So far I didn't find a single good reason to add 32bit support for > >> ARMv8 Amlogic based SoCs, if you have a solid reason please share. > > > > I totally agree with you, but I suppose it's fully related to 'big' > > Amlogic SoC like S905_ or A311_ series. A113L (aka 'a1') is > > a cost-efficient dual-core SoC which is used for small, cheap solutions > > with cheap components. Every cent is important during BoM development. > > That's why usually ODMs install small ROM and RAM capacity, and each > > megabyte is important for RAM/ROM kernel and rootfs footprints. > > Why am I talking about rootfs? For such small projects a good > > choice is buildroot rootfs assembling framework. Unfortunatelly, > > buildroot doesn't support 'compat' mode when kernel and userspace have > > a different bitness. In the internal project, we save several > > percents of ROM/RAM free space using 32-bit configuration (mostly rootfs > > ROM space, to be honest). Therefore, for such 'little' cost-efficient > > SoCs we can make an exception and support 32-bit configuration, from my > > point of view. > > > > What do you think about that? > > I would argue that is a problem with buildroot, and using a 32-bit > kernel is not something we should encourage over fixing buildroot > to do it right, or building the kernel separately from the rootfs. > > We do allow building support for a couple of ARMv8 SoCs in 32-bit > mode, but that is usually because they ship with a 32-bit bootrom > and cannot actually run a 64-bit kernel. To be honest, I didn't know about this principle. It looks like a very rational approach "start from max supported bitness". Based on overall maintainers opinion, we have to prepare a patch for buildroot to support compat mode :) [...]
On Mon, Feb 27, 2023 at 03:46:50PM +0100, neil.armstrong@linaro.org wrote: > On 27/02/2023 15:28, Dmitry Rokosov wrote: > > Hello Neil! > > > > On Mon, Feb 27, 2023 at 09:15:04AM +0100, neil.armstrong@linaro.org wrote: > > > > [...] > > > > > I'm aware Amlogic also runs their kernel as 32bit to gain a few kbytes > > > of memory, but those processors are ARMv8 and the arm64 arch code > > > has been designed for those CPUs. > > > > > > So far I didn't find a single good reason to add 32bit support for > > > ARMv8 Amlogic based SoCs, if you have a solid reason please share. > > > > I totally agree with you, but I suppose it's fully related to 'big' > > Amlogic SoC like S905_ or A311_ series. A113L (aka 'a1') is > > a cost-efficient dual-core SoC which is used for small, cheap solutions > > with cheap components. Every cent is important during BoM development. > > That's why usually ODMs install small ROM and RAM capacity, and each > > megabyte is important for RAM/ROM kernel and rootfs footprints. > > Do you have figures ? is 32bit ARM kernel really lighter when ARM64 one is correctly configured ? As I mentioned in the previous message, we have reached good results for userspace profile. For the kernel, we have disabled heavy cost memory features (like dynamic tracing) at both configurations, and get more or less the same results for memory footprint (pre-allocated for kernel sections). I can provide rootfs figures/results, but I'm afraid it's so workload dependent and very custom for us. > > > Why am I talking about rootfs? For such small projects a good > > choice is buildroot rootfs assembling framework. Unfortunatelly, > > buildroot doesn't support 'compat' mode when kernel and userspace have > > a different bitness. > > well this is a buildroot problem... the kernel itself is perfectly capable > of running an AArch32 userspace. Based on your and Arnd arguments, agree, this is a buildroot problem with compat mode definition. Buildroot must have it for such configurations with mainline kernel. [...]
On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: > On Mon, Feb 27, 2023 at 03:58:50PM +0100, Arnd Bergmann wrote: >> >> I would argue that is a problem with buildroot, and using a 32-bit >> kernel is not something we should encourage over fixing buildroot >> to do it right, or building the kernel separately from the rootfs. >> >> We do allow building support for a couple of ARMv8 SoCs in 32-bit >> mode, but that is usually because they ship with a 32-bit bootrom >> and cannot actually run a 64-bit kernel. > > To be honest, I didn't know about this principle. It looks like a very > rational approach "start from max supported bitness". > Based on overall maintainers opinion, we have to prepare a patch for > buildroot to support compat mode :) That would be great, thanks a lot! For what it's worth, the main arguments in favor of running a 64-bit kernel with compat user space over a 32-bit kernel are support for: - larger RAM sizes without highmem (most 32-bit kernels only support 768MB of lowmem, and highmem sucks) - larger virtual address space (4GB vs 3GB or less) - CPU specific errata workarounds (arch/arm/ only has those for 32-bit cpus) - mitigations for common attacks such as spectre - security hardening that depends on larger address space (KASLR, BTI, ptrauth, PAN, ...) - emulating instructions that were removed in Armv8 (setend, swp, ...) Most of these don't apply in userspace, so the incentive to run smaller 32-bit userland on systems with less than 1GB of RAM usually outweighs the benefits of 64-bit userspace. Arnd
On Mon, Feb 27, 2023 at 05:15:58PM +0100, Arnd Bergmann wrote: > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: > > On Mon, Feb 27, 2023 at 03:58:50PM +0100, Arnd Bergmann wrote: > >> > >> I would argue that is a problem with buildroot, and using a 32-bit > >> kernel is not something we should encourage over fixing buildroot > >> to do it right, or building the kernel separately from the rootfs. > >> > >> We do allow building support for a couple of ARMv8 SoCs in 32-bit > >> mode, but that is usually because they ship with a 32-bit bootrom > >> and cannot actually run a 64-bit kernel. > > > > To be honest, I didn't know about this principle. It looks like a very > > rational approach "start from max supported bitness". > > Based on overall maintainers opinion, we have to prepare a patch for > > buildroot to support compat mode :) > > That would be great, thanks a lot! > > For what it's worth, the main arguments in favor of running a 64-bit > kernel with compat user space over a 32-bit kernel are support for: > > - larger RAM sizes without highmem (most 32-bit kernels only > support 768MB of lowmem, and highmem sucks) > - larger virtual address space (4GB vs 3GB or less) > - CPU specific errata workarounds (arch/arm/ only has those for 32-bit cpus) > - mitigations for common attacks such as spectre > - security hardening that depends on larger address space > (KASLR, BTI, ptrauth, PAN, ...) > - emulating instructions that were removed in Armv8 (setend, swp, ...) > > Most of these don't apply in userspace, so the incentive to > run smaller 32-bit userland on systems with less than 1GB of > RAM usually outweighs the benefits of 64-bit userspace. Thank you very for the detailed clarification! It's strong arguments.
On 27/02/2023 17:15, Arnd Bergmann wrote: > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: >> On Mon, Feb 27, 2023 at 03:58:50PM +0100, Arnd Bergmann wrote: >>> >>> I would argue that is a problem with buildroot, and using a 32-bit >>> kernel is not something we should encourage over fixing buildroot >>> to do it right, or building the kernel separately from the rootfs. >>> >>> We do allow building support for a couple of ARMv8 SoCs in 32-bit >>> mode, but that is usually because they ship with a 32-bit bootrom >>> and cannot actually run a 64-bit kernel. >> >> To be honest, I didn't know about this principle. It looks like a very >> rational approach "start from max supported bitness". >> Based on overall maintainers opinion, we have to prepare a patch for >> buildroot to support compat mode :) > > That would be great, thanks a lot! > > For what it's worth, the main arguments in favor of running a 64-bit > kernel with compat user space over a 32-bit kernel are support for: > > - larger RAM sizes without highmem (most 32-bit kernels only > support 768MB of lowmem, and highmem sucks) > - larger virtual address space (4GB vs 3GB or less) > - CPU specific errata workarounds (arch/arm/ only has those for 32-bit cpus) > - mitigations for common attacks such as spectre > - security hardening that depends on larger address space > (KASLR, BTI, ptrauth, PAN, ...) > - emulating instructions that were removed in Armv8 (setend, swp, ...) > > Most of these don't apply in userspace, so the incentive to > run smaller 32-bit userland on systems with less than 1GB of > RAM usually outweighs the benefits of 64-bit userspace. Thanks for the details! Neil > > Arnd
On Mon, Feb 27, 2023 at 05:38:49PM +0100, Neil Armstrong wrote: > On 27/02/2023 17:15, Arnd Bergmann wrote: > > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: > > > On Mon, Feb 27, 2023 at 03:58:50PM +0100, Arnd Bergmann wrote: > > > > > > > > I would argue that is a problem with buildroot, and using a 32-bit > > > > kernel is not something we should encourage over fixing buildroot > > > > to do it right, or building the kernel separately from the rootfs. > > > > > > > > We do allow building support for a couple of ARMv8 SoCs in 32-bit > > > > mode, but that is usually because they ship with a 32-bit bootrom > > > > and cannot actually run a 64-bit kernel. > > > > > > To be honest, I didn't know about this principle. It looks like a very > > > rational approach "start from max supported bitness". > > > Based on overall maintainers opinion, we have to prepare a patch for > > > buildroot to support compat mode :) > > > > That would be great, thanks a lot! > > > > For what it's worth, the main arguments in favor of running a 64-bit > > kernel with compat user space over a 32-bit kernel are support for: > > > > - larger RAM sizes without highmem (most 32-bit kernels only > > support 768MB of lowmem, and highmem sucks) > > - larger virtual address space (4GB vs 3GB or less) > > - CPU specific errata workarounds (arch/arm/ only has those for 32-bit cpus) > > - mitigations for common attacks such as spectre > > - security hardening that depends on larger address space > > (KASLR, BTI, ptrauth, PAN, ...) > > - emulating instructions that were removed in Armv8 (setend, swp, ...) > > > > Most of these don't apply in userspace, so the incentive to > > run smaller 32-bit userland on systems with less than 1GB of > > RAM usually outweighs the benefits of 64-bit userspace. > > Thanks for the details! Looks like Thomas has already prepared a basic patch series for buildroot, but maintainers declined it. https://lore.kernel.org/all/20220730194331.GA2515056@scaer/
On Mon, Feb 27, 2023, at 17:50, Dmitry Rokosov wrote: > On Mon, Feb 27, 2023 at 05:38:49PM +0100, Neil Armstrong wrote: >> On 27/02/2023 17:15, Arnd Bergmann wrote: >> > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: >> > >> > Most of these don't apply in userspace, so the incentive to >> > run smaller 32-bit userland on systems with less than 1GB of >> > RAM usually outweighs the benefits of 64-bit userspace. >> >> Thanks for the details! > > Looks like Thomas has already prepared a basic patch series for buildroot, > but maintainers declined it. > > https://lore.kernel.org/all/20220730194331.GA2515056@scaer/ I see. I know very little about buildroot, but it sounds like there are other ways of doing the same thing here. In general, this is pretty much an Arm specific problem. While you clearly want compat mode for small userland on any architecture but don't want 32-bit kernels, arm is the only one that has a different kernel "ARCH=" value and needs a separate gcc toolchain. If the problem is only the toolchain, an easy way out may be to use clang instead of gcc as your compiler, as a single clang binary can target both 32-bit userland and 64-bit kernel on all supported architectures. Arnd
On Mon, Feb 27, 2023 at 07:19:38PM +0100, Arnd Bergmann wrote: > On Mon, Feb 27, 2023, at 17:50, Dmitry Rokosov wrote: > > On Mon, Feb 27, 2023 at 05:38:49PM +0100, Neil Armstrong wrote: > >> On 27/02/2023 17:15, Arnd Bergmann wrote: > >> > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: > >> > > >> > Most of these don't apply in userspace, so the incentive to > >> > run smaller 32-bit userland on systems with less than 1GB of > >> > RAM usually outweighs the benefits of 64-bit userspace. > >> > >> Thanks for the details! > > > > Looks like Thomas has already prepared a basic patch series for buildroot, > > but maintainers declined it. > > > > https://lore.kernel.org/all/20220730194331.GA2515056@scaer/ > > I see. I know very little about buildroot, but it sounds like > there are other ways of doing the same thing here. In general, > this is pretty much an Arm specific problem. While you clearly > want compat mode for small userland on any architecture but don't > want 32-bit kernels, arm is the only one that has a different > kernel "ARCH=" value and needs a separate gcc toolchain. > > If the problem is only the toolchain, an easy way out may > be to use clang instead of gcc as your compiler, as a single > clang binary can target both 32-bit userland and 64-bit kernel > on all supported architectures. Agreed with you. We will try different local approaches to support compat build configurations. For now, prebuilt toolchain (buildroot make sdk goal) is best way from my point of view. Anyway, we will try to solve this problem in the our sandbox and stay on the 64-bit kernel. Thank you for all the helpful details you shared, appreciate it!
Dmitry Rokosov <ddrokosov@sberdevices.ru> writes: > On Mon, Feb 27, 2023 at 07:19:38PM +0100, Arnd Bergmann wrote: >> On Mon, Feb 27, 2023, at 17:50, Dmitry Rokosov wrote: >> > On Mon, Feb 27, 2023 at 05:38:49PM +0100, Neil Armstrong wrote: >> >> On 27/02/2023 17:15, Arnd Bergmann wrote: >> >> > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: >> >> > >> >> > Most of these don't apply in userspace, so the incentive to >> >> > run smaller 32-bit userland on systems with less than 1GB of >> >> > RAM usually outweighs the benefits of 64-bit userspace. >> >> >> >> Thanks for the details! >> > >> > Looks like Thomas has already prepared a basic patch series for buildroot, >> > but maintainers declined it. >> > >> > https://lore.kernel.org/all/20220730194331.GA2515056@scaer/ >> >> I see. I know very little about buildroot, but it sounds like >> there are other ways of doing the same thing here. In general, >> this is pretty much an Arm specific problem. While you clearly >> want compat mode for small userland on any architecture but don't >> want 32-bit kernels, arm is the only one that has a different >> kernel "ARCH=" value and needs a separate gcc toolchain. >> >> If the problem is only the toolchain, an easy way out may >> be to use clang instead of gcc as your compiler, as a single >> clang binary can target both 32-bit userland and 64-bit kernel >> on all supported architectures. > > Agreed with you. We will try different local approaches to support > compat build configurations. For now, prebuilt toolchain (buildroot make > sdk goal) is best way from my point of view. Anyway, we will try to > solve this problem in the our sandbox and stay on the 64-bit kernel. > Thank you for all the helpful details you shared, appreciate it! Just to clarify one thing... More specifically, this is a buildroot *build system* problem. If you build the kernel separately from the rootfs, it works fine. I use 32-bit buildroot (and debian) rootfs images all the time on Amlogic SoCs with 64-bit kernels and it works fine. Kevin
Hello Kevin, On Thu, Mar 09, 2023 at 01:52:41PM -0800, Kevin Hilman wrote: > Dmitry Rokosov <ddrokosov@sberdevices.ru> writes: > > > On Mon, Feb 27, 2023 at 07:19:38PM +0100, Arnd Bergmann wrote: > >> On Mon, Feb 27, 2023, at 17:50, Dmitry Rokosov wrote: > >> > On Mon, Feb 27, 2023 at 05:38:49PM +0100, Neil Armstrong wrote: > >> >> On 27/02/2023 17:15, Arnd Bergmann wrote: > >> >> > On Mon, Feb 27, 2023, at 16:51, Dmitry Rokosov wrote: > >> >> > > >> >> > Most of these don't apply in userspace, so the incentive to > >> >> > run smaller 32-bit userland on systems with less than 1GB of > >> >> > RAM usually outweighs the benefits of 64-bit userspace. > >> >> > >> >> Thanks for the details! > >> > > >> > Looks like Thomas has already prepared a basic patch series for buildroot, > >> > but maintainers declined it. > >> > > >> > https://lore.kernel.org/all/20220730194331.GA2515056@scaer/ > >> > >> I see. I know very little about buildroot, but it sounds like > >> there are other ways of doing the same thing here. In general, > >> this is pretty much an Arm specific problem. While you clearly > >> want compat mode for small userland on any architecture but don't > >> want 32-bit kernels, arm is the only one that has a different > >> kernel "ARCH=" value and needs a separate gcc toolchain. > >> > >> If the problem is only the toolchain, an easy way out may > >> be to use clang instead of gcc as your compiler, as a single > >> clang binary can target both 32-bit userland and 64-bit kernel > >> on all supported architectures. > > > > Agreed with you. We will try different local approaches to support > > compat build configurations. For now, prebuilt toolchain (buildroot make > > sdk goal) is best way from my point of view. Anyway, we will try to > > solve this problem in the our sandbox and stay on the 64-bit kernel. > > Thank you for all the helpful details you shared, appreciate it! > > Just to clarify one thing... > > More specifically, this is a buildroot *build system* problem. If you > build the kernel separately from the rootfs, it works fine. > > I use 32-bit buildroot (and debian) rootfs images all the time on > Amlogic SoCs with 64-bit kernels and it works fine. You are totally right. It's one of the possible ways. But in the our internal project we build kernel + roofs + uboot together in the one buildroot project ('repo' based). So we will try to stay in the such paradigm, but will use multi-arch toolchain, maybe. Anyway, thanks a lot for sharing your experience.