| Message ID | 20240727074526.1771247-5-masahiroy@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | kbuild: cross-compile linux-headers package | expand |
On Sat, Jul 27, 2024 at 04:42:04PM +0900, Masahiro Yamada wrote: > There are known limitations: > > - GCC plugins > > It would possible to rebuild GCC plugins for the target architecture > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > installed, but gcc on the installed system emits > "cc1: error: incompatible gcc/plugin versions". I did not find a > solution for this because 'gcc' on a foreign architecture is a > different compiler after all. Do you mean having a plugins as part of a distro package? Does anyone do this?
On Sat, Jul 27, 2024 at 04:42:04PM +0900, Masahiro Yamada wrote: > A long standing issue in the upstream kernel packaging is that the > linux-headers package is not cross-compiled. > > For example, you can cross-build Debian packages for arm64 by running > the following command: > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > However, the generated linux-headers-*_arm64.deb is useless because the > host programs in it were built for your build machine architecture > (likely x86), not arm64. > > The Debian kernel maintains its own Makefiles to cross-compile host > tools without relying on Kbuild. [1] > > Instead of adding such full custom Makefiles, this commit adds a small > piece of code to cross-compile host programs located under the scripts/ > directory. > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > would also cross-compile scripts/basic/fixdep, which needs to be native > to process the if_changed_dep macro. (This approach may work under some > circumstances; you can execute foreign architecture programs with the > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > but it would require installing QEMU and libc for that architecture.) > > A trick is to use the external module build (KBUILD_EXTMOD=), which > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > userspace programs (CONFIG_CC_CAN_LINK=y). > > There are known limitations: > > - GCC plugins > > It would possible to rebuild GCC plugins for the target architecture > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > installed, but gcc on the installed system emits > "cc1: error: incompatible gcc/plugin versions". I did not find a > solution for this because 'gcc' on a foreign architecture is a > different compiler after all. > > - objtool and resolve_btfids > > These are built by the tools build system. They are not covered by > the current solution. > > I only tested this with Debian, but it should work for other package > systems as well. > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > 1 file changed, 34 insertions(+) > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > index cc335945dfbc..0b56d3d7b48f 100755 > --- a/scripts/package/install-extmod-build > +++ b/scripts/package/install-extmod-build > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > fi > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > +# the case for package building. It does not cross-compile when CC=clang. > +# > +# This caters to host programs that participate in Kbuild. objtool and > +# resolve_btfids are out of scope. Just for clarification: Why do you call both "out of scope" here? Because they're not being built by kbuild, or because they will never be needed for building oot kmods? > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > + echo "Rebuilding host programs with ${CC}..." > + > + cat <<-'EOF' > "${destdir}/Kbuild" > + subdir-y := scripts > + EOF > + > + # HOSTCXX is not overridden. The C++ compiler is used to build: > + # - scripts/kconfig/qconf, which is unneeded for external module builds > + # - GCC plugins, which will not work on the installed system even with > + # being rebuilt. > + # > + # Use the single-target build to avoid the modpost invocation, which > + # would overwrite Module.symvers. > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > + > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > + subdir-y := basic > + hostprogs-always-y := mod/modpost > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > + EOF > + > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > + > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > +fi > + > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete > -- > 2.43.0 > > Thanks for fixing this; looks good to me. Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
On Tue, Jul 30, 2024 at 10:03 AM Kees Cook <kees@kernel.org> wrote: > > On Sat, Jul 27, 2024 at 04:42:04PM +0900, Masahiro Yamada wrote: > > There are known limitations: > > > > - GCC plugins > > > > It would possible to rebuild GCC plugins for the target architecture > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > installed, but gcc on the installed system emits > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > solution for this because 'gcc' on a foreign architecture is a > > different compiler after all. > > Do you mean having a plugins as part of a distro package? Does anyone do > this? I think the use of GCC plugins is not so common in distributions, presumably due to its strong limitation. In my quick research, Debian, Ubuntu, Fedora disable CONFIG_GCC_PLUGINS. Arch Linux enables CONFIG_GCC_PLUGINS. > -- > Kees Cook >
On Thu, Aug 1, 2024 at 6:10 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > On Sat, Jul 27, 2024 at 04:42:04PM +0900, Masahiro Yamada wrote: > > A long standing issue in the upstream kernel packaging is that the > > linux-headers package is not cross-compiled. > > > > For example, you can cross-build Debian packages for arm64 by running > > the following command: > > > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > > > However, the generated linux-headers-*_arm64.deb is useless because the > > host programs in it were built for your build machine architecture > > (likely x86), not arm64. > > > > The Debian kernel maintains its own Makefiles to cross-compile host > > tools without relying on Kbuild. [1] > > > > Instead of adding such full custom Makefiles, this commit adds a small > > piece of code to cross-compile host programs located under the scripts/ > > directory. > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > > would also cross-compile scripts/basic/fixdep, which needs to be native > > to process the if_changed_dep macro. (This approach may work under some > > circumstances; you can execute foreign architecture programs with the > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > > but it would require installing QEMU and libc for that architecture.) > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > > userspace programs (CONFIG_CC_CAN_LINK=y). > > > > There are known limitations: > > > > - GCC plugins > > > > It would possible to rebuild GCC plugins for the target architecture > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > installed, but gcc on the installed system emits > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > solution for this because 'gcc' on a foreign architecture is a > > different compiler after all. > > > > - objtool and resolve_btfids > > > > These are built by the tools build system. They are not covered by > > the current solution. > > > > I only tested this with Debian, but it should work for other package > > systems as well. > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > --- > > > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > > 1 file changed, 34 insertions(+) > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > > index cc335945dfbc..0b56d3d7b48f 100755 > > --- a/scripts/package/install-extmod-build > > +++ b/scripts/package/install-extmod-build > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > > fi > > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > > +# the case for package building. It does not cross-compile when CC=clang. > > +# > > +# This caters to host programs that participate in Kbuild. objtool and > > +# resolve_btfids are out of scope. > > Just for clarification: Why do you call both "out of scope" here? > Because they're not being built by kbuild, or because they will never be > needed for building oot kmods? I meant the former. Debian applies a tricky patch to the tools build system in order to cross-compile objtool: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/patches/debian/fixdep-allow-overriding-hostcc-and-hostld.patch It is not an elegant solution, though. I still believe the right thing to do is converting Makefiles for objtool and resolve_btfids into Kbuild style. objtool and resolve_btfids are necessary for building external modules, when CONFIG_OBJTOOL=y and CONFIG_DEBUG_INFO_BTF=y, respectively. If these comments are confusing, I can delete them. > > > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > > + echo "Rebuilding host programs with ${CC}..." > > + > > + cat <<-'EOF' > "${destdir}/Kbuild" > > + subdir-y := scripts > > + EOF > > + > > + # HOSTCXX is not overridden. The C++ compiler is used to build: > > + # - scripts/kconfig/qconf, which is unneeded for external module builds > > + # - GCC plugins, which will not work on the installed system even with > > + # being rebuilt. > > + # > > + # Use the single-target build to avoid the modpost invocation, which > > + # would overwrite Module.symvers. > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > + > > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > > + subdir-y := basic > > + hostprogs-always-y := mod/modpost > > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > > + EOF > > + > > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > + > > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > > +fi > > + > > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete > > -- > > 2.43.0 > > > > > > Thanks for fixing this; looks good to me. > > Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> >
On Thu, Aug 01, 2024 at 11:37:30AM +0900, Masahiro Yamada wrote: > On Thu, Aug 1, 2024 at 6:10 AM Nicolas Schier <nicolas@fjasle.eu> wrote: > > > > On Sat, Jul 27, 2024 at 04:42:04PM +0900, Masahiro Yamada wrote: > > > A long standing issue in the upstream kernel packaging is that the > > > linux-headers package is not cross-compiled. > > > > > > For example, you can cross-build Debian packages for arm64 by running > > > the following command: > > > > > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > > > > > However, the generated linux-headers-*_arm64.deb is useless because the > > > host programs in it were built for your build machine architecture > > > (likely x86), not arm64. > > > > > > The Debian kernel maintains its own Makefiles to cross-compile host > > > tools without relying on Kbuild. [1] > > > > > > Instead of adding such full custom Makefiles, this commit adds a small > > > piece of code to cross-compile host programs located under the scripts/ > > > directory. > > > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > > > would also cross-compile scripts/basic/fixdep, which needs to be native > > > to process the if_changed_dep macro. (This approach may work under some > > > circumstances; you can execute foreign architecture programs with the > > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > > > but it would require installing QEMU and libc for that architecture.) > > > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which > > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > > > userspace programs (CONFIG_CC_CAN_LINK=y). > > > > > > There are known limitations: > > > > > > - GCC plugins > > > > > > It would possible to rebuild GCC plugins for the target architecture > > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > > installed, but gcc on the installed system emits > > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > > solution for this because 'gcc' on a foreign architecture is a > > > different compiler after all. > > > > > > - objtool and resolve_btfids > > > > > > These are built by the tools build system. They are not covered by > > > the current solution. > > > > > > I only tested this with Debian, but it should work for other package > > > systems as well. > > > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > > --- > > > > > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > > > 1 file changed, 34 insertions(+) > > > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > > > index cc335945dfbc..0b56d3d7b48f 100755 > > > --- a/scripts/package/install-extmod-build > > > +++ b/scripts/package/install-extmod-build > > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > > > fi > > > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > > > > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > > > +# the case for package building. It does not cross-compile when CC=clang. > > > +# > > > +# This caters to host programs that participate in Kbuild. objtool and > > > +# resolve_btfids are out of scope. > > > > Just for clarification: Why do you call both "out of scope" here? > > Because they're not being built by kbuild, or because they will never be > > needed for building oot kmods? > > > I meant the former. > > > Debian applies a tricky patch to the tools build system > in order to cross-compile objtool: > > https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/patches/debian/fixdep-allow-overriding-hostcc-and-hostld.patch > > It is not an elegant solution, though. > > > I still believe the right thing to do is > converting Makefiles for objtool and resolve_btfids into Kbuild style. > > > objtool and resolve_btfids are necessary for building external modules, > when CONFIG_OBJTOOL=y and CONFIG_DEBUG_INFO_BTF=y, respectively. > If these comments are confusing, I can delete them. I think it's good to mention that cross-built linux-headers package is still broken for CONFIG_OBJTOOL=y and CONFIG_DEBUG_INFO_BTF=y. I think I'd add a sentence to the commit message and keep the comment here as it is. Kind regards, Nicolas
On 7/27/24 12:42 AM, Masahiro Yamada wrote: > A long standing issue in the upstream kernel packaging is that the > linux-headers package is not cross-compiled. > > For example, you can cross-build Debian packages for arm64 by running > the following command: > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > However, the generated linux-headers-*_arm64.deb is useless because the > host programs in it were built for your build machine architecture > (likely x86), not arm64. > > The Debian kernel maintains its own Makefiles to cross-compile host > tools without relying on Kbuild. [1] > > Instead of adding such full custom Makefiles, this commit adds a small > piece of code to cross-compile host programs located under the scripts/ > directory. > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > would also cross-compile scripts/basic/fixdep, which needs to be native > to process the if_changed_dep macro. (This approach may work under some > circumstances; you can execute foreign architecture programs with the > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > but it would require installing QEMU and libc for that architecture.) > > A trick is to use the external module build (KBUILD_EXTMOD=), which > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > userspace programs (CONFIG_CC_CAN_LINK=y). > > There are known limitations: > > - GCC plugins > > It would possible to rebuild GCC plugins for the target architecture > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > installed, but gcc on the installed system emits > "cc1: error: incompatible gcc/plugin versions". I did not find a > solution for this because 'gcc' on a foreign architecture is a > different compiler after all. > > - objtool and resolve_btfids > > These are built by the tools build system. They are not covered by > the current solution. > > I only tested this with Debian, but it should work for other package > systems as well. > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > 1 file changed, 34 insertions(+) > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > index cc335945dfbc..0b56d3d7b48f 100755 > --- a/scripts/package/install-extmod-build > +++ b/scripts/package/install-extmod-build > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > fi > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > +# the case for package building. It does not cross-compile when CC=clang. > +# > +# This caters to host programs that participate in Kbuild. objtool and > +# resolve_btfids are out of scope. > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > + echo "Rebuilding host programs with ${CC}..." > + > + cat <<-'EOF' > "${destdir}/Kbuild" > + subdir-y := scripts > + EOF > + > + # HOSTCXX is not overridden. The C++ compiler is used to build: > + # - scripts/kconfig/qconf, which is unneeded for external module builds > + # - GCC plugins, which will not work on the installed system even with > + # being rebuilt. > + # > + # Use the single-target build to avoid the modpost invocation, which > + # would overwrite Module.symvers. > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > + > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > + subdir-y := basic > + hostprogs-always-y := mod/modpost > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > + EOF > + > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > + > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > +fi > + > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete This patch causes a build error when cross-compiling for RISC-V. I'm using the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain. When trying to build .debs with: make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1 "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg I get the following error: Rebuilding host programs with riscv64-unknown-linux-gnu-gcc... HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch] HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10: fatal error: openssl/opensslv.h: No such file or directory 25 | #include <openssl/opensslv.h> | ^~~~~~~~~~~~~~~~~~~~ compilation terminated. make[7]: *** [scripts/Makefile.host:116: debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file] Error 1 make[6]: *** [scripts/Makefile.build:478: debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts] Error 2 make[5]: *** [Makefile:1936: debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3] Error 2 make[4]: *** [Makefile:2063: run-command] Error 2 make[3]: *** [debian/rules:61: binary-headers] Error 2 dpkg-buildpackage: error: make -f debian/rules binary subprocess returned exit status 2 make[2]: *** [scripts/Makefile.package:121: bindeb-pkg] Error 2 make[1]: *** [/home/re/xfer/linux/Makefile:1557: bindeb-pkg] Error 2 make: *** [Makefile:224: __sub-make] Error 2
On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote: > On 7/27/24 12:42 AM, Masahiro Yamada wrote: > > A long standing issue in the upstream kernel packaging is that the > > linux-headers package is not cross-compiled. > > > > For example, you can cross-build Debian packages for arm64 by running > > the following command: > > > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > > > However, the generated linux-headers-*_arm64.deb is useless because the > > host programs in it were built for your build machine architecture > > (likely x86), not arm64. > > > > The Debian kernel maintains its own Makefiles to cross-compile host > > tools without relying on Kbuild. [1] > > > > Instead of adding such full custom Makefiles, this commit adds a small > > piece of code to cross-compile host programs located under the scripts/ > > directory. > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > > would also cross-compile scripts/basic/fixdep, which needs to be native > > to process the if_changed_dep macro. (This approach may work under some > > circumstances; you can execute foreign architecture programs with the > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > > but it would require installing QEMU and libc for that architecture.) > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > > userspace programs (CONFIG_CC_CAN_LINK=y). > > > > There are known limitations: > > > > - GCC plugins > > > > It would possible to rebuild GCC plugins for the target architecture > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > installed, but gcc on the installed system emits > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > solution for this because 'gcc' on a foreign architecture is a > > different compiler after all. > > > > - objtool and resolve_btfids > > > > These are built by the tools build system. They are not covered by > > the current solution. > > > > I only tested this with Debian, but it should work for other package > > systems as well. > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > --- > > > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > > 1 file changed, 34 insertions(+) > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > > index cc335945dfbc..0b56d3d7b48f 100755 > > --- a/scripts/package/install-extmod-build > > +++ b/scripts/package/install-extmod-build > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > > fi > > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > > +# the case for package building. It does not cross-compile when CC=clang. > > +# > > +# This caters to host programs that participate in Kbuild. objtool and > > +# resolve_btfids are out of scope. > > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > > + echo "Rebuilding host programs with ${CC}..." > > + > > + cat <<-'EOF' > "${destdir}/Kbuild" > > + subdir-y := scripts > > + EOF > > + > > + # HOSTCXX is not overridden. The C++ compiler is used to build: > > + # - scripts/kconfig/qconf, which is unneeded for external module builds > > + # - GCC plugins, which will not work on the installed system even with > > + # being rebuilt. > > + # > > + # Use the single-target build to avoid the modpost invocation, which > > + # would overwrite Module.symvers. > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > + > > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > > + subdir-y := basic > > + hostprogs-always-y := mod/modpost > > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > > + EOF > > + > > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > + > > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > > +fi > > + > > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete > > This patch causes a build error when cross-compiling for RISC-V. I'm using > the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain. > When trying to build .debs with: > > make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1 > "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg > > I get the following error: > > Rebuilding host programs with riscv64-unknown-linux-gnu-gcc... > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o > YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch] > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o > LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o > HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file > > debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10: > fatal error: openssl/opensslv.h: No such file or directory > 25 | #include <openssl/opensslv.h> > | ^~~~~~~~~~~~~~~~~~~~ > compilation terminated. I guess you have openssl/opensslv.h available on your system, do you? (In Debian/Ubuntu package libssl-dev or similar) Can you natively build a kernel with a similar kernel config? Kind regards, Nicolas
On 10/17/24 12:24 PM, Nicolas Schier wrote: > On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote: >> On 7/27/24 12:42 AM, Masahiro Yamada wrote: >>> A long standing issue in the upstream kernel packaging is that the >>> linux-headers package is not cross-compiled. >>> >>> For example, you can cross-build Debian packages for arm64 by running >>> the following command: >>> >>> $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg >>> >>> However, the generated linux-headers-*_arm64.deb is useless because the >>> host programs in it were built for your build machine architecture >>> (likely x86), not arm64. >>> >>> The Debian kernel maintains its own Makefiles to cross-compile host >>> tools without relying on Kbuild. [1] >>> >>> Instead of adding such full custom Makefiles, this commit adds a small >>> piece of code to cross-compile host programs located under the scripts/ >>> directory. >>> >>> A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it >>> would also cross-compile scripts/basic/fixdep, which needs to be native >>> to process the if_changed_dep macro. (This approach may work under some >>> circumstances; you can execute foreign architecture programs with the >>> help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, >>> but it would require installing QEMU and libc for that architecture.) >>> >>> A trick is to use the external module build (KBUILD_EXTMOD=), which >>> does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link >>> userspace programs (CONFIG_CC_CAN_LINK=y). >>> >>> There are known limitations: >>> >>> - GCC plugins >>> >>> It would possible to rebuild GCC plugins for the target architecture >>> by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages >>> installed, but gcc on the installed system emits >>> "cc1: error: incompatible gcc/plugin versions". I did not find a >>> solution for this because 'gcc' on a foreign architecture is a >>> different compiler after all. >>> >>> - objtool and resolve_btfids >>> >>> These are built by the tools build system. They are not covered by >>> the current solution. >>> >>> I only tested this with Debian, but it should work for other package >>> systems as well. >>> >>> [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 >>> >>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> >>> --- >>> >>> scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ >>> 1 file changed, 34 insertions(+) >>> >>> diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build >>> index cc335945dfbc..0b56d3d7b48f 100755 >>> --- a/scripts/package/install-extmod-build >>> +++ b/scripts/package/install-extmod-build >>> @@ -43,4 +43,38 @@ mkdir -p "${destdir}" >>> fi >>> } | tar -c -f - -T - | tar -xf - -C "${destdir}" >>> +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host >>> +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually >>> +# the case for package building. It does not cross-compile when CC=clang. >>> +# >>> +# This caters to host programs that participate in Kbuild. objtool and >>> +# resolve_btfids are out of scope. >>> +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then >>> + echo "Rebuilding host programs with ${CC}..." >>> + >>> + cat <<-'EOF' > "${destdir}/Kbuild" >>> + subdir-y := scripts >>> + EOF >>> + >>> + # HOSTCXX is not overridden. The C++ compiler is used to build: >>> + # - scripts/kconfig/qconf, which is unneeded for external module builds >>> + # - GCC plugins, which will not work on the installed system even with >>> + # being rebuilt. >>> + # >>> + # Use the single-target build to avoid the modpost invocation, which >>> + # would overwrite Module.symvers. >>> + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ >>> + >>> + cat <<-'EOF' > "${destdir}/scripts/Kbuild" >>> + subdir-y := basic >>> + hostprogs-always-y := mod/modpost >>> + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) >>> + EOF >>> + >>> + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. >>> + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ >>> + >>> + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" >>> +fi >>> + >>> find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete >> This patch causes a build error when cross-compiling for RISC-V. I'm using >> the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain. >> When trying to build .debs with: >> >> make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1 >> "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg >> >> I get the following error: >> >> Rebuilding host programs with riscv64-unknown-linux-gnu-gcc... >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o >> YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch] >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o >> LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o >> HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler >> HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file >> >> debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10: >> fatal error: openssl/opensslv.h: No such file or directory >> 25 | #include <openssl/opensslv.h> >> | ^~~~~~~~~~~~~~~~~~~~ >> compilation terminated. > I guess you have openssl/opensslv.h available on your system, do you? (In > Debian/Ubuntu package libssl-dev or similar) > > Can you natively build a kernel with a similar kernel config? > > Kind regards, > Nicolas Yes, I have /usr/include/openssl/opensslv.h on my system. But that's the x86 version. The cross compiler can't use that. A native build works fine. Ron
On Thu, Oct 17, 2024 at 12:34:49PM -0700, Ron Economos wrote: > On 10/17/24 12:24 PM, Nicolas Schier wrote: > > On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote: > > > On 7/27/24 12:42 AM, Masahiro Yamada wrote: > > > > A long standing issue in the upstream kernel packaging is that the > > > > linux-headers package is not cross-compiled. > > > > > > > > For example, you can cross-build Debian packages for arm64 by running > > > > the following command: > > > > > > > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > > > > > > > However, the generated linux-headers-*_arm64.deb is useless because the > > > > host programs in it were built for your build machine architecture > > > > (likely x86), not arm64. > > > > > > > > The Debian kernel maintains its own Makefiles to cross-compile host > > > > tools without relying on Kbuild. [1] > > > > > > > > Instead of adding such full custom Makefiles, this commit adds a small > > > > piece of code to cross-compile host programs located under the scripts/ > > > > directory. > > > > > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > > > > would also cross-compile scripts/basic/fixdep, which needs to be native > > > > to process the if_changed_dep macro. (This approach may work under some > > > > circumstances; you can execute foreign architecture programs with the > > > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > > > > but it would require installing QEMU and libc for that architecture.) > > > > > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which > > > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > > > > userspace programs (CONFIG_CC_CAN_LINK=y). > > > > > > > > There are known limitations: > > > > > > > > - GCC plugins > > > > > > > > It would possible to rebuild GCC plugins for the target architecture > > > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > > > installed, but gcc on the installed system emits > > > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > > > solution for this because 'gcc' on a foreign architecture is a > > > > different compiler after all. > > > > > > > > - objtool and resolve_btfids > > > > > > > > These are built by the tools build system. They are not covered by > > > > the current solution. > > > > > > > > I only tested this with Debian, but it should work for other package > > > > systems as well. > > > > > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > > > > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > > > --- > > > > > > > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > > > > 1 file changed, 34 insertions(+) > > > > > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > > > > index cc335945dfbc..0b56d3d7b48f 100755 > > > > --- a/scripts/package/install-extmod-build > > > > +++ b/scripts/package/install-extmod-build > > > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > > > > fi > > > > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > > > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > > > > +# the case for package building. It does not cross-compile when CC=clang. > > > > +# > > > > +# This caters to host programs that participate in Kbuild. objtool and > > > > +# resolve_btfids are out of scope. > > > > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > > > > + echo "Rebuilding host programs with ${CC}..." > > > > + > > > > + cat <<-'EOF' > "${destdir}/Kbuild" > > > > + subdir-y := scripts > > > > + EOF > > > > + > > > > + # HOSTCXX is not overridden. The C++ compiler is used to build: > > > > + # - scripts/kconfig/qconf, which is unneeded for external module builds > > > > + # - GCC plugins, which will not work on the installed system even with > > > > + # being rebuilt. > > > > + # > > > > + # Use the single-target build to avoid the modpost invocation, which > > > > + # would overwrite Module.symvers. > > > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > > > + > > > > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > > > > + subdir-y := basic > > > > + hostprogs-always-y := mod/modpost > > > > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > > > > + EOF > > > > + > > > > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > > > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > > > + > > > > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > > > > +fi > > > > + > > > > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete > > > This patch causes a build error when cross-compiling for RISC-V. I'm using > > > the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain. > > > When trying to build .debs with: > > > > > > make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1 > > > "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg > > > > > > I get the following error: > > > > > > Rebuilding host programs with riscv64-unknown-linux-gnu-gcc... > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o > > > YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch] > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o > > > LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o > > > HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file > > > > > > debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10: > > > fatal error: openssl/opensslv.h: No such file or directory > > > 25 | #include <openssl/opensslv.h> > > > | ^~~~~~~~~~~~~~~~~~~~ > > > compilation terminated. > > I guess you have openssl/opensslv.h available on your system, do you? (In > > Debian/Ubuntu package libssl-dev or similar) > > > > Can you natively build a kernel with a similar kernel config? > > > > Kind regards, > > Nicolas > > Yes, I have /usr/include/openssl/opensslv.h on my system. But that's the x86 > version. The cross compiler can't use that. You'll need to add the package for cross-compilation. If you are using ubuntu and the ubuntu riscv64 toolchain, you can add the riscv64 architecture `dpkg --add-architecture riscv64`, swap out your sources.list file to specify the architecture `sed -i 's/^deb/deb [arch=amd64]/' /etc/apt/sources.list`, add the riscv64 debs to your sources.list: deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy main restricted multiverse universe deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-updates main deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-security main Then `apt update` and `apt install libssl-dev:riscv64`. I imagine there is a similar procedure for other distros. If using a custom compiler, you'll need to copy over the installed headers to whatever location your compiler is looking for them. - Charlie > > A native build works fine. > > Ron > >
On Fri, Nov 15, 2024 at 5:57 AM Charlie Jenkins <charlie@rivosinc.com> wrote: > > On Thu, Oct 17, 2024 at 12:34:49PM -0700, Ron Economos wrote: > > On 10/17/24 12:24 PM, Nicolas Schier wrote: > > > On Thu, Oct 17, 2024 at 07:45:57AM -0700 Ron Economos wrote: > > > > On 7/27/24 12:42 AM, Masahiro Yamada wrote: > > > > > A long standing issue in the upstream kernel packaging is that the > > > > > linux-headers package is not cross-compiled. > > > > > > > > > > For example, you can cross-build Debian packages for arm64 by running > > > > > the following command: > > > > > > > > > > $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg > > > > > > > > > > However, the generated linux-headers-*_arm64.deb is useless because the > > > > > host programs in it were built for your build machine architecture > > > > > (likely x86), not arm64. > > > > > > > > > > The Debian kernel maintains its own Makefiles to cross-compile host > > > > > tools without relying on Kbuild. [1] > > > > > > > > > > Instead of adding such full custom Makefiles, this commit adds a small > > > > > piece of code to cross-compile host programs located under the scripts/ > > > > > directory. > > > > > > > > > > A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it > > > > > would also cross-compile scripts/basic/fixdep, which needs to be native > > > > > to process the if_changed_dep macro. (This approach may work under some > > > > > circumstances; you can execute foreign architecture programs with the > > > > > help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, > > > > > but it would require installing QEMU and libc for that architecture.) > > > > > > > > > > A trick is to use the external module build (KBUILD_EXTMOD=), which > > > > > does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link > > > > > userspace programs (CONFIG_CC_CAN_LINK=y). > > > > > > > > > > There are known limitations: > > > > > > > > > > - GCC plugins > > > > > > > > > > It would possible to rebuild GCC plugins for the target architecture > > > > > by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages > > > > > installed, but gcc on the installed system emits > > > > > "cc1: error: incompatible gcc/plugin versions". I did not find a > > > > > solution for this because 'gcc' on a foreign architecture is a > > > > > different compiler after all. > > > > > > > > > > - objtool and resolve_btfids > > > > > > > > > > These are built by the tools build system. They are not covered by > > > > > the current solution. > > > > > > > > > > I only tested this with Debian, but it should work for other package > > > > > systems as well. > > > > > > > > > > [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 > > > > > > > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > > > > > --- > > > > > > > > > > scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ > > > > > 1 file changed, 34 insertions(+) > > > > > > > > > > diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build > > > > > index cc335945dfbc..0b56d3d7b48f 100755 > > > > > --- a/scripts/package/install-extmod-build > > > > > +++ b/scripts/package/install-extmod-build > > > > > @@ -43,4 +43,38 @@ mkdir -p "${destdir}" > > > > > fi > > > > > } | tar -c -f - -T - | tar -xf - -C "${destdir}" > > > > > +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host > > > > > +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually > > > > > +# the case for package building. It does not cross-compile when CC=clang. > > > > > +# > > > > > +# This caters to host programs that participate in Kbuild. objtool and > > > > > +# resolve_btfids are out of scope. > > > > > +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then > > > > > + echo "Rebuilding host programs with ${CC}..." > > > > > + > > > > > + cat <<-'EOF' > "${destdir}/Kbuild" > > > > > + subdir-y := scripts > > > > > + EOF > > > > > + > > > > > + # HOSTCXX is not overridden. The C++ compiler is used to build: > > > > > + # - scripts/kconfig/qconf, which is unneeded for external module builds > > > > > + # - GCC plugins, which will not work on the installed system even with > > > > > + # being rebuilt. > > > > > + # > > > > > + # Use the single-target build to avoid the modpost invocation, which > > > > > + # would overwrite Module.symvers. > > > > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > > > > + > > > > > + cat <<-'EOF' > "${destdir}/scripts/Kbuild" > > > > > + subdir-y := basic > > > > > + hostprogs-always-y := mod/modpost > > > > > + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) > > > > > + EOF > > > > > + > > > > > + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. > > > > > + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ > > > > > + > > > > > + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" > > > > > +fi > > > > > + > > > > > find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete > > > > This patch causes a build error when cross-compiling for RISC-V. I'm using > > > > the cross-compiler from https://github.com/riscv-collab/riscv-gnu-toolchain. > > > > When trying to build .debs with: > > > > > > > > make CROSS_COMPILE=riscv64-unknown-linux-gnu- ARCH=riscv INSTALL_MOD_STRIP=1 > > > > "KCFLAGS=-mtune=sifive-7-series" LOCALVERSION= bindeb-pkg > > > > > > > > I get the following error: > > > > > > > > Rebuilding host programs with riscv64-unknown-linux-gnu-gcc... > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms.o > > > > YACC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.[ch] > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/parse.tab.o > > > > LEX debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.c > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/lex.lex.o > > > > HOSTLD debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/genksyms/genksyms > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/genheaders/genheaders > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/selinux/mdp/mdp > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/kallsyms > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sorttable > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/asn1_compiler > > > > HOSTCC debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file > > > > > > > > debian/linux-headers-6.12.0-rc3/usr/src/linux-headers-6.12.0-rc3/scripts/sign-file.c:25:10: > > > > fatal error: openssl/opensslv.h: No such file or directory > > > > 25 | #include <openssl/opensslv.h> > > > > | ^~~~~~~~~~~~~~~~~~~~ > > > > compilation terminated. > > > I guess you have openssl/opensslv.h available on your system, do you? (In > > > Debian/Ubuntu package libssl-dev or similar) > > > > > > Can you natively build a kernel with a similar kernel config? > > > > > > Kind regards, > > > Nicolas > > > > Yes, I have /usr/include/openssl/opensslv.h on my system. But that's the x86 > > version. The cross compiler can't use that. > > You'll need to add the package for cross-compilation. If you are using > ubuntu and the ubuntu riscv64 toolchain, you can add the riscv64 > architecture `dpkg --add-architecture riscv64`, swap out your > sources.list file to specify the architecture `sed -i 's/^deb/deb > [arch=amd64]/' /etc/apt/sources.list`, add the riscv64 debs to your > sources.list: > > deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy main restricted multiverse universe > deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-updates main > deb [arch=riscv64] http://ports.ubuntu.com/ubuntu-ports jammy-security main > > Then `apt update` and `apt install libssl-dev:riscv64`. I imagine there > is a similar procedure for other distros. If using a custom compiler, > you'll need to copy over the installed headers to whatever location your > compiler is looking for them. Right, this build dependency is enforced since commit e2c318225ac13083cdcb4780cdf5b90edaa8644d > > - Charlie > > > > > A native build works fine. > > > > Ron > > > >
diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index cc335945dfbc..0b56d3d7b48f 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -43,4 +43,38 @@ mkdir -p "${destdir}" fi } | tar -c -f - -T - | tar -xf - -C "${destdir}" +# When ${CC} and ${HOSTCC} differ, we are likely cross-compiling. Rebuild host +# programs using ${CC}. This assumes CC=${CROSS_COMPILE}gcc, which is usually +# the case for package building. It does not cross-compile when CC=clang. +# +# This caters to host programs that participate in Kbuild. objtool and +# resolve_btfids are out of scope. +if [ "${CC}" != "${HOSTCC}" ] && is_enabled CONFIG_CC_CAN_LINK; then + echo "Rebuilding host programs with ${CC}..." + + cat <<-'EOF' > "${destdir}/Kbuild" + subdir-y := scripts + EOF + + # HOSTCXX is not overridden. The C++ compiler is used to build: + # - scripts/kconfig/qconf, which is unneeded for external module builds + # - GCC plugins, which will not work on the installed system even with + # being rebuilt. + # + # Use the single-target build to avoid the modpost invocation, which + # would overwrite Module.symvers. + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + + cat <<-'EOF' > "${destdir}/scripts/Kbuild" + subdir-y := basic + hostprogs-always-y := mod/modpost + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) + EOF + + # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. + "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + + rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" +fi + find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete
A long standing issue in the upstream kernel packaging is that the linux-headers package is not cross-compiled. For example, you can cross-build Debian packages for arm64 by running the following command: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg However, the generated linux-headers-*_arm64.deb is useless because the host programs in it were built for your build machine architecture (likely x86), not arm64. The Debian kernel maintains its own Makefiles to cross-compile host tools without relying on Kbuild. [1] Instead of adding such full custom Makefiles, this commit adds a small piece of code to cross-compile host programs located under the scripts/ directory. A straightforward solution is to pass HOSTCC=${CROSS_COMPILE}gcc, but it would also cross-compile scripts/basic/fixdep, which needs to be native to process the if_changed_dep macro. (This approach may work under some circumstances; you can execute foreign architecture programs with the help of binfmt_misc because Debian systems enable CONFIG_BINFMT_MISC, but it would require installing QEMU and libc for that architecture.) A trick is to use the external module build (KBUILD_EXTMOD=), which does not rebuild scripts/basic/fixdep. ${CC} needs to be able to link userspace programs (CONFIG_CC_CAN_LINK=y). There are known limitations: - GCC plugins It would possible to rebuild GCC plugins for the target architecture by passing HOSTCXX=${CROSS_COMPILE}g++ with necessary packages installed, but gcc on the installed system emits "cc1: error: incompatible gcc/plugin versions". I did not find a solution for this because 'gcc' on a foreign architecture is a different compiler after all. - objtool and resolve_btfids These are built by the tools build system. They are not covered by the current solution. I only tested this with Debian, but it should work for other package systems as well. [1]: https://salsa.debian.org/kernel-team/linux/-/blob/debian/6.9.9-1/debian/rules.real#L586 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/install-extmod-build | 34 ++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)