| Message ID | 20250224132132.1765115-6-ardb+git@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | x86/build: Get rid of vmlinux postlink step | expand |
* Ard Biesheuvel <ardb+git@google.com> wrote: > From: Ard Biesheuvel <ardb@kernel.org> > > Kbuild supports an architecture specific Makefile.postlink file that is > invoked for the vmlinux target after it has been built. This Makefile > takes 'vmlinux' (which has just been built) as the target, and mangles > the file and/or constructs other intermediate artifacts from it. > > This violates the general philosophy of Make, which is based on rules > and dependencies, and artifacts that are rebuilt only when any of their > dependencies have been updated. > > Instead, the different incarnations of vmlinux that are consumed by > different stages of the build should be emitted as distinct files, where > rules and dependencies are used to define one in terms of the other. > > This also works around an error observed here [0], where vmlinux is > deleted by Make because a subsequent step that consumes it as input > throws an error. > > So refactor the vmlinux shell scripts and build rules so that > architectures that rely on --emit-relocs to construct vmlinux with > static relocations preserved will get a separate vmlinux.unstripped file > carrying those relocations. This removes the need for an imperative > postlink step, given that any rules that depend on the unstripped > vmlinux can now simply depend on vmlinux.unstripped, rather than inject > a build step into Makefile.postlink Nice! Does this also result in any measurable speedup of the build, or are the steps still similar? But in terms of justification for upstreaming, the reduction in complexity alone makes it worth it IMO: 19 files changed, 52 insertions(+), 87 deletions(-) Thanks, Ingo
On Mon, 24 Feb 2025 at 10:51, Ingo Molnar <mingo@kernel.org> wrote: > > But in terms of justification for upstreaming, the reduction in > complexity alone makes it worth it IMO: > > 19 files changed, 52 insertions(+), 87 deletions(-) Yeah, absolutely. Our fancy make build rules still have too many of the phony forced targets, but this is a few less of them and makes the build confirm (more) to the usual rules. I do wonder if we could just get rid of that CONFIG_ARCH_VMLINUX_NEEDS_RELOCS entirely and make it just be how all architectures do it. Yes, it was apparently "just" riscv/s390/x86/mips that did that 'strip_relocs' hack, but at the same time that whole pass *feels* entirely generic. IOW, this all makes me just wonder why other architectures don't do it? Linus
On Mon, 24 Feb 2025 at 21:00, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > On Mon, 24 Feb 2025 at 10:51, Ingo Molnar <mingo@kernel.org> wrote: > > > > But in terms of justification for upstreaming, the reduction in > > complexity alone makes it worth it IMO: > > > > 19 files changed, 52 insertions(+), 87 deletions(-) > > Yeah, absolutely. Our fancy make build rules still have too many of > the phony forced targets, but this is a few less of them and makes the > build confirm (more) to the usual rules. > > I do wonder if we could just get rid of that > CONFIG_ARCH_VMLINUX_NEEDS_RELOCS entirely and make it just be how all > architectures do it. > > Yes, it was apparently "just" riscv/s390/x86/mips that did that > 'strip_relocs' hack, but at the same time that whole pass *feels* > entirely generic. > TL;DR it is not It is only needed on architectures that use --emit-relocs in the first place, e.g., to construct bespoke KASLR tables. This is actually a somewhat dubious practice, because these are static relocations, i.e., what the linker consumes as input, and they are emitted along with vmlinux as output. [*] This feature was (AFAIK) never really intended for constructing dynamic relocation tables as some architectures in Linux do. On those architectures, these static relocations need to be stripped again, to avoid bloating vmlinux with useless data. On architectures that rely on PIE linking (such as arm64), the linker will emit a dynamic relocation table that is more suitable for use at boot time, i.e., it only contains absolute relocations (as RIP-relative ones never require any fixing up at boot), and uses RELR format to pack them very densely, removing the need for our own special format. Architectures that do not implement KASLR in the first place have no need for these static relocations either. PIE linking is generally a better choice than relying on --emit-relocs, but it is highly ISA dependent whether that requires a costlier kind of codegen. On arm64, we don't even bother generating -fPIE code because ordinary code can be linked in PIE mode. OTOH, on x86, we'd need full 64-bit PIC/PIE codegen in order to link with PIE, whereas we currently rely on the 'kernel' code model to generate 32-bit wide absolute symbol references that can only refer to the top 2G of the 64-bit address space. [*] Using static relocations to describe a fully linked binary such as vmlinux is problematic because a) it covers all external symbol references, including relative ones that don't need fixing up, but more importantly, b) the linker may perform relaxations that result in the code going out of sync with the relocation that annotates it (this is not entirely avoidable if the relaxed version of the code cannot even be described by any relocation specified by the ELF psABI)
On Mon, 24 Feb 2025 at 19:51, Ingo Molnar <mingo@kernel.org> wrote: > ... > > > > So refactor the vmlinux shell scripts and build rules so that > > architectures that rely on --emit-relocs to construct vmlinux with > > static relocations preserved will get a separate vmlinux.unstripped file > > carrying those relocations. This removes the need for an imperative > > postlink step, given that any rules that depend on the unstripped > > vmlinux can now simply depend on vmlinux.unstripped, rather than inject > > a build step into Makefile.postlink > > Nice! Does this also result in any measurable speedup of the build, or > are the steps still similar? > The steps are essentially the same, only described in a slightly better way (and with an additional artifact so the used disk space goes up slightly too)
From: Ard Biesheuvel <ardb@kernel.org> Kbuild supports an architecture specific Makefile.postlink file that is invoked for the vmlinux target after it has been built. This Makefile takes 'vmlinux' (which has just been built) as the target, and mangles the file and/or constructs other intermediate artifacts from it. This violates the general philosophy of Make, which is based on rules and dependencies, and artifacts that are rebuilt only when any of their dependencies have been updated. Instead, the different incarnations of vmlinux that are consumed by different stages of the build should be emitted as distinct files, where rules and dependencies are used to define one in terms of the other. This also works around an error observed here [0], where vmlinux is deleted by Make because a subsequent step that consumes it as input throws an error. So refactor the vmlinux shell scripts and build rules so that architectures that rely on --emit-relocs to construct vmlinux with static relocations preserved will get a separate vmlinux.unstripped file carrying those relocations. This removes the need for an imperative postlink step, given that any rules that depend on the unstripped vmlinux can now simply depend on vmlinux.unstripped, rather than inject a build step into Makefile.postlink S390 should be able to do the same. MIPS and RISC-V perform some post-build checks on vmlinux, which is reasonable in principle for a postlink step, although deleting vmlinux when the check fails is equally unhelpful. [0] https://lore.kernel.org/all/Z5ARucnUgqjwBnrp@gmail.com/T/#m731ed0206949fc3f39fcc8a7b82fe348a8fc80c4 Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Ingo Molnar <mingo@kernel.org> Ard Biesheuvel (4): Kbuild/link-vmlinux.sh: Make output file name configurable Kbuild: Introduce Kconfig symbol for linking vmlinux with relocations Kbuild: Create intermediate vmlinux build with relocations preserved x86: Get rid of Makefile.postlink Makefile | 4 ++ arch/Kconfig | 7 ++++ arch/mips/Kconfig | 1 + arch/mips/Makefile | 4 -- arch/mips/Makefile.postlink | 2 +- arch/riscv/Kconfig | 1 + arch/riscv/Makefile | 2 +- arch/riscv/Makefile.postlink | 11 +----- arch/riscv/boot/Makefile | 5 +-- arch/s390/Kconfig | 1 + arch/s390/Makefile | 2 +- arch/s390/Makefile.postlink | 4 +- arch/x86/Kconfig | 1 + arch/x86/Makefile | 6 --- arch/x86/Makefile.postlink | 40 -------------------- arch/x86/boot/compressed/Makefile | 8 ++-- scripts/Makefile.lib | 2 +- scripts/Makefile.vmlinux | 27 +++++++++---- scripts/link-vmlinux.sh | 11 +++--- 19 files changed, 52 insertions(+), 87 deletions(-) delete mode 100644 arch/x86/Makefile.postlink