| Message ID | 20231119100024.2370992-1-masahiroy@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi | expand |
On Sun, 19 Nov 2023 at 05:00, Masahiro Yamada <masahiroy@kernel.org> wrote: > > A common issue in Makefile is a race in parallel building. > > You need to be careful to prevent multiple threads from writing to the > same file simultaneously. > > Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not > generate invalid images") addressed such a bad scenario. > > A similar symptom occurs with the following command: > > $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi > [ snip ] > SORTTAB vmlinux > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > GZIP arch/riscv/boot/Image.gz > AS arch/riscv/boot/loader.o > AS arch/riscv/boot/loader.o > Kernel: arch/riscv/boot/Image is ready > PAD arch/riscv/boot/vmlinux.bin > GZIP arch/riscv/boot/vmlinuz > Kernel: arch/riscv/boot/loader is ready > OBJCOPY arch/riscv/boot/loader.bin > Kernel: arch/riscv/boot/loader.bin is ready > Kernel: arch/riscv/boot/Image.gz is ready > OBJCOPY arch/riscv/boot/vmlinuz.o > LD arch/riscv/boot/vmlinuz.efi.elf > OBJCOPY arch/riscv/boot/vmlinuz.efi > Kernel: arch/riscv/boot/vmlinuz.efi is ready > > The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times. > (also "AS arch/riscv/boot/loader.o" twice.) > > It indicates that 5 threads simultaneously enter arch/riscv/boot/ > and write to arch/riscv/boot/Image. > > It occasionally leads to a build failure: > > $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi > [ snip ] > SORTTAB vmlinux > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > PAD arch/riscv/boot/vmlinux.bin > truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin' > make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1 > make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin' > make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2 > make[1]: *** Waiting for unfinished jobs.... > Kernel: arch/riscv/boot/Image is ready > GZIP arch/riscv/boot/Image.gz > AS arch/riscv/boot/loader.o > AS arch/riscv/boot/loader.o > Kernel: arch/riscv/boot/loader is ready > OBJCOPY arch/riscv/boot/loader.bin > Kernel: arch/riscv/boot/loader.bin is ready > Kernel: arch/riscv/boot/Image.gz is ready > make: *** [Makefile:234: __sub-make] Error 2 > > Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends > on loader. Such dependencies are not specified in arch/riscv/Makefile. > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > Changes in v2: > - Fix commit log > Acked-by: Ard Biesheuvel <ardb@kernel.org> > arch/riscv/Makefile | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile > index 5cbe596345c1..1d6ed27e0a2a 100644 > --- a/arch/riscv/Makefile > +++ b/arch/riscv/Makefile > @@ -163,6 +163,8 @@ BOOT_TARGETS := Image Image.gz loader loader.bin xipImage vmlinuz.efi > > all: $(notdir $(KBUILD_IMAGE)) > > +loader.bin: loader > +Image.gz loader vmlinuz.efi: Image > $(BOOT_TARGETS): vmlinux > $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ > @$(kecho) ' Kernel: $(boot)/$@ is ready' > -- > 2.40.1 >
On 2023-11-19 4:00 AM, Masahiro Yamada wrote: > A common issue in Makefile is a race in parallel building. > > You need to be careful to prevent multiple threads from writing to the > same file simultaneously. > > Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not > generate invalid images") addressed such a bad scenario. > > A similar symptom occurs with the following command: > > $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi > [ snip ] > SORTTAB vmlinux > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > GZIP arch/riscv/boot/Image.gz > AS arch/riscv/boot/loader.o > AS arch/riscv/boot/loader.o > Kernel: arch/riscv/boot/Image is ready > PAD arch/riscv/boot/vmlinux.bin > GZIP arch/riscv/boot/vmlinuz > Kernel: arch/riscv/boot/loader is ready > OBJCOPY arch/riscv/boot/loader.bin > Kernel: arch/riscv/boot/loader.bin is ready > Kernel: arch/riscv/boot/Image.gz is ready > OBJCOPY arch/riscv/boot/vmlinuz.o > LD arch/riscv/boot/vmlinuz.efi.elf > OBJCOPY arch/riscv/boot/vmlinuz.efi > Kernel: arch/riscv/boot/vmlinuz.efi is ready > > The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times. > (also "AS arch/riscv/boot/loader.o" twice.) > > It indicates that 5 threads simultaneously enter arch/riscv/boot/ > and write to arch/riscv/boot/Image. > > It occasionally leads to a build failure: > > $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi > [ snip ] > SORTTAB vmlinux > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > OBJCOPY arch/riscv/boot/Image > PAD arch/riscv/boot/vmlinux.bin > truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin' > make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1 > make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin' > make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2 > make[1]: *** Waiting for unfinished jobs.... > Kernel: arch/riscv/boot/Image is ready > GZIP arch/riscv/boot/Image.gz > AS arch/riscv/boot/loader.o > AS arch/riscv/boot/loader.o > Kernel: arch/riscv/boot/loader is ready > OBJCOPY arch/riscv/boot/loader.bin > Kernel: arch/riscv/boot/loader.bin is ready > Kernel: arch/riscv/boot/Image.gz is ready > make: *** [Makefile:234: __sub-make] Error 2 > > Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends > on loader. Such dependencies are not specified in arch/riscv/Makefile. > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> > --- > > Changes in v2: > - Fix commit log > > arch/riscv/Makefile | 2 ++ > 1 file changed, 2 insertions(+) Reviewed-by: Samuel Holland <samuel.holland@sifive.com> Tested-by: Samuel Holland <samuel.holland@sifive.com>
Hello: This patch was applied to riscv/linux.git (fixes) by Palmer Dabbelt <palmer@rivosinc.com>: On Sun, 19 Nov 2023 19:00:24 +0900 you wrote: > A common issue in Makefile is a race in parallel building. > > You need to be careful to prevent multiple threads from writing to the > same file simultaneously. > > Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not > generate invalid images") addressed such a bad scenario. > > [...] Here is the summary with links: - [v2] riscv: add dependency among Image(.gz), loader(.bin), and vmlinuz.efi https://git.kernel.org/riscv/c/c4db7ff7a9ed You are awesome, thank you!
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 5cbe596345c1..1d6ed27e0a2a 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -163,6 +163,8 @@ BOOT_TARGETS := Image Image.gz loader loader.bin xipImage vmlinuz.efi all: $(notdir $(KBUILD_IMAGE)) +loader.bin: loader +Image.gz loader vmlinuz.efi: Image $(BOOT_TARGETS): vmlinux $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ @$(kecho) ' Kernel: $(boot)/$@ is ready'
A common issue in Makefile is a race in parallel building. You need to be careful to prevent multiple threads from writing to the same file simultaneously. Commit 3939f3345050 ("ARM: 8418/1: add boot image dependencies to not generate invalid images") addressed such a bad scenario. A similar symptom occurs with the following command: $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi [ snip ] SORTTAB vmlinux OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image GZIP arch/riscv/boot/Image.gz AS arch/riscv/boot/loader.o AS arch/riscv/boot/loader.o Kernel: arch/riscv/boot/Image is ready PAD arch/riscv/boot/vmlinux.bin GZIP arch/riscv/boot/vmlinuz Kernel: arch/riscv/boot/loader is ready OBJCOPY arch/riscv/boot/loader.bin Kernel: arch/riscv/boot/loader.bin is ready Kernel: arch/riscv/boot/Image.gz is ready OBJCOPY arch/riscv/boot/vmlinuz.o LD arch/riscv/boot/vmlinuz.efi.elf OBJCOPY arch/riscv/boot/vmlinuz.efi Kernel: arch/riscv/boot/vmlinuz.efi is ready The log "OBJCOPY arch/riscv/boot/Image" is displayed 5 times. (also "AS arch/riscv/boot/loader.o" twice.) It indicates that 5 threads simultaneously enter arch/riscv/boot/ and write to arch/riscv/boot/Image. It occasionally leads to a build failure: $ make -j$(nproc) ARCH=riscv Image Image.gz loader loader.bin vmlinuz.efi [ snip ] SORTTAB vmlinux OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image OBJCOPY arch/riscv/boot/Image PAD arch/riscv/boot/vmlinux.bin truncate: Invalid number: 'arch/riscv/boot/vmlinux.bin' make[2]: *** [drivers/firmware/efi/libstub/Makefile.zboot:13: arch/riscv/boot/vmlinux.bin] Error 1 make[2]: *** Deleting file 'arch/riscv/boot/vmlinux.bin' make[1]: *** [arch/riscv/Makefile:167: vmlinuz.efi] Error 2 make[1]: *** Waiting for unfinished jobs.... Kernel: arch/riscv/boot/Image is ready GZIP arch/riscv/boot/Image.gz AS arch/riscv/boot/loader.o AS arch/riscv/boot/loader.o Kernel: arch/riscv/boot/loader is ready OBJCOPY arch/riscv/boot/loader.bin Kernel: arch/riscv/boot/loader.bin is ready Kernel: arch/riscv/boot/Image.gz is ready make: *** [Makefile:234: __sub-make] Error 2 Image.gz, loader, vmlinuz.efi depend on Image. loader.bin depends on loader. Such dependencies are not specified in arch/riscv/Makefile. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Changes in v2: - Fix commit log arch/riscv/Makefile | 2 ++ 1 file changed, 2 insertions(+)