| Message ID | 20250122-kernel-compress-fast-v2-1-bde9d71b6b88@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level | expand |
On 1/22/25 16:08, Jann Horn wrote: > Allow choosing the LZ4 compression level using the new config flag > CONFIG_LZ4_LEVEL. > > This will allow developers who want to quickly iterate with one-off kernel > builds to speed up kernel compression at the cost of creating a larger > kernel image. > > LZ4 at level 9 (which the kernel uses by default) takes ~2.8 > seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 > just needs ~220ms wall clock time. > > Signed-off-by: Jann Horn <jannh@google.com> > --- > My patch provides kconfig for choosing compression level of LZ4; Tor Vic > mentioned using kconfig for choosing the compression level of ZSTD. > > Does my current approach of having a kconfig option specifically for > LZ4 compression level look good (and we can add the same for ZSTD later > if necessary); or should there be a single kconfig option that sets a > compression level that is used across all compression algorithms? > I guess doing it separately for different algorithms makes more sense > because different compression levels might mean different things for > different algorithms? Hi Jann, Yes, different algos have different ways of specifying compression levels. You cannot have one single option for all of the available compressors. > > Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size > rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on > its own somehow? Experimentally it seems like the build system triggers > a rebuild when the config option changes, but I'm not sure how that > actually works. Here is my take for zstd which I've been using for a while now. (zstd levels above 19 need the '--ultra' switch, hence the honkytonk below). -- From 2e9a0e058e10a428d2ee079b969627e475cc0f2e Mon Sep 17 00:00:00 2001 From: Tor Vic <torvic9@mailbox.org> Date: Tue, 6 Aug 2024 11:24:32 +0200 Subject: [PATCH] Kbuild, zstd: Allow users to choose a compression level for the kernel image Signed-off-by: Tor Vic <torvic9@mailbox.org> --- init/Kconfig | 13 +++++++++++++ scripts/Makefile.lib | 9 ++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/init/Kconfig b/init/Kconfig index 105cdd05db759..9c39bc17964de 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -380,6 +380,19 @@ config KERNEL_UNCOMPRESSED endchoice +menu "ZSTD compression options" + depends on KERNEL_ZSTD + +config ZSTD_COMPRESSION_LVL + int "Compression level (1-22)" + range 1 22 + default "22" + help + Choose a compression level for zstd kernel compression. + Default is 22, which is the maximum. + +endmenu + config DEFAULT_INIT string "Default init path" default "" diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 7395200538da8..58884363ebc62 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -516,6 +516,13 @@ quiet_cmd_xzmisc = XZMISC $@ # decompression is used, like initramfs decompression, zstd22 should likely not # be used because it would require zstd to allocate a 128 MB buffer. +ifdef CONFIG_ZSTD_COMPRESSION_LVL +zstd_comp_lvl := $(CONFIG_ZSTD_COMPRESSION_LVL) +ifeq ($(call test-gt, $(zstd_comp_lvl), 19),y) +zstd_comp_lvl += --ultra +endif +endif + quiet_cmd_zstd = ZSTD $@ cmd_zstd = cat $(real-prereqs) | $(ZSTD) -19 > $@ @@ -523,7 +530,7 @@ quiet_cmd_zstd22 = ZSTD22 $@ cmd_zstd22 = cat $(real-prereqs) | $(ZSTD) -22 --ultra > $@ quiet_cmd_zstd22_with_size = ZSTD22 $@ - cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@ + cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -T0 -$(zstd_comp_lvl); $(size_append); } > $@ # ASM offsets # --------------------------------------------------------------------------- --
diff --git a/init/Kconfig b/init/Kconfig index a20e6efd3f0fbdd7f0df2448854cc30734a0ee4f..3fd93bbe98390997e7e2d41426241b90d3b33053 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -358,6 +358,9 @@ config KERNEL_LZ4 is about 8% bigger than LZO. But the decompression speed is faster than LZO. + If you are working on the kernel and recompiling frequently, you can + adjust the compression level with CONFIG_LZ4_LEVEL. + config KERNEL_ZSTD bool "ZSTD" depends on HAVE_KERNEL_ZSTD @@ -380,6 +383,19 @@ config KERNEL_UNCOMPRESSED endchoice +config LZ4_LEVEL + int "LZ4 compression level" + range 1 12 + depends on KERNEL_LZ4 + default 9 + help + This option controls the compression level used for LZ4 invocations, + in particular for compressing the kernel image. + + If you are working on the kernel and recompiling frequently, you can + set this to a low value like 1 to speed up kernel builds at the cost + of using more disk space. + config DEFAULT_INIT string "Default init path" default "" diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 7395200538da89a2f6e6d21f8959f3f60d291d79..c3eedbc4acf133d5f9e7f13d637af09ad1968930 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -425,10 +425,10 @@ quiet_cmd_lzo_with_size = LZO $@ cmd_lzo_with_size = { cat $(real-prereqs) | $(KLZOP) -9; $(size_append); } > $@ quiet_cmd_lz4 = LZ4 $@ - cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -9 - - > $@ + cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - - > $@ quiet_cmd_lz4_with_size = LZ4 $@ - cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -9 - -; \ + cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - -; \ $(size_append); } > $@ # U-Boot mkimage
Allow choosing the LZ4 compression level using the new config flag CONFIG_LZ4_LEVEL. This will allow developers who want to quickly iterate with one-off kernel builds to speed up kernel compression at the cost of creating a larger kernel image. LZ4 at level 9 (which the kernel uses by default) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 just needs ~220ms wall clock time. Signed-off-by: Jann Horn <jannh@google.com> --- My patch provides kconfig for choosing compression level of LZ4; Tor Vic mentioned using kconfig for choosing the compression level of ZSTD. Does my current approach of having a kconfig option specifically for LZ4 compression level look good (and we can add the same for ZSTD later if necessary); or should there be a single kconfig option that sets a compression level that is used across all compression algorithms? I guess doing it separately for different algorithms makes more sense because different compression levels might mean different things for different algorithms? Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on its own somehow? Experimentally it seems like the build system triggers a rebuild when the config option changes, but I'm not sure how that actually works. --- Changes in v2: - removed x86/EFI subsystems from recipients since I'm no longer touching those - rewrote patch to make LZ4 level configurable instead of offering uncompressed builds on x86 (Ard) - Link to v1: https://lore.kernel.org/r/20250121-kernel-compress-fast-v1-1-fa693b6167d4@google.com --- init/Kconfig | 16 ++++++++++++++++ scripts/Makefile.lib | 4 ++-- 2 files changed, 18 insertions(+), 2 deletions(-) --- base-commit: 95ec54a420b8f445e04a7ca0ea8deb72c51fe1d3 change-id: 20250121-kernel-compress-fast-350ce5801c28