| Message ID | 20241114145645.563356-1-parth105105@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] kbuild: switch from lz4c to lz4 for compression | expand |
On Thu, Nov 14, 2024 at 03:56:44PM +0100, Parth Pancholi wrote: > From: Parth Pancholi <parth.pancholi@toradex.com> > > Replace lz4c with lz4 for kernel image compression. > Although lz4 and lz4c are functionally similar, lz4c has been deprecated > upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4 > and lz4c have been packaged together, making it safe to update the > requirement from lz4c to lz4. > > Consequently, some distributions and build systems, such as OpenEmbedded, > have fully transitioned to using lz4. OpenEmbedded core adopted this > change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of > lz4c"), causing compatibility issues when building the mainline kernel > in the latest OpenEmbedded environment, as seen in the errors below. > > This change also updates the LZ4 compression commands to make it backward > compatible by replacing stdin and stdout with the '-' option, due to some > unclear reason, the stdout keyword does not work for lz4 and '-' works for > both. In addition, this modifies the legacy '-c1' with '-9' which is also > compatible with both. This fixes the mainline kernel build failures with > the latest master OpenEmbedded builds associated with the mentioned > compatibility issues. > > LZ4 arch/arm/boot/compressed/piggy_data > /bin/sh: 1: lz4c: not found > ... > ... > ERROR: oe_runmake failed > > Cc: stable@vger.kernel.org What bug does this resolve that it needs to be backported to stable kernels? thanks, greg k-h
Hello Greg, On Thu, Nov 14, 2024 at 05:02:01PM +0100, Greg KH wrote: > On Thu, Nov 14, 2024 at 03:56:44PM +0100, Parth Pancholi wrote: > > From: Parth Pancholi <parth.pancholi@toradex.com> > > > > Replace lz4c with lz4 for kernel image compression. > > Although lz4 and lz4c are functionally similar, lz4c has been deprecated > > upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4 > > and lz4c have been packaged together, making it safe to update the > > requirement from lz4c to lz4. > > > > Consequently, some distributions and build systems, such as OpenEmbedded, > > have fully transitioned to using lz4. OpenEmbedded core adopted this > > change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of > > lz4c"), causing compatibility issues when building the mainline kernel > > in the latest OpenEmbedded environment, as seen in the errors below. > > > > This change also updates the LZ4 compression commands to make it backward > > compatible by replacing stdin and stdout with the '-' option, due to some > > unclear reason, the stdout keyword does not work for lz4 and '-' works for > > both. In addition, this modifies the legacy '-c1' with '-9' which is also > > compatible with both. This fixes the mainline kernel build failures with > > the latest master OpenEmbedded builds associated with the mentioned > > compatibility issues. > > > > LZ4 arch/arm/boot/compressed/piggy_data > > /bin/sh: 1: lz4c: not found > > ... > > ... > > ERROR: oe_runmake failed > > > > Cc: stable@vger.kernel.org > > What bug does this resolve that it needs to be backported to stable > kernels? This is not solving any existing actual bug, and therefore there is no fixes tag. The issue here is that the kernel build system is using lz4c, that is deprecated since 2018, and now distributions are actively moving away from it. openSUSE Tumbleweed and OE already removed it, so you would not be able to compile a stable kernel on such distribution when using lz4 unless we backport such a patch. Everything should be properly documented in the commit message already. My understanding is that something like that would be a reason for backporting to stable, if my understanding is not correct we'll remove the cc:stable and send a v3. Francesco
On Fri, Nov 15, 2024 at 09:39:40AM +0100, Francesco Dolcini wrote: > Hello Greg, > > On Thu, Nov 14, 2024 at 05:02:01PM +0100, Greg KH wrote: > > On Thu, Nov 14, 2024 at 03:56:44PM +0100, Parth Pancholi wrote: > > > From: Parth Pancholi <parth.pancholi@toradex.com> > > > > > > Replace lz4c with lz4 for kernel image compression. > > > Although lz4 and lz4c are functionally similar, lz4c has been deprecated > > > upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4 > > > and lz4c have been packaged together, making it safe to update the > > > requirement from lz4c to lz4. > > > > > > Consequently, some distributions and build systems, such as OpenEmbedded, > > > have fully transitioned to using lz4. OpenEmbedded core adopted this > > > change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of > > > lz4c"), causing compatibility issues when building the mainline kernel > > > in the latest OpenEmbedded environment, as seen in the errors below. > > > > > > This change also updates the LZ4 compression commands to make it backward > > > compatible by replacing stdin and stdout with the '-' option, due to some > > > unclear reason, the stdout keyword does not work for lz4 and '-' works for > > > both. In addition, this modifies the legacy '-c1' with '-9' which is also > > > compatible with both. This fixes the mainline kernel build failures with > > > the latest master OpenEmbedded builds associated with the mentioned > > > compatibility issues. > > > > > > LZ4 arch/arm/boot/compressed/piggy_data > > > /bin/sh: 1: lz4c: not found > > > ... > > > ... > > > ERROR: oe_runmake failed > > > > > > Cc: stable@vger.kernel.org > > > > What bug does this resolve that it needs to be backported to stable > > kernels? > > This is not solving any existing actual bug, and therefore there is no > fixes tag. > > The issue here is that the kernel build system is using lz4c, that is > deprecated since 2018, and now distributions are actively moving away from it. > > openSUSE Tumbleweed and OE already removed it, so you would not be able > to compile a stable kernel on such distribution when using lz4 unless we > backport such a patch. > > Everything should be properly documented in the commit message already. > > My understanding is that something like that would be a reason for > backporting to stable, if my understanding is not correct we'll remove > the cc:stable and send a v3. Please read: https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html for what meets stable kernel requirements. I don't think that this patch is that. thanks, greg k-h
On Fri, Nov 15, 2024 at 10:22:13AM +0100, Greg KH wrote: > On Fri, Nov 15, 2024 at 09:39:40AM +0100, Francesco Dolcini wrote: > > On Thu, Nov 14, 2024 at 05:02:01PM +0100, Greg KH wrote: > > > On Thu, Nov 14, 2024 at 03:56:44PM +0100, Parth Pancholi wrote: > > > > From: Parth Pancholi <parth.pancholi@toradex.com> > > > > > > > > Replace lz4c with lz4 for kernel image compression. > > > > Although lz4 and lz4c are functionally similar, lz4c has been deprecated > > > > upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4 > > > > and lz4c have been packaged together, making it safe to update the > > > > requirement from lz4c to lz4. > > > > > > > > Consequently, some distributions and build systems, such as OpenEmbedded, > > > > have fully transitioned to using lz4. OpenEmbedded core adopted this > > > > change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of > > > > lz4c"), causing compatibility issues when building the mainline kernel > > > > in the latest OpenEmbedded environment, as seen in the errors below. > > > > > > > > This change also updates the LZ4 compression commands to make it backward > > > > compatible by replacing stdin and stdout with the '-' option, due to some > > > > unclear reason, the stdout keyword does not work for lz4 and '-' works for > > > > both. In addition, this modifies the legacy '-c1' with '-9' which is also > > > > compatible with both. This fixes the mainline kernel build failures with > > > > the latest master OpenEmbedded builds associated with the mentioned > > > > compatibility issues. > > > > > > > > LZ4 arch/arm/boot/compressed/piggy_data > > > > /bin/sh: 1: lz4c: not found > > > > ... > > > > ... > > > > ERROR: oe_runmake failed > > > > > > > > Cc: stable@vger.kernel.org > > > > > > What bug does this resolve that it needs to be backported to stable > > > kernels? > > > > This is not solving any existing actual bug, and therefore there is no > > fixes tag. > > > > The issue here is that the kernel build system is using lz4c, that is > > deprecated since 2018, and now distributions are actively moving away from it. > > > > openSUSE Tumbleweed and OE already removed it, so you would not be able > > to compile a stable kernel on such distribution when using lz4 unless we > > backport such a patch. > > > > Everything should be properly documented in the commit message already. > > > > My understanding is that something like that would be a reason for > > backporting to stable, if my understanding is not correct we'll remove > > the cc:stable and send a v3. > > Please read: > https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html > for what meets stable kernel requirements. I don't think that this > patch is that. Greg, ack. Masahiro, can you please let me know if we should send a v3 with the stable tag removed or you can remove it yourself when applying? Francesco
On Fri, Nov 15, 2024 at 6:45 PM Francesco Dolcini <francesco@dolcini.it> wrote: > > On Fri, Nov 15, 2024 at 10:22:13AM +0100, Greg KH wrote: > > On Fri, Nov 15, 2024 at 09:39:40AM +0100, Francesco Dolcini wrote: > > > On Thu, Nov 14, 2024 at 05:02:01PM +0100, Greg KH wrote: > > > > On Thu, Nov 14, 2024 at 03:56:44PM +0100, Parth Pancholi wrote: > > > > > From: Parth Pancholi <parth.pancholi@toradex.com> > > > > > > > > > > Replace lz4c with lz4 for kernel image compression. > > > > > Although lz4 and lz4c are functionally similar, lz4c has been deprecated > > > > > upstream since 2018. Since as early as Ubuntu 16.04 and Fedora 25, lz4 > > > > > and lz4c have been packaged together, making it safe to update the > > > > > requirement from lz4c to lz4. > > > > > > > > > > Consequently, some distributions and build systems, such as OpenEmbedded, > > > > > have fully transitioned to using lz4. OpenEmbedded core adopted this > > > > > change in commit fe167e082cbd ("bitbake.conf: require lz4 instead of > > > > > lz4c"), causing compatibility issues when building the mainline kernel > > > > > in the latest OpenEmbedded environment, as seen in the errors below. > > > > > > > > > > This change also updates the LZ4 compression commands to make it backward > > > > > compatible by replacing stdin and stdout with the '-' option, due to some > > > > > unclear reason, the stdout keyword does not work for lz4 and '-' works for > > > > > both. In addition, this modifies the legacy '-c1' with '-9' which is also > > > > > compatible with both. This fixes the mainline kernel build failures with > > > > > the latest master OpenEmbedded builds associated with the mentioned > > > > > compatibility issues. > > > > > > > > > > LZ4 arch/arm/boot/compressed/piggy_data > > > > > /bin/sh: 1: lz4c: not found > > > > > ... > > > > > ... > > > > > ERROR: oe_runmake failed > > > > > > > > > > Cc: stable@vger.kernel.org > > > > > > > > What bug does this resolve that it needs to be backported to stable > > > > kernels? > > > > > > This is not solving any existing actual bug, and therefore there is no > > > fixes tag. > > > > > > The issue here is that the kernel build system is using lz4c, that is > > > deprecated since 2018, and now distributions are actively moving away from it. > > > > > > openSUSE Tumbleweed and OE already removed it, so you would not be able > > > to compile a stable kernel on such distribution when using lz4 unless we > > > backport such a patch. > > > > > > Everything should be properly documented in the commit message already. > > > > > > My understanding is that something like that would be a reason for > > > backporting to stable, if my understanding is not correct we'll remove > > > the cc:stable and send a v3. > > > > Please read: > > https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html > > for what meets stable kernel requirements. I don't think that this > > patch is that. > > Greg, ack. > > Masahiro, can you please let me know if we should send a v3 with the stable > tag removed or you can remove it yourself when applying? > I applied this with the stable tag removed. Thanks. (I guess someone may want to backport this eventually, as such distros cannot build stable kernels with ld4 compression.)
diff --git a/Makefile b/Makefile index 79192a3024bf..7630f763f5b2 100644 --- a/Makefile +++ b/Makefile @@ -508,7 +508,7 @@ KGZIP = gzip KBZIP2 = bzip2 KLZOP = lzop LZMA = lzma -LZ4 = lz4c +LZ4 = lz4 XZ = xz ZSTD = zstd diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 01a9f567d5af..fe5e132fcea8 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -371,10 +371,10 @@ quiet_cmd_lzo_with_size = LZO $@ cmd_lzo_with_size = { cat $(real-prereqs) | $(KLZOP) -9; $(size_append); } > $@ quiet_cmd_lz4 = LZ4 $@ - cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -c1 stdin stdout > $@ + cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -9 - - > $@ quiet_cmd_lz4_with_size = LZ4 $@ - cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -c1 stdin stdout; \ + cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -9 - -; \ $(size_append); } > $@ # U-Boot mkimage