diff mbox series

[v6,11/12] kbuild: deb-pkg: improve the usability of source package

Message ID 20230215012034.403356-11-masahiroy@kernel.org (mailing list archive)
State New, archived
Headers show
Series [v6,01/12] kbuild: add a tool to list files ignored by git | expand

Commit Message

Masahiro Yamada Feb. 15, 2023, 1:20 a.m. UTC 
Improve the source package support in case the dpkg-buildpackage is
directly used to build binary packages.

For cross-compiling, you can set CROSS_COMPILE via the environment
variable, but it is better to set it automatically - set it to
${DEB_HOST_GNU_TYPE}- if we are cross-compiling but not from the top
Makefile.

The generated source package may be carried to a different build
environment, which may have a different compiler installed.
Run olddefconfig first to set new CONFIG options to their default
values without prompting.

Take KERNELRELEASE and KBUILD_BUILD_VERSION from the version field of
debian/changelog in case it is updated afterwards.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

Changes in v6:
  - New patch

 scripts/package/deb-build-option | 16 ++++++++++++++++
 scripts/package/mkdebian         |  9 ++++-----
 2 files changed, 20 insertions(+), 5 deletions(-)
 create mode 100755 scripts/package/deb-build-option

Comments

Peter Ujfalusi March 8, 2023, 12:29 p.m. UTC | #1 
On 15/02/2023 03:20, Masahiro Yamada wrote:
> Improve the source package support in case the dpkg-buildpackage is
> directly used to build binary packages.
> 
> For cross-compiling, you can set CROSS_COMPILE via the environment
> variable, but it is better to set it automatically - set it to
> ${DEB_HOST_GNU_TYPE}- if we are cross-compiling but not from the top
> Makefile.
> 
> The generated source package may be carried to a different build
> environment, which may have a different compiler installed.
> Run olddefconfig first to set new CONFIG options to their default
> values without prompting.
> 
> Take KERNELRELEASE and KBUILD_BUILD_VERSION from the version field of
> debian/changelog in case it is updated afterwards.

I just arrived to this patch when trying to update the kernel for SOF
from 6.2.0 base to 6.3.0-rc1:
https://github.com/thesofproject/linux/pull/4223#issuecomment-1459997676

Our CI is building a kernel deb package from PRs to test with this command:
make bindeb-pkg LOCALVERSION=-$PRIDV-default KDEB_PKGVERSION=$(make
kernelversion)-$GITSHA

The files, directories and the name of the deb package follows the rule
set up by scripts/setlocalversion, taking into account the LOCALVERSION,
CONFIG_LOCALVERSION and CONFIG_LOCALVERSION_AUTO, but the built kernel
is using the version derived from KDEB_PKGVERSION and the system is not
able to boot up since the modules are in a different directory then they
should be.

Something like this:
make bindeb-pkg LOCALVERSION=-testbuild
vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty
/lib/modules/6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty/

strings vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty | grep 6.3
6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty

Then with the KDEB_PKGVERSION:
make bindeb-pkg LOCALVERSION=-testbuild KDEB_PKGVERSION=$(make
kernelversion)-notworking
vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty
/lib/modules/6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty/

strings vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty | grep 6.3
6.3.0-rc1

in this case modprobe will not find the modules, it is looking for them
under /lib/modules/6.3.0-rc1

Reverting this patch makes the created  deb package usable, the system
is bootable.

We use the LOCALVERSION to feed in PR inforamtion and the
KDEB_PKGVERSION is used fro additional information, like build ID.

Do you have any suggestion on how to get the functionality back or we
must stop using the KDEB_PKGVERSION altogether?

Thank you,
Masahiro Yamada March 8, 2023, 5:01 p.m. UTC | #2 
On Wed, Mar 8, 2023 at 9:29 PM Péter Ujfalusi
<peter.ujfalusi@linux.intel.com> wrote:
>
>
>
> On 15/02/2023 03:20, Masahiro Yamada wrote:
> > Improve the source package support in case the dpkg-buildpackage is
> > directly used to build binary packages.
> >
> > For cross-compiling, you can set CROSS_COMPILE via the environment
> > variable, but it is better to set it automatically - set it to
> > ${DEB_HOST_GNU_TYPE}- if we are cross-compiling but not from the top
> > Makefile.
> >
> > The generated source package may be carried to a different build
> > environment, which may have a different compiler installed.
> > Run olddefconfig first to set new CONFIG options to their default
> > values without prompting.
> >
> > Take KERNELRELEASE and KBUILD_BUILD_VERSION from the version field of
> > debian/changelog in case it is updated afterwards.
>
> I just arrived to this patch when trying to update the kernel for SOF
> from 6.2.0 base to 6.3.0-rc1:
> https://github.com/thesofproject/linux/pull/4223#issuecomment-1459997676
>
> Our CI is building a kernel deb package from PRs to test with this command:
> make bindeb-pkg LOCALVERSION=-$PRIDV-default KDEB_PKGVERSION=$(make
> kernelversion)-$GITSHA
>
> The files, directories and the name of the deb package follows the rule
> set up by scripts/setlocalversion, taking into account the LOCALVERSION,
> CONFIG_LOCALVERSION and CONFIG_LOCALVERSION_AUTO, but the built kernel
> is using the version derived from KDEB_PKGVERSION and the system is not
> able to boot up since the modules are in a different directory then they
> should be.
>
> Something like this:
> make bindeb-pkg LOCALVERSION=-testbuild
> vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty
> /lib/modules/6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty/
>
> strings vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty | grep 6.3
> 6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty
>
> Then with the KDEB_PKGVERSION:
> make bindeb-pkg LOCALVERSION=-testbuild KDEB_PKGVERSION=$(make
> kernelversion)-notworking
> vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty
> /lib/modules/6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty/
>
> strings vmlinuz-6.3.0-rc1-testbuild-05827-g863ad81a6aa9-dirty | grep 6.3
> 6.3.0-rc1
>
> in this case modprobe will not find the modules, it is looking for them
> under /lib/modules/6.3.0-rc1
>
> Reverting this patch makes the created  deb package usable, the system
> is bootable.
>
> We use the LOCALVERSION to feed in PR inforamtion and the
> KDEB_PKGVERSION is used fro additional information, like build ID.
>
> Do you have any suggestion on how to get the functionality back or we
> must stop using the KDEB_PKGVERSION altogether?


I am aware of this issue.

KERNELVERSION does not need to match the package version.

I should have taken it from the <abiver> in the package name
(linux-image-<abiver>_<origver>-<rev>_<arch>.deb)
if I had wanted to do this dynamically by debian/control.

I wrongly took it from <origver>.

I will change this when I solve another issue required by Linus Tolvalds.


Please wait.






>
> Thank you,
> --
> Péter
>
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> > Changes in v6:
> >   - New patch
> >
> >  scripts/package/deb-build-option | 16 ++++++++++++++++
> >  scripts/package/mkdebian         |  9 ++++-----
> >  2 files changed, 20 insertions(+), 5 deletions(-)
> >  create mode 100755 scripts/package/deb-build-option
> >
> > diff --git a/scripts/package/deb-build-option b/scripts/package/deb-build-option
> > new file mode 100755
> > index 000000000000..b079b0d121d4
> > --- /dev/null
> > +++ b/scripts/package/deb-build-option
> > @@ -0,0 +1,16 @@
> > +#!/bin/sh
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +
> > +# Set up CROSS_COMPILE if we are cross-compiling, but not called from the
> > +# kernel toplevel Makefile
> > +if [ -z "${CROSS_COMPILE}${cross_compiling}" -a "${DEB_HOST_ARCH}" != "${DEB_BUILD_ARCH}" ]; then
> > +     echo CROSS_COMPILE=${DEB_HOST_GNU_TYPE}-
> > +fi
> > +
> > +version=$(dpkg-parsechangelog -S Version)
> > +version_upstream="${version%-*}"
> > +debian_revision="${version#${version_upstream}}"
> > +debian_revision="${debian_revision#*-}"
> > +
> > +echo KERNELRELEASE=${version_upstream}
> > +echo KBUILD_BUILD_VERSION=${debian_revision}
> > diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
> > index eef20d5dc288..f74380036bb5 100755
> > --- a/scripts/package/mkdebian
> > +++ b/scripts/package/mkdebian
> > @@ -90,10 +90,8 @@ rm -rf debian
> >  version=$KERNELRELEASE
> >  if [ -n "$KDEB_PKGVERSION" ]; then
> >       packageversion=$KDEB_PKGVERSION
> > -     revision=${packageversion##*-}
> >  else
> > -     revision=$($srctree/init/build-version)
> > -     packageversion=$version-$revision
> > +     packageversion=$version-$($srctree/init/build-version)
> >  fi
> >  sourcename=${KDEB_SOURCENAME:-linux-upstream}
> >
> > @@ -244,8 +242,9 @@ srctree ?= .
> >
> >  build-indep:
> >  build-arch:
> > -     \$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
> > -     KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile
> > +     \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \
> > +     \$(shell \$(srctree)/scripts/package/deb-build-option) \
> > +     olddefconfig all
> >
> >  build: build-arch
> >
>
> --
> Péter
diff mbox series

Patch

diff --git a/scripts/package/deb-build-option b/scripts/package/deb-build-option
new file mode 100755
index 000000000000..b079b0d121d4
--- /dev/null
+++ b/scripts/package/deb-build-option
@@ -0,0 +1,16 @@ 
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+
+# Set up CROSS_COMPILE if we are cross-compiling, but not called from the
+# kernel toplevel Makefile
+if [ -z "${CROSS_COMPILE}${cross_compiling}" -a "${DEB_HOST_ARCH}" != "${DEB_BUILD_ARCH}" ]; then
+	echo CROSS_COMPILE=${DEB_HOST_GNU_TYPE}-
+fi
+
+version=$(dpkg-parsechangelog -S Version)
+version_upstream="${version%-*}"
+debian_revision="${version#${version_upstream}}"
+debian_revision="${debian_revision#*-}"
+
+echo KERNELRELEASE=${version_upstream}
+echo KBUILD_BUILD_VERSION=${debian_revision}
diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian
index eef20d5dc288..f74380036bb5 100755
--- a/scripts/package/mkdebian
+++ b/scripts/package/mkdebian
@@ -90,10 +90,8 @@  rm -rf debian
 version=$KERNELRELEASE
 if [ -n "$KDEB_PKGVERSION" ]; then
 	packageversion=$KDEB_PKGVERSION
-	revision=${packageversion##*-}
 else
-	revision=$($srctree/init/build-version)
-	packageversion=$version-$revision
+	packageversion=$version-$($srctree/init/build-version)
 fi
 sourcename=${KDEB_SOURCENAME:-linux-upstream}
 
@@ -244,8 +242,9 @@  srctree ?= .
 
 build-indep:
 build-arch:
-	\$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
-	KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile
+	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \
+	\$(shell \$(srctree)/scripts/package/deb-build-option) \
+	olddefconfig all
 
 build: build-arch