| Message ID | 20230315173524.63179-2-bage@linutronix.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | kbuild: deb-pkg: Build parallely with current dpkg-buildpackage | expand |
On Thu, Mar 16, 2023 at 2:35 AM Bastian Germann <bage@linutronix.de> wrote: > > With dpkg-buildpackage v1.21.10 or later, the only way to build the > deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS > environment variable or using the --jobs-force option, see dpkg commit > 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to > non-force mode"). The package ignores the usual parallel build option > that is described in Debian Policy. > > Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable > that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied > from Debian Policy. > > Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options > Signed-off-by: Bastian Germann <bage@linutronix.de> > --- I was about to pick this up, then I stopped it. If I apply this patch, "make -j2 bindeb-pkg" shows a new warning. make[3]: warning: -j16 forced in makefile: resetting jobserver mode. Please note `nproc` returns 16 on my machine. "make bindeb-pkg" also runs in 16 threads because $(DEB_BUILD_OPTIONS) always contains "parallel=16" even if -j<N> option is given at all. In other words, this patch will take away user's freedom to choose the number of threads. > scripts/package/mkdebian | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian > index f74380036bb5..ed5c2b65798b 100755 > --- a/scripts/package/mkdebian > +++ b/scripts/package/mkdebian > @@ -238,6 +238,11 @@ fi > cat <<EOF > debian/rules > #!$(command -v $MAKE) -f > > +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) > + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) > + MAKEFLAGS += -j\$(NUMJOBS) > +endif > + > srctree ?= . > > build-indep: > -- > 2.39.2 >
Am 21.03.23 um 11:18 schrieb Masahiro Yamada: > "make bindeb-pkg" also runs in 16 threads because > $(DEB_BUILD_OPTIONS) always contains "parallel=16" > even if -j<N> option is given at all. > > In other words, this patch will take away user's freedom > to choose the number of threads. Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, which is now called with the defaults. I think you should see the warning with older dpkg-buildpackage versions without the patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in effect currently.
On Tue, Mar 21, 2023 at 7:28 PM Bastian Germann <bage@linutronix.de> wrote: > > Am 21.03.23 um 11:18 schrieb Masahiro Yamada: > > "make bindeb-pkg" also runs in 16 threads because > > $(DEB_BUILD_OPTIONS) always contains "parallel=16" > > even if -j<N> option is given at all. > > > > In other words, this patch will take away user's freedom > > to choose the number of threads. > > Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, > which is now called with the defaults. Yes. Agreed. > I think you should see the warning with older dpkg-buildpackage versions without the > patch applied as well because their default is dpkg-buildpackage -j`nproc` and is in > effect currently.
diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index f74380036bb5..ed5c2b65798b 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -238,6 +238,11 @@ fi cat <<EOF > debian/rules #!$(command -v $MAKE) -f +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + MAKEFLAGS += -j\$(NUMJOBS) +endif + srctree ?= . build-indep:
With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option, see dpkg commit 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to non-force mode"). The package ignores the usual parallel build option that is described in Debian Policy. Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied from Debian Policy. Link: https://www.debian.org/doc/debian-policy/ch-source.html#debian-rules-and-deb-build-options Signed-off-by: Bastian Germann <bage@linutronix.de> --- scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+)