diff mbox series

[12/12] tests: Add a BTF reproducible generation test

Message ID 20240412211604.789632-13-acme@kernel.org (mailing list archive)
State Not Applicable
Headers show
Series [01/12] core: Allow asking for a reproducible build | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch

Commit Message

Arnaldo Carvalho de Melo April 12, 2024, 9:16 p.m. UTC 
From: Arnaldo Carvalho de Melo <acme@redhat.com>

  $ time tests/reproducible_build.sh vmlinux
  Parallel reproducible DWARF Loading/Serial BTF encoding: Ok

  real  1m13.844s
  user  3m3.601s
  sys   0m9.049s
  $

If the number of threads started by pahole is different than what was
requests via its -j command line option, it will fail as well as if the
output of 'bpftool btf dump' differs from the BTF encoded totally
serially to one of the detached BTF encoded using reproducible DWARF
loading/BTF encoding.

Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Kui-Feng Lee <kuifeng@fb.com>
Cc: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tests/reproducible_build.sh | 56 +++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
 create mode 100755 tests/reproducible_build.sh

Comments

Alan Maguire April 15, 2024, 10:26 a.m. UTC | #1 
On 12/04/2024 22:16, Arnaldo Carvalho de Melo wrote:
> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
>   $ time tests/reproducible_build.sh vmlinux
>   Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
> 
>   real  1m13.844s
>   user  3m3.601s
>   sys   0m9.049s
>   $
> 
> If the number of threads started by pahole is different than what was
> requests via its -j command line option, it will fail as well as if the
> output of 'bpftool btf dump' differs from the BTF encoded totally
> serially to one of the detached BTF encoded using reproducible DWARF
> loading/BTF encoding.
> 
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Kui-Feng Lee <kuifeng@fb.com>
> Cc: Thomas Weißschuh <linux@weissschuh.net>
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> ---
>  tests/reproducible_build.sh | 56 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 56 insertions(+)
>  create mode 100755 tests/reproducible_build.sh
> 

great to have a test for this! a few small things below but
for the series

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Tested-by: Alan Maguire <alan.maguire@oracle.com>

> diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
> new file mode 100755
> index 0000000000000000..9c72d548c2a21136
> --- /dev/null
> +++ b/tests/reproducible_build.sh
> @@ -0,0 +1,56 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding
> +# Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
> +
> +vmlinux=$1

nit: might be worth having a usage check/error for the vmlinux binary here.

> +outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX)
> +
> +echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: "
> +
> +test -n "$VERBOSE" && printf "\nserial encoding...\n"
> +
> +pahole --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux

suggestion here; what about adding --btf_features=all as this would mean
we'd be encoding all the standard kernel features? we'd need to do the
same below in the thread loop. without that we're missing out on a few
features that the kernel builds use in BTF encoding, and we probably
want to ensure that we're testing as close to a real kernel BTF encoding
scenario as possible.

> +bpftool btf dump file $outdir/vmlinux.btf.serial > $outdir/bpftool.output.vmlinux.btf.serial
> +
> +nr_proc=$(getconf _NPROCESSORS_ONLN)
> +
> +for threads in $(seq $nr_proc) ; do
> +	test -n "$VERBOSE" && echo $threads threads encoding
> +	pahole -j$threads --reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
> +	pahole=$!
> +	# HACK: Wait a bit for pahole to start its threads
> +	sleep 0.3s
> +	# PID part to remove ps output headers
> +	nr_threads_started=$(ps -L -C pahole | grep -v PID | wc -l)
> +
> +	if [ $threads -gt 1 ] ; then
> +		((nr_threads_started -= 1))
> +	fi
> +
> +	if [ $threads != $nr_threads_started ] ; then
> +		echo "ERROR: pahole asked to start $threads encoding threads, started $nr_threads_started"
> +		exit 1;
> +	fi
> +
> +	# ps -L -C pahole | grep -v PID | nl
> +	test -n "$VERBOSE" && echo $nr_threads_started threads started
> +	wait $pahole
> +	rm -f $outdir/bpftool.output.vmlinux.btf.parallel.reproducible
> +	bpftool btf dump file $outdir/vmlinux.btf.parallel.reproducible > $outdir/bpftool.output.vmlinux.btf.parallel.reproducible
> +	test -n "$VERBOSE" && echo "diff from serial encoding:"
> +	diff -u $outdir/bpftool.output.vmlinux.btf.serial $outdir/bpftool.output.vmlinux.btf.parallel.reproducible > $outdir/diff
> +	if [ -s $outdir/diff ] ; then
> +		echo "ERROR: BTF generated from DWARF in parallel is different from the one generated in serial!"
> +		exit 1
> +	fi
> +	test -n "$VERBOSE" && echo -----------------------------
> +done
> +
> +rm $outdir/*
> +rmdir $outdir
> +
> +echo "Ok"
> +
> +exit 0
Arnaldo Carvalho de Melo April 15, 2024, 5:26 p.m. UTC | #2 
On Mon, Apr 15, 2024 at 11:26:44AM +0100, Alan Maguire wrote:
> On 12/04/2024 22:16, Arnaldo Carvalho de Melo wrote:
> > From: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> >   $ time tests/reproducible_build.sh vmlinux
> >   Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
> > 
> >   real  1m13.844s
> >   user  3m3.601s
> >   sys   0m9.049s
> >   $
> > 
> > If the number of threads started by pahole is different than what was
> > requests via its -j command line option, it will fail as well as if the
> > output of 'bpftool btf dump' differs from the BTF encoded totally
> > serially to one of the detached BTF encoded using reproducible DWARF
> > loading/BTF encoding.
> > 
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Kui-Feng Lee <kuifeng@fb.com>
> > Cc: Thomas Weißschuh <linux@weissschuh.net>
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > ---
> >  tests/reproducible_build.sh | 56 +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 56 insertions(+)
> >  create mode 100755 tests/reproducible_build.sh
> > 
> 
> great to have a test for this! a few small things below but
> for the series
> 
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> Tested-by: Alan Maguire <alan.maguire@oracle.com>
> 
> > diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
> > new file mode 100755
> > index 0000000000000000..9c72d548c2a21136
> > --- /dev/null
> > +++ b/tests/reproducible_build.sh
> > @@ -0,0 +1,56 @@
> > +#!/bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +#
> > +# Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding
> > +# Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
> > +
> > +vmlinux=$1
> 
> nit: might be worth having a usage check/error for the vmlinux binary here.

Right, I'm doing that now.

Suggested-by: Alan Maguire <alan.maguire@oracle.com>

⬢[acme@toolbox pahole]$ tests/reproducible_build.sh 
Please specify a vmlinux file to operate on
⬢[acme@toolbox pahole]$ tests/reproducible_build.sh vmlinux1
vmlinux1 file not available, please specify another
⬢[acme@toolbox pahole]$ tests/reproducible_build.sh vmlinux
Parallel reproducible DWARF Loading/Serial BTF encoding: ^C
⬢[acme@toolbox pahole]$

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
index 9c72d548c2a21136..b821e28e7ce7bf8c 100755
--- a/tests/reproducible_build.sh
+++ b/tests/reproducible_build.sh
@@ -5,6 +5,17 @@
 # Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
 
 vmlinux=$1
+
+if [ -z "$vmlinux" ] ; then
+	echo "Please specify a vmlinux file to operate on"
+	exit 1
+fi
+
+if [ ! -f "$vmlinux" ] ; then
+	echo "$vmlinux file not available, please specify another"
+	exit 1
+fi
+
 outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX)
 
 echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: "
Arnaldo Carvalho de Melo April 15, 2024, 5:33 p.m. UTC | #3 
On Mon, Apr 15, 2024 at 11:26:44AM +0100, Alan Maguire wrote:
> On 12/04/2024 22:16, Arnaldo Carvalho de Melo wrote:
> > From: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> >   $ time tests/reproducible_build.sh vmlinux
> >   Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
> > 
> >   real  1m13.844s
> >   user  3m3.601s
> >   sys   0m9.049s
> >   $
> > 
> > If the number of threads started by pahole is different than what was
> > requests via its -j command line option, it will fail as well as if the
> > output of 'bpftool btf dump' differs from the BTF encoded totally
> > serially to one of the detached BTF encoded using reproducible DWARF
> > loading/BTF encoding.
> > 
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Kui-Feng Lee <kuifeng@fb.com>
> > Cc: Thomas Weißschuh <linux@weissschuh.net>
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > ---
> >  tests/reproducible_build.sh | 56 +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 56 insertions(+)
> >  create mode 100755 tests/reproducible_build.sh
> > 
> 
> great to have a test for this! a few small things below but
> for the series
> 
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> Tested-by: Alan Maguire <alan.maguire@oracle.com>
> 
> > diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
> > new file mode 100755
> > index 0000000000000000..9c72d548c2a21136
> > --- /dev/null
> > +++ b/tests/reproducible_build.sh
> > @@ -0,0 +1,56 @@
> > +#!/bin/bash
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +#
> > +# Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding
> > +# Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
> > +
> > +vmlinux=$1
> 
> nit: might be worth having a usage check/error for the vmlinux binary here.
> 
> > +outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX)
> > +
> > +echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: "
> > +
> > +test -n "$VERBOSE" && printf "\nserial encoding...\n"
> > +
> > +pahole --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux
> 
> suggestion here; what about adding --btf_features=all as this would mean
> we'd be encoding all the standard kernel features? we'd need to do the
> same below in the thread loop. without that we're missing out on a few
> features that the kernel builds use in BTF encoding, and we probably
> want to ensure that we're testing as close to a real kernel BTF encoding
> scenario as possible.

⬢[acme@toolbox pahole]$ time tests/reproducible_build.sh vmlinux
Parallel reproducible DWARF Loading/Serial BTF encoding: Ok

real	1m24.903s
user	3m8.143s
sys	0m47.329s
⬢[acme@toolbox pahole]$ 
⬢[acme@toolbox pahole]$ time VERBOSE=1 tests/reproducible_build.sh vmlinux
Parallel reproducible DWARF Loading/Serial BTF encoding: 
serial encoding...
1 threads encoding
1 threads started
diff from serial encoding:
-----------------------------
2 threads encoding
2 threads started
diff from serial encoding:
-----------------------------
3 threads encoding
3 threads started
^X^C

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
index b821e28e7ce7bf8c..8cc36fe4c75e8b75 100755
--- a/tests/reproducible_build.sh
+++ b/tests/reproducible_build.sh
@@ -22,14 +22,14 @@ echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: "
 
 test -n "$VERBOSE" && printf "\nserial encoding...\n"
 
-pahole --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux
+pahole --btf_features=all --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux
 bpftool btf dump file $outdir/vmlinux.btf.serial > $outdir/bpftool.output.vmlinux.btf.serial
 
 nr_proc=$(getconf _NPROCESSORS_ONLN)
 
 for threads in $(seq $nr_proc) ; do
 	test -n "$VERBOSE" && echo $threads threads encoding
-	pahole -j$threads --reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
+	pahole -j$threads --reproducible_build --btf_features=all --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
 	pahole=$!
 	# HACK: Wait a bit for pahole to start its threads
 	sleep 0.3s
diff mbox series

Patch

diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh
new file mode 100755
index 0000000000000000..9c72d548c2a21136
--- /dev/null
+++ b/tests/reproducible_build.sh
@@ -0,0 +1,56 @@ 
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding
+# Arnaldo Carvalho de Melo <acme@redhat.com> (C) 2024-
+
+vmlinux=$1
+outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX)
+
+echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: "
+
+test -n "$VERBOSE" && printf "\nserial encoding...\n"
+
+pahole --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux
+bpftool btf dump file $outdir/vmlinux.btf.serial > $outdir/bpftool.output.vmlinux.btf.serial
+
+nr_proc=$(getconf _NPROCESSORS_ONLN)
+
+for threads in $(seq $nr_proc) ; do
+	test -n "$VERBOSE" && echo $threads threads encoding
+	pahole -j$threads --reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux &
+	pahole=$!
+	# HACK: Wait a bit for pahole to start its threads
+	sleep 0.3s
+	# PID part to remove ps output headers
+	nr_threads_started=$(ps -L -C pahole | grep -v PID | wc -l)
+
+	if [ $threads -gt 1 ] ; then
+		((nr_threads_started -= 1))
+	fi
+
+	if [ $threads != $nr_threads_started ] ; then
+		echo "ERROR: pahole asked to start $threads encoding threads, started $nr_threads_started"
+		exit 1;
+	fi
+
+	# ps -L -C pahole | grep -v PID | nl
+	test -n "$VERBOSE" && echo $nr_threads_started threads started
+	wait $pahole
+	rm -f $outdir/bpftool.output.vmlinux.btf.parallel.reproducible
+	bpftool btf dump file $outdir/vmlinux.btf.parallel.reproducible > $outdir/bpftool.output.vmlinux.btf.parallel.reproducible
+	test -n "$VERBOSE" && echo "diff from serial encoding:"
+	diff -u $outdir/bpftool.output.vmlinux.btf.serial $outdir/bpftool.output.vmlinux.btf.parallel.reproducible > $outdir/diff
+	if [ -s $outdir/diff ] ; then
+		echo "ERROR: BTF generated from DWARF in parallel is different from the one generated in serial!"
+		exit 1
+	fi
+	test -n "$VERBOSE" && echo -----------------------------
+done
+
+rm $outdir/*
+rmdir $outdir
+
+echo "Ok"
+
+exit 0