| Message ID | 20240416143718.2857981-3-alan.maguire@oracle.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Delegated to: | BPF |
| Headers | show |
| Series | pahole: support nonstandard btf_features | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Not a local patch |
On Tue, Apr 16, 2024 at 03:37:17PM +0100, Alan Maguire wrote: > ...as a non-standard feature, so it will not be enabled for > "--btf_features=all" How did you test this? ⬢[acme@toolbox pahole]$ pahole --btf_features_strict=bgasd Feature 'bgasd' in 'bgasd' is not supported. Supported BTF features are: encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func,reproducible_build ⬢[acme@toolbox pahole]$ pahole -j --btf_features=all,reproducible_build --btf_encode_detached=vmlinux.btf.parallel.reproducible_build-via-btf_features vmlinux ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel.reproducible_build-via-btf_features > output.vmlinux.btf.parallel.reproducible_build-via-btf_features ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel.reproducible ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel.reproducible_build output.vmlinux.btf.parallel.reproducible_build-via-btf_features | head --- output.vmlinux.btf.parallel.reproducible_build 2024-04-16 12:20:28.513462223 -0300 +++ output.vmlinux.btf.parallel.reproducible_build-via-btf_features 2024-04-16 12:23:37.792962930 -0300 @@ -265,7 +265,7 @@ 'target' type_id=33 bits_offset=32 'key' type_id=43 bits_offset=64 [164] PTR '(anon)' type_id=163 -[165] PTR '(anon)' type_id=35751 +[165] PTR '(anon)' type_id=14983 [166] STRUCT 'static_key' size=16 vlen=2 'enabled' type_id=88 bits_offset=0 ⬢[acme@toolbox pahole]$ I'm double checking things now... - Arnaldo > Signed-off-by: Alan Maguire <alan.maguire@oracle.com> > --- > man-pages/pahole.1 | 8 ++++++++ > pahole.c | 1 + > 2 files changed, 9 insertions(+) > > diff --git a/man-pages/pahole.1 b/man-pages/pahole.1 > index 2c08e97..64de343 100644 > --- a/man-pages/pahole.1 > +++ b/man-pages/pahole.1 > @@ -310,6 +310,14 @@ Encode BTF using the specified feature list, or specify 'all' for all standard f > in different CUs. > .fi > > +Supported non-standard features (not enabled for 'all') > + > +.nf > + reproducible_build Ensure generated BTF is consistent every time; > + without this parallel BTF encoding can result in > + inconsistent BTF ids. > +.fi > + > So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. > > .TP > diff --git a/pahole.c b/pahole.c > index 890ef81..38cc636 100644 > --- a/pahole.c > +++ b/pahole.c > @@ -1286,6 +1286,7 @@ struct btf_feature { > BTF_FEATURE(enum64, skip_encoding_btf_enum64, true, true), > BTF_FEATURE(optimized_func, btf_gen_optimized, false, true), > BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false, true), > + BTF_FEATURE(reproducible_build, reproducible_build, false, false), > }; > > #define BTF_MAX_FEATURE_STR 1024
On 16/04/2024 16:24, Arnaldo Carvalho de Melo wrote: > On Tue, Apr 16, 2024 at 03:37:17PM +0100, Alan Maguire wrote: >> ...as a non-standard feature, so it will not be enabled for >> "--btf_features=all" > > How did you test this? > > ⬢[acme@toolbox pahole]$ pahole --btf_features_strict=bgasd > Feature 'bgasd' in 'bgasd' is not supported. Supported BTF features are: > encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func,reproducible_build > ⬢[acme@toolbox pahole]$ pahole -j --btf_features=all,reproducible_build --btf_encode_detached=vmlinux.btf.parallel.reproducible_build-via-btf_features vmlinux > ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel.reproducible_build-via-btf_features > output.vmlinux.btf.parallel.reproducible_build-via-btf_features > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel.reproducible > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel.reproducible_build output.vmlinux.btf.parallel.reproducible_build-via-btf_features | head > --- output.vmlinux.btf.parallel.reproducible_build 2024-04-16 12:20:28.513462223 -0300 > +++ output.vmlinux.btf.parallel.reproducible_build-via-btf_features 2024-04-16 12:23:37.792962930 -0300 > @@ -265,7 +265,7 @@ > 'target' type_id=33 bits_offset=32 > 'key' type_id=43 bits_offset=64 > [164] PTR '(anon)' type_id=163 > -[165] PTR '(anon)' type_id=35751 > +[165] PTR '(anon)' type_id=14983 > [166] STRUCT 'static_key' size=16 vlen=2 > 'enabled' type_id=88 bits_offset=0 > ⬢[acme@toolbox pahole]$ > > I'm double checking things now... > The test worked for me on x86_64/aarch64. Did you test with patch 3 applied? Because the test in its original state prior to patch 3 sets --reproducible_build before setting --btf_features=all, you won't get a reproducible build since the command line is saying "enable reproducible builds but also enable standard features only"; the second action undoes the first. switching to using --btf_features=all,reproducible_build fixes things for me. > - Arnaldo > >> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> >> --- >> man-pages/pahole.1 | 8 ++++++++ >> pahole.c | 1 + >> 2 files changed, 9 insertions(+) >> >> diff --git a/man-pages/pahole.1 b/man-pages/pahole.1 >> index 2c08e97..64de343 100644 >> --- a/man-pages/pahole.1 >> +++ b/man-pages/pahole.1 >> @@ -310,6 +310,14 @@ Encode BTF using the specified feature list, or specify 'all' for all standard f >> in different CUs. >> .fi >> >> +Supported non-standard features (not enabled for 'all') >> + >> +.nf >> + reproducible_build Ensure generated BTF is consistent every time; >> + without this parallel BTF encoding can result in >> + inconsistent BTF ids. >> +.fi >> + >> So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. >> >> .TP >> diff --git a/pahole.c b/pahole.c >> index 890ef81..38cc636 100644 >> --- a/pahole.c >> +++ b/pahole.c >> @@ -1286,6 +1286,7 @@ struct btf_feature { >> BTF_FEATURE(enum64, skip_encoding_btf_enum64, true, true), >> BTF_FEATURE(optimized_func, btf_gen_optimized, false, true), >> BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false, true), >> + BTF_FEATURE(reproducible_build, reproducible_build, false, false), >> }; >> >> #define BTF_MAX_FEATURE_STR 1024 > > >
On Tue, Apr 16, 2024 at 04:41:20PM +0100, Alan Maguire wrote: > On 16/04/2024 16:24, Arnaldo Carvalho de Melo wrote: > > On Tue, Apr 16, 2024 at 03:37:17PM +0100, Alan Maguire wrote: > >> ...as a non-standard feature, so it will not be enabled for > >> "--btf_features=all" > > > > How did you test this? > > > > ⬢[acme@toolbox pahole]$ pahole --btf_features_strict=bgasd > > Feature 'bgasd' in 'bgasd' is not supported. Supported BTF features are: > > encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func,reproducible_build > > ⬢[acme@toolbox pahole]$ pahole -j --btf_features=all,reproducible_build --btf_encode_detached=vmlinux.btf.parallel.reproducible_build-via-btf_features vmlinux > > ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel.reproducible_build-via-btf_features > output.vmlinux.btf.parallel.reproducible_build-via-btf_features > > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel.reproducible > > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel.reproducible_build output.vmlinux.btf.parallel.reproducible_build-via-btf_features | head > > --- output.vmlinux.btf.parallel.reproducible_build 2024-04-16 12:20:28.513462223 -0300 > > +++ output.vmlinux.btf.parallel.reproducible_build-via-btf_features 2024-04-16 12:23:37.792962930 -0300 > > @@ -265,7 +265,7 @@ > > 'target' type_id=33 bits_offset=32 > > 'key' type_id=43 bits_offset=64 > > [164] PTR '(anon)' type_id=163 > > -[165] PTR '(anon)' type_id=35751 > > +[165] PTR '(anon)' type_id=14983 > > [166] STRUCT 'static_key' size=16 vlen=2 > > 'enabled' type_id=88 bits_offset=0 > > ⬢[acme@toolbox pahole]$ > > > > I'm double checking things now... > The test worked for me on x86_64/aarch64. Did you test with patch 3 > applied? Because the test in its original state prior to patch 3 sets With all patches applied: ⬢[acme@toolbox pahole]$ git log --oneline -5 ecd3b0852ab1f1ff (HEAD -> master) tests/reproducible_build: use --btf_features=all,reproducible_build f9c8f5856b2aafea pahole: Add reproducible_build to --btf_features 0412978e8e6f8f76 pahole: Allow --btf_features to not participate in "all" a9738ddc828d5ea0 tests/reproducible_build: Use --btf_features=all when encoding d7edf9ae0388fb97 tests/reproducible_build: Validate the vmlinux file ⬢[acme@toolbox pahole]$ I made some mistake: ⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.serial --btf_features=all vmlinux ⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel --btf_features=all -j vmlinux ⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel--reproducible_build --btf_features=all --reproducible_build -j vmlinux ⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel--btf_features=all,reproducible_build --btf_features=all,reproducible_build -j vmlinux ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.serial > output.vmlinux.btf.serial ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel > output.vmlinux.btf.parallel ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel--reproducible_build > output.vmlinux.btf.parallel--reproducible_build ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel--btf_features=all,reproducible_build > output.vmlinux.btf.parallel--btf_features=all,reproducible_build ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel | wc -l 629165 ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel--reproducible_build | wc -l 0 ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel--btf_features=all,reproducible_build | wc -l 0 ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel--reproducible_build output.vmlinux.btf.parallel--btf_features=all,reproducible_build | wc -l 0 ⬢[acme@toolbox pahole]$ And of course: ⬢[acme@toolbox pahole]$ time tests/reproducible_build.sh vmlinux Parallel reproducible DWARF Loading/Serial BTF encoding: Ok real 1m25.241s user 3m10.144s sys 0m48.104s ⬢[acme@toolbox pahole]$ > --reproducible_build before setting --btf_features=all, you won't get a > reproducible build since the command line is saying "enable reproducible > builds but also enable standard features only"; the second action undoes Makes sense. > the first. switching to using --btf_features=all,reproducible_build > fixes things for me. Ok, all tested now, will push to the 'next' branch, waiting for Eduard's review. - Arnaldo > > - Arnaldo > > > >> Signed-off-by: Alan Maguire <alan.maguire@oracle.com> > >> --- > >> man-pages/pahole.1 | 8 ++++++++ > >> pahole.c | 1 + > >> 2 files changed, 9 insertions(+) > >> > >> diff --git a/man-pages/pahole.1 b/man-pages/pahole.1 > >> index 2c08e97..64de343 100644 > >> --- a/man-pages/pahole.1 > >> +++ b/man-pages/pahole.1 > >> @@ -310,6 +310,14 @@ Encode BTF using the specified feature list, or specify 'all' for all standard f > >> in different CUs. > >> .fi > >> > >> +Supported non-standard features (not enabled for 'all') > >> + > >> +.nf > >> + reproducible_build Ensure generated BTF is consistent every time; > >> + without this parallel BTF encoding can result in > >> + inconsistent BTF ids. > >> +.fi > >> + > >> So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. > >> > >> .TP > >> diff --git a/pahole.c b/pahole.c > >> index 890ef81..38cc636 100644 > >> --- a/pahole.c > >> +++ b/pahole.c > >> @@ -1286,6 +1286,7 @@ struct btf_feature { > >> BTF_FEATURE(enum64, skip_encoding_btf_enum64, true, true), > >> BTF_FEATURE(optimized_func, btf_gen_optimized, false, true), > >> BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false, true), > >> + BTF_FEATURE(reproducible_build, reproducible_build, false, false), > >> }; > >> > >> #define BTF_MAX_FEATURE_STR 1024 > > > > > >
diff --git a/man-pages/pahole.1 b/man-pages/pahole.1 index 2c08e97..64de343 100644 --- a/man-pages/pahole.1 +++ b/man-pages/pahole.1 @@ -310,6 +310,14 @@ Encode BTF using the specified feature list, or specify 'all' for all standard f in different CUs. .fi +Supported non-standard features (not enabled for 'all') + +.nf + reproducible_build Ensure generated BTF is consistent every time; + without this parallel BTF encoding can result in + inconsistent BTF ids. +.fi + So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. .TP diff --git a/pahole.c b/pahole.c index 890ef81..38cc636 100644 --- a/pahole.c +++ b/pahole.c @@ -1286,6 +1286,7 @@ struct btf_feature { BTF_FEATURE(enum64, skip_encoding_btf_enum64, true, true), BTF_FEATURE(optimized_func, btf_gen_optimized, false, true), BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false, true), + BTF_FEATURE(reproducible_build, reproducible_build, false, false), }; #define BTF_MAX_FEATURE_STR 1024
...as a non-standard feature, so it will not be enabled for "--btf_features=all" Signed-off-by: Alan Maguire <alan.maguire@oracle.com> --- man-pages/pahole.1 | 8 ++++++++ pahole.c | 1 + 2 files changed, 9 insertions(+)