| Message ID | e9c64e9b5c073dabd457ff45128aabcab7630098.1717477560.git.Tony.Ambardar@gmail.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 7bdcedd5c8fb88e7176b93812b139eca5fe0aa46 |
| Delegated to: | BPF |
| Headers | show |
| Series | bpf: Fix linker optimization removing kfuncs | expand |
On Mon, Jun 03, 2024 at 10:23:16PM -0700, Tony Ambardar wrote: > BPF kfuncs are often not directly referenced and may be inadvertently > removed by optimization steps during kernel builds, thus the __bpf_kfunc > tag mitigates against this removal by including the __used macro. However, > this macro alone does not prevent removal during linking, and may still > yield build warnings (e.g. on mips64el): > > LD vmlinux > BTFIDS vmlinux > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key > WARN: resolve_btfids: unresolved symbol bpf_key_put > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages > NM System.map > SORTTAB vmlinux > OBJCOPY vmlinux.32 > > Update the __bpf_kfunc tag to better guard against linker optimization by > including the new __retain compiler macro, which fixes the warnings above. > > Verify the __retain macro with readelf by checking object flags for 'R': > > $ readelf -Wa kernel/trace/bpf_trace.o > Section Headers: > [Nr] Name Type Address Off Size ES Flg Lk Inf Al > ... > [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 > ... > Key to Flags: > ... > R (retain), D (mbind), p (processor specific) > > Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ > Reported-by: kernel test robot <lkp@intel.com> > Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ > Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") > Cc: stable@vger.kernel.org # v6.6+ > Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com> tested on mips64 cross build and the warnings are gone and related functions are in the vmlinux patchset looks good to me Tested-by: Jiri Olsa <jolsa@kernel.org> Reviewed-by: Jiri Olsa <jolsa@kernel.org> thanks, jirka > --- > include/linux/btf.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/include/linux/btf.h b/include/linux/btf.h > index f9e56fd12a9f..7c3e40c3295e 100644 > --- a/include/linux/btf.h > +++ b/include/linux/btf.h > @@ -82,7 +82,7 @@ > * as to avoid issues such as the compiler inlining or eliding either a static > * kfunc, or a global kfunc in an LTO build. > */ > -#define __bpf_kfunc __used noinline > +#define __bpf_kfunc __used __retain noinline > > #define __bpf_kfunc_start_defs() \ > __diag_push(); \ > -- > 2.34.1 >
Hi Tony, On Mon, 3 Jun 2024, Tony Ambardar wrote: > BPF kfuncs are often not directly referenced and may be inadvertently > removed by optimization steps during kernel builds, thus the __bpf_kfunc > tag mitigates against this removal by including the __used macro. However, > this macro alone does not prevent removal during linking, and may still > yield build warnings (e.g. on mips64el): > > LD vmlinux > BTFIDS vmlinux > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key > WARN: resolve_btfids: unresolved symbol bpf_key_put > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages > NM System.map > SORTTAB vmlinux > OBJCOPY vmlinux.32 > > Update the __bpf_kfunc tag to better guard against linker optimization by > including the new __retain compiler macro, which fixes the warnings above. > > Verify the __retain macro with readelf by checking object flags for 'R': > > $ readelf -Wa kernel/trace/bpf_trace.o > Section Headers: > [Nr] Name Type Address Off Size ES Flg Lk Inf Al > ... > [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 > ... > Key to Flags: > ... > R (retain), D (mbind), p (processor specific) > > Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ > Reported-by: kernel test robot <lkp@intel.com> > Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ > Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") > Cc: stable@vger.kernel.org # v6.6+ > Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com> Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in v6.10-rc5. This is causing build failures on ARM with CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y: net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11859 | { | ^ net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11872 | { | ^ net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11885 | { | ^ net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] 11906 | { | ^ net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] 12092 | { | ^ net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] 713 | { | ^ net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] 736 | { | ^ net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] 769 | { | ^ [...] My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04). > --- a/include/linux/btf.h > +++ b/include/linux/btf.h > @@ -82,7 +82,7 @@ > * as to avoid issues such as the compiler inlining or eliding either a static > * kfunc, or a global kfunc in an LTO build. > */ > -#define __bpf_kfunc __used noinline > +#define __bpf_kfunc __used __retain noinline > > #define __bpf_kfunc_start_defs() \ > __diag_push(); \ Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
On Tue, Jun 25, 2024 at 12:46:48PM +0200, Geert Uytterhoeven wrote: > Hi Tony, > > On Mon, 3 Jun 2024, Tony Ambardar wrote: > > BPF kfuncs are often not directly referenced and may be inadvertently > > removed by optimization steps during kernel builds, thus the __bpf_kfunc > > tag mitigates against this removal by including the __used macro. However, > > this macro alone does not prevent removal during linking, and may still > > yield build warnings (e.g. on mips64el): > > > > LD vmlinux > > BTFIDS vmlinux > > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature > > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key > > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key > > WARN: resolve_btfids: unresolved symbol bpf_key_put > > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next > > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new > > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr > > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire > > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages > > NM System.map > > SORTTAB vmlinux > > OBJCOPY vmlinux.32 > > > > Update the __bpf_kfunc tag to better guard against linker optimization by > > including the new __retain compiler macro, which fixes the warnings above. > > > > Verify the __retain macro with readelf by checking object flags for 'R': > > > > $ readelf -Wa kernel/trace/bpf_trace.o > > Section Headers: > > [Nr] Name Type Address Off Size ES Flg Lk Inf Al > > ... > > [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 > > ... > > Key to Flags: > > ... > > R (retain), D (mbind), p (processor specific) > > > > Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ > > Reported-by: kernel test robot <lkp@intel.com> > > Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ > > Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") > > Cc: stable@vger.kernel.org # v6.6+ > > Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com> > > Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 > ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in > v6.10-rc5. > > This is causing build failures on ARM with > CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y: > > net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 11859 | { > | ^ > net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 11872 | { > | ^ > net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 11885 | { > | ^ > net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 11906 | { > | ^ > net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 12092 | { > | ^ > net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 713 | { > | ^ > net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 736 | { > | ^ > net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] > 769 | { > | ^ > [...] > > My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04). hum, so it'd mean __has_attribute(__retain__) returns true while gcc still ignores the retain attribute.. like in this bug which seems similar: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587 but not sure how it got fixed.. any chance you can upgrade gcc and retest? jirka > > > --- a/include/linux/btf.h > > +++ b/include/linux/btf.h > > @@ -82,7 +82,7 @@ > > * as to avoid issues such as the compiler inlining or eliding either a static > > * kfunc, or a global kfunc in an LTO build. > > */ > > -#define __bpf_kfunc __used noinline > > +#define __bpf_kfunc __used __retain noinline > > > > #define __bpf_kfunc_start_defs() \ > > __diag_push(); \ > > Gr{oetje,eeting}s, > > Geert > > -- > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org > > In personal conversations with technical people, I call myself a hacker. But > when I'm talking to journalists I just say "programmer" or something like that. > -- Linus Torvalds
Hi Jiri, On Wed, Jun 26, 2024 at 11:52 AM Jiri Olsa <olsajiri@gmail.com> wrote: > On Tue, Jun 25, 2024 at 12:46:48PM +0200, Geert Uytterhoeven wrote: > > On Mon, 3 Jun 2024, Tony Ambardar wrote: > > > BPF kfuncs are often not directly referenced and may be inadvertently > > > removed by optimization steps during kernel builds, thus the __bpf_kfunc > > > tag mitigates against this removal by including the __used macro. However, > > > this macro alone does not prevent removal during linking, and may still > > > yield build warnings (e.g. on mips64el): > > > > > > LD vmlinux > > > BTFIDS vmlinux > > > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature > > > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key > > > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key > > > WARN: resolve_btfids: unresolved symbol bpf_key_put > > > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next > > > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new > > > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr > > > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire > > > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages > > > NM System.map > > > SORTTAB vmlinux > > > OBJCOPY vmlinux.32 > > > > > > Update the __bpf_kfunc tag to better guard against linker optimization by > > > including the new __retain compiler macro, which fixes the warnings above. > > > > > > Verify the __retain macro with readelf by checking object flags for 'R': > > > > > > $ readelf -Wa kernel/trace/bpf_trace.o > > > Section Headers: > > > [Nr] Name Type Address Off Size ES Flg Lk Inf Al > > > ... > > > [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 > > > ... > > > Key to Flags: > > > ... > > > R (retain), D (mbind), p (processor specific) > > > > > > Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ > > > Reported-by: kernel test robot <lkp@intel.com> > > > Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ > > > Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") > > > Cc: stable@vger.kernel.org # v6.6+ > > > Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com> > > > > Thanks for your patch, which is now commit 7bdcedd5c8fb88e7 > > ("bpf: Harden __bpf_kfunc tag against linker kfunc removal") in > > v6.10-rc5. > > > > This is causing build failures on ARM with > > CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y: > > > > net/core/filter.c:11859:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 11859 | { > > | ^ > > net/core/filter.c:11872:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 11872 | { > > | ^ > > net/core/filter.c:11885:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 11885 | { > > | ^ > > net/core/filter.c:11906:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 11906 | { > > | ^ > > net/core/filter.c:12092:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 12092 | { > > | ^ > > net/core/xdp.c:713:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 713 | { > > | ^ > > net/core/xdp.c:736:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 736 | { > > | ^ > > net/core/xdp.c:769:1: error: ‘retain’ attribute ignored [-Werror=attributes] > > 769 | { > > | ^ > > [...] > > > > My compiler is arm-linux-gnueabihf-gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04). > > hum, so it'd mean __has_attribute(__retain__) returns true while gcc still > ignores the retain attribute.. like in this bug which seems similar: > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587 > but not sure how it got fixed.. any chance you can upgrade gcc and retest? Indeed, __has_attribute(__retain__) returns true, while the attribute is not supported. My test program: cat > /tmp/a.c <<EOF #if __has_attribute(__retain__) #warning __retain__ OK #else #warning No __retain__ #endif int x __attribute__((__retain__)); EOF $ arm-linux-gnueabihf-gcc-11 -c /tmp/a.c # gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)) /tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~ /tmp/a.c:7:1: warning: ‘retain’ attribute ignored [-Wattributes] 7 | int x __attribute__((__retain__)); | ^~~ Oops $ arm-linux-gnueabihf-gcc-12 -c /tmp/a.c # gcc version 12.3.0 (Ubuntu 12.3.0-1ubuntu1~22.04) /tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~ Fixed It works fine with the native gcc-11: $ gcc-11 -c /tmp/a.c # gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04) /tmp/a.c:2:2: warning: #warning __retain__ OK [-Wcpp] 2 | #warning __retain__ OK | ^~~~~~~ I gave it a try on all installed gcc-11 compilers. /usr/bin/aarch64-linux-gnu-gcc-11 /usr/bin/alpha-linux-gnu-gcc-11 /usr/bin/arm-linux-gnueabi-gcc-11 /usr/bin/arm-linux-gnueabihf-gcc-11 /usr/bin/hppa64-linux-gnu-gcc-11 /usr/bin/hppa-linux-gnu-gcc-11 /usr/bin/m68k-linux-gnu-gcc-11 /usr/bin/powerpc64le-linux-gnu-gcc-11 /usr/bin/powerpc64-linux-gnu-gcc-11 /usr/bin/powerpc-linux-gnu-gcc-11 /usr/bin/riscv64-linux-gnu-gcc-11 /usr/bin/s390x-linux-gnu-gcc-11 /usr/bin/sh4-linux-gnu-gcc-11 /usr/bin/sparc64-linux-gnu-gcc-11 /usr/bin/x86_64-linux-gnu-gcc-11 /usr/bin/x86_64-linux-gnux32-gcc-11 All of them failed (incl. x32), except for the native x86_64-linux-gnu-gcc-11. It works fine with all installed gcc-12 compilers (arm-linux-gnueabihf-gcc-12, m68k-linux-gnu-gcc-12, x86_64-linux-gnu-gcc-12). With gcc-9, the absence of __retain__ is detected correctly. Gr{oetje,eeting}s, Geert
diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..7c3e40c3295e 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -82,7 +82,7 @@ * as to avoid issues such as the compiler inlining or eliding either a static * kfunc, or a global kfunc in an LTO build. */ -#define __bpf_kfunc __used noinline +#define __bpf_kfunc __used __retain noinline #define __bpf_kfunc_start_defs() \ __diag_push(); \
BPF kfuncs are often not directly referenced and may be inadvertently removed by optimization steps during kernel builds, thus the __bpf_kfunc tag mitigates against this removal by including the __used macro. However, this macro alone does not prevent removal during linking, and may still yield build warnings (e.g. on mips64el): LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages NM System.map SORTTAB vmlinux OBJCOPY vmlinux.32 Update the __bpf_kfunc tag to better guard against linker optimization by including the new __retain compiler macro, which fixes the warnings above. Verify the __retain macro with readelf by checking object flags for 'R': $ readelf -Wa kernel/trace/bpf_trace.o Section Headers: [Nr] Name Type Address Off Size ES Flg Lk Inf Al ... [178] .text.bpf_key_put PROGBITS 00000000 6420 0050 00 AXR 0 0 8 ... Key to Flags: ... R (retain), D (mbind), p (processor specific) Link: https://lore.kernel.org/bpf/ZlmGoT9KiYLZd91S@krava/T/ Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/r/202401211357.OCX9yllM-lkp@intel.com/ Fixes: 57e7c169cd6a ("bpf: Add __bpf_kfunc tag for marking kernel functions as kfuncs") Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Tony Ambardar <Tony.Ambardar@gmail.com> --- include/linux/btf.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)