diff mbox series

bpftool: Fix failure with static linkage

Message ID 20241204213059.2792453-1-leo.yan@arm.com (mailing list archive)
State Changes Requested
Delegated to: BPF
Headers show
Series bpftool: Fix failure with static linkage | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch
bpf/vmtest-bpf-next-VM_Test-30 success Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-31 success Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2
bpf/vmtest-bpf-next-VM_Test-35 success Logs for x86_64-llvm-17 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-37 success Logs for x86_64-llvm-17 / veristat-meta
bpf/vmtest-bpf-next-VM_Test-39 success Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2
bpf/vmtest-bpf-next-VM_Test-38 success Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-46 success Logs for x86_64-llvm-18 / veristat-meta
bpf/vmtest-bpf-next-VM_Test-45 success Logs for x86_64-llvm-18 / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-36 success Logs for x86_64-llvm-17 / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-32 success Logs for x86_64-llvm-17 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-33 success Logs for x86_64-llvm-17 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-34 success Logs for x86_64-llvm-17 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-44 success Logs for x86_64-llvm-18 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-40 success Logs for x86_64-llvm-18 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-42 success Logs for x86_64-llvm-18 / test (test_progs_cpuv4, false, 360) / test_progs_cpuv4 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-41 success Logs for x86_64-llvm-18 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-43 success Logs for x86_64-llvm-18 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-0 success Logs for Lint
bpf/vmtest-bpf-next-PR fail PR summary
bpf/vmtest-bpf-next-VM_Test-3 success Logs for Validate matrix.py
bpf/vmtest-bpf-next-VM_Test-2 success Logs for Unittests
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-6 success Logs for aarch64-gcc / test
bpf/vmtest-bpf-next-VM_Test-4 fail Logs for aarch64-gcc / build / build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-8 success Logs for aarch64-gcc / veristat-meta
bpf/vmtest-bpf-next-VM_Test-5 success Logs for aarch64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-7 success Logs for aarch64-gcc / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-10 success Logs for s390x-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-11 success Logs for s390x-gcc / test
bpf/vmtest-bpf-next-VM_Test-9 fail Logs for s390x-gcc / build / build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-12 success Logs for s390x-gcc / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-13 success Logs for s390x-gcc / veristat-meta
bpf/vmtest-bpf-next-VM_Test-16 success Logs for x86_64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-14 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-15 fail Logs for x86_64-gcc / build / build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-18 success Logs for x86_64-gcc / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-17 success Logs for x86_64-gcc / test
bpf/vmtest-bpf-next-VM_Test-20 fail Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-19 success Logs for x86_64-gcc / veristat-meta
bpf/vmtest-bpf-next-VM_Test-22 success Logs for x86_64-llvm-17 / test
bpf/vmtest-bpf-next-VM_Test-21 fail Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2
bpf/vmtest-bpf-next-VM_Test-23 success Logs for x86_64-llvm-17 / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-25 fail Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-27 success Logs for x86_64-llvm-18 / test
bpf/vmtest-bpf-next-VM_Test-24 success Logs for x86_64-llvm-17 / veristat-meta
bpf/vmtest-bpf-next-VM_Test-26 fail Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2
bpf/vmtest-bpf-next-VM_Test-28 success Logs for x86_64-llvm-18 / veristat-kernel
bpf/vmtest-bpf-next-VM_Test-29 success Logs for x86_64-llvm-18 / veristat-meta

Commit Message

Leo Yan Dec. 4, 2024, 9:30 p.m. UTC
When building perf with static linkage:

  make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
  ...
  LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
  (.text+0x113): undefined reference to `ZSTD_createCCtx'
  /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
  /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
  /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
  /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
  /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
  (.text+0xbfc): undefined reference to `ZSTD_decompress'
  /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
  (.text+0xd45): undefined reference to `ZSTD_decompress'
  /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
  collect2: error: ld returned 1 exit status

Building bpftool with static linkage also fails with the same errors:

  make O=/build -C tools/bpf/bpftool/ V=1

To fix the issue, explicitly link libzstd.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/bpf/bpftool/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Namhyung Kim Dec. 4, 2024, 9:36 p.m. UTC | #1
Hi Leo,

On Wed, Dec 04, 2024 at 09:30:59PM +0000, Leo Yan wrote:
> When building perf with static linkage:
> 
>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
>   ...
>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
>   (.text+0xd45): undefined reference to `ZSTD_decompress'
>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
>   collect2: error: ld returned 1 exit status
> 
> Building bpftool with static linkage also fails with the same errors:
> 
>   make O=/build -C tools/bpf/bpftool/ V=1
> 
> To fix the issue, explicitly link libzstd.

I was about to report exactly the same. :)

> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>

Tested-by: Namhyung Kim <namhyung@kernel.org>

Thanks,
Namhyung

> ---
>  tools/bpf/bpftool/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index a4263dfb5e03..65b2671941e0 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -130,8 +130,8 @@ include $(FEATURES_DUMP)
>  endif
>  endif
>  
> -LIBS = $(LIBBPF) -lelf -lz
> -LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
> +LIBS = $(LIBBPF) -lelf -lz -lzstd
> +LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz -lzstd
>  ifeq ($(feature-libcap), 1)
>  CFLAGS += -DUSE_LIBCAP
>  LIBS += -lcap
> -- 
> 2.34.1
>
Quentin Monnet Dec. 4, 2024, 10:08 p.m. UTC | #2
2024-12-04 13:36 UTC-0800 ~ Namhyung Kim <namhyung@kernel.org>
> Hi Leo,
> 
> On Wed, Dec 04, 2024 at 09:30:59PM +0000, Leo Yan wrote:
>> When building perf with static linkage:
>>
>>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
>>   ...
>>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
>>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
>>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
>>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
>>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
>>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
>>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
>>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
>>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
>>   (.text+0xd45): undefined reference to `ZSTD_decompress'
>>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
>>   collect2: error: ld returned 1 exit status
>>
>> Building bpftool with static linkage also fails with the same errors:
>>
>>   make O=/build -C tools/bpf/bpftool/ V=1
>>
>> To fix the issue, explicitly link libzstd.
> 
> I was about to report exactly the same. :)

Thank you both. This has been reported before [0] but I didn't find the
time to look into a proper fix.

The tricky part is that static linkage works well without libzstd for
older versions of elfutils [1], but newer versions now require this
library. Which means that we don't want to link against libzstd
unconditionally, or users trying to build bpftool may have to install
unnecessary dependencies. Instead we should add a new probe under
tools/build/feature (Note that we already have several combinations in
there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
the best approach in terms of new combinations).

Thanks,
Quentin


[0] https://github.com/libbpf/bpftool/issues/152
[1] https://github.com/libbpf/bpftool/issues/152#issuecomment-2343131810
Namhyung Kim Dec. 4, 2024, 10:25 p.m. UTC | #3
Hello,

On Wed, Dec 04, 2024 at 10:08:15PM +0000, Quentin Monnet wrote:
> 2024-12-04 13:36 UTC-0800 ~ Namhyung Kim <namhyung@kernel.org>
> > Hi Leo,
> > 
> > On Wed, Dec 04, 2024 at 09:30:59PM +0000, Leo Yan wrote:
> >> When building perf with static linkage:
> >>
> >>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
> >>   ...
> >>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
> >>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
> >>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
> >>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
> >>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
> >>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
> >>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
> >>   (.text+0xd45): undefined reference to `ZSTD_decompress'
> >>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
> >>   collect2: error: ld returned 1 exit status
> >>
> >> Building bpftool with static linkage also fails with the same errors:
> >>
> >>   make O=/build -C tools/bpf/bpftool/ V=1
> >>
> >> To fix the issue, explicitly link libzstd.
> > 
> > I was about to report exactly the same. :)
> 
> Thank you both. This has been reported before [0] but I didn't find the
> time to look into a proper fix.
> 
> The tricky part is that static linkage works well without libzstd for
> older versions of elfutils [1], but newer versions now require this
> library. Which means that we don't want to link against libzstd
> unconditionally, or users trying to build bpftool may have to install
> unnecessary dependencies. Instead we should add a new probe under
> tools/build/feature (Note that we already have several combinations in
> there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
> the best approach in terms of new combinations).

I think you can use pkg-config if available.

  $ pkg-config --static --libs libelf
  -lelf -lz -lzstd -pthread 

Thanks,
Namhyung

> 
> [0] https://github.com/libbpf/bpftool/issues/152
> [1] https://github.com/libbpf/bpftool/issues/152#issuecomment-2343131810
Quentin Monnet Dec. 4, 2024, 10:55 p.m. UTC | #4
2024-12-04 14:25 UTC-0800 ~ Namhyung Kim <namhyung@kernel.org>
> Hello,
> 
> On Wed, Dec 04, 2024 at 10:08:15PM +0000, Quentin Monnet wrote:
>> 2024-12-04 13:36 UTC-0800 ~ Namhyung Kim <namhyung@kernel.org>
>>> Hi Leo,
>>>
>>> On Wed, Dec 04, 2024 at 09:30:59PM +0000, Leo Yan wrote:
>>>> When building perf with static linkage:
>>>>
>>>>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
>>>>   ...
>>>>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
>>>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
>>>>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
>>>>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
>>>>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
>>>>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
>>>>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
>>>>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
>>>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
>>>>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
>>>>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
>>>>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
>>>>   (.text+0xd45): undefined reference to `ZSTD_decompress'
>>>>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
>>>>   collect2: error: ld returned 1 exit status
>>>>
>>>> Building bpftool with static linkage also fails with the same errors:
>>>>
>>>>   make O=/build -C tools/bpf/bpftool/ V=1
>>>>
>>>> To fix the issue, explicitly link libzstd.
>>>
>>> I was about to report exactly the same. :)
>>
>> Thank you both. This has been reported before [0] but I didn't find the
>> time to look into a proper fix.
>>
>> The tricky part is that static linkage works well without libzstd for
>> older versions of elfutils [1], but newer versions now require this
>> library. Which means that we don't want to link against libzstd
>> unconditionally, or users trying to build bpftool may have to install
>> unnecessary dependencies. Instead we should add a new probe under
>> tools/build/feature (Note that we already have several combinations in
>> there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
>> the best approach in terms of new combinations).
> 
> I think you can use pkg-config if available.
> 
>   $ pkg-config --static --libs libelf
>   -lelf -lz -lzstd -pthread 
> 


That's another dependency that I'd like to avoid if I can :)

Quentin
Leo Yan Dec. 5, 2024, 10:23 a.m. UTC | #5
On Wed, Dec 04, 2024 at 10:55:32PM +0000, Quentin Monnet wrote:

[...]

> >>> I was about to report exactly the same. :)
> >>
> >> Thank you both. This has been reported before [0] but I didn't find the
> >> time to look into a proper fix.
> >>
> >> The tricky part is that static linkage works well without libzstd for
> >> older versions of elfutils [1], but newer versions now require this
> >> library. Which means that we don't want to link against libzstd
> >> unconditionally, or users trying to build bpftool may have to install
> >> unnecessary dependencies. Instead we should add a new probe under
> >> tools/build/feature (Note that we already have several combinations in
> >> there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
> >> the best approach in terms of new combinations).
> >
> > I think you can use pkg-config if available.
> >
> >   $ pkg-config --static --libs libelf
> >   -lelf -lz -lzstd -pthread
> 
> That's another dependency that I'd like to avoid if I can :)

Seems to me, pkg-config is the right tool for doing such kind thing -
not only it is nature for local building, it is also friendly for build
system (e.g. buildroot, OpenEmbedded / Yocto).  Though I have no deep
knowledge for building.

I am a bit confused why this issue is related to build features libbfd,
libbfd-liberty, libbfd-liberty-z.  Should not the issue is related to
libelf?  build/feature has several libelf checking, maybe we can add new
one libelf-zstd?

Thanks,
Leo
Quentin Monnet Dec. 5, 2024, 10:58 a.m. UTC | #6
2024-12-05 10:23 UTC+0000 ~ Leo Yan <leo.yan@arm.com>
> On Wed, Dec 04, 2024 at 10:55:32PM +0000, Quentin Monnet wrote:
> 
> [...]
> 
>>>>> I was about to report exactly the same. :)
>>>>
>>>> Thank you both. This has been reported before [0] but I didn't find the
>>>> time to look into a proper fix.
>>>>
>>>> The tricky part is that static linkage works well without libzstd for
>>>> older versions of elfutils [1], but newer versions now require this
>>>> library. Which means that we don't want to link against libzstd
>>>> unconditionally, or users trying to build bpftool may have to install
>>>> unnecessary dependencies. Instead we should add a new probe under
>>>> tools/build/feature (Note that we already have several combinations in
>>>> there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
>>>> the best approach in terms of new combinations).
>>>
>>> I think you can use pkg-config if available.
>>>
>>>   $ pkg-config --static --libs libelf
>>>   -lelf -lz -lzstd -pthread
>>
>> That's another dependency that I'd like to avoid if I can :)
> 
> Seems to me, pkg-config is the right tool for doing such kind thing -
> not only it is nature for local building, it is also friendly for build
> system (e.g. buildroot, OpenEmbedded / Yocto).  Though I have no deep
> knowledge for building.


pkg-config would be nice but is not always installed by default. We've
been handling build options without it so far, if we can fix the current
issue without struggling too much with probes I'd just as well avoid
adding a build dependency.


> I am a bit confused why this issue is related to build features libbfd,
> libbfd-liberty, libbfd-liberty-z.  Should not the issue is related to
> libelf?  build/feature has several libelf checking, maybe we can add new
> one libelf-zstd?


Apologies, I was the one getting confused. You're correct, this affects
libelf and not libbfd, and yes libelf-zstd is likely the way to go.

Thank you,
Quentin
Stanislav Fomichev Dec. 5, 2024, 7:24 p.m. UTC | #7
On 12/04, Leo Yan wrote:
> When building perf with static linkage:
> 
>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
>   ...
>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
>   (.text+0xd45): undefined reference to `ZSTD_decompress'
>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
>   collect2: error: ld returned 1 exit status
> 
> Building bpftool with static linkage also fails with the same errors:
> 
>   make O=/build -C tools/bpf/bpftool/ V=1
> 
> To fix the issue, explicitly link libzstd.
> 
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---
>  tools/bpf/bpftool/Makefile | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index a4263dfb5e03..65b2671941e0 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -130,8 +130,8 @@ include $(FEATURES_DUMP)
>  endif
>  endif
>  
> -LIBS = $(LIBBPF) -lelf -lz
> -LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
> +LIBS = $(LIBBPF) -lelf -lz -lzstd
> +LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz -lzstd
>  ifeq ($(feature-libcap), 1)
>  CFLAGS += -DUSE_LIBCAP
>  LIBS += -lcap
> -- 
> 2.34.1
> 

I'm not sure we 'offically' support -static builds, but this seems to be
ok. Tangential: maybe time to switch to pkg-config for bpftool? IIRC,
there is some flag to query for static lib dependencies... Will leave
it up to Quentin.
Andrii Nakryiko Dec. 10, 2024, 6:26 p.m. UTC | #8
On Wed, Dec 4, 2024 at 2:08 PM Quentin Monnet <qmo@kernel.org> wrote:
>
> 2024-12-04 13:36 UTC-0800 ~ Namhyung Kim <namhyung@kernel.org>
> > Hi Leo,
> >
> > On Wed, Dec 04, 2024 at 09:30:59PM +0000, Leo Yan wrote:
> >> When building perf with static linkage:
> >>
> >>   make O=/build LDFLAGS="-static" -C tools/perf VF=1 DEBUG=1
> >>   ...
> >>   LINK    /build/util/bpf_skel/.tmp/bootstrap/bpftool
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
> >>   (.text+0x113): undefined reference to `ZSTD_createCCtx'
> >>   /usr/bin/ld: (.text+0x2a9): undefined reference to `ZSTD_compressStream2'
> >>   /usr/bin/ld: (.text+0x2b4): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: (.text+0x2db): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x5a0): undefined reference to `ZSTD_compressStream2'
> >>   /usr/bin/ld: (.text+0x5ab): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: (.text+0x6b9): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x835): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x86f): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0x91b): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: (.text+0xa12): undefined reference to `ZSTD_freeCCtx'
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
> >>   (.text+0xbfc): undefined reference to `ZSTD_decompress'
> >>   /usr/bin/ld: (.text+0xc04): undefined reference to `ZSTD_isError'
> >>   /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress_elf':
> >>   (.text+0xd45): undefined reference to `ZSTD_decompress'
> >>   /usr/bin/ld: (.text+0xd4d): undefined reference to `ZSTD_isError'
> >>   collect2: error: ld returned 1 exit status
> >>
> >> Building bpftool with static linkage also fails with the same errors:
> >>
> >>   make O=/build -C tools/bpf/bpftool/ V=1
> >>
> >> To fix the issue, explicitly link libzstd.
> >
> > I was about to report exactly the same. :)
>
> Thank you both. This has been reported before [0] but I didn't find the
> time to look into a proper fix.
>
> The tricky part is that static linkage works well without libzstd for
> older versions of elfutils [1], but newer versions now require this
> library. Which means that we don't want to link against libzstd
> unconditionally, or users trying to build bpftool may have to install
> unnecessary dependencies. Instead we should add a new probe under
> tools/build/feature (Note that we already have several combinations in
> there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
> the best approach in terms of new combinations).
>

So what's the conclusion here? Do we apply this as a fix, or someone
needs to add more feature probing?

> Thanks,
> Quentin
>
>
> [0] https://github.com/libbpf/bpftool/issues/152
> [1] https://github.com/libbpf/bpftool/issues/152#issuecomment-2343131810
Leo Yan Dec. 10, 2024, 8:19 p.m. UTC | #9
Hi Andrii,

On Tue, Dec 10, 2024 at 10:26:22AM -0800, Andrii Nakryiko wrote:

[...]

> > The tricky part is that static linkage works well without libzstd for
> > older versions of elfutils [1], but newer versions now require this
> > library. Which means that we don't want to link against libzstd
> > unconditionally, or users trying to build bpftool may have to install
> > unnecessary dependencies. Instead we should add a new probe under
> > tools/build/feature (Note that we already have several combinations in
> > there, libbfd, libbfd-liberty, libbfd-liberty-z, and I'm not sure what's
> > the best approach in terms of new combinations).
> >
> 
> So what's the conclusion here? Do we apply this as a fix, or someone
> needs to add more feature probing?

I am working on a new build feature.  Based on that, it will refine for
perf build and bpftool build.  Once get ready, I will send out for
review.

Thanks,
Leo
diff mbox series

Patch

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index a4263dfb5e03..65b2671941e0 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -130,8 +130,8 @@  include $(FEATURES_DUMP)
 endif
 endif
 
-LIBS = $(LIBBPF) -lelf -lz
-LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
+LIBS = $(LIBBPF) -lelf -lz -lzstd
+LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz -lzstd
 ifeq ($(feature-libcap), 1)
 CFLAGS += -DUSE_LIBCAP
 LIBS += -lcap