mbox series

[bpf-next,v6,0/4] bpf: add cpu cycles kfuncss

Message ID 20241115194841.2108634-1-vadfed@meta.com (mailing list archive)
Headers show
Series bpf: add cpu cycles kfuncss | expand

Message

Vadim Fedorenko Nov. 15, 2024, 7:48 p.m. UTC 
This patchset adds 2 kfuncs to provide a way to precisely measure the
time spent running some code. The first patch provides a way to get cpu
cycles counter which is used to feed CLOCK_MONOTONIC_RAW. On x86
architecture it is effectively rdtsc_ordered() function while on other
architectures it falls back to __arch_get_hw_counter(). The second patch
adds a kfunc to convert cpu cycles to nanoseconds using shift/mult
constants discovered by kernel. JIT version is done for x86 for now, on
other architectures it falls back to slightly simplified version of
vdso_calc_ns.

Selftests are also added to check whether the JIT implementation is
correct and to show the simplest usage example.

Change log:
v5 -> v6:
* added cover letter
* add comment about dropping S64_MAX manipulation in jitted
  implementation of rdtsc_oredered (Alexey)
* add comment about using 'lfence;rdtsc' variant (Alexey)
* change the check in fixup_kfunc_call() (Eduard)
* make __arch_get_hw_counter() call more aligned with vDSO
  implementation (Yonghong)
v4 -> v5:
* use #if instead of #ifdef with IS_ENABLED
v3 -> v4:
* change name of the helper to bpf_get_cpu_cycles (Andrii)
* Hide the helper behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing
  it on architectures which do not have vDSO functions and data
* reduce the scope of check of inlined functions in verifier to only 2,
  which are actually inlined.
* change helper name to bpf_cpu_cycles_to_ns.
* hide it behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing on
  unsupported architectures.
v2 -> v3:
* change name of the helper to bpf_get_cpu_cycles_counter to
* explicitly mention what counter it provides (Andrii)
* move kfunc definition to bpf.h to use it in JIT.
* introduce another kfunc to convert cycles into nanoseconds as
* more meaningful time units for generic tracing use case (Andrii)
v1 -> v2:
* Fix incorrect function return value type to u64
* Introduce bpf_jit_inlines_kfunc_call() and use it in
	mark_fastcall_pattern_for_call() to avoid clobbering in case
	of running programs with no JIT (Eduard)
* Avoid rewriting instruction and check function pointer directly
	in JIT (Alexei)
* Change includes to fix compile issues on non x86 architectures

Vadim Fedorenko (4):
  bpf: add bpf_get_cpu_cycles kfunc
  bpf: add bpf_cpu_cycles_to_ns helper
  selftests/bpf: add selftest to check rdtsc jit
  selftests/bpf: add usage example for cpu cycles kfuncs

 arch/x86/net/bpf_jit_comp.c                   |  60 ++++++++++
 arch/x86/net/bpf_jit_comp32.c                 |  33 ++++++
 include/linux/bpf.h                           |   6 +
 include/linux/filter.h                        |   1 +
 kernel/bpf/core.c                             |  11 ++
 kernel/bpf/helpers.c                          |  32 ++++++
 kernel/bpf/verifier.c                         |  41 ++++++-
 .../bpf/prog_tests/test_cpu_cycles.c          |  35 ++++++
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/test_cpu_cycles.c     |  25 +++++
 .../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
 11 files changed, 344 insertions(+), 6 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c

Comments

Borislav Petkov Nov. 15, 2024, 8:15 p.m. UTC | #1 
On Fri, Nov 15, 2024 at 11:48:37AM -0800, Vadim Fedorenko wrote:
>  arch/x86/net/bpf_jit_comp.c                   |  60 ++++++++++
>  arch/x86/net/bpf_jit_comp32.c                 |  33 ++++++
>  include/linux/bpf.h                           |   6 +
>  include/linux/filter.h                        |   1 +
>  kernel/bpf/core.c                             |  11 ++
>  kernel/bpf/helpers.c                          |  32 ++++++
>  kernel/bpf/verifier.c                         |  41 ++++++-
>  .../bpf/prog_tests/test_cpu_cycles.c          |  35 ++++++
>  .../selftests/bpf/prog_tests/verifier.c       |   2 +
>  .../selftests/bpf/progs/test_cpu_cycles.c     |  25 +++++
>  .../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
>  11 files changed, 344 insertions(+), 6 deletions(-)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
>  create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c

For your whole set:

s/boot_cpu_has/cpu_feature_enabled/g

Thx.
Vadim Fedorenko Nov. 15, 2024, 8:30 p.m. UTC | #2 
On 15/11/2024 20:15, Borislav Petkov wrote:
> On Fri, Nov 15, 2024 at 11:48:37AM -0800, Vadim Fedorenko wrote:
>>   arch/x86/net/bpf_jit_comp.c                   |  60 ++++++++++
>>   arch/x86/net/bpf_jit_comp32.c                 |  33 ++++++
>>   include/linux/bpf.h                           |   6 +
>>   include/linux/filter.h                        |   1 +
>>   kernel/bpf/core.c                             |  11 ++
>>   kernel/bpf/helpers.c                          |  32 ++++++
>>   kernel/bpf/verifier.c                         |  41 ++++++-
>>   .../bpf/prog_tests/test_cpu_cycles.c          |  35 ++++++
>>   .../selftests/bpf/prog_tests/verifier.c       |   2 +
>>   .../selftests/bpf/progs/test_cpu_cycles.c     |  25 +++++
>>   .../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
>>   11 files changed, 344 insertions(+), 6 deletions(-)
>>   create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
>>   create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
>>   create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c
> 
> For your whole set:
> 
> s/boot_cpu_has/cpu_feature_enabled/g

thanks, will change it for the next version
> 
> Thx.
>
Andrii Nakryiko Nov. 15, 2024, 10:21 p.m. UTC | #3 
On Fri, Nov 15, 2024 at 11:49 AM Vadim Fedorenko <vadfed@meta.com> wrote:
>
> This patchset adds 2 kfuncs to provide a way to precisely measure the
> time spent running some code. The first patch provides a way to get cpu
> cycles counter which is used to feed CLOCK_MONOTONIC_RAW. On x86
> architecture it is effectively rdtsc_ordered() function while on other
> architectures it falls back to __arch_get_hw_counter(). The second patch
> adds a kfunc to convert cpu cycles to nanoseconds using shift/mult
> constants discovered by kernel. JIT version is done for x86 for now, on
> other architectures it falls back to slightly simplified version of
> vdso_calc_ns.
>
> Selftests are also added to check whether the JIT implementation is
> correct and to show the simplest usage example.
>
> Change log:
> v5 -> v6:
> * added cover letter
> * add comment about dropping S64_MAX manipulation in jitted
>   implementation of rdtsc_oredered (Alexey)
> * add comment about using 'lfence;rdtsc' variant (Alexey)
> * change the check in fixup_kfunc_call() (Eduard)
> * make __arch_get_hw_counter() call more aligned with vDSO
>   implementation (Yonghong)
> v4 -> v5:
> * use #if instead of #ifdef with IS_ENABLED
> v3 -> v4:
> * change name of the helper to bpf_get_cpu_cycles (Andrii)
> * Hide the helper behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing
>   it on architectures which do not have vDSO functions and data
> * reduce the scope of check of inlined functions in verifier to only 2,
>   which are actually inlined.
> * change helper name to bpf_cpu_cycles_to_ns.
> * hide it behind CONFIG_GENERIC_GETTIMEOFDAY to avoid exposing on
>   unsupported architectures.
> v2 -> v3:
> * change name of the helper to bpf_get_cpu_cycles_counter to
> * explicitly mention what counter it provides (Andrii)
> * move kfunc definition to bpf.h to use it in JIT.
> * introduce another kfunc to convert cycles into nanoseconds as
> * more meaningful time units for generic tracing use case (Andrii)
> v1 -> v2:
> * Fix incorrect function return value type to u64
> * Introduce bpf_jit_inlines_kfunc_call() and use it in
>         mark_fastcall_pattern_for_call() to avoid clobbering in case
>         of running programs with no JIT (Eduard)
> * Avoid rewriting instruction and check function pointer directly
>         in JIT (Alexei)
> * Change includes to fix compile issues on non x86 architectures
>
> Vadim Fedorenko (4):
>   bpf: add bpf_get_cpu_cycles kfunc
>   bpf: add bpf_cpu_cycles_to_ns helper
>   selftests/bpf: add selftest to check rdtsc jit
>   selftests/bpf: add usage example for cpu cycles kfuncs
>
>  arch/x86/net/bpf_jit_comp.c                   |  60 ++++++++++
>  arch/x86/net/bpf_jit_comp32.c                 |  33 ++++++
>  include/linux/bpf.h                           |   6 +
>  include/linux/filter.h                        |   1 +
>  kernel/bpf/core.c                             |  11 ++
>  kernel/bpf/helpers.c                          |  32 ++++++
>  kernel/bpf/verifier.c                         |  41 ++++++-
>  .../bpf/prog_tests/test_cpu_cycles.c          |  35 ++++++
>  .../selftests/bpf/prog_tests/verifier.c       |   2 +
>  .../selftests/bpf/progs/test_cpu_cycles.c     |  25 +++++
>  .../selftests/bpf/progs/verifier_cpu_cycles.c | 104 ++++++++++++++++++
>  11 files changed, 344 insertions(+), 6 deletions(-)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/test_cpu_cycles.c
>  create mode 100644 tools/testing/selftests/bpf/progs/test_cpu_cycles.c
>  create mode 100644 tools/testing/selftests/bpf/progs/verifier_cpu_cycles.c
>
> --
> 2.43.5
>

typo in subject: kfuncss -> kfuncs

LGTM overall, thanks a lot for adding this!

For the series:

Acked-by: Andrii Nakryiko <andrii@kernel.org>