[bpf-next,00/13] uprobes: Add support to optimize usdt probes on x86_64

Message

Jiri Olsa Dec. 11, 2024, 1:33 p.m. UTC

hi,
this patchset adds support to optimize usdt probes on top of 5-byte
nop instruction.

The generic approach (optimize all uprobes) is hard due to emulating
possible multiple original instructions and its related issues. The
usdt case, which stores 5-byte nop seems much easier, so starting
with that.

The basic idea is to replace breakpoint exception with syscall which
is faster on x86_64. For more details please see changelog of patch 8.

The run_bench_uprobes.sh benchmark triggers uprobe (on top of different
original instructions) in a loop and counts how many of those happened
per second (the unit below is million loops).

There's big speed up if you consider current usdt implementation
(uprobe-nop) compared to proposed usdt (uprobe-nop5):

  # ./benchs/run_bench_uprobes.sh 

      usermode-count :  233.831 ± 0.257M/s
      syscall-count  :   12.107 ± 0.038M/s
  --> uprobe-nop     :    3.246 ± 0.004M/s
      uprobe-push    :    3.057 ± 0.000M/s
      uprobe-ret     :    1.113 ± 0.003M/s
  --> uprobe-nop5    :    6.751 ± 0.037M/s
      uretprobe-nop  :    1.740 ± 0.015M/s
      uretprobe-push :    1.677 ± 0.018M/s
      uretprobe-ret  :    0.852 ± 0.005M/s
      uretprobe-nop5 :    6.769 ± 0.040M/s


v1 changes:
- rebased on top of bpf-next/master
- couple of function/variable renames [Andrii]
- added nop5 emulation [Andrii]
- added checks to arch_uprobe_verify_opcode [Andrii]
- fixed arch_uprobe_is_callable/find_nearest_page [Andrii]
- used CALL_INSN_OPCODE [Masami]
- added uprobe-nop5 benchmark [Andrii]
- using atomic64_t in tramp_area [Andri]
- using single page for all uprobe trampoline mappings

thanks,
jirka


---
Jiri Olsa (13):
      uprobes: Rename arch_uretprobe_trampoline function
      uprobes: Make copy_from_page global
      uprobes: Add nbytes argument to uprobe_write_opcode
      uprobes: Add arch_uprobe_verify_opcode function
      uprobes: Add mapping for optimized uprobe trampolines
      uprobes/x86: Add uprobe syscall to speed up uprobe
      uprobes/x86: Add support to emulate nop5 instruction
      uprobes/x86: Add support to optimize uprobes
      selftests/bpf: Use 5-byte nop for x86 usdt probes
      selftests/bpf: Add uprobe/usdt optimized test
      selftests/bpf: Add hit/attach/detach race optimized uprobe test
      selftests/bpf: Add uprobe syscall sigill signal test
      selftests/bpf: Add 5-byte nop uprobe trigger bench

 arch/x86/entry/syscalls/syscall_64.tbl                  |   1 +
 arch/x86/include/asm/uprobes.h                          |   7 +++
 arch/x86/kernel/uprobes.c                               | 255 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 include/linux/syscalls.h                                |   2 +
 include/linux/uprobes.h                                 |  25 +++++++-
 kernel/events/uprobes.c                                 | 191 +++++++++++++++++++++++++++++++++++++++++++++++++++-----
 kernel/fork.c                                           |   1 +
 kernel/sys_ni.c                                         |   1 +
 tools/testing/selftests/bpf/bench.c                     |  12 ++++
 tools/testing/selftests/bpf/benchs/bench_trigger.c      |  42 +++++++++++++
 tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh |   2 +-
 tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 326 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/uprobe_optimized.c    |  29 +++++++++
 tools/testing/selftests/bpf/sdt.h                       |   9 ++-
 14 files changed, 880 insertions(+), 23 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/uprobe_optimized.c