[bpf-next,00/16] bpf: Use bpf_mem_cache_alloc/free in bpf_local_storage

Message

Martin KaFai Lau March 6, 2023, 8:42 a.m. UTC

From: Martin KaFai Lau <[e-mail]>

This set is to use bpf_mem_cache_alloc/free in bpf_local_storage.
The primary motivation is to solve the deadlock/recursion issue
when bpf_task_storage is used in a bpf tracing prog [1]. This set
also comes with a micro-benchmark to test the storage creation.

Patch 1 to 4 are some general cleanup after bpf_local_storage
has been extended to multiple kernel objects (sk, task, inode,
and then cgrp).

Patch 5 to 11 is to refactor the memory free logic into the new
bpf_selem_free() and bpf_local_storage_free() functions. Together
with the existing bpf_selem_alloc() and bpf_local_storage_alloc(),
it should provide an easier way to change the alloc/free path in
the future.

Patch 12 to 13 is to use bpf_mem_cache_alloc/free.

The remaining patches are selftests and benchmark.

[1]: https://lore.kernel.org/bpf/[e-mail]/

Martin KaFai Lau (16):
  bpf: Move a few bpf_local_storage functions to static scope
  bpf: Refactor codes into bpf_local_storage_destroy
  bpf: Remove __bpf_local_storage_map_alloc
  bpf: Remove the preceding __ from __bpf_selem_unlink_storage
  bpf: Remember smap in bpf_local_storage
  bpf: Repurpose use_trace_rcu to reuse_now in bpf_local_storage
  bpf: Remove bpf_selem_free_fields*_rcu
  bpf: Add bpf_selem_free_rcu callback
  bpf: Add bpf_selem_free()
  bpf: Add bpf_local_storage_rcu callback
  bpf: Add bpf_local_storage_free()
  bpf: Use bpf_mem_cache_alloc/free in bpf_selem_alloc/free
  bpf: Use bpf_mem_cache_alloc/free for bpf_local_storage
  selftests/bpf: Replace CHECK with ASSERT in test_local_storage
  selftests/bpf: Check freeing sk->sk_local_storage with
    sk_local_storage->smap is NULL
  selftests/bpf: Add local-storage-create benchmark

 include/linux/bpf_local_storage.h             |  22 +-
 include/linux/bpf_mem_alloc.h                 |   5 +
 kernel/bpf/bpf_cgrp_storage.c                 |  11 +-
 kernel/bpf/bpf_inode_storage.c                |  10 +-
 kernel/bpf/bpf_local_storage.c                | 292 ++++++++++--------
 kernel/bpf/bpf_task_storage.c                 |  11 +-
 net/core/bpf_sk_storage.c                     |  12 +-
 tools/testing/selftests/bpf/Makefile          |   2 +
 tools/testing/selftests/bpf/bench.c           |   2 +
 .../bpf/benchs/bench_local_storage_create.c   | 141 +++++++++
 .../bpf/prog_tests/test_local_storage.c       |  49 ++-
 .../bpf/progs/bench_local_storage_create.c    |  57 ++++
 .../selftests/bpf/progs/local_storage.c       |  29 +-
 13 files changed, 429 insertions(+), 214 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_local_storage_create.c
 create mode 100644 tools/testing/selftests/bpf/progs/bench_local_storage_create.c