[bpf-next,v6,5/8] bpf: Use smaller low/high marks for percpu allocation

Commit Message

Yonghong Song Dec. 22, 2023, 3:17 a.m. UTC

Currently, refill low/high marks are set with the assumption
of normal non-percpu memory allocation. For example, for
an allocation size 256, for non-percpu memory allocation,
low mark is 32 and high mark is 96, resulting in the
batch allocation of 48 elements and the allocated memory
will be 48 * 256 = 12KB for this particular cpu.
Assuming an 128-cpu system, the total memory consumption
across all cpus will be 12K * 128 = 1.5MB memory.

This might be okay for non-percpu allocation, but may not be
good for percpu allocation, which will consume 1.5MB * 128 = 192MB
memory in the worst case if every cpu has a chance of memory
allocation.

In practice, percpu allocation is very rare compared to
non-percpu allocation. So let us have smaller low/high marks
which can avoid unnecessary memory consumption.

Signed-off-by: Yonghong Song <[e-mail]>
---
 kernel/bpf/memalloc.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Comments

Hou Tao Dec. 22, 2023, 9:36 a.m. UTC | #1

On 12/22/2023 11:17 AM, Yonghong Song wrote:
> Currently, refill low/high marks are set with the assumption
> of normal non-percpu memory allocation. For example, for
> an allocation size 256, for non-percpu memory allocation,
> low mark is 32 and high mark is 96, resulting in the
> batch allocation of 48 elements and the allocated memory
> will be 48 * 256 = 12KB for this particular cpu.
> Assuming an 128-cpu system, the total memory consumption
> across all cpus will be 12K * 128 = 1.5MB memory.
>
> This might be okay for non-percpu allocation, but may not be
> good for percpu allocation, which will consume 1.5MB * 128 = 192MB
> memory in the worst case if every cpu has a chance of memory
> allocation.
>
> In practice, percpu allocation is very rare compared to
> non-percpu allocation. So let us have smaller low/high marks
> which can avoid unnecessary memory consumption.
>
> Signed-off-by: Yonghong Song <[e-mail]>

Acked-by: Hou Tao <[e-mail]>

Patch

diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c
index a8ee6fb8401c..460c8f38fed6 100644
--- a/kernel/bpf/memalloc.c
+++ b/kernel/bpf/memalloc.c
@@ -464,11 +464,17 @@  static void notrace irq_work_raise(struct bpf_mem_cache *c)
  * consume ~ 11 Kbyte per cpu.
  * Typical case will be between 11K and 116K closer to 11K.
  * bpf progs can and should share bpf_mem_cache when possible.
+ *
+ * Percpu allocation is typically rare. To avoid potential unnecessary large
+ * memory consumption, set low_mark = 1 and high_mark = 3, resulting in c->batch = 1.
  */
 static void init_refill_work(struct bpf_mem_cache *c)
 {
 	init_irq_work(&c->refill_work, bpf_mem_refill);
-	if (c->unit_size <= 256) {
+	if (c->percpu_size) {
+		c->low_watermark = 1;
+		c->high_watermark = 3;
+	} else if (c->unit_size <= 256) {
 		c->low_watermark = 32;
 		c->high_watermark = 96;
 	} else {