[bpf] Use call_rcu_hurry() with synchronize_rcu_mult()

Commit Message

Paul E. McKenney May 18, 2023, 2:47 p.m. UTC

The bpf_struct_ops_map_free() function must wait for both an RCU grace
period and an RCU Tasks grace period, and so it passes call_rcu() and
call_rcu_tasks() to synchronize_rcu_mult().  This works, but on ChromeOS
and Android platforms call_rcu() can have lazy semantics, resulting in
multi-second delays between call_rcu() invocation and invocation of the
corresponding callback.

Therefore, substitute call_rcu_hurry() for call_rcu().

Signed-off-by: Paul E. McKenney <[e-mail]>
Cc: Martin KaFai Lau <[e-mail]>
Cc: Alexei Starovoitov <[e-mail]>
Cc: Daniel Borkmann <[e-mail]>
Cc: Andrii Nakryiko <[e-mail]>
Cc: Song Liu <[e-mail]>
Cc: Yonghong Song <[e-mail]>
Cc: John Fastabend <[e-mail]>
Cc: KP Singh <[e-mail]>
Cc: Stanislav Fomichev <[e-mail]>
Cc: Hao Luo <[e-mail]>
Cc: Jiri Olsa <[e-mail]>
Cc: <[e-mail]>
Cc: <[e-mail]>

Comments

Martin KaFai Lau May 18, 2023, 4:51 p.m. UTC | #1

On 5/18/23 7:47 AM, Paul E. McKenney wrote:
> The bpf_struct_ops_map_free() function must wait for both an RCU grace
> period and an RCU Tasks grace period, and so it passes call_rcu() and
> call_rcu_tasks() to synchronize_rcu_mult().  This works, but on ChromeOS
> and Android platforms call_rcu() can have lazy semantics, resulting in
> multi-second delays between call_rcu() invocation and invocation of the
> corresponding callback.
> 
> Therefore, substitute call_rcu_hurry() for call_rcu().

My understanding on the net-effect is to free up the struct_ops resources faster.

I believe call_rcu() should be fine. struct_ops freeing should not happen very 
often. For example, when a bpf written tcp congestion control (struct_ops) is 
registered, it will stay in the kernel for a long time. A couple seconds delay 
in releasing the struct_ops should be acceptable.

Paul E. McKenney May 18, 2023, 5:25 p.m. UTC | #2

On Thu, May 18, 2023 at 09:51:39AM -0700, Martin KaFai Lau wrote:
> On 5/18/23 7:47 AM, Paul E. McKenney wrote:
> > The bpf_struct_ops_map_free() function must wait for both an RCU grace
> > period and an RCU Tasks grace period, and so it passes call_rcu() and
> > call_rcu_tasks() to synchronize_rcu_mult().  This works, but on ChromeOS
> > and Android platforms call_rcu() can have lazy semantics, resulting in
> > multi-second delays between call_rcu() invocation and invocation of the
> > corresponding callback.
> > 
> > Therefore, substitute call_rcu_hurry() for call_rcu().
> 
> My understanding on the net-effect is to free up the struct_ops resources faster.
> 
> I believe call_rcu() should be fine. struct_ops freeing should not happen
> very often. For example, when a bpf written tcp congestion control
> (struct_ops) is registered, it will stay in the kernel for a long time. A
> couple seconds delay in releasing the struct_ops should be acceptable.

Very good, and sorry for the noise!

							Thanx, Paul

Patch

diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c
index d3f0a4825fa6..bacffd6cae60 100644
--- a/kernel/bpf/bpf_struct_ops.c
+++ b/kernel/bpf/bpf_struct_ops.c
@@ -634,7 +634,7 @@  static void bpf_struct_ops_map_free(struct bpf_map *map)
 	 * in the tramopline image to finish before releasing
 	 * the trampoline image.
 	 */
-	synchronize_rcu_mult(call_rcu, call_rcu_tasks);
+	synchronize_rcu_mult(call_rcu_hurry, call_rcu_tasks);
 
 	__bpf_struct_ops_map_free(map);
 }