[v2,2/6] rcu-tasks: Add data to eliminate RCU-tasks/do_exit() deadlocks

Commit Message

Boqun Feng Feb. 17, 2024, 1:27 a.m. UTC

From: "Paul E. McKenney" <[e-mail]>

Holding a mutex across synchronize_rcu_tasks() and acquiring
that same mutex in code called from do_exit() after its call to
exit_tasks_rcu_start() but before its call to exit_tasks_rcu_stop()
results in deadlock.  This is by design, because tasks that are far
enough into do_exit() are no longer present on the tasks list, making
it a bit difficult for RCU Tasks to find them, let alone wait on them
to do a voluntary context switch.  However, such deadlocks are becoming
more frequent.  In addition, lockdep currently does not detect such
deadlocks and they can be difficult to reproduce.

In addition, if a task voluntarily context switches during that time
(for example, if it blocks acquiring a mutex), then this task is in an
RCU Tasks quiescent state.  And with some adjustments, RCU Tasks could
just as well take advantage of that fact.

This commit therefore adds the data structures that will be needed
to rely on these quiescent states and to eliminate these deadlocks.

Link: https://lore.kernel.org/all/[e-mail]/

Reported-by: Chen Zhongjin <[e-mail]>
Reported-by: Yang Jihong <[e-mail]>
Signed-off-by: Paul E. McKenney <[e-mail]>
Tested-by: Yang Jihong <[e-mail]>
Tested-by: Chen Zhongjin <[e-mail]>
Signed-off-by: Boqun Feng <[e-mail]>
---
 include/linux/sched.h | 2 ++
 kernel/rcu/tasks.h    | 2 ++
 2 files changed, 4 insertions(+)

Comments

Frederic Weisbecker Feb. 22, 2024, 4:54 p.m. UTC | #1

Le Fri, Feb 16, 2024 at 05:27:37PM -0800, Boqun Feng a écrit :
> From: "Paul E. McKenney" <[e-mail]>
> 
> Holding a mutex across synchronize_rcu_tasks() and acquiring
> that same mutex in code called from do_exit() after its call to
> exit_tasks_rcu_start() but before its call to exit_tasks_rcu_stop()
> results in deadlock.  This is by design, because tasks that are far
> enough into do_exit() are no longer present on the tasks list, making
> it a bit difficult for RCU Tasks to find them, let alone wait on them
> to do a voluntary context switch.  However, such deadlocks are becoming
> more frequent.  In addition, lockdep currently does not detect such
> deadlocks and they can be difficult to reproduce.
> 
> In addition, if a task voluntarily context switches during that time
> (for example, if it blocks acquiring a mutex), then this task is in an
> RCU Tasks quiescent state.  And with some adjustments, RCU Tasks could
> just as well take advantage of that fact.
> 
> This commit therefore adds the data structures that will be needed
> to rely on these quiescent states and to eliminate these deadlocks.
> 
> Link: https://lore.kernel.org/all/[e-mail]/
> 
> Reported-by: Chen Zhongjin <[e-mail]>
> Reported-by: Yang Jihong <[e-mail]>
> Signed-off-by: Paul E. McKenney <[e-mail]>
> Tested-by: Yang Jihong <[e-mail]>
> Tested-by: Chen Zhongjin <[e-mail]>
> Signed-off-by: Boqun Feng <[e-mail]>

Reviewed-by: Frederic Weisbecker <[e-mail]>

Paul E. McKenney Feb. 22, 2024, 8:46 p.m. UTC | #2

On Thu, Feb 22, 2024 at 05:54:48PM +0100, Frederic Weisbecker wrote:
> Le Fri, Feb 16, 2024 at 05:27:37PM -0800, Boqun Feng a écrit :
> > From: "Paul E. McKenney" <[e-mail]>
> > 
> > Holding a mutex across synchronize_rcu_tasks() and acquiring
> > that same mutex in code called from do_exit() after its call to
> > exit_tasks_rcu_start() but before its call to exit_tasks_rcu_stop()
> > results in deadlock.  This is by design, because tasks that are far
> > enough into do_exit() are no longer present on the tasks list, making
> > it a bit difficult for RCU Tasks to find them, let alone wait on them
> > to do a voluntary context switch.  However, such deadlocks are becoming
> > more frequent.  In addition, lockdep currently does not detect such
> > deadlocks and they can be difficult to reproduce.
> > 
> > In addition, if a task voluntarily context switches during that time
> > (for example, if it blocks acquiring a mutex), then this task is in an
> > RCU Tasks quiescent state.  And with some adjustments, RCU Tasks could
> > just as well take advantage of that fact.
> > 
> > This commit therefore adds the data structures that will be needed
> > to rely on these quiescent states and to eliminate these deadlocks.
> > 
> > Link: https://lore.kernel.org/all/[e-mail]/
> > 
> > Reported-by: Chen Zhongjin <[e-mail]>
> > Reported-by: Yang Jihong <[e-mail]>
> > Signed-off-by: Paul E. McKenney <[e-mail]>
> > Tested-by: Yang Jihong <[e-mail]>
> > Tested-by: Chen Zhongjin <[e-mail]>
> > Signed-off-by: Boqun Feng <[e-mail]>
> 
> Reviewed-by: Frederic Weisbecker <[e-mail]>

Thank you, I have recorded your three review tags.

							Thanx, Paul

Patch

diff --git a/include/linux/sched.h b/include/linux/sched.h
index ffe8f618ab86..5eeebed2dd9b 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -858,6 +858,8 @@  struct task_struct {
 	u8				rcu_tasks_idx;
 	int				rcu_tasks_idle_cpu;
 	struct list_head		rcu_tasks_holdout_list;
+	int				rcu_tasks_exit_cpu;
+	struct list_head		rcu_tasks_exit_list;
 #endif /* #ifdef CONFIG_TASKS_RCU */
 
 #ifdef CONFIG_TASKS_TRACE_RCU
diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h
index 732ad5b39946..b7d5f2757053 100644
--- a/kernel/rcu/tasks.h
+++ b/kernel/rcu/tasks.h
@@ -32,6 +32,7 @@  typedef void (*postgp_func_t)(struct rcu_tasks *rtp);
  * @rtp_irq_work: IRQ work queue for deferred wakeups.
  * @barrier_q_head: RCU callback for barrier operation.
  * @rtp_blkd_tasks: List of tasks blocked as readers.
+ * @rtp_exit_list: List of tasks in the latter portion of do_exit().
  * @cpu: CPU number corresponding to this entry.
  * @rtpp: Pointer to the rcu_tasks structure.
  */
@@ -46,6 +47,7 @@  struct rcu_tasks_percpu {
 	struct irq_work rtp_irq_work;
 	struct rcu_head barrier_q_head;
 	struct list_head rtp_blkd_tasks;
+	struct list_head rtp_exit_list;
 	int cpu;
 	struct rcu_tasks *rtpp;
 };