[v2,01/14] thread_info: Add helpers to test and clear TIF_NOTIFY_IPI

Commit Message

K Prateek Nayak June 13, 2024, 6:16 p.m. UTC

From: "Gautham R. Shenoy" <[e-mail]>

Introduce the notion of TIF_NOTIFY_IPI flag. When a processor in
TIF_POLLING mode needs to process an IPI, the sender sets NEED_RESCHED
bit in idle task's thread_info to pull the target out of idle and avoids
sending an interrupt to the idle CPU. When NEED_RESCHED is set, the
scheduler assumes that a new task has been queued on the idle CPU and
calls schedule_idle(), however, it is not necessary that an IPI on an
idle CPU will necessarily end up waking a task on the said CPU. To avoid
spurious calls to schedule_idle() assuming an IPI on an idle CPU will
always wake a task on the said CPU, TIF_NOTIFY_IPI will be used to pull
a TIF_POLLING CPU out of idle.

Since the IPI handlers are processed before the call to schedule_idle(),
schedule_idle() will be called only if one of the handlers have woken up
a new task on the CPU and has set NEED_RESCHED.

Add tif_notify_ipi() and current_clr_notify_ipi() helpers to test if
TIF_NOTIFY_IPI is set in the current task's thread_info, and to clear it
respectively. These interfaces will be used in subsequent patches as
TIF_NOTIFY_IPI notion is integrated in the scheduler and in the idle
path.

[ prateek: Split the changes into a separate patch, add commit log ]

Cc: Richard Henderson <[e-mail]>
Cc: Ivan Kokshaysky <[e-mail]>
Cc: Matt Turner <[e-mail]>
Cc: Russell King <[e-mail]>
Cc: Guo Ren <[e-mail]>
Cc: Michal Simek <[e-mail]>
Cc: Dinh Nguyen <[e-mail]>
Cc: Jonas Bonn <[e-mail]>
Cc: Stefan Kristiansson <[e-mail]>
Cc: Stafford Horne <[e-mail]>
Cc: "James E.J. Bottomley" <[e-mail]>
Cc: Helge Deller <[e-mail]>
Cc: Michael Ellerman <[e-mail]>
Cc: Nicholas Piggin <[e-mail]>
Cc: Christophe Leroy <[e-mail]>
Cc: "Naveen N. Rao" <[e-mail]>
Cc: Yoshinori Sato <[e-mail]>
Cc: Rich Felker <[e-mail]>
Cc: John Paul Adrian Glaubitz <[e-mail]>
Cc: "David S. Miller" <[e-mail]>
Cc: Andreas Larsson <[e-mail]>
Cc: Thomas Gleixner <[e-mail]>
Cc: Ingo Molnar <[e-mail]>
Cc: Borislav Petkov <[e-mail]>
Cc: Dave Hansen <[e-mail]>
Cc: "H. Peter Anvin" <[e-mail]>
Cc: "Rafael J. Wysocki" <[e-mail]>
Cc: Daniel Lezcano <[e-mail]>
Cc: Peter Zijlstra <[e-mail]>
Cc: Juri Lelli <[e-mail]>
Cc: Vincent Guittot <[e-mail]>
Cc: Dietmar Eggemann <[e-mail]>
Cc: Steven Rostedt <[e-mail]>
Cc: Ben Segall <[e-mail]>
Cc: Mel Gorman <[e-mail]>
Cc: Daniel Bristot de Oliveira <[e-mail]>
Cc: Valentin Schneider <[e-mail]>
Cc: Andrew Donnellan <[e-mail]>
Cc: Benjamin Gray <[e-mail]>
Cc: Frederic Weisbecker <[e-mail]>
Cc: Xin Li <[e-mail]>
Cc: Kees Cook <[e-mail]>
Cc: Rick Edgecombe <[e-mail]>
Cc: Tony Battersby <[e-mail]>
Cc: Bjorn Helgaas <[e-mail]>
Cc: Brian Gerst <[e-mail]>
Cc: Leonardo Bras <[e-mail]>
Cc: Imran Khan <[e-mail]>
Cc: "Paul E. McKenney" <[e-mail]>
Cc: Rik van Riel <[e-mail]>
Cc: Tim Chen <[e-mail]>
Cc: David Vernet <[e-mail]>
Cc: Julia Lawall <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Signed-off-by: Gautham R. Shenoy <[e-mail]>
Co-developed-by: K Prateek Nayak <[e-mail]>
Signed-off-by: K Prateek Nayak <[e-mail]>
---
v1..v2:
o No changes.
---
 include/linux/thread_info.h | 43 +++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

Patch

diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h
index 9ea0b28068f4..1e10dd8c0227 100644
--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -195,6 +195,49 @@  static __always_inline bool tif_need_resched(void)
 
 #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */
 
+#ifdef TIF_NOTIFY_IPI
+
+#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H
+
+static __always_inline bool tif_notify_ipi(void)
+{
+	return arch_test_bit(TIF_NOTIFY_IPI,
+			     (unsigned long *)(&current_thread_info()->flags));
+}
+
+static __always_inline void current_clr_notify_ipi(void)
+{
+	arch_clear_bit(TIF_NOTIFY_IPI,
+		       (unsigned long *)(&current_thread_info()->flags));
+}
+
+#else
+
+static __always_inline bool tif_notify_ipi(void)
+{
+	return test_bit(TIF_NOTIFY_IPI,
+			(unsigned long *)(&current_thread_info()->flags));
+}
+
+static __always_inline void current_clr_notify_ipi(void)
+{
+	clear_bit(TIF_NOTIFY_IPI,
+		  (unsigned long *)(&current_thread_info()->flags));
+}
+
+#endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */
+
+#else /* !TIF_NOTIFY_IPI */
+
+static __always_inline bool tif_notify_ipi(void)
+{
+	return false;
+}
+
+static __always_inline void current_clr_notify_ipi(void) { }
+
+#endif /* TIF_NOTIFY_IPI */
+
 #ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES
 static inline int arch_within_stack_frames(const void * const stack,
 					   const void * const stackend,