diff mbox series

[10/10] x86/rcu: Add THUNK rcu_read_unlock_special_thunk

Message ID 20240328075318.83039-11-jiangshanlai@gmail.com (mailing list archive)
State Superseded
Headers show
Series rcu/x86: Use per-cpu rcu preempt count | expand

Commit Message

Lai Jiangshan March 28, 2024, 7:53 a.m. UTC 
From: Lai Jiangshan <jiangshan.ljs@antgroup.com>

Add rcu_read_unlock_special_thunk(), so that the inlined rcu_read_unlock()
doesn't need any code to save the caller-saved registers.

Make rcu_read_unlock() only two instructions in the slow path at the
caller site.

Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Lai Jiangshan <jiangshan.ljs@antgroup.com>
---
 arch/x86/entry/thunk_64.S          | 5 +++++
 arch/x86/include/asm/rcu_preempt.h | 4 +++-
 2 files changed, 8 insertions(+), 1 deletion(-)

Comments

kernel test robot March 29, 2024, 2:46 p.m. UTC | #1 
Hi Lai,

kernel test robot noticed the following build errors:

[auto build test ERROR on paulmck-rcu/dev]
[also build test ERROR on tip/locking/core tip/sched/core tip/x86/asm tip/master linus/master v6.9-rc1 next-20240328]
[cannot apply to tip/x86/core tip/auto-latest]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Lai-Jiangshan/lib-Use-rcu_preempt_depth-to-replace-current-rcu_read_lock_nesting/20240328-155513
base:   https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev
patch link:    https://lore.kernel.org/r/20240328075318.83039-11-jiangshanlai%40gmail.com
patch subject: [PATCH 10/10] x86/rcu: Add THUNK rcu_read_unlock_special_thunk
config: i386-alldefconfig (https://download.01.org/0day-ci/archive/20240329/202403292233.1m5tWJY5-lkp@intel.com/config)
compiler: gcc-12 (Ubuntu 12.3.0-9ubuntu2) 12.3.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240329/202403292233.1m5tWJY5-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202403292233.1m5tWJY5-lkp@intel.com/

All errors (new ones prefixed by >>):

   ld: init/main.o: in function `rest_init':
>> main.c:(.ref.text+0x96): undefined reference to `rcu_read_unlock_special_thunk'
>> ld: main.c:(.ref.text+0x9d): undefined reference to `rcu_read_unlock_special_thunk'
   ld: arch/x86/events/intel/ds.o: in function `intel_pmu_drain_bts_buffer':
>> ds.c:(.text+0x2cd9): undefined reference to `rcu_read_unlock_special_thunk'
   ld: arch/x86/kernel/nmi.o: in function `nmi_handle':
>> nmi.c:(.text+0x2e9): undefined reference to `rcu_read_unlock_special_thunk'
   ld: arch/x86/kernel/alternative.o: in function `__text_poke':
>> alternative.c:(.text+0x531): undefined reference to `rcu_read_unlock_special_thunk'
   ld: arch/x86/mm/init.o:init.c:(.init.text+0xf1): more undefined references to `rcu_read_unlock_special_thunk' follow
diff mbox series

Patch

diff --git a/arch/x86/entry/thunk_64.S b/arch/x86/entry/thunk_64.S
index 119ebdc3d362..10c60369a67c 100644
--- a/arch/x86/entry/thunk_64.S
+++ b/arch/x86/entry/thunk_64.S
@@ -13,3 +13,8 @@  THUNK preempt_schedule_thunk, preempt_schedule
 THUNK preempt_schedule_notrace_thunk, preempt_schedule_notrace
 EXPORT_SYMBOL(preempt_schedule_thunk)
 EXPORT_SYMBOL(preempt_schedule_notrace_thunk)
+
+#ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT
+THUNK rcu_read_unlock_special_thunk, rcu_read_unlock_special
+EXPORT_SYMBOL_GPL(rcu_read_unlock_special_thunk)
+#endif /* #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT */
diff --git a/arch/x86/include/asm/rcu_preempt.h b/arch/x86/include/asm/rcu_preempt.h
index cb25ebe038a5..acdd73b74c05 100644
--- a/arch/x86/include/asm/rcu_preempt.h
+++ b/arch/x86/include/asm/rcu_preempt.h
@@ -97,9 +97,11 @@  static __always_inline bool pcpu_rcu_preempt_count_dec_and_test(void)
 			       __percpu_arg([var]));
 }
 
+extern asmlinkage void rcu_read_unlock_special_thunk(void);
+
 #define pcpu_rcu_read_unlock_special()						\
 do {										\
-	rcu_read_unlock_special();						\
+	asm volatile ("call rcu_read_unlock_special_thunk" : ASM_CALL_CONSTRAINT);\
 } while (0)
 
 #endif // #ifdef CONFIG_PCPU_RCU_PREEMPT_COUNT