| Message ID | 20240903-fix_fencei_optimization-v2-1-8025f20171fc@rivosinc.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 7c1e5b9690b0e14acead4ff98d8a6c40f2dff54b |
| Headers | show |
| Series | [v2] riscv: Disable preemption while handling PR_RISCV_CTX_SW_FENCEI_OFF | expand |
Hello: This patch was applied to riscv/linux.git (fixes) by Palmer Dabbelt <palmer@rivosinc.com>: On Tue, 03 Sep 2024 15:52:34 -0700 you wrote: > The icache will be flushed in switch_to() if force_icache_flush is true, > or in flush_icache_deferred() if icache_stale_mask is set. Between > setting force_icache_flush to false and calculating the new > icache_stale_mask, preemption needs to be disabled. There are two > reasons for this: > > 1. If CPU migration happens between force_icache_flush = false, and the > icache_stale_mask is set, an icache flush will not be emitted. > 2. smp_processor_id() is used in set_icache_stale_mask() to mark the > current CPU as not needing another flush since a flush will have > happened either by userspace or by the kernel when performing the > migration. smp_processor_id() is currently called twice with preemption > enabled which causes a race condition. It allows > icache_stale_mask to be populated with inconsistent CPU ids. > > [...] Here is the summary with links: - [v2] riscv: Disable preemption while handling PR_RISCV_CTX_SW_FENCEI_OFF https://git.kernel.org/riscv/c/7c1e5b9690b0 You are awesome, thank you!
diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index a03c994eed3b..b81672729887 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -158,6 +158,7 @@ void __init riscv_init_cbo_blocksizes(void) #ifdef CONFIG_SMP static void set_icache_stale_mask(void) { + int cpu = get_cpu(); cpumask_t *mask; bool stale_cpu; @@ -168,10 +169,11 @@ static void set_icache_stale_mask(void) * concurrently on different harts. */ mask = ¤t->mm->context.icache_stale_mask; - stale_cpu = cpumask_test_cpu(smp_processor_id(), mask); + stale_cpu = cpumask_test_cpu(cpu, mask); cpumask_setall(mask); - cpumask_assign_cpu(smp_processor_id(), mask, stale_cpu); + cpumask_assign_cpu(cpu, mask, stale_cpu); + put_cpu(); } #endif @@ -239,14 +241,12 @@ int riscv_set_icache_flush_ctx(unsigned long ctx, unsigned long scope) case PR_RISCV_CTX_SW_FENCEI_OFF: switch (scope) { case PR_RISCV_SCOPE_PER_PROCESS: - current->mm->context.force_icache_flush = false; - set_icache_stale_mask(); + current->mm->context.force_icache_flush = false; break; case PR_RISCV_SCOPE_PER_THREAD: - current->thread.force_icache_flush = false; - set_icache_stale_mask(); + current->thread.force_icache_flush = false; break; default: return -EINVAL;
The icache will be flushed in switch_to() if force_icache_flush is true, or in flush_icache_deferred() if icache_stale_mask is set. Between setting force_icache_flush to false and calculating the new icache_stale_mask, preemption needs to be disabled. There are two reasons for this: 1. If CPU migration happens between force_icache_flush = false, and the icache_stale_mask is set, an icache flush will not be emitted. 2. smp_processor_id() is used in set_icache_stale_mask() to mark the current CPU as not needing another flush since a flush will have happened either by userspace or by the kernel when performing the migration. smp_processor_id() is currently called twice with preemption enabled which causes a race condition. It allows icache_stale_mask to be populated with inconsistent CPU ids. Resolve these two issues by setting the icache_stale_mask before setting force_icache_flush to false, and using get_cpu()/put_cpu() to obtain the smp_processor_id(). Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> Fixes: 6b9391b581fd ("riscv: Include riscv_set_icache_flush_ctx prctl") --- Changes in v2: - This patch has been split into a different series from the other patch in the v1. This patch is unchanged. --- arch/riscv/mm/cacheflush.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) --- base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba change-id: 20240812-fix_fencei_optimization-3f81ac200505