| Message ID | 20221030124517.2370-1-jszhang@kernel.org (mailing list archive) |
|---|---|
| State | Under Review, archived |
| Delegated to: | Palmer Dabbelt |
| Headers | show |
| Series | [v3] riscv: fix race when vmap stack overflow | expand |
Reviewed-by: Guo Ren <guoren@kernel.org> On Sun, Oct 30, 2022 at 8:54 PM Jisheng Zhang <jszhang@kernel.org> wrote: > > Currently, when detecting vmap stack overflow, riscv firstly switches > to the so called shadow stack, then use this shadow stack to call the > get_overflow_stack() to get the overflow stack. However, there's > a race here if two or more harts use the same shadow stack at the same > time. > > To solve this race, we introduce spin_shadow_stack atomic var, which > will be swap between its own address and 0 in atomic way, when the > var is set, it means the shadow_stack is being used; when the var > is cleared, it means the shadow_stack isn't being used. > > Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") > Signed-off-by: Jisheng Zhang <jszhang@kernel.org> > Suggested-by: Guo Ren <guoren@kernel.org> > --- > Since v2: > - use REG_AMOSWAP > - add comment to the purpose of smp_store_release() > > Since v1: > - use smp_store_release directly > - use unsigned int instead of atomic_t > > > arch/riscv/include/asm/asm.h | 1 + > arch/riscv/kernel/entry.S | 4 ++++ > arch/riscv/kernel/traps.c | 9 +++++++++ > 3 files changed, 14 insertions(+) > > diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h > index 1b471ff73178..acf563072b8b 100644 > --- a/arch/riscv/include/asm/asm.h > +++ b/arch/riscv/include/asm/asm.h > @@ -23,6 +23,7 @@ > #define REG_L __REG_SEL(ld, lw) > #define REG_S __REG_SEL(sd, sw) > #define REG_SC __REG_SEL(sc.d, sc.w) > +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w) > #define REG_ASM __REG_SEL(.dword, .word) > #define SZREG __REG_SEL(8, 4) > #define LGREG __REG_SEL(3, 2) > diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S > index b9eda3fcbd6d..ea6b78dac739 100644 > --- a/arch/riscv/kernel/entry.S > +++ b/arch/riscv/kernel/entry.S > @@ -404,6 +404,10 @@ handle_syscall_trace_exit: > > #ifdef CONFIG_VMAP_STACK > handle_kernel_stack_overflow: > +1: la sp, spin_shadow_stack > + REG_AMOSWAP sp, sp, (sp) > + bnez sp, 1b > + > la sp, shadow_stack > addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE > > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c > index f3e96d60a2ff..dea47f329708 100644 > --- a/arch/riscv/kernel/traps.c > +++ b/arch/riscv/kernel/traps.c > @@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void) > OVERFLOW_STACK_SIZE; > } > > +unsigned long spin_shadow_stack; > + > asmlinkage void handle_bad_stack(struct pt_regs *regs) > { > unsigned long tsk_stk = (unsigned long)current->stack; > unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack); > > + /* > + * to ensure spin flag is set after the sp is used in entry.S: > + * //load per-cpu overflow stack. > + * REG_L sp, -8(sp) > + */ > + smp_store_release(&spin_shadow_stack, 0); > + > console_verbose(); > > pr_emerg("Insufficient stack space to handle exception!\n"); > -- > 2.37.2 >
On Sun, Oct 30, 2022 at 09:23:51PM +0800, Guo Ren wrote: > Reviewed-by: Guo Ren <guoren@kernel.org> > Gentle ping? This patch is still missing in mainline. > On Sun, Oct 30, 2022 at 8:54 PM Jisheng Zhang <jszhang@kernel.org> wrote: > > > > Currently, when detecting vmap stack overflow, riscv firstly switches > > to the so called shadow stack, then use this shadow stack to call the > > get_overflow_stack() to get the overflow stack. However, there's > > a race here if two or more harts use the same shadow stack at the same > > time. > > > > To solve this race, we introduce spin_shadow_stack atomic var, which > > will be swap between its own address and 0 in atomic way, when the > > var is set, it means the shadow_stack is being used; when the var > > is cleared, it means the shadow_stack isn't being used. > > > > Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") > > Signed-off-by: Jisheng Zhang <jszhang@kernel.org> > > Suggested-by: Guo Ren <guoren@kernel.org> > > --- > > Since v2: > > - use REG_AMOSWAP > > - add comment to the purpose of smp_store_release() > > > > Since v1: > > - use smp_store_release directly > > - use unsigned int instead of atomic_t > > > > > > arch/riscv/include/asm/asm.h | 1 + > > arch/riscv/kernel/entry.S | 4 ++++ > > arch/riscv/kernel/traps.c | 9 +++++++++ > > 3 files changed, 14 insertions(+) > > > > diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h > > index 1b471ff73178..acf563072b8b 100644 > > --- a/arch/riscv/include/asm/asm.h > > +++ b/arch/riscv/include/asm/asm.h > > @@ -23,6 +23,7 @@ > > #define REG_L __REG_SEL(ld, lw) > > #define REG_S __REG_SEL(sd, sw) > > #define REG_SC __REG_SEL(sc.d, sc.w) > > +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w) > > #define REG_ASM __REG_SEL(.dword, .word) > > #define SZREG __REG_SEL(8, 4) > > #define LGREG __REG_SEL(3, 2) > > diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S > > index b9eda3fcbd6d..ea6b78dac739 100644 > > --- a/arch/riscv/kernel/entry.S > > +++ b/arch/riscv/kernel/entry.S > > @@ -404,6 +404,10 @@ handle_syscall_trace_exit: > > > > #ifdef CONFIG_VMAP_STACK > > handle_kernel_stack_overflow: > > +1: la sp, spin_shadow_stack > > + REG_AMOSWAP sp, sp, (sp) > > + bnez sp, 1b > > + > > la sp, shadow_stack > > addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE > > > > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c > > index f3e96d60a2ff..dea47f329708 100644 > > --- a/arch/riscv/kernel/traps.c > > +++ b/arch/riscv/kernel/traps.c > > @@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void) > > OVERFLOW_STACK_SIZE; > > } > > > > +unsigned long spin_shadow_stack; > > + > > asmlinkage void handle_bad_stack(struct pt_regs *regs) > > { > > unsigned long tsk_stk = (unsigned long)current->stack; > > unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack); > > > > + /* > > + * to ensure spin flag is set after the sp is used in entry.S: > > + * //load per-cpu overflow stack. > > + * REG_L sp, -8(sp) > > + */ > > + smp_store_release(&spin_shadow_stack, 0); > > + > > console_verbose(); > > > > pr_emerg("Insufficient stack space to handle exception!\n"); > > -- > > 2.37.2 > > > > > -- > Best Regards > Guo Ren
On Wed, 23 Nov 2022 07:23:51 PST (-0800), jszhang@kernel.org wrote: > On Sun, Oct 30, 2022 at 09:23:51PM +0800, Guo Ren wrote: >> Reviewed-by: Guo Ren <guoren@kernel.org> >> > > Gentle ping? This patch is still missing in mainline. Sorry, this one took a few looks to figure out. I just sent out a v4, it's pretty much the same thing but by the time I'd figured out what this was doing it seemed easier to just write up the comments directly. https://lore.kernel.org/r/20221130022442.18358-1-palmer@rivosinc.com/ I'll try and watch the threads, LMK if I screwed anything up. I've already got this queued up in staging so it's being tested, should be fast to take it onto fixes. Thanks! > >> On Sun, Oct 30, 2022 at 8:54 PM Jisheng Zhang <jszhang@kernel.org> wrote: >> > >> > Currently, when detecting vmap stack overflow, riscv firstly switches >> > to the so called shadow stack, then use this shadow stack to call the >> > get_overflow_stack() to get the overflow stack. However, there's >> > a race here if two or more harts use the same shadow stack at the same >> > time. >> > >> > To solve this race, we introduce spin_shadow_stack atomic var, which >> > will be swap between its own address and 0 in atomic way, when the >> > var is set, it means the shadow_stack is being used; when the var >> > is cleared, it means the shadow_stack isn't being used. >> > >> > Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") >> > Signed-off-by: Jisheng Zhang <jszhang@kernel.org> >> > Suggested-by: Guo Ren <guoren@kernel.org> >> > --- >> > Since v2: >> > - use REG_AMOSWAP >> > - add comment to the purpose of smp_store_release() >> > >> > Since v1: >> > - use smp_store_release directly >> > - use unsigned int instead of atomic_t >> > >> > >> > arch/riscv/include/asm/asm.h | 1 + >> > arch/riscv/kernel/entry.S | 4 ++++ >> > arch/riscv/kernel/traps.c | 9 +++++++++ >> > 3 files changed, 14 insertions(+) >> > >> > diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h >> > index 1b471ff73178..acf563072b8b 100644 >> > --- a/arch/riscv/include/asm/asm.h >> > +++ b/arch/riscv/include/asm/asm.h >> > @@ -23,6 +23,7 @@ >> > #define REG_L __REG_SEL(ld, lw) >> > #define REG_S __REG_SEL(sd, sw) >> > #define REG_SC __REG_SEL(sc.d, sc.w) >> > +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w) >> > #define REG_ASM __REG_SEL(.dword, .word) >> > #define SZREG __REG_SEL(8, 4) >> > #define LGREG __REG_SEL(3, 2) >> > diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S >> > index b9eda3fcbd6d..ea6b78dac739 100644 >> > --- a/arch/riscv/kernel/entry.S >> > +++ b/arch/riscv/kernel/entry.S >> > @@ -404,6 +404,10 @@ handle_syscall_trace_exit: >> > >> > #ifdef CONFIG_VMAP_STACK >> > handle_kernel_stack_overflow: >> > +1: la sp, spin_shadow_stack >> > + REG_AMOSWAP sp, sp, (sp) >> > + bnez sp, 1b >> > + >> > la sp, shadow_stack >> > addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE >> > >> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c >> > index f3e96d60a2ff..dea47f329708 100644 >> > --- a/arch/riscv/kernel/traps.c >> > +++ b/arch/riscv/kernel/traps.c >> > @@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void) >> > OVERFLOW_STACK_SIZE; >> > } >> > >> > +unsigned long spin_shadow_stack; >> > + >> > asmlinkage void handle_bad_stack(struct pt_regs *regs) >> > { >> > unsigned long tsk_stk = (unsigned long)current->stack; >> > unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack); >> > >> > + /* >> > + * to ensure spin flag is set after the sp is used in entry.S: >> > + * //load per-cpu overflow stack. >> > + * REG_L sp, -8(sp) >> > + */ >> > + smp_store_release(&spin_shadow_stack, 0); >> > + >> > console_verbose(); >> > >> > pr_emerg("Insufficient stack space to handle exception!\n"); >> > -- >> > 2.37.2 >> > >> >> >> -- >> Best Regards >> Guo Ren
diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h index 1b471ff73178..acf563072b8b 100644 --- a/arch/riscv/include/asm/asm.h +++ b/arch/riscv/include/asm/asm.h @@ -23,6 +23,7 @@ #define REG_L __REG_SEL(ld, lw) #define REG_S __REG_SEL(sd, sw) #define REG_SC __REG_SEL(sc.d, sc.w) +#define REG_AMOSWAP __REG_SEL(amoswap.d, amoswap.w) #define REG_ASM __REG_SEL(.dword, .word) #define SZREG __REG_SEL(8, 4) #define LGREG __REG_SEL(3, 2) diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index b9eda3fcbd6d..ea6b78dac739 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -404,6 +404,10 @@ handle_syscall_trace_exit: #ifdef CONFIG_VMAP_STACK handle_kernel_stack_overflow: +1: la sp, spin_shadow_stack + REG_AMOSWAP sp, sp, (sp) + bnez sp, 1b + la sp, shadow_stack addi sp, sp, SHADOW_OVERFLOW_STACK_SIZE diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index f3e96d60a2ff..dea47f329708 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -221,11 +221,20 @@ asmlinkage unsigned long get_overflow_stack(void) OVERFLOW_STACK_SIZE; } +unsigned long spin_shadow_stack; + asmlinkage void handle_bad_stack(struct pt_regs *regs) { unsigned long tsk_stk = (unsigned long)current->stack; unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack); + /* + * to ensure spin flag is set after the sp is used in entry.S: + * //load per-cpu overflow stack. + * REG_L sp, -8(sp) + */ + smp_store_release(&spin_shadow_stack, 0); + console_verbose(); pr_emerg("Insufficient stack space to handle exception!\n");
Currently, when detecting vmap stack overflow, riscv firstly switches to the so called shadow stack, then use this shadow stack to call the get_overflow_stack() to get the overflow stack. However, there's a race here if two or more harts use the same shadow stack at the same time. To solve this race, we introduce spin_shadow_stack atomic var, which will be swap between its own address and 0 in atomic way, when the var is set, it means the shadow_stack is being used; when the var is cleared, it means the shadow_stack isn't being used. Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Suggested-by: Guo Ren <guoren@kernel.org> --- Since v2: - use REG_AMOSWAP - add comment to the purpose of smp_store_release() Since v1: - use smp_store_release directly - use unsigned int instead of atomic_t arch/riscv/include/asm/asm.h | 1 + arch/riscv/kernel/entry.S | 4 ++++ arch/riscv/kernel/traps.c | 9 +++++++++ 3 files changed, 14 insertions(+)