| Message ID | 20231020-delay-verw-v1-2-cff54096326d@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Delay VERW | expand |
On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > + USER_CLEAR_CPU_BUFFERS > + > jmp .Lnative_iret > > > @@ -774,6 +780,9 @@ native_irq_return_ldt: > */ > popq %rax /* Restore user RAX */ > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > + USER_CLEAR_CPU_BUFFERS > + I'm thinking the comments add unnecessary noise here. The macro name is self-documenting enough. The detail about what mitigations are being done can go above the macro definition itself, which the reader can refer to if they want more detail about what the macro is doing and why. Speaking of the macro name, I think just "CLEAR_CPU_BUFFERS" is sufficient. The "USER_" prefix makes it harder to read IMO.
On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: > @@ -663,6 +665,10 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) > /* Restore RDI. */ > popq %rdi > swapgs > + > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > + USER_CLEAR_CPU_BUFFERS > + > jmp .Lnative_iret > > > @@ -774,6 +780,9 @@ native_irq_return_ldt: > */ > popq %rax /* Restore user RAX */ > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > + USER_CLEAR_CPU_BUFFERS > + Can the above two USER_CLEAR_CPU_BUFFERS be replaced with a single one just above native_irq_return_iret? Otherwise the native_irq_return_ldt case ends up getting two VERWs. > /* > * RSP now points to an ordinary IRET frame, except that the page > * is read-only and RSP[31:16] are preloaded with the userspace > @@ -1502,6 +1511,9 @@ nmi_restore: > std > movq $0, 5*8(%rsp) /* clear "NMI executing" */ > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > + USER_CLEAR_CPU_BUFFERS > + > /* > * iretq reads the "iret" frame and exits the NMI stack in a > * single instruction. We are returning to kernel mode, so this This isn't needed here. This is the NMI return-to-kernel path. The NMI return-to-user path is already mitigated as it goes through swapgs_restore_regs_and_return_to_usermode.
On Mon, Oct 23, 2023 at 11:22:11AM -0700, Josh Poimboeuf wrote: > On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + USER_CLEAR_CPU_BUFFERS > > + > > jmp .Lnative_iret > > > > > > @@ -774,6 +780,9 @@ native_irq_return_ldt: > > */ > > popq %rax /* Restore user RAX */ > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + USER_CLEAR_CPU_BUFFERS > > + > > I'm thinking the comments add unnecessary noise here. The macro name is > self-documenting enough. > > The detail about what mitigations are being done can go above the macro > definition itself, which the reader can refer to if they want more > detail about what the macro is doing and why. Sure, I will move the comments to definition. > Speaking of the macro name, I think just "CLEAR_CPU_BUFFERS" is > sufficient. The "USER_" prefix makes it harder to read IMO. Ok, will drop "USER_".
On 10/23/23 11:22, Josh Poimboeuf wrote: > On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: >> + /* Mitigate CPU data sampling attacks .e.g. MDS */ >> + USER_CLEAR_CPU_BUFFERS >> + >> jmp .Lnative_iret >> >> >> @@ -774,6 +780,9 @@ native_irq_return_ldt: >> */ >> popq %rax /* Restore user RAX */ >> >> + /* Mitigate CPU data sampling attacks .e.g. MDS */ >> + USER_CLEAR_CPU_BUFFERS >> + > > I'm thinking the comments add unnecessary noise here. The macro name is > self-documenting enough. > > The detail about what mitigations are being done can go above the macro > definition itself, which the reader can refer to if they want more > detail about what the macro is doing and why. > > Speaking of the macro name, I think just "CLEAR_CPU_BUFFERS" is > sufficient. The "USER_" prefix makes it harder to read IMO. Yes, please. The "USER_" prefix should be reserved for things that are uniquely for the unambiguous return-to-userspace paths.
On Mon, Oct 23, 2023 at 11:35:21AM -0700, Josh Poimboeuf wrote: > On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: > > @@ -663,6 +665,10 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) > > /* Restore RDI. */ > > popq %rdi > > swapgs > > + > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + USER_CLEAR_CPU_BUFFERS > > + > > jmp .Lnative_iret > > > > > > @@ -774,6 +780,9 @@ native_irq_return_ldt: > > */ > > popq %rax /* Restore user RAX */ > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + USER_CLEAR_CPU_BUFFERS > > + > > Can the above two USER_CLEAR_CPU_BUFFERS be replaced with a single one > just above native_irq_return_iret? Otherwise the native_irq_return_ldt > case ends up getting two VERWs. Wouldn't that make interrupts returning to kernel also execute VERWs? idtentry_body error_return restore_regs_and_return_to_kernel verw native_irq_return_ldt doesn't look to be a common case. Anyways, I will see how to remove the extra VERW. > > /* > > * RSP now points to an ordinary IRET frame, except that the page > > * is read-only and RSP[31:16] are preloaded with the userspace > > @@ -1502,6 +1511,9 @@ nmi_restore: > > std > > movq $0, 5*8(%rsp) /* clear "NMI executing" */ > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > + USER_CLEAR_CPU_BUFFERS > > + > > /* > > * iretq reads the "iret" frame and exits the NMI stack in a > > * single instruction. We are returning to kernel mode, so this > > This isn't needed here. This is the NMI return-to-kernel path. Yes, the VERW here can be omitted. But probably need to check if an NMI occuring between VERW and ring transition will still execute VERW after the NMI.
On Mon, Oct 23, 2023 at 02:04:10PM -0700, Pawan Gupta wrote: > On Mon, Oct 23, 2023 at 11:35:21AM -0700, Josh Poimboeuf wrote: > > On Fri, Oct 20, 2023 at 01:45:03PM -0700, Pawan Gupta wrote: > > > @@ -663,6 +665,10 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) > > > /* Restore RDI. */ > > > popq %rdi > > > swapgs > > > + > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > > + USER_CLEAR_CPU_BUFFERS > > > + > > > jmp .Lnative_iret > > > > > > > > > @@ -774,6 +780,9 @@ native_irq_return_ldt: > > > */ > > > popq %rax /* Restore user RAX */ > > > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > > + USER_CLEAR_CPU_BUFFERS > > > + > > > > Can the above two USER_CLEAR_CPU_BUFFERS be replaced with a single one > > just above native_irq_return_iret? Otherwise the native_irq_return_ldt > > case ends up getting two VERWs. > > Wouldn't that make interrupts returning to kernel also execute VERWs? > > idtentry_body > error_return > restore_regs_and_return_to_kernel > verw > > native_irq_return_ldt doesn't look to be a common case. Anyways, I will > see how to remove the extra VERW. Ah, right. > > > /* > > > * RSP now points to an ordinary IRET frame, except that the page > > > * is read-only and RSP[31:16] are preloaded with the userspace > > > @@ -1502,6 +1511,9 @@ nmi_restore: > > > std > > > movq $0, 5*8(%rsp) /* clear "NMI executing" */ > > > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > > + USER_CLEAR_CPU_BUFFERS > > > + > > > /* > > > * iretq reads the "iret" frame and exits the NMI stack in a > > > * single instruction. We are returning to kernel mode, so this > > > > This isn't needed here. This is the NMI return-to-kernel path. > > Yes, the VERW here can be omitted. But probably need to check if an NMI > occuring between VERW and ring transition will still execute VERW after > the NMI. That window does exist, though I'm not sure it's worth worrying about.
On Mon, Oct 23, 2023 at 02:47:52PM -0700, Josh Poimboeuf wrote: > > > > /* > > > > * RSP now points to an ordinary IRET frame, except that the page > > > > * is read-only and RSP[31:16] are preloaded with the userspace > > > > @@ -1502,6 +1511,9 @@ nmi_restore: > > > > std > > > > movq $0, 5*8(%rsp) /* clear "NMI executing" */ > > > > > > > > + /* Mitigate CPU data sampling attacks .e.g. MDS */ > > > > + USER_CLEAR_CPU_BUFFERS > > > > + > > > > /* > > > > * iretq reads the "iret" frame and exits the NMI stack in a > > > > * single instruction. We are returning to kernel mode, so this > > > > > > This isn't needed here. This is the NMI return-to-kernel path. > > > > Yes, the VERW here can be omitted. But probably need to check if an NMI > > occuring between VERW and ring transition will still execute VERW after > > the NMI. > > That window does exist, though I'm not sure it's worth worrying about. I am in favor of omitting the VERW here, unless someone objects with a rationale. IMO, precisely timing the NMIs in such a narrow window is impractical.
On 10/23/23 15:30, Pawan Gupta wrote: >>>>> /* >>>>> * iretq reads the "iret" frame and exits the NMI stack in a >>>>> * single instruction. We are returning to kernel mode, so this >>>> This isn't needed here. This is the NMI return-to-kernel path. >>> Yes, the VERW here can be omitted. But probably need to check if an NMI >>> occuring between VERW and ring transition will still execute VERW after >>> the NMI. >> That window does exist, though I'm not sure it's worth worrying about. > I am in favor of omitting the VERW here, unless someone objects with a > rationale. IMO, precisely timing the NMIs in such a narrow window is > impractical. I'd bet that given the right PMU event you could make this pretty reliable. But normal users can't do that by default. That leaves the NMI watchdog which (I bet) you can still time, but which is pretty low frequency. Are there any other NMI sources that a normal user can cause problems with? Let's at least leave a marker in here that folks can grep for: /* Skip CLEAR_CPU_BUFFERS since it will rarely help */ and some nice logic in the changelog that they can dig out if need be. But, basically it sounds like the logic is: 1. It's rare to get an NMI after VERW but before returning to userspace 2. There is no known way to make that NMI less rare or target it 3. It would take a large number of these precisely-timed NMIs to mount an actual attack. There's presumably not enough bandwidth. Anything else?
On Mon, Oct 23, 2023 at 03:45:41PM -0700, Dave Hansen wrote: > On 10/23/23 15:30, Pawan Gupta wrote: > >>>>> /* > >>>>> * iretq reads the "iret" frame and exits the NMI stack in a > >>>>> * single instruction. We are returning to kernel mode, so this > >>>> This isn't needed here. This is the NMI return-to-kernel path. > >>> Yes, the VERW here can be omitted. But probably need to check if an NMI > >>> occuring between VERW and ring transition will still execute VERW after > >>> the NMI. > >> That window does exist, though I'm not sure it's worth worrying about. > > I am in favor of omitting the VERW here, unless someone objects with a > > rationale. IMO, precisely timing the NMIs in such a narrow window is > > impractical. > > I'd bet that given the right PMU event you could make this pretty > reliable. But normal users can't do that by default. That leaves the > NMI watchdog which (I bet) you can still time, but which is pretty low > frequency. > > Are there any other NMI sources that a normal user can cause problems with? Generating recoverable parity check errors using rowhammer? But, thats probably going too far for very little gain. > Let's at least leave a marker in here that folks can grep for: > > /* Skip CLEAR_CPU_BUFFERS since it will rarely help */ Sure. > and some nice logic in the changelog that they can dig out if need be. > > But, basically it sounds like the logic is: > > 1. It's rare to get an NMI after VERW but before returning to userspace > 2. There is no known way to make that NMI less rare or target it > 3. It would take a large number of these precisely-timed NMIs to mount > an actual attack. There's presumably not enough bandwidth. Thanks for this. > Anything else? 4. The NMI in question occurs after a VERW, i.e. when user state is restored and most interesting data is already scrubbed. Whats left is only the data that NMI touches, and that may or may not be interesting.
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 43606de22511..e72ac30f0714 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -223,6 +223,8 @@ syscall_return_via_sysret: SYM_INNER_LABEL(entry_SYSRETQ_unsafe_stack, SYM_L_GLOBAL) ANNOTATE_NOENDBR swapgs + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS sysretq SYM_INNER_LABEL(entry_SYSRETQ_end, SYM_L_GLOBAL) ANNOTATE_NOENDBR @@ -663,6 +665,10 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) /* Restore RDI. */ popq %rdi swapgs + + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS + jmp .Lnative_iret @@ -774,6 +780,9 @@ native_irq_return_ldt: */ popq %rax /* Restore user RAX */ + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS + /* * RSP now points to an ordinary IRET frame, except that the page * is read-only and RSP[31:16] are preloaded with the userspace @@ -1502,6 +1511,9 @@ nmi_restore: std movq $0, 5*8(%rsp) /* clear "NMI executing" */ + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS + /* * iretq reads the "iret" frame and exits the NMI stack in a * single instruction. We are returning to kernel mode, so this @@ -1520,6 +1532,8 @@ SYM_CODE_START(ignore_sysret) UNWIND_HINT_END_OF_STACK ENDBR mov $-ENOSYS, %eax + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS sysretl SYM_CODE_END(ignore_sysret) #endif diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S index 70150298f8bd..d2ccd9148239 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -271,6 +271,8 @@ SYM_INNER_LABEL(entry_SYSRETL_compat_unsafe_stack, SYM_L_GLOBAL) xorl %r9d, %r9d xorl %r10d, %r10d swapgs + /* Mitigate CPU data sampling attacks .e.g. MDS */ + USER_CLEAR_CPU_BUFFERS sysretl SYM_INNER_LABEL(entry_SYSRETL_compat_end, SYM_L_GLOBAL) ANNOTATE_NOENDBR
Mitigation for MDS is to use VERW instruction to clear any secrets in CPU Buffers. Any memory accesses after VERW execution can still remain in CPU buffers. It is safer to execute VERW late in return to user path to minimize the window in which kernel data can end up in CPU buffers. There are not many kernel secrets to be had after SWITCH_TO_USER_CR3. Add support for deploying VERW mitigation after user register state is restored. This helps minimize the chances of kernel data ending up into CPU buffers after executing VERW. Note that the mitigation at the new location is not yet enabled. Suggested-by: Dave Hansen <dave.hansen@intel.com> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com> --- arch/x86/entry/entry_64.S | 14 ++++++++++++++ arch/x86/entry/entry_64_compat.S | 2 ++ 2 files changed, 16 insertions(+)