| Message ID | 20240701231027.61930-1-andrii@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | perf,x86: avoid missing caller address in stack traces captured in uprobe | expand |
+Josj +LKML On Mon, Jul 01, 2024 at 04:10:27PM -0700, Andrii Nakryiko wrote: > When tracing user functions with uprobe functionality, it's common to > install the probe (e.g., a BPF program) at the first instruction of the > function. This is often going to be `push %rbp` instruction in function > preamble, which means that within that function frame pointer hasn't > been established yet. This leads to consistently missing an actual > caller of the traced function, because perf_callchain_user() only > records current IP (capturing traced function) and then following frame > pointer chain (which would be caller's frame, containing the address of > caller's caller). > > So when we have target_1 -> target_2 -> target_3 call chain and we are > tracing an entry to target_3, captured stack trace will report > target_1 -> target_3 call chain, which is wrong and confusing. > > This patch proposes a x86-64-specific heuristic to detect `push %rbp` > instruction being traced. Given entire kernel implementation of user > space stack trace capturing works under assumption that user space code > was compiled with frame pointer register (%rbp) preservation, it seems > pretty reasonable to use this instruction as a strong indicator that > this is the entry to the function. In that case, return address is still > pointed to by %rsp, so we fetch it and add to stack trace before > proceeding to unwind the rest using frame pointer-based logic. > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org> > --- > arch/x86/events/core.c | 20 ++++++++++++++++++++ > include/linux/uprobes.h | 2 ++ > kernel/events/uprobes.c | 2 ++ > 3 files changed, 24 insertions(+) > > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c > index 5b0dd07b1ef1..82d5570b58ff 100644 > --- a/arch/x86/events/core.c > +++ b/arch/x86/events/core.c > @@ -2884,6 +2884,26 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs > return; > > pagefault_disable(); > + > +#ifdef CONFIG_UPROBES > + /* > + * If we are called from uprobe handler, and we are indeed at the very > + * entry to user function (which is normally a `push %rbp` instruction, > + * under assumption of application being compiled with frame pointers), > + * we should read return address from *regs->sp before proceeding > + * to follow frame pointers, otherwise we'll skip immediate caller > + * as %rbp is not yet setup. > + */ > + if (current->utask) { > + struct arch_uprobe *auprobe = current->utask->auprobe; > + u64 ret_addr; > + > + if (auprobe && auprobe->insn[0] == 0x55 /* push %rbp */ && > + !__get_user(ret_addr, (const u64 __user *)regs->sp)) This u64 is wrong, perf_callchain_user() is always native size. Additionally, I suppose you should also add a hunk to perf_callchain_user32(), which is the compat case. > + perf_callchain_store(entry, ret_addr); > + } > +#endif > + > while (entry->nr < entry->max_stack) { > if (!valid_user_frame(fp, sizeof(frame))) > break; > diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h > index b503fafb7fb3..a270a5892ab4 100644 > --- a/include/linux/uprobes.h > +++ b/include/linux/uprobes.h > @@ -76,6 +76,8 @@ struct uprobe_task { > struct uprobe *active_uprobe; > unsigned long xol_vaddr; > > + struct arch_uprobe *auprobe; > + > struct return_instance *return_instances; > unsigned int depth; > }; > diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c > index 99be2adedbc0..6e22e4d80f1e 100644 > --- a/kernel/events/uprobes.c > +++ b/kernel/events/uprobes.c > @@ -2082,6 +2082,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) > bool need_prep = false; /* prepare return uprobe, when needed */ > > down_read(&uprobe->register_rwsem); > + current->utask->auprobe = &uprobe->arch; > for (uc = uprobe->consumers; uc; uc = uc->next) { > int rc = 0; > > @@ -2096,6 +2097,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) > > remove &= rc; > } > + current->utask->auprobe = NULL; > > if (need_prep && !remove) > prepare_uretprobe(uprobe, regs); /* put bp at return */ > -- > 2.43.0 >
On Tue, Jul 2, 2024 at 2:50 AM Peter Zijlstra <peterz@infradead.org> wrote: > > > +Josj +LKML > ack, will add for next revision > On Mon, Jul 01, 2024 at 04:10:27PM -0700, Andrii Nakryiko wrote: > > When tracing user functions with uprobe functionality, it's common to > > install the probe (e.g., a BPF program) at the first instruction of the > > function. This is often going to be `push %rbp` instruction in function > > preamble, which means that within that function frame pointer hasn't > > been established yet. This leads to consistently missing an actual > > caller of the traced function, because perf_callchain_user() only > > records current IP (capturing traced function) and then following frame > > pointer chain (which would be caller's frame, containing the address of > > caller's caller). > > > > So when we have target_1 -> target_2 -> target_3 call chain and we are > > tracing an entry to target_3, captured stack trace will report > > target_1 -> target_3 call chain, which is wrong and confusing. > > > > This patch proposes a x86-64-specific heuristic to detect `push %rbp` > > instruction being traced. Given entire kernel implementation of user > > space stack trace capturing works under assumption that user space code > > was compiled with frame pointer register (%rbp) preservation, it seems > > pretty reasonable to use this instruction as a strong indicator that > > this is the entry to the function. In that case, return address is still > > pointed to by %rsp, so we fetch it and add to stack trace before > > proceeding to unwind the rest using frame pointer-based logic. > > > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org> > > --- > > arch/x86/events/core.c | 20 ++++++++++++++++++++ > > include/linux/uprobes.h | 2 ++ > > kernel/events/uprobes.c | 2 ++ > > 3 files changed, 24 insertions(+) > > > > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c > > index 5b0dd07b1ef1..82d5570b58ff 100644 > > --- a/arch/x86/events/core.c > > +++ b/arch/x86/events/core.c > > @@ -2884,6 +2884,26 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs > > return; > > > > pagefault_disable(); > > + > > +#ifdef CONFIG_UPROBES > > + /* > > + * If we are called from uprobe handler, and we are indeed at the very > > + * entry to user function (which is normally a `push %rbp` instruction, > > + * under assumption of application being compiled with frame pointers), > > + * we should read return address from *regs->sp before proceeding > > + * to follow frame pointers, otherwise we'll skip immediate caller > > + * as %rbp is not yet setup. > > + */ > > + if (current->utask) { > > + struct arch_uprobe *auprobe = current->utask->auprobe; > > + u64 ret_addr; > > + > > + if (auprobe && auprobe->insn[0] == 0x55 /* push %rbp */ && > > + !__get_user(ret_addr, (const u64 __user *)regs->sp)) > > This u64 is wrong, perf_callchain_user() is always native size. > > Additionally, I suppose you should also add a hunk to > perf_callchain_user32(), which is the compat case. > Ah, I misunderstood the purpose of perf_callchain_user32(), and so assumed u64 is correct here. I get it now, perf_callchain_user32() is compat 32-in-64 case, but the general case can be either 32 or 64 bit. Will fix it, thanks! > > + perf_callchain_store(entry, ret_addr); > > + } > > +#endif > > + > > while (entry->nr < entry->max_stack) { > > if (!valid_user_frame(fp, sizeof(frame))) > > break; > > diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h > > index b503fafb7fb3..a270a5892ab4 100644 > > --- a/include/linux/uprobes.h > > +++ b/include/linux/uprobes.h > > @@ -76,6 +76,8 @@ struct uprobe_task { > > struct uprobe *active_uprobe; > > unsigned long xol_vaddr; > > > > + struct arch_uprobe *auprobe; > > + > > struct return_instance *return_instances; > > unsigned int depth; > > }; > > diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c > > index 99be2adedbc0..6e22e4d80f1e 100644 > > --- a/kernel/events/uprobes.c > > +++ b/kernel/events/uprobes.c > > @@ -2082,6 +2082,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) > > bool need_prep = false; /* prepare return uprobe, when needed */ > > > > down_read(&uprobe->register_rwsem); > > + current->utask->auprobe = &uprobe->arch; > > for (uc = uprobe->consumers; uc; uc = uc->next) { > > int rc = 0; > > > > @@ -2096,6 +2097,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) > > > > remove &= rc; > > } > > + current->utask->auprobe = NULL; > > > > if (need_prep && !remove) > > prepare_uretprobe(uprobe, regs); /* put bp at return */ > > -- > > 2.43.0 > >
diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5b0dd07b1ef1..82d5570b58ff 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2884,6 +2884,26 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs return; pagefault_disable(); + +#ifdef CONFIG_UPROBES + /* + * If we are called from uprobe handler, and we are indeed at the very + * entry to user function (which is normally a `push %rbp` instruction, + * under assumption of application being compiled with frame pointers), + * we should read return address from *regs->sp before proceeding + * to follow frame pointers, otherwise we'll skip immediate caller + * as %rbp is not yet setup. + */ + if (current->utask) { + struct arch_uprobe *auprobe = current->utask->auprobe; + u64 ret_addr; + + if (auprobe && auprobe->insn[0] == 0x55 /* push %rbp */ && + !__get_user(ret_addr, (const u64 __user *)regs->sp)) + perf_callchain_store(entry, ret_addr); + } +#endif + while (entry->nr < entry->max_stack) { if (!valid_user_frame(fp, sizeof(frame))) break; diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index b503fafb7fb3..a270a5892ab4 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -76,6 +76,8 @@ struct uprobe_task { struct uprobe *active_uprobe; unsigned long xol_vaddr; + struct arch_uprobe *auprobe; + struct return_instance *return_instances; unsigned int depth; }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 99be2adedbc0..6e22e4d80f1e 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2082,6 +2082,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) bool need_prep = false; /* prepare return uprobe, when needed */ down_read(&uprobe->register_rwsem); + current->utask->auprobe = &uprobe->arch; for (uc = uprobe->consumers; uc; uc = uc->next) { int rc = 0; @@ -2096,6 +2097,7 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) remove &= rc; } + current->utask->auprobe = NULL; if (need_prep && !remove) prepare_uretprobe(uprobe, regs); /* put bp at return */
When tracing user functions with uprobe functionality, it's common to install the probe (e.g., a BPF program) at the first instruction of the function. This is often going to be `push %rbp` instruction in function preamble, which means that within that function frame pointer hasn't been established yet. This leads to consistently missing an actual caller of the traced function, because perf_callchain_user() only records current IP (capturing traced function) and then following frame pointer chain (which would be caller's frame, containing the address of caller's caller). So when we have target_1 -> target_2 -> target_3 call chain and we are tracing an entry to target_3, captured stack trace will report target_1 -> target_3 call chain, which is wrong and confusing. This patch proposes a x86-64-specific heuristic to detect `push %rbp` instruction being traced. Given entire kernel implementation of user space stack trace capturing works under assumption that user space code was compiled with frame pointer register (%rbp) preservation, it seems pretty reasonable to use this instruction as a strong indicator that this is the entry to the function. In that case, return address is still pointed to by %rsp, so we fetch it and add to stack trace before proceeding to unwind the rest using frame pointer-based logic. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> --- arch/x86/events/core.c | 20 ++++++++++++++++++++ include/linux/uprobes.h | 2 ++ kernel/events/uprobes.c | 2 ++ 3 files changed, 24 insertions(+)