From patchwork Thu Apr 20 11:25:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 13218548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 976B1C77B76 for ; Thu, 20 Apr 2023 11:26:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234440AbjDTL0g (ORCPT ); Thu, 20 Apr 2023 07:26:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234119AbjDTL0V (ORCPT ); Thu, 20 Apr 2023 07:26:21 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBAA91FF9; Thu, 20 Apr 2023 04:25:37 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 759B4617F4; Thu, 20 Apr 2023 11:25:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88365C433EF; Thu, 20 Apr 2023 11:25:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681989936; bh=TB7j0KwDnAm0k2wPYr3mSy3J/1Abax6W0iRp8ay6JNM=; h=From:To:Cc:Subject:Date:From; b=DoEidhVOQjKJN4MyDOIuWH/ojO3qAWFNDU8nFcxQeiB5SkZEkmL63TmAcEjcDCzsL WcIdjfO2fs7rKez+9Lr17kjHZXQeCeqjf2dLrHHtObOH4IRmbG0eRFzGWPr9cxGz5z EELgYqVvZzFbsxzfwV6fBGypathB4estsmr/9DA5Q9/3oYRFbis3gLt7U7rVxEkweN f6m9eSyK4b9l3RxKoY8msMfeFhnNIVSd7eLCDIOV4sVsADMzNe4TpT3KISHQ5C5TCb WoZ2X9u8GPkhqXbFfgY8LexdNeIPfOLc39eNTZMsvRMhJBKsgycH+7ltaOkI3T4bBf bT4HwbJqJ8B9w== From: "Masami Hiramatsu (Google)" To: linux-trace-kernel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Steven Rostedt , mhiramat@kernel.org, Florent Revest , Mark Rutland , Will Deacon , Mathieu Desnoyers , Martin KaFai Lau , bpf@vger.kernel.org Subject: [PATCH v5 0/9] tracing: Add fprobe events Date: Thu, 20 Apr 2023 20:25:31 +0900 Message-ID: <168198993129.1795549.8306571027057356176.stgit@mhiramat.roam.corp.google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-kernel@vger.kernel.org Hi, Here is the 5th version of improve fprobe and add a basic fprobe event support for ftrace (tracefs) and perf. Here is the previous version. https://lore.kernel.org/all/168035963900.397811.6647648816464443553.stgit@mhiramat.roam.corp.google.com/ I updated the first 3 patches and added 6 additional patches to support tracepoint probe (fprobe on tracepoint) and BTF support for function entry and tracepoint. This allows us to trace function entries with parameter names (if BTF is available). With this fprobe events, we can continue to trace function entry/exit even if the CONFIG_KPROBES_ON_FTRACE is not available. Since CONFIG_KPROBES_ON_FTRACE requires the CONFIG_DYNAMIC_FTRACE_WITH_REGS, it is not available if the architecture only supports CONFIG_DYNAMIC_FTRACE_WITH_ARGS (e.g. arm64). And that means kprobe events can not probe function entry/exit effectively on such architecture. But this problem can be solved if the dynamic events supports fprobe events because fprobe events doesn't use kprobe but ftrace via fprobe. Fprobe events allows user to add new events on the entry and exit of kernel functions (which can be ftraced). Unlike kprobe events, the fprobe events can only probe the function entry and exit, the IP address will have some offsets from the symbol address. And it can only trace the function args, return value, and stacks. (no registers) For probing function body, users can continue to use the kprobe events. The tracepoint probe events (tprobe events) also allows user to add new events dynamically on the tracepoint. Most of the tracepoint already has trace-events, so this feature is useful if you only want to know a specific parameter, or trace the tracepoints which has no trace-events (e.g. sched_*_tp tracepoints only exposes the tracepoints.) The fprobe events syntax is; f[:[GRP/][EVENT]] FUNCTION [FETCHARGS] f[MAXACTIVE][:[GRP/][EVENT]] FUNCTION%return [FETCHARGS] And tracepoint probe events syntax is; t[:[GRP/][EVENT]] TRACEPOINT [FETCHARGS] This series also includes BTF argument support for fprobe/tracepoint events, and kprobe events. This allows us to fetch a specific function parameter by name, and all parameters by '$$args'. Note that enabling this feature, you need to enable CONFIG_BPF_SYSCALL and confirm that your arch supports CONFIG_HAVE_FUNCTION_ARG_ACCESS_API. E.g. # echo 't kfree ptr' >> dynamic_events # echo 'f kfree object' >> dynamic_events # cat dynamic_events t:tracepoints/kfree kfree ptr=ptr f:fprobes/kfree__entry kfree object=object # echo 1 > events/fprobes/enable # echo 1 > events/tracepoints/enable # echo > trace # head -n 20 trace | tail # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | tail-84 [000] ..... 1324.561958: kfree__entry: (kfree+0x4/0x140) object=0xffff888006383c00 tail-84 [000] ...1. 1324.561961: kfree: (__probestub_kfree+0x4/0x10) ptr=0xffff888006383c00 tail-84 [000] ..... 1324.561988: kfree__entry: (kfree+0x4/0x140) object=0x0 tail-84 [000] ...1. 1324.561988: kfree: (__probestub_kfree+0x4/0x10) ptr=0x0 tail-84 [000] ..... 1324.561989: kfree__entry: (kfree+0x4/0x140) object=0xffff88800671e600 tail-84 [000] ...1. 1324.561989: kfree: (__probestub_kfree+0x4/0x10) ptr=0xffff88800671e600 tail-84 [000] ..... 1324.562368: kfree__entry: (kfree+0x4/0x140) object=0xffff8880065e0580 tail-84 [000] ...1. 1324.562369: kfree: (__probestub_kfree+0x4/0x10) ptr=0xffff8880065e0580 Thank you, --- Masami Hiramatsu (Google) (9): fprobe: Pass return address to the handlers tracing/probes: Add fprobe events for tracing function entry and exit. selftests/ftrace: Add fprobe related testcases tracing/probes: Add tracepoint support on fprobe_event tracing/probes: Move event parameter fetching code to common parser tracing/probes: Support function parameters if BTF is available tracing/probes: Add $$args meta argument for all function args selftests/ftrace: Add tracepoint probe test case selftests/ftrace: Add BTF arguments test cases include/linux/fprobe.h | 11 include/linux/rethook.h | 2 include/linux/trace_events.h | 3 include/linux/tracepoint-defs.h | 1 include/linux/tracepoint.h | 5 kernel/kprobes.c | 1 kernel/trace/Kconfig | 25 kernel/trace/Makefile | 1 kernel/trace/bpf_trace.c | 6 kernel/trace/fprobe.c | 17 kernel/trace/rethook.c | 3 kernel/trace/trace.c | 13 kernel/trace/trace.h | 11 kernel/trace/trace_eprobe.c | 44 - kernel/trace/trace_fprobe.c | 1214 ++++++++++++++++++++ kernel/trace/trace_kprobe.c | 33 - kernel/trace/trace_probe.c | 448 ++++++- kernel/trace/trace_probe.h | 42 + kernel/trace/trace_uprobe.c | 8 lib/test_fprobe.c | 10 samples/fprobe/fprobe_example.c | 6 .../ftrace/test.d/dynevent/add_remove_btfarg.tc | 54 + .../ftrace/test.d/dynevent/add_remove_fprobe.tc | 26 .../ftrace/test.d/dynevent/add_remove_tprobe.tc | 27 .../ftrace/test.d/dynevent/fprobe_syntax_errors.tc | 98 ++ .../ftrace/test.d/dynevent/tprobe_syntax_errors.tc | 82 + .../ftrace/test.d/kprobe/kprobe_syntax_errors.tc | 12 27 files changed, 2064 insertions(+), 139 deletions(-) create mode 100644 kernel/trace/trace_fprobe.c create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_btfarg.tc create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_fprobe.tc create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_tprobe.tc create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/fprobe_syntax_errors.tc create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/tprobe_syntax_errors.tc -- Masami Hiramatsu (Google)