From patchwork Fri Oct 8 09:13:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544643 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F546C433F5 for ; Fri, 8 Oct 2021 09:13:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6360560FDC for ; Fri, 8 Oct 2021 09:13:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235706AbhJHJPl (ORCPT ); Fri, 8 Oct 2021 05:15:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60413 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235819AbhJHJPk (ORCPT ); Fri, 8 Oct 2021 05:15:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t8XYUgR9kHcOYFdSNrWNV6bYolumP9ymhRdSASBvoKM=; b=NvEV3wvzZGZ4sG2PhwJMZJozhTmoMRzYiJxYq7m78HbBGfeBIYfi3fBbq6IuaFujgKEjGe 3jdcv2iJSFs/wIH6+2Zuh2SIU1nmDYdG77ngb+ThxfhMwJQGRz+1Au2k5TxkeqPnLhs/gn +E+9Osxuznz9QM+IbGjtTD93CDb+KJI= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-459-GBpginFRMAWEJPG0WcOUIg-1; Fri, 08 Oct 2021 05:13:44 -0400 X-MC-Unique: GBpginFRMAWEJPG0WcOUIg-1 Received: by mail-wr1-f69.google.com with SMTP id d13-20020adf9b8d000000b00160a94c235aso6810529wrc.2 for ; Fri, 08 Oct 2021 02:13:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t8XYUgR9kHcOYFdSNrWNV6bYolumP9ymhRdSASBvoKM=; b=d1WC9KothNPwQFOBQ5Iw6cCf/LG/zOTz3Lp/ufE4dWGj4sBmfl1tbIw5505eOecAG8 ZyP09FQQHzW2XFl//BPAy7y8gHZ1v3703iiNcnVi4Ih7XnZsVGaJDFKPX5eTK/Alk1Wy trb8H9XerFpCnyZFWBShaxlm4WfU9LHu1h8gHyCf8k5nnFJzUL/sWupBPVHyfbwTjote BaK8rpZxHFeg1Nah4tGC196Enzk4dII7i+6XWO2m3veWfJLDmNJf7Zque6t9zO1Z7glL l56r4REsGigshV/ZqfO+i8mWRY6jLEleKYm5o7d20ZgI6isg8BRcYjga6GxpDnUafc+5 e82g== X-Gm-Message-State: AOAM532DhlCY3Qfbzfh0EGpgA5pQ3w7+i87tJaH05xw6KNB6i1d9TNhV qWsoFCdNH/3+mGSMcC+KfxjA40Xvnez5UCyQ1JQdqEJO2VuP2gqC6HmZZdCLQ4VU7oQHRKTzBQp 7cWufOay26im7 X-Received: by 2002:adf:8b1a:: with SMTP id n26mr2553757wra.182.1633684423778; Fri, 08 Oct 2021 02:13:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7pqH/9VGiNhSSQLuJGc71f9A4S88upuxhvoibmjXrXfdo84xNDH3ZpV69N8QYUBNTaQgOdQ== X-Received: by 2002:adf:8b1a:: with SMTP id n26mr2553740wra.182.1633684423641; Fri, 08 Oct 2021 02:13:43 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id g25sm1822738wrc.88.2021.10.08.02.13.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:13:43 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 1/8] x86/ftrace: Remove extra orig rax move Date: Fri, 8 Oct 2021 11:13:29 +0200 Message-Id: <20211008091336.33616-2-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org There's identical move 2 lines earlier. Signed-off-by: Jiri Olsa --- arch/x86/kernel/ftrace_64.S | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index 7c273846c687..a8eb084a7a9a 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -251,7 +251,6 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL) * If ORIG_RAX is anything but zero, make this a call to that. * See arch_ftrace_set_direct_caller(). */ - movq ORIG_RAX(%rsp), %rax testq %rax, %rax SYM_INNER_LABEL(ftrace_regs_caller_jmp, SYM_L_GLOBAL) jnz 1f From patchwork Fri Oct 8 09:13:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544645 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD05C433EF for ; Fri, 8 Oct 2021 09:13:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D2CB360F6C for ; Fri, 8 Oct 2021 09:13:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236070AbhJHJPr (ORCPT ); Fri, 8 Oct 2021 05:15:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:46857 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235847AbhJHJPr (ORCPT ); Fri, 8 Oct 2021 05:15:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684431; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eqAj4ZrmrtKDThZEc8WrAKFFOfUsJKx4HE/+LaO0ups=; b=AWPM/0dJV6o4t4FM7esqEkgvfUfAx3yN2UoiW1bGjGEAHTKUnaKcJ/xa/RFlUSyTwuXj5w xkzvOw8Vd78XvFAbI7wnvnwZN4ZAlGTl+ButRGqJPYhzMGttiqzFBSfrLGqDMxCcoIXSMm 3yZp6q1cL4sFHpziQlEcaKLBXA1V4/k= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-218-GIGKOTafOwS_8GgpMbbU9g-1; Fri, 08 Oct 2021 05:13:51 -0400 X-MC-Unique: GIGKOTafOwS_8GgpMbbU9g-1 Received: by mail-wr1-f70.google.com with SMTP id p12-20020adfc38c000000b00160d6a7e293so4314875wrf.18 for ; Fri, 08 Oct 2021 02:13:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eqAj4ZrmrtKDThZEc8WrAKFFOfUsJKx4HE/+LaO0ups=; b=zO9CU6R/PvZ/nDWgoc65IxH+842vDUY3ekMjAvTQst/yLNF2stjHqAI65AhKQ5iSz4 uTXTyXxN0PXOLiitjewEyXMVVywsVu6L6vsnGDHJeJA8mvX65rq7oRsp9sQwgmotNIQW t5ySB0/OVK7zjSRQPwXPx0F+DK6GF+oYdXpN1t4xPczI+UrSqov8WA6w74gT7KpQjEBs 5dwZn6CFnEur/iEVZ7PZm7AYzLktBxpcH7JKs+eDkAm1yB0wMU+L6NjAXjuvm7y9/gbX cuLwMgLGJgoN+/7cz8zwkAA8vmu//C1p5jIMlizpFlbtM39WfI8R+vuDlA1dcf42joGq eD8w== X-Gm-Message-State: AOAM530POei4trPC+auLu8c9myeVPV/A6bE8nUlm1ZEr+hUu0o/VQVAA YWsT63o70T+5OvjZ89eY42n58bx6ewcliBp/qHL0jNKQ20680XWE5ka3nb+RG3m255MinF4d98s s/7mPDA6Gan82 X-Received: by 2002:adf:bc48:: with SMTP id a8mr2596059wrh.397.1633684429747; Fri, 08 Oct 2021 02:13:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+TIty7irIaIflePLhh/gHbVH4LIWV87ZMOeBrKD6N733Xj1c4RMbUdKNhdpfkePK8v0WAOQ== X-Received: by 2002:adf:bc48:: with SMTP id a8mr2596045wrh.397.1633684429598; Fri, 08 Oct 2021 02:13:49 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id q7sm1884016wrs.73.2021.10.08.02.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:13:49 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 2/8] x86/ftrace: Remove fault protection code in prepare_ftrace_return Date: Fri, 8 Oct 2021 11:13:30 +0200 Message-Id: <20211008091336.33616-3-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: "Steven Rostedt (VMware)" Removing the fault protection code when writing return_hooker to stack. As Steven noted: > That protection was there from the beginning due to being "paranoid", > considering ftrace was bricking network cards. But that protection > would not have even protected against that. Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Jiri Olsa --- arch/x86/kernel/ftrace.c | 38 +++----------------------------------- 1 file changed, 3 insertions(+), 35 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 1b3ce3b4a2a2..c555624da989 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -625,12 +625,10 @@ int ftrace_disable_ftrace_graph_caller(void) * Hook the return address and push it in the stack of return addrs * in current thread info. */ -void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, +void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer) { unsigned long return_hooker = (unsigned long)&return_to_handler; - unsigned long old; - int faulted; /* * When resuming from suspend-to-ram, this function can be indirectly @@ -650,37 +648,7 @@ void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, if (unlikely(atomic_read(¤t->tracing_graph_pause))) return; - /* - * Protect against fault, even if it shouldn't - * happen. This tool is too much intrusive to - * ignore such a protection. - */ - asm volatile( - "1: " _ASM_MOV " (%[parent]), %[old]\n" - "2: " _ASM_MOV " %[return_hooker], (%[parent])\n" - " movl $0, %[faulted]\n" - "3:\n" - - ".section .fixup, \"ax\"\n" - "4: movl $1, %[faulted]\n" - " jmp 3b\n" - ".previous\n" - - _ASM_EXTABLE(1b, 4b) - _ASM_EXTABLE(2b, 4b) - - : [old] "=&r" (old), [faulted] "=r" (faulted) - : [parent] "r" (parent), [return_hooker] "r" (return_hooker) - : "memory" - ); - - if (unlikely(faulted)) { - ftrace_graph_stop(); - WARN_ON(1); - return; - } - - if (function_graph_enter(old, self_addr, frame_pointer, parent)) - *parent = old; + if (!function_graph_enter(*parent, ip, frame_pointer, parent)) + *parent = return_hooker; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ From patchwork Fri Oct 8 09:13:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544647 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 863CEC433F5 for ; Fri, 8 Oct 2021 09:14:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E7EC6101E for ; Fri, 8 Oct 2021 09:14:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235847AbhJHJPy (ORCPT ); Fri, 8 Oct 2021 05:15:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34276 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236409AbhJHJPx (ORCPT ); Fri, 8 Oct 2021 05:15:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nte6bf5BUe/Hi+6frLVZIwG1fCExMFjvtESCzT+gsiE=; b=Cbq7i80GMqfuglrxRwQydnHKPRk53PDPbezk3Dkpt4kjEHhW0kwfYsmAjbz9FHyMdTCbgu U54hyMFSm5A6vq0gEqRYj4tqF5lrdEr+GU8WZo+WpLuCKcez2jegK9MJ7AaQE346YqCwzw x6XcWyE8dpKNnWmBfiETAAwCQD3ObWQ= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-572-HKnVeYsrNr6REdCeWpYaRA-1; Fri, 08 Oct 2021 05:13:57 -0400 X-MC-Unique: HKnVeYsrNr6REdCeWpYaRA-1 Received: by mail-wr1-f72.google.com with SMTP id 75-20020adf82d1000000b00160cbb0f800so6040169wrc.22 for ; Fri, 08 Oct 2021 02:13:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nte6bf5BUe/Hi+6frLVZIwG1fCExMFjvtESCzT+gsiE=; b=kN/JxVo7UrToGA/EkSWVcMhQ2gARX1k3OjBhjjJzGjMvu4CfCXsMqKhCR5tCschMIz WtIDsRgeW3F2qlrMWaIOifdk1jQwMzLXg2HOuQ9UCVZ9B8SMnlz51EpwY+zrR3JJdBmS KYXfz+CetrHt9GY7xyoaZVtDn8QqLPdHt4xU7RWR+0kyZbGaXCS/Njf2XYBjJjJvToD/ cYdYmWUFM2ry7NE/SaaXMPMbQ+KDcvS441Lx84MOWLcjVlmW7Bf/mpY6PbyiDkwJnchB XLQ4Vvc2qk2ASQ/R6AN54UsLY/OGM8fbcEkDCTAo8wWz5YNIsHUfV/oiNJoLPEocUL3G PTKA== X-Gm-Message-State: AOAM532HKC+FIM9H2TY2ABHHV8KnAB37bDfttmZpGOJXbXyaNKuUtFNh d8q2t6gWjSC14IK1ilPexUjvYw/FzQIBQzmhV8iZURE3jz6Zeb/swyAn8zrqgrfCc4xYdhMJ/W3 uTrjMmWgNmmCR X-Received: by 2002:a7b:cf03:: with SMTP id l3mr2103415wmg.25.1633684435689; Fri, 08 Oct 2021 02:13:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZXmpaccR+EZkTBzTcC/asawRUjB1I7LYk15gS6XGiTzOUzFr8mgaQEfjnECpG/wTXS8BAWA== X-Received: by 2002:a7b:cf03:: with SMTP id l3mr2103397wmg.25.1633684435508; Fri, 08 Oct 2021 02:13:55 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id v3sm584758wrg.23.2021.10.08.02.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:13:55 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 3/8] x86/ftrace: Make function graph use ftrace directly Date: Fri, 8 Oct 2021 11:13:31 +0200 Message-Id: <20211008091336.33616-4-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: "Steven Rostedt (VMware)" We don't need special hook for graph tracer entry point, but instead we can use graph_ops::func function to install the return_hooker. This moves the graph tracing setup _before_ the direct trampoline prepares the stack, so the return_hooker will be called when the direct trampoline is finished. This simplifies the code, because we don't need to take into account the direct trampoline setup when preparing the graph tracer hooker and we can allow function graph tracer on entries registered with direct trampoline. [fixed compile error reported by kernel test robot ] Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Jiri Olsa --- arch/x86/include/asm/ftrace.h | 9 +++++++-- arch/x86/kernel/ftrace.c | 37 ++++++++++++++++++++++++++++++++--- arch/x86/kernel/ftrace_64.S | 29 +-------------------------- include/linux/ftrace.h | 9 +++++++++ kernel/trace/fgraph.c | 6 ++++-- 5 files changed, 55 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h index 9f3130f40807..024d9797646e 100644 --- a/arch/x86/include/asm/ftrace.h +++ b/arch/x86/include/asm/ftrace.h @@ -57,6 +57,13 @@ arch_ftrace_get_regs(struct ftrace_regs *fregs) #define ftrace_instruction_pointer_set(fregs, _ip) \ do { (fregs)->regs.ip = (_ip); } while (0) + +struct ftrace_ops; +#define ftrace_graph_func ftrace_graph_func +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs); +#else +#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR #endif #ifdef CONFIG_DYNAMIC_FTRACE @@ -65,8 +72,6 @@ struct dyn_arch_ftrace { /* No extra data needed for x86 */ }; -#define FTRACE_GRAPH_TRAMP_ADDR FTRACE_GRAPH_ADDR - #endif /* CONFIG_DYNAMIC_FTRACE */ #endif /* __ASSEMBLY__ */ #endif /* CONFIG_FUNCTION_TRACER */ diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index c555624da989..804fcc6ef2c7 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -527,7 +527,7 @@ static void *addr_from_call(void *ptr) return ptr + CALL_INSN_SIZE + call.disp; } -void prepare_ftrace_return(unsigned long self_addr, unsigned long *parent, +void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer); /* @@ -541,7 +541,8 @@ static void *static_tramp_func(struct ftrace_ops *ops, struct dyn_ftrace *rec) void *ptr; if (ops && ops->trampoline) { -#ifdef CONFIG_FUNCTION_GRAPH_TRACER +#if !defined(CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS) && \ + defined(CONFIG_FUNCTION_GRAPH_TRACER) /* * We only know about function graph tracer setting as static * trampoline. @@ -589,8 +590,9 @@ void arch_ftrace_trampoline_free(struct ftrace_ops *ops) #ifdef CONFIG_FUNCTION_GRAPH_TRACER #ifdef CONFIG_DYNAMIC_FTRACE -extern void ftrace_graph_call(void); +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +extern void ftrace_graph_call(void); static const char *ftrace_jmp_replace(unsigned long ip, unsigned long addr) { return text_gen_insn(JMP32_INSN_OPCODE, (void *)ip, (void *)addr); @@ -618,7 +620,17 @@ int ftrace_disable_ftrace_graph_caller(void) return ftrace_mod_jmp(ip, &ftrace_stub); } +#else /* !CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ +int ftrace_enable_ftrace_graph_caller(void) +{ + return 0; +} +int ftrace_disable_ftrace_graph_caller(void) +{ + return 0; +} +#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* !CONFIG_DYNAMIC_FTRACE */ /* @@ -629,6 +641,7 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, unsigned long frame_pointer) { unsigned long return_hooker = (unsigned long)&return_to_handler; + int bit; /* * When resuming from suspend-to-ram, this function can be indirectly @@ -648,7 +661,25 @@ void prepare_ftrace_return(unsigned long ip, unsigned long *parent, if (unlikely(atomic_read(¤t->tracing_graph_pause))) return; + bit = ftrace_test_recursion_trylock(ip, *parent); + if (bit < 0) + return; + if (!function_graph_enter(*parent, ip, frame_pointer, parent)) *parent = return_hooker; + + ftrace_test_recursion_unlock(bit); +} + +#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct ftrace_regs *fregs) +{ + struct pt_regs *regs = &fregs->regs; + unsigned long *stack = (unsigned long *)kernel_stack_pointer(regs); + + prepare_ftrace_return(ip, (unsigned long *)stack, 0); } +#endif + #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index a8eb084a7a9a..7a879901f103 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -174,11 +174,6 @@ SYM_INNER_LABEL(ftrace_caller_end, SYM_L_GLOBAL) SYM_FUNC_END(ftrace_caller); SYM_FUNC_START(ftrace_epilogue) -#ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - jmp ftrace_stub -#endif - /* * This is weak to keep gas from relaxing the jumps. * It is also used to copy the retq for trampolines. @@ -288,15 +283,6 @@ SYM_FUNC_START(__fentry__) cmpq $ftrace_stub, ftrace_trace_function jnz trace -fgraph_trace: -#ifdef CONFIG_FUNCTION_GRAPH_TRACER - cmpq $ftrace_stub, ftrace_graph_return - jnz ftrace_graph_caller - - cmpq $ftrace_graph_entry_stub, ftrace_graph_entry - jnz ftrace_graph_caller -#endif - SYM_INNER_LABEL(ftrace_stub, SYM_L_GLOBAL) retq @@ -314,25 +300,12 @@ trace: CALL_NOSPEC r8 restore_mcount_regs - jmp fgraph_trace + jmp ftrace_stub SYM_FUNC_END(__fentry__) EXPORT_SYMBOL(__fentry__) #endif /* CONFIG_DYNAMIC_FTRACE */ #ifdef CONFIG_FUNCTION_GRAPH_TRACER -SYM_FUNC_START(ftrace_graph_caller) - /* Saves rbp into %rdx and fills first parameter */ - save_mcount_regs - - leaq MCOUNT_REG_SIZE+8(%rsp), %rsi - movq $0, %rdx /* No framepointers needed */ - call prepare_ftrace_return - - restore_mcount_regs - - retq -SYM_FUNC_END(ftrace_graph_caller) - SYM_FUNC_START(return_to_handler) subq $24, %rsp diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 832e65f06754..1ca60ee70fb0 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -795,6 +795,15 @@ static inline bool is_ftrace_trampoline(unsigned long addr) } #endif /* CONFIG_DYNAMIC_FTRACE */ +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +#ifndef ftrace_graph_func +#define ftrace_graph_func ftrace_stub +#define FTRACE_OPS_GRAPH_STUB FTRACE_OPS_FL_STUB +#else +#define FTRACE_OPS_GRAPH_STUB 0 +#endif +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + /* totally disable ftrace - can not re-enable after this */ void ftrace_kill(void); diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index b8a0d1d564fb..22061d38fc00 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -115,6 +115,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, { struct ftrace_graph_ent trace; +#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS /* * Skip graph tracing if the return location is served by direct trampoline, * since call sequence and return addresses are unpredictable anyway. @@ -124,6 +125,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, if (ftrace_direct_func_count && ftrace_find_rec_direct(ret - MCOUNT_INSN_SIZE)) return -EBUSY; +#endif trace.func = func; trace.depth = ++current->curr_ret_depth; @@ -333,10 +335,10 @@ unsigned long ftrace_graph_ret_addr(struct task_struct *task, int *idx, #endif /* HAVE_FUNCTION_GRAPH_RET_ADDR_PTR */ static struct ftrace_ops graph_ops = { - .func = ftrace_stub, + .func = ftrace_graph_func, .flags = FTRACE_OPS_FL_INITIALIZED | FTRACE_OPS_FL_PID | - FTRACE_OPS_FL_STUB, + FTRACE_OPS_GRAPH_STUB, #ifdef FTRACE_GRAPH_TRAMP_ADDR .trampoline = FTRACE_GRAPH_TRAMP_ADDR, /* trampoline_size is only needed for dynamically allocated tramps */ From patchwork Fri Oct 8 09:13:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544649 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BADA5C433EF for ; Fri, 8 Oct 2021 09:14:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A429561108 for ; Fri, 8 Oct 2021 09:14:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236585AbhJHJQG (ORCPT ); Fri, 8 Oct 2021 05:16:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60051 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236555AbhJHJP6 (ORCPT ); Fri, 8 Oct 2021 05:15:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UUb/i9RBmi1Fh2lX7yds41KHjfH6nx836x27H5fnG4k=; b=SM6IO0K7+FnWRglD3StwEfyVuKraT9nXkD321ZV3nmeoOC+CGKo6AWxYRs6fzdE0VNF8es Ofb0wHDQ3j5ekpiGgtkCwzwe+ZXnSLjLEorbhRTIQAEmm+NRn/mOvs6pJ0CnD3CiRuxszT eNJXPcDlBQHd8s1SIYpZBLq7yJmYQ7Q= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-256-PPPEvYZlPCCd2NtMHinp2Q-1; Fri, 08 Oct 2021 05:14:02 -0400 X-MC-Unique: PPPEvYZlPCCd2NtMHinp2Q-1 Received: by mail-wr1-f69.google.com with SMTP id y12-20020a056000168c00b00160da4de2c7so3708879wrd.5 for ; Fri, 08 Oct 2021 02:14:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UUb/i9RBmi1Fh2lX7yds41KHjfH6nx836x27H5fnG4k=; b=XndNq/JUWfFoj4tt5/P8pTw6KAPI7Xi7JHw+l6JmUZkokgWtNV/OXx4bSW4YfTjZ/1 LpXDbQA6KwEsuZoh5XerMpq3zMByY49VNxgIKkpAOghU/3746PECxFzohZQYkSC/ig+i jNxHYdI9HIHbhsJXZ3qQLp5wIJWRWY97vks28tfywN+79a7EEURQAyQpT8x6Wfb6+rnl KBe7XUAHAMgqPzmU+wW4KxYkIFgaJcJWAg+xO2GQAzhcfQ2tygXv7gI53ywrK9LAc/QL E0AQOOCIA/YTs/B8hw789kg5BKFh8TpzbLDHp3m39L4lJ/Dw2KWmSxE5rHDIvwoFN0gF Rh0Q== X-Gm-Message-State: AOAM531OBfzxN3gHLg7jIOAbj8kfD5q5XhujNjwiuS+tDxV6d3vvdIiT Am6/leB+AI6+LFncTWDUv4KsZfLYxt6iUabWyufgaSJW17V5zZL4Wwuz00lp9p9XL3jOuJFwweH 01lgHl0IgGy+y X-Received: by 2002:adf:c986:: with SMTP id f6mr2691107wrh.216.1633684441557; Fri, 08 Oct 2021 02:14:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz74rdA9MIuzk3OP1VjqcSnfHAvyHf5rVZuc+qGDMTLv6UmeZvBAgk9qtrFDqvInkOO8GT8WA== X-Received: by 2002:adf:c986:: with SMTP id f6mr2691081wrh.216.1633684441342; Fri, 08 Oct 2021 02:14:01 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id k9sm1814581wrz.22.2021.10.08.02.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:14:01 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 4/8] tracing: Add trampoline/graph selftest Date: Fri, 8 Oct 2021 11:13:32 +0200 Message-Id: <20211008091336.33616-5-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Adding selftest for checking that direct trampoline can co-exist together with graph tracer on same function. This is supported for CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS config option, which is defined only for x86_64 for now. Signed-off-by: Jiri Olsa --- kernel/trace/trace_selftest.c | 54 ++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index adf7ef194005..917b7e3bf1ec 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -750,6 +750,8 @@ static struct fgraph_ops fgraph_ops __initdata = { .retfunc = &trace_graph_return, }; +noinline __noclone static void trace_direct_tramp(void) { } + /* * Pretty much the same than for the function tracer from which the selftest * has been borrowed. @@ -760,6 +762,7 @@ trace_selftest_startup_function_graph(struct tracer *trace, { int ret; unsigned long count; + char *func_name __maybe_unused; #ifdef CONFIG_DYNAMIC_FTRACE if (ftrace_filter_param) { @@ -808,8 +811,57 @@ trace_selftest_startup_function_graph(struct tracer *trace, goto out; } - /* Don't test dynamic tracing, the function tracer already did */ +#ifdef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS + tracing_reset_online_cpus(&tr->array_buffer); + set_graph_array(tr); + + /* + * Some archs *cough*PowerPC*cough* add characters to the + * start of the function names. We simply put a '*' to + * accommodate them. + */ + func_name = "*" __stringify(DYN_FTRACE_TEST_NAME); + ftrace_set_global_filter(func_name, strlen(func_name), 1); + + /* + * Register direct function together with graph tracer + * and make sure we get graph trace. + */ + ret = register_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, + (unsigned long) trace_direct_tramp); + if (ret) + goto out; + + ret = register_ftrace_graph(&fgraph_ops); + if (ret) { + warn_failed_init_tracer(trace, ret); + goto out; + } + + DYN_FTRACE_TEST_NAME(); + + count = 0; + + tracing_stop(); + /* check the trace buffer */ + ret = trace_test_buffer(&tr->array_buffer, &count); + + unregister_ftrace_graph(&fgraph_ops); + + ret = unregister_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, + (unsigned long) trace_direct_tramp); + if (ret) + goto out; + + tracing_start(); + if (!ret && !count) { + ret = -1; + goto out; + } +#endif + + /* Don't test dynamic tracing, the function tracer already did */ out: /* Stop it if we failed */ if (ret) From patchwork Fri Oct 8 09:13:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544651 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB2B3C433EF for ; Fri, 8 Oct 2021 09:14:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0DB161029 for ; Fri, 8 Oct 2021 09:14:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236770AbhJHJQH (ORCPT ); Fri, 8 Oct 2021 05:16:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59833 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237054AbhJHJQF (ORCPT ); Fri, 8 Oct 2021 05:16:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684449; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yPkTcBiMO+JyYGVU92LrpZqbnFZrp8nJBwqKNECsCoA=; b=JK/o7DHRARxi4NMUhlpDa2+ILvMVTyJkvSJsPtxSGYnNVVqWoR4sHYUtV/gLDAhlWVgdM0 wxG9bfKZk/ty6cXNvo015EcDQQqtitTcmO/TzH5p6x3fbHxrC5b/3h+rK7oBOSAL9TGZzS Bj2ptE8DNPb5qrpTDnHEszFmQEwSmmA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-422-cNI_turoNmCGe7OKu0_0CA-1; Fri, 08 Oct 2021 05:14:08 -0400 X-MC-Unique: cNI_turoNmCGe7OKu0_0CA-1 Received: by mail-wr1-f72.google.com with SMTP id v15-20020adfa1cf000000b00160940b17a2so6795525wrv.19 for ; Fri, 08 Oct 2021 02:14:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yPkTcBiMO+JyYGVU92LrpZqbnFZrp8nJBwqKNECsCoA=; b=AlpFHvujxB+GylIfTY0A04/K1X6Xe+5OI29FD9tWf8MX7tslFowRTtATTDyEp21JhY xnJYECDOU1KWTNh+vVpt0UEzowrRCrBdZXn6Qv4Zt4R1HZX8JVQDN6h4CdmlW/F98w8w 5I19HSL0T0KXw5iPCYPfJmJhF1ui5hvvWzcQ97gZIBdY2rHZgvR1COZdzCbwjbZCxbzA TMKdq26hFzPKy+0iWgpnr2mNKbbVZu6dwoSJpLs1iT4dhLNrren7wyZJuBd67Z+nRQTN h1XeEwol9Wxy1HFpQc2aIJ2Xv+EuQE4yNKjtDR7KZvlTguj/2uHegS46KazNuzUYJH+t 4Qzw== X-Gm-Message-State: AOAM531zbcfo9YKsKCSkjywV8Gy7rLLF4jTCFJQNGcjLVAo0/I9Q2/7m 6F0XlPt/UQXCHz5zg+a803SWk1hjPwA0nK0Xbqx3/wK57KOd60kdw8A4c7NPle7fLyCbxcpo2SP uDZtFzy0s3qMm X-Received: by 2002:a05:600c:3644:: with SMTP id y4mr2237324wmq.125.1633684447456; Fri, 08 Oct 2021 02:14:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzdcvuUF+VLaW1cxuahKdcY46l87LC9jTZJGT4qjR69eSkhdKCU3uwWXVMNvZd4KGqGqF1o1A== X-Received: by 2002:a05:600c:3644:: with SMTP id y4mr2237313wmq.125.1633684447312; Fri, 08 Oct 2021 02:14:07 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id u3sm1800794wmc.16.2021.10.08.02.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:14:07 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 5/8] ftrace: Add ftrace_add_rec_direct function Date: Fri, 8 Oct 2021 11:13:33 +0200 Message-Id: <20211008091336.33616-6-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Factor out the code that adds (ip, addr) tuple to direct_functions hash in new ftrace_add_rec_direct function. It will be used in following patches. Signed-off-by: Jiri Olsa --- kernel/trace/ftrace.c | 64 +++++++++++++++++++++++++------------------ 1 file changed, 37 insertions(+), 27 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 3eec6792f115..6032a5441770 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2394,6 +2394,39 @@ unsigned long ftrace_find_rec_direct(unsigned long ip) return entry->direct; } +static struct ftrace_func_entry* +ftrace_add_rec_direct(unsigned long ip, unsigned long addr, + struct ftrace_hash **free_hash) +{ + struct ftrace_func_entry *entry; + + if (ftrace_hash_empty(direct_functions) || + direct_functions->count > 2 * (1 << direct_functions->size_bits)) { + struct ftrace_hash *new_hash; + int size = ftrace_hash_empty(direct_functions) ? 0 : + direct_functions->count + 1; + + if (size < 32) + size = 32; + + new_hash = dup_hash(direct_functions, size); + if (!new_hash) + return NULL; + + *free_hash = direct_functions; + direct_functions = new_hash; + } + + entry = kmalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) + return NULL; + + entry->ip = ip; + entry->direct = addr; + __add_hash_entry(direct_functions, entry); + return entry; +} + static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { @@ -5110,39 +5143,16 @@ int register_ftrace_direct(unsigned long ip, unsigned long addr) } ret = -ENOMEM; - if (ftrace_hash_empty(direct_functions) || - direct_functions->count > 2 * (1 << direct_functions->size_bits)) { - struct ftrace_hash *new_hash; - int size = ftrace_hash_empty(direct_functions) ? 0 : - direct_functions->count + 1; - - if (size < 32) - size = 32; - - new_hash = dup_hash(direct_functions, size); - if (!new_hash) - goto out_unlock; - - free_hash = direct_functions; - direct_functions = new_hash; - } - - entry = kmalloc(sizeof(*entry), GFP_KERNEL); - if (!entry) - goto out_unlock; - direct = ftrace_find_direct_func(addr); if (!direct) { direct = ftrace_alloc_direct_func(addr); - if (!direct) { - kfree(entry); + if (!direct) goto out_unlock; - } } - entry->ip = ip; - entry->direct = addr; - __add_hash_entry(direct_functions, entry); + entry = ftrace_add_rec_direct(ip, addr, &free_hash); + if (!entry) + goto out_unlock; ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); if (ret) From patchwork Fri Oct 8 09:13:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544653 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0990C433F5 for ; Fri, 8 Oct 2021 09:14:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6A646113E for ; Fri, 8 Oct 2021 09:14:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236555AbhJHJQW (ORCPT ); Fri, 8 Oct 2021 05:16:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36668 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237070AbhJHJQL (ORCPT ); Fri, 8 Oct 2021 05:16:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QINuD3vOMUoOmNOM6OKfs1CcbZJa/bSQ5hGKOOlIS9A=; b=JAaarcknOJcRV24Inw3z+TpDRUhqCrtEvWyoUTl4UkYXixsrKbA/8OhBlakFqju1X109P5 xx6hvfsEY4l94cgh1j7n1qr8YA5QwdTAAhCGABeMv1LiRmmnVzoCZvALdpCjM/nw2f9+zP 7SyzbjlA0jXDGpznOUfTsoCHaN26bIg= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-fx142TPtMqWhqR4-UBkA0g-1; Fri, 08 Oct 2021 05:14:14 -0400 X-MC-Unique: fx142TPtMqWhqR4-UBkA0g-1 Received: by mail-wr1-f71.google.com with SMTP id k16-20020a5d6290000000b00160753b430fso6834765wru.11 for ; Fri, 08 Oct 2021 02:14:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QINuD3vOMUoOmNOM6OKfs1CcbZJa/bSQ5hGKOOlIS9A=; b=1q2UpnZAYc1jeeNWdCUbC9AVJPTOWBszZwkz8/cIeYN3CKXXZxCIknsOAS8pjcce1Z K8rCQlwk4rNxtqo6Ef8B3fdpmrXhoB7DgLeV10G/NDnFx4wz3z/kzRil9acRejp9M7x/ e2wN9qYqDLSdScneElQyZXQYbQ6J49HqyoaJXByIf6C0SIwkfQM2fO0cRw8X0Tatg075 bKbBzd7fV7U59d/pm/JG5KgelGA5x8PRCxBch526WuAz5G2XnAWk78s1LQjT4H62QfM+ GyofgGxVOSsjP/DAOLSJmwwkpu6ax4JlNJjRTbK3DZusHUUv/CvAcNHwwWPbCUrMk2Ql qFjw== X-Gm-Message-State: AOAM531FUQVfK3MuaGhYEAtSONunt2af7XyKFWOCPZkSZjMf1o/hz5/5 j+doFSyen1jkBClG3sOKwc1kZ3Dh7I5Kaj1S67VlgXB5lOANgps9LznHwQXbxXKNWtosbkKrhuH lg2wRoB+30msA X-Received: by 2002:a05:600c:4e86:: with SMTP id f6mr2303231wmq.52.1633684453455; Fri, 08 Oct 2021 02:14:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyyh4MhcoIXFiPd0LzRAmW9q86m6uZ3JWzopP6VBqE7452ZqdtaQYVVlAhFAeatz4Cura/cmQ== X-Received: by 2002:a05:600c:4e86:: with SMTP id f6mr2303206wmq.52.1633684453263; Fri, 08 Oct 2021 02:14:13 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id k9sm1815090wrz.22.2021.10.08.02.14.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:14:13 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 6/8] ftrace: Add multi direct register/unregister interface Date: Fri, 8 Oct 2021 11:13:34 +0200 Message-Id: <20211008091336.33616-7-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Adding interface to register multiple direct functions within single call. Adding following functions: register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) The register_ftrace_direct_multi registers direct function (addr) with all functions in ops filter. The ops filter can be updated before with ftrace_set_filter_ip calls. All requested functions must not have direct function currently registered, otherwise register_ftrace_direct_multi will fail. The unregister_ftrace_direct_multi unregisters ops related direct functions. Signed-off-by: Jiri Olsa --- include/linux/ftrace.h | 11 ++++ kernel/trace/ftrace.c | 142 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 153 insertions(+) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 1ca60ee70fb0..ba5d02ba8166 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -316,7 +316,10 @@ int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, unsigned long old_addr, unsigned long new_addr); unsigned long ftrace_find_rec_direct(unsigned long ip); +int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); +int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); #else +struct ftrace_ops; # define ftrace_direct_func_count 0 static inline int register_ftrace_direct(unsigned long ip, unsigned long addr) { @@ -346,6 +349,14 @@ static inline unsigned long ftrace_find_rec_direct(unsigned long ip) { return 0; } +static inline int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + return -ENODEV; +} +static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + return -ENODEV; +} #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ #ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 6032a5441770..f9df7bffb770 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5405,6 +5405,148 @@ int modify_ftrace_direct(unsigned long ip, return ret; } EXPORT_SYMBOL_GPL(modify_ftrace_direct); + +#define MULTI_FLAGS (FTRACE_OPS_FL_IPMODIFY | FTRACE_OPS_FL_DIRECT | \ + FTRACE_OPS_FL_SAVE_REGS) + +static int check_direct_multi(struct ftrace_ops *ops) +{ + if (!(ops->flags & FTRACE_OPS_FL_INITIALIZED)) + return -EINVAL; + if ((ops->flags & MULTI_FLAGS) != MULTI_FLAGS) + return -EINVAL; + return 0; +} + +static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long addr) +{ + struct ftrace_func_entry *entry, *del; + int size, i; + + size = 1 << hash->size_bits; + for (i = 0; i < size; i++) { + hlist_for_each_entry(entry, &hash->buckets[i], hlist) { + del = __ftrace_lookup_ip(direct_functions, entry->ip); + if (del && del->direct == addr) { + remove_hash_entry(direct_functions, del); + kfree(del); + } + } + } +} + +/** + * register_ftrace_direct_multi - Call a custom trampoline directly + * for multiple functions registered in @ops + * @ops: The address of the struct ftrace_ops object + * @addr: The address of the trampoline to call at @ops functions + * + * This is used to connect a direct calls to @addr from the nop locations + * of the functions registered in @ops (with by ftrace_set_filter_ip + * function). + * + * The location that it calls (@addr) must be able to handle a direct call, + * and save the parameters of the function being traced, and restore them + * (or inject new ones if needed), before returning. + * + * Returns: + * 0 on success + * -EINVAL - The @ops object was already registered with this call or + * when there are no functions in @ops object. + * -EBUSY - Another direct function is already attached (there can be only one) + * -ENODEV - @ip does not point to a ftrace nop location (or not supported) + * -ENOMEM - There was an allocation failure. + */ +int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + struct ftrace_hash *hash, *free_hash = NULL; + struct ftrace_func_entry *entry, *new; + int err = -EBUSY, size, i; + + if (ops->func || ops->trampoline) + return -EINVAL; + if (!(ops->flags & FTRACE_OPS_FL_INITIALIZED)) + return -EINVAL; + if (ops->flags & FTRACE_OPS_FL_ENABLED) + return -EINVAL; + + hash = ops->func_hash->filter_hash; + if (ftrace_hash_empty(hash)) + return -EINVAL; + + mutex_lock(&direct_mutex); + + /* Make sure requested entries are not already registered.. */ + size = 1 << hash->size_bits; + for (i = 0; i < size; i++) { + hlist_for_each_entry(entry, &hash->buckets[i], hlist) { + if (ftrace_find_rec_direct(entry->ip)) + goto out_unlock; + } + } + + /* ... and insert them to direct_functions hash. */ + err = -ENOMEM; + for (i = 0; i < size; i++) { + hlist_for_each_entry(entry, &hash->buckets[i], hlist) { + new = ftrace_add_rec_direct(entry->ip, addr, &free_hash); + if (!new) + goto out_remove; + entry->direct = addr; + } + } + + ops->func = call_direct_funcs; + ops->flags = MULTI_FLAGS; + ops->trampoline = FTRACE_REGS_ADDR; + + err = register_ftrace_function(ops); + + out_remove: + if (err) + remove_direct_functions_hash(hash, addr); + + out_unlock: + mutex_unlock(&direct_mutex); + + if (free_hash) { + synchronize_rcu_tasks(); + free_ftrace_hash(free_hash); + } + return err; +} +EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); + +/** + * unregister_ftrace_direct_multi - Remove calls to custom trampoline + * previously registered by register_ftrace_direct_multi for @ops object. + * @ops: The address of the struct ftrace_ops object + * + * This is used to remove a direct calls to @addr from the nop locations + * of the functions registered in @ops (with by ftrace_set_filter_ip + * function). + * + * Returns: + * 0 on success + * -EINVAL - The @ops object was not properly registered. + */ +int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + struct ftrace_hash *hash = ops->func_hash->filter_hash; + int err; + + if (check_direct_multi(ops)) + return -EINVAL; + if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) + return -EINVAL; + + mutex_lock(&direct_mutex); + err = unregister_ftrace_function(ops); + remove_direct_functions_hash(hash, addr); + mutex_unlock(&direct_mutex); + return err; +} +EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** From patchwork Fri Oct 8 09:13:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544655 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62DEEC433F5 for ; Fri, 8 Oct 2021 09:14:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D61060F6C for ; Fri, 8 Oct 2021 09:14:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235869AbhJHJQZ (ORCPT ); Fri, 8 Oct 2021 05:16:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42847 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237139AbhJHJQQ (ORCPT ); Fri, 8 Oct 2021 05:16:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EXKKrinDM77WFUunxS5Afbd05XO3wolRRteT5kMHCY4=; b=Ajw8oZXpyPKUYjVGXzPPs7IXyVn8FUQA18STNKYb905SMgRPsCDmwBmhqPejk9Ih7vqMfJ MLilOw5TuEtoA+fGGIBVunONW4AICAnUfRqjaO1MCsU8LHg+HqNYcDKOvdsdw1Fa+/KZsq sdIZieTNdU2SLBJf4DiOuUyG1s1zRa8= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-558-u8DjXsseNxCHZRO1CvBJlw-1; Fri, 08 Oct 2021 05:14:20 -0400 X-MC-Unique: u8DjXsseNxCHZRO1CvBJlw-1 Received: by mail-wr1-f70.google.com with SMTP id l6-20020adfa386000000b00160c4c1866eso6804130wrb.4 for ; Fri, 08 Oct 2021 02:14:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EXKKrinDM77WFUunxS5Afbd05XO3wolRRteT5kMHCY4=; b=zLtLECBHeASVzPFO3H3+qjU4xmv3AsRQXhD0LEwkSTzt/R9SNh40a33+ezvj3EicFD 0x+JNDQwNZR/kjle50VmaARS6HvdAJaXJ7ujN001++kqw6hWK9Dn5bIxAgdRXMUwNsRi XHr6FloCXLr0AOQueNzSL1p5wkyhGzxPdGfObNr/Jw47lVUpoJ1yQO72NFscgwVhSdSn DFPfWpU1vx3jQq1wHPSPGTcWVIuv8Zee7uoHd3yiDJGLDM42LOEoaHSQFm1bkEOZCL/2 pbbv5zxAbCGZQIaGjhSH2djMe4bz8zPHlUNZGKXTveFWfj8Qgt66Fnm1PvT8Yzr5gFy6 z1AQ== X-Gm-Message-State: AOAM533tpR7LrPcno/f98aKKQa6Dg79tuugQ86kToxnG3T4MpLFabUez tGeOFYe1QuO17W3ywrd3YYXtk1PYohzv/z6wP4Ih5aSp2q8TgQnPB0k7YtRQGdlOC3AxBkSg/3S 9KBlI6coejaN+ X-Received: by 2002:a05:600c:1907:: with SMTP id j7mr2161820wmq.184.1633684459439; Fri, 08 Oct 2021 02:14:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz84j7K2uFms5WNUH+r7ll2dxXog50Ofy2QktxnUnkOsHyey2rYBor+XjZJ2p6C+bIHLWsaKg== X-Received: by 2002:a05:600c:1907:: with SMTP id j7mr2161801wmq.184.1633684459246; Fri, 08 Oct 2021 02:14:19 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id t3sm7861433wmj.33.2021.10.08.02.14.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:14:19 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 7/8] ftrace: Add multi direct modify interface Date: Fri, 8 Oct 2021 11:13:35 +0200 Message-Id: <20211008091336.33616-8-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Adding interface to modify registered direct function for ftrace_ops. Adding following function: modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) The function changes the currently registered direct function for all attached functions. Signed-off-by: Jiri Olsa Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Steven Rostedt (VMware) Acked-by: Jiri Olsa --- include/linux/ftrace.h | 6 ++++ kernel/trace/ftrace.c | 62 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index ba5d02ba8166..c15b767f39cf 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -318,6 +318,8 @@ int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, unsigned long ftrace_find_rec_direct(unsigned long ip); int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); +int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); + #else struct ftrace_ops; # define ftrace_direct_func_count 0 @@ -357,6 +359,10 @@ static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigne { return -ENODEV; } +static inline int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + return -ENODEV; +} #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ #ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index f9df7bffb770..d92f2591c3fc 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5547,6 +5547,68 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) return err; } EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); + +/** + * modify_ftrace_direct_multi - Modify an existing direct 'multi' call + * to call something else + * @ops: The address of the struct ftrace_ops object + * @addr: The address of the new trampoline to call at @ops functions + * + * This is used to unregister currently registered direct caller and + * register new one @addr on functions registered in @ops object. + * + * Note there's window between ftrace_shutdown and ftrace_startup calls + * where there will be no callbacks called. + * + * Returns: zero on success. Non zero on error, which includes: + * -EINVAL - The @ops object was not properly registered. + */ +int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +{ + struct ftrace_hash *hash = ops->func_hash->filter_hash; + struct ftrace_func_entry *entry, *iter; + int i, size; + int err; + + if (check_direct_multi(ops)) + return -EINVAL; + if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) + return -EINVAL; + + mutex_lock(&direct_mutex); + mutex_lock(&ftrace_lock); + + /* + * Shutdown the ops, change 'direct' pointer for each + * ops entry in direct_functions hash and startup the + * ops back again. + * + * Note there is no callback called for @ops object after + * this ftrace_shutdown call until ftrace_startup is called + * later on. + */ + err = ftrace_shutdown(ops, 0); + if (err) + goto out_unlock; + + size = 1 << hash->size_bits; + for (i = 0; i < size; i++) { + hlist_for_each_entry(iter, &hash->buckets[i], hlist) { + entry = __ftrace_lookup_ip(direct_functions, iter->ip); + if (!entry) + continue; + entry->direct = addr; + } + } + + err = ftrace_startup(ops, 0); + + out_unlock: + mutex_unlock(&ftrace_lock); + mutex_unlock(&direct_mutex); + return err; +} +EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi); #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** From patchwork Fri Oct 8 09:13:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12544657 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4626C433EF for ; Fri, 8 Oct 2021 09:14:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0D126101E for ; Fri, 8 Oct 2021 09:14:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237631AbhJHJQg (ORCPT ); Fri, 8 Oct 2021 05:16:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:27675 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236616AbhJHJQX (ORCPT ); Fri, 8 Oct 2021 05:16:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633684467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aMLlR2SzoNjOmv5PwyT5gjKRoe76WYTK4npUjuAglpY=; b=DruIj8GO1QHQgEY4+gJywUw62sMNEM051XZ/KNcus5kVv9qdvYJ8WTdLAwaj6OuTS/NdBP uvkxhFKSp3fSOF4QpSFzgP4LQ+BusZq+m4HxvQhd3zcMjFxEwREOZhbbwuQcf7KD4uN2x0 vSpnbuihWGPZQ91QetQEkLE66nEieqw= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-196-6cpmE5i3Mharmdxqw0aehg-1; Fri, 08 Oct 2021 05:14:26 -0400 X-MC-Unique: 6cpmE5i3Mharmdxqw0aehg-1 Received: by mail-wr1-f72.google.com with SMTP id r16-20020adfb1d0000000b00160bf8972ceso6790612wra.13 for ; Fri, 08 Oct 2021 02:14:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aMLlR2SzoNjOmv5PwyT5gjKRoe76WYTK4npUjuAglpY=; b=PAkg5rRtCWXbEKnE2h9+wICwYSN8mfkW2EBXTXRzrUHJh09GTxUvwjcpDMhpX0YuJn dE3QpQaeapMEubOG7NHhTh77LInKLnYpEM1QRdTvzRAyp1ixSHa/KJiOwooZ0Bq26OXs BZIetG0Nz/xGureDzERQhdO7y0yv3n4kHljeA5z+wXAe5sIt1Y1WP4fMvIUY31jGlCdY KTZdAuZaKSaERd7+Hcmn1xGhMJZkAMoXw+bUwnKp0VqjLx5NuiDAYKNQkFxHd/267qFA 5WIL3/MEDPVbbYsuhGU/4eJ4PI3vLY1eUOsx5IJsU3MjrWPK4Eb2+cfPVS3/S+CO5zJY hkrA== X-Gm-Message-State: AOAM532uagMAbEVSzSh53SK18ZgyOAh3ElqmJs7862rkZ/TsRyt5ULTK N76Z3WyG8Bo8Sc23/DEE2I47D0/+pqgJC2GBVPSZRymcHeJ4fffSQ761di6QzBNLgfDc8RX421R L95/VG4aU3eAT X-Received: by 2002:a1c:f705:: with SMTP id v5mr2170519wmh.18.1633684465596; Fri, 08 Oct 2021 02:14:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz6GIVVwRsyGduGMdmAo2Z9uL4etWtj7XIEnA3tDdBWwJS4LKVXzwWWpvrqQgMbPnFVV8FHtA== X-Received: by 2002:a1c:f705:: with SMTP id v5mr2170500wmh.18.1633684465393; Fri, 08 Oct 2021 02:14:25 -0700 (PDT) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id z8sm608721wrq.16.2021.10.08.02.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Oct 2021 02:14:25 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: "Steven Rostedt (VMware)" Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Subject: [PATCH 8/8] ftrace/samples: Add multi direct interface test module Date: Fri, 8 Oct 2021 11:13:36 +0200 Message-Id: <20211008091336.33616-9-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211008091336.33616-1-jolsa@kernel.org> References: <20211008091336.33616-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Adding simple module that uses multi direct interface: register_ftrace_direct_multi unregister_ftrace_direct_multi The init function registers trampoline for 2 functions, and exit function unregisters them. Signed-off-by: Jiri Olsa --- samples/ftrace/Makefile | 1 + samples/ftrace/ftrace-direct-multi.c | 52 ++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 samples/ftrace/ftrace-direct-multi.c diff --git a/samples/ftrace/Makefile b/samples/ftrace/Makefile index 4ce896e10b2e..ab1d1c05c288 100644 --- a/samples/ftrace/Makefile +++ b/samples/ftrace/Makefile @@ -3,6 +3,7 @@ obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-too.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-modify.o +obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-multi.o CFLAGS_sample-trace-array.o := -I$(src) obj-$(CONFIG_SAMPLE_TRACE_ARRAY) += sample-trace-array.o diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c new file mode 100644 index 000000000000..2a5b1fb7ac14 --- /dev/null +++ b/samples/ftrace/ftrace-direct-multi.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include + +#include /* for handle_mm_fault() */ +#include +#include + +void my_direct_func(unsigned long ip) +{ + trace_printk("ip %lx\n", ip); +} + +extern void my_tramp(void *); + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:" +" pushq %rbp\n" +" movq %rsp, %rbp\n" +" pushq %rdi\n" +" movq 8(%rbp), %rdi\n" +" call my_direct_func\n" +" popq %rdi\n" +" leave\n" +" ret\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +static struct ftrace_ops direct; + +static int __init ftrace_direct_multi_init(void) +{ + ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); + ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); +} + +static void __exit ftrace_direct_multi_exit(void) +{ + unregister_ftrace_direct_multi(&direct, (unsigned long) my_tramp); +} + +module_init(ftrace_direct_multi_init); +module_exit(ftrace_direct_multi_exit); + +MODULE_AUTHOR("Jiri Olsa"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); +MODULE_LICENSE("GPL");