From patchwork Wed Jul 7 21:47:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364067 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60BF0C07E95 for ; Wed, 7 Jul 2021 21:48:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 46F5561C3C for ; Wed, 7 Jul 2021 21:48:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232124AbhGGVux (ORCPT ); Wed, 7 Jul 2021 17:50:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35270 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231458AbhGGVuw (ORCPT ); Wed, 7 Jul 2021 17:50:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QgBr4LV9SR6/Qelufj3zindVEajUhYS2+l3zjx7rHSU=; b=DO+ix10DTo0paD0JC3Z6gRQwV1hNlxs+Tw7p7SR+gW1udXDdL978RBzg1YefvZrWw7UwC0 Tm3WrdV120A6GWE3W1O5/eBYU+ojRRXnneekzZT+Yzmp3i6dzIVKIOMj0lLpZABGQxA6SO VwOs7aEQoUBDTLYweM2NkzN5uOtLyc8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-vkHe6EWrMRSDOmmSOmHnwA-1; Wed, 07 Jul 2021 17:48:10 -0400 X-MC-Unique: vkHe6EWrMRSDOmmSOmHnwA-1 Received: by mail-wm1-f70.google.com with SMTP id m5-20020a05600c3b05b029020c34fd31cfso693114wms.4 for ; Wed, 07 Jul 2021 14:48:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QgBr4LV9SR6/Qelufj3zindVEajUhYS2+l3zjx7rHSU=; b=LA/pbRbYXcogS9F4oaBXfbK5Lp18jTKBUXfg5YtK4N5ZbMGa/2RhCYUJ6WWW4ISHR5 nZPZGxJS/0cnDYPFj0/LfSygPbVTMdbbHBoHHluxRdBa2CkAh0S1xGtdjjMSj9I97bJN RIh3k+py3A4CQlt1XN0FrBLU35kKWVm1WDMEffsqobnzNBoAjRBIBnfB4ANpjJPskKgH dcamHP9PfCNuTIFpznPS+UKByP0ls5dZs0r4/9xmNYaTLNWviHMVbLpTGIMDJRJGW3JT WL+6jQBd16kYSs8xNK+Tba841M9eCDuZLS4nEokM1/kkcUxgHS8/DtgB5Iromznlfr3K 2XMQ== X-Gm-Message-State: AOAM532Xy5kszK0yGmQrMgD9oIfzLU+Tew2TmDgrTHkINIhHuFTuUaIR oUhez83bWcCtGG7Bosuzw33J3DyCw+6Hjd1ZVbfhLat/BRvCf+V4VX9PPYm+AEl6lOG++Fcmj94 PCGnPP8YYWlNQ X-Received: by 2002:a05:600c:3504:: with SMTP id h4mr1321810wmq.118.1625694489445; Wed, 07 Jul 2021 14:48:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxxvoBCI3Hwfj+8IIJ0979hrRBL+Mvswms4GyKib5rSXrEUo/bGjX5pem2iEJH+QhrrbUJegw== X-Received: by 2002:a05:600c:3504:: with SMTP id h4mr1321803wmq.118.1625694489310; Wed, 07 Jul 2021 14:48:09 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id v2sm177032wru.16.2021.07.07.14.48.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:48:09 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 1/7] bpf, x86: Store caller's ip in trampoline stack Date: Wed, 7 Jul 2021 23:47:45 +0200 Message-Id: <20210707214751.159713-2-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Storing caller's ip in trampoline's stack. Trampoline programs can reach the IP in (ctx - 8) address, so there's no change in program's arguments interface. The IP address is takes from [fp + 8], which is return address from the initial 'call fentry' call to trampoline. This IP address will be returned via bpf_get_func_ip helper helper, which is added in following patches. Signed-off-by: Jiri Olsa --- arch/x86/net/bpf_jit_comp.c | 19 +++++++++++++++++++ include/linux/bpf.h | 5 +++++ 2 files changed, 24 insertions(+) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index e835164189f1..c320b3ce7b58 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -1951,6 +1951,9 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (flags & BPF_TRAMP_F_CALL_ORIG) stack_size += 8; /* room for return value of orig_call */ + if (flags & BPF_TRAMP_F_IP_ARG) + stack_size += 8; /* room for IP address argument */ + if (flags & BPF_TRAMP_F_SKIP_FRAME) /* skip patched call instruction and point orig_call to actual * body of the kernel function. @@ -1964,6 +1967,22 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i EMIT4(0x48, 0x83, 0xEC, stack_size); /* sub rsp, stack_size */ EMIT1(0x53); /* push rbx */ + if (flags & BPF_TRAMP_F_IP_ARG) { + /* Store IP address of the traced function: + * mov rax, QWORD PTR [rbp + 8] + * sub rax, X86_PATCH_SIZE + * mov QWORD PTR [rbp - stack_size], rax + */ + emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); + EMIT4(0x48, 0x83, 0xe8, X86_PATCH_SIZE); + emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -stack_size); + + /* Continue with stack_size for regs storage, stack will + * be correctly restored with 'leave' instruction. + */ + stack_size -= 8; + } + save_regs(m, &prog, nr_args, stack_size); if (flags & BPF_TRAMP_F_CALL_ORIG) { diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f309fc1509f2..6b3da9bc3d16 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -554,6 +554,11 @@ struct btf_func_model { */ #define BPF_TRAMP_F_SKIP_FRAME BIT(2) +/* Store IP address of the caller on the trampoline stack, + * so it's available for trampoline's programs. + */ +#define BPF_TRAMP_F_IP_ARG BIT(3) + /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. Pick a number to fit into BPF_IMAGE_SIZE / 2 */ From patchwork Wed Jul 7 21:47:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364069 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45CE7C07E95 for ; Wed, 7 Jul 2021 21:48:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27E1561C3C for ; Wed, 7 Jul 2021 21:48:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232859AbhGGVvA (ORCPT ); Wed, 7 Jul 2021 17:51:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27879 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232774AbhGGVvA (ORCPT ); Wed, 7 Jul 2021 17:51:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694499; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jm5cd0zOZ7Itm+FjlMPQhsBe4LoJlMQjbASHrTs/ggA=; b=JfPvXLah9ma06orCRN/m6//y9eFiarHSrL/UZbWevl3/+fBORGTRW50wR5rT5ZrZtYzqfk bFWXWAi0dgi+NgBzMbpexx8Tx4XWW61IqBxPhuhsSWFO75TQDgz/6kNSe8KlrBzVTa1j82 0fuCuH1uM32sZwSIK/x5TEemamYeOd0= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-225-q-2uwREzMLqgui41YUFBwg-1; Wed, 07 Jul 2021 17:48:18 -0400 X-MC-Unique: q-2uwREzMLqgui41YUFBwg-1 Received: by mail-wm1-f72.google.com with SMTP id a129-20020a1ce3870000b02901f050bc61d2so1533532wmh.8 for ; Wed, 07 Jul 2021 14:48:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jm5cd0zOZ7Itm+FjlMPQhsBe4LoJlMQjbASHrTs/ggA=; b=JKx0zhrdX5blIT4frWEHDjPyRHxqIQcutFjEfkktKgwh8bTZLJJgWpPBHQJ0GVK2KW mNbqpGbFUyna5gaMM1gccEy0Xo7XwkZ7oWhZrW+Bil2IwkrmUGIAUV8KQEFsbn6Var0i UnmaHkEE7UHascZTcX/5PTfVI5S1TZUX9JKJVVVoetbPFvbJpBi5IVKs0CDZ8ZcTEr+U 0/hLPtmLZCpRzi2kBZq0hK2AydweaqMgV6vd8nMDqwE0lBXX/6Ma4VXPJ4MpQsri0FTa 34ItRejOdA6rxIyhB1yu0ycBEjcnU271nb9gbKSYdcgEi4C0KU7gbumwUK2wemq4rZ7P BSBw== X-Gm-Message-State: AOAM531CVYHjzxEEDr8lAaB8yPq8DpaNa+3cXLnJuW9p3LQ5gwrscfmX DCYAhK7tTIWkJZMcHl6d+6wcdw5sWFmBsHTFZwxBFitdd5s5hoNCdQlEey+sYl/rOTHAtxC4nLv WdNXNk28Bdmb/ X-Received: by 2002:a05:600c:4f15:: with SMTP id l21mr28700399wmq.72.1625694497181; Wed, 07 Jul 2021 14:48:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxM7lGZp/iD4L9G5xqaYUuQfm/4DcRAzM24ezG7RuatVD8l4V3ER8Jf4oCp6GSduKGjyxhVSA== X-Received: by 2002:a05:600c:4f15:: with SMTP id l21mr28700380wmq.72.1625694497005; Wed, 07 Jul 2021 14:48:17 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id u15sm7265281wmq.1.2021.07.07.14.48.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:48:16 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 2/7] bpf: Enable BPF_TRAMP_F_IP_ARG for trampolines with call_get_func_ip Date: Wed, 7 Jul 2021 23:47:46 +0200 Message-Id: <20210707214751.159713-3-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Enabling BPF_TRAMP_F_IP_ARG for trampolines that actually need it. The BPF_TRAMP_F_IP_ARG adds extra 3 instructions to trampoline code and is used only by programs with bpf_get_func_ip helper, which is added in following patch and sets call_get_func_ip bit. This patch ensures that BPF_TRAMP_F_IP_ARG flag is used only for trampolines that have programs with call_get_func_ip set. Signed-off-by: Jiri Olsa --- include/linux/filter.h | 3 ++- kernel/bpf/trampoline.c | 12 +++++++++--- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 472f97074da0..ba36989f711a 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -559,7 +559,8 @@ struct bpf_prog { kprobe_override:1, /* Do we override a kprobe? */ has_callchain_buf:1, /* callchain buffer allocated? */ enforce_expected_attach_type:1, /* Enforce expected_attach_type checking at attach time */ - call_get_stack:1; /* Do we call bpf_get_stack() or bpf_get_stackid() */ + call_get_stack:1, /* Do we call bpf_get_stack() or bpf_get_stackid() */ + call_get_func_ip:1; /* Do we call get_func_ip() */ enum bpf_prog_type type; /* Type of BPF program */ enum bpf_attach_type expected_attach_type; /* For some prog types */ u32 len; /* Number of filter blocks */ diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 28a3630c48ee..b2535acfe9db 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -172,7 +172,7 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr) } static struct bpf_tramp_progs * -bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total) +bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_arg) { const struct bpf_prog_aux *aux; struct bpf_tramp_progs *tprogs; @@ -189,8 +189,10 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total) *total += tr->progs_cnt[kind]; progs = tprogs[kind].progs; - hlist_for_each_entry(aux, &tr->progs_hlist[kind], tramp_hlist) + hlist_for_each_entry(aux, &tr->progs_hlist[kind], tramp_hlist) { + *ip_arg |= aux->prog->call_get_func_ip; *progs++ = aux->prog; + } } return tprogs; } @@ -333,9 +335,10 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr) struct bpf_tramp_image *im; struct bpf_tramp_progs *tprogs; u32 flags = BPF_TRAMP_F_RESTORE_REGS; + bool ip_arg = false; int err, total; - tprogs = bpf_trampoline_get_progs(tr, &total); + tprogs = bpf_trampoline_get_progs(tr, &total, &ip_arg); if (IS_ERR(tprogs)) return PTR_ERR(tprogs); @@ -357,6 +360,9 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr) tprogs[BPF_TRAMP_MODIFY_RETURN].nr_progs) flags = BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_SKIP_FRAME; + if (ip_arg) + flags |= BPF_TRAMP_F_IP_ARG; + err = arch_prepare_bpf_trampoline(im, im->image, im->image + PAGE_SIZE, &tr->func.model, flags, tprogs, tr->func.addr); From patchwork Wed Jul 7 21:47:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364073 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C77C07E95 for ; Wed, 7 Jul 2021 21:48:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 391DD61CB0 for ; Wed, 7 Jul 2021 21:48:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232057AbhGGVvN (ORCPT ); Wed, 7 Jul 2021 17:51:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24746 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231317AbhGGVvI (ORCPT ); Wed, 7 Jul 2021 17:51:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FQtsSphgnYl9qFWBMJGlJNrhM3rcnyrmEFUQ3XZzsHM=; b=H+8DEv4Tgf4o1wOnSTMF3pGxPfT/nMkGNJ9vWRj76NEZStIKrPdTQlBIOz1wEBTzm7+/hu kAw0zAmi9TrCThPSUUp87QQbDxCRVp/qO72jY8bw5il0Mu6uchAsQjIa58TVY0D83Fiogx 2hPBDorzBaYe4XK5zvlItFHXyGQapdE= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-182--mg0bQENPeGTKHDEl8Hz6A-1; Wed, 07 Jul 2021 17:48:26 -0400 X-MC-Unique: -mg0bQENPeGTKHDEl8Hz6A-1 Received: by mail-wm1-f70.google.com with SMTP id h22-20020a7bc9360000b0290215b0f3da63so595653wml.3 for ; Wed, 07 Jul 2021 14:48:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FQtsSphgnYl9qFWBMJGlJNrhM3rcnyrmEFUQ3XZzsHM=; b=Gqr7aKfDNzr7eQzoM1KSHkwz1qvv770uOS50ywJJ9K03xCWN/qz46JDzIDs4h2/ffb FLl5FL3GTEXvlWCE0UVPLdBHKJevNXezeKIYIWCqHTI/fPuLWbwrA5LEUN1v+1XJjMeX oO+Qz595Oro3wDR+6aBpYbClwGTTWgt2OJRyUntr7HqYyCf901sUIovEnE1U2xUz0tZ7 H8VNsVb86xeR8/MgUKxShoUM6ZLjJFH1DR76nFwZA04rJiuvH5c2HvYWnoHXNKY9Bjvy 2ilnGv8Xz1Megl42gLFefK6dZIYN7REDaUQcYKlxcJKDbBZva+EFxuQ0tXBDoh4KBtVz DeFg== X-Gm-Message-State: AOAM533nqcx1LlLmsJU/OrtO/26zduJf0UzO3otZpAPrM3cQcD7aUP34 c4Nphl/MRmvwjyfRe3fG3W+6s6zwX+8e2d4LP6zfKB/Q0pBENQc/I+k3utdI3XqY62MgKHPtY1o BNpeBlckvHxfA X-Received: by 2002:a7b:c40d:: with SMTP id k13mr29429804wmi.97.1625694505315; Wed, 07 Jul 2021 14:48:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx5550/Ol6QmHMH+/kX6iFJFZwn2w34mju8EOtgfB02HXSUyWqItABbxMNPOlFsZM17sDsXGw== X-Received: by 2002:a7b:c40d:: with SMTP id k13mr29429780wmi.97.1625694505151; Wed, 07 Jul 2021 14:48:25 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id u15sm7534378wmq.48.2021.07.07.14.48.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:48:24 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: kernel test robot , Dan Carpenter , netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 3/7] bpf: Add bpf_get_func_ip helper for tracing programs Date: Wed, 7 Jul 2021 23:47:47 +0200 Message-Id: <20210707214751.159713-4-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding bpf_get_func_ip helper for BPF_PROG_TYPE_TRACING programs, specifically for all trampoline attach types. The trampoline's caller IP address is stored in (ctx - 8) address. so there's no reason to actually call the helper, but rather fixup the call instruction and return [ctx - 8] value directly (suggested by Alexei). [fixed has_get_func_ip wrong return type] Reported-by: kernel test robot Reported-by: Dan Carpenter Signed-off-by: Jiri Olsa --- include/uapi/linux/bpf.h | 7 +++++ kernel/bpf/verifier.c | 53 ++++++++++++++++++++++++++++++++++ kernel/trace/bpf_trace.c | 15 ++++++++++ tools/include/uapi/linux/bpf.h | 7 +++++ 4 files changed, 82 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index bf9252c7381e..83e87ffdbb6e 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -4780,6 +4780,12 @@ union bpf_attr { * Execute close syscall for given FD. * Return * A syscall result. + * + * u64 bpf_get_func_ip(void *ctx) + * Description + * Get address of the traced function (for tracing programs). + * Return + * Address of the traced function. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -4951,6 +4957,7 @@ union bpf_attr { FN(sys_bpf), \ FN(btf_find_by_name_kind), \ FN(sys_close), \ + FN(get_func_ip), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index be38bb930bf1..f975a3aa9368 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5149,6 +5149,11 @@ static bool allow_tail_call_in_subprogs(struct bpf_verifier_env *env) return env->prog->jit_requested && IS_ENABLED(CONFIG_X86_64); } +static bool allow_get_func_ip_tracing(struct bpf_verifier_env *env) +{ + return env->prog->jit_requested && IS_ENABLED(CONFIG_X86_64); +} + static int check_map_func_compatibility(struct bpf_verifier_env *env, struct bpf_map *map, int func_id) { @@ -5955,6 +5960,32 @@ static int check_bpf_snprintf_call(struct bpf_verifier_env *env, return err; } +static int has_get_func_ip(struct bpf_verifier_env *env) +{ + enum bpf_attach_type eatype = env->prog->expected_attach_type; + enum bpf_prog_type type = resolve_prog_type(env->prog); + int func_id = BPF_FUNC_get_func_ip; + + if (type == BPF_PROG_TYPE_TRACING) { + if (eatype != BPF_TRACE_FENTRY && eatype != BPF_TRACE_FEXIT && + eatype != BPF_MODIFY_RETURN) { + verbose(env, "func %s#%d supported only for fentry/fexit/fmod_ret programs\n", + func_id_name(func_id), func_id); + return -ENOTSUPP; + } + if (!allow_get_func_ip_tracing(env)) { + verbose(env, "func %s#%d for tracing programs supported only for JITed x86_64\n", + func_id_name(func_id), func_id); + return -ENOTSUPP; + } + return 0; + } + + verbose(env, "func %s#%d not supported for program type %d\n", + func_id_name(func_id), func_id, type); + return -ENOTSUPP; +} + static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -6225,6 +6256,12 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn if (func_id == BPF_FUNC_get_stackid || func_id == BPF_FUNC_get_stack) env->prog->call_get_stack = true; + if (func_id == BPF_FUNC_get_func_ip) { + if (has_get_func_ip(env)) + return -ENOTSUPP; + env->prog->call_get_func_ip = true; + } + if (changes_data) clear_all_pkt_pointers(env); return 0; @@ -12369,6 +12406,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) { struct bpf_prog *prog = env->prog; bool expect_blinding = bpf_jit_blinding_enabled(prog); + enum bpf_prog_type prog_type = resolve_prog_type(prog); struct bpf_insn *insn = prog->insnsi; const struct bpf_func_proto *fn; const int insn_cnt = prog->len; @@ -12702,6 +12740,21 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } + /* Implement bpf_get_func_ip inline. */ + if (prog_type == BPF_PROG_TYPE_TRACING && + insn->imm == BPF_FUNC_get_func_ip) { + /* Load IP address from ctx - 8 */ + insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, -8); + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1); + if (!new_prog) + return -ENOMEM; + + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + continue; + } + patch_call_imm: fn = env->ops->get_func_proto(insn->imm, env->prog); /* all functions that have prototype and verifier allowed diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 64bd2d84367f..9edd3b1a00ad 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -948,6 +948,19 @@ const struct bpf_func_proto bpf_snprintf_btf_proto = { .arg5_type = ARG_ANYTHING, }; +BPF_CALL_1(bpf_get_func_ip_tracing, void *, ctx) +{ + /* Stub, the helper call is inlined in the program. */ + return 0; +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_tracing = { + .func = bpf_get_func_ip_tracing, + .gpl_only = true, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + const struct bpf_func_proto * bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -1058,6 +1071,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_for_each_map_elem_proto; case BPF_FUNC_snprintf: return &bpf_snprintf_proto; + case BPF_FUNC_get_func_ip: + return &bpf_get_func_ip_proto_tracing; default: return NULL; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index bf9252c7381e..83e87ffdbb6e 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -4780,6 +4780,12 @@ union bpf_attr { * Execute close syscall for given FD. * Return * A syscall result. + * + * u64 bpf_get_func_ip(void *ctx) + * Description + * Get address of the traced function (for tracing programs). + * Return + * Address of the traced function. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -4951,6 +4957,7 @@ union bpf_attr { FN(sys_bpf), \ FN(btf_find_by_name_kind), \ FN(sys_close), \ + FN(get_func_ip), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper From patchwork Wed Jul 7 21:47:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364075 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8517C07E95 for ; Wed, 7 Jul 2021 21:48:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B07C361CB0 for ; Wed, 7 Jul 2021 21:48:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233033AbhGGVvU (ORCPT ); Wed, 7 Jul 2021 17:51:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:25258 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232996AbhGGVvS (ORCPT ); Wed, 7 Jul 2021 17:51:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XyCTtNsvJM23bTJlhCXcXFu90hTwq9G3m+Uc2FcCDhw=; b=Se9zk1X8A3xV3LpnfZm66kVx8CDKosItrTUc5YEKUDGr8igVt18wHgL0zljSHbBQxSq1a/ dCSn7LeVJiEya86nYLzWUSU2CqWTSQUtYVkesjEggeKMtEKE9ofbrgtLF6v5697Qq2GYpN J2mQwptqLNCqa/4KGlheUD79kdq3cKw= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-XyNtHVkoNR-ZhVOQu4YK_g-1; Wed, 07 Jul 2021 17:48:35 -0400 X-MC-Unique: XyNtHVkoNR-ZhVOQu4YK_g-1 Received: by mail-wm1-f70.google.com with SMTP id t12-20020a7bc3cc0000b02901f290c9c44eso1533875wmj.7 for ; Wed, 07 Jul 2021 14:48:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XyCTtNsvJM23bTJlhCXcXFu90hTwq9G3m+Uc2FcCDhw=; b=p+kDaiFNVe8PWhWwkgXV21pdN74nu3LAmQz0tgEX8QMyyFEzJDy4Asn5bc4wLPO2X7 aoSZrcAtGARBWLU7bFD8WptJ9jGs7EaIhqNEiu1+PkN76IcTEwZcl2kUwn4628q7sYw0 gIdIrN72AJcBBMr1wj6dmr2lU/a+Vh5Vw1LCcL9idDa8qypD++6YyWGs+9F+wVVTCeze kXLQVynUxOFV3Q216AviTezKEeGp0kprBfOlQQKFNL52MgcNQV+nWAPDBTiHKlxoFoRd fUXahJlS5r0seOlQvSF38hzoUdlJo/9pGBLePdlYUK34hYSfogL093i+tbsRqfQw9VOp Sadg== X-Gm-Message-State: AOAM530WCe0WyTOitzwIVA7SxsSwBPlduN9w/g+pN+Y4C+S5n2zNfE7Z +M6t/ZNhTE59WwxY5/ajQ2kCNRvLtMLGBEXSsWLQKT9dSc/FG9lEJz4kGj01tQ8rX7zp6Qm6uFL wRe6zGUFH7VNK X-Received: by 2002:adf:a2db:: with SMTP id t27mr29810081wra.272.1625694514725; Wed, 07 Jul 2021 14:48:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy1NKGIS5pSSuMCbb3wlNDchZjdi7A4xHRDEuiUKRnEl86QJ3yOKStzcLekLUeKf1S8O2RVww== X-Received: by 2002:adf:a2db:: with SMTP id t27mr29810063wra.272.1625694514514; Wed, 07 Jul 2021 14:48:34 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id p9sm150545wrx.59.2021.07.07.14.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:48:34 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 4/7] bpf: Add bpf_get_func_ip helper for kprobe programs Date: Wed, 7 Jul 2021 23:47:48 +0200 Message-Id: <20210707214751.159713-5-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding bpf_get_func_ip helper for BPF_PROG_TYPE_KPROBE programs, so it's now possible to call bpf_get_func_ip from both kprobe and kretprobe programs. Taking the caller's address from 'struct kprobe::addr', which is defined for both kprobe and kretprobe. Signed-off-by: Jiri Olsa Reviewed-by: Masami Hiramatsu --- include/uapi/linux/bpf.h | 2 +- kernel/bpf/verifier.c | 2 ++ kernel/trace/bpf_trace.c | 17 +++++++++++++++++ tools/include/uapi/linux/bpf.h | 2 +- 4 files changed, 21 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 83e87ffdbb6e..4894f99a1993 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -4783,7 +4783,7 @@ union bpf_attr { * * u64 bpf_get_func_ip(void *ctx) * Description - * Get address of the traced function (for tracing programs). + * Get address of the traced function (for tracing and kprobe programs). * Return * Address of the traced function. */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f975a3aa9368..79eb9d81a198 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5979,6 +5979,8 @@ static int has_get_func_ip(struct bpf_verifier_env *env) return -ENOTSUPP; } return 0; + } else if (type == BPF_PROG_TYPE_KPROBE) { + return 0; } verbose(env, "func %s#%d not supported for program type %d\n", diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 9edd3b1a00ad..55acf56b0c3a 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -17,6 +17,7 @@ #include #include #include +#include #include @@ -961,6 +962,20 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_tracing = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_func_ip_kprobe, struct pt_regs *, regs) +{ + struct kprobe *kp = kprobe_running(); + + return kp ? (u64) kp->addr : 0; +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { + .func = bpf_get_func_ip_kprobe, + .gpl_only = true, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + const struct bpf_func_proto * bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -1092,6 +1107,8 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_override_return: return &bpf_override_return_proto; #endif + case BPF_FUNC_get_func_ip: + return &bpf_get_func_ip_proto_kprobe; default: return bpf_tracing_func_proto(func_id, prog); } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 83e87ffdbb6e..4894f99a1993 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -4783,7 +4783,7 @@ union bpf_attr { * * u64 bpf_get_func_ip(void *ctx) * Description - * Get address of the traced function (for tracing programs). + * Get address of the traced function (for tracing and kprobe programs). * Return * Address of the traced function. */ From patchwork Wed Jul 7 21:47:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364077 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2BD8C07E95 for ; Wed, 7 Jul 2021 21:48:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B72C461C3C for ; Wed, 7 Jul 2021 21:48:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233067AbhGGVv2 (ORCPT ); Wed, 7 Jul 2021 17:51:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58202 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233051AbhGGVv2 (ORCPT ); Wed, 7 Jul 2021 17:51:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Yu/yssvroh5wN9vZzKT4DxZ0/5KDe3VlIF6FttuA20I=; b=Olwck9iyluT/kl0MsYLZviPeqc12v6Wm9Xqf02Z5L9KDJI6WYMUO1qFSAzkCRAYuoD7HRH Pm/B/YYZ74NTDVofV2KSt6u+91RGC0Cahtl4UWZD/yHQabbr/CVKLqylPAPC37x9jb7iHe QUlZGiQTfe4gx+yNKkcUQys5EMaCDjc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-573-TMfgLiKSNSy8BFmuh6YO5g-1; Wed, 07 Jul 2021 17:48:45 -0400 X-MC-Unique: TMfgLiKSNSy8BFmuh6YO5g-1 Received: by mail-wm1-f71.google.com with SMTP id z4-20020a1ce2040000b02901ee8d8e151eso3020604wmg.1 for ; Wed, 07 Jul 2021 14:48:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Yu/yssvroh5wN9vZzKT4DxZ0/5KDe3VlIF6FttuA20I=; b=cYCu/GFiOnz7/V9MahWCT6w6OwjW8HwpxUUdzDwY1wTjCQrEBNOSBdu8X8z+Czw3G+ uRBmL2RSQ/DEpbw2vhoRh686o1iD2mhyfdPCMco/Hxn4rJsBBBxL2wOm6OLFTPIB03Nx ATCb/a3LlggCWeSRVaxZjYyJOBpCJVycFI7v0B+5QNS/nsQo+KEI64Bpd2b17erY5QUj eB00Q1Xy1ZTlWkl3cjX3KVhy7DMgxGeYOpalqplzqB0M4JXv1mo+koUozzWtaskebD6P Mo6e/lL5SKtVn6eUWSws6rI+9NXUgPtJLWhmJRlafI31k9NbqTf+5aBMOf8+GYwkI4+i yVPg== X-Gm-Message-State: AOAM531d5XT0ifXlEbxAPYbCMepd4IwU1oPXk0mZ2RYledPiSP/KFvLv qh6jSZWCHmp/XPOj/y9/JSbUqFOWF306FtsXHVuuGwYmSEcctnZrlfO+ya4kUAczzOH/TflUyKG 9/XcGoOx+pVQ6 X-Received: by 2002:a5d:638b:: with SMTP id p11mr30691602wru.380.1625694524795; Wed, 07 Jul 2021 14:48:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJycPz+sXvZJjhr8Xj/sy2ZBxPlZ6jAR/5pNIkKBzJPvo5FZIjuvJ/z1RZhekC6aT943k9yvPQ== X-Received: by 2002:a5d:638b:: with SMTP id p11mr30691594wru.380.1625694524675; Wed, 07 Jul 2021 14:48:44 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id o19sm7365480wmc.12.2021.07.07.14.48.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:48:44 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 5/7] selftests/bpf: Add test for bpf_get_func_ip helper Date: Wed, 7 Jul 2021 23:47:49 +0200 Message-Id: <20210707214751.159713-6-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding test for bpf_get_func_ip helper for fentry, fexit, kprobe, kretprobe and fmod_ret programs. Signed-off-by: Jiri Olsa --- .../bpf/prog_tests/get_func_ip_test.c | 42 +++++++++++++ .../selftests/bpf/progs/get_func_ip_test.c | 62 +++++++++++++++++++ 2 files changed, 104 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c create mode 100644 tools/testing/selftests/bpf/progs/get_func_ip_test.c diff --git a/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c new file mode 100644 index 000000000000..06d34f566bbb --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/get_func_ip_test.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "get_func_ip_test.skel.h" + +void test_get_func_ip_test(void) +{ + struct get_func_ip_test *skel = NULL; + __u32 duration = 0, retval; + int err, prog_fd, i; + __u64 *result; + + skel = get_func_ip_test__open_and_load(); + if (!ASSERT_OK_PTR(skel, "get_func_ip_test__open_and_load")) + goto cleanup; + + err = get_func_ip_test__attach(skel); + if (!ASSERT_OK(err, "get_func_ip_test__attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + prog_fd = bpf_program__fd(skel->progs.fmod_ret_test); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + + ASSERT_OK(err, "test_run"); + + result = (__u64 *)skel->bss; + for (i = 0; i < sizeof(*skel->bss) / sizeof(__u64); i++) { + if (!ASSERT_EQ(result[i], 1, "fentry_result")) + break; + } + + get_func_ip_test__detach(skel); + +cleanup: + get_func_ip_test__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c new file mode 100644 index 000000000000..8ca54390d2b1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +extern const void bpf_fentry_test1 __ksym; +extern const void bpf_fentry_test2 __ksym; +extern const void bpf_fentry_test3 __ksym; +extern const void bpf_fentry_test4 __ksym; +extern const void bpf_modify_return_test __ksym; + +__u64 test1_result = 0; +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test1, int a) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test1_result = (const void *) addr == &bpf_fentry_test1; + return 0; +} + +__u64 test2_result = 0; +SEC("fexit/bpf_fentry_test2") +int BPF_PROG(test2, int a) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test2_result = (const void *) addr == &bpf_fentry_test2; + return 0; +} + +__u64 test3_result = 0; +SEC("kprobe/bpf_fentry_test3") +int test3(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test3_result = (const void *) addr == &bpf_fentry_test3; + return 0; +} + +__u64 test4_result = 0; +SEC("kretprobe/bpf_fentry_test4") +int BPF_KRETPROBE(test4) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test4_result = (const void *) addr == &bpf_fentry_test4; + return 0; +} + +__u64 test5_result = 0; +SEC("fmod_ret/bpf_modify_return_test") +int BPF_PROG(fmod_ret_test, int a, int *b, int ret) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test5_result = (const void *) addr == &bpf_modify_return_test; + return ret; +} From patchwork Wed Jul 7 21:47:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364079 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25BD6C07E95 for ; Wed, 7 Jul 2021 21:49:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1019661CC0 for ; Wed, 7 Jul 2021 21:49:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233100AbhGGVvo (ORCPT ); Wed, 7 Jul 2021 17:51:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32547 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233072AbhGGVvn (ORCPT ); Wed, 7 Jul 2021 17:51:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N6Lkp9H+PeXEJohL6vruWVGim8bb8jO4Rintdh3j+00=; b=VP32trsuwK7tdMZr5qDxW/pTWB54JJJGJeHr3B6hRge1Y3c664xpmIWf2TiHOVG+ADcS8X steStxfGmnTPM10WdMKacyRu26gXzMRthaoZGrV2z5vCw7UiwOPNdfwhHGQxczVrp1nwuL ehLHt5Pw8oyJe0oJE5MUZmnysIrpOBg= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-315-MMn0LmJMPHGZL-tLnomIBg-1; Wed, 07 Jul 2021 17:49:01 -0400 X-MC-Unique: MMn0LmJMPHGZL-tLnomIBg-1 Received: by mail-wm1-f72.google.com with SMTP id t12-20020a7bc3cc0000b02901f290c9c44eso1534276wmj.7 for ; Wed, 07 Jul 2021 14:49:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N6Lkp9H+PeXEJohL6vruWVGim8bb8jO4Rintdh3j+00=; b=T/L0900DCyAJn7CFnZKCJ1eOVEbHCDffrqpEYyM4Md5Alc4+yoSa13pRoLZM2n13ZR 3i3nzpaIxusftyDI0jvQZelFCpLRNzj0Wjjs0VUJjHLKKNgocywHqMB6I+NtGlsiVi/n HuczWKEzCfJrp0UURzq4IOsv6AXUHUN7S0PhaOEFpHJQcH1qmwPrJq9ZkXaHOrE7f47j zVyoMKtzA57AzCHL2q52bh+BUYfSSLafcevGhU8VjS0JKX7K9e9iwqjRE+CB+hGQXgnj 2ld/B0Q3sRHdVK4tfMybO5yHgh6IIAo5JqOxaaJW00LXQKQl5IUa0YCTivAyhdZYxBDf DT1g== X-Gm-Message-State: AOAM530IIXjoA0S5wJZ+WcGTTbY8Cq2Cm6SobRcbNVXe72+Y37dsiz1n hBzpvsmaugVwpY3Og1RRz/qD4+Wo3/GWX+gr8vcd33t2DyGBMVw+4seX5vUphrI/mD1RNmWZnem Q3FXWCV1GtsAo X-Received: by 2002:adf:9466:: with SMTP id 93mr30832144wrq.340.1625694540668; Wed, 07 Jul 2021 14:49:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzs/zK/o/CEalUgy0txiMvjZKapygMpZixoKK0bABasdS9LEUn7Zpd1LxDLk2h3Osby8JMxBg== X-Received: by 2002:adf:9466:: with SMTP id 93mr30832125wrq.340.1625694540543; Wed, 07 Jul 2021 14:49:00 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id c16sm144524wmr.2.2021.07.07.14.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:49:00 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 6/7] libbpf: allow specification of "kprobe/function+offset" Date: Wed, 7 Jul 2021 23:47:50 +0200 Message-Id: <20210707214751.159713-7-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alan Maguire kprobes can be placed on most instructions in a function, not just entry, and ftrace and bpftrace support the function+offset notification for probe placement. Adding parsing of func_name into func+offset to bpf_program__attach_kprobe() allows the user to specify SEC("kprobe/bpf_fentry_test5+0x6") ...for example, and the offset can be passed to perf_event_open_probe() to support kprobe attachment. Signed-off-by: Alan Maguire Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 1e04ce724240..60c9e3e77684 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -10309,11 +10309,25 @@ struct bpf_link *bpf_program__attach_kprobe(struct bpf_program *prog, const char *func_name) { char errmsg[STRERR_BUFSIZE]; + char func[BPF_OBJ_NAME_LEN]; + unsigned long offset = 0; struct bpf_link *link; - int pfd, err; + int pfd, err, n; + + n = sscanf(func_name, "%[a-zA-Z0-9_.]+%lx", func, &offset); + if (n < 1) { + err = -EINVAL; + pr_warn("kprobe name is invalid: %s\n", func_name); + return libbpf_err_ptr(err); + } + if (retprobe && offset != 0) { + err = -EINVAL; + pr_warn("kretprobes do not support offset specification\n"); + return libbpf_err_ptr(err); + } - pfd = perf_event_open_probe(false /* uprobe */, retprobe, func_name, - 0 /* offset */, -1 /* pid */); + pfd = perf_event_open_probe(false /* uprobe */, retprobe, func, + offset, -1 /* pid */); if (pfd < 0) { pr_warn("prog '%s': failed to create %s '%s' perf event: %s\n", prog->name, retprobe ? "kretprobe" : "kprobe", func_name, From patchwork Wed Jul 7 21:47:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12364081 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1701DC07E95 for ; Wed, 7 Jul 2021 21:49:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EEFF661C3C for ; Wed, 7 Jul 2021 21:49:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232724AbhGGVvz (ORCPT ); Wed, 7 Jul 2021 17:51:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:33526 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230378AbhGGVvz (ORCPT ); Wed, 7 Jul 2021 17:51:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1625694554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zbTjAalRT8G7ucBl7KamxhzUVx0mh7mTfA4EZ32X+1U=; b=Qyh3ogfsjzwZ9ol+utWPc0bgKEDV+JcdUEzVZYC8x6CrBvjYvbn1aFQ0OrH+mOmb68z654 KvIH0Z9eS8zJJA8JkBHBx10Yzb//M7K9eLUiQexmhLAV7x8z0ZzdFzJL4BRlbfFuReLPef 7xbCDkE1fuLDUlg6MpDz7febFRxU4Ks= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-403-5_mVwFRdMY2fezgdbSFQgA-1; Wed, 07 Jul 2021 17:49:11 -0400 X-MC-Unique: 5_mVwFRdMY2fezgdbSFQgA-1 Received: by mail-wm1-f69.google.com with SMTP id p4-20020a05600c3584b0290210725ac6bcso2687141wmq.2 for ; Wed, 07 Jul 2021 14:49:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zbTjAalRT8G7ucBl7KamxhzUVx0mh7mTfA4EZ32X+1U=; b=mnXKSGD8o4WpDFYN5NwFhFfkgsq5MKDrk4SaJzqxtIXWIKKvLXxl1mffxoV0JcfQdI mMKmUa6FWRFjeNJ3IvLFV9qHSRmSPlMQZ+ZYe+QzRHar03vL/zDHnF25V5PCWuR+5e4J 4DN0xPcZD1piiX/6OZII8nZ/ht5L35zllA3bef2eeHuan2cGqDeuXsnNGy/uX7nEPSsT D1AaN66tjVZbB6gg717Y+2o5YSCUZl11oYkrfysY8jJj2xmvXdmMx6WqmUmh8t+Y/edR VFFLG6uk5FwQnI2V3E3W0ubTdOCj9Lke+Y2SlHtbWsQPibelDjvQRUFDCV9O+GPG2ntV 4ETg== X-Gm-Message-State: AOAM5316v85kI56TEASchQimZ4nq62RhlQfI81CaoTVCMH6qPC3yc3/g MZpUXJtSKqPI8bFnzGO40YPa1M4hfJcqYHcvwyopCJTZSbAU5VMH9IBM6U5tneJVt0LxyklY4kZ eiEZvBvHH/EdG X-Received: by 2002:a1c:4b08:: with SMTP id y8mr28161396wma.80.1625694550244; Wed, 07 Jul 2021 14:49:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwMElhZP2kH1WV42xtt2FEDaPsQDguhdbByE9+5Y4+JgnQ44u0SnjbpMEECslWGFOsiWd4PPQ== X-Received: by 2002:a1c:4b08:: with SMTP id y8mr28161385wma.80.1625694550135; Wed, 07 Jul 2021 14:49:10 -0700 (PDT) Received: from krava.redhat.com ([185.153.78.55]) by smtp.gmail.com with ESMTPSA id k5sm148994wmk.11.2021.07.07.14.49.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jul 2021 14:49:09 -0700 (PDT) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Masami Hiramatsu , Alan Maguire Subject: [PATCHv3 bpf-next 7/7] selftests/bpf: Add test for bpf_get_func_ip in kprobe+offset probe Date: Wed, 7 Jul 2021 23:47:51 +0200 Message-Id: <20210707214751.159713-8-jolsa@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210707214751.159713-1-jolsa@kernel.org> References: <20210707214751.159713-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding test for bpf_get_func_ip in kprobe+ofset probe. Because of the offset value it's arch specific, adding it only for x86_64 architecture. Signed-off-by: Jiri Olsa --- .../testing/selftests/bpf/progs/get_func_ip_test.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_test.c index 8ca54390d2b1..e8a9428a0ea3 100644 --- a/tools/testing/selftests/bpf/progs/get_func_ip_test.c +++ b/tools/testing/selftests/bpf/progs/get_func_ip_test.c @@ -10,6 +10,7 @@ extern const void bpf_fentry_test2 __ksym; extern const void bpf_fentry_test3 __ksym; extern const void bpf_fentry_test4 __ksym; extern const void bpf_modify_return_test __ksym; +extern const void bpf_fentry_test6 __ksym; __u64 test1_result = 0; SEC("fentry/bpf_fentry_test1") @@ -60,3 +61,15 @@ int BPF_PROG(fmod_ret_test, int a, int *b, int ret) test5_result = (const void *) addr == &bpf_modify_return_test; return ret; } + +#ifdef __x86_64__ +__u64 test6_result = 0; +SEC("kprobe/bpf_fentry_test6+0x5") +int test6(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test6_result = (const void *) addr == &bpf_fentry_test6 + 5; + return 0; +} +#endif