From patchwork Wed Feb 2 13:53:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732934 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04CF5C433FE for ; Wed, 2 Feb 2022 13:53:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344455AbiBBNxp (ORCPT ); Wed, 2 Feb 2022 08:53:45 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44499 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231912AbiBBNxo (ORCPT ); Wed, 2 Feb 2022 08:53:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810023; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iTqwruTWPfHMFTIBAKX8hnY20kDuzrc4oApkYAdRO6g=; b=iRI0cYP1pUemaRfko4RHOvJP0M//SqkNTd2mz7b+lUYyUn4PYctsilYSCoh9THo4J4LYnm mCVPSzHNY08NIGhpdySu4ZddtgIedmahyacLUTIkf+LE7uEzU7ma2CS+3gjEW3eFV3ByTj vGu8mslNKm0Vn9PhAUkdz96V4JmBXmw= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-596-NK7eBgTIOeKZeNDtwaxH2Q-1; Wed, 02 Feb 2022 08:53:42 -0500 X-MC-Unique: NK7eBgTIOeKZeNDtwaxH2Q-1 Received: by mail-ed1-f69.google.com with SMTP id l16-20020aa7c3d0000000b004070ea10e7fso10366264edr.3 for ; Wed, 02 Feb 2022 05:53:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iTqwruTWPfHMFTIBAKX8hnY20kDuzrc4oApkYAdRO6g=; b=rr8kkcL62EvpWFwxs3UwNxzRGl6VhjpuW4mH2INJtDGs7BdkJTjsFF/idqX5WOR4sa omsd3ZRuKCcNH2pvZPIYttdlVKzH+2nO5jko/D4o1JhxWToTZfUY8cFg5ReqdvMoPWId rJ3CKERw+6Ckv0I7VGfKlr1hzeIAHBK7T+qD+V86BuoBIcxBazLw5+UPnCkvcQEx+Epy jBfjq7YDm2Dl71Zs7m/q/Sc8DJ0zX88ZHDFpW3vBUQ20qW6CDQrLczlW0n1OVRjwms6U L4vt+b/qOOfH91OAIz3g6C6GDFrlBmwhhNQ0kL/YfQGZWBDZ/zW4hAYc3MvfbytSdPez ACBg== X-Gm-Message-State: AOAM532087M/nqNtPcNrFZ3poUS17k4B+Wi2yc73Tukg/JtkueCCDkoo APfiXuRD4f5cPQbRmZK8ZgZ3caOmuziJnUauU7SyFRm5Uh1f+KGiAl7IaZsTtMWatE1ORIwnQxe Y6GTIJoQMq3lQsHQg X-Received: by 2002:a17:906:3602:: with SMTP id q2mr24970101ejb.331.1643810021162; Wed, 02 Feb 2022 05:53:41 -0800 (PST) X-Google-Smtp-Source: ABdhPJyXzqoQS1ZvJxxLsZ+BwuIIPfRrZtm3dn+Sy3pGMKpT/12UlVJwv5gPSUhL3DCYSekTahKqjg== X-Received: by 2002:a17:906:3602:: with SMTP id q2mr24970087ejb.331.1643810020887; Wed, 02 Feb 2022 05:53:40 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id a18sm2368808edu.31.2022.02.02.05.53.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:53:40 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 1/8] bpf: Add support to attach kprobe program with fprobe Date: Wed, 2 Feb 2022 14:53:26 +0100 Message-Id: <20220202135333.190761-2-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding new link type BPF_LINK_TYPE_FPROBE that attaches kprobe program through fprobe API. The fprobe API allows to attach probe on multiple functions at once very fast, because it works on top of ftrace. On the other hand this limits the probe point to the function entry or return. The kprobe program gets the same pt_regs input ctx as when it's attached through the perf API. Adding new attach type BPF_TRACE_FPROBE that enables such link for kprobe program. User provides array of addresses or symbols with count to attach the kprobe program to. The new link_create uapi interface looks like: struct { __aligned_u64 syms; __aligned_u64 addrs; __u32 cnt; __u32 flags; } fprobe; The flags field allows single BPF_F_FPROBE_RETURN bit to create return fprobe. Signed-off-by: Masami Hiramatsu Signed-off-by: Jiri Olsa --- include/linux/bpf_types.h | 1 + include/uapi/linux/bpf.h | 13 ++ kernel/bpf/syscall.c | 248 ++++++++++++++++++++++++++++++++- tools/include/uapi/linux/bpf.h | 13 ++ 4 files changed, 270 insertions(+), 5 deletions(-) diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h index 48a91c51c015..e279cea46653 100644 --- a/include/linux/bpf_types.h +++ b/include/linux/bpf_types.h @@ -140,3 +140,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_XDP, xdp) #ifdef CONFIG_PERF_EVENTS BPF_LINK_TYPE(BPF_LINK_TYPE_PERF_EVENT, perf) #endif +BPF_LINK_TYPE(BPF_LINK_TYPE_FPROBE, fprobe) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index a7f0ddedac1f..c0912f0a3dfe 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -997,6 +997,7 @@ enum bpf_attach_type { BPF_SK_REUSEPORT_SELECT, BPF_SK_REUSEPORT_SELECT_OR_MIGRATE, BPF_PERF_EVENT, + BPF_TRACE_FPROBE, __MAX_BPF_ATTACH_TYPE }; @@ -1011,6 +1012,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_FPROBE = 8, MAX_BPF_LINK_TYPE, }; @@ -1118,6 +1120,11 @@ enum bpf_link_type { */ #define BPF_F_XDP_HAS_FRAGS (1U << 5) +/* link_create.fprobe.flags used in LINK_CREATE command for + * BPF_TRACE_FPROBE attach type to create return probe. + */ +#define BPF_F_FPROBE_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1472,6 +1479,12 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 syms; + __aligned_u64 addrs; + __u32 cnt; + __u32 flags; + } fprobe; }; } link_create; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 72ce1edde950..0cfbb112c8e1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -32,6 +32,7 @@ #include #include #include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -3015,8 +3016,235 @@ static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *pro fput(perf_file); return err; } +#else +static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_PERF_EVENTS */ +#ifdef CONFIG_FPROBE + +struct bpf_fprobe_link { + struct bpf_link link; + struct fprobe fp; + unsigned long *addrs; +}; + +static void bpf_fprobe_link_release(struct bpf_link *link) +{ + struct bpf_fprobe_link *fprobe_link; + + fprobe_link = container_of(link, struct bpf_fprobe_link, link); + unregister_fprobe(&fprobe_link->fp); +} + +static void bpf_fprobe_link_dealloc(struct bpf_link *link) +{ + struct bpf_fprobe_link *fprobe_link; + + fprobe_link = container_of(link, struct bpf_fprobe_link, link); + kfree(fprobe_link->addrs); + kfree(fprobe_link); +} + +static const struct bpf_link_ops bpf_fprobe_link_lops = { + .release = bpf_fprobe_link_release, + .dealloc = bpf_fprobe_link_dealloc, +}; + +static int fprobe_link_prog_run(struct bpf_fprobe_link *fprobe_link, + struct pt_regs *regs) +{ + int err; + + if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) { + err = 0; + goto out; + } + + rcu_read_lock(); + migrate_disable(); + err = bpf_prog_run(fprobe_link->link.prog, regs); + migrate_enable(); + rcu_read_unlock(); + + out: + __this_cpu_dec(bpf_prog_active); + return err; +} + +static void fprobe_link_entry_handler(struct fprobe *fp, unsigned long entry_ip, + struct pt_regs *regs) +{ + unsigned long saved_ip = instruction_pointer(regs); + struct bpf_fprobe_link *fprobe_link; + + /* + * Because fprobe's regs->ip is set to the next instruction of + * dynamic-ftrace insturction, correct entry ip must be set, so + * that the bpf program can access entry address via regs as same + * as kprobes. + */ + instruction_pointer_set(regs, entry_ip); + + fprobe_link = container_of(fp, struct bpf_fprobe_link, fp); + fprobe_link_prog_run(fprobe_link, regs); + + instruction_pointer_set(regs, saved_ip); +} + +static void fprobe_link_exit_handler(struct fprobe *fp, unsigned long entry_ip, + struct pt_regs *regs) +{ + unsigned long saved_ip = instruction_pointer(regs); + struct bpf_fprobe_link *fprobe_link; + + instruction_pointer_set(regs, entry_ip); + + fprobe_link = container_of(fp, struct bpf_fprobe_link, fp); + fprobe_link_prog_run(fprobe_link, regs); + + instruction_pointer_set(regs, saved_ip); +} + +static int fprobe_resolve_syms(const void *usyms, u32 cnt, + unsigned long *addrs) +{ + unsigned long addr, size; + const char **syms; + int err = -ENOMEM; + unsigned int i; + char *func; + + size = cnt * sizeof(*syms); + syms = kzalloc(size, GFP_KERNEL); + if (!syms) + return -ENOMEM; + + func = kzalloc(KSYM_NAME_LEN, GFP_KERNEL); + if (!func) + goto error; + + if (copy_from_user(syms, usyms, size)) { + err = -EFAULT; + goto error; + } + + for (i = 0; i < cnt; i++) { + err = strncpy_from_user(func, syms[i], KSYM_NAME_LEN); + if (err == KSYM_NAME_LEN) + err = -E2BIG; + if (err < 0) + goto error; + + err = -EINVAL; + if (func[0] == '\0') + goto error; + addr = kallsyms_lookup_name(func); + if (!addr) + goto error; + if (!kallsyms_lookup_size_offset(addr, &size, NULL)) + size = MCOUNT_INSN_SIZE; + addr = ftrace_location_range(addr, addr + size - 1); + if (!addr) + goto error; + addrs[i] = addr; + } + + err = 0; +error: + kfree(syms); + kfree(func); + return err; +} + +static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + struct bpf_fprobe_link *link = NULL; + struct bpf_link_primer link_primer; + unsigned long *addrs; + u32 flags, cnt, size; + void __user *uaddrs; + void __user *usyms; + int err; + + /* no support for 32bit archs yet */ + if (sizeof(u64) != sizeof(void *)) + return -EINVAL; + + if (prog->expected_attach_type != BPF_TRACE_FPROBE) + return -EINVAL; + + flags = attr->link_create.fprobe.flags; + if (flags & ~BPF_F_FPROBE_RETURN) + return -EINVAL; + + uaddrs = u64_to_user_ptr(attr->link_create.fprobe.addrs); + usyms = u64_to_user_ptr(attr->link_create.fprobe.syms); + if ((!uaddrs && !usyms) || (uaddrs && usyms)) + return -EINVAL; + + cnt = attr->link_create.fprobe.cnt; + if (!cnt) + return -EINVAL; + + size = cnt * sizeof(*addrs); + addrs = kzalloc(size, GFP_KERNEL); + if (!addrs) + return -ENOMEM; + + if (uaddrs) { + if (copy_from_user(addrs, uaddrs, size)) { + err = -EFAULT; + goto error; + } + } else { + err = fprobe_resolve_syms(usyms, cnt, addrs); + if (err) + goto error; + } + + link = kzalloc(sizeof(*link), GFP_KERNEL); + if (!link) { + err = -ENOMEM; + goto error; + } + + bpf_link_init(&link->link, BPF_LINK_TYPE_FPROBE, + &bpf_fprobe_link_lops, prog); + + err = bpf_link_prime(&link->link, &link_primer); + if (err) + goto error; + + if (flags & BPF_F_FPROBE_RETURN) + link->fp.exit_handler = fprobe_link_exit_handler; + else + link->fp.entry_handler = fprobe_link_entry_handler; + + link->addrs = addrs; + + err = register_fprobe_ips(&link->fp, addrs, cnt); + if (err) { + bpf_link_cleanup(&link_primer); + return err; + } + + return bpf_link_settle(&link_primer); + +error: + kfree(link); + kfree(addrs); + return err; +} +#else /* !CONFIG_FPROBE */ +static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) +{ + return -EOPNOTSUPP; +} +#endif + #define BPF_RAW_TRACEPOINT_OPEN_LAST_FIELD raw_tracepoint.prog_fd static int bpf_raw_tracepoint_open(const union bpf_attr *attr) @@ -4248,7 +4476,7 @@ static int tracing_bpf_link_attach(const union bpf_attr *attr, bpfptr_t uattr, return -EINVAL; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.iter_info_len +#define BPF_LINK_CREATE_LAST_FIELD link_create.fprobe.flags static int link_create(union bpf_attr *attr, bpfptr_t uattr) { enum bpf_prog_type ptype; @@ -4272,7 +4500,6 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) ret = tracing_bpf_link_attach(attr, uattr, prog); goto out; case BPF_PROG_TYPE_PERF_EVENT: - case BPF_PROG_TYPE_KPROBE: case BPF_PROG_TYPE_TRACEPOINT: if (attr->link_create.attach_type != BPF_PERF_EVENT) { ret = -EINVAL; @@ -4280,6 +4507,14 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) } ptype = prog->type; break; + case BPF_PROG_TYPE_KPROBE: + if (attr->link_create.attach_type != BPF_PERF_EVENT && + attr->link_create.attach_type != BPF_TRACE_FPROBE) { + ret = -EINVAL; + goto out; + } + ptype = prog->type; + break; default: ptype = attach_type_to_prog_type(attr->link_create.attach_type); if (ptype == BPF_PROG_TYPE_UNSPEC || ptype != prog->type) { @@ -4311,13 +4546,16 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) ret = bpf_xdp_link_attach(attr, prog); break; #endif -#ifdef CONFIG_PERF_EVENTS case BPF_PROG_TYPE_PERF_EVENT: case BPF_PROG_TYPE_TRACEPOINT: - case BPF_PROG_TYPE_KPROBE: ret = bpf_perf_link_attach(attr, prog); break; -#endif + case BPF_PROG_TYPE_KPROBE: + if (attr->link_create.attach_type == BPF_PERF_EVENT) + ret = bpf_perf_link_attach(attr, prog); + else + ret = bpf_fprobe_link_attach(attr, prog); + break; default: ret = -EINVAL; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index a7f0ddedac1f..c0912f0a3dfe 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -997,6 +997,7 @@ enum bpf_attach_type { BPF_SK_REUSEPORT_SELECT, BPF_SK_REUSEPORT_SELECT_OR_MIGRATE, BPF_PERF_EVENT, + BPF_TRACE_FPROBE, __MAX_BPF_ATTACH_TYPE }; @@ -1011,6 +1012,7 @@ enum bpf_link_type { BPF_LINK_TYPE_NETNS = 5, BPF_LINK_TYPE_XDP = 6, BPF_LINK_TYPE_PERF_EVENT = 7, + BPF_LINK_TYPE_FPROBE = 8, MAX_BPF_LINK_TYPE, }; @@ -1118,6 +1120,11 @@ enum bpf_link_type { */ #define BPF_F_XDP_HAS_FRAGS (1U << 5) +/* link_create.fprobe.flags used in LINK_CREATE command for + * BPF_TRACE_FPROBE attach type to create return probe. + */ +#define BPF_F_FPROBE_RETURN (1U << 0) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * @@ -1472,6 +1479,12 @@ union bpf_attr { */ __u64 bpf_cookie; } perf_event; + struct { + __aligned_u64 syms; + __aligned_u64 addrs; + __u32 cnt; + __u32 flags; + } fprobe; }; } link_create; From patchwork Wed Feb 2 13:53:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732935 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1359CC433FE for ; Wed, 2 Feb 2022 13:53:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344556AbiBBNxx (ORCPT ); Wed, 2 Feb 2022 08:53:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24964 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344585AbiBBNxu (ORCPT ); Wed, 2 Feb 2022 08:53:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810029; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rU7PA92V+49dsAGPuIeIV/v8wUH1DOur5mrpoKB+KuE=; b=cgTxAVNu0LLLKKOb9cjJx4NK9B5frmjjS1H/jdtHijQfEgTbhFyUOr0ZTx8Qoy9mBQlDx0 ZIj+P3NlPa9nampVnLrh3aBVzzwOEOEoqpvn5FsR+tnJLktuuKkYYYGyyyQlET3JRFCgwq aEptarO9NIbs/hSa8E/rciNYY4EPfhY= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-482-ncnZ5SREN_GLuFwXqpIJYQ-1; Wed, 02 Feb 2022 08:53:48 -0500 X-MC-Unique: ncnZ5SREN_GLuFwXqpIJYQ-1 Received: by mail-ed1-f69.google.com with SMTP id p17-20020aa7c891000000b004052d1936a5so10443568eds.7 for ; Wed, 02 Feb 2022 05:53:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rU7PA92V+49dsAGPuIeIV/v8wUH1DOur5mrpoKB+KuE=; b=scA9UXzNGaQWwPK9hPNEOhrVETOYL6YrFZWO6dSGeFwrkCLMVgLJRZOM7l0Hd1XTM5 PUfdkW8s+fmfyFdLX1uSBMMg2+eQJsgwuYg2VWBWl1H0qnqfJ5iu/+XBT2VNY35/xu7s xDJxzHn4BOU2dNJaAcfCjWxCnBlnk9zbOL3cEXZOVntc5DGaXGgORU479cqqn/YvleT7 ger6pes+tqjGaiqbceKboyeO/1BJ1CqwCxShXjMHxxp1w2w3gPkewob1xqrRjSb1YPUR EAMkJBGMtPVns4Q0XVbD+62pWD261tVrnzMMY3WX+Myz6PsqAmRvA9T9i+hw9P7rLFL6 qQtQ== X-Gm-Message-State: AOAM531iMzje1Xp6i91nWCqjJv5FsxeaFm4lY6cHFR/Ch/I7Xs18DdQX L+n9b7vtITTv4oB1CgwGthcxslczc5kGvEEemYdPtv1jHT0hX7bdb38S3rQUnZhqPllkWq/RP9h Js289qpoCeJd/AERc X-Received: by 2002:a17:907:2ce1:: with SMTP id hz1mr19339598ejc.681.1643810027517; Wed, 02 Feb 2022 05:53:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJwwQirmhCLHvQ2UdBTeDbmq1tdFdGGrZl5m4jins0KciHUjrD9AYG1LHNgFrDoNGbtQI+UUJg== X-Received: by 2002:a17:907:2ce1:: with SMTP id hz1mr19339588ejc.681.1643810027285; Wed, 02 Feb 2022 05:53:47 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id by4sm2868404edb.107.2022.02.02.05.53.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:53:46 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 2/8] bpf: Add bpf_get_func_ip kprobe helper for fprobe link Date: Wed, 2 Feb 2022 14:53:27 +0100 Message-Id: <20220202135333.190761-3-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding support to call get_func_ip_fprobe helper from kprobe programs attached by fprobe link. Also adding support to inline it, because it's single load instruction. Signed-off-by: Jiri Olsa --- kernel/bpf/verifier.c | 19 ++++++++++++++++++- kernel/trace/bpf_trace.c | 16 +++++++++++++++- 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1ae41d0cf96c..a745ded00635 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13625,7 +13625,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } - /* Implement bpf_get_func_ip inline. */ + /* Implement tracing bpf_get_func_ip inline. */ if (prog_type == BPF_PROG_TYPE_TRACING && insn->imm == BPF_FUNC_get_func_ip) { /* Load IP address from ctx - 16 */ @@ -13640,6 +13640,23 @@ static int do_misc_fixups(struct bpf_verifier_env *env) continue; } + /* Implement kprobe/fprobe bpf_get_func_ip inline. */ + if (prog_type == BPF_PROG_TYPE_KPROBE && + eatype == BPF_TRACE_FPROBE && + insn->imm == BPF_FUNC_get_func_ip) { + /* Load IP address from ctx (struct pt_regs) ip */ + insn_buf[0] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct pt_regs, ip)); + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, 1); + if (!new_prog) + return -ENOMEM; + + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; + continue; + } + patch_call_imm: fn = env->ops->get_func_proto(insn->imm, env->prog); /* all functions that have prototype and verifier allowed diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index a2024ba32a20..28e59e31e3db 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1036,6 +1036,19 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_kprobe = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_func_ip_fprobe, struct pt_regs *, regs) +{ + /* This helper call is inlined by verifier. */ + return regs->ip; +} + +static const struct bpf_func_proto bpf_get_func_ip_proto_fprobe = { + .func = bpf_get_func_ip_fprobe, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx) { struct bpf_trace_run_ctx *run_ctx; @@ -1279,7 +1292,8 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_override_return_proto; #endif case BPF_FUNC_get_func_ip: - return &bpf_get_func_ip_proto_kprobe; + return prog->expected_attach_type == BPF_TRACE_FPROBE ? + &bpf_get_func_ip_proto_fprobe : &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: return &bpf_get_attach_cookie_proto_trace; default: From patchwork Wed Feb 2 13:53:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732936 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79921C433EF for ; Wed, 2 Feb 2022 13:54:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344656AbiBBNyC (ORCPT ); Wed, 2 Feb 2022 08:54:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:37536 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344613AbiBBNx5 (ORCPT ); Wed, 2 Feb 2022 08:53:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nmdcIiImDczn6f3CR2CX3Iu2tcVi06TPYmGxM9JxMVg=; b=M8xO5tsYTUxBrXvqfdMjSepHHPwUF6f6/l9HPkObmbKLi+JgdaHCbOIAHH4d9Zj9Qt7e/5 eaMcC7d37pjeCDTyTl79NdAzb0PHzMQtLY/WdePI/eyYt7CnmMsV+AVVmO4vK7Wy1x9xb5 98oPwozvzm0hlJ6Am7latphK+E2ovpQ= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-83-zcUzQwcfMcuqUCsz55ulaw-1; Wed, 02 Feb 2022 08:53:55 -0500 X-MC-Unique: zcUzQwcfMcuqUCsz55ulaw-1 Received: by mail-ed1-f72.google.com with SMTP id l14-20020aa7cace000000b003f7f8e1cbbdso10415109edt.20 for ; Wed, 02 Feb 2022 05:53:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nmdcIiImDczn6f3CR2CX3Iu2tcVi06TPYmGxM9JxMVg=; b=juZexaqW0ruo9RWtLbpIjZ6wmd08c3EMv9cbfaOZ3Z2M2GAUAzw6r5db+5VEKyPjV0 sGtZLJ61U68uq1fgn16MyzLMo3XuCaVpwKKOyv38M0U4lpmMxms6Aq94N5GiQ3JflwW0 cQorVB9gOBHRQMAD/Ql1lIfcF++SEKts+42hCN02/v63OSLBPoElbAP/MALYJ3LIN+Yx fRe9i8IFpvHSkPnmeyHO9Pcc9rTSz8uwK/jvq5i9sXrvCiuA8ikERDoEwUgvpQj/Di/I qI9gkpuWy3gsza967xRrn1sKZ9PaNqXjFfQ+5qfyL5OWDfXRtXQ/nPxlfsMOuGU2nfIU ajTA== X-Gm-Message-State: AOAM531DxpfV829h878l43wvvBO0SXgEm5xNW7pvlcSwOudxPsK6pN4r Z5pLmaMPwKdri5MYByNtWRs59WbCjMxr11wQBphaTVkYngKvMvj2x2GbQJgJHHYMX54bycu9WJg rXr0lwV3yh5VoduY/ X-Received: by 2002:a17:906:604d:: with SMTP id p13mr24728338ejj.639.1643810033734; Wed, 02 Feb 2022 05:53:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJyBR9DR4TppVEV3M7QqQMxwZJh36m2BbKzoYg1EmmV3Cx7Lz1/QJk2/Gf1EUqW/sQJhcqZWZA== X-Received: by 2002:a17:906:604d:: with SMTP id p13mr24728329ejj.639.1643810033497; Wed, 02 Feb 2022 05:53:53 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id ce20sm3260162ejb.169.2022.02.02.05.53.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:53:53 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 3/8] bpf: Add bpf_cookie support to fprobe Date: Wed, 2 Feb 2022 14:53:28 +0100 Message-Id: <20220202135333.190761-4-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding support to call bpf_get_attach_cookie helper from kprobe program attached by fprobe link. The bpf_cookie is provided by array of u64 values, where each value is paired with provided function address with the same array index. Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/linux/bpf.h | 2 + include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 83 +++++++++++++++++++++++++++++++++- kernel/trace/bpf_trace.c | 16 ++++++- tools/include/uapi/linux/bpf.h | 1 + 5 files changed, 100 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 6eb0b180d33b..7b65f05c0487 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1301,6 +1301,8 @@ static inline void bpf_reset_run_ctx(struct bpf_run_ctx *old_ctx) #endif } +u64 bpf_fprobe_cookie(struct bpf_run_ctx *ctx, u64 ip); + /* BPF program asks to bypass CAP_NET_BIND_SERVICE in bind. */ #define BPF_RET_BIND_NO_CAP_NET_BIND_SERVICE (1 << 0) /* BPF program asks to set CN on the packet. */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index c0912f0a3dfe..0dc6aa4f9683 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1484,6 +1484,7 @@ union bpf_attr { __aligned_u64 addrs; __u32 cnt; __u32 flags; + __aligned_u64 bpf_cookies; } fprobe; }; } link_create; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 0cfbb112c8e1..6c5e74bc43b6 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -33,6 +33,8 @@ #include #include #include +#include +#include #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -3025,10 +3027,18 @@ static int bpf_perf_link_attach(const union bpf_attr *attr, struct bpf_prog *pro #ifdef CONFIG_FPROBE +struct bpf_fprobe_cookie { + unsigned long addr; + u64 bpf_cookie; +}; + struct bpf_fprobe_link { struct bpf_link link; struct fprobe fp; unsigned long *addrs; + struct bpf_run_ctx run_ctx; + struct bpf_fprobe_cookie *bpf_cookies; + u32 cnt; }; static void bpf_fprobe_link_release(struct bpf_link *link) @@ -3045,6 +3055,7 @@ static void bpf_fprobe_link_dealloc(struct bpf_link *link) fprobe_link = container_of(link, struct bpf_fprobe_link, link); kfree(fprobe_link->addrs); + kfree(fprobe_link->bpf_cookies); kfree(fprobe_link); } @@ -3053,9 +3064,37 @@ static const struct bpf_link_ops bpf_fprobe_link_lops = { .dealloc = bpf_fprobe_link_dealloc, }; +static int bpf_fprobe_cookie_cmp(const void *_a, const void *_b) +{ + const struct bpf_fprobe_cookie *a = _a; + const struct bpf_fprobe_cookie *b = _b; + + if (a->addr == b->addr) + return 0; + return a->addr < b->addr ? -1 : 1; +} + +u64 bpf_fprobe_cookie(struct bpf_run_ctx *ctx, u64 ip) +{ + struct bpf_fprobe_link *fprobe_link; + struct bpf_fprobe_cookie *val, key = { + .addr = (unsigned long) ip, + }; + + if (!ctx) + return 0; + fprobe_link = container_of(ctx, struct bpf_fprobe_link, run_ctx); + if (!fprobe_link->bpf_cookies) + return 0; + val = bsearch(&key, fprobe_link->bpf_cookies, fprobe_link->cnt, + sizeof(key), bpf_fprobe_cookie_cmp); + return val ? val->bpf_cookie : 0; +} + static int fprobe_link_prog_run(struct bpf_fprobe_link *fprobe_link, struct pt_regs *regs) { + struct bpf_run_ctx *old_run_ctx; int err; if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) { @@ -3063,12 +3102,16 @@ static int fprobe_link_prog_run(struct bpf_fprobe_link *fprobe_link, goto out; } + old_run_ctx = bpf_set_run_ctx(&fprobe_link->run_ctx); + rcu_read_lock(); migrate_disable(); err = bpf_prog_run(fprobe_link->link.prog, regs); migrate_enable(); rcu_read_unlock(); + bpf_reset_run_ctx(old_run_ctx); + out: __this_cpu_dec(bpf_prog_active); return err; @@ -3161,10 +3204,12 @@ static int fprobe_resolve_syms(const void *usyms, u32 cnt, static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { + struct bpf_fprobe_cookie *bpf_cookies = NULL; struct bpf_fprobe_link *link = NULL; struct bpf_link_primer link_primer; + void __user *ubpf_cookies; + u32 flags, cnt, i, size; unsigned long *addrs; - u32 flags, cnt, size; void __user *uaddrs; void __user *usyms; int err; @@ -3205,6 +3250,37 @@ static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *p goto error; } + ubpf_cookies = u64_to_user_ptr(attr->link_create.fprobe.bpf_cookies); + if (ubpf_cookies) { + u64 *tmp; + + err = -ENOMEM; + tmp = kzalloc(size, GFP_KERNEL); + if (!tmp) + goto error; + + if (copy_from_user(tmp, ubpf_cookies, size)) { + kfree(tmp); + err = -EFAULT; + goto error; + } + + size = cnt * sizeof(*bpf_cookies); + bpf_cookies = kzalloc(size, GFP_KERNEL); + if (!bpf_cookies) { + kfree(tmp); + goto error; + } + + for (i = 0; i < cnt; i++) { + bpf_cookies[i].addr = addrs[i]; + bpf_cookies[i].bpf_cookie = tmp[i]; + } + + sort(bpf_cookies, cnt, sizeof(*bpf_cookies), bpf_fprobe_cookie_cmp, NULL); + kfree(tmp); + } + link = kzalloc(sizeof(*link), GFP_KERNEL); if (!link) { err = -ENOMEM; @@ -3224,6 +3300,8 @@ static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *p link->fp.entry_handler = fprobe_link_entry_handler; link->addrs = addrs; + link->bpf_cookies = bpf_cookies; + link->cnt = cnt; err = register_fprobe_ips(&link->fp, addrs, cnt); if (err) { @@ -3236,6 +3314,7 @@ static int bpf_fprobe_link_attach(const union bpf_attr *attr, struct bpf_prog *p error: kfree(link); kfree(addrs); + kfree(bpf_cookies); return err; } #else /* !CONFIG_FPROBE */ @@ -4476,7 +4555,7 @@ static int tracing_bpf_link_attach(const union bpf_attr *attr, bpfptr_t uattr, return -EINVAL; } -#define BPF_LINK_CREATE_LAST_FIELD link_create.fprobe.flags +#define BPF_LINK_CREATE_LAST_FIELD link_create.fprobe.bpf_cookies static int link_create(union bpf_attr *attr, bpfptr_t uattr) { enum bpf_prog_type ptype; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 28e59e31e3db..b54b2ef93928 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1049,6 +1049,18 @@ static const struct bpf_func_proto bpf_get_func_ip_proto_fprobe = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_attach_cookie_fprobe, struct pt_regs *, regs) +{ + return bpf_fprobe_cookie(current->bpf_ctx, regs->ip); +} + +static const struct bpf_func_proto bpf_get_attach_cookie_proto_fprobe = { + .func = bpf_get_attach_cookie_fprobe, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, +}; + BPF_CALL_1(bpf_get_attach_cookie_trace, void *, ctx) { struct bpf_trace_run_ctx *run_ctx; @@ -1295,7 +1307,9 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return prog->expected_attach_type == BPF_TRACE_FPROBE ? &bpf_get_func_ip_proto_fprobe : &bpf_get_func_ip_proto_kprobe; case BPF_FUNC_get_attach_cookie: - return &bpf_get_attach_cookie_proto_trace; + return prog->expected_attach_type == BPF_TRACE_FPROBE ? + &bpf_get_attach_cookie_proto_fprobe : + &bpf_get_attach_cookie_proto_trace; default: return bpf_tracing_func_proto(func_id, prog); } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index c0912f0a3dfe..0dc6aa4f9683 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1484,6 +1484,7 @@ union bpf_attr { __aligned_u64 addrs; __u32 cnt; __u32 flags; + __aligned_u64 bpf_cookies; } fprobe; }; } link_create; From patchwork Wed Feb 2 13:53:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732937 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49B33C433F5 for ; Wed, 2 Feb 2022 13:54:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344603AbiBBNyF (ORCPT ); Wed, 2 Feb 2022 08:54:05 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39096 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344657AbiBBNyC (ORCPT ); Wed, 2 Feb 2022 08:54:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jBnKFQ30kOdAO0O49lD07YWI/2GEebyH6rvvven/zqI=; b=eMzvODrZA7c8Dm2KBjB8lx68J5wVJneIGfPr/pPvskFQN7sdS9xwNGg4uUiaNMsCCqAyqw 321hy5MO+bMNEBGtjlv4VVWGcTTe76CaLN9sN7/iViPufDukiddWDQdpQJsEx0R/910p7T m1Oh8+BBlAc23ipXrlx0ReHdoTzSGPY= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-672-q_05NFpMNWOyAh7nVC-BXQ-1; Wed, 02 Feb 2022 08:54:01 -0500 X-MC-Unique: q_05NFpMNWOyAh7nVC-BXQ-1 Received: by mail-ed1-f69.google.com with SMTP id l14-20020aa7cace000000b003f7f8e1cbbdso10415243edt.20 for ; Wed, 02 Feb 2022 05:54:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jBnKFQ30kOdAO0O49lD07YWI/2GEebyH6rvvven/zqI=; b=AaVisnFYaFe2FJOV2ujs8JiTxk9o6F2vDNn6mwABzpvefdvSrtfIpZwhVXk5d1S8/F r/C+KfrNDQzcLrrjSWPv6fvNyf8dQKO5DB5++nbEVQl3VeApacvbdVzLnqCV/jKPDdHw aqROicpPgevAk+aUAXnnRxrlcfgII0ZbuE5iHbqhZN+4jlG4n0+calPO/XNZOXrsN/lk cyIiwaknZXOq+LNuVoexbt+6RL9c0Yb3I8kSV/EzaxEVzheOHvtjhi3YbeLg721pzwNa xbpoy9IVt2efZNTwQ1/BVV3t2x40eOaNl/WM0LfZFlivUrWTBhycyK4vVOdRwev6Fst7 w5Rw== X-Gm-Message-State: AOAM5315p98Nu2HojaBS+CYekvPIHp+9UwkAJGCjhBnlTpCZG4sYa8YG 8kX9Sm5tca1sfg7H+kOXLdq5PpsJZ65NsIP7CT8ADH/k9MufdoGy7c72Tr3lbxyY7ZglwTD5OmG bDuI6YL952Me+NJSy X-Received: by 2002:a50:bb0a:: with SMTP id y10mr8988221ede.441.1643810039920; Wed, 02 Feb 2022 05:53:59 -0800 (PST) X-Google-Smtp-Source: ABdhPJwpHD34n0QXTYfKs783CZ7B9M9fR7e6UeoB6pLhcbQSrQnEbVZssHdo7gWyGcK97/TJlR+epw== X-Received: by 2002:a50:bb0a:: with SMTP id y10mr8988201ede.441.1643810039727; Wed, 02 Feb 2022 05:53:59 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id f4sm15819702ejh.93.2022.02.02.05.53.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:53:59 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 4/8] libbpf: Add libbpf__kallsyms_parse function Date: Wed, 2 Feb 2022 14:53:29 +0100 Message-Id: <20220202135333.190761-5-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Move the kallsyms parsing in internal libbpf__kallsyms_parse function, so it can be used from other places. It will be used in following changes. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++------------- tools/lib/bpf/libbpf_internal.h | 5 +++ 2 files changed, 43 insertions(+), 24 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 1b0936b016d9..7d595cfd03bc 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -7165,12 +7165,10 @@ static int bpf_object__sanitize_maps(struct bpf_object *obj) return 0; } -static int bpf_object__read_kallsyms_file(struct bpf_object *obj) +int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb) { char sym_type, sym_name[500]; unsigned long long sym_addr; - const struct btf_type *t; - struct extern_desc *ext; int ret, err = 0; FILE *f; @@ -7189,35 +7187,51 @@ static int bpf_object__read_kallsyms_file(struct bpf_object *obj) if (ret != 3) { pr_warn("failed to read kallsyms entry: %d\n", ret); err = -EINVAL; - goto out; + break; } - ext = find_extern_by_name(obj, sym_name); - if (!ext || ext->type != EXT_KSYM) - continue; - - t = btf__type_by_id(obj->btf, ext->btf_id); - if (!btf_is_var(t)) - continue; - - if (ext->is_set && ext->ksym.addr != sym_addr) { - pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n", - sym_name, ext->ksym.addr, sym_addr); - err = -EINVAL; - goto out; - } - if (!ext->is_set) { - ext->is_set = true; - ext->ksym.addr = sym_addr; - pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr); - } + err = cb(arg, sym_addr, sym_type, sym_name); + if (err) + break; } -out: fclose(f); return err; } +static int kallsyms_cb(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name) +{ + struct bpf_object *obj = arg; + const struct btf_type *t; + struct extern_desc *ext; + + ext = find_extern_by_name(obj, sym_name); + if (!ext || ext->type != EXT_KSYM) + return 0; + + t = btf__type_by_id(obj->btf, ext->btf_id); + if (!btf_is_var(t)) + return 0; + + if (ext->is_set && ext->ksym.addr != sym_addr) { + pr_warn("extern (ksym) '%s' resolution is ambiguous: 0x%llx or 0x%llx\n", + sym_name, ext->ksym.addr, sym_addr); + return -EINVAL; + } + if (!ext->is_set) { + ext->is_set = true; + ext->ksym.addr = sym_addr; + pr_debug("extern (ksym) %s=0x%llx\n", sym_name, sym_addr); + } + return 0; +} + +static int bpf_object__read_kallsyms_file(struct bpf_object *obj) +{ + return libbpf__kallsyms_parse(obj, kallsyms_cb); +} + static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, __u16 kind, struct btf **res_btf, struct module_btf **res_mod_btf) diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index bc86b82e90d1..fb3b07d401df 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -449,6 +449,11 @@ __s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name, extern enum libbpf_strict_mode libbpf_mode; +typedef int (*kallsyms_cb_t)(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name); + +int libbpf__kallsyms_parse(void *arg, kallsyms_cb_t cb); + /* handle direct returned errors */ static inline int libbpf_err(int ret) { From patchwork Wed Feb 2 13:53:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732939 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86939C4332F for ; Wed, 2 Feb 2022 13:54:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344683AbiBBNyV (ORCPT ); Wed, 2 Feb 2022 08:54:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:54023 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344671AbiBBNyJ (ORCPT ); Wed, 2 Feb 2022 08:54:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810049; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gqIBYhSED2KiLRiPVvQEcn95LWAaRUgov7svs5RNRz4=; b=Hu0l9O7ZHMS+sNZB3UsrrjPCVNpGtHd0VKhxdOYKbyv5Yi0f73V0nmPsetPrx/CKtNmhcT eLIQZxTFSuOO+wbdWs0VimFQQpVUg1i0Ly1oP0QjGN1qXiBX+vfe9Myj0UmItWbB+2BkLO jGnp8Bq7TIgi28atfiSpcmkuaiLDZ70= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-R791E_-2McuGOQM-HBYWUA-1; Wed, 02 Feb 2022 08:54:08 -0500 X-MC-Unique: R791E_-2McuGOQM-HBYWUA-1 Received: by mail-ej1-f72.google.com with SMTP id fx12-20020a170906b74c00b006bbeab42f06so4537249ejb.3 for ; Wed, 02 Feb 2022 05:54:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gqIBYhSED2KiLRiPVvQEcn95LWAaRUgov7svs5RNRz4=; b=YGCq1mEp8eR3//qOujpwEmimfPhgK6l1TjAiEVMARKJAkp/2YaeKjvcMaVno5uYqPe B76r31Ntsb4zgrGfkxfxC9dDggENBi/IY4MeXRscO/epyQD8dBCmfmVrsGE2pd/iV3m3 Xf5c/hoM3SV/4lXCRdyjCErbgj/ExH338F7eKOV3SLnN2FIBjZZ/0SsPgZ0RtwQ7bg33 Cduu/pHHVDn9pFweOI8ixtlnJcscFu9aQk2ASYX3Fgd7C4nNFO9BCoF65i3m9Qlx5MTK B9IK55XgmlH2VAtlq1UHjmBg+z17vtT5QweLA7wBs6wTtROWUftniJ+NXgNnhaNmlTsa duGA== X-Gm-Message-State: AOAM533VVtzhHnAKMXmPH7+SIxOK1LJqGkSdE8tOdO23g/y0SMXAKqfL bz3g0sN69I0soBqqMeev+q41ekxkmS8Th4XY+8NNod5SoMWOM1a40SwPFQkolwwMgZYMUD/EkC3 /C3MBaVTgtx4Ja784 X-Received: by 2002:a05:6402:509:: with SMTP id m9mr29640228edv.237.1643810046247; Wed, 02 Feb 2022 05:54:06 -0800 (PST) X-Google-Smtp-Source: ABdhPJwcUwK13/nY2kcApXDzGsFhV/cZCeMZ9htyubnhs5SDp70e98r81NKwfnOABmoL6EkvsLeXhg== X-Received: by 2002:a05:6402:509:: with SMTP id m9mr29640200edv.237.1643810046059; Wed, 02 Feb 2022 05:54:06 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id s7sm15703501ejo.212.2022.02.02.05.54.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:54:05 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 5/8] libbpf: Add bpf_link_create support for multi kprobes Date: Wed, 2 Feb 2022 14:53:30 +0100 Message-Id: <20220202135333.190761-6-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding new kprobe struct in bpf_link_create_opts object to pass multi kprobe data to link_create attr API. Signed-off-by: Jiri Olsa --- tools/lib/bpf/bpf.c | 7 +++++++ tools/lib/bpf/bpf.h | 9 ++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 418b259166f8..98156709a96c 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -853,6 +853,13 @@ int bpf_link_create(int prog_fd, int target_fd, if (!OPTS_ZEROED(opts, perf_event)) return libbpf_err(-EINVAL); break; + case BPF_TRACE_FPROBE: + attr.link_create.fprobe.syms = OPTS_GET(opts, fprobe.syms, 0); + attr.link_create.fprobe.addrs = OPTS_GET(opts, fprobe.addrs, 0); + attr.link_create.fprobe.cnt = OPTS_GET(opts, fprobe.cnt, 0); + attr.link_create.fprobe.flags = OPTS_GET(opts, fprobe.flags, 0); + attr.link_create.fprobe.bpf_cookies = OPTS_GET(opts, fprobe.bpf_cookies, 0); + break; default: if (!OPTS_ZEROED(opts, flags)) return libbpf_err(-EINVAL); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index c2e8327010f9..114e828ae027 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -413,10 +413,17 @@ struct bpf_link_create_opts { struct { __u64 bpf_cookie; } perf_event; + struct { + __u64 syms; + __u64 addrs; + __u32 cnt; + __u32 flags; + __u64 bpf_cookies; + } fprobe; }; size_t :0; }; -#define bpf_link_create_opts__last_field perf_event +#define bpf_link_create_opts__last_field fprobe.bpf_cookies LIBBPF_API int bpf_link_create(int prog_fd, int target_fd, enum bpf_attach_type attach_type, From patchwork Wed Feb 2 13:53:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732938 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33E94C4332F for ; Wed, 2 Feb 2022 13:54:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344615AbiBBNyR (ORCPT ); Wed, 2 Feb 2022 08:54:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24913 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344666AbiBBNyP (ORCPT ); Wed, 2 Feb 2022 08:54:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810055; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oO9kCxYzu+S91SbPZpT/UZqtqo7+3S7Tz0558nP+sM4=; b=fADYtXwDyZE3T8rGeH59pclR012WdVNgye3r58n3myzKPxo96XfI9aQWN+m9cUHnB0xaEQ CsGMzYn4sn9ZrkMeGlyl0v5O4zj+XYu22Outf9KfTUhM5brhKOFJyUvl5mmZt/Y6AED4yx a75elTzAG/ojJxT1QZmTMsp+cuaM3+U= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-9eHfSAxmO6GnHqawcV04Hg-1; Wed, 02 Feb 2022 08:54:14 -0500 X-MC-Unique: 9eHfSAxmO6GnHqawcV04Hg-1 Received: by mail-ed1-f70.google.com with SMTP id bc24-20020a056402205800b00407cf07b2e0so10480176edb.8 for ; Wed, 02 Feb 2022 05:54:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oO9kCxYzu+S91SbPZpT/UZqtqo7+3S7Tz0558nP+sM4=; b=ZcvlM91glVRB/3c020fdCxA/yJrGkVgcPuCNcdDpgkQTDxzW8127jP+oeh4/OnD3+1 2BH+rhK5zlYtFpcJtQbv9HfUevSgE1tmAq6P16ViAjwzUZqU4HnIHRcrn6H27aOoibs2 P+IJmWxqU/SYSJ/KhhrH1MOUbuBBp5Tq5xhL1ntpqF5Z2Bq1fzqqtrluWtK+Q9YZq20G 55lVOCXlk0uysXItQxpp56eNAEmsxx05HtPUyRGQ8ojehc8nylZumKZL1RlcKdtrdrSf mpihW3dnlLSY236f6nZUhwghtBpWGOT+cDx4yrVEhkjQcvQjCace0Vcp3dXe4qW5vEqr TGuA== X-Gm-Message-State: AOAM532cvdv7tpNQMd1MuE3W8rG3XSYVcPeVtmN+bGdu34b/Lma33MjX Hqldf9oARhM20wuW21WVG16SpS470s7qaV1FQ3vGAYfCPHYXO6YTLMzcYgMI0POH8J3XCjgArd0 96lIXBMprICyIrJtA X-Received: by 2002:a50:ee09:: with SMTP id g9mr1824081eds.387.1643810052945; Wed, 02 Feb 2022 05:54:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJz1Sui85jvHnptZNNYz6ZXNsvi0FZvRDG85OezXcljt811sTnYz4JPS1Z27F+E8W9jeUIvoQg== X-Received: by 2002:a50:ee09:: with SMTP id g9mr1824065eds.387.1643810052726; Wed, 02 Feb 2022 05:54:12 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id by22sm15763502ejb.5.2022.02.02.05.54.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:54:12 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 6/8] libbpf: Add bpf_program__attach_kprobe_opts for multi kprobes Date: Wed, 2 Feb 2022 14:53:31 +0100 Message-Id: <20220202135333.190761-7-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding support to bpf_program__attach_kprobe_opts to load kprobes to multiple functions. If the kprobe program has BPF_TRACE_FPROBE as expected_attach_type it will use the new fprobe link to attach the program. In this case it will use 'func_name' as pattern for functions to attach. Adding also support to use '*' wildcard in 'kprobe/kretprobe' section name by SEC macro, like: SEC("kprobe/bpf_fentry_test*") SEC("kretprobe/bpf_fentry_test*") This will set kprobe's expected_attach_type to BPF_TRACE_FPROBE, and attach it to provided functions pattern. Signed-off-by: Jiri Olsa --- tools/lib/bpf/libbpf.c | 136 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 133 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 7d595cfd03bc..6b343ef77ed8 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -8607,13 +8607,15 @@ static struct bpf_link *attach_trace(const struct bpf_program *prog, long cookie static struct bpf_link *attach_lsm(const struct bpf_program *prog, long cookie); static struct bpf_link *attach_iter(const struct bpf_program *prog, long cookie); +static int init_kprobe(struct bpf_program *prog, long cookie); + static const struct bpf_sec_def section_defs[] = { SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE | SEC_SLOPPY_PFX), SEC_DEF("sk_reuseport/migrate", SK_REUSEPORT, BPF_SK_REUSEPORT_SELECT_OR_MIGRATE, SEC_ATTACHABLE | SEC_SLOPPY_PFX), SEC_DEF("sk_reuseport", SK_REUSEPORT, BPF_SK_REUSEPORT_SELECT, SEC_ATTACHABLE | SEC_SLOPPY_PFX), - SEC_DEF("kprobe/", KPROBE, 0, SEC_NONE, attach_kprobe), + SEC_DEF("kprobe/", KPROBE, 0, SEC_NONE, attach_kprobe, .init_fn = init_kprobe), SEC_DEF("uprobe/", KPROBE, 0, SEC_NONE), - SEC_DEF("kretprobe/", KPROBE, 0, SEC_NONE, attach_kprobe), + SEC_DEF("kretprobe/", KPROBE, 0, SEC_NONE, attach_kprobe, .init_fn = init_kprobe), SEC_DEF("uretprobe/", KPROBE, 0, SEC_NONE), SEC_DEF("tc", SCHED_CLS, 0, SEC_NONE), SEC_DEF("classifier", SCHED_CLS, 0, SEC_NONE | SEC_SLOPPY_PFX | SEC_DEPRECATED), @@ -10031,6 +10033,123 @@ static int perf_event_kprobe_open_legacy(const char *probe_name, bool retprobe, return pfd; } +struct fprobe_resolve { + const char *name; + __u64 *addrs; + __u32 alloc; + __u32 cnt; +}; + +static bool glob_matches(const char *glob, const char *s) +{ + int n = strlen(glob); + + if (n == 1 && glob[0] == '*') + return true; + + if (glob[0] == '*' && glob[n - 1] == '*') { + const char *subs; + /* substring match */ + + /* this is hacky, but we don't want to allocate + * for no good reason + */ + ((char *)glob)[n - 1] = '\0'; + subs = strstr(s, glob + 1); + ((char *)glob)[n - 1] = '*'; + + return subs != NULL; + } else if (glob[0] == '*') { + size_t nn = strlen(s); + /* suffix match */ + + /* too short for a given suffix */ + if (nn < n - 1) + return false; + return strcmp(s + nn - (n - 1), glob + 1) == 0; + } else if (glob[n - 1] == '*') { + /* prefix match */ + return strncmp(s, glob, n - 1) == 0; + } else { + /* exact match */ + return strcmp(glob, s) == 0; + } +} + +static int resolve_fprobe_cb(void *arg, unsigned long long sym_addr, + char sym_type, const char *sym_name) +{ + struct fprobe_resolve *res = arg; + __u64 *p; + + if (!glob_matches(res->name, sym_name)) + return 0; + + if (res->cnt == res->alloc) { + res->alloc = max((__u32) 16, res->alloc * 3 / 2); + p = libbpf_reallocarray(res->addrs, res->alloc, sizeof(__u32)); + if (!p) + return -ENOMEM; + res->addrs = p; + } + res->addrs[res->cnt++] = sym_addr; + return 0; +} + +static struct bpf_link * +attach_fprobe_opts(const struct bpf_program *prog, + const char *func_name, + const struct bpf_kprobe_opts *kopts) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + struct fprobe_resolve res = { + .name = func_name, + }; + struct bpf_link *link = NULL; + char errmsg[STRERR_BUFSIZE]; + int err, link_fd, prog_fd; + bool retprobe; + + err = libbpf__kallsyms_parse(&res, resolve_fprobe_cb); + if (err) + goto error; + if (!res.cnt) { + err = -ENOENT; + goto error; + } + + retprobe = OPTS_GET(kopts, retprobe, false); + + opts.fprobe.addrs = (__u64) res.addrs; + opts.fprobe.cnt = res.cnt; + opts.flags = retprobe ? BPF_F_FPROBE_RETURN : 0; + + link = calloc(1, sizeof(*link)); + if (!link) { + err = -ENOMEM; + goto error; + } + link->detach = &bpf_link__detach_fd; + + prog_fd = bpf_program__fd(prog); + link_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FPROBE, &opts); + if (link_fd < 0) { + err = -errno; + pr_warn("prog '%s': failed to attach to %s: %s\n", + prog->name, res.name, + libbpf_strerror_r(err, errmsg, sizeof(errmsg))); + goto error; + } + link->fd = link_fd; + free(res.addrs); + return link; + +error: + free(link); + free(res.addrs); + return libbpf_err_ptr(err); +} + struct bpf_link * bpf_program__attach_kprobe_opts(const struct bpf_program *prog, const char *func_name, @@ -10047,6 +10166,9 @@ bpf_program__attach_kprobe_opts(const struct bpf_program *prog, if (!OPTS_VALID(opts, bpf_kprobe_opts)) return libbpf_err_ptr(-EINVAL); + if (prog->expected_attach_type == BPF_TRACE_FPROBE) + return attach_fprobe_opts(prog, func_name, opts); + retprobe = OPTS_GET(opts, retprobe, false); offset = OPTS_GET(opts, offset, 0); pe_opts.bpf_cookie = OPTS_GET(opts, bpf_cookie, 0); @@ -10112,6 +10234,14 @@ struct bpf_link *bpf_program__attach_kprobe(const struct bpf_program *prog, return bpf_program__attach_kprobe_opts(prog, func_name, &opts); } +static int init_kprobe(struct bpf_program *prog, long cookie) +{ + /* If we have wildcard, switch to fprobe link. */ + if (strchr(prog->sec_name, '*')) + bpf_program__set_expected_attach_type(prog, BPF_TRACE_FPROBE); + return 0; +} + static struct bpf_link *attach_kprobe(const struct bpf_program *prog, long cookie) { DECLARE_LIBBPF_OPTS(bpf_kprobe_opts, opts); @@ -10127,7 +10257,7 @@ static struct bpf_link *attach_kprobe(const struct bpf_program *prog, long cooki else func_name = prog->sec_name + sizeof("kprobe/") - 1; - n = sscanf(func_name, "%m[a-zA-Z0-9_.]+%li", &func, &offset); + n = sscanf(func_name, "%m[a-zA-Z0-9_.*]+%li", &func, &offset); if (n < 1) { err = -EINVAL; pr_warn("kprobe name is invalid: %s\n", func_name); From patchwork Wed Feb 2 13:53:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732940 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81AA9C433F5 for ; Wed, 2 Feb 2022 13:54:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343550AbiBBNya (ORCPT ); Wed, 2 Feb 2022 08:54:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34439 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344629AbiBBNyV (ORCPT ); Wed, 2 Feb 2022 08:54:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810061; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bLaeMerEotaa1PvsxAUE2R3tXfnbR0hxnx4SsL7NZbg=; b=T5AvYGZjIs9OJrp4dJ2aMexJ8FB4RHMhD63MfGewDrE6m8GARDkPqTywm+q8HbGRY30/D/ TTSn6/Ta1I7XJRa9Sxszg/Bbg51AfyBkf6rGL1UroKtDtka5n2hSS5y7qicXmJhVu0NAfj 35UwfjD1xmII8HZcLkkS3k15R6VOvgY= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-158-WXYkcK46MHiy0DXHUnni-g-1; Wed, 02 Feb 2022 08:54:20 -0500 X-MC-Unique: WXYkcK46MHiy0DXHUnni-g-1 Received: by mail-ej1-f70.google.com with SMTP id ky6-20020a170907778600b0068e4bd99fd1so8155819ejc.15 for ; Wed, 02 Feb 2022 05:54:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bLaeMerEotaa1PvsxAUE2R3tXfnbR0hxnx4SsL7NZbg=; b=uappf1PJPECMHEkXJh1bARzsE3PpFNc/SmJCJmUJ+bVoh1MUDXKIvpZLv0pz+TVqsD I9iQFLdiRpGZR4Ah8tXRunzCufsp1eTmuDItN+dlMCuTu8fjUi/8ghirZ4cqmHfhv6kk Uzj8U2LSAY6xxWVz5w0q3uzgJw2vXyBCLJVho6FYZtkHsF7c7bon/dCtTBT+dNyXGG8l tWIHNMUNiYRq08SsuyTyaP+IL3sh7Zscmk97aV3YupCjEQFYzZm57HQX6p/LsV3HRQwF x2riLRM+3SquEHpEgUIWzOmYLe8sdl1Q8bgj8d4LTKl+SwTXcHmTHnSrONQUmC8nhFMK NLYQ== X-Gm-Message-State: AOAM530n5kOGYSVHGl+8rQ1et0uArraSJ5Fc/6Rs/AxB9DGoSsGCqGhx X88CBeucYSBEXos6JeYSycNK9tBgOOjWvbPi5GmX0qHMY9BHkf6LU/39q5O3vqOQmgBKOP10IYf ik+Pmcfgz7efMPBpy X-Received: by 2002:a17:907:7da8:: with SMTP id oz40mr19493094ejc.328.1643810059319; Wed, 02 Feb 2022 05:54:19 -0800 (PST) X-Google-Smtp-Source: ABdhPJz1VYc7yglvMYwmqUOo2giJKTPpWpi3bTqqc0M8BtqUBkJ+RuokTlGaYir/ixadM+hilTAryg== X-Received: by 2002:a17:907:7da8:: with SMTP id oz40mr19493065ejc.328.1643810059057; Wed, 02 Feb 2022 05:54:19 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id jl17sm16087939ejc.13.2022.02.02.05.54.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:54:18 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 7/8] selftest/bpf: Add fprobe attach test Date: Wed, 2 Feb 2022 14:53:32 +0100 Message-Id: <20220202135333.190761-8-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding kprobe attach test that uses new fprobe interface to attach kprobe program to multiple functions. The test is attaching to bpf_fentry_test* functions and uses single trampoline program bpf_prog_test_run to trigger bpf_fentry_test* functions. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/fprobe_test.c | 117 ++++++++++++++++++ tools/testing/selftests/bpf/progs/fprobe.c | 58 +++++++++ 2 files changed, 175 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/fprobe_test.c create mode 100644 tools/testing/selftests/bpf/progs/fprobe.c diff --git a/tools/testing/selftests/bpf/prog_tests/fprobe_test.c b/tools/testing/selftests/bpf/prog_tests/fprobe_test.c new file mode 100644 index 000000000000..dcbde37ec369 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/fprobe_test.c @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "fprobe.skel.h" +#include "trace_helpers.h" + +static void test_skel_api(void) +{ + struct fprobe *skel = NULL; + __u32 duration = 0, retval; + int err, prog_fd; + + skel = fprobe__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fprobe__open_and_load")) + goto cleanup; + + err = fprobe__attach(skel); + if (!ASSERT_OK(err, "fprobe__attach")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test2_result, 8, "test2_result"); + ASSERT_EQ(skel->bss->test3_result, 8, "test3_result"); + +cleanup: + fprobe__destroy(skel); +} + +static void test_link_api(struct bpf_link_create_opts *opts) +{ + int err, prog_fd, link1_fd = -1, link2_fd = -1; + struct fprobe *skel = NULL; + __u32 duration = 0, retval; + + skel = fprobe__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test2); + link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FPROBE, opts); + if (!ASSERT_GE(link1_fd, 0, "link_fd")) + goto cleanup; + + opts->fprobe.flags = BPF_F_FPROBE_RETURN; + prog_fd = bpf_program__fd(skel->progs.test3); + link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FPROBE, opts); + if (!ASSERT_GE(link2_fd, 0, "link_fd")) + goto cleanup; + + skel->bss->test2_result = 0; + skel->bss->test3_result = 0; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test2_result, 8, "test2_result"); + ASSERT_EQ(skel->bss->test3_result, 8, "test3_result"); + +cleanup: + if (link1_fd != -1) + close(link1_fd); + if (link2_fd != -1) + close(link2_fd); + fprobe__destroy(skel); +} + +static void test_link_api_addrs(void) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + __u64 addrs[8]; + + kallsyms_find("bpf_fentry_test1", &addrs[0]); + kallsyms_find("bpf_fentry_test2", &addrs[1]); + kallsyms_find("bpf_fentry_test3", &addrs[2]); + kallsyms_find("bpf_fentry_test4", &addrs[3]); + kallsyms_find("bpf_fentry_test5", &addrs[4]); + kallsyms_find("bpf_fentry_test6", &addrs[5]); + kallsyms_find("bpf_fentry_test7", &addrs[6]); + kallsyms_find("bpf_fentry_test8", &addrs[7]); + + opts.fprobe.addrs = (__u64) addrs; + opts.fprobe.cnt = 8; + test_link_api(&opts); +} + +static void test_link_api_syms(void) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + const char *syms[8] = { + "bpf_fentry_test1", + "bpf_fentry_test2", + "bpf_fentry_test3", + "bpf_fentry_test4", + "bpf_fentry_test5", + "bpf_fentry_test6", + "bpf_fentry_test7", + "bpf_fentry_test8", + }; + + opts.fprobe.syms = (__u64) syms; + opts.fprobe.cnt = 8; + test_link_api(&opts); +} + +void test_fprobe_test(void) +{ + test_skel_api(); + test_link_api_syms(); + test_link_api_addrs(); +} diff --git a/tools/testing/selftests/bpf/progs/fprobe.c b/tools/testing/selftests/bpf/progs/fprobe.c new file mode 100644 index 000000000000..baf7086203f9 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fprobe.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +extern const void bpf_fentry_test1 __ksym; +extern const void bpf_fentry_test2 __ksym; +extern const void bpf_fentry_test3 __ksym; +extern const void bpf_fentry_test4 __ksym; +extern const void bpf_fentry_test5 __ksym; +extern const void bpf_fentry_test6 __ksym; +extern const void bpf_fentry_test7 __ksym; +extern const void bpf_fentry_test8 __ksym; + +/* No tests, just to trigger bpf_fentry_test* through tracing test_run */ +SEC("fentry/bpf_modify_return_test") +int BPF_PROG(test1) +{ + return 0; +} + +__u64 test2_result = 0; + +SEC("kprobe/bpf_fentry_test*") +int test2(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test2_result += (const void *) addr == &bpf_fentry_test1 || + (const void *) addr == &bpf_fentry_test2 || + (const void *) addr == &bpf_fentry_test3 || + (const void *) addr == &bpf_fentry_test4 || + (const void *) addr == &bpf_fentry_test5 || + (const void *) addr == &bpf_fentry_test6 || + (const void *) addr == &bpf_fentry_test7 || + (const void *) addr == &bpf_fentry_test8; + return 0; +} + +__u64 test3_result = 0; + +SEC("kretprobe/bpf_fentry_test*") +int test3(struct pt_regs *ctx) +{ + __u64 addr = bpf_get_func_ip(ctx); + + test3_result += (const void *) addr == &bpf_fentry_test1 || + (const void *) addr == &bpf_fentry_test2 || + (const void *) addr == &bpf_fentry_test3 || + (const void *) addr == &bpf_fentry_test4 || + (const void *) addr == &bpf_fentry_test5 || + (const void *) addr == &bpf_fentry_test6 || + (const void *) addr == &bpf_fentry_test7 || + (const void *) addr == &bpf_fentry_test8; + return 0; +} From patchwork Wed Feb 2 13:53:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Olsa X-Patchwork-Id: 12732941 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC1BBC433F5 for ; Wed, 2 Feb 2022 13:54:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237853AbiBBNye (ORCPT ); Wed, 2 Feb 2022 08:54:34 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:51528 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344737AbiBBNy2 (ORCPT ); Wed, 2 Feb 2022 08:54:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1643810067; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ndZKvuTAfuQTlyvC6PEP1xi3KVMCTEpQydu3XzbTIRw=; b=B+F3+zJYMQGXEcQEKAbkiBr4Xwk3mI8zsaxPXEudX+YwaZuKxJ2WARBO3SZ1JCrdts7Y9n gw5lghS74xdgUTXd0FJ5E/uQkwPfbbZ4WM3NrqXSnECQWeU3MGXjnDz3n1n94M2LqpRjPL G7mdP9lq5ixEUzif55s1rbQYcHe5nE4= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-563-Emtwb9R5NBK89B5voV1Ngw-1; Wed, 02 Feb 2022 08:54:26 -0500 X-MC-Unique: Emtwb9R5NBK89B5voV1Ngw-1 Received: by mail-ej1-f71.google.com with SMTP id i21-20020a1709063c5500b006b4c7308c19so8155150ejg.14 for ; Wed, 02 Feb 2022 05:54:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ndZKvuTAfuQTlyvC6PEP1xi3KVMCTEpQydu3XzbTIRw=; b=yBLFjg23p1dbijxr03wzKqnPghg5QiJs6vtDiq0CBIFCr0fBTFJqcRjXZ181/EJ2Y/ n7ZSlolIrhhMe1nuMjZYonSaEKUDdNqjGd5CmsBtjq3Wb4DA2vx1GoivFl1vJ/7p1C6X x/ym7+ikLivWV2DE34Us8sRDLhHWQ7wDnQwxNJ+X+W0GDKhTWEkIMNJWbW734r4ECr98 roHviFL0gVF8/yyiI8yTo9ehuVggSbPr/dImp54roNVwKULyYpN0nmB4QZ8YAQy+fusB x6mRxpN8Ysx3g1ShT4OQCMIgBD5XzSrg4WSLZ4ZWCsE/RD89CtIkcmyqC7FZvkyNpV3N jacA== X-Gm-Message-State: AOAM532lb5oibwkAVQE3l0jQzH7YaJQrrNtB4tNAweV/cZVr/GD7hNsU qHdgljBaWsGiZ34+kVmkguVVM8Di1VNgKKgRWhLFaz/KOsymDfmJSGHHgaAJmym9fRpj7ll8YVR 1wj+3FqkVZDowgzee X-Received: by 2002:a05:6402:16cf:: with SMTP id r15mr30306462edx.406.1643810065534; Wed, 02 Feb 2022 05:54:25 -0800 (PST) X-Google-Smtp-Source: ABdhPJy6fmv5Qiwb3TMVnK4uiLl9uYsqBio1+nlTWUQc1NRXW+Y88hbgmFWbH25fj5/DYfJhAAS34w== X-Received: by 2002:a05:6402:16cf:: with SMTP id r15mr30306452edx.406.1643810065387; Wed, 02 Feb 2022 05:54:25 -0800 (PST) Received: from krava.redhat.com (nat-pool-brq-u.redhat.com. [213.175.37.12]) by smtp.gmail.com with ESMTPSA id kw5sm8493321ejc.140.2022.02.02.05.54.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Feb 2022 05:54:24 -0800 (PST) From: Jiri Olsa X-Google-Original-From: Jiri Olsa To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Masami Hiramatsu Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, lkml , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Steven Rostedt , Jiri Olsa Subject: [PATCH 8/8] selftest/bpf: Add fprobe test for bpf_cookie values Date: Wed, 2 Feb 2022 14:53:33 +0100 Message-Id: <20220202135333.190761-9-jolsa@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220202135333.190761-1-jolsa@kernel.org> References: <20220202135333.190761-1-jolsa@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Adding bpf_cookie test for kprobe attached by fprobe link. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/bpf_cookie.c | 73 +++++++++++++++++++ .../selftests/bpf/progs/fprobe_bpf_cookie.c | 62 ++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/fprobe_bpf_cookie.c diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c index cd10df6cd0fc..bf70d859c598 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_cookie.c @@ -7,6 +7,7 @@ #include #include #include "test_bpf_cookie.skel.h" +#include "fprobe_bpf_cookie.skel.h" /* uprobe attach point */ static void trigger_func(void) @@ -63,6 +64,76 @@ static void kprobe_subtest(struct test_bpf_cookie *skel) bpf_link__destroy(retlink2); } +static void fprobe_subtest(void) +{ + DECLARE_LIBBPF_OPTS(bpf_link_create_opts, opts); + int err, prog_fd, link1_fd = -1, link2_fd = -1; + struct fprobe_bpf_cookie *skel = NULL; + __u32 duration = 0, retval; + __u64 addrs[8], cookies[8]; + + skel = fprobe_bpf_cookie__open_and_load(); + if (!ASSERT_OK_PTR(skel, "fentry_raw_skel_load")) + goto cleanup; + + kallsyms_find("bpf_fentry_test1", &addrs[0]); + kallsyms_find("bpf_fentry_test2", &addrs[1]); + kallsyms_find("bpf_fentry_test3", &addrs[2]); + kallsyms_find("bpf_fentry_test4", &addrs[3]); + kallsyms_find("bpf_fentry_test5", &addrs[4]); + kallsyms_find("bpf_fentry_test6", &addrs[5]); + kallsyms_find("bpf_fentry_test7", &addrs[6]); + kallsyms_find("bpf_fentry_test8", &addrs[7]); + + cookies[0] = 1; + cookies[1] = 2; + cookies[2] = 3; + cookies[3] = 4; + cookies[4] = 5; + cookies[5] = 6; + cookies[6] = 7; + cookies[7] = 8; + + opts.fprobe.addrs = (__u64) &addrs; + opts.fprobe.cnt = 8; + opts.fprobe.bpf_cookies = (__u64) &cookies; + prog_fd = bpf_program__fd(skel->progs.test2); + + link1_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FPROBE, &opts); + if (!ASSERT_GE(link1_fd, 0, "link1_fd")) + return; + + cookies[0] = 8; + cookies[1] = 7; + cookies[2] = 6; + cookies[3] = 5; + cookies[4] = 4; + cookies[5] = 3; + cookies[6] = 2; + cookies[7] = 1; + + opts.flags = BPF_F_FPROBE_RETURN; + prog_fd = bpf_program__fd(skel->progs.test3); + + link2_fd = bpf_link_create(prog_fd, 0, BPF_TRACE_FPROBE, &opts); + if (!ASSERT_GE(link2_fd, 0, "link2_fd")) + goto cleanup; + + prog_fd = bpf_program__fd(skel->progs.test1); + err = bpf_prog_test_run(prog_fd, 1, NULL, 0, + NULL, NULL, &retval, &duration); + ASSERT_OK(err, "test_run"); + ASSERT_EQ(retval, 0, "test_run"); + + ASSERT_EQ(skel->bss->test2_result, 8, "test2_result"); + ASSERT_EQ(skel->bss->test3_result, 8, "test3_result"); + +cleanup: + close(link1_fd); + close(link2_fd); + fprobe_bpf_cookie__destroy(skel); +} + static void uprobe_subtest(struct test_bpf_cookie *skel) { DECLARE_LIBBPF_OPTS(bpf_uprobe_opts, opts); @@ -249,6 +320,8 @@ void test_bpf_cookie(void) if (test__start_subtest("kprobe")) kprobe_subtest(skel); + if (test__start_subtest("rawkprobe")) + fprobe_subtest(); if (test__start_subtest("uprobe")) uprobe_subtest(skel); if (test__start_subtest("tracepoint")) diff --git a/tools/testing/selftests/bpf/progs/fprobe_bpf_cookie.c b/tools/testing/selftests/bpf/progs/fprobe_bpf_cookie.c new file mode 100644 index 000000000000..42cb109e5a30 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fprobe_bpf_cookie.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +extern const void bpf_fentry_test1 __ksym; +extern const void bpf_fentry_test2 __ksym; +extern const void bpf_fentry_test3 __ksym; +extern const void bpf_fentry_test4 __ksym; +extern const void bpf_fentry_test5 __ksym; +extern const void bpf_fentry_test6 __ksym; +extern const void bpf_fentry_test7 __ksym; +extern const void bpf_fentry_test8 __ksym; + +/* No tests, just to trigger bpf_fentry_test* through tracing test_run */ +SEC("fentry/bpf_modify_return_test") +int BPF_PROG(test1) +{ + return 0; +} + +__u64 test2_result = 0; + +SEC("kprobe/bpf_fentry_test*") +int test2(struct pt_regs *ctx) +{ + __u64 cookie = bpf_get_attach_cookie(ctx); + __u64 addr = bpf_get_func_ip(ctx); + + test2_result += (const void *) addr == &bpf_fentry_test1 && cookie == 1; + test2_result += (const void *) addr == &bpf_fentry_test2 && cookie == 2; + test2_result += (const void *) addr == &bpf_fentry_test3 && cookie == 3; + test2_result += (const void *) addr == &bpf_fentry_test4 && cookie == 4; + test2_result += (const void *) addr == &bpf_fentry_test5 && cookie == 5; + test2_result += (const void *) addr == &bpf_fentry_test6 && cookie == 6; + test2_result += (const void *) addr == &bpf_fentry_test7 && cookie == 7; + test2_result += (const void *) addr == &bpf_fentry_test8 && cookie == 8; + + return 0; +} + +__u64 test3_result = 0; + +SEC("kretprobe/bpf_fentry_test*") +int test3(struct pt_regs *ctx) +{ + __u64 cookie = bpf_get_attach_cookie(ctx); + __u64 addr = bpf_get_func_ip(ctx); + + test3_result += (const void *) addr == &bpf_fentry_test1 && cookie == 8; + test3_result += (const void *) addr == &bpf_fentry_test2 && cookie == 7; + test3_result += (const void *) addr == &bpf_fentry_test3 && cookie == 6; + test3_result += (const void *) addr == &bpf_fentry_test4 && cookie == 5; + test3_result += (const void *) addr == &bpf_fentry_test5 && cookie == 4; + test3_result += (const void *) addr == &bpf_fentry_test6 && cookie == 3; + test3_result += (const void *) addr == &bpf_fentry_test7 && cookie == 2; + test3_result += (const void *) addr == &bpf_fentry_test8 && cookie == 1; + + return 0; +}