From patchwork Mon Sep 2 06:58:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Malik X-Patchwork-Id: 13786796 X-Patchwork-Delegate: bpf@iogearbox.net Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D4C615688E for ; Mon, 2 Sep 2024 06:58:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260307; cv=none; b=p5fgwFLnOjvuhutci4PdZ4WL6aiSg3xYtC3hzzawRXYefjfnimUY3si6hZhROoQnHK9C1zDIVTdftSMNrIJDkO5JwQF6ZCfwuxWVNL3Q2cQOhgMXFfyYBoxJ4wlY1CzmNRd/w24LOArDOnzObKKHNke+Ijo6ZwaMWmtBHDjlwnw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260307; c=relaxed/simple; bh=I6l3pTB0xuYZaIp3pzq3X6tlwepFRLK+C7LUqzvqLuQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ea58fRsmW3sPOcEj0FCJdEW04MyHUeGETfRhSBZ9LlayZUYDAP5+FvUcdNXvU7xwNy2fsTvRQKYylC+ktSQZqQn9aJB8GQ1fjBAIJGgMbjgfWq1kNYFqL3K+Kauf27ZeZPJu5iDVZyNNFfz+KGf849DjpY6WDTav9tMhd8CbZkw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=A1yraSfq; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A1yraSfq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725260304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5r15w4xESAPeLKzc6LJNAB2Znmgl8v0yRC0d6ROn+Os=; b=A1yraSfq5JqU3LnwuRA+6hQWM8JZ5XlPnBVrPY/ryeUDlPoIUeSRpQLfciHbZJZSPgtl9L 4PvVoorSdhoGi/aZDUJD2dnaggYDn2YlWSO5C/PnII7KoX0Sjo1R9Tbei0QXHD2+QHcZMm PNGkO00glOFFsb1WqOXFT/dkl+6hehA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-581-l6KoKixiMVmtlsJf9uVhNA-1; Mon, 02 Sep 2024 02:58:23 -0400 X-MC-Unique: l6KoKixiMVmtlsJf9uVhNA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0E6B418EA893; Mon, 2 Sep 2024 06:58:21 +0000 (UTC) Received: from vmalik-fedora.redhat.com (unknown [10.45.225.48]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EDB7C19560AE; Mon, 2 Sep 2024 06:58:15 +0000 (UTC) From: Viktor Malik To: bpf@vger.kernel.org Cc: Andrii Nakryiko , Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Viktor Malik Subject: [RFC bpf-next 1/3] libbpf: Support aliased symbols in linker Date: Mon, 2 Sep 2024 08:58:01 +0200 Message-ID: <87e9970b63dede4a19ec62ec572e224eecc26fa3.1725016029.git.vmalik@redhat.com> In-Reply-To: References: Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC It is possible to create multiple BPF programs sharing the same instructions using the compiler `__attribute__((alias("...")))`: int BPF_PROG(prog) { [...] } int prog_alias() __attribute__((alias("prog"))); This may be convenient when creating multiple programs with the same instruction set attached to different events (such as bpftrace does). One problem in this situation is that Clang doesn't generate a BTF entry for `prog_alias` which makes libbpf linker fail when processing such a BPF object. This commits adds support for that by finding another symbol at the same address for which a BTF entry exists and using that entry in the linker. This allows to use the linker (e.g. via `bpftool gen object ...`) on BPF objects containing aliases. Note that this won't be sufficient for most programs as we also need to add support for handling relocations in the aliased programs. This will be added by the following commit. Signed-off-by: Viktor Malik --- tools/lib/bpf/linker.c | 68 +++++++++++++++++++++++------------------- 1 file changed, 38 insertions(+), 30 deletions(-) diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c index 9cd3d4109788..5ebc9ff1246e 100644 --- a/tools/lib/bpf/linker.c +++ b/tools/lib/bpf/linker.c @@ -1688,6 +1688,34 @@ static bool btf_is_non_static(const struct btf_type *t) || (btf_is_func(t) && btf_func_linkage(t) != BTF_FUNC_STATIC); } +static Elf64_Sym *find_sym_by_name(struct src_obj *obj, size_t sec_idx, + int sym_type, const char *sym_name) +{ + struct src_sec *symtab = &obj->secs[obj->symtab_sec_idx]; + Elf64_Sym *sym = symtab->data->d_buf; + int i, n = symtab->shdr->sh_size / symtab->shdr->sh_entsize; + int str_sec_idx = symtab->shdr->sh_link; + const char *name; + + for (i = 0; i < n; i++, sym++) { + if (sym->st_shndx != sec_idx) + continue; + if (ELF64_ST_TYPE(sym->st_info) != sym_type) + continue; + + name = elf_strptr(obj->elf, str_sec_idx, sym->st_name); + if (!name) + return NULL; + + if (strcmp(sym_name, name) != 0) + continue; + + return sym; + } + + return NULL; +} + static int find_glob_sym_btf(struct src_obj *obj, Elf64_Sym *sym, const char *sym_name, int *out_btf_sec_id, int *out_btf_id) { @@ -1695,6 +1723,7 @@ static int find_glob_sym_btf(struct src_obj *obj, Elf64_Sym *sym, const char *sy const struct btf_type *t; const struct btf_var_secinfo *vi; const char *name; + Elf64_Sym *s; if (!obj->btf) { pr_warn("failed to find BTF info for object '%s'\n", obj->filename); @@ -1710,8 +1739,15 @@ static int find_glob_sym_btf(struct src_obj *obj, Elf64_Sym *sym, const char *sy */ if (btf_is_non_static(t)) { name = btf__str_by_offset(obj->btf, t->name_off); - if (strcmp(name, sym_name) != 0) - continue; + if (strcmp(name, sym_name) != 0) { + /* the symbol that we look for may not have BTF as it may + * be an alias of another symbol; we check if this is + * the original symbol and if so, we use its BTF id + */ + s = find_sym_by_name(obj, sym->st_shndx, STT_FUNC, name); + if (!s || s->st_value != sym->st_value) + continue; + } /* remember and still try to find DATASEC */ btf_id = i; @@ -2132,34 +2168,6 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *ob return 0; } -static Elf64_Sym *find_sym_by_name(struct src_obj *obj, size_t sec_idx, - int sym_type, const char *sym_name) -{ - struct src_sec *symtab = &obj->secs[obj->symtab_sec_idx]; - Elf64_Sym *sym = symtab->data->d_buf; - int i, n = symtab->shdr->sh_size / symtab->shdr->sh_entsize; - int str_sec_idx = symtab->shdr->sh_link; - const char *name; - - for (i = 0; i < n; i++, sym++) { - if (sym->st_shndx != sec_idx) - continue; - if (ELF64_ST_TYPE(sym->st_info) != sym_type) - continue; - - name = elf_strptr(obj->elf, str_sec_idx, sym->st_name); - if (!name) - return NULL; - - if (strcmp(sym_name, name) != 0) - continue; - - return sym; - } - - return NULL; -} - static int linker_fixup_btf(struct src_obj *obj) { const char *sec_name; From patchwork Mon Sep 2 06:58:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Malik X-Patchwork-Id: 13786797 X-Patchwork-Delegate: bpf@iogearbox.net Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E22A715688E for ; Mon, 2 Sep 2024 06:58:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260314; cv=none; b=hbdCGjvT2UhDO0WVAbcVtvS/C6LLxuFnm5WK7y8/WqfFulnGLHYGXnBvRI7WJcGChjNI3TaiJIWHIeeLGcECOcuLyMhDaWeH8ZPkdllDNlN25qmkkwCqGB99hShxwBCmOPrmjs8eX44SQ5eOtM855RdG06K5CB5vLZvoM67nUVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260314; c=relaxed/simple; bh=e7piipeOWeW9DwA4UHV3SmhYZmVCbkjfVHMwGvwHuWA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Mme9GMfI6L9viPUMMG9Mwo+JWHtqRm8/B8zmv3e837crqKCzlnEvaMrbbj8MZ/IHsr15HZdo+SAhGQgLP307T73WQAJc+CyUbvuXZNwT/vH+zbhw3aSnl18IUep9p53tUrpj+lCkdro1+17nCes6SYWgj/8/eVRWyoPS0KDKvEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UxTDGCpD; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UxTDGCpD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725260312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n2jPEgl6JDbir6KjTm0KD/i+8R9qKa7QNmbU2dF4eKE=; b=UxTDGCpDaMRa8kb5ZrJsTs7ylpEJczz8Hun4ERE7Ec/0gkyh4dTG/BLxU28lVvRLa/bzx1 ifDn+NoG5bkMLkcM1p2aoMGddT2cgsYcqdv/fLnG60o+asMSpoxLeaWcg9AQukvz9UriGh VT0BDBcx8WJ7lEk+22gtfRCMb52/rFw= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-2bz0r4OxNeKi85WHTOtvhQ-1; Mon, 02 Sep 2024 02:58:27 -0400 X-MC-Unique: 2bz0r4OxNeKi85WHTOtvhQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C06AD19772C2; Mon, 2 Sep 2024 06:58:25 +0000 (UTC) Received: from vmalik-fedora.redhat.com (unknown [10.45.225.48]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 63E16195605A; Mon, 2 Sep 2024 06:58:21 +0000 (UTC) From: Viktor Malik To: bpf@vger.kernel.org Cc: Andrii Nakryiko , Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Viktor Malik Subject: [RFC bpf-next 2/3] libbpf: Handle relocations in aliased symbols Date: Mon, 2 Sep 2024 08:58:02 +0200 Message-ID: <113cf1f68a6e334d0f500d435095a2f61278afcc.1725016029.git.vmalik@redhat.com> In-Reply-To: References: Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC It is possible to create multiple BPF programs sharing the same instructions using the compiler `__attribute__((alias("...")))`: int BPF_PROG(prog) { [...] } int prog_alias() __attribute__((alias("prog"))); The problem is that libbpf currently assumes that there may not be multiple BPF programs sharing the same instruction set in a BPF object and therefore records relocations for a single (arbitrarily chosen) program only. This commit adds libbpf support for aliased programs by recording relocations for all the aliased programs. The core of the change is updating find_prog_by_sec_insn() to return a sequence of programs instead of one and updating its users (where relevant) to perform relocation recording for all of the returned programs. For BPF object not containing aliased symbols, this works exactly as before without additional overhead. Currently, the main user of this feature is bpftrace which allows to attach the same BPF program to many attach points. While this is simple for k(u)probes using k(u)probe_multi, other program types (fentry/fexit, (raw) tracepoints) do not have such features and currently require the BPF object to contain a copy of the program for each attach point. For example, considering the following bpftrace script collecting the number of hits of each VFS function using fentry over a one second period: $ bpftrace -e 'kfunc:vfs_* { @[func] = count() } i:s:1 { exit() }' [...] this change will allow to reduce the size of the in-memory BPF object that bpftrace generates from 60K to 9K. Signed-off-by: Viktor Malik --- tools/lib/bpf/libbpf.c | 143 ++++++++++++++++++++++++----------------- 1 file changed, 84 insertions(+), 59 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index e55353887439..91afacede687 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -4612,14 +4612,15 @@ static bool prog_contains_insn(const struct bpf_program *prog, size_t insn_idx) insn_idx < prog->sec_insn_off + prog->sec_insn_cnt; } -static struct bpf_program *find_prog_by_sec_insn(const struct bpf_object *obj, - size_t sec_idx, size_t insn_idx) +static int find_progs_by_sec_insn(const struct bpf_object *obj, + size_t sec_idx, size_t insn_idx, + struct bpf_program **progs) { - int l = 0, r = obj->nr_programs - 1, m; + int l = 0, r = obj->nr_programs - 1, m, nr_progs = 0; struct bpf_program *prog; if (!obj->nr_programs) - return NULL; + return 0; while (l < r) { m = l + (r - l + 1) / 2; @@ -4631,13 +4632,20 @@ static struct bpf_program *find_prog_by_sec_insn(const struct bpf_object *obj, else r = m - 1; } - /* matching program could be at index l, but it still might be the - * wrong one, so we need to double check conditions for the last time + /* there may be multiple (aliasing) programs sharing the same instructions, + * index l will contain the last one so we search for the first one, + * set *progs to point to it, and return the number of matching programs + * + * note that there may be no matching programs so we may return 0 here */ prog = &obj->programs[l]; - if (prog->sec_idx == sec_idx && prog_contains_insn(prog, insn_idx)) - return prog; - return NULL; + while (l >= 0 && prog->sec_idx == sec_idx && prog_contains_insn(prog, insn_idx)) { + prog = &obj->programs[--l]; + nr_progs++; + } + if (nr_progs) + *progs = &obj->programs[l + 1]; + return nr_progs; } static int @@ -4647,7 +4655,7 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Dat size_t sec_idx = shdr->sh_info, sym_idx; struct bpf_program *prog; struct reloc_desc *relos; - int err, i, nrels; + int err, i, j, nrels, nr_progs; const char *sym_name; __u32 insn_idx; Elf_Scn *scn; @@ -4715,27 +4723,33 @@ bpf_object__collect_prog_relos(struct bpf_object *obj, Elf64_Shdr *shdr, Elf_Dat pr_debug("sec '%s': relo #%d: insn #%u against '%s'\n", relo_sec_name, i, insn_idx, sym_name); - prog = find_prog_by_sec_insn(obj, sec_idx, insn_idx); - if (!prog) { + nr_progs = find_progs_by_sec_insn(obj, sec_idx, insn_idx, &prog); + if (!nr_progs) { pr_debug("sec '%s': relo #%d: couldn't find program in section '%s' for insn #%u, probably overridden weak function, skipping...\n", relo_sec_name, i, sec_name, insn_idx); continue; } - relos = libbpf_reallocarray(prog->reloc_desc, - prog->nr_reloc + 1, sizeof(*relos)); - if (!relos) - return -ENOMEM; - prog->reloc_desc = relos; - - /* adjust insn_idx to local BPF program frame of reference */ + /* adjust insn_idx to local BPF program frame of reference + * we can just do it once as all progs have the same sec_insn_off + */ insn_idx -= prog->sec_insn_off; - err = bpf_program__record_reloc(prog, &relos[prog->nr_reloc], - insn_idx, sym_name, sym, rel); - if (err) - return err; - prog->nr_reloc++; + for (j = 0; j < nr_progs; j++, prog++) { + relos = libbpf_reallocarray(prog->reloc_desc, + prog->nr_reloc + 1, sizeof(*relos)); + if (!relos) + return -ENOMEM; + prog->reloc_desc = relos; + + /* adjust insn_idx to local BPF program frame of reference */ + err = bpf_program__record_reloc(prog, &relos[prog->nr_reloc], + insn_idx, sym_name, sym, rel); + if (err) + return err; + + prog->nr_reloc++; + } } return 0; } @@ -5861,7 +5875,7 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) struct bpf_program *prog; struct bpf_insn *insn; const char *sec_name; - int i, err = 0, insn_idx, sec_idx, sec_num; + int i, j, err = 0, insn_idx, sec_idx, sec_num, nr_progs; if (obj->btf_ext->core_relo_info.len == 0) return 0; @@ -5899,8 +5913,8 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) if (rec->insn_off % BPF_INSN_SZ) return -EINVAL; insn_idx = rec->insn_off / BPF_INSN_SZ; - prog = find_prog_by_sec_insn(obj, sec_idx, insn_idx); - if (!prog) { + nr_progs = find_progs_by_sec_insn(obj, sec_idx, insn_idx, &prog); + if (!nr_progs) { /* When __weak subprog is "overridden" by another instance * of the subprog from a different object file, linker still * appends all the .BTF.ext info that used to belong to that @@ -5913,43 +5927,48 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path) sec_name, i, insn_idx); continue; } - /* no need to apply CO-RE relocation if the program is - * not going to be loaded - */ - if (!prog->autoload) - continue; /* adjust insn_idx from section frame of reference to the local * program's frame of reference; (sub-)program code is not yet - * relocated, so it's enough to just subtract in-section offset + * relocated, so it's enough to just subtract in-section offset; + * we can just do it once as all progs have the same sec_insn_off */ insn_idx = insn_idx - prog->sec_insn_off; if (insn_idx >= prog->insns_cnt) return -EINVAL; - insn = &prog->insns[insn_idx]; - err = record_relo_core(prog, rec, insn_idx); - if (err) { - pr_warn("prog '%s': relo #%d: failed to record relocation: %d\n", - prog->name, i, err); - goto out; - } + for (j = 0; j < nr_progs; j++, prog++) { + /* no need to apply CO-RE relocation if the program is + * not going to be loaded + */ + if (!prog->autoload) + continue; - if (prog->obj->gen_loader) - continue; + insn = &prog->insns[insn_idx]; - err = bpf_core_resolve_relo(prog, rec, i, obj->btf, cand_cache, &targ_res); - if (err) { - pr_warn("prog '%s': relo #%d: failed to relocate: %d\n", - prog->name, i, err); - goto out; - } + err = record_relo_core(prog, rec, insn_idx); + if (err) { + pr_warn("prog '%s': relo #%d: failed to record relocation: %d\n", + prog->name, i, err); + goto out; + } - err = bpf_core_patch_insn(prog->name, insn, insn_idx, rec, i, &targ_res); - if (err) { - pr_warn("prog '%s': relo #%d: failed to patch insn #%u: %d\n", - prog->name, i, insn_idx, err); - goto out; + if (prog->obj->gen_loader) + continue; + + err = bpf_core_resolve_relo(prog, rec, i, obj->btf, cand_cache, &targ_res); + if (err) { + pr_warn("prog '%s': relo #%d: failed to relocate: %d\n", + prog->name, i, err); + goto out; + } + + err = bpf_core_patch_insn(prog->name, insn, insn_idx, rec, i, &targ_res); + if (err) { + pr_warn("prog '%s': relo #%d: failed to patch insn #%u: %d\n", + prog->name, i, insn_idx, err); + goto out; + } } } } @@ -6350,7 +6369,7 @@ bpf_object__reloc_code(struct bpf_object *obj, struct bpf_program *main_prog, struct bpf_program *subprog; struct reloc_desc *relo; struct bpf_insn *insn; - int err; + int err, nr_subprogs; err = reloc_prog_func_and_line_info(obj, main_prog, prog); if (err) @@ -6406,12 +6425,15 @@ bpf_object__reloc_code(struct bpf_object *obj, struct bpf_program *main_prog, } /* we enforce that sub-programs should be in .text section */ - subprog = find_prog_by_sec_insn(obj, obj->efile.text_shndx, sub_insn_idx); - if (!subprog) { + nr_subprogs = find_progs_by_sec_insn(obj, obj->efile.text_shndx, sub_insn_idx, &subprog); + if (!nr_subprogs) { pr_warn("prog '%s': no .text section found yet sub-program call exists\n", prog->name); return -LIBBPF_ERRNO__RELOC; } + /* there may be multiple aliasing subprogs but it doesn't matter + * which one we use here as they must all have the same insns + */ /* if it's the first call instruction calling into this * subprogram (meaning this subprog hasn't been processed @@ -9725,7 +9747,7 @@ static int bpf_object__collect_st_ops_relos(struct bpf_object *obj, __u32 member_idx; Elf64_Sym *sym; Elf64_Rel *rel; - int i, nrels; + int i, nrels, nr_progs; btf = obj->btf; nrels = shdr->sh_size / shdr->sh_entsize; @@ -9789,12 +9811,15 @@ static int bpf_object__collect_st_ops_relos(struct bpf_object *obj, return -EINVAL; } - prog = find_prog_by_sec_insn(obj, shdr_idx, insn_idx); - if (!prog) { + nr_progs = find_progs_by_sec_insn(obj, shdr_idx, insn_idx, &prog); + if (!nr_progs) { pr_warn("struct_ops reloc %s: cannot find prog at shdr_idx %u to relocate func ptr %s\n", map->name, shdr_idx, name); return -EINVAL; } + /* there may be multiple aliasing progs but it doesn't matter + * which one we use here as they must all have the same insns + */ /* prevent the use of BPF prog with invalid type */ if (prog->type != BPF_PROG_TYPE_STRUCT_OPS) { From patchwork Mon Sep 2 06:58:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Malik X-Patchwork-Id: 13786798 X-Patchwork-Delegate: bpf@iogearbox.net Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E68EC15B0FE for ; Mon, 2 Sep 2024 06:58:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260318; cv=none; b=fcXUbO9l3uqKA+J4SYAwmaqoXbQPx/FPTnFw+P/hxoknuZtn/sHOFe+YWV8wdeC6n5kDzFZqjfqYVypszsEiBcG+XDW9e1sgqAplrFIURC6zch6j874aGFBfMdYIB4hrUtsuLDMOlCBKcRAKjHvDOkk4KAEm9mKMS/PP8ZtO+eU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725260318; c=relaxed/simple; bh=aKmShiE9OA1pMxhe+iNcsmE5IpCEX/+T4K3cwEW7a94=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gzzqD4J5Hv2bMRaKG7WClwk8LpJ1MhryoNS/LC9AEmUHIJt1SY8rw/9lT8MP/YjaQ6kHEfIHam3gtQyQ+Gmz5cctRvbwFUvqtqMBU9HjPwv4fhC7J2ODsoUUh4cMr0AduNkal9x68QjTi+wlXWwlCaVEURH77NrEg4Mfjnd0IN8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ZCrNfGqw; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZCrNfGqw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725260315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LFHU9WopqFuOOI3SIwujmRQA94qsaoktB/l3biNMxk8=; b=ZCrNfGqw6f+y/C5txOQi0q9gSqNpzqtHBd1yEcqLtHb30lfbXpjky9FA6l/LADoO+Damdc gXaIfalnjAv9vrjPfsN/yLbYN2pLBUcrCGLGjxk9QTASuNOj2Jp2axi/CFL7lc8amT12AE DJwInK2Z2H2H3mvVv8DcIvnFK8x/lBg= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-8-75Nqns5pNMS43L8RM3EKrA-1; Mon, 02 Sep 2024 02:58:33 -0400 X-MC-Unique: 75Nqns5pNMS43L8RM3EKrA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C30DE1955BF8; Mon, 2 Sep 2024 06:58:30 +0000 (UTC) Received: from vmalik-fedora.redhat.com (unknown [10.45.225.48]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3B0B519560A3; Mon, 2 Sep 2024 06:58:26 +0000 (UTC) From: Viktor Malik To: bpf@vger.kernel.org Cc: Andrii Nakryiko , Eduard Zingerman , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Viktor Malik Subject: [RFC bpf-next 3/3] selftests/bpf: Add tests for aliased programs Date: Mon, 2 Sep 2024 08:58:03 +0200 Message-ID: <6351e73916133f35582f100eb09828cbac6b0f68.1725016029.git.vmalik@redhat.com> In-Reply-To: References: Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC The previous two commits added libbpf support for BPF objects containing multiple programs sharing the same instructions using the compiler `__attribute__((alias("...")))`. This adds tests for such objects. The tests check that different kinds of relocations (namely for global vars, maps, subprogs, and CO-RE) are correctly performed in both the original and the aliasing BPF program and that both programs hit when the target event occurs. Signed-off-by: Viktor Malik --- .../selftests/bpf/prog_tests/fentry_alias.c | 41 +++++++++ .../selftests/bpf/progs/fentry_alias.c | 83 +++++++++++++++++++ 2 files changed, 124 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/fentry_alias.c create mode 100644 tools/testing/selftests/bpf/progs/fentry_alias.c diff --git a/tools/testing/selftests/bpf/prog_tests/fentry_alias.c b/tools/testing/selftests/bpf/prog_tests/fentry_alias.c new file mode 100644 index 000000000000..6b9c6895b374 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/fentry_alias.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include "fentry_alias.skel.h" + +void test_fentry_alias(void) +{ + struct fentry_alias *fentry_skel = NULL; + int err, prog_fd; + __u64 key = 0, val; + + LIBBPF_OPTS(bpf_test_run_opts, opts); + + fentry_skel = fentry_alias__open_and_load(); + if (!ASSERT_OK_PTR(fentry_skel, "fentry_skel_load")) + goto cleanup; + + err = fentry_alias__attach(fentry_skel); + if (!ASSERT_OK(err, "fentry_attach")) + goto cleanup; + + fentry_skel->bss->real_pid = getpid(); + + prog_fd = bpf_program__fd(fentry_skel->progs.test1); + err = bpf_prog_test_run_opts(prog_fd, &opts); + + ASSERT_EQ(fentry_skel->bss->test1_hit_cnt, 2, + "fentry_alias_test1_result"); + + err = bpf_map__lookup_elem(fentry_skel->maps.map, &key, sizeof(key), + &val, sizeof(val), 0); + ASSERT_OK(err, "hashmap lookup"); + ASSERT_EQ(val, 2, "fentry_alias_test2_result"); + + ASSERT_EQ(fentry_skel->bss->test3_hit_cnt, 2, + "fentry_alias_test3_result"); + + ASSERT_EQ(fentry_skel->bss->test4_hit_cnt, 2, + "fentry_alias_test4_result"); +cleanup: + fentry_alias__destroy(fentry_skel); +} diff --git a/tools/testing/selftests/bpf/progs/fentry_alias.c b/tools/testing/selftests/bpf/progs/fentry_alias.c new file mode 100644 index 000000000000..5d7b492c7f95 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/fentry_alias.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +struct task_struct { + int tgid; +} __attribute__((preserve_access_index)); + +int real_pid = 0; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 1); + __type(key, __u64); + __type(value, __u64); +} map SEC(".maps"); + +/* test1 - glob var relocations */ +__u64 test1_hit_cnt = 0; + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test1, int a) +{ + test1_hit_cnt++; + return 0; +} + +int test1_alias(int a) __attribute__((alias("test1"))); + +/* test2 - map relocations */ +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test2, int a) +{ + __u64 key = 0, *value, new_value; + + value = bpf_map_lookup_elem(&map, &key); + new_value = value ? *value + 1 : 1; + bpf_map_update_elem(&map, &key, &new_value, 0); + return 0; +} + +int test2_alias(int a) __attribute__((alias("test2"))); + +/* test3 - subprog relocations */ +__u64 test3_hit_cnt = 0; + +static __noinline void test3_subprog(void) +{ + test3_hit_cnt++; +} + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test3, int a) +{ + test3_subprog(); + return 0; +} + +int test3_alias(int a) __attribute__((alias("test3"))); + +/* test4 - CO-RE relocations */ +__u64 test4_hit_cnt = 0; + +SEC("fentry/bpf_fentry_test1") +int BPF_PROG(test4, int a) +{ + struct task_struct *task; + int pid; + + task = (void *)bpf_get_current_task(); + pid = BPF_CORE_READ(task, tgid); + + if (pid == real_pid) + test4_hit_cnt++; + + return 0; +} + +int test4_alias(int a) __attribute__((alias("test4")));