From patchwork Fri May 17 10:22:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666830 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C71AD3A1AB for ; Fri, 17 May 2024 10:23:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941407; cv=none; b=FcIRQd61tJKXaxFEJmRI5nMap11gu3NyKilUaZls5UhT6SOc6NHmeS1krahRY4Sj9Y/w9E2IsrHm6NPA4o0XjGOPBWk5oiS3cojlq/pF4CqEfSdU9n5spwXI0rERx8dHCxxDJMiac8I6mFFZTczyapyGb5yBXKg/I561Gsw1prY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941407; c=relaxed/simple; bh=OaRtFfQLVJ0N0DfXK04z5pWJ938PMap+bgOY+5rtWjU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LSB1M6fHbyWMdaawYl3uyezfJoepOBnz7JpYziUfHBE+zkEjSVdzKNEt/EsBhbqQX1y3EPianpxUWe/b7Shejoy5MiOmpbNEsEg5F0kMWqunPwT6B8ir4vcSvV2WiEkDo8k4Sv2cm5V3TTJMMovjc8HsyIxzfGycLmS5MJTnn+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ajjpxDZ3; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ajjpxDZ3" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H7KTb6031029; Fri, 17 May 2024 10:23:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=pGOrLBwQR6CdoB/RW8K2HFpL1mehqXtHwITRKHweT3E=; b=ajjpxDZ3zAceGLAZZ3Jht4gMYeyaYpEc60tvpXPorWeZvAmxr1UO2s1nHFJTGfv4E0c9 U8Fs8GdRe6xErF0kzYe4Cwi6UhCDGe1wjPCublanQWgx460/rPr7sRKK6FzxLTVeRU9w 1dv8DYRLXkPGLJXPCDC+2klpQ96qBBYAOQKQtPundTDVxHVYGWkHzuwUOv9oNv348W7f jM095VuN3gzXf1vrF8AOUkhtORV1/1J3hHUk3X+ymoddkxnOMDz/nWyONmJNngU3ORyn 5O6mgmq4QCAFCZbf72wnOpHAZaUUMI/5iEKVPgpedd5cNWC2Bwd0mj1f8+zps0YpORCm HA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3tx8q9u9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:22:59 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8v803000563; Fri, 17 May 2024 10:22:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqrva-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:22:58 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFa036134; Fri, 17 May 2024 10:22:58 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-2; Fri, 17 May 2024 10:22:57 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 01/11] libbpf: add btf__distill_base() creating split BTF with distilled base BTF Date: Fri, 17 May 2024 11:22:36 +0100 Message-Id: <20240517102246.4070184-2-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: xqK0wcU9Mcpd4XKVe5dQK6zmA5qKb42J X-Proofpoint-ORIG-GUID: xqK0wcU9Mcpd4XKVe5dQK6zmA5qKb42J X-Patchwork-Delegate: bpf@iogearbox.net To support more robust split BTF, adding supplemental context for the base BTF type ids that split BTF refers to is required. Without such references, a simple shuffling of base BTF type ids (without any other significant change) invalidates the split BTF. Here the attempt is made to store additional context to make split BTF more robust. This context comes in the form of distilled base BTF providing minimal information (name and - in some cases - size) for base INTs, FLOATs, STRUCTs, UNIONs, ENUMs and ENUM64s along with modified split BTF that points at that base and contains any additional types needed (such as TYPEDEF, PTR and anonymous STRUCT/UNION declarations). This information constitutes the minimal BTF representation needed to disambiguate or remove split BTF references to base BTF. The rules are as follows: - INT, FLOAT are recorded in full. - if a named base BTF STRUCT or UNION is referred to from split BTF, it will be encoded either as a zero-member sized STRUCT/UNION (preserving size for later relocation checks) or as a named FWD. Only base BTF STRUCT/UNIONs that are either embedded in split BTF STRUCT/UNIONs or that have multiple STRUCT/UNION instances of the same name need to preserve size information, so a FWD representation will be used in most cases. - if an ENUM[64] is named, a ENUM forward representation (an ENUM with no values) is used. - in all other cases, the type is added to the new split BTF. Avoiding struct/union/enum/enum64 expansion is important to keep the distilled base BTF representation to a minimum size. When successful, new representations of the distilled base BTF and new split BTF that refers to it are returned. Both need to be freed by the caller. So to take a simple example, with split BTF with a type referring to "struct sk_buff", we will generate distilled base BTF with a FWD struct sk_buff, and the split BTF will refer to it instead. Tools like pahole can utilize such split BTF to populate the .BTF section (split BTF) and an additional .BTF.base section. Then when the split BTF is loaded, the distilled base BTF can be used to relocate split BTF to reference the current (and possibly changed) base BTF. So for example if "struct sk_buff" was id 502 when the split BTF was originally generated, we can use the distilled base BTF to see that id 502 refers to a "struct sk_buff" and replace instances of id 502 with the current (relocated) base BTF sk_buff type id. Distilled base BTF is small; when building a kernel with all modules using distilled base BTF as a test, ovreall module size grew by only 5.3Mb total across ~2700 modules. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 409 ++++++++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 20 ++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 424 insertions(+), 6 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 2d0840ef599a..953929d196c3 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1771,9 +1771,8 @@ static int btf_rewrite_str(__u32 *str_off, void *ctx) return 0; } -int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type) { - struct btf_pipe p = { .src = src_btf, .dst = btf }; struct btf_type *t; int sz, err; @@ -1782,20 +1781,27 @@ int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_t return libbpf_err(sz); /* deconstruct BTF, if necessary, and invalidate raw_data */ - if (btf_ensure_modifiable(btf)) + if (btf_ensure_modifiable(p->dst)) return libbpf_err(-ENOMEM); - t = btf_add_type_mem(btf, sz); + t = btf_add_type_mem(p->dst, sz); if (!t) return libbpf_err(-ENOMEM); memcpy(t, src_type, sz); - err = btf_type_visit_str_offs(t, btf_rewrite_str, &p); + err = btf_type_visit_str_offs(t, btf_rewrite_str, p); if (err) return libbpf_err(err); - return btf_commit_type(btf, sz); + return btf_commit_type(p->dst, sz); +} + +int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +{ + struct btf_pipe p = { .src = src_btf, .dst = btf }; + + return btf_add_type(&p, src_type); } static int btf_rewrite_type_ids(__u32 *type_id, void *ctx) @@ -5212,3 +5218,394 @@ int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void return 0; } + +#define BTF_NEEDS_SIZE (1 << 31) /* flag set if either struct/union is + * embedded - and thus size info must + * be preserved - or if there are + * multiple instances of the same + * struct/union - where size can be + * used to clarify which is wanted. + */ +#define BTF_ID(id) (id & ~BTF_NEEDS_SIZE) + +struct btf_distill { + struct btf_pipe pipe; + int *ids; + unsigned int split_start_id; + unsigned int split_start_str; + unsigned int diff_id; +}; + +/* Check if a member of a split BTF struct/union refers to a base BTF + * struct/union. Members can be const/restrict/volatile/typedef + * reference types, but if a pointer is encountered, type is no longer + * considered embedded. + */ +static int btf_find_embedded_composite_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + const struct btf_type *t; + __u32 next_id = *id; + + do { + if (next_id == 0) + return 0; + t = btf_type_by_id(dist->pipe.src, next_id); + switch (btf_kind(t)) { + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_TYPEDEF: + case BTF_KIND_TYPE_TAG: + next_id = t->type; + break; + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + next_id = a->type; + break; + } + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + dist->ids[next_id] |= BTF_NEEDS_SIZE; + return 0; + default: + return 0; + } + + } while (1); + + return 0; +} + +/* Check if composite type has a duplicate-named type; if it does, retain + * size information to help guide later relocation towards the correct type. + * For example there are duplicate 'dma_chan' structs in vmlinux BTF; + * one is size 112, the other 16. Having size information allows relocation + * to choose the right one. + */ +static int btf_mark_composite_dups(struct btf_distill *dist, __u32 id) +{ + __u8 *cnt = calloc(dist->split_start_str, sizeof(__u8)); + struct btf_type *t; + int i; + + if (!cnt) + return -ENOMEM; + + /* First pass; collect name counts for composite types. */ + for (i = 1; i < dist->split_start_id; i++) { + t = btf_type_by_id(dist->pipe.src, i); + if (!btf_is_composite(t) || !t->name_off) + continue; + if (cnt[t->name_off] < 255) + cnt[t->name_off]++; + } + /* Second pass; mark composite types with multiple instances of the + * same name as needing size information. + */ + for (i = 1; i < dist->split_start_id; i++) { + /* id not needed or is already preserving size information */ + if (!dist->ids[i] || (dist->ids[i] & BTF_NEEDS_SIZE)) + continue; + t = btf_type_by_id(dist->pipe.src, i); + if (!btf_is_composite(t) || !t->name_off) + continue; + if (cnt[t->name_off] > 1) + dist->ids[i] |= BTF_NEEDS_SIZE; + } + free(cnt); + + return 0; +} + +static bool btf_is_eligible_named_fwd(const struct btf_type *t) +{ + return (btf_is_composite(t) || btf_is_any_enum(t)) && t->name_off != 0; +} + +static int btf_add_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + struct btf_type *t = btf_type_by_id(dist->pipe.src, *id); + int err; + + if (!*id) + return 0; + /* split BTF id, not needed */ + if (*id >= dist->split_start_id) + return 0; + /* already added ? */ + if (BTF_ID(dist->ids[*id]) > 0) + return 0; + + /* only a subset of base BTF types should be referenced from split + * BTF; ensure nothing unexpected is referenced. + */ + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_ARRAY: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_TYPEDEF: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_PTR: + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_FUNC_PROTO: + case BTF_KIND_TYPE_TAG: + dist->ids[*id] |= *id; + break; + default: + pr_warn("unexpected reference to base type[%u] of kind [%u] when creating distilled base BTF.\n", + *id, btf_kind(t)); + return -EINVAL; + } + + /* struct/union members not needed, except for anonymous structs + * and unions, which we need since name won't help us determine + * matches; so if a named struct/union, no need to recurse + * into members. + */ + if (btf_is_eligible_named_fwd(t)) + return 0; + + /* ensure references in type are added also. */ + err = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, ctx); + if (err < 0) + return err; + return 0; +} + +static int btf_add_distilled_types(struct btf_distill *dist) +{ + bool adding_to_base = dist->pipe.dst->start_id == 1; + int id = btf__type_cnt(dist->pipe.dst); + struct btf_type *t; + int i, err = 0; + + /* Add types for each of the required references to either distilled + * base or split BTF, depending on type characteristics. + */ + for (i = 1; i < dist->split_start_id; i++) { + const char *name; + int kind; + + if (!BTF_ID(dist->ids[i])) + continue; + t = btf_type_by_id(dist->pipe.src, i); + kind = btf_kind(t); + name = btf__name_by_offset(dist->pipe.src, t->name_off); + + /* Named int, float, fwd struct, union, enum[64] are added to + * base; everything else is added to split BTF. + */ + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + if ((adding_to_base && !t->name_off) || (!adding_to_base && t->name_off)) + continue; + break; + default: + if (adding_to_base) + continue; + break; + } + if (dist->ids[i] & BTF_NEEDS_SIZE) { + /* If a named struct/union in base BTF is referenced as a type + * in split BTF without use of a pointer - i.e. as an embedded + * struct/union - add an empty struct/union preserving size + * since size must be consistent when relocating split and + * possibly changed base BTF. Similarly, when a struct/union + * has multiple instances of the same name in the original + * base BTF, retain size to help relocation later pick the + * right struct/union. + */ + err = btf_add_composite(dist->pipe.dst, kind, name, t->size); + } else if (btf_is_eligible_named_fwd(t)) { + /* If not embedded, use a fwd for named struct/unions since we + * can match via name without any other details. + */ + switch (kind) { + case BTF_KIND_STRUCT: + err = btf__add_fwd(dist->pipe.dst, name, BTF_FWD_STRUCT); + break; + case BTF_KIND_UNION: + err = btf__add_fwd(dist->pipe.dst, name, BTF_FWD_UNION); + break; + case BTF_KIND_ENUM: + err = btf__add_enum(dist->pipe.dst, name, t->size); + break; + case BTF_KIND_ENUM64: + err = btf__add_enum(dist->pipe.dst, name, t->size); + break; + default: + pr_warn("unexpected kind [%u] when creating distilled base BTF.\n", + btf_kind(t)); + return -EINVAL; + } + } else { + err = btf_add_type(&dist->pipe, t); + } + if (err < 0) + break; + dist->ids[i] = id++; + } + return err; +} + +/* Split BTF ids without a mapping will be shifted downwards since distilled + * base BTF is smaller than the original base BTF. For those that have a + * mapping (either to base or updated split BTF), update the id based on + * that mapping. + */ +static int btf_update_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + + if (BTF_ID(dist->ids[*id])) + *id = BTF_ID(dist->ids[*id]); + else if (*id >= dist->split_start_id) + *id -= dist->diff_id; + return 0; +} + +/* Create updated split BTF with distilled base BTF; distilled base BTF + * consists of BTF information required to clarify the types that split + * BTF refers to, omitting unneeded details. Specifically it will contain + * base types and forward declarations of named structs, unions and enumerated + * types. Associated reference types like pointers, arrays and anonymous + * structs, unions and enumerated types will be added to split BTF. + * + * The only case where structs, unions or enumerated types are fully represented + * is when they are anonymous; in such cases, the anonymous type is added to + * split BTF in full. + * + * We return newly-created split BTF where the split BTF refers to a newly-created + * distilled base BTF. Both must be freed separately by the caller. + */ +int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf) +{ + struct btf *new_base = NULL, *new_split = NULL; + const struct btf *old_base; + unsigned int n = btf__type_cnt(src_btf); + struct btf_distill dist = {}; + struct btf_type *t; + int i, err = 0; + + /* src BTF must be split BTF. */ + old_base = btf__base_btf(src_btf); + if (!new_base_btf || !new_split_btf || !old_base) + return libbpf_err(-EINVAL); + + new_base = btf__new_empty(); + if (!new_base) + return libbpf_err(-ENOMEM); + dist.ids = calloc(n, sizeof(*dist.ids)); + if (!dist.ids) { + err = -ENOMEM; + goto err_out; + } + dist.pipe.src = src_btf; + dist.pipe.dst = new_base; + dist.pipe.str_off_map = hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL); + if (IS_ERR(dist.pipe.str_off_map)) { + err = -ENOMEM; + goto err_out; + } + dist.split_start_id = btf__type_cnt(old_base); + dist.split_start_str = old_base->hdr->str_len; + + /* Pass over src split BTF; generate the list of base BTF + * type ids it references; these will constitute our distilled + * BTF set to be distributed over base and split BTF as appropriate. + */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + + /* check if members of struct/union in split BTF refer to base BTF + * struct/union; if so, we will use an empty sized struct to represent + * it rather than a FWD because its size must match on later BTF + * relocation. + */ + if (btf_is_composite(t)) { + err = btf_type_visit_type_ids(t, btf_find_embedded_composite_type_ids, + &dist); + if (err < 0) + goto err_out; + } + err = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, &dist); + if (err < 0) + goto err_out; + } + /* Next check the distilled type id list for any struct/unions that + * have multiple instances of the same name in base BTF; size must be + * preserved for those cases also to guide relocation matching. + */ + err = btf_mark_composite_dups(&dist, i); + if (err < 0) + goto err_out; + + /* Next add types for each of the required references to base BTF and split BTF in turn. */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto err_out; + /* now create new split BTF with distilled base BTF as its base; we end up with + * split BTF that has base BTF that represents enough about its base references + * to allow it to be relocated with the base BTF available. + */ + new_split = btf__new_empty_split(new_base); + if (!new_split_btf) { + err = -errno; + goto err_out; + } + dist.pipe.dst = new_split; + /* First add all split types */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + err = btf_add_type(&dist.pipe, t); + if (err < 0) + goto err_out; + } + /* Now add distilled types to split BTF that are not added to base. */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto err_out; + + /* all split BTF ids will be shifted downwards since there are less base BTF ids + * in distilled base BTF. + */ + dist.diff_id = dist.split_start_id - btf__type_cnt(new_base); + + n = btf__type_cnt(new_split); + /* Now update base/split BTF ids. */ + for (i = 1; i < n; i++) { + t = btf_type_by_id(new_split, i); + + err = btf_type_visit_type_ids(t, btf_update_distilled_type_ids, &dist); + if (err < 0) + goto err_out; + } + free(dist.ids); + hashmap__free(dist.pipe.str_off_map); + *new_base_btf = new_base; + *new_split_btf = new_split; + return 0; +err_out: + free(dist.ids); + if (!IS_ERR(dist.pipe.str_off_map)) + hashmap__free(dist.pipe.str_off_map); + btf__free(new_split); + btf__free(new_base); + return libbpf_err(err); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c84..f3f149a09088 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -107,6 +107,26 @@ LIBBPF_API struct btf *btf__new_empty(void); */ LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf); +/** + * @brief **btf__distill_base()** creates new versions of the split BTF + * *src_btf* and its base BTF. The new base BTF will only contain the types + * needed to improve robustness of the split BTF to small changes in base BTF. + * When that split BTF is loaded against a (possibly changed) base, this + * distilled base BTF will help update references to that (possibly changed) + * base BTF. + * + * Both the new split and its associated new base BTF must be freed by + * the caller. + * + * If successful, 0 is returned and **new_base_btf** and **new_split_btf** + * will point at new base/split BTF. Both the new split and its associated + * new base BTF must be freed by the caller. + * + * A negative value is returned on error. + */ +LIBBPF_API int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf); + LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext); LIBBPF_API struct btf *btf__parse_split(const char *path, struct btf *base_btf); LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c1ce8aa3520b..9e69d6e2a512 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -419,6 +419,7 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: + btf__distill_base; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; From patchwork Fri May 17 10:22:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666832 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE03B36B1D for ; Fri, 17 May 2024 10:23:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941416; cv=none; b=cFXEP1guGrxbdOa5mXRudcX7dLoYuRnt97lusk8HWhQyg0dEC6IyDTeGtFDWZuIO9DgAJ/+imxEU3XwshBq6BQhcehxX8Qt5/oMkPvWYE4+MUG4Q5D6J7upKVWz1k/apzZErGMMghkoXcP3uL+iX7CVhV+f9sT4iYBrZTpU1NSk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941416; c=relaxed/simple; bh=kvZcZfElvXPoJHIb6kL3M0t1ACxvRp6K8QlXKKeKCyE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FOB0CYmxaLNx7HKuSL2wS3PD1TA9OZJRrpQXiNfN4h0l/d37R7FlOFVK/AvAzN9s9tET6QN1l8NkuFpLG7iXnRoWCYwb37Gkh3gj0CDmxqJOiJmLhQTb91E0pWo0WylsMY65B+PaROCsoUAFxtL2mu81ELg85vUP2aJRxiGEH9w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=fNzCeVli; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="fNzCeVli" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44HA3DI2020469; Fri, 17 May 2024 10:23:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=82V+LpPII3apndW5F1O3nzJBL+KgnFPQDc3BDb9mujM=; b=fNzCeVlijQF5tg5PI3ZQJ9FyZ9ghoP6mnGD9ilsjYxPldA4Dnyl7UhW6GiCxam4rDs69 JtENNZbo9QAZAiQwEUYtaVI2MBsM+J/quNDtv8JJypPnELQxfzYfh0xYr7wE1wRUVEZ1 whCfzTHQ1aXwlRm4ZmGtwx9MhNm0IwFy4AG7W/0V58DikP74VQlewP280g2tV5aC0Z4e fVtvrGFPgJHsycUzYYW+cHVofBBuubb3zS1y3OnLoVWC98wFKO404mmC8Kdq2UAURe3f wVLO2YI3O8O83ijApMm/JEp6Jvm/PUUFNEGy2F9QP+Hz9lIIDcM60GGqh58ZS731fVOO pw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3t4fhg67-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:05 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44HA2YPK000531; Fri, 17 May 2024 10:23:04 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqrx6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:04 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFc036134; Fri, 17 May 2024 10:23:03 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-3; Fri, 17 May 2024 10:23:03 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 02/11] selftests/bpf: test distilled base, split BTF generation Date: Fri, 17 May 2024 11:22:37 +0100 Message-Id: <20240517102246.4070184-3-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-ORIG-GUID: UN_nbyYQgt0NZwfkYlR_6KqNmegvpjLW X-Proofpoint-GUID: UN_nbyYQgt0NZwfkYlR_6KqNmegvpjLW X-Patchwork-Delegate: bpf@iogearbox.net Test generation of split+distilled base BTF, ensuring that - base BTF STRUCTs which are embedded in split BTF structs are represented as 0-member sized structs, allowing size checking - base BTF UNION which has a duplicate-named UNION is represented as a 0-member sized union, helping clarify which union is referred to - FWDs are used in place of full named struct/union declarations - FWDs are used in place of full named enum declarations - anonymous struct/unions are represented in full in split BTF - anonymous enums are represented in full in split BTF - types unreferenced from split BTF are not present in distilled base BTF Also test that with vmlinux BTF and split BTF based upon it, we only represent needed base types referenced from split BTF in distilled base. Signed-off-by: Alan Maguire --- .../selftests/bpf/prog_tests/btf_distill.c | 279 ++++++++++++++++++ 1 file changed, 279 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_distill.c diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c new file mode 100644 index 000000000000..4ab522df041c --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -0,0 +1,279 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include +#include +#include "btf_helpers.h" + +/* Fabricate base, split BTF with references to base types needed; then create + * split BTF with distilled base BTF and ensure expectations are met: + * - only referenced base types from split BTF are present + * - struct/union/enum are represented as FWDs unless anonymous, when they + * are represented in full, or if embedded in a split BTF struct or duplicated + * case they are represented by a STRUCT/UNION with specified size and vlen=0. + */ +static void test_distilled_base(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_ptr(btf1, 1); /* [2] ptr to int */ + btf__add_struct(btf1, "s1", 8); /* [3] struct s1 { */ + btf__add_field(btf1, "f1", 2, 0, 0); /* int *f1; */ + /* } */ + btf__add_struct(btf1, "", 12); /* [4] struct { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 3, 32, 0); /* struct s1 f2; */ + /* } */ + btf__add_int(btf1, "unsigned int", 4, 0); /* [5] unsigned int */ + btf__add_union(btf1, "u1", 12); /* [6] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 2, 0, 0); /* int *f2; */ + /* } */ + btf__add_union(btf1, "", 4); /* [7] union { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_enum(btf1, "e1", 4); /* [8] enum e1 { */ + btf__add_enum_value(btf1, "v1", 1); /* v1 = 1; */ + /* } */ + btf__add_enum(btf1, "", 4); /* [9] enum { */ + btf__add_enum_value(btf1, "av1", 2); /* av1 = 2; */ + /* } */ + btf__add_enum64(btf1, "e641", 8, true); /* [10] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1024); /* v1 = 1024; */ + /* } */ + btf__add_enum64(btf1, "", 8, true); /* [11] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1025); /* v1 = 1025; */ + /* } */ + btf__add_struct(btf1, "unneeded", 4); /* [12] struct unneeded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_struct(btf1, "embedded", 4); /* [13] struct embedded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_func_proto(btf1, 1); /* [14] int (*)(int *p1); */ + btf__add_func_param(btf1, "p1", 1); + + btf__add_array(btf1, 1, 1, 3); /* [15] int [3]; */ + + btf__add_struct(btf1, "from_proto", 4); /* [16] struct from_proto { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_union(btf1, "u1", 4); /* [17] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0"); + + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + + btf__add_ptr(btf2, 3); /* [18] ptr to struct s1 */ + /* add ptr to struct anon */ + btf__add_ptr(btf2, 4); /* [19] ptr to struct (anon) */ + btf__add_const(btf2, 6); /* [20] const union u1 */ + btf__add_restrict(btf2, 7); /* [21] restrict union (anon) */ + btf__add_volatile(btf2, 8); /* [22] volatile enum e1 */ + btf__add_typedef(btf2, "et", 9); /* [23] typedef enum (anon) */ + btf__add_const(btf2, 10); /* [24] const enum64 e641 */ + btf__add_ptr(btf2, 11); /* [25] restrict enum64 (anon) */ + btf__add_struct(btf2, "with_embedded", 4); /* [26] struct with_embedded { */ + btf__add_field(btf2, "f1", 13, 0, 0); /* struct embedded f1; */ + /* } */ + btf__add_func(btf2, "fn", BTF_FUNC_STATIC, 14); /* [27] int fn(int p1); */ + btf__add_typedef(btf2, "arraytype", 15); /* [28] typedef int[3] foo; */ + btf__add_func_proto(btf2, 1); /* [29] int (*)(struct from proto p1); */ + btf__add_func_param(btf2, "p1", 16); + + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[18] PTR '(anon)' type_id=3", + "[19] PTR '(anon)' type_id=4", + "[20] CONST '(anon)' type_id=6", + "[21] RESTRICT '(anon)' type_id=7", + "[22] VOLATILE '(anon)' type_id=8", + "[23] TYPEDEF 'et' type_id=9", + "[24] CONST '(anon)' type_id=10", + "[25] PTR '(anon)' type_id=11", + "[26] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[27] FUNC 'fn' type_id=14 linkage=static", + "[28] TYPEDEF 'arraytype' type_id=15", + "[29] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16"); + + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(8, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 's1' fwd_kind=struct", + "[3] UNION 'u1' size=12 vlen=0", + "[4] ENUM 'e1' encoding=UNSIGNED size=4 vlen=0", + "[5] ENUM 'e641' encoding=UNSIGNED size=8 vlen=0", + "[6] STRUCT 'embedded' size=4 vlen=0", + "[7] FWD 'from_proto' fwd_kind=struct", + /* split BTF; these types should match split BTF above from 17-28, with + * updated type id references + */ + "[8] PTR '(anon)' type_id=2", + "[9] PTR '(anon)' type_id=20", + "[10] CONST '(anon)' type_id=3", + "[11] RESTRICT '(anon)' type_id=21", + "[12] VOLATILE '(anon)' type_id=4", + "[13] TYPEDEF 'et' type_id=22", + "[14] CONST '(anon)' type_id=5", + "[15] PTR '(anon)' type_id=23", + "[16] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=6 bits_offset=0", + "[17] FUNC 'fn' type_id=24 linkage=static", + "[18] TYPEDEF 'arraytype' type_id=25", + "[19] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=7", + /* split BTF types added from original base BTF below */ + "[20] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=32", + "[21] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[22] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[23] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[24] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[25] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* create split reference BTF from vmlinux + split BTF with a few type references; + * ensure the resultant split reference BTF is as expected, containing only types + * needed to disambiguate references from split BTF. + */ +static void test_distilled_base_vmlinux(void) +{ + struct btf *split_btf = NULL, *vmlinux_btf = btf__load_vmlinux_btf(); + struct btf *split_dist = NULL, *base_dist = NULL; + __s32 int_id, sk_buff_id; + + if (!ASSERT_OK_PTR(vmlinux_btf, "load_vmlinux")) + return; + int_id = btf__find_by_name_kind(vmlinux_btf, "int", BTF_KIND_INT); + if (!ASSERT_GT(int_id, 0, "find_int")) + goto cleanup; + sk_buff_id = btf__find_by_name_kind(vmlinux_btf, "sk_buff", BTF_KIND_STRUCT); + if (!ASSERT_GT(sk_buff_id, 0, "find_sk_buff_id")) + goto cleanup; + split_btf = btf__new_empty_split(vmlinux_btf); + if (!ASSERT_OK_PTR(split_btf, "new_split")) + goto cleanup; + btf__add_typedef(split_btf, "myint", int_id); + btf__add_ptr(split_btf, sk_buff_id); + + if (!ASSERT_EQ(btf__distill_base(split_btf, &base_dist, &split_dist), 0, + "distill_vmlinux_base")) + goto cleanup; + + if (!ASSERT_OK_PTR(split_dist, "split_distilled") || + !ASSERT_OK_PTR(base_dist, "base_dist")) + goto cleanup; + VALIDATE_RAW_BTF( + split_dist, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 'sk_buff' fwd_kind=struct", + "[3] TYPEDEF 'myint' type_id=1", + "[4] PTR '(anon)' type_id=2"); + +cleanup: + btf__free(split_dist); + btf__free(base_dist); + btf__free(split_btf); + btf__free(vmlinux_btf); +} + +void test_btf_distill(void) +{ + if (test__start_subtest("distilled_base")) + test_distilled_base(); + if (test__start_subtest("distilled_base_vmlinux")) + test_distilled_base_vmlinux(); +} From patchwork Fri May 17 10:22:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666831 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BED1536B09 for ; Fri, 17 May 2024 10:23:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941416; cv=none; b=fJMnzQGl8BLn6LYXc3ZztRYJFNj3AFGSwnD5x6T98nmOvaA6wbxr2f+9uK7s4a6IvCWW0ZXLw6tHPYf3N+wEvpKXA+SACUiNtXn257vEasC7FIw7Y8utDa6ELa+qs5lVIpH768/7VQCeHFKUR9tMiNNvfeQEk8jEOob5NvpgHRo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941416; c=relaxed/simple; bh=ktTHd6NG9BujSe63E5rLvF/t448hfQJWGHk3Upee4c4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nxXIuWy6tT/kk47kFvQxYRPff5XanYAJImcTthq4PDMxmISMyZ8qRL4Gfe399Ch2rse2h6f0XhCqjEEBEqE79Oce6eA+Xxg5zkXlHjXwPU5ppBjN64VW2sziHLzHq9qnZw5ouyqWKosYA1uOcAI2P+NUPn57Zdhk0tPtezRxa8I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=V6yeW3tp; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="V6yeW3tp" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H83FNN013318; Fri, 17 May 2024 10:23:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=K8UcUXxloosLUdwZLJNBsQV9zRGHU/7zSuOp5+5+tWA=; b=V6yeW3tpZZ5b9afJJZpGIFkvWfO5ggiGv34MgXoBlRc0J3jVWbGGIXqKWTX0VGeZVcTW W48g/doYAza0aGoXA5XqNRxC+znf8l0DaLD1vpxCwMmtWUmk7zirIWS82p+RRwmefBkq Heh+kKml5CjuwEGIZEQvz7EABcXZWJ7AnoLuEhM5HmJ2/JnpC8TvA4qBr+XSk0thjbK3 MekjKeR00hgrvF5cERvIzPiR4S5q7/jOGfy7kZnbqf8EfVP/oPhShBMDVqUtOhMmwKxb RFlVOl2UaSAekddP8cpleXV82YZgEx9pvlplBCa+2xpBkH/WLQsz1ZNKbjOZWp9WFHh6 bQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3tx399y2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:11 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44HA5iIh000505; Fri, 17 May 2024 10:23:10 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqs16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:10 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFe036134; Fri, 17 May 2024 10:23:09 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-4; Fri, 17 May 2024 10:23:09 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 03/11] libbpf: add btf__parse_opts() API for flexible BTF parsing Date: Fri, 17 May 2024 11:22:38 +0100 Message-Id: <20240517102246.4070184-4-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: St8UHLwcO_6ivb9Zx-u0DJxZF40zSouo X-Proofpoint-ORIG-GUID: St8UHLwcO_6ivb9Zx-u0DJxZF40zSouo X-Patchwork-Delegate: bpf@iogearbox.net Options cover existing parsing scenarios (ELF, raw, retrieving .BTF.ext) and also allow specification of the ELF section name containing BTF. This will allow consumers to retrieve BTF from .BTF.base sections (BTF_BASE_ELF_SEC) also. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 49 +++++++++++++++++++++++++++------------- tools/lib/bpf/btf.h | 31 +++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 65 insertions(+), 16 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 953929d196c3..feba071087a5 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1084,7 +1084,7 @@ struct btf *btf__new_split(const void *data, __u32 size, struct btf *base_btf) return libbpf_ptr(btf_new(data, size, base_btf)); } -static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, +static struct btf *btf_parse_elf(const char *path, const char *btf_sec, struct btf *base_btf, struct btf_ext **btf_ext) { Elf_Data *btf_data = NULL, *btf_ext_data = NULL; @@ -1146,7 +1146,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, idx, path); goto done; } - if (strcmp(name, BTF_ELF_SEC) == 0) { + if (strcmp(name, btf_sec) == 0) { btf_data = elf_getdata(scn, 0); if (!btf_data) { pr_warn("failed to get section(%d, %s) data from %s\n", @@ -1166,7 +1166,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, } if (!btf_data) { - pr_warn("failed to find '%s' ELF section in %s\n", BTF_ELF_SEC, path); + pr_warn("failed to find '%s' ELF section in %s\n", btf_sec, path); err = -ENODATA; goto done; } @@ -1212,12 +1212,12 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse_elf(path, NULL, btf_ext)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, NULL, btf_ext)); } struct btf *btf__parse_elf_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse_elf(path, base_btf, NULL)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, base_btf, NULL)); } static struct btf *btf_parse_raw(const char *path, struct btf *base_btf) @@ -1293,31 +1293,48 @@ struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf) return libbpf_ptr(btf_parse_raw(path, base_btf)); } -static struct btf *btf_parse(const char *path, struct btf *base_btf, struct btf_ext **btf_ext) +struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts) { - struct btf *btf; + struct btf *btf, *base_btf; + const char *btf_sec; + struct btf_ext **btf_ext; int err; + if (!OPTS_VALID(opts, btf_parse_opts)) + return libbpf_err_ptr(-EINVAL); + base_btf = OPTS_GET(opts, base_btf, NULL); + btf_sec = OPTS_GET(opts, btf_sec, NULL); + btf_ext = OPTS_GET(opts, btf_ext, NULL); + if (btf_ext) *btf_ext = NULL; - - btf = btf_parse_raw(path, base_btf); + if (!btf_sec) { + btf = btf_parse_raw(path, base_btf); + err = libbpf_get_error(btf); + if (!err) + return btf; + if (err != -EPROTO) + return libbpf_err_ptr(err); + } + btf = btf_parse_elf(path, btf_sec ?: BTF_ELF_SEC, base_btf, btf_ext); err = libbpf_get_error(btf); - if (!err) - return btf; - if (err != -EPROTO) - return ERR_PTR(err); - return btf_parse_elf(path, base_btf, btf_ext); + if (err) + return libbpf_err_ptr(err); + return btf; } struct btf *btf__parse(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse(path, NULL, btf_ext)); + LIBBPF_OPTS(btf_parse_opts, opts, .btf_ext = btf_ext); + + return btf__parse_opts(path, &opts); } struct btf *btf__parse_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse(path, base_btf, NULL)); + LIBBPF_OPTS(btf_parse_opts, opts, .base_btf = base_btf); + + return btf__parse_opts(path, &opts); } static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian); diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index f3f149a09088..8e1702ad5ef4 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -18,6 +18,7 @@ extern "C" { #define BTF_ELF_SEC ".BTF" #define BTF_EXT_ELF_SEC ".BTF.ext" +#define BTF_BASE_ELF_SEC ".BTF.base" #define MAPS_ELF_SEC ".maps" struct btf; @@ -134,6 +135,36 @@ LIBBPF_API struct btf *btf__parse_elf_split(const char *path, struct btf *base_b LIBBPF_API struct btf *btf__parse_raw(const char *path); LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf); +struct btf_parse_opts { + size_t sz; + /* use base BTF to parse split BTF */ + struct btf *base_btf; + /* retrieve optional .BTF.ext info */ + struct btf_ext **btf_ext; + /* BTF section name; if NULL, try parsing raw BTF, falling back to parsing + * .BTF ELF section if that fails also. If set, parse named ELF section. + */ + const char *btf_sec; + size_t :0; +}; + +#define btf_parse_opts__last_field btf_sec + +/* @brief **btf__parse_opts()** parses BTF information from either a + * raw BTF file (*btf_sec* is NULL) or from the specified BTF section, + * also retrieving .BTF.ext info if *btf_ext* is non-NULL. If + * *base_btf* is specified, use it to parse split BTF from the + * specified location. + * + * @return new BTF object instance which has to be eventually freed with + * **btf__free()** + * + * On error, NULL is returned, with the `errno` variable set to the + * error code. + */ + +LIBBPF_API struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts); + LIBBPF_API struct btf *btf__load_vmlinux_btf(void); LIBBPF_API struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_btf); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 9e69d6e2a512..fd7bfeaba542 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -420,6 +420,7 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: btf__distill_base; + btf__parse_opts; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; From patchwork Fri May 17 10:22:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666836 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A53DB364DA for ; Fri, 17 May 2024 10:24:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941443; cv=none; b=I7DU4JrlTzW7o3Eyst1gXI3cOnaqdOpcuj8DBGnlOPa9cqR2uugNHFBkVYr7IJOkp8mAjSKh0c1WlWghaFzHc23MGvQBR9nier94w1nJiIN0iVQ+e4YhHzQR/Jfe8ebXTRmw9+YYtNTM0e6wprRwacSlRXOjCgegfvVy1oOd6k8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941443; c=relaxed/simple; bh=qcWPhLjDHY6CcpCjNU1MQZYsT6uLYlDoZuk3WyLhdhA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oAlB+++8l2lWzy0jOy0/AOEzHnXMbKNUGkZcMalv02n5fbCn/6F2BlSFCBWyWhF2FmxnfgWCp8EG8JnBTtnSERNmItRiXgQKs9PHgoUJstKKGkUHy20pVkh1hoH1JNvONN2E7WCKO1Q/jGobZA2fHZwn/94OFXiKmv7Swk7e0R0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=G8zoWyrp; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="G8zoWyrp" Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H5Oudj007575; Fri, 17 May 2024 10:23:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=JJJz8ZRYZQlbtouiAH1SaVGrf5QMTghBmUWnxKuSwfQ=; b=G8zoWyrplyHTAZy61rHcEz8K2f5ja7zeqaX7H4dCJBGlcEF7jcd7nhU5CdVUFr98V9jw 5h9ihoBkyL+GGkNXNqTUD2We4KiLjD7Vtpkc+tv9aCimBBkWNa5ORKhpwzzl6G8S0qgQ OsAyGp9gexwSJo8rhhGQWfmNfV/ZLfdo19xR28zgznnU/6aIHoq+gKWoItTBPdc+Xz51 JPv86fyMQeQ8UweUT6/9aT5i2hL+eRk95C19sVXY3Ga2sbRAJrVzqTFwIFKnbIcvw/eR l0AclDKs3LbGkTios6Zn7WH/pSaLi7gFv7Gdu7xCuirrjr9xSo7eiRNSw8bnK7crVopV BA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3srryre1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:16 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44HA2YPP000531; Fri, 17 May 2024 10:23:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqs39-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:16 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFg036134; Fri, 17 May 2024 10:23:15 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-5; Fri, 17 May 2024 10:23:15 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire , Quentin Monnet Subject: [PATCH v4 bpf-next 04/11] bpftool: support displaying raw split BTF using base BTF section as base Date: Fri, 17 May 2024 11:22:39 +0100 Message-Id: <20240517102246.4070184-5-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-ORIG-GUID: QsZti-CaIHlxpPJkOJju6iWiJSpSV50C X-Proofpoint-GUID: QsZti-CaIHlxpPJkOJju6iWiJSpSV50C X-Patchwork-Delegate: bpf@iogearbox.net If no base BTF can be found, fall back to checking for the .BTF.base section and use it to display split BTF. Signed-off-by: Alan Maguire Acked-by: Quentin Monnet --- tools/bpf/bpftool/btf.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index af047dedde38..67be216023e8 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -756,6 +756,14 @@ static int do_dump(int argc, char **argv) base = get_vmlinux_btf_from_sysfs(); btf = btf__parse_split(*argv, base ?: base_btf); + /* Finally check for presence of base BTF section */ + if (!btf && !base && !base_btf) { + LIBBPF_OPTS(btf_parse_opts, optp, .btf_sec = BTF_BASE_ELF_SEC); + + base_btf = btf__parse_opts(*argv, &optp); + if (base_btf) + btf = btf__parse_split(*argv, base_btf); + } if (!btf) { err = -errno; p_err("failed to load BTF from %s: %s", From patchwork Fri May 17 10:22:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666833 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E48936137 for ; Fri, 17 May 2024 10:23:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941427; cv=none; b=Z9wDyk93o29MDJ5cQfZsxt0VFeJlfHaURKyjoUPiBgoMchoRlxOfEV7Xh5HS4ysh5iNSncTG62aqlyiOh+T8JLV5imKXYlYOMDabG9s3Mj58YkNaLAPcNB/a7T7A0T1+DbO2Ex39MMnLuB73WfC+agKKkRD9FAlwyO9Kisd2UZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941427; c=relaxed/simple; bh=/+yXlg1Ts9s3xPeHYINfynhz5+7pZAQ2giUpuazZaGU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PIt3ycJLNHZjVNYrkdaF3Ix9UzRV6RqwLDPGntUZI+mDz3xk38b9XXkZOlgNGL6nB+s4l2k4pvDGYXQwTJ6xjLh96BD5ip182s2AtmLyrLcaRBULsJNPJJIteHKQdYyKLyeJ+LEBYTBiebIbcVNJfXUZrN488XoqWxRM6x33p+Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=QiOEXJlq; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QiOEXJlq" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8mOmE031133; Fri, 17 May 2024 10:23:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=Pgy5/CZsZxyM/Pmv8NhpXNnUAQhuoc8JW/QL2mqOBME=; b=QiOEXJlqOngq4GFp0eRTta2TBfwZk3mC99AnCP4N2OW5kumZFep8r860RRbg1Tyi/gNg A5OiZ3dAHtLst11GjoP47ILbKcgfZnYNSwqOEvFZrDT0JwCpCe6mmeCYQ8U3lI/H4U7V iTH/5A7EF56Pjz/IW0mNDymM8y5nsWtIWtYdUCmmU/cI8lBC2iMPBl4deAZGDE5MrMVd VzjpOs+tkjqJ7YOPT5BXVWNR9kntgS3TOP/WUoGNsb6+cITndAxqbL3TEKyJpgVsuOjU mvJu16RFAvS07w/kvGKA2bIQnyljpl0tGuQvSXawkN7+sP6k7Bg++/oHYUAlwPdiqEGy Ew== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3t4fhg6n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:22 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H9wkBn001205; Fri, 17 May 2024 10:23:21 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqs58-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:21 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFi036134; Fri, 17 May 2024 10:23:20 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-6; Fri, 17 May 2024 10:23:20 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 05/11] resolve_btfids: use .BTF.base ELF section as base BTF if -B option is used Date: Fri, 17 May 2024 11:22:40 +0100 Message-Id: <20240517102246.4070184-6-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-ORIG-GUID: Jh1SCqI9eZ_p6mNvvejngkVIucr3CNkY X-Proofpoint-GUID: Jh1SCqI9eZ_p6mNvvejngkVIucr3CNkY X-Patchwork-Delegate: bpf@iogearbox.net When resolving BTF ids, use the BTF in the module .BTF.base section when passed the -B option. Both references to base BTF from split BTF and BTF ids will be relocated for base vmlinux on module load. Signed-off-by: Alan Maguire --- tools/bpf/resolve_btfids/main.c | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index d9520cb826b3..4f2ff52c3ea7 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -115,6 +115,7 @@ struct object { const char *path; const char *btf; const char *base_btf_path; + int base; struct { int fd; @@ -527,22 +528,37 @@ static int symbols_resolve(struct object *obj) int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; struct btf *base_btf = NULL; - int err, type_id; + int err = 0, type_id; struct btf *btf; __u32 nr_types; if (obj->base_btf_path) { - base_btf = btf__parse(obj->base_btf_path, NULL); - err = libbpf_get_error(base_btf); + LIBBPF_OPTS(btf_parse_opts, optp); + const char *path; + + if (obj->base) { + optp.btf_sec = BTF_BASE_ELF_SEC; + path = obj->path; + base_btf = btf__parse_opts(path, &optp); + /* fall back to normal base parsing if no BTF_BASE_ELF_SEC */ + } + if (!base_btf) { + optp.btf_sec = BTF_ELF_SEC; + path = obj->base_btf_path; + base_btf = btf__parse_opts(path, &optp); + } + if (!base_btf) + err = -errno; if (err) { pr_err("FAILED: load base BTF from %s: %s\n", - obj->base_btf_path, strerror(-err)); + path, strerror(-err)); return -1; } } btf = btf__parse_split(obj->btf ?: obj->path, base_btf); - err = libbpf_get_error(btf); + if (!btf) + err = -errno; if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); @@ -781,6 +797,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_STRING('b', "btf_base", &obj.base_btf_path, "file", "path of file providing base BTF"), + OPT_INCR('B', "base", &obj.base, + "use " BTF_BASE_ELF_SEC " ELF section BTF as base"), OPT_END() }; int err = -1; From patchwork Fri May 17 10:22:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666834 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1154374D2 for ; Fri, 17 May 2024 10:23:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941430; cv=none; b=DIKbAqPy8ytOVMD47y3MSV427pvegUbQknPMDcI9JExDH9aRw5c7SHqY5e3FXzbvtN6a/wlI5x7AVvmxlSbAdKoL/foN8dZHk81FFI4/r2vRy/vcyazbSmpMyfmanr1Lpl4fvGfuT9BjPMHR+/fmstZVoLw4of2x3pvtFBDbMZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941430; c=relaxed/simple; bh=WzkgXZptTdWjHkDVG6m9RG3J7L/zsTAQSp+f4G1tW0s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lItpOaa0HQ2m4IWOet316aBKu1JEeWOxSuC0gXZvKbtlu/m3Z+dJAqq0urMnuxMDb9mdVlCaH+QBz+UMmgryRW+PEwkVhp98W+KVmLYbxSP5guzDOby/qUyi2GcUXowSp+g06WsXTXaTb7LJ8b83eXJjZqde0+SwlDWS9YCVUII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=BuN9NhZd; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="BuN9NhZd" Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H7naVq013696; Fri, 17 May 2024 10:23:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=q65yVUupx7svNlcNd4D8Q2Kl9eKAp2/EdU6z1PrTsrE=; b=BuN9NhZdkQRN5kpzh9GuuEs1yy0MI+f22zSyzNf3DXNPzmRwqmapcoKuzSY0bYF8tkYz fIZpLUJHccIsT8fR9MSrtpYKW0Y9B6FKlY/uqF0YvVdgTN3PdJ8oOO6654m1oCmTHloT IllHP6/XzM4fxFuOUEJMOkDddxtvfyjXTzfYIXJzf5QLleUXngLh7ui1LHQ2S4Jw5D/W B+VLolhOH5d9Op52Z1FuASWMINe0mFMgk7xHBbfXrbK5VSJQLWnbYAf8UCXM001X7t0k CS1xDyq1JQ7vg6y71f3p1Pm7EpbnljFFJDf9GNxXL8+z7cN1lP/wx+JoCSiELS/qg+/p +g== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3rh7hmpe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:27 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8qw5O001269; Fri, 17 May 2024 10:23:27 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqs6q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:27 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFk036134; Fri, 17 May 2024 10:23:26 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-7; Fri, 17 May 2024 10:23:26 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 06/11] kbuild, bpf: add module-specific pahole/resolve_btfids flags for distilled base BTF Date: Fri, 17 May 2024 11:22:41 +0100 Message-Id: <20240517102246.4070184-7-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-ORIG-GUID: 7n9Ipo_I9VCkMt0DkxYSS4RDtxkLGBYs X-Proofpoint-GUID: 7n9Ipo_I9VCkMt0DkxYSS4RDtxkLGBYs X-Patchwork-Delegate: bpf@iogearbox.net Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids uses the "-B" option to specify that the BTF.ids section should be populated with split BTF relative to the added .BTF.base section rather than relative to the vmlinux base. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire --- scripts/Makefile.btf | 7 +++++++ scripts/Makefile.modfinal | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index bca8a8f26ea4..9f1e3163e718 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -21,8 +21,15 @@ else # Switch to using --btf_features for v1.26 and later. pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func +ifneq ($(KBUILD_EXTMOD),) +module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base +module-resolve-btfids-flags-$(call test-ge, $(pahole-ver), 126) = -B +endif + endif pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust export PAHOLE_FLAGS := $(pahole-flags-y) +export MODULE_PAHOLE_FLAGS := $(module-pahole-flags-y) +export MODULE_RESOLVE_BTFIDS_FLAGS := $(module-resolve-btfids-flags-y) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 79fcf2731686..86189ddf39e8 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -39,8 +39,8 @@ quiet_cmd_btf_ko = BTF [M] $@ if [ ! -f vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \ - $(RESOLVE_BTFIDS) -b vmlinux $@; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) $(MODULE_RESOLVE_BTFIDS_FLAGS) -b vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies From patchwork Fri May 17 10:22:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666835 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C001364BE for ; Fri, 17 May 2024 10:23:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941438; cv=none; b=gMPc6W0pX+sZxX6TDQqHta4Ko8F4OhdBQmtFJB39jTeGP2vEmGrzR5ZJ8n3RaoV5CO2xhKdNvftvWIyJiuj3aGndEXjH7ydgWUQpd/xAIGkyF5AXblD4zHFi09NlwZTySE5wF6/s6cf5Ock3tiF/xq+7LkoCpEpe/E45TchufRA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941438; c=relaxed/simple; bh=vjP75I9vBEYz3gHe8evFH1NrRmjB191s37yFmquVqQE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qh5T3kIuL1RDhkEoevEZcHusZ6gwOPLRZ13S13Zt0jQ7omJHRs2SjfasELyoE5GxMvTXRhH0E5n678DAt6oR1VwzPeKCX0fQdUKJg5W4+/LRi8Vbq4GbbcDvV7ZlGYJ/EGbiACQiFgFZyoF/kBQlW10NjLFFT1W2fJ5ToI64E/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=FWwU8GYI; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="FWwU8GYI" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H83Nq7013515; Fri, 17 May 2024 10:23:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=D+iQ4WaQQEoyoQmMB7u63DB7Fi+HVS3ZTBL84y3w3W0=; b=FWwU8GYIE+ONBLAMK0E90Zw7j4hX1Tg0Kl1DSdrVQDbGw7h34EOfYic4tlSSsxw7OWlq zYNGUmkLZlX2qkdoi/YBFZCumEjXtKWuZzNthzMHj+Kd9E8Vl7yjgUUpCEekwerDWflk peprR/veGfPTa3rRkd8a84rDYDdRCYsblymDXJR0BtCjD+BDvetJXHF7ysCqpRkzu8Nh cAuN9RAQbljeDZjSjEro8t+wTxgXMj/nfwCcnIerNEUD1MKPnB7GNz0RS8U3YRv9zR2s FvoWBVSfwo9OPf7QcXppR42SzxcluD9FHX2Cc1e4Eu/UNYGtmdq6nl6cNi4edB+GTsfp Vw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3tx399yt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:33 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44HA13SW000529; Fri, 17 May 2024 10:23:32 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqs8g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:32 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFm036134; Fri, 17 May 2024 10:23:31 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-8; Fri, 17 May 2024 10:23:31 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 07/11] libbpf: split BTF relocation Date: Fri, 17 May 2024 11:22:42 +0100 Message-Id: <20240517102246.4070184-8-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: mQ3nIThklVZ7SF0v_y7j1-l3u80Ev7d3 X-Proofpoint-ORIG-GUID: mQ3nIThklVZ7SF0v_y7j1-l3u80Ev7d3 X-Patchwork-Delegate: bpf@iogearbox.net Map distilled base BTF type ids referenced in split BTF and their references to the base BTF passed in, and if the mapping succeeds, reparent the split BTF to the base BTF. Relocation is done by first verifying that distilled base BTF only consists of named INT, FLOAT, ENUM, FWD, STRUCT and UNION kinds; then we sort these to speed lookups. Once sorted, the base BTF is iterated, and for each relevant kind we check for an equivalent in distilled base BTF. When found, the mapping from distilled -> base BTF id and string offset is recorded. Once all mappings are established, we can update type ids and string offsets in split BTF and reparent it to the new base. Signed-off-by: Alan Maguire --- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 17 ++ tools/lib/bpf/btf.h | 8 + tools/lib/bpf/btf_relocate.c | 318 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_internal.h | 3 + 6 files changed, 348 insertions(+), 1 deletion(-) create mode 100644 tools/lib/bpf/btf_relocate.c diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index b6619199a706..336da6844d42 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o + usdt.o zip.o elf.o features.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index feba071087a5..a4d6c46cc251 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5626,3 +5626,20 @@ int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, btf__free(new_base); return libbpf_err(err); } + +const struct btf_header *btf_header(const struct btf *btf) +{ + return btf->hdr; +} + +void btf_set_base_btf(struct btf *btf, const struct btf *base_btf) +{ + btf->base_btf = (struct btf *)base_btf; + btf->start_id = btf__type_cnt(base_btf); + btf->start_str_off = base_btf->hdr->str_len; +} + +int btf__relocate(struct btf *btf, const struct btf *base_btf) +{ + return btf_relocate(btf, base_btf, NULL); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e1702ad5ef4..f75db650e426 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -282,6 +282,14 @@ struct btf_dedup_opts { LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts); +/** + * @brief **btf__relocate()** will check the split BTF *btf* for references + * to base BTF kinds, and verify those references are compatible with + * *base_btf*; if they are, *btf* is adjusted such that is re-parented to + * *base_btf* and type ids and strings are adjusted to accommodate this. + */ +LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf); + struct btf_dump; struct btf_dump_opts { diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c new file mode 100644 index 000000000000..c06851f05472 --- /dev/null +++ b/tools/lib/bpf/btf_relocate.c @@ -0,0 +1,318 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include "btf.h" +#include "bpf.h" +#include "libbpf.h" +#include "libbpf_internal.h" + +struct btf; + +struct btf_relocate { + __u32 search_id; /* must be first field; see search below */ + struct btf *btf; + const struct btf *base_btf; + const struct btf *dist_base_btf; + unsigned int nr_base_types; + unsigned int nr_split_types; + unsigned int nr_dist_base_types; + int str_start; + int str_diff; + __u32 *map; + __u32 *str_map; + __u32 *dist_base_index; +}; + +static int btf_relocate_rewrite_type_id(__u32 *id, void *ctx) +{ + struct btf_relocate *r = ctx; + + *id = r->map[*id]; + return 0; +} + +/* Simple string comparison used for sorting within BTF, since all distilled types are + * named. + */ +static int cmp_btf_types(const void *id1, const void *id2, void *priv) +{ + const struct btf *btf = priv; + const struct btf_type *t1 = btf_type_by_id(btf, *(__u32 *)id1); + const struct btf_type *t2 = btf_type_by_id(btf, *(__u32 *)id2); + + return strcmp(btf__name_by_offset(btf, t1->name_off), + btf__name_by_offset(btf, t2->name_off)); +} + +/* Comparison between base BTF type (search type) and distilled base types (target). + * Because there is no bsearch_r() we need to use the search key - which also is + * the first element of struct btf_relocate * - as a means to retrieve the + * struct btf_relocate *. + */ +static int cmp_base_and_distilled_btf_types(const void *idbase, const void *iddist) +{ + struct btf_relocate *r = (struct btf_relocate *)idbase; + const struct btf_type *tbase = btf_type_by_id(r->base_btf, *(__u32 *)idbase); + const struct btf_type *tdist = btf_type_by_id(r->dist_base_btf, *(__u32 *)iddist); + + return strcmp(btf__name_by_offset(r->base_btf, tbase->name_off), + btf__name_by_offset(r->dist_base_btf, tdist->name_off)); +} + +/* Build a map from distilled base BTF ids to base BTF ids. To do so, iterate + * through base BTF looking up distilled type (using binary search) equivalents. + */ +static int btf_relocate_map_distilled_base(struct btf_relocate *r) +{ + struct btf_type *t; + const char *name; + __u32 id; + + /* generate a sort index array of type ids sorted by name for distilled + * base BTF to speed lookups. + */ + for (id = 1; id < r->nr_dist_base_types; id++) + r->dist_base_index[id] = id; + qsort_r(r->dist_base_index, r->nr_dist_base_types, sizeof(__u32), cmp_btf_types, + (struct btf *)r->dist_base_btf); + + for (id = 1; id < r->nr_base_types; id++) { + struct btf_type *dist_t; + int dist_kind, kind; + bool compat_kind; + __u32 *dist_id; + + t = btf_type_by_id(r->base_btf, id); + kind = btf_kind(t); + /* distilled base consists of named types only. */ + if (!t->name_off) + continue; + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_FWD: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + break; + default: + continue; + } + r->search_id = id; + dist_id = bsearch(&r->search_id, r->dist_base_index, r->nr_dist_base_types, + sizeof(__u32), cmp_base_and_distilled_btf_types); + if (!dist_id) + continue; + if (!*dist_id || *dist_id > r->nr_dist_base_types) { + pr_warn("base BTF id [%d] maps to invalid distilled base BTF id [%d]\n", + id, *dist_id); + return -EINVAL; + } + /* validate that kinds are compatible */ + dist_t = btf_type_by_id(r->dist_base_btf, *dist_id); + dist_kind = btf_kind(dist_t); + name = btf__name_by_offset(r->dist_base_btf, dist_t->name_off); + compat_kind = dist_kind == kind; + if (!compat_kind) { + switch (dist_kind) { + case BTF_KIND_FWD: + compat_kind = kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION; + break; + case BTF_KIND_ENUM: + compat_kind = kind == BTF_KIND_ENUM64; + break; + default: + break; + } + if (!compat_kind) { + pr_warn("kind incompatibility (%d != %d) between distilled base type '%s'[%d] and base type [%d]\n", + dist_kind, kind, name, *dist_id, id); + return -EINVAL; + } + } + /* validate that int, float struct, union sizes are compatible; + * distilled base BTF encodes an empty STRUCT/UNION with + * specific size for cases where a type is embedded in a split + * type (so has to preserve size info). Do not error out + * on mismatch as another size match may occur for an + * identically-named type. + */ + switch (btf_kind(dist_t)) { + case BTF_KIND_INT: + if (*(__u32 *)(t + 1) != *(__u32 *)(dist_t + 1)) + continue; + if (t->size != dist_t->size) + continue; + break; + case BTF_KIND_FLOAT: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + if (t->size != dist_t->size) + continue; + break; + default: + break; + } + /* map id and name */ + r->map[*dist_id] = id; + r->str_map[dist_t->name_off] = t->name_off; + } + /* ensure all distilled BTF ids have a mapping... */ + for (id = 1; id < r->nr_dist_base_types; id++) { + if (r->map[id]) + continue; + t = btf_type_by_id(r->dist_base_btf, id); + name = btf__name_by_offset(r->dist_base_btf, t->name_off); + pr_warn("distilled base BTF type '%s' [%d] is not mapped to base BTF id\n", + name, id); + return -EINVAL; + } + return 0; +} + +/* distilled base should only have named int/float/enum/fwd/struct/union types. */ +static int btf_relocate_validate_distilled_base(struct btf_relocate *r) +{ + unsigned int i; + + for (i = 1; i < r->nr_dist_base_types; i++) { + struct btf_type *t = btf_type_by_id(r->dist_base_btf, i); + int kind = btf_kind(t); + + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_FWD: + if (t->name_off) + break; + pr_warn("type [%d], kind [%d] is invalid for distilled base BTF; it is anonymous\n", + i, kind); + return -EINVAL; + default: + pr_warn("type [%d] in distilled based BTF has unexpected kind [%d]\n", + i, kind); + return -EINVAL; + } + } + return 0; +} + +static int btf_rewrite_strs(__u32 *str_off, void *ctx) +{ + struct btf_relocate *r = ctx; + int off; + + if (!*str_off) + return 0; + if (*str_off >= r->str_start) { + *str_off += r->str_diff; + } else { + off = r->str_map[*str_off]; + if (!off) { + pr_warn("string '%s' [offset %d] is not mapped to base BTF", + btf__str_by_offset(r->btf, off), *str_off); + return -ENOENT; + } + *str_off = off; + } + return 0; +} + +static int btf_relocate_finalize(struct btf_relocate *r) +{ + const struct btf_header *dist_base_hdr; + const struct btf_header *base_hdr; + struct btf_type *t; + int i, err; + + dist_base_hdr = btf_header(r->dist_base_btf); + base_hdr = btf_header(r->base_btf); + r->str_start = dist_base_hdr->str_len; + r->str_diff = base_hdr->str_len - dist_base_hdr->str_len; + for (i = 0; i < r->nr_split_types; i++) { + t = btf_type_by_id(r->btf, i + r->nr_dist_base_types); + err = btf_type_visit_str_offs(t, btf_rewrite_strs, r); + if (err) + break; + } + btf_set_base_btf(r->btf, r->base_btf); + + return err; +} + +/* If successful, output of relocation is updated BTF with base BTF pointing + * at base_btf, and type ids, strings adjusted accordingly + */ +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids) +{ + unsigned int nr_types = btf__type_cnt(btf); + struct btf_relocate r = {}; + struct btf_type *t; + int diff_id, err = 0; + __u32 id, i; + + r.dist_base_btf = btf__base_btf(btf); + if (!base_btf || r.dist_base_btf == base_btf) + return 0; + + r.nr_dist_base_types = btf__type_cnt(r.dist_base_btf); + r.nr_base_types = btf__type_cnt(base_btf); + r.nr_split_types = nr_types - r.nr_dist_base_types; + r.btf = btf; + r.base_btf = base_btf; + + r.map = calloc(nr_types, sizeof(*r.map)); + r.str_map = calloc(btf_header(r.dist_base_btf)->str_len, sizeof(*r.str_map)); + r.dist_base_index = calloc(r.nr_dist_base_types, sizeof(*r.dist_base_index)); + if (!r.map || !r.str_map || !r.dist_base_index) { + err = -ENOMEM; + goto err_out; + } + + err = btf_relocate_validate_distilled_base(&r); + if (err) + goto err_out; + + diff_id = r.nr_base_types - r.nr_dist_base_types; + /* Split BTF ids will start from after last base BTF id. */ + for (id = r.nr_dist_base_types; id < nr_types; id++) + r.map[id] = id + diff_id; + + /* Build a map from distilled base ids to actual base BTF ids; it is used + * to update split BTF id references. + */ + err = btf_relocate_map_distilled_base(&r); + if (err) + goto err_out; + + /* Next, rewrite type ids in split BTF, replacing split ids with updated + * ids based on number of types in base BTF, and base ids with + * relocated ids from base_btf. + */ + for (i = 0, id = r.nr_dist_base_types; i < r.nr_split_types; i++, id++) { + t = btf_type_by_id(btf, id); + err = btf_type_visit_type_ids(t, btf_relocate_rewrite_type_id, &r); + if (err) + goto err_out; + } + /* Finally reset base BTF to base_btf; as part of this operation, string + * offsets are also updated, and we are done. + */ + err = btf_relocate_finalize(&r); +err_out: + if (!err && map_ids) + *map_ids = r.map; + else + free(r.map); + free(r.str_map); + free(r.dist_base_index); + return err; +} diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index fd7bfeaba542..849cbe0def00 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -421,6 +421,7 @@ LIBBPF_1.5.0 { global: btf__distill_base; btf__parse_opts; + btf__relocate; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index a0dcfb82e455..3b98edae254f 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -234,6 +234,9 @@ struct btf_type; struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id); const char *btf_kind_str(const struct btf_type *t); const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id); +const struct btf_header *btf_header(const struct btf *btf); +void btf_set_base_btf(struct btf *btf, const struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t) { From patchwork Fri May 17 10:22:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666837 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8AA0364BE for ; Fri, 17 May 2024 10:24:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941444; cv=none; b=sRGlANZc++xrCtWmP0VsqFcDaGFPLdo6UAH5ymOevQybwHg3O6drBuG+2P3g2BxkuyN5pAezJ99QzeGcdR8bOpQ1ZnpNSCSDIzlidRaF9ek+AdkmGtqDLO7mM0zh2f0XNUjfRf9hshg/ixffI9SXjpsth9SZfsTw5LZd3OnAW8Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941444; c=relaxed/simple; bh=DiS6lwhbi6MtEiLrvEax32w8r/IwjnZcNvxoRJTS8l4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dxzmUM6zGsbg2RQMcUEmvh5LEEoFoCLE9bb+k6stvBOJ0pxI/oHBMZERiwwxRftBSqo6Un2OvM2EAZl46+9vWiVEMZk36KhmLTMBE5hBL5j87RuZLESk0VADG9VlaOfzizt4L4C8JWmxV6Ln6DProvbHB5ZBsjVKGcUO6ZxYkjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KxAplLWk; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KxAplLWk" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H9doDN026934; Fri, 17 May 2024 10:23:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=Xrqxmo/cOr7wDIK2H4sXnWi4svfwfASWxHqpe9g3XUk=; b=KxAplLWk3N7doxzyTkNsFAMFqpA4xE31au/NjdUPREDRRrMzZ7baYhSaSZ3qxbfCMXIF BFen52MutblSfzQaXGMy6uskt60smqlGthk89mtrUtTqwLQLd5+hJ1/7FuEem2QIrfeN P4EE0W+YU7i+u8S4a4ogiIx9o2r7kYToQSH1+GyzjsOKCQc/K7lG4zvfWXCf1gWVVMmz URi0XEv1huFkjdtqRt031TuLwZkCbHJ0j/++YP+lS9uccTtVN2p3haNN6ooBV6hil1jB wwPyjhbEL4WANUSf9t2cYq7hmoUEag09cKYbxSj2ng6A9FqaxIWeLOexCdvuyEL8/KPq Jg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y4c8r75vy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:38 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44HAAoBF000425; Fri, 17 May 2024 10:23:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqsa2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:37 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFo036134; Fri, 17 May 2024 10:23:37 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-9; Fri, 17 May 2024 10:23:36 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 08/11] selftests/bpf: extend distilled BTF tests to cover BTF relocation Date: Fri, 17 May 2024 11:22:43 +0100 Message-Id: <20240517102246.4070184-9-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-ORIG-GUID: Hx6-gf5UlTYNw6gOPDNg6aTI3mtjCztL X-Proofpoint-GUID: Hx6-gf5UlTYNw6gOPDNg6aTI3mtjCztL X-Patchwork-Delegate: bpf@iogearbox.net Ensure relocated BTF looks as expected; in this case identical to original split BTF, with a few duplicate anonymous types added to split BTF by the relocation process. Signed-off-by: Alan Maguire Acked-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/btf_distill.c | 67 +++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c index 4ab522df041c..442627719edc 100644 --- a/tools/testing/selftests/bpf/prog_tests/btf_distill.c +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -218,6 +218,73 @@ static void test_distilled_base(void) "\t'p1' type_id=1", "[25] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + if (!ASSERT_EQ(btf__relocate(btf4, btf1), 0, "relocate_split")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] UNION 'u1' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[18] PTR '(anon)' type_id=3", + "[19] PTR '(anon)' type_id=30", + "[20] CONST '(anon)' type_id=6", + "[21] RESTRICT '(anon)' type_id=31", + "[22] VOLATILE '(anon)' type_id=8", + "[23] TYPEDEF 'et' type_id=32", + "[24] CONST '(anon)' type_id=10", + "[25] PTR '(anon)' type_id=33", + "[26] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[27] FUNC 'fn' type_id=34 linkage=static", + "[28] TYPEDEF 'arraytype' type_id=35", + "[29] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16", + /* below here are (duplicate) anon base types added by distill + * process to split BTF. + */ + "[30] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[31] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[32] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[33] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[34] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[35] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + cleanup: btf__free(btf4); btf__free(btf3); From patchwork Fri May 17 10:22:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666838 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2C8D364AB for ; Fri, 17 May 2024 10:24:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941449; cv=none; b=lEXkfRi134rxacFVxkp/M+XKZOMJc0+vWF20FBAdN2U9VS2YMEEEL0rfmQs3cSiTmFosCiUGRQ5oMykRBRTpaQM1BR5egc/7SLdrrCAg8OCxa9ZlqNUrxS1D48nadoyswMHS8vgZjoejpPdMRqA3VvcOw1fq2PdsZKsEeMbJzL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941449; c=relaxed/simple; bh=cBcfmcVBrgf6azkPS8GIkEODkINDbJ+MQWMXCGDO06I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tJe/w30DyS1PHXGhEQEBIDlWVn/mObv1KUysA8XKsfF1Quig2/HQFcgxQh9H2VXylirD6+IyguBjWdavfc0+u3TZ6Vr2LhrezfO1Wec0dL5YKuHJtj8rKlzg1rSIbaYxKe7IY17oEoNWlNFs31B0Z9odtDnkGC4lAsgUQpvQWgE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=nLs91guR; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="nLs91guR" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H6dJiS007909; Fri, 17 May 2024 10:23:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=NLTtqzFIHAlwjSGVFNQNXoDScRZ3Sw+F0BFfEuNRB7Q=; b=nLs91guRJzc04FF1s6nOoVKBPl+oilYcY4VjRDKpP/d4Mc/t6X+j7mofWaFApZCeg088 o4BijAXVKY161y5OlysAOwoD+7/Cf3Wx3vv0zj7C1wzJlLxc8RbL4dJHdXprmwtBygmx D57EeXjM/2NC+gxO8RqznQH7mbk1nRY8oEk6+dGGIa/V+A0zV4Domr4Yl0L5JMhUVl+x 4qG/ug0MV/fx1auHMqckT/z2v0uDemSIRU41vdtDn8z0vqD1CHJshsYZABPjm3Wxh0Gh uEChGqRBojVUk0VKwROAeem7GX/D7YlRfpqyCrsFEIaQ4s0RF8ScJ190t73UB7XJso4H nA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3tx39a01-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:44 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8wsAT001127; Fri, 17 May 2024 10:23:43 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqsc4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:43 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFq036134; Fri, 17 May 2024 10:23:42 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-10; Fri, 17 May 2024 10:23:42 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 09/11] module, bpf: store BTF base pointer in struct module Date: Fri, 17 May 2024 11:22:44 +0100 Message-Id: <20240517102246.4070184-10-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: gk_gf3-iOqMOyEtj24GH3o7fM7i7PVyk X-Proofpoint-ORIG-GUID: gk_gf3-iOqMOyEtj24GH3o7fM7i7PVyk X-Patchwork-Delegate: bpf@iogearbox.net ...as this will allow split BTF modules with a base BTF representation (rather than the full vmlinux BTF at time of BTF encoding) to resolve their references to kernel types in a way that is more resilient to small changes in kernel types. This will allow modules that are not built every time the kernel is to provide more resilient BTF, rather than have it invalidated every time BTF ids for core kernel types change. Signed-off-by: Alan Maguire --- include/linux/module.h | 2 ++ kernel/module/main.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/module.h b/include/linux/module.h index 1153b0d99a80..f127a79a95d9 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -510,6 +510,8 @@ struct module { #ifdef CONFIG_DEBUG_INFO_BTF_MODULES unsigned int btf_data_size; void *btf_data; + unsigned int btf_base_data_size; + void *btf_base_data; #endif #ifdef CONFIG_JUMP_LABEL struct jump_entry *jump_entries; diff --git a/kernel/module/main.c b/kernel/module/main.c index e1e8a7a9d6c1..e18683abec07 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2148,6 +2148,8 @@ static int find_module_sections(struct module *mod, struct load_info *info) #endif #ifdef CONFIG_DEBUG_INFO_BTF_MODULES mod->btf_data = any_section_objs(info, ".BTF", 1, &mod->btf_data_size); + mod->btf_base_data = any_section_objs(info, ".BTF.base", 1, + &mod->btf_base_data_size); #endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries = section_objs(info, "__jump_table", @@ -2587,8 +2589,9 @@ static noinline int do_init_module(struct module *mod) } #ifdef CONFIG_DEBUG_INFO_BTF_MODULES - /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointer */ + /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointers */ mod->btf_data = NULL; + mod->btf_base_data = NULL; #endif /* * We want to free module_init, but be aware that kallsyms may be From patchwork Fri May 17 10:22:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666839 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A041E364BE for ; Fri, 17 May 2024 10:24:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941455; cv=none; b=N2frfvNOqRq8psMgAr3aZw4qEwNgj86Ofe2WB5MAqGEsjUe4VYsFCEZNh73jnkF9THzcmNYlvtAgRzi9nBU8wVdo0uuyZXQVS4nA24jFh13orrkazwGaOVn/QygebedUu4jnrJ3AwCmzsa4jr1iz/yzMw/rzAMMkbFM6OrPh7CI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941455; c=relaxed/simple; bh=3L4Xd/7h4rs1mNscDXbgIoDOjSyoZ981oNioNIQpWMM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=md9PDICzjfnPG1Jy6wr+em1RTw2/d/h1Kebm7LvCW8SXiUewe1ejPTUsxc1hbcJS2T6M9F7DeaVXlM+07cxDJTG0HZEPcHS/JSEvkTzoXeqf8eIv+hE7a1xiHA8svNvpyw4AEs2UUGTOVf60eutflD0/yuvrmWvvHKdJFaeBwn4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ZKSft3Fl; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ZKSft3Fl" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H7edAZ000878; Fri, 17 May 2024 10:23:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=VUgbFYtQDCo3T5hz1pND9iBxOW613nMWMWJpHs6gevE=; b=ZKSft3Fl6TGMu2dcWWQ6t7qwM5Zzzb2cvs9YmAbh8omfsVQ6FI/5iP9jrHqcXLEDnS19 /eZpXme1FFFbBriaqFkUo8ixp2CQa3Ab0R50h6T6evfoXwsjkeSF6e1MPQM2FvNTdMhI isIFg28r5kYDbdi1T7qrWPIJBmCqixXSVpwAr+8AMzbB8YnZhlAF0TXi5SCbpJPF2alm f9nsIYsxTl8wlWBqB7EgI7O/ZAmOqNvVu25KQjqKEEydZMJi1VLQQu5mbNw9Wr2AI60Y QR2i3ye4UXM6b6k5orGyOhpPQ/55DISBihUkdq7WarRK0PfQ5IuIDR2mfXzxfGXgtei9 AQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3txc7k36-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:49 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8iBZP000491; Fri, 17 May 2024 10:23:48 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqse2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:48 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFs036134; Fri, 17 May 2024 10:23:47 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-11; Fri, 17 May 2024 10:23:47 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v4 bpf-next 10/11] libbpf,bpf: share BTF relocate-related code with kernel Date: Fri, 17 May 2024 11:22:45 +0100 Message-Id: <20240517102246.4070184-11-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: hvhziwSxy-mJ0K9YlqUQ29lx1HT25UGM X-Proofpoint-ORIG-GUID: hvhziwSxy-mJ0K9YlqUQ29lx1HT25UGM X-Patchwork-Delegate: bpf@iogearbox.net Share relocation implementation with the kernel. As part of this, we also need the type/string visitation functions so add them to a btf_common.c file that also gets shared with the kernel. Relocation code in kernel and userspace is identical save for the impementation of the reparenting of split BTF to the relocated base BTF and retrieval of BTF header from "struct btf"; these small functions need separate user-space and kernel implementations. One other wrinkle on the kernel side is we have to map .BTF.ids in modules as they were generated with the type ids used at BTF encoding time. btf_relocate() optionally returns an array mapping from old BTF ids to relocated ids, so we use that to fix up these references where needed for kfuncs. Signed-off-by: Alan Maguire --- include/linux/btf.h | 45 ++++++++++ kernel/bpf/Makefile | 8 ++ kernel/bpf/btf.c | 166 +++++++++++++++++++++++++---------- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 130 --------------------------- tools/lib/bpf/btf_common.c | 143 ++++++++++++++++++++++++++++++ tools/lib/bpf/btf_relocate.c | 23 +++++ 7 files changed, 341 insertions(+), 176 deletions(-) create mode 100644 tools/lib/bpf/btf_common.c diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..69aa760f5972 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -140,6 +140,8 @@ extern const struct file_operations btf_fops; const char *btf_get_name(const struct btf *btf); void btf_get(struct btf *btf); void btf_put(struct btf *btf); +const struct btf_header *btf_header(const struct btf *btf); +void btf_set_base_btf(struct btf *btf, struct btf *base_btf); int btf_new_fd(const union bpf_attr *attr, bpfptr_t uattr, u32 uattr_sz); struct btf *btf_get_by_fd(int fd); int btf_get_info_by_fd(const struct btf *btf, @@ -214,6 +216,7 @@ bool btf_is_kernel(const struct btf *btf); bool btf_is_module(const struct btf *btf); struct module *btf_try_get_module(const struct btf *btf); u32 btf_nr_types(const struct btf *btf); +struct btf *btf_base_btf(const struct btf *btf); bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s, const struct btf_member *m, u32 expected_offset, u32 expected_size); @@ -339,6 +342,11 @@ static inline u8 btf_int_offset(const struct btf_type *t) return BTF_INT_OFFSET(*(u32 *)(t + 1)); } +static inline __u8 btf_int_bits(const struct btf_type *t) +{ + return BTF_INT_BITS(*(__u32 *)(t + 1)); +} + static inline bool btf_type_is_scalar(const struct btf_type *t) { return btf_type_is_int(t) || btf_type_is_enum(t); @@ -478,6 +486,11 @@ static inline struct btf_param *btf_params(const struct btf_type *t) return (struct btf_param *)(t + 1); } +static inline struct btf_decl_tag *btf_decl_tag(const struct btf_type *t) +{ + return (struct btf_decl_tag *)(t + 1); +} + static inline int btf_id_cmp_func(const void *a, const void *b) { const int *pa = a, *pb = b; @@ -515,9 +528,17 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf * } #endif +typedef int (*type_id_visit_fn)(__u32 *type_id, void *ctx); +typedef int (*str_off_visit_fn)(__u32 *str_off, void *ctx); + #ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); +void btf_set_base_btf(struct btf *btf, struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx); +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx); const char *btf_name_by_offset(const struct btf *btf, u32 offset); +const char *btf_str_by_offset(const struct btf *btf, u32 offset); struct btf *btf_parse_vmlinux(void); struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog); u32 *btf_kfunc_id_set_contains(const struct btf *btf, u32 kfunc_btf_id, @@ -543,6 +564,30 @@ static inline const struct btf_type *btf_type_by_id(const struct btf *btf, { return NULL; } + +static inline void btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + return; +} + +static inline int btf_relocate(void *log, struct btf *btf, const struct btf *base_btf, + __u32 **map_ids) +{ + return 0; +} + +static inline int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, + void *ctx) +{ + return 0; +} + +static inline int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, + void *ctx) +{ + return 0; +} + static inline const char *btf_name_by_offset(const struct btf *btf, u32 offset) { diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 7eb9ad3a3ae6..612eef1228ca 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -52,3 +52,11 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o $(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_common.o +$(obj)/btf_common.o: $(srctree)/tools/lib/bpf/btf_common.c FORCE + $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +$(obj)/btf_relocate.o: $(srctree)/tools/lib/bpf/btf_relocate.c FORCE + $(call if_changed_rule,cc_o_c) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 821063660d9f..ebc127da4d79 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -274,6 +274,7 @@ struct btf { u32 start_str_off; /* first string offset (0 for base BTF) */ char name[MODULE_NAME_LEN]; bool kernel_btf; + __u32 *base_map; /* map from distilled base BTF -> vmlinux BTF ids */ }; enum verifier_phase { @@ -530,6 +531,11 @@ static bool btf_type_is_decl_tag_target(const struct btf_type *t) btf_type_is_var(t) || btf_type_is_typedef(t); } +static bool btf_is_vmlinux(const struct btf *btf) +{ + return btf->kernel_btf && !btf->base_btf; +} + u32 btf_nr_types(const struct btf *btf) { u32 total = 0; @@ -772,7 +778,7 @@ static bool __btf_name_char_ok(char c, bool first) return true; } -static const char *btf_str_by_offset(const struct btf *btf, u32 offset) +const char *btf_str_by_offset(const struct btf *btf, u32 offset) { while (offset < btf->start_str_off) btf = btf->base_btf; @@ -1735,7 +1741,12 @@ static void btf_free(struct btf *btf) kvfree(btf->types); kvfree(btf->resolved_sizes); kvfree(btf->resolved_ids); - kvfree(btf->data); + /* vmlinux does not allocate btf->data, it simply points it at + * __start_BTF. + */ + if (!btf_is_vmlinux(btf)) + kvfree(btf->data); + kvfree(btf->base_map); kfree(btf); } @@ -1764,6 +1775,23 @@ void btf_put(struct btf *btf) } } +struct btf *btf_base_btf(const struct btf *btf) +{ + return btf->base_btf; +} + +const struct btf_header *btf_header(const struct btf *btf) +{ + return &btf->hdr; +} + +void btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + btf->base_btf = base_btf; + btf->start_id = btf_nr_types(base_btf); + btf->start_str_off = base_btf->hdr.str_len; +} + static int env_resolve_init(struct btf_verifier_env *env) { struct btf *btf = env->btf; @@ -5982,23 +6010,15 @@ int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_ty BTF_ID_LIST(bpf_ctx_convert_btf_id) BTF_ID(struct, bpf_ctx_convert) -struct btf *btf_parse_vmlinux(void) +static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name, + void *data, unsigned int data_size) { - struct btf_verifier_env *env = NULL; - struct bpf_verifier_log *log; struct btf *btf = NULL; int err; if (!IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) return ERR_PTR(-ENOENT); - env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); - if (!env) - return ERR_PTR(-ENOMEM); - - log = &env->log; - log->level = BPF_LOG_KERNEL; - btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6006,10 +6026,10 @@ struct btf *btf_parse_vmlinux(void) } env->btf = btf; - btf->data = __start_BTF; - btf->data_size = __stop_BTF - __start_BTF; + btf->data = data; + btf->data_size = data_size; btf->kernel_btf = true; - snprintf(btf->name, sizeof(btf->name), "vmlinux"); + snprintf(btf->name, sizeof(btf->name), "%s", name); err = btf_parse_hdr(env); if (err) @@ -6029,20 +6049,11 @@ struct btf *btf_parse_vmlinux(void) if (err) goto errout; - /* btf_parse_vmlinux() runs under bpf_verifier_lock */ - bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); - refcount_set(&btf->refcnt, 1); - err = btf_alloc_id(btf); - if (err) - goto errout; - - btf_verifier_env_free(env); return btf; errout: - btf_verifier_env_free(env); if (btf) { kvfree(btf->types); kfree(btf); @@ -6050,19 +6061,59 @@ struct btf *btf_parse_vmlinux(void) return ERR_PTR(err); } +struct btf *btf_parse_vmlinux(void) +{ + struct btf_verifier_env *env = NULL; + struct bpf_verifier_log *log; + struct btf *btf; + int err; + + env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); + if (!env) + return ERR_PTR(-ENOMEM); + + log = &env->log; + log->level = BPF_LOG_KERNEL; + btf = btf_parse_base(env, "vmlinux", __start_BTF, __stop_BTF - __start_BTF); + if (!IS_ERR(btf)) { + /* btf_parse_vmlinux() runs under bpf_verifier_lock */ + bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); + err = btf_alloc_id(btf); + if (err) { + btf_free(btf); + btf = ERR_PTR(err); + } + } + btf_verifier_env_free(env); + return btf; +} + #ifdef CONFIG_DEBUG_INFO_BTF_MODULES -static struct btf *btf_parse_module(const char *module_name, const void *data, unsigned int data_size) +/* If .BTF_ids section was created with distilled base BTF, both base and + * split BTF ids will need to be mapped to actual base/split ids for + * BTF now that it has been relocated. + */ +static __u32 btf_id_map(const struct btf *btf, __u32 id) { + if (!btf->base_btf || !btf->base_map) + return id; + return btf->base_map[id]; +} + +static struct btf *btf_parse_module(const char *module_name, const void *data, + unsigned int data_size, void *base_data, + unsigned int base_data_size) +{ + struct btf *btf = NULL, *vmlinux_btf, *base_btf = NULL; struct btf_verifier_env *env = NULL; struct bpf_verifier_log *log; - struct btf *btf = NULL, *base_btf; - int err; + int err = 0; - base_btf = bpf_get_btf_vmlinux(); - if (IS_ERR(base_btf)) - return base_btf; - if (!base_btf) + vmlinux_btf = bpf_get_btf_vmlinux(); + if (IS_ERR(vmlinux_btf)) + return vmlinux_btf; + if (!vmlinux_btf) return ERR_PTR(-EINVAL); env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); @@ -6072,6 +6123,16 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u log = &env->log; log->level = BPF_LOG_KERNEL; + if (base_data) { + base_btf = btf_parse_base(env, ".BTF.base", base_data, base_data_size); + if (IS_ERR(base_btf)) { + err = PTR_ERR(base_btf); + goto errout; + } + } else { + base_btf = vmlinux_btf; + } + btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6111,12 +6172,22 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u if (err) goto errout; + if (base_btf != vmlinux_btf) { + err = btf_relocate(btf, vmlinux_btf, &btf->base_map); + if (err) + goto errout; + btf_free(base_btf); + base_btf = vmlinux_btf; + } + btf_verifier_env_free(env); refcount_set(&btf->refcnt, 1); return btf; errout: btf_verifier_env_free(env); + if (base_btf != vmlinux_btf) + btf_free(base_btf); if (btf) { kvfree(btf->data); kvfree(btf->types); @@ -7669,7 +7740,8 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op, err = -ENOMEM; goto out; } - btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size); + btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size, + mod->btf_base_data, mod->btf_base_data_size); if (IS_ERR(btf)) { kfree(btf_mod); if (!IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH)) { @@ -7993,7 +8065,7 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, bool add_filter = !!kset->filter; struct btf_kfunc_set_tab *tab; struct btf_id_set8 *set; - u32 set_cnt; + u32 set_cnt, i; int ret; if (hook >= BTF_KFUNC_HOOK_MAX) { @@ -8039,21 +8111,15 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, goto end; } - /* We don't need to allocate, concatenate, and sort module sets, because - * only one is allowed per hook. Hence, we can directly assign the - * pointer and return. - */ - if (!vmlinux_set) { - tab->sets[hook] = add_set; - goto do_add_filter; - } - /* In case of vmlinux sets, there may be more than one set being * registered per hook. To create a unified set, we allocate a new set * and concatenate all individual sets being registered. While each set * is individually sorted, they may become unsorted when concatenated, * hence re-sorting the final set again is required to make binary * searching the set using btf_id_set8_contains function work. + * + * For module sets, we need to allocate as we may need to relocate + * BTF ids. */ set_cnt = set ? set->cnt : 0; @@ -8083,11 +8149,14 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, /* Concatenate the two sets */ memcpy(set->pairs + set->cnt, add_set->pairs, add_set->cnt * sizeof(set->pairs[0])); + /* Now that the set is copied, update with relocated BTF ids */ + for (i = set->cnt; i < set->cnt + add_set->cnt; i++) + set->pairs[i].id = btf_id_map(btf, set->pairs[i].id); + set->cnt += add_set->cnt; sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL); -do_add_filter: if (add_filter) { hook_filter = &tab->hook_filters[hook]; hook_filter->filters[hook_filter->nr_filters++] = kset->filter; @@ -8207,7 +8276,7 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook, return PTR_ERR(btf); for (i = 0; i < kset->set->cnt; i++) { - ret = btf_check_kfunc_protos(btf, kset->set->pairs[i].id, + ret = btf_check_kfunc_protos(btf, btf_id_map(btf, kset->set->pairs[i].id), kset->set->pairs[i].flags); if (ret) goto err_out; @@ -8306,7 +8375,7 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c { struct btf_id_dtor_kfunc_tab *tab; struct btf *btf; - u32 tab_cnt; + u32 tab_cnt, i; int ret; btf = btf_get_module_btf(owner); @@ -8357,6 +8426,13 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c btf->dtor_kfunc_tab = tab; memcpy(tab->dtors + tab->cnt, dtors, add_cnt * sizeof(tab->dtors[0])); + + /* remap BTF ids based on BTF relocation (if any) */ + for (i = tab_cnt; i < tab_cnt + add_cnt; i++) { + tab->dtors[i].btf_id = btf_id_map(btf, tab->dtors[i].btf_id); + tab->dtors[i].kfunc_btf_id = btf_id_map(btf, tab->dtors[i].kfunc_btf_id); + } + tab->cnt += add_cnt; sort(tab->dtors, tab->cnt, sizeof(tab->dtors[0]), btf_id_cmp_func, NULL); diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index 336da6844d42..567abaa52131 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o btf_relocate.o + usdt.o zip.o elf.o features.o btf_common.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index a4d6c46cc251..fc2a60759244 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5026,136 +5026,6 @@ struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_bt return btf__parse_split(path, vmlinux_btf); } -int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) -{ - int i, n, err; - - switch (btf_kind(t)) { - case BTF_KIND_INT: - case BTF_KIND_FLOAT: - case BTF_KIND_ENUM: - case BTF_KIND_ENUM64: - return 0; - - case BTF_KIND_FWD: - case BTF_KIND_CONST: - case BTF_KIND_VOLATILE: - case BTF_KIND_RESTRICT: - case BTF_KIND_PTR: - case BTF_KIND_TYPEDEF: - case BTF_KIND_FUNC: - case BTF_KIND_VAR: - case BTF_KIND_DECL_TAG: - case BTF_KIND_TYPE_TAG: - return visit(&t->type, ctx); - - case BTF_KIND_ARRAY: { - struct btf_array *a = btf_array(t); - - err = visit(&a->type, ctx); - err = err ?: visit(&a->index_type, ctx); - return err; - } - - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - err = visit(&t->type, ctx); - if (err) - return err; - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_DATASEC: { - struct btf_var_secinfo *m = btf_var_secinfos(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - default: - return -EINVAL; - } -} - -int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) -{ - int i, n, err; - - err = visit(&t->name_off, ctx); - if (err) - return err; - - switch (btf_kind(t)) { - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM: { - struct btf_enum *m = btf_enum(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM64: { - struct btf_enum64 *m = btf_enum64(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - default: - break; - } - - return 0; -} - int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx) { const struct btf_ext_info *seg; diff --git a/tools/lib/bpf/btf_common.c b/tools/lib/bpf/btf_common.c new file mode 100644 index 000000000000..93f207257dce --- /dev/null +++ b/tools/lib/bpf/btf_common.c @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +/* Copyright (c) 2021 Facebook */ +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifdef __KERNEL__ +#include +#include + +#define btf_var_secinfos btf_type_var_secinfo + +#else +#include "btf.h" +#include "libbpf_internal.h" +#endif + +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) +{ + int i, n, err; + + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + return 0; + + case BTF_KIND_FWD: + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_VAR: + case BTF_KIND_DECL_TAG: + case BTF_KIND_TYPE_TAG: + return visit(&t->type, ctx); + + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + err = visit(&a->type, ctx); + err = err ?: visit(&a->index_type, ctx); + return err; + } + + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + err = visit(&t->type, ctx); + if (err) + return err; + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + case BTF_KIND_DATASEC: { + struct btf_var_secinfo *m = btf_var_secinfos(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + default: + return -EINVAL; + } +} + +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) +{ + int i, n, err; + + err = visit(&t->name_off, ctx); + if (err) + return err; + + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM: { + struct btf_enum *m = btf_enum(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM64: { + struct btf_enum64 *m = btf_enum64(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + default: + break; + } + + return 0; +} diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c index c06851f05472..a6534832296d 100644 --- a/tools/lib/bpf/btf_relocate.c +++ b/tools/lib/bpf/btf_relocate.c @@ -5,11 +5,34 @@ #define _GNU_SOURCE #endif +#ifdef __KERNEL__ +#include +#include +#include +#include +#include +#include + +#define btf_type_by_id (struct btf_type *)btf_type_by_id +#define btf__type_cnt btf_nr_types +#define btf__base_btf btf_base_btf +#define btf__name_by_offset btf_name_by_offset +#define btf__str_by_offset btf_str_by_offset +#define btf_kflag btf_type_kflag + +#define calloc(nmemb, sz) kvcalloc(nmemb, sz, GFP_KERNEL | __GFP_NOWARN) +#define free(ptr) kvfree(ptr) +#define qsort_r(base, num, sz, cmp, priv) sort_r(base, num, sz, (cmp_r_func_t)cmp, NULL, priv) + +#else + #include "btf.h" #include "bpf.h" #include "libbpf.h" #include "libbpf_internal.h" +#endif /* __KERNEL__ */ + struct btf; struct btf_relocate { From patchwork Fri May 17 10:22:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13666840 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4A8A36B09 for ; Fri, 17 May 2024 10:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941465; cv=none; b=IR/AuP2jdMfTscxyusdqm0dkw5HZ6dZ58OsDCoIuWmzZMDuYjau4VTVIjhVKPLh+9/WRzTr55q5Grmuy3dTcPurVaX55oSDuiWd/f9p07JOnqLNFr2mDmvJ+G01JfHaQy4utAY5wJfuxP/3dVRPf1n4FwYAH6N4mGFgRhtN1T/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715941465; c=relaxed/simple; bh=W2KYWf7jkaxjP5UiW9SmpJStNluQGARweN5/o9Eq4F4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ngUa4hJj9ERFHWF6YsruTmBY7WgxcHvTmwzKNeDQVnvb3Ebm7G69hIkn1+92P9FswlZ6knSz5qpgxjkRPEwPHOKD0UKTAvQKrGLKc5jP7YYQzpoz18waXgJHvllD6Ea3L+B81vPq1Dk02yaC8loSG3gWRWChPd324hBcozxRn9o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=QgxJ+8ch; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QgxJ+8ch" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8eGBM010363; Fri, 17 May 2024 10:23:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=p2UJbJlF6rRg4Jf7pqaapdQb8+hPIgQ18ebit3RnUAg=; b=QgxJ+8choiWayWwn641Bhd1c2uZ9JF9wtsF6C1bJ1ewGOiTC91XyoinXlaA/dilWDSmy fduYmM/XLnPSQkAj52WFWUzJ5l5ZxvDqbuiaROavC7NbyEwrViNb0ef4AsJS9XlJO1SX Jh0Z+sm4S1peuDIX4DVgk9EaY2+UIYcqgvY55Nz65DUHzpyP9dQYOdg8v1UCzXK1Z4Us TU0ybkLqO6CkbDasExGrjRxaCz3XogfAlk5d9fNBKbRn/IRWxdj7QWm+qI0FG66upJQz G0aKJHardYKJjrcx0lolSm+3xTuNrcBHPZ0LoAUGrsBhYQn04qzlQ0qCgI4XSa+VLnCm 1g== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y3t4rym6a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:54 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44H8UhqO000426; Fri, 17 May 2024 10:23:54 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3y4fsuqsfy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2024 10:23:54 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44HAMqFu036134; Fri, 17 May 2024 10:23:53 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-196-17.vpn.oracle.com [10.175.196.17]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3y4fsuqrr2-12; Fri, 17 May 2024 10:23:53 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire , Quentin Monnet Subject: [PATCH v4 bpf-next 11/11] bpftool: support displaying relocated-with-base split BTF Date: Fri, 17 May 2024 11:22:46 +0100 Message-Id: <20240517102246.4070184-12-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240517102246.4070184-1-alan.maguire@oracle.com> References: <20240517102246.4070184-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-17_03,2024-05-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405170082 X-Proofpoint-GUID: yWC4ceirDVrMQaDo89Tk7JC7_chTJJ5P X-Proofpoint-ORIG-GUID: yWC4ceirDVrMQaDo89Tk7JC7_chTJJ5P X-Patchwork-Delegate: bpf@iogearbox.net If the -R option is used, we can display BTF that has been generated with distilled base BTF in its relocated form. For example for bpf_testmod.ko (which is built as an out-of-tree module, so has a distilled .BTF.base section: bpftool btf dump file bpf_testmod.ko Alternatively, we can display content relocated with (a possibly changed) base BTF via bpftool btf dump -R /sys/kernel/btf/vmlinux bpf_testmod.ko The latter mirrors how the kernel will handle such split BTF; it relocates its representation with the running kernel, and if successful, renumbers BTF ids to reference the current vmlinux BTF. Signed-off-by: Alan Maguire Acked-by: Eduard Zingerman Reviewed-by: Quentin Monnet --- tools/bpf/bpftool/Documentation/bpftool-btf.rst | 15 ++++++++++++++- tools/bpf/bpftool/bash-completion/bpftool | 7 ++++--- tools/bpf/bpftool/btf.c | 11 ++++++++++- tools/bpf/bpftool/main.c | 14 +++++++++++++- tools/bpf/bpftool/main.h | 2 ++ 5 files changed, 43 insertions(+), 6 deletions(-) diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst index 3f6bca03ad2e..b11abebeae81 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst @@ -16,7 +16,7 @@ SYNOPSIS **bpftool** [*OPTIONS*] **btf** *COMMAND* -*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } } +*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } { **-R** | **relocate-base-btf** } } *COMMANDS* := { **dump** | **help** } @@ -87,6 +87,19 @@ OPTIONS BTF object is passed through other handles, this option becomes necessary. +-R, --relocate-base-btf *FILE* + When split BTF is generated with distilled base BTF for relocation, + the latter is stored in a .BTF.base section and allows us to later + relocate split BTF and a potentially-changed base BTF by using + information in the .BTF.base section about the base types referenced + from split BTF. Relocation is carried out against the split BTF + supplied via this parameter and the split BTF will then refer to + the base types supplied in *FILE*. + + If this option is not used, split BTF is shown relative to the + .BTF.base, which contains just enough information to support later + relocation. + EXAMPLES ======== **# bpftool btf dump id 1226** diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index be99d49b8714..702335848e3c 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -262,7 +262,7 @@ _bpftool() # Deal with options if [[ ${words[cword]} == -* ]]; then local c='--version --json --pretty --bpffs --mapcompat --debug \ - --use-loader --base-btf' + --use-loader --base-btf --relocate-base-btf' COMPREPLY=( $( compgen -W "$c" -- "$cur" ) ) return 0 fi @@ -283,7 +283,7 @@ _bpftool() _sysfs_get_netdevs return 0 ;; - file|pinned|-B|--base-btf) + file|pinned|-B|-R|--base-btf|--relocate-base-btf) _filedir return 0 ;; @@ -297,7 +297,8 @@ _bpftool() local i pprev for (( i=1; i < ${#words[@]}; )); do if [[ ${words[i]::1} == - ]] && - [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]]; then + [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]] && + [[ ${words[i]} != "-R" ]] && [[ ${words[i]} != "--relocate-base-btf" ]]; then words=( "${words[@]:0:i}" "${words[@]:i+1}" ) [[ $i -le $cword ]] && cword=$(( cword - 1 )) else diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 67be216023e8..aaa8e4cb9ffd 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -763,6 +763,14 @@ static int do_dump(int argc, char **argv) base_btf = btf__parse_opts(*argv, &optp); if (base_btf) btf = btf__parse_split(*argv, base_btf); + if (btf && relocate_base_btf) { + err = btf__relocate(btf, relocate_base_btf); + if (err) { + p_err("could not relocate BTF from '%s' with base BTF '%s': %s\n", + *argv, relocate_base_btf_path, strerror(-err)); + goto done; + } + } } if (!btf) { err = -errno; @@ -1203,7 +1211,8 @@ static int do_help(int argc, char **argv) " " HELP_SPEC_MAP "\n" " " HELP_SPEC_PROGRAM "\n" " " HELP_SPEC_OPTIONS " |\n" - " {-B|--base-btf} }\n" + " {-B|--base-btf} |\n" + " {-R|--relocate-base-btf} }\n" "", bin_name, "btf"); diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c index 08d0ac543c67..69d4906bec5c 100644 --- a/tools/bpf/bpftool/main.c +++ b/tools/bpf/bpftool/main.c @@ -32,6 +32,8 @@ bool verifier_logs; bool relaxed_maps; bool use_loader; struct btf *base_btf; +struct btf *relocate_base_btf; +const char *relocate_base_btf_path; struct hashmap *refs_table; static void __noreturn clean_and_exit(int i) @@ -448,6 +450,7 @@ int main(int argc, char **argv) { "debug", no_argument, NULL, 'd' }, { "use-loader", no_argument, NULL, 'L' }, { "base-btf", required_argument, NULL, 'B' }, + { "relocate-base-btf", required_argument, NULL, 'R' }, { 0 } }; bool version_requested = false; @@ -473,7 +476,7 @@ int main(int argc, char **argv) bin_name = "bpftool"; opterr = 0; - while ((opt = getopt_long(argc, argv, "VhpjfLmndB:l", + while ((opt = getopt_long(argc, argv, "VhpjfLmndB:lR:", options, NULL)) >= 0) { switch (opt) { case 'V': @@ -519,6 +522,15 @@ int main(int argc, char **argv) case 'L': use_loader = true; break; + case 'R': + relocate_base_btf_path = optarg; + relocate_base_btf = btf__parse(optarg, NULL); + if (!relocate_base_btf) { + p_err("failed to parse base BTF for relocation at '%s': %d\n", + optarg, -errno); + return -1; + } + break; default: p_err("unrecognized option '%s'", argv[optind - 1]); if (json_output) diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h index 9eb764fe4cc8..bbf8194a2d76 100644 --- a/tools/bpf/bpftool/main.h +++ b/tools/bpf/bpftool/main.h @@ -83,6 +83,8 @@ extern bool verifier_logs; extern bool relaxed_maps; extern bool use_loader; extern struct btf *base_btf; +extern struct btf *relocate_base_btf; +extern const char *relocate_base_btf_path; extern struct hashmap *refs_table; void __printf(1, 2) p_err(const char *fmt, ...);