From patchwork Fri May 10 10:30:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661292 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82C4155C3B for ; Fri, 10 May 2024 10:33:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337193; cv=none; b=a01w1d+9Ubx2gdM9Ma7R/v1mV0GHux/d6n8GBMfCxaBNJYSRQgn9aGsOk3NF5MUMeLpT4hKdmdHfHxauasf58ADKkVBkMcuUgXi8CZQFUD0SJ1+XMvVgHglOtQN+aweO901Ek5rQU4rBWfSl/PVpiWYaqrdPjL1dzjM4JvYkcnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337193; c=relaxed/simple; bh=Vkhnbm9YakwwMiUNS8J2o3GMq5Jo/AKxZ5erQkzcbnI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mPnRZCcHnTybLfr3sJHLxbL8LMkOOqRTxG+sXLxsfN1dtK8ZmrPhGXcCRhvGlKaknOw7/dHv+aSz2ueZZDQlYjxgHbE/98vmOUL9YvPOcx8PpQ2BMPFs8zEjvQT7ggiBA3RL9UYVJXiZUc0PVP/vLdIykiieNVHCMxK4GogkLhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=PdkchoEO; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="PdkchoEO" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAWgCj015434; Fri, 10 May 2024 10:32:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=8M29G8Ns7/gvJIhUFbEZ8l73MJEcMhbsL4RGWETlEvA=; b=PdkchoEOn86Y2sd/HIYbTVtYhmvPFN9dWOK1o03O8QjSlt2Gmo2WzqZCSMdtebXCPgoZ XpQl3yvJFw69C0sUfpZAiV4nCLWJ3NIL7iZW/ZNAlkoiXR5rA7408IjBRmL1leRU+wPL IRdNqLjuw+2F6LgTvDqoxbdsEYcEbzWABRQ1SCMe28tXU22b9VTOnSQQPkj151r8CXs8 ke2BlmEKPPm4NAJvG82frzx17OTmszv4V4ACVsp2I9rXoWJqby8hVu5mldcCXYbFdvXW 76+/mNUouk3PoIpYmr4hbmmdysAQ019a6p9Jy4kJ0kvYminaCwKqJvUAgKnR4dVd8u7k ew== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1f4x0c4g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:32:45 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A9Mx0Z019119; Fri, 10 May 2024 10:31:06 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcmje-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:06 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hT011786; Fri, 10 May 2024 10:31:05 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-2; Fri, 10 May 2024 10:31:05 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 01/11] libbpf: add btf__distill_base() creating split BTF with distilled base BTF Date: Fri, 10 May 2024 11:30:42 +0100 Message-Id: <20240510103052.850012-2-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: oNNwPGN7S97RpvHiyhohibNCIINxZmXn X-Proofpoint-ORIG-GUID: oNNwPGN7S97RpvHiyhohibNCIINxZmXn X-Patchwork-Delegate: bpf@iogearbox.net To support more robust split BTF, adding supplemental context for the base BTF type ids that split BTF refers to is required. Without such references, a simple shuffling of base BTF type ids (without any other significant change) invalidates the split BTF. Here the attempt is made to store additional context to make split BTF more robust. This context comes in the form of distilled base BTF providing minimal information (name and - in some cases - size) for base INTs, FLOATs, STRUCTs, UNIONs, ENUMs and ENUM64s along with modified split BTF that points at that base and contains any additional types needed (such as TYPEDEF, PTR and anonymous STRUCT/UNION declarations). This information constitutes the minimal BTF representation needed to disambiguate or remove split BTF references to base BTF. The rules are as follows: - INT, FLOAT are recorded in full. - if a named base BTF STRUCT or UNION is referred to from split BTF, it will be encoded either as a zero-member sized STRUCT/UNION (preserving size for later relocation checks) or as a named FWD. Only base BTF STRUCT/UNIONs that are embedded in split BTF STRUCT/UNIONs need to preserve size information, so a FWD representation will be used in most cases. - if an ENUM[64] is named, a ENUM forward representation (an ENUM with no values) is used. - in all other cases, the type is added to the new split BTF. Avoiding struct/union/enum/enum64 expansion is important to keep the distilled base BTF representation to a minimum size. When successful, new representations of the distilled base BTF and new split BTF that refers to it are returned. Both need to be freed by the caller. So to take a simple example, with split BTF with a type referring to "struct sk_buff", we will generate distilled base BTF with a FWD struct sk_buff, and the split BTF will refer to it instead. Tools like pahole can utilize such split BTF to populate the .BTF section (split BTF) and an additional .BTF.base section. Then when the split BTF is loaded, the distilled base BTF can be used to relocate split BTF to reference the current (and possibly changed) base BTF. So for example if "struct sk_buff" was id 502 when the split BTF was originally generated, we can use the distilled base BTF to see that id 502 refers to a "struct sk_buff" and replace instances of id 502 with the current (relocated) base BTF sk_buff type id. Distilled base BTF is small; when building a kernel with all modules using distilled base BTF as a test, ovreall module size grew by only 5.3Mb total across ~2700 modules. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 346 ++++++++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 20 +++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 361 insertions(+), 6 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 2d0840ef599a..65abd555fa36 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1771,9 +1771,8 @@ static int btf_rewrite_str(__u32 *str_off, void *ctx) return 0; } -int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type) { - struct btf_pipe p = { .src = src_btf, .dst = btf }; struct btf_type *t; int sz, err; @@ -1782,20 +1781,27 @@ int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_t return libbpf_err(sz); /* deconstruct BTF, if necessary, and invalidate raw_data */ - if (btf_ensure_modifiable(btf)) + if (btf_ensure_modifiable(p->dst)) return libbpf_err(-ENOMEM); - t = btf_add_type_mem(btf, sz); + t = btf_add_type_mem(p->dst, sz); if (!t) return libbpf_err(-ENOMEM); memcpy(t, src_type, sz); - err = btf_type_visit_str_offs(t, btf_rewrite_str, &p); + err = btf_type_visit_str_offs(t, btf_rewrite_str, p); if (err) return libbpf_err(err); - return btf_commit_type(btf, sz); + return btf_commit_type(p->dst, sz); +} + +int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +{ + struct btf_pipe p = { .src = src_btf, .dst = btf }; + + return btf_add_type(&p, src_type); } static int btf_rewrite_type_ids(__u32 *type_id, void *ctx) @@ -5212,3 +5218,331 @@ int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void return 0; } + +#define BTF_EMBEDDED_COMPOSITE (1 << 31) /* flag set if struct/union is embedded */ +#define BTF_ID(id) (id & ~BTF_EMBEDDED_COMPOSITE) + +struct btf_distill { + struct btf_pipe pipe; + int *ids; + unsigned int split_start_id; + unsigned int diff_id; +}; + +/* Check if a member of a split BTF struct/union refers to a base BTF + * struct/union. Members can be const/restrict/volatile/typedef + * reference types, but if a pointer is encountered, type is no longer + * considered embedded. + */ +static int btf_find_embedded_composite_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + const struct btf_type *t; + __u32 next_id = *id; + + do { + if (next_id == 0) + return 0; + t = btf_type_by_id(dist->pipe.src, next_id); + switch (btf_kind(t)) { + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_TYPEDEF: + case BTF_KIND_TYPE_TAG: + next_id = t->type; + break; + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + next_id = a->type; + break; + } + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + dist->ids[next_id] |= BTF_EMBEDDED_COMPOSITE; + return 0; + default: + return 0; + } + + } while (1); + + return 0; +} + +static bool btf_is_eligible_named_fwd(const struct btf_type *t) +{ + return (btf_is_composite(t) || btf_is_any_enum(t)) && t->name_off != 0; +} + +static int btf_add_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + struct btf_type *t = btf_type_by_id(dist->pipe.src, *id); + int err; + + if (!*id) + return 0; + /* split BTF id, not needed */ + if (*id >= dist->split_start_id) + return 0; + /* already added ? */ + if (BTF_ID(dist->ids[*id]) > 0) + return 0; + + /* only a subset of base BTF types should be referenced from split + * BTF; ensure nothing unexpected is referenced. + */ + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_ARRAY: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_TYPEDEF: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_PTR: + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_FUNC_PROTO: + dist->ids[*id] |= *id; + break; + default: + pr_warn("unexpected reference to base type[%u] of kind [%u] when creating distilled base BTF.\n", + *id, btf_kind(t)); + return -EINVAL; + } + + /* struct/union members not needed, except for anonymous structs + * and unions, which we need since name won't help us determine + * matches; so if a named struct/union, no need to recurse + * into members. + */ + if (btf_is_eligible_named_fwd(t)) + return 0; + + /* ensure references in type are added also. */ + err = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, ctx); + if (err < 0) + return err; + return 0; +} + +static int btf_add_distilled_types(struct btf_distill *dist) +{ + bool adding_to_base = dist->pipe.dst->start_id == 1; + int id = btf__type_cnt(dist->pipe.dst); + struct btf_type *t; + int i, err = 0; + + /* Add types for each of the required references to either distilled + * base or split BTF, depending on type characteristics. + */ + for (i = 1; i < dist->split_start_id; i++) { + const char *name; + int kind; + + if (!BTF_ID(dist->ids[i])) + continue; + t = btf_type_by_id(dist->pipe.src, i); + kind = btf_kind(t); + name = btf__name_by_offset(dist->pipe.src, t->name_off); + + /* Named int, float, fwd struct, union, enum[64] are added to + * base; everything else is added to split BTF. + */ + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_FWD: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + if ((adding_to_base && !t->name_off) || (!adding_to_base && t->name_off)) + continue; + break; + default: + if (adding_to_base) + continue; + break; + } + if (dist->ids[i] & BTF_EMBEDDED_COMPOSITE) { + /* If a named struct/union in base BTF is referenced as a type + * in split BTF without use of a pointer - i.e. as an embedded + * struct/union - add an empty struct/union preserving size + * since size must be consistent when relocating split and + * possibly changed base BTF. + */ + err = btf_add_composite(dist->pipe.dst, kind, name, t->size); + } else if (btf_is_eligible_named_fwd(t)) { + /* If not embedded, use a fwd for named struct/unions since we + * can match via name without any other details. + */ + switch (kind) { + case BTF_KIND_STRUCT: + err = btf__add_fwd(dist->pipe.dst, name, BTF_FWD_STRUCT); + break; + case BTF_KIND_UNION: + err = btf__add_fwd(dist->pipe.dst, name, BTF_FWD_UNION); + break; + case BTF_KIND_ENUM: + err = btf__add_enum(dist->pipe.dst, name, sizeof(int)); + break; + case BTF_KIND_ENUM64: + err = btf__add_enum(dist->pipe.dst, name, sizeof(__u64)); + break; + default: + pr_warn("unexpected kind [%u] when creating distilled base BTF.\n", + btf_kind(t)); + return -EINVAL; + } + } else { + err = btf_add_type(&dist->pipe, t); + } + if (err < 0) + break; + dist->ids[i] = id++; + } + return err; +} + +/* Split BTF ids without a mapping will be shifted downwards since distilled + * base BTF is smaller than the original base BTF. For those that have a + * mapping (either to base or updated split BTF), update the id based on + * that mapping. + */ +static int btf_update_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + + if (BTF_ID(dist->ids[*id])) + *id = BTF_ID(dist->ids[*id]); + else if (*id >= dist->split_start_id) + *id -= dist->diff_id; + return 0; +} + +/* Create updated split BTF with distilled base BTF; distilled base BTF + * consists of BTF information required to clarify the types that split + * BTF refers to, omitting unneeded details. Specifically it will contain + * base types and forward declarations of named structs, unions and enumerated + * types. Associated reference types like pointers, arrays and anonymous + * structs, unions and enumerated types will be added to split BTF. + * + * The only case where structs, unions or enumerated types are fully represented + * is when they are anonymous; in such cases, the anonymous type is added to + * split BTF in full. + * + * We return newly-created split BTF where the split BTF refers to a newly-created + * distilled base BTF. Both must be freed separately by the caller. + */ +int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf) +{ + struct btf *new_base = NULL, *new_split = NULL; + unsigned int n = btf__type_cnt(src_btf); + struct btf_distill dist = {}; + struct btf_type *t; + int i, err = 0; + + /* src BTF must be split BTF. */ + if (!new_base_btf || !new_split_btf || !btf__base_btf(src_btf)) + return libbpf_err(-EINVAL); + + new_base = btf__new_empty(); + if (!new_base) + return libbpf_err(-ENOMEM); + dist.ids = calloc(n, sizeof(*dist.ids)); + if (!dist.ids) { + err = -ENOMEM; + goto err_out; + } + dist.pipe.src = src_btf; + dist.pipe.dst = new_base; + dist.pipe.str_off_map = hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL); + if (IS_ERR(dist.pipe.str_off_map)) { + err = -ENOMEM; + goto err_out; + } + dist.split_start_id = btf__type_cnt(btf__base_btf(src_btf)); + + /* Pass over src split BTF; generate the list of base BTF + * type ids it references; these will constitute our distilled + * BTF set to be distributed over base and split BTF as appropriate. + */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + + /* check if members of struct/union in split BTF refer to base BTF + * struct/union; if so, we will use an empty sized struct to represent + * it rather than a FWD because its size must match on later BTF + * relocation. + */ + if (btf_is_composite(t)) { + err = btf_type_visit_type_ids(t, btf_find_embedded_composite_type_ids, + &dist); + if (err < 0) + goto err_out; + } + err = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, &dist); + if (err < 0) + goto err_out; + } + /* Next add types for each of the required references to base BTF and split BTF in turn. */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto err_out; + /* now create new split BTF with distilled base BTF as its base; we end up with + * split BTF that has base BTF that represents enough about its base references + * to allow it to be relocated with the base BTF available. + */ + new_split = btf__new_empty_split(new_base); + if (!new_split_btf) { + err = -errno; + goto err_out; + } + dist.pipe.dst = new_split; + /* First add all split types */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + err = btf_add_type(&dist.pipe, t); + if (err < 0) + goto err_out; + } + /* Now add distilled types to split BTF that are not added to base. */ + err = btf_add_distilled_types(&dist); + if (err < 0) + goto err_out; + + /* all split BTF ids will be shifted downwards since there are less base BTF ids + * in distilled base BTF. + */ + dist.diff_id = dist.split_start_id - btf__type_cnt(new_base); + + n = btf__type_cnt(new_split); + /* Now update base/split BTF ids. */ + for (i = 1; i < n; i++) { + t = btf_type_by_id(new_split, i); + + err = btf_type_visit_type_ids(t, btf_update_distilled_type_ids, &dist); + if (err < 0) + goto err_out; + } + free(dist.ids); + hashmap__free(dist.pipe.str_off_map); + *new_base_btf = new_base; + *new_split_btf = new_split; + return 0; +err_out: + free(dist.ids); + if (!IS_ERR(dist.pipe.str_off_map)) + hashmap__free(dist.pipe.str_off_map); + btf__free(new_split); + btf__free(new_base); + return libbpf_err(err); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c84..f3f149a09088 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -107,6 +107,26 @@ LIBBPF_API struct btf *btf__new_empty(void); */ LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf); +/** + * @brief **btf__distill_base()** creates new versions of the split BTF + * *src_btf* and its base BTF. The new base BTF will only contain the types + * needed to improve robustness of the split BTF to small changes in base BTF. + * When that split BTF is loaded against a (possibly changed) base, this + * distilled base BTF will help update references to that (possibly changed) + * base BTF. + * + * Both the new split and its associated new base BTF must be freed by + * the caller. + * + * If successful, 0 is returned and **new_base_btf** and **new_split_btf** + * will point at new base/split BTF. Both the new split and its associated + * new base BTF must be freed by the caller. + * + * A negative value is returned on error. + */ +LIBBPF_API int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf); + LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext); LIBBPF_API struct btf *btf__parse_split(const char *path, struct btf *base_btf); LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c1ce8aa3520b..9e69d6e2a512 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -419,6 +419,7 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: + btf__distill_base; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; From patchwork Fri May 10 10:30:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661287 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E591364 for ; Fri, 10 May 2024 10:31:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337098; cv=none; b=tuxA2CeTTjk3qH9omdhbUIlSAYR4c5pkbQFIdQxxJ33vdTKg8yG2HjmxFg/vv0p129blS3iGTo84IOdj5F91GJOrhppQadycfKebz/gFiV7k9X4DPAjmNnSt8rnRg4h1MoOQSLpRvkks+ovK7t7K4ENUzEaOswUQVvrAK6DdlGA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337098; c=relaxed/simple; bh=Lu0eOBGIu8MNznZieN2LLBnGj7jlgZurq6T25UqSK3o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pBDZaW/jkleXzKBrz8XWsuJyGb5AVZr2pMX7wMsq4g+kq3qTZEgEm+Bvl1t+eXjN96k4rW481HeMxWPfkOOZ1EW2iwlNpotI33CCAZJ1M3+o9NFS3aQNSQi0J+izVvPElLMAXlHhdhppIgyxxzfoNOjlMX4iFnK/KytLY7C7ZUU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=TVLRrB59; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="TVLRrB59" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAT4jS003371; Fri, 10 May 2024 10:31:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=fhGdVuHoMZxVgqRziuSFyqySG+M0JMa8TseVzMQvlH0=; b=TVLRrB59Me1ivGqKOi2MESU05+LOTFmHWDaGD4cI+cHRi62ZMwvuSVfiyvZ9hiMbenm+ rjGVpWKU0Hug3Bp5FEzYij803XmgcuJE3Q+dXrTocA+I2ZUO05EHuTS2pslkWj+eGxvt z8ZFLdWyCxBRczlC/YP6LInlBQRPrl+PrF4324ujoXl1roSf64ZMLdPz18HErt4bw3qk OicXPZAmIr+zZd6fRPh8yoSzDkEyFzFPi7XOWX+iXOB0Kw+CK6j1c935FCvbSntjaEu6 diaOiaGhZbKh5sYzq9d9faeEh8jVBZgPmP/AdnrSD/G115uHxQpMoa8CvP7W/wLd7bdM 0w== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1hps80uv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:14 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44AANoIP019720; Fri, 10 May 2024 10:31:13 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcmqw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:13 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hV011786; Fri, 10 May 2024 10:31:12 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-3; Fri, 10 May 2024 10:31:12 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 02/11] selftests/bpf: test distilled base, split BTF generation Date: Fri, 10 May 2024 11:30:43 +0100 Message-Id: <20240510103052.850012-3-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: SJXIgAa_BGD1x6BqfFSDhxPL1oC5VHSs X-Proofpoint-ORIG-GUID: SJXIgAa_BGD1x6BqfFSDhxPL1oC5VHSs X-Patchwork-Delegate: bpf@iogearbox.net Test generation of split+distilled base BTF, ensuring that - base BTF STRUCTs which are embedded in split BTF structs are represented as 0-member sized structs, allowing size checking - FWDs are used in place of full named struct/union declarations - FWDs are used in place of full named enum declarations - anonymous struct/unions are represented in full in split BTF - anonymous enums are represented in full in split BTF - types unreferenced from split BTF are not present in distilled base BTF Also test that with vmlinux BTF and split BTF based upon it, we only represent needed base types referenced from split BTF in distilled base. Signed-off-by: Alan Maguire --- .../selftests/bpf/prog_tests/btf_distill.c | 272 ++++++++++++++++++ 1 file changed, 272 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_distill.c diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c new file mode 100644 index 000000000000..400df8740943 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -0,0 +1,272 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include +#include +#include "btf_helpers.h" + +/* Fabricate base, split BTF with references to base types needed; then create + * split BTF with distilled base BTF and ensure expectations are met: + * - only referenced base types from split BTF are present + * - struct/union/enum are represented as FWDs unless anonymous, when they + * are represented in full, or if embedded in a split BTF struct, in which + * case they are represented by a STRUCT with specified size and vlen=0. + */ +static void test_distilled_base(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_ptr(btf1, 1); /* [2] ptr to int */ + btf__add_struct(btf1, "s1", 8); /* [3] struct s1 { */ + btf__add_field(btf1, "f1", 2, 0, 0); /* int *f1; */ + /* } */ + btf__add_struct(btf1, "", 12); /* [4] struct { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 3, 32, 0); /* struct s1 f2; */ + /* } */ + btf__add_int(btf1, "unsigned int", 4, 0); /* [5] unsigned int */ + btf__add_union(btf1, "u1", 12); /* [6] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 2, 0, 0); /* int *f2; */ + /* } */ + btf__add_union(btf1, "", 4); /* [7] union { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_enum(btf1, "e1", 4); /* [8] enum e1 { */ + btf__add_enum_value(btf1, "v1", 1); /* v1 = 1; */ + /* } */ + btf__add_enum(btf1, "", 4); /* [9] enum { */ + btf__add_enum_value(btf1, "av1", 2); /* av1 = 2; */ + /* } */ + btf__add_enum64(btf1, "e641", 8, true); /* [10] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1024); /* v1 = 1024; */ + /* } */ + btf__add_enum64(btf1, "", 8, true); /* [11] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1025); /* v1 = 1025; */ + /* } */ + btf__add_struct(btf1, "unneeded", 4); /* [12] struct unneeded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_struct(btf1, "embedded", 4); /* [13] struct embedded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_func_proto(btf1, 1); /* [14] int (*)(int *p1); */ + btf__add_func_param(btf1, "p1", 1); + + btf__add_array(btf1, 1, 1, 3); /* [15] int [3]; */ + + btf__add_struct(btf1, "from_proto", 4); /* [16] struct from_proto { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0"); + + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + + btf__add_ptr(btf2, 3); /* [17] ptr to struct s1 */ + /* add ptr to struct anon */ + btf__add_ptr(btf2, 4); /* [18] ptr to struct (anon) */ + btf__add_const(btf2, 6); /* [19] const union u1 */ + btf__add_restrict(btf2, 7); /* [20] restrict union (anon) */ + btf__add_volatile(btf2, 8); /* [21] volatile enum e1 */ + btf__add_typedef(btf2, "et", 9); /* [22] typedef enum (anon) */ + btf__add_const(btf2, 10); /* [23] const enum64 e641 */ + btf__add_ptr(btf2, 11); /* [24] restrict enum64 (anon) */ + btf__add_struct(btf2, "with_embedded", 4); /* [25] struct with_embedded { */ + btf__add_field(btf2, "f1", 13, 0, 0); /* struct embedded f1; */ + /* } */ + btf__add_func(btf2, "fn", BTF_FUNC_STATIC, 14); /* [26] int fn(int p1); */ + btf__add_typedef(btf2, "arraytype", 15); /* [27] typedef int[3] foo; */ + btf__add_func_proto(btf2, 1); /* [28] int (*)(struct from proto p1); */ + btf__add_func_param(btf2, "p1", 16); + + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] PTR '(anon)' type_id=3", + "[18] PTR '(anon)' type_id=4", + "[19] CONST '(anon)' type_id=6", + "[20] RESTRICT '(anon)' type_id=7", + "[21] VOLATILE '(anon)' type_id=8", + "[22] TYPEDEF 'et' type_id=9", + "[23] CONST '(anon)' type_id=10", + "[24] PTR '(anon)' type_id=11", + "[25] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[26] FUNC 'fn' type_id=14 linkage=static", + "[27] TYPEDEF 'arraytype' type_id=15", + "[28] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16"); + + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split") || + !ASSERT_EQ(8, btf__type_cnt(btf3), "distilled_base_type_cnt")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 's1' fwd_kind=struct", + "[3] FWD 'u1' fwd_kind=union", + "[4] ENUM 'e1' encoding=UNSIGNED size=4 vlen=0", + "[5] ENUM 'e641' encoding=UNSIGNED size=8 vlen=0", + "[6] STRUCT 'embedded' size=4 vlen=0", + "[7] FWD 'from_proto' fwd_kind=struct", + /* split BTF; these types should match split BTF above from 17-28, with + * updated type id references + */ + "[8] PTR '(anon)' type_id=2", + "[9] PTR '(anon)' type_id=20", + "[10] CONST '(anon)' type_id=3", + "[11] RESTRICT '(anon)' type_id=21", + "[12] VOLATILE '(anon)' type_id=4", + "[13] TYPEDEF 'et' type_id=22", + "[14] CONST '(anon)' type_id=5", + "[15] PTR '(anon)' type_id=23", + "[16] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=6 bits_offset=0", + "[17] FUNC 'fn' type_id=24 linkage=static", + "[18] TYPEDEF 'arraytype' type_id=25", + "[19] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=7", + /* split BTF types added from original base BTF below */ + "[20] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=32", + "[21] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[22] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[23] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[24] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[25] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* create split reference BTF from vmlinux + split BTF with a few type references; + * ensure the resultant split reference BTF is as expected, containing only types + * needed to disambiguate references from split BTF. + */ +static void test_distilled_base_vmlinux(void) +{ + struct btf *split_btf = NULL, *vmlinux_btf = btf__load_vmlinux_btf(); + struct btf *split_dist = NULL, *base_dist = NULL; + __s32 int_id, sk_buff_id; + + if (!ASSERT_OK_PTR(vmlinux_btf, "load_vmlinux")) + return; + int_id = btf__find_by_name_kind(vmlinux_btf, "int", BTF_KIND_INT); + if (!ASSERT_GT(int_id, 0, "find_int")) + goto cleanup; + sk_buff_id = btf__find_by_name_kind(vmlinux_btf, "sk_buff", BTF_KIND_STRUCT); + if (!ASSERT_GT(sk_buff_id, 0, "find_sk_buff_id")) + goto cleanup; + split_btf = btf__new_empty_split(vmlinux_btf); + if (!ASSERT_OK_PTR(split_btf, "new_split")) + goto cleanup; + btf__add_typedef(split_btf, "myint", int_id); + btf__add_ptr(split_btf, sk_buff_id); + + if (!ASSERT_EQ(btf__distill_base(split_btf, &base_dist, &split_dist), 0, + "distill_vmlinux_base")) + goto cleanup; + + if (!ASSERT_OK_PTR(split_dist, "split_distilled") || + !ASSERT_OK_PTR(base_dist, "base_dist")) + goto cleanup; + VALIDATE_RAW_BTF( + split_dist, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 'sk_buff' fwd_kind=struct", + "[3] TYPEDEF 'myint' type_id=1", + "[4] PTR '(anon)' type_id=2"); + +cleanup: + btf__free(split_dist); + btf__free(base_dist); + btf__free(split_btf); + btf__free(vmlinux_btf); +} + +void test_btf_distill(void) +{ + if (test__start_subtest("distilled_base")) + test_distilled_base(); + if (test__start_subtest("distilled_base_vmlinux")) + test_distilled_base_vmlinux(); +} From patchwork Fri May 10 10:30:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661291 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EDC255C3B for ; Fri, 10 May 2024 10:32:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337175; cv=none; b=NQ3vFjfKUF2TP39UQWEDKVulGz320886FEocfNrvtU7UIlebT9qMjz/Xd+zd1mMSL0TWJ10BZD0pqcqjCc8vyzen0s+GMbDpK24NAXZu1mvBvbjoNkcT6QADdEvmkQMLibyjIw1NCFeK9LaAqJtKdqyPSid4Y186fD++sIpG7rA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337175; c=relaxed/simple; bh=D2oHaI4C8APcpBl2Vc4xlfszus1cjoweC40RbL9PUaM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hVoyLTAC2ffjoR7bWLwZTq+okpLqndHWfrMz/lYAXYLy/lc4GOhb4+6f+SSJOedTXWg1L20qGEbwOEyNipUKKQ5a4SoLVwal4iqmqGNMecFiBEjWAWWMYcxuGoFKQWKesbnYuYeHh7e8xb1STtrK24cGQZ4eDluo8COgJnOQTQE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=FcIvQNiS; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="FcIvQNiS" Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAWIgd012335; Fri, 10 May 2024 10:32:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=xv1TpMO2gyI2UgBxwRCKPJln9rX0JmAthyTwMJ5GRFA=; b=FcIvQNiSGIQ08CkZjaLn4b+F1msvzwu1TdzIKSJVv+r/3ep0pdYC3NvM5zx0szTL57Uy OpjQGud3g5K5tRF4SNdosKUR1407/scoGl6Fr3D5BAW9eQPO4sshThDxAKS+CjjdsnJa zxMxTMcV6Dpy1dEU+PJk4kTE+VbRLOxi6/0endC1jDqOSQklgnLHUZ1fm4PLsEBChLbk hLpjY6l0W3rkIq3psOFSFgwTdZacr2Env4WEh9/ViTED8Px47nxFaSG5dblPTa+O0Pu5 2z3z+J4mKoMMCG7h7DB4+V/+19u2w3EtFmpExzFuhgEg98XfyQ1PIcfTRQrUPtpMAtDu fw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1dfsrkcw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:32:18 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A9wlwT019113; Fri, 10 May 2024 10:31:19 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcmut-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:19 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hX011786; Fri, 10 May 2024 10:31:18 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-4; Fri, 10 May 2024 10:31:18 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 03/11] libbpf: add btf__parse_opts() API for flexible BTF parsing Date: Fri, 10 May 2024 11:30:44 +0100 Message-Id: <20240510103052.850012-4-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: kNXZeVJnjhAS9-2Ta_rqwIOYSTuXUaSg X-Proofpoint-ORIG-GUID: kNXZeVJnjhAS9-2Ta_rqwIOYSTuXUaSg X-Patchwork-Delegate: bpf@iogearbox.net Options cover existing parsing scenarios (ELF, raw, retrieving .BTF.ext) and also allow specification of the ELF section name containing BTF. This will allow consumers to retrieve BTF from .BTF.base sections (BTF_BASE_ELF_SEC) also. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 49 +++++++++++++++++++++++++++------------- tools/lib/bpf/btf.h | 31 +++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 65 insertions(+), 16 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 65abd555fa36..1eb66a7a4c46 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1084,7 +1084,7 @@ struct btf *btf__new_split(const void *data, __u32 size, struct btf *base_btf) return libbpf_ptr(btf_new(data, size, base_btf)); } -static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, +static struct btf *btf_parse_elf(const char *path, const char *btf_sec, struct btf *base_btf, struct btf_ext **btf_ext) { Elf_Data *btf_data = NULL, *btf_ext_data = NULL; @@ -1146,7 +1146,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, idx, path); goto done; } - if (strcmp(name, BTF_ELF_SEC) == 0) { + if (strcmp(name, btf_sec) == 0) { btf_data = elf_getdata(scn, 0); if (!btf_data) { pr_warn("failed to get section(%d, %s) data from %s\n", @@ -1166,7 +1166,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, } if (!btf_data) { - pr_warn("failed to find '%s' ELF section in %s\n", BTF_ELF_SEC, path); + pr_warn("failed to find '%s' ELF section in %s\n", btf_sec, path); err = -ENODATA; goto done; } @@ -1212,12 +1212,12 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse_elf(path, NULL, btf_ext)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, NULL, btf_ext)); } struct btf *btf__parse_elf_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse_elf(path, base_btf, NULL)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, base_btf, NULL)); } static struct btf *btf_parse_raw(const char *path, struct btf *base_btf) @@ -1293,31 +1293,48 @@ struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf) return libbpf_ptr(btf_parse_raw(path, base_btf)); } -static struct btf *btf_parse(const char *path, struct btf *base_btf, struct btf_ext **btf_ext) +struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts) { - struct btf *btf; + struct btf *btf, *base_btf; + const char *btf_sec; + struct btf_ext **btf_ext; int err; + if (!OPTS_VALID(opts, btf_parse_opts)) + return libbpf_err_ptr(-EINVAL); + base_btf = OPTS_GET(opts, base_btf, NULL); + btf_sec = OPTS_GET(opts, btf_sec, NULL); + btf_ext = OPTS_GET(opts, btf_ext, NULL); + if (btf_ext) *btf_ext = NULL; - - btf = btf_parse_raw(path, base_btf); + if (!btf_sec) { + btf = btf_parse_raw(path, base_btf); + err = libbpf_get_error(btf); + if (!err) + return btf; + if (err != -EPROTO) + return libbpf_err_ptr(err); + } + btf = btf_parse_elf(path, btf_sec ?: BTF_ELF_SEC, base_btf, btf_ext); err = libbpf_get_error(btf); - if (!err) - return btf; - if (err != -EPROTO) - return ERR_PTR(err); - return btf_parse_elf(path, base_btf, btf_ext); + if (err) + return libbpf_err_ptr(err); + return btf; } struct btf *btf__parse(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse(path, NULL, btf_ext)); + LIBBPF_OPTS(btf_parse_opts, opts, .btf_ext = btf_ext); + + return btf__parse_opts(path, &opts); } struct btf *btf__parse_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse(path, base_btf, NULL)); + LIBBPF_OPTS(btf_parse_opts, opts, .base_btf = base_btf); + + return btf__parse_opts(path, &opts); } static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian); diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index f3f149a09088..8e1702ad5ef4 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -18,6 +18,7 @@ extern "C" { #define BTF_ELF_SEC ".BTF" #define BTF_EXT_ELF_SEC ".BTF.ext" +#define BTF_BASE_ELF_SEC ".BTF.base" #define MAPS_ELF_SEC ".maps" struct btf; @@ -134,6 +135,36 @@ LIBBPF_API struct btf *btf__parse_elf_split(const char *path, struct btf *base_b LIBBPF_API struct btf *btf__parse_raw(const char *path); LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf); +struct btf_parse_opts { + size_t sz; + /* use base BTF to parse split BTF */ + struct btf *base_btf; + /* retrieve optional .BTF.ext info */ + struct btf_ext **btf_ext; + /* BTF section name; if NULL, try parsing raw BTF, falling back to parsing + * .BTF ELF section if that fails also. If set, parse named ELF section. + */ + const char *btf_sec; + size_t :0; +}; + +#define btf_parse_opts__last_field btf_sec + +/* @brief **btf__parse_opts()** parses BTF information from either a + * raw BTF file (*btf_sec* is NULL) or from the specified BTF section, + * also retrieving .BTF.ext info if *btf_ext* is non-NULL. If + * *base_btf* is specified, use it to parse split BTF from the + * specified location. + * + * @return new BTF object instance which has to be eventually freed with + * **btf__free()** + * + * On error, NULL is returned, with the `errno` variable set to the + * error code. + */ + +LIBBPF_API struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts); + LIBBPF_API struct btf *btf__load_vmlinux_btf(void); LIBBPF_API struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_btf); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 9e69d6e2a512..fd7bfeaba542 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -420,6 +420,7 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: btf__distill_base; + btf__parse_opts; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; From patchwork Fri May 10 10:30:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661288 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7A522233B for ; Fri, 10 May 2024 10:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337110; cv=none; b=KpK1WH/ukJwEqzuwUcuLEDX2kkRbW46KAWat1wj1REbi9oBlw1gzFnnUgvClq8EJUhPJ2BA0q6Gou2B7KRqh0d02kN9f9gds2YBYkAlQ39be+kwGC44BAv9d5a65jGTSaINruXHzwBKYTlfRckbZzrKdsoY+jNEjh+66AIDeRcA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337110; c=relaxed/simple; bh=n+0E3E5edjklp7dGzvm9IgDdc/nnGhrGYusc7hvVL74=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qJ8kgqldAbFWUB9X9qZr0dzgwceQ/dum1FLZ9+mTu0wbtQFvognonNvlj3YmdEPUgjLuWZZcC/3bo/cvz2xvRN2Bjd23UO++E9GgZy7585mQjWOPxpWhoBUj+fPheDj683ihXcCns2frh1T7uRsa8bPHXMDT/VpQYwdiEstyaqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=J9gn4ME0; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="J9gn4ME0" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AASsrJ015469; Fri, 10 May 2024 10:31:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=0YNCbKpyb3opOuzUulyTslCsmRcngO1VBVACpcslM94=; b=J9gn4ME0nLFC+t7d4Yyhp3W/BAKw+hxrVyPIvHamaxA8EAlupvJZYCzlGcImAC9EsAUB uB+hF7WkkIwxLDvDggn6CRHdDGwHV6xeD7z3arMf3z6E7qi2KdmbPnDayeVwTVKbwCNc s4YoLj73SQYiw4c8u9C0TqYCuaokC9Vzhz0zMMyf7Y6/hvYvD9zWWP9Hl2oU743bJiDl yir9T6YBYrITWYY2q7ainR7ZEMd+/tp3cvQzEd6/K3mHAxJV9I7qzvynDS7TwxV63P6a hny+TZxWxR9cGoW/HVrX0yhVqGJhznMeQMxq+eCFxq2S4mWf2SJB7tXV5hyK5fN/2g00 gA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1a0j8pes-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:27 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44AANoIa019720; Fri, 10 May 2024 10:31:26 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcmxr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:26 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hZ011786; Fri, 10 May 2024 10:31:25 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-5; Fri, 10 May 2024 10:31:25 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 04/11] bpftool: support displaying raw split BTF using base BTF section as base Date: Fri, 10 May 2024 11:30:45 +0100 Message-Id: <20240510103052.850012-5-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-ORIG-GUID: wsMT7UfLGIKGujK34Co_sXOiohV_drhB X-Proofpoint-GUID: wsMT7UfLGIKGujK34Co_sXOiohV_drhB X-Patchwork-Delegate: bpf@iogearbox.net If no base BTF can be found, fall back to checking for the .BTF.base section and use it to display split BTF. Signed-off-by: Alan Maguire Acked-by: Quentin Monnet --- tools/bpf/bpftool/btf.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 91fcb75babe3..0ca1f2417801 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -631,6 +631,14 @@ static int do_dump(int argc, char **argv) base = get_vmlinux_btf_from_sysfs(); btf = btf__parse_split(*argv, base ?: base_btf); + /* Finally check for presence of base BTF section */ + if (!btf && !base && !base_btf) { + LIBBPF_OPTS(btf_parse_opts, optp, .btf_sec = BTF_BASE_ELF_SEC); + + base_btf = btf__parse_opts(*argv, &optp); + if (base_btf) + btf = btf__parse_split(*argv, base_btf); + } if (!btf) { err = -errno; p_err("failed to load BTF from %s: %s", From patchwork Fri May 10 10:30:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661294 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DD4B14F137 for ; Fri, 10 May 2024 10:36:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337417; cv=none; b=kE9xnyQbFKzYInXLlxQR3v7Jo6w33Y/9jXyjhKr8JZ/KIYoEWXexL+Tw6mz6iB3n0ZlFZd9s2xyB8QD9BLOBnXtQ4AMLiocYIVISR5+aEWsRFUfC5x+ueJ4t7Z5/IGwABZ7dol5CwjCCA9zBqYeLz63aVRyijvNcukQdN8ag23E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337417; c=relaxed/simple; bh=/+yXlg1Ts9s3xPeHYINfynhz5+7pZAQ2giUpuazZaGU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=k9ubC5GfR4hxq1IItil/FNo+U8pmpuVrliL2ypeZw7dQ7P2amhNaR7eJzHOxeq8Di+CmsssZp0OOc0ETyDRSLgNCjFX8wvzEJ6ze3FQbbCSKATibo4VDwzJK645sQ7WAUAuEZe63l9UQ0brGBHQIfRWGI5f77sOob2tU3KpnXL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=XgvQ0k/f; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="XgvQ0k/f" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAaE9n005658; Fri, 10 May 2024 10:36:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=Pgy5/CZsZxyM/Pmv8NhpXNnUAQhuoc8JW/QL2mqOBME=; b=XgvQ0k/fVbP9+l5V6uUJKRbbzfKP5N62eKczD6qb1Qq3HlCW6qiXkN4J4RmYi6LMozVk 3NpmfqMZTjawNUWaRjvqorSNcGmbVxJX12gbfbk5cuapPmWZ1aC5qVoqe36/RI2oM7in iX2SQONmLRO99XqsdLG3zbUz9uf405WSArKtdzlQ4Z8lL7kmRZ6pHpcUqF0RjEyeMHW+ X2o28FLPBo/PhJtEVgOQFjQwbdN4RAFdJjpRltTvesT32/wmTYsC41idTUX5rtYmBWx+ Mq0a5MlLQDsSVxKLPQx4t+mbffOWeqm6cnM3uH192H26/gAxJHPGVp7wX6+DyDG0ZH+E Bg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1hyjg01a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:36:32 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A96QMm019773; Fri, 10 May 2024 10:31:31 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcn05-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:31 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hb011786; Fri, 10 May 2024 10:31:30 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-6; Fri, 10 May 2024 10:31:30 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 05/11] resolve_btfids: use .BTF.base ELF section as base BTF if -B option is used Date: Fri, 10 May 2024 11:30:46 +0100 Message-Id: <20240510103052.850012-6-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-ORIG-GUID: Dhj3pnPyBRt78lcAF78IdIPTWcFekWnS X-Proofpoint-GUID: Dhj3pnPyBRt78lcAF78IdIPTWcFekWnS X-Patchwork-Delegate: bpf@iogearbox.net When resolving BTF ids, use the BTF in the module .BTF.base section when passed the -B option. Both references to base BTF from split BTF and BTF ids will be relocated for base vmlinux on module load. Signed-off-by: Alan Maguire --- tools/bpf/resolve_btfids/main.c | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index d9520cb826b3..4f2ff52c3ea7 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -115,6 +115,7 @@ struct object { const char *path; const char *btf; const char *base_btf_path; + int base; struct { int fd; @@ -527,22 +528,37 @@ static int symbols_resolve(struct object *obj) int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; struct btf *base_btf = NULL; - int err, type_id; + int err = 0, type_id; struct btf *btf; __u32 nr_types; if (obj->base_btf_path) { - base_btf = btf__parse(obj->base_btf_path, NULL); - err = libbpf_get_error(base_btf); + LIBBPF_OPTS(btf_parse_opts, optp); + const char *path; + + if (obj->base) { + optp.btf_sec = BTF_BASE_ELF_SEC; + path = obj->path; + base_btf = btf__parse_opts(path, &optp); + /* fall back to normal base parsing if no BTF_BASE_ELF_SEC */ + } + if (!base_btf) { + optp.btf_sec = BTF_ELF_SEC; + path = obj->base_btf_path; + base_btf = btf__parse_opts(path, &optp); + } + if (!base_btf) + err = -errno; if (err) { pr_err("FAILED: load base BTF from %s: %s\n", - obj->base_btf_path, strerror(-err)); + path, strerror(-err)); return -1; } } btf = btf__parse_split(obj->btf ?: obj->path, base_btf); - err = libbpf_get_error(btf); + if (!btf) + err = -errno; if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); @@ -781,6 +797,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_STRING('b', "btf_base", &obj.base_btf_path, "file", "path of file providing base BTF"), + OPT_INCR('B', "base", &obj.base, + "use " BTF_BASE_ELF_SEC " ELF section BTF as base"), OPT_END() }; int err = -1; From patchwork Fri May 10 10:30:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661295 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1122114F139 for ; Fri, 10 May 2024 10:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337420; cv=none; b=clVtQz0ynZO2qwVNB/9DMS5kJ13I8LrCRa2Cb0PvOWeNzlkUi8bRAl0yl8VB0EmcYZbkF354a0f0FQESHNfWWkLO4WQo54QZrd43gpBBxwhwsvdYYgNbn8YPZidwNA9FKGf966stg22eJODRwSQiIXF1NO6J243QMlK3toGv6mI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337420; c=relaxed/simple; bh=yVC6tmcPxKskNY1hTzSU52cyP+qgu3OTkqLXyU4/+KU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nz+n89lVGrnWgv+P8yV42MwnptSIvEJmzreMtfSYG8uKsn0XD6Jmk1ihMn9ilC+ndmcwG00xMOPDGyfdSpUN9ux9BNTiS1DrAlXbd+Dk1mBoS9clPjzMuMpXdxKdrS8T/pnVULmBwtSTdQVsjCeMEI40TBtTbJL0Ec99ftplF8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=AdTono6X; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="AdTono6X" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAT4df003368; Fri, 10 May 2024 10:36:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=ccdujVrnaoL1Mchau8WyPceo/j5fIB7Xc7uyGgVrubI=; b=AdTono6XApF087nLxptT006tyuBjVVAGxtedGdAOwrEQVeZt4GWNsYtTxtkWxWgKcF5K P/Ry7Q7tpg0dRJnt0YewgKxfAE2eZg2h4IMUfUUxjgj13LPSsastG5ZfzQ4n4Ujxqupg dGx59DMFDUWTkUTY5tVp302wIy+sT+FZ18vsXMwA580q3EJYMKdBBdLToqeQqEVbfETs hTTLTTcLGwr1M7hXs4iKDzuA6v0fFu/G1iI5XorzgLGFMSt9PbEj2vKoN0o5FEdak7wq RIcfcrJv7HhzuPQUFbJdijCrGqyAs7Z0BO1yxDZHl+Ay1QE1EKE8Iwusw3L3RkKGih17 Yw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1hps817v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:36:37 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAUiNU019716; Fri, 10 May 2024 10:31:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcn22-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:36 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hd011786; Fri, 10 May 2024 10:31:35 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-7; Fri, 10 May 2024 10:31:35 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 06/11] kbuild, bpf: add module-specific pahole/resolve_btfids flags for distilled base BTF Date: Fri, 10 May 2024 11:30:47 +0100 Message-Id: <20240510103052.850012-7-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: deo8LT0kD4jzXB_TduwqLgvyzhIYZFry X-Proofpoint-ORIG-GUID: deo8LT0kD4jzXB_TduwqLgvyzhIYZFry X-Patchwork-Delegate: bpf@iogearbox.net Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids uses the "-B" option to specify that the BTF.ids section should be populated with split BTF relative to the added .BTF.base section rather than relative to the vmlinux base. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire --- scripts/Makefile.btf | 7 +++++++ scripts/Makefile.modfinal | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index 2d6e5ed9081e..48a11bbae38f 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -23,8 +23,15 @@ else # Switch to using --btf_features for v1.26 and later. pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func +ifneq ($(KBUILD_EXTMOD),) +module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base +module-resolve-btfids-flags-$(call test-ge, $(pahole-ver), 126) = -B +endif + endif pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust export PAHOLE_FLAGS := $(pahole-flags-y) +export MODULE_PAHOLE_FLAGS := $(module-pahole-flags-y) +export MODULE_RESOLVE_BTFIDS_FLAGS := $(module-resolve-btfids-flags-y) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 79fcf2731686..86189ddf39e8 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -39,8 +39,8 @@ quiet_cmd_btf_ko = BTF [M] $@ if [ ! -f vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \ - $(RESOLVE_BTFIDS) -b vmlinux $@; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) $(MODULE_RESOLVE_BTFIDS_FLAGS) -b vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies From patchwork Fri May 10 10:30:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661293 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEB917FBDA for ; Fri, 10 May 2024 10:33:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337194; cv=none; b=hvsYEXC/9eQ0vguItXSiu7zwI5lrUzhqxfWBENOMy52Z5z0Si08YyX1/1ZbmxpQsNYbQSSnLvL7J6ObeyhelQ/vK2x7QdJHtcuCYRgW5NCbtZib1d57DIMVO3OtelFV3HMlKeEOU9+yOCzAqPYFnseNZR04GpjIqS4fcu5L6OAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337194; c=relaxed/simple; bh=9DK92P5f8nQGX2U7cJ7RiJxx/xJ/PD7fwSg01su+Eyk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NS26KQ4XUWrg/P4lUUUVEAU2TYf2WaueM2sazSM1YuH/001swQUlrFC793XvVPWnhLRcV/AbuXp8p2fOTB0nD2+sVqgkE/f/liq5TT/DlL2hWVmq1tQV/50xI4IkVBaIzTXDHpW0uzo8Og2tCrFEMt/Y4ode/0wCMX+SQw/VnGc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=jliKIwL4; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jliKIwL4" Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAWHoX012274; Fri, 10 May 2024 10:32:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=cAIqpD2xcEzIfOhdfFUDXI5xF/b+gVID+5C3LF9kfhk=; b=jliKIwL4Okh6WT78Oc9tzYO6lvEfQh1m3RjpqOFGPCBpmOGRQJbpl/cXkByY0t6jwt/2 452egQBKlncRXy5SCtUw7nOoGtI6prxo1hYqlsZYLPudJEvqwnE9RJQr4puXD6wUWgLS xtHDJZUI6T/KiRzo9luo9656wObjJDmoxn6pTmgN3P9ydMSZF93Iha7BiM75GcaZSeTl gXNnRuo/hNkWVMflFLiPX98xzcLIDr5SmykhrxmBC3qYTUEUf3JInBRChhIsY+9pbwuL 8+cOT6RYMW53hhmknvnNxNfxNvzfm79huEnGiJW1jORRIKFSITmPhoOBv+QxaVLqPYNq MA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1dfsrkdk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:32:24 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A98bib019108; Fri, 10 May 2024 10:31:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcn41-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:41 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hf011786; Fri, 10 May 2024 10:31:40 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-8; Fri, 10 May 2024 10:31:40 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 07/11] libbpf: split BTF relocation Date: Fri, 10 May 2024 11:30:48 +0100 Message-Id: <20240510103052.850012-8-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: P-Zu09S7bgsDtxD7MGPrEsaqjncxbLjg X-Proofpoint-ORIG-GUID: P-Zu09S7bgsDtxD7MGPrEsaqjncxbLjg X-Patchwork-Delegate: bpf@iogearbox.net Map distilled base BTF type ids referenced in split BTF and their references to the base BTF passed in, and if the mapping succeeds, reparent the split BTF to the base BTF. Relocation is done by first verifying that distilled base BTF only consists of named INT, FLOAT, ENUM, FWD, STRUCT and UNION kinds; then we sort these to speed lookups. Once sorted, the base BTF is iterated, and for each relevant kind we check for an equivalent in distilled base BTF. When found, the mapping from distilled -> base BTF id is recorded. Once all mappings are established, we can update type ids in split BTF and reparent it to the new base. Signed-off-by: Alan Maguire --- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 59 +++++++ tools/lib/bpf/btf.h | 8 + tools/lib/bpf/btf_relocate.c | 264 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_internal.h | 2 + 6 files changed, 335 insertions(+), 1 deletion(-) create mode 100644 tools/lib/bpf/btf_relocate.c diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index b6619199a706..336da6844d42 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o + usdt.o zip.o elf.o features.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 1eb66a7a4c46..16bb1c538fa7 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5563,3 +5563,62 @@ int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, btf__free(new_base); return libbpf_err(err); } + +struct btf_rewrite_strs { + struct btf *btf; + const struct btf *old_base_btf; + int str_start; + int str_diff; +}; + +static int btf_rewrite_strs(__u32 *str_off, void *ctx) +{ + struct btf_rewrite_strs *r = ctx; + const char *s; + int off; + + if (!*str_off) + return 0; + if (*str_off >= r->str_start) { + *str_off += r->str_diff; + } else { + s = btf__str_by_offset(r->old_base_btf, *str_off); + if (!s) + return -ENOENT; + off = btf__add_str(r->btf, s); + if (off < 0) + return off; + *str_off = off; + } + return 0; +} + +int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + struct btf_rewrite_strs r = {}; + struct btf_type *t; + int i, err; + + r.old_base_btf = btf__base_btf(btf); + if (!r.old_base_btf) + return -EINVAL; + r.btf = btf; + r.str_start = r.old_base_btf->hdr->str_len; + r.str_diff = base_btf->hdr->str_len - r.old_base_btf->hdr->str_len; + btf->base_btf = base_btf; + btf->start_id = btf__type_cnt(base_btf); + btf->start_str_off = base_btf->hdr->str_len; + + for (i = 0; i < btf->nr_types; i++) { + t = (struct btf_type *)btf__type_by_id(btf, i + btf->start_id); + err = btf_type_visit_str_offs(t, btf_rewrite_strs, &r); + if (err) + break; + } + return err; +} + +int btf__relocate(struct btf *btf, const struct btf *base_btf) +{ + return btf_relocate(btf, base_btf, NULL); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e1702ad5ef4..f75db650e426 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -282,6 +282,14 @@ struct btf_dedup_opts { LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts); +/** + * @brief **btf__relocate()** will check the split BTF *btf* for references + * to base BTF kinds, and verify those references are compatible with + * *base_btf*; if they are, *btf* is adjusted such that is re-parented to + * *base_btf* and type ids and strings are adjusted to accommodate this. + */ +LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf); + struct btf_dump; struct btf_dump_opts { diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c new file mode 100644 index 000000000000..54949975398b --- /dev/null +++ b/tools/lib/bpf/btf_relocate.c @@ -0,0 +1,264 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include "btf.h" +#include "bpf.h" +#include "libbpf.h" +#include "libbpf_internal.h" + +struct btf; + +struct btf_relocate { + __u32 search_id; /* must be first field; see search below */ + struct btf *btf; + const struct btf *base_btf; + const struct btf *dist_base_btf; + unsigned int nr_base_types; + unsigned int nr_dist_base_types; + __u32 *map; + __u32 *dist_base_index; +}; + +static int btf_relocate_rewrite_type_id(__u32 *id, void *ctx) +{ + struct btf_relocate *r = ctx; + + *id = r->map[*id]; + return 0; +} + +/* Simple string comparison used for sorting within BTF, since all distilled types are + * named. + */ +static int cmp_btf_types(const void *id1, const void *id2, void *priv) +{ + const struct btf *btf = priv; + const struct btf_type *t1 = btf_type_by_id(btf, *(__u32 *)id1); + const struct btf_type *t2 = btf_type_by_id(btf, *(__u32 *)id2); + + return strcmp(btf__name_by_offset(btf, t1->name_off), + btf__name_by_offset(btf, t2->name_off)); +} + +/* Comparison between base BTF type (search type) and distilled base types (target). + * Because there is no bsearch_r() we need to use the search key - which also is + * the first element of struct btf_relocate * - as a means to retrieve the + * struct btf_relocate *. + */ +static int cmp_base_and_distilled_btf_types(const void *idbase, const void *iddist) +{ + struct btf_relocate *r = (struct btf_relocate *)idbase; + const struct btf_type *tbase = btf_type_by_id(r->base_btf, *(__u32 *)idbase); + const struct btf_type *tdist = btf_type_by_id(r->dist_base_btf, *(__u32 *)iddist); + + return strcmp(btf__name_by_offset(r->base_btf, tbase->name_off), + btf__name_by_offset(r->dist_base_btf, tdist->name_off)); +} + +/* Build a map from distilled base BTF ids to base BTF ids. To do so, iterate + * through base BTF looking up distilled type (using binary search) equivalents. + */ +static int btf_relocate_map_distilled_base(struct btf_relocate *r) +{ + struct btf_type *t; + const char *name; + __u32 id; + + /* generate a sort index array of type ids sorted by name for distilled + * base BTF to speed lookups. + */ + for (id = 1; id < r->nr_dist_base_types; id++) + r->dist_base_index[id] = id; + qsort_r(r->dist_base_index, r->nr_dist_base_types, sizeof(__u32), cmp_btf_types, + (struct btf *)r->dist_base_btf); + + for (id = 1; id < r->nr_base_types; id++) { + struct btf_type *dist_t; + int dist_kind, kind; + bool compat_kind; + __u32 *dist_id; + + t = btf_type_by_id(r->base_btf, id); + kind = btf_kind(t); + /* distilled base consists of named types only. */ + if (!t->name_off) + continue; + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_FWD: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + break; + default: + continue; + } + r->search_id = id; + dist_id = bsearch(&r->search_id, r->dist_base_index, r->nr_dist_base_types, + sizeof(__u32), cmp_base_and_distilled_btf_types); + if (!dist_id) + continue; + if (!*dist_id || *dist_id > r->nr_dist_base_types) { + pr_warn("base BTF id [%d] maps to invalid distilled base BTF id [%d]\n", + id, *dist_id); + return -EINVAL; + } + /* validate that kinds are compatible */ + dist_t = btf_type_by_id(r->dist_base_btf, *dist_id); + dist_kind = btf_kind(dist_t); + name = btf__name_by_offset(r->dist_base_btf, dist_t->name_off); + compat_kind = dist_kind == kind; + if (!compat_kind) { + switch (dist_kind) { + case BTF_KIND_FWD: + compat_kind = kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION; + break; + case BTF_KIND_ENUM: + compat_kind = kind == BTF_KIND_ENUM64; + break; + default: + break; + } + if (!compat_kind) { + pr_warn("kind incompatibility (%d != %d) between distilled base type '%s'[%d] and base type [%d]\n", + dist_kind, kind, name, *dist_id, id); + return -EINVAL; + } + } + /* validate that int, float struct, union sizes are compatible; + * distilled base BTF encodes an empty STRUCT/UNION with + * specific size for cases where a type is embedded in a split + * type (so has to preserve size info). Do not error out + * on mismatch as another size match may occur for an + * identically-named type. + */ + switch (btf_kind(dist_t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + if (t->size == dist_t->size) + break; + continue; + default: + break; + } + r->map[*dist_id] = id; + } + /* ensure all distilled BTF ids have a mapping... */ + for (id = 1; id < r->nr_dist_base_types; id++) { + t = btf_type_by_id(r->dist_base_btf, id); + name = btf__name_by_offset(r->dist_base_btf, t->name_off); + if (!r->map[id]) { + pr_warn("distilled base BTF type '%s' [%d] is not mapped to base BTF id\n", + name, id); + return -EINVAL; + } + } + return 0; +} + +/* distilled base should only have named int/float/enum/fwd/struct/union types. */ +static int btf_relocate_validate_distilled_base(struct btf_relocate *r) +{ + unsigned int i; + + for (i = 1; i < r->nr_dist_base_types; i++) { + struct btf_type *t = btf_type_by_id(r->dist_base_btf, i); + int kind = btf_kind(t); + + switch (kind) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_FWD: + if (t->name_off) + break; + pr_warn("type [%d], kind [%d] is invalid for distilled base BTF; it is anonymous\n", + i, kind); + return -EINVAL; + default: + pr_warn("type [%d] in distilled based BTF has unexpected kind [%d]\n", + i, kind); + return -EINVAL; + } + } + return 0; +} + +/* If successful, output of relocation is updated BTF with base BTF pointing + * at base_btf, and type ids, strings adjusted accordingly + */ +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids) +{ + const struct btf *dist_base_btf = btf__base_btf(btf); + unsigned int nr_types = btf__type_cnt(btf); + unsigned int nr_split_types; + struct btf_relocate r = {}; + struct btf_type *t; + int diff_id, err = 0; + __u32 id, i; + + if (!base_btf || dist_base_btf == base_btf) + return 0; + + r.nr_dist_base_types = btf__type_cnt(dist_base_btf); + r.nr_base_types = btf__type_cnt(base_btf); + nr_split_types = nr_types - r.nr_dist_base_types; + r.btf = btf; + r.dist_base_btf = dist_base_btf; + r.base_btf = base_btf; + + r.map = calloc(nr_types, sizeof(*r.map)); + r.dist_base_index = calloc(r.nr_dist_base_types, sizeof(*r.dist_base_index)); + if (!r.map || !r.dist_base_index) { + err = -ENOMEM; + goto err_out; + } + + err = btf_relocate_validate_distilled_base(&r); + if (err) + goto err_out; + + diff_id = r.nr_base_types - r.nr_dist_base_types; + /* Split BTF ids will start from after last base BTF id. */ + for (id = r.nr_dist_base_types; id < nr_types; id++) + r.map[id] = id + diff_id; + + /* Build a map from distilled base ids to actual base BTF ids; it is used + * to update split BTF id references. + */ + err = btf_relocate_map_distilled_base(&r); + if (err) + goto err_out; + + /* Next, rewrite type ids in split BTF, replacing split ids with updated + * ids based on number of types in base BTF, and base ids with + * relocated ids from base_btf. + */ + for (i = 0, id = r.nr_dist_base_types; i < nr_split_types; i++, id++) { + t = btf_type_by_id(btf, id); + err = btf_type_visit_type_ids(t, btf_relocate_rewrite_type_id, &r); + if (err) + goto err_out; + } + /* Finally reset base BTF to base_btf; as part of this operation, string + * offsets are also updated, and we are done. + */ + err = btf_set_base_btf(r.btf, (struct btf *)r.base_btf); +err_out: + if (!err && map_ids) + *map_ids = r.map; + else + free(r.map); + free(r.dist_base_index); + return err; +} diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index fd7bfeaba542..849cbe0def00 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -421,6 +421,7 @@ LIBBPF_1.5.0 { global: btf__distill_base; btf__parse_opts; + btf__relocate; bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index a0dcfb82e455..e38e1b01e86e 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -234,6 +234,8 @@ struct btf_type; struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id); const char *btf_kind_str(const struct btf_type *t); const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id); +int btf_set_base_btf(struct btf *btf, struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t) { From patchwork Fri May 10 10:30:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661289 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3FDE7F7DB for ; Fri, 10 May 2024 10:32:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337147; cv=none; b=U4xwlYSH6xHgkYqJqjbgNALahYlHLVUT0XU/MxYfHSfY14dFaMCYEsVQIUc3T2G8wKS+2Ol0T2Ya/xwtEnLIudX25WFziC/io6rjQzFC5dbHPh1jKD3OuFPeFZPpB6UqGwaMYixbACIteY79WgfQOfQXwbt6+1Eif2JesVb+/e4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337147; c=relaxed/simple; bh=a/FTdW+K8MYPUxvv2VtY8wlrjYWbRGxQQ4H4Wh8mP2E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZyQ313QRn8H/w8wBJ+JoqPsEnDz1P0WFSIFa453nDKbR2W92d5tkMcFGzQefSBe7voqRGueqzaHeezbDFVsbRP9NakzJ8SG78w7RsIw/hmmX1NJZXDnnZTZQdb7lhXKDxFMYh77v6inlz13NTR62UDKp2gfsb4Xz4usHmFUcz0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=UfSV5NHY; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="UfSV5NHY" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AATv10004001; Fri, 10 May 2024 10:31:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=e7hjPRua42vJK0UPlnG4KSHOFG+k2R8Dv4x9nUKjD54=; b=UfSV5NHYwmmbSlWFhC7HqKBHIxCBQtI9W/ikdSfLUR82s5K3t6Kns0tOw5PAzD33s6sO /b8SRvmOguk/8cBkuyLTAyjonQvi8WhPjBQZJ2AEedu1lEoqxIZxkzI5ax+9uvDVYfYx secF4LUCgIfO7I+LFKrd/hhXqS/cFhOucDRZWe8riXxLaObVSOkNotbsRJb2+/C7nAiE 05GnPcwlxjhg4hcXFRriIHzIdHUJ2Df4YDlNmOy5aAKKACiRVfa4tgGIochw8n6yWibu ++mAA9tfR/UJ8xDrGD6h0HVhPmFPjDrahx7ASp6IiwTGCFBrE9MpKEl2mRirPzdddZ03 VA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1hps80vh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:47 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A8Wjb5019738; Fri, 10 May 2024 10:31:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcn65-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:46 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hh011786; Fri, 10 May 2024 10:31:45 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-9; Fri, 10 May 2024 10:31:45 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 08/11] selftests/bpf: extend distilled BTF tests to cover BTF relocation Date: Fri, 10 May 2024 11:30:49 +0100 Message-Id: <20240510103052.850012-9-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-GUID: 2ZEBa2n4Bi2JGtfumXTyL1xmCHsUjrp4 X-Proofpoint-ORIG-GUID: 2ZEBa2n4Bi2JGtfumXTyL1xmCHsUjrp4 X-Patchwork-Delegate: bpf@iogearbox.net Ensure relocated BTF looks as expected; in this case identical to original split BTF, with a few duplicate anonymous types added to split BTF by the relocation process. Signed-off-by: Alan Maguire Acked-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/btf_distill.c | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c index 400df8740943..5989bafe5de7 100644 --- a/tools/testing/selftests/bpf/prog_tests/btf_distill.c +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -211,6 +211,71 @@ static void test_distilled_base(void) "\t'p1' type_id=1", "[25] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + if (!ASSERT_EQ(btf__relocate(btf4, btf1), 0, "relocate_split")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] STRUCT 'from_proto' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[17] PTR '(anon)' type_id=3", + "[18] PTR '(anon)' type_id=29", + "[19] CONST '(anon)' type_id=6", + "[20] RESTRICT '(anon)' type_id=30", + "[21] VOLATILE '(anon)' type_id=8", + "[22] TYPEDEF 'et' type_id=31", + "[23] CONST '(anon)' type_id=10", + "[24] PTR '(anon)' type_id=32", + "[25] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[26] FUNC 'fn' type_id=33 linkage=static", + "[27] TYPEDEF 'arraytype' type_id=34", + "[28] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=16", + /* below here are (duplicate) anon base types added by distill + * process to split BTF. + */ + "[29] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[30] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[31] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[32] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[33] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[34] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + cleanup: btf__free(btf4); btf__free(btf3); From patchwork Fri May 10 10:30:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661290 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7B747F7DB for ; Fri, 10 May 2024 10:32:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337150; cv=none; b=f3l+xlRdo8VRCO/YFGJCSwaUlVpi21XSJM1OB4I2tmiHs1ncjYXjSxO9W/xMIgr8WWsL3aLMBmSO6kJUWgzVUmYtNMtDAy2K5pIoQu2+TKU96rJY0bGWXw660Tre5x5sW1CHwKgGLh+P5RVg11BUx0YDJQfYgpJEsw4P6pZZcug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337150; c=relaxed/simple; bh=cBcfmcVBrgf6azkPS8GIkEODkINDbJ+MQWMXCGDO06I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GTuLyNuYz3TQba+dNsFwJ96K9CZbTsBFHKBzxW/uyqYdA9vKfp9i5fvvpj3eJD2eQ1omBHtC1AAnXVYlLsrItjRnFWdFbRX/eWTxeuSKo8hEbzpRyxSnrbJOGVHSIphd5jPKD+XDcD1v4VfyeEA5IF371TzpQKh6WuYE5/i8f5k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KEmpschX; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KEmpschX" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AASw9m015485; Fri, 10 May 2024 10:31:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=NLTtqzFIHAlwjSGVFNQNXoDScRZ3Sw+F0BFfEuNRB7Q=; b=KEmpschXAVTjv3bXXv+cC007wNqU7T3AFY0K5vowjlG4BOEzSOU6miXdxxiSLOGctxre LQZLsSE+aJJLVAY3x5bpOAicnfqF7nxcd5hCvW2nt10oofhlUOQ9h8IXg9aZRZYBRDiV 1CjWsiEHTvcTB1+QoyRXz7ypVGP9sMyLfLyt5eoh0gWCPHSt7d4qhtfRUy7Bb4J/pVoe YVZIyqt+U4div4yumponQVFaABhkflwTMOUJ85u29mz2u+JHel2CICuIiM0Yi8ZcuqKf 9be6PiBxiXWcmLfa3cadG9e/riNoWbcLRLz8QQEHR+jHm/2zFsg0nRGLheCsOjgNsqJ4 dg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1a0j8pf8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:53 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44AA00xO019748; Fri, 10 May 2024 10:31:52 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcn8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:52 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hj011786; Fri, 10 May 2024 10:31:51 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-10; Fri, 10 May 2024 10:31:51 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 09/11] module, bpf: store BTF base pointer in struct module Date: Fri, 10 May 2024 11:30:50 +0100 Message-Id: <20240510103052.850012-10-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-ORIG-GUID: q31h4cfS-aMvnfzix_m4bxfnMlWIDvgu X-Proofpoint-GUID: q31h4cfS-aMvnfzix_m4bxfnMlWIDvgu X-Patchwork-Delegate: bpf@iogearbox.net ...as this will allow split BTF modules with a base BTF representation (rather than the full vmlinux BTF at time of BTF encoding) to resolve their references to kernel types in a way that is more resilient to small changes in kernel types. This will allow modules that are not built every time the kernel is to provide more resilient BTF, rather than have it invalidated every time BTF ids for core kernel types change. Signed-off-by: Alan Maguire --- include/linux/module.h | 2 ++ kernel/module/main.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/module.h b/include/linux/module.h index 1153b0d99a80..f127a79a95d9 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -510,6 +510,8 @@ struct module { #ifdef CONFIG_DEBUG_INFO_BTF_MODULES unsigned int btf_data_size; void *btf_data; + unsigned int btf_base_data_size; + void *btf_base_data; #endif #ifdef CONFIG_JUMP_LABEL struct jump_entry *jump_entries; diff --git a/kernel/module/main.c b/kernel/module/main.c index e1e8a7a9d6c1..e18683abec07 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2148,6 +2148,8 @@ static int find_module_sections(struct module *mod, struct load_info *info) #endif #ifdef CONFIG_DEBUG_INFO_BTF_MODULES mod->btf_data = any_section_objs(info, ".BTF", 1, &mod->btf_data_size); + mod->btf_base_data = any_section_objs(info, ".BTF.base", 1, + &mod->btf_base_data_size); #endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries = section_objs(info, "__jump_table", @@ -2587,8 +2589,9 @@ static noinline int do_init_module(struct module *mod) } #ifdef CONFIG_DEBUG_INFO_BTF_MODULES - /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointer */ + /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointers */ mod->btf_data = NULL; + mod->btf_base_data = NULL; #endif /* * We want to free module_init, but be aware that kallsyms may be From patchwork Fri May 10 10:30:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661299 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C99557EEF8 for ; Fri, 10 May 2024 10:43:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337811; cv=none; b=Qxdf8YB5QGXVurTwaobb60bg92EPecAMb9mdZsrPlgVSxQRGaMq9gRQqOD69PBbw+aWoeHyfFlPZ2hKld735BlDWFV2984oQ2j07OUA0UWaTGfaZGELxDYr1YRNrqkdJh/HXEH8nBZFAA9SWQuKSHV8tN6uqYqEut+XXYBunKwY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337811; c=relaxed/simple; bh=k5rg9m6RX2epqggh1noLXeYi/qn10S0ejy3TbCLtkBc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UIfdt1G+sFHPjXUMxMmQ6Betu6uNUSVWYkLFpCxNNDpsaKhZy4UJZzYsBt3KdJlHmVDsYsndPOUk3DHsUARr05qioV+Lk1Yt3ABebBkPx2HCPJv/veSDzfRTvugGPtAOAryya1GLCl2dB6JVPX1iyzRXihzeUiUjxuZHQHwDBaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=K+tUCmAm; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="K+tUCmAm" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAfLJ1002909; Fri, 10 May 2024 10:43:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=lrMjvyWZtIjmSvAZAfd7LIo8zdYuOWFKnHkWj3krOMk=; b=K+tUCmAm7P2nB8XmN4qv3ZMFTooczWcxKawX4P+2JWV5JQQpnHErTH4mwN24O1PhR1wO /FcYuEQRMZmhbJbqoMGJRRyfIUhFnmC90HcVEYRnoLBwr7lab1ehkkXZi2/m5Gxf3Ryr 33Ofq5IlcPbyGsxxiUa48nhu+0P65KQmj6rhgSh8XDfAECEYRi/tCO0C8LdcIR8k9yO1 Rj0/FJ3dTcHCW15Pb7tqdMeftsFtGnSiHAStMBhbZYUOYytF3a/Rgmrfdv/RFGleWSEJ hZJ5ZD/sVIt5vRzIJnVELwaEvggZNiIz9kEckjGnCzFtiC2rRfPQmQs0epMGbn9gzzDq 6g== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1h49r282-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:43:04 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A9Mx0x019119; Fri, 10 May 2024 10:31:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcnad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:31:58 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hl011786; Fri, 10 May 2024 10:31:57 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-11; Fri, 10 May 2024 10:31:57 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 10/11] libbpf,bpf: share BTF relocate-related code with kernel Date: Fri, 10 May 2024 11:30:51 +0100 Message-Id: <20240510103052.850012-11-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-ORIG-GUID: w7zuDRVl-G_LqLhuzkFabS50dh9kPMVv X-Proofpoint-GUID: w7zuDRVl-G_LqLhuzkFabS50dh9kPMVv X-Patchwork-Delegate: bpf@iogearbox.net Share relocation implementation with the kernel. As part of this, we also need the type/string visitation functions so add them to a btf_common.c file that also gets shared with the kernel. Relocation code in kernel and userspace is identical save for the impementation of the reparenting of split BTF to the relocated base BTF; this depends on struct btf internals so is different in kernel and userspace. One other wrinkle on the kernel side is we have to map .BTF.ids in modules as they were generated with the type ids used at BTF encoding time. btf_relocate() optionally returns an array mapping from old BTF ids to relocated ids, so we use that to fix up these references where needed for kfuncs. Signed-off-by: Alan Maguire --- include/linux/btf.h | 32 +++++ kernel/bpf/Makefile | 8 ++ kernel/bpf/btf.c | 227 ++++++++++++++++++++++++++++------- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 130 -------------------- tools/lib/bpf/btf_common.c | 146 ++++++++++++++++++++++ tools/lib/bpf/btf_relocate.c | 32 +++++ 7 files changed, 402 insertions(+), 175 deletions(-) create mode 100644 tools/lib/bpf/btf_common.c diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..1cc20844f163 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -214,6 +214,7 @@ bool btf_is_kernel(const struct btf *btf); bool btf_is_module(const struct btf *btf); struct module *btf_try_get_module(const struct btf *btf); u32 btf_nr_types(const struct btf *btf); +struct btf *btf_base_btf(const struct btf *btf); bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s, const struct btf_member *m, u32 expected_offset, u32 expected_size); @@ -515,8 +516,15 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf * } #endif +typedef int (*type_id_visit_fn)(__u32 *type_id, void *ctx); +typedef int (*str_off_visit_fn)(__u32 *str_off, void *ctx); + #ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); +int btf_set_base_btf(struct btf *btf, struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx); +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx); const char *btf_name_by_offset(const struct btf *btf, u32 offset); struct btf *btf_parse_vmlinux(void); struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog); @@ -543,6 +551,30 @@ static inline const struct btf_type *btf_type_by_id(const struct btf *btf, { return NULL; } + +static inline int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + return 0; +} + +static inline int btf_relocate(void *log, struct btf *btf, const struct btf *base_btf, + __u32 **map_ids) +{ + return 0; +} + +static inline int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, + void *ctx) +{ + return 0; +} + +static inline int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, + void *ctx) +{ + return 0; +} + static inline const char *btf_name_by_offset(const struct btf *btf, u32 offset) { diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 7eb9ad3a3ae6..612eef1228ca 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -52,3 +52,11 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o $(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_common.o +$(obj)/btf_common.o: $(srctree)/tools/lib/bpf/btf_common.c FORCE + $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +$(obj)/btf_relocate.o: $(srctree)/tools/lib/bpf/btf_relocate.c FORCE + $(call if_changed_rule,cc_o_c) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 821063660d9f..82bd2a275a12 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -274,6 +274,7 @@ struct btf { u32 start_str_off; /* first string offset (0 for base BTF) */ char name[MODULE_NAME_LEN]; bool kernel_btf; + __u32 *base_map; /* map from distilled base BTF -> vmlinux BTF ids */ }; enum verifier_phase { @@ -1735,7 +1736,13 @@ static void btf_free(struct btf *btf) kvfree(btf->types); kvfree(btf->resolved_sizes); kvfree(btf->resolved_ids); - kvfree(btf->data); + /* only split BTF allocates data, but btf->data is non-NULL for + * vmlinux BTF too. + */ + if (btf->base_btf) + kvfree(btf->data); + if (btf->kernel_btf) + kvfree(btf->base_map); kfree(btf); } @@ -1764,6 +1771,90 @@ void btf_put(struct btf *btf) } } +struct btf *btf_base_btf(const struct btf *btf) +{ + return btf->base_btf; +} + +struct btf_rewrite_strs { + struct btf *btf; + const struct btf *old_base_btf; + int str_start; + int str_diff; + __u32 *str_map; +}; + +static __u32 btf_find_str(struct btf *btf, const char *s) +{ + __u32 offset = 0; + + while (offset < btf->hdr.str_len) { + while (!btf->strings[offset]) + offset++; + if (strcmp(s, &btf->strings[offset]) == 0) + return offset; + while (btf->strings[offset]) + offset++; + } + return -ENOENT; +} + +static int btf_rewrite_strs(__u32 *str_off, void *ctx) +{ + struct btf_rewrite_strs *r = ctx; + const char *s; + int off; + + if (!*str_off) + return 0; + if (*str_off >= r->str_start) { + *str_off += r->str_diff; + } else { + s = btf_str_by_offset(r->old_base_btf, *str_off); + if (!s) + return -ENOENT; + if (r->str_map[*str_off]) { + off = r->str_map[*str_off]; + } else { + off = btf_find_str(r->btf->base_btf, s); + if (off < 0) + return off; + r->str_map[*str_off] = off; + } + *str_off = off; + } + return 0; +} + +int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + struct btf_rewrite_strs r = {}; + struct btf_type *t; + int i, err; + + r.old_base_btf = btf_base_btf(btf); + if (!r.old_base_btf) + return -EINVAL; + r.btf = btf; + r.str_start = r.old_base_btf->hdr.str_len; + r.str_diff = base_btf->hdr.str_len - r.old_base_btf->hdr.str_len; + r.str_map = kvcalloc(r.old_base_btf->hdr.str_len, sizeof(*r.str_map), + GFP_KERNEL | __GFP_NOWARN); + if (!r.str_map) + return -ENOMEM; + btf->base_btf = base_btf; + btf->start_id = btf_nr_types(base_btf); + btf->start_str_off = base_btf->hdr.str_len; + for (i = 0; i < btf->nr_types; i++) { + t = (struct btf_type *)btf_type_by_id(btf, i + btf->start_id); + err = btf_type_visit_str_offs((struct btf_type *)t, btf_rewrite_strs, &r); + if (err) + break; + } + kvfree(r.str_map); + return err; +} + static int env_resolve_init(struct btf_verifier_env *env) { struct btf *btf = env->btf; @@ -5982,23 +6073,15 @@ int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_ty BTF_ID_LIST(bpf_ctx_convert_btf_id) BTF_ID(struct, bpf_ctx_convert) -struct btf *btf_parse_vmlinux(void) +static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name, + void *data, unsigned int data_size) { - struct btf_verifier_env *env = NULL; - struct bpf_verifier_log *log; struct btf *btf = NULL; int err; if (!IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) return ERR_PTR(-ENOENT); - env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); - if (!env) - return ERR_PTR(-ENOMEM); - - log = &env->log; - log->level = BPF_LOG_KERNEL; - btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6006,10 +6089,10 @@ struct btf *btf_parse_vmlinux(void) } env->btf = btf; - btf->data = __start_BTF; - btf->data_size = __stop_BTF - __start_BTF; + btf->data = data; + btf->data_size = data_size; btf->kernel_btf = true; - snprintf(btf->name, sizeof(btf->name), "vmlinux"); + snprintf(btf->name, sizeof(btf->name), "%s", name); err = btf_parse_hdr(env); if (err) @@ -6029,20 +6112,11 @@ struct btf *btf_parse_vmlinux(void) if (err) goto errout; - /* btf_parse_vmlinux() runs under bpf_verifier_lock */ - bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); - refcount_set(&btf->refcnt, 1); - err = btf_alloc_id(btf); - if (err) - goto errout; - - btf_verifier_env_free(env); return btf; errout: - btf_verifier_env_free(env); if (btf) { kvfree(btf->types); kfree(btf); @@ -6050,19 +6124,59 @@ struct btf *btf_parse_vmlinux(void) return ERR_PTR(err); } +struct btf *btf_parse_vmlinux(void) +{ + struct btf_verifier_env *env = NULL; + struct bpf_verifier_log *log; + struct btf *btf; + int err; + + env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); + if (!env) + return ERR_PTR(-ENOMEM); + + log = &env->log; + log->level = BPF_LOG_KERNEL; + btf = btf_parse_base(env, "vmlinux", __start_BTF, __stop_BTF - __start_BTF); + if (!IS_ERR(btf)) { + /* btf_parse_vmlinux() runs under bpf_verifier_lock */ + bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); + err = btf_alloc_id(btf); + if (err) { + btf_free(btf); + btf = ERR_PTR(err); + } + } + btf_verifier_env_free(env); + return btf; +} + #ifdef CONFIG_DEBUG_INFO_BTF_MODULES -static struct btf *btf_parse_module(const char *module_name, const void *data, unsigned int data_size) +/* If .BTF_ids section was created with distilled base BTF, both base and + * split BTF ids will need to be mapped to actual base/split ids for + * BTF now that it has been relocated. + */ +static __u32 btf_id_map(const struct btf *btf, __u32 id) +{ + if (!btf->base_btf || !btf->base_map) + return id; + return btf->base_map[id]; +} + +static struct btf *btf_parse_module(const char *module_name, const void *data, + unsigned int data_size, void *base_data, + unsigned int base_data_size) { + struct btf *btf = NULL, *vmlinux_btf, *base_btf = NULL; struct btf_verifier_env *env = NULL; struct bpf_verifier_log *log; - struct btf *btf = NULL, *base_btf; - int err; + int err = 0; - base_btf = bpf_get_btf_vmlinux(); - if (IS_ERR(base_btf)) - return base_btf; - if (!base_btf) + vmlinux_btf = bpf_get_btf_vmlinux(); + if (IS_ERR(vmlinux_btf)) + return vmlinux_btf; + if (!vmlinux_btf) return ERR_PTR(-EINVAL); env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); @@ -6072,6 +6186,16 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u log = &env->log; log->level = BPF_LOG_KERNEL; + if (base_data) { + base_btf = btf_parse_base(env, ".BTF.base", base_data, base_data_size); + if (IS_ERR(base_btf)) { + err = PTR_ERR(base_btf); + goto errout; + } + } else { + base_btf = vmlinux_btf; + } + btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6111,12 +6235,22 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u if (err) goto errout; + if (base_btf != vmlinux_btf) { + err = btf_relocate(btf, vmlinux_btf, &btf->base_map); + if (err) + goto errout; + btf_free(base_btf); + base_btf = vmlinux_btf; + } + btf_verifier_env_free(env); refcount_set(&btf->refcnt, 1); return btf; errout: btf_verifier_env_free(env); + if (base_btf != vmlinux_btf) + btf_free(base_btf); if (btf) { kvfree(btf->data); kvfree(btf->types); @@ -7669,7 +7803,8 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op, err = -ENOMEM; goto out; } - btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size); + btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size, + mod->btf_base_data, mod->btf_base_data_size); if (IS_ERR(btf)) { kfree(btf_mod); if (!IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH)) { @@ -7993,7 +8128,7 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, bool add_filter = !!kset->filter; struct btf_kfunc_set_tab *tab; struct btf_id_set8 *set; - u32 set_cnt; + u32 set_cnt, i; int ret; if (hook >= BTF_KFUNC_HOOK_MAX) { @@ -8039,21 +8174,15 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, goto end; } - /* We don't need to allocate, concatenate, and sort module sets, because - * only one is allowed per hook. Hence, we can directly assign the - * pointer and return. - */ - if (!vmlinux_set) { - tab->sets[hook] = add_set; - goto do_add_filter; - } - /* In case of vmlinux sets, there may be more than one set being * registered per hook. To create a unified set, we allocate a new set * and concatenate all individual sets being registered. While each set * is individually sorted, they may become unsorted when concatenated, * hence re-sorting the final set again is required to make binary * searching the set using btf_id_set8_contains function work. + * + * For module sets, we need to allocate as we may need to relocate + * BTF ids. */ set_cnt = set ? set->cnt : 0; @@ -8083,11 +8212,14 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, /* Concatenate the two sets */ memcpy(set->pairs + set->cnt, add_set->pairs, add_set->cnt * sizeof(set->pairs[0])); + /* Now that the set is copied, update with relocated BTF ids */ + for (i = set->cnt; i < set->cnt + add_set->cnt; i++) + set->pairs[i].id = btf_id_map(btf, set->pairs[i].id); + set->cnt += add_set->cnt; sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL); -do_add_filter: if (add_filter) { hook_filter = &tab->hook_filters[hook]; hook_filter->filters[hook_filter->nr_filters++] = kset->filter; @@ -8207,7 +8339,7 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook, return PTR_ERR(btf); for (i = 0; i < kset->set->cnt; i++) { - ret = btf_check_kfunc_protos(btf, kset->set->pairs[i].id, + ret = btf_check_kfunc_protos(btf, btf_id_map(btf, kset->set->pairs[i].id), kset->set->pairs[i].flags); if (ret) goto err_out; @@ -8306,7 +8438,7 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c { struct btf_id_dtor_kfunc_tab *tab; struct btf *btf; - u32 tab_cnt; + u32 tab_cnt, i; int ret; btf = btf_get_module_btf(owner); @@ -8357,6 +8489,13 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c btf->dtor_kfunc_tab = tab; memcpy(tab->dtors + tab->cnt, dtors, add_cnt * sizeof(tab->dtors[0])); + + /* remap BTF ids based on BTF relocation (if any) */ + for (i = tab_cnt; i < tab_cnt + add_cnt; i++) { + tab->dtors[i].btf_id = btf_id_map(btf, tab->dtors[i].btf_id); + tab->dtors[i].kfunc_btf_id = btf_id_map(btf, tab->dtors[i].kfunc_btf_id); + } + tab->cnt += add_cnt; sort(tab->dtors, tab->cnt, sizeof(tab->dtors[0]), btf_id_cmp_func, NULL); diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index 336da6844d42..567abaa52131 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o btf_relocate.o + usdt.o zip.o elf.o features.o btf_common.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 16bb1c538fa7..fab1635a9843 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5026,136 +5026,6 @@ struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_bt return btf__parse_split(path, vmlinux_btf); } -int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) -{ - int i, n, err; - - switch (btf_kind(t)) { - case BTF_KIND_INT: - case BTF_KIND_FLOAT: - case BTF_KIND_ENUM: - case BTF_KIND_ENUM64: - return 0; - - case BTF_KIND_FWD: - case BTF_KIND_CONST: - case BTF_KIND_VOLATILE: - case BTF_KIND_RESTRICT: - case BTF_KIND_PTR: - case BTF_KIND_TYPEDEF: - case BTF_KIND_FUNC: - case BTF_KIND_VAR: - case BTF_KIND_DECL_TAG: - case BTF_KIND_TYPE_TAG: - return visit(&t->type, ctx); - - case BTF_KIND_ARRAY: { - struct btf_array *a = btf_array(t); - - err = visit(&a->type, ctx); - err = err ?: visit(&a->index_type, ctx); - return err; - } - - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - err = visit(&t->type, ctx); - if (err) - return err; - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_DATASEC: { - struct btf_var_secinfo *m = btf_var_secinfos(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - default: - return -EINVAL; - } -} - -int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) -{ - int i, n, err; - - err = visit(&t->name_off, ctx); - if (err) - return err; - - switch (btf_kind(t)) { - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM: { - struct btf_enum *m = btf_enum(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM64: { - struct btf_enum64 *m = btf_enum64(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - default: - break; - } - - return 0; -} - int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx) { const struct btf_ext_info *seg; diff --git a/tools/lib/bpf/btf_common.c b/tools/lib/bpf/btf_common.c new file mode 100644 index 000000000000..ddec3f3ac423 --- /dev/null +++ b/tools/lib/bpf/btf_common.c @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +/* Copyright (c) 2021 Facebook */ +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifdef __KERNEL__ +#include +#include + +static inline struct btf_var_secinfo *btf_var_secinfos(const struct btf_type *t) +{ + return (struct btf_var_secinfo *)(t + 1); +} + +#else +#include "btf.h" +#include "libbpf_internal.h" +#endif + +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) +{ + int i, n, err; + + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + return 0; + + case BTF_KIND_FWD: + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_VAR: + case BTF_KIND_DECL_TAG: + case BTF_KIND_TYPE_TAG: + return visit(&t->type, ctx); + + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + err = visit(&a->type, ctx); + err = err ?: visit(&a->index_type, ctx); + return err; + } + + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + err = visit(&t->type, ctx); + if (err) + return err; + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + case BTF_KIND_DATASEC: { + struct btf_var_secinfo *m = btf_var_secinfos(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + default: + return -EINVAL; + } +} + +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) +{ + int i, n, err; + + err = visit(&t->name_off, ctx); + if (err) + return err; + + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM: { + struct btf_enum *m = btf_enum(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM64: { + struct btf_enum64 *m = btf_enum64(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + default: + break; + } + + return 0; +} diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c index 54949975398b..4a1fcb260f7f 100644 --- a/tools/lib/bpf/btf_relocate.c +++ b/tools/lib/bpf/btf_relocate.c @@ -5,11 +5,43 @@ #define _GNU_SOURCE #endif +#ifdef __KERNEL__ +#include +#include +#include +#include +#include +#include + +#define btf_type_by_id (struct btf_type *)btf_type_by_id +#define btf__type_cnt btf_nr_types +#define btf__base_btf btf_base_btf +#define btf__name_by_offset btf_name_by_offset +#define btf_kflag btf_type_kflag + +#define calloc(nmemb, sz) kvcalloc(nmemb, sz, GFP_KERNEL | __GFP_NOWARN) +#define free(ptr) kvfree(ptr) +#define qsort_r(base, num, sz, cmp, priv) sort_r(base, num, sz, (cmp_r_func_t)cmp, NULL, priv) + +static inline __u8 btf_int_bits(const struct btf_type *t) +{ + return BTF_INT_BITS(*(__u32 *)(t + 1)); +} + +static inline struct btf_decl_tag *btf_decl_tag(const struct btf_type *t) +{ + return (struct btf_decl_tag *)(t + 1); +} + +#else + #include "btf.h" #include "bpf.h" #include "libbpf.h" #include "libbpf_internal.h" +#endif /* __KERNEL__ */ + struct btf; struct btf_relocate { From patchwork Fri May 10 10:30:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13661298 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 327CC14F137 for ; Fri, 10 May 2024 10:43:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337811; cv=none; b=lETb/fH2jl1WJHfX5pvAEd/kFO1XMWUtsnwr4YubleqHL+Rh5wEMKH2ZCdgcjNedj2Awoz5S9UXVf6IlZkPU0u/lOHM7W0j+F1hUPXbuT3uMkYsK+tFyOyIizJSbVMFxAHH+yLF9+GjrpxATLFCJnL7ZxWm3+zPs10L2z8Dt9Jk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715337811; c=relaxed/simple; bh=9JfDxMtfmqJhDMeZyk7ksnoqImcLxnm+jr1GDjkqBgY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R7aRUHlWAW2GgtCGt3ttFhYf8d2/oRwSFXAD62jm420qqAipzeF7xAh3wnUsnjy1dLlpO/SdpVbi/GSkF0hvYgooZn/4LdV+BrvNDtASvsNZtJHYrdugb8dHvbif4COXNG4yFLTgbqJtNeotmGBfjUaX5qxBlr02manHK/n+rGE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=LOLRoIAY; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="LOLRoIAY" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 44AAfLJ4002909; Fri, 10 May 2024 10:43:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=tlpXrsO7MPu5YDaqTyIG/U365+CiySUGupQ2MheEDJs=; b=LOLRoIAYJ2KTTv2bogUlRqXaTnXA5Kiey5PoYmp+MraIqfXBzdwL2v0mbyG1+/8md+eF aCh4HMH1wPeS1XhUKLPOAajIUGwnXFrGC53osI7asHBFlEsgKDO+nP+DNNdEYViBH/3E 46dnvw3FIPfFDrvpCC+Q3sozkOizrjJkpm8wnxq3T9WWCbkLilKjqGgZyULj3tD1sLE7 2g2XaAedG1cIjwHJbZzhgMJjTn0NNdFJlZEP7eHp5XVLm63CRgVXcFPk8GxYPmCtzCU4 Wuns8pH/l1aqOAlkEBq0BMkurJTFPow2UcbjU1ROTb6pgaYnkyuh6TetvKh6fYaHl82v MQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3y1h49r282-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:43:05 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 44A9fMnF019784; Fri, 10 May 2024 10:32:04 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xysfpcngt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 10 May 2024 10:32:04 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 44AAV0hn011786; Fri, 10 May 2024 10:32:03 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-161-199.vpn.oracle.com [10.175.161.199]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xysfpcm4p-12; Fri, 10 May 2024 10:32:03 +0000 From: Alan Maguire To: andrii@kernel.org, jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com Cc: eddyz87@gmail.com, mykolal@fb.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v3 bpf-next 11/11] bpftool: support displaying relocated-with-base split BTF Date: Fri, 10 May 2024 11:30:52 +0100 Message-Id: <20240510103052.850012-12-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240510103052.850012-1-alan.maguire@oracle.com> References: <20240510103052.850012-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-05-10_07,2024-05-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000 definitions=main-2405100074 X-Proofpoint-ORIG-GUID: NfHriO6xhJZWtWSqCo1TOqPfgYI1cvnb X-Proofpoint-GUID: NfHriO6xhJZWtWSqCo1TOqPfgYI1cvnb X-Patchwork-Delegate: bpf@iogearbox.net If the -R option is used, we can display BTF that has been generated with distilled base BTF in its relocated form. For example for bpf_testmod.ko (which is built as an out-of-tree module, so has a distilled .BTF.base section: bpftool btf dump file bpf_testmod.ko Alternatively, we can display content relocated with (a possibly changed) base BTF via bpftool btf dump -R /sys/kernel/btf/vmlinux bpf_testmod.ko The latter mirrors how the kernel will handle such split BTF; it relocates its representation with the running kernel, and if successful, renumbers BTF ids to reference the current vmlinux BTF. Signed-off-by: Alan Maguire Acked-by: Eduard Zingerman Reviewed-by: Quentin Monnet --- tools/bpf/bpftool/Documentation/bpftool-btf.rst | 15 ++++++++++++++- tools/bpf/bpftool/bash-completion/bpftool | 7 ++++--- tools/bpf/bpftool/btf.c | 11 ++++++++++- tools/bpf/bpftool/main.c | 14 +++++++++++++- tools/bpf/bpftool/main.h | 2 ++ 5 files changed, 43 insertions(+), 6 deletions(-) diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst index eaba24320fb2..fd6bb1280e7b 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst @@ -16,7 +16,7 @@ SYNOPSIS **bpftool** [*OPTIONS*] **btf** *COMMAND* -*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } } +*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } { **-R** | **relocate-base-btf** } } *COMMANDS* := { **dump** | **help** } @@ -85,6 +85,19 @@ OPTIONS BTF object is passed through other handles, this option becomes necessary. +-R, --relocate-base-btf *FILE* + When split BTF is generated with distilled base BTF for relocation, + the latter is stored in a .BTF.base section and allows us to later + relocate split BTF and a potentially-changed base BTF by using + information in the .BTF.base section about the base types referenced + from split BTF. Relocation is carried out against the split BTF + supplied via this parameter and the split BTF will then refer to + the base types supplied in *FILE*. + + If this option is not used, split BTF is shown relative to the + .BTF.base, which contains just enough information to support later + relocation. + EXAMPLES ======== **# bpftool btf dump id 1226** diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 04afe2ac2228..878cf3d49a76 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -262,7 +262,7 @@ _bpftool() # Deal with options if [[ ${words[cword]} == -* ]]; then local c='--version --json --pretty --bpffs --mapcompat --debug \ - --use-loader --base-btf' + --use-loader --base-btf --relocate-base-btf' COMPREPLY=( $( compgen -W "$c" -- "$cur" ) ) return 0 fi @@ -283,7 +283,7 @@ _bpftool() _sysfs_get_netdevs return 0 ;; - file|pinned|-B|--base-btf) + file|pinned|-B|-R|--base-btf|--relocate-base-btf) _filedir return 0 ;; @@ -297,7 +297,8 @@ _bpftool() local i pprev for (( i=1; i < ${#words[@]}; )); do if [[ ${words[i]::1} == - ]] && - [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]]; then + [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]] && + [[ ${words[i]} != "-R" ]] && [[ ${words[i]} != "--relocate-base-btf" ]]; then words=( "${words[@]:0:i}" "${words[@]:i+1}" ) [[ $i -le $cword ]] && cword=$(( cword - 1 )) else diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 0ca1f2417801..34f60d9e433d 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -638,6 +638,14 @@ static int do_dump(int argc, char **argv) base_btf = btf__parse_opts(*argv, &optp); if (base_btf) btf = btf__parse_split(*argv, base_btf); + if (btf && relocate_base_btf) { + err = btf__relocate(btf, relocate_base_btf); + if (err) { + p_err("could not relocate BTF from '%s' with base BTF '%s': %s\n", + *argv, relocate_base_btf_path, strerror(-err)); + goto done; + } + } } if (!btf) { err = -errno; @@ -1075,7 +1083,8 @@ static int do_help(int argc, char **argv) " " HELP_SPEC_MAP "\n" " " HELP_SPEC_PROGRAM "\n" " " HELP_SPEC_OPTIONS " |\n" - " {-B|--base-btf} }\n" + " {-B|--base-btf} |\n" + " {-R|--relocate-base-btf} }\n" "", bin_name, "btf"); diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c index 08d0ac543c67..69d4906bec5c 100644 --- a/tools/bpf/bpftool/main.c +++ b/tools/bpf/bpftool/main.c @@ -32,6 +32,8 @@ bool verifier_logs; bool relaxed_maps; bool use_loader; struct btf *base_btf; +struct btf *relocate_base_btf; +const char *relocate_base_btf_path; struct hashmap *refs_table; static void __noreturn clean_and_exit(int i) @@ -448,6 +450,7 @@ int main(int argc, char **argv) { "debug", no_argument, NULL, 'd' }, { "use-loader", no_argument, NULL, 'L' }, { "base-btf", required_argument, NULL, 'B' }, + { "relocate-base-btf", required_argument, NULL, 'R' }, { 0 } }; bool version_requested = false; @@ -473,7 +476,7 @@ int main(int argc, char **argv) bin_name = "bpftool"; opterr = 0; - while ((opt = getopt_long(argc, argv, "VhpjfLmndB:l", + while ((opt = getopt_long(argc, argv, "VhpjfLmndB:lR:", options, NULL)) >= 0) { switch (opt) { case 'V': @@ -519,6 +522,15 @@ int main(int argc, char **argv) case 'L': use_loader = true; break; + case 'R': + relocate_base_btf_path = optarg; + relocate_base_btf = btf__parse(optarg, NULL); + if (!relocate_base_btf) { + p_err("failed to parse base BTF for relocation at '%s': %d\n", + optarg, -errno); + return -1; + } + break; default: p_err("unrecognized option '%s'", argv[optind - 1]); if (json_output) diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h index 9eb764fe4cc8..bbf8194a2d76 100644 --- a/tools/bpf/bpftool/main.h +++ b/tools/bpf/bpftool/main.h @@ -83,6 +83,8 @@ extern bool verifier_logs; extern bool relaxed_maps; extern bool use_loader; extern struct btf *base_btf; +extern struct btf *relocate_base_btf; +extern const char *relocate_base_btf_path; extern struct hashmap *refs_table; void __printf(1, 2) p_err(const char *fmt, ...);