From patchwork Fri May 17 19:05:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13667332 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ED9313E058 for ; Fri, 17 May 2024 19:06:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972770; cv=none; b=uZSzMs7kJzctY6dQxVHwWC0+AIPyhaplYHxTEuEMYSh4sDLkd44A6dt3yvc+0+M+qvrePgrlqQOcNB3gG28nUrTKgdzQC/1HeeTYuYqjdeMj+XR26ZRVyFZxkIctSvOVWKAMpFpvUcOfZhEgYSvxcXSIqeu8o0/hRiH3i8Tmj7U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972770; c=relaxed/simple; bh=SjTjv8qA/NC8e8exkpdLAO5LcqGblPiHgvV2yQrS0uE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rZUg5J0fZgQeJElZ/plHDH1HmgXEXuI7RX+Xe/otZ/VuX8JnnFl9Imon2o9IHZ1nc9R1E7PRBTPTkDz4OFRioD+fu1k+fcDV+KGCW21ed5RFD1ct8KT1DKnf21JZV+m+wKGOyudiakC/m43idp8dG5fRzRM4ml1lvGWUuX2cY+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FgXBz/sT; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FgXBz/sT" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1ec92e355bfso19900385ad.3 for ; Fri, 17 May 2024 12:06:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715972767; x=1716577567; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W6Fxrb/2/88WVf9ERm3nTOgjhdEBZvp0au6T7fTa2N0=; b=FgXBz/sT1p1Gs6hyxwqzrJhi3T1fNlmDX2nfo5nEHrrq5zvoxh0nva9AuQdF0Q6Cbb hX7iXMxScI3Br+Hr40KUZANsp4RdhFFd1Om/edgcMskbObMNtBlBKHqfEWV1C11lKZJ3 f/A/87nQn6n4R0DFSSc5ilaT6grG9Gl4jQ1gJajO6bkuJpErhRMEwJaMYXP0s7G4RkU5 iym/E/G2SxD5EEBlwXLwixS0KKOZWlYoUXBCSg/gOFYhgRLTnP/77PvJgS26ipchRxEv BcAYFcM8/6jYjj9ntnVO6yE2GA41YL9cl4FxqMsVJj6jgq/rZysN18KRCRlr5pcKLQcf rOgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715972767; x=1716577567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W6Fxrb/2/88WVf9ERm3nTOgjhdEBZvp0au6T7fTa2N0=; b=fOnWTPgXayhjrwCSR3os3CD7/mjuMF4ZNWmn4uAjyfDabmP4k/PNHY1K53FUKg6NTn CdP35O0lszKA/ewqojQ7uDcVdy4If3NfEdqgfdBbHgL6nOwKXL86o7Jg7jG+WFZoHoEz 2YkVw0Mq0ybkYigdscecRrPsnVh8hkXT9ty5Kk0rcEnL8Laix5LJmU+r8GXeQ1+hJxjm E9U6Ktgrv7ZroDkv/T2Rve1p+d4qh0ePVHzlezMFEsmt0NUiesLb2yM4ougozbnxWLOt kHGf8mD4t60apHDUYDw4lM76lXIwQpUsi9YEUiDyjeCcVErb5plK2Sv+PgS8jlB9kzoQ 0v+g== X-Gm-Message-State: AOJu0YxJvdCH1sNdfApXdyieLrlb0ium0csWdOVfCOrP7JghOu9UFOjw f47zuvB8j8FC/Q5ZYWbahFP/xoI+udeoMMpK/9LZ1Ys1BTeG6VnCBfC7iA== X-Google-Smtp-Source: AGHT+IHGk7NXWWrDSShWrP5XtKBto39jqlo57ii6ZtLam1vIKn9DMPu5Q1laMCf758OBKNnT+splOg== X-Received: by 2002:a17:90b:d0b:b0:2a2:c16:d673 with SMTP id 98e67ed59e1d1-2b6ccc73005mr21319428a91.36.1715972767000; Fri, 17 May 2024 12:06:07 -0700 (PDT) Received: from badger.hitronhub.home ([2604:3d08:6979:1160::3424]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2b9ddbcf05csm5459747a91.45.2024.05.17.12.06.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 12:06:06 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, alan.maguire@oracle.com, Eduard Zingerman Subject: [PATCH bpf-next v2 1/4] libbpf: put forward declarations to btf_dump->emit_queue Date: Fri, 17 May 2024 12:05:52 -0700 Message-Id: <20240517190555.4032078-2-eddyz87@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517190555.4032078-1-eddyz87@gmail.com> References: <20240517190555.4032078-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net As a preparatory step for introducing public API for BTF topological ordering / single type printing, this commit removes ordering logic from btf_dump_emit_type() and moves parts necessary to compute forward declarations to btf_dump_order_type(). Before this commit the topological ordering of types was effectively done twice: - first time in btf_dump_order_type(), which ordered types using only "strong" links (one structure embedded in another); - second time in btf_dump_emit_type() to emit forward declarations. After this commit: - btf_dump_emit_type() is responsible only for printing declaration / forward declaration of a single type; - btf_dump->emit_queue now contains pairs of form {id, forward declaration flag}; - btf_dump->emit_state is no longer necessary, as EMITTED state is effectively replaced by ORDERED state in btf_dump->order_state; Notable changes to btf_dump_order_type() logic: - no need to return strong / weak result, emit forward declaration to the queue for weak links instead; - track containing type id ('cont_id'), as btf_dump_emit_type() did, to avoid unnecessary forward declarations in recursive structs; - PTRs can no longer be marked ORDERED (see comment in the code); - incorporate btf_dump_emit_missing_aliases() and btf_dump_is_blacklisted() checks from btf_dump_emit_type(). When called for e.g. PTR type pointing to a struct btf_dump__dump_type() would previously result in an empty emit queue: btf_dump_order_type() would have reached struct with 'through_ptr' == true, thus not adding it to the queue. To mimic such behavior this patch adds a type filter to btf_dump__dump_type(): only STRUCT, UNION, TYPEDEF, ENUM, ENUM64, FWD types are ordered. The downside of a single pass algorithm is that for the following situation old logic would have avoided extra forward declaration: struct bar {}; struct foo { /* Suppose btf_dump__dump_type(foo) */ struct bar *a; /* is called first. */ struct bar b; }; The btf_dump_order_type() would have ordered 'bar' before 'foo', btf_dump_emit_type() would have been first called for 'bar' thus avoiding forward declaration for 'sturct bar' when processing 'foo'. In contrast, new logic would act as follows: - when processing foo->a forward declaration for 'bar' would be enqueued; - when processing foo->b full declaration for 'bar' would be enqueued; In practice this does not seem to be a big issue, number of forward declarations in vmlinux.h (for BPF selftests config) compared: llvm gcc - before this patch: 1772 1235 - after this patch: 1786 1249 Signed-off-by: Eduard Zingerman --- tools/lib/bpf/btf_dump.c | 351 ++++++++++++++++++--------------------- 1 file changed, 161 insertions(+), 190 deletions(-) diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c index 5dbca76b953f..10532ae9ff14 100644 --- a/tools/lib/bpf/btf_dump.c +++ b/tools/lib/bpf/btf_dump.c @@ -36,24 +36,16 @@ enum btf_dump_type_order_state { ORDERED, }; -enum btf_dump_type_emit_state { - NOT_EMITTED, - EMITTING, - EMITTED, -}; - /* per-type auxiliary state */ struct btf_dump_type_aux_state { /* topological sorting state */ enum btf_dump_type_order_state order_state: 2; - /* emitting state used to determine the need for forward declaration */ - enum btf_dump_type_emit_state emit_state: 2; - /* whether forward declaration was already emitted */ - __u8 fwd_emitted: 1; /* whether unique non-duplicate name was already assigned */ __u8 name_resolved: 1; /* whether type is referenced from any other type */ __u8 referenced: 1; + /* whether forward declaration was already ordered */ + __u8 fwd_ordered: 1; }; /* indent string length; one indent string is added for each indent level */ @@ -93,7 +85,10 @@ struct btf_dump { size_t cached_names_cap; /* topo-sorted list of dependent type definitions */ - __u32 *emit_queue; + struct { + __u32 id:31; + __u32 fwd:1; + } *emit_queue; int emit_queue_cap; int emit_queue_cnt; @@ -208,7 +203,6 @@ static int btf_dump_resize(struct btf_dump *d) if (d->last_id == 0) { /* VOID is special */ d->type_states[0].order_state = ORDERED; - d->type_states[0].emit_state = EMITTED; } /* eagerly determine referenced types for anon enums */ @@ -255,8 +249,8 @@ void btf_dump__free(struct btf_dump *d) free(d); } -static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr); -static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id); +static int btf_dump_order_type(struct btf_dump *d, __u32 id, __u32 cont_id, bool through_ptr); +static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd); /* * Dump BTF type in a compilable C syntax, including all the necessary @@ -276,6 +270,7 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id); */ int btf_dump__dump_type(struct btf_dump *d, __u32 id) { + const struct btf_type *t; int err, i; if (id >= btf__type_cnt(d->btf)) @@ -286,12 +281,23 @@ int btf_dump__dump_type(struct btf_dump *d, __u32 id) return libbpf_err(err); d->emit_queue_cnt = 0; - err = btf_dump_order_type(d, id, false); - if (err < 0) - return libbpf_err(err); + t = btf_type_by_id(d->btf, id); + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_TYPEDEF: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_FWD: + err = btf_dump_order_type(d, id, id, false); + if (err < 0) + return libbpf_err(err); + default: + break; + }; for (i = 0; i < d->emit_queue_cnt; i++) - btf_dump_emit_type(d, d->emit_queue[i], 0 /*top-level*/); + btf_dump_emit_type(d, d->emit_queue[i].id, d->emit_queue[i].fwd); return 0; } @@ -374,9 +380,9 @@ static int btf_dump_mark_referenced(struct btf_dump *d) return 0; } -static int btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id) +static int __btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id, bool fwd) { - __u32 *new_queue; + typeof(d->emit_queue[0]) *new_queue = NULL; size_t new_cap; if (d->emit_queue_cnt >= d->emit_queue_cap) { @@ -388,10 +394,45 @@ static int btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id) d->emit_queue_cap = new_cap; } - d->emit_queue[d->emit_queue_cnt++] = id; + d->emit_queue[d->emit_queue_cnt].id = id; + d->emit_queue[d->emit_queue_cnt].fwd = fwd; + d->emit_queue_cnt++; return 0; } +static int btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id) +{ + return __btf_dump_add_emit_queue_id(d, id, false); +} + +static int btf_dump_add_emit_queue_fwd(struct btf_dump *d, __u32 id) +{ + struct btf_dump_type_aux_state *tstate = &d->type_states[id]; + + if (tstate->fwd_ordered) + return 0; + + tstate->fwd_ordered = 1; + return __btf_dump_add_emit_queue_id(d, id, true); +} + +static bool btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id, bool dry_run); + +static bool btf_dump_is_blacklisted(struct btf_dump *d, __u32 id) +{ + const struct btf_type *t = btf__type_by_id(d->btf, id); + + /* __builtin_va_list is a compiler built-in, which causes compilation + * errors, when compiling w/ different compiler, then used to compile + * original code (e.g., GCC to compile kernel, Clang to use generated + * C header from BTF). As it is built-in, it should be already defined + * properly internally in compiler. + */ + if (t->name_off == 0) + return false; + return strcmp(btf_name_of(d, t->name_off), "__builtin_va_list") == 0; +} + /* * Determine order of emitting dependent types and specified type to satisfy * C compilation rules. This is done through topological sorting with an @@ -441,32 +482,33 @@ static int btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id) * The rule is as follows. Given a chain of BTF types from X to Y, if there is * BTF_KIND_PTR type in the chain and at least one non-anonymous type * Z (excluding X, including Y), then link is weak. Otherwise, it's strong. - * Weak/strong relationship is determined recursively during DFS traversal and - * is returned as a result from btf_dump_order_type(). + * Weak/strong relationship is determined recursively during DFS traversal. + * + * When type id is reached via a weak link a forward declaration for + * that type is added to the emit queue, otherwise "full" declaration + * is added to the emit queue. + * + * We also keep track of "containing struct/union type ID" to determine when + * we reference it from inside and thus can avoid emitting unnecessary forward + * declaration. * * btf_dump_order_type() is trying to avoid unnecessary forward declarations, * but it is not guaranteeing that no extraneous forward declarations will be * emitted. * * To avoid extra work, algorithm marks some of BTF types as ORDERED, when - * it's done with them, but not for all (e.g., VOLATILE, CONST, RESTRICT, - * ARRAY, FUNC_PROTO), as weak/strong semantics for those depends on the + * it's done with them, but not for all (e.g., PTR, VOLATILE, CONST, RESTRICT, + * ARRAY, FUNC_PROTO, TYPEDEF), as weak/strong semantics for those depends on the * entire graph path, so depending where from one came to that BTF type, it * might cause weak or strong ordering. For types like STRUCT/UNION/INT/ENUM, * once they are processed, there is no need to do it again, so they are - * marked as ORDERED. We can mark PTR as ORDERED as well, as it semi-forces - * weak link, unless subsequent referenced STRUCT/UNION/ENUM is anonymous. But - * in any case, once those are processed, no need to do it again, as the - * result won't change. + * marked as ORDERED. * * Returns: - * - 1, if type is part of strong link (so there is strong topological - * ordering requirements); - * - 0, if type is part of weak link (so can be satisfied through forward - * declaration); + * - 0, on success; * - <0, on error (e.g., unsatisfiable type loop detected). */ -static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr) +static int btf_dump_order_type(struct btf_dump *d, __u32 id, __u32 cont_id, bool through_ptr) { /* * Order state is used to detect strong link cycles, but only for BTF @@ -486,48 +528,78 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr) /* return true, letting typedefs know that it's ok to be emitted */ if (tstate->order_state == ORDERED) - return 1; + return 0; t = btf__type_by_id(d->btf, id); if (tstate->order_state == ORDERING) { /* type loop, but resolvable through fwd declaration */ - if (btf_is_composite(t) && through_ptr && t->name_off != 0) - return 0; + if (btf_is_composite(t) && through_ptr && t->name_off != 0) { + if (id != cont_id) + return btf_dump_add_emit_queue_fwd(d, id); + else + return 0; + } pr_warn("unsatisfiable type cycle, id:[%u]\n", id); return -ELOOP; } switch (btf_kind(t)) { case BTF_KIND_INT: + tstate->order_state = ORDERED; + if (btf_dump_emit_missing_aliases(d, id, true)) + return btf_dump_add_emit_queue_id(d, id); + else + return 0; case BTF_KIND_FLOAT: tstate->order_state = ORDERED; return 0; case BTF_KIND_PTR: - err = btf_dump_order_type(d, t->type, true); - tstate->order_state = ORDERED; + /* Depending on whether pointer is a part of a recursive struct + * declaration, it might not necessitate generation of a forward + * declaration for the target type, e.g.: + * + * struct foo { + * struct foo *p; // no need for forward declaration + * } + * + * struct bar { + * struct foo *p; // forward declaration is needed + * } + * + * Hence, don't mark pointer as ORDERED, to allow traversal + * to the target type and comparison with 'cont_id'. + */ + err = btf_dump_order_type(d, t->type, cont_id, true); return err; case BTF_KIND_ARRAY: - return btf_dump_order_type(d, btf_array(t)->type, false); + return btf_dump_order_type(d, btf_array(t)->type, cont_id, false); case BTF_KIND_STRUCT: case BTF_KIND_UNION: { const struct btf_member *m = btf_members(t); + __u32 new_cont_id; + /* * struct/union is part of strong link, only if it's embedded * (so no ptr in a path) or it's anonymous (so has to be * defined inline, even if declared through ptr) */ - if (through_ptr && t->name_off != 0) - return 0; + if (through_ptr && t->name_off != 0) { + if (id != cont_id) + return btf_dump_add_emit_queue_fwd(d, id); + else + return 0; + } tstate->order_state = ORDERING; + new_cont_id = t->name_off == 0 ? cont_id : id; vlen = btf_vlen(t); for (i = 0; i < vlen; i++, m++) { - err = btf_dump_order_type(d, m->type, false); + err = btf_dump_order_type(d, m->type, new_cont_id, false); if (err < 0) return err; } @@ -539,7 +611,7 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr) } tstate->order_state = ORDERED; - return 1; + return 0; } case BTF_KIND_ENUM: case BTF_KIND_ENUM64: @@ -555,51 +627,53 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr) return err; } tstate->order_state = ORDERED; - return 1; - - case BTF_KIND_TYPEDEF: { - int is_strong; - - is_strong = btf_dump_order_type(d, t->type, through_ptr); - if (is_strong < 0) - return is_strong; + return 0; - /* typedef is similar to struct/union w.r.t. fwd-decls */ - if (through_ptr && !is_strong) + case BTF_KIND_TYPEDEF: + /* Do not mark typedef as ORDERED, always emit a forward declaration for + * it instead. Otherwise the following situation would be troublesome: + * + * typedef struct foo foo_alias; + * + * struct foo {}; + * + * struct root { + * foo_alias *a; + * foo_alias b; + * }; + * + */ + if (btf_dump_is_blacklisted(d, id)) return 0; - /* typedef is always a named definition */ - err = btf_dump_add_emit_queue_id(d, id); - if (err) + err = btf_dump_order_type(d, t->type, t->type, through_ptr); + if (err < 0) return err; - d->type_states[id].order_state = ORDERED; - return 1; - } + err = btf_dump_add_emit_queue_fwd(d, id); + if (err) + return err; + return 0; case BTF_KIND_VOLATILE: case BTF_KIND_CONST: case BTF_KIND_RESTRICT: case BTF_KIND_TYPE_TAG: - return btf_dump_order_type(d, t->type, through_ptr); + return btf_dump_order_type(d, t->type, cont_id, through_ptr); case BTF_KIND_FUNC_PROTO: { const struct btf_param *p = btf_params(t); - bool is_strong; - err = btf_dump_order_type(d, t->type, through_ptr); + err = btf_dump_order_type(d, t->type, cont_id, through_ptr); if (err < 0) return err; - is_strong = err > 0; vlen = btf_vlen(t); for (i = 0; i < vlen; i++, p++) { - err = btf_dump_order_type(d, p->type, through_ptr); + err = btf_dump_order_type(d, p->type, cont_id, through_ptr); if (err < 0) return err; - if (err > 0) - is_strong = true; } - return is_strong; + return err; } case BTF_KIND_FUNC: case BTF_KIND_VAR: @@ -613,9 +687,6 @@ static int btf_dump_order_type(struct btf_dump *d, __u32 id, bool through_ptr) } } -static void btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id, - const struct btf_type *t); - static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id, const struct btf_type *t); static void btf_dump_emit_struct_def(struct btf_dump *d, __u32 id, @@ -649,73 +720,33 @@ static const char *btf_dump_ident_name(struct btf_dump *d, __u32 id); static size_t btf_dump_name_dups(struct btf_dump *d, struct hashmap *name_map, const char *orig_name); -static bool btf_dump_is_blacklisted(struct btf_dump *d, __u32 id) -{ - const struct btf_type *t = btf__type_by_id(d->btf, id); - - /* __builtin_va_list is a compiler built-in, which causes compilation - * errors, when compiling w/ different compiler, then used to compile - * original code (e.g., GCC to compile kernel, Clang to use generated - * C header from BTF). As it is built-in, it should be already defined - * properly internally in compiler. - */ - if (t->name_off == 0) - return false; - return strcmp(btf_name_of(d, t->name_off), "__builtin_va_list") == 0; -} - /* * Emit C-syntax definitions of types from chains of BTF types. * - * High-level handling of determining necessary forward declarations are handled - * by btf_dump_emit_type() itself, but all nitty-gritty details of emitting type + * All nitty-gritty details of emitting type * declarations/definitions in C syntax are handled by a combo of * btf_dump_emit_type_decl()/btf_dump_emit_type_chain() w/ delegation to * corresponding btf_dump_emit_*_{def,fwd}() functions. * - * We also keep track of "containing struct/union type ID" to determine when - * we reference it from inside and thus can avoid emitting unnecessary forward - * declaration. - * * This algorithm is designed in such a way, that even if some error occurs * (either technical, e.g., out of memory, or logical, i.e., malformed BTF * that doesn't comply to C rules completely), algorithm will try to proceed * and produce as much meaningful output as possible. */ -static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id) +static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd) { - struct btf_dump_type_aux_state *tstate = &d->type_states[id]; - bool top_level_def = cont_id == 0; const struct btf_type *t; __u16 kind; - if (tstate->emit_state == EMITTED) - return; - t = btf__type_by_id(d->btf, id); kind = btf_kind(t); - if (tstate->emit_state == EMITTING) { - if (tstate->fwd_emitted) - return; - + if (fwd) { switch (kind) { case BTF_KIND_STRUCT: case BTF_KIND_UNION: - /* - * if we are referencing a struct/union that we are - * part of - then no need for fwd declaration - */ - if (id == cont_id) - return; - if (t->name_off == 0) { - pr_warn("anonymous struct/union loop, id:[%u]\n", - id); - return; - } btf_dump_emit_struct_fwd(d, id, t); btf_dump_printf(d, ";\n\n"); - tstate->fwd_emitted = 1; break; case BTF_KIND_TYPEDEF: /* @@ -723,11 +754,8 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id) * was emitted, but it can be used only for "weak" * references through pointer only, not for embedding */ - if (!btf_dump_is_blacklisted(d, id)) { - btf_dump_emit_typedef_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - } - tstate->fwd_emitted = 1; + btf_dump_emit_typedef_def(d, id, t, 0); + btf_dump_printf(d, ";\n\n"); break; default: break; @@ -739,36 +767,18 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id) switch (kind) { case BTF_KIND_INT: /* Emit type alias definitions if necessary */ - btf_dump_emit_missing_aliases(d, id, t); - - tstate->emit_state = EMITTED; + btf_dump_emit_missing_aliases(d, id, false); break; case BTF_KIND_ENUM: case BTF_KIND_ENUM64: - if (top_level_def) { - btf_dump_emit_enum_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - } - tstate->emit_state = EMITTED; - break; - case BTF_KIND_PTR: - case BTF_KIND_VOLATILE: - case BTF_KIND_CONST: - case BTF_KIND_RESTRICT: - case BTF_KIND_TYPE_TAG: - btf_dump_emit_type(d, t->type, cont_id); - break; - case BTF_KIND_ARRAY: - btf_dump_emit_type(d, btf_array(t)->type, cont_id); + btf_dump_emit_enum_def(d, id, t, 0); + btf_dump_printf(d, ";\n\n"); break; case BTF_KIND_FWD: btf_dump_emit_fwd_def(d, id, t); btf_dump_printf(d, ";\n\n"); - tstate->emit_state = EMITTED; break; case BTF_KIND_TYPEDEF: - tstate->emit_state = EMITTING; - btf_dump_emit_type(d, t->type, id); /* * typedef can server as both definition and forward * declaration; at this stage someone depends on @@ -776,55 +786,14 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, __u32 cont_id) * through pointer), so unless we already did it, * emit typedef as a forward declaration */ - if (!tstate->fwd_emitted && !btf_dump_is_blacklisted(d, id)) { - btf_dump_emit_typedef_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - } - tstate->emit_state = EMITTED; + btf_dump_emit_typedef_def(d, id, t, 0); + btf_dump_printf(d, ";\n\n"); break; case BTF_KIND_STRUCT: case BTF_KIND_UNION: - tstate->emit_state = EMITTING; - /* if it's a top-level struct/union definition or struct/union - * is anonymous, then in C we'll be emitting all fields and - * their types (as opposed to just `struct X`), so we need to - * make sure that all types, referenced from struct/union - * members have necessary forward-declarations, where - * applicable - */ - if (top_level_def || t->name_off == 0) { - const struct btf_member *m = btf_members(t); - __u16 vlen = btf_vlen(t); - int i, new_cont_id; - - new_cont_id = t->name_off == 0 ? cont_id : id; - for (i = 0; i < vlen; i++, m++) - btf_dump_emit_type(d, m->type, new_cont_id); - } else if (!tstate->fwd_emitted && id != cont_id) { - btf_dump_emit_struct_fwd(d, id, t); - btf_dump_printf(d, ";\n\n"); - tstate->fwd_emitted = 1; - } - - if (top_level_def) { - btf_dump_emit_struct_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - tstate->emit_state = EMITTED; - } else { - tstate->emit_state = NOT_EMITTED; - } - break; - case BTF_KIND_FUNC_PROTO: { - const struct btf_param *p = btf_params(t); - __u16 n = btf_vlen(t); - int i; - - btf_dump_emit_type(d, t->type, cont_id); - for (i = 0; i < n; i++, p++) - btf_dump_emit_type(d, p->type, cont_id); - + btf_dump_emit_struct_def(d, id, t, 0); + btf_dump_printf(d, ";\n\n"); break; - } default: break; } @@ -1037,19 +1006,21 @@ static const char *missing_base_types[][2] = { { "__Poly128_t", "unsigned __int128" }, }; -static void btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id, - const struct btf_type *t) +static bool btf_dump_emit_missing_aliases(struct btf_dump *d, __u32 id, bool dry_run) { const char *name = btf_dump_type_name(d, id); int i; for (i = 0; i < ARRAY_SIZE(missing_base_types); i++) { - if (strcmp(name, missing_base_types[i][0]) == 0) { + if (strcmp(name, missing_base_types[i][0]) != 0) + continue; + if (!dry_run) btf_dump_printf(d, "typedef %s %s;\n\n", missing_base_types[i][1], name); - break; - } + return true; } + + return false; } static void btf_dump_emit_enum_fwd(struct btf_dump *d, __u32 id, From patchwork Fri May 17 19:05:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13667333 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B3856A005 for ; Fri, 17 May 2024 19:06:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972770; cv=none; b=rPje7QYbhr4td815kfrUzo8oCtBknJSfyORwBwejOudTHtf0tLS+hibIqW/b0eHYF3bjjVaFOJdKLquOFk6SUfuLFDmbof/dHrNNcRoWvRRPE9VTs1Jvyi/zBtKOYbe1UHI2vSzoVkWT/oJ4gcwMLBLw2klbnkZLQek3yUy8LhI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972770; c=relaxed/simple; bh=hDg8uRncuM+2DdOmQDZqVQJvOuV9MpfOZM/VsSe0G00=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fw0hjODmOUQhzK/MgAxIoxomhrPaqHPej8HXnQuH3N/+OvANLM+896ZaBOrHqFKYOr1hVM6OEoIL15e9QZLzzYvNNpO6AwEi72NBBboqKvDY04ryBMqvddmZAGd7RxzHjdjKD84+W4xLwnQ8GazZlr53AqIZN6KWh0YMd7EuTGk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jvRPXdeR; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jvRPXdeR" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1f2ecea41deso1460965ad.1 for ; Fri, 17 May 2024 12:06:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715972768; x=1716577568; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ozI7d8wwJtrIshDd63M8LYnf1u/sIwIfUY2uiayDpAA=; b=jvRPXdeR9Fc3cj4eRPiBdrBBNtLsxr+vQwpKpyrow9ML7Q4BjZtSfxxAyCjuIrcEhc lBzQY+r8b+OSkPQiTzVZ27qAIN/b9DNqr/Rn/Zv5SyveNVpD9v0EsQx9wm2LOL4Pfrsj oD2qJuy0t213ZlljqCAdXQJBYWeEAxfa1QwBGDnHCdm3POERN3p8uPO7jMWaBmxqCPbI trfGa6K54OI/Z3ZmHlsGMbCJlGiqIUOYhDa/7NnnAFwjXL0AENUO5ZuDTqbsO6xYKGZ0 NfKmMkg8TAInMJd6g/jZOdE5MrM5S2QnExe5m+w88UKux/1cXPq3pfWMIrrC+l3W+L4F mfGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715972768; x=1716577568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ozI7d8wwJtrIshDd63M8LYnf1u/sIwIfUY2uiayDpAA=; b=e0X3jsa0fDDBouboPW3kBAuOPvIpeshfeV7bf4Qcv/XBrOCECOQdQ63ecbVMF9G2i9 3xGHIMQN0hTyAyXUlT5xrlxtOAsL+2TLYLk5EDtknl8VEnXTF0/7+ByCkWRvgBqbYMQ3 3R14g9O99zfc/SLBIEVtn+hrOsjxX9NsTLoUEPGbCPCYiEFRMbz9/BufP+2pngmIfq8d Z1YiiCCAeOt8s73PrOX6pZQHANCVUsBOBwZNKGajDCt+Zne7H+xmZpoAa3/Qdgjb3alF +eMRZsAK+Msz13fGC4IwVCy1JL1aLek7lZT6MN65vAAeNbpZQO+1SzdhuLJdnoTsCpHk Y4Tw== X-Gm-Message-State: AOJu0YwCJS7+djTW8TdI6fyDDvoAXhYkXySkoSIxvMDUYr5W2o3mEFCS 2FYzAE2ZChUjwWPSyPpXkK8bssRwXuiE9pB9Dyt+KJKrhjScHBZio5IxmA== X-Google-Smtp-Source: AGHT+IFoouJm/BtC2yHetJ1oxElKlTHgeYbJFCfyZIzvKuUCPdWiefEiLMUsix8BixBJasBcSAiKdw== X-Received: by 2002:a17:90a:348d:b0:2b4:32ae:4712 with SMTP id 98e67ed59e1d1-2b6cc141d7emr22286658a91.9.1715972768240; Fri, 17 May 2024 12:06:08 -0700 (PDT) Received: from badger.hitronhub.home ([2604:3d08:6979:1160::3424]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2b9ddbcf05csm5459747a91.45.2024.05.17.12.06.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 12:06:07 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, alan.maguire@oracle.com, Eduard Zingerman , Andrii Nakryiko Subject: [PATCH bpf-next v2 2/4] libbpf: API to access btf_dump emit queue and print single type Date: Fri, 17 May 2024 12:05:53 -0700 Message-Id: <20240517190555.4032078-3-eddyz87@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517190555.4032078-1-eddyz87@gmail.com> References: <20240517190555.4032078-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add several API functions to allow more flexibility with btf dump: - int btf_dump__order_type(struct btf_dump *d, __u32 id); adds a type and all it's dependencies to the emit queue in topological order; - struct btf_dump_emit_queue_item *btf_dump__emit_queue(struct btf_dump *d); __u32 btf_dump__emit_queue_cnt(struct btf_dump *d); provide access to the emit queue owned by btf_dump object; - int btf_dump__dump_one_type(struct btf_dump *d, __u32 id, bool fwd); prints a given type in C format (skipping any dependencies). This API should allow to do the following on the libbpf client side: - filter printed types using arbitrary criteria; - add arbitrary type attributes or pre-processor statements for selected types. This is a follow-up to the following discussion: https://lore.kernel.org/bpf/20240503111836.25275-1-jose.marchesi@oracle.com/ Suggested-by: Andrii Nakryiko Signed-off-by: Eduard Zingerman --- tools/lib/bpf/btf.h | 33 ++++++++++++++++++++++ tools/lib/bpf/btf_dump.c | 61 ++++++++++++++++++++++------------------ tools/lib/bpf/libbpf.map | 4 +++ 3 files changed, 71 insertions(+), 27 deletions(-) diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c84..81d70ac35562 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -249,6 +249,39 @@ LIBBPF_API void btf_dump__free(struct btf_dump *d); LIBBPF_API int btf_dump__dump_type(struct btf_dump *d, __u32 id); +/* Dumps C language definition or forward declaration for type **id**: + * - returns 1 if type is printable; + * - returns 0 if type is non-printable. + */ +LIBBPF_API int btf_dump__dump_one_type(struct btf_dump *d, __u32 id, bool fwd); + +/* **struct btf_dump** tracks a list of types that should be dumped, + * these types are sorted in the topological order satisfying C language semantics: + * - if type A includes type B (e.g. A is a struct with a field of type B), + * then B comes before A; + * - if type A references type B via a pointer + * (e.g. A is a struct with a field of type pointer to B), + * then B's forward declaration comes before A. + * + * **struct btf_dump_emit_queue_item** represents a single entry of the emit queue. + */ +struct btf_dump_emit_queue_item { + __u32 id:31; + __u32 fwd:1; +}; + +/* Adds type **id** and it's dependencies to the emit queue. */ +LIBBPF_API int btf_dump__order_type(struct btf_dump *d, __u32 id); + +/* Provides access to currently accumulated emit queue, + * returned pointer is owned by **struct btf_dump** and should not be + * freed explicitly. + */ +LIBBPF_API struct btf_dump_emit_queue_item *btf_dump__emit_queue(struct btf_dump *d); + +/* Returns the size of currently accumulated emit queue */ +LIBBPF_API __u32 btf_dump__emit_queue_cnt(struct btf_dump *d); + struct btf_dump_emit_type_decl_opts { /* size of this struct, for forward/backward compatiblity */ size_t sz; diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c index 10532ae9ff14..c3af6bb606a0 100644 --- a/tools/lib/bpf/btf_dump.c +++ b/tools/lib/bpf/btf_dump.c @@ -85,10 +85,7 @@ struct btf_dump { size_t cached_names_cap; /* topo-sorted list of dependent type definitions */ - struct { - __u32 id:31; - __u32 fwd:1; - } *emit_queue; + struct btf_dump_emit_queue_item *emit_queue; int emit_queue_cap; int emit_queue_cnt; @@ -250,7 +247,6 @@ void btf_dump__free(struct btf_dump *d) } static int btf_dump_order_type(struct btf_dump *d, __u32 id, __u32 cont_id, bool through_ptr); -static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd); /* * Dump BTF type in a compilable C syntax, including all the necessary @@ -296,12 +292,32 @@ int btf_dump__dump_type(struct btf_dump *d, __u32 id) break; }; - for (i = 0; i < d->emit_queue_cnt; i++) - btf_dump_emit_type(d, d->emit_queue[i].id, d->emit_queue[i].fwd); + for (i = 0; i < d->emit_queue_cnt; i++) { + err = btf_dump__dump_one_type(d, d->emit_queue[i].id, d->emit_queue[i].fwd); + if (err < 0) + return libbpf_err(err); + if (err > 0) + btf_dump_printf(d, ";\n\n"); + } return 0; } +int btf_dump__order_type(struct btf_dump *d, __u32 id) +{ + return btf_dump_order_type(d, id, id, false); +} + +struct btf_dump_emit_queue_item *btf_dump__emit_queue(struct btf_dump *d) +{ + return d->emit_queue; +} + +__u32 btf_dump__emit_queue_cnt(struct btf_dump *d) +{ + return d->emit_queue_cnt; +} + /* * Mark all types that are referenced from any other type. This is used to * determine top-level anonymous enums that need to be emitted as an @@ -382,7 +398,7 @@ static int btf_dump_mark_referenced(struct btf_dump *d) static int __btf_dump_add_emit_queue_id(struct btf_dump *d, __u32 id, bool fwd) { - typeof(d->emit_queue[0]) *new_queue = NULL; + struct btf_dump_emit_queue_item *new_queue = NULL; size_t new_cap; if (d->emit_queue_cnt >= d->emit_queue_cap) { @@ -733,7 +749,7 @@ static size_t btf_dump_name_dups(struct btf_dump *d, struct hashmap *name_map, * that doesn't comply to C rules completely), algorithm will try to proceed * and produce as much meaningful output as possible. */ -static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd) +int btf_dump__dump_one_type(struct btf_dump *d, __u32 id, bool fwd) { const struct btf_type *t; __u16 kind; @@ -746,8 +762,7 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd) case BTF_KIND_STRUCT: case BTF_KIND_UNION: btf_dump_emit_struct_fwd(d, id, t); - btf_dump_printf(d, ";\n\n"); - break; + return 1; case BTF_KIND_TYPEDEF: /* * for typedef fwd_emitted means typedef definition @@ -755,29 +770,23 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd) * references through pointer only, not for embedding */ btf_dump_emit_typedef_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - break; + return 1; default: - break; + return 0; } - - return; } switch (kind) { case BTF_KIND_INT: /* Emit type alias definitions if necessary */ - btf_dump_emit_missing_aliases(d, id, false); - break; + return btf_dump_emit_missing_aliases(d, id, false); case BTF_KIND_ENUM: case BTF_KIND_ENUM64: btf_dump_emit_enum_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - break; + return 1; case BTF_KIND_FWD: btf_dump_emit_fwd_def(d, id, t); - btf_dump_printf(d, ";\n\n"); - break; + return 1; case BTF_KIND_TYPEDEF: /* * typedef can server as both definition and forward @@ -787,15 +796,13 @@ static void btf_dump_emit_type(struct btf_dump *d, __u32 id, bool fwd) * emit typedef as a forward declaration */ btf_dump_emit_typedef_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - break; + return 1; case BTF_KIND_STRUCT: case BTF_KIND_UNION: btf_dump_emit_struct_def(d, id, t, 0); - btf_dump_printf(d, ";\n\n"); - break; + return 1; default: - break; + return 0; } } diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c1ce8aa3520b..137e4cbaa7a7 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -422,4 +422,8 @@ LIBBPF_1.5.0 { bpf_program__attach_sockmap; ring__consume_n; ring_buffer__consume_n; + btf_dump__emit_queue; + btf_dump__emit_queue_cnt; + btf_dump__order_type; + btf_dump__dump_one_type; } LIBBPF_1.4.0; From patchwork Fri May 17 19:05:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13667334 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DBFC13E3E6 for ; Fri, 17 May 2024 19:06:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972771; cv=none; b=qv+HUsaHF8pggkcNPjdOtJCJoVa/ZCNTq31qS9S9t2FWntD85G2xMY9gWiLNKYpuWu7oh8ZmiwUsLq7OY7l9Fgk5l+BKzvervbxLKpYV7YcTXy0q3EgzQJrK7DNu5rMKRmsj3ijVQEUlERiO8K7kEPFnKTKmOyrJygX8slM90/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972771; c=relaxed/simple; bh=ZioFnZ3d1Vg35xANGhPTZ32CY+yYa1owDklEmy5rVm4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pkh1BLx9fhTYkwCyC19qlOZJPVdp/68tAMnto3OjTBMLa6G7teULYbyweiQ1rIBWMcp7zoBCARnMQdTVv24orFHTtXge3W6PRXCFLiTthy6hg6SM0qv9R64ziSPD8WaX4oPGi78Pi1l/j+32K4S64SCg6v8flaTfo6d08VAhCsE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WfjVjc2g; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WfjVjc2g" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1ed835f3c3cso20318635ad.3 for ; Fri, 17 May 2024 12:06:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715972769; x=1716577569; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZYh5uMJCF1r0KBlzHzNOaPta0GxxeMYUw422sU1IaAI=; b=WfjVjc2gGa4CtxH2Rdbv0s6QHfy/O7hOzZCFnDiz3DEoxbSgdzMog2cO8kMOX34nom CbNZnfne5r+ZlH3LUNsGHQoG9TGlYBTf1h7t6LbQ/Tu2uGCugfhIG5vnhh/RrzX4tR0e VR6ESYUzLhBGKu3gaJODD2MEzgfofNnHu6sMfQiEStxDDf2Oa46WH6nnZITYJlahrDJX HQW1SK5QMOBKRh/hQMxS/mkol6VAhe2BsnrbTveyekKOPJCh6xcGiiPgmgKZ8Rof2s0w unKsjc1yP1H3G6lP2751wXnhZ3LtoZPus9+CTr7o176hrMowrjjOiCBKPxw6iNtFGIWp uhig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715972769; x=1716577569; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZYh5uMJCF1r0KBlzHzNOaPta0GxxeMYUw422sU1IaAI=; b=Vf/YA7Eogfzp8o2l0iab8QfPpHGbDki2Iy2HiAGj34gDFXhMAq/scwtkAdP45IQ3+b 6YNfJVjqbg3GqLFkJc8y1ggIv9VechRdrqQLbDmzca+fPiJd22NFQe9TpXPEG1DB2ih/ cMXTQtL3R/lz11T20mtXfo4K3GoP5tLhYBiZjpoiT+yXRqH4APCswiR+LYbZwd9w+Skv R+3HCfuT1eicnJ3OGf3G4BTo1Sav/+kCHqd5b4tmBTL5Rl6tgcHCrooo9vf+yxPtcuJK 9J1ptWuwqfR+6BX/BouBASL3J8HqjSniXGg5+j9K+Zz+L+S5PecuUUz9itB9svgJV8u8 7fDQ== X-Gm-Message-State: AOJu0YyyeckH1nqbmYaoeP9CyLjilfr/C1ky7X2WwGLDlh8ei5CcA8cN aeCuCsnTHlA6ffzKHwNHdP4WvtUbtC709qVLc6sL7Y5KSqTnPpdnldXE1A== X-Google-Smtp-Source: AGHT+IGykXHx9HX0S8BVJLL7/281g2RzA7QR39GlnQwPp2clhYCO4W6rIq9LGPQmysIYIq70ijKeTg== X-Received: by 2002:a17:90b:3a8c:b0:2b3:2a3b:dd0f with SMTP id 98e67ed59e1d1-2b6cc14b8b5mr23311286a91.8.1715972769301; Fri, 17 May 2024 12:06:09 -0700 (PDT) Received: from badger.hitronhub.home ([2604:3d08:6979:1160::3424]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2b9ddbcf05csm5459747a91.45.2024.05.17.12.06.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 12:06:08 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, alan.maguire@oracle.com, Eduard Zingerman Subject: [PATCH bpf-next v2 3/4] selftests/bpf: tests for btf_dump emit queue API Date: Fri, 17 May 2024 12:05:54 -0700 Message-Id: <20240517190555.4032078-4-eddyz87@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517190555.4032078-1-eddyz87@gmail.com> References: <20240517190555.4032078-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Test cases for btf_dump__dump_one_type() and company: - peek a type, save it's dependencies to the emit queue, print the content of the emit queue, expect only relevant dependencies; - print a forward declaration for a single type, expect no dependencies in the output; - print a full dependencies for a type, expect no dependencies in the output. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/prog_tests/btf_dump.c | 86 +++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dump.c b/tools/testing/selftests/bpf/prog_tests/btf_dump.c index e9ea38aa8248..04f32472e221 100644 --- a/tools/testing/selftests/bpf/prog_tests/btf_dump.c +++ b/tools/testing/selftests/bpf/prog_tests/btf_dump.c @@ -255,6 +255,90 @@ static void test_btf_dump_incremental(void) btf__free(btf); } +static void test_btf_dump_emit_queue(void) +{ + struct btf_dump_emit_queue_item *items; + struct btf_dump *d = NULL; + struct btf *btf = NULL; + u32 union_simple; + int err, i, cnt; + + btf = btf__parse_elf("btf_dump_test_case_syntax.bpf.o", NULL); + if (!ASSERT_OK_PTR(btf, "btf_parse_elf")) { + err = -PTR_ERR(btf); + btf = NULL; + goto err_out; + } + + union_simple = btf__find_by_name_kind(btf, "union_simple", BTF_KIND_UNION); + ASSERT_GT(union_simple, 0, "'union_simple' id"); + + dump_buf_file = open_memstream(&dump_buf, &dump_buf_sz); + if (!ASSERT_OK_PTR(dump_buf_file, "dump_memstream")) + return; + d = btf_dump__new(btf, btf_dump_printf, dump_buf_file, NULL); + if (!ASSERT_OK(libbpf_get_error(d), "btf_dump__new")) + goto err_out; + + err = btf_dump__order_type(d, union_simple); + ASSERT_OK(err, "order type 'union_simple'"); + cnt = btf_dump__emit_queue_cnt(d); + items = btf_dump__emit_queue(d); + for (i = 1; i < cnt; i++) { + err = btf_dump__dump_one_type(d, items[i].id, items[i].fwd); + if (err > 0) + fprintf(dump_buf_file, ";\n\n"); + } + + fflush(dump_buf_file); + dump_buf[dump_buf_sz] = 0; + + ASSERT_STREQ(dump_buf, +"union union_empty {};\n" +"\n" +"union union_simple {\n" +" void *ptr;\n" +" int num;\n" +" int_t num2;\n" +" union union_empty u;\n" +"};\n\n", "c_dump1"); + + btf_dump__free(d); + d = btf_dump__new(btf, btf_dump_printf, dump_buf_file, NULL); + if (!ASSERT_OK(libbpf_get_error(d), "btf_dump__new")) { + d = NULL; + goto err_out; + } + err = btf_dump__order_type(d, union_simple); + ASSERT_OK(err, "order type 'union_simple'"); + + rewind(dump_buf_file); + btf_dump__dump_one_type(d, union_simple, true); + fflush(dump_buf_file); + dump_buf[dump_buf_sz] = 0; + + ASSERT_STREQ(dump_buf, "union union_simple", "c_dump2"); + + rewind(dump_buf_file); + btf_dump__dump_one_type(d, union_simple, false); + fflush(dump_buf_file); + dump_buf[dump_buf_sz] = 0; + + ASSERT_STREQ(dump_buf, +"union union_simple {\n" +" void *ptr;\n" +" int num;\n" +" int_t num2;\n" +" union union_empty u;\n" +"}", "c_dump3"); + +err_out: + fclose(dump_buf_file); + free(dump_buf); + btf_dump__free(d); + btf__free(btf); +} + #define STRSIZE 4096 static void btf_dump_snprintf(void *ctx, const char *fmt, va_list args) @@ -873,6 +957,8 @@ void test_btf_dump() { } if (test__start_subtest("btf_dump: incremental")) test_btf_dump_incremental(); + if (test__start_subtest("btf_dump: emit queue")) + test_btf_dump_emit_queue(); btf = libbpf_find_kernel_btf(); if (!ASSERT_OK_PTR(btf, "no kernel BTF found")) From patchwork Fri May 17 19:05:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13667335 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A73086A005 for ; Fri, 17 May 2024 19:06:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972773; cv=none; b=f3Gh0OOudctuZ7qA1nouXeLXpqBlDKJIaVO+wjqOfoug6GmftTrSjJbR8rfnkMw94gPHf3evNlIMVMaPXUUG2LA6AcHapECRApDmk/8V0ZNDYF7w14Kw4JDuJWC0nVkkQ6ZGb6eDdaPHa4BETzsF5hzaxRom5WJPHsTc7AXlhLM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715972773; c=relaxed/simple; bh=vRFX16U6ZbdJl60Rk6Q+Np2bjeuuL0xQY/q2key0MFU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=C81ZfRLDUC+KvwBoZcmnygSkbgP+ng6rsZFH1rrzVQd+KIaFmBQAkMT17ofZ/TH/h01ge+KG1KJ59+ufnTai86e2EuWwi2cjfMb7FfmjfkkpoviBpPz0I8kYKC6JiOrpGf7ProLt7dH7TyUpN+fQEsJCZozrc6RoJrEOZsJO8pA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ehAW/4YU; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ehAW/4YU" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2b432be5cc9so70858a91.3 for ; Fri, 17 May 2024 12:06:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715972770; x=1716577570; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ppcRlxhCgqPeJ/eCCIUZ+FHjvruC3n/kAnhmMV53fKg=; b=ehAW/4YUSE3aBB+p7qM3GIz3RQCBclOppzqKJYGMDu2q9CyzvX/jnP6uaj+9Ts9VBa bpRTO/WZykRbvMbhc/ZcadsZ3CmkCQCT9n3pfIX4yPfTExLp2tDmw3UpfQYqeDSp3mZS AzzwuP6+MU3NNWU+yp0dYHUFyXvU8oKQHX2sEW/AjyEkTawWDtaEflWAdLNV15EPTNRl /5zUeQrrx9KHaqDlVbaOcr5kJDepzz45QBHiX7jPwo5333qAL4ntcmqNwyocKV8wbHLN Y7yhRG7pSJvdYVju1oEkgWHyHimkcfutPPMGEzXcIEqDzU6z9HUAGjBdY7VrSY7dDGp2 JZjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715972770; x=1716577570; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ppcRlxhCgqPeJ/eCCIUZ+FHjvruC3n/kAnhmMV53fKg=; b=CdwmZZKyS1bcA4348VmoF4wnjv9cTNdG/gtvXdG2oLYhnsvXkCEQud/2qXQka6nIWI VqURss6j9XPnxnhIbVJiHxPBaCphA+6EYvFIDXgUNVZiVVFa1iq1avroyB0BBMsJftnF +lHajhj6xUzcQQc44iNuwaQw6rAnL9Wn1qHzjd6oVnQAuNv0aP3aivkXb/VlGAxY6uZz oTBmbeUqx05gGjn3a3pZuIYupmS4T+i0OZ7LNWgP6o9PPUbQIlDI6qABufmPVdbaHgUN Y1czzLIVVkWQ45nQuKR2NMngPGSCKrjfbj/9Njtg30D9IaMie5a/orYcl7b8IVMDF9jb 4KXw== X-Gm-Message-State: AOJu0Yyw6fR+/UwUC1d7W7+b33D5que+ymq6s9SKDx8yOQAH4AdlS/uB gLlS6DbvodeOtl+rCFuL0H0V1IG3SgWXYlxlGVv6301RYmOOQbPNR82L3g== X-Google-Smtp-Source: AGHT+IEG90JIPJu+D1jH1HGz328jn30MkLnCYb5jjG4WQGm2FhaUl8UU/lcdo6GMPdL0qWp/Pt4+2w== X-Received: by 2002:a17:90a:d686:b0:2b6:95f6:b79f with SMTP id 98e67ed59e1d1-2b6ccef2f54mr20868672a91.33.1715972770500; Fri, 17 May 2024 12:06:10 -0700 (PDT) Received: from badger.hitronhub.home ([2604:3d08:6979:1160::3424]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2b9ddbcf05csm5459747a91.45.2024.05.17.12.06.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 May 2024 12:06:10 -0700 (PDT) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, jose.marchesi@oracle.com, alan.maguire@oracle.com, Eduard Zingerman Subject: [PATCH bpf-next v2 4/4] selftests/bpf: corner case for typedefs handling in btf_dump Date: Fri, 17 May 2024 12:05:55 -0700 Message-Id: <20240517190555.4032078-5-eddyz87@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240517190555.4032078-1-eddyz87@gmail.com> References: <20240517190555.4032078-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This corner case stood out while testing: typedef struct foo foo_alias; struct foo {}; struct root { foo_alias *a; foo_alias b; }; btf_dump_order_type() visits root->a first and root->b next. If 'foo_alias' is marked as ORDERED when visiting root->a type 'foo' will only have a forward declaration. Signed-off-by: Eduard Zingerman --- .../selftests/bpf/progs/btf_dump_test_case_ordering.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/btf_dump_test_case_ordering.c b/tools/testing/selftests/bpf/progs/btf_dump_test_case_ordering.c index 7c95702ee4cb..e542cb8fb3f4 100644 --- a/tools/testing/selftests/bpf/progs/btf_dump_test_case_ordering.c +++ b/tools/testing/selftests/bpf/progs/btf_dump_test_case_ordering.c @@ -47,12 +47,22 @@ struct callback_head { void (*func)(struct callback_head *); }; +typedef struct foo foo_alias; + +struct foo {}; + +struct typedef_ptr_and_full { + foo_alias *a; + foo_alias b; +}; + struct root_struct { struct s4 s4; struct list_head l; struct hlist_node n; struct hlist_head h; struct callback_head cb; + struct typedef_ptr_and_full td; }; /*------ END-EXPECTED-OUTPUT ------ */