From patchwork Tue Oct 17 16:22:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13425607 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43F674734E for ; Tue, 17 Oct 2023 16:23:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NOBKbfNO" Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82132B0 for ; Tue, 17 Oct 2023 09:23:13 -0700 (PDT) Received: by mail-yb1-xb31.google.com with SMTP id 3f1490d57ef6-d9a4c0d89f7so6887106276.1 for ; Tue, 17 Oct 2023 09:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697559792; x=1698164592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2IFN/2HC9JygClV/G6CWtisNQ7HtXVVuY/UWh7b7UCo=; b=NOBKbfNOsskjHRhZYGfveQBY/ePY5NlfafVFcZNFWr+n5GGAP/FTjTAbiUZ9gxG/pl tBXUqpGKA65VSfAklOZEif/Neh07ZXv1cGWxfnLxpoE+xTCfPWA2iyBp+WHy02/M6VJr qSTA9vvpY3NYD6F0oxSAPa5OA/QkvRKDMU/sJ04ubojnHf6Ux0LdzVgTrEGi0QLnfVA0 J3IXBd51kzS3YEHmz8rpbi/C+Xratb0XLMLZn4gFISimW0kYENLYTxRRoWI1a36lFHvS xONWBDuo6D6tN74HN+9YA24S15pa2R7KGt+Msn+tcaQ0defNxBD1Z/wgtWoitT1To5Q3 2HkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697559792; x=1698164592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2IFN/2HC9JygClV/G6CWtisNQ7HtXVVuY/UWh7b7UCo=; b=Yj+xgVXAcAIDnnveop4lGz53uDUG65BbAwKv8f64ozt2tExYQ5xTCp6EhfukT8tMAq OMwHoEeR6mgHCILidU2CKqCoMM5wZaoDKsi/OoQ5/NCVyOlrBYzh+mlOemnd64jvAnC0 X6O0Jw39MvCT1g/AkVFt9XRP/TF/C7HZIsXWChtR1JQtEQ5iFxGP1H/WWeT1aIdAIUG+ ceQDdCEML8+jCbALukWLsh4M9s+bZgCJbRg0u19V9okPtxEt+AGD3NL0LyHPJOC/e+uk f9cLRtLUwYJpOPIYQxKGKm7XnJ8+6Wy5KhqMyAHWoG6TRaphgaGXliK3//xHekxdGDzZ Tayg== X-Gm-Message-State: AOJu0Yx+eqTP0qMAIWO77UvsDZoHyQ8v2hIbztQJRx9dl6Ioc0OjoJZs 4c00MeF8Q1Gprg0/WhgNGXeLSGzpmjo= X-Google-Smtp-Source: AGHT+IGj6DNsckG6XRRw1wnFZ8yXeasydHAZlismm6IcJDt1CaGMgBzud8g6YGBz+fVBZI+3cPpYNA== X-Received: by 2002:a25:d250:0:b0:d9a:4839:68fe with SMTP id j77-20020a25d250000000b00d9a483968femr2614674ybg.43.1697559792377; Tue, 17 Oct 2023 09:23:12 -0700 (PDT) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:ed01:b54a:4364:93cc]) by smtp.gmail.com with ESMTPSA id r189-20020a2544c6000000b00d814d8dfd69sm623645yba.27.2023.10.17.09.23.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 09:23:12 -0700 (PDT) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v5 2/9] bpf: add struct_ops_tab to btf. Date: Tue, 17 Oct 2023 09:22:59 -0700 Message-Id: <20231017162306.176586-3-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017162306.176586-1-thinker.li@gmail.com> References: <20231017162306.176586-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module. Every struct_ops type should have an associated module BTF to provide type information since we are going to allow modules to define and register new struct_ops types. Once this change is made, the bpf_struct_ops subsystem knows where to look up type info with just a bpf_struct_ops. The subsystem looks up struct_ops types from a given module BTF although it is always btf_vmlinux now. Once start using struct_ops_tab, btfs other than btf_vmlinux can be used as well. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 5 +-- include/linux/btf.h | 6 ++++ kernel/bpf/bpf_struct_ops.c | 17 ++++----- kernel/bpf/btf.c | 70 +++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 2 +- 5 files changed, 89 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 30063a760b5a..e6a648af2daa 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1626,6 +1626,7 @@ struct bpf_struct_ops { void (*unreg)(void *kdata); int (*update)(void *kdata, void *old_kdata); int (*validate)(void *kdata); + struct btf *btf; const struct btf_type *type; const struct btf_type *value_type; const char *name; @@ -1636,7 +1637,7 @@ struct bpf_struct_ops { #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL) #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA)) -const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id); +const struct bpf_struct_ops *bpf_struct_ops_find(struct btf *btf, u32 type_id); void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log); bool bpf_struct_ops_get(const void *kdata); void bpf_struct_ops_put(const void *kdata); @@ -1679,7 +1680,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); #endif #else -static inline const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id) +static inline const struct bpf_struct_ops *bpf_struct_ops_find(struct btf *btf, u32 type_id) { return NULL; } diff --git a/include/linux/btf.h b/include/linux/btf.h index 928113a80a95..aa2ba77648be 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -571,4 +571,10 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type return btf_type_is_struct(t); } +struct bpf_struct_ops; + +int btf_add_struct_ops(struct bpf_struct_ops *st_ops); +const struct bpf_struct_ops ** +btf_get_struct_ops(struct btf *btf, u32 *ret_cnt); + #endif diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 627cf1ea840a..7758f66ad734 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -185,6 +185,7 @@ static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, pr_warn("Error in init bpf_struct_ops %s\n", st_ops->name); } else { + st_ops->btf = btf; st_ops->type_id = type_id; st_ops->type = t; st_ops->value_id = value_id; @@ -221,7 +222,7 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) extern struct btf *btf_vmlinux; static const struct bpf_struct_ops * -bpf_struct_ops_find_value(u32 value_id) +bpf_struct_ops_find_value(struct btf *btf, u32 value_id) { unsigned int i; @@ -236,7 +237,7 @@ bpf_struct_ops_find_value(u32 value_id) return NULL; } -const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id) +const struct bpf_struct_ops *bpf_struct_ops_find(struct btf *btf, u32 type_id) { unsigned int i; @@ -316,7 +317,7 @@ static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map) } } -static int check_zero_holes(const struct btf_type *t, void *data) +static int check_zero_holes(const struct btf *btf, const struct btf_type *t, void *data) { const struct btf_member *member; u32 i, moff, msize, prev_mend = 0; @@ -328,8 +329,8 @@ static int check_zero_holes(const struct btf_type *t, void *data) memchr_inv(data + prev_mend, 0, moff - prev_mend)) return -EINVAL; - mtype = btf_type_by_id(btf_vmlinux, member->type); - mtype = btf_resolve_size(btf_vmlinux, mtype, &msize); + mtype = btf_type_by_id(btf, member->type); + mtype = btf_resolve_size(btf, mtype, &msize); if (IS_ERR(mtype)) return PTR_ERR(mtype); prev_mend = moff + msize; @@ -395,12 +396,12 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, if (*(u32 *)key != 0) return -E2BIG; - err = check_zero_holes(st_ops->value_type, value); + err = check_zero_holes(st_ops->btf, st_ops->value_type, value); if (err) return err; uvalue = value; - err = check_zero_holes(t, uvalue->data); + err = check_zero_holes(st_ops->btf, t, uvalue->data); if (err) return err; @@ -671,7 +672,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) struct bpf_map *map; int ret; - st_ops = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id); + st_ops = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id, btf_vmlinux); if (!st_ops) return ERR_PTR(-ENOTSUPP); diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index f93e835d90af..be5144dbb53d 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -241,6 +241,12 @@ struct btf_id_dtor_kfunc_tab { struct btf_id_dtor_kfunc dtors[]; }; +struct btf_struct_ops_tab { + u32 cnt; + u32 capacity; + struct bpf_struct_ops *ops[]; +}; + struct btf { void *data; struct btf_type **types; @@ -258,6 +264,7 @@ struct btf { struct btf_kfunc_set_tab *kfunc_set_tab; struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; struct btf_struct_metas *struct_meta_tab; + struct btf_struct_ops_tab *struct_ops_tab; /* split BTF support */ struct btf *base_btf; @@ -1688,11 +1695,20 @@ static void btf_free_struct_meta_tab(struct btf *btf) btf->struct_meta_tab = NULL; } +static void btf_free_struct_ops_tab(struct btf *btf) +{ + struct btf_struct_ops_tab *tab = btf->struct_ops_tab; + + kfree(tab); + btf->struct_ops_tab = NULL; +} + static void btf_free(struct btf *btf) { btf_free_struct_meta_tab(btf); btf_free_dtor_kfunc_tab(btf); btf_free_kfunc_set_tab(btf); + btf_free_struct_ops_tab(btf); kvfree(btf->types); kvfree(btf->resolved_sizes); kvfree(btf->resolved_ids); @@ -8601,3 +8617,57 @@ bool btf_type_ids_nocast_alias(struct bpf_verifier_log *log, return !strncmp(reg_name, arg_name, cmp_len); } + +int btf_add_struct_ops(struct bpf_struct_ops *st_ops) +{ + struct btf_struct_ops_tab *tab, *new_tab; + struct btf *btf = st_ops->btf; + int i; + + if (!btf) + return -ENOENT; + + /* Assume this function is called for a module when the module is + * loading. + */ + + tab = btf->struct_ops_tab; + if (!tab) { + tab = kzalloc(offsetof(struct btf_struct_ops_tab, ops[4]), + GFP_KERNEL); + if (!tab) + return -ENOMEM; + tab->capacity = 4; + btf->struct_ops_tab = tab; + } + + for (i = 0; i < tab->cnt; i++) + if (tab->ops[i] == st_ops) + return -EEXIST; + + if (tab->cnt == tab->capacity) { + new_tab = krealloc(tab, sizeof(*tab) + + sizeof(struct bpf_struct_ops *) * + tab->capacity * 2, GFP_KERNEL); + if (!new_tab) + return -ENOMEM; + tab = new_tab; + tab->capacity *= 2; + btf->struct_ops_tab = tab; + } + + btf->struct_ops_tab->ops[btf->struct_ops_tab->cnt++] = st_ops; + + return 0; +} + +const struct bpf_struct_ops **btf_get_struct_ops(struct btf *btf, u32 *ret_cnt) +{ + if (!btf) + return NULL; + if (!btf->struct_ops_tab) + return NULL; + + *ret_cnt = btf->struct_ops_tab->cnt; + return (const struct bpf_struct_ops **)btf->struct_ops_tab->ops; +} diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a7178ecf676d..6564a03c425d 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -19631,7 +19631,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) } btf_id = prog->aux->attach_btf_id; - st_ops = bpf_struct_ops_find(btf_id); + st_ops = bpf_struct_ops_find(btf_vmlinux, btf_id); if (!st_ops) { verbose(env, "attach_btf_id %u is not a supported struct\n", btf_id);