From patchwork Thu Jan 18 01:49:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522290 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f181.google.com (mail-yw1-f181.google.com [209.85.128.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 625741C06 for ; Thu, 18 Jan 2024 01:49:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542590; cv=none; b=Bxl4rpThUnlWNCfYd0nyQdrRTnAX0yH8j/Hs1du+qAxy0MiwrUIXKQ+oXNGw9toblspCcMhI+w9I2pnUrn+6GEs87YjsaoJnBCmgkSHar9AO3gXySc5jhfJw49anTmxZ7w5e6Kn5IcQek10Bnt4CBs2VlTHJIpwsJIH5g7PsBug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542590; c=relaxed/simple; bh=WhyulcBtGgmzeiUjv5gl+rwywGAD/2U+TEUq6nyaCwk=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=V6hjWjLMObr/e1PfE1oW7uEkP91mTx/bHGzDfrv/K+TAgmL5vL+jDctysEfuGFbaXSW62PcxvkjvmjsKgmuNJzZ1H2TxnCTXJy3CEqgxEGJZIvpqCuKwVEErmezX11K5aNLgMhaSO6XlFSNSeJia/m2726+JSsZwLQvSPED8AOA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CKoup1F+; arc=none smtp.client-ip=209.85.128.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CKoup1F+" Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-5fc2e997804so53564767b3.3 for ; Wed, 17 Jan 2024 17:49:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542588; x=1706147388; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rM52mOpOXRsJ8K9AGybBKYuwruVPoyAuVHFy5zVCFqU=; b=CKoup1F+G0JEHJ9LArpCdt/BwKBjtoMOvUyCLbCLZLNq85qVXW4XYKbnxtV8h1U6C1 ZYXpnW1SG+Xp5OHnkeWj9qE9QnhKFg9zYBwcnmmbNID4pwNY/kloLdxsSXMMWqzL97dT 4s1w1xwVxyYQa72uc5Si1Z75b31qOklGddDyR1x6lmnltkSaW3wyuVIlY0/JYteGmNn5 WVcz63/m0ERU6NRE+9nnKRJgersXCT8QCWnbW/aDoF17WrpLOZWBnzv+ou6tMcEr85Go 1nk1eG4E8GpAdqGvTv2mKX7MdJ+tcE+iochZGqK7RLQuyKtta0Sw+Tg3a+wqb0Whywvi HOEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542588; x=1706147388; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rM52mOpOXRsJ8K9AGybBKYuwruVPoyAuVHFy5zVCFqU=; b=uIOTJW2b6va+M2ZhtyvO+ZykDcJFcSZAzBpKfcDzMFMFornxML7k0f8oEax+BYriAy Dbu0QDTm0mJhQtzmstdC1xx0qqkOtVC+TL+SUuwEs5e/IEqO/ZWDVbQOXP3OwhUn8R3a oSgAQBIbdtbUxRQhV6cf0mkV4aJxJeoEoOGEiPkyzWS+ihKR6RThzLKp3hJqktoMa6Pw 7cR5dbx5NldjLeyqEovLuwHLJE+jii5rIwqm7+jwKz+XYxgyw4HTL5po5uS0Hk2p99mL 10vh6fnG9w/rPJSF6MrKyuO85Qr6T0Hn86AY3GrmrlhKWfLgnsgsM34bHH4WkAIAFOuL yMOA== X-Gm-Message-State: AOJu0YwqX/Jt0KApUFLVv0Vs8U4eLh7EzV/IzHScnG7Eu9mtRVi/Mzkh 57wwR6teORySUVW42Hyb3Ox6Ujm7RWYRwLizO2DTNjCq2IUyfblQJxTFzv4G X-Google-Smtp-Source: AGHT+IExp5niQYIq0V4ae7gBN7Nnr/a301URgRKO4bZrrxq+toQ6tCCNv0idC0YQ2QFB7JC9Xr2OtA== X-Received: by 2002:a81:4746:0:b0:5fa:c41d:dc1c with SMTP id u67-20020a814746000000b005fac41ddc1cmr125459ywa.50.1705542587906; Wed, 17 Jan 2024 17:49:47 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:46 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 01/14] bpf: refactory struct_ops type initialization to a function. Date: Wed, 17 Jan 2024 17:49:17 -0800 Message-Id: <20240118014930.1992551-2-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Move the majority of the code to bpf_struct_ops_init_one(), which can then be utilized for the initialization of newly registered dynamically allocated struct_ops types in the following patches. Signed-off-by: Kui-Feng Lee --- include/linux/btf.h | 1 + kernel/bpf/bpf_struct_ops.c | 157 +++++++++++++++++++----------------- kernel/bpf/btf.c | 5 ++ 3 files changed, 89 insertions(+), 74 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 59d404e22814..1d852dad7473 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -137,6 +137,7 @@ struct btf_struct_metas { extern const struct file_operations btf_fops; +const char *btf_get_name(const struct btf *btf); void btf_get(struct btf *btf); void btf_put(struct btf *btf); int btf_new_fd(const union bpf_attr *attr, bpfptr_t uattr, u32 uattr_sz); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 02068bd0e4d9..96cba76f4ac3 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -110,102 +110,111 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = { static const struct btf_type *module_type; -void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) +static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, + struct btf *btf, + struct bpf_verifier_log *log) { - s32 type_id, value_id, module_id; const struct btf_member *member; - struct bpf_struct_ops *st_ops; const struct btf_type *t; + s32 type_id, value_id; char value_name[128]; const char *mname; - u32 i, j; + int i; - /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */ -#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name); -#include "bpf_struct_ops_types.h" -#undef BPF_STRUCT_OPS_TYPE + if (strlen(st_ops->name) + VALUE_PREFIX_LEN >= + sizeof(value_name)) { + pr_warn("struct_ops name %s is too long\n", + st_ops->name); + return; + } + sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name); - module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT); - if (module_id < 0) { - pr_warn("Cannot find struct module in btf_vmlinux\n"); + value_id = btf_find_by_name_kind(btf, value_name, + BTF_KIND_STRUCT); + if (value_id < 0) { + pr_warn("Cannot find struct %s in %s\n", + value_name, btf_get_name(btf)); return; } - module_type = btf_type_by_id(btf, module_id); - for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - st_ops = bpf_struct_ops[i]; + type_id = btf_find_by_name_kind(btf, st_ops->name, + BTF_KIND_STRUCT); + if (type_id < 0) { + pr_warn("Cannot find struct %s in %s\n", + st_ops->name, btf_get_name(btf)); + return; + } + t = btf_type_by_id(btf, type_id); + if (btf_type_vlen(t) > BPF_STRUCT_OPS_MAX_NR_MEMBERS) { + pr_warn("Cannot support #%u members in struct %s\n", + btf_type_vlen(t), st_ops->name); + return; + } - if (strlen(st_ops->name) + VALUE_PREFIX_LEN >= - sizeof(value_name)) { - pr_warn("struct_ops name %s is too long\n", + for_each_member(i, t, member) { + const struct btf_type *func_proto; + + mname = btf_name_by_offset(btf, member->name_off); + if (!*mname) { + pr_warn("anon member in struct %s is not supported\n", st_ops->name); - continue; + break; } - sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name); - value_id = btf_find_by_name_kind(btf, value_name, - BTF_KIND_STRUCT); - if (value_id < 0) { - pr_warn("Cannot find struct %s in btf_vmlinux\n", - value_name); - continue; + if (__btf_member_bitfield_size(t, member)) { + pr_warn("bit field member %s in struct %s is not supported\n", + mname, st_ops->name); + break; } - type_id = btf_find_by_name_kind(btf, st_ops->name, - BTF_KIND_STRUCT); - if (type_id < 0) { - pr_warn("Cannot find struct %s in btf_vmlinux\n", - st_ops->name); - continue; - } - t = btf_type_by_id(btf, type_id); - if (btf_type_vlen(t) > BPF_STRUCT_OPS_MAX_NR_MEMBERS) { - pr_warn("Cannot support #%u members in struct %s\n", - btf_type_vlen(t), st_ops->name); - continue; + func_proto = btf_type_resolve_func_ptr(btf, + member->type, + NULL); + if (func_proto && + btf_distill_func_proto(log, btf, + func_proto, mname, + &st_ops->func_models[i])) { + pr_warn("Error in parsing func ptr %s in struct %s\n", + mname, st_ops->name); + break; } + } - for_each_member(j, t, member) { - const struct btf_type *func_proto; + if (i == btf_type_vlen(t)) { + if (st_ops->init(btf)) { + pr_warn("Error in init bpf_struct_ops %s\n", + st_ops->name); + } else { + st_ops->type_id = type_id; + st_ops->type = t; + st_ops->value_id = value_id; + st_ops->value_type = btf_type_by_id(btf, + value_id); + } + } +} - mname = btf_name_by_offset(btf, member->name_off); - if (!*mname) { - pr_warn("anon member in struct %s is not supported\n", - st_ops->name); - break; - } +void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) +{ + struct bpf_struct_ops *st_ops; + s32 module_id; + u32 i; - if (__btf_member_bitfield_size(t, member)) { - pr_warn("bit field member %s in struct %s is not supported\n", - mname, st_ops->name); - break; - } + /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */ +#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name); +#include "bpf_struct_ops_types.h" +#undef BPF_STRUCT_OPS_TYPE - func_proto = btf_type_resolve_func_ptr(btf, - member->type, - NULL); - if (func_proto && - btf_distill_func_proto(log, btf, - func_proto, mname, - &st_ops->func_models[j])) { - pr_warn("Error in parsing func ptr %s in struct %s\n", - mname, st_ops->name); - break; - } - } + module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT); + if (module_id < 0) { + pr_warn("Cannot find struct module in %s\n", btf_get_name(btf)); + return; + } + module_type = btf_type_by_id(btf, module_id); - if (j == btf_type_vlen(t)) { - if (st_ops->init(btf)) { - pr_warn("Error in init bpf_struct_ops %s\n", - st_ops->name); - } else { - st_ops->type_id = type_id; - st_ops->type = t; - st_ops->value_id = value_id; - st_ops->value_type = btf_type_by_id(btf, - value_id); - } - } + for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { + st_ops = bpf_struct_ops[i]; + bpf_struct_ops_init_one(st_ops, btf, log); } } diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 51e8b4bee0c8..1f94c250ab49 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -1707,6 +1707,11 @@ static void btf_free_rcu(struct rcu_head *rcu) btf_free(btf); } +const char *btf_get_name(const struct btf *btf) +{ + return btf->name; +} + void btf_get(struct btf *btf) { refcount_inc(&btf->refcnt); From patchwork Thu Jan 18 01:49:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522291 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f181.google.com (mail-yw1-f181.google.com [209.85.128.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88D2428FD for ; Thu, 18 Jan 2024 01:49:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542592; cv=none; b=jo8GLPJHWqBMweYr6Yrk0JDC5lvQSI/pJsZxZzzyekxiHdKhJP7YDR8HpksoDuj3TTg0j/AZGaJ0gjx0mwhpj3VAVLF33QVdTKEawg5Ch58gAaCkCTHn7WXmf7N2G1TEufdEav5pFxLaTUpXcHgtygqs3N90MIqV0C1vve80Rv0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542592; c=relaxed/simple; bh=95a0yqhJqEJ2eSnmB0Bwv/kLPu1mLnX1I+z6KNpil6k=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=IVXT65PbKqlNzqo45O498LVEksFo4ILvvEpPZrm1OJPrIaQ5a+4aVutpRjF5lsoQllonD46kKdSWzXllwqMJHYYM9ImMAZT+QdwqfHY0ULrzB+xaYKqnLohY1aW2l5AZPz3spiFjizSDJcB+s+gjoTebjjLZqz03FaA7IlL52Ug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hZM42qMr; arc=none smtp.client-ip=209.85.128.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hZM42qMr" Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-5ff6d3504d5so10993187b3.3 for ; Wed, 17 Jan 2024 17:49:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542590; x=1706147390; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XDr+NjtX2UDLWx9szIwoRGkmVQoh5QgUN+RdNYwqcvY=; b=hZM42qMr/7vBoofNyCakWAadJiT6QAUDG8BJHs8x0HHE0HQoKo5eBLKMacWaoucU00 2IkS3a0y9vpw3EISLH5zC0V3hRE3MlA4FoDEMCXW//Tm3/VLakGk1a5+8t1bNsKomCNM 3SIlf8UAled9eRxHino0MfqgjHKK2hasfvzTOC1xZyYcLxyRm8dClrXaStp0OH7QjReL SG1mxNLbD/OBjMeZkU+VOtyL+ZGoLIU2nmfLSVAzKI5ifCMX76fGel3L6H3GVo9ujCPg e9348imA+4EIm1mE3FE9JgO1ul9YggceimcNJ5ZXyrJGFXR0ORfV8/4xgGtR3tyGPhr+ dWvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542590; x=1706147390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XDr+NjtX2UDLWx9szIwoRGkmVQoh5QgUN+RdNYwqcvY=; b=K+fW/b6ueu8ocm157F4Ylhm1DKWBhnsEm9qId7rM0iIva4Uox/UnKuOHo+ZBEhcfXj 8K/N2ZC+Pp1k4y1JZZYeLmPqOyYGseGRdrlmkOh7qeCmwYl8b22JeZikSMS6KEiuuqpZ NgN5dxxQfKwXPETLDArli53g9b0yRdFUtRYGN6KtHy0Uz83PeD5T28f8onuT+rvzPPPW t31uCl+QqsB3X3oXS6W+EO2+5NS9TSMgwonH60Emw0yxzpxUUMhNR6G64+0IJdPbJn5r Q/ROyRkRCN55uIt3QW2huO95ZOBrPewSjaCvT57/t64HDjZquLHGBpU4O1xU8nXLXql0 84qA== X-Gm-Message-State: AOJu0YyqKm1PweRH4Z/rmF8Idf1JCxsBvE5Wq5VRoIjPdL++OJocwllD vmZAyKYdN6ollHl/K663bICSurmEgTU+96R45LPzKCdUOXbGA/nSujrzRoye X-Google-Smtp-Source: AGHT+IEQZFnakNp2YCjhk5y17jUZfhUadA1ILoaz7yXpUybXc4LklfCQnFgJnEHL8Q3/0mv0Sgy/HA== X-Received: by 2002:a81:6504:0:b0:5f1:f638:2bd8 with SMTP id z4-20020a816504000000b005f1f6382bd8mr112896ywb.31.1705542590047; Wed, 17 Jan 2024 17:49:50 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:49 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 02/14] bpf: get type information with BTF_ID_LIST Date: Wed, 17 Jan 2024 17:49:18 -0800 Message-Id: <20240118014930.1992551-3-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Get ready to remove bpf_struct_ops_init() in the future. By using BTF_ID_LIST, it is possible to gather type information while building instead of runtime. Signed-off-by: Kui-Feng Lee --- kernel/bpf/bpf_struct_ops.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 96cba76f4ac3..5b3ebcb435d0 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -108,7 +108,12 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = { #endif }; -static const struct btf_type *module_type; +BTF_ID_LIST(st_ops_ids) +BTF_ID(struct, module) + +enum { + IDX_MODULE_ID, +}; static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, struct btf *btf, @@ -197,7 +202,6 @@ static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) { struct bpf_struct_ops *st_ops; - s32 module_id; u32 i; /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */ @@ -205,13 +209,6 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) #include "bpf_struct_ops_types.h" #undef BPF_STRUCT_OPS_TYPE - module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT); - if (module_id < 0) { - pr_warn("Cannot find struct module in %s\n", btf_get_name(btf)); - return; - } - module_type = btf_type_by_id(btf, module_id); - for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { st_ops = bpf_struct_ops[i]; bpf_struct_ops_init_one(st_ops, btf, log); @@ -387,6 +384,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; const struct bpf_struct_ops *st_ops = st_map->st_ops; struct bpf_struct_ops_value *uvalue, *kvalue; + const struct btf_type *module_type; const struct btf_member *member; const struct btf_type *t = st_ops->type; struct bpf_tramp_links *tlinks; @@ -434,6 +432,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, image = st_map->image; image_end = st_map->image + PAGE_SIZE; + module_type = btf_type_by_id(btf_vmlinux, st_ops_ids[IDX_MODULE_ID]); for_each_member(i, t, member) { const struct btf_type *mtype, *ptype; struct bpf_prog *prog; From patchwork Thu Jan 18 01:49:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522292 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7A7328FE; Thu, 18 Jan 2024 01:49:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542594; cv=none; b=cTGqdTQ2rFvaUcbRIKZ9KjfUma8vLgDKryD6U5goYbdeKyOQYWOTA+AHnOuEdMR23xO+evXkknNoWNljvQEjaN0GD7fwPkf2RtO+qArh7KqiIRJNitZk+5qNAfqYUM1/U7g7BghCdXClyRcFntlRqaUXWa3gG2wp6OYYGqJV/vo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542594; c=relaxed/simple; bh=GQqUpGmmWvoC7Uskakk3F89C5axhb1Eztk/uDdT/f8o=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=eqLyc8zXZGTHsHfAsOzNcl2umD3cImon+BcGzaNb/EAy0dVld8h4FQby2MtWXxcAD6KW1+Y6Qkzr/egnQE/NDjF4HGXbUl6kooOnTES/g2PAVJy75SLYv6DK/xym7skxoK1vNZfI2Fr95Ej/vfndWdttSuMYq7yRlU/P+GXZLbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Pc0FVIpL; arc=none smtp.client-ip=209.85.128.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Pc0FVIpL" Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-5e734251f48so2284597b3.1; Wed, 17 Jan 2024 17:49:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542591; x=1706147391; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fGhCNS+OXSjb3TUD6BdQSRVIp9jmREbnxhNj9/+DHWw=; b=Pc0FVIpLKfMiAAGIjBEP0UgHUYWGSrsXJz52L0Lhdb6+DUe5pNTkyu/cKEQ7XF82FP wEMaKr5MPsEeeQeZaDO8IesVPayzHCyzi8PWJXg8SXKGOiOPF47EpMAKy7aJxlN+NH8l IQsBg1qKgUrFFh/OKCJcV8X6pwVcqp+9+B54i4B3EJ34eEPUFPAHnwRgs7CXez2GfY7j uKhJ3LEy/FQDKjOw1PaZ8wBQuuXarFU7gjeYCylercwM53zTfIYNhJXtUVwejNiA0OWo EC/WdL1iD+jP9OMmes7GsVSlSbDYuuSWB9ylM04gQdpGqS3W6q1e/bX4wUmpC7S/1taN w9QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542591; x=1706147391; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fGhCNS+OXSjb3TUD6BdQSRVIp9jmREbnxhNj9/+DHWw=; b=NcNHWEH9SXy/CextvxesOPeqGX3WHtD8cJrv4uI3m6fv8pXD93V0d2RQw5ZUVLWmxI BKdqIz/8Le1XAr7dotYZt8Dmurj8VvfKgSB57qqGfa42BgxvzrEiO23kpPLe3K+UZPlj ndFi+HAc9MgjWTAu3swyCXxBc0/2nGcnOzl2+xb9KEbn8e/C2keDn6qNeJsnmpWp1E6N R7bxSNHdbghRxRwUbMSBWKHKJurn+14Pm6ShpX6bTaxANhm4bLXf5GYZPMHS7VKGIaKA uIl3tCipjlV5BUScMMNMGk/98YIxIBzOpV26JYB2n8SovxGfNt15829VZ589gu6zSKeE PcYQ== X-Gm-Message-State: AOJu0YyiYKtpkvBfQb7UMz0V3qs0lKTccNwqrcHKPxknBoLpZibFHISn PEXg22DJiq15c7LDiYgC/FG8iK70tmX1hLnlIDEtqe9YDfkLs0CmB5fSOKEy X-Google-Smtp-Source: AGHT+IHv2w+r9Wmoemnj/ttxx+1FC7npdfzEU81kvRYSRTfFBoR43x2E+Wu+1pxeCw9RTW3QN9mRFg== X-Received: by 2002:a0d:d757:0:b0:5ff:7c9d:96f with SMTP id z84-20020a0dd757000000b005ff7c9d096fmr128118ywd.42.1705542591474; Wed, 17 Jan 2024 17:49:51 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:50 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee , netdev@vger.kernel.org Subject: [PATCH bpf-next v16 03/14] bpf, net: introduce bpf_struct_ops_desc. Date: Wed, 17 Jan 2024 17:49:19 -0800 Message-Id: <20240118014930.1992551-4-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Move some of members of bpf_struct_ops to bpf_struct_ops_desc. type_id is unavailabe in bpf_struct_ops anymore. Modules should get it from the btf received by kmod's init function. Cc: netdev@vger.kernel.org Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 20 ++++----- kernel/bpf/bpf_struct_ops.c | 80 +++++++++++++++++----------------- kernel/bpf/verifier.c | 8 ++-- net/bpf/bpf_dummy_struct_ops.c | 11 ++++- net/ipv4/bpf_tcp_ca.c | 8 +++- 5 files changed, 73 insertions(+), 54 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index e30100597d0a..ed4352f56d21 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1673,18 +1673,23 @@ struct bpf_struct_ops { void (*unreg)(void *kdata); int (*update)(void *kdata, void *old_kdata); int (*validate)(void *kdata); - const struct btf_type *type; - const struct btf_type *value_type; + void *cfi_stubs; const char *name; struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS]; +}; + +struct bpf_struct_ops_desc { + struct bpf_struct_ops *st_ops; + + const struct btf_type *type; + const struct btf_type *value_type; u32 type_id; u32 value_id; - void *cfi_stubs; }; #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL) #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA)) -const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id); +const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id); void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log); bool bpf_struct_ops_get(const void *kdata); void bpf_struct_ops_put(const void *kdata); @@ -1728,7 +1733,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); #endif #else -static inline const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id) +static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id) { return NULL; } @@ -1750,11 +1755,6 @@ static inline int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, { return -EINVAL; } -static inline int bpf_struct_ops_link_create(union bpf_attr *attr) -{ - return -EOPNOTSUPP; -} - #endif #if defined(CONFIG_CGROUP_BPF) && defined(CONFIG_BPF_LSM) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 5b3ebcb435d0..9774f7824e8b 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -32,7 +32,7 @@ struct bpf_struct_ops_value { struct bpf_struct_ops_map { struct bpf_map map; struct rcu_head rcu; - const struct bpf_struct_ops *st_ops; + const struct bpf_struct_ops_desc *st_ops_desc; /* protect map_update */ struct mutex lock; /* link has all the bpf_links that is populated @@ -92,9 +92,9 @@ enum { __NR_BPF_STRUCT_OPS_TYPE, }; -static struct bpf_struct_ops * const bpf_struct_ops[] = { +static struct bpf_struct_ops_desc bpf_struct_ops[] = { #define BPF_STRUCT_OPS_TYPE(_name) \ - [BPF_STRUCT_OPS_TYPE_##_name] = &bpf_##_name, + [BPF_STRUCT_OPS_TYPE_##_name] = { .st_ops = &bpf_##_name }, #include "bpf_struct_ops_types.h" #undef BPF_STRUCT_OPS_TYPE }; @@ -115,10 +115,11 @@ enum { IDX_MODULE_ID, }; -static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, - struct btf *btf, - struct bpf_verifier_log *log) +static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, + struct btf *btf, + struct bpf_verifier_log *log) { + struct bpf_struct_ops *st_ops = st_ops_desc->st_ops; const struct btf_member *member; const struct btf_type *t; s32 type_id, value_id; @@ -190,18 +191,18 @@ static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops, pr_warn("Error in init bpf_struct_ops %s\n", st_ops->name); } else { - st_ops->type_id = type_id; - st_ops->type = t; - st_ops->value_id = value_id; - st_ops->value_type = btf_type_by_id(btf, - value_id); + st_ops_desc->type_id = type_id; + st_ops_desc->type = t; + st_ops_desc->value_id = value_id; + st_ops_desc->value_type = btf_type_by_id(btf, + value_id); } } } void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) { - struct bpf_struct_ops *st_ops; + struct bpf_struct_ops_desc *st_ops_desc; u32 i; /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */ @@ -210,14 +211,14 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) #undef BPF_STRUCT_OPS_TYPE for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - st_ops = bpf_struct_ops[i]; - bpf_struct_ops_init_one(st_ops, btf, log); + st_ops_desc = &bpf_struct_ops[i]; + bpf_struct_ops_desc_init(st_ops_desc, btf, log); } } extern struct btf *btf_vmlinux; -static const struct bpf_struct_ops * +static const struct bpf_struct_ops_desc * bpf_struct_ops_find_value(u32 value_id) { unsigned int i; @@ -226,14 +227,14 @@ bpf_struct_ops_find_value(u32 value_id) return NULL; for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - if (bpf_struct_ops[i]->value_id == value_id) - return bpf_struct_ops[i]; + if (bpf_struct_ops[i].value_id == value_id) + return &bpf_struct_ops[i]; } return NULL; } -const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id) +const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id) { unsigned int i; @@ -241,8 +242,8 @@ const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id) return NULL; for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - if (bpf_struct_ops[i]->type_id == type_id) - return bpf_struct_ops[i]; + if (bpf_struct_ops[i].type_id == type_id) + return &bpf_struct_ops[i]; } return NULL; @@ -302,7 +303,7 @@ static void *bpf_struct_ops_map_lookup_elem(struct bpf_map *map, void *key) static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map) { - const struct btf_type *t = st_map->st_ops->type; + const struct btf_type *t = st_map->st_ops_desc->type; u32 i; for (i = 0; i < btf_type_vlen(t); i++) { @@ -382,11 +383,12 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, void *value, u64 flags) { struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; - const struct bpf_struct_ops *st_ops = st_map->st_ops; + const struct bpf_struct_ops_desc *st_ops_desc = st_map->st_ops_desc; + const struct bpf_struct_ops *st_ops = st_ops_desc->st_ops; struct bpf_struct_ops_value *uvalue, *kvalue; const struct btf_type *module_type; const struct btf_member *member; - const struct btf_type *t = st_ops->type; + const struct btf_type *t = st_ops_desc->type; struct bpf_tramp_links *tlinks; void *udata, *kdata; int prog_fd, err; @@ -399,7 +401,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, if (*(u32 *)key != 0) return -E2BIG; - err = check_zero_holes(st_ops->value_type, value); + err = check_zero_holes(st_ops_desc->value_type, value); if (err) return err; @@ -492,7 +494,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, } if (prog->type != BPF_PROG_TYPE_STRUCT_OPS || - prog->aux->attach_btf_id != st_ops->type_id || + prog->aux->attach_btf_id != st_ops_desc->type_id || prog->expected_attach_type != i) { bpf_prog_put(prog); err = -EINVAL; @@ -588,7 +590,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key) BPF_STRUCT_OPS_STATE_TOBEFREE); switch (prev_state) { case BPF_STRUCT_OPS_STATE_INUSE: - st_map->st_ops->unreg(&st_map->kvalue.data); + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); bpf_map_put(map); return 0; case BPF_STRUCT_OPS_STATE_TOBEFREE: @@ -669,22 +671,22 @@ static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) { - const struct bpf_struct_ops *st_ops; + const struct bpf_struct_ops_desc *st_ops_desc; size_t st_map_size; struct bpf_struct_ops_map *st_map; const struct btf_type *t, *vt; struct bpf_map *map; int ret; - st_ops = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id); - if (!st_ops) + st_ops_desc = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id); + if (!st_ops_desc) return ERR_PTR(-ENOTSUPP); - vt = st_ops->value_type; + vt = st_ops_desc->value_type; if (attr->value_size != vt->size) return ERR_PTR(-EINVAL); - t = st_ops->type; + t = st_ops_desc->type; st_map_size = sizeof(*st_map) + /* kvalue stores the @@ -696,7 +698,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) if (!st_map) return ERR_PTR(-ENOMEM); - st_map->st_ops = st_ops; + st_map->st_ops_desc = st_ops_desc; map = &st_map->map; ret = bpf_jit_charge_modmem(PAGE_SIZE); @@ -733,8 +735,8 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) static u64 bpf_struct_ops_map_mem_usage(const struct bpf_map *map) { struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; - const struct bpf_struct_ops *st_ops = st_map->st_ops; - const struct btf_type *vt = st_ops->value_type; + const struct bpf_struct_ops_desc *st_ops_desc = st_map->st_ops_desc; + const struct btf_type *vt = st_ops_desc->value_type; u64 usage; usage = sizeof(*st_map) + @@ -808,7 +810,7 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) /* st_link->map can be NULL if * bpf_struct_ops_link_create() fails to register. */ - st_map->st_ops->unreg(&st_map->kvalue.data); + st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data); bpf_map_put(&st_map->map); } kfree(st_link); @@ -855,7 +857,7 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map if (!bpf_struct_ops_valid_to_reg(new_map)) return -EINVAL; - if (!st_map->st_ops->update) + if (!st_map->st_ops_desc->st_ops->update) return -EOPNOTSUPP; mutex_lock(&update_mutex); @@ -868,12 +870,12 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map old_st_map = container_of(old_map, struct bpf_struct_ops_map, map); /* The new and old struct_ops must be the same type. */ - if (st_map->st_ops != old_st_map->st_ops) { + if (st_map->st_ops_desc != old_st_map->st_ops_desc) { err = -EINVAL; goto err_out; } - err = st_map->st_ops->update(st_map->kvalue.data, old_st_map->kvalue.data); + err = st_map->st_ops_desc->st_ops->update(st_map->kvalue.data, old_st_map->kvalue.data); if (err) goto err_out; @@ -924,7 +926,7 @@ int bpf_struct_ops_link_create(union bpf_attr *attr) if (err) goto err_out; - err = st_map->st_ops->reg(st_map->kvalue.data); + err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data); if (err) { bpf_link_cleanup(&link_primer); link = NULL; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index adbf330d364b..a187317500dd 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -20229,6 +20229,7 @@ static void print_verification_stats(struct bpf_verifier_env *env) static int check_struct_ops_btf_id(struct bpf_verifier_env *env) { const struct btf_type *t, *func_proto; + const struct bpf_struct_ops_desc *st_ops_desc; const struct bpf_struct_ops *st_ops; const struct btf_member *member; struct bpf_prog *prog = env->prog; @@ -20241,14 +20242,15 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) } btf_id = prog->aux->attach_btf_id; - st_ops = bpf_struct_ops_find(btf_id); - if (!st_ops) { + st_ops_desc = bpf_struct_ops_find(btf_id); + if (!st_ops_desc) { verbose(env, "attach_btf_id %u is not a supported struct\n", btf_id); return -ENOTSUPP; } + st_ops = st_ops_desc->st_ops; - t = st_ops->type; + t = st_ops_desc->type; member_idx = prog->expected_attach_type; if (member_idx >= btf_type_vlen(t)) { verbose(env, "attach to invalid member idx %u of struct %s\n", diff --git a/net/bpf/bpf_dummy_struct_ops.c b/net/bpf/bpf_dummy_struct_ops.c index 8906f7bdf4a9..ba2c58dba2da 100644 --- a/net/bpf/bpf_dummy_struct_ops.c +++ b/net/bpf/bpf_dummy_struct_ops.c @@ -22,6 +22,8 @@ struct bpf_dummy_ops_test_args { struct bpf_dummy_ops_state state; }; +static struct btf *bpf_dummy_ops_btf; + static struct bpf_dummy_ops_test_args * dummy_ops_init_args(const union bpf_attr *kattr, unsigned int nr) { @@ -90,9 +92,15 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, void *image = NULL; unsigned int op_idx; int prog_ret; + s32 type_id; int err; - if (prog->aux->attach_btf_id != st_ops->type_id) + type_id = btf_find_by_name_kind(bpf_dummy_ops_btf, + bpf_bpf_dummy_ops.name, + BTF_KIND_STRUCT); + if (type_id < 0) + return -EINVAL; + if (prog->aux->attach_btf_id != type_id) return -EOPNOTSUPP; func_proto = prog->aux->attach_func_proto; @@ -148,6 +156,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, static int bpf_dummy_init(struct btf *btf) { + bpf_dummy_ops_btf = btf; return 0; } diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index ae8b15e6896f..dffd8828079b 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -20,6 +20,7 @@ static u32 unsupported_ops[] = { static const struct btf_type *tcp_sock_type; static u32 tcp_sock_id, sock_id; +static const struct btf_type *tcp_congestion_ops_type; static int bpf_tcp_ca_init(struct btf *btf) { @@ -36,6 +37,11 @@ static int bpf_tcp_ca_init(struct btf *btf) tcp_sock_id = type_id; tcp_sock_type = btf_type_by_id(btf, tcp_sock_id); + type_id = btf_find_by_name_kind(btf, "tcp_congestion_ops", BTF_KIND_STRUCT); + if (type_id < 0) + return -EINVAL; + tcp_congestion_ops_type = btf_type_by_id(btf, type_id); + return 0; } @@ -149,7 +155,7 @@ static u32 prog_ops_moff(const struct bpf_prog *prog) u32 midx; midx = prog->expected_attach_type; - t = bpf_tcp_congestion_ops.type; + t = tcp_congestion_ops_type; m = &btf_type_member(t)[midx]; return __btf_member_bit_offset(t, m) / 8; From patchwork Thu Jan 18 01:49:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522293 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f177.google.com (mail-yw1-f177.google.com [209.85.128.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3534128FD for ; Thu, 18 Jan 2024 01:49:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542595; cv=none; b=YS7mpb/EtoWLnSh0C7xoVjoX35TLUjrw4IQTl0QoueehOcdUGGlWxtAICyywkb7DJtgpjZZUOmmnJEF4FlRdFzcaO9NNnfNTWiuBTh6hU/7QTAl4Ss/cJuBIbKq3B5rfpjd8v0qTdyjHcOr5kHleHyBe1Wk/xDsrIGRQP6RbJhM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542595; c=relaxed/simple; bh=YRIGwvZW7MeY7BnLmba86/sxXf/RVcO9TPOGwErh2E0=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=TIAVubnh7d0VpaREyDiWNuRpstwvxqUN/vbYoxnHKs36UgTFnkYrHGZfsm9uFrWZ7XdrCNi1i+L5YwTgez5uWleGzgL9UlSFZJMwGUYSE/skHtblxUo/aMOb9d3i6Yh7uy9EbH0yGXizoEp7cjdWG8gJcSkvxSaVQMI+Wya9YIY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Fi/ivfIN; arc=none smtp.client-ip=209.85.128.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Fi/ivfIN" Received: by mail-yw1-f177.google.com with SMTP id 00721157ae682-5e734251f48so2284697b3.1 for ; Wed, 17 Jan 2024 17:49:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542593; x=1706147393; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4fe4Cc6sFReNUBcJKDRt3tmmSRTHio7j2z7H1Z+m2ik=; b=Fi/ivfINa0/+l5QQ4KP51MNITHsgT4e9dAVJz8JrS9QhluPfA+VG7t0G03isEh4Ja3 taA6aUxGuheOAkx6EeFsKlVESQviENNEpDiibBh7f1BALGmuoVrG1bInwKiBb+C16IqC HFvmbCnuA9heDh12IrKv3wbtbQGzaEiztxhzi1T3uBwnHPQ4GCCfX+kihFsC73eLkfQh 0UTkVZ3jTK0fVMZr+CzNjnG8IiZd07/qyQcSzUFHvnZhY67Q5HFBqNFew+5IAMCIgXMX ML+dqdhH17msiMUlG7qVDgMKe8dQZGGqIPf61hfTcy4WcavGu2o7ypW7/MhwkL3/Cs8O Bv4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542593; x=1706147393; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4fe4Cc6sFReNUBcJKDRt3tmmSRTHio7j2z7H1Z+m2ik=; b=f0mGXS9WssGRQRgOMD+03XHfDgis50zjNsSa9S2hFI8t1GJvl5Rww+wNsntBkZrw7I cXze3bV50Q+gSoqoFgbTG/4DYPVtkL2vpr/z24ZsK88HWs1flBs1+zr0r50eCI3RraRJ 8fZ353Y8V2DFLWQWt7HjsGcmfOcC5auUZzaJz9QaBPKLWeQJGw3gy3bBqiLIZgdjywNU vBrb4yFws+cPjz8DGtGE6IQYb4lTF4dVMtu4dkto8p2J59M7ikKn9uAlIO6MDjiiTOGo reiYT5Ni3gryRHz2eOtdrxPYn5725p4daWbYc2Z4n7sC3twKdxdwevWOBakIjZkD0cyy MkAg== X-Gm-Message-State: AOJu0YxJ6jpV5mMyYj+rtwvKLhVHQn04sUoidETn2s+9xif20JzWYN0X Z6Sz8C41ELrdxVu0WXFMzSzScjJbPVybFuwvmXx/uDvLRZKGm7LjErn7L1Dl X-Google-Smtp-Source: AGHT+IFXgsfqiOre38poyEaAhuYPj8SeBkoV6lkWz9bXsc5WWK5EXTox9hFlqUSu4Js4OZ2W1MV6Vg== X-Received: by 2002:a81:5dc5:0:b0:5fa:d966:e223 with SMTP id r188-20020a815dc5000000b005fad966e223mr126407ywb.44.1705542592866; Wed, 17 Jan 2024 17:49:52 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:52 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 04/14] bpf: add struct_ops_tab to btf. Date: Wed, 17 Jan 2024 17:49:20 -0800 Message-Id: <20240118014930.1992551-5-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module. It is a preparation work for supporting kernel module struct_ops in a latter patch. Each struct_ops will be registered under its own kernel module btf and will be stored in the newly added btf->struct_ops_tab. The bpf verifier and bpf syscall (e.g. prog and map cmd) can find the struct_ops and its btf type/size/id... information from btf->struct_ops_tab. Signed-off-by: Kui-Feng Lee --- include/linux/btf.h | 2 ++ kernel/bpf/btf.c | 62 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+) diff --git a/include/linux/btf.h b/include/linux/btf.h index 1d852dad7473..a68604904f4e 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -584,4 +584,6 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type return btf_type_is_struct(t); } +struct bpf_struct_ops_desc; + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 1f94c250ab49..3fa84c44b882 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -241,6 +241,12 @@ struct btf_id_dtor_kfunc_tab { struct btf_id_dtor_kfunc dtors[]; }; +struct btf_struct_ops_tab { + u32 cnt; + u32 capacity; + struct bpf_struct_ops_desc ops[]; +}; + struct btf { void *data; struct btf_type **types; @@ -258,6 +264,7 @@ struct btf { struct btf_kfunc_set_tab *kfunc_set_tab; struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; struct btf_struct_metas *struct_meta_tab; + struct btf_struct_ops_tab *struct_ops_tab; /* split BTF support */ struct btf *base_btf; @@ -1688,11 +1695,20 @@ static void btf_free_struct_meta_tab(struct btf *btf) btf->struct_meta_tab = NULL; } +static void btf_free_struct_ops_tab(struct btf *btf) +{ + struct btf_struct_ops_tab *tab = btf->struct_ops_tab; + + kfree(tab); + btf->struct_ops_tab = NULL; +} + static void btf_free(struct btf *btf) { btf_free_struct_meta_tab(btf); btf_free_dtor_kfunc_tab(btf); btf_free_kfunc_set_tab(btf); + btf_free_struct_ops_tab(btf); kvfree(btf->types); kvfree(btf->resolved_sizes); kvfree(btf->resolved_ids); @@ -8471,3 +8487,49 @@ bool btf_type_ids_nocast_alias(struct bpf_verifier_log *log, return !strncmp(reg_name, arg_name, cmp_len); } + +static int +btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops) +{ + struct btf_struct_ops_tab *tab, *new_tab; + int i; + + if (!btf) + return -ENOENT; + + /* Assume this function is called for a module when the module is + * loading. + */ + + tab = btf->struct_ops_tab; + if (!tab) { + tab = kzalloc(offsetof(struct btf_struct_ops_tab, ops[4]), + GFP_KERNEL); + if (!tab) + return -ENOMEM; + tab->capacity = 4; + btf->struct_ops_tab = tab; + } + + for (i = 0; i < tab->cnt; i++) + if (tab->ops[i].st_ops == st_ops) + return -EEXIST; + + if (tab->cnt == tab->capacity) { + new_tab = krealloc(tab, + offsetof(struct btf_struct_ops_tab, + ops[tab->capacity * 2]), + GFP_KERNEL); + if (!new_tab) + return -ENOMEM; + tab = new_tab; + tab->capacity *= 2; + btf->struct_ops_tab = tab; + } + + tab->ops[btf->struct_ops_tab->cnt].st_ops = st_ops; + + btf->struct_ops_tab->cnt++; + + return 0; +} From patchwork Thu Jan 18 01:49:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522294 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4D4A5665 for ; Thu, 18 Jan 2024 01:49:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542597; cv=none; b=jKeKy1rni6rZ6A8JrmHztSpI/f4HNK5Ut77KtuxlKo1YF7fH4Z2frotXHoJ9XPZJVTBhQqK0oMYuQHQR+8c+3NsBdeXrf3UHB1pGBAXvKQDsCOnbDhvP3rM8DSYSWLV8SBsBTAz9RtCxVkejpEDfEhVdIg0mqUj8kSE64l8Jutk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542597; c=relaxed/simple; bh=XbywLo7TLmwiVSYMFXgs0Q7tcoif4V/s/i7wQ9R8joQ=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=CS0X2RGIfWrGTkztG25UCFWCemmQ94dTRtSLaBvu8J24vrum+wZvL4Gc5DMmjiBmCLCKqOKL3Kvg7gHn9DpflPe16PzCqtRo/PgaPJC8Mk1PTVI4Os2/xOMaV3Q41bokftESAoGm5ri+qluNa8ha2vlQjER5K/01BfAzqim19lw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NzwWyFfE; arc=none smtp.client-ip=209.85.128.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NzwWyFfE" Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-5ff7dc53ce0so6440307b3.1 for ; Wed, 17 Jan 2024 17:49:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542594; x=1706147394; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6cX5edNAyRXlgv6ZNVD4EGw6vjLdbpUYx/HNHgTpU3w=; b=NzwWyFfENzr7WKXhRHx1OJ+U/XBH/25DvK3mOay1heXyqDBCQ3NTPWI8D9x259AFmm MKQVahEJe4AJNj1XILAoDHJAiCpVf6++z4tDK3wVi/TVK3ww1eZTMAhXv9R0C9kDZo3h tOePJdLo0W/JYndGNCxKPJW6+tPIH4Hmek2QzqK2juAgSUsg3vCk5/YrRedjDfEkGAOg rTJxJpzVM126VC4oAf74JACzjGxgQIFL5d+VgZo/gfaMTC/KUr+yBEYNVwgXoWoVsRYA jPrJ9XVuv/a/kgnBTf8dtPOFJ0Z+7ifP0EHODAB1DcP92P7RuXaECU7CfOwa1alUobnI g5Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542594; x=1706147394; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6cX5edNAyRXlgv6ZNVD4EGw6vjLdbpUYx/HNHgTpU3w=; b=KdAOA5fy215/+4zLjmrZYMsRnohRa7d6RCe2lspZfIeONx7x6j6hDh8jMwXXl1JnWL 9EOTjJYyZnFBwEBuyzmmIRI/Q5Xj3dMGpGrcW+oZ4Qtd44K71smnREQXaXZVO6NQISeF ch6P7ItRmpf/zspJbj3BuG9AJT/mfBPMmTFQOP0AKFKrCxmR+fU6HA/r276vACMGbf5d Dfi1DoIIzCoekjceTq9l0TauLjcOKWAs1nmaPu78giiAgFIyl1EH1AyDT0llyDGgLwSR wuHekkhNEpypV42qPEFNfm7d5dlQJjYDL9XraPA6kzMvxwOGCcDXv9Qtc26CRePZBC1r rNfA== X-Gm-Message-State: AOJu0YwyS77kitX84Sol21za4tOdp4aIeQWVVclFl+wmqaJ8HAWFzVmL t/3SXLv9fufl2Rxbjv/6+fq99DTovOBmDb53DQ6LELecbMSLfarXMikmkZCQ X-Google-Smtp-Source: AGHT+IGGCBX5PK+iBV47VkR2gXsnuFPz2D1uUn3LyycuNXFo1mTY+FUZSmh7ePCcukrat/p8zo6JSw== X-Received: by 2002:a81:fd0c:0:b0:5d7:1940:8df8 with SMTP id g12-20020a81fd0c000000b005d719408df8mr103428ywn.95.1705542594300; Wed, 17 Jan 2024 17:49:54 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:53 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 05/14] bpf: make struct_ops_map support btfs other than btf_vmlinux. Date: Wed, 17 Jan 2024 17:49:21 -0800 Message-Id: <20240118014930.1992551-6-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Once new struct_ops can be registered from modules, btf_vmlinux is no longer the only btf that struct_ops_map would face. st_map should remember what btf it should use to get type information. Signed-off-by: Kui-Feng Lee --- kernel/bpf/bpf_struct_ops.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 9774f7824e8b..5ddcca4c4fba 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -46,6 +46,8 @@ struct bpf_struct_ops_map { * "links[]". */ void *image; + /* The owner moduler's btf. */ + struct btf *btf; /* uvalue->data stores the kernel struct * (e.g. tcp_congestion_ops) that is more useful * to userspace than the kvalue. For example, @@ -314,7 +316,7 @@ static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map) } } -static int check_zero_holes(const struct btf_type *t, void *data) +static int check_zero_holes(const struct btf *btf, const struct btf_type *t, void *data) { const struct btf_member *member; u32 i, moff, msize, prev_mend = 0; @@ -326,8 +328,8 @@ static int check_zero_holes(const struct btf_type *t, void *data) memchr_inv(data + prev_mend, 0, moff - prev_mend)) return -EINVAL; - mtype = btf_type_by_id(btf_vmlinux, member->type); - mtype = btf_resolve_size(btf_vmlinux, mtype, &msize); + mtype = btf_type_by_id(btf, member->type); + mtype = btf_resolve_size(btf, mtype, &msize); if (IS_ERR(mtype)) return PTR_ERR(mtype); prev_mend = moff + msize; @@ -401,12 +403,12 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, if (*(u32 *)key != 0) return -E2BIG; - err = check_zero_holes(st_ops_desc->value_type, value); + err = check_zero_holes(st_map->btf, st_ops_desc->value_type, value); if (err) return err; uvalue = value; - err = check_zero_holes(t, uvalue->data); + err = check_zero_holes(st_map->btf, t, uvalue->data); if (err) return err; @@ -442,7 +444,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, u32 moff; moff = __btf_member_bit_offset(t, member) / 8; - ptype = btf_type_resolve_ptr(btf_vmlinux, member->type, NULL); + ptype = btf_type_resolve_ptr(st_map->btf, member->type, NULL); if (ptype == module_type) { if (*(void **)(udata + moff)) goto reset_unlock; @@ -467,8 +469,8 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, if (!ptype || !btf_type_is_func_proto(ptype)) { u32 msize; - mtype = btf_type_by_id(btf_vmlinux, member->type); - mtype = btf_resolve_size(btf_vmlinux, mtype, &msize); + mtype = btf_type_by_id(st_map->btf, member->type); + mtype = btf_resolve_size(st_map->btf, mtype, &msize); if (IS_ERR(mtype)) { err = PTR_ERR(mtype); goto reset_unlock; @@ -607,6 +609,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key) static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key, struct seq_file *m) { + struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; void *value; int err; @@ -616,7 +619,8 @@ static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key, err = bpf_struct_ops_map_sys_lookup_elem(map, key, value); if (!err) { - btf_type_seq_show(btf_vmlinux, map->btf_vmlinux_value_type_id, + btf_type_seq_show(st_map->btf, + map->btf_vmlinux_value_type_id, value, m); seq_puts(m, "\n"); } @@ -726,6 +730,8 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); } + st_map->btf = btf_vmlinux; + mutex_init(&st_map->lock); bpf_map_init_from_attr(map, attr); From patchwork Thu Jan 18 01:49:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522295 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15A266106 for ; Thu, 18 Jan 2024 01:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542598; cv=none; b=q2et7JKqWFENF0T1ItEyu80nAinhdRVXRU+QdmdbUmc2RtWbogW5gm+vqj3QCsqvBU6ky9/B2IjAjWc9K/bMGOMG5SXvAdEPgOSolMrQ0+CbNQz8FOnXu+p7ve++l7gU0mypWau3PiEddBovvAekarASSdEe47hj1sdFFTtIp0w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542598; c=relaxed/simple; bh=IOkgiO07bBK/wrJ4r9fDxQUgBloiAHVwSLk7tTq7/Os=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=hexM7aVmgXTBvnhc6lNv1YGdgiljBZy4+s2/gt3aseukU8dmvttOMbKUVN2dhrTi6M+b5rJgaG1jLhOcMcaKOVB4uhe4zN8SfScNhwYqrQMgqOBb8JK1ZJY+rmiPpgGYK82ZeV41hAQtod0J017WkF4ArIRiF5B1AX01cozUeF0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OjrNvu7q; arc=none smtp.client-ip=209.85.128.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OjrNvu7q" Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-5e54d40cca2so91055987b3.3 for ; Wed, 17 Jan 2024 17:49:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542595; x=1706147395; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nOEWFr262czTiNDRv4aa6h4SqGe3Y26acn2oWpaKI20=; b=OjrNvu7q7FFi7C4KFCWljTF4FxWmRzB2zLDmXXSSU6kMIcC+D1cbBa3qS68tW+fwYm Gxmu9326GK2F/Ks03hku198JU8ukggL5wswukdWg2i24L3+HQqLnsa8JDjrJg8VtoYfY +bnwPWsM2nLio6H7gQjQvQBqEz41zSdxvhPHcqUie5Rmm1KcVApH7MzYrZ3RdkFKI5lT XmxGe3q37bDMg0uNrOBDt02dcUW/ispWQRMAzQTdFq3jVmwJfnnwkjn2+1WaX4JTgcRm 1PXlXXPadvO+pjJXBWEKAB4a00H1XcKpnuACAuji08cqBYKBXbHIxwhSqrQLmajM4Nci pEYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542595; x=1706147395; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nOEWFr262czTiNDRv4aa6h4SqGe3Y26acn2oWpaKI20=; b=sDReBo1qCiKIgd5vk4foP2l4/rYO0WzuCP9OgGWsS6e8fseTwktRVK8oeBozK8UAih HEinhOpmW9DJh2jryTKlZj172SLdqutMUQvCY1RtJAJut+xFymXERlabQcD6q8NigTaI TxqzY2vW77p+fQ4oI7y30P74jqri/yN1V3ngy8UV1IHOYhcIEQ9Kly7BYGJAxxEpUzML PhKWpFTaeNSYpgQiOFDs1sX5FM+83LA1EbIN0Z7ugjGPLtGrBhZoUVer4xwh3Iid2jEW iJZFg2eHiEZFKWnvfNHTkBO4YB6zmrrr07+yR1Y+fsNaHCLzaGcvYu7dpB1ALq/NP9OZ 7Ruw== X-Gm-Message-State: AOJu0Yw0KtYyH+ierl4tKsL5+E7rxVC7lMJcvp9GihLLcs9cp2KlYbWX prw7oiYSzFX0AKZ/zGHbQtYb08bzUy59vKGYnnqu8odeIMOpl4/hIVCXHw1J X-Google-Smtp-Source: AGHT+IHRGt3aedlVoJa2qLqspyceUocMHB4QDeiaffKxwTteV+pDjoe0rPEQ6z77nvdjbdpuunHtnA== X-Received: by 2002:a0d:d5c8:0:b0:5ff:51f1:c2 with SMTP id x191-20020a0dd5c8000000b005ff51f100c2mr85410ywd.75.1705542595578; Wed, 17 Jan 2024 17:49:55 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:55 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 06/14] bpf: pass btf object id in bpf_map_info. Date: Wed, 17 Jan 2024 17:49:22 -0800 Message-Id: <20240118014930.1992551-7-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Include btf object id (btf_obj_id) in bpf_map_info so that tools (ex: bpftools struct_ops dump) know the correct btf from the kernel to look up type information of struct_ops types. Since struct_ops types can be defined and registered in a module. The type information of a struct_ops type are defined in the btf of the module defining it. The userspace tools need to know which btf is for the module defining a struct_ops type. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 2 +- kernel/bpf/bpf_struct_ops.c | 7 +++++++ kernel/bpf/syscall.c | 2 ++ tools/include/uapi/linux/bpf.h | 2 +- 5 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ed4352f56d21..1e969d035b42 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1732,6 +1732,7 @@ struct bpf_dummy_ops { int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); #endif +void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map); #else static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id) { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 754e68ca8744..8eb949b52102 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6487,7 +6487,7 @@ struct bpf_map_info { __u32 btf_id; __u32 btf_key_type_id; __u32 btf_value_type_id; - __u32 :32; /* alignment pad */ + __u32 btf_vmlinux_id; __u64 map_extra; } __attribute__((aligned(8))); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 5ddcca4c4fba..5e98af4fc2e2 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -947,3 +947,10 @@ int bpf_struct_ops_link_create(union bpf_attr *attr) kfree(link); return err; } + +void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map) +{ + struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; + + info->btf_vmlinux_id = btf_obj_id(st_map->btf); +} diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index a1f18681721c..05fe78ecc441 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -4687,6 +4687,8 @@ static int bpf_map_get_info_by_fd(struct file *file, info.btf_value_type_id = map->btf_value_type_id; } info.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; + if (map->map_type == BPF_MAP_TYPE_STRUCT_OPS) + bpf_map_struct_ops_info_fill(&info, map); if (bpf_map_is_offloaded(map)) { err = bpf_map_offload_info_fill(&info, map); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 7f24d898efbb..1949d881f230 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -6487,7 +6487,7 @@ struct bpf_map_info { __u32 btf_id; __u32 btf_key_type_id; __u32 btf_value_type_id; - __u32 :32; /* alignment pad */ + __u32 btf_vmlinux_id; __u64 map_extra; } __attribute__((aligned(8))); From patchwork Thu Jan 18 01:49:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522296 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f181.google.com (mail-yw1-f181.google.com [209.85.128.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 334C9539A for ; Thu, 18 Jan 2024 01:49:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542599; cv=none; b=UnaPp0JqLqv1pR9ECfIv+i7bFazeggE6qaYsezO5SGywNVo1MtDfFBqTeiqNrBSmGKDbo5XjPjGCVHBSxF9l8BRLMstPvh+gZFN/hRbR104+J+VYAAWBOeflP9nZ69+14hA4awNxlRkyT1n3h/NOeApSwedR0TOeGFOvt8zg+aM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542599; c=relaxed/simple; bh=lHlr6xo/jV93GA/GDJi41M1rVRXfLgqfjWdlAq1EEEo=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=GadRBI194XF7+jP2kDRrB2dJh4jQlQibkbwXvrrw0XQQd2mhO4eunu+TkL0wJvEtsCqznil68hCoyrfbJNS0f0YyjrfmHKF/K+ppOMiEWDkm820izXgTSv8Hf4NbaQggjpMTa4xNR2QMiM9HiZOJiTonEgGGFth1ifcXt/bn6D4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dcPOuDxg; arc=none smtp.client-ip=209.85.128.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dcPOuDxg" Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-5ff847429d4so2654697b3.1 for ; Wed, 17 Jan 2024 17:49:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542597; x=1706147397; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wrl7sbOFgHUbQRf8u0fI0nyuTg4aRrb+2j5aLreyuQA=; b=dcPOuDxgYJ5ANoUFBqmdRPS1HcYQ7wYbGr+1IMa0NtzUgrMna6aOVwpRnpnHgjUm3E hkA2O29eY4KV1A4v9XMrKscS+gXR2TzINL6jxwvlrC/HHbM8C78Bk55De3a5QEYY+qvC qgZerItmNewF2LgRCkr7gpVEGTuATyIIIz0RD4/5kAU8keEo/ahi07y3UxJtUJkcBf8B SE3zeGz6GpNoOx5DQgUmSCBEcTVuuF5zxVhvXkgPNZh/XPJ6H79+gsvpLbnq6/PZn7Yt uED1rUzdzvH8FijzNTz+GGnMTXGeOaLLur29ooNXcu2Yz+qe+hilDwjYf8jbPC9uUND8 MsQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542597; x=1706147397; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wrl7sbOFgHUbQRf8u0fI0nyuTg4aRrb+2j5aLreyuQA=; b=LooRlpsQFiYoiH+RuA+EkvismG+HWBDjnYPcV8qc2uMkNt8TBE7ZwiVJZk6CV5N/I9 R0idCtMY6iLNX+xKp4wdiu1jU4LBgXpw1QqpBLM+HGpN5Z0osuPcTH1h3ixKPY+b+zGM 97tPOgv5Jay+r/VkQbuuTeE9WkNDAqNDSEwooFmci5/K8ZdWFkcq5wWNnmx8QGIpjGek jXBESaL3My/157+oNBCOKTpCkEaNo2NS0GveeUcXFCCRq/B/HdU8KMuPgdlgCtmQjdc6 sG436xe23MEoO2qzB0asgz0fQs+PxrvfeKprkGLkegQvouOWSQbrVbmuCQEyCcXQflTR 2lAQ== X-Gm-Message-State: AOJu0Yz2qirq6RrbBP85ZFV4SQoDMc1kCPwj/H6CgkRoi5KXJIhgHZGw CVFTq0ZSR34FBLfGBA0xBewoArlE6XJKVP6JsQ+otK+xPFZHljANUdXfCBVH X-Google-Smtp-Source: AGHT+IG83Q6KvKHEfQEE+IN7GYg/YqqxTa2x6nPatGi7KXCO75wcFbZZ09jWV0Cbv0w0i6vaWmPF1Q== X-Received: by 2002:a0d:f685:0:b0:5d7:1941:356b with SMTP id g127-20020a0df685000000b005d71941356bmr123426ywf.82.1705542596965; Wed, 17 Jan 2024 17:49:56 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:56 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 07/14] bpf: lookup struct_ops types from a given module BTF. Date: Wed, 17 Jan 2024 17:49:23 -0800 Message-Id: <20240118014930.1992551-8-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee This is a preparation for searching for struct_ops types from a specified module. BTF is always btf_vmlinux now. This patch passes a pointer of BTF to bpf_struct_ops_find_value() and bpf_struct_ops_find(). Once the new registration API of struct_ops types is used, other BTFs besides btf_vmlinux can also be passed to them. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 4 ++-- kernel/bpf/bpf_struct_ops.c | 11 ++++++----- kernel/bpf/verifier.c | 2 +- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 1e969d035b42..3d1c1014fdb2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1689,7 +1689,7 @@ struct bpf_struct_ops_desc { #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL) #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA)) -const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id); +const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id); void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log); bool bpf_struct_ops_get(const void *kdata); void bpf_struct_ops_put(const void *kdata); @@ -1734,7 +1734,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, #endif void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map); #else -static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id) +static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id) { return NULL; } diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 5e98af4fc2e2..7505f515aac3 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -221,11 +221,11 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) extern struct btf *btf_vmlinux; static const struct bpf_struct_ops_desc * -bpf_struct_ops_find_value(u32 value_id) +bpf_struct_ops_find_value(struct btf *btf, u32 value_id) { unsigned int i; - if (!value_id || !btf_vmlinux) + if (!value_id || !btf) return NULL; for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { @@ -236,11 +236,12 @@ bpf_struct_ops_find_value(u32 value_id) return NULL; } -const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id) +const struct bpf_struct_ops_desc * +bpf_struct_ops_find(struct btf *btf, u32 type_id) { unsigned int i; - if (!type_id || !btf_vmlinux) + if (!type_id || !btf) return NULL; for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { @@ -682,7 +683,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) struct bpf_map *map; int ret; - st_ops_desc = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id); + st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, attr->btf_vmlinux_value_type_id); if (!st_ops_desc) return ERR_PTR(-ENOTSUPP); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a187317500dd..0744a1f194fa 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -20242,7 +20242,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) } btf_id = prog->aux->attach_btf_id; - st_ops_desc = bpf_struct_ops_find(btf_id); + st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id); if (!st_ops_desc) { verbose(env, "attach_btf_id %u is not a supported struct\n", btf_id); From patchwork Thu Jan 18 01:49:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522297 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B051D6132 for ; Thu, 18 Jan 2024 01:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542601; cv=none; b=CAddGDQfL2XbT3T0hCvh+tWAtS/ZjLLJ+xIcW67ZaS2pgk2r3yBgUlRCIRSG9MqkuzsvL/ua24PSX1R1Sq4OMl9FoxDwfaEbufjO/zAyoAjPTRM9uDPyPVe71crLsEvoUrfr+rhBePo2Nl66UCPKHzDpenc/U799/QcZbLR2gPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542601; c=relaxed/simple; bh=WvCkmgY55g8iBKkRmUDJdCrbZ5jw3c9lemvyV/n4qe8=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=d2Jr+nyO8m4KFIf5jU2Z/joQVtMUwKBJksSU3gaFaNjNbVqUxhC93DbMnPAMfKBcemJrEGImLcBKWqu4hvclNIF1zbs6yE1DwDovm/y4rkqpC2BDkQJbECsXS2TcrpiUqy6Um9PUQUXP9WfYJ8er8MQfwmukxVbjtbPT4a+2gpU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JXMCeZkc; arc=none smtp.client-ip=209.85.128.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JXMCeZkc" Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-5eefd0da5c0so121718717b3.2 for ; Wed, 17 Jan 2024 17:49:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542598; x=1706147398; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bBK9IYm2aHddPI6w+02qZMwThls/3Mjssq7z6W9IB2I=; b=JXMCeZkcY+wo/dhAFhxAx8yvqpz1mE2YkD0Xm1+QBCwVFA//LCiWYwAD/Y7/SPtFCO R9dyJOtna6f8ijkXbFZK38TCimL5BkuMv4/yOVM8cNrjY/kGZ00vB52yulFZUMMr8+hN FUd0M2Km27XzaYWCrsUQCxVwlQvlZyLiKAJPh3/G4E96FKJB+wAQHKGxWdeyu4yOCsju o2uhdm2fuH9aG1wd6G725E6NhCfZuwoHpoukTq+gvUz2h1FE09btTRzpGZXP2pzheuBJ 8zrtFAI5SnonVx82dHsER1clxjbj8kQAKcgotoPrc8NtAYb+Nv2sP/Y3Rtsx7CvAkxiG kLEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542598; x=1706147398; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bBK9IYm2aHddPI6w+02qZMwThls/3Mjssq7z6W9IB2I=; b=vl+IAQdN7aCZsc1NGespEkdezb9oeQJLbXoLENterpKjtsxvwAJ4SG5fw29knYg6dR TdO18DrlBAf5rUAe4QXfVVVMpMzMuQuNELh28Kwk09+y37bShOTaTXgfuiNHxjiaNTnV yozksog6gG+wuSIBA4eJO+3SAcyp1kUhAbnTYaDZGByzhxZDWbDHVuUeVBB2caLMKTg5 fZIXD8tYSIaGsjmuNhN2mBk2svaCbmZVdplhy8B2l4JD+UzTq/qrbZZBrzN4ePhKcvax PrXi6fSYzDECcrkGLBlbY8R9H5jrR8NmM4KSIVDYTN4unZsTIzjohToNt7eg7uKCbHVE Uq7Q== X-Gm-Message-State: AOJu0Yx9M49lZJJ1wle7GYNAjzOGWtHdxSmEeW1W4NrooP1NdlNpUMEA c9af6rl3CeWUuaf/Jr0BvPq47+Nrrk4+8E0hS34NXM5Ff/GNV2Zin4UV0Dr9 X-Google-Smtp-Source: AGHT+IHcYAGjH8WhxpzRiPhyUa4QW9vyIxsgdPAsKZ02R4o49qf/oGWe8mKMxx1Ca5Tq9l53awQ54w== X-Received: by 2002:a81:6544:0:b0:5ff:48b2:9551 with SMTP id z65-20020a816544000000b005ff48b29551mr156407ywb.66.1705542598288; Wed, 17 Jan 2024 17:49:58 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:57 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 08/14] bpf: pass attached BTF to the bpf_struct_ops subsystem Date: Wed, 17 Jan 2024 17:49:24 -0800 Message-Id: <20240118014930.1992551-9-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Pass the fd of a btf from the userspace to the bpf() syscall, and then convert the fd into a btf. The btf is generated from the module that defines the target BPF struct_ops type. In order to inform the kernel about the module that defines the target struct_ops type, the userspace program needs to provide a btf fd for the respective module's btf. This btf contains essential information on the types defined within the module, including the target struct_ops type. A btf fd must be provided to the kernel for struct_ops maps and for the bpf programs attached to those maps. In the case of the bpf programs, the attach_btf_obj_fd parameter is passed as part of the bpf_attr and is converted into a btf. This btf is then stored in the prog->aux->attach_btf field. Here, it just let the verifier access attach_btf directly. In the case of struct_ops maps, a btf fd is passed as value_type_btf_obj_fd of bpf_attr. The bpf_struct_ops_map_alloc() function converts the fd to a btf and stores it as st_map->btf. A flag BPF_F_VTYPE_BTF_OBJ_FD is added for map_flags to indicate that the value of value_type_btf_obj_fd is set. Signed-off-by: Kui-Feng Lee --- include/uapi/linux/bpf.h | 8 +++++ kernel/bpf/bpf_struct_ops.c | 65 ++++++++++++++++++++++++---------- kernel/bpf/syscall.c | 2 +- kernel/bpf/verifier.c | 9 +++-- tools/include/uapi/linux/bpf.h | 8 +++++ 5 files changed, 70 insertions(+), 22 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 8eb949b52102..adb4dc8582f7 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1330,6 +1330,9 @@ enum { /* Get path from provided FD in BPF_OBJ_PIN/BPF_OBJ_GET commands */ BPF_F_PATH_FD = (1U << 14), + +/* Flag for value_type_btf_obj_fd, the fd is available */ + BPF_F_VTYPE_BTF_OBJ_FD = (1U << 15), }; /* Flags for BPF_PROG_QUERY. */ @@ -1403,6 +1406,11 @@ union bpf_attr { * to using 5 hash functions). */ __u64 map_extra; + + __s32 value_type_btf_obj_fd; /* fd pointing to a BTF + * type data for + * btf_vmlinux_value_type_id. + */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 7505f515aac3..3b8d689ece5d 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -641,6 +641,7 @@ static void __bpf_struct_ops_map_free(struct bpf_map *map) bpf_jit_uncharge_modmem(PAGE_SIZE); } bpf_map_area_free(st_map->uvalue); + btf_put(st_map->btf); bpf_map_area_free(st_map); } @@ -669,7 +670,8 @@ static void bpf_struct_ops_map_free(struct bpf_map *map) static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) { if (attr->key_size != sizeof(unsigned int) || attr->max_entries != 1 || - (attr->map_flags & ~BPF_F_LINK) || !attr->btf_vmlinux_value_type_id) + (attr->map_flags & ~(BPF_F_LINK | BPF_F_VTYPE_BTF_OBJ_FD)) || + !attr->btf_vmlinux_value_type_id) return -EINVAL; return 0; } @@ -681,15 +683,36 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) struct bpf_struct_ops_map *st_map; const struct btf_type *t, *vt; struct bpf_map *map; + struct btf *btf; int ret; - st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, attr->btf_vmlinux_value_type_id); - if (!st_ops_desc) - return ERR_PTR(-ENOTSUPP); + if (attr->map_flags & BPF_F_VTYPE_BTF_OBJ_FD) { + /* The map holds btf for its whole life time. */ + btf = btf_get_by_fd(attr->value_type_btf_obj_fd); + if (IS_ERR(btf)) + return ERR_CAST(btf); + if (!btf_is_module(btf)) { + btf_put(btf); + return ERR_PTR(-EINVAL); + } + } else { + btf = bpf_get_btf_vmlinux(); + if (IS_ERR(btf)) + return ERR_CAST(btf); + btf_get(btf); + } + + st_ops_desc = bpf_struct_ops_find_value(btf, attr->btf_vmlinux_value_type_id); + if (!st_ops_desc) { + ret = -ENOTSUPP; + goto errout; + } vt = st_ops_desc->value_type; - if (attr->value_size != vt->size) - return ERR_PTR(-EINVAL); + if (attr->value_size != vt->size) { + ret = -EINVAL; + goto errout; + } t = st_ops_desc->type; @@ -700,17 +723,17 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) (vt->size - sizeof(struct bpf_struct_ops_value)); st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE); - if (!st_map) - return ERR_PTR(-ENOMEM); + if (!st_map) { + ret = -ENOMEM; + goto errout; + } st_map->st_ops_desc = st_ops_desc; map = &st_map->map; ret = bpf_jit_charge_modmem(PAGE_SIZE); - if (ret) { - __bpf_struct_ops_map_free(map); - return ERR_PTR(ret); - } + if (ret) + goto errout_free; st_map->image = arch_alloc_bpf_trampoline(PAGE_SIZE); if (!st_map->image) { @@ -719,24 +742,30 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) * here. */ bpf_jit_uncharge_modmem(PAGE_SIZE); - __bpf_struct_ops_map_free(map); - return ERR_PTR(-ENOMEM); + ret = -ENOMEM; + goto errout_free; } st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); st_map->links = bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *), NUMA_NO_NODE); if (!st_map->uvalue || !st_map->links) { - __bpf_struct_ops_map_free(map); - return ERR_PTR(-ENOMEM); + ret = -ENOMEM; + goto errout_free; } - - st_map->btf = btf_vmlinux; + st_map->btf = btf; mutex_init(&st_map->lock); bpf_map_init_from_attr(map, attr); return map; + +errout_free: + __bpf_struct_ops_map_free(map); +errout: + btf_put(btf); + + return ERR_PTR(ret); } static u64 bpf_struct_ops_map_mem_usage(const struct bpf_map *map) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 05fe78ecc441..00b294ce0a61 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1123,7 +1123,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, return ret; } -#define BPF_MAP_CREATE_LAST_FIELD map_extra +#define BPF_MAP_CREATE_LAST_FIELD value_type_btf_obj_fd /* called via syscall */ static int map_create(union bpf_attr *attr) { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0744a1f194fa..ff41f7736618 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -20234,6 +20234,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) const struct btf_member *member; struct bpf_prog *prog = env->prog; u32 btf_id, member_idx; + struct btf *btf; const char *mname; if (!prog->gpl_compatible) { @@ -20241,8 +20242,10 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) return -EINVAL; } + btf = prog->aux->attach_btf ?: bpf_get_btf_vmlinux(); + btf_id = prog->aux->attach_btf_id; - st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id); + st_ops_desc = bpf_struct_ops_find(btf, btf_id); if (!st_ops_desc) { verbose(env, "attach_btf_id %u is not a supported struct\n", btf_id); @@ -20259,8 +20262,8 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) } member = &btf_type_member(t)[member_idx]; - mname = btf_name_by_offset(btf_vmlinux, member->name_off); - func_proto = btf_type_resolve_func_ptr(btf_vmlinux, member->type, + mname = btf_name_by_offset(btf, member->name_off); + func_proto = btf_type_resolve_func_ptr(btf, member->type, NULL); if (!func_proto) { verbose(env, "attach to invalid member %s(@idx %u) of struct %s\n", diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 1949d881f230..7f64865bf455 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1330,6 +1330,9 @@ enum { /* Get path from provided FD in BPF_OBJ_PIN/BPF_OBJ_GET commands */ BPF_F_PATH_FD = (1U << 14), + +/* Flag for value_type_btf_obj_fd, the fd is available */ + BPF_F_VTYPE_BTF_OBJ_FD = (1U << 15), }; /* Flags for BPF_PROG_QUERY. */ @@ -1403,6 +1406,11 @@ union bpf_attr { * to using 5 hash functions). */ __u64 map_extra; + + __s32 value_type_btf_obj_fd; /* fd pointing to a BTF + * type data for + * btf_vmlinux_value_type_id. + */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ From patchwork Thu Jan 18 01:49:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522298 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0E178BF6 for ; Thu, 18 Jan 2024 01:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542602; cv=none; b=LaW1QhHxg//+Ugm/k5ynKikLB+APKW/Gvj5hjZ/rOYV46G/odBVdf6FOI33BfOa3HnIgBM7nwVj5GZLJ0g4UpMz+c8M4+0AWQSnBYjezAhg0jxORZqpWAz3RPHSgIeZmc6hQ9tCxTlHpInjZ9Sv5wUMtLGXKUHwXmE6+7Yw0VUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542602; c=relaxed/simple; bh=NooAWNO3LTebAmeOJ+n2t/0P2AmiQhkSeC0rq6h0eM0=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=EhwaLEso4ATa8WYFZf8iOcSg/eQvxkOLoQGwsMAHmHhxQrdi+6PvNvqbXriNvWdNUoCYEcit8Vo8dzxp17GYeWxXEXppTegnbuo6OZhNaPBUI7FUEl6Ht4t09F+4RDv9fLB9CFkW+cIUNyX6nbIFCyolIatLHS2eFM04m0oufbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lAnJy+L4; arc=none smtp.client-ip=209.85.128.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lAnJy+L4" Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-5ff6245c1deso17911607b3.1 for ; Wed, 17 Jan 2024 17:50:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542599; x=1706147399; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uGt/aSa+2dNBp0tw2j5P8nfr1B8a4C0rQJteFRrhxiI=; b=lAnJy+L4Htpc++HltHjfvN5vUMdXnO0b0xp5wdomaRKr1XJ0Kyt6xVli10p6ham4R/ MHrEgeR1yzgWIWBU8pz93btM9o945Z0csWhd3KuruaDLoIg0hBewXW12IM1ChsJuVvB6 i8EDk8ffi7mHrNuzzRUTR0CPmX+1yNqDTM++ust13OWRWGQNylpOHBo4JAK48qpUa3Bv onFISjCelNCWwisb4x5z+IJMsv8Yc4sxBjYhNZd44/ks2Uda1J7UH5bG7Q+Dyh4lVYW5 jdab8id6mPXAFokSi7YjpEuOvIxS4tBB1mOxviQiKxTGxss1ANGt4wbHbv6UkMhNiCGW SDbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542599; x=1706147399; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uGt/aSa+2dNBp0tw2j5P8nfr1B8a4C0rQJteFRrhxiI=; b=ELLaDNCZVgjei023b4ykPGapbVQyrDtq0hNTQg96ihUh0AMO0ym/Qc5P2lJF8+sBeY AU0Ag/fDmF8TfD8GH6byYWe8AFOyIFjjhUK7BOM766FFHEeY637DcqpZlOlNjWhgMEc1 TdPzdtXuYusGnVo2R4Q+NnvDzb87bVG67N5GDiSQUZoFLLKyzqIIn5Vp4PNtgMjzClvx yGxl/AqRoMvpIvnDtkwrEqZT/JVCn5gS+B6ar+1wVtVHCeba7eSh/vUGce41+1fSQjnA 5AV2YUJqz7i2/d+WSZBImzoez512Pk0Y+aZj+daYJUSQqXYJMnfyKqJcE4iOyQHNtd52 Kllw== X-Gm-Message-State: AOJu0YwtBwRKRhSVS4v2AOzbfFwrun6cdjo/4YoAb5sv6fQKH4CAjUQI w6ruZCn6WTlelTN7aN5Q0FuXDfOPqq7u99So2TftOD8PqkcsyMRJ0GfkMjro X-Google-Smtp-Source: AGHT+IFAX6XuoEvKDNOe2nfWxMiQkyiasbiYbPbXTW3yhw49aPH5AhIxEkOmmYmlI0iFLUotL7Rdmw== X-Received: by 2002:a81:4891:0:b0:5ff:830e:70b with SMTP id v139-20020a814891000000b005ff830e070bmr127842ywa.33.1705542599680; Wed, 17 Jan 2024 17:49:59 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:49:59 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 09/14] bpf: hold module refcnt in bpf_struct_ops map creation and prog verification. Date: Wed, 17 Jan 2024 17:49:25 -0800 Message-Id: <20240118014930.1992551-10-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee To ensure that a module remains accessible whenever a struct_ops object of a struct_ops type provided by the module is still in use. struct bpf_struct_ops_map doesn't hold a refcnt to btf anymore since a module will hold a refcnt to it's btf already. But, struct_ops programs are different. They hold their associated btf, not the module since they need only btf to assure their types (signatures). However, verifier holds the refcnt of the associated module of a struct_ops type temporarily when verify a struct_ops prog. Verifier needs the help from the verifier operators (struct bpf_verifier_ops) provided by the owner module to verify data access of a prog, provide information, and generate code. This patch also add a count of links (links_cnt) to bpf_struct_ops_map. It avoids bpf_struct_ops_map_put_progs() from accessing btf after calling module_put() in bpf_struct_ops_map_free(). Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 1 + include/linux/bpf_verifier.h | 1 + kernel/bpf/bpf_struct_ops.c | 31 +++++++++++++++++++++++++------ kernel/bpf/verifier.c | 10 ++++++++++ 4 files changed, 37 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3d1c1014fdb2..a977ed75288c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1674,6 +1674,7 @@ struct bpf_struct_ops { int (*update)(void *kdata, void *old_kdata); int (*validate)(void *kdata); void *cfi_stubs; + struct module *owner; const char *name; struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS]; }; diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index d07d857ca67f..e6cf025c9446 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -662,6 +662,7 @@ struct bpf_verifier_env { u32 prev_insn_idx; struct bpf_prog *prog; /* eBPF program being verified */ const struct bpf_verifier_ops *ops; + struct module *attach_btf_mod; /* The owner module of prog->aux->attach_btf */ struct bpf_verifier_stack_elem *head; /* stack of verifier states to be processed */ int stack_size; /* number of states to be processed */ bool strict_alignment; /* perform strict pointer alignment checks */ diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 3b8d689ece5d..61486f6595ea 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -40,6 +40,7 @@ struct bpf_struct_ops_map { * (in kvalue.data). */ struct bpf_link **links; + u32 links_cnt; /* image is a page that has all the trampolines * that stores the func args before calling the bpf_prog. * A PAGE_SIZE "image" is enough to store all trampoline for @@ -306,10 +307,9 @@ static void *bpf_struct_ops_map_lookup_elem(struct bpf_map *map, void *key) static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map) { - const struct btf_type *t = st_map->st_ops_desc->type; u32 i; - for (i = 0; i < btf_type_vlen(t); i++) { + for (i = 0; i < st_map->links_cnt; i++) { if (st_map->links[i]) { bpf_link_put(st_map->links[i]); st_map->links[i] = NULL; @@ -641,12 +641,20 @@ static void __bpf_struct_ops_map_free(struct bpf_map *map) bpf_jit_uncharge_modmem(PAGE_SIZE); } bpf_map_area_free(st_map->uvalue); - btf_put(st_map->btf); bpf_map_area_free(st_map); } static void bpf_struct_ops_map_free(struct bpf_map *map) { + struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map; + + /* st_ops->owner was acquired during map_alloc to implicitly holds + * the btf's refcnt. The acquire was only done when btf_is_module() + * st_map->btf cannot be NULL here. + */ + if (btf_is_module(st_map->btf)) + module_put(st_map->st_ops_desc->st_ops->owner); + /* The struct_ops's function may switch to another struct_ops. * * For example, bpf_tcp_cc_x->init() may switch to @@ -682,6 +690,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) size_t st_map_size; struct bpf_struct_ops_map *st_map; const struct btf_type *t, *vt; + struct module *mod = NULL; struct bpf_map *map; struct btf *btf; int ret; @@ -695,11 +704,20 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) btf_put(btf); return ERR_PTR(-EINVAL); } + + mod = btf_try_get_module(btf); + if (!mod) { + btf_put(btf); + return ERR_PTR(-EINVAL); + } + /* mod holds a refcnt to btf. We don't need an extra refcnt + * here. + */ + btf_put(btf); } else { btf = bpf_get_btf_vmlinux(); if (IS_ERR(btf)) return ERR_CAST(btf); - btf_get(btf); } st_ops_desc = bpf_struct_ops_find_value(btf, attr->btf_vmlinux_value_type_id); @@ -746,8 +764,9 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) goto errout_free; } st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); + st_map->links_cnt = btf_type_vlen(t); st_map->links = - bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *), + bpf_map_area_alloc(st_map->links_cnt * sizeof(struct bpf_links *), NUMA_NO_NODE); if (!st_map->uvalue || !st_map->links) { ret = -ENOMEM; @@ -763,7 +782,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) errout_free: __bpf_struct_ops_map_free(map); errout: - btf_put(btf); + module_put(mod); return ERR_PTR(ret); } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index ff41f7736618..60f08f468399 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -20243,6 +20243,14 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) } btf = prog->aux->attach_btf ?: bpf_get_btf_vmlinux(); + if (btf_is_module(btf)) { + /* Make sure st_ops is valid through the lifetime of env */ + env->attach_btf_mod = btf_try_get_module(btf); + if (!env->attach_btf_mod) { + verbose(env, "owner module of btf is not found\n"); + return -ENOTSUPP; + } + } btf_id = prog->aux->attach_btf_id; st_ops_desc = bpf_struct_ops_find(btf, btf_id); @@ -20968,6 +20976,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3 env->prog->expected_attach_type = 0; *prog = env->prog; + + module_put(env->attach_btf_mod); err_unlock: if (!is_priv) mutex_unlock(&bpf_verifier_lock); From patchwork Thu Jan 18 01:49:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522299 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A0528F63 for ; Thu, 18 Jan 2024 01:50:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542604; cv=none; b=T46AQgMrWE+Fyao6OvOPpAKy2LhB1oM/b7zsj2rDTCJi3ncU+K7R9FEtKsraQ9ryKPHkuFrO6fnqOjyoOqifLKncemu1zARhgSLrGalUoCNKRdb25pXP5AdIkQp76lF+NTzfsYvrFJGZNwSdg2ns3yJ0IhXnqs2trV2TxItplI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542604; c=relaxed/simple; bh=YkXgjQhs99cvbtD+yNvzYQVEedtPu49AtL14kMNPrr8=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=EvRHIzyxUcuLjpPG3VnIiXvvOG8bkQjOVUA2Ud2ye/Sr2+Z26/A3NM2l1v+1IHeBtbkUS0237Jg7fJGmjTGg5LPN4XI/Q6CkuXzP6F0Jc7giNlUMxmT4RDy2C6kJPEChxUrKm2MHIbQAChyghEnE44Ti+mjPSjGn9Jy6+qQfehg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=buTdl2xH; arc=none smtp.client-ip=209.85.128.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="buTdl2xH" Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-5ff45c65e60so27821027b3.0 for ; Wed, 17 Jan 2024 17:50:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542601; x=1706147401; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cWE4VyuLc5E19tVmnfM5joUqQyzcZVY0l3eaNkNzmcY=; b=buTdl2xHqYzTS5WOdgf2XJ+Rk7F7MYkt13jkhLE0qgKSWOgz+HHVZG22l6MK5TRc4c d8U0ciUHyHeXzgyaqwH5czSqKn8JZgWBvE2MGnGsjRk/wj+cxVMgVC9cuA0llIRVYFFf 0QvMckWxEO5binB2sCyE2xEuiiKX8YA/+8Jwcs2t9xa6xbgRXNikX32XDe62BFBXAMNh tjRi90YQoYB6hjgHnv1QAsv+xkE0ra/AWuebNRhsp0DY8wvogqD5ssJF7Vyx3IKGVzCs AR5RXsTIzSLTSeEQQW6CyDoVw8lAxd9wJY2EJu/Q6g7LokZSscXO3hE5mc3iavusQ3Ic aTuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542601; x=1706147401; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cWE4VyuLc5E19tVmnfM5joUqQyzcZVY0l3eaNkNzmcY=; b=F8n5l1ezJVf5FRq7IAi6hkE5wB+RsbIwgSr7hICxjkjlFxJu1qRB6bv2q2xTaY/5sC BQEb0XNqhNfKuAf0/O6bzVTepsRD9/lTgTvBeurn7ajDOSU+7npRU5Wc0nHbVAYuD3Zb FZB0jXrYkIvPnghu1BtqgFHI0kHsmrt42mTW7rF9GbGfUxpZR0kBYVZ4spX4k7CH6FOm 1b5kBmpblY1yeDByTLY4bfYmDAl/rny16/+LmRxzRNiFLE9pMihn8Xdg7WBkkX09XHXQ ZIve/Lx4emwVyhHwZNULwPGCpePP6kTrcUUHaesBl980hmYuMXV/QviikdyP4w6JzXwF Qcmw== X-Gm-Message-State: AOJu0YyAM1ylk1zm0yZTpEJIAftbsV+2s9deOtVbyxDhR7x5MuZY2BrT Jn9S3zADdKS9413HAA2+wquj8PBQkEvf+Wqo1+EUBlNL69AAk4KipYFE8ieG X-Google-Smtp-Source: AGHT+IH4zGlG7TwVkWuzvDmIC27HfD+hYUcA2iUYtIuImeA/QhiJtHX/UmRt+D9TSoTmTuApp976Aw== X-Received: by 2002:a0d:e644:0:b0:5f8:420e:468b with SMTP id p65-20020a0de644000000b005f8420e468bmr89685ywe.45.1705542601186; Wed, 17 Jan 2024 17:50:01 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.49.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:50:00 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 10/14] bpf: validate value_type Date: Wed, 17 Jan 2024 17:49:26 -0800 Message-Id: <20240118014930.1992551-11-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee A value_type should consist of three components: refcnt, state, and data. refcnt and state has been move to struct bpf_struct_ops_common_value to make it easier to check the value type. Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 12 +++++ kernel/bpf/bpf_struct_ops.c | 93 ++++++++++++++++++++++++------------- 2 files changed, 72 insertions(+), 33 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index a977ed75288c..1cfbb89944c5 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1688,6 +1688,18 @@ struct bpf_struct_ops_desc { u32 value_id; }; +enum bpf_struct_ops_state { + BPF_STRUCT_OPS_STATE_INIT, + BPF_STRUCT_OPS_STATE_INUSE, + BPF_STRUCT_OPS_STATE_TOBEFREE, + BPF_STRUCT_OPS_STATE_READY, +}; + +struct bpf_struct_ops_common_value { + refcount_t refcnt; + enum bpf_struct_ops_state state; +}; + #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL) #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA)) const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 61486f6595ea..f5763f240722 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -13,19 +13,8 @@ #include #include -enum bpf_struct_ops_state { - BPF_STRUCT_OPS_STATE_INIT, - BPF_STRUCT_OPS_STATE_INUSE, - BPF_STRUCT_OPS_STATE_TOBEFREE, - BPF_STRUCT_OPS_STATE_READY, -}; - -#define BPF_STRUCT_OPS_COMMON_VALUE \ - refcount_t refcnt; \ - enum bpf_struct_ops_state state - struct bpf_struct_ops_value { - BPF_STRUCT_OPS_COMMON_VALUE; + struct bpf_struct_ops_common_value common; char data[] ____cacheline_aligned_in_smp; }; @@ -81,8 +70,8 @@ static DEFINE_MUTEX(update_mutex); #define BPF_STRUCT_OPS_TYPE(_name) \ extern struct bpf_struct_ops bpf_##_name; \ \ -struct bpf_struct_ops_##_name { \ - BPF_STRUCT_OPS_COMMON_VALUE; \ +struct bpf_struct_ops_##_name { \ + struct bpf_struct_ops_common_value common; \ struct _name data ____cacheline_aligned_in_smp; \ }; #include "bpf_struct_ops_types.h" @@ -113,11 +102,49 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = { BTF_ID_LIST(st_ops_ids) BTF_ID(struct, module) +BTF_ID(struct, bpf_struct_ops_common_value) enum { IDX_MODULE_ID, + IDX_ST_OPS_COMMON_VALUE_ID, }; +extern struct btf *btf_vmlinux; + +static bool is_valid_value_type(struct btf *btf, s32 value_id, + const struct btf_type *type, + const char *value_name) +{ + const struct btf_type *common_value_type; + const struct btf_member *member; + const struct btf_type *vt, *mt; + + vt = btf_type_by_id(btf, value_id); + if (btf_vlen(vt) != 2) { + pr_warn("The number of %s's members should be 2, but we get %d\n", + value_name, btf_vlen(vt)); + return false; + } + member = btf_type_member(vt); + mt = btf_type_by_id(btf, member->type); + common_value_type = btf_type_by_id(btf_vmlinux, + st_ops_ids[IDX_ST_OPS_COMMON_VALUE_ID]); + if (mt != common_value_type) { + pr_warn("The first member of %s should be bpf_struct_ops_common_value\n", + value_name); + return false; + } + member++; + mt = btf_type_by_id(btf, member->type); + if (mt != type) { + pr_warn("The second member of %s should be %s\n", + value_name, btf_name_by_offset(btf, type->name_off)); + return false; + } + + return true; +} + static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, struct btf *btf, struct bpf_verifier_log *log) @@ -138,14 +165,6 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, } sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name); - value_id = btf_find_by_name_kind(btf, value_name, - BTF_KIND_STRUCT); - if (value_id < 0) { - pr_warn("Cannot find struct %s in %s\n", - value_name, btf_get_name(btf)); - return; - } - type_id = btf_find_by_name_kind(btf, st_ops->name, BTF_KIND_STRUCT); if (type_id < 0) { @@ -160,6 +179,16 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, return; } + value_id = btf_find_by_name_kind(btf, value_name, + BTF_KIND_STRUCT); + if (value_id < 0) { + pr_warn("Cannot find struct %s in %s\n", + value_name, btf_get_name(btf)); + return; + } + if (!is_valid_value_type(btf, value_id, t, value_name)) + return; + for_each_member(i, t, member) { const struct btf_type *func_proto; @@ -219,8 +248,6 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) } } -extern struct btf *btf_vmlinux; - static const struct bpf_struct_ops_desc * bpf_struct_ops_find_value(struct btf *btf, u32 value_id) { @@ -276,7 +303,7 @@ int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, kvalue = &st_map->kvalue; /* Pair with smp_store_release() during map_update */ - state = smp_load_acquire(&kvalue->state); + state = smp_load_acquire(&kvalue->common.state); if (state == BPF_STRUCT_OPS_STATE_INIT) { memset(value, 0, map->value_size); return 0; @@ -287,7 +314,7 @@ int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, */ uvalue = value; memcpy(uvalue, st_map->uvalue, map->value_size); - uvalue->state = state; + uvalue->common.state = state; /* This value offers the user space a general estimate of how * many sockets are still utilizing this struct_ops for TCP @@ -295,7 +322,7 @@ int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, * should sufficiently meet our present goals. */ refcnt = atomic64_read(&map->refcnt) - atomic64_read(&map->usercnt); - refcount_set(&uvalue->refcnt, max_t(s64, refcnt, 0)); + refcount_set(&uvalue->common.refcnt, max_t(s64, refcnt, 0)); return 0; } @@ -413,7 +440,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, if (err) return err; - if (uvalue->state || refcount_read(&uvalue->refcnt)) + if (uvalue->common.state || refcount_read(&uvalue->common.refcnt)) return -EINVAL; tlinks = kcalloc(BPF_TRAMP_MAX, sizeof(*tlinks), GFP_KERNEL); @@ -425,7 +452,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, mutex_lock(&st_map->lock); - if (kvalue->state != BPF_STRUCT_OPS_STATE_INIT) { + if (kvalue->common.state != BPF_STRUCT_OPS_STATE_INIT) { err = -EBUSY; goto unlock; } @@ -540,7 +567,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, * * Pair with smp_load_acquire() during lookup_elem(). */ - smp_store_release(&kvalue->state, BPF_STRUCT_OPS_STATE_READY); + smp_store_release(&kvalue->common.state, BPF_STRUCT_OPS_STATE_READY); goto unlock; } @@ -558,7 +585,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, * It ensures the above udata updates (e.g. prog->aux->id) * can be seen once BPF_STRUCT_OPS_STATE_INUSE is set. */ - smp_store_release(&kvalue->state, BPF_STRUCT_OPS_STATE_INUSE); + smp_store_release(&kvalue->common.state, BPF_STRUCT_OPS_STATE_INUSE); goto unlock; } @@ -588,7 +615,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key) if (st_map->map.map_flags & BPF_F_LINK) return -EOPNOTSUPP; - prev_state = cmpxchg(&st_map->kvalue.state, + prev_state = cmpxchg(&st_map->kvalue.common.state, BPF_STRUCT_OPS_STATE_INUSE, BPF_STRUCT_OPS_STATE_TOBEFREE); switch (prev_state) { @@ -850,7 +877,7 @@ static bool bpf_struct_ops_valid_to_reg(struct bpf_map *map) return map->map_type == BPF_MAP_TYPE_STRUCT_OPS && map->map_flags & BPF_F_LINK && /* Pair with smp_store_release() during map_update */ - smp_load_acquire(&st_map->kvalue.state) == BPF_STRUCT_OPS_STATE_READY; + smp_load_acquire(&st_map->kvalue.common.state) == BPF_STRUCT_OPS_STATE_READY; } static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link) From patchwork Thu Jan 18 01:49:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522300 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDCA39447; Thu, 18 Jan 2024 01:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542605; cv=none; b=b53nEEfs1vGoJO2zozyufhTnqc6kSyCLhL1dIVQMSCtzJElMwSKMCyQiddClPkdvgnjYeZgMVRSrdYlF7lJjhQLx2wnyHRsPJ8MWD1dThoEK+lZUuPU9TO8VcQLIGBRQoU3rEvt+yRAj6BlbQ1s5Q4BZ8RMA1qDCfvZc0i0C938= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542605; c=relaxed/simple; bh=QuC/8z2TBNlN0bcUFbBbq/XlNkBlGgUhTW/rQpTz3Ig=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=lspxjiumRwH2XQ6Tenxjtr/lwDDkb+Q/w4va0N0NGyvaEqycqKEQ+u462zeBefAyrzzyFixmRrUHHdtYVfEqvg0+TgyVgtT1ObsE8VY3iArLkEus3TccnkcWVFa0pgRUzR90mx2e5jQ9BR3yOUVgWjoVS6Cx1xA0AFIRpi4BHuA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LJNTw322; arc=none smtp.client-ip=209.85.128.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LJNTw322" Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-5f75aee31d2so106259307b3.2; Wed, 17 Jan 2024 17:50:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542602; x=1706147402; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ub8NhHaYdoCeOA+v/XQVWByRmt9C4vKs1WE/5o0pmpQ=; b=LJNTw322XA02cuKKQXIOrDM6eaknqPj8iP4rQUXdf1/cPmjOHYfj4LJtfhPraOyRY3 lxMNj5nGmhJwACv71RHeCiylhExaJH0seK6DTL/dEHCVcVA5gFnZdmYeDgeRFHtpWXaG esbKpYDhequul6pE4JPGgfcJ9gGZbHe/QhDU94hhr6ZZQj6GZk7KQSIjgOwfB+s0o3Hc 6oq0W15eMQTb4/nnTy/2XRqWHhBYqQhS1D/TaYpATiuQwxaGvB5QTufKdZm0X+PYxjnB POd2UoRWTDMwdOoRtKDxiq1yGDMrkQl+R9lhb9IyqLxO2RHAnQBhN/McA9trFYBQZqjS fdoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542602; x=1706147402; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ub8NhHaYdoCeOA+v/XQVWByRmt9C4vKs1WE/5o0pmpQ=; b=lNH62ZAeX8LhC/3LyEmMloLPOQ8dC3NBp6Dne0K80UomVXtN03L7T7jwigCfO9ESjr JOSb4OjFRrnRHzfDg5V/8NC4TDk8DlxPUcwHrwQLeOex5vila/Iw9H6R6soRGdW3LLCF JRWnyvyk4RLUC/5hV+9w61lai5Iq3Rl+CyEs7v4nztqdmDdL3rHN7lqqH4N+hlzQBZmt BAR+b+9kXda76RDl49dLD/cZkCnu/IrAKt3rMkCa7Npk3STwrhr3eQ8aR3a7/mjf0tA4 /DdXM6stLUR1Rdps2Blr/ph/pORbIvjWlWPqCQjuZSvtWjKqRAzkF8v3FoMVR7FndHf7 iW2w== X-Gm-Message-State: AOJu0Yzv2ZbXoOSHwo1JYlPoA3MlHUncb6/XvpuO67mSjiYYuDPl2OeT 1MepTk8FzGCYecKNOhIgJeIRLhQDLSZHj6+C9fyBZSKVeH899o2NFLuA51Oi X-Google-Smtp-Source: AGHT+IGBCyrKErZNSvysKjdlfYNvdXG6QHqplzQwRbuMTLWZVgFZkH6frrUn0JraRWJBhOITtB/oVw== X-Received: by 2002:a0d:d94e:0:b0:5f6:e673:3f22 with SMTP id b75-20020a0dd94e000000b005f6e6733f22mr133688ywe.37.1705542602526; Wed, 17 Jan 2024 17:50:02 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.50.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:50:02 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee , netdev@vger.kernel.org Subject: [PATCH bpf-next v16 11/14] bpf, net: switch to dynamic registration Date: Wed, 17 Jan 2024 17:49:27 -0800 Message-Id: <20240118014930.1992551-12-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Replace the static list of struct_ops types with per-btf struct_ops_tab to enable dynamic registration. Both bpf_dummy_ops and bpf_tcp_ca now utilize the registration function instead of being listed in bpf_struct_ops_types.h. Cc: netdev@vger.kernel.org Signed-off-by: Kui-Feng Lee --- include/linux/bpf.h | 37 ++++++++--- include/linux/btf.h | 12 ++++ kernel/bpf/bpf_struct_ops.c | 100 ++++-------------------------- kernel/bpf/bpf_struct_ops_types.h | 12 ---- kernel/bpf/btf.c | 84 +++++++++++++++++++++++-- net/bpf/bpf_dummy_struct_ops.c | 11 +++- net/ipv4/bpf_tcp_ca.c | 12 +++- 7 files changed, 150 insertions(+), 118 deletions(-) delete mode 100644 kernel/bpf/bpf_struct_ops_types.h diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 1cfbb89944c5..a2522fcfe57c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1700,10 +1700,22 @@ struct bpf_struct_ops_common_value { enum bpf_struct_ops_state state; }; +/* This macro helps developer to register a struct_ops type and generate + * type information correctly. Developers should use this macro to register + * a struct_ops type instead of calling register_bpf_struct_ops() directly. + */ +#define REGISTER_BPF_STRUCT_OPS(st_ops, type) \ + ({ \ + struct bpf_struct_ops_##type { \ + struct bpf_struct_ops_common_value common; \ + struct type data ____cacheline_aligned_in_smp; \ + }; \ + BTF_TYPE_EMIT(struct bpf_struct_ops_##type); \ + register_bpf_struct_ops(st_ops); \ + }) + #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL) #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA)) -const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id); -void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log); bool bpf_struct_ops_get(const void *kdata); void bpf_struct_ops_put(const void *kdata); int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key, @@ -1745,16 +1757,11 @@ struct bpf_dummy_ops { int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); #endif +int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, + struct btf *btf, + struct bpf_verifier_log *log); void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map); #else -static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id) -{ - return NULL; -} -static inline void bpf_struct_ops_init(struct btf *btf, - struct bpf_verifier_log *log) -{ -} static inline bool bpf_try_module_get(const void *data, struct module *owner) { return try_module_get(owner); @@ -1769,6 +1776,10 @@ static inline int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, { return -EINVAL; } +static inline int bpf_struct_ops_link_create(union bpf_attr *attr) +{ + return -EOPNOTSUPP; +} #endif #if defined(CONFIG_CGROUP_BPF) && defined(CONFIG_BPF_LSM) @@ -1784,6 +1795,12 @@ static inline int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog, static inline void bpf_trampoline_unlink_cgroup_shim(struct bpf_prog *prog) { } +static inline int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, + struct btf *btf, + struct bpf_verifier_log *log) { + return -EOPNOTSUPP; +} + #endif struct bpf_array { diff --git a/include/linux/btf.h b/include/linux/btf.h index a68604904f4e..bc3576b067f6 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -496,6 +496,7 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) } struct bpf_verifier_log; +struct bpf_struct_ops; #ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); @@ -520,6 +521,9 @@ btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_type); bool btf_types_are_same(const struct btf *btf1, u32 id1, const struct btf *btf2, u32 id2); +int register_bpf_struct_ops(struct bpf_struct_ops *st_ops); +const struct bpf_struct_ops_desc *bpf_struct_ops_find_value(struct btf *btf, u32 value_id); +const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -572,6 +576,14 @@ static inline bool btf_types_are_same(const struct btf *btf1, u32 id1, { return false; } +static inline int register_bpf_struct_ops(struct bpf_struct_ops *st_ops) +{ + return -ENOTSUPP; +} +static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id) +{ + return NULL; +} #endif static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index f5763f240722..1ebd7e3acb67 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -62,35 +62,6 @@ static DEFINE_MUTEX(update_mutex); #define VALUE_PREFIX "bpf_struct_ops_" #define VALUE_PREFIX_LEN (sizeof(VALUE_PREFIX) - 1) -/* bpf_struct_ops_##_name (e.g. bpf_struct_ops_tcp_congestion_ops) is - * the map's value exposed to the userspace and its btf-type-id is - * stored at the map->btf_vmlinux_value_type_id. - * - */ -#define BPF_STRUCT_OPS_TYPE(_name) \ -extern struct bpf_struct_ops bpf_##_name; \ - \ -struct bpf_struct_ops_##_name { \ - struct bpf_struct_ops_common_value common; \ - struct _name data ____cacheline_aligned_in_smp; \ -}; -#include "bpf_struct_ops_types.h" -#undef BPF_STRUCT_OPS_TYPE - -enum { -#define BPF_STRUCT_OPS_TYPE(_name) BPF_STRUCT_OPS_TYPE_##_name, -#include "bpf_struct_ops_types.h" -#undef BPF_STRUCT_OPS_TYPE - __NR_BPF_STRUCT_OPS_TYPE, -}; - -static struct bpf_struct_ops_desc bpf_struct_ops[] = { -#define BPF_STRUCT_OPS_TYPE(_name) \ - [BPF_STRUCT_OPS_TYPE_##_name] = { .st_ops = &bpf_##_name }, -#include "bpf_struct_ops_types.h" -#undef BPF_STRUCT_OPS_TYPE -}; - const struct bpf_verifier_ops bpf_struct_ops_verifier_ops = { }; @@ -145,9 +116,9 @@ static bool is_valid_value_type(struct btf *btf, s32 value_id, return true; } -static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, - struct btf *btf, - struct bpf_verifier_log *log) +int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, + struct btf *btf, + struct bpf_verifier_log *log) { struct bpf_struct_ops *st_ops = st_ops_desc->st_ops; const struct btf_member *member; @@ -161,7 +132,7 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, sizeof(value_name)) { pr_warn("struct_ops name %s is too long\n", st_ops->name); - return; + return -EINVAL; } sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name); @@ -170,13 +141,13 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (type_id < 0) { pr_warn("Cannot find struct %s in %s\n", st_ops->name, btf_get_name(btf)); - return; + return -EINVAL; } t = btf_type_by_id(btf, type_id); if (btf_type_vlen(t) > BPF_STRUCT_OPS_MAX_NR_MEMBERS) { pr_warn("Cannot support #%u members in struct %s\n", btf_type_vlen(t), st_ops->name); - return; + return -EINVAL; } value_id = btf_find_by_name_kind(btf, value_name, @@ -184,10 +155,10 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (value_id < 0) { pr_warn("Cannot find struct %s in %s\n", value_name, btf_get_name(btf)); - return; + return -EINVAL; } if (!is_valid_value_type(btf, value_id, t, value_name)) - return; + return -EINVAL; for_each_member(i, t, member) { const struct btf_type *func_proto; @@ -196,13 +167,13 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (!*mname) { pr_warn("anon member in struct %s is not supported\n", st_ops->name); - break; + return -EOPNOTSUPP; } if (__btf_member_bitfield_size(t, member)) { pr_warn("bit field member %s in struct %s is not supported\n", mname, st_ops->name); - break; + return -EOPNOTSUPP; } func_proto = btf_type_resolve_func_ptr(btf, @@ -214,7 +185,7 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, &st_ops->func_models[i])) { pr_warn("Error in parsing func ptr %s in struct %s\n", mname, st_ops->name); - break; + return -EINVAL; } } @@ -222,6 +193,7 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (st_ops->init(btf)) { pr_warn("Error in init bpf_struct_ops %s\n", st_ops->name); + return -EINVAL; } else { st_ops_desc->type_id = type_id; st_ops_desc->type = t; @@ -230,54 +202,8 @@ static void bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, value_id); } } -} -void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log) -{ - struct bpf_struct_ops_desc *st_ops_desc; - u32 i; - - /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */ -#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name); -#include "bpf_struct_ops_types.h" -#undef BPF_STRUCT_OPS_TYPE - - for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - st_ops_desc = &bpf_struct_ops[i]; - bpf_struct_ops_desc_init(st_ops_desc, btf, log); - } -} - -static const struct bpf_struct_ops_desc * -bpf_struct_ops_find_value(struct btf *btf, u32 value_id) -{ - unsigned int i; - - if (!value_id || !btf) - return NULL; - - for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - if (bpf_struct_ops[i].value_id == value_id) - return &bpf_struct_ops[i]; - } - - return NULL; -} - -const struct bpf_struct_ops_desc * -bpf_struct_ops_find(struct btf *btf, u32 type_id) -{ - unsigned int i; - - if (!type_id || !btf) - return NULL; - - for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) { - if (bpf_struct_ops[i].type_id == type_id) - return &bpf_struct_ops[i]; - } - - return NULL; + return 0; } static int bpf_struct_ops_map_get_next_key(struct bpf_map *map, void *key, diff --git a/kernel/bpf/bpf_struct_ops_types.h b/kernel/bpf/bpf_struct_ops_types.h deleted file mode 100644 index 5678a9ddf817..000000000000 --- a/kernel/bpf/bpf_struct_ops_types.h +++ /dev/null @@ -1,12 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* internal file - do not include directly */ - -#ifdef CONFIG_BPF_JIT -#ifdef CONFIG_NET -BPF_STRUCT_OPS_TYPE(bpf_dummy_ops) -#endif -#ifdef CONFIG_INET -#include -BPF_STRUCT_OPS_TYPE(tcp_congestion_ops) -#endif -#endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 3fa84c44b882..860b2946de3c 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -5792,8 +5793,6 @@ struct btf *btf_parse_vmlinux(void) /* btf_parse_vmlinux() runs under bpf_verifier_lock */ bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); - bpf_struct_ops_init(btf, log); - refcount_set(&btf->refcnt, 1); err = btf_alloc_id(btf); @@ -8489,10 +8488,11 @@ bool btf_type_ids_nocast_alias(struct bpf_verifier_log *log, } static int -btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops) +btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops, + struct bpf_verifier_log *log) { struct btf_struct_ops_tab *tab, *new_tab; - int i; + int i, err; if (!btf) return -ENOENT; @@ -8529,7 +8529,83 @@ btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops) tab->ops[btf->struct_ops_tab->cnt].st_ops = st_ops; + err = bpf_struct_ops_desc_init(&tab->ops[btf->struct_ops_tab->cnt], btf, log); + if (err) + return err; + btf->struct_ops_tab->cnt++; return 0; } + +const struct bpf_struct_ops_desc * +bpf_struct_ops_find_value(struct btf *btf, u32 value_id) +{ + const struct bpf_struct_ops_desc *st_ops_list; + unsigned int i; + u32 cnt; + + if (!value_id) + return NULL; + if (!btf->struct_ops_tab) + return NULL; + + cnt = btf->struct_ops_tab->cnt; + st_ops_list = btf->struct_ops_tab->ops; + for (i = 0; i < cnt; i++) { + if (st_ops_list[i].value_id == value_id) + return &st_ops_list[i]; + } + + return NULL; +} + +const struct bpf_struct_ops_desc * +bpf_struct_ops_find(struct btf *btf, u32 type_id) +{ + const struct bpf_struct_ops_desc *st_ops_list; + unsigned int i; + u32 cnt; + + if (!type_id) + return NULL; + if (!btf->struct_ops_tab) + return NULL; + + cnt = btf->struct_ops_tab->cnt; + st_ops_list = btf->struct_ops_tab->ops; + for (i = 0; i < cnt; i++) { + if (st_ops_list[i].type_id == type_id) + return &st_ops_list[i]; + } + + return NULL; +} + +int register_bpf_struct_ops(struct bpf_struct_ops *st_ops) +{ + struct bpf_verifier_log *log; + struct btf *btf; + int err = 0; + + btf = btf_get_module_btf(st_ops->owner); + if (!btf) + return -EINVAL; + + log = kzalloc(sizeof(*log), GFP_KERNEL | __GFP_NOWARN); + if (!log) { + err = -ENOMEM; + goto errout; + } + + log->level = BPF_LOG_KERNEL; + + err = btf_add_struct_ops(btf, st_ops, log); + +errout: + kfree(log); + btf_put(btf); + + return err; +} +EXPORT_SYMBOL_GPL(register_bpf_struct_ops); diff --git a/net/bpf/bpf_dummy_struct_ops.c b/net/bpf/bpf_dummy_struct_ops.c index ba2c58dba2da..42034c54a70f 100644 --- a/net/bpf/bpf_dummy_struct_ops.c +++ b/net/bpf/bpf_dummy_struct_ops.c @@ -7,7 +7,7 @@ #include #include -extern struct bpf_struct_ops bpf_bpf_dummy_ops; +static struct bpf_struct_ops bpf_bpf_dummy_ops; /* A common type for test_N with return value in bpf_dummy_ops */ typedef int (*dummy_ops_test_ret_fn)(struct bpf_dummy_ops_state *state, ...); @@ -256,7 +256,7 @@ static struct bpf_dummy_ops __bpf_bpf_dummy_ops = { .test_sleepable = bpf_dummy_test_sleepable, }; -struct bpf_struct_ops bpf_bpf_dummy_ops = { +static struct bpf_struct_ops bpf_bpf_dummy_ops = { .verifier_ops = &bpf_dummy_verifier_ops, .init = bpf_dummy_init, .check_member = bpf_dummy_ops_check_member, @@ -265,4 +265,11 @@ struct bpf_struct_ops bpf_bpf_dummy_ops = { .unreg = bpf_dummy_unreg, .name = "bpf_dummy_ops", .cfi_stubs = &__bpf_bpf_dummy_ops, + .owner = THIS_MODULE, }; + +static int __init bpf_dummy_struct_ops_init(void) +{ + return REGISTER_BPF_STRUCT_OPS(&bpf_bpf_dummy_ops, bpf_dummy_ops); +} +late_initcall(bpf_dummy_struct_ops_init); diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index dffd8828079b..7e053b9faca3 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -12,7 +12,7 @@ #include /* "extern" is to avoid sparse warning. It is only used in bpf_struct_ops.c. */ -extern struct bpf_struct_ops bpf_tcp_congestion_ops; +static struct bpf_struct_ops bpf_tcp_congestion_ops; static u32 unsupported_ops[] = { offsetof(struct tcp_congestion_ops, get_info), @@ -345,7 +345,7 @@ static struct tcp_congestion_ops __bpf_ops_tcp_congestion_ops = { .release = __bpf_tcp_ca_release, }; -struct bpf_struct_ops bpf_tcp_congestion_ops = { +static struct bpf_struct_ops bpf_tcp_congestion_ops = { .verifier_ops = &bpf_tcp_ca_verifier_ops, .reg = bpf_tcp_ca_reg, .unreg = bpf_tcp_ca_unreg, @@ -356,10 +356,16 @@ struct bpf_struct_ops bpf_tcp_congestion_ops = { .validate = bpf_tcp_ca_validate, .name = "tcp_congestion_ops", .cfi_stubs = &__bpf_ops_tcp_congestion_ops, + .owner = THIS_MODULE, }; static int __init bpf_tcp_ca_kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set); + ret = ret ?: REGISTER_BPF_STRUCT_OPS(&bpf_tcp_congestion_ops, tcp_congestion_ops); + + return ret; } late_initcall(bpf_tcp_ca_kfunc_init); From patchwork Thu Jan 18 01:49:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522301 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f173.google.com (mail-yw1-f173.google.com [209.85.128.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F1098F67 for ; Thu, 18 Jan 2024 01:50:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542606; cv=none; b=hCuYMjPFjgw/TjGoXVdI+cH0TAiODs6b2oXUTUBhrSIZzb1kiCCDNIOqlhuF2oCqYaJ5IoKjuNNvCK3fHvN85zfAsdqBHejUwPFhikIeyX/cflEHQwWJ+V3xy6ofGNfezjj/aUP4jkg/07Uew7CFznnpmEp/gCRoIYvHz+SlnQo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542606; c=relaxed/simple; bh=ujTTC1QvWPZdRdqKzNzrhwwiLxXuFR9mmb7ab1O/4UU=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=TU1XFT1akascln/jj42VghXYhVblNWkZjC15LoU2JW6vxcq3+m1Pq7IT3/MJcEsv4KvIlp4s558ncriUDV4TAzNBqLGxvNHO+6vlprCSh+AwnuUrbIC53/JrrBrGOsX0fsryfhHv/dQCTgOEK/gdInYlmF6tMvG8EEx7aN56SCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=G7bKqv8T; arc=none smtp.client-ip=209.85.128.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G7bKqv8T" Received: by mail-yw1-f173.google.com with SMTP id 00721157ae682-5ff828b93f0so4431707b3.3 for ; Wed, 17 Jan 2024 17:50:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542604; x=1706147404; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XWorhE91exuesnp0l3cNzkw3ftaMNHgcCmrux9EtVTY=; b=G7bKqv8TL9n9EwSiMOhjMGtMXuhqnj+n9FPIWwobYGvotxJIA2W0/hWWqb0pb/BMyN j0E5smR8n/AniDfG4TzDih0DyGc+aUF2Yd0rgFsxoHhTBtaIM+M5eEy383nYkG2wIBEI DOtO3gsjkiaIjDDlpIn0IqtqavUTs0R6hsgyxcC2pNQf2aSHNtru6rdarAoKO5ukphAq uOqmr4kA5IxC/cXDPMXjSpDHfRAy34UlQMcKTYgImoXWHVPXxBzaJQwWz+mW4RXVoxoT qu624u7Pe75CUkOyJdk3pKal7SOoNMvoUd9PI9diUwn8DkPXpP3/2NP9FWSM9cRZrtrO PeiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542604; x=1706147404; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XWorhE91exuesnp0l3cNzkw3ftaMNHgcCmrux9EtVTY=; b=Gl74ADr8pOgE2heFlKXThNzc0Y4eDHLuASHmO6H2jNbmRb2XZR49BzJLa4B4YJiBmX JNwzX72YzKz8pClTIyBsV9i43RKaaBigOXh6JwpXPtLc5b4OhrmdVEojcLk6WLids2uW YmUZbG2DVrEr4j/AkdU69IRKR+ix5J1hStOt6wkAoIVAvYeDxeRjAx7DLzXBW6lFldw1 Nq1MFRe84AHWsQ0KrpTK1sniajEx/qESDGplJHQKcxp6264lrkGYZwMxKVISkcaXLeBU 9cfeU4JiyzJo0KEerb/SafXQ1OCDo21SLAYrQN+TdxiTPPHJg2OnS3gSqQ4wlCN2oouR nA6Q== X-Gm-Message-State: AOJu0YwTM7AjvblyOEOHn37lzG3G8hderAKlpw7SODHLnd/mzbKof0JD PLcw1seeDFnUIr7Z9H0TM4F/gOBZlMVFXh/1TJIDIR7yNI8z+u1AJ2d09nSH X-Google-Smtp-Source: AGHT+IHkce2+P69wgMbeuH7r9JLLU25qus4ZzDNcEY0xRQD2VYnSHTw82DWuAFs07pIo4VmQJXLlyw== X-Received: by 2002:a0d:c905:0:b0:5ff:638f:441e with SMTP id l5-20020a0dc905000000b005ff638f441emr122540ywd.103.1705542603880; Wed, 17 Jan 2024 17:50:03 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.50.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:50:03 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 12/14] libbpf: Find correct module BTFs for struct_ops maps and progs. Date: Wed, 17 Jan 2024 17:49:28 -0800 Message-Id: <20240118014930.1992551-13-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Locate the module BTFs for struct_ops maps and progs and pass them to the kernel. This ensures that the kernel correctly resolves type IDs from the appropriate module BTFs. For the map of a struct_ops object, the FD of the module BTF is set to bpf_map to keep a reference to the module BTF. The FD is passed to the kernel as value_type_btf_obj_fd when the struct_ops object is loaded. For a bpf_struct_ops prog, attach_btf_obj_fd of bpf_prog is the FD of a module BTF in the kernel. Signed-off-by: Kui-Feng Lee Acked-by: Andrii Nakryiko --- tools/lib/bpf/bpf.c | 4 +++- tools/lib/bpf/bpf.h | 4 +++- tools/lib/bpf/libbpf.c | 41 ++++++++++++++++++++++++++--------- tools/lib/bpf/libbpf_probes.c | 1 + 4 files changed, 38 insertions(+), 12 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 9dc9625651dc..3a35472a17c5 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -169,7 +169,8 @@ int bpf_map_create(enum bpf_map_type map_type, __u32 max_entries, const struct bpf_map_create_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, map_extra); + const size_t attr_sz = offsetofend(union bpf_attr, + value_type_btf_obj_fd); union bpf_attr attr; int fd; @@ -191,6 +192,7 @@ int bpf_map_create(enum bpf_map_type map_type, attr.btf_key_type_id = OPTS_GET(opts, btf_key_type_id, 0); attr.btf_value_type_id = OPTS_GET(opts, btf_value_type_id, 0); attr.btf_vmlinux_value_type_id = OPTS_GET(opts, btf_vmlinux_value_type_id, 0); + attr.value_type_btf_obj_fd = OPTS_GET(opts, value_type_btf_obj_fd, -1); attr.inner_map_fd = OPTS_GET(opts, inner_map_fd, 0); attr.map_flags = OPTS_GET(opts, map_flags, 0); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index d0f53772bdc0..53b0bee053ad 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -51,8 +51,10 @@ struct bpf_map_create_opts { __u32 numa_node; __u32 map_ifindex; + __s32 value_type_btf_obj_fd; + size_t:0; }; -#define bpf_map_create_opts__last_field map_ifindex +#define bpf_map_create_opts__last_field value_type_btf_obj_fd LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index c5a42ac309fd..4ee5e9412ddc 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -527,6 +527,7 @@ struct bpf_map { struct bpf_map_def def; __u32 numa_node; __u32 btf_var_idx; + int mod_btf_fd; __u32 btf_key_type_id; __u32 btf_value_type_id; __u32 btf_vmlinux_value_type_id; @@ -930,22 +931,29 @@ find_member_by_name(const struct btf *btf, const struct btf_type *t, return NULL; } +static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, + __u16 kind, struct btf **res_btf, + struct module_btf **res_mod_btf); + #define STRUCT_OPS_VALUE_PREFIX "bpf_struct_ops_" static int find_btf_by_prefix_kind(const struct btf *btf, const char *prefix, const char *name, __u32 kind); static int -find_struct_ops_kern_types(const struct btf *btf, const char *tname, +find_struct_ops_kern_types(struct bpf_object *obj, const char *tname, + struct module_btf **mod_btf, const struct btf_type **type, __u32 *type_id, const struct btf_type **vtype, __u32 *vtype_id, const struct btf_member **data_member) { const struct btf_type *kern_type, *kern_vtype; const struct btf_member *kern_data_member; + struct btf *btf; __s32 kern_vtype_id, kern_type_id; __u32 i; - kern_type_id = btf__find_by_name_kind(btf, tname, BTF_KIND_STRUCT); + kern_type_id = find_ksym_btf_id(obj, tname, BTF_KIND_STRUCT, + &btf, mod_btf); if (kern_type_id < 0) { pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n", tname); @@ -999,14 +1007,16 @@ static bool bpf_map__is_struct_ops(const struct bpf_map *map) } /* Init the map's fields that depend on kern_btf */ -static int bpf_map__init_kern_struct_ops(struct bpf_map *map, - const struct btf *btf, - const struct btf *kern_btf) +static int bpf_map__init_kern_struct_ops(struct bpf_map *map) { const struct btf_member *member, *kern_member, *kern_data_member; const struct btf_type *type, *kern_type, *kern_vtype; __u32 i, kern_type_id, kern_vtype_id, kern_data_off; + struct bpf_object *obj = map->obj; + const struct btf *btf = obj->btf; struct bpf_struct_ops *st_ops; + const struct btf *kern_btf; + struct module_btf *mod_btf; void *data, *kern_data; const char *tname; int err; @@ -1014,16 +1024,19 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map, st_ops = map->st_ops; type = st_ops->type; tname = st_ops->tname; - err = find_struct_ops_kern_types(kern_btf, tname, + err = find_struct_ops_kern_types(obj, tname, &mod_btf, &kern_type, &kern_type_id, &kern_vtype, &kern_vtype_id, &kern_data_member); if (err) return err; + kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux; + pr_debug("struct_ops init_kern %s: type_id:%u kern_type_id:%u kern_vtype_id:%u\n", map->name, st_ops->type_id, kern_type_id, kern_vtype_id); + map->mod_btf_fd = mod_btf ? mod_btf->fd : -1; map->def.value_size = kern_vtype->size; map->btf_vmlinux_value_type_id = kern_vtype_id; @@ -1099,6 +1112,8 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map, return -ENOTSUP; } + if (mod_btf) + prog->attach_btf_obj_fd = mod_btf->fd; prog->attach_btf_id = kern_type_id; prog->expected_attach_type = kern_member_idx; @@ -1141,8 +1156,7 @@ static int bpf_object__init_kern_struct_ops_maps(struct bpf_object *obj) if (!bpf_map__is_struct_ops(map)) continue; - err = bpf_map__init_kern_struct_ops(map, obj->btf, - obj->btf_vmlinux); + err = bpf_map__init_kern_struct_ops(map); if (err) return err; } @@ -5161,8 +5175,13 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.numa_node = map->numa_node; create_attr.map_extra = map->map_extra; - if (bpf_map__is_struct_ops(map)) + if (bpf_map__is_struct_ops(map)) { create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; + if (map->mod_btf_fd >= 0) { + create_attr.value_type_btf_obj_fd = map->mod_btf_fd; + create_attr.map_flags |= BPF_F_VTYPE_BTF_OBJ_FD; + } + } if (obj->btf && btf__fd(obj->btf) >= 0) { create_attr.btf_fd = btf__fd(obj->btf); @@ -9842,7 +9861,9 @@ static int libbpf_find_attach_btf_id(struct bpf_program *prog, const char *attac *btf_obj_fd = 0; *btf_type_id = 1; } else { - err = find_kernel_btf_id(prog->obj, attach_name, attach_type, btf_obj_fd, btf_type_id); + err = find_kernel_btf_id(prog->obj, attach_name, + attach_type, btf_obj_fd, + btf_type_id); } if (err) { pr_warn("prog '%s': failed to find kernel BTF type ID of '%s': %d\n", diff --git a/tools/lib/bpf/libbpf_probes.c b/tools/lib/bpf/libbpf_probes.c index 9c4db90b92b6..98373d126d9d 100644 --- a/tools/lib/bpf/libbpf_probes.c +++ b/tools/lib/bpf/libbpf_probes.c @@ -326,6 +326,7 @@ static int probe_map_create(enum bpf_map_type map_type) case BPF_MAP_TYPE_STRUCT_OPS: /* we'll get -ENOTSUPP for invalid BTF type ID for struct_ops */ opts.btf_vmlinux_value_type_id = 1; + opts.value_type_btf_obj_fd = -1; exp_err = -524; /* -ENOTSUPP */ break; case BPF_MAP_TYPE_BLOOM_FILTER: From patchwork Thu Jan 18 01:49:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522302 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 957119455 for ; Thu, 18 Jan 2024 01:50:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542607; cv=none; b=fRhXT2l/ZTgDy5/5hSu7/7g7uK2iErjsXFBIwkBDOSleYMauXV2a0GxDYUWJMmR5IPqK/rnZrVO/XbcxYXbckIZUW0AbDrJtTggCVsBblXUvbVvQTegBpS+mCcWN8uupVz/KbP5TlrfGoo/OnExTDIsmHiXcrW+hWdTNTGX57NI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542607; c=relaxed/simple; bh=qV/bNE7mVSdiBN51PE1Eq8GIQzmn+HJcIEvE8ef2rCI=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=YDx5PCSoqPaR5dmX6D0tzEvkZIiLsu2RIjvS+DVrnc2Pkc2I3xZwho4wiWCvGKX8hi/7y8uDF1oI6BeZ9rcY9CflpEbwCJzQt8qmt1wldPZ52ikczwQy7cnl4FelOHdME2ijuUaR6l20kBWS151sMYjHBsG63t93giMsabKbmS0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iiikFk1m; arc=none smtp.client-ip=209.85.128.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iiikFk1m" Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-5f2aab1c0c5so2503707b3.0 for ; Wed, 17 Jan 2024 17:50:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542605; x=1706147405; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vNpUUEEGm/sXghb4A1ayJ7iYZn64QQerqvktJXH1aV4=; b=iiikFk1mvzJ+XOMic7cFgBQ+CA6uV0gsCOMI/giNPkxt/Ki3xyYMs7AyX6zkXs47FL B909oD9CAzChtHrgfhDJhKzMK/yR7eMNNGITj30q5urKnWrCgjlZK3sK8h3CyrqDGglk mq1h4sOcmC+QjB5K3hXaWhurYIHaCocOPTnz/fAURE1eoAxfqMG1GrywDWRQ0V8mr/qZ V+8WYYfTiPCpybQSPVposJQ7atCNNZnnrc4syKBR98nrorkkTaQ7mwA4Iz1ma3bvipcZ /U+eTAcHzO3/ScYc2CclhwAVKmCQOFeDPFFjEBu6oNqV9jLZJfyFkQjKo61iZyUR2xDx 8ifQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542605; x=1706147405; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vNpUUEEGm/sXghb4A1ayJ7iYZn64QQerqvktJXH1aV4=; b=cezoiPjGIETp4UZcDv/NCOGgPLtlMtV9IQoFPNTZIibSAOPNcw+GvBJNVEcgvtDXjJ 4Lz2CKaWlZ9eRmKdqlxazifJAwhmKVFJ01Oxl9kIeHi6PADWx/KIUSV3FmD64emzcchN pJrZaFQQO06aIH389z/B7mBI8gh++K9AMQmV5BGHQeFPVWFhI8sCe9URBvWnkMQ5e4r2 7Qr5+pcb6Xmol8ju5RQ8JaUUUghTyozx2aKcFkms7uLbAZ73a4wIewZkWOiNkirw5MMZ tbH4MJynFLnAQES8CVqV7X/JNGHQxyaZm9YqYmvT2OUICb6fVwuAoPy/0rMUCmsUA0AM 74XQ== X-Gm-Message-State: AOJu0YzTLeJEp4ngMbXv6AwOeaJpzxf04EolYQRSXmm2P1SVniTdd5a2 Ox6EBKzYyrmiw5YP0y+n2ruE/9cL4lo212ttsWw+2IHLGmD60SBprVKO+jRL X-Google-Smtp-Source: AGHT+IHB3WKAaZWTBpu0GJwwsL74OgeLMurOT+GB4rA4ejsgzUBBk74FimJplB0lV8f3p7EK2JbrLw== X-Received: by 2002:a0d:d843:0:b0:5ff:88b6:8775 with SMTP id a64-20020a0dd843000000b005ff88b68775mr88644ywe.44.1705542605234; Wed, 17 Jan 2024 17:50:05 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.50.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:50:04 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 13/14] bpf: export btf_ctx_access to modules. Date: Wed, 17 Jan 2024 17:49:29 -0800 Message-Id: <20240118014930.1992551-14-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee The module requires the use of btf_ctx_access() to invoke bpf_tracing_btf_ctx_access() from a module. This function is valuable for implementing validation functions that ensure proper access to ctx. Signed-off-by: Kui-Feng Lee --- kernel/bpf/btf.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 860b2946de3c..81db591b4a22 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6142,6 +6142,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, __btf_name_by_offset(btf, t->name_off)); return true; } +EXPORT_SYMBOL_GPL(btf_ctx_access); enum bpf_struct_walk_result { /* < 0 error */ From patchwork Thu Jan 18 01:49:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kui-Feng Lee X-Patchwork-Id: 13522303 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com [209.85.128.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6A6D947F for ; Thu, 18 Jan 2024 01:50:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542609; cv=none; b=CDQmVOpJyjoDbI3r9x+3fQKCRLMX7hNwzq8JwPUWQsmrDfW6tvsb3IW2DAF1n3lOU5Sqpxmh4ouY0smSpVkwgcRKGc5e/qx/OnPoq6eJabAe8/xvhpH4tfs1UPygabLVrnRQakIp7bqaaNHTMGBHLRhEDFdFswfEKKi3ktbpeEc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705542609; c=relaxed/simple; bh=iyhHvIPvaBBtfmoJDpxo9OZbZ7hgn3EAMrvPEF5dFGQ=; h=Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Google-Smtp-Source:X-Received:Received:From: To:Cc:Subject:Date:Message-Id:X-Mailer:In-Reply-To:References: MIME-Version:Content-Transfer-Encoding; b=qXNsK4In+Pk9tMOT71ntQAPQ9JMQJMLotTQ4Ayipf8n+k2dUrxB0kxy/v3nlkJ4suOwZx3MqGhU/UJIdLXvzZEBZhtyW9laFeVm0zQpnrArMGLU+QaKskM3mfxElm/fDCkU859kuHBK55eR6uC6vvGtm3kBxiidi2aiFVtdiU4A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SpZCp9ni; arc=none smtp.client-ip=209.85.128.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SpZCp9ni" Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-5fc2e997804so53566697b3.3 for ; Wed, 17 Jan 2024 17:50:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705542606; x=1706147406; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U/0XNGTvS6j7zkOcjeg62IEHvsHjUequp8YwrgIRsYg=; b=SpZCp9niSV6VVDh5W08moV60czyqIXO3zMkABALBTZs+BEHEa5Ok5CmSgTaLpBt1qe oJqJ8B/WQ6yn8IpAWaneXMjtgWw1U1uBi6JUWMuhxwa5AI3wgTWh5ea/tlDnRZhxFOTj QtGE3cJ/obOxQHyXsf04Aoh3qO5brhcW2lb4b4bQg2L1P6lK2BeYB/05OrLSdiHlavqx m1gbIy6VtpSCUGlDpelvqovq+9EW/tDLshAqvzLhH0tgDJdg4n6y1ZzPYmHDMIsP8X/L lSRc48ZK/8uMxgYfMTJJc9u9Eap6HkUf7viNyHxA5Q/80wDxYmnfN+1zHnUdLj1X3OQm W/wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705542606; x=1706147406; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U/0XNGTvS6j7zkOcjeg62IEHvsHjUequp8YwrgIRsYg=; b=EJD9Wh4OdRGY7XPeqCgOfZ3oBlfbL7I4bQ9HsMetXu27/8QbFp3teTDcSfT2RQQzpu qqFYo1D5BYI8RDXpcJLT18mNs7YUrWRkLoArROcAFGG0uAGPwnWi4HsLODjJ8ki2LgnU gY8qoA65THM2RK6uYR/EkN3nCBgtnKOrTijCswWa/X3siSrHhWaKuQ3KqTk64xEkzXfe q2Yl6tKFQzlX3BvmzkWPB/EwxhYkQfilld45I+JCB6DxvrlGUqYO53wMnl8H2bE0lxIT ViHbJnXGUGguwJ6nd2QWhtrrywfVPdR8fArIWUKxDhfwh4+g37w/05e89Cd1YuSNPgb0 ooZA== X-Gm-Message-State: AOJu0Yz3YQoP+R2u1DC8Q5gujADhdhEX+vYZJw4pLFL4mTKzm2/C2IUK WEvwPVx7Exd3AWW+xxsi1eEGo9Ba/Euo6ZQ1WeNT9hQJFyc3l5xZPHEN4yB5 X-Google-Smtp-Source: AGHT+IH0FgckQgiL+DxRmx3xBOTinZBWZxnuWzSuPOeIqQKuED3tcw+kZ5zctn5/TEW9j1KhRl08mA== X-Received: by 2002:a81:6544:0:b0:5f0:3620:2909 with SMTP id z65-20020a816544000000b005f036202909mr148736ywb.66.1705542606486; Wed, 17 Jan 2024 17:50:06 -0800 (PST) Received: from kickker.attlocal.net ([2600:1700:6cf8:1240:8b90:cd6a:b588:8d99]) by smtp.gmail.com with ESMTPSA id cb9-20020a05690c090900b005e5fff5c537sm6248606ywb.85.2024.01.17.17.50.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jan 2024 17:50:06 -0800 (PST) From: thinker.li@gmail.com To: bpf@vger.kernel.org, ast@kernel.org, martin.lau@linux.dev, song@kernel.org, kernel-team@meta.com, andrii@kernel.org, drosen@google.com Cc: sinquersw@gmail.com, kuifeng@meta.com, Kui-Feng Lee Subject: [PATCH bpf-next v16 14/14] selftests/bpf: test case for register_bpf_struct_ops(). Date: Wed, 17 Jan 2024 17:49:30 -0800 Message-Id: <20240118014930.1992551-15-thinker.li@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240118014930.1992551-1-thinker.li@gmail.com> References: <20240118014930.1992551-1-thinker.li@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Kui-Feng Lee Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module. Signed-off-by: Kui-Feng Lee --- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 66 ++++++++++++++++ .../selftests/bpf/bpf_testmod/bpf_testmod.h | 5 ++ .../bpf/prog_tests/test_struct_ops_module.c | 75 +++++++++++++++++++ .../selftests/bpf/progs/struct_ops_module.c | 30 ++++++++ 4 files changed, 176 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 91907b321f91..fe945d093378 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ +#include #include #include #include @@ -520,11 +521,75 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_static_unused_arg) BTF_ID_FLAGS(func, bpf_kfunc_call_test_offset) BTF_SET8_END(bpf_testmod_check_kfunc_ids) +static int bpf_testmod_ops_init(struct btf *btf) +{ + return 0; +} + +static bool bpf_testmod_ops_is_valid_access(int off, int size, + enum bpf_access_type type, + const struct bpf_prog *prog, + struct bpf_insn_access_aux *info) +{ + return bpf_tracing_btf_ctx_access(off, size, type, prog, info); +} + +static int bpf_testmod_ops_init_member(const struct btf_type *t, + const struct btf_member *member, + void *kdata, const void *udata) +{ + return 0; +} + static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = { .owner = THIS_MODULE, .set = &bpf_testmod_check_kfunc_ids, }; +static const struct bpf_verifier_ops bpf_testmod_verifier_ops = { + .is_valid_access = bpf_testmod_ops_is_valid_access, +}; + +static int bpf_dummy_reg(void *kdata) +{ + struct bpf_testmod_ops *ops = kdata; + int r; + + r = ops->test_2(4, 3); + + return 0; +} + +static void bpf_dummy_unreg(void *kdata) +{ +} + +static int bpf_testmod_test_1(void) +{ + return 0; +} + +static int bpf_testmod_test_2(int a, int b) +{ + return 0; +} + +static struct bpf_testmod_ops __bpf_testmod_ops = { + .test_1 = bpf_testmod_test_1, + .test_2 = bpf_testmod_test_2, +}; + +struct bpf_struct_ops bpf_bpf_testmod_ops = { + .verifier_ops = &bpf_testmod_verifier_ops, + .init = bpf_testmod_ops_init, + .init_member = bpf_testmod_ops_init_member, + .reg = bpf_dummy_reg, + .unreg = bpf_dummy_unreg, + .cfi_stubs = &__bpf_testmod_ops, + .name = "bpf_testmod_ops", + .owner = THIS_MODULE, +}; + extern int bpf_fentry_test1(int a); static int bpf_testmod_init(void) @@ -535,6 +600,7 @@ static int bpf_testmod_init(void) ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_testmod_kfunc_set); ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set); ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set); + ret = ret ?: REGISTER_BPF_STRUCT_OPS(&bpf_bpf_testmod_ops, bpf_testmod_ops); if (ret < 0) return ret; if (bpf_fentry_test1(0) < 0) diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h index f32793efe095..ca5435751c79 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h @@ -28,4 +28,9 @@ struct bpf_iter_testmod_seq { int cnt; }; +struct bpf_testmod_ops { + int (*test_1)(void); + int (*test_2)(int a, int b); +}; + #endif /* _BPF_TESTMOD_H */ diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c new file mode 100644 index 000000000000..333d70c5f708 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c @@ -0,0 +1,75 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include + +#include "struct_ops_module.skel.h" + +static void check_map_info(struct bpf_map_info *info) +{ + struct bpf_btf_info btf_info; + char btf_name[256]; + u32 btf_info_len = sizeof(btf_info); + int err, fd; + + fd = bpf_btf_get_fd_by_id(info->btf_vmlinux_id); + if (!ASSERT_GE(fd, 0, "get_value_type_btf_obj_fd")) + return; + + memset(&btf_info, 0, sizeof(btf_info)); + btf_info.name = ptr_to_u64(btf_name); + btf_info.name_len = sizeof(btf_name); + err = bpf_btf_get_info_by_fd(fd, &btf_info, &btf_info_len); + if (!ASSERT_OK(err, "get_value_type_btf_obj_info")) + goto cleanup; + + if (!ASSERT_EQ(strcmp(btf_name, "bpf_testmod"), 0, "get_value_type_btf_obj_name")) + goto cleanup; + +cleanup: + close(fd); +} + +static void test_struct_ops_load(void) +{ + DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts); + struct struct_ops_module *skel; + struct bpf_map_info info = {}; + struct bpf_link *link; + int err; + u32 len; + + skel = struct_ops_module__open_opts(&opts); + if (!ASSERT_OK_PTR(skel, "struct_ops_module_open")) + return; + + err = struct_ops_module__load(skel); + if (!ASSERT_OK(err, "struct_ops_module_load")) + goto cleanup; + + len = sizeof(info); + err = bpf_map_get_info_by_fd(bpf_map__fd(skel->maps.testmod_1), &info, + &len); + if (!ASSERT_OK(err, "bpf_map_get_info_by_fd")) + goto cleanup; + + link = bpf_map__attach_struct_ops(skel->maps.testmod_1); + ASSERT_OK_PTR(link, "attach_test_mod_1"); + + /* test_2() will be called from bpf_dummy_reg() in bpf_testmod.c */ + ASSERT_EQ(skel->bss->test_2_result, 7, "test_2_result"); + + bpf_link__destroy(link); + + check_map_info(&info); + +cleanup: + struct_ops_module__destroy(skel); +} + +void serial_test_struct_ops_module(void) +{ + if (test__start_subtest("test_struct_ops_load")) + test_struct_ops_load(); +} + diff --git a/tools/testing/selftests/bpf/progs/struct_ops_module.c b/tools/testing/selftests/bpf/progs/struct_ops_module.c new file mode 100644 index 000000000000..cb305d04342f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_module.c @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include "../bpf_testmod/bpf_testmod.h" + +char _license[] SEC("license") = "GPL"; + +int test_2_result = 0; + +SEC("struct_ops/test_1") +int BPF_PROG(test_1) +{ + return 0xdeadbeef; +} + +SEC("struct_ops/test_2") +int BPF_PROG(test_2, int a, int b) +{ + test_2_result = a + b; + return a + b; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_1 = { + .test_1 = (void *)test_1, + .test_2 = (void *)test_2, +}; +