From patchwork Fri Dec 6 04:03:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13896379 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D308B339AB for ; Fri, 6 Dec 2024 04:03:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457806; cv=none; b=L8zotfj6lb45FwtCEFCehvfXLdCWsTQ6Ej6A/TXnjrvUMldlBeRLVmZwht+nlBJOzdtuom5D12UihXAIwlbtDpz7ivb85xteC4eJzZHyUH8pMwIvuRXEoiBRErGEbh57l2n20tGP8QCe62+CrRDP0K5IhqHZGxrdlBh6KySeCgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457806; c=relaxed/simple; bh=TlC1hQwWMq+Mk+iKtgbwNYHf3xze7SGO+PXOV4DIyIo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bZQz4Z+uhNJalR6at0bONQACvmDAKNM9a4RcHRL4MhURMT+GqD8VB8ynX9T7Elg9tAlMogASjMrxonHeRRtzf3l9f++KPHUomP/oXW4CDlgSX44ry5uOdaNVJIWdbbhIesul57UsEaWHV6IVu1ScMB5upCUycBEVuf7MiTdO+Hg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=biIDGLUH; arc=none smtp.client-ip=209.85.216.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="biIDGLUH" Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2eed4fa2b3aso1313116a91.0 for ; Thu, 05 Dec 2024 20:03:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733457804; x=1734062604; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wCeClI9N6IGYROGb0Jn7lpSIINKCLvjsqLrZAWgTIeU=; b=biIDGLUHACoBC/mNcXM/fC+uKcgnWDfpC80M2ofyYWBlVzF1uz5khT4hjaNVFpceA5 dRTiLGu6APnGucXYedWd36lqf3qeidSATybNDCGcRsTNteh1XdnQPrvUp3YudTCAeV5T SGpenhTq6/rHZR5C5QxBXQ15go0S1mGLvJdN/gIsFCQGL4N5GGYZqi4NIYTm6wWtYDRe BVt4EcDCLlIzRcETjmH6FCmISO8Wgc9rURaS5Lpdyi37f1z3YwblZTSFcq/8XL9jpHVq mH1kND2PxCuu4FapIz5WaxQ6PBfVSYgrWIY88W4T+phdrgVNwCMP7MpDxHf65dhLxgvM k5Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733457804; x=1734062604; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wCeClI9N6IGYROGb0Jn7lpSIINKCLvjsqLrZAWgTIeU=; b=on6r/ymghDFxZ6NHmsVBYYIPHB5qGRJmcFASeoj3/OfYEiyywKPh+P2sssK7b73MzU InlpncBpWiWwpLCgB8bm2X0vWgijOGTEeq8rXhbUJ3FWs5H7yfbdPhGkv+r3KY7+gpUD 6/ON2xLZHJKk/cBiR4f7XUBqnmVrviZoJtWx6G+781EPuPopkFu37pa3RVgeqjb1h3SB 5QcHoT4hUqamHOR2qe7bvAjapEdrkOqIDUFmpFQorKH/WP9jqUc5m2dx4qQNRndHXwl5 TXWhK8ptJpbmfMZNDERYQip+017dzKD+hhe/pn/E/VUUCH2ZX+/5cdMQmv2ExouOuw3U RLHQ== X-Gm-Message-State: AOJu0Yy9vmbeMZhdUJUa2OOd/zOLBjHNGZ3m5rmMpNT3B8tzfDIZ4f/g gRnzP2+aMnb4MXekZI6hDaUY60c43skgM2RsG5RwKq5HdxtimTOQILiBOQ== X-Gm-Gg: ASbGncv6Qscz3Rxo6kxK38IIM+FeiURfhN7jl+MujGG0TpcNdNfHCkDzqk1ySKonXC5 QtFRKKrXItPAPodDdZgtUq2R0u5+DvJItt6Zeq71r1hxBzLzatvN89WI75UDWDWylrSkYjVpncw VoSDqNhPL9GDeHERGPr1Guh2VMp68iJMlxEaMnZdJcsW0P5Svlv4m3HpHnpwkL3mYH5pLfXRJAd ZPfCNJga2NlFT/ERWE7FsSAogkgDeqPKvF+Gb4c4hM2kg== X-Google-Smtp-Source: AGHT+IGTDESjykmtNVUxiOHZ9hQK3VVjEo0wccEUHlkmkmzDpHfGPTEcfxU46BcIm4dfIRfUCs+HXQ== X-Received: by 2002:a17:90b:2d43:b0:2ee:ad18:b30d with SMTP id 98e67ed59e1d1-2ef69654504mr2152790a91.6.1733457803789; Thu, 05 Dec 2024 20:03:23 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97ffsm4101846a91.10.2024.12.05.20.03.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Dec 2024 20:03:23 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, mejedi@gmail.com, Eduard Zingerman Subject: [PATCH bpf 1/4] bpf: add find_containing_subprog() utility function Date: Thu, 5 Dec 2024 20:03:04 -0800 Message-ID: <20241206040307.568065-2-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206040307.568065-1-eddyz87@gmail.com> References: <20241206040307.568065-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add a utility function, looking for a subprogram containing a given instruction index, rewrite find_subprog() to use this function. Signed-off-by: Eduard Zingerman --- kernel/bpf/verifier.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 01fbef9576e0..277c1892bb9a 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -2597,16 +2597,36 @@ static int cmp_subprogs(const void *a, const void *b) ((struct bpf_subprog_info *)b)->start; } +/* Find subprogram that contains instruction at 'off' */ +static struct bpf_subprog_info *find_containing_subprog(struct bpf_verifier_env *env, int off) +{ + struct bpf_subprog_info *vals = env->subprog_info; + int l, r, m; + + if (off >= env->prog->len || off < 0 || env->subprog_cnt == 0) + return NULL; + + l = 0; + r = env->subprog_cnt - 1; + while (l < r) { + m = l + (r - l + 1) / 2; + if (vals[m].start <= off) + l = m; + else + r = m - 1; + } + return &vals[l]; +} + +/* Find subprogram that starts exactly at 'off' */ static int find_subprog(struct bpf_verifier_env *env, int off) { struct bpf_subprog_info *p; - p = bsearch(&off, env->subprog_info, env->subprog_cnt, - sizeof(env->subprog_info[0]), cmp_subprogs); - if (!p) + p = find_containing_subprog(env, off); + if (!p || p->start != off) return -ENOENT; return p - env->subprog_info; - } static int add_subprog(struct bpf_verifier_env *env, int off) From patchwork Fri Dec 6 04:03:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13896380 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F409175BF for ; Fri, 6 Dec 2024 04:03:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457807; cv=none; b=UVFaB86S6aO3xNVbmDNCp2alKKbmM75shNdqwq0jpkXlS8IKSJVC4DziheuUh+Ll82i081bJzaoa15KU8Zy0N/4QT3Dik8uP4SHFPCe7lndR0dKxCO8FDBPVpTatenUjkFbsJVRVS1TMXX0ynJhKSkY/XQ11YPoKU+0ucs5b7Jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457807; c=relaxed/simple; bh=a9a5bDwU6ShIZlOC+79WCuNVJCFXLQBDM/WxaKA1E5M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZXkUZgc1k/RNKwlTUZhY/0rMB8rk41zDsTAbXpVnPyKpE1//tPS/bk7WCdgySYvJHR7XXIZ43f7O8/6LWsUXK8aDo4VtJPkiYfeYYtBoC39N9qiywBKcmUgjvS00MS+LAo1SbkZdRkkoCsfILGW6dUhwY+0coM2zP53YcjX5M3A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RIKHbirg; arc=none smtp.client-ip=209.85.216.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RIKHbirg" Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2ef6af22ea8so328049a91.0 for ; Thu, 05 Dec 2024 20:03:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733457805; x=1734062605; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aF6MVln4LC9hUMsccdpl2mvsSb4BSYKAburlVkwvkJQ=; b=RIKHbirg5abHWSgwckgxT6yW1UyvKeJpYvZx+ZcidlgCPYlxLO/Fi6wqyc3iTd7hwO 8Iagscydl8w3CjkdDCcF/fC/ho4q7qVbtmkPiUZ3WMHwj/mOIW4jsmRmGiemcNW2hiOl SLG15DJqW+L8AvR/TcYnz375aOjw4HvPhRIjGDcy7ajAGhJCtVWoZtcRLBne0ug1H1l2 49grlqfkU4onVgX7eCpl4l7ZjdTtN8/icZGl73VxbQVabIedFoyvIaaQnHA0WyhabFfc QgQ2DIdK2KvHnokab4aUw0DlK8j4bQ6QJpZX+DoJYvxcTYlUJMGELj6x2wxwI+vsBA0O 0YDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733457805; x=1734062605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aF6MVln4LC9hUMsccdpl2mvsSb4BSYKAburlVkwvkJQ=; b=IEZW4w45FO30+tJF7bb1E296h+frU/T/nYIzQroQwrjMaU5BSMLL+XWGg6TXL7QOAY aXKAsTDrZzh6sxSCHDpg11QUkp3EdXoP2wPGx6etJCsaALHQiNcQDW8oyelj0kf7VbSy btHa9mzs2x50e5C9VuID5BOegdB05/ZAmU2r/Q9oIjNCVwVss12ozzNW7fd+VOv/ZnOh egQbIwRBjQ6m8TIjsmPxzgNguppMZr512oXgmsBYvczaDRw8Ahd+OKhxkQhUyk3eiN9Z dOmEW6WRIAkXDNwJrlYqcW/WU4CJIzLhduOV+mq/M9H05G+2WbAn7pTVdlveOY3dgrn1 NaSw== X-Gm-Message-State: AOJu0YzLIIZH5klpJ8S4xHEVaTTdJCQ0swQP/0j126Hhw0APwpv6bMss FfvRZl5GYrFnqTBjCqaZv739Ci3z2ZdvS5p7oi4RH/zWOplhhYhFvV0Mzg== X-Gm-Gg: ASbGncszXp4xJuxg3EZM3YJoYHmi7xCrGdCuQf1shfTRPSJw3SHt8Na3ovuZxZZHfaV nhTQ8/vPHIQXQWlb8v/bv7boCeKji2e+bcUgcZFwKEdegTJdau2L8CepX73cDYyjakErKlB3XB5 67uuDNfocAKLqGPsLep9Jyadzm48CO34N/8vlgAE3HyuAzuMB3OvGAI6a+KUTMHKz9fpJnHlpCW sjkykkPn3qGeWhCspquJalgJND00vhlGYt0224ZuwTyhQ== X-Google-Smtp-Source: AGHT+IF6dEn3GMKxbEVCLY+KzpwHP1Mxs6tvHjhXddFterPX+t1UzIKtZPerYepfGfZdAACr2ldA9A== X-Received: by 2002:a17:90b:1a86:b0:2ee:c291:7674 with SMTP id 98e67ed59e1d1-2ef69b3594amr2876799a91.14.1733457805248; Thu, 05 Dec 2024 20:03:25 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97ffsm4101846a91.10.2024.12.05.20.03.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Dec 2024 20:03:24 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, mejedi@gmail.com, Eduard Zingerman Subject: [PATCH bpf 2/4] bpf: refactor bpf_helper_changes_pkt_data to use helper number Date: Thu, 5 Dec 2024 20:03:05 -0800 Message-ID: <20241206040307.568065-3-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206040307.568065-1-eddyz87@gmail.com> References: <20241206040307.568065-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Use BPF helper number instead of function pointer in bpf_helper_changes_pkt_data(). This would simplify usage of this function in verifier.c:check_cfg() (in a follow-up patch), where only helper number is easily available and there is no real need to lookup helper proto. Signed-off-by: Eduard Zingerman --- include/linux/filter.h | 2 +- kernel/bpf/core.c | 2 +- kernel/bpf/verifier.c | 2 +- net/core/filter.c | 63 +++++++++++++++++++----------------------- 4 files changed, 31 insertions(+), 38 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index 3a21947f2fd4..0477254bc2d3 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -1122,7 +1122,7 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena); bool bpf_jit_supports_private_stack(void); u64 bpf_arch_uaddress_limit(void); void arch_bpf_stack_walk(bool (*consume_fn)(void *cookie, u64 ip, u64 sp, u64 bp), void *cookie); -bool bpf_helper_changes_pkt_data(void *func); +bool bpf_helper_changes_pkt_data(enum bpf_func_id func_id); static inline bool bpf_dump_raw_ok(const struct cred *cred) { diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index a2327c4fdc8b..6fa8041d4831 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2936,7 +2936,7 @@ void __weak bpf_jit_compile(struct bpf_prog *prog) { } -bool __weak bpf_helper_changes_pkt_data(void *func) +bool __weak bpf_helper_changes_pkt_data(enum bpf_func_id func_id) { return false; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 277c1892bb9a..ad3f6d28e8e4 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10728,7 +10728,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn } /* With LD_ABS/IND some JITs save/restore skb from r1. */ - changes_data = bpf_helper_changes_pkt_data(fn->func); + changes_data = bpf_helper_changes_pkt_data(func_id); if (changes_data && fn->arg1_type != ARG_PTR_TO_CTX) { verbose(env, "kernel subsystem misconfigured func %s#%d: r1 != ctx\n", func_id_name(func_id), func_id); diff --git a/net/core/filter.c b/net/core/filter.c index 6625b3f563a4..efb75eed2e35 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -7899,42 +7899,35 @@ static const struct bpf_func_proto bpf_tcp_raw_check_syncookie_ipv6_proto = { #endif /* CONFIG_INET */ -bool bpf_helper_changes_pkt_data(void *func) -{ - if (func == bpf_skb_vlan_push || - func == bpf_skb_vlan_pop || - func == bpf_skb_store_bytes || - func == bpf_skb_change_proto || - func == bpf_skb_change_head || - func == sk_skb_change_head || - func == bpf_skb_change_tail || - func == sk_skb_change_tail || - func == bpf_skb_adjust_room || - func == sk_skb_adjust_room || - func == bpf_skb_pull_data || - func == sk_skb_pull_data || - func == bpf_clone_redirect || - func == bpf_l3_csum_replace || - func == bpf_l4_csum_replace || - func == bpf_xdp_adjust_head || - func == bpf_xdp_adjust_meta || - func == bpf_msg_pull_data || - func == bpf_msg_push_data || - func == bpf_msg_pop_data || - func == bpf_xdp_adjust_tail || -#if IS_ENABLED(CONFIG_IPV6_SEG6_BPF) - func == bpf_lwt_seg6_store_bytes || - func == bpf_lwt_seg6_adjust_srh || - func == bpf_lwt_seg6_action || -#endif -#ifdef CONFIG_INET - func == bpf_sock_ops_store_hdr_opt || -#endif - func == bpf_lwt_in_push_encap || - func == bpf_lwt_xmit_push_encap) +bool bpf_helper_changes_pkt_data(enum bpf_func_id func_id) +{ + switch (func_id) { + case BPF_FUNC_clone_redirect: + case BPF_FUNC_l3_csum_replace: + case BPF_FUNC_l4_csum_replace: + case BPF_FUNC_lwt_push_encap: + case BPF_FUNC_lwt_seg6_action: + case BPF_FUNC_lwt_seg6_adjust_srh: + case BPF_FUNC_lwt_seg6_store_bytes: + case BPF_FUNC_msg_pop_data: + case BPF_FUNC_msg_pull_data: + case BPF_FUNC_msg_push_data: + case BPF_FUNC_skb_adjust_room: + case BPF_FUNC_skb_change_head: + case BPF_FUNC_skb_change_proto: + case BPF_FUNC_skb_change_tail: + case BPF_FUNC_skb_pull_data: + case BPF_FUNC_skb_store_bytes: + case BPF_FUNC_skb_vlan_pop: + case BPF_FUNC_skb_vlan_push: + case BPF_FUNC_store_hdr_opt: + case BPF_FUNC_xdp_adjust_head: + case BPF_FUNC_xdp_adjust_meta: + case BPF_FUNC_xdp_adjust_tail: return true; - - return false; + default: + return false; + } } const struct bpf_func_proto bpf_event_output_data_proto __weak; From patchwork Fri Dec 6 04:03:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13896381 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A2A8194120 for ; Fri, 6 Dec 2024 04:03:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457809; cv=none; b=SuCMtzIdAkr7B2nptjHn64AnU0QGM2MSHqTJE25KlSrcTN8duxxhxpYbB2tT3t1EA9u7i0NamJWITxcLi41bBUbQ298wHnMfe+PU8ENY5dz358LG2wffuLRZM6q226aCDPNpKnJ4/KEwQ4MDpiN4k9+/hAb5MmLWp4vLZIPQtEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457809; c=relaxed/simple; bh=ETgZc24McfcPAcX2yubnpzSEOfbz3gxmaXfMfYvsdak=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=e/AzDRqfz0JZwdGI1h8tBkGQvnixDMq3Tf8vvcAVKDL60Lm4PMntRhX0ImMJXJQvyPZKkndvO/H1NitAP3/TdsZeOaDKCVxrq8BgpGpNKfKsxVzW5+wWaFsTSM4Qj9DNit53wqLtzEeyu0IvYh65FVBo8k+AfHDPhRzhbOycbko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kQedV5Lp; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kQedV5Lp" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-724f42c1c38so1487113b3a.1 for ; Thu, 05 Dec 2024 20:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733457807; x=1734062607; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IZ4GQ7+KCdCyL8YNQozZccMYcpExv9M3xriogwMx2ho=; b=kQedV5LpwIORYhLuzZSn1kOGUFb0L3Yh7tsYqrxgoGAPIbmjPg7nObsqp4T0TCzoAo TrBRxaApVQNBloQ7QWb1WNQDUt3pKf4NmuEJKCP8JfmpIr4mLnjZzrGsCPcpS5ULL2E3 c3vnsSXlyWU3usqLkoSsNs+k3Jy9rS4CzkpjeXHBKjQxrRrKDoiPWV/bekQRV0lvkqz2 KvBejwEHMNHpNWGE96jPKHsuS3AbPiHga+8z7wf3Q2M7EdCywC1LAoP4t7MNfhxIM5YE 1y8gV6o43JKNqwik1aAxkGZs7KxttYRyAO90Ci073tgnMXki2PrxsMPpHewBoFXeiKMY xRmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733457807; x=1734062607; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IZ4GQ7+KCdCyL8YNQozZccMYcpExv9M3xriogwMx2ho=; b=jRx4jBEHRN1b5sYbO8msALLSFQHmDYYhoEOEbBgNOitojGFU6R5ETHJhkLPTfccoMm LyMNWsx2vj+GuxSuY58qtCU3GKegGOw141c4mg/gCKwGNz34ODIvSoVx385xR8JrMkHT yj3bzomwTPkomjYl9scYnlcWqHm3ncL95sWCLg8iOBSD4Pljlbz4IDewYX4l2eRaC1q0 +yej+xB+FiBWeTbzZde8d+Q/2Av8MA8Zqoctu7n8rbAhvAWezNcjWwmvydqg/N3CdkwB g2iFqz/cuem1guu+VMJiUfTlygkr/zzzixA+ihTOAvIB0ggV+CJFLOtyC6VKSVdFHe6v VexA== X-Gm-Message-State: AOJu0YyKZmlXMSKR9aMjFqF3kO6pIjYXoHxpgpVt9zNqnHsDxQNtS6pu 9vaJTQJOAoIGm20BziOyT/bXL9HxCGB56KPPyqQogBX+u+b/eVOZJadc6Q== X-Gm-Gg: ASbGncsptF8L3WNA+VUtVIMNhTtthF8YbIEHEfYXmqkJjd1/FzVMtH3E0txJhcSNfBp kKgZraj4ZaIjTM3uUPTOd65Vxkgm2sEBnJhBpjs+2v2AZ1pqwvG/lEJfiPfb/xcq9NHZ0RRpikY OYBIq2q7radZOWgy/iWPNJIhL99uflV00QQy7ciuh3t7ozUUCMUbYXbFCNiIVPmQDIa7/xrpQSF LE3ZwZ0hgZgAWzVR/qZBCLdXm5b37dnrYfZA5eXVzYQaQ== X-Google-Smtp-Source: AGHT+IEwSQWlwAJ/53FpebvrKhZHIbI33908doWKwtLNnZl5Bo2Vm0RBWF+riPP4TBR1BSG72xlyEQ== X-Received: by 2002:a17:90b:2f0b:b0:2ee:d35c:39ab with SMTP id 98e67ed59e1d1-2ef6a6c1436mr2300485a91.22.1733457807223; Thu, 05 Dec 2024 20:03:27 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97ffsm4101846a91.10.2024.12.05.20.03.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Dec 2024 20:03:26 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, mejedi@gmail.com, Eduard Zingerman Subject: [PATCH bpf 3/4] bpf: track changes_pkt_data property for global functions Date: Thu, 5 Dec 2024 20:03:06 -0800 Message-ID: <20241206040307.568065-4-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206040307.568065-1-eddyz87@gmail.com> References: <20241206040307.568065-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net When processing calls to certain helpers, verifier invalidates all packet pointers in a current state. For example, consider the following program: __attribute__((__noinline__)) long skb_pull_data(struct __sk_buff *sk, __u32 len) { return bpf_skb_pull_data(sk, len); } SEC("tc") int test_invalidate_checks(struct __sk_buff *sk) { int *p = (void *)(long)sk->data; if ((void *)(p + 1) > (void *)(long)sk->data_end) return TCX_DROP; skb_pull_data(sk, 0); *p = 42; return TCX_PASS; } After a call to bpf_skb_pull_data() the pointer 'p' can't be used safely. See function filter.c:bpf_helper_changes_pkt_data() for a list of such helpers. At the moment verifier does packet pointers invalidation only upon processing calls to helper functions. This means that calls to helpers done from global sub-programs do not invalidate pointers in the caller state. E.g. the following program above is unsafe, but is not rejected by verifier. This commit fixes the omission by computing field bpf_subprog_info->changes_pkt_data for each sub-program before main verification pass. changes_pkt_data should be set if: - subprogram calls helper for which bpf_helper_changes_pkt_data returns true; - subprogram calls a global function, for which bpf_subprog_info->changes_pkt_data should be set. The verifier.c:check_cfg() pass is modified to compute this information. The commit relies on depth first instruction traversal done by check_cfg() and absence of recursive function calls: - check_cfg() would eventually visit every call to subprogram S in a state when S is fully explored; - when S is fully explored: - every direct helper call within S is explored (and thus changes_pkt_data is set if needed); - every call to subprogram S1 called by S was visited with S1 fully explored (and thus S inherits changes_pkt_data from S1). The downside of such approach is that dead code elimination is not taken into account: if a helper call inside global function is dead because of current configuration, verifier would conservatively assume that the call occurs for the purpose of the changes_pkt_data computation. Reported-by: Nick Zavaritsky Closes: https://lore.kernel.org/bpf/0498CA22-5779-4767-9C0C-A9515CEA711F@gmail.com/ Signed-off-by: Eduard Zingerman --- include/linux/bpf_verifier.h | 1 + kernel/bpf/verifier.c | 32 +++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index f4290c179bee..48b7b2eeb7e2 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -659,6 +659,7 @@ struct bpf_subprog_info { bool args_cached: 1; /* true if bpf_fastcall stack region is used by functions that can't be inlined */ bool keep_fastcall_stack: 1; + bool changes_pkt_data: 1; enum priv_stack_mode priv_stack_mode; u8 arg_cnt; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index ad3f6d28e8e4..6a29b68cebd6 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10042,6 +10042,8 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, verbose(env, "Func#%d ('%s') is global and assumed valid.\n", subprog, sub_name); + if (env->subprog_info[subprog].changes_pkt_data) + clear_all_pkt_pointers(env); /* mark global subprog for verifying after main prog */ subprog_aux(env, subprog)->called = true; clear_caller_saved_regs(env, caller->regs); @@ -16246,6 +16248,29 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char return 0; } +static void mark_subprog_changes_pkt_data(struct bpf_verifier_env *env, int off) +{ + struct bpf_subprog_info *subprog; + + subprog = find_containing_subprog(env, off); + subprog->changes_pkt_data = true; +} + +/* 't' is an index of a call-site. + * 'w' is a callee entry point. + * Eventually this function would be called when env->cfg.insn_state[w] == EXPLORED. + * Rely on DFS traversal order and absence of recursive calls to guarantee that + * callee's change_pkt_data marks would be correct at that moment. + */ +static void merge_callee_effects(struct bpf_verifier_env *env, int t, int w) +{ + struct bpf_subprog_info *caller, *callee; + + caller = find_containing_subprog(env, t); + callee = find_containing_subprog(env, w); + caller->changes_pkt_data |= callee->changes_pkt_data; +} + /* non-recursive DFS pseudo code * 1 procedure DFS-iterative(G,v): * 2 label v as discovered @@ -16379,6 +16404,7 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns, bool visit_callee) { int ret, insn_sz; + int w; insn_sz = bpf_is_ldimm64(&insns[t]) ? 2 : 1; ret = push_insn(t, t + insn_sz, FALLTHROUGH, env); @@ -16390,8 +16416,10 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns, mark_jmp_point(env, t + insn_sz); if (visit_callee) { + w = t + insns[t].imm + 1; mark_prune_point(env, t); - ret = push_insn(t, t + insns[t].imm + 1, BRANCH, env); + merge_callee_effects(env, t, w); + ret = push_insn(t, w, BRANCH, env); } return ret; } @@ -16708,6 +16736,8 @@ static int visit_insn(int t, struct bpf_verifier_env *env) mark_prune_point(env, t); mark_jmp_point(env, t); } + if (bpf_helper_call(insn) && bpf_helper_changes_pkt_data(insn->imm)) + mark_subprog_changes_pkt_data(env, t); if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) { struct bpf_kfunc_call_arg_meta meta; From patchwork Fri Dec 6 04:03:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eduard Zingerman X-Patchwork-Id: 13896382 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59264194158 for ; Fri, 6 Dec 2024 04:03:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457810; cv=none; b=EyQv2sJ3/uwdwK8KIs0sew6cea9iuUCYhdjW4yYW9z1YH5SHpt72ennGG+tHwI9KdRnfRDerSz3oRTHnkFGCLg2t8hAGCWf5ggVi9ywR5lg6b+xLmgE0XVauBNNkdMXflG7e9aKt0HUkA5OALy6nOAmQH941lYOgyraMqUgMcSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733457810; c=relaxed/simple; bh=lW4mQ58JGfw7JGH8bL/drjsNfSgfvzBEqWD7FVkfokg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X0XBmWN1naoB3GyRKhdwh7V8AEgvHaWbU1vd+1Ebt++xqC8LQuHFQsD0LYtUEz7e5ZtIQrgeo5mcfYLaExT/gp6bugKKVPnmOW+TUq83j5et3K99Q1qGGDGyMA5U5HVqHxWBSOsMlVq6YSZpAvA6PIPEFtktLSZZLSsQUOVuTNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=PTXn0Nv/; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PTXn0Nv/" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-725b93a59feso307244b3a.2 for ; Thu, 05 Dec 2024 20:03:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733457808; x=1734062608; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4VTXn9EJbSSdNCqgtyoG6wj3OIgIOOdlJuFJ/UTprec=; b=PTXn0Nv/cu8G1TC73omlZTna4JpaUirZLxjLhavY2O7iiuZNEAqwM1Kioxwa7riPs4 SW1x1dXYyVw05lg5opoVhKVmbGkp5H+H9krDWaXODt7INev885vixrCp4QNZO6Mcrnq3 CGGynNlRdXeYayw9Z2MBKoomvyG9AxD/BeZxfi7IfM7tKtNd+d/I5BsuEBGiaHtZWPR/ Ae/anKe1LrBYNWdFXXOOj5YroF1D4O/R8bnMAdyhGr67ibiYLDWsXKa4d4WRteTLEfz+ P3BQ/FO9RidTS97Sk0FBHPJa2aslETlvYEiIXEDcYLQrPEMqyflcScvepUfd+rrAA3ns sZ0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733457808; x=1734062608; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4VTXn9EJbSSdNCqgtyoG6wj3OIgIOOdlJuFJ/UTprec=; b=H3UuJnRoM4yVgEC8oz5AgYKv9cwKmtt8nW7g947bCpCj839tEaMdbQ+UXdWV4jitfZ WJ6p/aONqfev2QdpAv6LHD4tpQDmxqC6MUZLAxUECIMwWbG2sDqRnPMO5X004Ck3buwz g/4x0XXw7r2bV4VD3tf3Prh9lhC0Ld3AfZIMOwi0WOX8x250TCcYgcvTE84nnpGJVIjq UlN6Gu5pnKU1WxTZVzXvD8X/9NCVdTOSBkjrdrhaFF47lccEz2bxnQ8uA7M7nNH89bpS lfNEed4rGLDQYsDF5v+ZWx/nmb9VOoIf4IlqnVRkMUaK4Yhu70t0ZuH9jn6HIsogBsuZ P6NQ== X-Gm-Message-State: AOJu0YzPIMTEdQwIGs7Kuc5UgzKXLSYp19E7yF2sWQUMWYywXJWvPakc RL5fgfjPEpzrZSEgGXcCtZDmVS5JjXdH7HfOc02tmNcwfIM2t3C8n6iFIg== X-Gm-Gg: ASbGncuDL5v1lAOA3m6UAFTtOYr9cWxpREZo22d0a6bp8clxemCwYYUI0/6jdsZJaMK qpN0og8wMpaRPBYoR/C/f4GLNycBdODIKNPW4z9yKluNHMmjxGY5xq4QFVm7s/Er9Z+K/4AUf1T q0/zu7Dv8k25UyefpJgJz/xYyQlhUpUo1WN5H7bSd6rtNnRAueoLZo2QPZqdCrtPRcp71r5rl1c kWDx0dytQg9Fr/ifNPYrIJ8nsXuhmYQCWjaC0OMeNof2w== X-Google-Smtp-Source: AGHT+IEZuyqETvKywL0wrGsW6tHpMr+J3LuRJXV9wb/hOlVDhJgJutOO2JUFNC9XnDQ0qVaFRgr/Qw== X-Received: by 2002:a17:90b:2803:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2ef69e167fdmr2907136a91.16.1733457808359; Thu, 05 Dec 2024 20:03:28 -0800 (PST) Received: from honey-badger.. ([38.34.87.7]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef26ff97ffsm4101846a91.10.2024.12.05.20.03.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Dec 2024 20:03:27 -0800 (PST) From: Eduard Zingerman To: bpf@vger.kernel.org, ast@kernel.org Cc: andrii@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, kernel-team@fb.com, yonghong.song@linux.dev, mejedi@gmail.com, Eduard Zingerman Subject: [PATCH bpf 4/4] selftests/bpf: test for changing packet data from global functions Date: Thu, 5 Dec 2024 20:03:07 -0800 Message-ID: <20241206040307.568065-5-eddyz87@gmail.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241206040307.568065-1-eddyz87@gmail.com> References: <20241206040307.568065-1-eddyz87@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Check if verifier is aware of packet pointers invalidation done in global functions. Based on a test shared by Nick Zavaritsky in [0]. [0] https://lore.kernel.org/bpf/0498CA22-5779-4767-9C0C-A9515CEA711F@gmail.com/ Suggested-by: Nick Zavaritsky Signed-off-by: Eduard Zingerman --- .../selftests/bpf/progs/verifier_sock.c | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/verifier_sock.c b/tools/testing/selftests/bpf/progs/verifier_sock.c index d3e70e38e442..51826379a1aa 100644 --- a/tools/testing/selftests/bpf/progs/verifier_sock.c +++ b/tools/testing/selftests/bpf/progs/verifier_sock.c @@ -1037,4 +1037,32 @@ __naked void sock_create_read_src_port(void) : __clobber_all); } +__noinline +long skb_pull_data2(struct __sk_buff *sk, __u32 len) +{ + return bpf_skb_pull_data(sk, len); +} + +__noinline +long skb_pull_data1(struct __sk_buff *sk, __u32 len) +{ + return skb_pull_data2(sk, len); +} + +/* global function calls bpf_skb_pull_data(), which invalidates packet + * pointers established before global function call. + */ +SEC("tc") +__failure __msg("invalid mem access") +int invalidate_pkt_pointers_from_global_func(struct __sk_buff *sk) +{ + int *p = (void *)(long)sk->data; + + if ((void *)(p + 1) > (void *)(long)sk->data_end) + return TCX_DROP; + skb_pull_data1(sk, 0); + *p = 42; /* this is unsafe */ + return TCX_PASS; +} + char _license[] SEC("license") = "GPL";