[bpf,v3] bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4

Commit Message

Yonghong Song Dec. 1, 2023, 2:46 a.m. UTC

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9
("bpf: Support new 32bit offset jmp instruction") added support for new
32bit offset jmp instruction. Unfortunately, in function
bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset
(plus/minor a small delta) compares to 16-bit offset bound
[S16_MIN, S16_MAX], which caused the following verification failure:
  $ ./test_progs-cpuv4 -t verif_scale_pyperf180
  ...
  insn 10 cannot be patched due to 16-bit range
  ...
  libbpf: failed to load object 'pyperf180.bpf.o'
  scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
  #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied
in bpf-next) needs to be applied to bpf tree for testing purpose.

The fix is rather simple. For 32bit offset branch insn, the adjusted
offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

  [1] https://lore.kernel.org/all/[e-mail]
  [2] https://lore.kernel.org/bpf/[e-mail]

Fixes: 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")
Signed-off-by: Yonghong Song <[e-mail]>
---
 kernel/bpf/core.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

Changelogs:
  v2 -> v3:
    - not doing init for off_min/off_max at decl time and rather do
      it at 'off' assignment time to make code better.
  v1 -> v2:
    - Change off from s32 to s64 to capture overflow case

Comments

Andrii Nakryiko Dec. 1, 2023, 11:53 p.m. UTC | #1

On Thu, Nov 30, 2023 at 6:47 PM Yonghong Song <[e-mail]> wrote:
>
> Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9
> ("bpf: Support new 32bit offset jmp instruction") added support for new
> 32bit offset jmp instruction. Unfortunately, in function
> bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset
> (plus/minor a small delta) compares to 16-bit offset bound
> [S16_MIN, S16_MAX], which caused the following verification failure:
>   $ ./test_progs-cpuv4 -t verif_scale_pyperf180
>   ...
>   insn 10 cannot be patched due to 16-bit range
>   ...
>   libbpf: failed to load object 'pyperf180.bpf.o'
>   scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
>   #405     verif_scale_pyperf180:FAIL
>
> Note that due to recent llvm18 development, the patch [2] (already applied
> in bpf-next) needs to be applied to bpf tree for testing purpose.
>
> The fix is rather simple. For 32bit offset branch insn, the adjusted
> offset compares to [S32_MIN, S32_MAX] and then verification succeeded.
>
>   [1] https://lore.kernel.org/all/[e-mail]
>   [2] https://lore.kernel.org/bpf/[e-mail]
>
> Fixes: 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")
> Signed-off-by: Yonghong Song <[e-mail]>
> ---
>  kernel/bpf/core.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
>
> Changelogs:
>   v2 -> v3:
>     - not doing init for off_min/off_max at decl time and rather do
>       it at 'off' assignment time to make code better.
>   v1 -> v2:
>     - Change off from s32 to s64 to capture overflow case
>
> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index cd3afe57ece3..e76762b5ad28 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c
> @@ -371,14 +371,19 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,
>  static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
>                                 s32 end_new, s32 curr, const bool probe_pass)
>  {
> -       const s32 off_min = S16_MIN, off_max = S16_MAX;
> +       s64 off_min, off_max;
>         s32 delta = end_new - end_old;
> -       s32 off;
> +       s64 off;

I combined off declaration with off_min, off_max on the same line.
Pushed to bpf, thanks!


>
> -       if (insn->code == (BPF_JMP32 | BPF_JA))
> +       if (insn->code == (BPF_JMP32 | BPF_JA)) {
>                 off = insn->imm;
> -       else
> +               off_min = S32_MIN;
> +               off_max = S32_MAX;
> +       } else {
>                 off = insn->off;
> +               off_min = S16_MIN;
> +               off_max = S16_MAX;
> +       }
>
>         if (curr < pos && curr + off + 1 >= end_old)
>                 off += delta;
> --
> 2.34.1
>

[e-mail] Dec. 2, 2023, midnight UTC | #2

Hello:

This patch was applied to bpf/bpf.git (master)
by Andrii Nakryiko <[e-mail]>:

On Thu, 30 Nov 2023 18:46:40 -0800 you wrote:
> Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9
> ("bpf: Support new 32bit offset jmp instruction") added support for new
> 32bit offset jmp instruction. Unfortunately, in function
> bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset
> (plus/minor a small delta) compares to 16-bit offset bound
> [S16_MIN, S16_MAX], which caused the following verification failure:
>   $ ./test_progs-cpuv4 -t verif_scale_pyperf180
>   ...
>   insn 10 cannot be patched due to 16-bit range
>   ...
>   libbpf: failed to load object 'pyperf180.bpf.o'
>   scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
>   #405     verif_scale_pyperf180:FAIL
> 
> [...]

Here is the summary with links:
  - [bpf,v3] bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4
    https://git.kernel.org/bpf/bpf/c/dfce9cb31405

You are awesome, thank you!

Patch

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index cd3afe57ece3..e76762b5ad28 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -371,14 +371,19 @@  static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,
 static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 				s32 end_new, s32 curr, const bool probe_pass)
 {
-	const s32 off_min = S16_MIN, off_max = S16_MAX;
+	s64 off_min, off_max;
 	s32 delta = end_new - end_old;
-	s32 off;
+	s64 off;
 
-	if (insn->code == (BPF_JMP32 | BPF_JA))
+	if (insn->code == (BPF_JMP32 | BPF_JA)) {
 		off = insn->imm;
-	else
+		off_min = S32_MIN;
+		off_max = S32_MAX;
+	} else {
 		off = insn->off;
+		off_min = S16_MIN;
+		off_max = S16_MAX;
+	}
 
 	if (curr < pos && curr + off + 1 >= end_old)
 		off += delta;