| Message ID | 20240323154652.54572-3-puranjay12@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | bpf,riscv: Add support for BPF Arena | expand |
On 2024/3/23 23:46, Puranjay Mohan wrote: > LLVM generates bpf_addr_space_cast instruction while translating [snip] > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { > + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); combine mv and zextw will be better > + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); > + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); > + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); > + emit_mv(rd, RV_REG_T1, ctx); ditto, but for or and mv
Pu Lehui <pulehui@huawei.com> writes: > On 2024/3/23 23:46, Puranjay Mohan wrote: >> LLVM generates bpf_addr_space_cast instruction while translating > [snip] >> >> /* Convert from ninsns to bytes. */ >> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >> index f51b832eafb6..3c389e75cb96 100644 >> --- a/arch/riscv/net/bpf_jit_comp64.c >> +++ b/arch/riscv/net/bpf_jit_comp64.c >> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >> /* dst = src */ >> case BPF_ALU | BPF_MOV | BPF_X: >> case BPF_ALU64 | BPF_MOV | BPF_X: >> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >> + insn->imm == 1U << 16) { >> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); > combine mv and zextw will be better Do you suggest doing: emit_zextw(RV_REG_T1, rs, ctx); Will do it in next version. >> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >> + emit_mv(rd, RV_REG_T1, ctx); > ditto, but for or and mv How would we combine or and mv? also, we have a beq above and in one case both or and mv should happen, but in other case only mv should happen. Thanks, Puranjay
On Sat, Mar 23, 2024 at 8:47 AM Puranjay Mohan <puranjay12@gmail.com> wrote: > > LLVM generates bpf_addr_space_cast instruction while translating > pointers between native (zero) address space and > __attribute__((address_space(N))). The addr_space=0 is reserved as > bpf_arena address space. > > rY = addr_space_cast(rX, 0, 1) is processed by the verifier and > converted to normal 32-bit move: wX = wY > > rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: > > Here I explain using symbolic language what the JIT is supposed to do: > We have: > src = [src_upper32][src_lower32] // 64 bit src kernel pointer > uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start This is a bit misleading. src_lower32 are always equal to uvm_lower32 and src_upper32 are either zero or uvm_upper32. Hence most of the time llvm doesn't generate this insn, since it knows that upper 32 bit are uvm_upper32. > The JIT has to make the dst reg like following > dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 > dst = [00000000000][00000000000] // if src_lower32 == 0 > > Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> > --- > arch/riscv/net/bpf_jit.h | 1 + > arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ > arch/riscv/net/bpf_jit_core.c | 1 + > 3 files changed, 17 insertions(+) > > diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h > index 8a47da08dd9c..5fc374ed98ea 100644 > --- a/arch/riscv/net/bpf_jit.h > +++ b/arch/riscv/net/bpf_jit.h > @@ -82,6 +82,7 @@ struct rv_jit_context { > unsigned long flags; > int stack_size; > u64 arena_vm_start; > + u64 user_vm_start; > }; > > /* Convert from ninsns to bytes. */ > diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c > index f51b832eafb6..3c389e75cb96 100644 > --- a/arch/riscv/net/bpf_jit_comp64.c > +++ b/arch/riscv/net/bpf_jit_comp64.c > @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, > /* dst = src */ > case BPF_ALU | BPF_MOV | BPF_X: > case BPF_ALU64 | BPF_MOV | BPF_X: > + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && > + insn->imm == 1U << 16) { Let's add a generic helper like insn_is_zext(), call it insn_is_cast_user() ? and use it in all JIT-s ? I should have added it right away when I did x86 part. Sorry. And a comment next to the helper that addr space cast 0->1 is for converting bpf arena pointers to user vma. Hence the name. Same comments for arm64 JIT arena support. pw-bot: cr
On 2024/3/24 0:49, Puranjay Mohan wrote: > Pu Lehui <pulehui@huawei.com> writes: > >> On 2024/3/23 23:46, Puranjay Mohan wrote: >>> LLVM generates bpf_addr_space_cast instruction while translating >> [snip] >>> >>> /* Convert from ninsns to bytes. */ >>> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c >>> index f51b832eafb6..3c389e75cb96 100644 >>> --- a/arch/riscv/net/bpf_jit_comp64.c >>> +++ b/arch/riscv/net/bpf_jit_comp64.c >>> @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, >>> /* dst = src */ >>> case BPF_ALU | BPF_MOV | BPF_X: >>> case BPF_ALU64 | BPF_MOV | BPF_X: >>> + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && >>> + insn->imm == 1U << 16) { >>> + emit_mv(RV_REG_T1, rs, ctx); > + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); >> combine mv and zextw will be better > > Do you suggest doing: > > emit_zextw(RV_REG_T1, rs, ctx); > > Will do it in next version. > >>> + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); >>> + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); >>> + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); >>> + emit_mv(rd, RV_REG_T1, ctx); >> ditto, but for or and mv > > How would we combine or and mv? > also, we have a beq above and in one case both or and mv should happen, > but in other case only mv should happen. > Okay, another branch is that t1 is zero, but not rd. > Thanks, > Puranjay
diff --git a/arch/riscv/net/bpf_jit.h b/arch/riscv/net/bpf_jit.h index 8a47da08dd9c..5fc374ed98ea 100644 --- a/arch/riscv/net/bpf_jit.h +++ b/arch/riscv/net/bpf_jit.h @@ -82,6 +82,7 @@ struct rv_jit_context { unsigned long flags; int stack_size; u64 arena_vm_start; + u64 user_vm_start; }; /* Convert from ninsns to bytes. */ diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c index f51b832eafb6..3c389e75cb96 100644 --- a/arch/riscv/net/bpf_jit_comp64.c +++ b/arch/riscv/net/bpf_jit_comp64.c @@ -1083,6 +1083,16 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, /* dst = src */ case BPF_ALU | BPF_MOV | BPF_X: case BPF_ALU64 | BPF_MOV | BPF_X: + if (BPF_CLASS(insn->code) == BPF_ALU64 && insn->off == BPF_ADDR_SPACE_CAST && + insn->imm == 1U << 16) { + emit_mv(RV_REG_T1, rs, ctx); + emit_zextw(RV_REG_T1, RV_REG_T1, ctx); + emit_imm(rd, (ctx->user_vm_start >> 32) << 32, ctx); + emit(rv_beq(RV_REG_T1, RV_REG_ZERO, 4), ctx); + emit_or(RV_REG_T1, rd, RV_REG_T1, ctx); + emit_mv(rd, RV_REG_T1, ctx); + break; + } if (imm == 1) { /* Special mov32 for zext */ emit_zextw(rd, rd, ctx); @@ -2010,3 +2020,8 @@ bool bpf_jit_supports_ptr_xchg(void) { return true; } + +bool bpf_jit_supports_arena(void) +{ + return true; +} diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 9b6696b1290a..aaef1d0c7c46 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -82,6 +82,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) } ctx->arena_vm_start = bpf_arena_get_kern_vm_start(prog->aux->arena); + ctx->user_vm_start = bpf_arena_get_user_vm_start(prog->aux->arena); ctx->prog = prog; ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL); if (!ctx->offset) {
LLVM generates bpf_addr_space_cast instruction while translating pointers between native (zero) address space and __attribute__((address_space(N))). The addr_space=0 is reserved as bpf_arena address space. rY = addr_space_cast(rX, 0, 1) is processed by the verifier and converted to normal 32-bit move: wX = wY rY = addr_space_cast(rX, 1, 0) has to be converted by JIT: Here I explain using symbolic language what the JIT is supposed to do: We have: src = [src_upper32][src_lower32] // 64 bit src kernel pointer uvm = [uvm_upper32][uvm_lower32] // 64 bit user_vm_start The JIT has to make the dst reg like following dst = [uvm_upper32][src_lower32] // if src_lower32 != 0 dst = [00000000000][00000000000] // if src_lower32 == 0 Signed-off-by: Puranjay Mohan <puranjay12@gmail.com> --- arch/riscv/net/bpf_jit.h | 1 + arch/riscv/net/bpf_jit_comp64.c | 15 +++++++++++++++ arch/riscv/net/bpf_jit_core.c | 1 + 3 files changed, 17 insertions(+)