diff mbox series

[bpf-next,v3] bpf, docs: Add docs on extended 64-bit immediate instructions

Message ID 20230325224305.2157-1-dthaler1968@googlemail.com (mailing list archive)
State Superseded
Delegated to: BPF
Headers show
Series [bpf-next,v3] bpf, docs: Add docs on extended 64-bit immediate instructions | expand

Checks

Context Check Description
netdev/tree_selection success Clearly marked for bpf-next
netdev/apply success Patch already applied to bpf-next
bpf/vmtest-bpf-next-PR fail merge-conflict
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-2 success Logs for build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-3 success Logs for build for aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-4 success Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-5 success Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-6 success Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-7 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-8 success Logs for test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-9 success Logs for test_maps on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-10 success Logs for test_maps on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-11 success Logs for test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-12 success Logs for test_maps on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-13 success Logs for test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-14 success Logs for test_progs on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-15 success Logs for test_progs on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-16 fail Logs for test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-17 success Logs for test_progs on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-18 success Logs for test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-19 success Logs for test_progs_no_alu32 on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-20 fail Logs for test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-21 success Logs for test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-22 success Logs for test_progs_no_alu32 on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-23 success Logs for test_progs_no_alu32_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-24 success Logs for test_progs_no_alu32_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-25 success Logs for test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-26 success Logs for test_progs_no_alu32_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-27 success Logs for test_progs_parallel on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-28 success Logs for test_progs_parallel on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-29 success Logs for test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-30 success Logs for test_progs_parallel on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-31 success Logs for test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-32 success Logs for test_verifier on aarch64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-33 success Logs for test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-34 success Logs for test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-35 success Logs for test_verifier on x86_64 with llvm-16

Commit Message

Dave Thaler March 25, 2023, 10:43 p.m. UTC 
From: Dave Thaler <dthaler@microsoft.com>

Add docs on extended 64-bit immediate instructions, including six instructions
previously undocumented.  Include a brief description of map objects, and variables,
as used by those instructions.

---
V1 -> V2: rebased on top of latest master

V2 -> V3: addressed comments from Alexei

Signed-off-by: Dave Thaler <dthaler@microsoft.com>
---
 Documentation/bpf/instruction-set.rst | 56 +++++++++++++++++++++++----
 Documentation/bpf/linux-notes.rst     | 13 +++++++
 2 files changed, 61 insertions(+), 8 deletions(-)

Comments

David Vernet March 26, 2023, 3:28 a.m. UTC | #1 
On Sat, Mar 25, 2023 at 10:43:05PM +0000, Dave Thaler wrote:
> From: Dave Thaler <dthaler@microsoft.com>
> 
> Add docs on extended 64-bit immediate instructions, including six instructions
> previously undocumented.  Include a brief description of map objects, and variables,
> as used by those instructions.
> 
> ---
> V1 -> V2: rebased on top of latest master
> 
> V2 -> V3: addressed comments from Alexei
> 
> Signed-off-by: Dave Thaler <dthaler@microsoft.com>
> ---
>  Documentation/bpf/instruction-set.rst | 56 +++++++++++++++++++++++----
>  Documentation/bpf/linux-notes.rst     | 13 +++++++
>  2 files changed, 61 insertions(+), 8 deletions(-)
> 
> diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
> index db8789e6969..2c8347d63e7 100644
> --- a/Documentation/bpf/instruction-set.rst
> +++ b/Documentation/bpf/instruction-set.rst
> @@ -385,14 +385,54 @@ and loaded back to ``R0``.
>  -----------------------------
>  
>  Instructions with the ``BPF_IMM`` 'mode' modifier use the wide instruction
> -encoding for an extra imm64 value.
> -
> -There is currently only one such instruction.
> -
> -``BPF_LD | BPF_DW | BPF_IMM`` means::
> -
> -  dst = imm64
> -
> +encoding defined in `Instruction encoding`_, and use the 'src' field of the
> +basic instruction to hold an opcode subtype.
> +
> +The following instructions are defined, and use additional concepts defined below:

nit: Perhaps this is a bit clearer? Wdyt?

The following opcode subtypes are defined for `BPF_IMM | BPF_DQ |
BPF_LD` instructions, with new terms such as "map" defined further
below:

> +
> +=========================  ======  ===  =========================================  ===========  ==============
> +opcode construction        opcode  src  pseudocode                                 imm type     dst type
> +=========================  ======  ===  =========================================  ===========  ==============
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x0  dst = imm64                                integer      integer
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x1  dst = map_by_fd(imm)                       map fd       map
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x2  dst = map_val(map_by_fd(imm)) + next_imm   map fd       data pointer
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x3  dst = var_addr(imm)                        variable id  data pointer
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x4  dst = code_addr(imm)                       integer      code pointer
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x5  dst = map_by_idx(imm)                      map index    map
> +BPF_IMM | BPF_DW | BPF_LD  0x18    0x6  dst = map_val(map_by_idx(imm)) + next_imm  map index    data pointer
> +=========================  ======  ===  =========================================  ===========  ==============
> +
> +where
> +
> +* map_by_fd(imm) means to convert a 32-bit POSIX file descriptor into an address of a map object (see `Map objects`_)
> +* map_by_idx(imm) means to convert a 32-bit index into an address of a map object
> +* map_val(map) gets the address of the first value in a given map object
> +* var_addr(imm) gets the address of a platform variable (see `Platform Variables`_) with a given id
> +* code_addr(imm) gets the address of the instruction at a specified relative offset in number of (64-bit) instructions
> +* the 'imm type' can be used by disassemblers for display
> +* the 'dst type' can be used for verification and JIT compilation purposes
> +
> +Map objects
> +~~~~~~~~~~~
> +
> +Maps are shared memory regions accessible by eBPF programs on some platforms, where we use the term "map object"
> +to refer to an object containing the data and metadata (e.g., size) about the memory region.
> +A map can have various semantics as defined in a separate document, and may or may not have a single
> +contiguous memory region, but the 'map_val(map)' is currently only defined for maps that do have a single
> +contiguous memory region.
> +
> +Each map object can have a POSIX file descriptor (fd) if supported by the platform,
> +where 'map_by_fd(imm)' means to get the map with the specified file descriptor.
> +Each BPF program can also be defined to use a set of maps associated with the program
> +at load time, and 'map_by_idx(imm)' means to get the map with the given index in the set
> +associated with the BPF program containing the instruction.
> +
> +Platform Variables
> +~~~~~~~~~~~~~~~~~~
> +
> +Platform variables are memory regions, identified by integer ids, exposed by the runtime and accessible by BPF programs on
> +some platforms.  The 'var_addr(imm)' operation means to get the address of the memory region
> +identified by the given id.
>  
>  Legacy BPF Packet access instructions
>  -------------------------------------
> diff --git a/Documentation/bpf/linux-notes.rst b/Documentation/bpf/linux-notes.rst
> index 956b0c86699..2d161467105 100644
> --- a/Documentation/bpf/linux-notes.rst
> +++ b/Documentation/bpf/linux-notes.rst
> @@ -12,6 +12,19 @@ Byte swap instructions
>  
>  ``BPF_FROM_LE`` and ``BPF_FROM_BE`` exist as aliases for ``BPF_TO_LE`` and ``BPF_TO_BE`` respectively.
>  
> +Map objects
> +===========
> +
> +Linux only supports the 'map_val(map)' operation on array maps with a single element.
> +
> +Linux uses an fd_array to store maps associated with a BPF program. Thus,
> +map_by_index(index) uses the fd at that index in the array.
> +
> +Variables
> +=========
> +
> +Linux uses BTF ids to identify variables.

Not quite sure exactly what this means. Linux uses BTF ids to identify
_types_, right? This doesn't seem like something that needs to be
specified as Linux specific either, even if it's not yet supported
elsewhere. Certain legacy things such as Linux-specific helpers make
sense, but not sure about BTF ids.

> +
>  Legacy BPF Packet access instructions
>  =====================================
>  
> -- 
> 2.33.4
> 
> -- 
> Bpf mailing list
> Bpf@ietf.org
> https://www.ietf.org/mailman/listinfo/bpf
David Vernet March 26, 2023, 5 a.m. UTC | #2 
On Sat, Mar 25, 2023 at 10:28:39PM -0500, David Vernet wrote:
> On Sat, Mar 25, 2023 at 10:43:05PM +0000, Dave Thaler wrote:
> > From: Dave Thaler <dthaler@microsoft.com>
> > 
> > Add docs on extended 64-bit immediate instructions, including six instructions
> > previously undocumented.  Include a brief description of map objects, and variables,
> > as used by those instructions.
> > 
> > ---
> > V1 -> V2: rebased on top of latest master
> > 
> > V2 -> V3: addressed comments from Alexei
> > 
> > Signed-off-by: Dave Thaler <dthaler@microsoft.com>
> > ---
> >  Documentation/bpf/instruction-set.rst | 56 +++++++++++++++++++++++----
> >  Documentation/bpf/linux-notes.rst     | 13 +++++++
> >  2 files changed, 61 insertions(+), 8 deletions(-)
> > 
> > diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
> > index db8789e6969..2c8347d63e7 100644
> > --- a/Documentation/bpf/instruction-set.rst
> > +++ b/Documentation/bpf/instruction-set.rst
> > @@ -385,14 +385,54 @@ and loaded back to ``R0``.
> >  -----------------------------
> >  
> >  Instructions with the ``BPF_IMM`` 'mode' modifier use the wide instruction
> > -encoding for an extra imm64 value.
> > -
> > -There is currently only one such instruction.
> > -
> > -``BPF_LD | BPF_DW | BPF_IMM`` means::
> > -
> > -  dst = imm64
> > -
> > +encoding defined in `Instruction encoding`_, and use the 'src' field of the
> > +basic instruction to hold an opcode subtype.
> > +
> > +The following instructions are defined, and use additional concepts defined below:
> 
> nit: Perhaps this is a bit clearer? Wdyt?
> 
> The following opcode subtypes are defined for `BPF_IMM | BPF_DQ |
> BPF_LD` instructions, with new terms such as "map" defined further
> below:

Oops, meant to say `BPF_IMM | BPF_DW | BPF_LD`

> 
> > +
> > +=========================  ======  ===  =========================================  ===========  ==============
> > +opcode construction        opcode  src  pseudocode                                 imm type     dst type
> > +=========================  ======  ===  =========================================  ===========  ==============
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x0  dst = imm64                                integer      integer
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x1  dst = map_by_fd(imm)                       map fd       map
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x2  dst = map_val(map_by_fd(imm)) + next_imm   map fd       data pointer
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x3  dst = var_addr(imm)                        variable id  data pointer
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x4  dst = code_addr(imm)                       integer      code pointer
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x5  dst = map_by_idx(imm)                      map index    map
> > +BPF_IMM | BPF_DW | BPF_LD  0x18    0x6  dst = map_val(map_by_idx(imm)) + next_imm  map index    data pointer
> > +=========================  ======  ===  =========================================  ===========  ==============
> > +
> > +where
> > +
> > +* map_by_fd(imm) means to convert a 32-bit POSIX file descriptor into an address of a map object (see `Map objects`_)
> > +* map_by_idx(imm) means to convert a 32-bit index into an address of a map object
> > +* map_val(map) gets the address of the first value in a given map object
> > +* var_addr(imm) gets the address of a platform variable (see `Platform Variables`_) with a given id
> > +* code_addr(imm) gets the address of the instruction at a specified relative offset in number of (64-bit) instructions
> > +* the 'imm type' can be used by disassemblers for display
> > +* the 'dst type' can be used for verification and JIT compilation purposes
> > +
> > +Map objects
> > +~~~~~~~~~~~
> > +
> > +Maps are shared memory regions accessible by eBPF programs on some platforms, where we use the term "map object"
> > +to refer to an object containing the data and metadata (e.g., size) about the memory region.
> > +A map can have various semantics as defined in a separate document, and may or may not have a single
> > +contiguous memory region, but the 'map_val(map)' is currently only defined for maps that do have a single
> > +contiguous memory region.
> > +
> > +Each map object can have a POSIX file descriptor (fd) if supported by the platform,
> > +where 'map_by_fd(imm)' means to get the map with the specified file descriptor.
> > +Each BPF program can also be defined to use a set of maps associated with the program
> > +at load time, and 'map_by_idx(imm)' means to get the map with the given index in the set
> > +associated with the BPF program containing the instruction.
> > +
> > +Platform Variables
> > +~~~~~~~~~~~~~~~~~~
> > +
> > +Platform variables are memory regions, identified by integer ids, exposed by the runtime and accessible by BPF programs on
> > +some platforms.  The 'var_addr(imm)' operation means to get the address of the memory region
> > +identified by the given id.
> >  
> >  Legacy BPF Packet access instructions
> >  -------------------------------------
> > diff --git a/Documentation/bpf/linux-notes.rst b/Documentation/bpf/linux-notes.rst
> > index 956b0c86699..2d161467105 100644
> > --- a/Documentation/bpf/linux-notes.rst
> > +++ b/Documentation/bpf/linux-notes.rst
> > @@ -12,6 +12,19 @@ Byte swap instructions
> >  
> >  ``BPF_FROM_LE`` and ``BPF_FROM_BE`` exist as aliases for ``BPF_TO_LE`` and ``BPF_TO_BE`` respectively.
> >  
> > +Map objects
> > +===========
> > +
> > +Linux only supports the 'map_val(map)' operation on array maps with a single element.
> > +
> > +Linux uses an fd_array to store maps associated with a BPF program. Thus,
> > +map_by_index(index) uses the fd at that index in the array.
> > +
> > +Variables
> > +=========
> > +
> > +Linux uses BTF ids to identify variables.
> 
> Not quite sure exactly what this means. Linux uses BTF ids to identify
> _types_, right? This doesn't seem like something that needs to be
> specified as Linux specific either, even if it's not yet supported
> elsewhere. Certain legacy things such as Linux-specific helpers make
> sense, but not sure about BTF ids.

Ok, I read over this some more and I think I understand what the
intention was here. You were specifying that for the following
instruction encoding:

=========================  ======  ===  ===================  ===========  ============
opcode construction        opcode  src  pseudocode           imm type     dst type
=========================  ======  ===  ===================  ===========  ============
BPF_IMM | BPF_DW | BPF_LD  0x18    0x3  dst = var_addr(imm)  variable id  data pointer

imm is a BTF ID, and it is Linux-specific that var_addr(imm) happens to
be mapping a BTF ID to an actual variable address that is stored in dst.
Is that right? If so, I think this section needs to include a bit more
context so that it makes sense on its own. What about something like
this:

The following 64-bit immediate instruction specifies that a variable
address, which corresponds to some integer stored in the immediate
field, should be loaded:

=========================  ======  ===  =========================================  ===========  ==============
opcode construction        opcode  src  pseudocode                                 imm type     dst type
=========================  ======  ===  =========================================  ===========  ==============
BPF_IMM | BPF_DW | BPF_LD  0x18    0x3  dst = var_addr(imm)                        variable id  data pointer

On Linux, this integer is a BTF ID.

> > +
> >  Legacy BPF Packet access instructions
> >  =====================================
> >  
> > -- 
> > 2.33.4
> > 
> > -- 
> > Bpf mailing list
> > Bpf@ietf.org
> > https://www.ietf.org/mailman/listinfo/bpf
diff mbox series

Patch

diff --git a/Documentation/bpf/instruction-set.rst b/Documentation/bpf/instruction-set.rst
index db8789e6969..2c8347d63e7 100644
--- a/Documentation/bpf/instruction-set.rst
+++ b/Documentation/bpf/instruction-set.rst
@@ -385,14 +385,54 @@  and loaded back to ``R0``.
 -----------------------------
 
 Instructions with the ``BPF_IMM`` 'mode' modifier use the wide instruction
-encoding for an extra imm64 value.
-
-There is currently only one such instruction.
-
-``BPF_LD | BPF_DW | BPF_IMM`` means::
-
-  dst = imm64
-
+encoding defined in `Instruction encoding`_, and use the 'src' field of the
+basic instruction to hold an opcode subtype.
+
+The following instructions are defined, and use additional concepts defined below:
+
+=========================  ======  ===  =========================================  ===========  ==============
+opcode construction        opcode  src  pseudocode                                 imm type     dst type
+=========================  ======  ===  =========================================  ===========  ==============
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x0  dst = imm64                                integer      integer
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x1  dst = map_by_fd(imm)                       map fd       map
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x2  dst = map_val(map_by_fd(imm)) + next_imm   map fd       data pointer
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x3  dst = var_addr(imm)                        variable id  data pointer
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x4  dst = code_addr(imm)                       integer      code pointer
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x5  dst = map_by_idx(imm)                      map index    map
+BPF_IMM | BPF_DW | BPF_LD  0x18    0x6  dst = map_val(map_by_idx(imm)) + next_imm  map index    data pointer
+=========================  ======  ===  =========================================  ===========  ==============
+
+where
+
+* map_by_fd(imm) means to convert a 32-bit POSIX file descriptor into an address of a map object (see `Map objects`_)
+* map_by_idx(imm) means to convert a 32-bit index into an address of a map object
+* map_val(map) gets the address of the first value in a given map object
+* var_addr(imm) gets the address of a platform variable (see `Platform Variables`_) with a given id
+* code_addr(imm) gets the address of the instruction at a specified relative offset in number of (64-bit) instructions
+* the 'imm type' can be used by disassemblers for display
+* the 'dst type' can be used for verification and JIT compilation purposes
+
+Map objects
+~~~~~~~~~~~
+
+Maps are shared memory regions accessible by eBPF programs on some platforms, where we use the term "map object"
+to refer to an object containing the data and metadata (e.g., size) about the memory region.
+A map can have various semantics as defined in a separate document, and may or may not have a single
+contiguous memory region, but the 'map_val(map)' is currently only defined for maps that do have a single
+contiguous memory region.
+
+Each map object can have a POSIX file descriptor (fd) if supported by the platform,
+where 'map_by_fd(imm)' means to get the map with the specified file descriptor.
+Each BPF program can also be defined to use a set of maps associated with the program
+at load time, and 'map_by_idx(imm)' means to get the map with the given index in the set
+associated with the BPF program containing the instruction.
+
+Platform Variables
+~~~~~~~~~~~~~~~~~~
+
+Platform variables are memory regions, identified by integer ids, exposed by the runtime and accessible by BPF programs on
+some platforms.  The 'var_addr(imm)' operation means to get the address of the memory region
+identified by the given id.
 
 Legacy BPF Packet access instructions
 -------------------------------------
diff --git a/Documentation/bpf/linux-notes.rst b/Documentation/bpf/linux-notes.rst
index 956b0c86699..2d161467105 100644
--- a/Documentation/bpf/linux-notes.rst
+++ b/Documentation/bpf/linux-notes.rst
@@ -12,6 +12,19 @@  Byte swap instructions
 
 ``BPF_FROM_LE`` and ``BPF_FROM_BE`` exist as aliases for ``BPF_TO_LE`` and ``BPF_TO_BE`` respectively.
 
+Map objects
+===========
+
+Linux only supports the 'map_val(map)' operation on array maps with a single element.
+
+Linux uses an fd_array to store maps associated with a BPF program. Thus,
+map_by_index(index) uses the fd at that index in the array.
+
+Variables
+=========
+
+Linux uses BTF ids to identify variables.
+
 Legacy BPF Packet access instructions
 =====================================