@@ -249,39 +249,65 @@ Where size is one of: ``BPF_B``, ``BPF_H``, ``BPF_W``, or ``BPF_DW``.
Atomic operations
-----------------
-eBPF includes atomic operations, which use the immediate field for extra
-encoding::
+Atomic operations are operations that operate on memory and can not be
+interrupted or corrupted by other access to the same memory region
+by other eBPF programs or means outside of this specification.
- .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_W | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg
- .imm = BPF_ADD, .code = BPF_ATOMIC | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg
+All atomic operations supported by eBPF are encoded as store operations
+that use the ``BPF_ATOMIC`` mode modifier as follows:
-The basic atomic operations supported are::
+ * ``BPF_ATOMIC | BPF_W | BPF_STX`` for 32-bit operations
+ * ``BPF_ATOMIC | BPF_DW | BPF_STX`` for 64-bit operations
+ * 8-bit and 16-bit wide atomic operations are not supported.
- BPF_ADD
- BPF_AND
- BPF_OR
- BPF_XOR
+The imm field is used to encode the actual atomic operation.
+Simple atomic operation use a subset of the values defined to encode
+arithmetic operations in the imm field to encode the atomic operation:
-Each having equivalent semantics with the ``BPF_ADD`` example, that is: the
-memory location addresed by ``dst_reg + off`` is atomically modified, with
-``src_reg`` as the other operand. If the ``BPF_FETCH`` flag is set in the
-immediate, then these operations also overwrite ``src_reg`` with the
-value that was in memory before it was modified.
+ ======== ===== ===========
+ imm value description
+ ======== ===== ===========
+ BPF_ADD 0x00 atomic add
+ BPF_OR 0x40 atomic or
+ BPF_AND 0x50 atomic and
+ BPF_XOR 0xa0 atomic xor
+ ======== ===== ===========
-The more special operations are::
- BPF_XCHG
+``BPF_ATOMIC | BPF_W | BPF_STX`` with imm = BPF_ADD means::
-This atomically exchanges ``src_reg`` with the value addressed by ``dst_reg +
-off``. ::
+ *(u32 *)(dst_reg + off16) += src_reg
- BPF_CMPXCHG
+``BPF_ATOMIC | BPF_DW | BPF_STX`` with imm = BPF ADD means::
-This atomically compares the value addressed by ``dst_reg + off`` with
-``R0``. If they match it is replaced with ``src_reg``. In either case, the
-value that was there before is zero-extended and loaded back to ``R0``.
+ *(u64 *)(dst_reg + off16) += src_reg
-Note that 1 and 2 byte atomic operations are not supported.
+``BPF_XADD`` is a deprecated name for ``BPF_ATOMIC | BPF_ADD``.
+
+In addition to the simple atomic operations, there also is a modifier and
+two complex atomic operations:
+
+ =========== ================ ===========================
+ imm value description
+ =========== ================ ===========================
+ BPF_FETCH 0x01 modifier: return old value
+ BPF_XCHG 0xe0 | BPF_FETCH atomic exchange
+ BPF_CMPXCHG 0xf0 | BPF_FETCH atomic compare and exchange
+ =========== ================ ===========================
+
+The ``BPF_FETCH`` modifier is optional for simple atomic operations, and
+always set for the complex atomic operations. If the ``BPF_FETCH`` flag
+is set, then the operation also overwrites ``src_reg`` with the value that
+was in memory before it was modified.
+
+The ``BPF_XCHG`` operation atomically exchanges ``src_reg`` with the value
+addressed by ``dst_reg + off``.
+
+The ``BPF_CMPXCHG`` operation atomically compares the value addressed by
+``dst_reg + off`` with ``R0``. If they match, the value addressed by
+``dst_reg + off`` is replaced with ``src_reg``. In either case, the
+value that was at ``dst_reg + off`` before the operation is zero-extended
+and loaded back to ``R0``.
Clang can generate atomic instructions by default when ``-mcpu=v3`` is
enabled. If a lower version for ``-mcpu`` is set, the only atomic instruction
@@ -289,10 +315,6 @@ Clang can generate is ``BPF_ADD`` *without* ``BPF_FETCH``. If you need to enable
the atomics features, while keeping a lower ``-mcpu`` version, you can use
``-Xclang -target-feature -Xclang +alu32``.
-You may encounter ``BPF_XADD`` - this is a legacy name for ``BPF_ATOMIC``,
-referring to the exclusive-add operation encoded when the immediate field is
-zero.
-
64-bit immediate instructions
-----------------------------
Use proper tables and RST markup to document the atomic instructions in a structured way. Signed-off-by: Christoph Hellwig <hch@lst.de> --- Documentation/bpf/instruction-set.rst | 76 +++++++++++++++++---------- 1 file changed, 49 insertions(+), 27 deletions(-)