| Message ID | 20221117154446.3684330-1-mtahhan@redhat.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | BPF |
| Headers | show |
| Series | [bpf-next,v2,1/1] docs: BPF_MAP_TYPE_XSKMAP | expand |
Hi Maryam, On Thu, 17 Nov 2022 10:44:46 -0500, mtahhan@redhat.com wrote: > From: Maryam Tahhan <mtahhan@redhat.com> > > Add documentation for BPF_MAP_TYPE_XSKMAP > including kernel version introduced, usage > and examples. > > Signed-off-by: Maryam Tahhan <mtahhan@redhat.com> > > --- > v2: > - Fixed typos + incorrect return type references. > - Adjusted examples to use __u32 and fixed references to key_size. > - Changed `AF_XDP socket` references to XSK. > - Added note re map key and value size. > --- > Documentation/bpf/map_xskmap.rst | 167 +++++++++++++++++++++++++++++++ > 1 file changed, 167 insertions(+) > create mode 100644 Documentation/bpf/map_xskmap.rst > > diff --git a/Documentation/bpf/map_xskmap.rst b/Documentation/bpf/map_xskmap.rst > new file mode 100644 [...] > +Kernel BPF > +---------- > +.. c:function:: > + long bpf_redirect_map(struct bpf_map *map, u32 key, u64 flags) > + > + Redirect the packet to the endpoint referenced by ``map`` at index ``key``. > + For ``BPF_MAP_TYPE_XSKMAP`` this map contains references to XSK FDs > + for sockets attached to a netdev's queues. > + > + .. note:: > + If the map is empty at an index, the packet is dropped. This means that it is > + necessary to have an XDP program loaded with at least one XSK in the > + XSKMAP to be able to get any traffic to user space through the socket. > + > +.. c:function:: > + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) > + > + XSK entry references of type ``struct xdp_sock *`` can be retrieved using the > + ``bpf_map_lookup_elem()`` helper. > + > +Userspace > +--------- > +.. note:: > + XSK entries can only be updated/deleted from user space and not from > + an eBPF program. Trying to call these functions from a kernel eBPF program will > + result in the program failing to load and a verifier warning. > + > +.. c:function:: > + int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags) > + > + XSK entries can be added or updated using the ``bpf_map_update_elem()`` > + helper. The ``key`` parameter is equal to the queue_id of the queue the XSK > + is attaching to. And the ``value`` parameter is the FD value of that socket. > + > + Under the hood, the XSKMAP update function uses the XSK FD value to retrieve the > + associated ``struct xdp_sock`` instance. > + > + The flags argument can be one of the following: > + > + - BPF_ANY: Create a new element or update an existing element. > + - BPF_NOEXIST: Create a new element only if it did not exist. > + - BPF_EXIST: Update an existing element. > + > +.. c:function:: > + int bpf_map_lookup_elem(int fd, const void *key, void *value) > + So you have two declarations of bpf_map_lookup_elem() in map_xskmap.rst. This will cause "make htmldocs" with Sphinx >=3.1 to emit a warning of: /linux/Documentation/bpf/map_xskmap.rst:100: WARNING: Duplicate C declaration, also defined at map_xskmap:71. Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value)'. , in addition to a bunch of similar warnings observed at bpf-next: /linux/Documentation/bpf/map_cpumap.rst:50: WARNING: Duplicate C declaration, also defined at map_array:43. Declaration is '.. c:function:: int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags);'. /linux/Documentation/bpf/map_cpumap.rst:72: WARNING: Duplicate C declaration, also defined at map_array:35. Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value);'. /linux/Documentation/bpf/map_hash.rst:37: WARNING: Duplicate C declaration, also defined at map_array:43. Declaration is '.. c:function:: long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags)'. ... [bunch of similar warnings] You might want to say you don't care, but they would annoy those who do test "make htmldocs". So let me explain why sphinx complains. C domain declarations in kernel documentation are for kernel APIs. By default, c:function declarations belong to the top-level namespace, which is intended for kernel APIs. IIUC, most APIs described in map*.rst files don't belong to kernel. So I think the way to go is to use the c:namespace directive. See: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#namespacing As mentioned there, namespacing works with Sphinx >=3.1. Currently, kernel documentation build scripts support only the "c:namespace" directive, which means you can't switch namespaces in the middle of a .rst file. This limitation comes from the fact that Sphinx 1.7.9 is still in the list for htmldocs at the moment and build scripts emulate namespacing for Sphinx <3.1 in a limited way. So please avoid putting function declarations of the same name in a .rst file. The other duplicate warnings shown above can be silenced by the change attached below. It is only as a suggestion and I'm not putting a S-o-b tag. Hope this helps, Akira -------- diff --git a/Documentation/bpf/map_array.rst b/Documentation/bpf/map_array.rst index 97bb80333254..68545702ca78 100644 --- a/Documentation/bpf/map_array.rst +++ b/Documentation/bpf/map_array.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_ARRAY + ================================================ BPF_MAP_TYPE_ARRAY and BPF_MAP_TYPE_PERCPU_ARRAY ================================================ diff --git a/Documentation/bpf/map_cpumap.rst b/Documentation/bpf/map_cpumap.rst index 61a797a86342..25e05d14ec82 100644 --- a/Documentation/bpf/map_cpumap.rst +++ b/Documentation/bpf/map_cpumap.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_CPUMAP + =================== BPF_MAP_TYPE_CPUMAP =================== diff --git a/Documentation/bpf/map_hash.rst b/Documentation/bpf/map_hash.rst index e85120878b27..3ac93ccf2b0e 100644 --- a/Documentation/bpf/map_hash.rst +++ b/Documentation/bpf/map_hash.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_HASH + =============================================== BPF_MAP_TYPE_HASH, with PERCPU and LRU Variants =============================================== diff --git a/Documentation/bpf/map_lpm_trie.rst b/Documentation/bpf/map_lpm_trie.rst index 31be1aa7ba2c..c934c3e2bcb7 100644 --- a/Documentation/bpf/map_lpm_trie.rst +++ b/Documentation/bpf/map_lpm_trie.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_LPM_TRIE + ===================== BPF_MAP_TYPE_LPM_TRIE ===================== diff --git a/Documentation/bpf/map_of_maps.rst b/Documentation/bpf/map_of_maps.rst index 07212b9227a9..f59cd0e3a72c 100644 --- a/Documentation/bpf/map_of_maps.rst +++ b/Documentation/bpf/map_of_maps.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_OF_MAPS + ======================================================== BPF_MAP_TYPE_ARRAY_OF_MAPS and BPF_MAP_TYPE_HASH_OF_MAPS ======================================================== diff --git a/Documentation/bpf/map_queue_stack.rst b/Documentation/bpf/map_queue_stack.rst index f20e31a647b9..abc8ed569900 100644 --- a/Documentation/bpf/map_queue_stack.rst +++ b/Documentation/bpf/map_queue_stack.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0-only .. Copyright (C) 2022 Red Hat, Inc. +.. c:namespace:: BPF.MAP_QUEUE_STACK + ========================================= BPF_MAP_TYPE_QUEUE and BPF_MAP_TYPE_STACK ========================================= --
Akira Yokosawa <akiyks@gmail.com> writes: > > So you have two declarations of bpf_map_lookup_elem() in map_xskmap.rst. > > This will cause "make htmldocs" with Sphinx >=3.1 to emit a warning of: > > /linux/Documentation/bpf/map_xskmap.rst:100: WARNING: Duplicate C declaration, also defined at map_xskmap:71. > Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value)'. > > , in addition to a bunch of similar warnings observed at bpf-next: > > /linux/Documentation/bpf/map_cpumap.rst:50: WARNING: Duplicate C declaration, also defined at map_array:43. > Declaration is '.. c:function:: int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags);'. > /linux/Documentation/bpf/map_cpumap.rst:72: WARNING: Duplicate C declaration, also defined at map_array:35. > Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value);'. > /linux/Documentation/bpf/map_hash.rst:37: WARNING: Duplicate C declaration, also defined at map_array:43. > Declaration is '.. c:function:: long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags)'. > ... [bunch of similar warnings] That's unfortunate, and I'm responsible for some of those. Not sure how we'd know to check for warnings with Sphinx >= 3.1 when Documentation/doc-guide/sphinx.rst and Documentation/sphinx/requirements.txt both specify version 2.4.4 > You might want to say you don't care, but they would annoy those > who do test "make htmldocs". > > So let me explain why sphinx complains. > > C domain declarations in kernel documentation are for kernel APIs. > By default, c:function declarations belong to the top-level namespace, > which is intended for kernel APIs. > > IIUC, most APIs described in map*.rst files don't belong to kernel. > So I think the way to go is to use the c:namespace directive. > > See: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#namespacing > > As mentioned there, namespacing works with Sphinx >=3.1. > Currently, kernel documentation build scripts support only the > "c:namespace" directive, which means you can't switch namespaces in the > middle of a .rst file. This limitation comes from the fact that Sphinx > 1.7.9 is still in the list for htmldocs at the moment and build scripts > emulate namespacing for Sphinx <3.1 in a limited way. What's the reason for keeping support for Sphinx 1.7.9 and pinning to 2.4.4 in Documentation/sphinx/requirements.txt if we want to support Sphinx >= 3.1? Given that the latest Sphinx release is 5.3.0, and Python 2 support was dropped in Sphinx 2.0.0 it seems that we need to have a higher minimum version and a higher default version. > So please avoid putting function declarations of the same name in > a .rst file. The same function name, with different signature gets used as a BPF helper and as a userspace function. We'd really like to be able to document the semantics of both for a given BPF map type, all on the same page. Is there a better way for us to highlight the function signature, without using the c:function:: directive, since they're not really function declarations? > The other duplicate warnings shown above can be silenced by the > change attached below. It is only as a suggestion and I'm not putting > a S-o-b tag. > > Hope this helps, > > Akira
Hi Donald, On Fri, 18 Nov 2022 09:33:21 +0000, Donald Hunter wrote: > Akira Yokosawa <akiyks@gmail.com> writes: >> >> So you have two declarations of bpf_map_lookup_elem() in map_xskmap.rst. >> >> This will cause "make htmldocs" with Sphinx >=3.1 to emit a warning of: >> >> /linux/Documentation/bpf/map_xskmap.rst:100: WARNING: Duplicate C declaration, also defined at map_xskmap:71. >> Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value)'. >> >> , in addition to a bunch of similar warnings observed at bpf-next: >> >> /linux/Documentation/bpf/map_cpumap.rst:50: WARNING: Duplicate C declaration, also defined at map_array:43. >> Declaration is '.. c:function:: int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags);'. >> /linux/Documentation/bpf/map_cpumap.rst:72: WARNING: Duplicate C declaration, also defined at map_array:35. >> Declaration is '.. c:function:: int bpf_map_lookup_elem(int fd, const void *key, void *value);'. >> /linux/Documentation/bpf/map_hash.rst:37: WARNING: Duplicate C declaration, also defined at map_array:43. >> Declaration is '.. c:function:: long bpf_map_update_elem(struct bpf_map *map, const void *key, const void *value, u64 flags)'. >> ... [bunch of similar warnings] > > That's unfortunate, and I'm responsible for some of those. Not sure how > we'd know to check for warnings with Sphinx >= 3.1 when > Documentation/doc-guide/sphinx.rst and > Documentation/sphinx/requirements.txt both specify version 2.4.4 Sorry, I didn't mean to blame anyone. :-/ I think I need to share some background. Please read on. > >> You might want to say you don't care, but they would annoy those >> who do test "make htmldocs". >> >> So let me explain why sphinx complains. >> >> C domain declarations in kernel documentation are for kernel APIs. >> By default, c:function declarations belong to the top-level namespace, >> which is intended for kernel APIs. >> >> IIUC, most APIs described in map*.rst files don't belong to kernel. >> So I think the way to go is to use the c:namespace directive. >> >> See: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#namespacing >> >> As mentioned there, namespacing works with Sphinx >=3.1. >> Currently, kernel documentation build scripts support only the >> "c:namespace" directive, which means you can't switch namespaces in the >> middle of a .rst file. This limitation comes from the fact that Sphinx >> 1.7.9 is still in the list for htmldocs at the moment and build scripts >> emulate namespacing for Sphinx <3.1 in a limited way. > > What's the reason for keeping support for Sphinx 1.7.9 and pinning to > 2.4.4 in Documentation/sphinx/requirements.txt if we want to support > Sphinx >= 3.1? Given that the latest Sphinx release is 5.3.0, and Python > 2 support was dropped in Sphinx 2.0.0 it seems that we need to have a > higher minimum version and a higher default version. Middle term, progressing to recent versions of Sphinx is highly hoped as we'd really like to utilize the namespacing capability in its full strength. Unfortunately, as is mentioned in ./scripts/sphinx_pre: Please note that Sphinx >= 3.0 will currently produce false-positive warning when the same name is used for more than one type (functions, structs, enums,...). This is known Sphinx bug. For more details, see: https://github.com/sphinx-doc/sphinx/pull/8313 , later Sphinx emits a dozen of false positive warnings of duplicates. Hence we stick to Sphinx 2.4.4 in doc-guide documents. 2.4.4 or 1.7.9 is good enough for catching easy-to-fix errors in .rst files. See https://lore.kernel.org/linux-doc/20220702122311.358c0219@sal.lan/ for Mauro's thoughts on these issues. What I'm doing now is manually distinguishing false and real positives and asking fixes of the latter, so that transition to later Sphinx versions would be smooth as possible. On of such fixes is commit c18c20f16219 ("mm, slab: remove duplicate kernel-doc comment for ksize()") which landed v6.1-rc5. > >> So please avoid putting function declarations of the same name in >> a .rst file. > > The same function name, with different signature gets used as a BPF > helper and as a userspace function. We'd really like to be able to > document the semantics of both for a given BPF map type, all on the same > page. > > Is there a better way for us to highlight the function signature, > without using the c:function:: directive, since they're not really > function declarations? If you don't feel like bothering with namespacing, your option would be to use the code-block directive. I see a couple of APIs presented as plain literal blocks in map_cgrp_storage.rst. If you change them to "code-block"s as follows: diff --git a/Documentation/bpf/map_cgrp_storage.rst b/Documentation/bpf/map_cgrp_storage.rst index 5d3f603efffa..be31f250453b 100644 --- a/Documentation/bpf/map_cgrp_storage.rst +++ b/Documentation/bpf/map_cgrp_storage.rst @@ -18,14 +18,18 @@ Usage ===== The map key must be ``sizeof(int)`` representing a cgroup fd. -To access the storage in a program, use ``bpf_cgrp_storage_get``:: +To access the storage in a program, use ``bpf_cgrp_storage_get``: + +.. code-block:: c void *bpf_cgrp_storage_get(struct bpf_map *map, struct cgroup *cgroup, void *value, u64 flags) ``flags`` could be 0 or ``BPF_LOCAL_STORAGE_GET_F_CREATE`` which indicates that a new local storage will be created if one does not exist. -The local storage can be removed with ``bpf_cgrp_storage_delete``:: +The local storage can be removed with ``bpf_cgrp_storage_delete``: + +.. code-block:: c long bpf_cgrp_storage_delete(struct bpf_map *map, struct cgroup *cgroup) ------------ , you can get highlighted signatures. I'm not sure if you like the way they are highlighted, though. Hope this helps. Please feel free to ask if you have further questions. Akira > >> The other duplicate warnings shown above can be silenced by the >> change attached below. It is only as a suggestion and I'm not putting >> a S-o-b tag. >> >> Hope this helps, >> >> Akira
diff --git a/Documentation/bpf/map_xskmap.rst b/Documentation/bpf/map_xskmap.rst new file mode 100644 index 000000000000..4fd2f58aa629 --- /dev/null +++ b/Documentation/bpf/map_xskmap.rst @@ -0,0 +1,167 @@ +.. SPDX-License-Identifier: GPL-2.0-only +.. Copyright (C) 2022 Red Hat, Inc. + +=================== +BPF_MAP_TYPE_XSKMAP +=================== + +.. note:: + - ``BPF_MAP_TYPE_XSKMAP`` was introduced in kernel version 4.18 + +The ``BPF_MAP_TYPE_XSKMAP`` is used as a backend map for XDP BPF helper +call ``bpf_redirect_map()`` and ``XDP_REDIRECT`` action, like 'devmap' and 'cpumap'. +This map type redirects raw XDP frames to AF_XDP sockets (XSKs). An AF_XDP socket +binds to a single netdev queue. A mapping of XSKs to queues is shown below: + +.. code-block:: none + + +---------------------------------------------------+ + | xsk A | xsk B | xsk C |<---+ Userspace + =========================================================|========== + | Queue 0 | Queue 1 | Queue 2 | | Kernel + +---------------------------------------------------+ | + | Netdev eth0 | | + +---------------------------------------------------+ | + | +=============+ | | + | | key | xsk | | | + | +---------+ +=============+ | | + | | | | 0 | xsk A | | | + | | | +-------------+ | | + | | | | 1 | xsk B | | | + | | eBPF |-- redirect -->+-------------+-------------+ + | | prog | | 2 | xsk C | | + | | | +-------------+ | + | | | | + | | | | + | +---------+ | + | | + +---------------------------------------------------+ + +.. note:: + An AF_XDP socket that is bound to a certain <netdev/queue_id> will *only* + accept XDP frames from that <netdev/queue_id>. If an XDP program tries to redirect + from a <netdev/queue_id> other than what the socket is bound to, the frame will + not be received on the socket. + +Typically a XSKMAP is created per netdev. This map contains an array of XSK File +Descriptors (FDs). The number of array elements is typically set or adjusted using +the ``max_entries`` map parameter. For AF_XDP ``max_entries`` is equal to the number +of queues supported by the netdev. + +.. note:: + Both the map key and map value size must be 4 bytes. + +Usage +===== + +Kernel BPF +---------- +.. c:function:: + long bpf_redirect_map(struct bpf_map *map, u32 key, u64 flags) + + Redirect the packet to the endpoint referenced by ``map`` at index ``key``. + For ``BPF_MAP_TYPE_XSKMAP`` this map contains references to XSK FDs + for sockets attached to a netdev's queues. + + .. note:: + If the map is empty at an index, the packet is dropped. This means that it is + necessary to have an XDP program loaded with at least one XSK in the + XSKMAP to be able to get any traffic to user space through the socket. + +.. c:function:: + void *bpf_map_lookup_elem(struct bpf_map *map, const void *key) + + XSK entry references of type ``struct xdp_sock *`` can be retrieved using the + ``bpf_map_lookup_elem()`` helper. + +Userspace +--------- +.. note:: + XSK entries can only be updated/deleted from user space and not from + an eBPF program. Trying to call these functions from a kernel eBPF program will + result in the program failing to load and a verifier warning. + +.. c:function:: + int bpf_map_update_elem(int fd, const void *key, const void *value, __u64 flags) + + XSK entries can be added or updated using the ``bpf_map_update_elem()`` + helper. The ``key`` parameter is equal to the queue_id of the queue the XSK + is attaching to. And the ``value`` parameter is the FD value of that socket. + + Under the hood, the XSKMAP update function uses the XSK FD value to retrieve the + associated ``struct xdp_sock`` instance. + + The flags argument can be one of the following: + + - BPF_ANY: Create a new element or update an existing element. + - BPF_NOEXIST: Create a new element only if it did not exist. + - BPF_EXIST: Update an existing element. + +.. c:function:: + int bpf_map_lookup_elem(int fd, const void *key, void *value) + + Returns ``struct xdp_sock *`` or negative error in case of failure. + +.. c:function:: + int bpf_map_delete_elem(int fd, const void *key) + + XSK entries can be deleted using the ``bpf_map_delete_elem()`` + helper. This helper will return 0 on success, or negative error in case of + failure. + +.. note:: + When `libxdp`_ deletes a XSK it also removes the associated socket + entry from the XSKMAP. + +Examples +======== +Kernel +------ + +The following code snippet shows how to declare a ``BPF_MAP_TYPE_XSKMAP`` called +``xsks_map`` and how to redirect packets to a XSK. + +.. code-block:: c + + struct { + __uint(type, BPF_MAP_TYPE_XSKMAP); + __type(key, __u32); + __type(value, __u32); + __uint(max_entries, 64); + } xsks_map SEC(".maps"); + + + SEC("xdp") + int xsk_redir_prog(struct xdp_md *ctx) + { + __u32 index = ctx->rx_queue_index; + + if (bpf_map_lookup_elem(&xsks_map, &index)) + return bpf_redirect_map(&xsks_map, index, 0); + return XDP_PASS; + } + +Userspace +--------- + +The following code snippet shows how to update a XSKMAP with a XSK entry. + +.. code-block:: c + + int update_xsks_map(struct bpf_map *xsks_map, int queue_id, int xsk_fd) + { + int ret; + + ret = bpf_map_update_elem(bpf_map__fd(xsks_map), &queue_id, &xsk_fd, 0); + if (ret < 0) { + fprintf(stderr, "Failed to update xsks_map: %s\n", + strerror(errno)); + } + + return ret; + } + +.. note:: + The most comprehensive resource for using XSKMAPs is `libxdp`_. + +.. _libxdp: https://github.com/xdp-project/xdp-tools/tree/master/lib/libxdp