[RFC,net-next,v7,13/14] net: add devmem TCP documentation

Commit Message

Mina Almasry March 26, 2024, 10:50 p.m. UTC

Add documentation outlining the usage and details of devmem TCP.

Signed-off-by: Mina Almasry <[e-mail]>

---

v7:
- Applied docs suggestions (Jakub).

v2:

- Missing spdx (simon)
- add to index.rst (simon)

---
 Documentation/networking/devmem.rst | 256 ++++++++++++++++++++++++++++
 Documentation/networking/index.rst  |   1 +
 2 files changed, 257 insertions(+)
 create mode 100644 Documentation/networking/devmem.rst

Comments

Randy Dunlap March 27, 2024, 12:38 a.m. UTC | #1

Hi,

On 3/26/24 15:50, Mina Almasry wrote:
> Add documentation outlining the usage and details of devmem TCP.
> 
> Signed-off-by: Mina Almasry <[e-mail]>
> 
> ---
> 
> v7:
> - Applied docs suggestions (Jakub).
> 
> v2:
> 
> - Missing spdx (simon)
> - add to index.rst (simon)
> 
> ---
>  Documentation/networking/devmem.rst | 256 ++++++++++++++++++++++++++++
>  Documentation/networking/index.rst  |   1 +
>  2 files changed, 257 insertions(+)
>  create mode 100644 Documentation/networking/devmem.rst
> 
> diff --git a/Documentation/networking/devmem.rst b/Documentation/networking/devmem.rst
> new file mode 100644
> index 000000000000..b0899e8e9e83
> --- /dev/null
> +++ b/Documentation/networking/devmem.rst
> @@ -0,0 +1,256 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=================
> +Device Memory TCP
> +=================
> +
> +
> +Intro
> +=====
> +
> +Device memory TCP (devmem TCP) enables receiving data directly into device
> +memory (dmabuf). The feature is currently implemented for TCP sockets.
> +
> +
> +Opportunity
> +-----------
> +
> +A large number of data transfers have device memory as the source and/or
> +destination. Accelerators drastically increased the prevalence of such
> +transfers.  Some examples include:
> +
> +- Distributed training, where ML accelerators, such as GPUs on different hosts,
> +  exchange data.
> +
> +- Distributed raw block storage applications transfer large amounts of data with
> +  remote SSDs, much of this data does not require host processing.

            SSDs. Much

> +
> +Typically the Device-to-Device data transfers the network are implemented as the

                                                 in the network
?

> +following low level operations: Device-to-Host copy, Host-to-Host network

             low-level

> +transfer, and Host-to-Device copy.
> +
> +The flow involving host copies is suboptimal, especially for bulk data transfers,
> +and can put significant strains on system resources such as host memory
> +bandwidth and PCIe bandwidth.
> +
> +Devmem TCP optimizes this use case by implementing socket APIs that enable
> +the user to receive incoming network packets directly into device memory.
> +
> +Packet payloads go directly from the NIC to device memory.
> +
> +Packet headers go to host memory and are processed by the TCP/IP stack
> +normally. The NIC must support header split to achieve this.
> +
> +Advantages:
> +
> +- Alleviate host memory bandwidth pressure, compared to existing
> +  network-transfer + device-copy semantics.
> +
> +- Alleviate PCIe bandwidth pressure, by limiting data transfer to the lowest
> +  level of the PCIe tree, compared to traditional path which sends data through

                                      to the

> +  the root complex.
> +
> +
> +More Info
> +---------
> +
> +  slides, video
> +    https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html
> +
> +  patchset
> +    [RFC PATCH v6 00/12] Device Memory TCP
> +    https://lore.kernel.org/netdev/[e-mail]/
> +
> +
> +Interface
> +=========
> +
> +Example
> +-------
> +
> +tools/testing/selftests/net/ncdevmem.c:do_server shows an example of setting up
> +the RX path of this API.
> +
> +NIC Setup
> +---------
> +
> +Header split, flow steering, & RSS are required features for devmem TCP.
> +
> +Header split is used to split incoming packets into a header buffer in host
> +memory, and a payload buffer in device memory.
> +
> +Flow steering & RSS are used to ensure that only flows targeting devmem land on> +RX queue bound to devmem.

   an RX queue
?

> +
> +Enable header split & flow steering::
> +
> +	# enable header split
> +	ethtool -G eth1 tcp-data-split on
> +
> +
> +	# enable flow steering
> +	ethtool -K eth1 ntuple on
> +
> +Configure RSS to steer all traffic away from the target RX queue (queue 15 in
> +this example)::
> +
> +	ethtool --set-rxfh-indir eth1 equal 15
> +
> +
> +The user must bind a dmabuf to any number of RX queues on a given NIC using
> +netlink API::

   the netlink API::

> +
> +	/* Bind dmabuf to NIC RX queue 15 */
> +	struct netdev_queue *queues;
> +	queues = malloc(sizeof(*queues) * 1);
> +
> +	queues[0]._present.type = 1;
> +	queues[0]._present.idx = 1;
> +	queues[0].type = NETDEV_RX_QUEUE_TYPE_RX;
> +	queues[0].idx = 15;
> +
> +	*ys = ynl_sock_create(&ynl_netdev_family, &yerr);
> +
> +	req = netdev_bind_rx_req_alloc();
> +	netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */);
> +	netdev_bind_rx_req_set_dmabuf_fd(req, dmabuf_fd);
> +	__netdev_bind_rx_req_set_queues(req, queues, n_queue_index);
> +
> +	rsp = netdev_bind_rx(*ys, req);
> +
> +	dmabuf_id = rsp->dmabuf_id;
> +
> +
> +The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf
> +that has been bound.
> +
> +Socket Setup
> +------------
> +
> +The socket must be flow steering to the dmabuf bound RX queue::

                      flow steered
?

> +
> +	ethtool -N eth1 flow-type tcp4 ... queue 15,
> +
> +
> +Receiving data
> +--------------
> +
> +The user application must signal to the kernel that it is capable of receiving
> +devmem data by passing the MSG_SOCK_DEVMEM flag to recvmsg::
> +
> +	ret = recvmsg(fd, &msg, MSG_SOCK_DEVMEM);
> +
> +Applications that do not specify the MSG_SOCK_DEVMEM flag will receive an EFAULT
> +on devmem data.
> +
> +Devmem data is received directly into the dmabuf bound to the NIC in 'NIC
> +Setup', and the kernel signals such to the user via the SCM_DEVMEM_* cmsgs::
> +
> +		for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) {
> +			if (cm->cmsg_level != SOL_SOCKET ||
> +				(cm->cmsg_type != SCM_DEVMEM_DMABUF &&
> +				 cm->cmsg_type != SCM_DEVMEM_LINEAR))
> +				continue;
> +
> +			dmabuf_cmsg = (struct dmabuf_cmsg *)CMSG_DATA(cm);
> +
> +			if (cm->cmsg_type == SCM_DEVMEM_DMABUF) {
> +				/* Frag landed in dmabuf.
> +				 *
> +				 * dmabuf_cmsg->dmabuf_id is the dmabuf the
> +				 * frag landed on.
> +				 *
> +				 * dmabuf_cmsg->frag_offset is the offset into
> +				 * the dmabuf where the frag starts.
> +				 *
> +				 * dmabuf_cmsg->frag_size is the size of the
> +				 * frag.
> +				 *
> +				 * dmabuf_cmsg->frag_token is a token used to
> +				 * refer to this frag for later freeing.
> +				 */
> +
> +				struct dmabuf_token token;
> +				token.token_start = dmabuf_cmsg->frag_token;
> +				token.token_count = 1;
> +				continue;
> +			}
> +
> +			if (cm->cmsg_type == SCM_DEVMEM_LINEAR)
> +				/* Frag landed in linear buffer.
> +				 *
> +				 * dmabuf_cmsg->frag_size is the size of the
> +				 * frag.
> +				 */
> +				continue;
> +
> +		}
> +
> +Applications may receive 2 cmsgs:
> +
> +- SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf indicated
> +  by dmabuf_id.
> +
> +- SCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer.
> +  This typically happens when the NIC is unable to split the packet at the
> +  header boundary, such that part (or all) of the payload landed in host
> +  memory.
> +
> +Applications may receive no SO_DEVMEM_* cmsgs. That indicates non-devmem,
> +regular TCP data that landed on an RX queue not bound to a dmabuf.
> +
> +
> +Freeing frags
> +-------------
> +
> +Frags received via SCM_DEVMEM_DMABUF are pinned by the kernel while the user
> +processes the frag. The user must return the frag to the kernel via
> +SO_DEVMEM_DONTNEED::
> +
> +	ret = setsockopt(client_fd, SOL_SOCKET, SO_DEVMEM_DONTNEED, &token,
> +			 sizeof(token));
> +
> +The user must ensure the tokens are returned to the kernel in a timely manner.
> +Failure to do so will exhaust the limited dmabuf that is bound to the RX queue
> +and will lead to packet drops.
> +
> +
> +Implementation & Caveats
> +========================
> +
> +Unreadable skbs
> +---------------
> +
> +Devmem payloads are inaccessible to the kernel processing the packets. This
> +results in a few quirks for payloads of devmem skbs:
> +
> +- Loopback is not functional. Loopback relies on copying the payload, which is
> +  not possible with devmem skbs.
> +
> +- Software checksum calculation fails.
> +
> +- TCP Dump and bpf can't access devmem packet payloads.
> +
> +
> +Testing
> +=======
> +
> +More realistic example code can be found in the kernel source under
> +tools/testing/selftests/net/ncdevmem.c
> +
> +ncdevmem is a devmem TCP netcat. It works very similarly to netcat, but
> +receives data directly into a udmabuf.
> +
> +To run ncdevmem, you need to run it a server on the machine under test, and you

                                    it on a server

> +need to run netcat on a peer to provide the TX data.
> +
> +ncdevmem has a validation mode as well that expects a repeating pattern of
> +incoming data and validates it as such::
> +
> +	# On server:
> +	ncdevmem -s <server IP> -c <client IP> -f eth1 -d 3 -n 0000:06:00.0 -l \
> +		 -p 5201 -v 7
> +
> +	# On client:
> +	yes $(echo -e \\x01\\x02\\x03\\x04\\x05\\x06) | \
> +		tr \\n \\0 | head -c 5G | nc <server IP> 5201 -p 5201

Patch

diff --git a/Documentation/networking/devmem.rst b/Documentation/networking/devmem.rst
new file mode 100644
index 000000000000..b0899e8e9e83
--- /dev/null
+++ b/Documentation/networking/devmem.rst
@@ -0,0 +1,256 @@ 
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Device Memory TCP
+=================
+
+
+Intro
+=====
+
+Device memory TCP (devmem TCP) enables receiving data directly into device
+memory (dmabuf). The feature is currently implemented for TCP sockets.
+
+
+Opportunity
+-----------
+
+A large number of data transfers have device memory as the source and/or
+destination. Accelerators drastically increased the prevalence of such
+transfers.  Some examples include:
+
+- Distributed training, where ML accelerators, such as GPUs on different hosts,
+  exchange data.
+
+- Distributed raw block storage applications transfer large amounts of data with
+  remote SSDs, much of this data does not require host processing.
+
+Typically the Device-to-Device data transfers the network are implemented as the
+following low level operations: Device-to-Host copy, Host-to-Host network
+transfer, and Host-to-Device copy.
+
+The flow involving host copies is suboptimal, especially for bulk data transfers,
+and can put significant strains on system resources such as host memory
+bandwidth and PCIe bandwidth.
+
+Devmem TCP optimizes this use case by implementing socket APIs that enable
+the user to receive incoming network packets directly into device memory.
+
+Packet payloads go directly from the NIC to device memory.
+
+Packet headers go to host memory and are processed by the TCP/IP stack
+normally. The NIC must support header split to achieve this.
+
+Advantages:
+
+- Alleviate host memory bandwidth pressure, compared to existing
+  network-transfer + device-copy semantics.
+
+- Alleviate PCIe bandwidth pressure, by limiting data transfer to the lowest
+  level of the PCIe tree, compared to traditional path which sends data through
+  the root complex.
+
+
+More Info
+---------
+
+  slides, video
+    https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html
+
+  patchset
+    [RFC PATCH v6 00/12] Device Memory TCP
+    https://lore.kernel.org/netdev/[e-mail]/
+
+
+Interface
+=========
+
+Example
+-------
+
+tools/testing/selftests/net/ncdevmem.c:do_server shows an example of setting up
+the RX path of this API.
+
+NIC Setup
+---------
+
+Header split, flow steering, & RSS are required features for devmem TCP.
+
+Header split is used to split incoming packets into a header buffer in host
+memory, and a payload buffer in device memory.
+
+Flow steering & RSS are used to ensure that only flows targeting devmem land on
+RX queue bound to devmem.
+
+Enable header split & flow steering::
+
+	# enable header split
+	ethtool -G eth1 tcp-data-split on
+
+
+	# enable flow steering
+	ethtool -K eth1 ntuple on
+
+Configure RSS to steer all traffic away from the target RX queue (queue 15 in
+this example)::
+
+	ethtool --set-rxfh-indir eth1 equal 15
+
+
+The user must bind a dmabuf to any number of RX queues on a given NIC using
+netlink API::
+
+	/* Bind dmabuf to NIC RX queue 15 */
+	struct netdev_queue *queues;
+	queues = malloc(sizeof(*queues) * 1);
+
+	queues[0]._present.type = 1;
+	queues[0]._present.idx = 1;
+	queues[0].type = NETDEV_RX_QUEUE_TYPE_RX;
+	queues[0].idx = 15;
+
+	*ys = ynl_sock_create(&ynl_netdev_family, &yerr);
+
+	req = netdev_bind_rx_req_alloc();
+	netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */);
+	netdev_bind_rx_req_set_dmabuf_fd(req, dmabuf_fd);
+	__netdev_bind_rx_req_set_queues(req, queues, n_queue_index);
+
+	rsp = netdev_bind_rx(*ys, req);
+
+	dmabuf_id = rsp->dmabuf_id;
+
+
+The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf
+that has been bound.
+
+Socket Setup
+------------
+
+The socket must be flow steering to the dmabuf bound RX queue::
+
+	ethtool -N eth1 flow-type tcp4 ... queue 15,
+
+
+Receiving data
+--------------
+
+The user application must signal to the kernel that it is capable of receiving
+devmem data by passing the MSG_SOCK_DEVMEM flag to recvmsg::
+
+	ret = recvmsg(fd, &msg, MSG_SOCK_DEVMEM);
+
+Applications that do not specify the MSG_SOCK_DEVMEM flag will receive an EFAULT
+on devmem data.
+
+Devmem data is received directly into the dmabuf bound to the NIC in 'NIC
+Setup', and the kernel signals such to the user via the SCM_DEVMEM_* cmsgs::
+
+		for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) {
+			if (cm->cmsg_level != SOL_SOCKET ||
+				(cm->cmsg_type != SCM_DEVMEM_DMABUF &&
+				 cm->cmsg_type != SCM_DEVMEM_LINEAR))
+				continue;
+
+			dmabuf_cmsg = (struct dmabuf_cmsg *)CMSG_DATA(cm);
+
+			if (cm->cmsg_type == SCM_DEVMEM_DMABUF) {
+				/* Frag landed in dmabuf.
+				 *
+				 * dmabuf_cmsg->dmabuf_id is the dmabuf the
+				 * frag landed on.
+				 *
+				 * dmabuf_cmsg->frag_offset is the offset into
+				 * the dmabuf where the frag starts.
+				 *
+				 * dmabuf_cmsg->frag_size is the size of the
+				 * frag.
+				 *
+				 * dmabuf_cmsg->frag_token is a token used to
+				 * refer to this frag for later freeing.
+				 */
+
+				struct dmabuf_token token;
+				token.token_start = dmabuf_cmsg->frag_token;
+				token.token_count = 1;
+				continue;
+			}
+
+			if (cm->cmsg_type == SCM_DEVMEM_LINEAR)
+				/* Frag landed in linear buffer.
+				 *
+				 * dmabuf_cmsg->frag_size is the size of the
+				 * frag.
+				 */
+				continue;
+
+		}
+
+Applications may receive 2 cmsgs:
+
+- SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf indicated
+  by dmabuf_id.
+
+- SCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer.
+  This typically happens when the NIC is unable to split the packet at the
+  header boundary, such that part (or all) of the payload landed in host
+  memory.
+
+Applications may receive no SO_DEVMEM_* cmsgs. That indicates non-devmem,
+regular TCP data that landed on an RX queue not bound to a dmabuf.
+
+
+Freeing frags
+-------------
+
+Frags received via SCM_DEVMEM_DMABUF are pinned by the kernel while the user
+processes the frag. The user must return the frag to the kernel via
+SO_DEVMEM_DONTNEED::
+
+	ret = setsockopt(client_fd, SOL_SOCKET, SO_DEVMEM_DONTNEED, &token,
+			 sizeof(token));
+
+The user must ensure the tokens are returned to the kernel in a timely manner.
+Failure to do so will exhaust the limited dmabuf that is bound to the RX queue
+and will lead to packet drops.
+
+
+Implementation & Caveats
+========================
+
+Unreadable skbs
+---------------
+
+Devmem payloads are inaccessible to the kernel processing the packets. This
+results in a few quirks for payloads of devmem skbs:
+
+- Loopback is not functional. Loopback relies on copying the payload, which is
+  not possible with devmem skbs.
+
+- Software checksum calculation fails.
+
+- TCP Dump and bpf can't access devmem packet payloads.
+
+
+Testing
+=======
+
+More realistic example code can be found in the kernel source under
+tools/testing/selftests/net/ncdevmem.c
+
+ncdevmem is a devmem TCP netcat. It works very similarly to netcat, but
+receives data directly into a udmabuf.
+
+To run ncdevmem, you need to run it a server on the machine under test, and you
+need to run netcat on a peer to provide the TX data.
+
+ncdevmem has a validation mode as well that expects a repeating pattern of
+incoming data and validates it as such::
+
+	# On server:
+	ncdevmem -s <server IP> -c <client IP> -f eth1 -d 3 -n 0000:06:00.0 -l \
+		 -p 5201 -v 7
+
+	# On client:
+	yes $(echo -e \\x01\\x02\\x03\\x04\\x05\\x06) | \
+		tr \\n \\0 | head -c 5G | nc <server IP> 5201 -p 5201
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index 473d72c36d61..6b47a51caad6 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -48,6 +48,7 @@  Contents:
    cdc_mbim
    dccp
    dctcp
+   devmem
    dns_resolver
    driver
    eql