[PATCHSET,v2,0/3] Support for provided buffers for send

Message

Jens Axboe Feb. 19, 2024, 9:25 p.m. UTC

Hi,

We never supported provided buffers for sends, because it didn't seem
to make a lot of sense. But it actually does make a lot of sense! If
an app is receiving data, doing something with it, and then sending
either the same or another buffer out based on that, then if we use
provided buffers for sends we can guarantee that the sends are
serialized. This is because provided buffer rings are FIFO ordered,
as it's a ring buffer, and hence it doesn't really matter if you
have more than one send inflight.

This provides a nice efficiency win, but more importantly, it reduces
the complexity in the application as it no longer needs to track a
potential backlog of sends. The app just sets up a send based buffer
ring, exactly like it does for incoming data. And that's it, no more
dealing with serialized sends.

In some testing with proxy [1], in basic shuffling of packets I see
a 36% improvement with this over manually dealing with sends. That's
a pretty big win on top of making the app simpler.

This also opens the door for multishot send requests, which is an
interesting future venue to pursue!

You can also find the patches here:

https://git.kernel.dk/cgit/linux/log/?h=io_uring-send-queue

[1] https://git.kernel.dk/cgit/liburing/tree/examples/proxy.c

Changes since v1:

- Forgot that the sendmsg side uses the generic helper for this, whereas
  recvmsg rolls its own to deal with provided buffers. Unify them in patch
  1 instead.

 include/uapi/linux/io_uring.h |   1 +
 io_uring/io_uring.c           |   3 +-
 io_uring/net.c                | 309 +++++++++++++++++++---------------
 io_uring/opdef.c              |   2 +
 4 files changed, 175 insertions(+), 140 deletions(-)