[PATCHSET,0/3] io_uring: support for linked SQEs

Message

Jens Axboe May 17, 2019, 9:41 p.m. UTC

We currently support a barrier operation, which ensures that previous
commands have finished before they are executed. This patchset
introduces support for linked commands. Linked commands form a
dependency between commands, but not for the execution pipeline in
general. Example:

[[ ReadA, WriteA], [ReadB, WriteB], [ReadC, WriteC]]

This is a weak ascii attempt at showing a series of reads and writes,
where each write depends on the previous read. WriteA will not be
started before ReadA is complete. Think of a copy like operation, where
ReadA is "read X bytes from file Y at offset Z" and WriteB is then
"write X bytes to file A at offset B", with the two sharing the same
iovec. While WriteA depends on the completion of ReadA, there are no
dependencies between ReadA and ReadB, and they may execute in parallel.

In terms of user API, ReadA will have IOSQE_IO_LINK set. When that is
set, the next command depends on it. If the next one also has
IOSQE_IO_LINK set, then the dependency continues. If it doesn't, then
the chain stops with the next command.

I wrote a trivial cp(2) implementation using linked reads and writes,
you can find that here:

http://git.kernel.dk/cgit/liburing/tree/examples/link-cp.c

in the liburing git repo. There's also a test program in there
exercising various types of chains.

I think this is a very powerful concept, and something that could even
be extended to be BPF driven at completion time if dependencies need
further linkage outside of what can be statically provided initially.
A potential series could be [Open file, Read, Close file] where the
fd needs to be carried forward, for example.

Patches are against current master from Linus, and are also in my
io_uring-next branch.

Comments

Christoph Hellwig May 22, 2019, 4:34 p.m. UTC | #1

This just apparead in your for-linus tree.  I don't think queueing
up unreviewed features after -rc1 is acceptable..

Jens Axboe May 22, 2019, 4:49 p.m. UTC | #2

On 5/22/19 10:34 AM, Christoph Hellwig wrote:
> This just apparead in your for-linus tree.  I don't think queueing
> up unreviewed features after -rc1 is acceptable..

I'm not pushing it for 5.2, nor was it my intent. I think I messed
up the branching, those bits will end up in for-5.3/io_uring.