io_uring: EIO on write to a file on a ram dev

Commit Message

Roman Penyaev Dec. 2, 2019, 11:42 a.m. UTC

Hi Jens,

I stumbled upon EIO error while writing to a file which is on ram device
(actually any dev which is in bio mode, i.e. uses make_request_fn hook).
All details will be clear from the reproduction description.

How to reproduce:

   mkfs.ext4 /dev/ram0
   mount /dev/ram0 /mnt

   # Works fine, since we take async path in io_uring and create kiocb
   # without IOCB_NOWAIT
   fio --rw=write --ioengine=io_uring --size=1M --direct 1 --name=job 
--filename=/mnt/file

   # Repeat, all fs blocks are allocated and pure nowait & direct path
   # is taken and eventually we get -EIO
   fio --rw=write --ioengine=io_uring --size=1M --direct 1 --name=job 
--filename=/mnt/file


The culprit is the following check:

    generic_make_request_checks():
         if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_mq(q))
		    goto not_supported;


Probably the solution can be to complete bio with BLK_STS_AGAIN which
can be then correctly handled in dio_bio_complete() and eventually
async path in io_uring will be taken:

Not clear is that an intention not to inherit the NOWAIT flag inside
__blkdev_direct_IO or the flag is simply lost?

--
Roman

Patch

--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -889,20 +889,31 @@  generic_make_request_checks(struct bio *bio)
         /*
          * For a REQ_NOWAIT based request, return -EOPNOTSUPP
          * if queue is not a request based queue.
          */
-       if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_mq(q))
-               goto not_supported;
+       if ((bio->bi_opf & REQ_NOWAIT) && !queue_is_mq(q)) {
+               status = BLK_STS_AGAIN;
+               goto end_io;
+       }

(and this works) but I'm afraid that should degrade performance, since
on each IO we hit the error deeply in the stack, rewind the stack and
repeat the whole procedure from a worker.


Other confusing thing is that when io_uring writes directly to /dev/ram0
no errors are returned, because eventually bio is created and does not
inherit NOWAIT flag on this path:

    __blkdev_direct_IO():
         bio->bi_opf = dio_bio_write_op(iocb);

as opposed to writing directly to a file on the same ram dev, where all
flags are inherited from dio:

    __blockdev_direct_IO
    do_blockdev_direct_IO
    dio_send_cur_page
    dio_new_bio
    dio_bio_alloc():
         bio_set_op_attrs(bio, dio->op, dio->op_flags);