| Message ID | 776a5fa2-818c-de42-2ac3-a4588df218ca@kernel.dk (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | brd: mark as nowait compatible | expand |
On Wed, Feb 15, 2023 at 04:43:47PM -0700, Jens Axboe wrote: > By default, non-mq drivers do not support nowait. This causes io_uring > to use a slower path as the driver cannot be trust not to block. brd > can safely set the nowait flag, as worst case all it does is a NOIO > allocation. But a NOIO allocation can block. I think we need to do a GFP_NOWAIT allocation in brd_insert_page if the NOWAIT flag is set.
On 2/15/23 11:08 PM, Christoph Hellwig wrote: > On Wed, Feb 15, 2023 at 04:43:47PM -0700, Jens Axboe wrote: >> By default, non-mq drivers do not support nowait. This causes io_uring >> to use a slower path as the driver cannot be trust not to block. brd >> can safely set the nowait flag, as worst case all it does is a NOIO >> allocation. > > But a NOIO allocation can block. I think we need to do a > GFP_NOWAIT allocation in brd_insert_page if the NOWAIT flag is set. I did consider that, but we do allocations almost everywhere and as long as we're not waiting on IO, it's mostly considered acceptable. But I can make that change, no reason not to.
On Thu, Feb 16, 2023 at 06:11:41AM -0700, Jens Axboe wrote:
> I did consider that, but we do allocations almost everywhere and
Do we? All the code I've touched for nowait goes to great length
to avoid blocking allocations.
On 2/16/23 7:40 AM, Christoph Hellwig wrote: > On Thu, Feb 16, 2023 at 06:11:41AM -0700, Jens Axboe wrote: >> I did consider that, but we do allocations almost everywhere and > > Do we? All the code I've touched for nowait goes to great length > to avoid blocking allocations. It is prudent, I'm not disagreeing. I've got a v2, will send it out shortly.
diff --git a/drivers/block/brd.c b/drivers/block/brd.c index 20acc4a1fd6d..82419e345777 100644 --- a/drivers/block/brd.c +++ b/drivers/block/brd.c @@ -412,6 +412,7 @@ static int brd_alloc(int i) /* Tell the block layer that this is not a rotational device */ blk_queue_flag_set(QUEUE_FLAG_NONROT, disk->queue); blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, disk->queue); + blk_queue_flag_set(QUEUE_FLAG_NOWAIT, disk->queue); err = add_disk(disk); if (err) goto out_cleanup_disk;
By default, non-mq drivers do not support nowait. This causes io_uring to use a slower path as the driver cannot be trust not to block. brd can safely set the nowait flag, as worst case all it does is a NOIO allocation. For io_uring, this makes a substantial difference. Before: submitter=0, tid=453, file=/dev/ram0, node=-1 polled=0, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=440.03K, BW=1718MiB/s, IOS/call=32/31 IOPS=428.96K, BW=1675MiB/s, IOS/call=32/32 IOPS=442.59K, BW=1728MiB/s, IOS/call=32/31 IOPS=419.65K, BW=1639MiB/s, IOS/call=32/32 IOPS=426.82K, BW=1667MiB/s, IOS/call=32/31 and after: submitter=0, tid=354, file=/dev/ram0, node=-1 polled=0, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=3.37M, BW=13.15GiB/s, IOS/call=32/31 IOPS=3.45M, BW=13.46GiB/s, IOS/call=32/31 IOPS=3.43M, BW=13.42GiB/s, IOS/call=32/32 IOPS=3.43M, BW=13.39GiB/s, IOS/call=32/31 IOPS=3.43M, BW=13.38GiB/s, IOS/call=32/31 or about an 8x in difference. Signed-off-by: Jens Axboe <axboe@kernel.dk> ---