[RFC,v5,0/5] introduce io_uring_cmd_import_fixed_vec

Message ID	20250319061251.21452-1-sidong.yang@furiosa.ai (mailing list archive)
Headers	show Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC92E22094 for <linux-btrfs@vger.kernel.org>; Wed, 19 Mar 2025 06:13:23 +0000 (UTC) From: Sidong Yang <sidong.yang@furiosa.ai> To: Josef Bacik <josef@toxicpanda.com>, David Sterba <dsterba@suse.com>, Jens Axboe <axboe@kernel.dk>, Pavel Begunkov <asml.silence@gmail.com> Cc: linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, io-uring@vger.kernel.org, Sidong Yang <sidong.yang@furiosa.ai> Subject: [RFC PATCH v5 0/5] introduce io_uring_cmd_import_fixed_vec Date: Wed, 19 Mar 2025 06:12:46 +0000 Message-ID: <20250319061251.21452-1-sidong.yang@furiosa.ai> Precedence: bulk MIME-Version: 1.0 Content-Transfer-Encoding: 8bit
Series	introduce io_uring_cmd_import_fixed_vec \| expand [RFC,v5,0/5] introduce io_uring_cmd_import_fixed_vec [RFC,v5,1/5] io_uring: rename the data cmd cache [RFC,v5,2/5] io_uring/cmd: don't expose entire cmd async data [RFC,v5,3/5] io_uring/cmd: add iovec cache for commands [RFC,v5,4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec [RFC,v5,5/5] btrfs: ioctl: introduce btrfs_uring_import_iovec()

Sidong Yang March 19, 2025, 6:12 a.m. UTC

This patche series introduce io_uring_cmd_import_vec. With this function,
Multiple fixed buffer could be used in uring cmd. It's vectored version
for io_uring_cmd_import_fixed(). Also this patch series includes a usage
for new api for encoded read/write in btrfs by using uring cmd.

There was approximately 10 percent of performance improvements through benchmark.
The benchmark code is in
https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c

./main -l
Elapsed time: 0.598997 seconds
./main -l -f
Elapsed time: 0.540332 seconds

Additionally, There is a commit that fixed a memory bug in btrfs uring encoded
read.

v2:
 - don't export iou_vc, use bvec for btrfs
 - use io_is_compat for checking compat
 - reduce allocation/free for import fixed vec

v3:
 - add iou_vec cache in io_uring_cmd and use it
 - also encoded write fixed supported

v4:
 - add a patch that introduce io_async_cmd
 - add a patch that fixes a bug in btrfs encoded read

v5:
 - use Pavel's original patches rebased for axboe/for-6.15/io_uring-reg-vec
 - pop a patch that fixes btrfs encoded read
 
Pavel Begunkov (4):
  io_uring: rename the data cmd cache
  io_uring/cmd: don't expose entire cmd async data
  io_uring/cmd: add iovec cache for commands
  io_uring/cmd: introduce io_uring_cmd_import_fixed_vec

Sidong Yang (1):
  btrfs: ioctl: introduce btrfs_uring_import_iovec()

 fs/btrfs/ioctl.c               | 34 ++++++++++++++------
 include/linux/io_uring/cmd.h   | 13 ++++++++
 include/linux/io_uring_types.h |  2 +-
 io_uring/io_uring.c            |  6 ++--
 io_uring/opdef.c               |  3 +-
 io_uring/uring_cmd.c           | 59 +++++++++++++++++++++++++++++-----
 io_uring/uring_cmd.h           | 17 ++++++++++
 7 files changed, 112 insertions(+), 22 deletions(-)

Jens Axboe March 19, 2025, 3:26 p.m. UTC | #1

On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
> This patche series introduce io_uring_cmd_import_vec. With this function,
> Multiple fixed buffer could be used in uring cmd. It's vectored version
> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
> for new api for encoded read/write in btrfs by using uring cmd.
> 
> There was approximately 10 percent of performance improvements through benchmark.
> The benchmark code is in
> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
> 
> [...]

Applied, thanks!

[1/5] io_uring: rename the data cmd cache
      commit: 575e7b0629d4bd485517c40ff20676180476f5f9
[2/5] io_uring/cmd: don't expose entire cmd async data
      commit: 5f14404bfa245a156915ee44c827edc56655b067
[3/5] io_uring/cmd: add iovec cache for commands
      commit: fe549edab6c3b7995b58450e31232566b383a249
[4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
      commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d

Best regards,

Jens Axboe March 19, 2025, 3:27 p.m. UTC | #2

On 3/19/25 9:26 AM, Jens Axboe wrote:
> 
> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
>> This patche series introduce io_uring_cmd_import_vec. With this function,
>> Multiple fixed buffer could be used in uring cmd. It's vectored version
>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
>> for new api for encoded read/write in btrfs by using uring cmd.
>>
>> There was approximately 10 percent of performance improvements through benchmark.
>> The benchmark code is in
>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
>>
>> [...]
> 
> Applied, thanks!
> 
> [1/5] io_uring: rename the data cmd cache
>       commit: 575e7b0629d4bd485517c40ff20676180476f5f9
> [2/5] io_uring/cmd: don't expose entire cmd async data
>       commit: 5f14404bfa245a156915ee44c827edc56655b067
> [3/5] io_uring/cmd: add iovec cache for commands
>       commit: fe549edab6c3b7995b58450e31232566b383a249
> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
>       commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d

1-4 look pretty straight forward to me - I'll be happy to queue the
btrfs one as well if the btrfs people are happy with it, just didn't
want to assume anything here.

David Sterba March 19, 2025, 5:07 p.m. UTC | #3

On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
> On 3/19/25 9:26 AM, Jens Axboe wrote:
> > 
> > On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
> >> This patche series introduce io_uring_cmd_import_vec. With this function,
> >> Multiple fixed buffer could be used in uring cmd. It's vectored version
> >> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
> >> for new api for encoded read/write in btrfs by using uring cmd.
> >>
> >> There was approximately 10 percent of performance improvements through benchmark.
> >> The benchmark code is in
> >> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
> >>
> >> [...]
> > 
> > Applied, thanks!
> > 
> > [1/5] io_uring: rename the data cmd cache
> >       commit: 575e7b0629d4bd485517c40ff20676180476f5f9
> > [2/5] io_uring/cmd: don't expose entire cmd async data
> >       commit: 5f14404bfa245a156915ee44c827edc56655b067
> > [3/5] io_uring/cmd: add iovec cache for commands
> >       commit: fe549edab6c3b7995b58450e31232566b383a249
> > [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
> >       commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
> 
> 1-4 look pretty straight forward to me - I'll be happy to queue the
> btrfs one as well if the btrfs people are happy with it, just didn't
> want to assume anything here.

For 6.15 is too late so it makes more sense to take it through the btrfs
patches targetting 6.16.

Jens Axboe March 19, 2025, 5:10 p.m. UTC | #4

On 3/19/25 11:07 AM, David Sterba wrote:
> On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
>> On 3/19/25 9:26 AM, Jens Axboe wrote:
>>>
>>> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
>>>> This patche series introduce io_uring_cmd_import_vec. With this function,
>>>> Multiple fixed buffer could be used in uring cmd. It's vectored version
>>>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
>>>> for new api for encoded read/write in btrfs by using uring cmd.
>>>>
>>>> There was approximately 10 percent of performance improvements through benchmark.
>>>> The benchmark code is in
>>>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
>>>>
>>>> [...]
>>>
>>> Applied, thanks!
>>>
>>> [1/5] io_uring: rename the data cmd cache
>>>       commit: 575e7b0629d4bd485517c40ff20676180476f5f9
>>> [2/5] io_uring/cmd: don't expose entire cmd async data
>>>       commit: 5f14404bfa245a156915ee44c827edc56655b067
>>> [3/5] io_uring/cmd: add iovec cache for commands
>>>       commit: fe549edab6c3b7995b58450e31232566b383a249
>>> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
>>>       commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
>>
>> 1-4 look pretty straight forward to me - I'll be happy to queue the
>> btrfs one as well if the btrfs people are happy with it, just didn't
>> want to assume anything here.
> 
> For 6.15 is too late so it makes more sense to take it through the btrfs
> patches targetting 6.16.

No problem - Sidong, guessing you probably want to resend patch 5/5 once
btrfs has a next branch based on 6.15-rc1 or later.

Sidong Yang March 20, 2025, 1:47 a.m. UTC | #5

On Wed, Mar 19, 2025 at 11:10:07AM -0600, Jens Axboe wrote:
> On 3/19/25 11:07 AM, David Sterba wrote:
> > On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
> >> On 3/19/25 9:26 AM, Jens Axboe wrote:
> >>>
> >>> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
> >>>> This patche series introduce io_uring_cmd_import_vec. With this function,
> >>>> Multiple fixed buffer could be used in uring cmd. It's vectored version
> >>>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
> >>>> for new api for encoded read/write in btrfs by using uring cmd.
> >>>>
> >>>> There was approximately 10 percent of performance improvements through benchmark.
> >>>> The benchmark code is in
> >>>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
> >>>>
> >>>> [...]
> >>>
> >>> Applied, thanks!
> >>>
> >>> [1/5] io_uring: rename the data cmd cache
> >>>       commit: 575e7b0629d4bd485517c40ff20676180476f5f9
> >>> [2/5] io_uring/cmd: don't expose entire cmd async data
> >>>       commit: 5f14404bfa245a156915ee44c827edc56655b067
> >>> [3/5] io_uring/cmd: add iovec cache for commands
> >>>       commit: fe549edab6c3b7995b58450e31232566b383a249
> >>> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
> >>>       commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
> >>
> >> 1-4 look pretty straight forward to me - I'll be happy to queue the
> >> btrfs one as well if the btrfs people are happy with it, just didn't
> >> want to assume anything here.
> > 
> > For 6.15 is too late so it makes more sense to take it through the btrfs
> > patches targetting 6.16.
> 
> No problem - Sidong, guessing you probably want to resend patch 5/5 once
> btrfs has a next branch based on 6.15-rc1 or later.

Thanks, I'll resend only patch 5/5 then.

Thanks,
Sidong
> 
> -- 
> Jens Axboe
>

Pavel Begunkov March 20, 2025, 11:53 a.m. UTC | #6

On 3/19/25 15:27, Jens Axboe wrote:
> On 3/19/25 9:26 AM, Jens Axboe wrote:
>>
>> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
>>> This patche series introduce io_uring_cmd_import_vec. With this function,
>>> Multiple fixed buffer could be used in uring cmd. It's vectored version
>>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
>>> for new api for encoded read/write in btrfs by using uring cmd.
>>>
>>> There was approximately 10 percent of performance improvements through benchmark.
>>> The benchmark code is in
>>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
>>>
>>> [...]
>>
>> Applied, thanks!
>>
>> [1/5] io_uring: rename the data cmd cache
>>        commit: 575e7b0629d4bd485517c40ff20676180476f5f9
>> [2/5] io_uring/cmd: don't expose entire cmd async data
>>        commit: 5f14404bfa245a156915ee44c827edc56655b067
>> [3/5] io_uring/cmd: add iovec cache for commands
>>        commit: fe549edab6c3b7995b58450e31232566b383a249
>> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
>>        commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
> 
> 1-4 look pretty straight forward to me - I'll be happy to queue the
> btrfs one as well if the btrfs people are happy with it, just didn't
> want to assume anything here.

fwiw, finally got time to wire a hacky test with a separate cmd,
works fine, but I'll need to setup btrfs to test the last patch.

Pavel Begunkov March 20, 2025, 12:04 p.m. UTC | #7

On 3/20/25 01:47, Sidong Yang wrote:
> On Wed, Mar 19, 2025 at 11:10:07AM -0600, Jens Axboe wrote:
>> On 3/19/25 11:07 AM, David Sterba wrote:
>>> On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
>>>> On 3/19/25 9:26 AM, Jens Axboe wrote:
>>>>>
>>>>> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
>>>>>> This patche series introduce io_uring_cmd_import_vec. With this function,
>>>>>> Multiple fixed buffer could be used in uring cmd. It's vectored version
>>>>>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
>>>>>> for new api for encoded read/write in btrfs by using uring cmd.
>>>>>>
>>>>>> There was approximately 10 percent of performance improvements through benchmark.
>>>>>> The benchmark code is in
>>>>>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
>>>>>>
>>>>>> [...]
>>>>>
>>>>> Applied, thanks!
>>>>>
>>>>> [1/5] io_uring: rename the data cmd cache
>>>>>        commit: 575e7b0629d4bd485517c40ff20676180476f5f9
>>>>> [2/5] io_uring/cmd: don't expose entire cmd async data
>>>>>        commit: 5f14404bfa245a156915ee44c827edc56655b067
>>>>> [3/5] io_uring/cmd: add iovec cache for commands
>>>>>        commit: fe549edab6c3b7995b58450e31232566b383a249
>>>>> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
>>>>>        commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
>>>>
>>>> 1-4 look pretty straight forward to me - I'll be happy to queue the
>>>> btrfs one as well if the btrfs people are happy with it, just didn't
>>>> want to assume anything here.
>>>
>>> For 6.15 is too late so it makes more sense to take it through the btrfs
>>> patches targetting 6.16.
>>
>> No problem - Sidong, guessing you probably want to resend patch 5/5 once
>> btrfs has a next branch based on 6.15-rc1 or later.
> 
> Thanks, I'll resend only patch 5/5 then.

And please do send the fix, that should always be done first,
especially if it conflicts with the current patch as they usually
go to different trees and the fix might need to be backported.

Sidong Yang March 20, 2025, 4:10 p.m. UTC | #8

On Thu, Mar 20, 2025 at 12:04:33PM +0000, Pavel Begunkov wrote:
> On 3/20/25 01:47, Sidong Yang wrote:
> > On Wed, Mar 19, 2025 at 11:10:07AM -0600, Jens Axboe wrote:
> > > On 3/19/25 11:07 AM, David Sterba wrote:
> > > > On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
> > > > > On 3/19/25 9:26 AM, Jens Axboe wrote:
> > > > > > 
> > > > > > On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
> > > > > > > This patche series introduce io_uring_cmd_import_vec. With this function,
> > > > > > > Multiple fixed buffer could be used in uring cmd. It's vectored version
> > > > > > > for io_uring_cmd_import_fixed(). Also this patch series includes a usage
> > > > > > > for new api for encoded read/write in btrfs by using uring cmd.
> > > > > > > 
> > > > > > > There was approximately 10 percent of performance improvements through benchmark.
> > > > > > > The benchmark code is in
> > > > > > > https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
> > > > > > > 
> > > > > > > [...]
> > > > > > 
> > > > > > Applied, thanks!
> > > > > > 
> > > > > > [1/5] io_uring: rename the data cmd cache
> > > > > >        commit: 575e7b0629d4bd485517c40ff20676180476f5f9
> > > > > > [2/5] io_uring/cmd: don't expose entire cmd async data
> > > > > >        commit: 5f14404bfa245a156915ee44c827edc56655b067
> > > > > > [3/5] io_uring/cmd: add iovec cache for commands
> > > > > >        commit: fe549edab6c3b7995b58450e31232566b383a249
> > > > > > [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
> > > > > >        commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
> > > > > 
> > > > > 1-4 look pretty straight forward to me - I'll be happy to queue the
> > > > > btrfs one as well if the btrfs people are happy with it, just didn't
> > > > > want to assume anything here.
> > > > 
> > > > For 6.15 is too late so it makes more sense to take it through the btrfs
> > > > patches targetting 6.16.
> > > 
> > > No problem - Sidong, guessing you probably want to resend patch 5/5 once
> > > btrfs has a next branch based on 6.15-rc1 or later.
> > 
> > Thanks, I'll resend only patch 5/5 then.
> 
> And please do send the fix, that should always be done first,
> especially if it conflicts with the current patch as they usually
> go to different trees and the fix might need to be backported.

Sorry to forget to cc you Pavel.
https://lore.kernel.org/linux-btrfs/20250319180416.GL32661@twin.jikos.cz/T/#t

Thanks,
Sidong
> 
> -- 
> Pavel Begunkov
>

Pavel Begunkov March 21, 2025, 9:56 a.m. UTC | #9

On 3/20/25 16:10, Sidong Yang wrote:
> On Thu, Mar 20, 2025 at 12:04:33PM +0000, Pavel Begunkov wrote:
>> On 3/20/25 01:47, Sidong Yang wrote:
>>> On Wed, Mar 19, 2025 at 11:10:07AM -0600, Jens Axboe wrote:
>>>> On 3/19/25 11:07 AM, David Sterba wrote:
>>>>> On Wed, Mar 19, 2025 at 09:27:37AM -0600, Jens Axboe wrote:
>>>>>> On 3/19/25 9:26 AM, Jens Axboe wrote:
>>>>>>>
>>>>>>> On Wed, 19 Mar 2025 06:12:46 +0000, Sidong Yang wrote:
>>>>>>>> This patche series introduce io_uring_cmd_import_vec. With this function,
>>>>>>>> Multiple fixed buffer could be used in uring cmd. It's vectored version
>>>>>>>> for io_uring_cmd_import_fixed(). Also this patch series includes a usage
>>>>>>>> for new api for encoded read/write in btrfs by using uring cmd.
>>>>>>>>
>>>>>>>> There was approximately 10 percent of performance improvements through benchmark.
>>>>>>>> The benchmark code is in
>>>>>>>> https://github.com/SidongYang/btrfs-encoded-io-test/blob/main/main.c
>>>>>>>>
>>>>>>>> [...]
>>>>>>>
>>>>>>> Applied, thanks!
>>>>>>>
>>>>>>> [1/5] io_uring: rename the data cmd cache
>>>>>>>         commit: 575e7b0629d4bd485517c40ff20676180476f5f9
>>>>>>> [2/5] io_uring/cmd: don't expose entire cmd async data
>>>>>>>         commit: 5f14404bfa245a156915ee44c827edc56655b067
>>>>>>> [3/5] io_uring/cmd: add iovec cache for commands
>>>>>>>         commit: fe549edab6c3b7995b58450e31232566b383a249
>>>>>>> [4/5] io_uring/cmd: introduce io_uring_cmd_import_fixed_vec
>>>>>>>         commit: b24cb04c1e072ecd859a98b2e4258ca8fe8d2d4d
>>>>>>
>>>>>> 1-4 look pretty straight forward to me - I'll be happy to queue the
>>>>>> btrfs one as well if the btrfs people are happy with it, just didn't
>>>>>> want to assume anything here.
>>>>>
>>>>> For 6.15 is too late so it makes more sense to take it through the btrfs
>>>>> patches targetting 6.16.
>>>>
>>>> No problem - Sidong, guessing you probably want to resend patch 5/5 once
>>>> btrfs has a next branch based on 6.15-rc1 or later.
>>>
>>> Thanks, I'll resend only patch 5/5 then.
>>
>> And please do send the fix, that should always be done first,
>> especially if it conflicts with the current patch as they usually
>> go to different trees and the fix might need to be backported.
> 
> Sorry to forget to cc you Pavel.
> https://lore.kernel.org/linux-btrfs/20250319180416.GL32661@twin.jikos.cz/T/#t

Ah, you already did, great!

[RFC,v5,0/5] introduce io_uring_cmd_import_fixed_vec

Message

Comments