| Message ID | 20220515131230.155267-1-haoxu.linux@icloud.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | fixed worker | expand |
On 5/15/22 21:12, Hao Xu wrote: > From: Hao Xu <howeyxu@tencent.com> > > This is the second version of fixed worker implementation. > Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. > normal workers: > ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 > time spent: 10464397 usecs IOPS: 1911242 > time spent: 9610976 usecs IOPS: 2080954 > time spent: 9807361 usecs IOPS: 2039284 > > fixed workers: > ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 > time spent: 17314274 usecs IOPS: 1155116 > time spent: 17016942 usecs IOPS: 1175299 > time spent: 17908684 usecs IOPS: 1116776 > > About 2x improvement. From perf result, almost no acct->lock contension. > Test program: https://github.com/HowHsu/liburing/tree/fixed_worker > liburing/test/nop_wqe.c > > v3->v4: > - make work in fixed worker's private worfixed worker > - tweak the io_wqe_acct struct to make it clearer > Hi Jens and Pavel, Any comments on this series? There are two coding style issue and I'm going to send v5, before this I'd like to get some comment if there is any. Thanks, Hao
On 5/31/22 1:05 AM, Hao Xu wrote: > On 5/15/22 21:12, Hao Xu wrote: >> From: Hao Xu <howeyxu@tencent.com> >> >> This is the second version of fixed worker implementation. >> Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. >> normal workers: >> ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 >> time spent: 10464397 usecs IOPS: 1911242 >> time spent: 9610976 usecs IOPS: 2080954 >> time spent: 9807361 usecs IOPS: 2039284 >> >> fixed workers: >> ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 >> time spent: 17314274 usecs IOPS: 1155116 >> time spent: 17016942 usecs IOPS: 1175299 >> time spent: 17908684 usecs IOPS: 1116776 >> >> About 2x improvement. From perf result, almost no acct->lock contension. >> Test program: https://github.com/HowHsu/liburing/tree/fixed_worker >> liburing/test/nop_wqe.c >> >> v3->v4: >> - make work in fixed worker's private worfixed worker >> - tweak the io_wqe_acct struct to make it clearer >> > > Hi Jens and Pavel, > Any comments on this series? There are two coding style issue and I'm > going to send v5, before this I'd like to get some comment if there is > any. I'll try to find some time to review it, doing a conference this week. Rebasing on the current for-5.20/io_uring branch would be a good idea anyway. Also, looks like your numbers are still swapped in the above, since fixed workers are still presented as taking longer / running slower?
On 5/31/22 16:46, Jens Axboe wrote: > On 5/31/22 1:05 AM, Hao Xu wrote: >> On 5/15/22 21:12, Hao Xu wrote: >>> From: Hao Xu <howeyxu@tencent.com> >>> >>> This is the second version of fixed worker implementation. >>> Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. >>> normal workers: >>> ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 >>> time spent: 10464397 usecs IOPS: 1911242 >>> time spent: 9610976 usecs IOPS: 2080954 >>> time spent: 9807361 usecs IOPS: 2039284 >>> >>> fixed workers: >>> ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 >>> time spent: 17314274 usecs IOPS: 1155116 >>> time spent: 17016942 usecs IOPS: 1175299 >>> time spent: 17908684 usecs IOPS: 1116776 >>> >>> About 2x improvement. From perf result, almost no acct->lock contension. >>> Test program: https://github.com/HowHsu/liburing/tree/fixed_worker >>> liburing/test/nop_wqe.c >>> >>> v3->v4: >>> - make work in fixed worker's private worfixed worker >>> - tweak the io_wqe_acct struct to make it clearer >>> >> >> Hi Jens and Pavel, >> Any comments on this series? There are two coding style issue and I'm >> going to send v5, before this I'd like to get some comment if there is >> any. > > I'll try to find some time to review it, doing a conference this week. No worries. > Rebasing on the current for-5.20/io_uring branch would be a good idea > anyway. I'll do that. > > Also, looks like your numbers are still swapped in the above, since > fixed workers are still presented as taking longer / running slower? > Thanks for pointing it out.. I'll make the change in next version.
On 5/31/22 2:55 AM, Hao Xu wrote: > On 5/31/22 16:46, Jens Axboe wrote: >> On 5/31/22 1:05 AM, Hao Xu wrote: >>> On 5/15/22 21:12, Hao Xu wrote: >>>> From: Hao Xu <howeyxu@tencent.com> >>>> >>>> This is the second version of fixed worker implementation. >>>> Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. >>>> normal workers: >>>> ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 >>>> time spent: 10464397 usecs IOPS: 1911242 >>>> time spent: 9610976 usecs IOPS: 2080954 >>>> time spent: 9807361 usecs IOPS: 2039284 >>>> >>>> fixed workers: >>>> ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 >>>> time spent: 17314274 usecs IOPS: 1155116 >>>> time spent: 17016942 usecs IOPS: 1175299 >>>> time spent: 17908684 usecs IOPS: 1116776 >>>> >>>> About 2x improvement. From perf result, almost no acct->lock contension. >>>> Test program: https://github.com/HowHsu/liburing/tree/fixed_worker >>>> liburing/test/nop_wqe.c >>>> >>>> v3->v4: >>>> - make work in fixed worker's private worfixed worker >>>> - tweak the io_wqe_acct struct to make it clearer >>>> >>> >>> Hi Jens and Pavel, >>> Any comments on this series? There are two coding style issue and I'm >>> going to send v5, before this I'd like to get some comment if there is >>> any. >> >> I'll try to find some time to review it, doing a conference this week. > > No worries. > >> Rebasing on the current for-5.20/io_uring branch would be a good idea >> anyway. > > I'll do that. When you do, most/all patches also have: From: Hao Xu <haoxu.linux@gmail.com> From: Hao Xu <howeyxu@tencent.com> which is a bit confusing, so probably choose one and go with that :-)
On 5/31/22 17:02, Jens Axboe wrote: > On 5/31/22 2:55 AM, Hao Xu wrote: >> On 5/31/22 16:46, Jens Axboe wrote: >>> On 5/31/22 1:05 AM, Hao Xu wrote: >>>> On 5/15/22 21:12, Hao Xu wrote: >>>>> From: Hao Xu <howeyxu@tencent.com> >>>>> >>>>> This is the second version of fixed worker implementation. >>>>> Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. >>>>> normal workers: >>>>> ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 >>>>> time spent: 10464397 usecs IOPS: 1911242 >>>>> time spent: 9610976 usecs IOPS: 2080954 >>>>> time spent: 9807361 usecs IOPS: 2039284 >>>>> >>>>> fixed workers: >>>>> ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 >>>>> time spent: 17314274 usecs IOPS: 1155116 >>>>> time spent: 17016942 usecs IOPS: 1175299 >>>>> time spent: 17908684 usecs IOPS: 1116776 >>>>> >>>>> About 2x improvement. From perf result, almost no acct->lock contension. >>>>> Test program: https://github.com/HowHsu/liburing/tree/fixed_worker >>>>> liburing/test/nop_wqe.c >>>>> >>>>> v3->v4: >>>>> - make work in fixed worker's private worfixed worker >>>>> - tweak the io_wqe_acct struct to make it clearer >>>>> >>>> >>>> Hi Jens and Pavel, >>>> Any comments on this series? There are two coding style issue and I'm >>>> going to send v5, before this I'd like to get some comment if there is >>>> any. >>> >>> I'll try to find some time to review it, doing a conference this week. >> >> No worries. >> >>> Rebasing on the current for-5.20/io_uring branch would be a good idea >>> anyway. >> >> I'll do that. > > When you do, most/all patches also have: > > From: Hao Xu <haoxu.linux@gmail.com> > > From: Hao Xu <howeyxu@tencent.com> > > which is a bit confusing, so probably choose one and go with that :-) > Sure
From: Hao Xu <howeyxu@tencent.com> This is the second version of fixed worker implementation. Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers. normal workers: ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3 time spent: 10464397 usecs IOPS: 1911242 time spent: 9610976 usecs IOPS: 2080954 time spent: 9807361 usecs IOPS: 2039284 fixed workers: ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3 time spent: 17314274 usecs IOPS: 1155116 time spent: 17016942 usecs IOPS: 1175299 time spent: 17908684 usecs IOPS: 1116776 About 2x improvement. From perf result, almost no acct->lock contension. Test program: https://github.com/HowHsu/liburing/tree/fixed_worker liburing/test/nop_wqe.c v3->v4: - make work in fixed worker's private worfixed worker - tweak the io_wqe_acct struct to make it clearer Hao Xu (11): io-wq: add a worker flag for individual exit io-wq: change argument of create_io_worker() for convienence io-wq: add infra data structure for fixed workers io-wq: tweak io_get_acct() io-wq: fixed worker initialization io-wq: fixed worker exit io-wq: implement fixed worker logic io-wq: batch the handling of fixed worker private works io_uring: add register fixed worker interface io-wq: add an work list for fixed worker io_uring: cancel works in exec work list for fixed worker fs/io-wq.c | 481 ++++++++++++++++++++++++++++++---- fs/io-wq.h | 8 + fs/io_uring.c | 71 +++++ include/uapi/linux/io_uring.h | 11 + 4 files changed, 525 insertions(+), 46 deletions(-) base-commit: fa5da31df51f8f581ec1776e613c1bcabbe9559f