| Message ID | 20211220141734.12206-1-joshi.k@samsung.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | uring-passthru for nvme | expand |
On 12/20/21 7:17 AM, Kanchan Joshi wrote: > Here is a revamped series on uring-passthru which is on top of Jens > "nvme-passthru-wip.2" branch. > https://git.kernel.dk/cgit/linux-block/commit/?h=nvme-passthru-wip.2 > > This scales much better than before with the addition of following: > - plugging > - passthru polling (sync and async; sync part comes from a patch that > Keith did earlier) > - bio-cache (this is regardless of irq/polling since we submit/complete in > task-contex anyway. Currently kicks in when fixed-buffer option is > also passed, but that's primarily to keep the plumbing simple) > > Also the feedback from Christoph (previous fixed-buffer series) is in > which has streamlined the plumbing. > > I look forward to further feedback/comments. > > KIOPS(512b) on P5800x looked like this: > > QD uring pt uring-poll pt-poll > 8 538 589 831 902 > 64 967 1131 1351 1378 > 256 1043 1230 1376 1429 These are nice results! Can you share all the job files or fio invocations for each of these? I guess it's just two variants, with QD varied between them? We really (REALLY) should turn the nvme-wip branch into something coherent, but at least with this we have some idea of an end result and something that is testable. This looks so much better from the performance POV than the earlier versions, passthrough _should_ be faster than non-pt.
On Tue, Dec 21, 2021 at 9:15 AM Jens Axboe <axboe@kernel.dk> wrote: > > On 12/20/21 7:17 AM, Kanchan Joshi wrote: > > Here is a revamped series on uring-passthru which is on top of Jens > > "nvme-passthru-wip.2" branch. > > https://git.kernel.dk/cgit/linux-block/commit/?h=nvme-passthru-wip.2 > > > > This scales much better than before with the addition of following: > > - plugging > > - passthru polling (sync and async; sync part comes from a patch that > > Keith did earlier) > > - bio-cache (this is regardless of irq/polling since we submit/complete in > > task-contex anyway. Currently kicks in when fixed-buffer option is > > also passed, but that's primarily to keep the plumbing simple) > > > > Also the feedback from Christoph (previous fixed-buffer series) is in > > which has streamlined the plumbing. > > > > I look forward to further feedback/comments. > > > > KIOPS(512b) on P5800x looked like this: > > > > QD uring pt uring-poll pt-poll > > 8 538 589 831 902 > > 64 967 1131 1351 1378 > > 256 1043 1230 1376 1429 > > These are nice results! Can you share all the job files or fio > invocations for each of these? I guess it's just two variants, with QD > varied between them? Yes, just two variants with three QD/batch combinations. Here are all the job files for the above data: https://github.com/joshkan/fio/tree/nvme-passthru-wip-polling/pt-perf-jobs > We really (REALLY) should turn the nvme-wip branch into something > coherent, but at least with this we have some idea of an end result and > something that is testable. This looks so much better from the > performance POV than the earlier versions, passthrough _should_ be > faster than non-pt. > It'd be great to know how it performs in your setup. And please let me know how I can help in making things more coherent.