| Message ID | 20181213075727.23540-1-xiaoguangrong@tencent.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | optimize waiting for free thread to do compression | expand |
On Thu, Dec 13, 2018 at 03:57:25PM +0800, guangrong.xiao@gmail.com wrote: > From: Xiao Guangrong <xiaoguangrong@tencent.com> > > Currently we have two behaviors if all threads are busy to do compression, > the main thread mush wait one of them becoming free if @compress-wait-thread > set to on or the main thread can directly return without wait and post > the page out as normal one > > Both of them have its profits and short-comes, however, if the bandwidth is > not limited extremely so that compression can not use out all of it bandwidth, > at the same time, the migration process is easily throttled if we posted too > may pages as normal pages. None of them can work properly under this case > > In order to use the bandwidth more effectively, we introduce the third > behavior, compress-wait-thread-adaptive, which make the main thread wait > if there is no bandwidth left or let the page go out as normal page if there > has enough bandwidth to make sure the migration process will not be > throttled > > Another patch introduces a new statistic, pages-per-second, as bandwidth > or mbps is not enough to measure the performance of posting pages out as > we have compression, xbzrle, which can significantly reduce the amount of > the data size, instead, pages-per-second if the one we want > > Performance data > ================ > We have limited the bandwidth to 300 > > Used Bandwidth Pages-per-Second > compress-wait-thread = on 951.66 mbps 131784 > > compress-wait-thread = off 2491.74 mbps 93495 > compress-wait-thread-adaptive 1982.94 mbps 162529 > = on Sounds reasonable to me, though it looks like there're only three options: on, off, adaptive. Should we squash the two parameters? Thanks,
On 12/21/18 4:11 PM, Peter Xu wrote: > On Thu, Dec 13, 2018 at 03:57:25PM +0800, guangrong.xiao@gmail.com wrote: >> From: Xiao Guangrong <xiaoguangrong@tencent.com> >> >> Currently we have two behaviors if all threads are busy to do compression, >> the main thread mush wait one of them becoming free if @compress-wait-thread >> set to on or the main thread can directly return without wait and post >> the page out as normal one >> >> Both of them have its profits and short-comes, however, if the bandwidth is >> not limited extremely so that compression can not use out all of it bandwidth, >> at the same time, the migration process is easily throttled if we posted too >> may pages as normal pages. None of them can work properly under this case >> >> In order to use the bandwidth more effectively, we introduce the third >> behavior, compress-wait-thread-adaptive, which make the main thread wait >> if there is no bandwidth left or let the page go out as normal page if there >> has enough bandwidth to make sure the migration process will not be >> throttled >> >> Another patch introduces a new statistic, pages-per-second, as bandwidth >> or mbps is not enough to measure the performance of posting pages out as >> we have compression, xbzrle, which can significantly reduce the amount of >> the data size, instead, pages-per-second if the one we want >> >> Performance data >> ================ >> We have limited the bandwidth to 300 >> >> Used Bandwidth Pages-per-Second >> compress-wait-thread = on 951.66 mbps 131784 >> >> compress-wait-thread = off 2491.74 mbps 93495 >> compress-wait-thread-adaptive 1982.94 mbps 162529 >> = on > > Sounds reasonable to me, though it looks like there're only three > options: on, off, adaptive. Should we squash the two parameters? Thanks for your review, Peter! That's good to me, if other guys do not have objects, i will do it in the next version.
From: Xiao Guangrong <xiaoguangrong@tencent.com> Currently we have two behaviors if all threads are busy to do compression, the main thread mush wait one of them becoming free if @compress-wait-thread set to on or the main thread can directly return without wait and post the page out as normal one Both of them have its profits and short-comes, however, if the bandwidth is not limited extremely so that compression can not use out all of it bandwidth, at the same time, the migration process is easily throttled if we posted too may pages as normal pages. None of them can work properly under this case In order to use the bandwidth more effectively, we introduce the third behavior, compress-wait-thread-adaptive, which make the main thread wait if there is no bandwidth left or let the page go out as normal page if there has enough bandwidth to make sure the migration process will not be throttled Another patch introduces a new statistic, pages-per-second, as bandwidth or mbps is not enough to measure the performance of posting pages out as we have compression, xbzrle, which can significantly reduce the amount of the data size, instead, pages-per-second if the one we want Performance data ================ We have limited the bandwidth to 300 Used Bandwidth Pages-per-Second compress-wait-thread = on 951.66 mbps 131784 compress-wait-thread = off 2491.74 mbps 93495 compress-wait-thread-adaptive 1982.94 mbps 162529 = on Xiao Guangrong (2): migration: introduce compress-wait-thread-adaptive migration: introduce pages-per-second hmp.c | 13 ++++++ migration/migration.c | 49 +++++++++++++++++++- migration/migration.h | 12 +++++ migration/ram.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++---- qapi/migration.json | 31 +++++++++++-- 5 files changed, 215 insertions(+), 14 deletions(-)