mbox series

[v4,0/6] *** Add Multifd support for TLS migration ***

Message ID 1600139042-104593-1-git-send-email-zhengchuan@huawei.com (mailing list archive)
Headers show
Series *** Add Multifd support for TLS migration *** | expand

Message

Zheng Chuan Sept. 15, 2020, 3:03 a.m. UTC
v3 -> v4:
    still use MigrationState field and save hostname in migration_tls_channel_connect
    remove MigrationState from MultiFDSendParams
 
v2 -> v3:
    rebase patches on master

v1 -> v2:
    fix memoryleak of MigrationState hostname
    add tls_hostname into MultiFDSendParams for handshake use
    fix function alignment
    squash Patch005 and Patch006
    add ioc into trace-events

TLS migration could easily reach bottleneck of cpu because of encryption
and decryption in migration thread.
In our test, the tls migration could only reach 300MB/s under bandwidth
of 500MB/s.

Inspired by multifd, we add multifd support for tls migration to make fully
use of given net bandwidth at the cost of multi-cpus and could reduce
at most of 100% migration time with 4U16G test vm.

Evaluate migration time of migration vm.
The VM specifications for migration are as follows:
- VM use 4-K page;
- the number of VCPU is 4;
- the total memory is 16Gigabit;
- use 'mempress' tool to pressurize VM(mempress 4096 100);
- migration flag is 73755 (8219 + 65536 (TLS)) vs 204827 (8219 + 65536 (TLS) + 131072(Multifd))

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|                      |         TLS           |      MultiFD + TLS (2 channel)    |
--------------------------------------------------------t---------------------------
| mempress 1024 120    |       25.035s         |           15.067s                 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
| mempress 1024 200    |       48.798s         |           25.334s                 |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Chuan Zheng (6):
  migration/tls: save hostname into MigrationState
  migration/tls: extract migration_tls_client_create for common-use
  migration/tls: add tls_hostname into MultiFDSendParams
  migration/tls: extract cleanup function for common-use
  migration/tls: add support for multifd tls-handshake
  migration/tls: add trace points for multifd-tls

 migration/channel.c    |   1 +
 migration/migration.c  |   1 +
 migration/migration.h  |   5 ++
 migration/multifd.c    | 124 +++++++++++++++++++++++++++++++++++++++++++------
 migration/multifd.h    |   2 +
 migration/tls.c        |  28 +++++++----
 migration/tls.h        |   6 +++
 migration/trace-events |   4 ++
 8 files changed, 149 insertions(+), 22 deletions(-)

Comments

Dr. David Alan Gilbert Sept. 24, 2020, 9:03 a.m. UTC | #1
* Chuan Zheng (zhengchuan@huawei.com) wrote:
> v3 -> v4:
>     still use MigrationState field and save hostname in migration_tls_channel_connect
>     remove MigrationState from MultiFDSendParams
>  
> v2 -> v3:
>     rebase patches on master
> 
> v1 -> v2:
>     fix memoryleak of MigrationState hostname
>     add tls_hostname into MultiFDSendParams for handshake use
>     fix function alignment
>     squash Patch005 and Patch006
>     add ioc into trace-events
> 
> TLS migration could easily reach bottleneck of cpu because of encryption
> and decryption in migration thread.
> In our test, the tls migration could only reach 300MB/s under bandwidth
> of 500MB/s.
> 
> Inspired by multifd, we add multifd support for tls migration to make fully
> use of given net bandwidth at the cost of multi-cpus and could reduce
> at most of 100% migration time with 4U16G test vm.
> 
> Evaluate migration time of migration vm.
> The VM specifications for migration are as follows:
> - VM use 4-K page;
> - the number of VCPU is 4;
> - the total memory is 16Gigabit;
> - use 'mempress' tool to pressurize VM(mempress 4096 100);
> - migration flag is 73755 (8219 + 65536 (TLS)) vs 204827 (8219 + 65536 (TLS) + 131072(Multifd))
> 
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> |                      |         TLS           |      MultiFD + TLS (2 channel)    |
> --------------------------------------------------------t---------------------------
> | mempress 1024 120    |       25.035s         |           15.067s                 |
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> | mempress 1024 200    |       48.798s         |           25.334s                 |
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Queued.

> Chuan Zheng (6):
>   migration/tls: save hostname into MigrationState
>   migration/tls: extract migration_tls_client_create for common-use
>   migration/tls: add tls_hostname into MultiFDSendParams
>   migration/tls: extract cleanup function for common-use
>   migration/tls: add support for multifd tls-handshake
>   migration/tls: add trace points for multifd-tls
> 
>  migration/channel.c    |   1 +
>  migration/migration.c  |   1 +
>  migration/migration.h  |   5 ++
>  migration/multifd.c    | 124 +++++++++++++++++++++++++++++++++++++++++++------
>  migration/multifd.h    |   2 +
>  migration/tls.c        |  28 +++++++----
>  migration/tls.h        |   6 +++
>  migration/trace-events |   4 ++
>  8 files changed, 149 insertions(+), 22 deletions(-)
> 
> -- 
> 1.8.3.1
> 
>