[v2,0/4] Live Migration Acceleration with IAA Compression

Message

Yuan Liu Nov. 9, 2023, 3:46 p.m. UTC

Hi,

I am writing to submit a code change aimed at enhancing live migration
acceleration by leveraging the compression capability of the Intel
In-Memory Analytics Accelerator (IAA).

The implementation of the IAA (de)compression code is based on Intel Query
Processing Library (QPL), an open-source software project designed for
IAA high-level software programming. https://github.com/intel/qpl

In the previous version, there was some discussion about whether to
introduce a new compression algorithm for IAA. Because the compression
algorithm of IAA hardware is based on deflate, and QPL already supports
Zlib, so in this version, I implemented IAA as an accelerator for the
Zlib compression method. However, due to some reasons, QPL is currently
not compatible with the existing Zlib method that Zlib compressed data
can be decompressed by QPl and vice versa, I am still working on
resolving compatibility issues.

I have some concerns about the existing Zlib compression
  1. Will you consider supporting one channel to support multi-stream
     compression? Of course, this may lead to a reduction in compression
     ratio, but it will allow the hardware to process each stream
     concurrently. We can have each stream process multiple pages,
     reducing the loss of compression ratio. For example, 128 pages are
     divided into 16 streams for independent compression.

  2. Will you consider using QPL/IAA as an independent compression
     algorithm instead of an accelerator? In this way, we can better
     utilize hardware performance and some features, such as IAA's
     canned mode, which can be dynamically generated by some statistics
     of data. A huffman table to improve the compression ratio.

Test condition:
  1. Host CPUs are based on Sapphire Rapids, and frequency locked to 3.4G
  2. VM type, 16 vCPU and 64G memory
  3. The Idle workload means no workload is running in the VM
  4. The Redis workload means YCSB workloadb + Redis Server are running
     in the VM, about 20G or more memory will be used.
  5. Source side migartion configuration commands
     a. migrate_set_capability multifd on
     b. migrate_set_parameter multifd-channels 2/4/8
     c. migrate_set_parameter downtime-limit 300
     d. migrate_set_parameter multifd-compression zlib
     e. migrate_set_parameter multifd-compression-accel none/qpl
     f. migrate_set_parameter max-bandwidth 100G
  6. Desitination side migration configuration commands
     a. migrate_set_capability multifd on
     b. migrate_set_parameter multifd-channels 2/4/8
     c. migrate_set_parameter multifd-compression zlib
     d. migrate_set_parameter multifd-compression-accel none/qpl
     e. migrate_set_parameter max-bandwidth 100G

Early migration result, each result is the average of three tests

 +--------+-------------+--------+--------+---------+----+-----+
 |        | The number  |total   |downtime|network  |pages per |
 |        | of channels |time(ms)|(ms)    |bandwidth|second    |
 |        | and mode    |        |        |(mbps)   |          |
 |        +-------------+-----------------+---------+----------+
 |        | 2 chl, Zlib | 20647  | 22     | 195     | 137767   |
 |        +-------------+--------+--------+---------+----------+
 | Idle   | 2 chl, IAA  | 17022  | 36     | 286     | 460289   |
 |workload+-------------+--------+--------+---------+----------+
 |        | 4 chl, Zlib | 18835  | 29     | 241     | 299028   |
 |        +-------------+--------+--------+---------+----------+
 |        | 4 chl, IAA  | 16280  | 32     | 298     | 652456   |
 |        +-------------+--------+--------+---------+----------+
 |        | 8 chl, Zlib | 17379  | 32     | 275     | 470591   |
 |        +-------------+--------+--------+---------+----------+
 |        | 8 chl, IAA  | 15551  | 46     | 313     | 1315784  |
 +--------+-------------+--------+--------+---------+----------+

 +--------+-------------+--------+--------+---------+----+-----+
 |        | The number  |total   |downtime|network  |pages per |
 |        | of channels |time(ms)|(ms)    |bandwidth|second    |
 |        | and mode    |        |        |(mbps)   |          |
 |        +-------------+-----------------+---------+----------+
 |        | 2 chl, Zlib | 100% failure, timeout is 120s        |
 |        +-------------+--------+--------+---------+----------+
 | Redis  | 2 chl, IAA  | 62737  | 115    | 4547    | 387911   |
 |workload+-------------+--------+--------+---------+----------+
 |        | 4 chl, Zlib | 30% failure, timeout is 120s         |
 |        +-------------+--------+--------+---------+----------+
 |        | 4 chl, IAA  | 54645  | 177    | 5382    | 656865   |
 |        +-------------+--------+--------+---------+----------+
 |        | 8 chl, Zlib | 93488  | 74     | 1264    | 129486   |
 |        +-------------+--------+--------+---------+----------+
 |        | 8 chl, IAA  | 24367  | 303    | 6901    | 964380   |
 +--------+-------------+--------+--------+---------+----------+

v2:
  - add support for multifd compression accelerator
  - add support for the QPL accelerator in the multifd
    compression accelerator
  - fixed the issue that QPL was compiled into the migration
    module by default

Yuan Liu (4):
  migration: Introduce multifd-compression-accel parameter
  multifd: Implement multifd compression accelerator
  configure: add qpl option
  multifd: Introduce QPL compression accelerator

 hw/core/qdev-properties-system.c    |  11 +
 include/hw/qdev-properties-system.h |   4 +
 meson.build                         |   7 +
 meson_options.txt                   |   2 +
 migration/meson.build               |   1 +
 migration/migration-hmp-cmds.c      |  10 +
 migration/multifd-qpl.c             | 326 ++++++++++++++++++++++++++++
 migration/multifd.c                 |  38 +++-
 migration/multifd.h                 |   8 +
 migration/options.c                 |  24 ++
 migration/options.h                 |   1 +
 qapi/migration.json                 |  26 ++-
 scripts/meson-buildoptions.sh       |   3 +
 13 files changed, 458 insertions(+), 3 deletions(-)
 create mode 100644 migration/multifd-qpl.c

--
2.39.3