| Message ID | 20230906150853.22176-1-avihaih@nvidia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vfio/migration: Block VFIO migration with postcopy and background snapshot | expand |
On 9/6/23 17:08, Avihai Horon wrote: > Hello, > > Recently added VFIO migration is not compatible with some of the > pre-existing migration features. This was overlooked and today these > combinations are not blocked by QEMU. This series fixes it. > > Postcopy migration: > VFIO migration is not compatible with postcopy migration. A VFIO device > in the destination can't handle page faults for pages that have not been > sent yet. Doing such migration will cause the VM to crash in the > destination. > > Background snapshot: > Background snapshot allows creating a snapshot of the VM while it's > running and keeping it small by not including dirty RAM pages. > > The way it works is by first stopping the VM, saving the non-iterable > devices' state and then starting the VM and saving the RAM while write > protecting it with UFFD. The resulting snapshot represents the VM state > at snapshot start. > > VFIO migration is not compatible with background snapshot. > First of all, VFIO device state is not even saved in background snapshot > because only non-iterable device state is saved. But even if it was > saved, after starting the VM, a VFIO device could dirty pages without it > being detected by UFFD write protection. This would corrupt the > snapshot, as the RAM in it would not represent the RAM at snapshot > start. > > This series fixes it by blocking these combinations. This is done by > adding a .save_prepare() handler to struct SaveVMHandler. The > .save_prepare() handler is called early, even before migration starts, > and allows VFIO migration to check the migration capabilities and fail > migration if needed. > > Note that this series is based on the P2P series [1] sent a few weeks > ago. > > Comments and suggestions will be greatly appreciated. Applied to vfio-next. Thanks, C.
[ ... ]
> Applied to vfio-next.
On that topic I am preparing a PR.
Juan, Peter, Leonardo, is it ok for you if these migration changes
go through the VFIO tree ?
Thanks,
C.
On Thu, Sep 07, 2023 at 11:07:10AM +0200, Cédric Le Goater wrote: > [ ... ] > > > Applied to vfio-next. > > On that topic I am preparing a PR. > > Juan, Peter, Leonardo, is it ok for you if these migration changes > go through the VFIO tree ? All good here. Thanks,
Hello, Recently added VFIO migration is not compatible with some of the pre-existing migration features. This was overlooked and today these combinations are not blocked by QEMU. This series fixes it. Postcopy migration: VFIO migration is not compatible with postcopy migration. A VFIO device in the destination can't handle page faults for pages that have not been sent yet. Doing such migration will cause the VM to crash in the destination. Background snapshot: Background snapshot allows creating a snapshot of the VM while it's running and keeping it small by not including dirty RAM pages. The way it works is by first stopping the VM, saving the non-iterable devices' state and then starting the VM and saving the RAM while write protecting it with UFFD. The resulting snapshot represents the VM state at snapshot start. VFIO migration is not compatible with background snapshot. First of all, VFIO device state is not even saved in background snapshot because only non-iterable device state is saved. But even if it was saved, after starting the VM, a VFIO device could dirty pages without it being detected by UFFD write protection. This would corrupt the snapshot, as the RAM in it would not represent the RAM at snapshot start. This series fixes it by blocking these combinations. This is done by adding a .save_prepare() handler to struct SaveVMHandler. The .save_prepare() handler is called early, even before migration starts, and allows VFIO migration to check the migration capabilities and fail migration if needed. Note that this series is based on the P2P series [1] sent a few weeks ago. Comments and suggestions will be greatly appreciated. Thanks. Changes from v2 [3]: * Added a new patch that moves more migration initializations to migrate_init(). (Cedric) * Consolidated the two call sites of qemu_savevm_state_prepare() into migrate_init(). (Peter) * Added R-bs and Tested-by tags. Changes from v1 [2]: * Adopted Peter's suggestion to block migration upon migrate command using a new .save_prepare() handler in SaveVMHandlers. * Added R-bs by Cedric. [1] https://lore.kernel.org/qemu-devel/20230802081449.2528-1-avihaih@nvidia.com/ [2] https://lore.kernel.org/qemu-devel/20230828151842.11303-1-avihaih@nvidia.com/ [3] https://lore.kernel.org/qemu-devel/20230831125702.11263-1-avihaih@nvidia.com/ Avihai Horon (6): migration: Add migration prefix to functions in target.c vfio/migration: Fail adding device with enable-migration=on and existing blocker migration: Move more initializations to migrate_init() migration: Add .save_prepare() handler to struct SaveVMHandlers vfio/migration: Block VFIO migration with postcopy migration vfio/migration: Block VFIO migration with background snapshot include/migration/register.h | 5 +++++ migration/migration.h | 6 +++--- migration/savevm.h | 1 + hw/vfio/common.c | 7 +++++-- hw/vfio/migration.c | 31 +++++++++++++++++++++++++++++++ migration/migration.c | 33 ++++++++++++++++++++++----------- migration/savevm.c | 32 ++++++++++++++++++++++++++++---- migration/target.c | 8 ++++---- 8 files changed, 99 insertions(+), 24 deletions(-)