From patchwork Thu Oct 27 06:42:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 9398957 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7716560231 for ; Thu, 27 Oct 2016 06:58:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3DB172A056 for ; Thu, 27 Oct 2016 06:58:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 328772A05A; Thu, 27 Oct 2016 06:58:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 84B042A056 for ; Thu, 27 Oct 2016 06:58:29 +0000 (UTC) Received: from localhost ([::1]:39398 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzedw-0006b1-Hc for patchwork-qemu-devel@patchwork.kernel.org; Thu, 27 Oct 2016 02:58:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32908) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzeQd-00045H-1n for qemu-devel@nongnu.org; Thu, 27 Oct 2016 02:44:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bzeQb-0003rQ-CE for qemu-devel@nongnu.org; Thu, 27 Oct 2016 02:44:43 -0400 Received: from szxga03-in.huawei.com ([119.145.14.66]:59377) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1bzeQY-0003dZ-Vn for qemu-devel@nongnu.org; Thu, 27 Oct 2016 02:44:41 -0400 Received: from 172.24.1.137 (EHLO szxeml427-hub.china.huawei.com) ([172.24.1.137]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id CKK24296; Thu, 27 Oct 2016 14:44:19 +0800 (CST) Received: from localhost (10.177.24.212) by szxeml427-hub.china.huawei.com (10.82.67.182) with Microsoft SMTP Server id 14.3.235.1; Thu, 27 Oct 2016 14:44:08 +0800 From: zhanghailiang To: , Date: Thu, 27 Oct 2016 14:42:54 +0800 Message-ID: <1477550589-16288-4-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 2.7.2.windows.1 In-Reply-To: <1477550589-16288-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1477550589-16288-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.24.212] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.66 Subject: [Qemu-devel] [PATCH COLO-Frame (Base) v23 03/18] migration: Enter into COLO mode after migration if COLO is enabled X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xiecl.fnst@cn.fujitsu.com, zhanghailiang , lizhijian@cn.fujitsu.com, qemu-devel@nongnu.org, dgilbert@redhat.com, Gonglei Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add a new migration state: MIGRATION_STATUS_COLO. Migration source side enters this state after the first live migration successfully finished if COLO is enabled by command 'migrate_set_capability x-colo on'. We reuse migration thread, so the process of checkpointing will be handled in migration thread. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Gonglei Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah --- v23: - Add Reviewed-by tag v22: - Remove unnecessary 'migrate_set_state(&s->state, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED)' v21: - Don't mark the image with BDRV_O_INACTIVE flag if we are going into colo v19: - fix title to make it more exact. v11: - Rebase to master - Add Reviewed-by tag v10: - Simplify process by dropping colo thread and reusing migration thread. (Dave's suggestion) --- include/migration/colo.h | 3 +++ migration/colo.c | 29 +++++++++++++++++++++++++++++ migration/migration.c | 38 +++++++++++++++++++++++++++++++++----- migration/trace-events | 3 +++ qapi-schema.json | 4 +++- stubs/migration-colo.c | 9 +++++++++ 6 files changed, 80 insertions(+), 6 deletions(-) diff --git a/include/migration/colo.h b/include/migration/colo.h index 1c899a0..bf84b99 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -19,4 +19,7 @@ bool colo_supported(void); void colo_info_init(void); +void migrate_start_colo_process(MigrationState *s); +bool migration_in_colo_state(void); + #endif diff --git a/migration/colo.c b/migration/colo.c index d215057..f480431 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -11,9 +11,38 @@ */ #include "qemu/osdep.h" +#include "sysemu/sysemu.h" #include "migration/colo.h" +#include "trace.h" bool colo_supported(void) { return false; } + +bool migration_in_colo_state(void) +{ + MigrationState *s = migrate_get_current(); + + return (s->state == MIGRATION_STATUS_COLO); +} + +static void colo_process_checkpoint(MigrationState *s) +{ + qemu_mutex_lock_iothread(); + vm_start(); + qemu_mutex_unlock_iothread(); + trace_colo_vm_state_change("stop", "run"); + + /* TODO: COLO checkpoint savevm loop */ + +} + +void migrate_start_colo_process(MigrationState *s) +{ + qemu_mutex_unlock_iothread(); + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_COLO); + colo_process_checkpoint(s); + qemu_mutex_lock_iothread(); +} diff --git a/migration/migration.c b/migration/migration.c index f7dd9c6..462007d 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -695,6 +695,10 @@ MigrationInfo *qmp_query_migrate(Error **errp) get_xbzrle_cache_stats(info); break; + case MIGRATION_STATUS_COLO: + info->has_status = true; + /* TODO: display COLO specific information (checkpoint info etc.) */ + break; case MIGRATION_STATUS_COMPLETED: get_xbzrle_cache_stats(info); @@ -1113,7 +1117,8 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, params.shared = has_inc && inc; if (migration_is_setup_or_active(s->state) || - s->state == MIGRATION_STATUS_CANCELLING) { + s->state == MIGRATION_STATUS_CANCELLING || + s->state == MIGRATION_STATUS_COLO) { error_setg(errp, QERR_MIGRATION_ACTIVE); return; } @@ -1660,7 +1665,11 @@ static void migration_completion(MigrationState *s, int current_active_state, if (!ret) { ret = vm_stop_force_state(RUN_STATE_FINISH_MIGRATE); - if (ret >= 0) { + /* + * Don't mark the image with BDRV_O_INACTIVE flag if + * we will go into COLO stage later. + */ + if (ret >= 0 && !migrate_colo_enabled()) { ret = bdrv_inactivate_all(); } if (ret >= 0) { @@ -1702,8 +1711,11 @@ static void migration_completion(MigrationState *s, int current_active_state, goto fail_invalidate; } - migrate_set_state(&s->state, current_active_state, - MIGRATION_STATUS_COMPLETED); + if (!migrate_colo_enabled()) { + migrate_set_state(&s->state, current_active_state, + MIGRATION_STATUS_COMPLETED); + } + return; fail_invalidate: @@ -1748,6 +1760,7 @@ static void *migration_thread(void *opaque) bool entered_postcopy = false; /* The active state we expect to be in; ACTIVE or POSTCOPY_ACTIVE */ enum MigrationStatus current_active_state = MIGRATION_STATUS_ACTIVE; + bool enable_colo = migrate_colo_enabled(); rcu_register_thread(); @@ -1856,7 +1869,13 @@ static void *migration_thread(void *opaque) end_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_mutex_lock_iothread(); - qemu_savevm_state_cleanup(); + /* + * The resource has been allocated by migration will be reused in COLO + * process, so don't release them. + */ + if (!enable_colo) { + qemu_savevm_state_cleanup(); + } if (s->state == MIGRATION_STATUS_COMPLETED) { uint64_t transferred_bytes = qemu_ftell(s->to_dst_file); s->total_time = end_time - s->total_time; @@ -1869,6 +1888,15 @@ static void *migration_thread(void *opaque) } runstate_set(RUN_STATE_POSTMIGRATE); } else { + if (s->state == MIGRATION_STATUS_ACTIVE && enable_colo) { + migrate_start_colo_process(s); + qemu_savevm_state_cleanup(); + /* + * Fixme: we will run VM in COLO no matter its old running state. + * After exited COLO, we will keep running. + */ + old_vm_running = true; + } if (old_vm_running && !entered_postcopy) { vm_start(); } else { diff --git a/migration/trace-events b/migration/trace-events index dfee75a..a29e5a0 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -207,3 +207,6 @@ migration_tls_outgoing_handshake_complete(void) "" migration_tls_incoming_handshake_start(void) "" migration_tls_incoming_handshake_error(const char *err) "err=%s" migration_tls_incoming_handshake_complete(void) "" + +# migration/colo.c +colo_vm_state_change(const char *old, const char *new) "Change '%s' => '%s'" diff --git a/qapi-schema.json b/qapi-schema.json index 0fb4d7e..8b21215 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -459,12 +459,14 @@ # # @failed: some error occurred during migration process. # +# @colo: VM is in the process of fault tolerance. (since 2.8) +# # Since: 2.3 # ## { 'enum': 'MigrationStatus', 'data': [ 'none', 'setup', 'cancelling', 'cancelled', - 'active', 'postcopy-active', 'completed', 'failed' ] } + 'active', 'postcopy-active', 'completed', 'failed', 'colo' ] } ## # @MigrationInfo diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c index d215057..0c8eef4 100644 --- a/stubs/migration-colo.c +++ b/stubs/migration-colo.c @@ -17,3 +17,12 @@ bool colo_supported(void) { return false; } + +bool migration_in_colo_state(void) +{ + return false; +} + +void migrate_start_colo_process(MigrationState *s) +{ +}