From patchwork Tue Oct 18 12:10:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 9382013 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EDF0260839 for ; Tue, 18 Oct 2016 12:11:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDD8129575 for ; Tue, 18 Oct 2016 12:11:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D29F729577; Tue, 18 Oct 2016 12:11:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0A30929575 for ; Tue, 18 Oct 2016 12:11:49 +0000 (UTC) Received: from localhost ([::1]:41050 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bwTFE-0006ny-A7 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 18 Oct 2016 08:11:48 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36357) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bwTEh-0006ky-Sk for qemu-devel@nongnu.org; Tue, 18 Oct 2016 08:11:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bwTEd-00051x-KF for qemu-devel@nongnu.org; Tue, 18 Oct 2016 08:11:15 -0400 Received: from szxga03-in.huawei.com ([119.145.14.66]:20789) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1bwTEc-0004xN-Qf for qemu-devel@nongnu.org; Tue, 18 Oct 2016 08:11:11 -0400 Received: from 172.24.1.137 (EHLO szxeml428-hub.china.huawei.com) ([172.24.1.137]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id CJR35512; Tue, 18 Oct 2016 20:10:54 +0800 (CST) Received: from localhost (10.177.24.212) by szxeml428-hub.china.huawei.com (10.82.67.183) with Microsoft SMTP Server id 14.3.235.1; Tue, 18 Oct 2016 20:10:47 +0800 From: zhanghailiang To: , Date: Tue, 18 Oct 2016 20:10:00 +0800 Message-ID: <1476792613-11712-5-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 2.7.2.windows.1 In-Reply-To: <1476792613-11712-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1476792613-11712-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.24.212] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.66 Subject: [Qemu-devel] [PATCH COLO-Frame (Base) v21 04/17] migration: Switch to COLO process after finishing loadvm X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xiecl.fnst@cn.fujitsu.com, zhanghailiang , lizhijian@cn.fujitsu.com, qemu-devel@nongnu.org, dgilbert@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Switch from normal migration loadvm process into COLO checkpoint process if COLO mode is enabled. We add three new members to struct MigrationIncomingState, 'have_colo_incoming_thread' and 'colo_incoming_thread' record the COLO related thread for secondary VM, 'migration_incoming_co' records the original migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Amit Shah --- v19: - Fix the title and comments v12: - Add Reviewed-by tag v11: - We moved the place of bdrv_invalidate_cache_all(), but done the deleting work in other patch. Fix it. - Add documentation for colo in 'MigrationStatus' (Eric's review comment) v10: - fix a bug about fd leak which is found by Dave. --- include/migration/colo.h | 7 +++++++ include/migration/migration.h | 7 +++++++ migration/colo-comm.c | 10 ++++++++++ migration/colo.c | 21 +++++++++++++++++++++ migration/migration.c | 12 ++++++++++++ stubs/migration-colo.c | 10 ++++++++++ 6 files changed, 67 insertions(+) diff --git a/include/migration/colo.h b/include/migration/colo.h index bf84b99..b40676c 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -15,6 +15,8 @@ #include "qemu-common.h" #include "migration/migration.h" +#include "qemu/coroutine_int.h" +#include "qemu/thread.h" bool colo_supported(void); void colo_info_init(void); @@ -22,4 +24,9 @@ void colo_info_init(void); void migrate_start_colo_process(MigrationState *s); bool migration_in_colo_state(void); +/* loadvm */ +bool migration_incoming_enable_colo(void); +void migration_incoming_exit_colo(void); +void *colo_process_incoming_thread(void *opaque); +bool migration_incoming_in_colo_state(void); #endif diff --git a/include/migration/migration.h b/include/migration/migration.h index 34f442b..c309d23 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -21,6 +21,7 @@ #include "migration/vmstate.h" #include "qapi-types.h" #include "exec/cpu-common.h" +#include "qemu/coroutine_int.h" #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002 @@ -107,6 +108,12 @@ struct MigrationIncomingState { QEMUBH *bh; int state; + + bool have_colo_incoming_thread; + QemuThread colo_incoming_thread; + /* The coroutine we should enter (back) after failover */ + Coroutine *migration_incoming_co; + /* See savevm.c */ LoadStateEntry_Head loadvm_handlers; }; diff --git a/migration/colo-comm.c b/migration/colo-comm.c index a2d5185..3af9333 100644 --- a/migration/colo-comm.c +++ b/migration/colo-comm.c @@ -49,3 +49,13 @@ void colo_info_init(void) { vmstate_register(NULL, 0, &colo_state, &colo_info); } + +bool migration_incoming_enable_colo(void) +{ + return colo_info.colo_requested; +} + +void migration_incoming_exit_colo(void) +{ + colo_info.colo_requested = 0; +} diff --git a/migration/colo.c b/migration/colo.c index fd3ceeb..968cd51 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -27,6 +27,13 @@ bool migration_in_colo_state(void) return (s->state == MIGRATION_STATUS_COLO); } +bool migration_incoming_in_colo_state(void) +{ + MigrationIncomingState *mis = migration_incoming_get_current(); + + return mis && (mis->state == MIGRATION_STATUS_COLO); +} + static void colo_process_checkpoint(MigrationState *s) { qemu_mutex_lock_iothread(); @@ -48,3 +55,17 @@ void migrate_start_colo_process(MigrationState *s) colo_process_checkpoint(s); qemu_mutex_lock_iothread(); } + +void *colo_process_incoming_thread(void *opaque) +{ + MigrationIncomingState *mis = opaque; + + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_COLO); + + /* TODO: COLO checkpoint restore loop */ + + migration_incoming_exit_colo(); + + return NULL; +} diff --git a/migration/migration.c b/migration/migration.c index 462007d..13ea7e9 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -407,6 +407,18 @@ static void process_incoming_migration_co(void *opaque) /* Else if something went wrong then just fall out of the normal exit */ } + /* we get COLO info, and know if we are in COLO mode */ + if (!ret && migration_incoming_enable_colo()) { + mis->migration_incoming_co = qemu_coroutine_self(); + qemu_thread_create(&mis->colo_incoming_thread, "COLO incoming", + colo_process_incoming_thread, mis, QEMU_THREAD_JOINABLE); + mis->have_colo_incoming_thread = true; + qemu_coroutine_yield(); + + /* Wait checkpoint incoming thread exit before free resource */ + qemu_thread_join(&mis->colo_incoming_thread); + } + qemu_fclose(f); free_xbzrle_decoded_buf(); diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c index 0c8eef4..7b72395 100644 --- a/stubs/migration-colo.c +++ b/stubs/migration-colo.c @@ -23,6 +23,16 @@ bool migration_in_colo_state(void) return false; } +bool migration_incoming_in_colo_state(void) +{ + return false; +} + void migrate_start_colo_process(MigrationState *s) { } + +void *colo_process_incoming_thread(void *opaque) +{ + return NULL; +}