From patchwork Fri Jan 29 05:27:18 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 8159251 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id ACF069F440 for ; Fri, 29 Jan 2016 05:30:10 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E77AA200E9 for ; Fri, 29 Jan 2016 05:30:08 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 05C8B201F2 for ; Fri, 29 Jan 2016 05:30:07 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1ag-0007Zs-Ol; Fri, 29 Jan 2016 05:27:26 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1ae-0007ZJ-CL for xen-devel@lists.xen.org; Fri, 29 Jan 2016 05:27:24 +0000 Received: from [85.158.143.35] by server-3.bemta-4.messagelabs.com id 85/A1-31122-B38FAA65; Fri, 29 Jan 2016 05:27:23 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-3.tower-21.messagelabs.com!1454045239!12339147!1 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 9016 invoked from network); 29 Jan 2016 05:27:21 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-3.tower-21.messagelabs.com with SMTP; 29 Jan 2016 05:27:21 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="3106327" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 29 Jan 2016 13:27:16 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 1FD0B418251A; Fri, 29 Jan 2016 13:26:23 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Fri, 29 Jan 2016 13:26:58 +0800 From: Wen Congyang To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 29 Jan 2016 13:27:18 +0800 Message-ID: <1454045254-3711-3-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> References: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 1FD0B418251A.A9167 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Ian Campbell , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Ian Jackson , Yang Hongyang Subject: [Xen-devel] [PATCH v7 02/18] tools/libxl: move remus code into libxl_remus.c X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After previous refactoring, we are now able to move all remus code into a separate file libxl_remus.c. Export following functions for internal use: - Remus callbacks * libxl__remus_domain_suspend_callback * libxl__remus_domain_resume_callback * libxl__remus_domain_save_checkpoint_callback * libxl__remus_domain_restore_checkpoint_callback - setup/teardown Remus: * libxl__remus_setup * libxl__remus_teardown Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Campbell CC: Ian Jackson CC: Wei Liu Acked-by:Ian Campbell Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxl/Makefile | 2 +- tools/libxl/libxl.c | 69 --------- tools/libxl/libxl_create.c | 27 ---- tools/libxl/libxl_dom.c | 223 --------------------------- tools/libxl/libxl_internal.h | 15 +- tools/libxl/libxl_remus.c | 348 +++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 362 insertions(+), 322 deletions(-) create mode 100644 tools/libxl/libxl_remus.c diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile index 620720e..7d64ecc 100644 --- a/tools/libxl/Makefile +++ b/tools/libxl/Makefile @@ -64,7 +64,7 @@ else LIBXL_OBJS-y += libxl_no_convert_callout.o endif -LIBXL_OBJS-y += libxl_remus_device.o libxl_remus_disk_drbd.o +LIBXL_OBJS-y += libxl_remus.o libxl_remus_device.o libxl_remus_disk_drbd.o LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o libxl_x86.o libxl_psr.o LIBXL_OBJS-$(CONFIG_ARM) += libxl_nocpuid.o libxl_arm.o libxl_libfdt_compat.o diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 5346a0c..6347097 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -831,12 +831,6 @@ out: return ptr; } -static void libxl__remus_setup(libxl__egc *egc, - libxl__domain_suspend_state *dss); -static void remus_setup_done(libxl__egc *egc, - libxl__remus_devices_state *rds, int rc); -static void remus_setup_failed(libxl__egc *egc, - libxl__remus_devices_state *rds, int rc); static void remus_failover_cb(libxl__egc *egc, libxl__domain_suspend_state *dss, int rc); @@ -893,69 +887,6 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, return AO_CREATE_FAIL(rc); } -static void libxl__remus_setup(libxl__egc *egc, - libxl__domain_suspend_state *dss) -{ - /* Convenience aliases */ - libxl__remus_devices_state *const rds = &dss->rds; - const libxl_domain_remus_info *const info = dss->remus; - - STATE_AO_GC(dss->ao); - - if (libxl_defbool_val(info->netbuf)) { - if (!libxl__netbuffer_enabled(gc)) { - LOG(ERROR, "Remus: No support for network buffering"); - goto out; - } - rds->device_kind_flags |= (1 << LIBXL__DEVICE_KIND_VIF); - } - - if (libxl_defbool_val(info->diskbuf)) - rds->device_kind_flags |= (1 << LIBXL__DEVICE_KIND_VBD); - - rds->ao = ao; - rds->domid = dss->domid; - rds->callback = remus_setup_done; - - dss->sws.checkpoint_callback = remus_checkpoint_stream_written; - - libxl__remus_devices_setup(egc, rds); - return; - -out: - dss->callback(egc, dss, ERROR_FAIL); -} - -static void remus_setup_done(libxl__egc *egc, - libxl__remus_devices_state *rds, int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - STATE_AO_GC(dss->ao); - - if (!rc) { - libxl__domain_save(egc, dss); - return; - } - - LOG(ERROR, "Remus: failed to setup device for guest with domid %u, rc %d", - dss->domid, rc); - rds->callback = remus_setup_failed; - libxl__remus_devices_teardown(egc, rds); -} - -static void remus_setup_failed(libxl__egc *egc, - libxl__remus_devices_state *rds, int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - STATE_AO_GC(dss->ao); - - if (rc) - LOG(ERROR, "Remus: failed to teardown device after setup failed" - " for guest with domid %u, rc %d", dss->domid, rc); - - dss->callback(egc, dss, rc); -} - static void remus_failover_cb(libxl__egc *egc, libxl__domain_suspend_state *dss, int rc) { diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index 8b1efe5..6f1cf93 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -697,33 +697,6 @@ static int store_libxl_entry(libxl__gc *gc, uint32_t domid, libxl_device_model_version_to_string(b_info->device_model_version)); } -/*----- remus asynchronous checkpoint callback -----*/ - -static void remus_checkpoint_stream_done( - libxl__egc *egc, libxl__stream_read_state *srs, int rc); - -static void libxl__remus_domain_restore_checkpoint_callback(void *data) -{ - libxl__save_helper_state *shs = data; - libxl__domain_create_state *dcs = shs->caller_state; - libxl__egc *egc = shs->egc; - STATE_AO_GC(dcs->ao); - - libxl__stream_read_start_checkpoint(egc, &dcs->srs); -} - -static void remus_checkpoint_stream_done( - libxl__egc *egc, libxl__stream_read_state *stream, int rc) -{ - libxl__xc_domain_saverestore_async_callback_done(egc, &stream->shs, rc); -} - -static void libxl__remus_restore_setup(libxl__egc *egc, - libxl__domain_create_state *dcs) -{ - dcs->srs.checkpoint_callback = remus_checkpoint_stream_done; -} - /*----- main domain creation -----*/ /* We have a linear control flow; only one event callback is diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 9e28bc4..81bd464 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -1479,196 +1479,6 @@ int libxl__save_emulator_xenstore_data(libxl__domain_suspend_state *dss, return rc; } -/*----- remus callbacks -----*/ -static void remus_domain_suspend_callback_common_done(libxl__egc *egc, - libxl__domain_suspend_state *dss, int ok); -static void remus_devices_postsuspend_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc); -static void remus_devices_preresume_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc); - -static void libxl__remus_domain_suspend_callback(void *data) -{ - libxl__save_helper_state *shs = data; - libxl__egc *egc = shs->egc; - libxl__domain_suspend_state *dss = shs->caller_state; - - dss->callback_common_done = remus_domain_suspend_callback_common_done; - libxl__domain_suspend(egc, dss); -} - -static void remus_domain_suspend_callback_common_done(libxl__egc *egc, - libxl__domain_suspend_state *dss, int rc) -{ - if (rc) - goto out; - - libxl__remus_devices_state *const rds = &dss->rds; - rds->callback = remus_devices_postsuspend_cb; - libxl__remus_devices_postsuspend(egc, rds); - return; - -out: - dss->rc = rc; - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); -} - -static void remus_devices_postsuspend_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - - if (rc) - goto out; - - rc = 0; - -out: - if (rc) - dss->rc = rc; - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); -} - -static void libxl__remus_domain_resume_callback(void *data) -{ - libxl__save_helper_state *shs = data; - libxl__egc *egc = shs->egc; - libxl__domain_suspend_state *dss = shs->caller_state; - STATE_AO_GC(dss->ao); - - libxl__remus_devices_state *const rds = &dss->rds; - rds->callback = remus_devices_preresume_cb; - libxl__remus_devices_preresume(egc, rds); -} - -static void remus_devices_preresume_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - STATE_AO_GC(dss->ao); - - if (rc) - goto out; - - /* Resumes the domain and the device model */ - rc = libxl__domain_resume(gc, dss->domid, /* Fast Suspend */1); - if (rc) - goto out; - - rc = 0; - -out: - if (rc) - dss->rc = rc; - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); -} - -/*----- remus asynchronous checkpoint callback -----*/ - -static void remus_devices_commit_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc); -static void remus_next_checkpoint(libxl__egc *egc, libxl__ev_time *ev, - const struct timeval *requested_abs, - int rc); - -static void libxl__remus_domain_save_checkpoint_callback(void *data) -{ - libxl__save_helper_state *shs = data; - libxl__domain_suspend_state *dss = shs->caller_state; - libxl__egc *egc = shs->egc; - STATE_AO_GC(dss->ao); - - libxl__stream_write_start_checkpoint(egc, &dss->sws); -} - -void remus_checkpoint_stream_written( - libxl__egc *egc, libxl__stream_write_state *sws, int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(sws, *dss, sws); - - /* Convenience aliases */ - libxl__remus_devices_state *const rds = &dss->rds; - - STATE_AO_GC(dss->ao); - - if (rc) { - LOG(ERROR, "Failed to save device model. Terminating Remus.."); - goto out; - } - - rds->callback = remus_devices_commit_cb; - libxl__remus_devices_commit(egc, rds); - - return; - -out: - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, 0); -} - -static void remus_devices_commit_cb(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - - STATE_AO_GC(dss->ao); - - if (rc) { - LOG(ERROR, "Failed to do device commit op." - " Terminating Remus.."); - goto out; - } - - /* - * At this point, we have successfully checkpointed the guest and - * committed it at the backup. We'll come back after the checkpoint - * interval to checkpoint the guest again. Until then, let the guest - * continue execution. - */ - - /* Set checkpoint interval timeout */ - rc = libxl__ev_time_register_rel(ao, &dss->checkpoint_timeout, - remus_next_checkpoint, - dss->interval); - - if (rc) - goto out; - - return; - -out: - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, 0); -} - -static void remus_next_checkpoint(libxl__egc *egc, libxl__ev_time *ev, - const struct timeval *requested_abs, - int rc) -{ - libxl__domain_suspend_state *dss = - CONTAINER_OF(ev, *dss, checkpoint_timeout); - - STATE_AO_GC(dss->ao); - - if (rc == ERROR_TIMEDOUT) /* As intended */ - rc = 0; - - /* - * Time to checkpoint the guest again. We return 1 to libxc - * (xc_domain_save.c). in order to continue executing the infinite loop - * (suspend, checkpoint, resume) in xc_domain_save(). - */ - - if (rc) - dss->rc = rc; - - libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); -} - /*----- main code for saving, in order of execution -----*/ void libxl__domain_save(libxl__egc *egc, libxl__domain_suspend_state *dss) @@ -1782,13 +1592,6 @@ static void stream_done(libxl__egc *egc, domain_save_done(egc, sws->dss, rc); } -static void libxl__remus_teardown(libxl__egc *egc, - libxl__domain_suspend_state *dss, - int rc); -static void remus_teardown_done(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc); - static void domain_save_done(libxl__egc *egc, libxl__domain_suspend_state *dss, int rc) { @@ -1817,32 +1620,6 @@ static void domain_save_done(libxl__egc *egc, dss->callback(egc, dss, rc); } -static void libxl__remus_teardown(libxl__egc *egc, - libxl__domain_suspend_state *dss, - int rc) -{ - EGC_GC; - - LOG(WARN, "Remus: Domain suspend terminated with rc %d," - " teardown Remus devices...", rc); - dss->rds.callback = remus_teardown_done; - libxl__remus_devices_teardown(egc, &dss->rds); -} - -static void remus_teardown_done(libxl__egc *egc, - libxl__remus_devices_state *rds, - int rc) -{ - libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); - STATE_AO_GC(dss->ao); - - if (rc) - LOG(ERROR, "Remus: failed to teardown device for guest with domid %u," - " rc %d", dss->domid, rc); - - dss->callback(egc, dss, rc); -} - /*==================== Miscellaneous ====================*/ char *libxl__uuid2string(libxl__gc *gc, const libxl_uuid uuid) diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index abc0eac..7005d6b 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3506,9 +3506,20 @@ _hidden void libxl__domain_suspend(libxl__egc *egc, /* used by libxc to suspend the guest during migration */ _hidden void libxl__domain_suspend_callback(void *data); +/* Remus callbacks for save */ +_hidden void libxl__remus_domain_suspend_callback(void *data); +_hidden void libxl__remus_domain_resume_callback(void *data); +_hidden void libxl__remus_domain_save_checkpoint_callback(void *data); +/* Remus setup and teardown*/ +_hidden void libxl__remus_setup(libxl__egc *egc, + libxl__domain_suspend_state *dss); +_hidden void libxl__remus_teardown(libxl__egc *egc, + libxl__domain_suspend_state *dss, + int rc); /* Remus callbacks for restore */ -_hidden void remus_checkpoint_stream_written( - libxl__egc *egc, libxl__stream_write_state *sws, int rc); +_hidden void libxl__remus_domain_restore_checkpoint_callback(void *data); +_hidden void libxl__remus_restore_setup(libxl__egc *egc, + libxl__domain_create_state *dcs); /* diff --git a/tools/libxl/libxl_remus.c b/tools/libxl/libxl_remus.c new file mode 100644 index 0000000..e3caf7d --- /dev/null +++ b/tools/libxl/libxl_remus.c @@ -0,0 +1,348 @@ +/* + * Copyright (C) 2009 Citrix Ltd. + * Author Vincent Hanquez + * Yang Hongyang + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#include "libxl_osdeps.h" /* must come before any other headers */ + +#include "libxl_internal.h" + +/*-------------------- Remus setup and teardown ---------------------*/ + +static void remus_setup_done(libxl__egc *egc, + libxl__remus_devices_state *rds, int rc); +static void remus_setup_failed(libxl__egc *egc, + libxl__remus_devices_state *rds, int rc); +static void remus_checkpoint_stream_written( + libxl__egc *egc, libxl__stream_write_state *sws, int rc); + +void libxl__remus_setup(libxl__egc *egc, + libxl__domain_suspend_state *dss) +{ + /* Convenience aliases */ + libxl__remus_devices_state *const rds = &dss->rds; + const libxl_domain_remus_info *const info = dss->remus; + + STATE_AO_GC(dss->ao); + + if (libxl_defbool_val(info->netbuf)) { + if (!libxl__netbuffer_enabled(gc)) { + LOG(ERROR, "Remus: No support for network buffering"); + goto out; + } + rds->device_kind_flags |= (1 << LIBXL__DEVICE_KIND_VIF); + } + + if (libxl_defbool_val(info->diskbuf)) + rds->device_kind_flags |= (1 << LIBXL__DEVICE_KIND_VBD); + + rds->ao = ao; + rds->domid = dss->domid; + rds->callback = remus_setup_done; + + dss->sws.checkpoint_callback = remus_checkpoint_stream_written; + + libxl__remus_devices_setup(egc, rds); + return; + +out: + dss->callback(egc, dss, ERROR_FAIL); +} + +static void remus_setup_done(libxl__egc *egc, + libxl__remus_devices_state *rds, int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + STATE_AO_GC(dss->ao); + + if (!rc) { + libxl__domain_save(egc, dss); + return; + } + + LOG(ERROR, "Remus: failed to setup device for guest with domid %u, rc %d", + dss->domid, rc); + rds->callback = remus_setup_failed; + libxl__remus_devices_teardown(egc, rds); +} + +static void remus_setup_failed(libxl__egc *egc, + libxl__remus_devices_state *rds, int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + STATE_AO_GC(dss->ao); + + if (rc) + LOG(ERROR, "Remus: failed to teardown device after setup failed" + " for guest with domid %u, rc %d", dss->domid, rc); + + dss->callback(egc, dss, rc); +} + +static void remus_teardown_done(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc); +void libxl__remus_teardown(libxl__egc *egc, + libxl__domain_suspend_state *dss, + int rc) +{ + EGC_GC; + + LOG(WARN, "Remus: Domain suspend terminated with rc %d," + " teardown Remus devices...", rc); + dss->rds.callback = remus_teardown_done; + libxl__remus_devices_teardown(egc, &dss->rds); +} + +static void remus_teardown_done(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + STATE_AO_GC(dss->ao); + + if (rc) + LOG(ERROR, "Remus: failed to teardown device for guest with domid %u," + " rc %d", dss->domid, rc); + + dss->callback(egc, dss, rc); +} + +/*---------------------- remus callbacks (save) -----------------------*/ + +static void remus_domain_suspend_callback_common_done(libxl__egc *egc, + libxl__domain_suspend_state *dss, int ok); +static void remus_devices_postsuspend_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc); +static void remus_devices_preresume_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc); + +void libxl__remus_domain_suspend_callback(void *data) +{ + libxl__save_helper_state *shs = data; + libxl__egc *egc = shs->egc; + libxl__domain_suspend_state *dss = shs->caller_state; + + dss->callback_common_done = remus_domain_suspend_callback_common_done; + libxl__domain_suspend(egc, dss); +} + +static void remus_domain_suspend_callback_common_done(libxl__egc *egc, + libxl__domain_suspend_state *dss, int rc) +{ + if (rc) + goto out; + + libxl__remus_devices_state *const rds = &dss->rds; + rds->callback = remus_devices_postsuspend_cb; + libxl__remus_devices_postsuspend(egc, rds); + return; + +out: + dss->rc = rc; + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); +} + +static void remus_devices_postsuspend_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + + if (rc) + goto out; + + rc = 0; + +out: + if (rc) + dss->rc = rc; + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); +} + +void libxl__remus_domain_resume_callback(void *data) +{ + libxl__save_helper_state *shs = data; + libxl__egc *egc = shs->egc; + libxl__domain_suspend_state *dss = shs->caller_state; + STATE_AO_GC(dss->ao); + + libxl__remus_devices_state *const rds = &dss->rds; + rds->callback = remus_devices_preresume_cb; + libxl__remus_devices_preresume(egc, rds); +} + +static void remus_devices_preresume_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + STATE_AO_GC(dss->ao); + + if (rc) + goto out; + + /* Resumes the domain and the device model */ + rc = libxl__domain_resume(gc, dss->domid, /* Fast Suspend */1); + if (rc) + goto out; + + rc = 0; + +out: + if (rc) + dss->rc = rc; + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); +} + +/*----- remus asynchronous checkpoint callback -----*/ + +static void remus_devices_commit_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc); +static void remus_next_checkpoint(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, + int rc); + +void libxl__remus_domain_save_checkpoint_callback(void *data) +{ + libxl__save_helper_state *shs = data; + libxl__domain_suspend_state *dss = shs->caller_state; + libxl__egc *egc = shs->egc; + STATE_AO_GC(dss->ao); + + libxl__stream_write_start_checkpoint(egc, &dss->sws); +} + +static void remus_checkpoint_stream_written( + libxl__egc *egc, libxl__stream_write_state *sws, int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(sws, *dss, sws); + + /* Convenience aliases */ + libxl__remus_devices_state *const rds = &dss->rds; + + STATE_AO_GC(dss->ao); + + if (rc) { + LOG(ERROR, "Failed to save device model. Terminating Remus.."); + goto out; + } + + rds->callback = remus_devices_commit_cb; + libxl__remus_devices_commit(egc, rds); + + return; + +out: + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, 0); +} + +static void remus_devices_commit_cb(libxl__egc *egc, + libxl__remus_devices_state *rds, + int rc) +{ + libxl__domain_suspend_state *dss = CONTAINER_OF(rds, *dss, rds); + + STATE_AO_GC(dss->ao); + + if (rc) { + LOG(ERROR, "Failed to do device commit op." + " Terminating Remus.."); + goto out; + } + + /* + * At this point, we have successfully checkpointed the guest and + * committed it at the backup. We'll come back after the checkpoint + * interval to checkpoint the guest again. Until then, let the guest + * continue execution. + */ + + /* Set checkpoint interval timeout */ + rc = libxl__ev_time_register_rel(ao, &dss->checkpoint_timeout, + remus_next_checkpoint, + dss->interval); + + if (rc) + goto out; + + return; + +out: + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, 0); +} + +static void remus_next_checkpoint(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, + int rc) +{ + libxl__domain_suspend_state *dss = + CONTAINER_OF(ev, *dss, checkpoint_timeout); + + STATE_AO_GC(dss->ao); + + if (rc == ERROR_TIMEDOUT) /* As intended */ + rc = 0; + + /* + * Time to checkpoint the guest again. We return 1 to libxc + * (xc_domain_save.c). in order to continue executing the infinite loop + * (suspend, checkpoint, resume) in xc_domain_save(). + */ + + if (rc) + dss->rc = rc; + + libxl__xc_domain_saverestore_async_callback_done(egc, &dss->sws.shs, !rc); +} + +/*---------------------- remus callbacks (restore) -----------------------*/ + +/*----- remus asynchronous checkpoint callback -----*/ + +static void remus_checkpoint_stream_done( + libxl__egc *egc, libxl__stream_read_state *srs, int rc); + +void libxl__remus_domain_restore_checkpoint_callback(void *data) +{ + libxl__save_helper_state *shs = data; + libxl__domain_create_state *dcs = shs->caller_state; + libxl__egc *egc = shs->egc; + STATE_AO_GC(dcs->ao); + + libxl__stream_read_start_checkpoint(egc, &dcs->srs); +} + +static void remus_checkpoint_stream_done( + libxl__egc *egc, libxl__stream_read_state *stream, int rc) +{ + libxl__xc_domain_saverestore_async_callback_done(egc, &stream->shs, rc); +} + +void libxl__remus_restore_setup(libxl__egc *egc, + libxl__domain_create_state *dcs) +{ + dcs->srs.checkpoint_callback = remus_checkpoint_stream_done; +} + +/* + * Local variables: + * mode: C + * c-basic-offset: 4 + * indent-tabs-mode: nil + * End: + */