From patchwork Mon Feb 22 02:52:07 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 8370921 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id DE64E9F314 for ; Mon, 22 Feb 2016 02:54:53 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5BE7C20411 for ; Mon, 22 Feb 2016 02:54:52 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C64EB203C2 for ; Mon, 22 Feb 2016 02:54:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aXgbT-000119-Nr; Mon, 22 Feb 2016 02:52:03 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aXgbS-00010h-1U for xen-devel@lists.xen.org; Mon, 22 Feb 2016 02:52:02 +0000 Received: from [193.109.254.147] by server-7.bemta-14.messagelabs.com id EB/F1-29985-1D77AC65; Mon, 22 Feb 2016 02:52:01 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-4.tower-27.messagelabs.com!1456109514!24622642!2 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 5091 invoked from network); 22 Feb 2016 02:51:57 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-4.tower-27.messagelabs.com with SMTP; 22 Feb 2016 02:51:57 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="3766001" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 22 Feb 2016 10:51:49 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 4CD8B42AC848; Mon, 22 Feb 2016 10:49:56 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Mon, 22 Feb 2016 10:51:47 +0800 From: Wen Congyang To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Mon, 22 Feb 2016 10:52:07 +0800 Message-ID: <1456109555-28299-4-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1456109555-28299-1-git-send-email-wency@cn.fujitsu.com> References: <1456109555-28299-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 4CD8B42AC848.ACD76 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v10 03/31] tools/libxl: Add back channel to allow migration target send data back X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In COLO mode, secondary needs to send the following data to primary: 1. In libxl Secondary sends the following CHECKPOINT_CONTEXT to primary: CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED 2. In libxc Secondary sends the dirty pfn list to primary But the io_fd only can be written in primary, and only can be read in secondary. Save recv_fd in domain_suspend_state, and send_fd in domain_create_state. Extend libxl_domain_create_restore API, add a send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate the API change. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Acked-by: Wei Liu --- tools/libxl/libxl.c | 3 +-- tools/libxl/libxl.h | 29 ++++++++++++++++++++++++++++- tools/libxl/libxl_create.c | 10 ++++++---- tools/libxl/libxl_internal.h | 2 ++ tools/libxl/xl_cmdimpl.c | 8 +++++++- tools/ocaml/libs/xl/xenlight_stubs.c | 2 +- 6 files changed, 45 insertions(+), 9 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 4cdc169..e9ab78c 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -834,7 +834,6 @@ out: static void remus_failover_cb(libxl__egc *egc, libxl__domain_save_state *dss, int rc); -/* TODO: Explicit Checkpoint acknowledgements via recv_fd. */ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, uint32_t domid, int send_fd, int recv_fd, const libxl_asyncop_how *ao_how) @@ -871,7 +870,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->callback = remus_failover_cb; dss->domid = domid; dss->fd = send_fd; - /* TODO do something with recv_fd */ + dss->recv_fd = recv_fd; dss->type = type; dss->live = 1; dss->debug = 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index f9e3ef5..5a286ec 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -639,6 +639,15 @@ typedef struct libxl__ctx libxl_ctx; #define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1 /* + * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_BACK_FD 1 + * + * If this is defined, libxl_domain_create_restore()'s API has changed to + * include a send_back_fd param which used for libxl migration back channel + * during COLO. + */ +#define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_BACK_FD 1 + +/* * LIBXL_HAVE_CREATEINFO_PVH * If this is defined, then libxl supports creation of a PVH guest. */ @@ -1165,6 +1174,7 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, LIBXL_EXTERNAL_CALLERS_ONLY; int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, + int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1185,7 +1195,7 @@ int static inline libxl_domain_create_restore_0x040200( libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore( - ctx, d_config, domid, restore_fd, ¶ms, ao_how, aop_console_how); + ctx, d_config, domid, restore_fd, -1, ¶ms, ao_how, aop_console_how); libxl_domain_restore_params_dispose(¶ms); return ret; @@ -1193,6 +1203,23 @@ int static inline libxl_domain_create_restore_0x040200( #define libxl_domain_create_restore libxl_domain_create_restore_0x040200 +#elif defined(LIBXL_API_VERSION) && LIBXL_API_VERSION >= 0x040400 \ + && LIBXL_API_VERSION < 0x040700 + +int static inline libxl_domain_create_restore_0x040400( + libxl_ctx *ctx, libxl_domain_config *d_config, + uint32_t *domid, int restore_fd, + const libxl_domain_restore_params *params, + const libxl_asyncop_how *ao_how, + const libxl_asyncprogress_how *aop_console_how) + LIBXL_EXTERNAL_CALLERS_ONLY +{ + return libxl_domain_create_restore(ctx, d_config, domid, restore_fd, + -1, params, ao_how, aop_console_how); +} + +#define libxl_domain_create_restore libxl_domain_create_restore_0x040400 + #endif int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index f1028bc..525bf85 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1572,7 +1572,7 @@ static void domain_create_cb(libxl__egc *egc, int rc, uint32_t domid); static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1587,6 +1587,7 @@ static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_domain_config_init(&cdcs->dcs.guest_config_saved); libxl_domain_config_copy(ctx, &cdcs->dcs.guest_config_saved, d_config); cdcs->dcs.restore_fd = cdcs->dcs.libxc_fd = restore_fd; + cdcs->dcs.send_back_fd = send_back_fd; if (restore_fd > -1) { cdcs->dcs.restore_params = *params; rc = libxl__fd_flags_modify_save(gc, cdcs->dcs.restore_fd, @@ -1765,18 +1766,19 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, -1, NULL, + return do_domain_create(ctx, d_config, domid, -1, -1, NULL, ao_how, aop_console_how); } int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, + int send_back_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, restore_fd, params, - ao_how, aop_console_how); + return do_domain_create(ctx, d_config, domid, restore_fd, send_back_fd, + params, ao_how, aop_console_how); } int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 53a6388..3d3e8e8 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3134,6 +3134,7 @@ struct libxl__domain_save_state { uint32_t domid; int fd; int fdfl; /* original flags on fd */ + int recv_fd; libxl_domain_type type; int live; int debug; @@ -3467,6 +3468,7 @@ struct libxl__domain_create_state { libxl_domain_config guest_config_saved; /* vanilla config */ int restore_fd, libxc_fd; int restore_fdfl; /* original flags of restore_fd */ + int send_back_fd; libxl_domain_restore_params restore_params; uint32_t domid_soft_reset; libxl__domain_create_cb *callback; diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 09037ca..df7268b 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -159,6 +159,7 @@ struct domain_create { char *extra_config; /* extra config string */ const char *restore_file; int migrate_fd; /* -1 means none */ + int send_back_fd; /* -1 means none */ char **migration_domname_r; /* from malloc */ }; @@ -2686,6 +2687,7 @@ static uint32_t create_domain(struct domain_create *dom_info) int config_len = 0; int restore_fd = -1; int restore_fd_to_close = -1; + int send_back_fd = -1; const libxl_asyncprogress_how *autoconnect_console_how; struct save_file_header hdr; uint32_t domid_soft_reset = INVALID_DOMID; @@ -2703,6 +2705,7 @@ static uint32_t create_domain(struct domain_create *dom_info) if (migrate_fd >= 0) { restore_source = ""; restore_fd = migrate_fd; + send_back_fd = dom_info->send_back_fd; } else { restore_source = restore_file; restore_fd = open(restore_file, O_RDONLY); @@ -2891,7 +2894,7 @@ start: ret = libxl_domain_create_restore(ctx, &d_config, &domid, restore_fd, - ¶ms, + send_back_fd, ¶ms, 0, autoconnect_console_how); libxl_domain_restore_params_dispose(¶ms); @@ -4454,6 +4457,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.monitor = monitor; dom_info.paused = 1; dom_info.migrate_fd = recv_fd; + dom_info.send_back_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; @@ -4627,6 +4631,7 @@ int main_restore(int argc, char **argv) dom_info.config_file = config_file; dom_info.restore_file = checkpoint_file; dom_info.migrate_fd = -1; + dom_info.send_back_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; @@ -5094,6 +5099,7 @@ int main_create(int argc, char **argv) dom_info.quiet = quiet; dom_info.config_file = filename; dom_info.migrate_fd = -1; + dom_info.send_back_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 4133527..98b52b9 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -538,7 +538,7 @@ value stub_libxl_domain_create_restore(value ctx, value domain_config, value par caml_enter_blocking_section(); ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, - &c_params, ao_how, NULL); + -1, &c_params, ao_how, NULL); caml_leave_blocking_section(); free(ao_how);