From patchwork Fri Jan 29 05:27:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 8159291 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id E0A68BEEE5 for ; Fri, 29 Jan 2016 05:30:18 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id ADCFC200E9 for ; Fri, 29 Jan 2016 05:30:17 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E589A20123 for ; Fri, 29 Jan 2016 05:30:15 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1az-0007rA-5z; Fri, 29 Jan 2016 05:27:45 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1au-0007d8-7u for xen-devel@lists.xen.org; Fri, 29 Jan 2016 05:27:40 +0000 Received: from [85.158.143.35] by server-1.bemta-4.messagelabs.com id 8A/4C-09708-C48FAA65; Fri, 29 Jan 2016 05:27:40 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-13.tower-21.messagelabs.com!1454045252!11153695!3 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 62213 invoked from network); 29 Jan 2016 05:27:38 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-13.tower-21.messagelabs.com with SMTP; 29 Jan 2016 05:27:38 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="3106353" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 29 Jan 2016 13:27:32 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 35910418AD6B; Fri, 29 Jan 2016 13:26:33 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Fri, 29 Jan 2016 13:27:08 +0800 From: Wen Congyang To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 29 Jan 2016 13:27:27 +0800 Message-ID: <1454045254-3711-12-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> References: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 35910418AD6B.A8ECB X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v7 11/18] tools/libxl: Add back channel to allow migration target send data back X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In COLO mode, secondary needs to send the following data to primary: 1. In libxl Secondary sends the following CHECKPOINT_CONTEXT to primary: CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED 2. In libxc Secondary sends the dirty pfn list to primary But the io_fd only can be written in primary, and only can be read in secondary. Save recv_fd in domain_suspend_state, and send_fd in domain_create_state. Extend libxl_domain_create_restore API, add a send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate the API change. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Acked-by: Wei Liu --- tools/libxl/libxl.c | 2 +- tools/libxl/libxl.h | 30 ++++++++++++++++++++++++++++-- tools/libxl/libxl_create.c | 9 +++++---- tools/libxl/libxl_internal.h | 2 ++ tools/libxl/xl_cmdimpl.c | 8 +++++++- tools/ocaml/libs/xl/xenlight_stubs.c | 2 +- 6 files changed, 44 insertions(+), 9 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index fc7844d..e286329 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -871,7 +871,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->callback = remus_failover_cb; dss->domid = domid; dss->fd = send_fd; - /* TODO do something with recv_fd */ + dss->recv_fd = recv_fd; dss->type = type; dss->live = 1; dss->debug = 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 6225db1..5e4aede 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -639,6 +639,15 @@ typedef struct libxl__ctx libxl_ctx; #define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1 /* + * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_FD 1 + * + * If this is defined, libxl_domain_create_restore()'s API has changed to + * include a send_fd param which used for libxl migration back channel + * during COLO. + */ +#define LIBXL_HAVE_DOMAIN_CREATE_RESTORE_SEND_FD 1 + +/* * LIBXL_HAVE_CREATEINFO_PVH * If this is defined, then libxl supports creation of a PVH guest. */ @@ -1152,7 +1161,7 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncprogress_how *aop_console_how) LIBXL_EXTERNAL_CALLERS_ONLY; int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1173,7 +1182,7 @@ int static inline libxl_domain_create_restore_0x040200( libxl_domain_restore_params_init(¶ms); ret = libxl_domain_create_restore( - ctx, d_config, domid, restore_fd, ¶ms, ao_how, aop_console_how); + ctx, d_config, domid, restore_fd, -1, ¶ms, ao_how, aop_console_how); libxl_domain_restore_params_dispose(¶ms); return ret; @@ -1181,6 +1190,23 @@ int static inline libxl_domain_create_restore_0x040200( #define libxl_domain_create_restore libxl_domain_create_restore_0x040200 +#elif defined(LIBXL_API_VERSION) && LIBXL_API_VERSION >= 0x040400 \ + && LIBXL_API_VERSION < 0x040700 + +int static inline libxl_domain_create_restore_0x040400( + libxl_ctx *ctx, libxl_domain_config *d_config, + uint32_t *domid, int restore_fd, + const libxl_domain_restore_params *params, + const libxl_asyncop_how *ao_how, + const libxl_asyncprogress_how *aop_console_how) + LIBXL_EXTERNAL_CALLERS_ONLY +{ + return libxl_domain_create_restore(ctx, d_config, domid, restore_fd, + -1, params, ao_how, aop_console_how); +} + +#define libxl_domain_create_restore libxl_domain_create_restore_0x040400 + #endif int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index 0d20c2d..eb869ea 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1563,7 +1563,7 @@ static void domain_create_cb(libxl__egc *egc, int rc, uint32_t domid); static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) @@ -1578,6 +1578,7 @@ static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_domain_config_init(&cdcs->dcs.guest_config_saved); libxl_domain_config_copy(ctx, &cdcs->dcs.guest_config_saved, d_config); cdcs->dcs.restore_fd = cdcs->dcs.libxc_fd = restore_fd; + cdcs->dcs.send_fd = send_fd; if (restore_fd > -1) { cdcs->dcs.restore_params = *params; rc = libxl__fd_flags_modify_save(gc, cdcs->dcs.restore_fd, @@ -1756,17 +1757,17 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, -1, NULL, + return do_domain_create(ctx, d_config, domid, -1, -1, NULL, ao_how, aop_console_how); } int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, - uint32_t *domid, int restore_fd, + uint32_t *domid, int restore_fd, int send_fd, const libxl_domain_restore_params *params, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - return do_domain_create(ctx, d_config, domid, restore_fd, params, + return do_domain_create(ctx, d_config, domid, restore_fd, send_fd, params, ao_how, aop_console_how); } diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index adc426a..8e9e57d 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3119,6 +3119,7 @@ struct libxl__domain_save_state { uint32_t domid; int fd; int fdfl; /* original flags on fd */ + int recv_fd; libxl_domain_type type; int live; int debug; @@ -3453,6 +3454,7 @@ struct libxl__domain_create_state { libxl_domain_config guest_config_saved; /* vanilla config */ int restore_fd, libxc_fd; int restore_fdfl; /* original flags of restore_fd */ + int send_fd; libxl_domain_restore_params restore_params; uint32_t domid_soft_reset; libxl__domain_create_cb *callback; diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 04cbcf3..59182b7 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -159,6 +159,7 @@ struct domain_create { char *extra_config; /* extra config string */ const char *restore_file; int migrate_fd; /* -1 means none */ + int send_fd; /* -1 means none */ char **migration_domname_r; /* from malloc */ }; @@ -2686,6 +2687,7 @@ static uint32_t create_domain(struct domain_create *dom_info) int config_len = 0; int restore_fd = -1; int restore_fd_to_close = -1; + int send_fd = -1; const libxl_asyncprogress_how *autoconnect_console_how; struct save_file_header hdr; uint32_t domid_soft_reset = INVALID_DOMID; @@ -2703,6 +2705,7 @@ static uint32_t create_domain(struct domain_create *dom_info) if (migrate_fd >= 0) { restore_source = ""; restore_fd = migrate_fd; + send_fd = dom_info->send_fd; } else { restore_source = restore_file; restore_fd = open(restore_file, O_RDONLY); @@ -2893,7 +2896,7 @@ start: ret = libxl_domain_create_restore(ctx, &d_config, &domid, restore_fd, - ¶ms, + send_fd, ¶ms, 0, autoconnect_console_how); libxl_domain_restore_params_dispose(¶ms); @@ -4449,6 +4452,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.monitor = monitor; dom_info.paused = 1; dom_info.migrate_fd = recv_fd; + dom_info.send_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; @@ -4622,6 +4626,7 @@ int main_restore(int argc, char **argv) dom_info.config_file = config_file; dom_info.restore_file = checkpoint_file; dom_info.migrate_fd = -1; + dom_info.send_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; @@ -5089,6 +5094,7 @@ int main_create(int argc, char **argv) dom_info.quiet = quiet; dom_info.config_file = filename; dom_info.migrate_fd = -1; + dom_info.send_fd = -1; dom_info.vnc = vnc; dom_info.vncautopass = vncautopass; dom_info.console_autoconnect = console_autoconnect; diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c b/tools/ocaml/libs/xl/xenlight_stubs.c index 4133527..1c52c2a 100644 --- a/tools/ocaml/libs/xl/xenlight_stubs.c +++ b/tools/ocaml/libs/xl/xenlight_stubs.c @@ -537,7 +537,7 @@ value stub_libxl_domain_create_restore(value ctx, value domain_config, value par restore_fd = Int_val(Field(params, 0)); caml_enter_blocking_section(); - ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, + ret = libxl_domain_create_restore(CTX, &c_dconfig, &c_domid, restore_fd, -1, &c_params, ao_how, NULL); caml_leave_blocking_section();