From patchwork Fri Mar 25 06:44:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changlong Xie X-Patchwork-Id: 8668681 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C267E9F6E1 for ; Fri, 25 Mar 2016 06:44:57 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7BF4F20395 for ; Fri, 25 Mar 2016 06:44:56 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 20F562037E for ; Fri, 25 Mar 2016 06:44:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLST-0006og-4J; Fri, 25 Mar 2016 06:42:57 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLSR-0006d1-MW for xen-devel@lists.xen.org; Fri, 25 Mar 2016 06:42:55 +0000 Received: from [85.158.143.35] by server-1.bemta-6.messagelabs.com id 51/F1-29237-FEDD4F65; Fri, 25 Mar 2016 06:42:55 +0000 X-Env-Sender: xiecl.fnst@cn.fujitsu.com X-Msg-Ref: server-7.tower-21.messagelabs.com!1458888168!5949034!3 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18075 invoked from network); 25 Mar 2016 06:42:54 -0000 Received: from unknown (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-7.tower-21.messagelabs.com with SMTP; 25 Mar 2016 06:42:53 -0000 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="4958506" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 25 Mar 2016 14:42:31 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id 04A9E408D26A; Fri, 25 Mar 2016 14:42:27 +0800 (CST) Received: from changlox.g08.fujitsu.local (10.167.225.55) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Fri, 25 Mar 2016 14:42:26 +0800 From: Changlong Xie To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 25 Mar 2016 14:44:24 +0800 Message-ID: <1458888273-7469-18-git-send-email-xiecl.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> References: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.55] X-yoursite-MailScanner-ID: 04A9E408D26A.A3C4F X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: xiecl.fnst@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Li Zhijian , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Anthony Perard , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v13 17/26] implement the cmdline for COLO X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wen Congyang Add a new option -c to the command 'xl remus'. If you want to use COLO HA instead of Remus HA, please use -c option. Update man pages to reflect the addition of a new option to 'xl remus' command. Also add a new option --colo to the internal command 'xl migrate-receive'. Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- docs/man/xl.pod.1 | 13 ++++++++-- tools/libxl/libxl.c | 22 ++++++++++++++-- tools/libxl/libxl_create.c | 1 - tools/libxl/xl_cmdimpl.c | 65 +++++++++++++++++++++++++++++++++++----------- tools/libxl/xl_cmdtable.c | 4 ++- 5 files changed, 84 insertions(+), 21 deletions(-) diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1 index dc6213e..a992a45 100644 --- a/docs/man/xl.pod.1 +++ b/docs/man/xl.pod.1 @@ -447,12 +447,16 @@ Print huge (!) amount of debug during the migration process. =item B [I] I I -Enable Remus HA for domain. By default B relies on ssh as a transport -mechanism between the two hosts. +Enable Remus HA or COLO HA for domain. By default B relies on ssh as a +transport mechanism between the two hosts. N.B: Remus support in xl is still in experimental (proof-of-concept) phase. Disk replication support is limited to DRBD disks. + COLO support in xl is still in experimental (proof-of-concept) phase. + There is no support for network or disk, so the guest will corrupt its + disk and confuse its network peers at the moment. + B =over 4 @@ -498,6 +502,11 @@ Disable network output buffering. Requires enabling unsafe mode. Disable disk replication. Requires enabling unsafe mode. +=item B<-c> + +Enable COLO HA. This conflicts with B<-i> and B<-b>, and memory +checkpoint compression must be disabled. + =back =item B I diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 272c6a5..349a3c6 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -848,12 +848,27 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, goto out; } + /* The caller must set this defbool */ + if (libxl_defbool_is_default(info->colo)) { + LOG(ERROR, "colo mode must be enabled/disabled"); + rc = ERROR_FAIL; + goto out; + } + libxl_defbool_setdefault(&info->allow_unsafe, false); libxl_defbool_setdefault(&info->blackhole, false); - libxl_defbool_setdefault(&info->compression, true); + libxl_defbool_setdefault(&info->compression, + !libxl_defbool_val(info->colo)); libxl_defbool_setdefault(&info->netbuf, true); libxl_defbool_setdefault(&info->diskbuf, true); + if (libxl_defbool_val(info->colo) && + libxl_defbool_val(info->compression)) { + LOG(ERROR, "cannot use memory checkpoint compression in COLO mode"); + rc = ERROR_FAIL; + goto out; + } + if (!libxl_defbool_val(info->allow_unsafe) && (libxl_defbool_val(info->blackhole) || !libxl_defbool_val(info->netbuf) || @@ -875,7 +890,10 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->live = 1; dss->debug = 0; dss->remus = info; - dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; + if (libxl_defbool_val(info->colo)) + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_COLO; + else + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; assert(info); diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index d6c794e..be604e5 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1893,7 +1893,6 @@ int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { - assert(send_back_fd == -1); return do_domain_create(ctx, d_config, domid, restore_fd, send_back_fd, params, ao_how, aop_console_how); } diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 2e64f44..25bd81a 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -4740,6 +4740,8 @@ static void migrate_receive(int debug, int daemonize, int monitor, char rc_buf; char *migration_domname; struct domain_create dom_info; + const char *ha = checkpointed == LIBXL_CHECKPOINTED_STREAM_COLO ? + "COLO" : "Remus"; signal(SIGPIPE, SIG_IGN); /* if we get SIGPIPE we'd rather just have it as an error */ @@ -4757,7 +4759,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.monitor = monitor; dom_info.paused = 1; dom_info.migrate_fd = recv_fd; - dom_info.send_back_fd = -1; + dom_info.send_back_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; @@ -4772,11 +4774,12 @@ static void migrate_receive(int debug, int daemonize, int monitor, switch (checkpointed) { case LIBXL_CHECKPOINTED_STREAM_REMUS: + case LIBXL_CHECKPOINTED_STREAM_COLO: /* If we are here, it means that the sender (primary) has crashed. * TODO: Split-Brain Check. */ - fprintf(stderr, "migration target: Remus Failover for domain %u\n", - domid); + fprintf(stderr, "migration target: %s Failover for domain %u\n", + ha, domid); /* * If domain renaming fails, lets just continue (as we need the domain @@ -4792,16 +4795,20 @@ static void migrate_receive(int debug, int daemonize, int monitor, rc = libxl_domain_rename(ctx, domid, migration_domname, common_domname); if (rc) - fprintf(stderr, "migration target (Remus): " + fprintf(stderr, "migration target (%s): " "Failed to rename domain from %s to %s:%d\n", - migration_domname, common_domname, rc); + ha, migration_domname, common_domname, rc); } + if (checkpointed == LIBXL_CHECKPOINTED_STREAM_COLO) + /* The guest is running after failover in COLO mode */ + exit(rc ? -ERROR_FAIL: 0); + rc = libxl_domain_unpause(ctx, domid); if (rc) - fprintf(stderr, "migration target (Remus): " + fprintf(stderr, "migration target (%s): " "Failed to unpause domain %s (id: %u):%d\n", - common_domname, domid, rc); + ha, common_domname, domid, rc); exit(rc ? -ERROR_FAIL: 0); default: @@ -4948,8 +4955,12 @@ int main_migrate_receive(int argc, char **argv) int debug = 0, daemonize = 1, monitor = 1; libxl_checkpointed_stream checkpointed = LIBXL_CHECKPOINTED_STREAM_NONE; int opt; + static struct option opts[] = { + {"colo", 0, 0, 0x100}, + COMMON_LONG_OPTS + }; - SWITCH_FOREACH_OPT(opt, "Fedr", NULL, "migrate-receive", 0) { + SWITCH_FOREACH_OPT(opt, "Fedr", opts, "migrate-receive", 0) { case 'F': daemonize = 0; break; @@ -4963,6 +4974,9 @@ int main_migrate_receive(int argc, char **argv) case 'r': checkpointed = LIBXL_CHECKPOINTED_STREAM_REMUS; break; + case 0x100: + checkpointed = LIBXL_CHECKPOINTED_STREAM_COLO; + break; } if (argc-optind != 0) { @@ -8338,11 +8352,8 @@ int main_remus(int argc, char **argv) int config_len; memset(&r_info, 0, sizeof(libxl_domain_remus_info)); - /* Defaults */ - r_info.interval = 200; - libxl_defbool_setdefault(&r_info.blackhole, false); - SWITCH_FOREACH_OPT(opt, "Fbundi:s:N:e", NULL, "remus", 2) { + SWITCH_FOREACH_OPT(opt, "Fbundi:s:N:ec", NULL, "remus", 2) { case 'i': r_info.interval = atoi(optarg); break; @@ -8370,11 +8381,32 @@ int main_remus(int argc, char **argv) case 'e': daemonize = 0; break; + case 'c': + libxl_defbool_set(&r_info.colo, true); } domid = find_domain(argv[optind]); host = argv[optind + 1]; + /* Defaults */ + libxl_defbool_setdefault(&r_info.blackhole, false); + libxl_defbool_setdefault(&r_info.colo, false); + if (!libxl_defbool_val(r_info.colo) && !r_info.interval) + r_info.interval = 200; + + if (libxl_defbool_val(r_info.colo)) { + if (r_info.interval || libxl_defbool_val(r_info.blackhole)) { + perror("Option -c conflicts with -i or -b"); + exit(-1); + } + + if (libxl_defbool_is_default(r_info.compression)) { + perror("COLO can't be used with memory compression. " + "Disable memory checkpoint compression now..."); + libxl_defbool_set(&r_info.compression, false); + } + } + if (!r_info.netbufscript) r_info.netbufscript = default_remus_netbufscript; @@ -8389,8 +8421,9 @@ int main_remus(int argc, char **argv) if (!ssh_command[0]) { rune = host; } else { - xasprintf(&rune, "exec %s %s xl migrate-receive -r %s", + xasprintf(&rune, "exec %s %s xl migrate-receive %s %s", ssh_command, host, + libxl_defbool_val(r_info.colo) ? "-c" : "-r", daemonize ? "" : " -e"); } @@ -8418,7 +8451,8 @@ int main_remus(int argc, char **argv) * domain to force failover */ if (libxl_domain_info(ctx, 0, domid)) { - fprintf(stderr, "Remus: Primary domain has been destroyed.\n"); + fprintf(stderr, "%s: Primary domain has been destroyed.\n", + libxl_defbool_val(r_info.colo) ? "COLO" : "Remus"); close(send_fd); return 0; } @@ -8430,7 +8464,8 @@ int main_remus(int argc, char **argv) if (rc == ERROR_GUEST_TIMEDOUT) fprintf(stderr, "Failed to suspend domain at primary.\n"); else { - fprintf(stderr, "Remus: Backup failed? resuming domain at primary.\n"); + fprintf(stderr, "%s: Backup failed? resuming domain at primary.\n", + libxl_defbool_val(r_info.colo) ? "COLO" : "Remus"); libxl_domain_resume(ctx, domid, 1, 0); } diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c index b14b881..5911ea8 100644 --- a/tools/libxl/xl_cmdtable.c +++ b/tools/libxl/xl_cmdtable.c @@ -499,7 +499,9 @@ struct cmd_spec cmd_table[] = { "-b Replicate memory checkpoints to /dev/null (blackhole).\n" " Works only in unsafe mode.\n" "-n Disable network output buffering. Works only in unsafe mode.\n" - "-d Disable disk replication. Works only in unsafe mode." + "-d Disable disk replication. Works only in unsafe mode.\n" + "-c Enable COLO HA. It is conflict with -i and -b, and memory\n" + " checkpoint must be disabled" }, #endif { "devd",