From patchwork Fri Mar 25 06:44:33 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changlong Xie X-Patchwork-Id: 8668701 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 4E902C0553 for ; Fri, 25 Mar 2016 06:45:05 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DE60820382 for ; Fri, 25 Mar 2016 06:45:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D09F203A1 for ; Fri, 25 Mar 2016 06:45:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLSX-000758-7a; Fri, 25 Mar 2016 06:43:01 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLSV-0006cl-4L for xen-devel@lists.xen.org; Fri, 25 Mar 2016 06:42:59 +0000 Received: from [85.158.143.35] by server-2.bemta-6.messagelabs.com id 30/32-09532-2FDD4F65; Fri, 25 Mar 2016 06:42:58 +0000 X-Env-Sender: xiecl.fnst@cn.fujitsu.com X-Msg-Ref: server-7.tower-21.messagelabs.com!1458888168!5949034!6 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 18189 invoked from network); 25 Mar 2016 06:42:57 -0000 Received: from unknown (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-7.tower-21.messagelabs.com with SMTP; 25 Mar 2016 06:42:57 -0000 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="4958520" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 25 Mar 2016 14:42:43 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id 8E424408D264; Fri, 25 Mar 2016 14:42:37 +0800 (CST) Received: from changlox.g08.fujitsu.local (10.167.225.55) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Fri, 25 Mar 2016 14:42:37 +0800 From: Changlong Xie To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 25 Mar 2016 14:44:33 +0800 Message-ID: <1458888273-7469-27-git-send-email-xiecl.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> References: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.55] X-yoursite-MailScanner-ID: 8E424408D264.A3CA2 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: xiecl.fnst@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Li Zhijian , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Anthony Perard , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v13 26/26] cmdline switches and config vars to control colo-proxy X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wen Congyang Add cmdline switches to 'xl migrate-receive' command to specify a domain-specific hotplug script to setup COLO proxy. Add a new config var 'colo.default.agentscript' to xl.conf, that allows the user to override the default global script used to setup COLO proxy. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie --- docs/man/xl.conf.pod.5 | 6 ++++++ docs/man/xl.pod.1 | 7 ++++-- tools/libxl/libxl.c | 6 ++++++ tools/libxl/libxl_colo_restore.c | 5 +++++ tools/libxl/libxl_create.c | 9 ++++++-- tools/libxl/libxl_types.idl | 1 + tools/libxl/xl.c | 3 +++ tools/libxl/xl.h | 1 + tools/libxl/xl_cmdimpl.c | 46 +++++++++++++++++++++++++++++++--------- 9 files changed, 70 insertions(+), 14 deletions(-) diff --git a/docs/man/xl.conf.pod.5 b/docs/man/xl.conf.pod.5 index 8ae19bb..8f7fd28 100644 --- a/docs/man/xl.conf.pod.5 +++ b/docs/man/xl.conf.pod.5 @@ -111,6 +111,12 @@ Configures the default script used by Remus to setup network buffering. Default: C +=item B + +Configures the default script used by COLO to setup colo-proxy. + +Default: C + =item B Configures the default output format used by xl when printing "machine diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1 index 6788465..6dac73e 100644 --- a/docs/man/xl.pod.1 +++ b/docs/man/xl.pod.1 @@ -458,8 +458,6 @@ Remus support in xl is still in experimental (proof-of-concept) phase. Disk replication support is limited to DRBD disks. COLO support in xl is still in experimental (proof-of-concept) phase. -There is no support for network, so the guest will confuse its network -peers at the moment. =back @@ -483,6 +481,11 @@ and it's used by secondary. =item B :Primary's modified contents will be buffered in this disk, and it's used by secondary. +(b) An example for COLO network configuration: vif =[ '...,forwarddev=xxx,...'] + +=item B :Forward devices for primary and secondary, there are +directly connected. + =back B diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 63fbe16..aabf3a7 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -3378,6 +3378,11 @@ void libxl__device_nic_add(libxl__egc *egc, uint32_t domid, flexarray_append(back, nic->ifname); } + if (nic->coloft_forwarddev) { + flexarray_append(back, "forwarddev"); + flexarray_append(back, nic->coloft_forwarddev); + } + flexarray_append(back, "mac"); flexarray_append(back,GCSPRINTF(LIBXL_MAC_FMT, LIBXL_MAC_BYTES(nic->mac))); if (nic->ip) { @@ -3500,6 +3505,7 @@ static int libxl__device_nic_from_xs_be(libxl__gc *gc, nic->ip = READ_BACKEND(NOGC, "ip"); nic->bridge = READ_BACKEND(NOGC, "bridge"); nic->script = READ_BACKEND(NOGC, "script"); + nic->coloft_forwarddev = READ_BACKEND(NOGC, "forwarddev"); /* vif_ioemu nics use the same xenstore entries as vif interfaces */ tmp = READ_BACKEND(gc, "type"); diff --git a/tools/libxl/libxl_colo_restore.c b/tools/libxl/libxl_colo_restore.c index c8ad796..3483f39 100644 --- a/tools/libxl/libxl_colo_restore.c +++ b/tools/libxl/libxl_colo_restore.c @@ -233,6 +233,11 @@ void libxl__colo_restore_setup(libxl__egc *egc, crcs->crs = crs; crs->qdisk_setuped = false; crs->qdisk_used = false; + if (dcs->colo_proxy_script) + crs->colo_proxy_script = libxl__strdup(gc, dcs->colo_proxy_script); + else + crs->colo_proxy_script = GCSPRINTF("%s/colo-proxy-setup", + libxl__xen_script_dir_path()); /* setup dsps */ crcs->dsps.ao = ao; diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index e2ec25c..d6028aa 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1690,6 +1690,7 @@ static void domain_create_cb(libxl__egc *egc, static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, uint32_t *domid, int restore_fd, int send_back_fd, const libxl_domain_restore_params *params, + const char *colo_proxy_script, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { @@ -1713,6 +1714,7 @@ static int do_domain_create(libxl_ctx *ctx, libxl_domain_config *d_config, } cdcs->dcs.callback = domain_create_cb; cdcs->dcs.domid_soft_reset = INVALID_DOMID; + cdcs->dcs.colo_proxy_script = colo_proxy_script; libxl__ao_progress_gethow(&cdcs->dcs.aop_console_how, aop_console_how); cdcs->domid_out = domid; @@ -1900,7 +1902,7 @@ int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncprogress_how *aop_console_how) { unset_disk_colo_restore(d_config); - return do_domain_create(ctx, d_config, domid, -1, -1, NULL, + return do_domain_create(ctx, d_config, domid, -1, -1, NULL, NULL, ao_how, aop_console_how); } @@ -1911,14 +1913,17 @@ int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, const libxl_asyncop_how *ao_how, const libxl_asyncprogress_how *aop_console_how) { + char *colo_proxy_script = NULL; + if (params->checkpointed_stream == LIBXL_CHECKPOINTED_STREAM_COLO) { + colo_proxy_script = params->colo_proxy_script; set_disk_colo_restore(d_config); } else { unset_disk_colo_restore(d_config); } return do_domain_create(ctx, d_config, domid, restore_fd, send_back_fd, - params, ao_how, aop_console_how); + params, colo_proxy_script, ao_how, aop_console_how); } int libxl_domain_soft_reset(libxl_ctx *ctx, diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 6893732..03eb860 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -386,6 +386,7 @@ libxl_domain_create_info = Struct("domain_create_info",[ libxl_domain_restore_params = Struct("domain_restore_params", [ ("checkpointed_stream", integer), ("stream_version", uint32, {'init_val': '1'}), + ("colo_proxy_script", string), ]) libxl_domain_sched_params = Struct("domain_sched_params",[ diff --git a/tools/libxl/xl.c b/tools/libxl/xl.c index dfae84a..a272258 100644 --- a/tools/libxl/xl.c +++ b/tools/libxl/xl.c @@ -45,6 +45,7 @@ char *default_bridge = NULL; char *default_gatewaydev = NULL; char *default_vifbackend = NULL; char *default_remus_netbufscript = NULL; +char *default_colo_proxy_script = NULL; enum output_format default_output_format = OUTPUT_FORMAT_JSON; int claim_mode = 1; bool progress_use_cr = 0; @@ -179,6 +180,8 @@ static void parse_global_config(const char *configfile, xlu_cfg_replace_string (config, "remus.default.netbufscript", &default_remus_netbufscript, 0); + xlu_cfg_replace_string (config, "colo.default.proxyscript", + &default_colo_proxy_script, 0); xlu_cfg_destroy(config); } diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h index 309627a..e601ca1 100644 --- a/tools/libxl/xl.h +++ b/tools/libxl/xl.h @@ -194,6 +194,7 @@ extern char *default_bridge; extern char *default_gatewaydev; extern char *default_vifbackend; extern char *default_remus_netbufscript; +extern char *default_colo_proxy_script; extern char *blkdev_start; enum output_format { diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 25bd81a..ba77862 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -158,6 +158,7 @@ struct domain_create { const char *config_file; char *extra_config; /* extra config string */ const char *restore_file; + char *colo_proxy_script; int migrate_fd; /* -1 means none */ int send_back_fd; /* -1 means none */ char **migration_domname_r; /* from malloc */ @@ -1053,6 +1054,8 @@ static int parse_nic_config(libxl_device_nic *nic, XLU_Config **config, char *to replace_string(&nic->model, oparg); } else if (MATCH_OPTION("rate", token, oparg)) { parse_vif_rate(config, oparg, nic); + } else if (MATCH_OPTION("forwarddev", token, oparg)) { + replace_string(&nic->coloft_forwarddev, oparg); } else if (MATCH_OPTION("accel", token, oparg)) { fprintf(stderr, "the accel parameter for vifs is currently not supported\n"); } else { @@ -3001,6 +3004,7 @@ start: params.checkpointed_stream = dom_info->checkpointed_stream; params.stream_version = (hdr.mandatory_flags & XL_MANDATORY_FLAG_STREAMv2) ? 2 : 1; + params.colo_proxy_script = dom_info->colo_proxy_script; ret = libxl_domain_create_restore(ctx, &d_config, &domid, restore_fd, @@ -4733,7 +4737,8 @@ static void migrate_domain(uint32_t domid, const char *rune, int debug, static void migrate_receive(int debug, int daemonize, int monitor, int send_fd, int recv_fd, - libxl_checkpointed_stream checkpointed) + libxl_checkpointed_stream checkpointed, + char *colo_proxy_script) { uint32_t domid; int rc, rc2; @@ -4762,6 +4767,7 @@ static void migrate_receive(int debug, int daemonize, int monitor, dom_info.send_back_fd = send_fd; dom_info.migration_domname_r = &migration_domname; dom_info.checkpointed_stream = checkpointed; + dom_info.colo_proxy_script = colo_proxy_script; rc = create_domain(&dom_info); if (rc < 0) { @@ -4955,8 +4961,10 @@ int main_migrate_receive(int argc, char **argv) int debug = 0, daemonize = 1, monitor = 1; libxl_checkpointed_stream checkpointed = LIBXL_CHECKPOINTED_STREAM_NONE; int opt; + char *script = NULL; static struct option opts[] = { {"colo", 0, 0, 0x100}, + {"coloft-script", 1, 0, 0x200}, COMMON_LONG_OPTS }; @@ -4977,6 +4985,9 @@ int main_migrate_receive(int argc, char **argv) case 0x100: checkpointed = LIBXL_CHECKPOINTED_STREAM_COLO; break; + case 0x200: + script = optarg; + break; } if (argc-optind != 0) { @@ -4985,7 +4996,7 @@ int main_migrate_receive(int argc, char **argv) } migrate_receive(debug, daemonize, monitor, STDOUT_FILENO, STDIN_FILENO, - checkpointed); + checkpointed, script); return 0; } @@ -8395,8 +8406,10 @@ int main_remus(int argc, char **argv) r_info.interval = 200; if (libxl_defbool_val(r_info.colo)) { - if (r_info.interval || libxl_defbool_val(r_info.blackhole)) { - perror("Option -c conflicts with -i or -b"); + if (r_info.interval || libxl_defbool_val(r_info.blackhole) || + !libxl_defbool_is_default(r_info.netbuf) || + !libxl_defbool_is_default(r_info.diskbuf)) { + perror("option -c is conflict with -i, -d, -n or -b"); exit(-1); } @@ -8407,8 +8420,12 @@ int main_remus(int argc, char **argv) } } - if (!r_info.netbufscript) - r_info.netbufscript = default_remus_netbufscript; + if (!r_info.netbufscript) { + if (libxl_defbool_val(r_info.colo)) + r_info.netbufscript = default_colo_proxy_script; + else + r_info.netbufscript = default_remus_netbufscript; + } if (libxl_defbool_val(r_info.blackhole)) { send_fd = open("/dev/null", O_RDWR, 0644); @@ -8421,10 +8438,19 @@ int main_remus(int argc, char **argv) if (!ssh_command[0]) { rune = host; } else { - xasprintf(&rune, "exec %s %s xl migrate-receive %s %s", - ssh_command, host, - libxl_defbool_val(r_info.colo) ? "-c" : "-r", - daemonize ? "" : " -e"); + if (!libxl_defbool_val(r_info.colo)) { + xasprintf(&rune, "exec %s %s xl migrate-receive %s %s", + ssh_command, host, + "-r", + daemonize ? "" : " -e"); + } else { + xasprintf(&rune, "exec %s %s xl migrate-receive %s %s %s %s", + ssh_command, host, + "--colo", + r_info.netbufscript ? "--coloft-script" : "", + r_info.netbufscript ? r_info.netbufscript : "", + daemonize ? "" : " -e"); + } } save_domain_core_begin(domid, NULL, &config_data, &config_len);