From patchwork Thu Feb 18 02:43:17 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 8345161 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 811BC9F372 for ; Thu, 18 Feb 2016 02:46:09 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 39AB2203DC for ; Thu, 18 Feb 2016 02:46:08 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7626203AB for ; Thu, 18 Feb 2016 02:46:06 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aWEYl-0007yb-Ef; Thu, 18 Feb 2016 02:43:15 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aWEYi-0007vT-25 for xen-devel@lists.xen.org; Thu, 18 Feb 2016 02:43:12 +0000 Received: from [85.158.137.68] by server-16.bemta-3.messagelabs.com id AF/D8-07451-FBF25C65; Thu, 18 Feb 2016 02:43:11 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-4.tower-31.messagelabs.com!1455763380!13590680!5 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15492 invoked from network); 18 Feb 2016 02:43:09 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-4.tower-31.messagelabs.com with SMTP; 18 Feb 2016 02:43:09 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="3658311" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 18 Feb 2016 10:42:50 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 0D00F42AC841; Thu, 18 Feb 2016 10:41:07 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Thu, 18 Feb 2016 10:42:45 +0800 From: Wen Congyang To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Thu, 18 Feb 2016 10:43:17 +0800 Message-ID: <1455763403-18641-8-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1455763403-18641-1-git-send-email-wency@cn.fujitsu.com> References: <1455763403-18641-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 0D00F42AC841.AE3A9 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Ian Campbell , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Ian Jackson , Yang Hongyang Subject: [Xen-devel] [PATCH v8 07/13] migration/save: pass checkpointed_stream from libxl to libxc X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Pass checkpointed_stream from libxl to libxc. It won't affact legacy migration because legacy migration won't use this param. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Campbell CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxc/include/xenguest.h | 6 ++++-- tools/libxc/xc_nomigrate.c | 3 ++- tools/libxc/xc_sr_common.h | 12 +++++++++++- tools/libxc/xc_sr_save.c | 17 +++++++++++------ tools/libxl/libxl.c | 2 ++ tools/libxl/libxl_dom_save.c | 11 ++++++++--- tools/libxl/libxl_internal.h | 1 + tools/libxl/libxl_save_callout.c | 2 +- tools/libxl/libxl_save_helper.c | 3 ++- tools/libxl/libxl_stream_write.c | 2 +- tools/libxl/libxl_types.idl | 1 + 11 files changed, 44 insertions(+), 16 deletions(-) diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h index d48b3ff..affc42b 100644 --- a/tools/libxc/include/xenguest.h +++ b/tools/libxc/include/xenguest.h @@ -29,7 +29,6 @@ #define XCFLAGS_HVM (1 << 2) #define XCFLAGS_STDVGA (1 << 3) #define XCFLAGS_CHECKPOINT_COMPRESS (1 << 4) -#define XCFLAGS_CHECKPOINTED (1 << 5) #define X86_64_B_SIZE 64 #define X86_32_B_SIZE 32 @@ -82,11 +81,14 @@ struct save_callbacks { * @parm xch a handle to an open hypervisor interface * @parm fd the file descriptor to save a domain to * @parm dom the id of the domain + * @param checkpointed_stream MIG_STREAM_NONE if the far end of the stream + * doesn't use checkpointing * @return 0 on success, -1 on failure */ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags /* XCFLAGS_xxx */, - struct save_callbacks* callbacks, int hvm); + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream); /* callbacks provided by xc_domain_restore */ struct restore_callbacks { diff --git a/tools/libxc/xc_nomigrate.c b/tools/libxc/xc_nomigrate.c index 902429e..c9124df 100644 --- a/tools/libxc/xc_nomigrate.c +++ b/tools/libxc/xc_nomigrate.c @@ -22,7 +22,8 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags, - struct save_callbacks* callbacks, int hvm) + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream) { errno = ENOSYS; return -1; diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h index 60b43e8..66f595f 100644 --- a/tools/libxc/xc_sr_common.h +++ b/tools/libxc/xc_sr_common.h @@ -180,6 +180,16 @@ struct xc_sr_context xc_dominfo_t dominfo; + /* + * migration stream + * 0: Plain VM + * 1: Remus + */ + enum { + MIG_STREAM_NONE, /* plain stream */ + MIG_STREAM_REMUS, + } migration_stream; + union /* Common save or restore data. */ { struct /* Save data. */ @@ -191,7 +201,7 @@ struct xc_sr_context bool live; /* Plain VM, or checkpoints over time. */ - bool checkpointed; + int checkpointed; /* Further debugging information in the stream. */ bool debug; diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c index ccb000e..e258b7c 100644 --- a/tools/libxc/xc_sr_save.c +++ b/tools/libxc/xc_sr_save.c @@ -629,7 +629,7 @@ static int send_domain_memory_live(struct xc_sr_context *ctx) if ( rc ) goto out; - if ( ctx->save.debug && !ctx->save.checkpointed ) + if ( ctx->save.debug && ctx->save.checkpointed != MIG_STREAM_NONE ) { rc = verify_frames(ctx); if ( rc ) @@ -758,7 +758,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( ctx->save.live ) rc = send_domain_memory_live(ctx); - else if ( ctx->save.checkpointed ) + else if ( ctx->save.checkpointed != MIG_STREAM_NONE ) rc = send_domain_memory_checkpointed(ctx); else rc = send_domain_memory_nonlive(ctx); @@ -778,7 +778,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc ) goto err; - if ( ctx->save.checkpointed ) + if ( ctx->save.checkpointed != MIG_STREAM_NONE ) { /* * We have now completed the initial live portion of the checkpoint @@ -799,7 +799,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc <= 0 ) goto err; } - } while ( ctx->save.checkpointed ); + } while ( ctx->save.checkpointed != MIG_STREAM_NONE ); xc_report_progress_single(xch, "End of stream"); @@ -829,7 +829,8 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags, - struct save_callbacks* callbacks, int hvm) + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream) { struct xc_sr_context ctx = { @@ -841,7 +842,11 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, ctx.save.callbacks = callbacks; ctx.save.live = !!(flags & XCFLAGS_LIVE); ctx.save.debug = !!(flags & XCFLAGS_DEBUG); - ctx.save.checkpointed = !!(flags & XCFLAGS_CHECKPOINTED); + ctx.save.checkpointed = checkpointed_stream; + + /* If altering migration_stream update this assert too. */ + assert(checkpointed_stream == MIG_STREAM_NONE || + checkpointed_stream == MIG_STREAM_REMUS); /* * TODO: Find some time to better tweak the live migration algorithm. diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index db5732c..58b4574 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -876,6 +876,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->live = 1; dss->debug = 0; dss->remus = info; + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; assert(info); @@ -936,6 +937,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, dss->type = type; dss->live = flags & LIBXL_SUSPEND_LIVE; dss->debug = flags & LIBXL_SUSPEND_DEBUG; + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; rc = libxl__fd_flags_modify_save(gc, dss->fd, ~(O_NONBLOCK|O_NDELAY), 0, diff --git a/tools/libxl/libxl_dom_save.c b/tools/libxl/libxl_dom_save.c index aead042..a385500 100644 --- a/tools/libxl/libxl_dom_save.c +++ b/tools/libxl/libxl_dom_save.c @@ -338,6 +338,12 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) unsigned int nr_vnodes = 0, nr_vmemranges = 0, nr_vcpus = 0; libxl__domain_suspend_state *dsps = &dss->dsps; + if (dss->checkpointed_stream != LIBXL_CHECKPOINTED_STREAM_NONE && !r_info) { + LOG(ERROR, "Migration stream is checkpointed, but there's no " + "checkpoint info!"); + goto out; + } + dss->rc = 0; logdirty_init(&dss->logdirty); dsps->ao = ao; @@ -376,14 +382,13 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) goto out; } - if (r_info != NULL) { + if (dss->checkpointed_stream == LIBXL_CHECKPOINTED_STREAM_REMUS) { dss->interval = r_info->interval; - dss->xcflags |= XCFLAGS_CHECKPOINTED; if (libxl_defbool_val(r_info->compression)) dss->xcflags |= XCFLAGS_CHECKPOINT_COMPRESS; } - if (r_info == NULL) + if (dss->checkpointed_stream == LIBXL_CHECKPOINTED_STREAM_NONE) callbacks->suspend = libxl__domain_suspend_callback; callbacks->switch_qemu_logdirty = libxl__domain_suspend_common_switch_qemu_logdirty; diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 82c3610..ac6457f 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3113,6 +3113,7 @@ struct libxl__domain_save_state { libxl_domain_type type; int live; int debug; + int checkpointed_stream; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/libxl/libxl_save_callout.c b/tools/libxl/libxl_save_callout.c index 94b6b67..7f1f5d4 100644 --- a/tools/libxl/libxl_save_callout.c +++ b/tools/libxl/libxl_save_callout.c @@ -85,7 +85,7 @@ void libxl__xc_domain_save(libxl__egc *egc, libxl__domain_save_state *dss, const unsigned long argnums[] = { dss->domid, 0, 0, dss->xcflags, dss->hvm, - cbflags, + cbflags, dss->checkpointed_stream, }; shs->ao = ao; diff --git a/tools/libxl/libxl_save_helper.c b/tools/libxl/libxl_save_helper.c index 39038f9..6bdcf13 100644 --- a/tools/libxl/libxl_save_helper.c +++ b/tools/libxl/libxl_save_helper.c @@ -253,6 +253,7 @@ int main(int argc, char **argv) uint32_t flags = strtoul(NEXTARG,0,10); int hvm = atoi(NEXTARG); unsigned cbflags = strtoul(NEXTARG,0,10); + int checkpointed_stream = strtoul(NEXTARG,0,10); assert(!*++argv); helper_setcallbacks_save(&helper_save_callbacks, cbflags); @@ -261,7 +262,7 @@ int main(int argc, char **argv) setup_signals(save_signal_handler); r = xc_domain_save(xch, io_fd, dom, max_iters, max_factor, flags, - &helper_save_callbacks, hvm); + &helper_save_callbacks, hvm, checkpointed_stream); complete(r); } else if (!strcmp(mode,"--restore-domain")) { diff --git a/tools/libxl/libxl_stream_write.c b/tools/libxl/libxl_stream_write.c index 9053146..f6ea55d 100644 --- a/tools/libxl/libxl_stream_write.c +++ b/tools/libxl/libxl_stream_write.c @@ -355,7 +355,7 @@ void libxl__xc_domain_save_done(libxl__egc *egc, void *dss_void, * If the stream is not still alive, we must not continue any work. */ if (libxl__stream_write_inuse(stream)) { - if (dss->remus) + if (dss->checkpointed_stream != LIBXL_CHECKPOINTED_STREAM_NONE) /* * For remus, if libxl__xc_domain_save_done() completes, * there was an error sending data to the secondary. diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index b8fb22f..605fb9a 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -228,6 +228,7 @@ libxl_hdtype = Enumeration("hdtype", [ (2, "AHCI"), ], init_val = "LIBXL_HDTYPE_IDE") +# Consistent with the values defined for migration_stream. libxl_checkpointed_stream = Enumeration("checkpointed_stream", [ (0, "NONE"), (1, "REMUS"),