From patchwork Fri Jan 29 05:27:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Congyang X-Patchwork-Id: 8159241 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 224439F440 for ; Fri, 29 Jan 2016 05:30:07 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id CE69920138 for ; Fri, 29 Jan 2016 05:30:05 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB91720123 for ; Fri, 29 Jan 2016 05:30:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1ax-0007og-NK; Fri, 29 Jan 2016 05:27:43 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP1at-0007cr-SQ for xen-devel@lists.xen.org; Fri, 29 Jan 2016 05:27:39 +0000 Received: from [85.158.143.35] by server-3.bemta-4.messagelabs.com id 86/B1-31122-B48FAA65; Fri, 29 Jan 2016 05:27:39 +0000 X-Env-Sender: wency@cn.fujitsu.com X-Msg-Ref: server-15.tower-21.messagelabs.com!1454045241!12756590!5 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 43542 invoked from network); 29 Jan 2016 05:27:38 -0000 Received: from cn.fujitsu.com (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-15.tower-21.messagelabs.com with SMTP; 29 Jan 2016 05:27:38 -0000 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="3106351" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 29 Jan 2016 13:27:32 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (unknown [10.167.33.80]) by cn.fujitsu.com (Postfix) with ESMTP id 1AEDE418AD64; Fri, 29 Jan 2016 13:26:29 +0800 (CST) Received: from G08FNSTD140052.g08.fujitsu.local (10.167.226.52) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Fri, 29 Jan 2016 13:27:04 +0800 From: Wen Congyang To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 29 Jan 2016 13:27:23 +0800 Message-ID: <1454045254-3711-8-git-send-email-wency@cn.fujitsu.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> References: <1454045254-3711-1-git-send-email-wency@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.52] X-yoursite-MailScanner-ID: 1AEDE418AD64.AA78D X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wency@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Ian Campbell , Wen Congyang , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Shriram Rajagopalan , Ian Jackson , Yang Hongyang Subject: [Xen-devel] [PATCH v7 07/18] migration/save: pass checkpointed_stream from libxl to libxc X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Pass checkpointed_stream from libxl to libxc. It won't affact legacy migration because legacy migration won't use this param. Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang CC: Ian Campbell CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper Reviewed-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu --- tools/libxc/include/xenguest.h | 6 ++++-- tools/libxc/xc_nomigrate.c | 3 ++- tools/libxc/xc_sr_common.h | 12 +++++++++++- tools/libxc/xc_sr_save.c | 18 ++++++++++++------ tools/libxl/libxl.c | 2 ++ tools/libxl/libxl_dom_save.c | 11 ++++++++--- tools/libxl/libxl_internal.h | 1 + tools/libxl/libxl_save_callout.c | 2 +- tools/libxl/libxl_save_helper.c | 3 ++- tools/libxl/libxl_stream_write.c | 2 +- tools/libxl/libxl_types.idl | 1 + 11 files changed, 45 insertions(+), 16 deletions(-) diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h index d48b3ff..affc42b 100644 --- a/tools/libxc/include/xenguest.h +++ b/tools/libxc/include/xenguest.h @@ -29,7 +29,6 @@ #define XCFLAGS_HVM (1 << 2) #define XCFLAGS_STDVGA (1 << 3) #define XCFLAGS_CHECKPOINT_COMPRESS (1 << 4) -#define XCFLAGS_CHECKPOINTED (1 << 5) #define X86_64_B_SIZE 64 #define X86_32_B_SIZE 32 @@ -82,11 +81,14 @@ struct save_callbacks { * @parm xch a handle to an open hypervisor interface * @parm fd the file descriptor to save a domain to * @parm dom the id of the domain + * @param checkpointed_stream MIG_STREAM_NONE if the far end of the stream + * doesn't use checkpointing * @return 0 on success, -1 on failure */ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags /* XCFLAGS_xxx */, - struct save_callbacks* callbacks, int hvm); + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream); /* callbacks provided by xc_domain_restore */ struct restore_callbacks { diff --git a/tools/libxc/xc_nomigrate.c b/tools/libxc/xc_nomigrate.c index 902429e..c9124df 100644 --- a/tools/libxc/xc_nomigrate.c +++ b/tools/libxc/xc_nomigrate.c @@ -22,7 +22,8 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags, - struct save_callbacks* callbacks, int hvm) + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream) { errno = ENOSYS; return -1; diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h index 60b43e8..66f595f 100644 --- a/tools/libxc/xc_sr_common.h +++ b/tools/libxc/xc_sr_common.h @@ -180,6 +180,16 @@ struct xc_sr_context xc_dominfo_t dominfo; + /* + * migration stream + * 0: Plain VM + * 1: Remus + */ + enum { + MIG_STREAM_NONE, /* plain stream */ + MIG_STREAM_REMUS, + } migration_stream; + union /* Common save or restore data. */ { struct /* Save data. */ @@ -191,7 +201,7 @@ struct xc_sr_context bool live; /* Plain VM, or checkpoints over time. */ - bool checkpointed; + int checkpointed; /* Further debugging information in the stream. */ bool debug; diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c index ccb000e..0bea97e 100644 --- a/tools/libxc/xc_sr_save.c +++ b/tools/libxc/xc_sr_save.c @@ -629,7 +629,8 @@ static int send_domain_memory_live(struct xc_sr_context *ctx) if ( rc ) goto out; - if ( ctx->save.debug && !ctx->save.checkpointed ) + if ( ctx->save.debug && + ctx->save.checkpointed != MIG_STREAM_NONE ) { rc = verify_frames(ctx); if ( rc ) @@ -758,7 +759,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( ctx->save.live ) rc = send_domain_memory_live(ctx); - else if ( ctx->save.checkpointed ) + else if ( ctx->save.checkpointed != MIG_STREAM_NONE ) rc = send_domain_memory_checkpointed(ctx); else rc = send_domain_memory_nonlive(ctx); @@ -778,7 +779,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc ) goto err; - if ( ctx->save.checkpointed ) + if ( ctx->save.checkpointed != MIG_STREAM_NONE ) { /* * We have now completed the initial live portion of the checkpoint @@ -799,7 +800,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc <= 0 ) goto err; } - } while ( ctx->save.checkpointed ); + } while ( ctx->save.checkpointed != MIG_STREAM_NONE ); xc_report_progress_single(xch, "End of stream"); @@ -829,7 +830,8 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iters, uint32_t max_factor, uint32_t flags, - struct save_callbacks* callbacks, int hvm) + struct save_callbacks* callbacks, int hvm, + int checkpointed_stream) { struct xc_sr_context ctx = { @@ -841,7 +843,11 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, ctx.save.callbacks = callbacks; ctx.save.live = !!(flags & XCFLAGS_LIVE); ctx.save.debug = !!(flags & XCFLAGS_DEBUG); - ctx.save.checkpointed = !!(flags & XCFLAGS_CHECKPOINTED); + ctx.save.checkpointed = checkpointed_stream; + + /* If altering migration_stream update this assert too. */ + assert(checkpointed_stream == MIG_STREAM_NONE || + checkpointed_stream == MIG_STREAM_REMUS); /* * TODO: Find some time to better tweak the live migration algorithm. diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 8707b08..fc7844d 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -876,6 +876,7 @@ int libxl_domain_remus_start(libxl_ctx *ctx, libxl_domain_remus_info *info, dss->live = 1; dss->debug = 0; dss->remus = info; + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_REMUS; assert(info); @@ -936,6 +937,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd, int flags, dss->type = type; dss->live = flags & LIBXL_SUSPEND_LIVE; dss->debug = flags & LIBXL_SUSPEND_DEBUG; + dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE; rc = libxl__fd_flags_modify_save(gc, dss->fd, ~(O_NONBLOCK|O_NDELAY), 0, diff --git a/tools/libxl/libxl_dom_save.c b/tools/libxl/libxl_dom_save.c index 02cc143..cd2e7de 100644 --- a/tools/libxl/libxl_dom_save.c +++ b/tools/libxl/libxl_dom_save.c @@ -338,6 +338,12 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) unsigned int nr_vnodes = 0, nr_vmemranges = 0, nr_vcpus = 0; libxl__domain_suspend_state *dsps = &dss->dsps; + if (dss->checkpointed_stream != LIBXL_CHECKPOINTED_STREAM_NONE && !r_info) { + LOG(ERROR, "Migration stream is checkpointed, but there's no " + "checkpoint info!"); + goto out; + } + dss->rc = 0; logdirty_init(&dss->logdirty); dsps->ao = ao; @@ -376,15 +382,14 @@ void libxl__domain_save(libxl__egc *egc, libxl__domain_save_state *dss) goto out; } - if (r_info != NULL) { + if (dss->checkpointed_stream == LIBXL_CHECKPOINTED_STREAM_REMUS) { dss->interval = r_info->interval; - dss->xcflags |= XCFLAGS_CHECKPOINTED; if (libxl_defbool_val(r_info->compression)) dss->xcflags |= XCFLAGS_CHECKPOINT_COMPRESS; } memset(callbacks, 0, sizeof(*callbacks)); - if (r_info != NULL) { + if (dss->checkpointed_stream == LIBXL_CHECKPOINTED_STREAM_REMUS) { callbacks->suspend = libxl__remus_domain_suspend_callback; callbacks->postcopy = libxl__remus_domain_resume_callback; callbacks->checkpoint = libxl__remus_domain_save_checkpoint_callback; diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index bc48bec..fbd1acb 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3111,6 +3111,7 @@ struct libxl__domain_save_state { libxl_domain_type type; int live; int debug; + int checkpointed_stream; const libxl_domain_remus_info *remus; /* private */ int rc; diff --git a/tools/libxl/libxl_save_callout.c b/tools/libxl/libxl_save_callout.c index 2d06b42..416b318 100644 --- a/tools/libxl/libxl_save_callout.c +++ b/tools/libxl/libxl_save_callout.c @@ -85,7 +85,7 @@ void libxl__xc_domain_save(libxl__egc *egc, libxl__domain_save_state *dss, const unsigned long argnums[] = { dss->domid, 0, 0, dss->xcflags, dss->hvm, - cbflags, + cbflags, dss->checkpointed_stream, }; shs->ao = ao; diff --git a/tools/libxl/libxl_save_helper.c b/tools/libxl/libxl_save_helper.c index 39038f9..6bdcf13 100644 --- a/tools/libxl/libxl_save_helper.c +++ b/tools/libxl/libxl_save_helper.c @@ -253,6 +253,7 @@ int main(int argc, char **argv) uint32_t flags = strtoul(NEXTARG,0,10); int hvm = atoi(NEXTARG); unsigned cbflags = strtoul(NEXTARG,0,10); + int checkpointed_stream = strtoul(NEXTARG,0,10); assert(!*++argv); helper_setcallbacks_save(&helper_save_callbacks, cbflags); @@ -261,7 +262,7 @@ int main(int argc, char **argv) setup_signals(save_signal_handler); r = xc_domain_save(xch, io_fd, dom, max_iters, max_factor, flags, - &helper_save_callbacks, hvm); + &helper_save_callbacks, hvm, checkpointed_stream); complete(r); } else if (!strcmp(mode,"--restore-domain")) { diff --git a/tools/libxl/libxl_stream_write.c b/tools/libxl/libxl_stream_write.c index 9053146..f6ea55d 100644 --- a/tools/libxl/libxl_stream_write.c +++ b/tools/libxl/libxl_stream_write.c @@ -355,7 +355,7 @@ void libxl__xc_domain_save_done(libxl__egc *egc, void *dss_void, * If the stream is not still alive, we must not continue any work. */ if (libxl__stream_write_inuse(stream)) { - if (dss->remus) + if (dss->checkpointed_stream != LIBXL_CHECKPOINTED_STREAM_NONE) /* * For remus, if libxl__xc_domain_save_done() completes, * there was an error sending data to the secondary. diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index b8fb22f..605fb9a 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -228,6 +228,7 @@ libxl_hdtype = Enumeration("hdtype", [ (2, "AHCI"), ], init_val = "LIBXL_HDTYPE_IDE") +# Consistent with the values defined for migration_stream. libxl_checkpointed_stream = Enumeration("checkpointed_stream", [ (0, "NONE"), (1, "REMUS"),