From patchwork Tue Dec 24 15:19:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11309513 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A37166C1 for ; Tue, 24 Dec 2019 15:20:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73B6A206D3 for ; Tue, 24 Dec 2019 15:20:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="H0As9JV8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73B6A206D3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ijlyP-0000Do-Og; Tue, 24 Dec 2019 15:19:49 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ijlyO-0000DP-2r for xen-devel@lists.xenproject.org; Tue, 24 Dec 2019 15:19:48 +0000 X-Inumbo-ID: ccc7aac7-2660-11ea-97e5-12813bfff9fa Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id ccc7aac7-2660-11ea-97e5-12813bfff9fa; Tue, 24 Dec 2019 15:19:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1577200778; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=DS7TGPbdI6Ja+UTk1N5gZTJcM0lO8dXhUM0YMqhj2aw=; b=H0As9JV8qOTq4FjsavGUIoQfKouz6XaazE5ZneY6USBcH3G+w3p1+8c7 P0BmbJG1dZnYl5hZhDjnrsrTXrwQ88m2Hl4PtOzFgQo6AS3qw7oo64ZWa QRAKAQQ05EUDTOjt12miy1Cbt4dBRTyxLkL+9dzIx+/RROEZY8D3QZ/Z/ c=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: qITykmmPSrYxJ1UVeKOFXXQnxV4iBtaACOTvUr4cavWkIWSjGaMjzHSXF50tgdURjxyOcmLXxV ihGnPIheZKLlUI2fgsZuy11yRCR4GsVBTEJiNfQhmAP+wcoV3Qw0U77ciCo4Lij/X6hvHuWtGO VbyvmIywaRZvK+7kwG5Leg/s7Jmj0txfHOr70gfHTIuobHhF9I4nYy+5pMbqLIeNMsF1DIkWd5 VE9gLEBS2G1nxVzHSaMGidGQcDA2cPZ9PzikFh5KhO0H+OvPkGweBpfAH2K2UH+rGsYhjqlT2g j8o= X-SBRS: 2.7 X-MesageID: 10482747 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,351,1571716800"; d="scan'208";a="10482747" From: Andrew Cooper To: Xen-devel Date: Tue, 24 Dec 2019 15:19:23 +0000 Message-ID: <20191224151932.6304-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20191224151932.6304-1-andrew.cooper3@citrix.com> References: <20191224151932.6304-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type' X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Originally, 'checkpointed' was a boolean signalling the difference between a plain and a Remus stream. COLO was added later, but several bits of code retained boolean-style logic. While correct, it is confusing to follow. Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no stream". Consolidate all the logic on the term 'stream_type', and rename STREAM_NONE to STREAM_PLAIN. Re-position the stream_type variable so it isn't duplicated in both the save and restore unions. No functional change. Signed-off-by: Andrew Cooper Acked-by: Ian Jackson --- CC: Ian Jackson CC: Wei Liu --- tools/libxc/include/xenguest.h | 15 +++++++------- tools/libxc/xc_nomigrate.c | 4 ++-- tools/libxc/xc_sr_common.h | 9 +++------ tools/libxc/xc_sr_restore.c | 33 +++++++++++++++++++------------ tools/libxc/xc_sr_save.c | 44 ++++++++++++++++++++++++----------------- tools/libxl/libxl_save_helper.c | 4 ++-- 6 files changed, 61 insertions(+), 48 deletions(-) diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h index b4b2e19619..9ba09af743 100644 --- a/tools/libxc/include/xenguest.h +++ b/tools/libxc/include/xenguest.h @@ -115,11 +115,12 @@ struct save_callbacks { void* data; }; +/* Type of stream. Plain, or using a continuous replication protocol? */ typedef enum { - XC_MIG_STREAM_NONE, /* plain stream */ - XC_MIG_STREAM_REMUS, - XC_MIG_STREAM_COLO, -} xc_migration_stream_t; + XC_STREAM_PLAIN, + XC_STREAM_REMUS, + XC_STREAM_COLO, +} xc_stream_type_t; /** * This function will save a running domain. @@ -127,14 +128,14 @@ typedef enum { * @parm xch a handle to an open hypervisor interface * @parm fd the file descriptor to save a domain to * @parm dom the id of the domain - * @param stream_type XC_MIG_STREAM_NONE if the far end of the stream + * @param stream_type XC_STREAM_PLAIN if the far end of the stream * doesn't use checkpointing * @return 0 on success, -1 on failure */ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t flags /* XCFLAGS_xxx */, struct save_callbacks* callbacks, int hvm, - xc_migration_stream_t stream_type, int recv_fd); + xc_stream_type_t stream_type, int recv_fd); /* callbacks provided by xc_domain_restore */ struct restore_callbacks { @@ -198,7 +199,7 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, uint32_t store_domid, unsigned int console_evtchn, unsigned long *console_mfn, uint32_t console_domid, unsigned int hvm, unsigned int pae, - xc_migration_stream_t stream_type, + xc_stream_type_t stream_type, struct restore_callbacks *callbacks, int send_back_fd); /** diff --git a/tools/libxc/xc_nomigrate.c b/tools/libxc/xc_nomigrate.c index 6d6169d5ad..3099e3278c 100644 --- a/tools/libxc/xc_nomigrate.c +++ b/tools/libxc/xc_nomigrate.c @@ -22,7 +22,7 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t flags, struct save_callbacks* callbacks, int hvm, - xc_migration_stream_t stream_type, int recv_fd) + xc_stream_type_t stream_type, int recv_fd) { errno = ENOSYS; return -1; @@ -33,7 +33,7 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, uint32_t store_domid, unsigned int console_evtchn, unsigned long *console_mfn, uint32_t console_domid, unsigned int hvm, unsigned int pae, - xc_migration_stream_t stream_type, + xc_stream_type_t stream_type, struct restore_callbacks *callbacks, int send_back_fd) { errno = ENOSYS; diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h index 19b053911f..4db63a63b2 100644 --- a/tools/libxc/xc_sr_common.h +++ b/tools/libxc/xc_sr_common.h @@ -205,6 +205,9 @@ struct xc_sr_context uint32_t domid; int fd; + /* Plain VM, or checkpoints over time. */ + xc_stream_type_t stream_type; + xc_dominfo_t dominfo; union /* Common save or restore data. */ @@ -219,9 +222,6 @@ struct xc_sr_context /* Live migrate vs non live suspend. */ bool live; - /* Plain VM, or checkpoints over time. */ - int checkpointed; - /* Further debugging information in the stream. */ bool debug; @@ -252,9 +252,6 @@ struct xc_sr_context uint32_t guest_type; uint32_t guest_page_size; - /* Plain VM, or checkpoints over time. */ - int checkpointed; - /* Currently buffering records between a checkpoint */ bool buffer_all_records; diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c index 98f3fe4098..7872b71ab5 100644 --- a/tools/libxc/xc_sr_restore.c +++ b/tools/libxc/xc_sr_restore.c @@ -511,7 +511,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx) int rc = 0, ret; unsigned int i; - if ( !ctx->restore.checkpointed ) + if ( ctx->stream_type == XC_STREAM_PLAIN ) { ERROR("Found checkpoint in non-checkpointed stream"); rc = -1; @@ -553,7 +553,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx) else ctx->restore.buffer_all_records = true; - if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) { #define HANDLE_CALLBACK_RETURN_VALUE(ret) \ do { \ @@ -672,7 +672,7 @@ static int setup(struct xc_sr_context *ctx) DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap, &ctx->restore.dirty_bitmap_hbuf); - if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) { dirty_bitmap = xc_hypercall_buffer_alloc_pages( xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size))); @@ -723,7 +723,7 @@ static void cleanup(struct xc_sr_context *ctx) for ( i = 0; i < ctx->restore.buffered_rec_num; i++ ) free(ctx->restore.buffered_records[i].data); - if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) xc_hypercall_buffer_free_pages( xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size))); @@ -793,7 +793,7 @@ static int restore(struct xc_sr_context *ctx) } while ( rec.type != REC_TYPE_END ); remus_failover: - if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) { /* With COLO, we have already called stream_complete */ rc = 0; @@ -834,13 +834,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, uint32_t store_domid, unsigned int console_evtchn, unsigned long *console_gfn, uint32_t console_domid, unsigned int hvm, unsigned int pae, - xc_migration_stream_t stream_type, + xc_stream_type_t stream_type, struct restore_callbacks *callbacks, int send_back_fd) { xen_pfn_t nr_pfns; struct xc_sr_context ctx = { .xch = xch, .fd = io_fd, + .stream_type = stream_type, }; /* GCC 4.4 (of CentOS 6.x vintage) can' t initialise anonymous unions. */ @@ -848,21 +849,27 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, ctx.restore.console_domid = console_domid; ctx.restore.xenstore_evtchn = store_evtchn; ctx.restore.xenstore_domid = store_domid; - ctx.restore.checkpointed = stream_type; ctx.restore.callbacks = callbacks; ctx.restore.send_back_fd = send_back_fd; - /* Sanity checks for callbacks. */ - if ( stream_type ) - assert(callbacks->checkpoint); - - if ( ctx.restore.checkpointed == XC_MIG_STREAM_COLO ) + /* Sanity check stream_type-related parameters */ + switch ( stream_type ) { - /* this is COLO restore */ + case XC_STREAM_COLO: assert(callbacks->suspend && callbacks->postcopy && callbacks->wait_checkpoint && callbacks->restore_results); + /* Fallthrough */ + case XC_STREAM_REMUS: + assert(callbacks->checkpoint); + /* Fallthrough */ + case XC_STREAM_PLAIN: + break; + + default: + assert(!"Bad stream_type"); + break; } DPRINTF("fd %d, dom %u, hvm %u, pae %u, stream_type %d", diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c index 9764aa743f..5467965b08 100644 --- a/tools/libxc/xc_sr_save.c +++ b/tools/libxc/xc_sr_save.c @@ -658,7 +658,7 @@ static int suspend_and_send_dirty(struct xc_sr_context *ctx) bitmap_or(dirty_bitmap, ctx->save.deferred_pages, ctx->save.p2m_size); - if ( !ctx->save.live && ctx->save.checkpointed == XC_MIG_STREAM_COLO ) + if ( !ctx->save.live && ctx->stream_type == XC_STREAM_COLO ) { rc = colo_merge_secondary_dirty_bitmap(ctx); if ( rc ) @@ -735,7 +735,7 @@ static int send_domain_memory_live(struct xc_sr_context *ctx) if ( rc ) goto out; - if ( ctx->save.debug && ctx->save.checkpointed != XC_MIG_STREAM_NONE ) + if ( ctx->save.debug && ctx->stream_type != XC_STREAM_PLAIN ) { rc = verify_frames(ctx); if ( rc ) @@ -861,7 +861,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( ctx->save.live ) rc = send_domain_memory_live(ctx); - else if ( ctx->save.checkpointed != XC_MIG_STREAM_NONE ) + else if ( ctx->stream_type != XC_STREAM_PLAIN ) rc = send_domain_memory_checkpointed(ctx); else rc = send_domain_memory_nonlive(ctx); @@ -881,7 +881,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc ) goto err; - if ( ctx->save.checkpointed != XC_MIG_STREAM_NONE ) + if ( ctx->stream_type != XC_STREAM_PLAIN ) { /* * We have now completed the initial live portion of the checkpoint @@ -894,7 +894,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc ) goto err; - if ( ctx->save.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) { rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data); if ( !rc ) @@ -908,14 +908,14 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) if ( rc <= 0 ) goto err; - if ( ctx->save.checkpointed == XC_MIG_STREAM_COLO ) + if ( ctx->stream_type == XC_STREAM_COLO ) { rc = ctx->save.callbacks->wait_checkpoint( ctx->save.callbacks->data); if ( rc <= 0 ) goto err; } - else if ( ctx->save.checkpointed == XC_MIG_STREAM_REMUS ) + else if ( ctx->stream_type == XC_STREAM_REMUS ) { rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data); if ( rc <= 0 ) @@ -928,7 +928,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) goto err; } } - } while ( ctx->save.checkpointed != XC_MIG_STREAM_NONE ); + } while ( ctx->stream_type != XC_STREAM_PLAIN ); xc_report_progress_single(xch, "End of stream"); @@ -958,32 +958,40 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type) int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t flags, struct save_callbacks *callbacks, - int hvm, xc_migration_stream_t stream_type, int recv_fd) + int hvm, xc_stream_type_t stream_type, int recv_fd) { struct xc_sr_context ctx = { .xch = xch, .fd = io_fd, + .stream_type = stream_type, }; /* GCC 4.4 (of CentOS 6.x vintage) can' t initialise anonymous unions. */ ctx.save.callbacks = callbacks; ctx.save.live = !!(flags & XCFLAGS_LIVE); ctx.save.debug = !!(flags & XCFLAGS_DEBUG); - ctx.save.checkpointed = stream_type; ctx.save.recv_fd = recv_fd; - /* If altering migration_stream update this assert too. */ - assert(stream_type == XC_MIG_STREAM_NONE || - stream_type == XC_MIG_STREAM_REMUS || - stream_type == XC_MIG_STREAM_COLO); + /* Sanity check stream_type-related parameters */ + switch ( stream_type ) + { + case XC_STREAM_COLO: + assert(callbacks->wait_checkpoint); + /* Fallthrough */ + case XC_STREAM_REMUS: + assert(callbacks->checkpoint && callbacks->postcopy); + /* Fallthrough */ + case XC_STREAM_PLAIN: + break; + + default: + assert(!"Bad stream_type"); + break; + } /* Sanity checks for callbacks. */ if ( hvm ) assert(callbacks->switch_qemu_logdirty); - if ( ctx.save.checkpointed ) - assert(callbacks->checkpoint && callbacks->postcopy); - if ( ctx.save.checkpointed == XC_MIG_STREAM_COLO ) - assert(callbacks->wait_checkpoint); DPRINTF("fd %d, dom %u, flags %u, hvm %d", io_fd, dom, flags, hvm); diff --git a/tools/libxl/libxl_save_helper.c b/tools/libxl/libxl_save_helper.c index 38089a002d..398df00dd6 100644 --- a/tools/libxl/libxl_save_helper.c +++ b/tools/libxl/libxl_save_helper.c @@ -254,7 +254,7 @@ int main(int argc, char **argv) uint32_t flags = strtoul(NEXTARG,0,10); int hvm = atoi(NEXTARG); unsigned cbflags = strtoul(NEXTARG,0,10); - xc_migration_stream_t stream_type = strtoul(NEXTARG,0,10); + xc_stream_type_t stream_type = strtoul(NEXTARG,0,10); assert(!*++argv); helper_setcallbacks_save(&helper_save_callbacks, cbflags); @@ -278,7 +278,7 @@ int main(int argc, char **argv) unsigned int hvm = strtoul(NEXTARG,0,10); unsigned int pae = strtoul(NEXTARG,0,10); unsigned cbflags = strtoul(NEXTARG,0,10); - xc_migration_stream_t stream_type = strtoul(NEXTARG,0,10); + xc_stream_type_t stream_type = strtoul(NEXTARG,0,10); assert(!*++argv); helper_setcallbacks_restore(&helper_restore_callbacks, cbflags);