From patchwork Fri Mar 25 06:44:18 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changlong Xie X-Patchwork-Id: 8668721 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 36B15C0554 for ; Fri, 25 Mar 2016 06:45:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 251C320395 for ; Fri, 25 Mar 2016 06:45:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD12C20380 for ; Fri, 25 Mar 2016 06:45:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLSP-0006gV-As; Fri, 25 Mar 2016 06:42:53 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajLSN-0006d1-Qe for xen-devel@lists.xen.org; Fri, 25 Mar 2016 06:42:51 +0000 Received: from [85.158.143.35] by server-1.bemta-6.messagelabs.com id 3C/E1-29237-BEDD4F65; Fri, 25 Mar 2016 06:42:51 +0000 X-Env-Sender: xiecl.fnst@cn.fujitsu.com X-Msg-Ref: server-13.tower-21.messagelabs.com!1458888168!5953439!1 X-Originating-IP: [59.151.112.132] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 62426 invoked from network); 25 Mar 2016 06:42:49 -0000 Received: from unknown (HELO heian.cn.fujitsu.com) (59.151.112.132) by server-13.tower-21.messagelabs.com with SMTP; 25 Mar 2016 06:42:49 -0000 X-IronPort-AV: E=Sophos;i="5.22,518,1449504000"; d="scan'208";a="4958501" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 25 Mar 2016 14:42:24 +0800 Received: from G08CNEXCHPEKD03.g08.fujitsu.local (unknown [10.167.33.85]) by cn.fujitsu.com (Postfix) with ESMTP id C6848408D26A; Fri, 25 Mar 2016 14:42:20 +0800 (CST) Received: from changlox.g08.fujitsu.local (10.167.225.55) by G08CNEXCHPEKD03.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.279.2; Fri, 25 Mar 2016 14:42:20 +0800 From: Changlong Xie To: xen devel , Konrad Rzeszutek Wilk , Andrew Cooper , Ian Campbell , Ian Jackson , Wei Liu Date: Fri, 25 Mar 2016 14:44:18 +0800 Message-ID: <1458888273-7469-12-git-send-email-xiecl.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> References: <1458888273-7469-1-git-send-email-xiecl.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.55] X-yoursite-MailScanner-ID: C6848408D26A.A2508 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: xiecl.fnst@cn.fujitsu.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 Cc: Lars Kurth , Changlong Xie , Wen Congyang , Li Zhijian , Gui Jianfeng , Jiang Yunhong , Dong Eddie , Anthony Perard , Shriram Rajagopalan , Yang Hongyang Subject: [Xen-devel] [PATCH v13 11/26] tools/libxl: add back channel support to read stream X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wen Congyang This is used by primay to read records sent by secondary. Note: The function libxl__stream_read_checkpoint_state() will be used in later patches called "secondary vm suspend/resume/checkpoint code" and "primary vm suspend/resume/checkpoint code". Signed-off-by: Yang Hongyang Signed-off-by: Wen Congyang Signed-off-by: Changlong Xie Acked-by: Ian Jackson --- tools/libxl/libxl_create.c | 1 + tools/libxl/libxl_internal.h | 4 ++ tools/libxl/libxl_stream_read.c | 94 +++++++++++++++++++++++++++++++++++++---- 3 files changed, 91 insertions(+), 8 deletions(-) diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index 09f2f13..4d2b95c 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -1034,6 +1034,7 @@ static void domcreate_bootloader_done(libxl__egc *egc, dcs->srs.dcs = dcs; dcs->srs.fd = restore_fd; dcs->srs.legacy = (dcs->restore_params.stream_version == 1); + dcs->srs.back_channel = false; dcs->srs.completion_callback = domcreate_stream_done; if (restore_fd >= 0) { diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 0e5f9f8..1fafba8 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -3444,6 +3444,7 @@ struct libxl__stream_read_state { libxl__domain_create_state *dcs; int fd; bool legacy; + bool back_channel; void (*completion_callback)(libxl__egc *egc, libxl__stream_read_state *srs, int rc); @@ -3455,6 +3456,7 @@ struct libxl__stream_read_state { bool running; bool in_checkpoint; bool sync_teardown; /* Only used to coordinate shutdown on error path. */ + bool in_checkpoint_state; libxl__save_helper_state shs; libxl__conversion_helper_state chs; @@ -3482,6 +3484,8 @@ _hidden void libxl__stream_read_start(libxl__egc *egc, libxl__stream_read_state *stream); _hidden void libxl__stream_read_start_checkpoint(libxl__egc *egc, libxl__stream_read_state *stream); +_hidden void libxl__stream_read_checkpoint_state(libxl__egc *egc, + libxl__stream_read_state *stream); _hidden void libxl__stream_read_abort(libxl__egc *egc, libxl__stream_read_state *stream, int rc); static inline bool diff --git a/tools/libxl/libxl_stream_read.c b/tools/libxl/libxl_stream_read.c index f4781eb..302ae53 100644 --- a/tools/libxl/libxl_stream_read.c +++ b/tools/libxl/libxl_stream_read.c @@ -118,6 +118,15 @@ * record, and therefore the buffered state is inconsistent. In * libxl__xc_domain_restore_done(), we just complete the stream and * stream->completion_callback() will be called to resume the guest + * + * For back channel stream: + * - libxl__stream_read_start() + * - Set up the stream to running state + * + * - libxl__stream_read_continue() + * - Set up reading the next record from a started stream. + * Add some codes to process_record() to handle the record. + * Then call stream->checkpoint_callback() to return. */ /* Success/error/cleanup handling. */ @@ -157,6 +166,10 @@ static void write_emulator_done(libxl__egc *egc, libxl__datacopier_state *dc, int rc, int onwrite, int errnoval); +/* Handlers for checkpoint state mini-loop */ +static void checkpoint_state_done(libxl__egc *egc, + libxl__stream_read_state *stream, int rc); + /*----- Helpers -----*/ /* Helper to set up reading some data from the stream. */ @@ -221,6 +234,14 @@ void libxl__stream_read_start(libxl__egc *egc, stream->running = true; stream->phase = SRS_PHASE_NORMAL; + dc->ao = stream->ao; + dc->copywhat = "restore v2 stream"; + dc->readfd = stream->fd; + dc->writefd = -1; + + if (stream->back_channel) + return; + if (stream->legacy) { /* Convert the legacy stream. */ libxl__conversion_helper_state *chs = &stream->chs; @@ -241,12 +262,6 @@ void libxl__stream_read_start(libxl__egc *egc, stream->fd = libxl__carefd_fd(stream->chs.v2_carefd); stream->dcs->libxc_fd = stream->fd; } - /* stream->fd is now a v2 stream. */ - - dc->ao = stream->ao; - dc->copywhat = "restore v2 stream"; - dc->readfd = stream->fd; - dc->writefd = -1; /* Start reading the stream header. */ rc = setup_read(stream, "stream header", @@ -532,6 +547,7 @@ static bool process_record(libxl__egc *egc, STATE_AO_GC(stream->ao); libxl__domain_create_state *dcs = stream->dcs; libxl__sr_record_buf *rec; + libxl_sr_checkpoint_state *srcs; bool further_action_needed = false; int rc = 0; @@ -602,6 +618,17 @@ static bool process_record(libxl__egc *egc, checkpoint_done(egc, stream, 0); break; + case REC_TYPE_CHECKPOINT_STATE: + if (!stream->in_checkpoint_state) { + LOG(ERROR, "Unexpected CHECKPOINT_STATE record in stream"); + rc = ERROR_FAIL; + goto err; + } + + srcs = rec->body; + checkpoint_state_done(egc, stream, srcs->id); + break; + default: LOG(ERROR, "Unrecognised record 0x%08x", rec->hdr.type); rc = ERROR_FAIL; @@ -713,6 +740,21 @@ static void stream_complete(libxl__egc *egc, return; } + if (stream->in_checkpoint_state) { + assert(rc); + + /* + * If an error is encountered while in a checkpoint, pass it + * back to libxc. The failure will come back around to us via + * 1. normal stream + * libxl__xc_domain_restore_done() + * 2. back_channel stream + * libxl__stream_read_abort() + */ + checkpoint_state_done(egc, stream, rc); + return; + } + stream_done(egc, stream, rc); } @@ -743,6 +785,7 @@ static void stream_done(libxl__egc *egc, assert(stream->running); assert(!stream->in_checkpoint); + assert(!stream->in_checkpoint_state); stream->running = false; if (stream->incoming_record) @@ -762,7 +805,19 @@ static void stream_done(libxl__egc *egc, LIBXL_STAILQ_FOREACH_SAFE(rec, &stream->record_queue, entry, trec) free_record(rec); - check_all_finished(egc, stream, rc); + if (!stream->back_channel) { + /* + * 1. In stream_done(), stream->running is set to false, so + * the stream itself is not in use. + * 2. Read stream is a back channel stream, this means it is + * only used by primary(save side) to read records sent by + * secondary(restore side), so it doesn't have restore helper. + * 3. Back channel stream doesn't support legacy stream, so + * there is no conversion helper. + * So we don't need invoke check_all_finished here + */ + check_all_finished(egc, stream, rc); + } } void libxl__xc_domain_restore_done(libxl__egc *egc, void *dcs_void, @@ -862,7 +917,30 @@ static void check_all_finished(libxl__egc *egc, libxl__conversion_helper_inuse(&stream->chs)) return; - stream->completion_callback(egc, stream, stream->rc); + if (stream->completion_callback) + /* back channel stream doesn't have completion_callback() */ + stream->completion_callback(egc, stream, stream->rc); +} + +/*----- Checkpoint state handlers -----*/ + +void libxl__stream_read_checkpoint_state(libxl__egc *egc, + libxl__stream_read_state *stream) +{ + assert(stream->running); + assert(!stream->in_checkpoint); + assert(!stream->in_checkpoint_state); + stream->in_checkpoint_state = true; + + setup_read_record(egc, stream); +} + +static void checkpoint_state_done(libxl__egc *egc, + libxl__stream_read_state *stream, int rc) +{ + assert(stream->in_checkpoint_state); + stream->in_checkpoint_state = false; + stream->checkpoint_callback(egc, stream, rc); } /*