From patchwork Fri Jun 8 08:10:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Wei W" X-Patchwork-Id: 10453991 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 65D666037F for ; Fri, 8 Jun 2018 08:43:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5459828068 for ; Fri, 8 Jun 2018 08:43:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4903B28524; Fri, 8 Jun 2018 08:43:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A83A628068 for ; Fri, 8 Jun 2018 08:43:25 +0000 (UTC) Received: from localhost ([::1]:34017 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fRCzU-00038L-V2 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 08 Jun 2018 04:43:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46520) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fRCsb-00066X-Fj for qemu-devel@nongnu.org; Fri, 08 Jun 2018 04:36:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fRCsa-0000RD-8i for qemu-devel@nongnu.org; Fri, 08 Jun 2018 04:36:17 -0400 Received: from mga09.intel.com ([134.134.136.24]:58067) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fRCsZ-0000OZ-S5 for qemu-devel@nongnu.org; Fri, 08 Jun 2018 04:36:16 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jun 2018 01:36:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,490,1520924400"; d="scan'208";a="65356036" Received: from devel-ww.sh.intel.com ([10.239.48.110]) by orsmga002.jf.intel.com with ESMTP; 08 Jun 2018 01:36:13 -0700 From: Wei Wang To: qemu-devel@nongnu.org, virtio-dev@lists.oasis-open.org, mst@redhat.com, quintela@redhat.com, dgilbert@redhat.com Date: Fri, 8 Jun 2018 16:10:41 +0800 Message-Id: <1528445443-43406-5-git-send-email-wei.w.wang@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1528445443-43406-1-git-send-email-wei.w.wang@intel.com> References: <1528445443-43406-1-git-send-email-wei.w.wang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.24 Subject: [Qemu-devel] [PATCH v8 4/6] migration/ram.c: add ram save state notifiers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yang.zhang.wz@gmail.com, quan.xu0@gmail.com, liliang.opensource@gmail.com, peterx@redhat.com, wei.w.wang@intel.com, pbonzini@redhat.com, nilal@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a ram save state notifier list, and expose RAMState for the notifer callbacks to use. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu --- include/migration/misc.h | 52 +++++++++++++++++++++++++++++++++++++++ migration/ram.c | 64 +++++++++++++++++------------------------------- 2 files changed, 75 insertions(+), 41 deletions(-) diff --git a/include/migration/misc.h b/include/migration/misc.h index 113320e..b970d7d 100644 --- a/include/migration/misc.h +++ b/include/migration/misc.h @@ -16,9 +16,61 @@ #include "exec/cpu-common.h" #include "qemu/notify.h" +#include "qemu/thread.h" /* migration/ram.c */ +typedef enum RamSaveState { + RAM_SAVE_ERR = 0, + RAM_SAVE_RESET = 1, + RAM_SAVE_BEFORE_SYNC_BITMAP = 2, + RAM_SAVE_AFTER_SYNC_BITMAP = 3, + RAM_SAVE_MAX = 4, +} RamSaveState; + +/* State of RAM for migration */ +typedef struct RAMState { + /* QEMUFile used for this migration */ + QEMUFile *f; + /* Last block that we have visited searching for dirty pages */ + RAMBlock *last_seen_block; + /* Last block from where we have sent data */ + RAMBlock *last_sent_block; + /* Last dirty target page we have sent */ + ram_addr_t last_page; + /* last ram version we have seen */ + uint32_t last_version; + /* We are in the first round */ + bool ram_bulk_stage; + /* How many times we have dirty too many pages */ + int dirty_rate_high_cnt; + /* ram save states used for notifiers */ + int ram_save_state; + /* these variables are used for bitmap sync */ + /* last time we did a full bitmap_sync */ + int64_t time_last_bitmap_sync; + /* bytes transferred at start_time */ + uint64_t bytes_xfer_prev; + /* number of dirty pages since start_time */ + uint64_t num_dirty_pages_period; + /* xbzrle misses since the beginning of the period */ + uint64_t xbzrle_cache_miss_prev; + /* number of iterations at the beginning of period */ + uint64_t iterations_prev; + /* Iterations since start */ + uint64_t iterations; + /* number of dirty bits in the bitmap */ + uint64_t migration_dirty_pages; + /* protects modification of the bitmap */ + QemuMutex bitmap_mutex; + /* The RAMBlock used in the last src_page_requests */ + RAMBlock *last_req_rb; + /* Queue of outstanding page requests from the destination */ + QemuMutex src_page_req_mutex; + QSIMPLEQ_HEAD(src_page_requests, RAMSrcPageRequest) src_page_requests; +} RAMState; + +void add_ram_save_state_change_notifier(Notifier *notify); void ram_mig_init(void); void qemu_guest_free_page_hint(void *addr, size_t len); diff --git a/migration/ram.c b/migration/ram.c index 237f11e..d45b5a4 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -56,6 +56,9 @@ #include "qemu/uuid.h" #include "savevm.h" +static NotifierList ram_save_state_notifiers = + NOTIFIER_LIST_INITIALIZER(ram_save_state_notifiers); + /***********************************************************/ /* ram save/restore */ @@ -267,49 +270,18 @@ struct RAMSrcPageRequest { QSIMPLEQ_ENTRY(RAMSrcPageRequest) next_req; }; -/* State of RAM for migration */ -struct RAMState { - /* QEMUFile used for this migration */ - QEMUFile *f; - /* Last block that we have visited searching for dirty pages */ - RAMBlock *last_seen_block; - /* Last block from where we have sent data */ - RAMBlock *last_sent_block; - /* Last dirty target page we have sent */ - ram_addr_t last_page; - /* last ram version we have seen */ - uint32_t last_version; - /* We are in the first round */ - bool ram_bulk_stage; - /* How many times we have dirty too many pages */ - int dirty_rate_high_cnt; - /* these variables are used for bitmap sync */ - /* last time we did a full bitmap_sync */ - int64_t time_last_bitmap_sync; - /* bytes transferred at start_time */ - uint64_t bytes_xfer_prev; - /* number of dirty pages since start_time */ - uint64_t num_dirty_pages_period; - /* xbzrle misses since the beginning of the period */ - uint64_t xbzrle_cache_miss_prev; - /* number of iterations at the beginning of period */ - uint64_t iterations_prev; - /* Iterations since start */ - uint64_t iterations; - /* number of dirty bits in the bitmap */ - uint64_t migration_dirty_pages; - /* protects modification of the bitmap */ - QemuMutex bitmap_mutex; - /* The RAMBlock used in the last src_page_requests */ - RAMBlock *last_req_rb; - /* Queue of outstanding page requests from the destination */ - QemuMutex src_page_req_mutex; - QSIMPLEQ_HEAD(src_page_requests, RAMSrcPageRequest) src_page_requests; -}; -typedef struct RAMState RAMState; - static RAMState *ram_state; +void add_ram_save_state_change_notifier(Notifier *notify) +{ + notifier_list_add(&ram_save_state_notifiers, notify); +} + +static void notify_ram_save_state_change_notifier(void) +{ + notifier_list_notify(&ram_save_state_notifiers, ram_state); +} + uint64_t ram_bytes_remaining(void) { return ram_state ? (ram_state->migration_dirty_pages * TARGET_PAGE_SIZE) : @@ -1139,6 +1111,9 @@ static void migration_bitmap_sync(RAMState *rs) int64_t end_time; uint64_t bytes_xfer_now; + rs->ram_save_state = RAM_SAVE_BEFORE_SYNC_BITMAP; + notify_ram_save_state_change_notifier(); + ram_counters.dirty_sync_count++; if (!rs->time_last_bitmap_sync) { @@ -1205,6 +1180,9 @@ static void migration_bitmap_sync(RAMState *rs) if (migrate_use_events()) { qapi_event_send_migration_pass(ram_counters.dirty_sync_count, NULL); } + + rs->ram_save_state = RAM_SAVE_AFTER_SYNC_BITMAP; + notify_ram_save_state_change_notifier(); } /** @@ -1961,6 +1939,8 @@ static void ram_state_reset(RAMState *rs) rs->last_page = 0; rs->last_version = ram_list.version; rs->ram_bulk_stage = true; + rs->ram_save_state = RAM_SAVE_RESET; + notify_ram_save_state_change_notifier(); } #define MAX_WAIT 50 /* ms, half buffered_file limit */ @@ -2709,6 +2689,8 @@ out: ret = qemu_file_get_error(f); if (ret < 0) { + rs->ram_save_state = RAM_SAVE_ERR; + notify_ram_save_state_change_notifier(); return ret; }