From patchwork Thu Jul 20 03:49:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhoujian (jay)" X-Patchwork-Id: 9853713 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9E6A260388 for ; Thu, 20 Jul 2017 03:51:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82982286EB for ; Thu, 20 Jul 2017 03:51:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7752228750; Thu, 20 Jul 2017 03:51:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E4163286EB for ; Thu, 20 Jul 2017 03:51:29 +0000 (UTC) Received: from localhost ([::1]:35964 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dY2Uq-0006eR-3c for patchwork-qemu-devel@patchwork.kernel.org; Wed, 19 Jul 2017 23:51:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41760) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dY2UK-0006eK-D8 for qemu-devel@nongnu.org; Wed, 19 Jul 2017 23:50:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dY2UH-0003Y3-Aj for qemu-devel@nongnu.org; Wed, 19 Jul 2017 23:50:56 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:4471) by eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71) (envelope-from ) id 1dY2UG-0003Uq-N7 for qemu-devel@nongnu.org; Wed, 19 Jul 2017 23:50:53 -0400 Received: from 172.30.72.54 (EHLO DGGEML403-HUB.china.huawei.com) ([172.30.72.54]) by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id ASL15832; Thu, 20 Jul 2017 11:50:38 +0800 (CST) Received: from localhost (10.177.19.14) by DGGEML403-HUB.china.huawei.com (10.3.17.33) with Microsoft SMTP Server id 14.3.301.0; Thu, 20 Jul 2017 11:50:29 +0800 From: Jay Zhou To: Date: Thu, 20 Jul 2017 11:49:29 +0800 Message-ID: <1500522569-10760-1-git-send-email-jianjay.zhou@huawei.com> X-Mailer: git-send-email 2.6.1.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.177.19.14] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020203.59702890.0003, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: ee182059538049fdb1c1f8aae79d4a78 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] [fuzzy] X-Received-From: 45.249.212.187 Subject: [Qemu-devel] [PATCH] migration: optimize the downtime X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: weidong.huang@huawei.com, zhang.zhanghailiang@huawei.com, quintela@redhat.com, wangxinxin.wang@huawei.com, armbru@redhat.com, dgilbert@redhat.com, arei.gonglei@huawei.com, Jay Zhou Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Qemu_savevm_state_cleanup() takes about 300ms in my ram migration tests with a 8U24G vm(20G is really occupied), the main cost comes from KVM_SET_USER_MEMORY_REGION ioctl when mem.memory_size = 0 in kvm_set_user_memory_region(). In kmod, the main cost is kvm_zap_obsolete_pages(), which traverses the active_mmu_pages list to zap the unsync sptes. I think it can be optimized: (1) source vm will be destroyed if the migration is successfully done, so the resources will be cleanuped automatically by the system (2) delay the cleanup if the migration failed Signed-off-by: Jay Zhou --- migration/migration.c | 16 +++++++++------- qmp.c | 10 ++++++++++ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index a0db40d..72832be 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1877,6 +1877,15 @@ static void *migration_thread(void *opaque) if (qemu_file_get_error(s->to_dst_file)) { migrate_set_state(&s->state, current_active_state, MIGRATION_STATUS_FAILED); + /* + * The resource has been allocated by migration will be reused in + * COLO process, so don't release them. + */ + if (!enable_colo) { + qemu_mutex_lock_iothread(); + qemu_savevm_state_cleanup(); + qemu_mutex_unlock_iothread(); + } trace_migration_thread_file_err(); break; } @@ -1916,13 +1925,6 @@ static void *migration_thread(void *opaque) end_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME); qemu_mutex_lock_iothread(); - /* - * The resource has been allocated by migration will be reused in COLO - * process, so don't release them. - */ - if (!enable_colo) { - qemu_savevm_state_cleanup(); - } if (s->state == MIGRATION_STATUS_COMPLETED) { uint64_t transferred_bytes = qemu_ftell(s->to_dst_file); s->total_time = end_time - s->total_time; diff --git a/qmp.c b/qmp.c index b86201e..0e68eaa 100644 --- a/qmp.c +++ b/qmp.c @@ -37,6 +37,8 @@ #include "qom/object_interfaces.h" #include "hw/mem/pc-dimm.h" #include "hw/acpi/acpi_dev_interface.h" +#include "migration/migration.h" +#include "migration/savevm.h" NameInfo *qmp_query_name(Error **errp) { @@ -200,6 +202,14 @@ void qmp_cont(Error **errp) if (runstate_check(RUN_STATE_INMIGRATE)) { autostart = 1; } else { + /* + * Delay the cleanup to reduce the downtime of migration. + * The resource has been allocated by migration will be reused + * in COLO process, so don't release them. + */ + if (runstate_check(RUN_STATE_POSTMIGRATE) && !migrate_colo_enabled()) { + qemu_savevm_state_cleanup(); + } vm_start(); } }