From patchwork Mon Feb 22 02:40:16 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 8370611 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 0BDDC9F314 for ; Mon, 22 Feb 2016 02:53:15 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3CE9120396 for ; Mon, 22 Feb 2016 02:53:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BF3720390 for ; Mon, 22 Feb 2016 02:53:13 +0000 (UTC) Received: from localhost ([::1]:46172 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aXgca-0001jM-Me for patchwork-qemu-devel@patchwork.kernel.org; Sun, 21 Feb 2016 21:53:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59945) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aXgRp-0002DM-Fy for qemu-devel@nongnu.org; Sun, 21 Feb 2016 21:42:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aXgRn-0007ZY-UE for qemu-devel@nongnu.org; Sun, 21 Feb 2016 21:42:05 -0500 Received: from szxga03-in.huawei.com ([119.145.14.66]:13896) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aXgRm-0007Yn-Hw for qemu-devel@nongnu.org; Sun, 21 Feb 2016 21:42:03 -0500 Received: from 172.24.1.48 (EHLO szxeml431-hub.china.huawei.com) ([172.24.1.48]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id BWL15763; Mon, 22 Feb 2016 10:41:29 +0800 (CST) Received: from localhost (10.177.24.212) by szxeml431-hub.china.huawei.com (10.82.67.208) with Microsoft SMTP Server id 14.3.235.1; Mon, 22 Feb 2016 10:41:19 +0800 From: zhanghailiang To: Date: Mon, 22 Feb 2016 10:40:16 +0800 Message-ID: <1456108832-24212-23-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 1.9.0.msysgit.0 In-Reply-To: <1456108832-24212-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1456108832-24212-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.24.212] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A090204.56CA755B.004E, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 4f97057348a06af5531f52076f940374 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.66 Cc: xiecl.fnst@cn.fujitsu.com, lizhijian@cn.fujitsu.com, quintela@redhat.com, armbru@redhat.com, yunhong.jiang@intel.com, eddie.dong@intel.com, peter.huangpeng@huawei.com, dgilbert@redhat.com, zhanghailiang , arei.gonglei@huawei.com, stefanha@redhat.com, amit.shah@redhat.com, zhangchen.fnst@cn.fujitsu.com, hongyang.yang@easystack.cn Subject: [Qemu-devel] [PATCH COLO-Frame v15 22/38] COLO failover: Shutdown related socket fd when do failover X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the net connection between COLO's two sides is broken while colo/colo incoming thread is blocked in 'read'/'write' socket fd. It will not detect this error until connect timeout. It will be a long time. Here we shutdown all the related socket file descriptors to wake up the blocking operation in failover BH. Besides, we should close the corresponding file descriptors after failvoer BH shutdown them, or there will be an error. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert Cc: Dr. David Alan Gilbert --- v13: - Add Reviewed-by tag - Use semaphore to notify colo/colo incoming loop that failover work is finished. v12: - Shutdown both QEMUFile's fd though they may use the same fd. (Dave's suggestion) v11: - Only shutdown fd for once --- include/migration/migration.h | 3 +++ migration/colo.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/include/migration/migration.h b/include/migration/migration.h index 14b9f3d..b34def6 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -112,6 +112,7 @@ struct MigrationIncomingState { QemuThread colo_incoming_thread; /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; + QemuSemaphore colo_incoming_sem; /* See savevm.c */ LoadStateEntry_Head loadvm_handlers; @@ -175,6 +176,8 @@ struct MigrationState QSIMPLEQ_HEAD(src_page_requests, MigrationSrcPageRequest) src_page_requests; /* The RAMBlock used in the last src_page_request */ RAMBlock *last_req_rb; + + QemuSemaphore colo_sem; }; void migrate_set_state(int *state, int old_state, int new_state); diff --git a/migration/colo.c b/migration/colo.c index 814480c..5c87a8e 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -59,6 +59,18 @@ static void secondary_vm_do_failover(void) /* recover runstate to normal migration finish state */ autostart = true; } + /* + * Make sure colo incoming thread not block in recv or send, + * If mis->from_src_file and mis->to_src_file use the same fd, + * The second shutdown() will return -1, we ignore this value, + * it is harmless. + */ + if (mis->from_src_file) { + qemu_file_shutdown(mis->from_src_file); + } + if (mis->to_src_file) { + qemu_file_shutdown(mis->to_src_file); + } old_state = failover_set_state(FAILOVER_STATUS_HANDLING, FAILOVER_STATUS_COMPLETED); @@ -67,6 +79,8 @@ static void secondary_vm_do_failover(void) "secondary VM", old_state); return; } + /* Notify COLO incoming thread that failover work is finished */ + qemu_sem_post(&mis->colo_incoming_sem); /* For Secondary VM, jump to incoming co */ if (mis->migration_incoming_co) { qemu_coroutine_enter(mis->migration_incoming_co, NULL); @@ -81,6 +95,18 @@ static void primary_vm_do_failover(void) migrate_set_state(&s->state, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED); + /* + * Make sure colo thread no block in recv or send, + * The s->rp_state.from_dst_file and s->to_dst_file may use the + * same fd, but we still shutdown the fd for twice, it is harmless. + */ + if (s->to_dst_file) { + qemu_file_shutdown(s->to_dst_file); + } + if (s->rp_state.from_dst_file) { + qemu_file_shutdown(s->rp_state.from_dst_file); + } + old_state = failover_set_state(FAILOVER_STATUS_HANDLING, FAILOVER_STATUS_COMPLETED); if (old_state != FAILOVER_STATUS_HANDLING) { @@ -88,6 +114,8 @@ static void primary_vm_do_failover(void) old_state); return; } + /* Notify COLO thread that failover work is finished */ + qemu_sem_post(&s->colo_sem); } void colo_do_failover(MigrationState *s) @@ -383,6 +411,14 @@ out: qsb_free(buffer); buffer = NULL; + /* Hope this not to be too long to wait here */ + qemu_sem_wait(&s->colo_sem); + qemu_sem_destroy(&s->colo_sem); + /* + * Must be called after failover BH is completed, + * Or the failover BH may shutdown the wrong fd, that + * re-used by other thread after we release here. + */ if (s->rp_state.from_dst_file) { qemu_fclose(s->rp_state.from_dst_file); } @@ -391,6 +427,7 @@ out: void migrate_start_colo_process(MigrationState *s) { qemu_mutex_unlock_iothread(); + qemu_sem_init(&s->colo_sem, 0); migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_COLO); colo_process_checkpoint(s); @@ -430,6 +467,8 @@ void *colo_process_incoming_thread(void *opaque) Error *local_err = NULL; int ret; + qemu_sem_init(&mis->colo_incoming_sem, 0); + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, MIGRATION_STATUS_COLO); @@ -561,6 +600,10 @@ out: */ colo_release_ram_cache(); + /* Hope this not to be too long to loop here */ + qemu_sem_wait(&mis->colo_incoming_sem); + qemu_sem_destroy(&mis->colo_incoming_sem); + /* Must be called after failover BH is completed */ if (mis->to_src_file) { qemu_fclose(mis->to_src_file); }