From patchwork Sat Aug 11 20:59:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10563565 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB2E313B4 for ; Sat, 11 Aug 2018 21:11:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE680298D9 for ; Sat, 11 Aug 2018 21:11:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D2BA029915; Sat, 11 Aug 2018 21:11:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6EF71298D9 for ; Sat, 11 Aug 2018 21:11:07 +0000 (UTC) Received: from localhost ([::1]:33229 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fobAA-0006QF-KS for patchwork-qemu-devel@patchwork.kernel.org; Sat, 11 Aug 2018 17:11:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48157) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1foazp-000500-7H for qemu-devel@nongnu.org; Sat, 11 Aug 2018 17:00:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1foazl-0001I3-W9 for qemu-devel@nongnu.org; Sat, 11 Aug 2018 17:00:25 -0400 Received: from mail-pl0-x242.google.com ([2607:f8b0:400e:c01::242]:36149) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1foazl-0001HR-Ln for qemu-devel@nongnu.org; Sat, 11 Aug 2018 17:00:21 -0400 Received: by mail-pl0-x242.google.com with SMTP id e11-v6so5386116plb.3 for ; Sat, 11 Aug 2018 14:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=t0ew/6qCUx4k4vyc0bty50OVKj22t3leUPx2lVu+1E0=; b=E69gCNRTKUeXvpVrfRVIbMaEBsj+QQJtQZcKorfg52yF/Lir4UW5amkob+zmIdIoW2 qVFkMO2H/0aqcgMLNRBJGntsp23uXmHa00k4TpcOUQQLBH0gTUkOBjm0dlhKIRpcAi2d ohVOM5/pFngnYSMJdT58foGqwlvt0VRwYMt7+n2g3c+YUTWlBWSFvrLOP2tzpQ8BH5xZ 9vf/rrvR38toKgErXYYhI+kE8gSH8F5Pd1HoTouc324xsooZlIMFXr97a3sWUbekUUVS LRwide2PqzNtVsQFoF5id1W9c0WyuLO3BwckxoWFYrokVpNeke7nJY9Srb89fqefxDBC J4qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=t0ew/6qCUx4k4vyc0bty50OVKj22t3leUPx2lVu+1E0=; b=D+QGfMBIC852ATSp+H32XF+lR0IjdUeEMbyWYWN4TacYVm60SeV1AbwSF4SbjRl2kl 0tUkXvwNph/QSHe2rfHP5+gCzGg6f2qBGzfk5ucGkR9DRmdIxClc1IuV450ujF5w4WIk L5QFYHwkkbZk11ebAr3JOvwYiNEOl9UVaknCANfbUxNrmMm+OCaMjCdZdYoyI64zo4h5 fOEWqhqUnocnvGsHWrL9rmasRHyeJfw0kxAw8SrBg+973h2C8U0wwJovSptUIU/ZC8WX i5di0tBwTYKmc2bUngV8v/R9ux0w9B5tzFrZJjNc2GHd3jZRbuNw0hNoog13KHsfI9lf heRQ== X-Gm-Message-State: AOUpUlFK8khhV9lJqZmEjTC9vSYyEhyBgy+xzPod0OGn4HjFvyCVWOuh SY/HHIhg5GRDjpq+EGBKgmgpg6zW X-Google-Smtp-Source: AA+uWPzJCiHINFJNrrZ4FbAGpOdunR/BBrM2QNyvMz+o4435VnbWnJtMYfY25tnG7CvNCCOT3bMz9Q== X-Received: by 2002:a17:902:a40b:: with SMTP id p11-v6mr11167271plq.228.1534021220008; Sat, 11 Aug 2018 14:00:20 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id h24-v6sm23502641pfk.113.2018.08.11.14.00.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 11 Aug 2018 14:00:18 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Sun, 12 Aug 2018 04:59:12 +0800 Message-Id: <20180811205924.4113-9-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180811205924.4113-1-zhangckid@gmail.com> References: <20180811205924.4113-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::242 Subject: [Qemu-devel] [PATCH V11 08/20] ram/COLO: Record the dirty pages that SVM received X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 43 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 40 insertions(+), 3 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index ce2cf99bf5..3dc403adb5 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3332,6 +3332,15 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, __func__, block->idstr); return NULL; } + + /* + * During colo checkpoint, we need bitmap of these migrated pages. + * It help us to decide which pages in ram cache should be flushed + * into VM's RAM later. + */ + if (!test_and_set_bit(offset >> TARGET_PAGE_BITS, block->bmap)) { + ram_state->migration_dirty_pages++; + } return block->colo_cache + offset; } @@ -3549,7 +3558,7 @@ int colo_init_ram_cache(void) RAMBlock *block; rcu_read_lock(); - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + RAMBLOCK_FOREACH_MIGRATABLE(block) { block->colo_cache = qemu_anon_ram_alloc(block->used_length, NULL, false); @@ -3562,10 +3571,29 @@ int colo_init_ram_cache(void) memcpy(block->colo_cache, block->host, block->used_length); } rcu_read_unlock(); + /* + * Record the dirty pages that sent by PVM, we use this dirty bitmap together + * with to decide which page in cache should be flushed into SVM's RAM. Here + * we use the same name 'ram_bitmap' as for migration. + */ + if (ram_bytes_total()) { + RAMBlock *block; + + RAMBLOCK_FOREACH_MIGRATABLE(block) { + unsigned long pages = block->max_length >> TARGET_PAGE_BITS; + + block->bmap = bitmap_new(pages); + bitmap_set(block->bmap, 0, pages); + } + } + ram_state = g_new0(RAMState, 1); + ram_state->migration_dirty_pages = 0; + return 0; out_locked: - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + + RAMBLOCK_FOREACH_MIGRATABLE(block) { if (block->colo_cache) { qemu_anon_ram_free(block->colo_cache, block->used_length); block->colo_cache = NULL; @@ -3581,14 +3609,23 @@ void colo_release_ram_cache(void) { RAMBlock *block; + RAMBLOCK_FOREACH_MIGRATABLE(block) { + g_free(block->bmap); + block->bmap = NULL; + } + rcu_read_lock(); - QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + + RAMBLOCK_FOREACH_MIGRATABLE(block) { if (block->colo_cache) { qemu_anon_ram_free(block->colo_cache, block->used_length); block->colo_cache = NULL; } } + rcu_read_unlock(); + g_free(ram_state); + ram_state = NULL; } /**