From patchwork Wed May 9 14:58:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fam Zheng X-Patchwork-Id: 10389717 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7D5D960318 for ; Wed, 9 May 2018 15:05:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A65228FB5 for ; Wed, 9 May 2018 15:05:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D34229004; Wed, 9 May 2018 15:05:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C523228FB5 for ; Wed, 9 May 2018 15:05:02 +0000 (UTC) Received: from localhost ([::1]:56992 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQeL-0005D4-Ug for patchwork-qemu-devel@patchwork.kernel.org; Wed, 09 May 2018 11:05:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56961) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fGQZ7-0008SX-J7 for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fGQZ6-0007wj-2N for qemu-devel@nongnu.org; Wed, 09 May 2018 10:59:37 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:37318 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fGQYy-0007gD-0P; Wed, 09 May 2018 10:59:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 60B8B81A88A2; Wed, 9 May 2018 14:59:27 +0000 (UTC) Received: from lemon.usersys.redhat.com (ovpn-12-170.pek2.redhat.com [10.72.12.170]) by smtp.corp.redhat.com (Postfix) with ESMTP id 14A8CD74BA; Wed, 9 May 2018 14:59:18 +0000 (UTC) From: Fam Zheng To: qemu-devel@nongnu.org Date: Wed, 9 May 2018 22:58:15 +0800 Message-Id: <20180509145815.3330-10-famz@redhat.com> In-Reply-To: <20180509145815.3330-1-famz@redhat.com> References: <20180509145815.3330-1-famz@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:59:27 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Wed, 09 May 2018 14:59:27 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'famz@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [PATCH v3 9/9] qemu-img: Convert with copy offloading X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Stefan Hajnoczi , qemu-block@nongnu.org, Peter Lieven , Max Reitz , Ronnie Sahlberg , Paolo Bonzini Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Update 178.out.qcow2 as we are now a little more efficient on the actual file size. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- qemu-img.c | 50 ++++++++++++++++++++++++++++++++++++++-- tests/qemu-iotests/178.out.qcow2 | 8 +++---- 2 files changed, 52 insertions(+), 6 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index ea62d2d61e..9c3678ebf0 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1544,6 +1544,7 @@ typedef struct ImgConvertState { bool compressed; bool target_has_backing; bool wr_in_order; + bool copy_range; int min_sparse; size_t cluster_sectors; size_t buf_sectors; @@ -1737,6 +1738,37 @@ static int coroutine_fn convert_co_write(ImgConvertState *s, int64_t sector_num, return 0; } +static int coroutine_fn convert_co_copy_range(ImgConvertState *s, int64_t sector_num, + int nb_sectors) +{ + int n, ret; + + while (nb_sectors > 0) { + BlockBackend *blk; + int src_cur; + int64_t bs_sectors, src_cur_offset; + int64_t offset; + + convert_select_part(s, sector_num, &src_cur, &src_cur_offset); + offset = (sector_num - src_cur_offset) << BDRV_SECTOR_BITS; + blk = s->src[src_cur]; + bs_sectors = s->src_sectors[src_cur]; + + n = MIN(nb_sectors, bs_sectors - (sector_num - src_cur_offset)); + + ret = blk_co_copy_range(blk, offset, s->target, + sector_num << BDRV_SECTOR_BITS, + n << BDRV_SECTOR_BITS, 0); + if (ret < 0) { + return ret; + } + + sector_num += n; + nb_sectors -= n; + } + return 0; +} + static void coroutine_fn convert_co_do_copy(void *opaque) { ImgConvertState *s = opaque; @@ -1759,6 +1791,7 @@ static void coroutine_fn convert_co_do_copy(void *opaque) int n; int64_t sector_num; enum ImgConvertBlockStatus status; + bool copy_range; qemu_co_mutex_lock(&s->lock); if (s->ret != -EINPROGRESS || s->sector_num >= s->total_sectors) { @@ -1788,7 +1821,9 @@ static void coroutine_fn convert_co_do_copy(void *opaque) s->allocated_sectors, 0); } - if (status == BLK_DATA) { +retry: + copy_range = s->copy_range && s->status == BLK_DATA; + if (status == BLK_DATA && !copy_range) { ret = convert_co_read(s, sector_num, n, buf); if (ret < 0) { error_report("error while reading sector %" PRId64 @@ -1810,7 +1845,15 @@ static void coroutine_fn convert_co_do_copy(void *opaque) } if (s->ret == -EINPROGRESS) { - ret = convert_co_write(s, sector_num, n, buf, status); + if (copy_range) { + ret = convert_co_copy_range(s, sector_num, n); + if (ret) { + s->copy_range = false; + goto retry; + } + } else { + ret = convert_co_write(s, sector_num, n, buf, status); + } if (ret < 0) { error_report("error while writing sector %" PRId64 ": %s", sector_num, strerror(-ret)); @@ -1933,6 +1976,7 @@ static int img_convert(int argc, char **argv) ImgConvertState s = (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ .min_sparse = 8, + .copy_range = true, .buf_sectors = IO_BUF_SIZE / BDRV_SECTOR_SIZE, .wr_in_order = true, .num_coroutines = 8, @@ -1973,6 +2017,7 @@ static int img_convert(int argc, char **argv) break; case 'c': s.compressed = true; + s.copy_range = false; break; case 'o': if (!is_valid_option_list(optarg)) { @@ -2014,6 +2059,7 @@ static int img_convert(int argc, char **argv) } s.min_sparse = sval / BDRV_SECTOR_SIZE; + s.copy_range = false; break; } case 'p': diff --git a/tests/qemu-iotests/178.out.qcow2 b/tests/qemu-iotests/178.out.qcow2 index d42d4a4597..3e72893451 100644 --- a/tests/qemu-iotests/178.out.qcow2 +++ b/tests/qemu-iotests/178.out.qcow2 @@ -54,7 +54,7 @@ wrote 64512/64512 bytes at offset 134217728 required size: 589824 fully allocated size: 1074135040 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 == qcow2 input image with internal snapshot (human) == @@ -107,7 +107,7 @@ wrote 64512/64512 bytes at offset 134217728 required size: 589824 fully allocated size: 1074135040 -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 == raw input image and a backing file (human) == @@ -187,7 +187,7 @@ wrote 64512/64512 bytes at offset 134217728 "fully-allocated": 1074135040 } -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 == qcow2 input image with internal snapshot (json) == @@ -254,7 +254,7 @@ wrote 64512/64512 bytes at offset 134217728 "fully-allocated": 1074135040 } -converted image file size in bytes: 524288 +converted image file size in bytes: 523264 == raw input image and a backing file (json) ==