From patchwork Tue Oct 1 19:27:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Shinkevich X-Patchwork-Id: 11169529 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7393413B1 for ; Tue, 1 Oct 2019 19:36:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51E9E20B7C for ; Tue, 1 Oct 2019 19:36:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51E9E20B7C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFNwg-0002vP-Oh for patchwork-qemu-devel@patchwork.kernel.org; Tue, 01 Oct 2019 15:36:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48313) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iFNo8-0004iF-Hk for qemu-devel@nongnu.org; Tue, 01 Oct 2019 15:27:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iFNo7-0001SC-10 for qemu-devel@nongnu.org; Tue, 01 Oct 2019 15:27:36 -0400 Received: from relay.sw.ru ([185.231.240.75]:45300) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iFNo6-0001Qi-PF; Tue, 01 Oct 2019 15:27:34 -0400 Received: from [172.16.25.136] (helo=dhcp-172-16-25-136.sw.ru) by relay.sw.ru with esmtp (Exim 4.92.2) (envelope-from ) id 1iFNo2-00060a-Vx; Tue, 01 Oct 2019 22:27:31 +0300 From: Andrey Shinkevich To: qemu-devel@nongnu.org, qemu-block@nongnu.org Subject: [PATCH 1/6] qcow2: multiple clusters write compressed Date: Tue, 1 Oct 2019 22:27:15 +0300 Message-Id: <1569958040-697220-2-git-send-email-andrey.shinkevich@virtuozzo.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com> References: <1569958040-697220-1-git-send-email-andrey.shinkevich@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, fam@euphon.net, vsementsov@virtuozzo.com, jsnow@redhat.com, armbru@redhat.com, dgilbert@redhat.com, stefanha@redhat.com, andrey.shinkevich@virtuozzo.com, den@openvz.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" QEMU currently supports writing compressed data of size less than or equal to one cluster. This patch allows writing QCOW2 compressed data that exceed one cluster. The implementation is simple, we split buffered data into separate clusters and write them using the existing functionality. For unaligned requests, we use a workaround that writes the data without compression. Suggested-by: Pavel Butsykin Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich --- block/qcow2.c | 113 +++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 85 insertions(+), 28 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 7961c05..54ccaf6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4152,10 +4152,8 @@ fail: return ret; } -/* XXX: put compressed sectors first, then all the cluster aligned - tables to avoid losing bytes in alignment */ static coroutine_fn int -qcow2_co_pwritev_compressed_part(BlockDriverState *bs, +qcow2_co_pwritev_compressed_task(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset) { @@ -4165,36 +4163,14 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs, uint8_t *buf, *out_buf; uint64_t cluster_offset; - if (has_data_file(bs)) { - return -ENOTSUP; - } - - if (bytes == 0) { - /* align end of file to a sector boundary to ease reading with - sector based I/Os */ - int64_t len = bdrv_getlength(bs->file->bs); - if (len < 0) { - return len; - } - return bdrv_co_truncate(bs->file, len, PREALLOC_MODE_OFF, NULL); - } - - if (offset_into_cluster(s, offset)) { - return -EINVAL; - } + assert(bytes <= s->cluster_size); buf = qemu_blockalign(bs, s->cluster_size); - if (bytes != s->cluster_size) { - if (bytes > s->cluster_size || - offset + bytes != bs->total_sectors << BDRV_SECTOR_BITS) - { - qemu_vfree(buf); - return -EINVAL; - } + if (bytes < s->cluster_size) { /* Zero-pad last write if image size is not cluster aligned */ memset(buf + bytes, 0, s->cluster_size - bytes); } - qemu_iovec_to_buf(qiov, qiov_offset, buf, bytes); + qemu_iovec_to_buf(qiov, qiov_offset, buf, s->cluster_size); out_buf = g_malloc(s->cluster_size); @@ -4228,6 +4204,9 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs, BLKDBG_EVENT(s->data_file, BLKDBG_WRITE_COMPRESSED); ret = bdrv_co_pwrite(s->data_file, cluster_offset, out_len, out_buf, 0); + if (ret == -ENOTSUP) { + ret = qcow2_co_pwritev_part(bs, offset, bytes, qiov, qiov_offset, 0); + } if (ret < 0) { goto fail; } @@ -4239,6 +4218,84 @@ fail: return ret; } +static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task) +{ + Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); + + assert(!t->cluster_type); + + return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov, + t->qiov_offset); +} + +/* + * XXX: put compressed sectors first, then all the cluster aligned + tables to avoid losing bytes in alignment + */ +static coroutine_fn int +qcow2_co_pwritev_compressed_part(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, size_t qiov_offset) +{ + BDRVQcow2State *s = bs->opaque; + QCowL2Meta *l2meta = NULL; + AioTaskPool *aio = NULL; + uint64_t curr_off = 0; + int ret; + + if (has_data_file(bs)) { + return -ENOTSUP; + } + + if (bytes == 0) { + /* + * align end of file to a sector boundary to ease reading with + * sector based I/Os + */ + int64_t cluster_offset = bdrv_getlength(bs->file->bs); + if (cluster_offset < 0) { + return cluster_offset; + } + return bdrv_co_truncate(bs->file, cluster_offset, PREALLOC_MODE_OFF, + NULL); + } + + if (offset_into_cluster(s, offset)) { + return -EINVAL; + } + + while (bytes && aio_task_pool_status(aio) == 0) { + uint32_t chunk_size = MIN(bytes, s->cluster_size); + + assert((curr_off & (BDRV_SECTOR_SIZE - 1)) == 0); + assert((chunk_size & (BDRV_SECTOR_SIZE - 1)) == 0); + + if (!aio && chunk_size != bytes) { + aio = aio_task_pool_new(QCOW2_MAX_WORKERS); + } + + ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_compressed_task_entry, + 0, 0, offset + curr_off, chunk_size, + qiov, qiov_offset, l2meta); + if (ret < 0) { + break; + } + qiov_offset += chunk_size; + curr_off += chunk_size; + bytes -= chunk_size; + } + + if (aio) { + aio_task_pool_wait_all(aio); + if (ret == 0) { + ret = aio_task_pool_status(aio); + } + g_free(aio); + } + + return ret; +} + static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs, uint64_t file_cluster_offset,