From patchwork Tue Oct 20 14:39:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhengui li X-Patchwork-Id: 11847101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96A6161C for ; Tue, 20 Oct 2020 14:41:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CCD12224B for ; Tue, 20 Oct 2020 14:41:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3CCD12224B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47426 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUspm-00006S-V5 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 20 Oct 2020 10:41:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUsoc-0006sS-AS; Tue, 20 Oct 2020 10:40:42 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:5706 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUsoX-0001u8-Gq; Tue, 20 Oct 2020 10:40:42 -0400 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 196C1B679D09F2141CA0; Tue, 20 Oct 2020 22:40:18 +0800 (CST) Received: from DESKTOP-80C7KIU.china.huawei.com (10.174.187.210) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.487.0; Tue, 20 Oct 2020 22:40:12 +0800 From: Zhengui li To: , , , Subject: [PATCH v2 1/2] qemu-img: add support for rate limit in qemu-img commit Date: Tue, 20 Oct 2020 14:39:58 +0000 Message-ID: <1603204799-17544-2-git-send-email-lizhengui@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1603204799-17544-1-git-send-email-lizhengui@huawei.com> References: <1603204799-17544-1-git-send-email-lizhengui@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.210] X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.191; envelope-from=lizhengui@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/20 09:15:07 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, lizhengui@huawei.com, qemu-devel@nongnu.org, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Zhengui add support for rate limit in qemu-img commit. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 4 +++- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 11 +++++++++-- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index c35bd64..bcb11b0 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -349,7 +349,7 @@ Command description: state after (the attempt at) repairing it. That is, a successful ``-r all`` will yield the exit code 0, independently of the image state before. -.. option:: commit [--object OBJECTDEF] [--image-opts] [-q] [-f FMT] [-t CACHE] [-b BASE] [-d] [-p] FILENAME +.. option:: commit [--object OBJECTDEF] [--image-opts] [-q] [-f FMT] [-t CACHE] [-b BASE] [-r RATE_LIMIT] [-d] [-p] FILENAME Commit the changes recorded in *FILENAME* in its base image or backing file. If the backing file is smaller than the snapshot, then the backing file will be @@ -371,6 +371,8 @@ Command description: garbage data when read. For this reason, ``-b`` implies ``-d`` (so that the top image stays valid). + The rate limit for the commit process is specified by ``-r``. + .. option:: compare [--object OBJECTDEF] [--image-opts] [-f FMT] [-F FMT] [-T SRC_CACHE] [-p] [-q] [-s] [-U] FILENAME1 FILENAME2 Check if two images have the same content. You can compare images with diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index b89c019..2a31806 100644 --- a/qemu-img-cmds.hx +++ b/qemu-img-cmds.hx @@ -34,9 +34,9 @@ SRST ERST DEF("commit", img_commit, - "commit [--object objectdef] [--image-opts] [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename") + "commit [--object objectdef] [--image-opts] [-q] [-f fmt] [-t cache] [-b base] [-r rate_limit] [-d] [-p] filename") SRST -.. option:: commit [--object OBJECTDEF] [--image-opts] [-q] [-f FMT] [-t CACHE] [-b BASE] [-d] [-p] FILENAME +.. option:: commit [--object OBJECTDEF] [--image-opts] [-q] [-f FMT] [-t CACHE] [-b BASE] [-r RATE_LIMIT] [-d] [-p] FILENAME ERST DEF("compare", img_compare, diff --git a/qemu-img.c b/qemu-img.c index 2103507..3023abe 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -980,6 +980,7 @@ static int img_commit(int argc, char **argv) CommonBlockJobCBInfo cbi; bool image_opts = false; AioContext *aio_context; + int64_t rate_limit = 0; fmt = NULL; cache = BDRV_DEFAULT_CACHE; @@ -991,7 +992,7 @@ static int img_commit(int argc, char **argv) {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS}, {0, 0, 0, 0} }; - c = getopt_long(argc, argv, ":f:ht:b:dpq", + c = getopt_long(argc, argv, ":f:ht:b:dpqr:", long_options, NULL); if (c == -1) { break; @@ -1026,6 +1027,12 @@ static int img_commit(int argc, char **argv) case 'q': quiet = true; break; + case 'r': + rate_limit = cvtnum("rate limit", optarg); + if (rate_limit < 0) { + return 1; + } + break; case OPTION_OBJECT: { QemuOpts *opts; opts = qemu_opts_parse_noisily(&qemu_object_opts, @@ -1099,7 +1106,7 @@ static int img_commit(int argc, char **argv) aio_context = bdrv_get_aio_context(bs); aio_context_acquire(aio_context); - commit_active_start("commit", bs, base_bs, JOB_DEFAULT, 0, + commit_active_start("commit", bs, base_bs, JOB_DEFAULT, rate_limit, BLOCKDEV_ON_ERROR_REPORT, NULL, common_block_job_cb, &cbi, false, &local_err); aio_context_release(aio_context); From patchwork Tue Oct 20 14:39:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhengui li X-Patchwork-Id: 11847105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64D6F14B7 for ; Tue, 20 Oct 2020 14:42:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E37722251 for ; Tue, 20 Oct 2020 14:42:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E37722251 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:48434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUspy-0000Y1-Um for patchwork-qemu-devel@patchwork.kernel.org; Tue, 20 Oct 2020 10:42:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUsob-0006ql-Mu; Tue, 20 Oct 2020 10:40:41 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:5707 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUsoZ-0001uy-7o; Tue, 20 Oct 2020 10:40:41 -0400 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 22A4C24200025F8071F5; Tue, 20 Oct 2020 22:40:23 +0800 (CST) Received: from DESKTOP-80C7KIU.china.huawei.com (10.174.187.210) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.487.0; Tue, 20 Oct 2020 22:40:14 +0800 From: Zhengui li To: , , , Subject: [PATCH v2 2/2] qemu-img: add support for rate limit in qemu-img convert Date: Tue, 20 Oct 2020 14:39:59 +0000 Message-ID: <1603204799-17544-3-git-send-email-lizhengui@huawei.com> X-Mailer: git-send-email 2.6.4.windows.1 In-Reply-To: <1603204799-17544-1-git-send-email-lizhengui@huawei.com> References: <1603204799-17544-1-git-send-email-lizhengui@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.210] X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.191; envelope-from=lizhengui@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/20 09:15:07 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xieyingtai@huawei.com, lizhengui@huawei.com, qemu-devel@nongnu.org, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Zhengui add support for rate limit in qemu-img convert. Signed-off-by: Zhengui --- docs/tools/qemu-img.rst | 6 +++++- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 27 ++++++++++++++++++++++++++- 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst index bcb11b0..b615aa8 100644 --- a/docs/tools/qemu-img.rst +++ b/docs/tools/qemu-img.rst @@ -188,6 +188,10 @@ Parameters to convert subcommand: allocated target image depending on the host support for getting allocation information. +.. option:: -r + + Rate limit for the convert process + .. option:: --salvage Try to ignore I/O errors when reading. Unless in quiet mode (``-q``), errors @@ -410,7 +414,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index 2a31806..7eb489b 100644 --- a/qemu-img-cmds.hx +++ b/qemu-img-cmds.hx @@ -46,9 +46,9 @@ SRST ERST DEF("convert", img_convert, - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-r rate_limit] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") SRST -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME ERST DEF("create", img_create, diff --git a/qemu-img.c b/qemu-img.c index 3023abe..a968c74 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -50,6 +50,8 @@ #include "block/qapi.h" #include "crypto/init.h" #include "trace/control.h" +#include "qemu/throttle.h" +#include "block/throttle-groups.h" #define QEMU_IMG_VERSION "qemu-img version " QEMU_FULL_VERSION \ "\n" QEMU_COPYRIGHT "\n" @@ -1669,6 +1671,7 @@ enum ImgConvertBlockStatus { }; #define MAX_COROUTINES 16 +#define CONVERT_THROTTLE_GROUP "img_convert" typedef struct ImgConvertState { BlockBackend **src; @@ -2184,6 +2187,17 @@ static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst) #define MAX_BUF_SECTORS 32768 +static void set_rate_limit(BlockBackend *blk, int64_t rate_limit) +{ + ThrottleConfig cfg; + + throttle_config_init(&cfg); + cfg.buckets[THROTTLE_BPS_WRITE].avg = rate_limit; + + blk_io_limits_enable(blk, CONVERT_THROTTLE_GROUP); + blk_set_io_limits(blk, &cfg); +} + static int img_convert(int argc, char **argv) { int c, bs_i, flags, src_flags = 0; @@ -2204,6 +2218,7 @@ static int img_convert(int argc, char **argv) bool force_share = false; bool explict_min_sparse = false; bool bitmaps = false; + int64_t rate_limit = 0; ImgConvertState s = (ImgConvertState) { /* Need at least 4k of zeros for sparse detection */ @@ -2226,7 +2241,7 @@ static int img_convert(int argc, char **argv) {"bitmaps", no_argument, 0, OPTION_BITMAPS}, {0, 0, 0, 0} }; - c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WU", + c = getopt_long(argc, argv, ":hf:O:B:Cco:l:S:pt:T:qnm:WUr:", long_options, NULL); if (c == -1) { break; @@ -2323,6 +2338,12 @@ static int img_convert(int argc, char **argv) case 'U': force_share = true; break; + case 'r': + rate_limit = cvtnum("rate limit", optarg); + if (rate_limit < 0) { + goto fail_getopt; + } + break; case OPTION_OBJECT: { QemuOpts *object_opts; object_opts = qemu_opts_parse_noisily(&qemu_object_opts, @@ -2712,6 +2733,10 @@ static int img_convert(int argc, char **argv) s.cluster_sectors = bdi.cluster_size / BDRV_SECTOR_SIZE; } + if (rate_limit) { + set_rate_limit(s.target, rate_limit); + } + ret = convert_do_copy(&s); /* Now copy the bitmaps */