From patchwork Tue Oct 31 13:54:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22529C4167B for ; Tue, 31 Oct 2023 13:57:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEY-0000iN-5a; Tue, 31 Oct 2023 09:56:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDy-0000VM-Tm; Tue, 31 Oct 2023 09:56:08 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDw-0000lI-Jt; Tue, 31 Oct 2023 09:56:06 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 123B842A7E; Tue, 31 Oct 2023 14:55:51 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 01/10] blockjob: introduce block-job-change QMP command Date: Tue, 31 Oct 2023 14:54:22 +0100 Message-Id: <20231031135431.393137-2-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org which will allow changing job-type-specific options after job creation. In the JobVerbTable, the same allow bits as for set-speed are used, because set-speed can be considered an existing change command. Signed-off-by: Fiona Ebner Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- Changes in v4: * Add note that change callback can be invoked before job coroutine is running. blockdev.c | 14 ++++++++++++++ blockjob.c | 20 ++++++++++++++++++++ include/block/blockjob.h | 11 +++++++++++ include/block/blockjob_int.h | 7 +++++++ job.c | 1 + qapi/block-core.json | 26 ++++++++++++++++++++++++++ qapi/job.json | 4 +++- 7 files changed, 82 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index a01c62596b..639000d7d8 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3382,6 +3382,20 @@ void qmp_block_job_dismiss(const char *id, Error **errp) job_dismiss_locked(&job, errp); } +void qmp_block_job_change(BlockJobChangeOptions *opts, Error **errp) +{ + BlockJob *job; + + JOB_LOCK_GUARD(); + job = find_block_job_locked(opts->id, errp); + + if (!job) { + return; + } + + block_job_change_locked(job, opts, errp); +} + void qmp_change_backing_file(const char *device, const char *image_node_name, const char *backing_file, diff --git a/blockjob.c b/blockjob.c index 807f992b59..11bae119bd 100644 --- a/blockjob.c +++ b/blockjob.c @@ -328,6 +328,26 @@ static bool block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) return block_job_set_speed_locked(job, speed, errp); } +void block_job_change_locked(BlockJob *job, BlockJobChangeOptions *opts, + Error **errp) +{ + const BlockJobDriver *drv = block_job_driver(job); + + GLOBAL_STATE_CODE(); + + if (job_apply_verb_locked(&job->job, JOB_VERB_CHANGE, errp)) { + return; + } + + if (drv->change) { + job_unlock(); + drv->change(job, opts, errp); + job_lock(); + } else { + error_setg(errp, "Job type does not support change"); + } +} + void block_job_ratelimit_processed_bytes(BlockJob *job, uint64_t n) { IO_CODE(); diff --git a/include/block/blockjob.h b/include/block/blockjob.h index 058b0c824c..95854f1477 100644 --- a/include/block/blockjob.h +++ b/include/block/blockjob.h @@ -172,6 +172,17 @@ bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs); */ bool block_job_set_speed_locked(BlockJob *job, int64_t speed, Error **errp); +/** + * block_job_change_locked: + * @job: The job to change. + * @opts: The new options. + * @errp: Error object. + * + * Change the job according to opts. + */ +void block_job_change_locked(BlockJob *job, BlockJobChangeOptions *opts, + Error **errp); + /** * block_job_query_locked: * @job: The job to get information about. diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index 104824040c..a4656d4cb5 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -67,6 +67,13 @@ struct BlockJobDriver { void (*attached_aio_context)(BlockJob *job, AioContext *new_context); void (*set_speed)(BlockJob *job, int64_t speed); + + /* + * Change the @job's options according to @opts. + * + * Note that this can already be called before the job coroutine is running. + */ + void (*change)(BlockJob *job, BlockJobChangeOptions *opts, Error **errp); }; /* diff --git a/job.c b/job.c index 72d57f0934..99a2e54b54 100644 --- a/job.c +++ b/job.c @@ -80,6 +80,7 @@ bool JobVerbTable[JOB_VERB__MAX][JOB_STATUS__MAX] = { [JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0}, [JOB_VERB_FINALIZE] = {0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0}, [JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + [JOB_VERB_CHANGE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, }; /* Transactional group of jobs */ diff --git a/qapi/block-core.json b/qapi/block-core.json index 89751d81f2..c6f31a9399 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3044,6 +3044,32 @@ { 'command': 'block-job-finalize', 'data': { 'id': 'str' }, 'allow-preconfig': true } +## +# @BlockJobChangeOptions: +# +# Block job options that can be changed after job creation. +# +# @id: The job identifier +# +# @type: The job type +# +# Since 8.2 +## +{ 'union': 'BlockJobChangeOptions', + 'base': { 'id': 'str', 'type': 'JobType' }, + 'discriminator': 'type', + 'data': {} } + +## +# @block-job-change: +# +# Change the block job's options. +# +# Since: 8.2 +## +{ 'command': 'block-job-change', + 'data': 'BlockJobChangeOptions', 'boxed': true } + ## # @BlockdevDiscardOptions: # diff --git a/qapi/job.json b/qapi/job.json index 7f0ba090de..b3957207a4 100644 --- a/qapi/job.json +++ b/qapi/job.json @@ -105,11 +105,13 @@ # # @finalize: see @job-finalize # +# @change: see @block-job-change (since 8.2) +# # Since: 2.12 ## { 'enum': 'JobVerb', 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', - 'finalize' ] } + 'finalize', 'change' ] } ## # @JOB_STATUS_CHANGE: From patchwork Tue Oct 31 13:54:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2633EC4167B for ; Tue, 31 Oct 2023 13:59:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEd-0001Fo-ON; Tue, 31 Oct 2023 09:56:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDz-0000Vs-Qj; Tue, 31 Oct 2023 09:56:08 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDw-0000lB-UG; Tue, 31 Oct 2023 09:56:07 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 8E90F42A69; Tue, 31 Oct 2023 14:55:50 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 02/10] block/mirror: set actively_synced even after the job is ready Date: Tue, 31 Oct 2023 14:54:23 +0100 Message-Id: <20231031135431.393137-3-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation to allow switching from background to active mode. This ensures that setting actively_synced will not be missed when the switch happens after the job is ready. Signed-off-by: Fiona Ebner Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- No changes in v4. block/mirror.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index dcd88de2e3..1c2c00ee1d 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1074,9 +1074,9 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) * the target in a consistent state. */ job_transition_to_ready(&s->common.job); - if (s->copy_mode != MIRROR_COPY_MODE_BACKGROUND) { - s->actively_synced = true; - } + } + if (s->copy_mode != MIRROR_COPY_MODE_BACKGROUND) { + s->actively_synced = true; } should_complete = s->should_complete || From patchwork Tue Oct 31 13:54:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93643C4332F for ; Tue, 31 Oct 2023 13:57:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEc-0001Ae-J7; Tue, 31 Oct 2023 09:56:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDy-0000UX-Ol; Tue, 31 Oct 2023 09:56:08 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDw-0000l6-Tn; Tue, 31 Oct 2023 09:56:06 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id B972542A6F; Tue, 31 Oct 2023 14:55:50 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 03/10] block/mirror: move dirty bitmap to filter Date: Tue, 31 Oct 2023 14:54:24 +0100 Message-Id: <20231031135431.393137-4-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation to allow switching to active mode without draining. Initialization of the bitmap in mirror_dirty_init() still happens with the original/backing BlockDriverState, which should be fine, because the mirror top has the same length. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Fiona Ebner Reviewed-by: Eric Blake --- Changes in v4: * Also set actively_synced to false when setting the bitmap in bdrv_mirror_top_do_write. block/mirror.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 1c2c00ee1d..914d723446 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1500,6 +1500,11 @@ bdrv_mirror_top_do_write(BlockDriverState *bs, MirrorMethod method, abort(); } + if (!copy_to_target && s->job && s->job->dirty_bitmap) { + s->job->actively_synced = false; + bdrv_set_dirty_bitmap(s->job->dirty_bitmap, offset, bytes); + } + if (ret < 0) { goto out; } @@ -1823,13 +1828,17 @@ static BlockJob *mirror_start_job( s->should_complete = true; } - s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp); + s->dirty_bitmap = bdrv_create_dirty_bitmap(s->mirror_top_bs, granularity, + NULL, errp); if (!s->dirty_bitmap) { goto fail; } - if (s->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING) { - bdrv_disable_dirty_bitmap(s->dirty_bitmap); - } + + /* + * The dirty bitmap is set by bdrv_mirror_top_do_write() when not in active + * mode. + */ + bdrv_disable_dirty_bitmap(s->dirty_bitmap); ret = block_job_add_bdrv(&s->common, "source", bs, 0, BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE | From patchwork Tue Oct 31 13:54:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9087AC4332F for ; Tue, 31 Oct 2023 13:59:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEe-0001H7-Iq; Tue, 31 Oct 2023 09:56:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDy-0000Ur-QV; Tue, 31 Oct 2023 09:56:08 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpDw-0000lN-Js; Tue, 31 Oct 2023 09:56:06 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 90ACE42A54; Tue, 31 Oct 2023 14:55:51 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 04/10] block/mirror: determine copy_to_target only once Date: Tue, 31 Oct 2023 14:54:25 +0100 Message-Id: <20231031135431.393137-5-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation to allow changing the copy_mode via QMP. When running in an iothread, it could be that copy_mode is changed from the main thread in between reading copy_mode in bdrv_mirror_top_pwritev() and reading copy_mode in bdrv_mirror_top_do_write(), so they might end up disagreeing about whether copy_to_target is true or false. Avoid that scenario by determining copy_to_target only once and passing it to bdrv_mirror_top_do_write() as an argument. Signed-off-by: Fiona Ebner Reviewed-by: Vladimir Sementsov-Ogievskiy --- No changes in v4. block/mirror.c | 41 ++++++++++++++++++----------------------- 1 file changed, 18 insertions(+), 23 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 914d723446..31da1526eb 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -1463,21 +1463,21 @@ bdrv_mirror_top_preadv(BlockDriverState *bs, int64_t offset, int64_t bytes, return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); } +static bool should_copy_to_target(MirrorBDSOpaque *s) +{ + return s->job && s->job->ret >= 0 && + !job_is_cancelled(&s->job->common.job) && + s->job->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING; +} + static int coroutine_fn GRAPH_RDLOCK bdrv_mirror_top_do_write(BlockDriverState *bs, MirrorMethod method, - uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, - int flags) + bool copy_to_target, uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) { MirrorOp *op = NULL; MirrorBDSOpaque *s = bs->opaque; int ret = 0; - bool copy_to_target = false; - - if (s->job) { - copy_to_target = s->job->ret >= 0 && - !job_is_cancelled(&s->job->common.job) && - s->job->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING; - } if (copy_to_target) { op = active_write_prepare(s->job, offset, bytes); @@ -1524,17 +1524,10 @@ static int coroutine_fn GRAPH_RDLOCK bdrv_mirror_top_pwritev(BlockDriverState *bs, int64_t offset, int64_t bytes, QEMUIOVector *qiov, BdrvRequestFlags flags) { - MirrorBDSOpaque *s = bs->opaque; QEMUIOVector bounce_qiov; void *bounce_buf; int ret = 0; - bool copy_to_target = false; - - if (s->job) { - copy_to_target = s->job->ret >= 0 && - !job_is_cancelled(&s->job->common.job) && - s->job->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING; - } + bool copy_to_target = should_copy_to_target(bs->opaque); if (copy_to_target) { /* The guest might concurrently modify the data to write; but @@ -1551,8 +1544,8 @@ bdrv_mirror_top_pwritev(BlockDriverState *bs, int64_t offset, int64_t bytes, flags &= ~BDRV_REQ_REGISTERED_BUF; } - ret = bdrv_mirror_top_do_write(bs, MIRROR_METHOD_COPY, offset, bytes, qiov, - flags); + ret = bdrv_mirror_top_do_write(bs, MIRROR_METHOD_COPY, copy_to_target, + offset, bytes, qiov, flags); if (copy_to_target) { qemu_iovec_destroy(&bounce_qiov); @@ -1575,15 +1568,17 @@ static int coroutine_fn GRAPH_RDLOCK bdrv_mirror_top_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int64_t bytes, BdrvRequestFlags flags) { - return bdrv_mirror_top_do_write(bs, MIRROR_METHOD_ZERO, offset, bytes, NULL, - flags); + bool copy_to_target = should_copy_to_target(bs->opaque); + return bdrv_mirror_top_do_write(bs, MIRROR_METHOD_ZERO, copy_to_target, + offset, bytes, NULL, flags); } static int coroutine_fn GRAPH_RDLOCK bdrv_mirror_top_pdiscard(BlockDriverState *bs, int64_t offset, int64_t bytes) { - return bdrv_mirror_top_do_write(bs, MIRROR_METHOD_DISCARD, offset, bytes, - NULL, 0); + bool copy_to_target = should_copy_to_target(bs->opaque); + return bdrv_mirror_top_do_write(bs, MIRROR_METHOD_DISCARD, copy_to_target, + offset, bytes, NULL, 0); } static void bdrv_mirror_top_refresh_filename(BlockDriverState *bs) From patchwork Tue Oct 31 13:54:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A947FC4332F for ; Tue, 31 Oct 2023 13:59:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEe-0001Gg-7i; Tue, 31 Oct 2023 09:56:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000XD-L6; Tue, 31 Oct 2023 09:56:16 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qE-Tb; Tue, 31 Oct 2023 09:56:11 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 0C4D742A60; Tue, 31 Oct 2023 14:55:52 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 05/10] mirror: implement mirror_change method Date: Tue, 31 Oct 2023 14:54:26 +0100 Message-Id: <20231031135431.393137-6-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org which allows switching the @copy-mode from 'background' to 'write-blocking'. This is useful for management applications, so they can start out in background mode to avoid limiting guest write speed and switch to active mode when certain criteria are fulfilled. In presence of an iothread, the copy_mode member is now shared between the iothread and the main thread, so turn accesses to it atomic. Signed-off-by: Fiona Ebner --- Changes in v4: * add comment about how copy_mode should be accessed * add GLOBAL_STATE_CODE annotation for mirror_change * use two spaces in between sentences in QAPI description * fix typo in QAPI description block/mirror.c | 44 +++++++++++++++++++++++++++++++++++++++++--- qapi/block-core.json | 13 ++++++++++++- 2 files changed, 53 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 31da1526eb..4016d89253 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -55,6 +55,10 @@ typedef struct MirrorBlockJob { BlockMirrorBackingMode backing_mode; /* Whether the target image requires explicit zero-initialization */ bool zero_target; + /* + * To be accesssed with atomics. Written only under the BQL (required by the + * current implementation of mirror_change()). + */ MirrorCopyMode copy_mode; BlockdevOnError on_source_error, on_target_error; /* Set when the target is synced (dirty bitmap is clean, nothing @@ -1075,7 +1079,7 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) */ job_transition_to_ready(&s->common.job); } - if (s->copy_mode != MIRROR_COPY_MODE_BACKGROUND) { + if (qatomic_read(&s->copy_mode) != MIRROR_COPY_MODE_BACKGROUND) { s->actively_synced = true; } @@ -1246,6 +1250,39 @@ static bool commit_active_cancel(Job *job, bool force) return force || !job_is_ready(job); } +static void mirror_change(BlockJob *job, BlockJobChangeOptions *opts, + Error **errp) +{ + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); + BlockJobChangeOptionsMirror *change_opts = &opts->u.mirror; + MirrorCopyMode current; + + /* + * The implementation relies on the fact that copy_mode is only written + * under the BQL. Otherwise, further synchronization would be required. + */ + + GLOBAL_STATE_CODE(); + + if (qatomic_read(&s->copy_mode) == change_opts->copy_mode) { + return; + } + + if (change_opts->copy_mode != MIRROR_COPY_MODE_WRITE_BLOCKING) { + error_setg(errp, "Change to copy mode '%s' is not implemented", + MirrorCopyMode_str(change_opts->copy_mode)); + return; + } + + current = qatomic_cmpxchg(&s->copy_mode, MIRROR_COPY_MODE_BACKGROUND, + change_opts->copy_mode); + if (current != MIRROR_COPY_MODE_BACKGROUND) { + error_setg(errp, "Expected current copy mode '%s', got '%s'", + MirrorCopyMode_str(MIRROR_COPY_MODE_BACKGROUND), + MirrorCopyMode_str(current)); + } +} + static const BlockJobDriver mirror_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), @@ -1260,6 +1297,7 @@ static const BlockJobDriver mirror_job_driver = { .cancel = mirror_cancel, }, .drained_poll = mirror_drained_poll, + .change = mirror_change, }; static const BlockJobDriver commit_active_job_driver = { @@ -1467,7 +1505,7 @@ static bool should_copy_to_target(MirrorBDSOpaque *s) { return s->job && s->job->ret >= 0 && !job_is_cancelled(&s->job->common.job) && - s->job->copy_mode == MIRROR_COPY_MODE_WRITE_BLOCKING; + qatomic_read(&s->job->copy_mode) == MIRROR_COPY_MODE_WRITE_BLOCKING; } static int coroutine_fn GRAPH_RDLOCK @@ -1813,7 +1851,7 @@ static BlockJob *mirror_start_job( s->is_none_mode = is_none_mode; s->backing_mode = backing_mode; s->zero_target = zero_target; - s->copy_mode = copy_mode; + qatomic_set(&s->copy_mode, copy_mode); s->base = base; s->base_overlay = bdrv_find_overlay(bs, base); s->granularity = granularity; diff --git a/qapi/block-core.json b/qapi/block-core.json index c6f31a9399..6369207be2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3044,6 +3044,17 @@ { 'command': 'block-job-finalize', 'data': { 'id': 'str' }, 'allow-preconfig': true } +## +# @BlockJobChangeOptionsMirror: +# +# @copy-mode: Switch to this copy mode. Currently, only the switch +# from 'background' to 'write-blocking' is implemented. +# +# Since: 8.2 +## +{ 'struct': 'BlockJobChangeOptionsMirror', + 'data': { 'copy-mode' : 'MirrorCopyMode' } } + ## # @BlockJobChangeOptions: # @@ -3058,7 +3069,7 @@ { 'union': 'BlockJobChangeOptions', 'base': { 'id': 'str', 'type': 'JobType' }, 'discriminator': 'type', - 'data': {} } + 'data': { 'mirror': 'BlockJobChangeOptionsMirror' } } ## # @block-job-change: From patchwork Tue Oct 31 13:54:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4580CC4332F for ; Tue, 31 Oct 2023 13:58:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEf-0001I2-7M; Tue, 31 Oct 2023 09:56:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000X1-6f; Tue, 31 Oct 2023 09:56:15 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qD-UM; Tue, 31 Oct 2023 09:56:11 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id DD8B242A84; Tue, 31 Oct 2023 14:55:51 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 06/10] qapi/block-core: use JobType for BlockJobInfo's type Date: Tue, 31 Oct 2023 14:54:27 +0100 Message-Id: <20231031135431.393137-7-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation to turn BlockJobInfo into a union with @type as the discriminator. That requires it to be an enum. Even without that requirement, it's nicer to have an enum instead of a str here. No functional change is intended. Signed-off-by: Fiona Ebner Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Markus Armbruster --- Changes in v4: * Mention that it's nicer to have an enum regardless in commit message. block/monitor/block-hmp-cmds.c | 4 ++-- blockjob.c | 2 +- qapi/block-core.json | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c index 7645c7e5fb..5b2c597e7a 100644 --- a/block/monitor/block-hmp-cmds.c +++ b/block/monitor/block-hmp-cmds.c @@ -846,7 +846,7 @@ void hmp_info_block_jobs(Monitor *mon, const QDict *qdict) } while (list) { - if (strcmp(list->value->type, "stream") == 0) { + if (list->value->type == JOB_TYPE_STREAM) { monitor_printf(mon, "Streaming device %s: Completed %" PRId64 " of %" PRId64 " bytes, speed limit %" PRId64 " bytes/s\n", @@ -858,7 +858,7 @@ void hmp_info_block_jobs(Monitor *mon, const QDict *qdict) monitor_printf(mon, "Type %s, device %s: Completed %" PRId64 " of %" PRId64 " bytes, speed limit %" PRId64 " bytes/s\n", - list->value->type, + JobType_str(list->value->type), list->value->device, list->value->offset, list->value->len, diff --git a/blockjob.c b/blockjob.c index 11bae119bd..9665b02627 100644 --- a/blockjob.c +++ b/blockjob.c @@ -388,7 +388,7 @@ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) &progress_total); info = g_new0(BlockJobInfo, 1); - info->type = g_strdup(job_type_str(&job->job)); + info->type = job_type(&job->job); info->device = g_strdup(job->job.id); info->busy = job->job.busy; info->paused = job->job.pause_count > 0; diff --git a/qapi/block-core.json b/qapi/block-core.json index 6369207be2..9d03210664 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1396,7 +1396,7 @@ # Since: 1.1 ## { 'struct': 'BlockJobInfo', - 'data': {'type': 'str', 'device': 'str', 'len': 'int', + 'data': {'type': 'JobType', 'device': 'str', 'len': 'int', 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', 'status': 'JobStatus', From patchwork Tue Oct 31 13:54:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AEA66C4332F for ; Tue, 31 Oct 2023 13:58:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEh-0001Kh-LU; Tue, 31 Oct 2023 09:56:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000X0-6e; Tue, 31 Oct 2023 09:56:15 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qC-ST; Tue, 31 Oct 2023 09:56:11 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id DA3D942A3D; Tue, 31 Oct 2023 14:55:51 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 07/10] qapi/block-core: turn BlockJobInfo into a union Date: Tue, 31 Oct 2023 14:54:28 +0100 Message-Id: <20231031135431.393137-8-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In preparation to additionally return job-type-specific information. Signed-off-by: Fiona Ebner Reviewed-by: Vladimir Sementsov-Ogievskiy --- No changes in v4. qapi/block-core.json | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 9d03210664..dca0e94bb0 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1395,13 +1395,15 @@ # # Since: 1.1 ## -{ 'struct': 'BlockJobInfo', - 'data': {'type': 'JobType', 'device': 'str', 'len': 'int', +{ 'union': 'BlockJobInfo', + 'base': {'type': 'JobType', 'device': 'str', 'len': 'int', 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', 'status': 'JobStatus', 'auto-finalize': 'bool', 'auto-dismiss': 'bool', - '*error': 'str' } } + '*error': 'str' }, + 'discriminator': 'type', + 'data': {} } ## # @query-block-jobs: From patchwork Tue Oct 31 13:54:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4EC54C00142 for ; Tue, 31 Oct 2023 13:57:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEb-0000xS-0p; Tue, 31 Oct 2023 09:56:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000X2-6y; Tue, 31 Oct 2023 09:56:15 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qB-Ic; Tue, 31 Oct 2023 09:56:11 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id BF0A142A62; Tue, 31 Oct 2023 14:55:51 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 08/10] blockjob: query driver-specific info via a new 'query' driver method Date: Tue, 31 Oct 2023 14:54:29 +0100 Message-Id: <20231031135431.393137-9-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Fiona Ebner --- No changes in v4. blockjob.c | 6 ++++++ include/block/blockjob_int.h | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/blockjob.c b/blockjob.c index 9665b02627..41719dcf7d 100644 --- a/blockjob.c +++ b/blockjob.c @@ -376,6 +376,7 @@ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) { BlockJobInfo *info; uint64_t progress_current, progress_total; + const BlockJobDriver *drv = block_job_driver(job); GLOBAL_STATE_CODE(); @@ -405,6 +406,11 @@ BlockJobInfo *block_job_query_locked(BlockJob *job, Error **errp) g_strdup(error_get_pretty(job->job.err)) : g_strdup(strerror(-job->job.ret)); } + if (drv->query) { + job_unlock(); + drv->query(job, info); + job_lock(); + } return info; } diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h index a4656d4cb5..18ee6f7bf0 100644 --- a/include/block/blockjob_int.h +++ b/include/block/blockjob_int.h @@ -74,6 +74,11 @@ struct BlockJobDriver { * Note that this can already be called before the job coroutine is running. */ void (*change)(BlockJob *job, BlockJobChangeOptions *opts, Error **errp); + + /* + * Query information specific to this kind of block job. + */ + void (*query)(BlockJob *job, BlockJobInfo *info); }; /* From patchwork Tue Oct 31 13:54:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 221F4C4332F for ; Tue, 31 Oct 2023 13:58:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEg-0001K7-W0; Tue, 31 Oct 2023 09:56:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000XH-RC; Tue, 31 Oct 2023 09:56:16 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qN-TS; Tue, 31 Oct 2023 09:56:12 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id ADC5442A76; Tue, 31 Oct 2023 14:55:53 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 09/10] mirror: return mirror-specific information upon query Date: Tue, 31 Oct 2023 14:54:30 +0100 Message-Id: <20231031135431.393137-10-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org To start out, only actively-synced is returned. For example, this is useful for jobs that started out in background mode and switched to active mode. Once actively-synced is true, it's clear that the mode switch has been completed. Note that completion of the switch might happen much earlier, e.g. if the switch happens before the job is ready, once all background operations have finished. It's assumed that whether the disks are actively-synced or not is more interesting than whether the mode switch completed. That information can still be added if required in the future. In presence of an iothread, the actively_synced member is now shared between the iothread and the main thread, so turn accesses to it atomic. Requires to adapt the output for iotest 109. Signed-off-by: Fiona Ebner --- Changes in v4: * mention that actively_synced should be accessed with atomics above field declaration block/mirror.c | 31 +++++++++++++++++++++++-------- qapi/block-core.json | 16 +++++++++++++++- tests/qemu-iotests/109.out | 24 ++++++++++++------------ 3 files changed, 50 insertions(+), 21 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 4016d89253..c839542774 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -61,8 +61,12 @@ typedef struct MirrorBlockJob { */ MirrorCopyMode copy_mode; BlockdevOnError on_source_error, on_target_error; - /* Set when the target is synced (dirty bitmap is clean, nothing - * in flight) and the job is running in active mode */ + /* + * To be accessed with atomics. + * + * Set when the target is synced (dirty bitmap is clean, nothing in flight) + * and the job is running in active mode. + */ bool actively_synced; bool should_complete; int64_t granularity; @@ -126,7 +130,7 @@ typedef enum MirrorMethod { static BlockErrorAction mirror_error_action(MirrorBlockJob *s, bool read, int error) { - s->actively_synced = false; + qatomic_set(&s->actively_synced, false); if (read) { return block_job_error_action(&s->common, s->on_source_error, true, error); @@ -966,7 +970,7 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) if (s->bdev_length == 0) { /* Transition to the READY state and wait for complete. */ job_transition_to_ready(&s->common.job); - s->actively_synced = true; + qatomic_set(&s->actively_synced, true); while (!job_cancel_requested(&s->common.job) && !s->should_complete) { job_yield(&s->common.job); } @@ -1080,7 +1084,7 @@ static int coroutine_fn mirror_run(Job *job, Error **errp) job_transition_to_ready(&s->common.job); } if (qatomic_read(&s->copy_mode) != MIRROR_COPY_MODE_BACKGROUND) { - s->actively_synced = true; + qatomic_set(&s->actively_synced, true); } should_complete = s->should_complete || @@ -1283,6 +1287,15 @@ static void mirror_change(BlockJob *job, BlockJobChangeOptions *opts, } } +static void mirror_query(BlockJob *job, BlockJobInfo *info) +{ + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); + + info->u.mirror = (BlockJobInfoMirror) { + .actively_synced = qatomic_read(&s->actively_synced), + }; +} + static const BlockJobDriver mirror_job_driver = { .job_driver = { .instance_size = sizeof(MirrorBlockJob), @@ -1298,6 +1311,7 @@ static const BlockJobDriver mirror_job_driver = { }, .drained_poll = mirror_drained_poll, .change = mirror_change, + .query = mirror_query, }; static const BlockJobDriver commit_active_job_driver = { @@ -1416,7 +1430,7 @@ do_sync_target_write(MirrorBlockJob *job, MirrorMethod method, bitmap_end = QEMU_ALIGN_UP(offset + bytes, job->granularity); bdrv_set_dirty_bitmap(job->dirty_bitmap, bitmap_offset, bitmap_end - bitmap_offset); - job->actively_synced = false; + qatomic_set(&job->actively_synced, false); action = mirror_error_action(job, false, -ret); if (action == BLOCK_ERROR_ACTION_REPORT) { @@ -1475,7 +1489,8 @@ static void coroutine_fn GRAPH_RDLOCK active_write_settle(MirrorOp *op) uint64_t end_chunk = DIV_ROUND_UP(op->offset + op->bytes, op->s->granularity); - if (!--op->s->in_active_write_counter && op->s->actively_synced) { + if (!--op->s->in_active_write_counter && + qatomic_read(&op->s->actively_synced)) { BdrvChild *source = op->s->mirror_top_bs->backing; if (QLIST_FIRST(&source->bs->parents) == source && @@ -1539,7 +1554,7 @@ bdrv_mirror_top_do_write(BlockDriverState *bs, MirrorMethod method, } if (!copy_to_target && s->job && s->job->dirty_bitmap) { - s->job->actively_synced = false; + qatomic_set(&s->job->actively_synced, false); bdrv_set_dirty_bitmap(s->job->dirty_bitmap, offset, bytes); } diff --git a/qapi/block-core.json b/qapi/block-core.json index dca0e94bb0..99961256f2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1352,6 +1352,20 @@ { 'enum': 'MirrorCopyMode', 'data': ['background', 'write-blocking'] } +## +# @BlockJobInfoMirror: +# +# Information specific to mirror block jobs. +# +# @actively-synced: Whether the source is actively synced to the +# target, i.e. same data and new writes are done synchronously to +# both. +# +# Since 8.2 +## +{ 'struct': 'BlockJobInfoMirror', + 'data': { 'actively-synced': 'bool' } } + ## # @BlockJobInfo: # @@ -1403,7 +1417,7 @@ 'auto-finalize': 'bool', 'auto-dismiss': 'bool', '*error': 'str' }, 'discriminator': 'type', - 'data': {} } + 'data': { 'mirror': 'BlockJobInfoMirror' } } ## # @query-block-jobs: diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out index 2611d6a40f..965c9a6a0a 100644 --- a/tests/qemu-iotests/109.out +++ b/tests/qemu-iotests/109.out @@ -38,7 +38,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -90,7 +90,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 197120, "offset": 197120, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 197120, "offset": 197120, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -142,7 +142,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -194,7 +194,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -246,7 +246,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 65536, "offset": 65536, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 65536, "offset": 65536, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -298,7 +298,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -349,7 +349,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -400,7 +400,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 31457280, "offset": 31457280, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 31457280, "offset": 31457280, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -451,7 +451,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -502,7 +502,7 @@ read 512/512 bytes at offset 0 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2048, "offset": 2048, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2048, "offset": 2048, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -533,7 +533,7 @@ WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} @@ -557,7 +557,7 @@ Images are identical. {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} {"execute":"query-block-jobs"} -{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} +{"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror", "actively-synced": false}]} {"execute":"quit"} {"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} From patchwork Tue Oct 31 13:54:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fiona Ebner X-Patchwork-Id: 13441557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72478C4332F for ; Tue, 31 Oct 2023 13:57:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qxpEf-0001I3-8G; Tue, 31 Oct 2023 09:56:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE5-0000XC-Kd; Tue, 31 Oct 2023 09:56:16 -0400 Received: from proxmox-new.maurer-it.com ([94.136.29.106]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qxpE0-0000qO-Tr; Tue, 31 Oct 2023 09:56:11 -0400 Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 72CFE42A5D; Tue, 31 Oct 2023 14:55:52 +0100 (CET) From: Fiona Ebner To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, armbru@redhat.com, eblake@redhat.com, hreitz@redhat.com, kwolf@redhat.com, vsementsov@yandex-team.ru, jsnow@redhat.com, den@virtuozzo.com, t.lamprecht@proxmox.com, alexander.ivanov@virtuozzo.com Subject: [PATCH v4 10/10] iotests: add test for changing mirror's copy_mode Date: Tue, 31 Oct 2023 14:54:31 +0100 Message-Id: <20231031135431.393137-11-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231031135431.393137-1-f.ebner@proxmox.com> References: <20231031135431.393137-1-f.ebner@proxmox.com> MIME-Version: 1.0 Received-SPF: pass client-ip=94.136.29.106; envelope-from=f.ebner@proxmox.com; helo=proxmox-new.maurer-it.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org One part of the test is using a throttled source to ensure that there are no obvious issues when changing the copy_mode while there are ongoing requests (source and target images are compared at the very end). The other part of the test is using a throttled target to ensure that the change to active mode actually happened. This is done by hitting the throttling limit, issuing a synchronous write and then immediately verifying the target side. QSD is used, because otherwise, a synchronous write would hang there. Signed-off-by: Fiona Ebner --- New in v4. .../tests/mirror-change-copy-mode | 192 ++++++++++++++++++ .../tests/mirror-change-copy-mode.out | 5 + 2 files changed, 197 insertions(+) create mode 100755 tests/qemu-iotests/tests/mirror-change-copy-mode create mode 100644 tests/qemu-iotests/tests/mirror-change-copy-mode.out diff --git a/tests/qemu-iotests/tests/mirror-change-copy-mode b/tests/qemu-iotests/tests/mirror-change-copy-mode new file mode 100755 index 0000000000..e9fef8c5e9 --- /dev/null +++ b/tests/qemu-iotests/tests/mirror-change-copy-mode @@ -0,0 +1,192 @@ +#!/usr/bin/env python3 +# group: rw +# +# Test for changing mirror copy mode from background to active +# +# Copyright (C) 2023 Proxmox Server Solutions GmbH +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import time + +import iotests +from iotests import qemu_img, qemu_io, QemuStorageDaemon + +iops_target = 8 +iops_source = iops_target * 2 +image_size = 1 * 1024 * 1024 +source_img = os.path.join(iotests.test_dir, 'source.' + iotests.imgfmt) +target_img = os.path.join(iotests.test_dir, 'target.' + iotests.imgfmt) +nbd_sock = os.path.join(iotests.sock_dir, 'nbd.sock') + +class TestMirrorChangeCopyMode(iotests.QMPTestCase): + + def setUp(self): + qemu_img('create', '-f', iotests.imgfmt, source_img, str(image_size)) + qemu_img('create', '-f', iotests.imgfmt, target_img, str(image_size)) + + self.qsd = QemuStorageDaemon('--nbd-server', + f'addr.type=unix,addr.path={nbd_sock}', + qmp=True) + + self.qsd.cmd('object-add', { + 'qom-type': 'throttle-group', + 'id': 'thrgr-target', + 'limits': { + 'iops-write': iops_target, + 'iops-write-max': iops_target + } + }) + + self.qsd.cmd('blockdev-add', { + 'node-name': 'target', + 'driver': 'throttle', + 'throttle-group': 'thrgr-target', + 'file': { + 'driver': iotests.imgfmt, + 'file': { + 'driver': 'file', + 'filename': target_img + } + } + }) + + self.qsd.cmd('block-export-add', { + 'id': 'exp0', + 'type': 'nbd', + 'node-name': 'target', + 'writable': True + }) + + self.vm = iotests.VM() + self.vm.add_args('-drive', + f'file={source_img},if=none,format={iotests.imgfmt},' + + f'iops_wr={iops_source},' + + f'iops_wr_max={iops_source},' + + 'id=source') + self.vm.launch() + + self.vm.cmd('blockdev-add', { + 'node-name': 'target', + 'driver': 'nbd', + 'export': 'target', + 'server': { + 'type': 'unix', + 'path': nbd_sock + } + }) + + + def tearDown(self): + self.vm.shutdown() + self.qsd.stop() + self.check_qemu_io_errors() + self.check_images_identical() + os.remove(source_img) + os.remove(target_img) + + # Once the VM is shut down we can parse the log and see if qemu-io ran + # without errors. + def check_qemu_io_errors(self): + self.assertFalse(self.vm.is_running()) + log = self.vm.get_log() + for line in log.split("\n"): + assert not line.startswith("Pattern verification failed") + + def check_images_identical(self): + qemu_img('compare', '-f', iotests.imgfmt, source_img, target_img) + + def start_mirror(self): + self.vm.cmd('blockdev-mirror', + job_id='mirror', + device='source', + target='target', + filter_node_name='mirror-top', + sync='full', + copy_mode='background') + + def test_background_to_active(self): + self.vm.hmp_qemu_io('source', f'write 0 {image_size}') + self.vm.hmp_qemu_io('target', f'write 0 {image_size}') + + self.start_mirror() + + result = self.vm.cmd('query-block-jobs') + assert not result[0]['actively-synced'] + + self.vm.event_wait('BLOCK_JOB_READY') + + result = self.vm.cmd('query-block-jobs') + assert not result[0]['actively-synced'] + + # Start some background requests. + reqs = 4 * iops_source + req_size = image_size // reqs + for i in range(0, reqs): + req = f'aio_write -P 7 {req_size * i} {req_size}' + self.vm.hmp_qemu_io('source', req) + + # Wait for the first few requests. + time.sleep(1) + self.vm.qtest(f'clock_step {1 * 1000 * 1000 * 1000}') + + result = self.vm.cmd('query-block-jobs') + # There should've been new requests. + assert result[0]['len'] > image_size + # To verify later that not all requests were completed at this point. + len_before_change = result[0]['len'] + + # Change the copy mode while requests are happening. + self.vm.cmd('block-job-change', + id='mirror', + type='mirror', + copy_mode='write-blocking') + + # Wait until image is actively synced. + while True: + time.sleep(0.1) + self.vm.qtest(f'clock_step {100 * 1000 * 1000}') + result = self.vm.cmd('query-block-jobs') + if result[0]['actively-synced']: + break + + # Because of throttling, not all request should've been completed above. + result = self.vm.cmd('query-block-jobs') + assert result[0]['len'] > len_before_change + + # Issue enough requests for a few seconds only touching the first half + # of the image. + reqs = 4 * iops_target + req_size = image_size // 2 // reqs + for i in range(0, reqs): + req = f'aio_write -P 19 {req_size * i} {req_size}' + self.vm.hmp_qemu_io('source', req) + + # Now issue a synchronous write in the second half of the image and + # immediately verify that it was written to the target too. This would + # fail without switching the copy mode. Note that this only produces a + # log line and the actual checking happens during tearDown(). + req_args = f'-P 37 {3 * (image_size // 4)} {req_size}' + self.vm.hmp_qemu_io('source', f'write {req_args}') + self.vm.hmp_qemu_io('target', f'read {req_args}') + + self.vm.cmd('block-job-cancel', device='mirror') + while len(self.vm.cmd('query-block-jobs')) > 0: + time.sleep(0.1) + +if __name__ == '__main__': + iotests.main(supported_fmts=['qcow2', 'raw'], + supported_protocols=['file']) diff --git a/tests/qemu-iotests/tests/mirror-change-copy-mode.out b/tests/qemu-iotests/tests/mirror-change-copy-mode.out new file mode 100644 index 0000000000..ae1213e6f8 --- /dev/null +++ b/tests/qemu-iotests/tests/mirror-change-copy-mode.out @@ -0,0 +1,5 @@ +. +---------------------------------------------------------------------- +Ran 1 tests + +OK