From patchwork Wed Sep 6 15:49:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DDA3EE14AC for ; Wed, 6 Sep 2023 15:51:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumw-0007Cw-E9; Wed, 06 Sep 2023 11:49:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumu-0007BG-9w; Wed, 06 Sep 2023 11:49:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdump-0002xo-8j; Wed, 06 Sep 2023 11:49:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujh-007L3X-0f; Wed, 06 Sep 2023 17:49:32 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, "Denis V. Lunev" , Vladimir Sementsov-Ogievskiy , Stefan Hajnoczi Subject: [PULL 01/18] MAINTAINERS: add tree to keep parallels format driver changes Date: Wed, 6 Sep 2023 17:49:34 +0200 Message-Id: <20230906154942.656537-2-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Driver changes are driving by me for now. At least we need to get functionally complete check and repair procedure for now. Signed-off-by: Denis V. Lunev Reviewed-by: Vladimir Sementsov-Ogievskiy CC: Stefan Hajnoczi --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 3b29568ed4..b471973e1e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3693,6 +3693,7 @@ S: Supported F: block/parallels.c F: block/parallels-ext.c F: docs/interop/parallels.txt +T: git https://src.openvz.org/scm/~den/qemu.git parallels qed M: Stefan Hajnoczi From patchwork Wed Sep 6 15:49:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 06B18EE14A5 for ; Wed, 6 Sep 2023 15:52:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumx-0007E0-9g; Wed, 06 Sep 2023 11:49:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumu-0007BI-AZ; Wed, 06 Sep 2023 11:49:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdump-0002xu-9G; Wed, 06 Sep 2023 11:49:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujh-007L3X-26; Wed, 06 Sep 2023 17:49:32 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 02/18] parallels: Fix comments formatting inside parallels driver Date: Wed, 6 Sep 2023 17:49:35 +0200 Message-Id: <20230906154942.656537-3-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov This patch is technically necessary as git patch rendering could result in moving some code from one place to the another and that hits checkpatch.pl warning. This problem specifically happens within next series. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 18e34aef28..c7b2ed5a54 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -188,7 +188,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, idx = sector_num / s->tracks; to_allocate = DIV_ROUND_UP(sector_num + *pnum, s->tracks) - idx; - /* This function is called only by parallels_co_writev(), which will never + /* + * This function is called only by parallels_co_writev(), which will never * pass a sector_num at or beyond the end of the image (because the block * layer never passes such a sector_num to that function). Therefore, idx * is always below s->bat_size. @@ -196,7 +197,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, * exceed the image end. Therefore, idx + to_allocate cannot exceed * s->bat_size. * Note that s->bat_size is an unsigned int, therefore idx + to_allocate - * will always fit into a uint32_t. */ + * will always fit into a uint32_t. + */ assert(idx < s->bat_size && idx + to_allocate <= s->bat_size); space = to_allocate * s->tracks; @@ -230,13 +232,15 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, } } - /* Try to read from backing to fill empty clusters + /* + * Try to read from backing to fill empty clusters * FIXME: 1. previous write_zeroes may be redundant * 2. most of data we read from backing will be rewritten by * parallels_co_writev. On aligned-to-cluster write we do not need * this read at all. * 3. it would be good to combine write of data from backing and new - * data into one write call */ + * data into one write call. + */ if (bs->backing) { int64_t nb_cow_sectors = to_allocate * s->tracks; int64_t nb_cow_bytes = nb_cow_sectors << BDRV_SECTOR_BITS; @@ -864,8 +868,10 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, s->data_end = ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE); } if (s->data_end < s->header_size) { - /* there is not enough unused space to fit to block align between BAT - and actual data. We can't avoid read-modify-write... */ + /* + * There is not enough unused space to fit to block align between BAT + * and actual data. We can't avoid read-modify-write... + */ s->header_size = size; } From patchwork Wed Sep 6 15:49:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E7B4EE14A0 for ; Wed, 6 Sep 2023 15:52:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumx-0007Dx-2T; Wed, 06 Sep 2023 11:49:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumu-0007BF-7n; Wed, 06 Sep 2023 11:49:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdump-0002xw-AQ; Wed, 06 Sep 2023 11:49:51 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduji-007L3X-0H; Wed, 06 Sep 2023 17:49:33 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" , Hanna Czenczek Subject: [PULL 03/18] parallels: Incorrect data end calculation in parallels_open() Date: Wed, 6 Sep 2023 17:49:36 +0200 Message-Id: <20230906154942.656537-4-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov The BDRVParallelsState structure contains data_end field that is measured in sectors. In parallels_open() initially this field is set by data_off field from parallels image header. According to the parallels format documentation, data_off field contains an offset, in sectors, from the start of the file to the start of the data area. For "WithoutFreeSpace" images: if data_off is zero, the offset is calculated as the end of the BAT table plus some padding to ensure sector size alignment. The parallels_open() function has code for handling zero value in data_off, but in the result data_end contains the offset in bytes. Replace the alignment to sector size by division by sector size and fix the comparision with s->header_size. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Reviewed-by: Hanna Czenczek Signed-off-by: Denis V. Lunev --- block/parallels.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index c7b2ed5a54..3c0dca3dbf 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -865,9 +865,9 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, } s->data_end = le32_to_cpu(ph.data_off); if (s->data_end == 0) { - s->data_end = ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE); + s->data_end = DIV_ROUND_UP(size, BDRV_SECTOR_SIZE); } - if (s->data_end < s->header_size) { + if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { /* * There is not enough unused space to fit to block align between BAT * and actual data. We can't avoid read-modify-write... From patchwork Wed Sep 6 15:49:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9A81EE14A5 for ; Wed, 6 Sep 2023 15:50:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumw-0007CX-0U; Wed, 06 Sep 2023 11:49:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdums-0007Ag-8B; Wed, 06 Sep 2023 11:49:50 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdump-0002xv-8m; Wed, 06 Sep 2023 11:49:49 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduji-007L3X-1Y; Wed, 06 Sep 2023 17:49:33 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 04/18] parallels: Check if data_end greater than the file size Date: Wed, 6 Sep 2023 17:49:37 +0200 Message-Id: <20230906154942.656537-5-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Initially data_end is set to the data_off image header field and must not be greater than the file size. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 3c0dca3dbf..6a3d41373a 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -874,6 +874,11 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, */ s->header_size = size; } + if (s->data_end > file_nb_sectors) { + error_setg(errp, "Invalid image: incorrect data_off field"); + ret = -EINVAL; + goto fail; + } ret = bdrv_pread(bs->file, 0, s->header_size, s->header, 0); if (ret < 0) { From patchwork Wed Sep 6 15:49:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8DD94EE14A5 for ; Wed, 6 Sep 2023 15:51:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumv-0007CT-C0; Wed, 06 Sep 2023 11:49:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumu-0007BH-9s; Wed, 06 Sep 2023 11:49:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdump-0002y3-Gf; Wed, 06 Sep 2023 11:49:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduji-007L3X-2x; Wed, 06 Sep 2023 17:49:33 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 05/18] parallels: Add "explicit" argument to parallels_check_leak() Date: Wed, 6 Sep 2023 17:49:38 +0200 Message-Id: <20230906154942.656537-6-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov In the on of the next patches we need to repair leaks without changing leaks and leaks_fixed info in res. Also we don't want to print any warning about leaks. Add "explicit" argument to skip info changing if the argument is false. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 6a3d41373a..8bb5d115fc 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -488,7 +488,7 @@ parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res, static int coroutine_fn GRAPH_RDLOCK parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, - BdrvCheckMode fix) + BdrvCheckMode fix, bool explicit) { BDRVParallelsState *s = bs->opaque; int64_t size; @@ -503,10 +503,13 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, if (size > res->image_end_offset) { int64_t count; count = DIV_ROUND_UP(size - res->image_end_offset, s->cluster_size); - fprintf(stderr, "%s space leaked at the end of the image %" PRId64 "\n", - fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR", - size - res->image_end_offset); - res->leaks += count; + if (explicit) { + fprintf(stderr, + "%s space leaked at the end of the image %" PRId64 "\n", + fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR", + size - res->image_end_offset); + res->leaks += count; + } if (fix & BDRV_FIX_LEAKS) { Error *local_err = NULL; @@ -521,7 +524,9 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, res->check_errors++; return ret; } - res->leaks_fixed += count; + if (explicit) { + res->leaks_fixed += count; + } } } @@ -574,7 +579,7 @@ parallels_co_check(BlockDriverState *bs, BdrvCheckResult *res, return ret; } - ret = parallels_check_leak(bs, res, fix); + ret = parallels_check_leak(bs, res, fix, true); if (ret < 0) { return ret; } From patchwork Wed Sep 6 15:49:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6ED18EE14A0 for ; Wed, 6 Sep 2023 15:51:53 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumx-0007E4-SA; Wed, 06 Sep 2023 11:49:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumu-0007Bl-M8; Wed, 06 Sep 2023 11:49:52 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumq-0002yB-E7; Wed, 06 Sep 2023 11:49:52 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujj-007L3X-0y; Wed, 06 Sep 2023 17:49:34 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 06/18] parallels: Add data_start field to BDRVParallelsState Date: Wed, 6 Sep 2023 17:49:39 +0200 Message-Id: <20230906154942.656537-7-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov In the next patch we will need the offset of the data area for host cluster index calculation. Add this field and setting up code. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 7 ++++--- block/parallels.h | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 8bb5d115fc..f7b44cb433 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -868,10 +868,11 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, ret = -ENOMEM; goto fail; } - s->data_end = le32_to_cpu(ph.data_off); - if (s->data_end == 0) { - s->data_end = DIV_ROUND_UP(size, BDRV_SECTOR_SIZE); + s->data_start = le32_to_cpu(ph.data_off); + if (s->data_start == 0) { + s->data_start = DIV_ROUND_UP(size, BDRV_SECTOR_SIZE); } + s->data_end = s->data_start; if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { /* * There is not enough unused space to fit to block align between BAT diff --git a/block/parallels.h b/block/parallels.h index f22f43f988..4e53e9572d 100644 --- a/block/parallels.h +++ b/block/parallels.h @@ -75,6 +75,7 @@ typedef struct BDRVParallelsState { uint32_t *bat_bitmap; unsigned int bat_size; + int64_t data_start; int64_t data_end; uint64_t prealloc_size; ParallelsPreallocMode prealloc_mode; From patchwork Wed Sep 6 15:49:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375748 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2DBEBEE14AD for ; Wed, 6 Sep 2023 15:51:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumz-0007Fe-69; Wed, 06 Sep 2023 11:49:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumw-0007Co-CC; Wed, 06 Sep 2023 11:49:54 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumq-0002yJ-EE; Wed, 06 Sep 2023 11:49:54 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujj-007L3X-2D; Wed, 06 Sep 2023 17:49:34 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 07/18] parallels: Add checking and repairing duplicate offsets in BAT Date: Wed, 6 Sep 2023 17:49:40 +0200 Message-Id: <20230906154942.656537-8-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Cluster offsets must be unique among all the BAT entries. Find duplicate offsets in the BAT and fix it by copying the content of the relevant cluster to a newly allocated cluster and set the new cluster offset to the duplicated entry. Add host_cluster_index() helper to deduplicate the code. When new clusters are allocated, the file size increases by 128 Mb. Call parallels_check_leak() to fix this leak. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index f7b44cb433..a78238eadd 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -136,6 +136,12 @@ static int cluster_remainder(BDRVParallelsState *s, int64_t sector_num, return MIN(nb_sectors, ret); } +static uint32_t host_cluster_index(BDRVParallelsState *s, int64_t off) +{ + off -= s->data_start << BDRV_SECTOR_BITS; + return off / s->cluster_size; +} + static int64_t block_status(BDRVParallelsState *s, int64_t sector_num, int nb_sectors, int *pnum) { @@ -533,6 +539,139 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, return 0; } +static int coroutine_fn GRAPH_RDLOCK +parallels_check_duplicate(BlockDriverState *bs, BdrvCheckResult *res, + BdrvCheckMode fix) +{ + BDRVParallelsState *s = bs->opaque; + int64_t host_off, host_sector, guest_sector; + unsigned long *bitmap; + uint32_t i, bitmap_size, cluster_index, bat_entry; + int n, ret = 0; + uint64_t *buf = NULL; + bool fixed = false; + + /* + * Create a bitmap of used clusters. + * If a bit is set, there is a BAT entry pointing to this cluster. + * Loop through the BAT entries, check bits relevant to an entry offset. + * If bit is set, this entry is duplicated. Otherwise set the bit. + * + * We shouldn't worry about newly allocated clusters outside the image + * because they are created higher then any existing cluster pointed by + * a BAT entry. + */ + bitmap_size = host_cluster_index(s, res->image_end_offset); + if (bitmap_size == 0) { + return 0; + } + if (res->image_end_offset % s->cluster_size) { + /* A not aligned image end leads to a bitmap shorter by 1 */ + bitmap_size++; + } + + bitmap = bitmap_new(bitmap_size); + + buf = qemu_blockalign(bs, s->cluster_size); + + for (i = 0; i < s->bat_size; i++) { + host_off = bat2sect(s, i) << BDRV_SECTOR_BITS; + if (host_off == 0) { + continue; + } + + cluster_index = host_cluster_index(s, host_off); + assert(cluster_index < bitmap_size); + if (!test_bit(cluster_index, bitmap)) { + bitmap_set(bitmap, cluster_index, 1); + continue; + } + + /* this cluster duplicates another one */ + fprintf(stderr, "%s duplicate offset in BAT entry %u\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i); + + res->corruptions++; + + if (!(fix & BDRV_FIX_ERRORS)) { + continue; + } + + /* + * Reset the entry and allocate a new cluster + * for the relevant guest offset. In this way we let + * the lower layer to place the new cluster properly. + * Copy the original cluster to the allocated one. + * But before save the old offset value for repairing + * if we have an error. + */ + bat_entry = s->bat_bitmap[i]; + parallels_set_bat_entry(s, i, 0); + + ret = bdrv_co_pread(bs->file, host_off, s->cluster_size, buf, 0); + if (ret < 0) { + res->check_errors++; + goto out_repair_bat; + } + + guest_sector = (i * (int64_t)s->cluster_size) >> BDRV_SECTOR_BITS; + host_sector = allocate_clusters(bs, guest_sector, s->tracks, &n); + if (host_sector < 0) { + res->check_errors++; + goto out_repair_bat; + } + host_off = host_sector << BDRV_SECTOR_BITS; + + ret = bdrv_co_pwrite(bs->file, host_off, s->cluster_size, buf, 0); + if (ret < 0) { + res->check_errors++; + goto out_repair_bat; + } + + if (host_off + s->cluster_size > res->image_end_offset) { + res->image_end_offset = host_off + s->cluster_size; + } + + /* + * In the future allocate_cluster() will reuse holed offsets + * inside the image. Keep the used clusters bitmap content + * consistent for the new allocated clusters too. + * + * Note, clusters allocated outside the current image are not + * considered, and the bitmap size doesn't change. + */ + cluster_index = host_cluster_index(s, host_off); + if (cluster_index < bitmap_size) { + bitmap_set(bitmap, cluster_index, 1); + } + + fixed = true; + res->corruptions_fixed++; + + } + + if (fixed) { + /* + * When new clusters are allocated, the file size increases by + * 128 Mb. We need to truncate the file to the right size. Let + * the leak fix code make its job without res changing. + */ + ret = parallels_check_leak(bs, res, fix, false); + } + +out_free: + g_free(buf); + g_free(bitmap); + return ret; +/* + * We can get here only from places where index and old_offset have + * meaningful values. + */ +out_repair_bat: + s->bat_bitmap[i] = bat_entry; + goto out_free; +} + static void parallels_collect_statistics(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -584,6 +723,11 @@ parallels_co_check(BlockDriverState *bs, BdrvCheckResult *res, return ret; } + ret = parallels_check_duplicate(bs, res, fix); + if (ret < 0) { + return ret; + } + parallels_collect_statistics(bs, res, fix); } From patchwork Wed Sep 6 15:49:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 27564EE14A5 for ; Wed, 6 Sep 2023 15:52:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumz-0007Fy-RY; Wed, 06 Sep 2023 11:49:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumw-0007Cd-8Z; Wed, 06 Sep 2023 11:49:54 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumq-0002ya-K3; Wed, 06 Sep 2023 11:49:54 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujk-007L3X-0G; Wed, 06 Sep 2023 17:49:35 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 08/18] parallels: Image repairing in parallels_open() Date: Wed, 6 Sep 2023 17:49:41 +0200 Message-Id: <20230906154942.656537-9-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Repair an image at opening if the image is unclean or out-of-image corruption was detected. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 70 +++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index a78238eadd..5100c8f903 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -951,7 +951,7 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, BDRVParallelsState *s = bs->opaque; ParallelsHeader ph; int ret, size, i; - int64_t file_nb_sectors; + int64_t file_nb_sectors, sector; QemuOpts *opts = NULL; Error *local_err = NULL; char *buf; @@ -1024,11 +1024,6 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, */ s->header_size = size; } - if (s->data_end > file_nb_sectors) { - error_setg(errp, "Invalid image: incorrect data_off field"); - ret = -EINVAL; - goto fail; - } ret = bdrv_pread(bs->file, 0, s->header_size, s->header, 0); if (ret < 0) { @@ -1036,33 +1031,8 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, } s->bat_bitmap = (uint32_t *)(s->header + 1); - for (i = 0; i < s->bat_size; i++) { - int64_t off = bat2sect(s, i); - if (off >= file_nb_sectors) { - if (flags & BDRV_O_CHECK) { - continue; - } - error_setg(errp, "parallels: Offset %" PRIi64 " in BAT[%d] entry " - "is larger than file size (%" PRIi64 ")", - off << BDRV_SECTOR_BITS, i, - file_nb_sectors << BDRV_SECTOR_BITS); - ret = -EINVAL; - goto fail; - } - if (off >= s->data_end) { - s->data_end = off + s->tracks; - } - } - if (le32_to_cpu(ph.inuse) == HEADER_INUSE_MAGIC) { - /* Image was not closed correctly. The check is mandatory */ s->header_unclean = true; - if ((flags & BDRV_O_RDWR) && !(flags & BDRV_O_CHECK)) { - error_setg(errp, "parallels: Image was not closed correctly; " - "cannot be opened read/write"); - ret = -EACCES; - goto fail; - } } opts = qemu_opts_create(¶llels_runtime_opts, NULL, 0, errp); @@ -1123,10 +1093,40 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, bdrv_get_device_or_node_name(bs)); ret = migrate_add_blocker(s->migration_blocker, errp); if (ret < 0) { - error_free(s->migration_blocker); + error_setg(errp, "Migration blocker error"); goto fail; } qemu_co_mutex_init(&s->lock); + + for (i = 0; i < s->bat_size; i++) { + sector = bat2sect(s, i); + if (sector + s->tracks > s->data_end) { + s->data_end = sector + s->tracks; + } + } + + /* + * We don't repair the image here if it's opened for checks. Also we don't + * want to change inactive images and can't change readonly images. + */ + if ((flags & (BDRV_O_CHECK | BDRV_O_INACTIVE)) || !(flags & BDRV_O_RDWR)) { + return 0; + } + + /* + * Repair the image if it's dirty or + * out-of-image corruption was detected. + */ + if (s->data_end > file_nb_sectors || s->header_unclean) { + BdrvCheckResult res; + ret = bdrv_check(bs, &res, BDRV_FIX_ERRORS | BDRV_FIX_LEAKS); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not repair corrupted image"); + migrate_del_blocker(s->migration_blocker); + goto fail; + } + } + return 0; fail_format: @@ -1134,6 +1134,12 @@ fail_format: fail_options: ret = -EINVAL; fail: + /* + * "s" object was allocated by g_malloc0 so we can safely + * try to free its fields even they were not allocated. + */ + error_free(s->migration_blocker); + g_free(s->bat_dirty_bmap); qemu_vfree(s->header); return ret; } From patchwork Wed Sep 6 15:49:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 36179EE14A0 for ; Wed, 6 Sep 2023 15:51:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdumy-0007F9-Kg; Wed, 06 Sep 2023 11:49:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumw-0007CY-3q; Wed, 06 Sep 2023 11:49:54 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdumq-0002yf-Sl; Wed, 06 Sep 2023 11:49:53 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qdujk-007L3X-1V; Wed, 06 Sep 2023 17:49:35 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 09/18] parallels: Use bdrv_co_getlength() in parallels_check_outside_image() Date: Wed, 6 Sep 2023 17:49:42 +0200 Message-Id: <20230906154942.656537-10-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov bdrv_co_getlength() should be used in coroutine context. Replace bdrv_getlength() by bdrv_co_getlength() in parallels_check_outside_image(). Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/parallels.c b/block/parallels.c index 5100c8f903..3414807089 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -500,7 +500,7 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, int64_t size; int ret; - size = bdrv_getlength(bs->file->bs); + size = bdrv_co_getlength(bs->file->bs); if (size < 0) { res->check_errors++; return size; From patchwork Wed Sep 6 15:54:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFED2EE14AD for ; Wed, 6 Sep 2023 15:55:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurg-0006WX-Pk; Wed, 06 Sep 2023 11:54:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurT-0006Pb-LN; Wed, 06 Sep 2023 11:54:37 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003mj-28; Wed, 06 Sep 2023 11:54:35 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo3-007ME4-1p; Wed, 06 Sep 2023 17:54:02 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 10/18] parallels: Add data_off check Date: Wed, 6 Sep 2023 17:54:05 +0200 Message-Id: <20230906155413.656644-1-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906154942.656537-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov data_off field of the parallels image header can be corrupted. Check if this field greater than the header + BAT size and less than file size. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/block/parallels.c b/block/parallels.c index 3414807089..103acb7b40 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -450,6 +450,81 @@ static void parallels_check_unclean(BlockDriverState *bs, } } +/* + * Returns true if data_off is correct, otherwise false. In both cases + * correct_offset is set to the proper value. + */ +static bool parallels_test_data_off(BDRVParallelsState *s, + int64_t file_nb_sectors, + uint32_t *correct_offset) +{ + uint32_t data_off, min_off; + bool old_magic; + + /* + * There are two slightly different image formats: with "WithoutFreeSpace" + * or "WithouFreSpacExt" magic words. Call the first one as "old magic". + * In such images data_off field can be zero. In this case the offset is + * calculated as the end of BAT table plus some padding to ensure sector + * size alignment. + */ + old_magic = !memcmp(s->header->magic, HEADER_MAGIC, 16); + + min_off = DIV_ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE); + if (!old_magic) { + min_off = ROUND_UP(min_off, s->cluster_size / BDRV_SECTOR_SIZE); + } + + if (correct_offset) { + *correct_offset = min_off; + } + + data_off = le32_to_cpu(s->header->data_off); + if (data_off == 0 && old_magic) { + return true; + } + + if (data_off < min_off || data_off > file_nb_sectors) { + return false; + } + + if (correct_offset) { + *correct_offset = data_off; + } + + return true; +} + +static int coroutine_fn GRAPH_RDLOCK +parallels_check_data_off(BlockDriverState *bs, BdrvCheckResult *res, + BdrvCheckMode fix) +{ + BDRVParallelsState *s = bs->opaque; + int64_t file_size; + uint32_t data_off; + + file_size = bdrv_co_nb_sectors(bs->file->bs); + if (file_size < 0) { + res->check_errors++; + return file_size; + } + + if (parallels_test_data_off(s, file_size, &data_off)) { + return 0; + } + + res->corruptions++; + if (fix & BDRV_FIX_ERRORS) { + s->header->data_off = cpu_to_le32(data_off); + res->corruptions_fixed++; + } + + fprintf(stderr, "%s data_off field has incorrect value\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR"); + + return 0; +} + static int coroutine_fn GRAPH_RDLOCK parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -713,6 +788,11 @@ parallels_co_check(BlockDriverState *bs, BdrvCheckResult *res, WITH_QEMU_LOCK_GUARD(&s->lock) { parallels_check_unclean(bs, res, fix); + ret = parallels_check_data_off(bs, res, fix); + if (ret < 0) { + return ret; + } + ret = parallels_check_outside_image(bs, res, fix); if (ret < 0) { return ret; From patchwork Wed Sep 6 15:54:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D79CEE14AC for ; Wed, 6 Sep 2023 15:56:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurd-0006T2-BJ; Wed, 06 Sep 2023 11:54:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurS-0006PK-2j; Wed, 06 Sep 2023 11:54:35 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003ml-1j; Wed, 06 Sep 2023 11:54:33 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo3-007ME4-35; Wed, 06 Sep 2023 17:54:02 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 11/18] parallels: Add data_off repairing to parallels_open() Date: Wed, 6 Sep 2023 17:54:06 +0200 Message-Id: <20230906155413.656644-2-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Place data_start/data_end calculation after reading the image header to s->header. Set s->data_start to the offset calculated in parallels_test_data_off(). Call bdrv_check() if data_off is incorrect. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- block/parallels.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/block/parallels.c b/block/parallels.c index 103acb7b40..48c32d6821 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -1032,9 +1032,11 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, ParallelsHeader ph; int ret, size, i; int64_t file_nb_sectors, sector; + uint32_t data_start; QemuOpts *opts = NULL; Error *local_err = NULL; char *buf; + bool data_off_is_correct; ret = bdrv_open_file_child(NULL, options, "file", bs, errp); if (ret < 0) { @@ -1092,18 +1094,6 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, ret = -ENOMEM; goto fail; } - s->data_start = le32_to_cpu(ph.data_off); - if (s->data_start == 0) { - s->data_start = DIV_ROUND_UP(size, BDRV_SECTOR_SIZE); - } - s->data_end = s->data_start; - if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { - /* - * There is not enough unused space to fit to block align between BAT - * and actual data. We can't avoid read-modify-write... - */ - s->header_size = size; - } ret = bdrv_pread(bs->file, 0, s->header_size, s->header, 0); if (ret < 0) { @@ -1115,6 +1105,18 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, s->header_unclean = true; } + data_off_is_correct = parallels_test_data_off(s, file_nb_sectors, + &data_start); + s->data_start = data_start; + s->data_end = s->data_start; + if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { + /* + * There is not enough unused space to fit to block align between BAT + * and actual data. We can't avoid read-modify-write... + */ + s->header_size = size; + } + opts = qemu_opts_create(¶llels_runtime_opts, NULL, 0, errp); if (!opts) { goto fail_options; @@ -1197,7 +1199,8 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, * Repair the image if it's dirty or * out-of-image corruption was detected. */ - if (s->data_end > file_nb_sectors || s->header_unclean) { + if (s->data_end > file_nb_sectors || s->header_unclean + || !data_off_is_correct) { BdrvCheckResult res; ret = bdrv_check(bs, &res, BDRV_FIX_ERRORS | BDRV_FIX_LEAKS); if (ret < 0) { From patchwork Wed Sep 6 15:54:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DFECEE14A0 for ; Wed, 6 Sep 2023 15:55:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurg-0006Wj-Uo; Wed, 06 Sep 2023 11:54:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurW-0006Py-8Q; Wed, 06 Sep 2023 11:54:39 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurD-0003mn-3n; Wed, 06 Sep 2023 11:54:37 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo4-007ME4-18; Wed, 06 Sep 2023 17:54:03 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 12/18] iotests: Add out-of-image check test for parallels format Date: Wed, 6 Sep 2023 17:54:07 +0200 Message-Id: <20230906155413.656644-3-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Fill the image with a pattern to generate entries in the BAT, set the first BAT entry outside the image, try to read the corrupted image. At the image opening it should be repaired, check for zeroes in the first cluster. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/tests/parallels-checks | 71 +++++++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 12 ++++ 2 files changed, 83 insertions(+) create mode 100755 tests/qemu-iotests/tests/parallels-checks create mode 100644 tests/qemu-iotests/tests/parallels-checks.out diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests/tests/parallels-checks new file mode 100755 index 0000000000..055ce34766 --- /dev/null +++ b/tests/qemu-iotests/tests/parallels-checks @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# group: rw quick +# +# Test qemu-img check for parallels format +# +# Copyright (C) 2022 Virtuozzo International GmbH +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=alexander.ivanov@virtuozzo.com + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ../common.rc +. ../common.filter + +_supported_fmt parallels +_supported_proto file +_supported_os Linux + +SIZE=$((4 * 1024 * 1024)) +IMGFMT=parallels +CLUSTER_SIZE_OFFSET=28 +BAT_OFFSET=64 + +_make_test_img $SIZE + +CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4) +CLUSTER_SIZE=$((CLUSTER_SIZE * 512)) +LAST_CLUSTER_OFF=$((SIZE - CLUSTER_SIZE)) +LAST_CLUSTER=$((LAST_CLUSTER_OFF/CLUSTER_SIZE)) + +echo "== TEST OUT OF IMAGE CHECK ==" + +echo "== write pattern ==" +{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== corrupt image ==" +cluster=$(($LAST_CLUSTER + 2)) +poke_file "$TEST_IMG" "$BAT_OFFSET" "\x$cluster\x00\x00\x00" + +echo "== read corrupted image with repairing ==" +{ $QEMU_IO -c "read -P 0x00 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iotests/tests/parallels-checks.out new file mode 100644 index 0000000000..ea4dcef0a6 --- /dev/null +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -0,0 +1,12 @@ +QA output created by parallels-checks +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST OUT OF IMAGE CHECK == +== write pattern == +wrote 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== corrupt image == +== read corrupted image with repairing == +Repairing cluster 0 is outside image +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** done From patchwork Wed Sep 6 15:54:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F2D1FEE14A8 for ; Wed, 6 Sep 2023 15:56:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdure-0006UK-Fp; Wed, 06 Sep 2023 11:54:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurU-0006Ps-Ia; Wed, 06 Sep 2023 11:54:37 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003mk-8G; Wed, 06 Sep 2023 11:54:36 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo4-007ME4-2O; Wed, 06 Sep 2023 17:54:03 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 13/18] iotests: Add leak check test for parallels format Date: Wed, 6 Sep 2023 17:54:08 +0200 Message-Id: <20230906155413.656644-4-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Write a pattern to the last cluster, extend the image by 1 claster, repair and check that the last cluster still has the same pattern. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/tests/parallels-checks | 27 +++++++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 22 +++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests/tests/parallels-checks index 055ce34766..8be282fabe 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -65,6 +65,33 @@ poke_file "$TEST_IMG" "$BAT_OFFSET" "\x$cluster\x00\x00\x00" echo "== read corrupted image with repairing ==" { $QEMU_IO -c "read -P 0x00 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +# Clear image +_make_test_img $SIZE + +echo "== TEST LEAK CHECK ==" + +echo "== write pattern to last cluster ==" +echo "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" +{ $QEMU_IO -c "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== extend image by 1 cluster ==" +fallocate -xl $((file_size + CLUSTER_SIZE)) "$TEST_IMG" + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== repair image ==" +_check_test_img -r all + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== check last cluster ==" +{ $QEMU_IO -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iotests/tests/parallels-checks.out index ea4dcef0a6..f2cb6dde85 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -9,4 +9,26 @@ wrote 4194304/4194304 bytes at offset 0 Repairing cluster 0 is outside image read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST LEAK CHECK == +== write pattern to last cluster == +write -P 0x11 3145728 1048576 +wrote 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +file size: 2097152 +== extend image by 1 cluster == +file size: 3145728 +== repair image == +Repairing space leaked at the end of the image 1048576 +The following inconsistencies were found and repaired: + + 1 leaked clusters + 0 corruptions + +Double checking the fixed image now... +No errors were found on the image. +file size: 2097152 +== check last cluster == +read 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done From patchwork Wed Sep 6 15:54:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375760 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02E63EE14A0 for ; Wed, 6 Sep 2023 15:56:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurf-0006VR-40; Wed, 06 Sep 2023 11:54:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurV-0006Pw-L9; Wed, 06 Sep 2023 11:54:39 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurD-0003mm-4W; Wed, 06 Sep 2023 11:54:37 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo5-007ME4-0S; Wed, 06 Sep 2023 17:54:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 14/18] iotests: Add test for BAT entries duplication check Date: Wed, 6 Sep 2023 17:54:09 +0200 Message-Id: <20230906155413.656644-5-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Fill a parallels image with a pattern and write another pattern to the second cluster. Corrupt the image and check if the pattern changes. Repair the image and check the patterns on guest and host sides. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/tests/parallels-checks | 32 +++++++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 31 ++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests/tests/parallels-checks index 8be282fabe..8a63c3daf4 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -92,6 +92,38 @@ echo "file size: $file_size" echo "== check last cluster ==" { $QEMU_IO -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +# Clear image +_make_test_img $SIZE + +echo "== TEST DUPLICATION CHECK ==" + +echo "== write pattern to whole image ==" +{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== write another pattern to second cluster ==" +{ $QEMU_IO -c "write -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== corrupt image ==" +poke_file "$TEST_IMG" "$(($BAT_OFFSET + 4))" "\x01\x00\x00\x00" + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== repair image ==" +_check_test_img -r all + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== check first cluster on host ==" +printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` + +echo "== check second cluster on host ==" +printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iotests/tests/parallels-checks.out index f2cb6dde85..b747bba1f3 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -31,4 +31,35 @@ file size: 2097152 == check last cluster == read 1048576/1048576 bytes at offset 3145728 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST DUPLICATION CHECK == +== write pattern to whole image == +wrote 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== write another pattern to second cluster == +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== corrupt image == +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== repair image == +Repairing duplicate offset in BAT entry 1 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== check first cluster on host == +content: 0x11 +== check second cluster on host == +content: 0x11 *** done From patchwork Wed Sep 6 15:54:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1EFE9EE14AD for ; Wed, 6 Sep 2023 15:56:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurd-0006Qt-6p; Wed, 06 Sep 2023 11:54:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurS-0006PJ-26; Wed, 06 Sep 2023 11:54:35 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003mo-1O; Wed, 06 Sep 2023 11:54:33 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo5-007ME4-1i; Wed, 06 Sep 2023 17:54:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 15/18] iotests: Refactor tests of parallels images checks (131) Date: Wed, 6 Sep 2023 17:54:10 +0200 Message-Id: <20230906155413.656644-6-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Replace hardcoded numbers by variables. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/131 | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index a847692b4c..601546c84c 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -44,31 +44,34 @@ _supported_os Linux inuse_offset=$((0x2c)) size=$((64 * 1024 * 1024)) -CLUSTER_SIZE=64k +CLUSTER_SIZE=$((64 * 1024)) IMGFMT=parallels _make_test_img $size +CLUSTER_HALF_SIZE=$((CLUSTER_SIZE / 2)) +CLUSTER_DBL_SIZE=$((CLUSTER_SIZE * 2)) + echo == read empty image == -{ $QEMU_IO -c "read -P 0 32k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 $CLUSTER_HALF_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == write more than 1 block in a row == -{ $QEMU_IO -c "write -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "write -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read less than block == -{ $QEMU_IO -c "read -P 0x11 32k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read exactly 1 block == -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read more than 1 block == -{ $QEMU_IO -c "read -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == check that there is no trash after written == -{ $QEMU_IO -c "read -P 0 160k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 $((CLUSTER_HALF_SIZE + CLUSTER_DBL_SIZE)) $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == check that there is no trash before written == -{ $QEMU_IO -c "read -P 0 0 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo "== Corrupt image ==" poke_file "$TEST_IMG" "$inuse_offset" "\x59\x6e\x6f\x74" -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir _check_test_img _check_test_img -r all -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo "== allocate with backing ==" # Verify that allocating clusters works fine even when there is a backing image. @@ -83,7 +86,7 @@ TEST_IMG="$TEST_IMG.base" _make_test_img $size # Write some data to the base image (which would trigger an assertion failure if # interpreted as a QEMUIOVector) -$QEMU_IO -c 'write -P 42 0 64k' "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -P 42 0 $CLUSTER_SIZE" "$TEST_IMG.base" | _filter_qemu_io # Parallels does not seem to support storing a backing filename in the image # itself, so we need to build our backing chain on the command line @@ -99,8 +102,8 @@ QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \ QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \ $QEMU_IO --image-opts "$imgopts" \ -c 'read -P 1 0 64' \ - -c "read -P 42 64 $((64 * 1024 - 64))" \ - -c "read -P 0 64k $((size - 64 * 1024))" \ + -c "read -P 42 64 $((CLUSTER_SIZE - 64))" \ + -c "read -P 0 $CLUSTER_SIZE $((size - CLUSTER_SIZE))" \ | _filter_qemu_io # success, all done From patchwork Wed Sep 6 15:54:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2CB0EE14A8 for ; Wed, 6 Sep 2023 15:56:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdure-0006Uw-Ob; Wed, 06 Sep 2023 11:54:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurT-0006Pc-NB; Wed, 06 Sep 2023 11:54:37 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003nG-MW; Wed, 06 Sep 2023 11:54:35 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo5-007ME4-31; Wed, 06 Sep 2023 17:54:04 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 16/18] iotests: Fix cluster size in parallels images tests (131) Date: Wed, 6 Sep 2023 17:54:11 +0200 Message-Id: <20230906155413.656644-7-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov In this test cluster size is 64k, but modern tools generate images with cluster size 1M. Calculate cluster size using track field from image header. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/131 | 5 ++++- tests/qemu-iotests/131.out | 44 +++++++++++++++++++------------------- 2 files changed, 26 insertions(+), 23 deletions(-) diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index 601546c84c..72f6535581 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -44,10 +44,13 @@ _supported_os Linux inuse_offset=$((0x2c)) size=$((64 * 1024 * 1024)) -CLUSTER_SIZE=$((64 * 1024)) IMGFMT=parallels _make_test_img $size +# get cluster size in sectors from "tracks" header field +CLUSTER_SIZE_OFFSET=28 +CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4) +CLUSTER_SIZE=$((CLUSTER_SIZE * 512)) CLUSTER_HALF_SIZE=$((CLUSTER_SIZE / 2)) CLUSTER_DBL_SIZE=$((CLUSTER_SIZE * 2)) diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out index de5ef7a8f5..98017a067e 100644 --- a/tests/qemu-iotests/131.out +++ b/tests/qemu-iotests/131.out @@ -1,26 +1,26 @@ QA output created by 131 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 == read empty image == -read 65536/65536 bytes at offset 32768 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 524288 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == write more than 1 block in a row == -wrote 131072/131072 bytes at offset 32768 -128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2097152/2097152 bytes at offset 524288 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read less than block == -read 32768/32768 bytes at offset 32768 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 524288 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read exactly 1 block == -read 65536/65536 bytes at offset 65536 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read more than 1 block == -read 131072/131072 bytes at offset 32768 -128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2097152/2097152 bytes at offset 524288 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == check that there is no trash after written == -read 32768/32768 bytes at offset 163840 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 2621440 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == check that there is no trash before written == -read 32768/32768 bytes at offset 0 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == Corrupt image == qemu-io: can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write ERROR image was not closed correctly @@ -35,19 +35,19 @@ The following inconsistencies were found and repaired: Double checking the fixed image now... No errors were found on the image. -read 65536/65536 bytes at offset 65536 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == allocate with backing == Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 -wrote 65536/65536 bytes at offset 0 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 64/64 bytes at offset 0 64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 64/64 bytes at offset 0 64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 65472/65472 bytes at offset 64 -63.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 67043328/67043328 bytes at offset 65536 -63.938 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048512/1048512 bytes at offset 64 +1023.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 66060288/66060288 bytes at offset 1048576 +63 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done From patchwork Wed Sep 6 15:54:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7FDB3EE14A5 for ; Wed, 6 Sep 2023 15:55:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdure-0006U9-6b; Wed, 06 Sep 2023 11:54:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurU-0006Pr-Hu; Wed, 06 Sep 2023 11:54:37 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurC-0003nF-MM; Wed, 06 Sep 2023 11:54:36 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo6-007ME4-11; Wed, 06 Sep 2023 17:54:05 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 17/18] iotests: Fix test 131 after repair was added to parallels_open() Date: Wed, 6 Sep 2023 17:54:12 +0200 Message-Id: <20230906155413.656644-8-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Images repairing in parallels_open() was added, thus parallels tests fail. Access to an image leads to repairing the image. Further image check don't detect any corruption. Remove reads after image creation in test 131. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/131 | 6 ++---- tests/qemu-iotests/131.out | 15 ++------------- 2 files changed, 4 insertions(+), 17 deletions(-) diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index 72f6535581..304bbb3f61 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -69,11 +69,9 @@ echo == check that there is no trash after written == echo == check that there is no trash before written == { $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir -echo "== Corrupt image ==" +echo "== corrupt image ==" poke_file "$TEST_IMG" "$inuse_offset" "\x59\x6e\x6f\x74" -{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir -_check_test_img -_check_test_img -r all +echo "== read corrupted image with repairing ==" { $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo "== allocate with backing ==" diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out index 98017a067e..d2904578df 100644 --- a/tests/qemu-iotests/131.out +++ b/tests/qemu-iotests/131.out @@ -21,20 +21,9 @@ read 524288/524288 bytes at offset 2621440 == check that there is no trash before written == read 524288/524288 bytes at offset 0 512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -== Corrupt image == -qemu-io: can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write -ERROR image was not closed correctly - -1 errors were found on the image. -Data may be corrupted, or further writes to the image may corrupt it. +== corrupt image == +== read corrupted image with repairing == Repairing image was not closed correctly -The following inconsistencies were found and repaired: - - 0 leaked clusters - 1 corruptions - -Double checking the fixed image now... -No errors were found on the image. read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == allocate with backing == From patchwork Wed Sep 6 15:54:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Denis V. Lunev" X-Patchwork-Id: 13375756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12F0FEE14A5 for ; Wed, 6 Sep 2023 15:55:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdurh-0006X3-AQ; Wed, 06 Sep 2023 11:54:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurV-0006Pv-HG; Wed, 06 Sep 2023 11:54:39 -0400 Received: from relay.virtuozzo.com ([130.117.225.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdurD-0003nU-1p; Wed, 06 Sep 2023 11:54:37 -0400 Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=iris.sw.ru) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1qduo6-007ME4-2H; Wed, 06 Sep 2023 17:54:05 +0200 From: "Denis V. Lunev" To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, stefanha@gmail.com, Alexander Ivanov , "Denis V . Lunev" Subject: [PULL 18/18] iotests: Add test for data_off check Date: Wed, 6 Sep 2023 17:54:13 +0200 Message-Id: <20230906155413.656644-9-den@openvz.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230906155413.656644-1-den@openvz.org> References: <20230906154942.656537-1-den@openvz.org> <20230906155413.656644-1-den@openvz.org> MIME-Version: 1.0 Received-SPF: pass client-ip=130.117.225.111; envelope-from=den@openvz.org; helo=relay.virtuozzo.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Alexander Ivanov Write a pattern to the first cluster. Corrupt the data_off field and check if the field was repaired on image opening and the pattern has not changed. Signed-off-by: Alexander Ivanov Reviewed-by: Denis V. Lunev Signed-off-by: Denis V. Lunev --- tests/qemu-iotests/tests/parallels-checks | 15 +++++++++++++++ tests/qemu-iotests/tests/parallels-checks.out | 10 ++++++++++ 2 files changed, 25 insertions(+) diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests/tests/parallels-checks index 8a63c3daf4..a7a1b357b5 100755 --- a/tests/qemu-iotests/tests/parallels-checks +++ b/tests/qemu-iotests/tests/parallels-checks @@ -44,6 +44,7 @@ _supported_os Linux SIZE=$((4 * 1024 * 1024)) IMGFMT=parallels CLUSTER_SIZE_OFFSET=28 +DATA_OFF_OFFSET=48 BAT_OFFSET=64 _make_test_img $SIZE @@ -124,6 +125,20 @@ printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` echo "== check second cluster on host ==" printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` +# Clear image +_make_test_img $SIZE + +echo "== TEST DATA_OFF CHECK ==" + +echo "== write pattern to first cluster ==" +{ $QEMU_IO -c "write -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== spoil data_off field ==" +poke_file "$TEST_IMG" "$DATA_OFF_OFFSET" "\xff\xff\xff\xff" + +echo "== check first cluster ==" +{ $QEMU_IO -c "read -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + # success, all done echo "*** done" rm -f $seq.full diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iotests/tests/parallels-checks.out index b747bba1f3..98a3a7f55e 100644 --- a/tests/qemu-iotests/tests/parallels-checks.out +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -62,4 +62,14 @@ read 1048576/1048576 bytes at offset 1048576 content: 0x11 == check second cluster on host == content: 0x11 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST DATA_OFF CHECK == +== write pattern to first cluster == +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== spoil data_off field == +== check first cluster == +Repairing data_off field has incorrect value +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done