From patchwork Fri May 19 09:34:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Nefedov X-Patchwork-Id: 9736765 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9F8BF6034C for ; Fri, 19 May 2017 09:36:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EB50286BE for ; Fri, 19 May 2017 09:36:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93593288EC; Fri, 19 May 2017 09:36:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A1C4C286BE for ; Fri, 19 May 2017 09:36:47 +0000 (UTC) Received: from localhost ([::1]:57475 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dBeL0-0006ni-Lp for patchwork-qemu-devel@patchwork.kernel.org; Fri, 19 May 2017 05:36:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48248) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dBeJY-0006k3-RY for qemu-devel@nongnu.org; Fri, 19 May 2017 05:35:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dBeJU-0006mv-Qo for qemu-devel@nongnu.org; Fri, 19 May 2017 05:35:16 -0400 Received: from mail-db5eur01on0136.outbound.protection.outlook.com ([104.47.2.136]:35055 helo=EUR01-DB5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dBeJU-0006mU-CX for qemu-devel@nongnu.org; Fri, 19 May 2017 05:35:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=bC8GLk9iHz3NvYOELkXeccICXZIApFGL4favqxeaCDo=; b=hdBPTyF1mq9mHS9UcIX0U3uu8STkP76Fgh+npTcmI5BYVQPeYZY39MHRbe+zuaOV1piYTqCXiJ/8hZ2YmWhlAKEOy61yaBkWj3iWuUlm9ds01vdy/46f75HLx2XZv6/s4LlihInJyXXpYeCG/qATQP6HlAMA60/r7d20nHNYtbQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=anton.nefedov@virtuozzo.com; Received: from xantnef-ws.sw.ru (195.214.232.6) by AM5PR0801MB1988.eurprd08.prod.outlook.com (2603:10a6:203:4b::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1101.14; Fri, 19 May 2017 09:35:09 +0000 From: Anton Nefedov To: Date: Fri, 19 May 2017 12:34:31 +0300 Message-ID: <1495186480-114192-5-git-send-email-anton.nefedov@virtuozzo.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1495186480-114192-1-git-send-email-anton.nefedov@virtuozzo.com> References: <1495186480-114192-1-git-send-email-anton.nefedov@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: HE1PR09CA0073.eurprd09.prod.outlook.com (2603:10a6:7:3d::17) To AM5PR0801MB1988.eurprd08.prod.outlook.com (2603:10a6:203:4b::15) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR0801MB1988: X-MS-Office365-Filtering-Correlation-Id: a540aaff-feaa-4f9e-5a4a-08d49e9a57c3 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:AM5PR0801MB1988; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1988; 3:zDukCurYmPjBfm1rfu6wQjZflEmX8SxNJNSXdMpr+CdRpvmBS8unCnU3SSJRDazpufawgKF0tQfjpzM/RRwqRnskhq5K3kE0dcT4HlVzja64DcNIr4H7+OvRkygRm7ehBC37DrIfJSlfE7BkCTenTEiEtTbWByjUSWWPWHeNZ8x2fxliezbrVOLPSXYhOO6JX+kjWY11kT6vmQ0u4xPe1qcyOOsw08dl1fk0Hpg50rzQxAKcIJMl1zkML3Dey937iIsq6NFW5NI4VOatiqAqQNol7kOHMSxbciFHM/vBkxswsIjGkQrHVinNOCHdc8EDsFBL4vBBr9reujHf8goq4w==; 25:znbHmmUlGNMSisPgvXNq4C4frB3iu6HnZHURkdcUfyCpyE/2dHmrDEsiG0Vb0AQA2eBDaP63SBtNuIVWnexA6tNYkcXbeGVKjtPokzg6wu3gCrXlD+pLnhKr1qvfGBxwK9aEA60Sr0oyRPU67tAwco7wgCKmZM3dKjURiXfDhM58svZkg1gsyh6bNxg8uZUaOKqt3dxXbgQZN4jpXmiNv0U+eV/BGHOzRK9NjJQM1QwI8kv/tIQvN5KqdMvVT2ucYsaK4CqR2vMztJnQa8SD4JbBXlawaSvinDcVXqbnCHafc7MDYG9w835PaV6FE3hfuls6YivnHlsZnkEZoafbwemOiM5TE8RrgcB8RlItXuJOMBzo6W0IqQBlYQgUczw2Mf3urVt+wpWpeGz/3MOITSshwyBlbkWhDpQG+tJpOhyntzx69O/jan//gN1CpDQGlSbMJpkpCNIuxLbgM1r89mB9/8iYU9qOLDab7QubBXw= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1988; 31:zxERpZepR3D2y2qbVx1vdDMqC+vdFCMkKHjVH5+yINBxDFU9zkkAqT49aiKuWebClp2zBc/Qsu93ESDo5t21b27LMkjvga45TSx2rywFtTJzg7GMSM9JXUrW/xbny0lae2bUJjEhpRqwt4P1cJw1BV8eQuhOME/Pjy9oUGayjwL5unw+p4532FCgHVbzVl1xX64ZLwYAHqLkFbW3KG5NqPsm/7e/3+qTJeEPmSK30wg=; 20:Q6R0kV7OiqV23G/xiJQfbyVQA1rkZbpKdA3i5mlLXRoB0rh3OJUexLQQP6iPNgOyNsnoj50WQcrWiiMY1vtLgiId0k/R9ffWvnbD/31yqzu+Ucz8EzSMBHMX6H2/nOw4K4i+kuPAaZemLC66hKdG+U71av84lkyYKHsMeVq9P8xCy+fdyGJdJR+F/F0cmubdogm65qPEvCiPAmKmtELonwEqibCTVPeupEjN6USOsuKeDhGCBzffn3nHRT7RVrqt21kYlERFrEkQ5tpVM0v9x2zFik2Jvady/bvcAzUOy3XTNIVko4dMPgNu/AP0Xibg9k9yeD5Zhp+i2DZNV9TNPTUDBF7j25dVfUz0y5LvGtLzWRo5eMRQloPQTpS5nV/y0hr5dNrmZ7pTXsndsgml052HkMKzdZh2imUcZIu0oWs= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(10201501046)(6041248)(20161123555025)(20161123558100)(20161123562025)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148); SRVR:AM5PR0801MB1988; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0801MB1988; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1988; 4:6Af5Fv5Uh7oceZ4irgvjNL+MoZvFcDj0NE9xxutAtyr2Y+Xbs/iK26EYVGMvY6JdKXrANCBwj/AivKgYN7uJtVV7EBoNOMUEg+BDCbQz1mdR4/4SsBQSqVD3ocBsJTCRmyI5FR/FMQfncrnKDL3WZ0+GUn0duQhFTGzfx2rZcyMEaixRxDXkvoo20aM+SiFN5IM+jPCF4pGvUuFVr2gBdr95MG6tHO71D7KtbIS70RvKICyUBUziHdK2BhPWZhwQYZ6l3/OjgLvoJaQpDpYyEpqWjOrOx8pf/+ywEuloANmmQEK3sqrQts8CYXzPe7VF0nORv4v9fyyxFsbWAxhDaUn0BhcQubJn2sIQIhK9BipVAkz3RDtkku+dvlfFYmRNbCjS3PTCfA89wXX7Xq03k5S1xvN6T6htMmKTiedLZHyJc3ptnRgfoR03OgFdX0lUHmAgvibCxW1zxqueeemCvXHa6cpo9Z/4eCHjBM3NsUH1FYF30hnYEg6X6XQlYUhTQjgpwjf8Fd2NvzpahDn9iPbHRPtz/WDdocxx1DJYh3ysmizKuTovrFYtFuzvbu2KsXgaX+vANMEmzo6hjZr3KSqXb05RIupdfvtxZc+LDRiBIdHxC+PzGQYYUAUT51kKtJTnv9p1U11B8HV/82fzFCMdSvQTkOGDeZLxQzINx5VeNeKeEoQJN1rCB/NzRAoUqIl8krjKm2L1oxzNV7+XmLl5gWcrFPReYSnsisru8+NA6+3cJ4IoL+k8ffKyxVv5QtkeAZGaaDOE7y2Ls2gNQgisr+/hvlnD3+5L7m3fHlw= X-Forefront-PRVS: 031257FE13 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(979002)(6009001)(39450400003)(39840400002)(39410400002)(39400400002)(199003)(189002)(48376002)(8676002)(7736002)(81166006)(53416004)(305945005)(50226002)(33646002)(5003940100001)(42186005)(47776003)(6116002)(3846002)(4326008)(25786009)(66066001)(189998001)(50466002)(2351001)(2906002)(478600001)(6486002)(6506006)(50986999)(575784001)(6512007)(54906002)(36756003)(5660300001)(86362001)(53936002)(2950100002)(110136004)(6666003)(6916009)(69596002)(107886003)(38730400002)(76176999)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM5PR0801MB1988; H:xantnef-ws.sw.ru; FPR:; SPF:None; MLV:ovrnspm; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0801MB1988; 23:Zg8ECujzkxsYpRVYN1vyaiAsTm9tbFb+0v2SWnU?= =?us-ascii?Q?AtYBQtus0EYYFxY+l4c/Guh3YBWwdTFyU0mQ1ytKRqAKb41vJPvtRfpvyn5S?= =?us-ascii?Q?8CSe3rl/X9ZJEZ/jQFaEvgHdSfgf9oF3oGFaPBSEdZiLVqv35EM2VbeeeiYC?= =?us-ascii?Q?y/XlO5FDtpTURy0DCWxhsIMAneXrYhadQcsH8/MO9h0sg8c0G0yS97aI7PQo?= =?us-ascii?Q?jeqLzCXAbOEjyYn6+D+mnXFB0Acsg/gxdqZX1ujg7x25HvQGf7jFXq+ssFzD?= =?us-ascii?Q?V0nNkFQEf2EsVrx1CZQR3ywOweFTM3rEgzKWAPaakVpIR281djhCCN9lCQlV?= =?us-ascii?Q?a9aiiSZjOxt65XzvcdKjDfKvb5UXS/GTFhH08hTjHp+BgU6aqO2qyUWDZDVc?= =?us-ascii?Q?QsvRDSG5UyOp4WoA3hG7pqvrqxg5vo000L1877bqoZOuCdIVfN+IzlG7lkEX?= =?us-ascii?Q?HqPquJRPvHEyer0RPN4344GFQJuEBFoaKC0c9MeHX9QMpaMXqlkr7gVo+lKp?= =?us-ascii?Q?fkep3G+M6BO5faD63l1lVOOFmGnv6iVcTG2J6G3dbEQ/Hpbxen1y4G54RCMq?= =?us-ascii?Q?ConFKz6WrOc1AX9Qqmg3/fcdvjl+JqGnJTwFhfdLGOKwJfF0sN8StKPAvp8I?= =?us-ascii?Q?/cvDHvexbzmhT6BcDzNNa5hRuTcsW6T+saPovh/e1rvOcIYSNWEuP2dewMK6?= =?us-ascii?Q?WgEY9rZD+RpXY54dKjBKZqFR+B7sgu55wxk7oNrjZScXbspcrjT1n6rHAEG+?= =?us-ascii?Q?W0I66d5Et+y3ITUEGcmtIhilyVOH+taQoXU9Dhh/DgknVyRQpCGam3sXZdew?= =?us-ascii?Q?LdyyfSL1NdI8GKlJJTErEG7xhD7llVXA63mqDLFX4H+nrz0+D0LiZ7MLpGXQ?= =?us-ascii?Q?+bvj2dwSV9r3JmZk+cdUYVo1z83orDvjPeP1GHavsd0SgUUWLyoxJ8q2hcrn?= =?us-ascii?Q?26EDs93PmevN/1XoOSEBRSCJ7+eJnuD9VZrs+GprrwG41//pwSY8SAw3DoUg?= =?us-ascii?Q?wEU9eMTHdLwX/CtomfII5iSNmJhLGboes4Y/N0HKvsdebA06DXP2ZVXnYLRs?= =?us-ascii?Q?O6Qr93FKl7/vOntrxGuNsDKuNL9LMIxcNC/LLm/K2OkoL2eDsdcgtP4ZJ7Iv?= =?us-ascii?Q?geoU5hIgZ+kV8duwaBBbm2ETrGs+2lw/zdR9bt2CnV9IBRyfBbIiq8XQFXTC?= =?us-ascii?Q?wD3YXeQkyhjAfBQxJuycIAQ+4jTAGYemq+tQLpot+3mhbURgxqQzAMsN79Qr?= =?us-ascii?Q?P1PAEmBQGvxLGSKkQ3Xo=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1988; 6:dPKSu4pW+QjB3I92NO37mdK2heMrIETr49i77EGaGay0ZPb/cZOgvgeGQpRTtlQUa1uatv2EyyzOZeS6ieO7j9eiOgSrxfri0j4OsRYfnty4Y77YBiK+EIKv4HkIxArZpwYCXMbYuRRZYDwlsCKKzoFeJnjFbESZwe+65igwoieUJq6vmRxjrK9f+g3i3oGW6Fag1PefplGUdGxeN3ohvn0yA3MRJRXU9lCa65q0zAC6VUR1sY9dE1BJyd+0k0aPXOp4Ks28oo9IEAmYA85rK7GI5YIhcaOh5o+zgOrqtVnLuzph+KGVMGeJRmc+gcMABYu/U76ythNaN48dVjuQ04FSYNqtypNIaMH1fzm9w6iFyrVTLSMJIUb600irWEoPForx+7iU0bQ0I+GoRfz2kUGJClFIhPJMX7Y+Bdm4RhMZeOR6FzUqg5+fOKOuzDetc//d+ukSiJsx/Po33jt/YraG6vmYrPWXIJUpc/P45B1qRsvBjYRnfc4DzR0ZZNtsDH35j6kllflJMcfX96I6Hg==; 5:CvYdIf1nseyxhePP32SexqFEmxTMwqsrruRiRMl/wqgt9TfY0n0D5m1HuefpEgJZdsFw4l/bJKE98lQdYY/ISp7Jnd2we651m2blfAEF3dQwkBdmXcWdk1PHB2iQBoP/bor0erqlJVzbaNkCxHKGDw==; 24:lWSLG8+HaFrS7Aju1RfFjcdSmc8cleU3M7DKe9DwULipcUw+o0gUjKfYCa+C8PS3yLQZOfbpHczO978QYtEk4HztpB/L8yqG8tFlZy+FQ/U= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0801MB1988; 7:Tiv9MxEl0yseBWpAY1SZ+RY/68dl680YoUlBIWO3LR++/xWlTZxsw4zsGRC3kzitr1kNn8Q9MNtjrF844AjKswBMGex/qEY0nm/Xzr29HRsF+RQb2Aueuv03Vewfjuk0QIAyk7HwXnVSXx6sWflUcfus2lorj5pRxAp0Ty3iZZzKGgmqkc42voGrgCoFTuJieMzOmm3/J8BjHzJdPPGFB+SDUopKhLC0fZICRu0aEluitPfZbd5/AaxevAbWas8PGFsYBmrjSoXiWCAS9JSecty7pmcOGojIbsFXTxxKDOr+cMES8dZvhaBp2NMSf6AqTf8x1n78Wb3FXu2k3Yfxhw==; 20:H9npG/N3roeBFYQ+smm0D2z3dkXSf6DhR9ru2zpjC+yhLAUmVOMBVLg+4ylR8NwpgrxuerzKvGzlz4XKJoe0et1FIuWZol8c+uW9GI/A/9PKBO64N9EVCoCt7g55lhBWI+dd0qw+Es6eRK8/RtHiSO/oRBChUpAI63ZPh6jGyHk= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2017 09:35:09.3829 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1988 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 104.47.2.136 Subject: [Qemu-devel] [PATCH v1 04/13] qcow2: preallocation at image expand X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, "Denis V. Lunev" , Anton Nefedov , den@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: "Denis V. Lunev" This patch adds image preallocation at expand to provide better locality of QCOW2 image file and optimize this procedure for some distributed storages where this procedure is slow. Image expand requests have to be suspended until the allocation is performed which is done via special QCowL2Meta. This meta is invisible to handle_dependencies() code. This is the main reason for also calling preallocation before metadata write: it might intersect with preallocation triggered by another IO, and has to yield Signed-off-by: Denis V. Lunev Signed-off-by: Anton Nefedov --- block/qcow2-cache.c | 3 + block/qcow2-cluster.c | 5 ++ block/qcow2-refcount.c | 14 +++++ block/qcow2.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ block/qcow2.h | 5 ++ 5 files changed, 178 insertions(+) diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c index 1d25147..aa9da5f 100644 --- a/block/qcow2-cache.c +++ b/block/qcow2-cache.c @@ -204,6 +204,9 @@ static int qcow2_cache_entry_flush(BlockDriverState *bs, Qcow2Cache *c, int i) return ret; } + /* check and preallocate extra space if touching a fresh metadata cluster */ + qcow2_handle_prealloc(bs, c->entries[i].offset, s->cluster_size); + if (c == s->refcount_block_cache) { BLKDBG_EVENT(bs->file, BLKDBG_REFBLOCK_UPDATE_PART); } else if (c == s->l2_table_cache) { diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index cf18dee..a4b6d40 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -108,6 +108,9 @@ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size, goto fail; } + qcow2_handle_prealloc(bs, new_l1_table_offset, + QEMU_ALIGN_UP(new_l1_size2, s->cluster_size)); + BLKDBG_EVENT(bs->file, BLKDBG_L1_GROW_WRITE_TABLE); for(i = 0; i < s->l1_size; i++) new_l1_table[i] = cpu_to_be64(new_l1_table[i]); @@ -1820,6 +1823,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, goto fail; } + qcow2_handle_prealloc(bs, offset, s->cluster_size); + ret = bdrv_pwrite_zeroes(bs->file, offset, s->cluster_size, 0); if (ret < 0) { if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 7c06061..873a1d2 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -547,6 +547,8 @@ static int alloc_refcount_block(BlockDriverState *bs, } /* Write refcount blocks to disk */ + qcow2_handle_prealloc(bs, meta_offset, blocks_clusters * s->cluster_size); + BLKDBG_EVENT(bs->file, BLKDBG_REFBLOCK_ALLOC_WRITE_BLOCKS); ret = bdrv_pwrite_sync(bs->file, meta_offset, new_blocks, blocks_clusters * s->cluster_size); @@ -561,6 +563,10 @@ static int alloc_refcount_block(BlockDriverState *bs, cpu_to_be64s(&new_table[i]); } + qcow2_handle_prealloc(bs, table_offset, + QEMU_ALIGN_UP(table_size * sizeof(uint64_t), + s->cluster_size)); + BLKDBG_EVENT(bs->file, BLKDBG_REFBLOCK_ALLOC_WRITE_TABLE); ret = bdrv_pwrite_sync(bs->file, table_offset, new_table, table_size * sizeof(uint64_t)); @@ -2104,6 +2110,8 @@ write_refblocks: goto fail; } + qcow2_handle_prealloc(bs, refblock_offset, s->cluster_size); + /* The size of *refcount_table is always cluster-aligned, therefore the * write operation will not overflow */ on_disk_refblock = (void *)((char *) *refcount_table + @@ -2158,6 +2166,8 @@ write_refblocks: } assert(reftable_size < INT_MAX / sizeof(uint64_t)); + qcow2_handle_prealloc(bs, reftable_offset, + reftable_size * sizeof(uint64_t)); ret = bdrv_pwrite(bs->file, reftable_offset, on_disk_reftable, reftable_size * sizeof(uint64_t)); if (ret < 0) { @@ -2845,6 +2855,10 @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order, cpu_to_be64s(&new_reftable[i]); } + qcow2_handle_prealloc(bs, new_reftable_offset, + QEMU_ALIGN_UP(new_reftable_size * sizeof(uint64_t), + s->cluster_size)); + ret = bdrv_pwrite(bs->file, new_reftable_offset, new_reftable, new_reftable_size * sizeof(uint64_t)); diff --git a/block/qcow2.c b/block/qcow2.c index b438f22..6e7ce96 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -464,6 +464,11 @@ static QemuOptsList qcow2_runtime_opts = { .type = QEMU_OPT_NUMBER, .help = "Clean unused cache entries after this time (in seconds)", }, + { + .name = QCOW2_OPT_PREALLOC_SIZE, + .type = QEMU_OPT_SIZE, + .help = "Preallocation amount at image expand", + }, { /* end of list */ } }, }; @@ -754,6 +759,13 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, r->discard_passthrough[QCOW2_DISCARD_OTHER] = qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false); + s->prealloc_size = + ROUND_UP(qemu_opt_get_size_del(opts, QCOW2_OPT_PREALLOC_SIZE, 0), + s->cluster_size); + if (s->prealloc_size && bs->file->bs->drv->bdrv_co_pwrite_zeroes == NULL) { + s->prealloc_size = 0; + } + ret = 0; fail: qemu_opts_del(opts); @@ -1597,6 +1609,135 @@ static void handle_cow_reduce(BlockDriverState *bs, QCowL2Meta *m) } } +/* + * Checks that the host space area specified by @m is not being preallocated + * at the moment, and does co_queue_wait() if it is. + * If the specified area is not allocated yet, allocates it + prealloc_size + * bytes ahead. + * + * Returns + * true if the space is allocated and contains zeroes + */ +static bool coroutine_fn handle_prealloc(BlockDriverState *bs, + const QCowL2Meta *m) +{ + BDRVQcow2State *s = bs->opaque; + BlockDriverState *file = bs->file->bs; + QCowL2Meta *old, *meta; + uint64_t start = m->alloc_offset; + uint64_t end = start + (m->nb_clusters << s->cluster_bits); + uint64_t nbytes; + int err; + + assert(offset_into_cluster(s, start) == 0); + +restart: + /* check that the request is not overlapped with any + currently running preallocations */ + QLIST_FOREACH(old, &s->cluster_allocs, next_in_flight) { + uint64_t old_start, old_end; + + old_start = old->alloc_offset; + old_end = old_start + (old->nb_clusters << s->cluster_bits); + + if (old == m || end <= old_start || start >= old_end) { + /* No intersection */ + continue; + } + + qemu_co_queue_wait(&old->dependent_requests, NULL); + goto restart; + } + + if (end <= bdrv_getlength(file)) { + /* No need to care, file size will not be changed */ + return false; + } + + meta = g_alloca(sizeof(*meta)); + *meta = (QCowL2Meta) { + /* this meta is invisible for handle_dependencies() */ + .alloc_offset = bdrv_getlength(file), + .nb_clusters = size_to_clusters(s, start + + (m->nb_clusters << s->cluster_bits) + + s->prealloc_size - bdrv_getlength(file)), + }; + qemu_co_queue_init(&meta->dependent_requests); + QLIST_INSERT_HEAD(&s->cluster_allocs, meta, next_in_flight); + + nbytes = meta->nb_clusters << s->cluster_bits; + + /* try to alloc host space in one chunk for better locality */ + err = file->drv->bdrv_co_pwrite_zeroes(file, meta->alloc_offset, nbytes, 0); + + QLIST_REMOVE(meta, next_in_flight); + qemu_co_queue_restart_all(&meta->dependent_requests); + + if (err == 0) { + file->total_sectors = + MAX(file->total_sectors, + (meta->alloc_offset + nbytes) / BDRV_SECTOR_SIZE); + return start >= meta->alloc_offset; + } + return false; +} + +typedef struct { + BlockDriverState *bs; + uint64_t offset; + uint64_t size; + int ret; +} PreallocCo; + +static void coroutine_fn handle_prealloc_co_entry(void* opaque) +{ + PreallocCo *prco = opaque; + BDRVQcow2State *s = prco->bs->opaque; + QCowL2Meta meta = { + /* this meta is invisible for handle_dependencies() */ + .alloc_offset = prco->offset, + .nb_clusters = size_to_clusters(s, prco->size) + }; + handle_prealloc(prco->bs, &meta); + prco->ret = 0; +} + +/* + * Context(coroutine)-independent interface around handle_prealloc(), see + * its description. + * Must be called on a first write on the newly allocated cluster(s). + * @offset and @size must be cluster_aligned + */ +void qcow2_handle_prealloc(BlockDriverState *bs, uint64_t offset, uint64_t size) +{ + BDRVQcow2State *s = bs->opaque; + PreallocCo prco = { + .bs = bs, + .offset = offset, + .size = size, + .ret = -EAGAIN + }; + + assert(offset_into_cluster(s, offset) == 0); + assert(offset_into_cluster(s, size) == 0); + + if (s->prealloc_size == 0 || + bs->file->bs->drv->bdrv_co_pwrite_zeroes == NULL) { + return; + } + + if (qemu_in_coroutine()) { + handle_prealloc_co_entry(&prco); + } else { + AioContext *aio_context = bdrv_get_aio_context(bs); + Coroutine *co = qemu_coroutine_create(handle_prealloc_co_entry, &prco); + qemu_coroutine_enter(co); + while (prco.ret == -EAGAIN) { + aio_poll(aio_context, true); + } + } +} + static void handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) { BDRVQcow2State *s = bs->opaque; @@ -1607,6 +1748,11 @@ static void handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) for (m = l2meta; m != NULL; m = m->next) { uint64_t bytes = m->nb_clusters << s->cluster_bits; + if (s->prealloc_size != 0 && handle_prealloc(bs, m)) { + handle_cow_reduce(bs, m); + continue; + } + if (m->cow_start.nb_bytes == 0 && m->cow_end.nb_bytes == 0) { continue; } @@ -2725,6 +2871,11 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, goto fail; } + qcow2_handle_prealloc(bs, start_of_cluster(s, cluster_offset), + QEMU_ALIGN_UP( + offset_into_cluster(s, cluster_offset) + out_len, + s->cluster_size)); + iov = (struct iovec) { .iov_base = out_buf, .iov_len = out_len, diff --git a/block/qcow2.h b/block/qcow2.h index ba15c08..a0d222d 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -97,6 +97,7 @@ #define QCOW2_OPT_L2_CACHE_SIZE "l2-cache-size" #define QCOW2_OPT_REFCOUNT_CACHE_SIZE "refcount-cache-size" #define QCOW2_OPT_CACHE_CLEAN_INTERVAL "cache-clean-interval" +#define QCOW2_OPT_PREALLOC_SIZE "prealloc-size" typedef struct QCowHeader { uint32_t magic; @@ -294,6 +295,8 @@ typedef struct BDRVQcow2State { * override) */ char *image_backing_file; char *image_backing_format; + + uint64_t prealloc_size; } BDRVQcow2State; typedef struct Qcow2COWRegion { @@ -493,6 +496,8 @@ int qcow2_mark_dirty(BlockDriverState *bs); int qcow2_mark_corrupt(BlockDriverState *bs); int qcow2_mark_consistent(BlockDriverState *bs); int qcow2_update_header(BlockDriverState *bs); +void qcow2_handle_prealloc(BlockDriverState *bs, + uint64_t offset, uint64_t size); void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset, int64_t size, const char *message_format, ...)