From patchwork Sat Mar 23 22:59:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nir Soffer X-Patchwork-Id: 10867319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71E251390 for ; Sun, 24 Mar 2019 01:22:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 56E1B29A20 for ; Sun, 24 Mar 2019 01:22:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A3C829B2E; Sun, 24 Mar 2019 01:22:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CD86129A20 for ; Sun, 24 Mar 2019 01:22:34 +0000 (UTC) Received: from localhost ([127.0.0.1]:50197 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h7rqL-0001Ej-DR for patchwork-qemu-devel@patchwork.kernel.org; Sat, 23 Mar 2019 21:22:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:55250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h7pji-0007kP-QJ for qemu-devel@nongnu.org; Sat, 23 Mar 2019 19:07:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h7pcl-0006Kk-Dh for qemu-devel@nongnu.org; Sat, 23 Mar 2019 19:00:24 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:34134) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h7pci-0006Ga-Gs; Sat, 23 Mar 2019 19:00:20 -0400 Received: by mail-wm1-x344.google.com with SMTP id o10so6816942wmc.1; Sat, 23 Mar 2019 16:00:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=dXvN8k/Xxx239HSFIMFN+wJjWLgqgZydI+6NI4QDH1Y=; b=ZPJDMCH8cCfvEAuB/sNs3Ny7+traUfNiKOLsI0o4sxdLjmtwfxev/kI0XYtyJukrIx c63h2p5KdVIzUFzkU/KlOS5rTOp5Lv6Ph+OYGzHMCsdOYmPjB+spNdX/DXh+uzv7SanA XbAqCBONCciaTcm3Mz36zPik4BNjCBgZbyn4FVBU7HrmcoVqxipqHJlYNB+yWgUSoaKC glS/w/rYynhyTNJDDQnP1UgyPeQ/gARhQtFQ7ALfHKCi3NbcnFL5Rp/8euykSOUCpUKT cMY/aBOMDBT+VApr3Gj/Tdj1DLHpsdFBOfA883tLH7hJm6+jp67lKsl+L9DW+1vXONj6 YQwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=dXvN8k/Xxx239HSFIMFN+wJjWLgqgZydI+6NI4QDH1Y=; b=kUCfOWfTGTV/AS/s1geyTIxiMjdm7RipnDWuzSNmWU0iwShqJddPus652UU5n7Rm06 8dCLXSpfxwXF3oYGK3qK7bqJ+cll7eg7rqvG2QNYHKxwttTyP5ni1Hi+jC3l0cqf54LK UDSmqTFkiNhx+lOD4wWY4mmlpguFmLaf0UqNm9SoEjcSA9Is1Yf811QxLkYkKdGEwtJR IeLj3PSi+rNApsleUTbEYso/AH6HWLq+GeWQrwWx4CaHOrwbdiGMbMpvydEZoG5m6Ajw +O1l64IBzILUEnbWUmO5l0/Uqcakm+Y0tZQXJYrVlEdy2oV+21Jdln4Q5utviJJrrWXo OmAw== X-Gm-Message-State: APjAAAXSn4Tts/WSzjOaCC5RyTXjG23rLmzYepVJKrEKBZUkpcJ4RbKP SgxUoNfS+PLFUzvtPVslihQA3UXn X-Google-Smtp-Source: APXvYqxeASWRWf02E6z0iTPTlePVNHZjAB7dUubb2h32ctnpxe1vrM2ackr6oeJW7AlMizRan1ROMg== X-Received: by 2002:a1c:f204:: with SMTP id s4mr6100878wmc.51.1553382014302; Sat, 23 Mar 2019 16:00:14 -0700 (PDT) Received: from lean.local (dsl217-132-181-48.bb.netvision.net.il. [217.132.181.48]) by smtp.gmail.com with ESMTPSA id y1sm25321732wrd.34.2019.03.23.16.00.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 23 Mar 2019 16:00:12 -0700 (PDT) From: Nir Soffer To: qemu-block@nongnu.org Date: Sun, 24 Mar 2019 00:59:48 +0200 Message-Id: <20190323225948.19737-1-nirsof@gmail.com> X-Mailer: git-send-email 2.17.2 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::344 X-Mailman-Approved-At: Sat, 23 Mar 2019 21:21:48 -0400 Subject: [Qemu-devel] [PATCH] qemu-img: Enable BDRV_REQ_MAY_UNMAP in convert X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , qemu-devel@nongnu.org, Nir Soffer , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP With Kevin's "block: Fix slow pre-zeroing in qemu-img convert"[1] we skip the pre zero step called like this: blk_make_zero(s->target, BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) And we write zeroes later using: blk_co_pwrite_zeroes(s->target, sector_num << BDRV_SECTOR_BITS, n << BDRV_SECTOR_BITS, 0); Since we use flags=0, this is translated to NBD_CMD_WRITE_ZEROES with NBD_CMD_FLAG_NO_HOLE flag, which cause the NBD server to allocated space instead of punching a hole. Here is an example failure: $ dd if=/dev/urandom of=src.img bs=1M count=5 $ truncate -s 50m src.img $ truncate -s 50m dst.img $ nbdkit -f -v -e '' -U nbd.sock file file=dst.img $ ./qemu-img convert -n src.img nbd:unix:nbd.sock We can see in nbdkit log that it received the NBD_CMD_FLAG_NO_HOLE (may_trim=0): nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d nbdkit: file[1]: debug: pwrite count=2097152 offset=0 nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152 nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304 nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=0 nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=0 nbdkit: file[1]: debug: flush And the image became fully allocated: $ qemu-img info dst.img virtual size: 50M (52428800 bytes) disk size: 50M With this change we see that nbdkit did not receive the NBD_CMD_FLAG_NO_HOLE (may_trim=1): nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d nbdkit: file[1]: debug: pwrite count=2097152 offset=0 nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152 nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304 nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=1 nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=1 nbdkit: file[1]: debug: flush And the file is sparse as expected: $ qemu-img info dst.img virtual size: 50M (52428800 bytes) disk size: 5.0M Tested on top of Kevin patches: http://lists.nongnu.org/archive/html/qemu-block/2019-03/msg00755.html I'm not sure this change is correct for all cases, posting for discussion. [1] http://lists.nongnu.org/archive/html/qemu-block/2019-03/msg00761.html Signed-off-by: Nir Soffer Reviewed-by: Eric Blake --- qemu-img.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/qemu-img.c b/qemu-img.c index 8ee63daeae..ca9deb3758 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1752,11 +1752,12 @@ static int coroutine_fn convert_co_write(ImgConvertState *s, int64_t sector_num, assert(!s->target_has_backing); break; } ret = blk_co_pwrite_zeroes(s->target, sector_num << BDRV_SECTOR_BITS, - n << BDRV_SECTOR_BITS, 0); + n << BDRV_SECTOR_BITS, + BDRV_REQ_MAY_UNMAP); if (ret < 0) { return ret; } break; }