From patchwork Tue Jun 27 14:49:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Lieven X-Patchwork-Id: 9812027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1888B6020A for ; Tue, 27 Jun 2017 14:51:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A93528543 for ; Tue, 27 Jun 2017 14:51:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F24112858E; Tue, 27 Jun 2017 14:51:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 300E020007 for ; Tue, 27 Jun 2017 14:51:25 +0000 (UTC) Received: from localhost ([::1]:53236 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dPrps-0004s1-5F for patchwork-qemu-devel@patchwork.kernel.org; Tue, 27 Jun 2017 10:51:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52954) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dProX-0004pu-IC for qemu-devel@nongnu.org; Tue, 27 Jun 2017 10:50:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dProT-0001SP-Ck for qemu-devel@nongnu.org; Tue, 27 Jun 2017 10:50:01 -0400 Received: from mx-v6.kamp.de ([2a02:248:0:51::16]:51177 helo=mx01.kamp.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dProT-0001M3-3C for qemu-devel@nongnu.org; Tue, 27 Jun 2017 10:49:57 -0400 Received: (qmail 30838 invoked by uid 89); 27 Jun 2017 14:49:54 -0000 Received: from [195.62.97.28] by client-16-kamp (envelope-from , uid 89) with qmail-scanner-2010/03/19-MF (clamdscan: 0.99.2/23512. avast: 1.2.2/17010300. spamassassin: 3.4.1. Clear:RC:1(195.62.97.28):. Processed in 0.074292 secs); 27 Jun 2017 14:49:54 -0000 Received: from smtp.kamp.de (HELO submission.kamp.de) ([195.62.97.28]) by mx01.kamp.de with ESMTPS (DHE-RSA-AES256-GCM-SHA384 encrypted); 27 Jun 2017 14:49:53 -0000 X-GL_Whitelist: yes Received: (qmail 30289 invoked from network); 27 Jun 2017 14:49:53 -0000 Received: from lieven-pc.kamp-intra.net (HELO ?172.21.12.60?) (pl@kamp.de@::ffff:172.21.12.60) by submission.kamp.de with ESMTPS (DHE-RSA-AES128-SHA encrypted) ESMTPA; 27 Jun 2017 14:49:53 -0000 To: Eric Blake , qemu-block@nongnu.org References: <1498566850-7934-1-git-send-email-pl@kamp.de> <1498566850-7934-2-git-send-email-pl@kamp.de> From: Peter Lieven Message-ID: <560f62fb-f6fb-42ae-2fd9-b10c9c478350@kamp.de> Date: Tue, 27 Jun 2017 16:49:53 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a02:248:0:51::16 Subject: Re: [Qemu-devel] [PATCH 1/4] block/qcow2: add compression_algorithm create option X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, lersek@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Am 27.06.2017 um 14:49 schrieb Eric Blake: > On 06/27/2017 07:34 AM, Peter Lieven wrote: >> this patch adds a new compression_algorithm option when creating qcow2 images. >> The current default for the compresison algorithm is zlib and zlib will be > s/compresison/compression/ > >> used when this option is omitted (like before). >> >> If the option is specified e.g. with: >> >> qemu-img create -f qcow2 -o compression_algorithm=zlib image.qcow2 1G >> >> then a new compression algorithm header extension is added and an incompatible >> feature bit is set. This means that if the header is present it must be parsed >> by Qemu on qcow2_open and it must be validated if the specified compression >> algorithm is supported by the current build of Qemu. >> >> This means if the compression_algorithm option is specified Qemu prior to this >> commit will not be able to open the created image. >> >> Signed-off-by: Peter Lieven >> --- >> block/qcow2.c | 93 ++++++++++++++++++++++++++++++++++++++++++++--- >> block/qcow2.h | 20 +++++++--- >> docs/interop/qcow2.txt | 8 +++- > Focusing on just the spec change first: > >> +++ b/docs/interop/qcow2.txt >> @@ -85,7 +85,11 @@ in the description of a field. >> be written to (unless for regaining >> consistency). >> >> - Bits 2-63: Reserved (set to 0) >> + Bit 2: Compress algorithm bit. If this bit is set then >> + the compress algorithm extension must be parsed >> + and checked for compatiblity. > s/compatiblity/compatibility/ > >> + >> + Bits 3-63: Reserved (set to 0) >> >> 80 - 87: compatible_features >> Bitmask of compatible features. An implementation can >> @@ -135,6 +139,8 @@ be stored. Each extension has a structure like the following: >> 0xE2792ACA - Backing file format name >> 0x6803f857 - Feature name table >> 0x23852875 - Bitmaps extension >> + 0xC0318300 - Compression Algorithm >> + 0xC03183xx - Reserved for compression algorithm params > s/params/parameters/ > > You have now introduced 256 different reserved headers, without > documenting any of their formats. You absolutely MUST include a > documentation of how the new 0xC0318300 header is laid out (see, for > example, our section on "Bitmaps extension"), along with text mentioning > that the new header MUST be present if incompatible-feature bit is set > and MUST be absent otherwise. But I also think that with a bit of > proper design work, you only need ONE header for all possible algorithm > parameters, rather than burning an additional 255 unspecified > reservations. That is, make sure your new header includes a common > prefix including a length field and the algorightm in use, and then the > length covers a variable-length suffix that can be parsed in a > per-algorithm-specific manner for whatever additional parameters are > needed for that algorithm. > Before I continue, can you please give feedback on the following spec change: Thanks, Peter diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index 80cdfd0..f1428e9 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -85,7 +85,11 @@ in the description of a field. be written to (unless for regaining consistency). - Bits 2-63: Reserved (set to 0) + Bit 2: Compression format bit. Iff this bit is set then + the compression format extension MUST be present + and MUST be parsed and checked for compatibility. + + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -135,6 +139,7 @@ be stored. Each extension has a structure like the following: 0xE2792ACA - Backing file format name 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension + 0xC0318300 - Compression format extension other - Unknown header extension, can be safely ignored @@ -208,6 +213,28 @@ The fields of the bitmaps extension are: starts. Must be aligned to a cluster boundary. +== Compression format extension == + +The compression format extension is an optional header extension. It provides +the ability to specify the compression algorithm and compression parameters +that are used for compressed clusters. This new header MUST be present if +the incompatible-feature bit "compression format bit" is set and MUST be absent +otherwise. + +The fields of the compression format extension are: + + Byte 0 - 15: compression_format_name (padded with zeros, but not + necessarily null terminated if it has full length) + + 16: compression_level (uint8_t) + 0 = default compression level + 1 = lowest compression level + x = highest compression level (the highest compression + level may vary for different compression formats) + + 17 - 23: Reserved for future use, must be zero. + + == Host cluster management == qcow2 manages the allocation of host clusters by maintaining a reference count