From patchwork Sun Jan 4 03:48:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Danielle Church X-Patchwork-Id: 5562601 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5C3F7BF6C3 for ; Sun, 4 Jan 2015 03:54:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 68341200F3 for ; Sun, 4 Jan 2015 03:54:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4CA572017A for ; Sun, 4 Jan 2015 03:54:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752156AbbADDyO (ORCPT ); Sat, 3 Jan 2015 22:54:14 -0500 Received: from cheri.shyou.org ([74.208.174.96]:45444 "EHLO cheri.shyou.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752039AbbADDyK (ORCPT ); Sat, 3 Jan 2015 22:54:10 -0500 X-Greylist: delayed 340 seconds by postgrey-1.27 at vger.kernel.org; Sat, 03 Jan 2015 22:54:10 EST Received: from localhost (localhost [127.0.0.1]) by cheri.shyou.org (Postfix) with ESMTP id 2106C1303D1 for ; Sat, 3 Jan 2015 22:48:26 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cheri.shyou.org; h=content-type:content-type:mime-version:user-agent:message-id :subject:subject:from:from:date:date:received:received; s=d1024; t=1420343297; bh=AtCepx/d1QQzJx62gvalub+XyGtsZ2jj87JOJNw+flg=; b= ntqjt8vvXcV0YY3hVdyitVDgQA1T8nZWjDcv1THcqaISjFQ+GjfocHRjD4oVQWse xO6Kmagz1lAwS2n2XP6AIeo+o2zBdEH5px1KWy9s8N4S2vP0U9JzLa61OYkRMjMr Yvk8ia8/r49jaKKTgvXRPqZPSeS0eJSUscvAi9zLBrA= X-Virus-Scanned: amavisd-new at shyou.org Received: from cheri.shyou.org ([127.0.0.1]) by localhost (cheri.shyou.org [127.0.0.1]) (amavisd-new, port 10026) with LMTP id It_NJTWp-PHq for ; Sat, 3 Jan 2015 22:48:17 -0500 (EST) Received: by cheri.shyou.org (Postfix, from userid 1000) id 3C4F41303C5; Sat, 3 Jan 2015 22:48:17 -0500 (EST) Date: Sat, 3 Jan 2015 22:48:17 -0500 (EST) From: Danielle Church To: linux-btrfs@vger.kernel.org Subject: [PATCH 1/2] btrfs: fix compress=zlib when stream end crosses page boundary Message-ID: User-Agent: Alpine 2.00 (LNX 1167 2008-08-23) MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In zlib_compress_pages(), if the last looped call to zlib_deflate ends between 1 and 5 bytes short of a page boundary, the final call with Z_FINISH is unable to write the final 6 bytes of the zlib stream and bails out. This causes compress_file_range() in inode.c to distrust the compressor and flag ths inode nocompress, with the end result that the rest of a potentially highly-compressible file is stored in expanded form. You can demonstrate this with the following script, setting MOUNTPOINT to an otherwise-unused btrfs mount with compress=zlib: MOUNTPOINT=/mnt sync; df $MOUNTPOINT { head -c 6630 /usr/src/linux/Documentation/BUG-HUNTING; cat /dev/zero; } | head -c 1310720 > $MOUNTPOINT/test sync; df $MOUNTPOINT rm -f $MOUNTPOINT/test The selection of 6630 bytes from BUG-HUNTING plus nulls provides for a first 128KiB chunk that compresses to just over 4 KiB, and the other nine chunks compress easily within a single page each, so this should compress to 11 pages or 44 KiB; instead it uses significantly more, depending on how many threads were able to compress chunks before nocompress was set. The following patch fixes this by moving the Z_FINISH calls to inside the page read/write loop, allowing it to call as many times as needed. Signed-off-by: Danielle Church --- fs/btrfs/zlib.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/zlib.c b/fs/btrfs/zlib.c index fb22fd8..1dc0455 100644 --- a/fs/btrfs/zlib.c +++ b/fs/btrfs/zlib.c @@ -89,6 +89,7 @@ static int zlib_compress_pages(struct list_head *ws, struct page *in_page = NULL; struct page *out_page = NULL; unsigned long bytes_left; + int deflate_flush = Z_SYNC_FLUSH; *out_pages = 0; *total_out = 0; @@ -120,8 +121,12 @@ static int zlib_compress_pages(struct list_head *ws, workspace->strm.avail_out = PAGE_CACHE_SIZE; workspace->strm.avail_in = min(len, PAGE_CACHE_SIZE); - while (workspace->strm.total_in < len) { - ret = zlib_deflate(&workspace->strm, Z_SYNC_FLUSH); + while (deflate_flush != Z_FINISH || ret != Z_STREAM_END) { + ret = zlib_deflate(&workspace->strm, deflate_flush); + /* we're all done, including the stream end */ + if (ret == Z_STREAM_END && deflate_flush == Z_FINISH) + break; + if (ret != Z_OK) { printk(KERN_DEBUG "BTRFS: deflate in loop returned %d\n", ret); @@ -159,9 +164,12 @@ static int zlib_compress_pages(struct list_head *ws, workspace->strm.avail_out = PAGE_CACHE_SIZE; workspace->strm.next_out = cpage_out; } - /* we're all done */ - if (workspace->strm.total_in >= len) - break; + /* we're all done with input data; keep looping until stream end is written */ + if (workspace->strm.total_in >= len) { + deflate_flush = Z_FINISH; + workspace->strm.avail_in = 0; + continue; + } /* we've read in a full page, get a new one */ if (workspace->strm.avail_in == 0) { @@ -181,11 +189,11 @@ static int zlib_compress_pages(struct list_head *ws, workspace->strm.next_in = data_in; } } - workspace->strm.avail_in = 0; - ret = zlib_deflate(&workspace->strm, Z_FINISH); - zlib_deflateEnd(&workspace->strm); + ret = zlib_deflateEnd(&workspace->strm); - if (ret != Z_STREAM_END) { + if (ret != Z_OK) { + printk(KERN_DEBUG "BTRFS: deflateEnd returned %d\n", + ret); ret = -EIO; goto out; }