From patchwork Wed Nov 2 04:07:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiaoguang Wang X-Patchwork-Id: 9408451 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DE517601C2 for ; Wed, 2 Nov 2016 04:14:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE28629DCC for ; Wed, 2 Nov 2016 04:14:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C2D8529DCF; Wed, 2 Nov 2016 04:14:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,LOTS_OF_MONEY, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2184C29DCC for ; Wed, 2 Nov 2016 04:14:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751453AbcKBEOV (ORCPT ); Wed, 2 Nov 2016 00:14:21 -0400 Received: from cn.fujitsu.com ([222.73.24.84]:51022 "EHLO song.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-OK) by vger.kernel.org with ESMTP id S1750698AbcKBEOU (ORCPT ); Wed, 2 Nov 2016 00:14:20 -0400 X-IronPort-AV: E=Sophos;i="5.20,367,1444665600"; d="scan'208";a="940481" Received: from unknown (HELO cn.fujitsu.com) ([10.167.250.3]) by song.cn.fujitsu.com with ESMTP; 02 Nov 2016 12:14:02 +0800 Received: from localhost.localdomain (unknown [10.167.226.107]) by cn.fujitsu.com (Postfix) with ESMTP id 09C9740142D3; Wed, 2 Nov 2016 12:14:02 +0800 (CST) From: Wang Xiaoguang To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, eguan@redhat.com, darrick.wong@oracle.com Subject: [PATCH v5 2/2] generic: make 17[1-4] work well when btrfs compression is enabled Date: Wed, 2 Nov 2016 12:07:27 +0800 Message-Id: <20161102040727.5765-2-wangxg.fnst@cn.fujitsu.com> X-Mailer: git-send-email 2.9.0 In-Reply-To: <20161102040727.5765-1-wangxg.fnst@cn.fujitsu.com> References: <20161102040727.5765-1-wangxg.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 09C9740142D3.A15B8 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: wangxg.fnst@cn.fujitsu.com Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc error occurs. Note _fill_fs is copied from tests/generic/256, but with some minor modifications. Signed-off-by: Wang Xiaoguang --- V2: In common/, I did't find an existing function suitable for these 4 test cases to fill fs, so I still use _pwrite_byte() with a big enough file length fo fill fs. Note, for btrfs, metadata space still is not full, only data space is full, but it's OK for these 4 test cases. All these 4 cases pass in xfs and btrfs(without compression), if btrfs has compression enabled, these 4 cases will fail for false enospc error, I have sent kernel patches to fix this bug. V3: Introduce _fill_fs in common/rc to fill fs. V4: Fix some issues suggested by Eryu and Darrick. V5: Put _fill_fs() in common/populate. --- common/populate | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/171 | 4 ++-- tests/generic/172 | 5 +++-- tests/generic/173 | 4 ++-- tests/generic/174 | 4 ++-- tests/generic/256 | 66 +++++------------------------------------------------- 6 files changed, 82 insertions(+), 68 deletions(-) diff --git a/common/populate b/common/populate index 3b9b531..78e9809 100644 --- a/common/populate +++ b/common/populate @@ -539,3 +539,70 @@ _scratch_fuzz_test() { (find "${SCRATCH_MNT}/test.1/" -type f -size -1048576k -print0 | xargs -0 cat) >/dev/null 2>&1 } +# Fill a file system by repeatedly creating files in the given folder +# starting with the given file size. Files are reduced in size when +# they can no longer fit until no more files can be created. +_fill_fs() +{ + local file_size=$1 + local dir=$2 + local block_size=$3 + local switch_user=$4 + local file_count=1 + local bytes_written=0 + local use_falloc=1; + + if [ $# -ne 4 ]; then + echo "Usage: _fill_fs filesize dir blocksize switch_user" + exit 1 + fi + + if [ $switch_user -eq 0 ]; then + mkdir -p $dir + else + _user_do "mkdir -p $dir" + fi + if [ ! -d $dir ]; then + return 0; + fi + + testio=`$XFS_IO_PROG -F -fc "falloc 0 $block_size" $dir/$$.xfs_io 2>&1` + echo $testio | grep -q "not found" && use_falloc=0 + echo $testio | grep -q "Operation not supported" && use_falloc=0 + + if [ $file_size -lt $block_size ]; then + $file_size = $block_size + fi + + while [ $file_size -ge $block_size ]; do + bytes_written=0 + if [ $switch_user -eq 0 ]; then + if [ $use_falloc -eq 0 ]; then + $XFS_IO_PROG -fc "pwrite -b 8388608 0 $file_size" \ + $dir/$file_count + else + $XFS_IO_PROG -fc "falloc 0 $file_size" \ + $dir/$file_count + fi + else + if [ $use_falloc -eq 0 ]; then + _user_do "$XFS_IO_PROG -f -c \"pwrite -b 8388608 0 \ + $file_size\" $dir/$file_count" + else + _user_do "$XFS_IO_PROG -f -c \"falloc 0 \ + $file_size\" $dir/$file_count" + fi + fi + + if [ -f $dir/$file_count ]; then + bytes_written=$(stat -c '%s' $dir/$file_count) + fi + + # If there was no room to make the file, then divide it in + # half, and keep going + if [ $bytes_written -lt $file_size ]; then + file_size=$((file_size / 2)) + fi + file_count=$((file_count + 1)) + done +} diff --git a/tests/generic/171 b/tests/generic/171 index a69f798..d0cd192 100755 --- a/tests/generic/171 +++ b/tests/generic/171 @@ -38,6 +38,7 @@ _cleanup() # get standard environment, filters and checks . ./common/rc +. ./common/populate . ./common/filter . ./common/attr . ./common/reflink @@ -76,8 +77,7 @@ sync echo "Allocate the rest of the space" nr_free=$(stat -f -c '%f' $testdir) -touch $testdir/file0 $testdir/file1 -_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> $seqres.full 2>&1 +_fill_fs $((nr_free * blksz)) $testdir/space $blksz 0 >> $seqres.full 2>&1 sync echo "CoW the big file" diff --git a/tests/generic/172 b/tests/generic/172 index 8192290..d943e64 100755 --- a/tests/generic/172 +++ b/tests/generic/172 @@ -38,6 +38,7 @@ _cleanup() # get standard environment, filters and checks . ./common/rc +. ./common/populate . ./common/filter . ./common/attr . ./common/reflink @@ -57,6 +58,7 @@ testdir=$SCRATCH_MNT/test-$seq mkdir $testdir echo "Reformat with appropriate size" +blksz="$(get_block_size $testdir)" umount $SCRATCH_MNT file_size=$((168 * 1024 * 1024)) @@ -72,8 +74,7 @@ _cp_reflink $testdir/bigfile $testdir/clonefile sync echo "Allocate the rest of the space" -touch $testdir/file0 $testdir/file1 -_pwrite_byte 0x61 0 $fs_size $testdir/eat_my_space >> $seqres.full 2>&1 +_fill_fs $fs_size $testdir/space $blksz 0 >> $seqres.full 2>&1 sync echo "CoW the big file" diff --git a/tests/generic/173 b/tests/generic/173 index e35597f..a414c8e 100755 --- a/tests/generic/173 +++ b/tests/generic/173 @@ -38,6 +38,7 @@ _cleanup() # get standard environment, filters and checks . ./common/rc +. ./common/populate . ./common/filter . ./common/attr . ./common/reflink @@ -76,8 +77,7 @@ sync echo "Allocate the rest of the space" nr_free=$(stat -f -c '%f' $testdir) -touch $testdir/file0 $testdir/file1 -_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> $seqres.full 2>&1 +_fill_fs $((blksz * nr_free)) $testdir/space $blksz 0 >> $seqres.full 2>&1 sync echo "mmap CoW the big file" diff --git a/tests/generic/174 b/tests/generic/174 index e58d64b..b55cb72 100755 --- a/tests/generic/174 +++ b/tests/generic/174 @@ -38,6 +38,7 @@ _cleanup() # get standard environment, filters and checks . ./common/rc +. ./common/populate . ./common/filter . ./common/attr . ./common/reflink @@ -77,8 +78,7 @@ sync echo "Allocate the rest of the space" nr_free=$(stat -f -c '%f' $testdir) -touch $testdir/file0 $testdir/file1 -_pwrite_byte 0x61 0 $((blksz * nr_free)) $testdir/eat_my_space >> $seqres.full 2>&1 +_fill_fs $((blksz * nr_free)) $testdir/space $blksz 0 >> $seqres.full 2>&1 sync echo "CoW the big file" diff --git a/tests/generic/256 b/tests/generic/256 index cfbf790..63f2d4f 100755 --- a/tests/generic/256 +++ b/tests/generic/256 @@ -39,6 +39,7 @@ trap "_cleanup ; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common/rc +. ./common/populate . ./common/filter . ./common/punch @@ -53,64 +54,6 @@ _require_test testfile=$TEST_DIR/256.$$ -# _fill_fs() -# -# Fills a file system by repeatedly creating files in the given folder -# starting with the given file size. Files are reduced in size when -# they can no longer fit untill no more files can be created. -# -# This routine is used by _test_full_fs_punch to test that a hole may -# still be punched when the disk is full by borrowing reserved blocks. -# All files are created as a non root user to prevent reserved blocks -# from being consumed. -# -_fill_fs() { - local file_size=$1 - local dir=$2 - local block_size=$3 - local file_count=1 - local bytes_written=0 - - if [ $# -ne 3 ] - then - echo "USAGE: _fill_fs filesize dir block size" - exit 1 - fi - - # Creation of files or folders - # must not be done as root or - # reserved blocks will be consumed - _user_do "mkdir -p $dir &> /dev/null" - if [ $? -ne 0 ] ; then - return 0 - fi - - if [ $file_size -lt $block_size ] - then - $file_size = $block_size - fi - - while [ $file_size -ge $block_size ] - do - bytes_written=0 - _user_do "$XFS_IO_PROG -f -c \"pwrite 0 $file_size\" $dir/$file_count.bin &> /dev/null" - - if [ -f $dir/$file_count.bin ] - then - bytes_written=`$XFS_IO_PROG -c "stat" $dir/$file_count.bin | grep stat.size | cut -d ' ' -f3` - fi - - # If there was no room to make the file, - # then divide it in half, and keep going - if [ $bytes_written -lt $file_size ] - then - file_size=$(( $file_size / 2 )) - fi - file_count=$(( $file_count + 1 )) - - done -} - # _test_full_fs_punch() # # This function will test that a hole may be punched @@ -144,7 +87,10 @@ _test_full_fs_punch() -c "fsync" $file_name &> /dev/null chmod 666 $file_name - _fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size + # All files are created as a non root user to prevent reserved blocks + # from being consumed. + _fill_fs $(( 1024 * 1024 * 1024 )) $path/fill $block_size 1 \ + > /dev/null 2>&1 for (( i=0; i<$iterations; i++ )) do @@ -159,7 +105,7 @@ _test_full_fs_punch() hole_offset=$(( $hole_offset + $hole_len + $hole_interval )) - _fill_fs $hole_len $path/fill.$i $block_size + _fill_fs $hole_len $path/fill.$i $block_size 1 > /dev/null 2>&1 done }