From patchwork Mon Oct 28 15:49:51 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 3103171 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 8F357BF924 for ; Mon, 28 Oct 2013 15:50:03 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D7EA120213 for ; Mon, 28 Oct 2013 15:50:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC5A620240 for ; Mon, 28 Oct 2013 15:49:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756711Ab3J1Ptz (ORCPT ); Mon, 28 Oct 2013 11:49:55 -0400 Received: from dkim2.fusionio.com ([66.114.96.54]:54243 "EHLO dkim2.fusionio.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756004Ab3J1Pty (ORCPT ); Mon, 28 Oct 2013 11:49:54 -0400 Received: from mx2.fusionio.com (unknown [10.101.1.160]) by dkim2.fusionio.com (Postfix) with ESMTP id 2C8759A06A7 for ; Mon, 28 Oct 2013 09:49:54 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fusionio.com; s=default; t=1382975394; bh=pSauxYC7ydVctJO3vSPXzdnEr6GGYeGsWBJOuooodTo=; h=From:To:Subject:Date; b=dOf8HnaBO8WMehaPOaHjpSOvJuWoZ+d7bC4EgvkIdZpTtPLdyNZh+7TeFe+l46uNS GtqbHGUs3xWnvivs1f2zT5N1QbH2DbpVlnnlAVrLrk1flqX8srCC7mOSQciR1MXrBq aHSYM2nLmXvw7RNcBqpkueGevL0aq7AKy/VjkNbo= X-ASG-Debug-ID: 1382975393-0421b57ae51cf90001-6jHSXT Received: from CAS1.int.fusionio.com (cas1.int.fusionio.com [10.101.1.40]) by mx2.fusionio.com with ESMTP id Afwadb3iwVCCt3jQ (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO); Mon, 28 Oct 2013 09:49:53 -0600 (MDT) X-Barracuda-Envelope-From: JBacik@fusionio.com Received: from localhost (10.101.1.160) by mail.fusionio.com (10.101.1.40) with Microsoft SMTP Server (TLS) id 14.3.158.1; Mon, 28 Oct 2013 09:49:52 -0600 From: Josef Bacik To: , Subject: [PATCH] xfstests: add generic/320 to test fsync() on directories V2 Date: Mon, 28 Oct 2013 11:49:51 -0400 X-ASG-Orig-Subj: [PATCH] xfstests: add generic/320 to test fsync() on directories V2 Message-ID: <1382975391-26843-1-git-send-email-jbacik@fusionio.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-Originating-IP: [10.101.1.160] X-Barracuda-Connect: cas1.int.fusionio.com[10.101.1.40] X-Barracuda-Start-Time: 1382975393 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://10.101.1.181:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at fusionio.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=MARKETING_SUBJECT X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.141858 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Btrfs had some issues with fsync()'ing directories and fsync()'ing after renames. These three new tests cover the 3 different issues we were seeing. This breaks out the dmflakey stuff into a common helper to be shared between generic/311 and generic/320. Thanks, Signed-off-by: Josef Bacik --- V1->V2: moved this out into its own test instead of adding it to 311 common/dmflakey | 74 ++++++++++++++++++++++++++ tests/generic/311 | 56 +++----------------- tests/generic/320 | 140 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/320.out | 9 ++++ tests/generic/group | 1 + 5 files changed, 231 insertions(+), 49 deletions(-) create mode 100644 common/dmflakey create mode 100644 tests/generic/320 create mode 100644 tests/generic/320.out diff --git a/common/dmflakey b/common/dmflakey new file mode 100644 index 0000000..bc984bd --- /dev/null +++ b/common/dmflakey @@ -0,0 +1,74 @@ +##/bin/bash +# +# Copyright (c) 2013 Fusion IO, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +# +# common functions for setting up and tearing down a dmflakey device + +FLAKEY_ALLOW_WRITES=0 +FLAKEY_DROP_WRITES=1 + +_init_flakey() +{ + local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV` + FLAKEY_DEV=/dev/mapper/flakey-test + FLAKEY_TABLE="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 180 0" + FLAKEY_TABLE_DROP="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 0 180 1 drop_writes" + $DMSETUP_PROG create flakey-test --table "$FLAKEY_TABLE" || \ + _fatal "failed to create flakey device" +} + +_mount_flakey() +{ + mount -t $FSTYP $MOUNT_OPTIONS $FLAKEY_DEV $SCRATCH_MNT +} + +_unmount_flakey() +{ + $UMOUNT_PROG $SCRATCH_MNT +} + +_cleanup_flakey() +{ + # If dmsetup load fails then we need to make sure to do resume here + # otherwise the umount will hang + $DMSETUP_PROG resume flakey-test > /dev/null 2>&1 + $UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1 + $DMSETUP_PROG remove flakey-test > /dev/null 2>&1 +} + +# _load_flakey_table [lockfs] +# +# This defaults to --nolockfs, which doesn't freeze_fs() before loading the new +# table, so it simulates power failure. +_load_flakey_table() +{ + + table="$FLAKEY_TABLE" + [ $1 -eq $FLAKEY_DROP_WRITES ] && table="$FLAKEY_TABLE_DROP" + + suspend_opt="--nolockfs" + [ $# -gt 1 ] && [ $2 -eq 1 ] && suspend_opt="" + + $DMSETUP_PROG suspend $suspend_opt flakey-test + [ $? -ne 0 ] && _fatal "failed to suspend flakey-test" + + $DMSETUP_PROG load flakey-test --table "$table" + [ $? -ne 0 ] && _fatal "failed to load table into flakey-test" + + $DMSETUP_PROG resume flakey-test + [ $? -ne 0 ] && _fatal "failed to resumeflakey-test" +} diff --git a/tests/generic/311 b/tests/generic/311 index 675d927..6802a96 100644 --- a/tests/generic/311 +++ b/tests/generic/311 @@ -41,17 +41,14 @@ status=1 # failure is the default! _cleanup() { - # If dmsetup load fails then we need to make sure to do resume here - # otherwise the umount will hang - $DMSETUP_PROG resume flakey-test > /dev/null 2>&1 - $UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1 - $DMSETUP_PROG remove flakey-test > /dev/null 2>&1 + _cleanup_flakey } trap "_cleanup; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common/rc . ./common/filter +. ./common/dmflakey # real QA test starts here _supported_fs generic @@ -63,49 +60,12 @@ _require_dm_flakey [ -x $here/src/fsync-tester ] || _notrun "fsync-tester not build" rm -f $seqres.full -BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV` -FLAKEY_DEV=/dev/mapper/flakey-test SEED=1 testfile=$SCRATCH_MNT/$seq.fsync -FLAKEY_TABLE="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 180 0" -FLAKEY_TABLE_DROP="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 0 180 1 drop_writes" -_TEST_OPTIONS="" - -_mount_flakey() -{ - mount -t $FSTYP $MOUNT_OPTIONS $FLAKEY_DEV $SCRATCH_MNT -} - -_unmount_flakey() -{ - $UMOUNT_PROG $SCRATCH_MNT -} - -_load_flakey_table() -{ - # _load_flakey_table
- - table="$FLAKEY_TABLE" - [ $1 -eq 1 ] && table="$FLAKEY_TABLE_DROP" - - suspend_opt="" - [ $nolockfs -eq 1 ] && suspend_opt="--nolockfs" - - $DMSETUP_PROG suspend $suspend_opt flakey-test - [ $? -ne 0 ] && _fatal "failed to suspend flakey-test" - - $DMSETUP_PROG load flakey-test --table "$table" - [ $? -ne 0 ] && _fatal "failed to load table into flakey-test" - - $DMSETUP_PROG resume flakey-test - [ $? -ne 0 ] && _fatal "failed to resumeflakey-test" -} _run_test() { # _run_test <0 - buffered | 1 - O_DIRECT> - allow_writes=0 - drop_writes=1 test_num=$1 direct_opt="" @@ -115,11 +75,11 @@ _run_test() [ $? -ne 0 ] && _fatal "fsync tester exited abnormally" _md5_checksum $testfile - _load_flakey_table $drop_writes + _load_flakey_table $FLAKEY_DROP_WRITES $lockfs _unmount_flakey #Ok mount so that any recovery that needs to happen is done - _load_flakey_table $allow_writes + _load_flakey_table $FLAKEY_ALLOW_WRITES _mount_flakey _md5_checksum $testfile @@ -134,23 +94,21 @@ _run_test() _scratch_mkfs >> $seqres.full 2>&1 # Create a basic flakey device that will never error out -$DMSETUP_PROG create flakey-test --table "$FLAKEY_TABLE" -[ $? -ne 0 ] && _fatal "failed to create flakey device" - +_init_flakey _mount_flakey buffered=0 direct=1 for i in $(seq 1 20); do - nolockfs=0 + lockfs=1 SEED=$i echo "Running test $i buffered, normal suspend" _run_test $i $buffered echo "Running test $i direct, normal suspend" _run_test $i $direct - nolockfs=1 + lockfs=0 echo "Running test $i buffered, nolockfs" _run_test $i $buffered echo "Running test $i direct, nolockfs" diff --git a/tests/generic/320 b/tests/generic/320 new file mode 100644 index 0000000..4b4a77c --- /dev/null +++ b/tests/generic/320 @@ -0,0 +1,140 @@ +#! /bin/bash +# FS QA Test No. 320 +# +# Runs various dir fsync tests to cover fsync'ing directory corner cases. +# +#----------------------------------------------------------------------- +# Copyright (c) 2013 Fusion IO. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_flakey +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmflakey + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_need_to_be_root +_require_scratch +_require_dm_flakey + +rm -f $seqres.full + +_clean_working_dir() +{ + _mount_flakey + rm -rf $SCRATCH_MNT/* + _unmount_flakey +} + +# Btrfs wasn't making sure the directory survived fsync +_directory_test() +{ + echo "fsync new directory" + _mount_flakey + mkdir $SCRATCH_MNT/bar + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar + _load_flakey_table $FLAKEY_DROP_WRITES + _unmount_flakey + + _load_flakey_table $FLAKEY_ALLOW_WRITES + _mount_flakey + _ls_l $SCRATCH_MNT | tail -n +2 | awk '{ print $1, $9 }' + _unmount_flakey + _check_scratch_fs $FLAKEY_DEV + [ $? -ne 0 ] && _fatal "fsck failed" +} + +# Btrfs was losing a rename into a new directory +_rename_test() +{ + echo "rename fsync test" + _mount_flakey + touch $SCRATCH_MNT/foo + mkdir $SCRATCH_MNT/bar + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo + mv $SCRATCH_MNT/foo $SCRATCH_MNT/bar/foo + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar/foo + _load_flakey_table $FLAKEY_DROP_WRITES + _unmount_flakey + + _load_flakey_table $FLAKEY_ALLOW_WRITES + _mount_flakey + _ls_l $SCRATCH_MNT | tail -n +2 | awk '{ print $1, $9 }' + _ls_l $SCRATCH_MNT/bar | tail -n +2 | awk '{ print $1, $9 }' + _unmount_flakey + _check_scratch_fs $FLAKEY_DEV + [ $? -ne 0 ] && _fatal "fsck failed" +} + +# Btrfs was failing to replay a log when we had a inode with a smaller inode +# number that is renamed into a directory with a higher inode number +_replay_rename_test() +{ + echo "replay rename fsync test" + _mount_flakey + touch $SCRATCH_MNT/foo + mkdir $SCRATCH_MNT/bar + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/foo + mv $SCRATCH_MNT/foo $SCRATCH_MNT/bar/foo + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar + + # This is to force btrfs to relog the entire inode including the ref so + # we are sure to try and replay the ref along with the dir_index item + setfattr -n user.foo -v blah $SCRATCH_MNT/bar/foo >> $seqres.full 2>&1 + + $XFS_IO_PROG -c "fsync" $SCRATCH_MNT/bar/foo + _load_flakey_table $FLAKEY_DROP_WRITES + _unmount_flakey + + _load_flakey_table $FLAKEY_ALLOW_WRITES + _mount_flakey + _ls_l $SCRATCH_MNT | tail -n +2 | awk '{ print $1, $9 }' + _ls_l $SCRATCH_MNT/bar | tail -n +2 | awk '{ print $1, $9 }' + _unmount_flakey + _check_scratch_fs $FLAKEY_DEV + [ $? -ne 0 ] && _fatal "fsck failed" +} + +_scratch_mkfs >> $seqres.full 2>&1 + +# Create a basic flakey device that will never error out +_init_flakey + +_directory_test +_clean_working_dir +_rename_test +_clean_working_dir +_replay_rename_test + +status=0 +exit diff --git a/tests/generic/320.out b/tests/generic/320.out new file mode 100644 index 0000000..6e563dd --- /dev/null +++ b/tests/generic/320.out @@ -0,0 +1,9 @@ +QA output created by 320 +fsync new directory +drwxr-xr-x bar +rename fsync test +drwxr-xr-x bar +-rw-r--r-- foo +replay rename fsync test +drwxr-xr-x bar +-rw-r--r-- foo diff --git a/tests/generic/group b/tests/generic/group index 3488c42..b48eab8 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -122,4 +122,5 @@ 317 auto metadata quick 318 acl attr auto quick 319 acl auto quick +320 auto quick metadata log