| Message ID | 20210901221006.125888-2-catherine.hoang@oracle.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Log Attribute Replay Test | expand |
On Wed, Sep 01, 2021 at 10:10:06PM +0000, Catherine Hoang wrote: > From: Allison Henderson <allison.henderson@oracle.com> > > This patch adds a test to exercise the log attribute error > inject and log replay. Attributes are added in increaseing > sizes up to 64k, and the error inject is used to replay them > from the log > > Signed-off-by: Allison Henderson <allison.henderson@oracle.com> > Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com> > --- > V2: Updated attr sizes > Added attr16k test > Removed rm -f $seqres.full > Added filtering for SCRATCH_MNT .... > +_test_attr_replay() > +{ > + attr_name=$1 > + attr_value=$2 > + touch $testfile.1 > + > + echo "Inject error" > + _scratch_inject_error "larp" > + > + echo "Set attribute" > + echo "$attr_value" | ${ATTR_PROG} -s "$attr_name" $testfile.1 2>&1 | \ > + _filter_scratch > + > + echo "FS should be shut down, touch will fail" > + touch $testfile.1 2>&1 | _filter_scratch > + > + echo "Remount to replay log" > + _scratch_inject_logprint >> $seqres.full Huh. That function name has nothing to do with remount or dumping the log. _scratch_remount_dump_log() would at least describe what it does (indeed, it is just scratch_unmount; scratch_dump_log; scratch_mount). Can you follow this up with another patch to rename _scratch_inject_logprint() to _scratch_remount_dump_log() and also do the same for the equivalent _test_inject_logprint() function? They should probably move to common/xfs from common/inject, too... > + > + echo "FS should be online, touch should succeed" > + touch $testfile.1 > + > + echo "Verify attr recovery" > + _getfattr --absolute-names $testfile.1 | _filter_scratch > + $ATTR_PROG -g $attr_name $testfile.1 | md5sum > + > + echo "" > +} Ok, so this tests just the "set" operation. FWIW, there is no need to echo test beahviour descriptions to the output file. Each of the "echo" statements here should just be comments. > + > + > +# real QA test starts here > +_supported_fs xfs > + > +_require_scratch > +_require_attrs > +_require_xfs_io_error_injection "larp" > +_require_xfs_sysfs debug/larp > + > +# turn on log attributes > +echo 1 > /sys/fs/xfs/debug/larp > + > +_scratch_unmount >/dev/null 2>&1 > + > +#attributes of increaseing sizes > +attr16="0123456789ABCDEF" > +attr64="$attr16$attr16$attr16$attr16" > +attr256="$attr64$attr64$attr64$attr64" > +attr1k="$attr256$attr256$attr256$attr256" > +attr4k="$attr1k$attr1k$attr1k$attr1k" > +attr8k="$attr4k$attr4k" > +attr16k="$attr8k$attr8k" > +attr32k="$attr16k$attr16k" > +attr64k="$attr32k$attr32k" > + > +echo "*** mkfs" > +_scratch_mkfs_xfs >/dev/null > + > +echo "*** mount FS" > +_scratch_mount > + > +testfile=$SCRATCH_MNT/testfile > +echo "*** make test file 1" > + > +_test_attr_replay "attr_name1" $attr16 > +_test_attr_replay "attr_name2" $attr64 > +_test_attr_replay "attr_name3" $attr256 > +_test_attr_replay "attr_name4" $attr1k > +_test_attr_replay "attr_name5" $attr4k > +_test_attr_replay "attr_name6" $attr8k > +_test_attr_replay "attr_name7" $attr16k > +_test_attr_replay "attr_name8" $attr32k > +_test_attr_replay "attr_name9" $attr64k Hmmm - all attributes have different names, so this only tests the "create new attribute" operation, not the "replace attribute" or "remove attribute" operations. Also, why were the given sizes chosen? It seems to me like we should be selecting the attribute sizes based on the different operations they trigger. For an empty 512 byte inode on 4kB block size fs, we have ~300 bytes available for local attr storage. Hence both attr16 and attr64 will be stored inline. attr256 will trigger sf-to-leaf transition with existing entries. attr1k will do a leaf internal addition. attr4k will be stored externally as a remote attr, as will all the remaining larger attrs. Hence this doesn't test the following cases: - empty to leaf transition on first attr insert - remote xattr insertion when empty - leaf split/addition due to filling a leaf block - extent format to btree format transistion (i.e. tree level increase) IOWs, for a 512 byte inode and 4kB block size fs, the tests really need to be: - empty, add inline attr (64 bytes) - empty, add internal attr (1kB) - empty, add remote attr (64kB) - inline, add inline attr (64 bytes) - inline, add internal attr (1kB) - inline, add remote attr (64kB) - extent, add internal attr (1kB) - extent, add multiple internal attr (inject error on split operation) - extent, add multiple internal attr (inject error on fork transition to btree format operation) - extent, add remote attr (64kB) - btree, add multiple internal (1kB) - btree, add remote attr (64kB) This covers all the different attr fork storage forms and transitions between the different forms. We then need to cover the same cases but in reverse for attr removal (e.g. recovery of leaf merge operations, btree to extent form conversion, etc). We also need to have coverage of attr overwrite recovery of all the attr formats (shortform, leaf internal and remote) because these both add and remove attributes of the same name. We probably want different points of error injection for these so that we can force it to recover from different points in the replace operation... Cheers, Dave.
On 9/1/21 4:36 PM, Dave Chinner wrote: > On Wed, Sep 01, 2021 at 10:10:06PM +0000, Catherine Hoang wrote: >> From: Allison Henderson <allison.henderson@oracle.com> >> >> This patch adds a test to exercise the log attribute error >> inject and log replay. Attributes are added in increaseing >> sizes up to 64k, and the error inject is used to replay them >> from the log >> >> Signed-off-by: Allison Henderson <allison.henderson@oracle.com> >> Signed-off-by: Catherine Hoang <catherine.hoang@oracle.com> >> --- >> V2: Updated attr sizes >> Added attr16k test >> Removed rm -f $seqres.full >> Added filtering for SCRATCH_MNT > .... >> +_test_attr_replay() >> +{ >> + attr_name=$1 >> + attr_value=$2 >> + touch $testfile.1 >> + >> + echo "Inject error" >> + _scratch_inject_error "larp" >> + >> + echo "Set attribute" >> + echo "$attr_value" | ${ATTR_PROG} -s "$attr_name" $testfile.1 2>&1 | \ >> + _filter_scratch >> + >> + echo "FS should be shut down, touch will fail" >> + touch $testfile.1 2>&1 | _filter_scratch >> + >> + echo "Remount to replay log" >> + _scratch_inject_logprint >> $seqres.full > > Huh. That function name has nothing to do with remount or dumping > the log. _scratch_remount_dump_log() would at least describe what it > does (indeed, it is just scratch_unmount; scratch_dump_log; > scratch_mount). Can you follow this up with another patch to > rename _scratch_inject_logprint() to _scratch_remount_dump_log() > and also do the same for the equivalent _test_inject_logprint() > function? They should probably move to common/xfs from > common/inject, too... Sure, we can do a separate clean up patch for that > >> + >> + echo "FS should be online, touch should succeed" >> + touch $testfile.1 >> + >> + echo "Verify attr recovery" >> + _getfattr --absolute-names $testfile.1 | _filter_scratch >> + $ATTR_PROG -g $attr_name $testfile.1 | md5sum >> + >> + echo "" >> +} > > Ok, so this tests just the "set" operation. > > FWIW, there is no need to echo test beahviour descriptions to the > output file. Each of the "echo" statements here should just be > comments. Ok, we can turn the echos into comments then > >> + >> + >> +# real QA test starts here >> +_supported_fs xfs >> + >> +_require_scratch >> +_require_attrs >> +_require_xfs_io_error_injection "larp" >> +_require_xfs_sysfs debug/larp >> + >> +# turn on log attributes >> +echo 1 > /sys/fs/xfs/debug/larp >> + >> +_scratch_unmount >/dev/null 2>&1 >> + >> +#attributes of increaseing sizes >> +attr16="0123456789ABCDEF" >> +attr64="$attr16$attr16$attr16$attr16" >> +attr256="$attr64$attr64$attr64$attr64" >> +attr1k="$attr256$attr256$attr256$attr256" >> +attr4k="$attr1k$attr1k$attr1k$attr1k" >> +attr8k="$attr4k$attr4k" >> +attr16k="$attr8k$attr8k" >> +attr32k="$attr16k$attr16k" >> +attr64k="$attr32k$attr32k" >> + >> +echo "*** mkfs" >> +_scratch_mkfs_xfs >/dev/null >> + >> +echo "*** mount FS" >> +_scratch_mount >> + >> +testfile=$SCRATCH_MNT/testfile >> +echo "*** make test file 1" >> + >> +_test_attr_replay "attr_name1" $attr16 >> +_test_attr_replay "attr_name2" $attr64 >> +_test_attr_replay "attr_name3" $attr256 >> +_test_attr_replay "attr_name4" $attr1k >> +_test_attr_replay "attr_name5" $attr4k >> +_test_attr_replay "attr_name6" $attr8k >> +_test_attr_replay "attr_name7" $attr16k >> +_test_attr_replay "attr_name8" $attr32k >> +_test_attr_replay "attr_name9" $attr64k > > Hmmm - all attributes have different names, so this only tests > the "create new attribute" operation, not the "replace attribute" > or "remove attribute" operations. > > Also, why were the given sizes chosen? It seems to me like we should > be selecting the attribute sizes based on the different operations > they trigger. > > For an empty 512 byte inode on 4kB block size fs, we have ~300 bytes > available for local attr storage. Hence both attr16 and attr64 will > be stored inline. attr256 will trigger sf-to-leaf transition with > existing entries. attr1k will do a leaf internal addition. attr4k > will be stored externally as a remote attr, as will all the > remaining larger attrs. > > Hence this doesn't test the following cases: > - empty to leaf transition on first attr insert > - remote xattr insertion when empty > - leaf split/addition due to filling a leaf block > - extent format to btree format transistion (i.e. tree level > increase) > > IOWs, for a 512 byte inode and 4kB block size fs, the tests really > need to be: > > - empty, add inline attr (64 bytes) > - empty, add internal attr (1kB) > - empty, add remote attr (64kB) > - inline, add inline attr (64 bytes) > - inline, add internal attr (1kB) > - inline, add remote attr (64kB) > - extent, add internal attr (1kB) > - extent, add multiple internal attr (inject error on split operation) > - extent, add multiple internal attr (inject error on fork > transition to btree format operation) > - extent, add remote attr (64kB) > - btree, add multiple internal (1kB) > - btree, add remote attr (64kB) > > This covers all the different attr fork storage forms and > transitions between the different forms. Ok, so if I'm understanding this description correctly, I think we can just add a file name parameter to the _test_attr_replay function, and then modify the test calls to look something like this: # test empty attr add touch empty_file1 touch empty_file2 touch empty_file3 _test_attr_replay empty_file1 "attr_name" $attr64 _test_attr_replay empty_file2 "attr_name" $attr1k _test_attr_replay empty_file3 "attr_name" $attr64k # test inline attr add touch inline_file _test_attr_replay inline_file "attr_name1" $attr64 _test_attr_replay inline_file "attr_name2" $attr1k _test_attr_replay inline_file "attr_name3" $attr64k _test_attr_replay inline_file "attr_name4" $attr1k # test split on leaf touch leaf_file echo "$attr16$att64" | ${ATTR_PROG} -s "attr_name" leaf_file 2>&1 | \ _filter_scratch _test_attr_replay inline_file "attr_name2" $attr256 # test fork transition touch fork_file echo "$attr1k" | ${ATTR_PROG} -s "attr_name" fork_file 2>&1 | \ _filter_scratch _test_attr_replay fork_file "attr_name2" $4k #test remote attr touch remote_file _test_attr_replay remote_file "attr_name1" $attr64k _test_attr_replay remote_file "attr_name2" $attr1k _test_attr_replay remote_file "attr_name3" $attr64k Does that reflect what you are meaning to describe? > > We then need to cover the same cases but in reverse for attr removal > (e.g. recovery of leaf merge operations, btree to extent form > conversion, etc). Sure, we can add a _test_attr_rmv_replay. Maybe _test_attr_replay should be renamed _test_attr_set_replay too. > > We also need to have coverage of attr overwrite recovery of all the > attr formats (shortform, leaf internal and remote) because these > both add and remove attributes of the same name. We probably want > different points of error injection for these so that we can force > it to recover from different points in the replace operation... So you want more error tags? Maybe one for the shortform, leaf and node? Thanks! Allison > > Cheers, > > Dave. >
diff --git a/tests/xfs/540 b/tests/xfs/540 new file mode 100755 index 00000000..b2fdc153 --- /dev/null +++ b/tests/xfs/540 @@ -0,0 +1,101 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Oracle and/or its affiliates. All Rights Reserved. +# +# FS QA Test 540 +# +# Log attribute replay test +# +. ./common/preamble +_begin_fstest auto quick attr + +# get standard environment, filters and checks +. ./common/filter +. ./common/attr +. ./common/inject + +_cleanup() +{ + echo "*** unmount" + _scratch_unmount 2>/dev/null + rm -f $tmp.* + echo 0 > /sys/fs/xfs/debug/larp +} + +_test_attr_replay() +{ + attr_name=$1 + attr_value=$2 + touch $testfile.1 + + echo "Inject error" + _scratch_inject_error "larp" + + echo "Set attribute" + echo "$attr_value" | ${ATTR_PROG} -s "$attr_name" $testfile.1 2>&1 | \ + _filter_scratch + + echo "FS should be shut down, touch will fail" + touch $testfile.1 2>&1 | _filter_scratch + + echo "Remount to replay log" + _scratch_inject_logprint >> $seqres.full + + echo "FS should be online, touch should succeed" + touch $testfile.1 + + echo "Verify attr recovery" + _getfattr --absolute-names $testfile.1 | _filter_scratch + $ATTR_PROG -g $attr_name $testfile.1 | md5sum + + echo "" +} + + +# real QA test starts here +_supported_fs xfs + +_require_scratch +_require_attrs +_require_xfs_io_error_injection "larp" +_require_xfs_sysfs debug/larp + +# turn on log attributes +echo 1 > /sys/fs/xfs/debug/larp + +_scratch_unmount >/dev/null 2>&1 + +#attributes of increaseing sizes +attr16="0123456789ABCDEF" +attr64="$attr16$attr16$attr16$attr16" +attr256="$attr64$attr64$attr64$attr64" +attr1k="$attr256$attr256$attr256$attr256" +attr4k="$attr1k$attr1k$attr1k$attr1k" +attr8k="$attr4k$attr4k" +attr16k="$attr8k$attr8k" +attr32k="$attr16k$attr16k" +attr64k="$attr32k$attr32k" + +echo "*** mkfs" +_scratch_mkfs_xfs >/dev/null + +echo "*** mount FS" +_scratch_mount + +testfile=$SCRATCH_MNT/testfile +echo "*** make test file 1" + +_test_attr_replay "attr_name1" $attr16 +_test_attr_replay "attr_name2" $attr64 +_test_attr_replay "attr_name3" $attr256 +_test_attr_replay "attr_name4" $attr1k +_test_attr_replay "attr_name5" $attr4k +_test_attr_replay "attr_name6" $attr8k +_test_attr_replay "attr_name7" $attr16k +_test_attr_replay "attr_name8" $attr32k +_test_attr_replay "attr_name9" $attr64k + + +echo "*** done" +status=0 +exit diff --git a/tests/xfs/540.out b/tests/xfs/540.out new file mode 100755 index 00000000..e5f39ccf --- /dev/null +++ b/tests/xfs/540.out @@ -0,0 +1,168 @@ +QA output created by 540 +*** mkfs +*** mount FS +*** make test file 1 +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name1" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 + +5a5e91e29ed5e8aa7a30547754b9e1ee - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name2" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 + +1d61ccb38292dc01ebe9ea93bbd2564f - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name3" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 + +f513471cb87436a3df3fb930d5babb9f - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name4" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 + +0bc08fada39bf76dc83c856ee2a7d7d5 - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name5" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 +user.attr_name5 + +258b2457eed1ce94e61168c734948198 - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name6" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 +user.attr_name5 +user.attr_name6 + +507b6ac60f89d347160ddc1be73193ad - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name7" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 +user.attr_name5 +user.attr_name6 +user.attr_name7 + +a856751ea45e3c121b7b4f1fa423defc - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name8" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 +user.attr_name5 +user.attr_name6 +user.attr_name7 +user.attr_name8 + +150bffbd81292b5f239923f6a54c0c1a - + +Inject error +Set attribute +attr_set: Input/output error +Could not set "attr_name9" for SCRATCH_MNT/testfile.1 +FS should be shut down, touch will fail +touch: cannot touch 'SCRATCH_MNT/testfile.1': Input/output error +Remount to replay log +FS should be online, touch should succeed +Verify attr recovery +# file: SCRATCH_MNT/testfile.1 +user.attr_name1 +user.attr_name2 +user.attr_name3 +user.attr_name4 +user.attr_name5 +user.attr_name6 +user.attr_name7 +user.attr_name8 +user.attr_name9 + +29133253befceb131c3b736f2687cff9 - + +*** done +*** unmount