| Message ID | 1512057927-15022-1-git-send-email-nborisov@suse.com (mailing list archive) |
|---|---|
| State | Accepted, archived |
| Headers | show |
On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > Fiemap gained support for passing in optional offset len > which denote the range requested, so this patch adds > testcases for this functionality. Aditionally, a special "ranged" > argument is added to the require_xfs_io_command which checks > for the presence of fiemap range support. > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> > --- > > For the time being this test is expected to fail on XFS. > > V7: > * Adjusted for 64k block size filesystem > * Only use/require a test device and not a scratch device v7 looks good to me, test passes on ppc64 hosts too, thanks! But it seems that it's not decided yet whether xfs should trim the ranged fiemap query request, I'd like to see the expected behavior decided first. (BTW, v7 passes with latest RHEL7 kernel too, seems extents are not trimmed there.) Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Dec 05, 2017 at 04:24:06PM +0800, Eryu Guan wrote: > On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > > Fiemap gained support for passing in optional offset len > > which denote the range requested, so this patch adds > > testcases for this functionality. Aditionally, a special "ranged" > > argument is added to the require_xfs_io_command which checks > > for the presence of fiemap range support. > > > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> > > --- > > > > For the time being this test is expected to fail on XFS. > > > > V7: > > * Adjusted for 64k block size filesystem > > * Only use/require a test device and not a scratch device > > v7 looks good to me, test passes on ppc64 hosts too, thanks! > > But it seems that it's not decided yet whether xfs should trim the > ranged fiemap query request, I'd like to see the expected behavior > decided first. (BTW, v7 passes with latest RHEL7 kernel too, seems > extents are not trimmed there.) The iomap callers need to tell iomap_apply if they want exactly the range they asked for and no more (most of the write paths) or if they can handle outside ranges (fiemap), and then we can turn on XFS_BMAPI_ENTIRE reporting in xfs_iomap_begin. --D > Thanks, > Eryu > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Dec 05, 2017 at 08:51:27AM -0800, Darrick J. Wong wrote: > On Tue, Dec 05, 2017 at 04:24:06PM +0800, Eryu Guan wrote: > > On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > > > Fiemap gained support for passing in optional offset len > > > which denote the range requested, so this patch adds > > > testcases for this functionality. Aditionally, a special "ranged" > > > argument is added to the require_xfs_io_command which checks > > > for the presence of fiemap range support. > > > > > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> > > > --- > > > > > > For the time being this test is expected to fail on XFS. > > > > > > V7: > > > * Adjusted for 64k block size filesystem > > > * Only use/require a test device and not a scratch device > > > > v7 looks good to me, test passes on ppc64 hosts too, thanks! > > > > But it seems that it's not decided yet whether xfs should trim the > > ranged fiemap query request, I'd like to see the expected behavior > > decided first. (BTW, v7 passes with latest RHEL7 kernel too, seems > > extents are not trimmed there.) > > The iomap callers need to tell iomap_apply if they want exactly the > range they asked for and no more (most of the write paths) or if they > can handle outside ranges (fiemap), and then we can turn on > XFS_BMAPI_ENTIRE reporting in xfs_iomap_begin. That means.. I can merge the test as-is? Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > Fiemap gained support for passing in optional offset len > which denote the range requested, so this patch adds > testcases for this functionality. Aditionally, a special "ranged" > argument is added to the require_xfs_io_command which checks > for the presence of fiemap range support. > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> > --- > > For the time being this test is expected to fail on XFS. > > V7: > * Adjusted for 64k block size filesystem > * Only use/require a test device and not a scratch device > > V6: > * Moved to generic tests and successfully ran against ext4/btrfs > * Use pwrite to create the holes to make it even more generic > > V5: > * Drop changes to existing generic punch hole tests since > the new fiemap implementation don't require them > * Merge the common/rc change with this patch > * Added Data + Hole + Data and Hole + Data tests as per Eryu's request > * Adjusted output of some tests which fall in a hole, since holes are > truncated to passed range > * Simplified the logic to check for fiemap range support (Eryu) > > V4: > * Added test description > * Added new test for past-eof behavior > * Removed supported_generic_fs line > * Switched to using the "ranged" param require > * Revert v3 changes > > V3: > * Adjusted tests for '-r' fiemap param > * Tests for invalid -r combination > > V2: No change > V1: No change > > common/rc | 7 ++ > tests/generic/900 | 107 +++++++++++++++++++++++++++ > tests/generic/900.out | 196 ++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/generic/group | 1 + > 4 files changed, 311 insertions(+) > create mode 100755 tests/generic/900 > create mode 100644 tests/generic/900.out > > diff --git a/common/rc b/common/rc > index 4c053a5..a0d162a 100644 > --- a/common/rc > +++ b/common/rc > @@ -2056,6 +2056,13 @@ _require_xfs_io_command() > -c "$command 4k 8k" $testfile 2>&1` > ;; > "fiemap") > + # If 'ranged' is passed as argument then we check to see if fiemap supports > + # ranged query params > + if echo "$param" | grep -q "ranged"; then > + param=$(echo "$param" | sed "s/ranged//") > + $XFS_IO_PROG -c "help fiemap" | grep -q "\[offset \[len\]\]" > + [ $? -eq 0 ] || _notrun "xfs_io $command ranged support is missing" > + fi > testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \ > -c "fiemap -v $param" $testfile 2>&1` > param_checked=1 > diff --git a/tests/generic/900 b/tests/generic/900 > new file mode 100755 > index 0000000..3f3019b > --- /dev/null > +++ b/tests/generic/900 > @@ -0,0 +1,107 @@ > +#! /bin/bash > +# FS QA Test No. 900 > + > +# Test for the new ranged query functionality in xfs_io's fiemap command. > +# This tests various combinations of hole + data layout being printed. > +# Also the test used 16k holes to be compatible with 16k block filesystems 64k blocks? > +#----------------------------------------------------------------------- > +# Copyright (c) 2017 SUSE Linux Products GmbH. All Rights Reserved. > +# Author: Nikolay Borisov <nborisov@suse.com> > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/punch > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_os Linux > +_require_test > +_require_xfs_io_command "truncate" > +# ranged is a special argument which checks if fiemap supports > +# [offset [len]] args > +_require_xfs_io_command "fiemap" "ranged" > + > +file=$TEST_DIR/fiemap.$seq > +rm -f $file > + > +# Create a file with 64k hole followed by 64k data, and this pattern > +# repeats till it reaches 4M file size, so each extent has 64k data. > +# But truncate file to its final size first, otherwise XFS would merge > +# some extents due to speculative preallocation. > +$XFS_IO_PROG -f -c "truncate 4m" $file > +for i in {0..31}; do > + $XFS_IO_PROG -c "pwrite $(($i*128+64))k 64k" $file >/dev/null; > +done > + > +# Query 1 data extent between 64k..64k range > +echo "Basic data extent" > +$XFS_IO_PROG -c "fiemap -v 64k 64k" $file | _filter_fiemap > + > +# Query data and hole extent > +echo "Data + Hole" > +$XFS_IO_PROG -c "fiemap -v 64k 80k" $file | _filter_fiemap > + > +echo "Hole + Data" > +$XFS_IO_PROG -c "fiemap -v 0 65k" $file | _filter_fiemap > + > +echo "Hole + Data + Hole" > +$XFS_IO_PROG -c "fiemap -v 0k 130k" $file | _filter_fiemap > + > +echo "Data + Hole + Data" > +$XFS_IO_PROG -c "fiemap -v 64k 192k" $file | _filter_fiemap > + > +echo "Beginning with a hole" > +$XFS_IO_PROG -c "fiemap -v 0 3k" $file | _filter_fiemap > + > +# Query for 0..160k that's 40 extents, more than the EXTENT_BATCH > +echo "Query more than 32 extents" > +$XFS_IO_PROG -c "fiemap -v 0 3m" $file | _filter_fiemap > + > +echo "Larger query than file size" > +$XFS_IO_PROG -c "fiemap -v 0 5m" $file | _filter_fiemap > + > +# mapping past eof shouldn't print anything" > +$XFS_IO_PROG -c "fiemap -v 5m" $file | _filter_fiemap > + > +echo "Skip first hole" > +# check everything without the first hole > +$XFS_IO_PROG -c "fiemap -v 64k" $file | _filter_fiemap > + > +# success, all done > +status=0 > +exit > diff --git a/tests/generic/900.out b/tests/generic/900.out > new file mode 100644 > index 0000000..3df6a3a > --- /dev/null > +++ b/tests/generic/900.out > @@ -0,0 +1,196 @@ > +QA output created by 900 > +Basic data extent > +0: [128..255]: data > +Data + Hole > +0: [128..255]: data > +1: [256..287]: hole > +Hole + Data > +0: [0..127]: hole > +1: [128..255]: data This fails on XFS: $ xfs_io -c "fiemap -v 0 65k" file moo: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..127]: hole 128 1: [128..135]: 74391184..74391191 8 0x0 (Note the records are trimmed to sector 135, or 65k) Compare to ext4: $ xfs_io -c "fiemap -v 0k 65k" file moo: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..127]: hole 128 1: [128..255]: 2510790784..2510790911 128 0x0 (ext4 doesn't bother trimming) Documentation/filesystems/fiemap.txt says "...the logical offset of the 1st returned extent may start before fm_start, and the range covered by the last returned extent may end after fm_length", so both behaviors are allowed. --D > +Hole + Data + Hole > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..259]: hole > +Data + Hole + Data > +0: [128..255]: data > +1: [256..383]: hole > +2: [384..511]: data > +Beginning with a hole > +Query more than 32 extents > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..383]: hole > +3: [384..511]: data > +4: [512..639]: hole > +5: [640..767]: data > +6: [768..895]: hole > +7: [896..1023]: data > +8: [1024..1151]: hole > +9: [1152..1279]: data > +10: [1280..1407]: hole > +11: [1408..1535]: data > +12: [1536..1663]: hole > +13: [1664..1791]: data > +14: [1792..1919]: hole > +15: [1920..2047]: data > +16: [2048..2175]: hole > +17: [2176..2303]: data > +18: [2304..2431]: hole > +19: [2432..2559]: data > +20: [2560..2687]: hole > +21: [2688..2815]: data > +22: [2816..2943]: hole > +23: [2944..3071]: data > +24: [3072..3199]: hole > +25: [3200..3327]: data > +26: [3328..3455]: hole > +27: [3456..3583]: data > +28: [3584..3711]: hole > +29: [3712..3839]: data > +30: [3840..3967]: hole > +31: [3968..4095]: data > +32: [4096..4223]: hole > +33: [4224..4351]: data > +34: [4352..4479]: hole > +35: [4480..4607]: data > +36: [4608..4735]: hole > +37: [4736..4863]: data > +38: [4864..4991]: hole > +39: [4992..5119]: data > +40: [5120..5247]: hole > +41: [5248..5375]: data > +42: [5376..5503]: hole > +43: [5504..5631]: data > +44: [5632..5759]: hole > +45: [5760..5887]: data > +46: [5888..6015]: hole > +47: [6016..6143]: data > +Larger query than file size > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..383]: hole > +3: [384..511]: data > +4: [512..639]: hole > +5: [640..767]: data > +6: [768..895]: hole > +7: [896..1023]: data > +8: [1024..1151]: hole > +9: [1152..1279]: data > +10: [1280..1407]: hole > +11: [1408..1535]: data > +12: [1536..1663]: hole > +13: [1664..1791]: data > +14: [1792..1919]: hole > +15: [1920..2047]: data > +16: [2048..2175]: hole > +17: [2176..2303]: data > +18: [2304..2431]: hole > +19: [2432..2559]: data > +20: [2560..2687]: hole > +21: [2688..2815]: data > +22: [2816..2943]: hole > +23: [2944..3071]: data > +24: [3072..3199]: hole > +25: [3200..3327]: data > +26: [3328..3455]: hole > +27: [3456..3583]: data > +28: [3584..3711]: hole > +29: [3712..3839]: data > +30: [3840..3967]: hole > +31: [3968..4095]: data > +32: [4096..4223]: hole > +33: [4224..4351]: data > +34: [4352..4479]: hole > +35: [4480..4607]: data > +36: [4608..4735]: hole > +37: [4736..4863]: data > +38: [4864..4991]: hole > +39: [4992..5119]: data > +40: [5120..5247]: hole > +41: [5248..5375]: data > +42: [5376..5503]: hole > +43: [5504..5631]: data > +44: [5632..5759]: hole > +45: [5760..5887]: data > +46: [5888..6015]: hole > +47: [6016..6143]: data > +48: [6144..6271]: hole > +49: [6272..6399]: data > +50: [6400..6527]: hole > +51: [6528..6655]: data > +52: [6656..6783]: hole > +53: [6784..6911]: data > +54: [6912..7039]: hole > +55: [7040..7167]: data > +56: [7168..7295]: hole > +57: [7296..7423]: data > +58: [7424..7551]: hole > +59: [7552..7679]: data > +60: [7680..7807]: hole > +61: [7808..7935]: data > +62: [7936..8063]: hole > +63: [8064..8191]: data > +Skip first hole > +0: [128..255]: data > +1: [256..383]: hole > +2: [384..511]: data > +3: [512..639]: hole > +4: [640..767]: data > +5: [768..895]: hole > +6: [896..1023]: data > +7: [1024..1151]: hole > +8: [1152..1279]: data > +9: [1280..1407]: hole > +10: [1408..1535]: data > +11: [1536..1663]: hole > +12: [1664..1791]: data > +13: [1792..1919]: hole > +14: [1920..2047]: data > +15: [2048..2175]: hole > +16: [2176..2303]: data > +17: [2304..2431]: hole > +18: [2432..2559]: data > +19: [2560..2687]: hole > +20: [2688..2815]: data > +21: [2816..2943]: hole > +22: [2944..3071]: data > +23: [3072..3199]: hole > +24: [3200..3327]: data > +25: [3328..3455]: hole > +26: [3456..3583]: data > +27: [3584..3711]: hole > +28: [3712..3839]: data > +29: [3840..3967]: hole > +30: [3968..4095]: data > +31: [4096..4223]: hole > +32: [4224..4351]: data > +33: [4352..4479]: hole > +34: [4480..4607]: data > +35: [4608..4735]: hole > +36: [4736..4863]: data > +37: [4864..4991]: hole > +38: [4992..5119]: data > +39: [5120..5247]: hole > +40: [5248..5375]: data > +41: [5376..5503]: hole > +42: [5504..5631]: data > +43: [5632..5759]: hole > +44: [5760..5887]: data > +45: [5888..6015]: hole > +46: [6016..6143]: data > +47: [6144..6271]: hole > +48: [6272..6399]: data > +49: [6400..6527]: hole > +50: [6528..6655]: data > +51: [6656..6783]: hole > +52: [6784..6911]: data > +53: [6912..7039]: hole > +54: [7040..7167]: data > +55: [7168..7295]: hole > +56: [7296..7423]: data > +57: [7424..7551]: hole > +58: [7552..7679]: data > +59: [7680..7807]: hole > +60: [7808..7935]: data > +61: [7936..8063]: hole > +62: [8064..8191]: data > diff --git a/tests/generic/group b/tests/generic/group > index 6c3bb03..a043e98 100644 > --- a/tests/generic/group > +++ b/tests/generic/group > @@ -472,3 +472,4 @@ > 467 auto quick exportfs > 468 shutdown auto quick metadata > 469 auto quick > +900 auto quick > -- > 2.7.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 6.12.2017 19:45, Darrick J. Wong wrote: > On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: >> Fiemap gained support for passing in optional offset len >> which denote the range requested, so this patch adds >> testcases for this functionality. Aditionally, a special "ranged" >> argument is added to the require_xfs_io_command which checks >> for the presence of fiemap range support. >> >> Signed-off-by: Nikolay Borisov <nborisov@suse.com> >> --- >> >> For the time being this test is expected to fail on XFS. >> >> V7: >> * Adjusted for 64k block size filesystem >> * Only use/require a test device and not a scratch device >> >> V6: >> * Moved to generic tests and successfully ran against ext4/btrfs >> * Use pwrite to create the holes to make it even more generic >> >> V5: >> * Drop changes to existing generic punch hole tests since >> the new fiemap implementation don't require them >> * Merge the common/rc change with this patch >> * Added Data + Hole + Data and Hole + Data tests as per Eryu's request >> * Adjusted output of some tests which fall in a hole, since holes are >> truncated to passed range >> * Simplified the logic to check for fiemap range support (Eryu) >> >> V4: >> * Added test description >> * Added new test for past-eof behavior >> * Removed supported_generic_fs line >> * Switched to using the "ranged" param require >> * Revert v3 changes >> >> V3: >> * Adjusted tests for '-r' fiemap param >> * Tests for invalid -r combination >> >> V2: No change >> V1: No change >> >> common/rc | 7 ++ >> tests/generic/900 | 107 +++++++++++++++++++++++++++ >> tests/generic/900.out | 196 ++++++++++++++++++++++++++++++++++++++++++++++++++ >> tests/generic/group | 1 + >> 4 files changed, 311 insertions(+) >> create mode 100755 tests/generic/900 >> create mode 100644 tests/generic/900.out >> >> diff --git a/common/rc b/common/rc >> index 4c053a5..a0d162a 100644 >> --- a/common/rc >> +++ b/common/rc >> @@ -2056,6 +2056,13 @@ _require_xfs_io_command() >> -c "$command 4k 8k" $testfile 2>&1` >> ;; >> "fiemap") >> + # If 'ranged' is passed as argument then we check to see if fiemap supports >> + # ranged query params >> + if echo "$param" | grep -q "ranged"; then >> + param=$(echo "$param" | sed "s/ranged//") >> + $XFS_IO_PROG -c "help fiemap" | grep -q "\[offset \[len\]\]" >> + [ $? -eq 0 ] || _notrun "xfs_io $command ranged support is missing" >> + fi >> testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \ >> -c "fiemap -v $param" $testfile 2>&1` >> param_checked=1 >> diff --git a/tests/generic/900 b/tests/generic/900 >> new file mode 100755 >> index 0000000..3f3019b >> --- /dev/null >> +++ b/tests/generic/900 >> @@ -0,0 +1,107 @@ >> +#! /bin/bash >> +# FS QA Test No. 900 >> + >> +# Test for the new ranged query functionality in xfs_io's fiemap command. >> +# This tests various combinations of hole + data layout being printed. >> +# Also the test used 16k holes to be compatible with 16k block filesystems > > 64k blocks? > >> +#----------------------------------------------------------------------- >> +# Copyright (c) 2017 SUSE Linux Products GmbH. All Rights Reserved. >> +# Author: Nikolay Borisov <nborisov@suse.com> >> +# >> +# This program is free software; you can redistribute it and/or >> +# modify it under the terms of the GNU General Public License as >> +# published by the Free Software Foundation. >> +# >> +# This program is distributed in the hope that it would be useful, >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> +# GNU General Public License for more details. >> +# >> +# You should have received a copy of the GNU General Public License >> +# along with this program; if not, write the Free Software Foundation, >> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA >> +#----------------------------------------------------------------------- >> +# >> + >> +seq=`basename $0` >> +seqres=$RESULT_DIR/$seq >> +echo "QA output created by $seq" >> + >> +here=`pwd` >> +tmp=/tmp/$$ >> +status=1 # failure is the default! >> +trap "_cleanup; exit \$status" 0 1 2 3 15 >> + >> +_cleanup() >> +{ >> + cd / >> + rm -f $tmp.* >> +} >> + >> +# get standard environment, filters and checks >> +. ./common/rc >> +. ./common/punch >> + >> +# remove previous $seqres.full before test >> +rm -f $seqres.full >> + >> +# real QA test starts here >> + >> +# Modify as appropriate. >> +_supported_os Linux >> +_require_test >> +_require_xfs_io_command "truncate" >> +# ranged is a special argument which checks if fiemap supports >> +# [offset [len]] args >> +_require_xfs_io_command "fiemap" "ranged" >> + >> +file=$TEST_DIR/fiemap.$seq >> +rm -f $file >> + >> +# Create a file with 64k hole followed by 64k data, and this pattern >> +# repeats till it reaches 4M file size, so each extent has 64k data. >> +# But truncate file to its final size first, otherwise XFS would merge >> +# some extents due to speculative preallocation. >> +$XFS_IO_PROG -f -c "truncate 4m" $file >> +for i in {0..31}; do >> + $XFS_IO_PROG -c "pwrite $(($i*128+64))k 64k" $file >/dev/null; >> +done >> + >> +# Query 1 data extent between 64k..64k range >> +echo "Basic data extent" >> +$XFS_IO_PROG -c "fiemap -v 64k 64k" $file | _filter_fiemap >> + >> +# Query data and hole extent >> +echo "Data + Hole" >> +$XFS_IO_PROG -c "fiemap -v 64k 80k" $file | _filter_fiemap >> + >> +echo "Hole + Data" >> +$XFS_IO_PROG -c "fiemap -v 0 65k" $file | _filter_fiemap >> + >> +echo "Hole + Data + Hole" >> +$XFS_IO_PROG -c "fiemap -v 0k 130k" $file | _filter_fiemap >> + >> +echo "Data + Hole + Data" >> +$XFS_IO_PROG -c "fiemap -v 64k 192k" $file | _filter_fiemap >> + >> +echo "Beginning with a hole" >> +$XFS_IO_PROG -c "fiemap -v 0 3k" $file | _filter_fiemap >> + >> +# Query for 0..160k that's 40 extents, more than the EXTENT_BATCH >> +echo "Query more than 32 extents" >> +$XFS_IO_PROG -c "fiemap -v 0 3m" $file | _filter_fiemap >> + >> +echo "Larger query than file size" >> +$XFS_IO_PROG -c "fiemap -v 0 5m" $file | _filter_fiemap >> + >> +# mapping past eof shouldn't print anything" >> +$XFS_IO_PROG -c "fiemap -v 5m" $file | _filter_fiemap >> + >> +echo "Skip first hole" >> +# check everything without the first hole >> +$XFS_IO_PROG -c "fiemap -v 64k" $file | _filter_fiemap >> + >> +# success, all done >> +status=0 >> +exit >> diff --git a/tests/generic/900.out b/tests/generic/900.out >> new file mode 100644 >> index 0000000..3df6a3a >> --- /dev/null >> +++ b/tests/generic/900.out >> @@ -0,0 +1,196 @@ >> +QA output created by 900 >> +Basic data extent >> +0: [128..255]: data >> +Data + Hole >> +0: [128..255]: data >> +1: [256..287]: hole >> +Hole + Data >> +0: [0..127]: hole >> +1: [128..255]: data > > This fails on XFS: > > $ xfs_io -c "fiemap -v 0 65k" file > moo: > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > 0: [0..127]: hole 128 > 1: [128..135]: 74391184..74391191 8 0x0 > > (Note the records are trimmed to sector 135, or 65k) > > Compare to ext4: > > $ xfs_io -c "fiemap -v 0k 65k" file > moo: > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > 0: [0..127]: hole 128 > 1: [128..255]: 2510790784..2510790911 128 0x0 > > (ext4 doesn't bother trimming) > > Documentation/filesystems/fiemap.txt says "...the logical offset of the > 1st returned extent may start before fm_start, and the range covered by > the last returned extent may end after fm_length", so both behaviors are > allowed. So how do you propose the test to deal with the fact that XFS is the only file system (of the mainstream ones at least) that trims the extents? > > --D > >> +Hole + Data + Hole >> +0: [0..127]: hole >> +1: [128..255]: data >> +2: [256..259]: hole >> +Data + Hole + Data >> +0: [128..255]: data >> +1: [256..383]: hole >> +2: [384..511]: data >> +Beginning with a hole >> +Query more than 32 extents >> +0: [0..127]: hole >> +1: [128..255]: data >> +2: [256..383]: hole >> +3: [384..511]: data >> +4: [512..639]: hole >> +5: [640..767]: data >> +6: [768..895]: hole >> +7: [896..1023]: data >> +8: [1024..1151]: hole >> +9: [1152..1279]: data >> +10: [1280..1407]: hole >> +11: [1408..1535]: data >> +12: [1536..1663]: hole >> +13: [1664..1791]: data >> +14: [1792..1919]: hole >> +15: [1920..2047]: data >> +16: [2048..2175]: hole >> +17: [2176..2303]: data >> +18: [2304..2431]: hole >> +19: [2432..2559]: data >> +20: [2560..2687]: hole >> +21: [2688..2815]: data >> +22: [2816..2943]: hole >> +23: [2944..3071]: data >> +24: [3072..3199]: hole >> +25: [3200..3327]: data >> +26: [3328..3455]: hole >> +27: [3456..3583]: data >> +28: [3584..3711]: hole >> +29: [3712..3839]: data >> +30: [3840..3967]: hole >> +31: [3968..4095]: data >> +32: [4096..4223]: hole >> +33: [4224..4351]: data >> +34: [4352..4479]: hole >> +35: [4480..4607]: data >> +36: [4608..4735]: hole >> +37: [4736..4863]: data >> +38: [4864..4991]: hole >> +39: [4992..5119]: data >> +40: [5120..5247]: hole >> +41: [5248..5375]: data >> +42: [5376..5503]: hole >> +43: [5504..5631]: data >> +44: [5632..5759]: hole >> +45: [5760..5887]: data >> +46: [5888..6015]: hole >> +47: [6016..6143]: data >> +Larger query than file size >> +0: [0..127]: hole >> +1: [128..255]: data >> +2: [256..383]: hole >> +3: [384..511]: data >> +4: [512..639]: hole >> +5: [640..767]: data >> +6: [768..895]: hole >> +7: [896..1023]: data >> +8: [1024..1151]: hole >> +9: [1152..1279]: data >> +10: [1280..1407]: hole >> +11: [1408..1535]: data >> +12: [1536..1663]: hole >> +13: [1664..1791]: data >> +14: [1792..1919]: hole >> +15: [1920..2047]: data >> +16: [2048..2175]: hole >> +17: [2176..2303]: data >> +18: [2304..2431]: hole >> +19: [2432..2559]: data >> +20: [2560..2687]: hole >> +21: [2688..2815]: data >> +22: [2816..2943]: hole >> +23: [2944..3071]: data >> +24: [3072..3199]: hole >> +25: [3200..3327]: data >> +26: [3328..3455]: hole >> +27: [3456..3583]: data >> +28: [3584..3711]: hole >> +29: [3712..3839]: data >> +30: [3840..3967]: hole >> +31: [3968..4095]: data >> +32: [4096..4223]: hole >> +33: [4224..4351]: data >> +34: [4352..4479]: hole >> +35: [4480..4607]: data >> +36: [4608..4735]: hole >> +37: [4736..4863]: data >> +38: [4864..4991]: hole >> +39: [4992..5119]: data >> +40: [5120..5247]: hole >> +41: [5248..5375]: data >> +42: [5376..5503]: hole >> +43: [5504..5631]: data >> +44: [5632..5759]: hole >> +45: [5760..5887]: data >> +46: [5888..6015]: hole >> +47: [6016..6143]: data >> +48: [6144..6271]: hole >> +49: [6272..6399]: data >> +50: [6400..6527]: hole >> +51: [6528..6655]: data >> +52: [6656..6783]: hole >> +53: [6784..6911]: data >> +54: [6912..7039]: hole >> +55: [7040..7167]: data >> +56: [7168..7295]: hole >> +57: [7296..7423]: data >> +58: [7424..7551]: hole >> +59: [7552..7679]: data >> +60: [7680..7807]: hole >> +61: [7808..7935]: data >> +62: [7936..8063]: hole >> +63: [8064..8191]: data >> +Skip first hole >> +0: [128..255]: data >> +1: [256..383]: hole >> +2: [384..511]: data >> +3: [512..639]: hole >> +4: [640..767]: data >> +5: [768..895]: hole >> +6: [896..1023]: data >> +7: [1024..1151]: hole >> +8: [1152..1279]: data >> +9: [1280..1407]: hole >> +10: [1408..1535]: data >> +11: [1536..1663]: hole >> +12: [1664..1791]: data >> +13: [1792..1919]: hole >> +14: [1920..2047]: data >> +15: [2048..2175]: hole >> +16: [2176..2303]: data >> +17: [2304..2431]: hole >> +18: [2432..2559]: data >> +19: [2560..2687]: hole >> +20: [2688..2815]: data >> +21: [2816..2943]: hole >> +22: [2944..3071]: data >> +23: [3072..3199]: hole >> +24: [3200..3327]: data >> +25: [3328..3455]: hole >> +26: [3456..3583]: data >> +27: [3584..3711]: hole >> +28: [3712..3839]: data >> +29: [3840..3967]: hole >> +30: [3968..4095]: data >> +31: [4096..4223]: hole >> +32: [4224..4351]: data >> +33: [4352..4479]: hole >> +34: [4480..4607]: data >> +35: [4608..4735]: hole >> +36: [4736..4863]: data >> +37: [4864..4991]: hole >> +38: [4992..5119]: data >> +39: [5120..5247]: hole >> +40: [5248..5375]: data >> +41: [5376..5503]: hole >> +42: [5504..5631]: data >> +43: [5632..5759]: hole >> +44: [5760..5887]: data >> +45: [5888..6015]: hole >> +46: [6016..6143]: data >> +47: [6144..6271]: hole >> +48: [6272..6399]: data >> +49: [6400..6527]: hole >> +50: [6528..6655]: data >> +51: [6656..6783]: hole >> +52: [6784..6911]: data >> +53: [6912..7039]: hole >> +54: [7040..7167]: data >> +55: [7168..7295]: hole >> +56: [7296..7423]: data >> +57: [7424..7551]: hole >> +58: [7552..7679]: data >> +59: [7680..7807]: hole >> +60: [7808..7935]: data >> +61: [7936..8063]: hole >> +62: [8064..8191]: data >> diff --git a/tests/generic/group b/tests/generic/group >> index 6c3bb03..a043e98 100644 >> --- a/tests/generic/group >> +++ b/tests/generic/group >> @@ -472,3 +472,4 @@ >> 467 auto quick exportfs >> 468 shutdown auto quick metadata >> 469 auto quick >> +900 auto quick >> -- >> 2.7.4 >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Dec 06, 2017 at 10:51:39PM +0200, Nikolay Borisov wrote: > > > On 6.12.2017 19:45, Darrick J. Wong wrote: > > On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > >> Fiemap gained support for passing in optional offset len > >> which denote the range requested, so this patch adds > >> testcases for this functionality. Aditionally, a special "ranged" > >> argument is added to the require_xfs_io_command which checks > >> for the presence of fiemap range support. > >> > >> Signed-off-by: Nikolay Borisov <nborisov@suse.com> > >> --- > >> > >> For the time being this test is expected to fail on XFS. > >> > >> V7: > >> * Adjusted for 64k block size filesystem > >> * Only use/require a test device and not a scratch device > >> > >> V6: > >> * Moved to generic tests and successfully ran against ext4/btrfs > >> * Use pwrite to create the holes to make it even more generic > >> > >> V5: > >> * Drop changes to existing generic punch hole tests since > >> the new fiemap implementation don't require them > >> * Merge the common/rc change with this patch > >> * Added Data + Hole + Data and Hole + Data tests as per Eryu's request > >> * Adjusted output of some tests which fall in a hole, since holes are > >> truncated to passed range > >> * Simplified the logic to check for fiemap range support (Eryu) > >> > >> V4: > >> * Added test description > >> * Added new test for past-eof behavior > >> * Removed supported_generic_fs line > >> * Switched to using the "ranged" param require > >> * Revert v3 changes > >> > >> V3: > >> * Adjusted tests for '-r' fiemap param > >> * Tests for invalid -r combination > >> > >> V2: No change > >> V1: No change > >> > >> common/rc | 7 ++ > >> tests/generic/900 | 107 +++++++++++++++++++++++++++ > >> tests/generic/900.out | 196 ++++++++++++++++++++++++++++++++++++++++++++++++++ > >> tests/generic/group | 1 + > >> 4 files changed, 311 insertions(+) > >> create mode 100755 tests/generic/900 > >> create mode 100644 tests/generic/900.out > >> > >> diff --git a/common/rc b/common/rc > >> index 4c053a5..a0d162a 100644 > >> --- a/common/rc > >> +++ b/common/rc > >> @@ -2056,6 +2056,13 @@ _require_xfs_io_command() > >> -c "$command 4k 8k" $testfile 2>&1` > >> ;; > >> "fiemap") > >> + # If 'ranged' is passed as argument then we check to see if fiemap supports > >> + # ranged query params > >> + if echo "$param" | grep -q "ranged"; then > >> + param=$(echo "$param" | sed "s/ranged//") > >> + $XFS_IO_PROG -c "help fiemap" | grep -q "\[offset \[len\]\]" > >> + [ $? -eq 0 ] || _notrun "xfs_io $command ranged support is missing" > >> + fi > >> testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \ > >> -c "fiemap -v $param" $testfile 2>&1` > >> param_checked=1 > >> diff --git a/tests/generic/900 b/tests/generic/900 > >> new file mode 100755 > >> index 0000000..3f3019b > >> --- /dev/null > >> +++ b/tests/generic/900 > >> @@ -0,0 +1,107 @@ > >> +#! /bin/bash > >> +# FS QA Test No. 900 > >> + > >> +# Test for the new ranged query functionality in xfs_io's fiemap command. > >> +# This tests various combinations of hole + data layout being printed. > >> +# Also the test used 16k holes to be compatible with 16k block filesystems > > > > 64k blocks? > > > >> +#----------------------------------------------------------------------- > >> +# Copyright (c) 2017 SUSE Linux Products GmbH. All Rights Reserved. > >> +# Author: Nikolay Borisov <nborisov@suse.com> > >> +# > >> +# This program is free software; you can redistribute it and/or > >> +# modify it under the terms of the GNU General Public License as > >> +# published by the Free Software Foundation. > >> +# > >> +# This program is distributed in the hope that it would be useful, > >> +# but WITHOUT ANY WARRANTY; without even the implied warranty of > >> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > >> +# GNU General Public License for more details. > >> +# > >> +# You should have received a copy of the GNU General Public License > >> +# along with this program; if not, write the Free Software Foundation, > >> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > >> +#----------------------------------------------------------------------- > >> +# > >> + > >> +seq=`basename $0` > >> +seqres=$RESULT_DIR/$seq > >> +echo "QA output created by $seq" > >> + > >> +here=`pwd` > >> +tmp=/tmp/$$ > >> +status=1 # failure is the default! > >> +trap "_cleanup; exit \$status" 0 1 2 3 15 > >> + > >> +_cleanup() > >> +{ > >> + cd / > >> + rm -f $tmp.* > >> +} > >> + > >> +# get standard environment, filters and checks > >> +. ./common/rc > >> +. ./common/punch > >> + > >> +# remove previous $seqres.full before test > >> +rm -f $seqres.full > >> + > >> +# real QA test starts here > >> + > >> +# Modify as appropriate. > >> +_supported_os Linux > >> +_require_test > >> +_require_xfs_io_command "truncate" > >> +# ranged is a special argument which checks if fiemap supports > >> +# [offset [len]] args > >> +_require_xfs_io_command "fiemap" "ranged" > >> + > >> +file=$TEST_DIR/fiemap.$seq > >> +rm -f $file > >> + > >> +# Create a file with 64k hole followed by 64k data, and this pattern > >> +# repeats till it reaches 4M file size, so each extent has 64k data. > >> +# But truncate file to its final size first, otherwise XFS would merge > >> +# some extents due to speculative preallocation. > >> +$XFS_IO_PROG -f -c "truncate 4m" $file > >> +for i in {0..31}; do > >> + $XFS_IO_PROG -c "pwrite $(($i*128+64))k 64k" $file >/dev/null; > >> +done > >> + > >> +# Query 1 data extent between 64k..64k range > >> +echo "Basic data extent" > >> +$XFS_IO_PROG -c "fiemap -v 64k 64k" $file | _filter_fiemap > >> + > >> +# Query data and hole extent > >> +echo "Data + Hole" > >> +$XFS_IO_PROG -c "fiemap -v 64k 80k" $file | _filter_fiemap > >> + > >> +echo "Hole + Data" > >> +$XFS_IO_PROG -c "fiemap -v 0 65k" $file | _filter_fiemap > >> + > >> +echo "Hole + Data + Hole" > >> +$XFS_IO_PROG -c "fiemap -v 0k 130k" $file | _filter_fiemap > >> + > >> +echo "Data + Hole + Data" > >> +$XFS_IO_PROG -c "fiemap -v 64k 192k" $file | _filter_fiemap > >> + > >> +echo "Beginning with a hole" > >> +$XFS_IO_PROG -c "fiemap -v 0 3k" $file | _filter_fiemap > >> + > >> +# Query for 0..160k that's 40 extents, more than the EXTENT_BATCH > >> +echo "Query more than 32 extents" > >> +$XFS_IO_PROG -c "fiemap -v 0 3m" $file | _filter_fiemap > >> + > >> +echo "Larger query than file size" > >> +$XFS_IO_PROG -c "fiemap -v 0 5m" $file | _filter_fiemap > >> + > >> +# mapping past eof shouldn't print anything" > >> +$XFS_IO_PROG -c "fiemap -v 5m" $file | _filter_fiemap > >> + > >> +echo "Skip first hole" > >> +# check everything without the first hole > >> +$XFS_IO_PROG -c "fiemap -v 64k" $file | _filter_fiemap > >> + > >> +# success, all done > >> +status=0 > >> +exit > >> diff --git a/tests/generic/900.out b/tests/generic/900.out > >> new file mode 100644 > >> index 0000000..3df6a3a > >> --- /dev/null > >> +++ b/tests/generic/900.out > >> @@ -0,0 +1,196 @@ > >> +QA output created by 900 > >> +Basic data extent > >> +0: [128..255]: data > >> +Data + Hole > >> +0: [128..255]: data > >> +1: [256..287]: hole > >> +Hole + Data > >> +0: [0..127]: hole > >> +1: [128..255]: data > > > > This fails on XFS: > > > > $ xfs_io -c "fiemap -v 0 65k" file > > moo: > > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > > 0: [0..127]: hole 128 > > 1: [128..135]: 74391184..74391191 8 0x0 > > > > (Note the records are trimmed to sector 135, or 65k) > > > > Compare to ext4: > > > > $ xfs_io -c "fiemap -v 0k 65k" file > > moo: > > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > > 0: [0..127]: hole 128 > > 1: [128..255]: 2510790784..2510790911 128 0x0 > > > > (ext4 doesn't bother trimming) > > > > Documentation/filesystems/fiemap.txt says "...the logical offset of the > > 1st returned extent may start before fm_start, and the range covered by > > the last returned extent may end after fm_length", so both behaviors are > > allowed. > > So how do you propose the test to deal with the fact that XFS is the > only file system (of the mainstream ones at least) that trims the extents? Put: test "$FSTYP" = "xfs" && _notrun "This is tested by xfs/XXX" in this test and then duplicate it as some xfs/ test with whatever the correct output is on xfs. Also add some checks for fiemap -a since we actually have attribute block maps on xfs. (let's see if hch/dchinner come in and NAK the whole thing on the grounds that FIEMAP is a debugging interface anyway...) --D > > > > --D > > > >> +Hole + Data + Hole > >> +0: [0..127]: hole > >> +1: [128..255]: data > >> +2: [256..259]: hole > >> +Data + Hole + Data > >> +0: [128..255]: data > >> +1: [256..383]: hole > >> +2: [384..511]: data > >> +Beginning with a hole > >> +Query more than 32 extents > >> +0: [0..127]: hole > >> +1: [128..255]: data > >> +2: [256..383]: hole > >> +3: [384..511]: data > >> +4: [512..639]: hole > >> +5: [640..767]: data > >> +6: [768..895]: hole > >> +7: [896..1023]: data > >> +8: [1024..1151]: hole > >> +9: [1152..1279]: data > >> +10: [1280..1407]: hole > >> +11: [1408..1535]: data > >> +12: [1536..1663]: hole > >> +13: [1664..1791]: data > >> +14: [1792..1919]: hole > >> +15: [1920..2047]: data > >> +16: [2048..2175]: hole > >> +17: [2176..2303]: data > >> +18: [2304..2431]: hole > >> +19: [2432..2559]: data > >> +20: [2560..2687]: hole > >> +21: [2688..2815]: data > >> +22: [2816..2943]: hole > >> +23: [2944..3071]: data > >> +24: [3072..3199]: hole > >> +25: [3200..3327]: data > >> +26: [3328..3455]: hole > >> +27: [3456..3583]: data > >> +28: [3584..3711]: hole > >> +29: [3712..3839]: data > >> +30: [3840..3967]: hole > >> +31: [3968..4095]: data > >> +32: [4096..4223]: hole > >> +33: [4224..4351]: data > >> +34: [4352..4479]: hole > >> +35: [4480..4607]: data > >> +36: [4608..4735]: hole > >> +37: [4736..4863]: data > >> +38: [4864..4991]: hole > >> +39: [4992..5119]: data > >> +40: [5120..5247]: hole > >> +41: [5248..5375]: data > >> +42: [5376..5503]: hole > >> +43: [5504..5631]: data > >> +44: [5632..5759]: hole > >> +45: [5760..5887]: data > >> +46: [5888..6015]: hole > >> +47: [6016..6143]: data > >> +Larger query than file size > >> +0: [0..127]: hole > >> +1: [128..255]: data > >> +2: [256..383]: hole > >> +3: [384..511]: data > >> +4: [512..639]: hole > >> +5: [640..767]: data > >> +6: [768..895]: hole > >> +7: [896..1023]: data > >> +8: [1024..1151]: hole > >> +9: [1152..1279]: data > >> +10: [1280..1407]: hole > >> +11: [1408..1535]: data > >> +12: [1536..1663]: hole > >> +13: [1664..1791]: data > >> +14: [1792..1919]: hole > >> +15: [1920..2047]: data > >> +16: [2048..2175]: hole > >> +17: [2176..2303]: data > >> +18: [2304..2431]: hole > >> +19: [2432..2559]: data > >> +20: [2560..2687]: hole > >> +21: [2688..2815]: data > >> +22: [2816..2943]: hole > >> +23: [2944..3071]: data > >> +24: [3072..3199]: hole > >> +25: [3200..3327]: data > >> +26: [3328..3455]: hole > >> +27: [3456..3583]: data > >> +28: [3584..3711]: hole > >> +29: [3712..3839]: data > >> +30: [3840..3967]: hole > >> +31: [3968..4095]: data > >> +32: [4096..4223]: hole > >> +33: [4224..4351]: data > >> +34: [4352..4479]: hole > >> +35: [4480..4607]: data > >> +36: [4608..4735]: hole > >> +37: [4736..4863]: data > >> +38: [4864..4991]: hole > >> +39: [4992..5119]: data > >> +40: [5120..5247]: hole > >> +41: [5248..5375]: data > >> +42: [5376..5503]: hole > >> +43: [5504..5631]: data > >> +44: [5632..5759]: hole > >> +45: [5760..5887]: data > >> +46: [5888..6015]: hole > >> +47: [6016..6143]: data > >> +48: [6144..6271]: hole > >> +49: [6272..6399]: data > >> +50: [6400..6527]: hole > >> +51: [6528..6655]: data > >> +52: [6656..6783]: hole > >> +53: [6784..6911]: data > >> +54: [6912..7039]: hole > >> +55: [7040..7167]: data > >> +56: [7168..7295]: hole > >> +57: [7296..7423]: data > >> +58: [7424..7551]: hole > >> +59: [7552..7679]: data > >> +60: [7680..7807]: hole > >> +61: [7808..7935]: data > >> +62: [7936..8063]: hole > >> +63: [8064..8191]: data > >> +Skip first hole > >> +0: [128..255]: data > >> +1: [256..383]: hole > >> +2: [384..511]: data > >> +3: [512..639]: hole > >> +4: [640..767]: data > >> +5: [768..895]: hole > >> +6: [896..1023]: data > >> +7: [1024..1151]: hole > >> +8: [1152..1279]: data > >> +9: [1280..1407]: hole > >> +10: [1408..1535]: data > >> +11: [1536..1663]: hole > >> +12: [1664..1791]: data > >> +13: [1792..1919]: hole > >> +14: [1920..2047]: data > >> +15: [2048..2175]: hole > >> +16: [2176..2303]: data > >> +17: [2304..2431]: hole > >> +18: [2432..2559]: data > >> +19: [2560..2687]: hole > >> +20: [2688..2815]: data > >> +21: [2816..2943]: hole > >> +22: [2944..3071]: data > >> +23: [3072..3199]: hole > >> +24: [3200..3327]: data > >> +25: [3328..3455]: hole > >> +26: [3456..3583]: data > >> +27: [3584..3711]: hole > >> +28: [3712..3839]: data > >> +29: [3840..3967]: hole > >> +30: [3968..4095]: data > >> +31: [4096..4223]: hole > >> +32: [4224..4351]: data > >> +33: [4352..4479]: hole > >> +34: [4480..4607]: data > >> +35: [4608..4735]: hole > >> +36: [4736..4863]: data > >> +37: [4864..4991]: hole > >> +38: [4992..5119]: data > >> +39: [5120..5247]: hole > >> +40: [5248..5375]: data > >> +41: [5376..5503]: hole > >> +42: [5504..5631]: data > >> +43: [5632..5759]: hole > >> +44: [5760..5887]: data > >> +45: [5888..6015]: hole > >> +46: [6016..6143]: data > >> +47: [6144..6271]: hole > >> +48: [6272..6399]: data > >> +49: [6400..6527]: hole > >> +50: [6528..6655]: data > >> +51: [6656..6783]: hole > >> +52: [6784..6911]: data > >> +53: [6912..7039]: hole > >> +54: [7040..7167]: data > >> +55: [7168..7295]: hole > >> +56: [7296..7423]: data > >> +57: [7424..7551]: hole > >> +58: [7552..7679]: data > >> +59: [7680..7807]: hole > >> +60: [7808..7935]: data > >> +61: [7936..8063]: hole > >> +62: [8064..8191]: data > >> diff --git a/tests/generic/group b/tests/generic/group > >> index 6c3bb03..a043e98 100644 > >> --- a/tests/generic/group > >> +++ b/tests/generic/group > >> @@ -472,3 +472,4 @@ > >> 467 auto quick exportfs > >> 468 shutdown auto quick metadata > >> 469 auto quick > >> +900 auto quick > >> -- > >> 2.7.4 > >> > >> -- > >> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > >> the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Dec 06, 2017 at 01:06:59PM -0800, Darrick J. Wong wrote: > On Wed, Dec 06, 2017 at 10:51:39PM +0200, Nikolay Borisov wrote: > > On 6.12.2017 19:45, Darrick J. Wong wrote: > > > On Thu, Nov 30, 2017 at 06:05:27PM +0200, Nikolay Borisov wrote: > > >> new file mode 100644 > > >> index 0000000..3df6a3a > > >> --- /dev/null > > >> +++ b/tests/generic/900.out > > >> @@ -0,0 +1,196 @@ > > >> +QA output created by 900 > > >> +Basic data extent > > >> +0: [128..255]: data > > >> +Data + Hole > > >> +0: [128..255]: data > > >> +1: [256..287]: hole > > >> +Hole + Data > > >> +0: [0..127]: hole > > >> +1: [128..255]: data > > > > > > This fails on XFS: > > > > > > $ xfs_io -c "fiemap -v 0 65k" file > > > moo: > > > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > > > 0: [0..127]: hole 128 > > > 1: [128..135]: 74391184..74391191 8 0x0 > > > > > > (Note the records are trimmed to sector 135, or 65k) > > > > > > Compare to ext4: > > > > > > $ xfs_io -c "fiemap -v 0k 65k" file > > > moo: > > > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > > > 0: [0..127]: hole 128 > > > 1: [128..255]: 2510790784..2510790911 128 0x0 > > > > > > (ext4 doesn't bother trimming) > > > > > > Documentation/filesystems/fiemap.txt says "...the logical offset of the > > > 1st returned extent may start before fm_start, and the range covered by > > > the last returned extent may end after fm_length", so both behaviors are > > > allowed. > > > > So how do you propose the test to deal with the fact that XFS is the > > only file system (of the mainstream ones at least) that trims the extents? > > Put: > > test "$FSTYP" = "xfs" && _notrun "This is tested by xfs/XXX" > > in this test and then duplicate it as some xfs/ test with whatever the > correct output is on xfs. Also add some checks for fiemap -a since we > actually have attribute block maps on xfs. No, please don't do stupid things like this to work around unpredictable output from a *newly implemented command*. There's a *simple answer* to this problem: fix the new command's output. That is: the user asked for a specific range, so the command itself should trim the map returned by the kernel to only display the exact range the user asked for. Then it doesn't matter if the underlying filesystem trims the extents or not, because the we're going to do that anyway in userspace. > (let's see if hch/dchinner come in and NAK the whole thing on the > grounds that FIEMAP is a debugging interface anyway...) I've stayed out of it as much as I can, but when I see spiralling stupidity^Wcomplexity like this I'm going to open my big mouth and put my foot as far into it as I can. :/ Cheers, Dave.
On 12/6/17 3:57 PM, Dave Chinner wrote: > There's a *simple answer* to this problem: fix the new command's > output. > > That is: the user asked for a specific range, so the command itself > should trim the map returned by the kernel to only display the exact > range the user asked for. Then it doesn't matter if the underlying > filesystem trims the extents or not, because the we're going to do > that anyway in userspace. I have a different opinion: xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. Ranged fiemap queries are a real thing; you put numbers into the kernel, and you get numbers out of the kernel. IMNSO, xfs_io should present to the user /what the kernel returned/, and not re-interpret it to fit some other notion of correctness if we don't like what the kernel told us. If you want to have some user-friendlier behavior where xfs_io layers behaviors on top of what the kernel provides, then add a "-t" argument for trim, but hiding ioctl inconsistencies by filtering them through xfs_io sounds like the wrong approach to me. -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Dec 06, 2017 at 04:01:58PM -0600, Eric Sandeen wrote: > On 12/6/17 3:57 PM, Dave Chinner wrote: > > > There's a *simple answer* to this problem: fix the new command's > > output. > > > > That is: the user asked for a specific range, so the command itself > > should trim the map returned by the kernel to only display the exact > > range the user asked for. Then it doesn't matter if the underlying > > filesystem trims the extents or not, because the we're going to do > > that anyway in userspace. > > I have a different opinion: > > xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. > > Ranged fiemap queries are a real thing; you put numbers into the kernel, > and you get numbers out of the kernel. > > IMNSO, xfs_io should present to the user /what the kernel returned/, > and not re-interpret it to fit some other notion of correctness if we > don't like what the kernel told us. I hardly think "trimming to the range the user asked for" is "re-interpreting what the kernel told us". It's limiting output range to exactly what the user asked for - the output is still correct regardless of how it's filtered to match what the user asked for.... > If you want to have some user-friendlier behavior where xfs_io layers > behaviors on top of what the kernel provides, then add a "-t" argument for trim, > but hiding ioctl inconsistencies by filtering them through xfs_io sounds > like the wrong approach to me. Just filter the last output in the test, then, so it looks like 2: [128..XXX] data There is absolutely no excuse for creating multiple tests to support a small difference in trailing extent range output from different filesystem. Cheers, Dave.
On 12/6/17 4:16 PM, Dave Chinner wrote: > On Wed, Dec 06, 2017 at 04:01:58PM -0600, Eric Sandeen wrote: >> On 12/6/17 3:57 PM, Dave Chinner wrote: >> >>> There's a *simple answer* to this problem: fix the new command's >>> output. >>> >>> That is: the user asked for a specific range, so the command itself >>> should trim the map returned by the kernel to only display the exact >>> range the user asked for. Then it doesn't matter if the underlying >>> filesystem trims the extents or not, because the we're going to do >>> that anyway in userspace. >> >> I have a different opinion: >> >> xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. >> >> Ranged fiemap queries are a real thing; you put numbers into the kernel, >> and you get numbers out of the kernel. >> >> IMNSO, xfs_io should present to the user /what the kernel returned/, >> and not re-interpret it to fit some other notion of correctness if we >> don't like what the kernel told us. > > I hardly think "trimming to the range the user asked for" is > "re-interpreting what the kernel told us". It's limiting output > range to exactly what the user asked for - the output is still > correct regardless of how it's filtered to match what the user asked > for.... > >> If you want to have some user-friendlier behavior where xfs_io layers >> behaviors on top of what the kernel provides, then add a "-t" argument for trim, >> but hiding ioctl inconsistencies by filtering them through xfs_io sounds >> like the wrong approach to me. > > Just filter the last output in the test, then, so it looks like > > 2: [128..XXX] data Would need to do the same for the first extent AIUI. > There is absolutely no excuse for creating multiple tests to support > a small difference in trailing extent range output from different > filesystem. Excuse #1 is to ensure that for filesystems which return extent data past the requested range, it's actually correct... If we trim/filter it away and it returns junk for any reason, we'd never know via this test. So I think filtering the first/last lengths is a reasonable way to force it into a common test, but it reduces the value of the test to some degree. -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Dec 07, 2017 at 09:16:03AM +1100, Dave Chinner wrote: > On Wed, Dec 06, 2017 at 04:01:58PM -0600, Eric Sandeen wrote: > > On 12/6/17 3:57 PM, Dave Chinner wrote: > > > > > There's a *simple answer* to this problem: fix the new command's > > > output. > > > > > > That is: the user asked for a specific range, so the command itself > > > should trim the map returned by the kernel to only display the exact > > > range the user asked for. Then it doesn't matter if the underlying > > > filesystem trims the extents or not, because the we're going to do > > > that anyway in userspace. > > > > I have a different opinion: > > > > xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. > > > > Ranged fiemap queries are a real thing; you put numbers into the kernel, > > and you get numbers out of the kernel. > > > > IMNSO, xfs_io should present to the user /what the kernel returned/, > > and not re-interpret it to fit some other notion of correctness if we > > don't like what the kernel told us. > > I hardly think "trimming to the range the user asked for" is > "re-interpreting what the kernel told us". It's limiting output > range to exactly what the user asked for - the output is still > correct regardless of how it's filtered to match what the user asked > for.... ...except that xfs_io is a tool we use to debug the garbage that FIEMAP sprays back at userspace, so it absolutely should not be hiding things like this from the test suite: $ xfs_io -c "fiemap -va 0k 65k" moofile moo: EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS 0: [0..18446744073709551615]: 1585456720..1585456719 0 0x301 1: [0..129]: hole 130 (Yay ext4. WTF does the extent have length 0?) So.... historically XFS trimmed the FIEMAP results. Fix the test to reflect the XFS behavior, get ext4 (and btrfs) to fix their weird implementations, and leave xfs_io alone. (Don't mind me shifting positions. :P) --D > > If you want to have some user-friendlier behavior where xfs_io layers > > behaviors on top of what the kernel provides, then add a "-t" argument for trim, > > but hiding ioctl inconsistencies by filtering them through xfs_io sounds > > like the wrong approach to me. > > Just filter the last output in the test, then, so it looks like > > 2: [128..XXX] data > > There is absolutely no excuse for creating multiple tests to support > a small difference in trailing extent range output from different > filesystem. > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > -- > To unsubscribe from this list: send the line "unsubscribe linux-xfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 12/6/17 4:46 PM, Darrick J. Wong wrote: > On Thu, Dec 07, 2017 at 09:16:03AM +1100, Dave Chinner wrote: >> On Wed, Dec 06, 2017 at 04:01:58PM -0600, Eric Sandeen wrote: >>> On 12/6/17 3:57 PM, Dave Chinner wrote: >>> >>>> There's a *simple answer* to this problem: fix the new command's >>>> output. >>>> >>>> That is: the user asked for a specific range, so the command itself >>>> should trim the map returned by the kernel to only display the exact >>>> range the user asked for. Then it doesn't matter if the underlying >>>> filesystem trims the extents or not, because the we're going to do >>>> that anyway in userspace. >>> >>> I have a different opinion: >>> >>> xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. >>> >>> Ranged fiemap queries are a real thing; you put numbers into the kernel, >>> and you get numbers out of the kernel. >>> >>> IMNSO, xfs_io should present to the user /what the kernel returned/, >>> and not re-interpret it to fit some other notion of correctness if we >>> don't like what the kernel told us. >> >> I hardly think "trimming to the range the user asked for" is >> "re-interpreting what the kernel told us". It's limiting output >> range to exactly what the user asked for - the output is still >> correct regardless of how it's filtered to match what the user asked >> for.... > > ...except that xfs_io is a tool we use to debug the garbage that FIEMAP > sprays back at userspace, so it absolutely should not be hiding things > like this from the test suite: That's the point I was trying, I guess badly, to make. If I send a patch for xfs which essentially does this in fiemap: if (ranged_query && last_block_in_range_allocated) last_extent.length += 42; have I introduced a bug or not? -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 7.12.2017 00:46, Darrick J. Wong wrote: > On Thu, Dec 07, 2017 at 09:16:03AM +1100, Dave Chinner wrote: >> On Wed, Dec 06, 2017 at 04:01:58PM -0600, Eric Sandeen wrote: >>> On 12/6/17 3:57 PM, Dave Chinner wrote: >>> >>>> There's a *simple answer* to this problem: fix the new command's >>>> output. >>>> >>>> That is: the user asked for a specific range, so the command itself >>>> should trim the map returned by the kernel to only display the exact >>>> range the user asked for. Then it doesn't matter if the underlying >>>> filesystem trims the extents or not, because the we're going to do >>>> that anyway in userspace. >>> >>> I have a different opinion: >>> >>> xfs_io is a debugging tool; the fiemap command sends an ioctl to the kernel. >>> >>> Ranged fiemap queries are a real thing; you put numbers into the kernel, >>> and you get numbers out of the kernel. >>> >>> IMNSO, xfs_io should present to the user /what the kernel returned/, >>> and not re-interpret it to fit some other notion of correctness if we >>> don't like what the kernel told us. >> >> I hardly think "trimming to the range the user asked for" is >> "re-interpreting what the kernel told us". It's limiting output >> range to exactly what the user asked for - the output is still >> correct regardless of how it's filtered to match what the user asked >> for.... > > ...except that xfs_io is a tool we use to debug the garbage that FIEMAP > sprays back at userspace, so it absolutely should not be hiding things > like this from the test suite: > > $ xfs_io -c "fiemap -va 0k 65k" moofile > moo: > EXT: FILE-OFFSET BLOCK-RANGE TOTAL FLAGS > 0: [0..18446744073709551615]: 1585456720..1585456719 0 0x301 > 1: [0..129]: hole 130 Isn't there something similar in xfs_bmapi_read: /* Reading past eof, act as though there's a hole up to end. */ if (eof) got.br_startoff = end; > > (Yay ext4. WTF does the extent have length 0?) > > So.... historically XFS trimmed the FIEMAP results. Fix the test to > reflect the XFS behavior, get ext4 (and btrfs) to fix their weird > implementations, and leave xfs_io alone. > > (Don't mind me shifting positions. :P) > > --D > >>> If you want to have some user-friendlier behavior where xfs_io layers >>> behaviors on top of what the kernel provides, then add a "-t" argument for trim, >>> but hiding ioctl inconsistencies by filtering them through xfs_io sounds >>> like the wrong approach to me. >> >> Just filter the last output in the test, then, so it looks like >> >> 2: [128..XXX] data >> >> There is absolutely no excuse for creating multiple tests to support >> a small difference in trailing extent range output from different >> filesystem. >> >> Cheers, >> >> Dave. >> -- >> Dave Chinner >> david@fromorbit.com >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 30.11.2017 18:05, Nikolay Borisov wrote: > Fiemap gained support for passing in optional offset len > which denote the range requested, so this patch adds > testcases for this functionality. Aditionally, a special "ranged" > argument is added to the require_xfs_io_command which checks > for the presence of fiemap range support. > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> SO what's the final verdit, even though to be honest I got a bit annoyed at the way the whole situation with fiemap untangled and dunno if I have the willpower to make any more adjustments to this test. > --- > > For the time being this test is expected to fail on XFS. > > V7: > * Adjusted for 64k block size filesystem > * Only use/require a test device and not a scratch device > > V6: > * Moved to generic tests and successfully ran against ext4/btrfs > * Use pwrite to create the holes to make it even more generic > > V5: > * Drop changes to existing generic punch hole tests since > the new fiemap implementation don't require them > * Merge the common/rc change with this patch > * Added Data + Hole + Data and Hole + Data tests as per Eryu's request > * Adjusted output of some tests which fall in a hole, since holes are > truncated to passed range > * Simplified the logic to check for fiemap range support (Eryu) > > V4: > * Added test description > * Added new test for past-eof behavior > * Removed supported_generic_fs line > * Switched to using the "ranged" param require > * Revert v3 changes > > V3: > * Adjusted tests for '-r' fiemap param > * Tests for invalid -r combination > > V2: No change > V1: No change > > common/rc | 7 ++ > tests/generic/900 | 107 +++++++++++++++++++++++++++ > tests/generic/900.out | 196 ++++++++++++++++++++++++++++++++++++++++++++++++++ > tests/generic/group | 1 + > 4 files changed, 311 insertions(+) > create mode 100755 tests/generic/900 > create mode 100644 tests/generic/900.out > > diff --git a/common/rc b/common/rc > index 4c053a5..a0d162a 100644 > --- a/common/rc > +++ b/common/rc > @@ -2056,6 +2056,13 @@ _require_xfs_io_command() > -c "$command 4k 8k" $testfile 2>&1` > ;; > "fiemap") > + # If 'ranged' is passed as argument then we check to see if fiemap supports > + # ranged query params > + if echo "$param" | grep -q "ranged"; then > + param=$(echo "$param" | sed "s/ranged//") > + $XFS_IO_PROG -c "help fiemap" | grep -q "\[offset \[len\]\]" > + [ $? -eq 0 ] || _notrun "xfs_io $command ranged support is missing" > + fi > testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \ > -c "fiemap -v $param" $testfile 2>&1` > param_checked=1 > diff --git a/tests/generic/900 b/tests/generic/900 > new file mode 100755 > index 0000000..3f3019b > --- /dev/null > +++ b/tests/generic/900 > @@ -0,0 +1,107 @@ > +#! /bin/bash > +# FS QA Test No. 900 > + > +# Test for the new ranged query functionality in xfs_io's fiemap command. > +# This tests various combinations of hole + data layout being printed. > +# Also the test used 16k holes to be compatible with 16k block filesystems > + > +#----------------------------------------------------------------------- > +# Copyright (c) 2017 SUSE Linux Products GmbH. All Rights Reserved. > +# Author: Nikolay Borisov <nborisov@suse.com> > +# > +# This program is free software; you can redistribute it and/or > +# modify it under the terms of the GNU General Public License as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it would be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program; if not, write the Free Software Foundation, > +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA > +#----------------------------------------------------------------------- > +# > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/punch > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_os Linux > +_require_test > +_require_xfs_io_command "truncate" > +# ranged is a special argument which checks if fiemap supports > +# [offset [len]] args > +_require_xfs_io_command "fiemap" "ranged" > + > +file=$TEST_DIR/fiemap.$seq > +rm -f $file > + > +# Create a file with 64k hole followed by 64k data, and this pattern > +# repeats till it reaches 4M file size, so each extent has 64k data. > +# But truncate file to its final size first, otherwise XFS would merge > +# some extents due to speculative preallocation. > +$XFS_IO_PROG -f -c "truncate 4m" $file > +for i in {0..31}; do > + $XFS_IO_PROG -c "pwrite $(($i*128+64))k 64k" $file >/dev/null; > +done > + > +# Query 1 data extent between 64k..64k range > +echo "Basic data extent" > +$XFS_IO_PROG -c "fiemap -v 64k 64k" $file | _filter_fiemap > + > +# Query data and hole extent > +echo "Data + Hole" > +$XFS_IO_PROG -c "fiemap -v 64k 80k" $file | _filter_fiemap > + > +echo "Hole + Data" > +$XFS_IO_PROG -c "fiemap -v 0 65k" $file | _filter_fiemap > + > +echo "Hole + Data + Hole" > +$XFS_IO_PROG -c "fiemap -v 0k 130k" $file | _filter_fiemap > + > +echo "Data + Hole + Data" > +$XFS_IO_PROG -c "fiemap -v 64k 192k" $file | _filter_fiemap > + > +echo "Beginning with a hole" > +$XFS_IO_PROG -c "fiemap -v 0 3k" $file | _filter_fiemap > + > +# Query for 0..160k that's 40 extents, more than the EXTENT_BATCH > +echo "Query more than 32 extents" > +$XFS_IO_PROG -c "fiemap -v 0 3m" $file | _filter_fiemap > + > +echo "Larger query than file size" > +$XFS_IO_PROG -c "fiemap -v 0 5m" $file | _filter_fiemap > + > +# mapping past eof shouldn't print anything" > +$XFS_IO_PROG -c "fiemap -v 5m" $file | _filter_fiemap > + > +echo "Skip first hole" > +# check everything without the first hole > +$XFS_IO_PROG -c "fiemap -v 64k" $file | _filter_fiemap > + > +# success, all done > +status=0 > +exit > diff --git a/tests/generic/900.out b/tests/generic/900.out > new file mode 100644 > index 0000000..3df6a3a > --- /dev/null > +++ b/tests/generic/900.out > @@ -0,0 +1,196 @@ > +QA output created by 900 > +Basic data extent > +0: [128..255]: data > +Data + Hole > +0: [128..255]: data > +1: [256..287]: hole > +Hole + Data > +0: [0..127]: hole > +1: [128..255]: data > +Hole + Data + Hole > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..259]: hole > +Data + Hole + Data > +0: [128..255]: data > +1: [256..383]: hole > +2: [384..511]: data > +Beginning with a hole > +Query more than 32 extents > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..383]: hole > +3: [384..511]: data > +4: [512..639]: hole > +5: [640..767]: data > +6: [768..895]: hole > +7: [896..1023]: data > +8: [1024..1151]: hole > +9: [1152..1279]: data > +10: [1280..1407]: hole > +11: [1408..1535]: data > +12: [1536..1663]: hole > +13: [1664..1791]: data > +14: [1792..1919]: hole > +15: [1920..2047]: data > +16: [2048..2175]: hole > +17: [2176..2303]: data > +18: [2304..2431]: hole > +19: [2432..2559]: data > +20: [2560..2687]: hole > +21: [2688..2815]: data > +22: [2816..2943]: hole > +23: [2944..3071]: data > +24: [3072..3199]: hole > +25: [3200..3327]: data > +26: [3328..3455]: hole > +27: [3456..3583]: data > +28: [3584..3711]: hole > +29: [3712..3839]: data > +30: [3840..3967]: hole > +31: [3968..4095]: data > +32: [4096..4223]: hole > +33: [4224..4351]: data > +34: [4352..4479]: hole > +35: [4480..4607]: data > +36: [4608..4735]: hole > +37: [4736..4863]: data > +38: [4864..4991]: hole > +39: [4992..5119]: data > +40: [5120..5247]: hole > +41: [5248..5375]: data > +42: [5376..5503]: hole > +43: [5504..5631]: data > +44: [5632..5759]: hole > +45: [5760..5887]: data > +46: [5888..6015]: hole > +47: [6016..6143]: data > +Larger query than file size > +0: [0..127]: hole > +1: [128..255]: data > +2: [256..383]: hole > +3: [384..511]: data > +4: [512..639]: hole > +5: [640..767]: data > +6: [768..895]: hole > +7: [896..1023]: data > +8: [1024..1151]: hole > +9: [1152..1279]: data > +10: [1280..1407]: hole > +11: [1408..1535]: data > +12: [1536..1663]: hole > +13: [1664..1791]: data > +14: [1792..1919]: hole > +15: [1920..2047]: data > +16: [2048..2175]: hole > +17: [2176..2303]: data > +18: [2304..2431]: hole > +19: [2432..2559]: data > +20: [2560..2687]: hole > +21: [2688..2815]: data > +22: [2816..2943]: hole > +23: [2944..3071]: data > +24: [3072..3199]: hole > +25: [3200..3327]: data > +26: [3328..3455]: hole > +27: [3456..3583]: data > +28: [3584..3711]: hole > +29: [3712..3839]: data > +30: [3840..3967]: hole > +31: [3968..4095]: data > +32: [4096..4223]: hole > +33: [4224..4351]: data > +34: [4352..4479]: hole > +35: [4480..4607]: data > +36: [4608..4735]: hole > +37: [4736..4863]: data > +38: [4864..4991]: hole > +39: [4992..5119]: data > +40: [5120..5247]: hole > +41: [5248..5375]: data > +42: [5376..5503]: hole > +43: [5504..5631]: data > +44: [5632..5759]: hole > +45: [5760..5887]: data > +46: [5888..6015]: hole > +47: [6016..6143]: data > +48: [6144..6271]: hole > +49: [6272..6399]: data > +50: [6400..6527]: hole > +51: [6528..6655]: data > +52: [6656..6783]: hole > +53: [6784..6911]: data > +54: [6912..7039]: hole > +55: [7040..7167]: data > +56: [7168..7295]: hole > +57: [7296..7423]: data > +58: [7424..7551]: hole > +59: [7552..7679]: data > +60: [7680..7807]: hole > +61: [7808..7935]: data > +62: [7936..8063]: hole > +63: [8064..8191]: data > +Skip first hole > +0: [128..255]: data > +1: [256..383]: hole > +2: [384..511]: data > +3: [512..639]: hole > +4: [640..767]: data > +5: [768..895]: hole > +6: [896..1023]: data > +7: [1024..1151]: hole > +8: [1152..1279]: data > +9: [1280..1407]: hole > +10: [1408..1535]: data > +11: [1536..1663]: hole > +12: [1664..1791]: data > +13: [1792..1919]: hole > +14: [1920..2047]: data > +15: [2048..2175]: hole > +16: [2176..2303]: data > +17: [2304..2431]: hole > +18: [2432..2559]: data > +19: [2560..2687]: hole > +20: [2688..2815]: data > +21: [2816..2943]: hole > +22: [2944..3071]: data > +23: [3072..3199]: hole > +24: [3200..3327]: data > +25: [3328..3455]: hole > +26: [3456..3583]: data > +27: [3584..3711]: hole > +28: [3712..3839]: data > +29: [3840..3967]: hole > +30: [3968..4095]: data > +31: [4096..4223]: hole > +32: [4224..4351]: data > +33: [4352..4479]: hole > +34: [4480..4607]: data > +35: [4608..4735]: hole > +36: [4736..4863]: data > +37: [4864..4991]: hole > +38: [4992..5119]: data > +39: [5120..5247]: hole > +40: [5248..5375]: data > +41: [5376..5503]: hole > +42: [5504..5631]: data > +43: [5632..5759]: hole > +44: [5760..5887]: data > +45: [5888..6015]: hole > +46: [6016..6143]: data > +47: [6144..6271]: hole > +48: [6272..6399]: data > +49: [6400..6527]: hole > +50: [6528..6655]: data > +51: [6656..6783]: hole > +52: [6784..6911]: data > +53: [6912..7039]: hole > +54: [7040..7167]: data > +55: [7168..7295]: hole > +56: [7296..7423]: data > +57: [7424..7551]: hole > +58: [7552..7679]: data > +59: [7680..7807]: hole > +60: [7808..7935]: data > +61: [7936..8063]: hole > +62: [8064..8191]: data > diff --git a/tests/generic/group b/tests/generic/group > index 6c3bb03..a043e98 100644 > --- a/tests/generic/group > +++ b/tests/generic/group > @@ -472,3 +472,4 @@ > 467 auto quick exportfs > 468 shutdown auto quick metadata > 469 auto quick > +900 auto quick > -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Dec 15, 2017 at 10:33:16AM +0200, Nikolay Borisov wrote: > > > On 30.11.2017 18:05, Nikolay Borisov wrote: > > Fiemap gained support for passing in optional offset len > > which denote the range requested, so this patch adds > > testcases for this functionality. Aditionally, a special "ranged" > > argument is added to the require_xfs_io_command which checks > > for the presence of fiemap range support. > > > > Signed-off-by: Nikolay Borisov <nborisov@suse.com> > > SO what's the final verdit, even though to be honest I got a bit annoyed > at the way the whole situation with fiemap untangled and dunno if I have > the willpower to make any more adjustments to this test. I tend to take the patch as-is for now, and let it fail on XFS with the hope that the failure will drive further discussions and fixes on the test or kernel when we have a final conclusion. Please let me know if I should drop the patch. Thanks, Eryu -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/common/rc b/common/rc index 4c053a5..a0d162a 100644 --- a/common/rc +++ b/common/rc @@ -2056,6 +2056,13 @@ _require_xfs_io_command() -c "$command 4k 8k" $testfile 2>&1` ;; "fiemap") + # If 'ranged' is passed as argument then we check to see if fiemap supports + # ranged query params + if echo "$param" | grep -q "ranged"; then + param=$(echo "$param" | sed "s/ranged//") + $XFS_IO_PROG -c "help fiemap" | grep -q "\[offset \[len\]\]" + [ $? -eq 0 ] || _notrun "xfs_io $command ranged support is missing" + fi testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \ -c "fiemap -v $param" $testfile 2>&1` param_checked=1 diff --git a/tests/generic/900 b/tests/generic/900 new file mode 100755 index 0000000..3f3019b --- /dev/null +++ b/tests/generic/900 @@ -0,0 +1,107 @@ +#! /bin/bash +# FS QA Test No. 900 + +# Test for the new ranged query functionality in xfs_io's fiemap command. +# This tests various combinations of hole + data layout being printed. +# Also the test used 16k holes to be compatible with 16k block filesystems + +#----------------------------------------------------------------------- +# Copyright (c) 2017 SUSE Linux Products GmbH. All Rights Reserved. +# Author: Nikolay Borisov <nborisov@suse.com> +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/punch + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_os Linux +_require_test +_require_xfs_io_command "truncate" +# ranged is a special argument which checks if fiemap supports +# [offset [len]] args +_require_xfs_io_command "fiemap" "ranged" + +file=$TEST_DIR/fiemap.$seq +rm -f $file + +# Create a file with 64k hole followed by 64k data, and this pattern +# repeats till it reaches 4M file size, so each extent has 64k data. +# But truncate file to its final size first, otherwise XFS would merge +# some extents due to speculative preallocation. +$XFS_IO_PROG -f -c "truncate 4m" $file +for i in {0..31}; do + $XFS_IO_PROG -c "pwrite $(($i*128+64))k 64k" $file >/dev/null; +done + +# Query 1 data extent between 64k..64k range +echo "Basic data extent" +$XFS_IO_PROG -c "fiemap -v 64k 64k" $file | _filter_fiemap + +# Query data and hole extent +echo "Data + Hole" +$XFS_IO_PROG -c "fiemap -v 64k 80k" $file | _filter_fiemap + +echo "Hole + Data" +$XFS_IO_PROG -c "fiemap -v 0 65k" $file | _filter_fiemap + +echo "Hole + Data + Hole" +$XFS_IO_PROG -c "fiemap -v 0k 130k" $file | _filter_fiemap + +echo "Data + Hole + Data" +$XFS_IO_PROG -c "fiemap -v 64k 192k" $file | _filter_fiemap + +echo "Beginning with a hole" +$XFS_IO_PROG -c "fiemap -v 0 3k" $file | _filter_fiemap + +# Query for 0..160k that's 40 extents, more than the EXTENT_BATCH +echo "Query more than 32 extents" +$XFS_IO_PROG -c "fiemap -v 0 3m" $file | _filter_fiemap + +echo "Larger query than file size" +$XFS_IO_PROG -c "fiemap -v 0 5m" $file | _filter_fiemap + +# mapping past eof shouldn't print anything" +$XFS_IO_PROG -c "fiemap -v 5m" $file | _filter_fiemap + +echo "Skip first hole" +# check everything without the first hole +$XFS_IO_PROG -c "fiemap -v 64k" $file | _filter_fiemap + +# success, all done +status=0 +exit diff --git a/tests/generic/900.out b/tests/generic/900.out new file mode 100644 index 0000000..3df6a3a --- /dev/null +++ b/tests/generic/900.out @@ -0,0 +1,196 @@ +QA output created by 900 +Basic data extent +0: [128..255]: data +Data + Hole +0: [128..255]: data +1: [256..287]: hole +Hole + Data +0: [0..127]: hole +1: [128..255]: data +Hole + Data + Hole +0: [0..127]: hole +1: [128..255]: data +2: [256..259]: hole +Data + Hole + Data +0: [128..255]: data +1: [256..383]: hole +2: [384..511]: data +Beginning with a hole +Query more than 32 extents +0: [0..127]: hole +1: [128..255]: data +2: [256..383]: hole +3: [384..511]: data +4: [512..639]: hole +5: [640..767]: data +6: [768..895]: hole +7: [896..1023]: data +8: [1024..1151]: hole +9: [1152..1279]: data +10: [1280..1407]: hole +11: [1408..1535]: data +12: [1536..1663]: hole +13: [1664..1791]: data +14: [1792..1919]: hole +15: [1920..2047]: data +16: [2048..2175]: hole +17: [2176..2303]: data +18: [2304..2431]: hole +19: [2432..2559]: data +20: [2560..2687]: hole +21: [2688..2815]: data +22: [2816..2943]: hole +23: [2944..3071]: data +24: [3072..3199]: hole +25: [3200..3327]: data +26: [3328..3455]: hole +27: [3456..3583]: data +28: [3584..3711]: hole +29: [3712..3839]: data +30: [3840..3967]: hole +31: [3968..4095]: data +32: [4096..4223]: hole +33: [4224..4351]: data +34: [4352..4479]: hole +35: [4480..4607]: data +36: [4608..4735]: hole +37: [4736..4863]: data +38: [4864..4991]: hole +39: [4992..5119]: data +40: [5120..5247]: hole +41: [5248..5375]: data +42: [5376..5503]: hole +43: [5504..5631]: data +44: [5632..5759]: hole +45: [5760..5887]: data +46: [5888..6015]: hole +47: [6016..6143]: data +Larger query than file size +0: [0..127]: hole +1: [128..255]: data +2: [256..383]: hole +3: [384..511]: data +4: [512..639]: hole +5: [640..767]: data +6: [768..895]: hole +7: [896..1023]: data +8: [1024..1151]: hole +9: [1152..1279]: data +10: [1280..1407]: hole +11: [1408..1535]: data +12: [1536..1663]: hole +13: [1664..1791]: data +14: [1792..1919]: hole +15: [1920..2047]: data +16: [2048..2175]: hole +17: [2176..2303]: data +18: [2304..2431]: hole +19: [2432..2559]: data +20: [2560..2687]: hole +21: [2688..2815]: data +22: [2816..2943]: hole +23: [2944..3071]: data +24: [3072..3199]: hole +25: [3200..3327]: data +26: [3328..3455]: hole +27: [3456..3583]: data +28: [3584..3711]: hole +29: [3712..3839]: data +30: [3840..3967]: hole +31: [3968..4095]: data +32: [4096..4223]: hole +33: [4224..4351]: data +34: [4352..4479]: hole +35: [4480..4607]: data +36: [4608..4735]: hole +37: [4736..4863]: data +38: [4864..4991]: hole +39: [4992..5119]: data +40: [5120..5247]: hole +41: [5248..5375]: data +42: [5376..5503]: hole +43: [5504..5631]: data +44: [5632..5759]: hole +45: [5760..5887]: data +46: [5888..6015]: hole +47: [6016..6143]: data +48: [6144..6271]: hole +49: [6272..6399]: data +50: [6400..6527]: hole +51: [6528..6655]: data +52: [6656..6783]: hole +53: [6784..6911]: data +54: [6912..7039]: hole +55: [7040..7167]: data +56: [7168..7295]: hole +57: [7296..7423]: data +58: [7424..7551]: hole +59: [7552..7679]: data +60: [7680..7807]: hole +61: [7808..7935]: data +62: [7936..8063]: hole +63: [8064..8191]: data +Skip first hole +0: [128..255]: data +1: [256..383]: hole +2: [384..511]: data +3: [512..639]: hole +4: [640..767]: data +5: [768..895]: hole +6: [896..1023]: data +7: [1024..1151]: hole +8: [1152..1279]: data +9: [1280..1407]: hole +10: [1408..1535]: data +11: [1536..1663]: hole +12: [1664..1791]: data +13: [1792..1919]: hole +14: [1920..2047]: data +15: [2048..2175]: hole +16: [2176..2303]: data +17: [2304..2431]: hole +18: [2432..2559]: data +19: [2560..2687]: hole +20: [2688..2815]: data +21: [2816..2943]: hole +22: [2944..3071]: data +23: [3072..3199]: hole +24: [3200..3327]: data +25: [3328..3455]: hole +26: [3456..3583]: data +27: [3584..3711]: hole +28: [3712..3839]: data +29: [3840..3967]: hole +30: [3968..4095]: data +31: [4096..4223]: hole +32: [4224..4351]: data +33: [4352..4479]: hole +34: [4480..4607]: data +35: [4608..4735]: hole +36: [4736..4863]: data +37: [4864..4991]: hole +38: [4992..5119]: data +39: [5120..5247]: hole +40: [5248..5375]: data +41: [5376..5503]: hole +42: [5504..5631]: data +43: [5632..5759]: hole +44: [5760..5887]: data +45: [5888..6015]: hole +46: [6016..6143]: data +47: [6144..6271]: hole +48: [6272..6399]: data +49: [6400..6527]: hole +50: [6528..6655]: data +51: [6656..6783]: hole +52: [6784..6911]: data +53: [6912..7039]: hole +54: [7040..7167]: data +55: [7168..7295]: hole +56: [7296..7423]: data +57: [7424..7551]: hole +58: [7552..7679]: data +59: [7680..7807]: hole +60: [7808..7935]: data +61: [7936..8063]: hole +62: [8064..8191]: data diff --git a/tests/generic/group b/tests/generic/group index 6c3bb03..a043e98 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -472,3 +472,4 @@ 467 auto quick exportfs 468 shutdown auto quick metadata 469 auto quick +900 auto quick
Fiemap gained support for passing in optional offset len which denote the range requested, so this patch adds testcases for this functionality. Aditionally, a special "ranged" argument is added to the require_xfs_io_command which checks for the presence of fiemap range support. Signed-off-by: Nikolay Borisov <nborisov@suse.com> --- For the time being this test is expected to fail on XFS. V7: * Adjusted for 64k block size filesystem * Only use/require a test device and not a scratch device V6: * Moved to generic tests and successfully ran against ext4/btrfs * Use pwrite to create the holes to make it even more generic V5: * Drop changes to existing generic punch hole tests since the new fiemap implementation don't require them * Merge the common/rc change with this patch * Added Data + Hole + Data and Hole + Data tests as per Eryu's request * Adjusted output of some tests which fall in a hole, since holes are truncated to passed range * Simplified the logic to check for fiemap range support (Eryu) V4: * Added test description * Added new test for past-eof behavior * Removed supported_generic_fs line * Switched to using the "ranged" param require * Revert v3 changes V3: * Adjusted tests for '-r' fiemap param * Tests for invalid -r combination V2: No change V1: No change common/rc | 7 ++ tests/generic/900 | 107 +++++++++++++++++++++++++++ tests/generic/900.out | 196 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/group | 1 + 4 files changed, 311 insertions(+) create mode 100755 tests/generic/900 create mode 100644 tests/generic/900.out