| Message ID | 20230110224906.1171483-4-david@fromorbit.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests: filesystem population fixes | expand |
Hi Dave! On Wed, 11 Jan 2023 09:49:06 +1100, Dave Chinner wrote: > From: Dave Chinner <dchinner@redhat.com> > > This creates a bunch of files in a dir, then deletes 97% of them > attempting to leave 1 allocated inode per inode chunk so that they > aren't freed. Performance is badly limited by task creation and > destruction for each inode created. Fix this by using "echo -n > > file" rather than touch so that the shell creates the empty files > without needing to fork/exec a separate task for each creation. > > This reduces runtime from 45s down to 15s. I see >7x improvement on a zram device, be it from a lower baseline :) Reviewed-by: David Disseldorp <ddiss@suse.de> Cheers, David
On Wed, Jan 11, 2023 at 09:49:06AM +1100, Dave Chinner wrote: > From: Dave Chinner <dchinner@redhat.com> > > This creates a bunch of files in a dir, then deletes 97% of them > attempting to leave 1 allocated inode per inode chunk so that they > aren't freed. Performance is badly limited by task creation and > destruction for each inode created. Fix this by using "echo -n > > file" rather than touch so that the shell creates the empty files > without needing to fork/exec a separate task for each creation. > > This reduces runtime from 45s down to 15s. > > Also add more debug with inode counts and internal superblock > counter information for determining why this test may ENOSPC on the > final creation loop. > > Signed-off-by: Dave Chinner <dchinner@redhat.com> > --- > tests/xfs/294 | 19 +++++++++++++++---- > 1 file changed, 15 insertions(+), 4 deletions(-) > > diff --git a/tests/xfs/294 b/tests/xfs/294 > index d381e2c85..1ce0d1cc5 100755 > --- a/tests/xfs/294 > +++ b/tests/xfs/294 > @@ -28,6 +28,13 @@ _require_test_program "punch-alternating" > _require_xfs_io_command "falloc" > _require_xfs_io_command "fpunch" > > +dump_freespace() > +{ > + df $SCRATCH_MNT > + df -i $SCRATCH_MNT > + $XFS_IO_PROG -rc "statfs -c" $SCRATCH_MNT > +} > + > # We want to mkfs with a very specific geometry > MKFS_OPTIONS="" > _scratch_mkfs "-d size=512m -n size=8192 -i size=1024" >> $seqres.full 2>&1 \ > @@ -37,7 +44,7 @@ _scratch_mount > # Make a ton of mostly-empty inode clusters so we can always > # make more inodes > mkdir $SCRATCH_MNT/tmp > -for I in `seq 1 10000`; do touch $SCRATCH_MNT/tmp/$I; done > +for I in `seq 1 10000`; do echo -n > $SCRATCH_MNT/tmp/$I; done Make sense to me, thanks for this improvement. Reviewed-by: Zorro Lang <zlang@redhat.com> > > # These mostly-empty clusters will live here: > mkdir $SCRATCH_MNT/clusters > @@ -50,7 +57,7 @@ rm -rf $SCRATCH_MNT/tmp > mkdir $SCRATCH_MNT/testdir > # roughly 20 chars per file > for I in `seq 1 100`; do > - touch $SCRATCH_MNT/testdir/12345678901234567890$I; > + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; > done > > # File to fragment: > @@ -63,7 +70,7 @@ space=$(stat -f -c '%f * %S * 95 / 100' $SCRATCH_MNT | $BC_PROG) > $XFS_IO_PROG -f -c "falloc 0 $space" $SCRATCH_MNT/fillfile || > _fail "Could not allocate space" > > -df -h $SCRATCH_MNT >> $seqres.full 2>&1 > +dump_freespace >> $seqres.full 2>&1 > > # Fill remaining space; let this run to failure > dd if=/dev/zero of=$SCRATCH_MNT/spacefile1 oflag=direct >> $seqres.full 2>&1 > @@ -75,12 +82,16 @@ $here/src/punch-alternating $SCRATCH_MNT/fragfile >> $seqres.full 2>&1 > # (and then some for good measure) > dd conv=fsync if=/dev/zero of=$SCRATCH_MNT/spacefile2 bs=1M count=64 >> $seqres.full 2>&1 > > +dump_freespace >> $seqres.full 2>&1 > + > # Now populate the directory so that it must allocate these > # fragmented blocks > for I in `seq 1 1400`; do > - touch $SCRATCH_MNT/testdir/12345678901234567890$I; > + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; > done > > +dump_freespace >> $seqres.full 2>&1 > + > # Now traverse that ugly thing! > find $SCRATCH_MNT/testdir | sort | _filter_scratch | md5sum > > -- > 2.38.1 >
diff --git a/tests/xfs/294 b/tests/xfs/294 index d381e2c85..1ce0d1cc5 100755 --- a/tests/xfs/294 +++ b/tests/xfs/294 @@ -28,6 +28,13 @@ _require_test_program "punch-alternating" _require_xfs_io_command "falloc" _require_xfs_io_command "fpunch" +dump_freespace() +{ + df $SCRATCH_MNT + df -i $SCRATCH_MNT + $XFS_IO_PROG -rc "statfs -c" $SCRATCH_MNT +} + # We want to mkfs with a very specific geometry MKFS_OPTIONS="" _scratch_mkfs "-d size=512m -n size=8192 -i size=1024" >> $seqres.full 2>&1 \ @@ -37,7 +44,7 @@ _scratch_mount # Make a ton of mostly-empty inode clusters so we can always # make more inodes mkdir $SCRATCH_MNT/tmp -for I in `seq 1 10000`; do touch $SCRATCH_MNT/tmp/$I; done +for I in `seq 1 10000`; do echo -n > $SCRATCH_MNT/tmp/$I; done # These mostly-empty clusters will live here: mkdir $SCRATCH_MNT/clusters @@ -50,7 +57,7 @@ rm -rf $SCRATCH_MNT/tmp mkdir $SCRATCH_MNT/testdir # roughly 20 chars per file for I in `seq 1 100`; do - touch $SCRATCH_MNT/testdir/12345678901234567890$I; + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; done # File to fragment: @@ -63,7 +70,7 @@ space=$(stat -f -c '%f * %S * 95 / 100' $SCRATCH_MNT | $BC_PROG) $XFS_IO_PROG -f -c "falloc 0 $space" $SCRATCH_MNT/fillfile || _fail "Could not allocate space" -df -h $SCRATCH_MNT >> $seqres.full 2>&1 +dump_freespace >> $seqres.full 2>&1 # Fill remaining space; let this run to failure dd if=/dev/zero of=$SCRATCH_MNT/spacefile1 oflag=direct >> $seqres.full 2>&1 @@ -75,12 +82,16 @@ $here/src/punch-alternating $SCRATCH_MNT/fragfile >> $seqres.full 2>&1 # (and then some for good measure) dd conv=fsync if=/dev/zero of=$SCRATCH_MNT/spacefile2 bs=1M count=64 >> $seqres.full 2>&1 +dump_freespace >> $seqres.full 2>&1 + # Now populate the directory so that it must allocate these # fragmented blocks for I in `seq 1 1400`; do - touch $SCRATCH_MNT/testdir/12345678901234567890$I; + echo -n > $SCRATCH_MNT/testdir/12345678901234567890$I; done +dump_freespace >> $seqres.full 2>&1 + # Now traverse that ugly thing! find $SCRATCH_MNT/testdir | sort | _filter_scratch | md5sum