| Message ID | 20180912101547.28835-1-zlang@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | shared/010: avoid dedupe testing blocked on large fs | expand |
On 9/12/18 5:15 AM, Zorro Lang wrote: > When test on large fs (--large-fs), xfstests preallocates a large > file in SCRATCH_MNT/ at first. Duperemove will take too long time > to deal with that large file (many days on 500T XFS). So move > working directory to a sub-dir underlying $SCRATCH_MNT/. > > Signed-off-by: Zorro Lang <zlang@redhat.com> Looks fine to me - Reviewed-by: Eric Sandeen <sandeen@redhat.com> But - I'm not sure what you mean about "testdir?" > --- > > Hi, > > Besides fix this issue, this patch fix another issue passingly. I left > a bad variable named "testdir" in this case. This patch can fix it. > > If maintainer feels I should fix it in another patch, please tell me:-P > > Thanks, > Zorro > > tests/shared/010 | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/tests/shared/010 b/tests/shared/010 > index 1817081b..04f55890 100755 > --- a/tests/shared/010 > +++ b/tests/shared/010 > @@ -65,15 +65,17 @@ function end_test() > sleep_time=$((50 * TIME_FACTOR)) > > # Start fsstress > +testdir="$SCRATCH_MNT/dir" > +mkdir $testdir > fsstress_opts="-r -n 1000 -p $((5 * LOAD_FACTOR))" > -$FSSTRESS_PROG $fsstress_opts -d $SCRATCH_MNT -l 0 >> $seqres.full 2>&1 & > +$FSSTRESS_PROG $fsstress_opts -d $testdir -l 0 >> $seqres.full 2>&1 & > dedup_pids="" > dupe_run=$TEST_DIR/${seq}-running > # Start several dedupe processes on same directory > touch $dupe_run > for ((i = 0; i < $((2 * LOAD_FACTOR)); i++)); do > while [ -e $dupe_run ]; do > - $DUPEREMOVE_PROG -dr --dedupe-options=same $SCRATCH_MNT/ \ > + $DUPEREMOVE_PROG -dr --dedupe-options=same $testdir \ > >>$seqres.full 2>&1 > done & > dedup_pids="$! $dedup_pids" >
On Wed, Sep 12, 2018 at 06:15:47PM +0800, Zorro Lang wrote: > When test on large fs (--large-fs), xfstests preallocates a large > file in SCRATCH_MNT/ at first. Duperemove will take too long time > to deal with that large file (many days on 500T XFS). So move > working directory to a sub-dir underlying $SCRATCH_MNT/. > > Signed-off-by: Zorro Lang <zlang@redhat.com> > --- > > Hi, > > Besides fix this issue, this patch fix another issue passingly. I left > a bad variable named "testdir" in this case. This patch can fix it. > > If maintainer feels I should fix it in another patch, please tell me:-P > > Thanks, > Zorro > > tests/shared/010 | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/tests/shared/010 b/tests/shared/010 > index 1817081b..04f55890 100755 > --- a/tests/shared/010 > +++ b/tests/shared/010 > @@ -65,15 +65,17 @@ function end_test() > sleep_time=$((50 * TIME_FACTOR)) > > # Start fsstress > +testdir="$SCRATCH_MNT/dir" Can we not call a temporary on the scratch device "testdir"? Because the "test dir" name specifically refers to the test device mount point, and using it for anything else (even though it's a different variable) is just confusing. Call it something like "local_dir" or "work_dir" or "stress_dir". But not "testdir". Cheers, Dave.
On Thu, Sep 13, 2018 at 11:28:26AM -0500, Eric Sandeen wrote: > On 9/12/18 5:15 AM, Zorro Lang wrote: > > When test on large fs (--large-fs), xfstests preallocates a large > > file in SCRATCH_MNT/ at first. Duperemove will take too long time > > to deal with that large file (many days on 500T XFS). So move > > working directory to a sub-dir underlying $SCRATCH_MNT/. > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > Looks fine to me - > > Reviewed-by: Eric Sandeen <sandeen@redhat.com> > > But - I'm not sure what you mean about "testdir?" Read shared/010 before merge this patch, you'll find a "$testdir" on this line: find $testdir -type f -exec md5sum {} \; > ${tmp}.md5sum But that "$testdir" never be assigned, it's a mistake I left :) Thanks, Zorro > > > --- > > > > Hi, > > > > Besides fix this issue, this patch fix another issue passingly. I left > > a bad variable named "testdir" in this case. This patch can fix it. > > > > If maintainer feels I should fix it in another patch, please tell me:-P > > > > Thanks, > > Zorro > > > > tests/shared/010 | 6 ++++-- > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > diff --git a/tests/shared/010 b/tests/shared/010 > > index 1817081b..04f55890 100755 > > --- a/tests/shared/010 > > +++ b/tests/shared/010 > > @@ -65,15 +65,17 @@ function end_test() > > sleep_time=$((50 * TIME_FACTOR)) > > > > # Start fsstress > > +testdir="$SCRATCH_MNT/dir" > > +mkdir $testdir > > fsstress_opts="-r -n 1000 -p $((5 * LOAD_FACTOR))" > > -$FSSTRESS_PROG $fsstress_opts -d $SCRATCH_MNT -l 0 >> $seqres.full 2>&1 & > > +$FSSTRESS_PROG $fsstress_opts -d $testdir -l 0 >> $seqres.full 2>&1 & > > dedup_pids="" > > dupe_run=$TEST_DIR/${seq}-running > > # Start several dedupe processes on same directory > > touch $dupe_run > > for ((i = 0; i < $((2 * LOAD_FACTOR)); i++)); do > > while [ -e $dupe_run ]; do > > - $DUPEREMOVE_PROG -dr --dedupe-options=same $SCRATCH_MNT/ \ > > + $DUPEREMOVE_PROG -dr --dedupe-options=same $testdir \ > > >>$seqres.full 2>&1 > > done & > > dedup_pids="$! $dedup_pids" > >
On Fri, Sep 14, 2018 at 09:01:21AM +1000, Dave Chinner wrote: > On Wed, Sep 12, 2018 at 06:15:47PM +0800, Zorro Lang wrote: > > When test on large fs (--large-fs), xfstests preallocates a large > > file in SCRATCH_MNT/ at first. Duperemove will take too long time > > to deal with that large file (many days on 500T XFS). So move > > working directory to a sub-dir underlying $SCRATCH_MNT/. > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > --- > > > > Hi, > > > > Besides fix this issue, this patch fix another issue passingly. I left > > a bad variable named "testdir" in this case. This patch can fix it. > > > > If maintainer feels I should fix it in another patch, please tell me:-P > > > > Thanks, > > Zorro > > > > tests/shared/010 | 6 ++++-- > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > diff --git a/tests/shared/010 b/tests/shared/010 > > index 1817081b..04f55890 100755 > > --- a/tests/shared/010 > > +++ b/tests/shared/010 > > @@ -65,15 +65,17 @@ function end_test() > > sleep_time=$((50 * TIME_FACTOR)) > > > > # Start fsstress > > +testdir="$SCRATCH_MNT/dir" > > Can we not call a temporary on the scratch device "testdir"? > > Because the "test dir" name specifically refers to the test device > mount point, and using it for anything else (even though it's a > different variable) is just confusing. > > Call it something like "local_dir" or "work_dir" or "stress_dir". > But not "testdir". Hmm... by running: # grep -rsn testdir xfstests/tests I found there're lots of "testdir"... do you need I change them all, or just begin to notice this issue from this patch? Thanks, Zorro > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com
On Fri, Sep 14, 2018 at 09:13:45AM +0800, Zorro Lang wrote: > On Fri, Sep 14, 2018 at 09:01:21AM +1000, Dave Chinner wrote: > > On Wed, Sep 12, 2018 at 06:15:47PM +0800, Zorro Lang wrote: > > > When test on large fs (--large-fs), xfstests preallocates a large > > > file in SCRATCH_MNT/ at first. Duperemove will take too long time > > > to deal with that large file (many days on 500T XFS). So move > > > working directory to a sub-dir underlying $SCRATCH_MNT/. > > > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > > --- > > > > > > Hi, > > > > > > Besides fix this issue, this patch fix another issue passingly. I left > > > a bad variable named "testdir" in this case. This patch can fix it. > > > > > > If maintainer feels I should fix it in another patch, please tell me:-P > > > > > > Thanks, > > > Zorro > > > > > > tests/shared/010 | 6 ++++-- > > > 1 file changed, 4 insertions(+), 2 deletions(-) > > > > > > diff --git a/tests/shared/010 b/tests/shared/010 > > > index 1817081b..04f55890 100755 > > > --- a/tests/shared/010 > > > +++ b/tests/shared/010 > > > @@ -65,15 +65,17 @@ function end_test() > > > sleep_time=$((50 * TIME_FACTOR)) > > > > > > # Start fsstress > > > +testdir="$SCRATCH_MNT/dir" > > > > Can we not call a temporary on the scratch device "testdir"? > > > > Because the "test dir" name specifically refers to the test device > > mount point, and using it for anything else (even though it's a > > different variable) is just confusing. > > > > Call it something like "local_dir" or "work_dir" or "stress_dir". > > But not "testdir". > > Hmm... by running: > # grep -rsn testdir xfstests/tests > > I found there're lots of "testdir"... do you need I change them all, or > just begin to notice this issue from this patch? Just this test for now, but it would be good to clean up all the other tests at some point. Cheers, Dave.
diff --git a/tests/shared/010 b/tests/shared/010 index 1817081b..04f55890 100755 --- a/tests/shared/010 +++ b/tests/shared/010 @@ -65,15 +65,17 @@ function end_test() sleep_time=$((50 * TIME_FACTOR)) # Start fsstress +testdir="$SCRATCH_MNT/dir" +mkdir $testdir fsstress_opts="-r -n 1000 -p $((5 * LOAD_FACTOR))" -$FSSTRESS_PROG $fsstress_opts -d $SCRATCH_MNT -l 0 >> $seqres.full 2>&1 & +$FSSTRESS_PROG $fsstress_opts -d $testdir -l 0 >> $seqres.full 2>&1 & dedup_pids="" dupe_run=$TEST_DIR/${seq}-running # Start several dedupe processes on same directory touch $dupe_run for ((i = 0; i < $((2 * LOAD_FACTOR)); i++)); do while [ -e $dupe_run ]; do - $DUPEREMOVE_PROG -dr --dedupe-options=same $SCRATCH_MNT/ \ + $DUPEREMOVE_PROG -dr --dedupe-options=same $testdir \ >>$seqres.full 2>&1 done & dedup_pids="$! $dedup_pids"
When test on large fs (--large-fs), xfstests preallocates a large file in SCRATCH_MNT/ at first. Duperemove will take too long time to deal with that large file (many days on 500T XFS). So move working directory to a sub-dir underlying $SCRATCH_MNT/. Signed-off-by: Zorro Lang <zlang@redhat.com> --- Hi, Besides fix this issue, this patch fix another issue passingly. I left a bad variable named "testdir" in this case. This patch can fix it. If maintainer feels I should fix it in another patch, please tell me:-P Thanks, Zorro tests/shared/010 | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)