| Message ID | 1427990168-21503-1-git-send-email-omzg@plexistor.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Apr 2, 2015 at 4:56 PM, Omer Zilberberg <omzg@plexistor.com> wrote: > Test was not run because directory parameter was omitted. > Also return value was not tested, so this problem was not caught. > > Signed-off-by: Omer Zilberberg <omzg@plexistor.com> Reviewed-by: Filipe Manana <fdmanana@suse.com> Tested-by: Filipe Manana <fdmanana@suse.com> > --- > tests/generic/076 | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/tests/generic/076 b/tests/generic/076 > index aa0aae0..3e1aa1a 100755 > --- a/tests/generic/076 > +++ b/tests/generic/076 > @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" > cat $SCRATCH_DEV >/dev/null & > pid=$! > > -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` > +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` > echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full > $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full > +rc=$? > +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" run_check $FSSTRESS_PROG $FSSTRESS_ARGS Would also do it (some other tests do this). Just a minor detail. Thanks. > _lets_get_pidst > > echo "*** done" > -- > 1.9.3 > > -- > To unsubscribe from this list: send the line "unsubscribe fstests" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Apr 02, 2015 at 10:05:40PM +0100, Filipe David Manana wrote: > On Thu, Apr 2, 2015 at 4:56 PM, Omer Zilberberg <omzg@plexistor.com> wrote: > > Test was not run because directory parameter was omitted. > > Also return value was not tested, so this problem was not caught. > > > > Signed-off-by: Omer Zilberberg <omzg@plexistor.com> > > Reviewed-by: Filipe Manana <fdmanana@suse.com> > Tested-by: Filipe Manana <fdmanana@suse.com> > > > --- > > tests/generic/076 | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/tests/generic/076 b/tests/generic/076 > > index aa0aae0..3e1aa1a 100755 > > --- a/tests/generic/076 > > +++ b/tests/generic/076 > > @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" > > cat $SCRATCH_DEV >/dev/null & > > pid=$! > > > > -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` > > +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` > > echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full > > $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full > > +rc=$? > > +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" > > run_check $FSSTRESS_PROG $FSSTRESS_ARGS > > Would also do it (some other tests do this). Please don't encourage run_check usage - it's cargo-cult programming at it's worst. It stops people from thinking about what errors they actually need to care about and capture exactly via other methods (such as output filtering). To demonstrate: fsstress only ever returns non-zero when there's a config or setup problem. This basically never happens once the test has been written. Further, fsstress does not collect runtime errors from child processes because it is designed to trigger errors and still continue onwards. Finally, the output stream is really only debug information, so that's not useful for anythign other than debugging, either. Hence there is never any need to check exit status of fsstress, nor do anything with the output stream except redirect it to the debug output file or the bit bucket. Cheers, Dave.
On 04/07/2015 05:29 AM, Dave Chinner wrote: > On Thu, Apr 02, 2015 at 10:05:40PM +0100, Filipe David Manana wrote: >> On Thu, Apr 2, 2015 at 4:56 PM, Omer Zilberberg <omzg@plexistor.com> wrote: >>> Test was not run because directory parameter was omitted. >>> Also return value was not tested, so this problem was not caught. >>> >>> Signed-off-by: Omer Zilberberg <omzg@plexistor.com> >> >> Reviewed-by: Filipe Manana <fdmanana@suse.com> >> Tested-by: Filipe Manana <fdmanana@suse.com> >> >>> --- >>> tests/generic/076 | 4 +++- >>> 1 file changed, 3 insertions(+), 1 deletion(-) >>> >>> diff --git a/tests/generic/076 b/tests/generic/076 >>> index aa0aae0..3e1aa1a 100755 >>> --- a/tests/generic/076 >>> +++ b/tests/generic/076 >>> @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" >>> cat $SCRATCH_DEV >/dev/null & >>> pid=$! >>> >>> -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` >>> +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` >>> echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full >>> $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full >>> +rc=$? >>> +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" >> >> run_check $FSSTRESS_PROG $FSSTRESS_ARGS >> >> Would also do it (some other tests do this). > > Please don't encourage run_check usage - it's cargo-cult programming > at it's worst. It stops people from thinking about what errors they > actually need to care about and capture exactly via other methods > (such as output filtering). > > To demonstrate: fsstress only ever returns non-zero when there's a > config or setup problem. This basically never happens once the test > has been written. Further, fsstress does not collect runtime errors > from child processes because it is designed to trigger errors and > still continue onwards. Finally, the output stream is really only > debug information, so that's not useful for anythign other than > debugging, either. > > Hence there is never any need to check exit status of fsstress, nor > do anything with the output stream except redirect it to the > debug output file or the bit bucket. run_check usage notwithstanding, I would argue that checking the exit status of fsstress would have prevented this problem - the dir argument was omitted during the introduction of load factor into fsstress in b84aade, and the test has been printing fsstress usage message to the debug output file ever since, when IMHO it should have failed. > > Cheers, > > Dave. > Thank you both for your comments. -- To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Apr 7, 2015 at 3:29 AM, Dave Chinner <david@fromorbit.com> wrote: > On Thu, Apr 02, 2015 at 10:05:40PM +0100, Filipe David Manana wrote: >> On Thu, Apr 2, 2015 at 4:56 PM, Omer Zilberberg <omzg@plexistor.com> wrote: >> > Test was not run because directory parameter was omitted. >> > Also return value was not tested, so this problem was not caught. >> > >> > Signed-off-by: Omer Zilberberg <omzg@plexistor.com> >> >> Reviewed-by: Filipe Manana <fdmanana@suse.com> >> Tested-by: Filipe Manana <fdmanana@suse.com> >> >> > --- >> > tests/generic/076 | 4 +++- >> > 1 file changed, 3 insertions(+), 1 deletion(-) >> > >> > diff --git a/tests/generic/076 b/tests/generic/076 >> > index aa0aae0..3e1aa1a 100755 >> > --- a/tests/generic/076 >> > +++ b/tests/generic/076 >> > @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" >> > cat $SCRATCH_DEV >/dev/null & >> > pid=$! >> > >> > -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` >> > +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` >> > echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full >> > $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full >> > +rc=$? >> > +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" >> >> run_check $FSSTRESS_PROG $FSSTRESS_ARGS >> >> Would also do it (some other tests do this). > > Please don't encourage run_check usage - it's cargo-cult programming > at it's worst. It stops people from thinking about what errors they > actually need to care about and capture exactly via other methods > (such as output filtering). > > To demonstrate: fsstress only ever returns non-zero when there's a > config or setup problem. That's exactly why here I don't see a problem with using run_check. Had we been using it, or checking the exit value with some other way, we would have know the test wasn't doing what is supposed to do (as Omer replied). Sure run_check aborts the test immediately if an error happens, preventing further test code from running, but in this case the test is not doing anything else after calling fsstress. Thanks. > This basically never happens once the test > has been written. Further, fsstress does not collect runtime errors > from child processes because it is designed to trigger errors and > still continue onwards. Finally, the output stream is really only > debug information, so that's not useful for anythign other than > debugging, either. > > Hence there is never any need to check exit status of fsstress, nor > do anything with the output stream except redirect it to the > debug output file or the bit bucket. > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com
On Tue, Apr 07, 2015 at 09:32:19AM +0300, Omer Zilberberg wrote: > On 04/07/2015 05:29 AM, Dave Chinner wrote: > > On Thu, Apr 02, 2015 at 10:05:40PM +0100, Filipe David Manana wrote: > >> On Thu, Apr 2, 2015 at 4:56 PM, Omer Zilberberg <omzg@plexistor.com> wrote: > >>> Test was not run because directory parameter was omitted. > >>> Also return value was not tested, so this problem was not caught. > >>> > >>> Signed-off-by: Omer Zilberberg <omzg@plexistor.com> > >> > >> Reviewed-by: Filipe Manana <fdmanana@suse.com> > >> Tested-by: Filipe Manana <fdmanana@suse.com> > >> > >>> --- > >>> tests/generic/076 | 4 +++- > >>> 1 file changed, 3 insertions(+), 1 deletion(-) > >>> > >>> diff --git a/tests/generic/076 b/tests/generic/076 > >>> index aa0aae0..3e1aa1a 100755 > >>> --- a/tests/generic/076 > >>> +++ b/tests/generic/076 > >>> @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" > >>> cat $SCRATCH_DEV >/dev/null & > >>> pid=$! > >>> > >>> -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` > >>> +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` > >>> echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full > >>> $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full > >>> +rc=$? > >>> +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" > >> > >> run_check $FSSTRESS_PROG $FSSTRESS_ARGS > >> > >> Would also do it (some other tests do this). > > > > Please don't encourage run_check usage - it's cargo-cult programming > > at it's worst. It stops people from thinking about what errors they > > actually need to care about and capture exactly via other methods > > (such as output filtering). > > > > To demonstrate: fsstress only ever returns non-zero when there's a > > config or setup problem. This basically never happens once the test > > has been written. Further, fsstress does not collect runtime errors > > from child processes because it is designed to trigger errors and > > still continue onwards. Finally, the output stream is really only > > debug information, so that's not useful for anythign other than > > debugging, either. > > > > Hence there is never any need to check exit status of fsstress, nor > > do anything with the output stream except redirect it to the > > debug output file or the bit bucket. > run_check usage notwithstanding, I would argue that checking the exit > status of fsstress would have prevented this problem - the dir argument > was omitted during the introduction of load factor into fsstress in b84aade, > and the test has been printing fsstress usage message to the debug output > file ever since, when IMHO it should have failed. So there was a bug in a commit, which we need to fix. That's not justification for peppering every program we run with "run_check" so that the exit value is tested just in case someone makes a mistake somewhere that wasn't caught by the reviewer.... Remember, the test harness infrastructure is designed specifcally so that we don't need to check the error status of every program we run. Programs need to give users obvious feedback of failure (i.e. stdout/stderr) because users *do not check return codes*, and the test harness is designed around ensuring programs generate useful error messages. IOWs, every place we use run_check to determine whether an operation passed or failed based on return code, we've failed to check if we informed the user of the error that occurred. Every utility a user runs should be reporting errors in text, not silently failing. Needing to use run_check to detect failures and advocating for widespread use is indicative of a bigger, more fundamental problem. i.e. if the only way we can check if am operation has succeed is to check the error code via run_check, we're giving ours users the big finger. Errors messages matter to users more than the return code of a program and hence our regression tests should be checking error messages, not return codes. Every time we use run_check we fail our users... Cheers, Dave.
diff --git a/tests/generic/076 b/tests/generic/076 index aa0aae0..3e1aa1a 100755 --- a/tests/generic/076 +++ b/tests/generic/076 @@ -74,9 +74,11 @@ echo "*** test concurrent block/fs access" cat $SCRATCH_DEV >/dev/null & pid=$! -FSSTRESS_ARGS=`_scale_fsstress_args -p 2 -n 2000 $FSSTRESS_AVOID` +FSSTRESS_ARGS=`_scale_fsstress_args -d $SCRATCH_MNT -p 2 -n 2000 $FSSTRESS_AVOID` echo "run fsstress with args: $FSSTRESS_ARGS" >>$seqres.full $FSSTRESS_PROG $FSSTRESS_ARGS >>$seqres.full +rc=$? +[ $rc != 0 ] && echo "fsstress returned $rc - see $seqres.full" _lets_get_pidst echo "*** done"
Test was not run because directory parameter was omitted. Also return value was not tested, so this problem was not caught. Signed-off-by: Omer Zilberberg <omzg@plexistor.com> --- tests/generic/076 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)