| Message ID | 169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests: testing improvements | expand |
On Tue, Jul 25, 2023 at 06:56:45PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong <djwong@kernel.org> > > Create a "-smoketest" parameter to check that will run generic > filesystem smoke testing for five minutes apiece. Since there are only > five smoke tests, this is effectively a 16min super-quick test. The code is setting SOAK_DURATION to 4 minutes, not 5 minutes. However, when I ran the moral equivalent: kvm-xfstests --soak-duration 4m --fail-loop-count 0 -c ext4/4k \ generic/475 generic/476 generic/521 generic/522 generic/642 It overall took 17 minutes to run, with just under a minute of test infrastructure overhead (in the check script and my wrapper scripts), with the actual test time as follows: ext4/4k: 5 tests, 975 seconds generic/475 Pass 242s generic/476 Pass 244s generic/521 Pass 241s generic/522 Pass 241s generic/642 Pass 7s Totals: 5 tests, 0 skipped, 0 failures, 0 errors, 975s The time which generic/642 ran was surprising so I took a closer look. It does claim to be in group "soak", and it even tries to canonicalize SOAK_DURATION (I'm not sure why, since the check script does this already). But generic/642 doesn't seem to use SOAK_DURATION. It does caculate a DURATION value, but it doesn't actually use SOAK_DURATION. So that sounds like a bug in the generic/642 test? There was also a bug xfstests's "make install" in that it doesn't actually install src/soak_duration.awk, but I'll send that a patch fixing that under separate cover. Darrick -- suppose changed the SOAK_DURATION down to 2 minutes? How much do you think that would materially affect the code coverage metrics, and the overall effectiveness of the smoke test? If we get generci/642 to honor SOAK_DURATION, using an overall 2 minute soak for each test would translate to the smoke test taking about 13 minutes, which would be great from a drive-by patch submitter perspective. - Ted > With gcov enabled, running these tests yields about ~75% coverage for > iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for > btrfs. Coverage was about ~65% for the pagecache.
diff --git a/README b/README index 9790334db1..d4ec73d10d 100644 --- a/README +++ b/README @@ -311,6 +311,15 @@ Running tests: The TEST and SCRATCH partitions should be pre-formatted with another base fs, where the overlay dirs will be created + - For infrequent filesystem developers who simply want to run a quick test + of the most commonly used filesystem functionality, use this command: + + ./check -smoketest <other config options> + + This configures fstests to run five tests to exercise the file I/O, + metadata, and crash recovery exercisers for four minutes apiece. This + should complete in approximately 20 minutes. + The check script tests the return value of each script, and compares the output against the expected output. If the output diff --git a/check b/check index 89e7e7bf20..c02e693642 100755 --- a/check +++ b/check @@ -68,6 +68,7 @@ check options -pvfs2 test PVFS2 -tmpfs test TMPFS -ubifs test ubifs + -smoketest run smoke tests for 4min each -l line mode diff -udiff show unified diff (default) -n show me, do not run tests @@ -290,7 +291,10 @@ while [ $# -gt 0 ]; do FSTYP=overlay export OVERLAY=true ;; - + -smoketest) + test -z "$SOAK_DURATION" && SOAK_DURATION="4m" + GROUP_LIST="smoketest" + ;; -g) group=$2 ; shift ; GROUP_LIST="$GROUP_LIST ${group//,/ }" ;; diff --git a/doc/group-names.txt b/doc/group-names.txt index 1c35a39432..c3dcca3755 100644 --- a/doc/group-names.txt +++ b/doc/group-names.txt @@ -118,6 +118,7 @@ selftest tests with fixed results, used to validate testing setup send btrfs send/receive shrinkfs decreasing the size of a filesystem shutdown FS_IOC_SHUTDOWN ioctl +smoketest Simple smoke tests snapshot btrfs snapshots soak long running soak tests whose runtime can be controlled directly by setting the SOAK_DURATION variable diff --git a/tests/generic/475 b/tests/generic/475 index 0cbf5131c2..ce7fe013b1 100755 --- a/tests/generic/475 +++ b/tests/generic/475 @@ -12,7 +12,7 @@ # testing efforts. # . ./common/preamble -_begin_fstest shutdown auto log metadata eio recoveryloop +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest # Override the default cleanup function. _cleanup() diff --git a/tests/generic/476 b/tests/generic/476 index 8e93b73457..b1ae4df4d4 100755 --- a/tests/generic/476 +++ b/tests/generic/476 @@ -8,7 +8,7 @@ # bugs in the write path. # . ./common/preamble -_begin_fstest auto rw long_rw stress soak +_begin_fstest auto rw long_rw stress soak smoketest # Override the default cleanup function. _cleanup() diff --git a/tests/generic/521 b/tests/generic/521 index 22dd31a8ec..0956e50171 100755 --- a/tests/generic/521 +++ b/tests/generic/521 @@ -7,7 +7,7 @@ # Long-soak directio fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter diff --git a/tests/generic/522 b/tests/generic/522 index f0cbcb245c..0e4e6009ed 100755 --- a/tests/generic/522 +++ b/tests/generic/522 @@ -7,7 +7,7 @@ # Long-soak buffered fsx test # . ./common/preamble -_begin_fstest soak long_rw +_begin_fstest soak long_rw smoketest # Import common functions. . ./common/filter diff --git a/tests/generic/642 b/tests/generic/642 index eba90903a3..e6a475a8b5 100755 --- a/tests/generic/642 +++ b/tests/generic/642 @@ -8,7 +8,7 @@ # bugs in the xattr code. # . ./common/preamble -_begin_fstest auto soak attr long_rw stress +_begin_fstest auto soak attr long_rw stress smoketest _cleanup() {