xfstests: aio random write and verify stress test

Commit Message

Zorro Lang May 13, 2019, 1:49 a.m. UTC

We found some AIO write related bugs recently, so I think a AIO
random write test is needed. By the new aio-aio-write-verify.c tool,
we can do this easily.

Signed-off-by: Zorro Lang <[e-mail]>
---

Long time passed, re-send this to review again.

Thanks,
Zorro

 tests/generic/998     | 87 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/998.out |  2 +
 tests/generic/group   |  1 +
 3 files changed, 90 insertions(+)
 create mode 100755 tests/generic/998
 create mode 100644 tests/generic/998.out

Comments

Amir Goldstein June 28, 2019, 8:03 a.m. UTC | #1

On Mon, May 13, 2019 at 4:57 AM Zorro Lang <[e-mail]> wrote:
>
> We found some AIO write related bugs recently, so I think a AIO
> random write test is needed. By the new aio-aio-write-verify.c tool,
> we can do this easily.
>

Is it just me or does this test gets killed by OOM killer for anyone else?
(generic/551).

It always gets killed on a 2GB RAM VM and once got killed on 8GB RAM VM.

At least I can verify that Darrick's fix is working as expected
("check: try to insulate the test framework from oom killer")

Thanks,
Amir.

Zorro Lang June 28, 2019, 8:24 a.m. UTC | #2

On Fri, Jun 28, 2019 at 11:03:57AM +0300, Amir Goldstein wrote:
> On Mon, May 13, 2019 at 4:57 AM Zorro Lang <[e-mail]> wrote:
> >
> > We found some AIO write related bugs recently, so I think a AIO
> > random write test is needed. By the new aio-aio-write-verify.c tool,
> > we can do this easily.
> >
> 
> Is it just me or does this test gets killed by OOM killer for anyone else?
> (generic/551).
> 
> It always gets killed on a 2GB RAM VM and once got killed on 8GB RAM VM.

Sorry, not only you. We've found that this case can cause OOM on low memeory
machine. I'm going to reduce the default test stress, to avoid the OOM. If
you already has a patch now, please feel free to cc me.

Thanks,
Zorro

> 
> At least I can verify that Darrick's fix is working as expected
> ("check: try to insulate the test framework from oom killer")
> 
> Thanks,
> Amir.

Patch

diff --git a/tests/generic/998 b/tests/generic/998
new file mode 100755
index 00000000..54b79691
--- /dev/null
+++ b/tests/generic/998
@@ -0,0 +1,87 @@ 
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test No. 998
+#
+# Randomly direct AIO write&verify stress test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_aiodio aio-dio-write-verify
+
+_scratch_mkfs > $seqres.full 2>&1
+_scratch_mount
+
+localfile=$SCRATCH_MNT/testfile
+diosize=`_min_dio_alignment $SCRATCH_DEV`
+
+# The maximum write size and offset are both 32k diosize. So the maximum
+# file size will be (32 * 2)k
+free_size_k=`df -kP $SCRATCH_MNT | grep -v Filesystem | awk '{print $4}'`
+max_io_size_b=$((32 * 1024))
+if [ $max_io_size_b -gt $((free_size_k * 1024 / 2 / diosize)) ]; then
+	max_io_size_b=$((free_size_k * 1024 / 2 / diosize))
+fi
+
+do_test()
+{
+	local num_oper
+	local oper_list=""
+	local size
+	local off
+	local truncsize
+
+	# the number of AIO write operation
+	num_oper=$((RANDOM % 64 + 1))
+
+	for ((i=0; i<num_oper; i++)); do
+		size=$(((RANDOM % max_io_size_b + 1) * diosize))
+		off=$((RANDOM % max_io_size_b * diosize))
+		oper_list="$oper_list -a size=$size,off=$off"
+	done
+	truncsize=$(((RANDOM * diosize + RANDOM % diosize) % max_io_size_b))
+
+	$AIO_TEST -t $truncsize $oper_list $localfile
+	if [ $? -ne 0 ];then
+		echo "$AIO_TEST -t $truncsize $oper_list $localfile"
+		echo "==========^^ Fail ^^=========="
+	fi
+}
+
+testimes=$((LOAD_FACTOR * 100))
+while [ $testimes -gt 0 ]; do
+	echo > $localfile
+	do_test
+	((testimes--))
+done
+
+echo "Silence is golden"
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/998.out b/tests/generic/998.out
new file mode 100644
index 00000000..d2679ae0
--- /dev/null
+++ b/tests/generic/998.out
@@ -0,0 +1,2 @@ 
+QA output created by 998
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index 0c3016fd..36b180c3 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -540,4 +540,5 @@ 
 535 auto quick log
 536 auto quick rw shutdown
 537 auto quick trim
+998 auto stress aio
 999 auto quick aio