diff mbox

[i-g-t,5/5] tests/kms_test_only: Validate TEST_ONLY correctness against full atomic commit

Message ID 1483099241-15898-6-git-send-email-mika.kahola@intel.com (mailing list archive)
State New, archived
Headers show

Commit Message

Mika Kahola Dec. 30, 2016, noon UTC
This test case adds TEST_ONLY flag to the following test cases to test
atomic commit correctness.

 - kms_plane_multiple
 - kms_atomic_transitions
 - kms_plane_scaling
 - kms_rotation_crc

The test randomly selects one of the above test cases and tests atomic
commit. If the test fails with TEST_ONLY flag the real deal atomic commit
is executed and the outcome is verified.

The test runs by default for 64 iterations.

Signed-off-by: Mika Kahola <mika.kahola@intel.com>
---
 tests/Makefile.sources |   1 +
 tests/kms_test_only.c  | 455 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 456 insertions(+)
 create mode 100644 tests/kms_test_only.c

Comments

Maarten Lankhorst Jan. 12, 2017, 10:11 a.m. UTC | #1
Op 30-12-16 om 13:00 schreef Mika Kahola:
> This test case adds TEST_ONLY flag to the following test cases to test
> atomic commit correctness.
>
>  - kms_plane_multiple
>  - kms_atomic_transitions
>  - kms_plane_scaling
>  - kms_rotation_crc
>
> The test randomly selects one of the above test cases and tests atomic
> commit. If the test fails with TEST_ONLY flag the real deal atomic commit
> is executed and the outcome is verified.
>
> The test runs by default for 64 iterations.
>
> Signed-off-by: Mika Kahola <mika.kahola@intel.com>
> ---
>  tests/Makefile.sources |   1 +
>  tests/kms_test_only.c  | 455 +++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 456 insertions(+)
>  create mode 100644 tests/kms_test_only.c
Hey,

What does this test add?

Any test should pass with PASS. This test retrying failed test-only tests won't add much there. No test should ever fail.

~Maarten
Maarten Lankhorst Jan. 12, 2017, 11:37 a.m. UTC | #2
Hey,

Op 12-01-17 om 11:28 schreef Mika Kahola:
> On Thu, 2017-01-12 at 11:11 +0100, Maarten Lankhorst wrote:
>> Op 30-12-16 om 13:00 schreef Mika Kahola:
>>> This test case adds TEST_ONLY flag to the following test cases to
>>> test
>>> atomic commit correctness.
>>>
>>>  - kms_plane_multiple
>>>  - kms_atomic_transitions
>>>  - kms_plane_scaling
>>>  - kms_rotation_crc
>>>
>>> The test randomly selects one of the above test cases and tests
>>> atomic
>>> commit. If the test fails with TEST_ONLY flag the real deal atomic
>>> commit
>>> is executed and the outcome is verified.
>>>
>>> The test runs by default for 64 iterations.
>>>
>>> Signed-off-by: Mika Kahola <mika.kahola@intel.com>
>>> ---
>>>  tests/Makefile.sources |   1 +
>>>  tests/kms_test_only.c  | 455
>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>  2 files changed, 456 insertions(+)
>>>  create mode 100644 tests/kms_test_only.c
>> Hey,
>>
>> What does this test add?
> This is for VIZ-6956. The idea here is to test atomic commits with
> TEST_ONLY first, and it that fails we try to commit in real. 
>> Any test should pass with PASS. This test retrying failed test-only
>> tests won't add much there. No test should ever fail.
> That's my thought also. You could argue whether or not this test adds
> any value. In my mind, this test only tests if TEST_ONLY flag is
> behaving ok :)
But you can already compare test passes vs test fails. There's no need for a separate test because any test failing should already be investigated.

I don't think a test that tests another test is useful, we already have infrastructure for that. :-)

~Maarten
Mika Kahola Jan. 23, 2017, 1:26 p.m. UTC | #3
On Thu, 2017-01-12 at 12:37 +0100, Maarten Lankhorst wrote:
> Hey,
> 
> Op 12-01-17 om 11:28 schreef Mika Kahola:
> > 
> > On Thu, 2017-01-12 at 11:11 +0100, Maarten Lankhorst wrote:
> > > 
> > > Op 30-12-16 om 13:00 schreef Mika Kahola:
> > > > 
> > > > This test case adds TEST_ONLY flag to the following test cases
> > > > to
> > > > test
> > > > atomic commit correctness.
> > > > 
> > > >  - kms_plane_multiple
> > > >  - kms_atomic_transitions
> > > >  - kms_plane_scaling
> > > >  - kms_rotation_crc
> > > > 
> > > > The test randomly selects one of the above test cases and tests
> > > > atomic
> > > > commit. If the test fails with TEST_ONLY flag the real deal
> > > > atomic
> > > > commit
> > > > is executed and the outcome is verified.
> > > > 
> > > > The test runs by default for 64 iterations.
> > > > 
> > > > Signed-off-by: Mika Kahola <mika.kahola@intel.com>
> > > > ---
> > > >  tests/Makefile.sources |   1 +
> > > >  tests/kms_test_only.c  | 455
> > > > +++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 456 insertions(+)
> > > >  create mode 100644 tests/kms_test_only.c
> > > Hey,
> > > 
> > > What does this test add?
> > This is for VIZ-6956. The idea here is to test atomic commits with
> > TEST_ONLY first, and it that fails we try to commit in real. 
> > > 
> > > Any test should pass with PASS. This test retrying failed test-
> > > only
> > > tests won't add much there. No test should ever fail.
> > That's my thought also. You could argue whether or not this test
> > adds
> > any value. In my mind, this test only tests if TEST_ONLY flag is
> > behaving ok :)
> But you can already compare test passes vs test fails. There's no
> need for a separate test because any test failing should already be
> investigated.
> 
> I don't think a test that tests another test is useful, we already
> have infrastructure for that. :-)
So what should we do about this test? Would it be enough if we just add
these TEST_ONLY flag tests to these igt tests and forget this one that
just tests if the test was successful or not?

> 
> ~Maarten
Maarten Lankhorst Jan. 23, 2017, 3:35 p.m. UTC | #4
Op 23-01-17 om 14:26 schreef Mika Kahola:
> On Thu, 2017-01-12 at 12:37 +0100, Maarten Lankhorst wrote:
>> Hey,
>>
>> Op 12-01-17 om 11:28 schreef Mika Kahola:
>>> On Thu, 2017-01-12 at 11:11 +0100, Maarten Lankhorst wrote:
>>>> Op 30-12-16 om 13:00 schreef Mika Kahola:
>>>>> This test case adds TEST_ONLY flag to the following test cases
>>>>> to
>>>>> test
>>>>> atomic commit correctness.
>>>>>
>>>>>  - kms_plane_multiple
>>>>>  - kms_atomic_transitions
>>>>>  - kms_plane_scaling
>>>>>  - kms_rotation_crc
>>>>>
>>>>> The test randomly selects one of the above test cases and tests
>>>>> atomic
>>>>> commit. If the test fails with TEST_ONLY flag the real deal
>>>>> atomic
>>>>> commit
>>>>> is executed and the outcome is verified.
>>>>>
>>>>> The test runs by default for 64 iterations.
>>>>>
>>>>> Signed-off-by: Mika Kahola <mika.kahola@intel.com>
>>>>> ---
>>>>>  tests/Makefile.sources |   1 +
>>>>>  tests/kms_test_only.c  | 455
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>  2 files changed, 456 insertions(+)
>>>>>  create mode 100644 tests/kms_test_only.c
>>>> Hey,
>>>>
>>>> What does this test add?
>>> This is for VIZ-6956. The idea here is to test atomic commits with
>>> TEST_ONLY first, and it that fails we try to commit in real. 
>>>> Any test should pass with PASS. This test retrying failed test-
>>>> only
>>>> tests won't add much there. No test should ever fail.
>>> That's my thought also. You could argue whether or not this test
>>> adds
>>> any value. In my mind, this test only tests if TEST_ONLY flag is
>>> behaving ok :)
>> But you can already compare test passes vs test fails. There's no
>> need for a separate test because any test failing should already be
>> investigated.
>>
>> I don't think a test that tests another test is useful, we already
>> have infrastructure for that. :-)
> So what should we do about this test? Would it be enough if we just add
> these TEST_ONLY flag tests to these igt tests and forget this one that
> just tests if the test was successful or not?
Yes. But we should also probably determine what we want to test.

Easiest might just be to do each commit twice, once with TEST_ONLY, followed by the same commit done normally.

Maybe we could create a test mode for this in igt core that can be toggled, that performs any atomic commit twice?

Plugging that in would be a whole lot less invasive, only adding a single line to each changed test.

~Maarten
diff mbox

Patch

diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index 6316ea6..ff599c3 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -112,6 +112,7 @@  TESTS_progs_M = \
 	kms_plane \
 	kms_plane_multiple \
 	kms_plane_lowres \
+	kms_test_only \
 	kms_properties \
 	kms_psr_sink_crc \
 	kms_render \
diff --git a/tests/kms_test_only.c b/tests/kms_test_only.c
new file mode 100644
index 0000000..1ae835e
--- /dev/null
+++ b/tests/kms_test_only.c
@@ -0,0 +1,455 @@ 
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "igt.h"
+#include "drmtest.h"
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+
+IGT_TEST_DESCRIPTION("Test atomic mode setting with a TEST_ONLY flag");
+
+#define LOOP_FOREVER     -1
+
+#define FAIL   -1
+#define SKIP    0
+#define SUCCESS 1
+
+/* Command line parameters. */
+struct {
+	int iterations;
+} opt = {
+	.iterations = 64,
+};
+
+static
+int parse_output(char *testname)
+{
+	FILE *fid;
+	char output[1024];
+	char result[32];
+
+	fid = popen(testname, "r");
+	igt_assert(fid != NULL);
+
+	while (fgets(output, sizeof(output)-1, fid) != NULL) {
+		if (strstr(output, "Subtest")) {
+			sscanf(output, "%*s %*s %s%*c", result);
+
+			if (strncmp(result, "FAIL", 4) == 0) {
+				pclose(fid);
+				return FAIL;
+			} else if (strncmp(result, "SKIP", 4) == 0) {
+				pclose(fid);
+				return SKIP;
+			} else if (strncmp(result, "SUCCESS", 7) == 0) {
+				pclose(fid);
+				return SUCCESS;
+			}
+		} else if (strstr(output, "Test requirement not met in function")) {
+			pclose(fid);
+			return SKIP;
+		}
+	}
+
+	pclose(fid);
+
+	return -EINVAL;
+}
+
+static
+void test_kms_rotation_crc(void)
+{
+	int ret;
+	char testname[256];
+
+	strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-180-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-180");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-180-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-180");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest cursor-rotation-180-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest cursor-rotation-180");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-270-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-270");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-90-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-90");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-270-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-270");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-90-pos-100-0-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest sprite-rotation-90-pos-100-0");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest bad-pixel-format-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest bad-pixel-format");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest bad-tiling-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest bad-tiling");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90-flip-stress-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90-flip-stress");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90-Y-tiled-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest primary-rotation-90-Y-tiled");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_rotation_crc --run-subtest exhaust-fences-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_rotation_crc --run-subtest exhaust-fences");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+}
+
+static
+void test_kms_atomic_transition(void)
+{
+	int ret;
+	char testname[256];
+
+	strcpy(testname, "kms_atomic_transition --run-subtest plane-all-transition-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_atomic_transition --run-subtest plane-all-transition");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_atomic_transition --run-subtest plane-all-transition-nonblocking-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_atomic_transition --run-subtest plane-all-transition-nonblocking");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_atomic_transition --run-subtest plane-all-modeset-transition-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_atomic_transition --run-subtest plane-all-modeset-transition");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	strcpy(testname, "kms_atomic_transition --run-subtest plane-toggle-modeset-transition-test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_atomic_transition --run-subtest plane-toggle-modeset-transition");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+
+	for (int i = 1; i <= I915_MAX_PIPES; i++) {
+		sprintf(testname, "kms_atomic_transition --run-subtest %ix-modeset-transitions-test-only", i);
+		ret = parse_output(testname);
+		if (ret == SKIP)
+			return;
+
+		if (ret == FAIL) {
+			igt_info("%s failed. Trying with real atomic commit\n", testname);
+			sprintf(testname, "kms_atomic_transition --run-subtest %ix-modeset-transitions", i);
+			ret = parse_output(testname);
+			igt_assert_eq(ret, FAIL);
+		}
+
+		sprintf(testname, "kms_atomic_transition --run-subtest %ix-modeset-transitions-nonblocking-test-only", i);
+		ret = parse_output(testname);
+		if (ret == SKIP)
+			return;
+
+		if (ret == FAIL) {
+			igt_info("%s failed. Trying with real atomic commit\n", testname);
+			sprintf(testname, "kms_atomic_transition --run-subtest %ix-modeset-transitions-nonblocking", i);
+			ret = parse_output(testname);
+			igt_assert_eq(ret, FAIL);
+		}
+	}
+}
+
+static
+void test_kms_plane_scaling(void)
+{
+	int ret;
+	char testname[256];
+
+	strcpy(testname, "kms_plane_scaling --test-only");
+	ret = parse_output(testname);
+	if (ret == SKIP)
+		return;
+
+	if (ret == FAIL) {
+		igt_info("%s failed. Trying with real atomic commit\n", testname);
+		strcpy(testname, "kms_plane_scaling");
+		ret = parse_output(testname);
+		igt_assert_eq(ret, FAIL);
+	}
+}
+
+static
+void test_kms_plane_multiple(void)
+{
+	char testname[256];
+	const char *tiling_list[4] = {"none", "x", "y", "yf"};
+	int ret;
+
+	for (int pipe = 0; pipe < I915_MAX_PIPES; pipe++) {
+		for (int tiling = 0; tiling < 4; tiling++) {
+			for (int plane = IGT_PLANE_PRIMARY; plane < IGT_MAX_PLANES; plane++) {
+				sprintf(testname, "kms_plane_multiple --run-subtest atomic-pipe-%s-tiling-%s-planes-%d --iterations 1 --test-only",
+					kmstest_pipe_name(pipe), tiling_list[tiling], plane);
+
+				ret = parse_output(testname);
+
+				if (ret == SKIP)
+					return;
+
+				if (ret == FAIL) {
+					igt_info("%s failed. Trying with real atomic commit\n", testname);
+					sprintf(testname, "kms_plane_multiple --run-subtest atomic-pipe-%s-tiling-%s-planes-%d --iterations 1",
+						kmstest_pipe_name(pipe), tiling_list[tiling], plane);
+					ret = parse_output(testname);
+					igt_assert_eq(ret, FAIL);
+				}
+			}
+		}
+	}
+}
+
+static
+int run_test(int index)
+{
+	switch (index) {
+	case 0:
+		igt_info("testing kms_plane_multiple\n");
+		test_kms_plane_multiple();
+		return 0;
+	case 1:
+		igt_info("testing kms_atomic_transition\n");
+		test_kms_atomic_transition();
+		return 1;
+	case 2:
+		igt_info("testing kms_plane_scaling\n");
+		test_kms_plane_scaling();
+		return 2;
+	case 3:
+		igt_info("testing kms_rotation_crc\n");
+		test_kms_rotation_crc();
+		return 3;
+	default:
+		igt_assert(false);
+	}
+}
+
+static int opt_handler(int option, int option_index, void *input)
+{
+	switch (option) {
+	case 'i':
+		opt.iterations = strtol(optarg, NULL, 0);
+
+		if (opt.iterations < LOOP_FOREVER || opt.iterations == 0) {
+			igt_info("incorrect number of iterations\n");
+			igt_assert(false);
+		}
+		break;
+	default:
+		igt_assert(false);
+	}
+
+	return 0;
+}
+
+const char *help_str =
+	"  --iterations Number of iterations for test coverage. -1 loop forever, default 64 iterations\n";
+
+int main(int argc, char *argv[])
+{
+	int i;
+	bool loop_forever;
+
+	struct option long_options[] = {
+		{ "iterations", required_argument, NULL, 'i'},
+		{ 0, 0, 0, 0 }
+	};
+
+	if (opt.iterations == LOOP_FOREVER) {
+		loop_forever = true;
+		opt.iterations =  1;
+	} else {
+		loop_forever = false;
+	}
+
+	igt_subtest_init_parse_opts(&argc, argv, "", long_options, help_str,
+				    opt_handler, NULL);
+
+	igt_skip_on_simulation();
+
+	srand(time(NULL));
+
+	i = 0;
+	while (i < opt.iterations || loop_forever) {
+		run_test(rand() % 4);
+		i++;
+	}
+
+	igt_success();
+
+	igt_exit();
+}