diff mbox

[i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

Message ID 1476703717-21149-1-git-send-email-mika.kahola@intel.com (mailing list archive)
State New, archived
Headers show

Commit Message

Mika Kahola Oct. 17, 2016, 11:28 a.m. UTC
This is a testcase with multiple planes. The idea here is the following

 - draw a uniform frame with blue color
 - grab crc for reference
 - put planes randomly on top with the same blue color
 - punch holes with black color into the primary framebuffer
 - ideally the planes should cover these holes so that the output is the
   identical to reference crc
 - composite all with one ioctl call
 - grab crc and verify that the reference crc is equal
 - repeat this for dozen iterations to maximize coverage

v3: Cleanup by removing separate plane array
    For atomic, pass DRM_MODE_PAGE_FLIP_EVENT
    Grab crc by using igt_pipe_crc_get_crc instead of igt_pipe_crc_collect_crc
    Rename nplanes variable to max_planes
    To optimize test execution, run iterations after the modeset

v2: Keep a logfile on random number seeds per subtest that are not skipped
    due to unmet test requirements

Signed-off-by: Mika Kahola <mika.kahola@intel.com>
---
 tests/Makefile.sources     |   1 +
 tests/kms_plane_multiple.c | 423 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 424 insertions(+)
 create mode 100644 tests/kms_plane_multiple.c

Comments

Daniel Vetter Oct. 17, 2016, 2:30 p.m. UTC | #1
On Mon, Oct 17, 2016 at 02:28:37PM +0300, Mika Kahola wrote:
> +	for (int i = 0; i < iterations; i++) {
> +		igt_info("%d/%d: Testing connector %s using pipe %s with %d planes\n",
> +			 i+1, iterations, igt_output_name(output),
> +			 kmstest_pipe_name(pipe), max_planes);
> +
> +		test_init(data, pipe);
> +
> +		test_grab_crc(data, output, pipe, &blue, tiling,
> +			      &test.reference_crc);
> +
> +		test_planes(data, pipe, &blue, tiling, max_planes, output);
> +
> +		if (test_atomic) {
> +			igt_display_commit_atomic(&data->display,
> +						  DRM_MODE_PAGE_FLIP_EVENT,
> +						  &data->display);
> +		} else
> +			igt_display_commit2(&data->display, COMMIT_LEGACY);
> +
> +		igt_pipe_crc_start(data->pipe_crc);
> +		n = igt_pipe_crc_get_crcs(data->pipe_crc, 1, &crc);
> +		igt_assert_eq(n, 1);
> +		igt_pipe_crc_stop(data->pipe_crc);

Comment on testing method here: With atomic we don't just require that the
result looks good at the end, but also that _every_ frame is perfect. That
means you need a slightly different test sequence:

1. Enable crc capture.

2. Create a new atomic state (randomized, whatever) which should in the
end still result in the same screen contents using the punchout box trick.
This depends upon the exact subtest.

3. Commit the state from step 2.

4. Wait to make sure the atomic commit has completed. If you do an async
commit, that means waiting for the flip_event to get signalled (didn't see
code for that anywhere).

5. Fetch all the crc values (if there's not a bug in your code or in the
kernel it should be just 1) and make sure they are _all_ the right value.
Your code here only grabs 1 crc after the atomic commit completed, which
means if there's tearing or underruns you'll miss them. Which means it
won't actually validate the crucial feature for which we've created
atomic!

6. Go back to 2.

7. After enough loops, stop crc capturing.

Note that this is the loop for ALLOW_MODESET==false atomic commits, i.e.
where every atomic commit should take at most 1 vblank interval. If any of
the commits take longer than that, there's a bug in either the kernel or
your testcase. Note that crc_start alone has a few vblank waits (due to
crc bugs on some platforms) which will break this.

Cheers, Daniel

> +
> +		igt_assert_crc_equal(&test.reference_crc, crc);
> +
> +		test_fini(data, output, max_planes);
> +	}
> +}
Mika Kahola Oct. 19, 2016, 9:10 a.m. UTC | #2
On Mon, 2016-10-17 at 16:30 +0200, Daniel Vetter wrote:
> On Mon, Oct 17, 2016 at 02:28:37PM +0300, Mika Kahola wrote:
> > 
> > +	for (int i = 0; i < iterations; i++) {
> > +		igt_info("%d/%d: Testing connector %s using pipe
> > %s with %d planes\n",
> > +			 i+1, iterations, igt_output_name(output),
> > +			 kmstest_pipe_name(pipe), max_planes);
> > +
> > +		test_init(data, pipe);
> > +
> > +		test_grab_crc(data, output, pipe, &blue, tiling,
> > +			      &test.reference_crc);
> > +
> > +		test_planes(data, pipe, &blue, tiling, max_planes,
> > output);
> > +
> > +		if (test_atomic) {
> > +			igt_display_commit_atomic(&data->display,
> > +						  DRM_MODE_PAGE_FL
> > IP_EVENT,
> > +						  &data->display);
> > +		} else
> > +			igt_display_commit2(&data->display,
> > COMMIT_LEGACY);
> > +
> > +		igt_pipe_crc_start(data->pipe_crc);
> > +		n = igt_pipe_crc_get_crcs(data->pipe_crc, 1,
> > &crc);
> > +		igt_assert_eq(n, 1);
> > +		igt_pipe_crc_stop(data->pipe_crc);
> Comment on testing method here: With atomic we don't just require
> that the
> result looks good at the end, but also that _every_ frame is perfect.
> That
> means you need a slightly different test sequence:
> 
> 1. Enable crc capture.
> 
> 2. Create a new atomic state (randomized, whatever) which should in
> the
> end still result in the same screen contents using the punchout box
> trick.
> This depends upon the exact subtest.
> 
> 3. Commit the state from step 2.
> 
> 4. Wait to make sure the atomic commit has completed. If you do an
> async
> commit, that means waiting for the flip_event to get signalled
> (didn't see
> code for that anywhere).
> 
> 5. Fetch all the crc values (if there's not a bug in your code or in
> the
> kernel it should be just 1) and make sure they are _all_ the right
> value.
> Your code here only grabs 1 crc after the atomic commit completed,
> which
> means if there's tearing or underruns you'll miss them. Which means
> it
> won't actually validate the crucial feature for which we've created
> atomic!
That's true, I missed that one. I'll make modification on the test
sequence and spin another round of this test.

Cheers,
Mika

> 
> 6. Go back to 2.
> 
> 7. After enough loops, stop crc capturing.
> 
> Note that this is the loop for ALLOW_MODESET==false atomic commits,
> i.e.
> where every atomic commit should take at most 1 vblank interval. If
> any of
> the commits take longer than that, there's a bug in either the kernel
> or
> your testcase. Note that crc_start alone has a few vblank waits (due
> to
> crc bugs on some platforms) which will break this.
> 
> Cheers, Daniel
> 
> > 
> > +
> > +		igt_assert_crc_equal(&test.reference_crc, crc);
> > +
> > +		test_fini(data, output, max_planes);
> > +	}
> > +}
diff mbox

Patch

diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index 6d081c3..ffd59c1 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -105,6 +105,7 @@  TESTS_progs_M = \
 	kms_pipe_color \
 	kms_pipe_crc_basic \
 	kms_plane \
+	kms_plane_multiple \
 	kms_properties \
 	kms_psr_sink_crc \
 	kms_render \
diff --git a/tests/kms_plane_multiple.c b/tests/kms_plane_multiple.c
new file mode 100644
index 0000000..0cb7552
--- /dev/null
+++ b/tests/kms_plane_multiple.c
@@ -0,0 +1,423 @@ 
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "igt.h"
+#include "drmtest.h"
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+
+IGT_TEST_DESCRIPTION("Test atomic mode setting with multiple planes ");
+
+#define SIZE 128
+
+typedef struct {
+	float red;
+	float green;
+	float blue;
+} color_t;
+
+typedef struct {
+	int drm_fd;
+	igt_display_t display;
+	igt_pipe_crc_t *pipe_crc;
+	igt_plane_t *plane[IGT_MAX_PLANES];
+	struct igt_fb fb[IGT_MAX_PLANES];
+} data_t;
+
+typedef struct {
+	data_t *data;
+	igt_crc_t reference_crc;
+} test_position_t;
+
+/* Command line parameters. */
+struct {
+	bool user_seed;
+	int seed;
+	bool user_logfile;
+	char logfile[SIZE];
+} opt = {
+	.user_seed = false,
+	.seed = 1,
+	.user_logfile = false,
+	.logfile = "kms_plane_multiple.log",
+};
+
+
+static int logwrite(const char *testname)
+{
+	time_t curr_time;
+	FILE *fid;
+	char *time_str;
+
+	fid = fopen(opt.logfile, "a");
+
+	if (fid == NULL) {
+		igt_debug("Could not open file %s\n", opt.logfile);
+		return -1;
+	}
+
+	curr_time = time(NULL);
+
+	time_str = ctime(&curr_time);
+	time_str[strlen(time_str)-1] = '\0';
+
+	fprintf(fid, "%s: kms_plane_multiple --run-subtest %s --seed %d\n",
+		time_str, testname, opt.seed);
+
+	fclose(fid);
+
+	return 0;
+}
+
+/*
+ * Common code across all tests, acting on data_t
+ */
+static void test_init(data_t *data, enum pipe pipe)
+{
+	data->pipe_crc = igt_pipe_crc_new(pipe, INTEL_PIPE_CRC_SOURCE_AUTO);
+}
+
+static void test_fini(data_t *data, igt_output_t *output, int max_planes)
+{
+	igt_plane_set_fb(data->plane[IGT_PLANE_PRIMARY], NULL);
+
+	for (int i = IGT_PLANE_2; i <= max_planes; i++)
+		igt_plane_set_fb(data->plane[i], NULL);
+
+	/* reset the constraint on the pipe */
+	igt_output_set_pipe(output, PIPE_ANY);
+
+	igt_pipe_crc_free(data->pipe_crc);
+}
+
+static bool test_is_atomic(const char *testname)
+{
+	if (strncmp(testname, "atomic", 6) == 0)
+		return true;
+
+	return false;
+}
+
+static void
+test_grab_crc(data_t *data, igt_output_t *output, enum pipe pipe,
+	      color_t *color, uint64_t tiling, igt_crc_t *crc /* out */)
+{
+	struct igt_fb fb;
+	drmModeModeInfo *mode;
+	igt_plane_t *primary;
+	int n;
+	bool test_atomic = test_is_atomic(igt_subtest_name());
+
+	igt_output_set_pipe(output, pipe);
+
+	primary = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
+
+	mode = igt_output_get_mode(output);
+
+	igt_create_color_fb(data->drm_fd, mode->hdisplay, mode->vdisplay,
+			    DRM_FORMAT_XRGB8888,
+			    LOCAL_DRM_FORMAT_MOD_NONE,
+			    color->red, color->green, color->blue,
+			    &fb);
+
+	igt_plane_set_fb(primary, &fb);
+
+	igt_display_commit2(&data->display,
+			    test_atomic ? COMMIT_ATOMIC : COMMIT_LEGACY);
+
+	igt_pipe_crc_start(data->pipe_crc);
+	n = igt_pipe_crc_get_crcs(data->pipe_crc, 1, &crc);
+	igt_assert_eq(n, 1);
+	igt_pipe_crc_stop(data->pipe_crc);
+
+	igt_plane_set_fb(primary, NULL);
+
+	igt_display_commit2(&data->display,
+			    test_atomic ? COMMIT_ATOMIC : COMMIT_LEGACY);
+
+	igt_remove_fb(data->drm_fd, &fb);
+}
+
+/*
+ * Multiple plane position test.
+ *   - We start by grabbing a reference CRC of a full blue fb being scanned
+ *     out on the primary plane
+ *   - Then we scannout number of planes:
+ *      * the primary plane uses a blue fb with a black rectangle hole
+ *      * planes, on top of the primary plane, with a blue fb that is set-up
+ *        to cover the black rectangles of the primary plane fb
+ *     The resulting CRC should be identical to the reference CRC
+ */
+
+static void
+create_fb_for_mode_position(data_t *data, drmModeModeInfo *mode,
+			    color_t *color, int *rect_x, int *rect_y,
+			    int rect_w, int rect_h, uint64_t tiling,
+			    int max_planes)
+{
+	unsigned int fb_id;
+	cairo_t *cr;
+
+	fb_id = igt_create_fb(data->drm_fd,
+			      mode->hdisplay, mode->vdisplay,
+			      DRM_FORMAT_XRGB8888,
+			      tiling,
+			      &data->fb[IGT_PLANE_PRIMARY]);
+	igt_assert(fb_id);
+
+	cr = igt_get_cairo_ctx(data->drm_fd, &data->fb[IGT_PLANE_PRIMARY]);
+	igt_paint_color(cr, rect_x[0], rect_y[0],
+			mode->hdisplay, mode->vdisplay,
+			color->red, color->green, color->blue);
+
+	for (int i = IGT_PLANE_2; i <= max_planes; i++)
+		igt_paint_color(cr, rect_x[i], rect_y[i],
+				rect_w, rect_h, 0.0, 0.0, 0.0);
+
+	igt_assert(cairo_status(cr) == 0);
+	cairo_destroy(cr);
+}
+
+
+static void
+test_planes(data_t *data, enum pipe pipe, color_t *color,
+	    uint64_t tiling, int max_planes, igt_output_t *output)
+{
+	drmModeModeInfo *mode;
+	int x[IGT_MAX_PLANES];
+	int y[IGT_MAX_PLANES];
+	int i;
+
+	igt_output_set_pipe(output, pipe);
+
+	mode = igt_output_get_mode(output);
+
+	/* sprite planes with random positions */
+	x[IGT_PLANE_PRIMARY] = 0;
+	y[IGT_PLANE_PRIMARY] = 0;
+	for (i = IGT_PLANE_2; i <= max_planes; i++) {
+		x[i] = rand() % (mode->hdisplay - SIZE);
+		y[i] = rand() % (mode->vdisplay - SIZE);
+
+		data->plane[i] = igt_output_get_plane(output, i);
+
+		igt_create_color_fb(data->drm_fd,
+				    SIZE, SIZE, /* width, height */
+				    data->plane[i]->is_cursor ? DRM_FORMAT_ARGB8888 : DRM_FORMAT_XRGB8888,
+				    tiling,
+				    color->red, color->green, color->blue,
+				    &data->fb[i]);
+
+		igt_plane_set_position(data->plane[i], x[i], y[i]);
+		igt_plane_set_fb(data->plane[i], &data->fb[i]);
+	}
+
+	/* primary plane */
+	data->plane[IGT_PLANE_PRIMARY] = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
+	create_fb_for_mode_position(data, mode, color, x, y,
+				    SIZE, SIZE, tiling, max_planes);
+	igt_plane_set_fb(data->plane[IGT_PLANE_PRIMARY], &data->fb[IGT_PLANE_PRIMARY]);
+}
+
+static void
+test_plane_position_with_output(data_t *data, enum pipe pipe, igt_output_t *output,
+				int max_planes, uint64_t tiling)
+{
+	test_position_t test = { .data = data };
+	igt_crc_t *crc = NULL;
+	color_t blue  = { 0.0f, 0.0f, 1.0f };
+	int n;
+	bool test_atomic = test_is_atomic(igt_subtest_name());
+	int iterations = 12;
+
+	if (test_atomic)
+		igt_require(data->display.is_atomic);
+
+	igt_info("running test for dozen iterations\n");
+
+	for (int i = 0; i < iterations; i++) {
+		igt_info("%d/%d: Testing connector %s using pipe %s with %d planes\n",
+			 i+1, iterations, igt_output_name(output),
+			 kmstest_pipe_name(pipe), max_planes);
+
+		test_init(data, pipe);
+
+		test_grab_crc(data, output, pipe, &blue, tiling,
+			      &test.reference_crc);
+
+		test_planes(data, pipe, &blue, tiling, max_planes, output);
+
+		if (test_atomic) {
+			igt_display_commit_atomic(&data->display,
+						  DRM_MODE_PAGE_FLIP_EVENT,
+						  &data->display);
+		} else
+			igt_display_commit2(&data->display, COMMIT_LEGACY);
+
+		igt_pipe_crc_start(data->pipe_crc);
+		n = igt_pipe_crc_get_crcs(data->pipe_crc, 1, &crc);
+		igt_assert_eq(n, 1);
+		igt_pipe_crc_stop(data->pipe_crc);
+
+		igt_assert_crc_equal(&test.reference_crc, crc);
+
+		test_fini(data, output, max_planes);
+	}
+}
+
+static void
+test_plane_position(data_t *data, enum pipe pipe, int max_planes,
+		    uint64_t tiling)
+{
+	igt_output_t *output;
+	int connected_outs;
+
+	igt_skip_on(pipe >= data->display.n_pipes);
+	igt_skip_on(max_planes >= data->display.pipes[pipe].n_planes);
+
+	if (!opt.user_seed)
+		opt.seed = time(NULL);
+
+	srand(opt.seed);
+	logwrite(igt_subtest_name());
+
+	connected_outs = 0;
+	for_each_connected_output(&data->display, output) {
+		test_plane_position_with_output(data, pipe, output,
+						max_planes, tiling);
+		connected_outs++;
+	}
+
+	igt_skip_on(connected_outs == 0);
+
+}
+
+static void
+run_tests_for_pipe_plane(data_t *data, enum pipe pipe, int max_planes)
+{
+	igt_subtest_f("legacy-pipe-%s-tiling-none-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_DRM_FORMAT_MOD_NONE);
+
+	igt_subtest_f("atomic-pipe-%s-tiling-none-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_X_TILED);
+
+	igt_subtest_f("legacy-pipe-%s-tiling-x-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_X_TILED);
+
+	igt_subtest_f("atomic-pipe-%s-tiling-x-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_X_TILED);
+
+	igt_subtest_f("legacy-pipe-%s-tiling-y-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_Y_TILED);
+
+	igt_subtest_f("atomic-pipe-%s-tiling-y-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_Y_TILED);
+
+	igt_subtest_f("legacy-pipe-%s-tiling-yf-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_Yf_TILED);
+
+	igt_subtest_f("atomic-pipe-%s-tiling-yf-planes-%d",
+		      kmstest_pipe_name(pipe), max_planes)
+		test_plane_position(data, pipe, max_planes, LOCAL_I915_FORMAT_MOD_Yf_TILED);
+}
+
+static void
+run_tests_for_pipe(data_t *data, enum pipe pipe)
+{
+	for (int planes = 1; planes < IGT_MAX_PLANES; planes++)
+		run_tests_for_pipe_plane(data, pipe, planes);
+}
+
+static data_t data;
+
+static int opt_handler(int option, int option_index, void *input)
+{
+	switch (option) {
+	case 's':
+		opt.user_seed = true;
+		opt.seed = strtol(optarg, NULL, 0);
+		break;
+	case 'l':
+		opt.user_logfile = true;
+		strcpy(opt.logfile, optarg);
+		break;
+	default:
+		igt_assert(false);
+	}
+
+	return 0;
+}
+
+const char *help_str =
+	"  --seed       Seed for random number generator\n"
+	"  --logfile    Logfile to store seeds for random number generator, default 'kms_plane_multiple.log'\n";
+
+int main(int argc, char *argv[])
+{
+	time_t t = time(NULL);
+	struct tm tm = *localtime(&t);
+
+	struct option long_options[] = {
+		{ "seed",    required_argument, NULL, 's'},
+		{ "logfile", required_argument, NULL, 'l'},
+		{ 0, 0, 0, 0 }
+	};
+
+	igt_subtest_init_parse_opts(&argc, argv, "", long_options, help_str,
+				    opt_handler, NULL);
+
+	if (!opt.user_logfile)
+		sprintf(opt.logfile, "kms_plane_multiple-%4d-%2d-%2d.log",
+			tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday);
+
+	igt_skip_on_simulation();
+
+	igt_fixture {
+		data.drm_fd = drm_open_driver_master(DRIVER_INTEL);
+
+		kmstest_set_vt_graphics_mode();
+
+		igt_require_pipe_crc();
+		igt_display_init(&data.display, data.drm_fd);
+	}
+
+	for (int pipe = 0; pipe < I915_MAX_PIPES; pipe++)
+		run_tests_for_pipe(&data, pipe);
+
+	igt_fixture {
+		igt_display_fini(&data.display);
+	}
+
+	igt_exit();
+}