diff mbox series

testsuite: Add facility to skip tests.

Message ID 20200516125412.12366-1-marius@devup.no (mailing list archive)
State New, archived
Headers show
Series testsuite: Add facility to skip tests. | expand

Commit Message

Marius Bakke May 16, 2020, 12:54 p.m. UTC
The Makefile helpfully warns that some tests will fail when
--sysconfdir != /etc, but there are no provisions to easily disable
those.  This commit provides an escape hatch.
---
 testsuite/testsuite.c | 9 +++++++++
 testsuite/testsuite.h | 1 +
 2 files changed, 10 insertions(+)

Comments

Lucas De Marchi July 8, 2020, 6:16 p.m. UTC | #1
On Sat, May 16, 2020 at 6:04 AM Marius Bakke <marius@devup.no> wrote:
>
> The Makefile helpfully warns that some tests will fail when
> --sysconfdir != /etc, but there are no provisions to easily disable
> those.  This commit provides an escape hatch.

nice... but are we missing a patch to actually make it skip?

> ---
>  testsuite/testsuite.c | 9 +++++++++
>  testsuite/testsuite.h | 1 +
>  2 files changed, 10 insertions(+)
>
> diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
> index e46f3d8..ff41057 100644
> --- a/testsuite/testsuite.c
> +++ b/testsuite/testsuite.c
> @@ -37,6 +37,7 @@
>  #include "testsuite.h"
>
>  static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
> +static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
>  static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
>  static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
>
> @@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
>         int err;
>         bool matchout, match_modules;
>
> +       if (t->skip == true) {

only if (t->skip)  would be less verbose and preferred I think.

thanks
Lucas de Marchi

> +               LOG("%sSKIPPED%s: %s\n",
> +                       ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
> +                       t->name);
> +               err = EXIT_SUCCESS;
> +               goto exit;
> +       }
> +
>         /* Close write-fds */
>         if (t->output.out != NULL)
>                 close(fdout[1]);
> diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
> index 7ed96bf..8029c64 100644
> --- a/testsuite/testsuite.h
> +++ b/testsuite/testsuite.h
> @@ -109,6 +109,7 @@ struct test {
>         const struct keyval *env_vars;
>         bool need_spawn;
>         bool expected_fail;
> +       bool skip;
>         bool print_outputs;
>  } __attribute__((aligned(8)));
>
> --
> 2.26.2
>
Marius Bakke Aug. 1, 2020, 3:58 p.m. UTC | #2
Lucas De Marchi <lucas.de.marchi@gmail.com> writes:

> On Sat, May 16, 2020 at 6:04 AM Marius Bakke <marius@devup.no> wrote:
>>
>> The Makefile helpfully warns that some tests will fail when
>> --sysconfdir != /etc, but there are no provisions to easily disable
>> those.  This commit provides an escape hatch.
>
> nice... but are we missing a patch to actually make it skip?

Uh...  That makes sense.  (I maintained a list down-stream, derp.)

>> @@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
>>         int err;
>>         bool matchout, match_modules;
>>
>> +       if (t->skip == true) {
>
> only if (t->skip)  would be less verbose and preferred I think.

Fixed in v2, thanks!

Marius Bakke (2):
  testsuite: Add facility to skip tests.
  testsuite: Automatically skip tests that fail when sysconfdir != /etc.

 Makefile.am                | 12 +++++-------
 configure.ac               |  2 ++
 testsuite/test-blacklist.c |  3 +++
 testsuite/test-depmod.c    | 12 ++++++++++++
 testsuite/test-modprobe.c  |  6 ++++++
 testsuite/testsuite.c      |  9 +++++++++
 testsuite/testsuite.h      |  1 +
 7 files changed, 38 insertions(+), 7 deletions(-)
diff mbox series

Patch

diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
index e46f3d8..ff41057 100644
--- a/testsuite/testsuite.c
+++ b/testsuite/testsuite.c
@@ -37,6 +37,7 @@ 
 #include "testsuite.h"
 
 static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
+static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
 static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
 static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
 
@@ -948,6 +949,14 @@  static inline int test_run_parent(const struct test *t, int fdout[2],
 	int err;
 	bool matchout, match_modules;
 
+	if (t->skip == true) {
+		LOG("%sSKIPPED%s: %s\n",
+			ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
+			t->name);
+		err = EXIT_SUCCESS;
+		goto exit;
+	}
+
 	/* Close write-fds */
 	if (t->output.out != NULL)
 		close(fdout[1]);
diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
index 7ed96bf..8029c64 100644
--- a/testsuite/testsuite.h
+++ b/testsuite/testsuite.h
@@ -109,6 +109,7 @@  struct test {
 	const struct keyval *env_vars;
 	bool need_spawn;
 	bool expected_fail;
+	bool skip;
 	bool print_outputs;
 } __attribute__((aligned(8)));