| Message ID | 20220414105322.577439-2-thuth@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: s390: selftests: Provide TAP output in tests | expand |
On 4/14/22 12:53, Thomas Huth wrote: > The memop test currently does not have any output (unless one of the > TEST_ASSERT statement fails), so it's hard to say for a user whether > a certain new sub-test has been included in the binary or not. Let's > make this a little bit more user-friendly and include some TAP output > via the kselftests.h interface. > > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > tools/testing/selftests/kvm/s390x/memop.c | 90 ++++++++++++++++++----- > 1 file changed, 73 insertions(+), 17 deletions(-) > > diff --git a/tools/testing/selftests/kvm/s390x/memop.c b/tools/testing/selftests/kvm/s390x/memop.c > index b04c2c1b3c30..a2783d9afcac 100644 > --- a/tools/testing/selftests/kvm/s390x/memop.c > +++ b/tools/testing/selftests/kvm/s390x/memop.c > @@ -12,6 +12,7 @@ > > #include "test_util.h" > #include "kvm_util.h" > +#include "kselftest.h" > > enum mop_target { > LOGICAL, > @@ -648,33 +649,88 @@ static void test_errors(void) > kvm_vm_free(t.kvm_vm); > } > > +struct testdef { > + const char *name; > + void (*test)(void); > + bool needs_extension; Please make this numeric. You could also rename it to required_extension or similar. > +} testlist[] = { > + { > + .name = "simple copy", > + .test = test_copy, > + .needs_extension = false, > + }, > + { > + .name = "copy with storage keys", > + .test = test_copy_key, > + .needs_extension = true, > + }, > + { > + .name = "copy with key storage protection override", > + .test = test_copy_key_storage_prot_override, > + .needs_extension = true, > + }, > + { > + .name = "copy with key fetch protection", > + .test = test_copy_key_fetch_prot, > + .needs_extension = true, > + }, > + { > + .name = "copy with key fetch protection override", > + .test = test_copy_key_fetch_prot_override, > + .needs_extension = true, > + }, > + { > + .name = "error checks with key", > + .test = test_errors_key, > + .needs_extension = true, > + }, > + { > + .name = "error checks with key storage protection override", > + .test = test_errors_key_storage_prot_override, > + .needs_extension = true, > + }, > + { > + .name = "error checks without key fetch prot override", > + .test = test_errors_key_fetch_prot_override_not_enabled, > + .needs_extension = true, > + }, > + { > + .name = "error checks with key fetch prot override", > + .test = test_errors_key_fetch_prot_override_enabled, > + .needs_extension = true, > + }, > + { > + .name = "generic error checks", > + .test = test_errors, > + .needs_extension = false, > + }, > +}; > + > int main(int argc, char *argv[]) > { > - int memop_cap, extension_cap; > + int memop_cap, extension_cap, idx; > > setbuf(stdout, NULL); /* Tell stdout not to buffer its content */ > > + ksft_print_header(); > + > memop_cap = kvm_check_cap(KVM_CAP_S390_MEM_OP); > extension_cap = kvm_check_cap(KVM_CAP_S390_MEM_OP_EXTENSION); > if (!memop_cap) { > - print_skip("CAP_S390_MEM_OP not supported"); > - exit(KSFT_SKIP); > + ksft_exit_skip("CAP_S390_MEM_OP not supported.\n"); > } > > - test_copy(); > - if (extension_cap > 0) { > - test_copy_key(); > - test_copy_key_storage_prot_override(); > - test_copy_key_fetch_prot(); > - test_copy_key_fetch_prot_override(); > - test_errors_key(); > - test_errors_key_storage_prot_override(); > - test_errors_key_fetch_prot_override_not_enabled(); > - test_errors_key_fetch_prot_override_enabled(); > - } else { > - print_skip("storage key memop extension not supported"); > + ksft_set_plan(ARRAY_SIZE(testlist)); > + > + for (idx = 0; idx < ARRAY_SIZE(testlist); idx++) { > + if (!testlist[idx].needs_extension || extension_cap) { Then check here that extension_cap >= the required extension. This way the test can easily be adapted in case of future extensions. > + testlist[idx].test(); > + ksft_test_result_pass("%s\n", testlist[idx].name); > + } else { > + ksft_test_result_skip("%s - storage key memop not supported\n", > + testlist[idx].name); > + } > } > - test_errors(); > > - return 0; > + ksft_finished(); > }
On 14/04/2022 14.48, Janis Schoetterl-Glausch wrote: > On 4/14/22 12:53, Thomas Huth wrote: >> The memop test currently does not have any output (unless one of the >> TEST_ASSERT statement fails), so it's hard to say for a user whether >> a certain new sub-test has been included in the binary or not. Let's >> make this a little bit more user-friendly and include some TAP output >> via the kselftests.h interface. >> >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> tools/testing/selftests/kvm/s390x/memop.c | 90 ++++++++++++++++++----- >> 1 file changed, 73 insertions(+), 17 deletions(-) >> >> diff --git a/tools/testing/selftests/kvm/s390x/memop.c b/tools/testing/selftests/kvm/s390x/memop.c >> index b04c2c1b3c30..a2783d9afcac 100644 >> --- a/tools/testing/selftests/kvm/s390x/memop.c >> +++ b/tools/testing/selftests/kvm/s390x/memop.c >> @@ -12,6 +12,7 @@ >> >> #include "test_util.h" >> #include "kvm_util.h" >> +#include "kselftest.h" >> >> enum mop_target { >> LOGICAL, >> @@ -648,33 +649,88 @@ static void test_errors(void) >> kvm_vm_free(t.kvm_vm); >> } >> >> +struct testdef { >> + const char *name; >> + void (*test)(void); >> + bool needs_extension; > > Please make this numeric. You could also rename it to required_extension or similar. [...] >> + >> + for (idx = 0; idx < ARRAY_SIZE(testlist); idx++) { >> + if (!testlist[idx].needs_extension || extension_cap) { > > Then check here that extension_cap >= the required extension. > This way the test can easily be adapted in case of future extensions. Not sure whether a ">=" will really be safe, since a future extension does not necessarily assert that previous extensions are available at the same time. But I can still turn the bool into a numeric to make it a little bit more flexible for future use. Thomas
On 4/19/22 19:40, Thomas Huth wrote: > On 14/04/2022 14.48, Janis Schoetterl-Glausch wrote: >> On 4/14/22 12:53, Thomas Huth wrote: >>> The memop test currently does not have any output (unless one of the >>> TEST_ASSERT statement fails), so it's hard to say for a user whether >>> a certain new sub-test has been included in the binary or not. Let's >>> make this a little bit more user-friendly and include some TAP output >>> via the kselftests.h interface. >>> >>> Signed-off-by: Thomas Huth <thuth@redhat.com> >>> --- >>> tools/testing/selftests/kvm/s390x/memop.c | 90 ++++++++++++++++++----- >>> 1 file changed, 73 insertions(+), 17 deletions(-) >>> >>> diff --git a/tools/testing/selftests/kvm/s390x/memop.c b/tools/testing/selftests/kvm/s390x/memop.c >>> index b04c2c1b3c30..a2783d9afcac 100644 >>> --- a/tools/testing/selftests/kvm/s390x/memop.c >>> +++ b/tools/testing/selftests/kvm/s390x/memop.c >>> @@ -12,6 +12,7 @@ >>> #include "test_util.h" >>> #include "kvm_util.h" >>> +#include "kselftest.h" >>> enum mop_target { >>> LOGICAL, >>> @@ -648,33 +649,88 @@ static void test_errors(void) >>> kvm_vm_free(t.kvm_vm); >>> } >>> +struct testdef { >>> + const char *name; >>> + void (*test)(void); >>> + bool needs_extension; >> >> Please make this numeric. You could also rename it to required_extension or similar. > [...] >>> + >>> + for (idx = 0; idx < ARRAY_SIZE(testlist); idx++) { >>> + if (!testlist[idx].needs_extension || extension_cap) { >> >> Then check here that extension_cap >= the required extension. >> This way the test can easily be adapted in case of future extensions. > > Not sure whether a ">=" will really be safe, since a future extension does not necessarily assert that previous extensions are available at the same time. Hmm, I intend for that to hold. In any case, for the existing extension we have committed to it, e.g. the documentation says: Absolute accesses are permitted for the VM ioctl if KVM_CAP_S390_MEM_OP_EXTENSION is > 0. So, if we introduce an extension and allow for it to be removed with a higher extension number, when we add testing support for that extension we'd have to change the capability check, but the existing test case would not break. I guess the most flexible way would be to initialize the array in the middle of main, then you could do .skip = !extension_cap and in the future whatever expression makes sense, but it's kinda ugly and should not be necessary anyway. > > But I can still turn the bool into a numeric to make it a little bit more flexible for future use. > > Thomas >
diff --git a/tools/testing/selftests/kvm/s390x/memop.c b/tools/testing/selftests/kvm/s390x/memop.c index b04c2c1b3c30..a2783d9afcac 100644 --- a/tools/testing/selftests/kvm/s390x/memop.c +++ b/tools/testing/selftests/kvm/s390x/memop.c @@ -12,6 +12,7 @@ #include "test_util.h" #include "kvm_util.h" +#include "kselftest.h" enum mop_target { LOGICAL, @@ -648,33 +649,88 @@ static void test_errors(void) kvm_vm_free(t.kvm_vm); } +struct testdef { + const char *name; + void (*test)(void); + bool needs_extension; +} testlist[] = { + { + .name = "simple copy", + .test = test_copy, + .needs_extension = false, + }, + { + .name = "copy with storage keys", + .test = test_copy_key, + .needs_extension = true, + }, + { + .name = "copy with key storage protection override", + .test = test_copy_key_storage_prot_override, + .needs_extension = true, + }, + { + .name = "copy with key fetch protection", + .test = test_copy_key_fetch_prot, + .needs_extension = true, + }, + { + .name = "copy with key fetch protection override", + .test = test_copy_key_fetch_prot_override, + .needs_extension = true, + }, + { + .name = "error checks with key", + .test = test_errors_key, + .needs_extension = true, + }, + { + .name = "error checks with key storage protection override", + .test = test_errors_key_storage_prot_override, + .needs_extension = true, + }, + { + .name = "error checks without key fetch prot override", + .test = test_errors_key_fetch_prot_override_not_enabled, + .needs_extension = true, + }, + { + .name = "error checks with key fetch prot override", + .test = test_errors_key_fetch_prot_override_enabled, + .needs_extension = true, + }, + { + .name = "generic error checks", + .test = test_errors, + .needs_extension = false, + }, +}; + int main(int argc, char *argv[]) { - int memop_cap, extension_cap; + int memop_cap, extension_cap, idx; setbuf(stdout, NULL); /* Tell stdout not to buffer its content */ + ksft_print_header(); + memop_cap = kvm_check_cap(KVM_CAP_S390_MEM_OP); extension_cap = kvm_check_cap(KVM_CAP_S390_MEM_OP_EXTENSION); if (!memop_cap) { - print_skip("CAP_S390_MEM_OP not supported"); - exit(KSFT_SKIP); + ksft_exit_skip("CAP_S390_MEM_OP not supported.\n"); } - test_copy(); - if (extension_cap > 0) { - test_copy_key(); - test_copy_key_storage_prot_override(); - test_copy_key_fetch_prot(); - test_copy_key_fetch_prot_override(); - test_errors_key(); - test_errors_key_storage_prot_override(); - test_errors_key_fetch_prot_override_not_enabled(); - test_errors_key_fetch_prot_override_enabled(); - } else { - print_skip("storage key memop extension not supported"); + ksft_set_plan(ARRAY_SIZE(testlist)); + + for (idx = 0; idx < ARRAY_SIZE(testlist); idx++) { + if (!testlist[idx].needs_extension || extension_cap) { + testlist[idx].test(); + ksft_test_result_pass("%s\n", testlist[idx].name); + } else { + ksft_test_result_skip("%s - storage key memop not supported\n", + testlist[idx].name); + } } - test_errors(); - return 0; + ksft_finished(); }
The memop test currently does not have any output (unless one of the TEST_ASSERT statement fails), so it's hard to say for a user whether a certain new sub-test has been included in the binary or not. Let's make this a little bit more user-friendly and include some TAP output via the kselftests.h interface. Signed-off-by: Thomas Huth <thuth@redhat.com> --- tools/testing/selftests/kvm/s390x/memop.c | 90 ++++++++++++++++++----- 1 file changed, 73 insertions(+), 17 deletions(-)