| Message ID | 20221124134429.612467-3-nrb@linux.ibm.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | s390x: test CMM during migration | expand |
On 24/11/2022 14.44, Nico Boehr wrote: > Add a test which modifies CMM page states while migration is in > progress. > > Signed-off-by: Nico Boehr <nrb@linux.ibm.com> > --- > s390x/Makefile | 1 + > s390x/migration-during-cmm.c | 121 +++++++++++++++++++++++++++++++++++ > s390x/unittests.cfg | 5 ++ > 3 files changed, 127 insertions(+) > create mode 100644 s390x/migration-during-cmm.c > > diff --git a/s390x/Makefile b/s390x/Makefile > index 401cb6371cee..64c7c04409ae 100644 > --- a/s390x/Makefile > +++ b/s390x/Makefile > @@ -39,6 +39,7 @@ tests += $(TEST_DIR)/panic-loop-extint.elf > tests += $(TEST_DIR)/panic-loop-pgm.elf > tests += $(TEST_DIR)/migration-sck.elf > tests += $(TEST_DIR)/exittime.elf > +tests += $(TEST_DIR)/migration-during-cmm.elf > > pv-tests += $(TEST_DIR)/pv-diags.elf > > diff --git a/s390x/migration-during-cmm.c b/s390x/migration-during-cmm.c > new file mode 100644 > index 000000000000..afe1f73605ba > --- /dev/null > +++ b/s390x/migration-during-cmm.c > @@ -0,0 +1,121 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Perform CMMA actions while migrating. > + * > + * Copyright IBM Corp. 2022 > + * > + * Authors: > + * Nico Boehr <nrb@linux.ibm.com> > + */ > + > +#include <libcflat.h> > +#include <smp.h> > +#include <asm-generic/barrier.h> > + > +#include "cmm.h" > + > +#define NUM_PAGES 128 > + > +/* > + * Allocate 3 pages more than we need so we can start at different offsets. This ensures page states I'd suggest to break the line after "offsets.", that's better to read. > + * change on every loop iteration. > + */ > +static uint8_t pagebuf[(NUM_PAGES + 3) * PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); > + > +static unsigned int thread_iters; > +static int thread_should_exit; > +static int thread_exited; > +struct cmm_verify_result result; > + > +static void test_cmm_during_migration(void) > +{ > + uint8_t *pagebuf_start; > + /* > + * The second CPU must not print on the console, otherwise it will race with "print to the console" ? ... no clue, I'm not a native speaker ;-) > + * the primary CPU on the SCLP buffer. > + */ > + while (!thread_should_exit) { > + /* > + * Start on a offset different from the last iteration so page states change with > + * every iteration. This is why pagebuf has 3 extra pages. > + */ > + pagebuf_start = pagebuf + (thread_iters % 4) * PAGE_SIZE; > + cmm_set_page_states(pagebuf_start, NUM_PAGES); > + > + /* > + * Always increment even if the verify fails. This ensures primary CPU knows where > + * we left off and can do an additional verify round after migration finished. > + */ > + thread_iters++; > + > + result = cmm_verify_page_states(pagebuf_start, NUM_PAGES); > + if (result.verify_failed) > + goto out; A "break" would be nicer than a goto. > + } > + > +out: > + WRITE_ONCE(thread_exited, 1); > +} > + > +int main(void) > +{ > + bool has_essa = check_essa_available(); > + struct psw psw; > + > + report_prefix_push("migration-during-cmm"); > + if (!has_essa) { > + report_skip("ESSA is not available"); > + goto error; > + } > + > + if (smp_query_num_cpus() == 1) { > + report_skip("need at least 2 cpus for this test"); > + goto error; > + } > + > + psw.mask = extract_psw_mask(); > + psw.addr = (unsigned long)test_cmm_during_migration; > + smp_cpu_setup(1, psw); > + > + puts("Please migrate me, then press return\n"); > + (void)getchar(); > + > + WRITE_ONCE(thread_should_exit, 1); > + > + while (!thread_exited) > + mb(); > + > + report_info("thread completed %u iterations", thread_iters); > + > + report_prefix_push("during migration"); > + cmm_report_verify(&result); > + report_prefix_pop(); > + > + /* > + * Verification of page states occurs on the thread. We don't know if we > + * were still migrating during the verification. > + * To be sure, make another verification round after the migration > + * finished to catch page states which might not have been migrated > + * correctly. > + */ > + report_prefix_push("after migration"); > + assert(thread_iters > 0); > + result = cmm_verify_page_states(pagebuf + ((thread_iters - 1) % 4) * PAGE_SIZE, NUM_PAGES); > + cmm_report_verify(&result); > + report_prefix_pop(); > + > + goto done; > + > +error: > + /* > + * If we just exit and don't ask migrate_cmd to migrate us, it > + * will just hang forever. Hence, also ask for migration when we > + * skip this test alltogether. s/alltogether/all together/ > + */ > + puts("Please migrate me, then press return\n"); > + (void)getchar(); > + > +done: > + report_prefix_pop(); > + return report_summary(); > +} > diff --git a/s390x/unittests.cfg b/s390x/unittests.cfg > index 3caf81eda396..f6889bd4da01 100644 > --- a/s390x/unittests.cfg > +++ b/s390x/unittests.cfg > @@ -208,3 +208,8 @@ groups = migration > [exittime] > file = exittime.elf > smp = 2 > + > +[migration-during-cmm] > +file = migration-during-cmm.elf > +groups = migration > +smp = 2 With the nits fixed: Reviewed-by: Thomas Huth <thuth@redhat.com>
On Fri, 25 Nov 2022 14:58:35 +0100 Thomas Huth <thuth@redhat.com> wrote: [...] > > + /* > > + * If we just exit and don't ask migrate_cmd to migrate us, it > > + * will just hang forever. Hence, also ask for migration when we > > + * skip this test alltogether. > > s/alltogether/all together/ nope, it's "altogether"
On 25/11/2022 15.05, Claudio Imbrenda wrote: > On Fri, 25 Nov 2022 14:58:35 +0100 > Thomas Huth <thuth@redhat.com> wrote: > > [...] > >>> + /* >>> + * If we just exit and don't ask migrate_cmd to migrate us, it >>> + * will just hang forever. Hence, also ask for migration when we >>> + * skip this test alltogether. >> >> s/alltogether/all together/ > > nope, it's "altogether" Drat, it must be close to the weekend already ;-) Thomas
On Thu, 24 Nov 2022 14:44:29 +0100 Nico Boehr <nrb@linux.ibm.com> wrote: > Add a test which modifies CMM page states while migration is in > progress. > > Signed-off-by: Nico Boehr <nrb@linux.ibm.com> > --- > s390x/Makefile | 1 + > s390x/migration-during-cmm.c | 121 +++++++++++++++++++++++++++++++++++ > s390x/unittests.cfg | 5 ++ > 3 files changed, 127 insertions(+) > create mode 100644 s390x/migration-during-cmm.c > > diff --git a/s390x/Makefile b/s390x/Makefile > index 401cb6371cee..64c7c04409ae 100644 > --- a/s390x/Makefile > +++ b/s390x/Makefile > @@ -39,6 +39,7 @@ tests += $(TEST_DIR)/panic-loop-extint.elf > tests += $(TEST_DIR)/panic-loop-pgm.elf > tests += $(TEST_DIR)/migration-sck.elf > tests += $(TEST_DIR)/exittime.elf > +tests += $(TEST_DIR)/migration-during-cmm.elf > > pv-tests += $(TEST_DIR)/pv-diags.elf > > diff --git a/s390x/migration-during-cmm.c b/s390x/migration-during-cmm.c > new file mode 100644 > index 000000000000..afe1f73605ba > --- /dev/null > +++ b/s390x/migration-during-cmm.c > @@ -0,0 +1,121 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Perform CMMA actions while migrating. > + * > + * Copyright IBM Corp. 2022 > + * > + * Authors: > + * Nico Boehr <nrb@linux.ibm.com> > + */ > + > +#include <libcflat.h> > +#include <smp.h> > +#include <asm-generic/barrier.h> > + > +#include "cmm.h" > + > +#define NUM_PAGES 128 > + > +/* > + * Allocate 3 pages more than we need so we can start at different offsets. This ensures page states > + * change on every loop iteration. > + */ > +static uint8_t pagebuf[(NUM_PAGES + 3) * PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); > + > +static unsigned int thread_iters; > +static int thread_should_exit; > +static int thread_exited; > +struct cmm_verify_result result; > + > +static void test_cmm_during_migration(void) > +{ > + uint8_t *pagebuf_start; > + /* > + * The second CPU must not print on the console, otherwise it will race with > + * the primary CPU on the SCLP buffer. > + */ > + while (!thread_should_exit) { I don't see any mb()s here, maybe use READ_ONCE > + /* > + * Start on a offset different from the last iteration so page states change with > + * every iteration. This is why pagebuf has 3 extra pages. > + */ > + pagebuf_start = pagebuf + (thread_iters % 4) * PAGE_SIZE; > + cmm_set_page_states(pagebuf_start, NUM_PAGES); > + > + /* > + * Always increment even if the verify fails. This ensures primary CPU knows where > + * we left off and can do an additional verify round after migration finished. > + */ > + thread_iters++; > + > + result = cmm_verify_page_states(pagebuf_start, NUM_PAGES); > + if (result.verify_failed) > + goto out; > + } > + > +out: > + WRITE_ONCE(thread_exited, 1); > +} > + > +int main(void) > +{ > + bool has_essa = check_essa_available(); > + struct psw psw; > + > + report_prefix_push("migration-during-cmm"); > + if (!has_essa) { > + report_skip("ESSA is not available"); > + goto error; > + } > + > + if (smp_query_num_cpus() == 1) { > + report_skip("need at least 2 cpus for this test"); > + goto error; > + } > + > + psw.mask = extract_psw_mask(); > + psw.addr = (unsigned long)test_cmm_during_migration; > + smp_cpu_setup(1, psw); > + > + puts("Please migrate me, then press return\n"); > + (void)getchar(); > + > + WRITE_ONCE(thread_should_exit, 1); > + > + while (!thread_exited) > + mb(); > + > + report_info("thread completed %u iterations", thread_iters); > + > + report_prefix_push("during migration"); > + cmm_report_verify(&result); > + report_prefix_pop(); > + > + /* > + * Verification of page states occurs on the thread. We don't know if we > + * were still migrating during the verification. > + * To be sure, make another verification round after the migration > + * finished to catch page states which might not have been migrated > + * correctly. > + */ > + report_prefix_push("after migration"); > + assert(thread_iters > 0); > + result = cmm_verify_page_states(pagebuf + ((thread_iters - 1) % 4) * PAGE_SIZE, NUM_PAGES); > + cmm_report_verify(&result); > + report_prefix_pop(); > + > + goto done; > + > +error: > + /* > + * If we just exit and don't ask migrate_cmd to migrate us, it > + * will just hang forever. Hence, also ask for migration when we > + * skip this test alltogether. > + */ > + puts("Please migrate me, then press return\n"); > + (void)getchar(); > + I wonder if this can be made easier to review and less error prone... maybe something like: static void migrate_once(void) { static bool migrated = false; if (migrated) return; puts("Please migrate me, then press return\n"); migrated = true; (void)getchar(); } then you can put it where needed in the code _and_ unconditionally at the end > +done: > + report_prefix_pop(); > + return report_summary(); > +} > diff --git a/s390x/unittests.cfg b/s390x/unittests.cfg > index 3caf81eda396..f6889bd4da01 100644 > --- a/s390x/unittests.cfg > +++ b/s390x/unittests.cfg > @@ -208,3 +208,8 @@ groups = migration > [exittime] > file = exittime.elf > smp = 2 > + > +[migration-during-cmm] > +file = migration-during-cmm.elf > +groups = migration > +smp = 2
diff --git a/s390x/Makefile b/s390x/Makefile index 401cb6371cee..64c7c04409ae 100644 --- a/s390x/Makefile +++ b/s390x/Makefile @@ -39,6 +39,7 @@ tests += $(TEST_DIR)/panic-loop-extint.elf tests += $(TEST_DIR)/panic-loop-pgm.elf tests += $(TEST_DIR)/migration-sck.elf tests += $(TEST_DIR)/exittime.elf +tests += $(TEST_DIR)/migration-during-cmm.elf pv-tests += $(TEST_DIR)/pv-diags.elf diff --git a/s390x/migration-during-cmm.c b/s390x/migration-during-cmm.c new file mode 100644 index 000000000000..afe1f73605ba --- /dev/null +++ b/s390x/migration-during-cmm.c @@ -0,0 +1,121 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Perform CMMA actions while migrating. + * + * Copyright IBM Corp. 2022 + * + * Authors: + * Nico Boehr <nrb@linux.ibm.com> + */ + +#include <libcflat.h> +#include <smp.h> +#include <asm-generic/barrier.h> + +#include "cmm.h" + +#define NUM_PAGES 128 + +/* + * Allocate 3 pages more than we need so we can start at different offsets. This ensures page states + * change on every loop iteration. + */ +static uint8_t pagebuf[(NUM_PAGES + 3) * PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + +static unsigned int thread_iters; +static int thread_should_exit; +static int thread_exited; +struct cmm_verify_result result; + +static void test_cmm_during_migration(void) +{ + uint8_t *pagebuf_start; + /* + * The second CPU must not print on the console, otherwise it will race with + * the primary CPU on the SCLP buffer. + */ + while (!thread_should_exit) { + /* + * Start on a offset different from the last iteration so page states change with + * every iteration. This is why pagebuf has 3 extra pages. + */ + pagebuf_start = pagebuf + (thread_iters % 4) * PAGE_SIZE; + cmm_set_page_states(pagebuf_start, NUM_PAGES); + + /* + * Always increment even if the verify fails. This ensures primary CPU knows where + * we left off and can do an additional verify round after migration finished. + */ + thread_iters++; + + result = cmm_verify_page_states(pagebuf_start, NUM_PAGES); + if (result.verify_failed) + goto out; + } + +out: + WRITE_ONCE(thread_exited, 1); +} + +int main(void) +{ + bool has_essa = check_essa_available(); + struct psw psw; + + report_prefix_push("migration-during-cmm"); + if (!has_essa) { + report_skip("ESSA is not available"); + goto error; + } + + if (smp_query_num_cpus() == 1) { + report_skip("need at least 2 cpus for this test"); + goto error; + } + + psw.mask = extract_psw_mask(); + psw.addr = (unsigned long)test_cmm_during_migration; + smp_cpu_setup(1, psw); + + puts("Please migrate me, then press return\n"); + (void)getchar(); + + WRITE_ONCE(thread_should_exit, 1); + + while (!thread_exited) + mb(); + + report_info("thread completed %u iterations", thread_iters); + + report_prefix_push("during migration"); + cmm_report_verify(&result); + report_prefix_pop(); + + /* + * Verification of page states occurs on the thread. We don't know if we + * were still migrating during the verification. + * To be sure, make another verification round after the migration + * finished to catch page states which might not have been migrated + * correctly. + */ + report_prefix_push("after migration"); + assert(thread_iters > 0); + result = cmm_verify_page_states(pagebuf + ((thread_iters - 1) % 4) * PAGE_SIZE, NUM_PAGES); + cmm_report_verify(&result); + report_prefix_pop(); + + goto done; + +error: + /* + * If we just exit and don't ask migrate_cmd to migrate us, it + * will just hang forever. Hence, also ask for migration when we + * skip this test alltogether. + */ + puts("Please migrate me, then press return\n"); + (void)getchar(); + +done: + report_prefix_pop(); + return report_summary(); +} diff --git a/s390x/unittests.cfg b/s390x/unittests.cfg index 3caf81eda396..f6889bd4da01 100644 --- a/s390x/unittests.cfg +++ b/s390x/unittests.cfg @@ -208,3 +208,8 @@ groups = migration [exittime] file = exittime.elf smp = 2 + +[migration-during-cmm] +file = migration-during-cmm.elf +groups = migration +smp = 2
Add a test which modifies CMM page states while migration is in progress. Signed-off-by: Nico Boehr <nrb@linux.ibm.com> --- s390x/Makefile | 1 + s390x/migration-during-cmm.c | 121 +++++++++++++++++++++++++++++++++++ s390x/unittests.cfg | 5 ++ 3 files changed, 127 insertions(+) create mode 100644 s390x/migration-during-cmm.c