From patchwork Thu Nov 24 16:10:32 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 9446049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CF0186071C for ; Thu, 24 Nov 2016 16:25:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C0EF32808F for ; Thu, 24 Nov 2016 16:25:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B5939280DE; Thu, 24 Nov 2016 16:25:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C4DCF2808F for ; Thu, 24 Nov 2016 16:25:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1c9wnm-0005ea-EQ; Thu, 24 Nov 2016 16:23:10 +0000 Received: from mail-wm0-x22c.google.com ([2a00:1450:400c:c09::22c]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1c9wcW-0003VX-0b for linux-arm-kernel@lists.infradead.org; Thu, 24 Nov 2016 16:11:48 +0000 Received: by mail-wm0-x22c.google.com with SMTP id a197so119263065wmd.0 for ; Thu, 24 Nov 2016 08:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=31kI1HCC6x9RDRo19d0ss0Rx6i6p+tckkCczobHa3YU=; b=QbW8BdQyKDupio+eqNZgFBp985Oj/sYZe8e80jxMRPjDabeBX5A/wVqspUInhX+ysW EpvK+AU6MDnKd+sWNKMsf+Tgbv3/TRRZM38D3B9Ef8yxjyafJFPf0nv8dIUTsPwYhqfz uNx1J89oEMlgwIosG+ECY6shMRB4fKPq1l7kE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=31kI1HCC6x9RDRo19d0ss0Rx6i6p+tckkCczobHa3YU=; b=TFPNwSZDHk6ynndY9M/Yr+w20BQrhYPguQyTkGOAG+4Z4QbPX45UM8DtBtc3lJtMtV QT+d6iTsoVsMGaPSvW2oik/aGobgs8einsrclw8mbPZQ8BJQ96pPP/YVKo5U7x8/HDQa jPy5/CkUEC/372YiCIVGSoW3Kq1NGdEgXMU25emsi0+2Fy4djjlXLUDqqFIOKMuYAlc7 PAWlggtZIkRMddtWMkw5LMdt6UO9u6sHUrIJesHu80u/gLhxBY604aIk17/O6MLu9hLK 79wHbxtQP7yraXLVmcSta5wiEMQZpYcvEAchDZk4A7HuRXSerAQpqiQndqAnEQeIPojh cTlw== X-Gm-Message-State: AKaTC01lqMPVjdDxNpNqPF6wWmXqUH35sioXMofC2ZTiNAXbgMZ0gk26B64Qb2HQJq+cBrMy X-Received: by 10.28.214.84 with SMTP id n81mr2901826wmg.120.1480003870019; Thu, 24 Nov 2016 08:11:10 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 135sm8916527wmh.14.2016.11.24.08.11.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Nov 2016 08:11:06 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 461E03E0585; Thu, 24 Nov 2016 16:11:00 +0000 (GMT) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, christoffer.dall@linaro.org, marc.zyngier@arm.com Subject: [kvm-unit-tests PATCH v7 10/11] arm/barrier-litmus-tests: add simple mp and sal litmus tests Date: Thu, 24 Nov 2016 16:10:32 +0000 Message-Id: <20161124161033.11456-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.10.1 In-Reply-To: <20161124161033.11456-1-alex.bennee@linaro.org> References: <20161124161033.11456-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20161124_081132_913415_65A1883D X-CRM114-Status: GOOD ( 24.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mttcg@listserver.greensocs.com, peter.maydell@linaro.org, claudio.fontana@huawei.com, nikunj@linux.vnet.ibm.com, jan.kiszka@siemens.com, mark.burton@greensocs.com, a.rigo@virtualopensystems.com, Will Deacon , qemu-devel@nongnu.org, cota@braap.org, serge.fdrv@gmail.com, pbonzini@redhat.com, bobby.prani@gmail.com, rth@twiddle.net, =?UTF-8?q?Alex=20Benn=C3=A9e?= , fred.konrad@greensocs.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP This adds a framework for adding simple barrier litmus tests against ARM. The litmus tests aren't as comprehensive as the academic exercises which will attempt to do all sorts of things to keep racing CPUs synced up. These tests do honour the "sync" parameter to do a poor-mans equivalent. The two litmus tests are: - message passing - store-after-load They both have case that should fail (although won't on single-threaded TCG setups). If barriers aren't working properly the store-after-load test will fail even on an x86 backend as x86 allows re-ording of non aliased stores. I've imported a few more of the barrier primatives from the Linux source tree so we consistently use macros. The arm64 barrier primitives trip up on -Wstrict-aliasing so this is disabled in the Makefile. Signed-off-by: Alex Bennée CC: Will Deacon --- v7 - merge in store-after-load - clean-up sync-up code - use new counter api - fix xfail for sal test v6 - add a unittest.cfg - -fno-strict-aliasing --- Makefile | 2 + arm/Makefile.common | 2 + arm/barrier-litmus-test.c | 437 ++++++++++++++++++++++++++++++++++++++++++++++ arm/unittests.cfg | 36 ++++ lib/arm/asm/barrier.h | 63 ++++++- lib/arm64/asm/barrier.h | 50 ++++++ 6 files changed, 589 insertions(+), 1 deletion(-) create mode 100644 arm/barrier-litmus-test.c diff --git a/Makefile b/Makefile index 5201472..53594a1 100644 --- a/Makefile +++ b/Makefile @@ -51,10 +51,12 @@ fomit_frame_pointer := $(call cc-option, $(frame-pointer-flag), "") fnostack_protector := $(call cc-option, -fno-stack-protector, "") fnostack_protector_all := $(call cc-option, -fno-stack-protector-all, "") wno_frame_address := $(call cc-option, -Wno-frame-address, "") +fno_strict_aliasing := $(call cc-option, -fno-strict-aliasing, "") CFLAGS += $(fomit_frame_pointer) CFLAGS += $(fno_stack_protector) CFLAGS += $(fno_stack_protector_all) CFLAGS += $(wno_frame_address) +CFLAGS += $(fno_strict_aliasing) CXXFLAGS += $(CFLAGS) diff --git a/arm/Makefile.common b/arm/Makefile.common index eb4cfdf..a508128 100644 --- a/arm/Makefile.common +++ b/arm/Makefile.common @@ -16,6 +16,7 @@ tests-common += $(TEST_DIR)/gic.flat tests-common += $(TEST_DIR)/tlbflush-code.flat tests-common += $(TEST_DIR)/tlbflush-data.flat tests-common += $(TEST_DIR)/locking-test.flat +tests-common += $(TEST_DIR)/barrier-litmus-test.flat all: test_cases @@ -87,3 +88,4 @@ $(TEST_DIR)/selftest.o $(cstart.o): $(asm-offsets) $(TEST_DIR)/tlbflush-code.elf: $(cstart.o) $(TEST_DIR)/tlbflush-code.o $(TEST_DIR)/tlbflush-data.elf: $(cstart.o) $(TEST_DIR)/tlbflush-data.o $(TEST_DIR)/locking-test.elf: $(cstart.o) $(TEST_DIR)/locking-test.o +$(TEST_DIR)/barrier-litmus-test.elf: $(cstart.o) $(TEST_DIR)/barrier-litmus-test.o diff --git a/arm/barrier-litmus-test.c b/arm/barrier-litmus-test.c new file mode 100644 index 0000000..2557a88 --- /dev/null +++ b/arm/barrier-litmus-test.c @@ -0,0 +1,437 @@ +/* + * ARM Barrier Litmus Tests + * + * This test provides a framework for testing barrier conditions on + * the processor. It's simpler than the more involved barrier testing + * frameworks as we are looking for simple failures of QEMU's TCG not + * weird edge cases the silicon gets wrong. + */ + +#include +#include +#include +#include +#include + +#define MAX_CPUS 8 + +/* Array size and access controls */ +static int array_size = 100000; +static int wait_if_ahead = 0; + +static cpumask_t cpu_mask; + +/* + * These test_array_* structures are a contiguous array modified by two or more + * competing CPUs. The padding is to ensure the variables do not share + * cache lines. + * + * All structures start zeroed. + */ + +typedef struct test_array +{ + volatile unsigned int x; + uint8_t dummy[64]; + volatile unsigned int y; + uint8_t dummy2[64]; + volatile unsigned int r[MAX_CPUS]; +} test_array; + +volatile test_array *array; + +/* Test definition structure + * + * The first function will always run on the primary CPU, it is + * usually the one that will detect any weirdness and trigger the + * failure of the test. + */ + +typedef void (*test_fn)(void); + +typedef struct { + const char *test_name; + bool should_pass; + test_fn main_fn; + test_fn secondary_fns[MAX_CPUS-1]; +} test_descr_t; + +/* Litmus tests */ + +static unsigned long sync_start(void) +{ + const unsigned long gate_mask = ~0x3ffff; + unsigned long gate, now; + gate = get_cntvct() & gate_mask; + do { + now =get_cntvct(); + } while ((now & gate_mask)==gate); + + return now; +} + +/* Simple Message Passing + * + * x is the message data + * y is the flag to indicate the data is ready + * + * Reading x == 0 when y == 1 is a failure. + */ + +void message_passing_write(void) +{ + int i; + + sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + entry->x = 1; + entry->y = 1; + } + + halt(); +} + +void message_passing_read(void) +{ + int i; + int errors = 0, ready = 0; + + sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + unsigned int x,y; + y = entry->y; + x = entry->x; + + if (y && !x) + errors++; + ready += y; + } + + report_xfail("mp: %d errors, %d ready", true, errors == 0, errors, ready); +} + +/* Simple Message Passing with barriers */ +void message_passing_write_barrier(void) +{ + int i; + sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + entry->x = 1; + smp_wmb(); + entry->y = 1; + } + + halt(); +} + +void message_passing_read_barrier(void) +{ + int i; + int errors = 0, ready = 0, not_ready = 0; + + sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + unsigned int x, y; + y = entry->y; + smp_rmb(); + x = entry->x; + + if (y && !x) + errors++; + + if (y) { + ready++; + } else { + not_ready++; + + if (not_ready > 2) { + entry = &array[i+1]; + do { + not_ready = 0; + } while (wait_if_ahead && !entry->y); + } + } + } + + report("mp barrier: %d errors, %d ready", errors == 0, errors, ready); +} + +/* Simple Message Passing with Acquire/Release */ +void message_passing_write_release(void) +{ + int i; + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + entry->x = 1; + smp_store_release(&entry->y, 1); + } + + halt(); +} + +void message_passing_read_acquire(void) +{ + int i; + int errors = 0, ready = 0, not_ready = 0; + + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + unsigned int x, y; + y = smp_load_acquire(&entry->y); + x = entry->x; + + if (y && !x) + errors++; + + if (y) { + ready++; + } else { + not_ready++; + + if (not_ready > 2) { + entry = &array[i+1]; + do { + not_ready = 0; + } while (wait_if_ahead && !entry->y); + } + } + } + + report("mp acqrel: %d errors, %d ready", errors == 0, errors, ready); +} + +/* + * Store after load + * + * T1: write 1 to x, load r from y + * T2: write 1 to y, load r from x + * + * Without memory fence r[0] && r[1] == 0 + * With memory fence both == 0 should be impossible + */ + +static void check_store_and_load_results(char *name, int thread, bool xfail, + unsigned long start, unsigned long end) +{ + int i; + int neither = 0; + int only_first = 0; + int only_second = 0; + int both = 0; + + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + if (entry->r[0] == 0 && + entry->r[1] == 0) { + neither++; + } else if (entry->r[0] && + entry->r[1]) { + both++; + } else if (entry->r[0]) { + only_first++; + } else { + only_second++; + } + } + + printf("T%d: %08lx->%08lx neither=%d only_t1=%d only_t2=%d both=%d\n", thread, + start, end, neither, only_first, only_second, both); + + if (thread == 1) { + if (xfail) { + report_xfail("%s: errors=%d", true, neither==0, + name, neither); + } else { + report("%s: errors=%d", neither==0, name, neither); + } + + } +} + +/* + * This attempts to synchronise the start of both threads to roughly + * the same time. On real hardware there is a little latency as the + * secondary vCPUs are powered up however this effect it much more + * exaggerated on a TCG host. + * + * Busy waits until the we pass a future point in time, returns final + * start time. + */ + +void store_and_load_1(void) +{ + int i; + unsigned long start, end; + + start = sync_start(); + for (i=0; ix = 1; + r = entry->y; + entry->r[0] = r; + } + end = get_cntvct(); + + smp_mb(); + + while (!cpumask_test_cpu(1, &cpu_mask)) + cpu_relax(); + + check_store_and_load_results("sal", 1, true, start, end); +} + +void store_and_load_2(void) +{ + int i; + unsigned long start, end; + + start = sync_start(); + for (i=0; iy = 1; + r = entry->x; + entry->r[1] = r; + } + end = get_cntvct(); + + check_store_and_load_results("sal", 2, true, start, end); + + cpumask_set_cpu(1, &cpu_mask); + + halt(); +} + +void store_and_load_barrier_1(void) +{ + int i; + unsigned long start, end; + + start = sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + unsigned int r; + entry->x = 1; + smp_mb(); + r = entry->y; + entry->r[0] = r; + } + end = get_cntvct(); + + smp_mb(); + + while (!cpumask_test_cpu(1, &cpu_mask)) + cpu_relax(); + + check_store_and_load_results("sal_barrier", 1, false, start, end); +} + +void store_and_load_barrier_2(void) +{ + int i; + unsigned long start, end; + + start = sync_start(); + for (i=0; i< array_size; i++) { + volatile test_array *entry = &array[i]; + unsigned int r; + entry->y = 1; + smp_mb(); + r = entry->x; + entry->r[1] = r; + } + end = get_cntvct(); + + check_store_and_load_results("sal_barrier", 2, false, start, end); + + cpumask_set_cpu(1, &cpu_mask); + + halt(); +} + + +/* Test array */ +static test_descr_t tests[] = { + + { "mp", false, + message_passing_read, + { message_passing_write } + }, + + { "mp_barrier", true, + message_passing_read_barrier, + { message_passing_write_barrier } + }, + + { "mp_acqrel", true, + message_passing_read_acquire, + { message_passing_write_release } + }, + + { "sal", false, + store_and_load_1, + { store_and_load_2 } + }, + + { "sal_barrier", true, + store_and_load_barrier_1, + { store_and_load_barrier_2 } + }, +}; + + +void setup_and_run_litmus(test_descr_t *test) +{ + array = calloc(array_size, sizeof(test_array)); + + if (array) { + int i = 0; + printf("Allocated test array @ %p\n", array); + + while (test->secondary_fns[i]) { + smp_boot_secondary(i+1, test->secondary_fns[i]); + i++; + } + + test->main_fn(); + } else { + report("%s: failed to allocate memory",false, test->test_name); + } +} + +int main(int argc, char **argv) +{ + int i; + unsigned int j; + test_descr_t *test = NULL; + + for (i=0; i4?4:$MAX_SMP)) extra_params = -append 'excl' groups = locking accel = tcg + +[barrier-litmus::mp] +file = barrier-litmus-test.flat +smp = 2 +extra_params = -append 'mp' +groups = barrier +accel = tcg + +[barrier-litmus::mp-barrier] +file = barrier-litmus-test.flat +smp = 2 +extra_params = -append 'mp_barrier' +groups = barrier +accel = tcg + +[barrier-litmus::mp-acqrel] +file = barrier-litmus-test.flat +smp = 2 +extra_params = -append 'mp_acqrel' +groups = barrier +accel = tcg + +[barrier-litmus::sal] +file = barrier-litmus-test.flat +smp = 2 +extra_params = -append 'sal' +groups = barrier +accel = tcg + +[barrier-litmus::sal-barrier] +file = barrier-litmus-test.flat +smp = 2 +extra_params = -append 'sal_barrier' +groups = barrier +accel = tcg + diff --git a/lib/arm/asm/barrier.h b/lib/arm/asm/barrier.h index 394a4a2..e3b7a2e 100644 --- a/lib/arm/asm/barrier.h +++ b/lib/arm/asm/barrier.h @@ -1,9 +1,11 @@ #ifndef _ASMARM_BARRIER_H_ #define _ASMARM_BARRIER_H_ /* - * Adapted form arch/arm/include/asm/barrier.h + * Adapted from arch/arm/include/asm/barrier.h */ +#include + #define sev() asm volatile("sev" : : : "memory") #define wfe() asm volatile("wfe" : : : "memory") #define wfi() asm volatile("wfi" : : : "memory") @@ -20,4 +22,63 @@ #define smp_rmb() smp_mb() #define smp_wmb() dmb(ishst) +extern void abort(void); + +static inline void __write_once_size(volatile void *p, void *res, int size) +{ + switch (size) { + case 1: *(volatile uint8_t *)p = *(uint8_t *)res; break; + case 2: *(volatile uint16_t *)p = *(uint16_t *)res; break; + case 4: *(volatile uint32_t *)p = *(uint32_t *)res; break; + case 8: *(volatile uint64_t *)p = *(uint64_t *)res; break; + default: + /* unhandled case */ + abort(); + } +} + +#define WRITE_ONCE(x, val) \ +({ \ + union { typeof(x) __val; char __c[1]; } __u = \ + { .__val = (typeof(x)) (val) }; \ + __write_once_size(&(x), __u.__c, sizeof(x)); \ + __u.__val; \ +}) + +#define smp_store_release(p, v) \ +do { \ + smp_mb(); \ + WRITE_ONCE(*p, v); \ +} while (0) + + +static inline +void __read_once_size(const volatile void *p, void *res, int size) +{ + switch (size) { + case 1: *(uint8_t *)res = *(volatile uint8_t *)p; break; + case 2: *(uint16_t *)res = *(volatile uint16_t *)p; break; + case 4: *(uint32_t *)res = *(volatile uint32_t *)p; break; + case 8: *(uint64_t *)res = *(volatile uint64_t *)p; break; + default: + /* unhandled case */ + abort(); + } +} + +#define READ_ONCE(x) \ +({ \ + union { typeof(x) __val; char __c[1]; } __u; \ + __read_once_size(&(x), __u.__c, sizeof(x)); \ + __u.__val; \ +}) + + +#define smp_load_acquire(p) \ +({ \ + typeof(*p) ___p1 = READ_ONCE(*p); \ + smp_mb(); \ + ___p1; \ +}) + #endif /* _ASMARM_BARRIER_H_ */ diff --git a/lib/arm64/asm/barrier.h b/lib/arm64/asm/barrier.h index dbdac9d..aafabdc 100644 --- a/lib/arm64/asm/barrier.h +++ b/lib/arm64/asm/barrier.h @@ -19,4 +19,54 @@ #define smp_rmb() dmb(ishld) #define smp_wmb() dmb(ishst) +#define smp_store_release(p, v) \ +do { \ + switch (sizeof(*p)) { \ + case 1: \ + asm volatile ("stlrb %w1, %0" \ + : "=Q" (*p) : "r" (v) : "memory"); \ + break; \ + case 2: \ + asm volatile ("stlrh %w1, %0" \ + : "=Q" (*p) : "r" (v) : "memory"); \ + break; \ + case 4: \ + asm volatile ("stlr %w1, %0" \ + : "=Q" (*p) : "r" (v) : "memory"); \ + break; \ + case 8: \ + asm volatile ("stlr %1, %0" \ + : "=Q" (*p) : "r" (v) : "memory"); \ + break; \ + } \ +} while (0) + +#define smp_load_acquire(p) \ +({ \ + union { typeof(*p) __val; char __c[1]; } __u; \ + switch (sizeof(*p)) { \ + case 1: \ + asm volatile ("ldarb %w0, %1" \ + : "=r" (*(u8 *)__u.__c) \ + : "Q" (*p) : "memory"); \ + break; \ + case 2: \ + asm volatile ("ldarh %w0, %1" \ + : "=r" (*(u16 *)__u.__c) \ + : "Q" (*p) : "memory"); \ + break; \ + case 4: \ + asm volatile ("ldar %w0, %1" \ + : "=r" (*(u32 *)__u.__c) \ + : "Q" (*p) : "memory"); \ + break; \ + case 8: \ + asm volatile ("ldar %0, %1" \ + : "=r" (*(u64 *)__u.__c) \ + : "Q" (*p) : "memory"); \ + break; \ + } \ + __u.__val; \ +}) + #endif /* _ASMARM64_BARRIER_H_ */