From patchwork Tue Sep 5 21:42:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13375099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2697CA100D for ; Tue, 5 Sep 2023 21:42:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 229608E0011; Tue, 5 Sep 2023 17:42:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B24E8E000B; Tue, 5 Sep 2023 17:42:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF9058E0011; Tue, 5 Sep 2023 17:42:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DB7C58E000B for ; Tue, 5 Sep 2023 17:42:51 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AAD03140922 for ; Tue, 5 Sep 2023 21:42:51 +0000 (UTC) X-FDA: 81203868942.20.29C50F2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 788974001B for ; Tue, 5 Sep 2023 21:42:49 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JEdXCOat; spf=pass (imf27.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693950169; a=rsa-sha256; cv=none; b=rmbXXrDqoobGkJpVubW6GQXy4CQUm5kouFiiPHznyramqWu1KpCA1e8C9qrfmaUGhM+lXE 6ibQJM0DRfMs28OpKioK5wO1uQ3h8Hq5AHFgRybdeWDMQpkbMrHKpb6fW0blnIPcTYuK2g HW721YXAZEhJQ+nXSzrRkKZqvn1C/UY= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JEdXCOat; spf=pass (imf27.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693950169; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eXvYIDhk9Xunttv7gLZeZ6m5tePV5F/iEIjQTI0WMa4=; b=rfmZ4ykL80spwnzbw+ANC2K5SSFWz23h2XeaOrFs4ccWMJDxWwoPgeUTNB7Sp+14yN/Hj5 U7ilzWseupB3+aQ2mmUs+u9x6yYwvg2XBYzLQNZqGCvOfsUuuxvLyd3D3De7HajPiPtnjn 6E1duldUqGHycM4ZGHz9ETAMqZFkB6E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1693950168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eXvYIDhk9Xunttv7gLZeZ6m5tePV5F/iEIjQTI0WMa4=; b=JEdXCOatA2jMcoVeya76c9mQfTHTUSW3YrzXtrrtXbUmqOSnnKFGu6P7cE5ic+0U0rRdON 4/lZn8QdEExkOaU2MY2KCXhmEmsOPOXf7Iv0qNqWnDd8Epb6lxUjz4NTsdHaExhO7f3xfI eLzkyRLHiyfrK6gQk/yZigEQm/DUEEc= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-73-sVDiqNIaNXCyPXss2LVgrw-1; Tue, 05 Sep 2023 17:42:47 -0400 X-MC-Unique: sVDiqNIaNXCyPXss2LVgrw-1 Received: by mail-qk1-f200.google.com with SMTP id af79cd13be357-76f097b28ecso19719385a.1 for ; Tue, 05 Sep 2023 14:42:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693950167; x=1694554967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eXvYIDhk9Xunttv7gLZeZ6m5tePV5F/iEIjQTI0WMa4=; b=fy78zhciwFUwf52RgDYbfA/luyYU+9TtIyT7mvJUCXKSJCDfPWm2ANjCZsYSmn1UxE viuOWxrqg9+r30DweVcJDk5hnmQI3TIKV0s5RRTzYc10mVvhukF3bG2nQM0pmbFveuDG gvREBzdMSE2ofXXKVlMchVhAwLD+lcX7ubzWFPHr6/9G7/QtTowFoLWoT+alHGVl4I5a IavVgFrP2z3EAoukrPkp2BnE/afrlc8P1uDRs3d0AQC2kA08gFWFWxSAnopZNyf3Yvmt bmOruAj6EL5SpAQn8guqjGJJ6f/DFcet9Dx7FU7ojlo1QpgqveYrZmw8qk6qTc7yPtNo CLLA== X-Gm-Message-State: AOJu0YzugkEKBL8EzzQfF6zkHEt1gUUXKlqGnfL94LERU6w13yM7+Cnb h0od/eOt8oqMrNMrdJ3hy/Eey0NKs9bKpWbfMWiMUj5IApd8PEPaK7ur0Dh5VH68XSBG9MPRTbx EFnEMjolm+JY= X-Received: by 2002:a05:620a:1a26:b0:76c:ed4e:ac10 with SMTP id bk38-20020a05620a1a2600b0076ced4eac10mr16770227qkb.6.1693950167013; Tue, 05 Sep 2023 14:42:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEGbQRi3lj2GCHZMrI9G7N8PmFSehzLMjlA7R138X4fKreXMS+kXyyfOvFWMrTvBreLVK2oMg== X-Received: by 2002:a05:620a:1a26:b0:76c:ed4e:ac10 with SMTP id bk38-20020a05620a1a2600b0076ced4eac10mr16770206qkb.6.1693950166761; Tue, 05 Sep 2023 14:42:46 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id i2-20020a37c202000000b007682af2c8aasm4396938qkm.126.2023.09.05.14.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 14:42:46 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Anish Moorthy , Axel Rasmussen , Alexander Viro , Mike Kravetz , Peter Zijlstra , Andrew Morton , Mike Rapoport , Christian Brauner , peterx@redhat.com, linux-fsdevel@vger.kernel.org, Andrea Arcangeli , Ingo Molnar , James Houghton , Nadav Amit Subject: [PATCH 7/7] selftests/mm: uffd perf test Date: Tue, 5 Sep 2023 17:42:35 -0400 Message-ID: <20230905214235.320571-8-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905214235.320571-1-peterx@redhat.com> References: <20230905214235.320571-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 788974001B X-Stat-Signature: 3zi8rzpzf56e3y3sgzjj4awn6ipfh19y X-Rspam-User: X-HE-Tag: 1693950169-948210 X-HE-Meta: U2FsdGVkX19+a9NzoQsr4jRGwtSwKUC3arBWKX5YeMAYtcgibuft0zwceGtpon6PUXJNT3SnZhH/nGqKXm8zoa2S+VDo4N/13mlskvdCucHKyQZ6jqQbHJXHmurn7IFX2slt0jmub113tY1Dc7y5KQwyi4NTBxM6A2MY52F1dAFrxsHIPeqnAO0HziPty/cYgGVoGEv9hKcDxlm/qgp8hr/bQnIyBo47BQogDf3plRb885BomsyhoocdM1McDLPp9UEl6l2Y5XvAFTDVCM86mX260khY3stPxzlzIiHozdte+L9f+fwyYTEr45N1ia3BftJ0QChzCelP4O2gcNSWAeabVTzwzw6x7i75W+R7Si708jlrkyZC+iVsM0Kee4uzqukAiCEGZF7oPn1HiBBa29dep/mBnJZNNO/Bm1rskV24Pq2jbp5QUEvmu4lZirwh7ebD2gAFl/lieDZL7DGbz1+/fbOsw72LQBVWm6viRC53vES2LOJdFYrPnbKXVtU9T1N0ESVM0pb0OKOK8gme97tZ+ijIFjIN1H6zO5z0t/Eec0ZGwN/nH4/dDtnewsFcnV1ZyaifwHBYJkF2mLDNGdpGRFNae4k6789ONhFTWGbdu7hZl5ry9/x6z3x2cndMxltZVNeESp9R9Zos0ozp/2Qh9L4rM84yDXCNIUWrUZcA4O+7JwF/iJmBpRXe6RcnO/BI5k/q7Wx+NzYh/DpBzdjjoqlToJ1BDR1zH++4kcPCzWGZ6ggZ+yuQvbCMvLRYAuhTnZGTVlVdzAfvi9wrW1ajwCOAGkCaKhaVpD3I4Wf6WwRxYDsPs9lfC3SoS+VwyGTztd7ILNtiO0vzwPpDyO8DkfIRUc2O/YNvrRo4mL7pPx4jK2PONxlznEOMiGXeDFpp305YWDFvsyczRqNWzunTHqIFfQ66bhcWvnqyIeFSjzFkfJu0UNoK0ixDC+FfSFk8/h63qowxYt5wMPU WJaawhTN BB25UMahvn4PHQ5JM7+4u7U2HI2YaZbRLQ4KWSki5pf0hdxq4JjK2kJxfzfKx/uRB1gE13gs62MBG5OMqiLxr5ztg+8oxwBasOMZlGWOuByKp3IzqdZHg+NMiyr1m6RlUPn9Adlm5yod08eFoi4cCqOUsaH3Kj1oDe1O+DO3g7vEryFL5Y8ogIANXDd21Jlfz5vy8JkEjRdVQgthoESgAJ7TNSRV1YFzXMNuoqSqqziV/ESSH6JUFZbQ5cbMhGIzjsF8Da0FB66B6P+dUFtRyBq1/Xx2+gFf32yj/X7KawFZXvnm5dRhhNlPKMbnackocH7indkkpf3QpSwL1Eayobb9MTQcRbHl2hpkwxT8wsLyJL8127XFwIzbXj76djg+bRlH+YLGndaJg6TrDheb/T4sof8MB3Ig2c7bNfWWdvkGeTbNaI16hepEbHGIJewFVuKhrCC4dnkfeyF2VSgxNgZcOwg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a simple perf test for userfaultfd missing mode, on private anon only. It mostly only tests the messaging, so memory type / fault type may not that much yet. Signed-off-by: Peter Xu --- tools/testing/selftests/mm/Makefile | 2 + tools/testing/selftests/mm/uffd-common.c | 18 ++ tools/testing/selftests/mm/uffd-common.h | 1 + tools/testing/selftests/mm/uffd-perf.c | 207 +++++++++++++++++++++++ 4 files changed, 228 insertions(+) create mode 100644 tools/testing/selftests/mm/uffd-perf.c diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile index 6a9fc5693145..acb22517d37e 100644 --- a/tools/testing/selftests/mm/Makefile +++ b/tools/testing/selftests/mm/Makefile @@ -64,6 +64,7 @@ TEST_GEN_FILES += thuge-gen TEST_GEN_FILES += transhuge-stress TEST_GEN_FILES += uffd-stress TEST_GEN_FILES += uffd-unit-tests +TEST_GEN_FILES += uffd-perf TEST_GEN_FILES += split_huge_page_test TEST_GEN_FILES += ksm_tests TEST_GEN_FILES += ksm_functional_tests @@ -120,6 +121,7 @@ $(TEST_GEN_FILES): vm_util.c $(OUTPUT)/uffd-stress: uffd-common.c $(OUTPUT)/uffd-unit-tests: uffd-common.c +$(OUTPUT)/uffd-perf: uffd-common.c ifeq ($(ARCH),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32)) diff --git a/tools/testing/selftests/mm/uffd-common.c b/tools/testing/selftests/mm/uffd-common.c index 851284395b29..afbf2f7add56 100644 --- a/tools/testing/selftests/mm/uffd-common.c +++ b/tools/testing/selftests/mm/uffd-common.c @@ -725,3 +725,21 @@ int uffd_get_features(uint64_t *features) return 0; } + +uint64_t get_usec(void) +{ + uint64_t val = 0; + struct timespec t; + int ret = clock_gettime(CLOCK_MONOTONIC, &t); + + if (ret == -1) { + perror("clock_gettime() failed"); + /* should never happen */ + exit(-1); + } + + val = t.tv_nsec / 1000; /* ns -> us */ + val += t.tv_sec * 1000000; /* s -> us */ + + return val; +} diff --git a/tools/testing/selftests/mm/uffd-common.h b/tools/testing/selftests/mm/uffd-common.h index 9d66ad5c52cb..4273201ae19f 100644 --- a/tools/testing/selftests/mm/uffd-common.h +++ b/tools/testing/selftests/mm/uffd-common.h @@ -123,6 +123,7 @@ int uffd_open_dev(unsigned int flags); int uffd_open_sys(unsigned int flags); int uffd_open(unsigned int flags); int uffd_get_features(uint64_t *features); +uint64_t get_usec(void); #define TEST_ANON 1 #define TEST_HUGETLB 2 diff --git a/tools/testing/selftests/mm/uffd-perf.c b/tools/testing/selftests/mm/uffd-perf.c new file mode 100644 index 000000000000..eda99718311a --- /dev/null +++ b/tools/testing/selftests/mm/uffd-perf.c @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Userfaultfd performance tests. + * + * Copyright (C) 2023 Red Hat, Inc. + */ + +#include "uffd-common.h" + +#ifdef __NR_userfaultfd + +#define DEF_MEM_SIZE_MB (512) +#define MB(x) ((x) * 1024 * 1024) +#define DEF_N_TESTS 5 + +static volatile bool perf_test_started; +static unsigned int n_uffd_threads, n_worker_threads; +static uint64_t nr_pages_per_worker; +static unsigned long n_tests = DEF_N_TESTS; + +static void setup_env(unsigned long mem_size_mb) +{ + /* Test private anon only for now */ + map_shared = false; + uffd_test_ops = &anon_uffd_test_ops; + page_size = psize(); + nr_cpus = n_uffd_threads; + nr_pages = MB(mem_size_mb) / page_size; + nr_pages_per_worker = nr_pages / n_worker_threads; + if (nr_pages_per_worker == 0) + err("each worker should at least own one page"); +} + +void *worker_fn(void *opaque) +{ + unsigned long i = (unsigned long) opaque; + unsigned long page_nr, start_nr, end_nr; + int v = 0; + + start_nr = i * nr_pages_per_worker; + end_nr = (i + 1) * nr_pages_per_worker; + + while (!perf_test_started); + + for (page_nr = start_nr; page_nr < end_nr; page_nr++) + v += *(volatile int *)(area_dst + page_nr * page_size); + + return NULL; +} + +static uint64_t run_perf(uint64_t mem_size_mb, bool poll) +{ + pthread_t worker_threads[n_worker_threads]; + pthread_t uffd_threads[n_uffd_threads]; + const char *errmsg = NULL; + struct uffd_args *args; + uint64_t start, end; + int i, ret; + + if (uffd_test_ctx_init(0, &errmsg)) + err("%s", errmsg); + + /* + * By default, uffd is opened with NONBLOCK mode; use block mode + * when test read() + */ + if (!poll) { + int flags = fcntl(uffd, F_GETFL); + + if (flags < 0) + err("fcntl(F_GETFL) failed"); + + if (flags & O_NONBLOCK) + flags &= ~O_NONBLOCK; + + if (fcntl(uffd, F_SETFL, flags)) + err("fcntl(F_SETFL) failed"); + } + + ret = uffd_register(uffd, area_dst, MB(mem_size_mb), + true, false, false); + if (ret) + err("uffd_register() failed"); + + args = calloc(nr_cpus, sizeof(struct uffd_args)); + if (!args) + err("calloc()"); + + for (i = 0; i < n_uffd_threads; i++) { + args[i].cpu = i; + uffd_fault_thread_create(&uffd_threads[i], NULL, + &args[i], poll); + } + + for (i = 0; i < n_worker_threads; i++) { + if (pthread_create(&worker_threads[i], NULL, + worker_fn, (void *)(uintptr_t)i)) + err("create uffd threads"); + } + + start = get_usec(); + perf_test_started = true; + for (i = 0; i < n_worker_threads; i++) + pthread_join(worker_threads[i], NULL); + end = get_usec(); + + for (i = 0; i < n_uffd_threads; i++) { + struct uffd_args *p = &args[i]; + + uffd_fault_thread_join(uffd_threads[i], i, poll); + + assert(p->wp_faults == 0 && p->minor_faults == 0); + } + + free(args); + + ret = uffd_unregister(uffd, area_dst, MB(mem_size_mb)); + if (ret) + err("uffd_unregister() failed"); + + return end - start; +} + +static void usage(const char *prog) +{ + printf("usage: %s \n", prog); + puts(""); + printf(" -m: size of memory to test (in MB, default: %u)\n", + DEF_MEM_SIZE_MB); + puts(" -p: use poll() (the default)"); + puts(" -r: use read()"); + printf(" -t: test rounds (default: %u)\n", DEF_N_TESTS); + puts(" -u: number of uffd threads (default: n_cpus)"); + puts(" -w: number of worker threads (default: n_cpus)"); + puts(""); + exit(KSFT_FAIL); +} + +int main(int argc, char *argv[]) +{ + unsigned long mem_size_mb = DEF_MEM_SIZE_MB; + uint64_t result, sum = 0; + bool use_poll = true; + int opt, count; + + n_uffd_threads = n_worker_threads = sysconf(_SC_NPROCESSORS_ONLN); + + while ((opt = getopt(argc, argv, "hm:prt:u:w:")) != -1) { + switch (opt) { + case 'm': + mem_size_mb = strtoul(optarg, NULL, 10); + break; + case 'p': + use_poll = true; + break; + case 'r': + use_poll = false; + break; + case 't': + n_tests = strtoul(optarg, NULL, 10); + break; + case 'u': + n_uffd_threads = strtoul(optarg, NULL, 10); + break; + case 'w': + n_worker_threads = strtoul(optarg, NULL, 10); + break; + case 'h': + default: + /* Unknown */ + usage(argv[0]); + break; + } + } + + setup_env(mem_size_mb); + + printf("Message mode: \t\t%s\n", use_poll ? "poll" : "read"); + printf("Mem size: \t\t%lu (MB)\n", mem_size_mb); + printf("Uffd threads: \t\t%u\n", n_uffd_threads); + printf("Worker threads: \t%u\n", n_worker_threads); + printf("Test rounds: \t\t%lu\n", n_tests); + printf("Time used (us): \t"); + + for (count = 0; count < n_tests; count++) { + result = run_perf(mem_size_mb, use_poll); + sum += result; + printf("%" PRIu64 ", ", result); + fflush(stdout); + } + printf("\b\b \n"); + printf("Average (us): \t\t%"PRIu64"\n", sum / n_tests); + + return KSFT_PASS; +} + +#else /* __NR_userfaultfd */ + +#warning "missing __NR_userfaultfd definition" + +int main(void) +{ + printf("Skipping %s (missing __NR_userfaultfd)\n", __file__); + return KSFT_SKIP; +} + +#endif /* __NR_userfaultfd */