From patchwork Thu Feb 25 07:29:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nadav Amit X-Patchwork-Id: 12103567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1820C433DB for ; Thu, 25 Feb 2021 07:34:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 39E9164E90 for ; Thu, 25 Feb 2021 07:34:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39E9164E90 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 291A08D0006; Thu, 25 Feb 2021 02:33:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F04F8D0005; Thu, 25 Feb 2021 02:33:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01C998D0006; Thu, 25 Feb 2021 02:33:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0076.hostedemail.com [216.40.44.76]) by kanga.kvack.org (Postfix) with ESMTP id E1B618D0005 for ; Thu, 25 Feb 2021 02:33:58 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A921F180477A2 for ; Thu, 25 Feb 2021 07:33:58 +0000 (UTC) X-FDA: 77855976156.15.FA64ACE Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf15.hostedemail.com (Postfix) with ESMTP id 20A57A0009D5 for ; Thu, 25 Feb 2021 07:33:56 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id q204so1992169pfq.10 for ; Wed, 24 Feb 2021 23:33:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ew5ULXh2s3debDfJGELT9nH6Qr39U3zvkA1enkJWaeA=; b=onvirPQoLiEDzNesEG2CCzrN3ULqiE3DB2mLthAl4jMogSYLK3XkbcJG1p5ojLSl63 KeMDMnPQNKt/fYg4GTGw3oc8DJgJJ40yDzAikB0a8WNjRgJz3nMfjUj4vdCqUG6gvPVA iGoLvshjoZUe4ePEJC49XUGLUapSIDowlKjRDSwWViktrLxLORKgkoO76zR99isB+5bT NAzCZb/jrKW6A7Yf8Psm8jHNxFNU1rkdtGMVb16r20KydB4kILjusiTWNcY2SrZb85Jm a/rSBPNHHWK4j7eWAB5zMgmNGTcyUgTXOwlfKpWJikWQVlb060EnO43EQjNBlVhaghg/ zxDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ew5ULXh2s3debDfJGELT9nH6Qr39U3zvkA1enkJWaeA=; b=mY0ObtDJLtKthpeAFDz80BBea3DSS8+HkCURQlo7CijlXyb7WvbOJBCWPIwkm+SOOA C+pQsbDX3jLcxo0q1Pda8jSGcrTKa14cBzNISnLkTPM+YU3WIAW3Fugf5JaKz83ytms5 qS+GJDYxKQtt6KJfFAy9h/qxWJ3CqJK7jRcoxcjRjF7R1Gg6ZFcSzgMQYGN1jUgmxZ2n CBH2F49o/gyKk4vlVz4mSNgnqITn2LFbS0WA8LJ1kCIb7TNK1wZbZiCgAK/yt+lebVY+ N5E+XKoT9HJ+MjtDq6SZG/EgYaph66lk6oaTmgB5qv0TCbS8S6Q0PMdJZ6ofH0FfOD/a WyKA== X-Gm-Message-State: AOAM531LyegxCItI2MQ0hVugyrJObGCwrxZQQEvjtRjS4/0TX4E0GXHV yW7spP+R0o6XFTJEXPZEhj6DWnoJqNtcLA== X-Google-Smtp-Source: ABdhPJyP/D1mqxF7RjwvOWEtaASpae014BCao/9yEM/ejxJFLMHEY4DszZ17KieyhQx2UezDqsNJrg== X-Received: by 2002:a63:546:: with SMTP id 67mr1803027pgf.173.1614238436911; Wed, 24 Feb 2021 23:33:56 -0800 (PST) Received: from sc2-haas01-esx0118.eng.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id w3sm4917561pjt.24.2021.02.24.23.33.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Feb 2021 23:33:56 -0800 (PST) From: Nadav Amit X-Google-Original-From: Nadav Amit To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Hugh Dickins , Andy Lutomirski , Thomas Gleixner , Peter Zijlstra , Ingo Molnar , Borislav Petkov , Nadav Amit , Sean Christopherson , Andrew Morton , x86@kernel.org Subject: [PATCH 6/6] testing/selftest: test vDSO prefetch_page() Date: Wed, 24 Feb 2021 23:29:10 -0800 Message-Id: <20210225072910.2811795-7-namit@vmware.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210225072910.2811795-1-namit@vmware.com> References: <20210225072910.2811795-1-namit@vmware.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 20A57A0009D5 X-Stat-Signature: zmgmipokk8on13nah3b17tenawr4mjhz Received-SPF: none (<>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from="<>"; helo=mail-pf1-f171.google.com; client-ip=209.85.210.171 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614238436-36764 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nadav Amit Test prefetch_page() in cases of invalid pointer, file-mmap and anonymous memory. Partial checks are also done with mincore syscall to ensure the output of prefetch_page() is consistent with mincore (taking into account the different semantics of the two). The tests are not fool-proof as they rely on the behavior of the page-cache and page reclamation mechanism to get a major page-fault. They should be robust in the sense of test being skipped if it failed. There is a question though on how to know how much memory to access in the test of anonymous memory to force the eviction of a page and trigger a refault. Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Sean Christopherson Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Andrew Morton Cc: x86@kernel.org Signed-off-by: Nadav Amit --- tools/testing/selftests/vDSO/Makefile | 2 + .../selftests/vDSO/vdso_test_prefetch_page.c | 265 ++++++++++++++++++ 2 files changed, 267 insertions(+) create mode 100644 tools/testing/selftests/vDSO/vdso_test_prefetch_page.c diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile index d53a4d8008f9..dcd1ede8c0f7 100644 --- a/tools/testing/selftests/vDSO/Makefile +++ b/tools/testing/selftests/vDSO/Makefile @@ -11,6 +11,7 @@ ifeq ($(ARCH),$(filter $(ARCH),x86 x86_64)) TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86 endif TEST_GEN_PROGS += $(OUTPUT)/vdso_test_correctness +TEST_GEN_PROGS += $(OUTPUT)/vdso_test_prefetch_page CFLAGS := -std=gnu99 CFLAGS_vdso_standalone_test_x86 := -nostdlib -fno-asynchronous-unwind-tables -fno-stack-protector @@ -33,3 +34,4 @@ $(OUTPUT)/vdso_test_correctness: vdso_test_correctness.c vdso_test_correctness.c \ -o $@ \ $(LDFLAGS_vdso_test_correctness) +$(OUTPUT)/vdso_test_prefetch_page: vdso_test_prefetch_page.c parse_vdso.c diff --git a/tools/testing/selftests/vDSO/vdso_test_prefetch_page.c b/tools/testing/selftests/vDSO/vdso_test_prefetch_page.c new file mode 100644 index 000000000000..35928c3f36ca --- /dev/null +++ b/tools/testing/selftests/vDSO/vdso_test_prefetch_page.c @@ -0,0 +1,265 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vdso_test_prefetch_page.c: Test vDSO's prefetch_page()) + */ + +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../kselftest.h" +#include "parse_vdso.h" + +const char *version = "LINUX_2.6"; +const char *name = "__vdso_prefetch_page"; + +struct getcpu_cache; +typedef long (*prefetch_page_t)(const void *p); + +#define MEM_SIZE_K (9500000ull) +#define PAGE_SIZE (4096ull) + +#define SKIP_MINCORE_BEFORE (1 << 0) +#define SKIP_MINCORE_AFTER (1 << 1) + +static prefetch_page_t prefetch_page; + +static const void *ptr_align(const void *p) +{ + return (const void *)((unsigned long)p & ~(PAGE_SIZE - 1)); +} + + +static int __test_prefetch(const void *p, bool expected_no_io, + const char *test_name, unsigned int skip_mincore) +{ + bool no_io; + char vec; + long r; + uint64_t start; + + p = ptr_align(p); + + /* + * First, run a sanity check to use mincore() to see if the page is in + * memory when we expect it not to be. We can only trust mincore to + * tell us when a page is already in memory when it should not be. + */ + if (!(skip_mincore & SKIP_MINCORE_BEFORE)) { + if (mincore((void *)p, PAGE_SIZE, &vec)) { + printf("[SKIP]\t%s: mincore failed: %s\n", test_name, + strerror(errno)); + return 0; + } + + no_io = vec & 1; + if (!skip_mincore && no_io && !expected_no_io) { + printf("[SKIP]\t%s: unexpected page state: %s\n", + test_name, + no_io ? "in memory" : "not in memory"); + return 0; + } + } + + /* + * Check we got the expected result from prefetch page. + */ + r = prefetch_page(p); + + no_io = r == 0; + if (no_io != expected_no_io) { + printf("[FAIL]\t%s: prefetch_page() returned %ld\n", + test_name, r); + return KSFT_FAIL; + } + + if (skip_mincore & SKIP_MINCORE_AFTER) + return 0; + + /* + * Check again using mincore that the page state is as expected. + * A bit racy. Skip the test if mincore fails. + */ + if (mincore((void *)p, PAGE_SIZE, &vec)) { + printf("[SKIP]\t%s: mincore failed: %s\n", test_name, + strerror(errno)); + return 0; + } + + no_io = vec & 1; + if (0 && no_io != expected_no_io) { + printf("[FAIL]\t%s: mincore reported page is %s\n", + test_name, no_io ? "in memory" : "not in memory"); + return KSFT_FAIL; + + } + return 0; +} + +#define test_prefetch(p, expected_no_io, test_name, skip_mincore) \ + do { \ + long _r = __test_prefetch(p, expected_no_io, \ + test_name, skip_mincore); \ + \ + if (_r) \ + return _r; \ + } while (0) + +static void wait_for_io_completion(const void *p) +{ + char vec; + int i; + + /* Wait to allow the I/O to complete */ + p = ptr_align(p); + + vec = 0; + + /* Wait for 5 seconds and keep probing the page to get it */ + for (i = 0; i < 5000; i++) { + if (mincore((void *)p, PAGE_SIZE, &vec) == 0 && (vec & 1)) + break; + prefetch_page(p); + usleep(1000); + } +} + +int main(int argc, char **argv) +{ + unsigned long sysinfo_ehdr; + long ret, i, test_ret = 0; + int fd, drop_fd; + char *p, vec; + + printf("[RUN]\tTesting vdso_prefetch_page\n"); + + sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR); + if (!sysinfo_ehdr) { + printf("[SKIP]\tAT_SYSINFO_EHDR is not present!\n"); + return KSFT_SKIP; + } + + vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR)); + + prefetch_page = (prefetch_page_t)vdso_sym(version, name); + if (!prefetch_page) { + printf("[SKIP]\tCould not find %s in vdso\n", name); + return KSFT_SKIP; + } + + test_prefetch(NULL, false, "NULL access", + SKIP_MINCORE_BEFORE|SKIP_MINCORE_AFTER); + + test_prefetch(name, true, "present", 0); + + p = mmap(0, PAGE_SIZE, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0); + if (p == MAP_FAILED) { + perror("mmap anon"); + return KSFT_FAIL; + } + + /* + * Mincore would not tell us that no I/O is needed to retrieve the page, + * so tell test_prefetch() to skip it. + */ + test_prefetch(p, true, "anon prefetch", SKIP_MINCORE_BEFORE); + + /* Drop the caches before testing file mmap */ + drop_fd = open("/proc/sys/vm/drop_caches", O_WRONLY); + if (drop_fd < 0) { + perror("open /proc/sys/vm/drop_caches"); + return KSFT_FAIL; + } + + sync(); + ret = write(drop_fd, "3", 1); + if (ret != 1) { + perror("write to /proc/sys/vm/drop_caches"); + return KSFT_FAIL; + } + + /* close, which would also flush */ + ret = close(drop_fd); + if (ret) { + perror("close /proc/sys/vm/drop_caches"); + return KSFT_FAIL; + } + + /* Using /etc/passwd as a file that should alway exist */ + fd = open("/etc/hosts", O_RDONLY); + if (fd < 0) { + perror("open /etc/passwd"); + return KSFT_FAIL; + } + + p = mmap(0, PAGE_SIZE, PROT_READ, MAP_SHARED, fd, 0); + if (p == MAP_FAILED) { + perror("mmap file"); + return KSFT_FAIL; + } + + test_prefetch(p, false, "Minor-fault (io) file prefetch", 0); + + wait_for_io_completion(p); + + test_prefetch(p, true, "Minor-fault (cached) file prefetch", 0); + + munmap(p, PAGE_SIZE); + + /* + * Try to lock all to avoid unrelated page-faults before we create + * memory pressure to prevent unrelated page-faults. + */ + mlockall(MCL_CURRENT); + + p = mmap(0, 1024 * MEM_SIZE_K, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0); + if (p == MAP_FAILED) { + perror("mmap file"); + return KSFT_FAIL; + } + + /* + * Write random value to avoid try to prevent KSM from deduplicating + * this page. + */ + *(volatile unsigned long *)p = 0x43454659; + ret = madvise(p, PAGE_SIZE, MADV_PAGEOUT); + if (ret != 0) { + perror("madvise(MADV_PAGEOUT)"); + return KSFT_FAIL; + } + + /* Wait to allow the page-out to complete */ + usleep(2000000); + + /* Cause some memory pressure */ + for (i = PAGE_SIZE; i < MEM_SIZE_K * 1024; i += PAGE_SIZE) + *(volatile unsigned long *)((unsigned long)p + i) = i + 1; + + /* Check if we managed to evict the page */ + ret = mincore(p, PAGE_SIZE, &vec); + if (ret != 0) { + perror("mincore"); + return KSFT_FAIL; + } + + test_prefetch(p, false, "Minor-fault (io) anon prefetch", 0); + wait_for_io_completion(p); + + test_prefetch(p, true, "Minor-fault (cached) anon prefetch", false); + + printf("[PASS]\tvdso_prefetch_page\n"); + return 0; +}