From patchwork Fri Feb 9 04:06:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13550860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C34FC48297 for ; Fri, 9 Feb 2024 04:07:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B13EB8D0010; Thu, 8 Feb 2024 23:07:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AC3838D0001; Thu, 8 Feb 2024 23:07:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93CED8D0010; Thu, 8 Feb 2024 23:07:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7FDA18D0001 for ; Thu, 8 Feb 2024 23:07:36 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 66945A0DAB for ; Fri, 9 Feb 2024 04:07:36 +0000 (UTC) X-FDA: 81770931312.21.F04CE3F Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf08.hostedemail.com (Postfix) with ESMTP id 7A889160002 for ; Fri, 9 Feb 2024 04:07:34 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Ko2v8iBr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707451654; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ax1i6Xx3KClBMGyRsPQIF9NlJODAQM+auENr+G97oYA=; b=yUVdBfuVPyalWL3vRflBMG1Jg5cseEckxzHESmUWcds9W8kU1++0mlgQigGJn+eJKPAzk/ Q2m67BZS2j01DRoRKDu7zoBlECoUu8KY1DB7T5HbIZCdzmcYaLyLLeMbajDH0/2fnMZCuY HhXccTTDvRoGKVzJuiCFYfPVajorvTo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Ko2v8iBr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707451654; a=rsa-sha256; cv=none; b=kNLpRlvYtNdzK43UoqWGauP60Ws/olHUIp5GWhibfjrHcUdqZEDBZwedTzfB2FtwBm7CzE yM8/jJA+88Z/BxBNHUn4UCctvvKHyCJlhFGSGa/FpS0SCPPsRJSDXt23y/RCdldmTYYnap X5kPK4hwzHam6z1GDR1ZjkYl6kuEDp4= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1d74045c463so4471685ad.3 for ; Thu, 08 Feb 2024 20:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707451653; x=1708056453; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ax1i6Xx3KClBMGyRsPQIF9NlJODAQM+auENr+G97oYA=; b=Ko2v8iBrEVfv+eXSI2BRv2liTQhh60VvJJznoDUcqZUQxrnT1HLWivE3zRa02HNRqB OVRZXk1hvyvIw5QdIu8caJe5lqX3yUwBdK7BKnQ5PxLnMWdQafo0VM/NY+vcCWNjV4WM ntqp8EHxAnupTK12wnsSgkFa3nW461xjLCzFk6nKxpdPZiT4zG0nt5FBmAGftYs2yTqg nKb+ktWtRIzOF2xiiqWV7ATj+PaleqQ93F05i/LUXGPjugCgCdtRsonxT2KXRMUEzfn3 BBkmf5zpjdrq0VvkdFtQKw8kuREZQZrrJG1OLbrrtr3cmuVNioBDHr6yitRw4kHAztUJ Jj6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707451653; x=1708056453; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ax1i6Xx3KClBMGyRsPQIF9NlJODAQM+auENr+G97oYA=; b=w3NIuC5P7924B6i1zoxDaXnUK1I52wasFU8+yK2Zf8kGfEymvzjrZjXWZ18UPOT9L0 YL5uza13oFwtut3BBu4b+6Q73YhyYm5e0kAcz7TB4kICF0ZMdVYV9z+dixCqV4FodbBd X6CcAkdFRl9yG4rrfel6xo8TIYDdJoxUuiiRZvJmr8BS0vzyD7gKoSPCEGT7OgdOsClO uZUXIqdNJLQCGP49GEHgIQpAggwCl88vO1HadAcBSUMGmy7E1WTQxEqLk6u0sJRiFhMS 3X4A8WVzrM3yFQ46Q1uaINi2I37lwSZO+JuAJRox4RrRVN03ix6JyIdMfUha5Dv0VcNR rk3w== X-Forwarded-Encrypted: i=1; AJvYcCUCuvCnxzwiKDDjCvhn9Arm+4B+KbUgYcTCLdVkU2Z8hnMhOjKCrhYE7tR9j0M8z/uAzDyrX6SbSd3hqaSCMzEgJbI= X-Gm-Message-State: AOJu0YwfzcJ/n2O34oBUGjP6Yt0/YYOktuBZHj4hAr+ClQmhi8FuIO8I Rtg5MC4kbsvyqVHj09R2NVUu3OZ6Tdg99INv/XLVN7JkRI2oQZUFg/3gYWAp X-Google-Smtp-Source: AGHT+IEGpYswlstUcGLeCwP8YO/u0UAf0+3x8WGh941h8RI1JhGiemicOLthwo5sUBqPjsP/3SF5Yw== X-Received: by 2002:a17:902:ec86:b0:1d8:b798:dfe3 with SMTP id x6-20020a170902ec8600b001d8b798dfe3mr539249plg.43.1707451653406; Thu, 08 Feb 2024 20:07:33 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCVs1Kr+GkxG5gGSe9IAqpYlusa71Kdv6s5vkhdFG/fLIWePlY2qWKxLMvnRUrNz4H/DAJGLF26mTDtKKjD8WWDBIctDQZzbsST9F4Ln+bPNO0wK+o8/lgyIJDSDX9j0UVSTG00H0rEVqkxYJfTYelhcv9T3cgo6Q3OyKabdF+pCiDx0TAFrFONzZioUBqDHvXhS2oIgVjsslE6CRoZx9oQ36cxG82cPnMKVzWA3lxyVdGIsESj7DqQMgkA+fPZOW4nam9SwLlwJAN1KWtzKdp0P8+g79UkteFT1wAcUGkp8s/fBeG9eZbxbKgPEzBihaEPIHFM4epKG+jmwpn4Na/ARvgwu54fm4mUdB5o9tfeL1v7IVfQwXQ== Received: from macbook-pro-49.dhcp.thefacebook.com ([2620:10d:c090:400::4:a894]) by smtp.gmail.com with ESMTPSA id q13-20020a170902a3cd00b001d963d963aasm561477plb.308.2024.02.08.20.07.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 08 Feb 2024 20:07:33 -0800 (PST) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, memxor@gmail.com, eddyz87@gmail.com, tj@kernel.org, brho@google.com, hannes@cmpxchg.org, lstoakes@gmail.com, akpm@linux-foundation.org, urezki@gmail.com, hch@infradead.org, linux-mm@kvack.org, kernel-team@fb.com Subject: [PATCH v2 bpf-next 19/20] selftests/bpf: Add bpf_arena_htab test. Date: Thu, 8 Feb 2024 20:06:07 -0800 Message-Id: <20240209040608.98927-20-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20240209040608.98927-1-alexei.starovoitov@gmail.com> References: <20240209040608.98927-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 7A889160002 X-Stat-Signature: qtzdp584bc19z3iypyb9iwdcofe693k9 X-Rspam-User: X-HE-Tag: 1707451654-620383 X-HE-Meta: U2FsdGVkX18q5/4IBMlX8zA0vgjrn/dlwgU9jo4gcYjoM2fAFvXL0Ag/T9p3UqatMv3KtjNkqCWfjZYvP/3+RFIKYObUkdO2r/QlslAMUPJ7YjoIKZzl2m0P7WrPaKin8/S3AORoQLJdPBHqISBrlWLmrIUyl9g0UE0/+EYct59dU1Koiu9dCSH5BlOXORWY7S0aeEDRv8MGru8md6YC7Oh+ei+TNzfcOisalpcBWZRwZu4nVlWRo7PUJWpxC/5qXpRTdU12KeoubacAK1csSgKSA09VsiUFB3ctcIsn4UHiwTda2PA5vEjxO1y/fGgVlDt2Ij+YWEAE6LldK2Wz9qGsCaTcwCrPzYfa1m088f00hHZnSfgBytuSaD+ziwGJO8AFJkUbEGY3Zha0Hcun5e7rnXZhBKcU5ToTOBPx70D71RRIdyiX5DwEM+TeEL/k3rvuReQx2YI4pRHaCd7YsWHbEfl2cf2BCsTKuFu4D0cfc8VabhUbVe608NSzTZcEYCSrEbfpJf/e0FKigT49lC+G+wX6FmdK7cNdgkLbF1JrdHIbDnFIrNSF78sCPfhOC8TwK6/mefiSXXzdSYKG6BtuB9yzYjFwgxOiqG35EG4uL6SLUM3GlP3wxVS9Ie6Qz7DEncR4A8fsOhit8eE5toW+WSCY7bF6MDv+SR/CeT4IXU/0qntAwrPGbrJNA1cO1a7EDjK7Ko3Z+38k8IiRP1Q+sTU3dcPFsedN167Lvbxqo253gqQCRVYFUJelGbbisj8TGBCCbGVmdfzmxmuI66EF5aENhrouBWVnv2s2jKsj+nSN9KPVCh6Q7GVhVxrYrx+4qwudg5TxwXhfHpYcTDm26K0EaJ6mqGqOabyIcsY7VurKNI1TMqVDEOIIf84yc4lZld2d57CbxL/O9NnLVCcTNNTCU8frMcBQdpzfrQEJcFDnGjFTP1htxvdXtD+09tgz88IBhgnWyiAMuUM S3CHs/R2 WcV81O5ZzAy31ddrSvYGoAMGd9/oJ9/GZnyOpvnG0RosNGEQ3etXPI8+0oG19dObTx99e8CyttDrv0nYYohDL6RbPHc+41z4UIwc8/Dg8uugDMAIqWUU4oUEDMdfu2vTLReCPokKN4f6vKw2blyR4qklaBeasxsSCkvy6Yp7idjA8HgyCPctcFFkCuuefd2zQyeXirHI3daCa69zSQrB1DXI3T4IlS4OG2EzyO0V5/cSr9CULBm1wt6ZjEcbFT3KvA8OCdducIRIUZHMZTNg3HhE+umW8EFIOj3eZCEKQQZK6mI8ntGpe+FQJwt3jJULiMUomLWdmxFZKjF+NAJ0VIM75hSS6M2fgXTo2UwXrycXDapoPYsvi8Fp9lqysu6pcuSJhnceJzH2vokhTMlqAUiEl9BcZ4X7Oo96yrwRhwuwMhw1na3CbTWmaKnctLG/L13DuoF6mch7wVEzP7lwlGuAl0siUofeQjC2DM6afqRdhJ1Fj98st/vwF/xdREqkupQ8wGrNyD4s8DVqJ6Hn0PbPXJENjFhnVj6zFpPrD+yC3Z5uBjpJRS56uxYaZ7Sce4f5u0qEEx/6pknRV9MHWYK9Y6YC5wOnBkD37 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov bpf_arena_htab.h - hash table implemented as bpf program Signed-off-by: Alexei Starovoitov --- tools/testing/selftests/bpf/bpf_arena_htab.h | 100 ++++++++++++++++++ .../selftests/bpf/prog_tests/arena_htab.c | 88 +++++++++++++++ .../testing/selftests/bpf/progs/arena_htab.c | 46 ++++++++ .../selftests/bpf/progs/arena_htab_asm.c | 5 + 4 files changed, 239 insertions(+) create mode 100644 tools/testing/selftests/bpf/bpf_arena_htab.h create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_htab.c create mode 100644 tools/testing/selftests/bpf/progs/arena_htab.c create mode 100644 tools/testing/selftests/bpf/progs/arena_htab_asm.c diff --git a/tools/testing/selftests/bpf/bpf_arena_htab.h b/tools/testing/selftests/bpf/bpf_arena_htab.h new file mode 100644 index 000000000000..acc01a876668 --- /dev/null +++ b/tools/testing/selftests/bpf/bpf_arena_htab.h @@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#pragma once +#include +#include "bpf_arena_alloc.h" +#include "bpf_arena_list.h" + +struct htab_bucket { + struct arena_list_head head; +}; +typedef struct htab_bucket __arena htab_bucket_t; + +struct htab { + htab_bucket_t *buckets; + int n_buckets; +}; +typedef struct htab __arena htab_t; + +static inline htab_bucket_t *__select_bucket(htab_t *htab, __u32 hash) +{ + htab_bucket_t *b = htab->buckets; + + cast_kern(b); + return &b[hash & (htab->n_buckets - 1)]; +} + +static inline arena_list_head_t *select_bucket(htab_t *htab, __u32 hash) +{ + return &__select_bucket(htab, hash)->head; +} + +struct hashtab_elem { + int hash; + int key; + int value; + struct arena_list_node hash_node; +}; +typedef struct hashtab_elem __arena hashtab_elem_t; + +static hashtab_elem_t *lookup_elem_raw(arena_list_head_t *head, __u32 hash, int key) +{ + hashtab_elem_t *l; + + list_for_each_entry(l, head, hash_node) + if (l->hash == hash && l->key == key) + return l; + + return NULL; +} + +static int htab_hash(int key) +{ + return key; +} + +__weak int htab_lookup_elem(htab_t *htab __arg_arena, int key) +{ + hashtab_elem_t *l_old; + arena_list_head_t *head; + + cast_kern(htab); + head = select_bucket(htab, key); + l_old = lookup_elem_raw(head, htab_hash(key), key); + if (l_old) + return l_old->value; + return 0; +} + +__weak int htab_update_elem(htab_t *htab __arg_arena, int key, int value) +{ + hashtab_elem_t *l_new = NULL, *l_old; + arena_list_head_t *head; + + cast_kern(htab); + head = select_bucket(htab, key); + l_old = lookup_elem_raw(head, htab_hash(key), key); + + l_new = bpf_alloc(sizeof(*l_new)); + if (!l_new) + return -ENOMEM; + l_new->key = key; + l_new->hash = htab_hash(key); + l_new->value = value; + + list_add_head(&l_new->hash_node, head); + if (l_old) { + list_del(&l_old->hash_node); + bpf_free(l_old); + } + return 0; +} + +void htab_init(htab_t *htab) +{ + void __arena *buckets = bpf_arena_alloc_pages(&arena, NULL, 2, NUMA_NO_NODE, 0); + + cast_user(buckets); + htab->buckets = buckets; + htab->n_buckets = 2 * PAGE_SIZE / sizeof(struct htab_bucket); +} diff --git a/tools/testing/selftests/bpf/prog_tests/arena_htab.c b/tools/testing/selftests/bpf/prog_tests/arena_htab.c new file mode 100644 index 000000000000..0766702de846 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/arena_htab.c @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include + +#include "arena_htab_asm.skel.h" +#include "arena_htab.skel.h" + +#define PAGE_SIZE 4096 + +#include "bpf_arena_htab.h" + +static void test_arena_htab_common(struct htab *htab) +{ + int i; + + printf("htab %p buckets %p n_buckets %d\n", htab, htab->buckets, htab->n_buckets); + ASSERT_OK_PTR(htab->buckets, "htab->buckets shouldn't be NULL"); + for (i = 0; htab->buckets && i < 16; i += 4) { + /* + * Walk htab buckets and link lists since all pointers are correct, + * though they were written by bpf program. + */ + int val = htab_lookup_elem(htab, i); + + ASSERT_EQ(i, val, "key == value"); + } +} + +static void test_arena_htab_llvm(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + struct arena_htab *skel; + struct htab *htab; + size_t arena_sz; + void *area; + int ret; + + skel = arena_htab__open_and_load(); + if (!ASSERT_OK_PTR(skel, "arena_htab__open_and_load")) + return; + + area = bpf_map__initial_value(skel->maps.arena, &arena_sz); + /* fault-in a page with pgoff == 0 as sanity check */ + *(volatile int *)area = 0x55aa; + + /* bpf prog will allocate more pages */ + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.arena_htab_llvm), &opts); + ASSERT_OK(ret, "ret"); + ASSERT_OK(opts.retval, "retval"); + if (skel->bss->skip) { + printf("%s:SKIP:compiler doesn't support arena_cast\n", __func__); + test__skip(); + goto out; + } + htab = skel->bss->htab_for_user; + test_arena_htab_common(htab); +out: + arena_htab__destroy(skel); +} + +static void test_arena_htab_asm(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + struct arena_htab_asm *skel; + struct htab *htab; + int ret; + + skel = arena_htab_asm__open_and_load(); + if (!ASSERT_OK_PTR(skel, "arena_htab_asm__open_and_load")) + return; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.arena_htab_asm), &opts); + ASSERT_OK(ret, "ret"); + ASSERT_OK(opts.retval, "retval"); + htab = skel->bss->htab_for_user; + test_arena_htab_common(htab); + arena_htab_asm__destroy(skel); +} + +void test_arena_htab(void) +{ + if (test__start_subtest("arena_htab_llvm")) + test_arena_htab_llvm(); + if (test__start_subtest("arena_htab_asm")) + test_arena_htab_asm(); +} diff --git a/tools/testing/selftests/bpf/progs/arena_htab.c b/tools/testing/selftests/bpf/progs/arena_htab.c new file mode 100644 index 000000000000..441fc502312f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/arena_htab.c @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include "bpf_experimental.h" + +struct { + __uint(type, BPF_MAP_TYPE_ARENA); + __uint(map_flags, BPF_F_MMAPABLE); + __uint(max_entries, 100); /* number of pages */ +} arena SEC(".maps"); + +#include "bpf_arena_htab.h" + +void __arena *htab_for_user; +bool skip = false; + +SEC("syscall") +int arena_htab_llvm(void *ctx) +{ +#if defined(__BPF_FEATURE_ARENA_CAST) || defined(BPF_ARENA_FORCE_ASM) + struct htab __arena *htab; + __u64 i; + + htab = bpf_alloc(sizeof(*htab)); + cast_kern(htab); + htab_init(htab); + + /* first run. No old elems in the table */ + bpf_for(i, 0, 1000) + htab_update_elem(htab, i, i); + + /* should replace all elems with new ones */ + bpf_for(i, 0, 1000) + htab_update_elem(htab, i, i); + cast_user(htab); + htab_for_user = htab; +#else + skip = true; +#endif + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/arena_htab_asm.c b/tools/testing/selftests/bpf/progs/arena_htab_asm.c new file mode 100644 index 000000000000..6cd70ea12f0d --- /dev/null +++ b/tools/testing/selftests/bpf/progs/arena_htab_asm.c @@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_ARENA_FORCE_ASM +#define arena_htab_llvm arena_htab_asm +#include "arena_htab.c"