From patchwork Mon Dec 14 20:11:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 11972945 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE1B8C2BBCF for ; Mon, 14 Dec 2020 20:16:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A33DC207A2 for ; Mon, 14 Dec 2020 20:16:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387805AbgLNUQP (ORCPT ); Mon, 14 Dec 2020 15:16:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2503051AbgLNUML (ORCPT ); Mon, 14 Dec 2020 15:12:11 -0500 Received: from mail-ot1-x344.google.com (mail-ot1-x344.google.com [IPv6:2607:f8b0:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45872C061793; Mon, 14 Dec 2020 12:11:31 -0800 (PST) Received: by mail-ot1-x344.google.com with SMTP id f16so17011024otl.11; Mon, 14 Dec 2020 12:11:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yygU5NAXWVy/uMevreNO6dRLr+BDbLez0blcakg1cFs=; b=o3HtlNau90LyxaDbf4q3WrIQ5NJOxBKtTWVEsFyG6i7g5jg62EXWKNPnVURdf0vWkR A8gf5eJQucjyyXCX7+1+pOAAZPjKDHiD8nq81CnC+8VI/XJb9/ehW2HltU6EoAgYDMS8 hcIx8q/2jHzdWIoQVUTHCL0onQklHltK3nXNVj+DeCNtL5gIjEum3BPlzfHLshsdV8Dt Cjm9PUkBIKvcp0jMGDoAUBsLnezElVRZPWEg/r349wgScm+HV5Sf8sNDWHpmSga+pKdV csolezKEoTihqPZ2l/wfeC+q+mCAPRZk9zyHKONHw2T4u1uwv6SFuHjQ52Z8R0qvubYH p81w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yygU5NAXWVy/uMevreNO6dRLr+BDbLez0blcakg1cFs=; b=UNxiQ/HLoI+yajGWzvChW6BjnC7jTEXqRSeodw3qMm/BosKYweR31bnV66NPMMJOBV t9PZnhNAX7mLEDQLR9D6PwG5Am91JYp0h+MDP0luJiPv9vFbFS6ZSen2+YY0IVOIIVAz Q0rCwvVaAAA796T+NMqhkeAvRi+zaRXhtHShbR+VFYxvQdkH7O5gT3M2/u9kD2rnxbdi JbZ9Ze2CkY05c9Ty/mCQitebJ+A9lchtketih6170GUkWUqeuzoqgI2UIcI8qR5FAX29 yaUiV/0a+8Txv4++D7GiK9ULTXxF8TAY87X5PY7skqFiyTUjEg1ZNCgxkI6cO+nkepip czZw== X-Gm-Message-State: AOAM531A6ZcbN8Phzk/RXeDl6OziauJEGxD7qI34rfcXFRGq6VHJ+f2n 7wJQlSnfvWluLVmJ5knKLvLUpd5IQ1HUhw== X-Google-Smtp-Source: ABdhPJyQ6MvXody+z+SEzCHIYo/RXfDxe1Hcq+dCQzigpgoBmeVIieqOlljnGlaU6MImq3TlkRL6vQ== X-Received: by 2002:a9d:6414:: with SMTP id h20mr20677067otl.28.1607976690422; Mon, 14 Dec 2020 12:11:30 -0800 (PST) Received: from unknown.attlocal.net ([2600:1700:65a0:ab60:3825:1c64:a3d3:108]) by smtp.gmail.com with ESMTPSA id h26sm3905850ots.9.2020.12.14.12.11.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 12:11:29 -0800 (PST) From: Cong Wang To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, Cong Wang , Song Liu , Alexei Starovoitov , Daniel Borkmann , Dongdong Wang Subject: [Patch bpf-next v2 1/5] bpf: use index instead of hash for map_locked[] Date: Mon, 14 Dec 2020 12:11:14 -0800 Message-Id: <20201214201118.148126-2-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201214201118.148126-1-xiyou.wangcong@gmail.com> References: <20201214201118.148126-1-xiyou.wangcong@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Cong Wang Commit 20b6cc34ea74 ("bpf: Avoid hashtab deadlock with map_locked") introduced a percpu counter map_locked to bail out NMI case. It uses the hash of each bucket for indexing, which requires callers of htab_lock_bucket()/htab_unlock_bucket() to pass it in. But hash value is not always available, especially when we traverse the whole hash table where we do not have keys to compute the hash. We can just compute the index of each bucket with its address and use index instead. This is a prerequisite for the following timeout map patch. Cc: Song Liu Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Dongdong Wang Signed-off-by: Cong Wang --- kernel/bpf/hashtab.c | 57 +++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 27 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 7e848200cd26..f0b7b54fa3a8 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -156,16 +156,17 @@ static void htab_init_buckets(struct bpf_htab *htab) } static inline int htab_lock_bucket(const struct bpf_htab *htab, - struct bucket *b, u32 hash, - unsigned long *pflags) + struct bucket *b, unsigned long *pflags) { unsigned long flags; + unsigned int index; - hash = hash & HASHTAB_MAP_LOCK_MASK; + index = b - htab->buckets; + index &= HASHTAB_MAP_LOCK_MASK; migrate_disable(); - if (unlikely(__this_cpu_inc_return(*(htab->map_locked[hash])) != 1)) { - __this_cpu_dec(*(htab->map_locked[hash])); + if (unlikely(__this_cpu_inc_return(*(htab->map_locked[index])) != 1)) { + __this_cpu_dec(*(htab->map_locked[index])); migrate_enable(); return -EBUSY; } @@ -180,15 +181,17 @@ static inline int htab_lock_bucket(const struct bpf_htab *htab, } static inline void htab_unlock_bucket(const struct bpf_htab *htab, - struct bucket *b, u32 hash, - unsigned long flags) + struct bucket *b, unsigned long flags) { - hash = hash & HASHTAB_MAP_LOCK_MASK; + unsigned int index; + + index = b - htab->buckets; + index &= HASHTAB_MAP_LOCK_MASK; if (htab_use_raw_lock(htab)) raw_spin_unlock_irqrestore(&b->raw_lock, flags); else spin_unlock_irqrestore(&b->lock, flags); - __this_cpu_dec(*(htab->map_locked[hash])); + __this_cpu_dec(*(htab->map_locked[index])); migrate_enable(); } @@ -710,7 +713,7 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) b = __select_bucket(htab, tgt_l->hash); head = &b->head; - ret = htab_lock_bucket(htab, b, tgt_l->hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return false; @@ -720,7 +723,7 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) break; } - htab_unlock_bucket(htab, b, tgt_l->hash, flags); + htab_unlock_bucket(htab, b, flags); return l == tgt_l; } @@ -1019,7 +1022,7 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, */ } - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1062,7 +1065,7 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, } ret = 0; err: - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); return ret; } @@ -1100,7 +1103,7 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, return -ENOMEM; memcpy(l_new->key + round_up(map->key_size, 8), value, map->value_size); - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1121,7 +1124,7 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, ret = 0; err: - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); if (ret) bpf_lru_push_free(&htab->lru, &l_new->lru_node); @@ -1156,7 +1159,7 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, b = __select_bucket(htab, hash); head = &b->head; - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1181,7 +1184,7 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, } ret = 0; err: - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); return ret; } @@ -1221,7 +1224,7 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, return -ENOMEM; } - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1245,7 +1248,7 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, } ret = 0; err: - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); if (l_new) bpf_lru_push_free(&htab->lru, &l_new->lru_node); return ret; @@ -1283,7 +1286,7 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) b = __select_bucket(htab, hash); head = &b->head; - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1296,7 +1299,7 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) ret = -ENOENT; } - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); return ret; } @@ -1318,7 +1321,7 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) b = __select_bucket(htab, hash); head = &b->head; - ret = htab_lock_bucket(htab, b, hash, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) return ret; @@ -1329,7 +1332,7 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) else ret = -ENOENT; - htab_unlock_bucket(htab, b, hash, flags); + htab_unlock_bucket(htab, b, flags); if (l) bpf_lru_push_free(&htab->lru, &l->lru_node); return ret; @@ -1480,7 +1483,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, head = &b->head; /* do not grab the lock unless need it (bucket_cnt > 0). */ if (locked) { - ret = htab_lock_bucket(htab, b, batch, &flags); + ret = htab_lock_bucket(htab, b, &flags); if (ret) goto next_batch; } @@ -1500,7 +1503,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, /* Note that since bucket_cnt > 0 here, it is implicit * that the locked was grabbed, so release it. */ - htab_unlock_bucket(htab, b, batch, flags); + htab_unlock_bucket(htab, b, flags); rcu_read_unlock(); bpf_enable_instrumentation(); goto after_loop; @@ -1511,7 +1514,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, /* Note that since bucket_cnt > 0 here, it is implicit * that the locked was grabbed, so release it. */ - htab_unlock_bucket(htab, b, batch, flags); + htab_unlock_bucket(htab, b, flags); rcu_read_unlock(); bpf_enable_instrumentation(); kvfree(keys); @@ -1564,7 +1567,7 @@ __htab_map_lookup_and_delete_batch(struct bpf_map *map, dst_val += value_size; } - htab_unlock_bucket(htab, b, batch, flags); + htab_unlock_bucket(htab, b, flags); locked = false; while (node_to_free) { From patchwork Mon Dec 14 20:11:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 11972947 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D953C2BB40 for ; Mon, 14 Dec 2020 20:16:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33D37207A0 for ; Mon, 14 Dec 2020 20:16:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2503081AbgLNUMV (ORCPT ); Mon, 14 Dec 2020 15:12:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2503077AbgLNUMN (ORCPT ); Mon, 14 Dec 2020 15:12:13 -0500 Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE8C0C061794; Mon, 14 Dec 2020 12:11:32 -0800 (PST) Received: by mail-ot1-x329.google.com with SMTP id a109so17027712otc.1; Mon, 14 Dec 2020 12:11:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4WBlcHYwI+3Y4HP+YoX+oWasgSpHPqx6MTYqxpSQ1yY=; b=Ni7fGlpTw8we4buhZJ8egClgs5vrGx5Wre44rUwdLhckcWC/MnDE9wqngJdUUeMtl8 XkZ7CBoSshouMqHbC7stypih2OtM1/UZxXDNnp44qjRic83B4gDueJT66T8Ycr6bMFJr c8Qng8c7U+mnueUMAcH5nfZUgP3ZlIz1LZIisgni48BkTuOXd8tqFpfKR6dxVdZoqbn1 0Cm/qgmhWofJf+7eaxbZ5dMM10ozkx6C6zrMJAl1pxNe+bhceO4T7ybRdGtyoTQep9gx V6S8YKjAUBEObDPbw2g3OPon4H551fLnPzpsLjVpGBrV0ExshEcmUFHY7v3f/ywSAPD2 9C1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4WBlcHYwI+3Y4HP+YoX+oWasgSpHPqx6MTYqxpSQ1yY=; b=os2xoFJHoS0C18xTtOVSJ85ZtK+z5hkKlRbnfAXM67zWP8Dxld2BHm1/vFX91Xo24Y 2OdUrygLq0ZOyfWH8CMmKi3Iz1KZ3ijbL/NnaDka3tIabVkYO32A64yOPhptlfM4fVEs SXN6eWkpc33vrmYDNIX80fnTbaL6q4rTyYcPypwHpKsoGZ1Ih6YPH+JSmEskLVFBzu0j hgcUOswUdy/6xZIB3If3E7W8lKvOQJEKDqO7aH1NTNjz/6SRPZ7RVCMswghN3XU17wgz SRJUTBfIwnYW3/BSg3HdGxHOFNpSF31etDo3URxU3V0PA+LcqSoxdJe+VLm6U1jnQYW5 dnVA== X-Gm-Message-State: AOAM531YtrXpC1vjo8auLjgibsxORJT5swCTgruTLthF16iTBox+nxKB HyrL1a+aWVscMjFSD7o2fF285YcPFoqzPg== X-Google-Smtp-Source: ABdhPJyLsZ5UBC30C+lm6Mij+WYsV1NDTcblfqDugRXKaCDtWMOREfnjFQeg4GslxkRuu8iDUexL1w== X-Received: by 2002:a05:6830:1c34:: with SMTP id f20mr18716328ote.147.1607976691783; Mon, 14 Dec 2020 12:11:31 -0800 (PST) Received: from unknown.attlocal.net ([2600:1700:65a0:ab60:3825:1c64:a3d3:108]) by smtp.gmail.com with ESMTPSA id h26sm3905850ots.9.2020.12.14.12.11.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 12:11:31 -0800 (PST) From: Cong Wang To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, Cong Wang , Alexei Starovoitov , Daniel Borkmann , Dongdong Wang Subject: [Patch bpf-next v2 2/5] bpf: introduce timeout map Date: Mon, 14 Dec 2020 12:11:15 -0800 Message-Id: <20201214201118.148126-3-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201214201118.148126-1-xiyou.wangcong@gmail.com> References: <20201214201118.148126-1-xiyou.wangcong@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Cong Wang This borrows the idea from conntrack and will be used for conntrack in bpf too. Each element in a timeout map has a user-specified timeout in secs, after it expires it will be automatically removed from the map. There are two cases here: 1. When the timeout map is idle, that is, no one updates or accesses it, we rely on the idle work to scan the whole hash table and remove these expired. The idle work is scheduled every 1 sec. 2. When the timeout map is actively accessed, we could reach expired elements before the idle work kicks in, we can simply skip them and schedule another work to do the actual removal work. We avoid taking locks on fast path. The timeout of each element can be set or updated via bpf_map_update_elem() and we reuse the upper 32-bit of the 64-bit flag for the timeout value. Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Dongdong Wang Signed-off-by: Cong Wang --- include/linux/bpf_types.h | 1 + include/uapi/linux/bpf.h | 3 +- kernel/bpf/hashtab.c | 244 ++++++++++++++++++++++++++++++++- kernel/bpf/syscall.c | 3 +- tools/include/uapi/linux/bpf.h | 1 + 5 files changed, 248 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h index 99f7fd657d87..00a3b17b6af2 100644 --- a/include/linux/bpf_types.h +++ b/include/linux/bpf_types.h @@ -125,6 +125,7 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_STACK, stack_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_STRUCT_OPS, bpf_struct_ops_map_ops) #endif BPF_MAP_TYPE(BPF_MAP_TYPE_RINGBUF, ringbuf_map_ops) +BPF_MAP_TYPE(BPF_MAP_TYPE_TIMEOUT_HASH, htab_timeout_map_ops) BPF_LINK_TYPE(BPF_LINK_TYPE_RAW_TRACEPOINT, raw_tracepoint) BPF_LINK_TYPE(BPF_LINK_TYPE_TRACING, tracing) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 30b477a26482..dedb47bc3f52 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -158,6 +158,7 @@ enum bpf_map_type { BPF_MAP_TYPE_RINGBUF, BPF_MAP_TYPE_INODE_STORAGE, BPF_MAP_TYPE_TASK_STORAGE, + BPF_MAP_TYPE_TIMEOUT_HASH, }; /* Note that tracing related programs such as @@ -393,7 +394,7 @@ enum bpf_link_type { */ #define BPF_PSEUDO_CALL 1 -/* flags for BPF_MAP_UPDATE_ELEM command */ +/* flags for BPF_MAP_UPDATE_ELEM command, upper 32 bits are timeout */ enum { BPF_ANY = 0, /* create new element or update existing */ BPF_NOEXIST = 1, /* create new element if it didn't exist */ diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index f0b7b54fa3a8..178cb376c397 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -8,6 +8,8 @@ #include #include #include +#include +#include #include #include #include "percpu_freelist.h" @@ -84,6 +86,8 @@ struct bucket { raw_spinlock_t raw_lock; spinlock_t lock; }; + struct llist_node gc_node; + atomic_t pending; }; #define HASHTAB_MAP_LOCK_COUNT 8 @@ -104,6 +108,9 @@ struct bpf_htab { u32 hashrnd; struct lock_class_key lockdep_key; int __percpu *map_locked[HASHTAB_MAP_LOCK_COUNT]; + struct llist_head gc_list; + struct work_struct gc_work; + struct delayed_work gc_idle_work; }; /* each htab element is struct htab_elem + key + value */ @@ -124,6 +131,7 @@ struct htab_elem { struct bpf_lru_node lru_node; }; u32 hash; + u64 expires; char key[] __aligned(8); }; @@ -143,6 +151,7 @@ static void htab_init_buckets(struct bpf_htab *htab) for (i = 0; i < htab->n_buckets; i++) { INIT_HLIST_NULLS_HEAD(&htab->buckets[i].head, i); + atomic_set(&htab->buckets[i].pending, 0); if (htab_use_raw_lock(htab)) { raw_spin_lock_init(&htab->buckets[i].raw_lock); lockdep_set_class(&htab->buckets[i].raw_lock, @@ -431,6 +440,14 @@ static int htab_map_alloc_check(union bpf_attr *attr) return 0; } +static void htab_sched_gc(struct bpf_htab *htab, struct bucket *b) +{ + if (atomic_fetch_or(1, &b->pending)) + return; + llist_add(&b->gc_node, &htab->gc_list); + queue_work(system_unbound_wq, &htab->gc_work); +} + static struct bpf_map *htab_map_alloc(union bpf_attr *attr) { bool percpu = (attr->map_type == BPF_MAP_TYPE_PERCPU_HASH || @@ -732,10 +749,13 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) { struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + bool is_timeout = map->map_type == BPF_MAP_TYPE_TIMEOUT_HASH; struct hlist_nulls_head *head; struct htab_elem *l, *next_l; u32 hash, key_size; + struct bucket *b; int i = 0; + u64 now; WARN_ON_ONCE(!rcu_read_lock_held()); @@ -746,7 +766,8 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) hash = htab_map_hash(key, key_size, htab->hashrnd); - head = select_bucket(htab, hash); + b = __select_bucket(htab, hash); + head = &b->head; /* lookup the key */ l = lookup_nulls_elem_raw(head, hash, key, key_size, htab->n_buckets); @@ -759,6 +780,13 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) struct htab_elem, hash_node); if (next_l) { + if (is_timeout) { + now = get_jiffies_64(); + if (time_after_eq64(now, next_l->expires)) { + htab_sched_gc(htab, b); + goto find_first_elem; + } + } /* if next elem in this hash list is non-zero, just return it */ memcpy(next_key, next_l->key, key_size); return 0; @@ -771,12 +799,20 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key) find_first_elem: /* iterate over buckets */ for (; i < htab->n_buckets; i++) { - head = select_bucket(htab, i); + b = __select_bucket(htab, i); + head = &b->head; /* pick first element in the bucket */ next_l = hlist_nulls_entry_safe(rcu_dereference_raw(hlist_nulls_first_rcu(head)), struct htab_elem, hash_node); if (next_l) { + if (is_timeout) { + now = get_jiffies_64(); + if (time_after_eq64(now, next_l->expires)) { + htab_sched_gc(htab, b); + continue; + } + } /* if it's not empty, just return it */ memcpy(next_key, next_l->key, key_size); return 0; @@ -975,18 +1011,31 @@ static int check_flags(struct bpf_htab *htab, struct htab_elem *l_old, return 0; } +static u32 fetch_timeout(u64 *map_flags) +{ + u32 timeout = (*map_flags) >> 32; + + *map_flags = (*map_flags) & 0xffffffff; + return timeout; +} + /* Called from syscall or from eBPF program */ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags) { struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + bool timeout_map = map->map_type == BPF_MAP_TYPE_TIMEOUT_HASH; struct htab_elem *l_new = NULL, *l_old; struct hlist_nulls_head *head; unsigned long flags; struct bucket *b; u32 key_size, hash; + u32 timeout; + u64 now; int ret; + timeout = fetch_timeout(&map_flags); + if (unlikely((map_flags & ~BPF_F_LOCK) > BPF_EXIST)) /* unknown flags */ return -EINVAL; @@ -1042,6 +1091,10 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, copy_map_value_locked(map, l_old->key + round_up(key_size, 8), value, false); + if (timeout_map) { + now = get_jiffies_64(); + l_old->expires = now + HZ * timeout; + } ret = 0; goto err; } @@ -1054,6 +1107,13 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, goto err; } + if (timeout_map) { + now = get_jiffies_64(); + if (l_old && time_after_eq64(now, l_old->expires)) + htab_sched_gc(htab, b); + l_new->expires = now + HZ * timeout; + } + /* add new element to the head of the list, so that * concurrent search will find it before old elem */ @@ -2180,3 +2240,183 @@ const struct bpf_map_ops htab_of_maps_map_ops = { .map_btf_name = "bpf_htab", .map_btf_id = &htab_of_maps_map_btf_id, }; + +static void __htab_gc_bucket(struct bpf_htab *htab, struct bucket *b) +{ + struct hlist_nulls_head *head = &b->head; + struct hlist_nulls_node *n; + u64 now = get_jiffies_64(); + struct htab_elem *l; + + hlist_nulls_for_each_entry_safe(l, n, head, hash_node) { + if (time_after_eq64(now, l->expires)) { + hlist_nulls_del_rcu(&l->hash_node); + free_htab_elem(htab, l); + } + } +} + +static void htab_gc(struct work_struct *work) +{ + struct llist_node *head; + struct bpf_htab *htab; + struct bucket *b, *n; + + htab = container_of(work, struct bpf_htab, gc_work); + head = llist_del_all(&htab->gc_list); + + llist_for_each_entry_safe(b, n, head, gc_node) { + unsigned long flags; + int ret; + + ret = htab_lock_bucket(htab, b, &flags); + if (ret) + continue; + __htab_gc_bucket(htab, b); + htab_unlock_bucket(htab, b, flags); + + atomic_set(&b->pending, 0); + + cond_resched(); + } +} + +static void htab_gc_idle(struct work_struct *work) +{ + struct bpf_htab *htab; + int i; + + htab = container_of(work, struct bpf_htab, gc_idle_work.work); + + for (i = 0; i < htab->n_buckets; i++) { + unsigned long flags; + struct bucket *b; + int ret; + + b = __select_bucket(htab, i); + if (hlist_nulls_empty(&b->head)) + continue; + if (atomic_read(&b->pending)) + continue; + ret = htab_lock_bucket(htab, b, &flags); + if (ret) + continue; + __htab_gc_bucket(htab, b); + htab_unlock_bucket(htab, b, flags); + cond_resched(); + } + + queue_delayed_work(system_power_efficient_wq, &htab->gc_idle_work, HZ); +} + +static void *__htab_timeout_map_lookup_elem(struct bpf_map *map, void *key) +{ + struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + struct hlist_nulls_head *head; + struct htab_elem *l; + struct bucket *b; + u32 key_size = map->key_size; + u32 hash; + + hash = htab_map_hash(key, key_size, htab->hashrnd); + b = __select_bucket(htab, hash); + head = &b->head; + + l = lookup_nulls_elem_raw(head, hash, key, key_size, htab->n_buckets); + if (l && time_after_eq64(get_jiffies_64(), l->expires)) { + htab_sched_gc(htab, b); + l = NULL; + } + + return l; +} + +static void *htab_timeout_map_lookup_elem(struct bpf_map *map, void *key) +{ + struct htab_elem *l = __htab_timeout_map_lookup_elem(map, key); + + if (l) + return l->key + round_up(map->key_size, 8); + return NULL; +} + +static int htab_timeout_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf) +{ + struct bpf_insn *insn = insn_buf; + const int ret = BPF_REG_0; + + BUILD_BUG_ON(!__same_type(&__htab_timeout_map_lookup_elem, + (void *(*)(struct bpf_map *map, void *key))NULL)); + *insn++ = BPF_EMIT_CALL(BPF_CAST_CALL(__htab_timeout_map_lookup_elem)); + *insn++ = BPF_JMP_IMM(BPF_JEQ, ret, 0, 1); + *insn++ = BPF_ALU64_IMM(BPF_ADD, ret, + offsetof(struct htab_elem, key) + + round_up(map->key_size, 8)); + return insn - insn_buf; +} + +static void htab_timeout_map_seq_show_elem(struct bpf_map *map, void *key, + struct seq_file *m) +{ + void *value; + + rcu_read_lock(); + + value = htab_timeout_map_lookup_elem(map, key); + if (!value) { + rcu_read_unlock(); + return; + } + + btf_type_seq_show(map->btf, map->btf_key_type_id, key, m); + seq_puts(m, ": "); + btf_type_seq_show(map->btf, map->btf_value_type_id, value, m); + seq_puts(m, "\n"); + + rcu_read_unlock(); +} + +static struct bpf_map *htab_timeout_map_alloc(union bpf_attr *attr) +{ + struct bpf_map *map = htab_map_alloc(attr); + struct bpf_htab *htab; + + if (!IS_ERR(map)) { + htab = container_of(map, struct bpf_htab, map); + init_llist_head(&htab->gc_list); + INIT_WORK(&htab->gc_work, htab_gc); + INIT_DEFERRABLE_WORK(&htab->gc_idle_work, htab_gc_idle); + queue_delayed_work(system_power_efficient_wq, + &htab->gc_idle_work, HZ); + } + + return map; +} + +static void htab_timeout_map_free(struct bpf_map *map) +{ + struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + + cancel_work_sync(&htab->gc_work); + cancel_delayed_work_sync(&htab->gc_idle_work); + + htab_map_free(map); +} + +static int htab_timeout_map_btf_id; +const struct bpf_map_ops htab_timeout_map_ops = { + .map_meta_equal = bpf_map_meta_equal, + .map_alloc_check = htab_map_alloc_check, + .map_alloc = htab_timeout_map_alloc, + .map_free = htab_timeout_map_free, + .map_get_next_key = htab_map_get_next_key, + .map_lookup_elem = htab_timeout_map_lookup_elem, + .map_update_elem = htab_map_update_elem, + .map_delete_elem = htab_map_delete_elem, + .map_gen_lookup = htab_timeout_map_gen_lookup, + .map_seq_show_elem = htab_timeout_map_seq_show_elem, + BATCH_OPS(htab), + .map_btf_name = "bpf_htab", + .map_btf_id = &htab_timeout_map_btf_id, + .iter_seq_info = &iter_seq_info, +}; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 287be337d5f6..9ebd2e380a57 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -778,7 +778,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, map->map_type != BPF_MAP_TYPE_CGROUP_STORAGE && map->map_type != BPF_MAP_TYPE_SK_STORAGE && map->map_type != BPF_MAP_TYPE_INODE_STORAGE && - map->map_type != BPF_MAP_TYPE_TASK_STORAGE) + map->map_type != BPF_MAP_TYPE_TASK_STORAGE && + map->map_type != BPF_MAP_TYPE_TIMEOUT_HASH) return -ENOTSUPP; if (map->spin_lock_off + sizeof(struct bpf_spin_lock) > map->value_size) { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 30b477a26482..684b8011a97a 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -158,6 +158,7 @@ enum bpf_map_type { BPF_MAP_TYPE_RINGBUF, BPF_MAP_TYPE_INODE_STORAGE, BPF_MAP_TYPE_TASK_STORAGE, + BPF_MAP_TYPE_TIMEOUT_HASH, }; /* Note that tracing related programs such as From patchwork Mon Dec 14 20:11:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 11972939 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE141C2BB40 for ; Mon, 14 Dec 2020 20:12:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BEB42168B for ; Mon, 14 Dec 2020 20:12:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2503076AbgLNUMU (ORCPT ); Mon, 14 Dec 2020 15:12:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2503078AbgLNUMO (ORCPT ); Mon, 14 Dec 2020 15:12:14 -0500 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9AADC06179C; Mon, 14 Dec 2020 12:11:33 -0800 (PST) Received: by mail-ot1-x32c.google.com with SMTP id j20so12559577otq.5; Mon, 14 Dec 2020 12:11:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GJIQSb408Ty07UVJPZWzak1MwnXpH47wjbyMmA0spq4=; b=qNHkGvqK+IKHMiz8hQn6ahIFoHOp9NYvrFtggOrt68UHmpe9hN/QlXQOAmToUcZjTl jo3jTSNv48icSB/8gHOygKKBbzBtl+gYWb05bx7/v6DCDUTBVNU8RL7lpx5zEr+RjuMR RQqvrCWc3sxk6NR0549X56SZoVgnwyeZO7wdJHnn6+u2J3Fh7CpJQH6tynZSAK9oPilj QMlu9m6jqrS27xW4heImiMNNbG8EIq6EaGtpc/sLle8uTRzOBuMuNz4bacAyhRe11Rqx nO9I/zhsiwMUL0wPzwVDEtmJSl+SX80YQI1YveBehzN1gTkl30zvqAzhJBq6iAREO+wT fKeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GJIQSb408Ty07UVJPZWzak1MwnXpH47wjbyMmA0spq4=; b=B7mVcdEQmrTXCZawUdSperw5ubP/8UEKpCEU4vvnz7i/R98z1hfOBX4Le47vrbcIZ5 rypcDdXQ69p2x7JAfREu4p8o+ClKgO5XtWfZ6m0qxgEpWzToPSxSIrRvoNLwx6faH9nZ 3sI1U7Ih4Oz9+0+ADTk43kWzq4axSK/+xyA4GawYomUKPHxn368DDEpoP1nqMtFnqoI1 j9pfX9DbdpqaaRZjF8ZzrIw64QUBq9fBj2ucFQDfCRagPPqb01uUQM3kPIYC8HEeDdcH M2I4HTHywz0bkgKXiyZdA56Hxu4KuO2E0hYYIxU/IW0/QUHHZz2nHjC78kuBY0Pu8mTl WvYg== X-Gm-Message-State: AOAM531GPGkLdutmKu5NLOTW4zAVmCiTHBHc26h/UOssl3W15kRiEjel pEGPVli/XkK/3OkPCqpMZKg6Rk7n9jkuew== X-Google-Smtp-Source: ABdhPJzYi5jZ+ElEupPhmr7mTv/+CDm0Gk7x9oUTD1f4+ngcii3H2+t5cOCtTtMlzPxrO4z4NEhdmQ== X-Received: by 2002:a9d:3d64:: with SMTP id a91mr20435167otc.144.1607976693095; Mon, 14 Dec 2020 12:11:33 -0800 (PST) Received: from unknown.attlocal.net ([2600:1700:65a0:ab60:3825:1c64:a3d3:108]) by smtp.gmail.com with ESMTPSA id h26sm3905850ots.9.2020.12.14.12.11.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 12:11:32 -0800 (PST) From: Cong Wang To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, Cong Wang , Andrey Ignatov , Alexei Starovoitov , Daniel Borkmann , Dongdong Wang Subject: [Patch bpf-next v2 3/5] selftests/bpf: update elem_size check in map ptr test Date: Mon, 14 Dec 2020 12:11:16 -0800 Message-Id: <20201214201118.148126-4-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201214201118.148126-1-xiyou.wangcong@gmail.com> References: <20201214201118.148126-1-xiyou.wangcong@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Cong Wang In map ptr test, a hard-coded 64 is used to check hash element size. Increase it to 72 as we increase the size of struct htab_elem. It seems struct htab_elem is not visible here. Cc: Andrey Ignatov Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Dongdong Wang Signed-off-by: Cong Wang Acked-by: Andrey Ignatov --- tools/testing/selftests/bpf/progs/map_ptr_kern.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c index d8850bc6a9f1..34f9880a1903 100644 --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c @@ -111,7 +111,7 @@ static inline int check_hash(void) VERIFY(check_default_noinline(&hash->map, map)); VERIFY(hash->n_buckets == MAX_ENTRIES); - VERIFY(hash->elem_size == 64); + VERIFY(hash->elem_size == 72); VERIFY(hash->count.counter == 0); for (i = 0; i < HALF_ENTRIES; ++i) { From patchwork Mon Dec 14 20:11:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 11972943 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89EBBC2BB40 for ; Mon, 14 Dec 2020 20:15:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 54FC1207A2 for ; Mon, 14 Dec 2020 20:15:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2503088AbgLNUMc (ORCPT ); Mon, 14 Dec 2020 15:12:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2503085AbgLNUMZ (ORCPT ); Mon, 14 Dec 2020 15:12:25 -0500 Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 393CDC0617A6; Mon, 14 Dec 2020 12:11:35 -0800 (PST) Received: by mail-ot1-x335.google.com with SMTP id y24so17031356otk.3; Mon, 14 Dec 2020 12:11:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o9Y3t8Nk2w6YOn+aCane90NBqrXU64pdv9z0KzYNQcM=; b=SKBHBLM5dDCb3JwuEfm+9iWPJnpkves4bhwMH4UENFEtBD1HsOtOFVouipB+HRHMFL LJag3tkLsWADTymPPEmypNx+9gTFTz7SvhbFUQJkNJGCigI82hh0pJAAA1+cePKq+FK2 T/JvMiCmF52feBh+8ShF5z265q7asjFdRKt9nSOw1iuTX88nzuTtF7DU3ohKO0wZvyI6 UYxjiBaex32AzfXsL5WEF55/g+oV2mP/Lw3aAvJAHdlwb4VuIllHzbF31V5PRU7j4FGW blKh3/tOkwq5jNC5aeXxYDqsFH8M9lq2jSEw1RhLevZLiV1cjNBmy0BeGr16FdDxYvip BGqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o9Y3t8Nk2w6YOn+aCane90NBqrXU64pdv9z0KzYNQcM=; b=LbzzPCuVy6AxtfABml9jWTgsLFZCff1fxm/iB4AxzW9ThSBzeHXGxYr1R38EHqUTBQ IswCaPetuWE2SzyXREzW/lNCWxZeeVBIj8rBJpjDhKj4B6eFVROGlOv4GUnrJ6H7fCkA Lcgdqd/iPvKYq3clzjAPL6uQsyI+bFdhDNUxMr9+KsUSzERAuoE7YCEMGLnkzI+xnuje 3BHIXkz8Sni+qhBGm7Bjt6KjKPHso/KeXWmDOah51u+jC536eUOI9jr8AcNeWFfxy8Sw OteusZ1hPNFMntG/jNqaEqKyDcgnyF29u11WnyfkGBKFa21LHYGEJpg4Myy9hCfS72JJ Ehaw== X-Gm-Message-State: AOAM53200C2s5jY7o9QiXxVzx/MQbPSZghzS1tVF6FU0v8p6dSMbUQk7 BddRfrKZuNtoVyvCDqvLlTPJydelxxOrVQ== X-Google-Smtp-Source: ABdhPJzzF+vFqL72EuNuS3aPwTru5+b6OBGT8hh3HicI2PU72klrBS8onnVhAcKcYD1dRj8NPFEQ3g== X-Received: by 2002:a05:6830:118b:: with SMTP id u11mr21016023otq.130.1607976694404; Mon, 14 Dec 2020 12:11:34 -0800 (PST) Received: from unknown.attlocal.net ([2600:1700:65a0:ab60:3825:1c64:a3d3:108]) by smtp.gmail.com with ESMTPSA id h26sm3905850ots.9.2020.12.14.12.11.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 12:11:33 -0800 (PST) From: Cong Wang To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, Cong Wang , Alexei Starovoitov , Daniel Borkmann , Dongdong Wang Subject: [Patch bpf-next v2 4/5] selftests/bpf: add a test case for bpf timeout map Date: Mon, 14 Dec 2020 12:11:17 -0800 Message-Id: <20201214201118.148126-5-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201214201118.148126-1-xiyou.wangcong@gmail.com> References: <20201214201118.148126-1-xiyou.wangcong@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Cong Wang Add a test case in test_maps.c for timeout map, which focuses on testing timeout. Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Dongdong Wang Signed-off-by: Cong Wang --- tools/testing/selftests/bpf/test_maps.c | 41 +++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index 0ad3e6305ff0..9fc7b5424fdf 100644 --- a/tools/testing/selftests/bpf/test_maps.c +++ b/tools/testing/selftests/bpf/test_maps.c @@ -1723,6 +1723,45 @@ static void test_reuseport_array(void) close(map_fd); } +static void test_timeout_map(void) +{ + int val1 = 1, val2 = 2, val3 = 3; + int key1 = 1, key2 = 2, key3 = 3; + int fd; + + fd = bpf_create_map(BPF_MAP_TYPE_TIMEOUT_HASH, sizeof(int), sizeof(int), + 3, map_flags); + if (fd < 0) { + printf("Failed to create timeout map '%s'!\n", strerror(errno)); + exit(1); + } + + /* Timeout after 1 secs */ + assert(bpf_map_update_elem(fd, &key1, &val1, (u64)1<<32) == 0); + /* Timeout after 2 secs */ + assert(bpf_map_update_elem(fd, &key2, &val2, (u64)2<<32) == 0); + /* Timeout after 10 secs */ + assert(bpf_map_update_elem(fd, &key3, &val3, (u64)10<<32) == 0); + + sleep(1); + assert(bpf_map_lookup_elem(fd, &key1, &val1) != 0); + val2 = 0; + assert(bpf_map_lookup_elem(fd, &key2, &val2) == 0 && val2 == 2); + + sleep(1); + assert(bpf_map_lookup_elem(fd, &key1, &val1) != 0); + assert(bpf_map_lookup_elem(fd, &key2, &val2) != 0); + + /* Modify timeout to expire it earlier */ + val3 = 0; + assert(bpf_map_lookup_elem(fd, &key3, &val3) == 0 && val3 == 3); + assert(bpf_map_update_elem(fd, &key3, &val3, (u64)1<<32) == 0); + sleep(1); + assert(bpf_map_lookup_elem(fd, &key3, &val3) != 0); + + close(fd); +} + static void run_all_tests(void) { test_hashmap(0, NULL); @@ -1752,6 +1791,8 @@ static void run_all_tests(void) test_stackmap(0, NULL); test_map_in_map(); + + test_timeout_map(); } #define DEFINE_TEST(name) extern void test_##name(void); From patchwork Mon Dec 14 20:11:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 11972941 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F7A8C4361B for ; Mon, 14 Dec 2020 20:12:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC106207A2 for ; Mon, 14 Dec 2020 20:12:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2503084AbgLNUMc (ORCPT ); Mon, 14 Dec 2020 15:12:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2503086AbgLNUMZ (ORCPT ); Mon, 14 Dec 2020 15:12:25 -0500 Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com [IPv6:2607:f8b0:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EA2FC0617A7; Mon, 14 Dec 2020 12:11:36 -0800 (PST) Received: by mail-ot1-x32e.google.com with SMTP id 11so17026629oty.9; Mon, 14 Dec 2020 12:11:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=opfRPH7/5Ygcvu8PVbcBIUkauhhzFK5c8dPVoi/d7pI=; b=E+jnUsXxWviLRRfLL+l4oUhWA62lMNJgnIiP668fkpej1LOjtj2lvGDF1w7P5F7vWV RWazJ9flLUdPyARwJC0m9pjcAYUBLGyXBrL/ANtPFCKZmrflZcydPy+Anu6JSnKSk9iD B6UanvngCAl9R9owd0gRDexcM9MXbhaXxgPcXhje3NCYWomdMEXUmlqGGSfBnQSXhF9z SZls60dzb/4MRNTRfI4DPoz+xdshUhDX0wC9DrqrV4wBllNewwKoHVleG1NQSPEH11vc cCO7T7QK6ixcvSBF2hhV/HrmYI9bySR45Z4Mef7CSaKcG8UbPgjUOLZ+XWQUtwrmW7lS NMCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=opfRPH7/5Ygcvu8PVbcBIUkauhhzFK5c8dPVoi/d7pI=; b=LIYC4Mq6Hh3a64sc2Z38WFYWuMfkzpnu11mzKBZ2rYMQhhGC+ZNt45VNRUpq3DmDv5 vPB9SHiKB1GNrxu1FwRtkQ7fP+xPxYliJcVSF3w0hYgMJjkOjXAsoJyKDea6c7iIc7/N GymKd5JNNuSI8cWUslALej6/Wb7b9m8rmyS3ZeAejdF9RK1KizFurK514QtnSX2/RUQR E/ow9m0NvV73IDsoJICG6DHYhAduKblGLx50BG3YGzySel3RhMPI5RXeaRkCKqJ2/JLI X0XlLuwEBXtU2Vc6SSV9YgwH1pt9XPEkpUdnvNAdioKREOCV6CfwZUU8NhIVrkBydx0C KTvA== X-Gm-Message-State: AOAM533Jxbpp3jxdNFQC5hE3IvvzvDeFc1ci8WScKeurJQYA1H5MTsfw d88Uq47jkKV6byWd2KlRnnie61ydFp9J9g== X-Google-Smtp-Source: ABdhPJzTnvIVLEIpm76mSDP8tTDwwGMBJlJBgfichUH2q/uxHdWfVt5bn1GAilUFyiUwyvqX66k9kw== X-Received: by 2002:a9d:27e9:: with SMTP id c96mr20504771otb.15.1607976695733; Mon, 14 Dec 2020 12:11:35 -0800 (PST) Received: from unknown.attlocal.net ([2600:1700:65a0:ab60:3825:1c64:a3d3:108]) by smtp.gmail.com with ESMTPSA id h26sm3905850ots.9.2020.12.14.12.11.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 12:11:35 -0800 (PST) From: Cong Wang To: netdev@vger.kernel.org Cc: bpf@vger.kernel.org, Cong Wang , Andrey Ignatov , Alexei Starovoitov , Daniel Borkmann , Dongdong Wang Subject: [Patch bpf-next v2 5/5] selftests/bpf: add timeout map check in map_ptr tests Date: Mon, 14 Dec 2020 12:11:18 -0800 Message-Id: <20201214201118.148126-6-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201214201118.148126-1-xiyou.wangcong@gmail.com> References: <20201214201118.148126-1-xiyou.wangcong@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Cong Wang Similar to regular hashmap test. Cc: Andrey Ignatov Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Dongdong Wang Signed-off-by: Cong Wang Acked-by: Andrey Ignatov --- .../selftests/bpf/progs/map_ptr_kern.c | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tools/testing/selftests/bpf/progs/map_ptr_kern.c b/tools/testing/selftests/bpf/progs/map_ptr_kern.c index 34f9880a1903..f158b4f7e6c8 100644 --- a/tools/testing/selftests/bpf/progs/map_ptr_kern.c +++ b/tools/testing/selftests/bpf/progs/map_ptr_kern.c @@ -648,6 +648,25 @@ static inline int check_ringbuf(void) return 1; } +struct { + __uint(type, BPF_MAP_TYPE_TIMEOUT_HASH); + __uint(max_entries, MAX_ENTRIES); + __type(key, __u32); + __type(value, __u32); +} m_timeout SEC(".maps"); + +static inline int check_timeout_hash(void) +{ + struct bpf_htab *timeout_hash = (struct bpf_htab *)&m_timeout; + struct bpf_map *map = (struct bpf_map *)&m_timeout; + + VERIFY(check_default(&timeout_hash->map, map)); + VERIFY(timeout_hash->n_buckets == MAX_ENTRIES); + VERIFY(timeout_hash->elem_size == 72); + + return 1; +} + SEC("cgroup_skb/egress") int cg_skb(void *ctx) { @@ -679,6 +698,7 @@ int cg_skb(void *ctx) VERIFY_TYPE(BPF_MAP_TYPE_SK_STORAGE, check_sk_storage); VERIFY_TYPE(BPF_MAP_TYPE_DEVMAP_HASH, check_devmap_hash); VERIFY_TYPE(BPF_MAP_TYPE_RINGBUF, check_ringbuf); + VERIFY_TYPE(BPF_MAP_TYPE_TIMEOUT_HASH, check_timeout_hash); return 1; }