From patchwork Sat Jul 9 15:44:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12912268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 497AAC43334 for ; Sat, 9 Jul 2022 15:45:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7DB06B0073; Sat, 9 Jul 2022 11:45:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A28FE6B0074; Sat, 9 Jul 2022 11:45:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87BFB6B0075; Sat, 9 Jul 2022 11:45:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7B13B6B0073 for ; Sat, 9 Jul 2022 11:45:03 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4206F346E4 for ; Sat, 9 Jul 2022 15:45:03 +0000 (UTC) X-FDA: 79667984886.18.DEC66C4 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf09.hostedemail.com (Postfix) with ESMTP id ECFF214006A for ; Sat, 9 Jul 2022 15:45:02 +0000 (UTC) Received: by mail-qk1-f179.google.com with SMTP id f12so946571qka.12 for ; Sat, 09 Jul 2022 08:45:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bkLct7+3x09q3TPrw9hr1g7AQwWmVQubLfPgu28Dhr0=; b=Q7yw0wFIPOJflUJiXHOo6vvgX5eAazVsQSyQo+3EM00n1vjxcSp0MPsktNjtXotXyb SCUm1SVXetSbg2CthqfDdW9tiiO05DFrQ7Cci270NKwZx45jnez/jmMz8iXGo5yMH6ns QP8p9SLRrv/9zuJ2IRfvXUBoggIehg3rBCQzNJd8e4fCoc2EiF+aFxN+A98p4M6qmWad HPgtddGUzz5sAScyXjvzJHvbb8mgTVJqDLbr0pwjNSeR3zpKLDjrTPgvWm7yeojDvnIH EfdXzndAdsmgSOGiFn2q7vCUx554Y9wFypQ0mpHKwERHFyhIdTBC9uipsW03M9EybnJC 3+CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bkLct7+3x09q3TPrw9hr1g7AQwWmVQubLfPgu28Dhr0=; b=SDmdiThqSSxKPkKNYW8vsc3p/eG2OG4yj5YFGksnZ30LbKfWy/8s8pStwoKdJjRO4I llroIOfkgyAoGw36HHeAJwEPB+Gyz0zX2EGQD9kMaaWrynYuHXpMQ3eTaaCf8WySdcXc 6aL38Ci3Hsf62PxvmIQEdUqQqVB6NWNzTACAg8ZC7Qq3jhR3oajLbOdFFLw78reOJ/zY WoAkbHHOQwXaBShFlf+8VKptwaStXn1UhXIdVuQtzBsY6k4n0H53gs352RFIVnGCumRV QarVQCa6yIIh0KpbS6P+oUexNG/SqNJMMGwsyd01Yyk2lL7xlh3iEQT5wV3xyeGomM37 4qRw== X-Gm-Message-State: AJIora8OfExSttfI98TbntNeD8iBoEYI/5aGAgp+uKsaA5f+COhPKE/i XzdsH9breI5riY7u8+yZQXk= X-Google-Smtp-Source: AGRyM1vM2NyKCJqvfdCCZjo75kvFKaaQ+FzJQlbCqBZZqXIDAZhZWRqSBi7BW6ePAsZ4AE6yx2IjCQ== X-Received: by 2002:a05:620a:258a:b0:6a7:9479:655e with SMTP id x10-20020a05620a258a00b006a79479655emr5699121qko.681.1657381502269; Sat, 09 Jul 2022 08:45:02 -0700 (PDT) Received: from vultr.guest ([2001:19f0:5:6e4b:5400:4ff:fe10:17bb]) by smtp.gmail.com with ESMTPSA id u14-20020a05620a430e00b006a6a6f148e6sm1682411qko.17.2022.07.09.08.45.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Jul 2022 08:45:01 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, quentin@isovalent.com, roman.gushchin@linux.dev, haoluo@google.com, shakeelb@google.com Cc: bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao , NeilBrown Subject: [PATCH bpf-next v3 1/2] bpf: Make non-preallocated allocation low priority Date: Sat, 9 Jul 2022 15:44:56 +0000 Message-Id: <20220709154457.57379-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220709154457.57379-1-laoar.shao@gmail.com> References: <20220709154457.57379-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657381503; a=rsa-sha256; cv=none; b=swduSrfkpUh1ycGQcpXQIderD9YFip/kLApyPME+3YNBjK5cqIMolBSdnl3jKJIqvo2wop ztEKimbwo2DOi2DkGd5sJoYB/ILcpJZsSAkaF6VQ9e+3mquyYpO7LQqFIlro3lmEqTgewK pXsK5aMPHCBJ/kiNZTFKrll1G8n7pNo= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q7yw0wFI; spf=pass (imf09.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657381503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bkLct7+3x09q3TPrw9hr1g7AQwWmVQubLfPgu28Dhr0=; b=fksblw0M9OWTAdpN0peE9JoPlxoRzU4lcVt+o78cedDlkkrRY6uTIwVpC/3dIsdJg1ZW/4 7XqxCWTGlxx3+Js8PZ6wPGE8I4NPO1YsWqd5+cvq0KgbFR6qIhhesIh6v2+Hx81mcC1/gc j4LbUxESQa/hABy9/GQDqBj2+QsqaK4= X-Stat-Signature: pbs7yrsqqppzr1khbu1y7s1813hs9owy X-Rspamd-Queue-Id: ECFF214006A Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q7yw0wFI; spf=pass (imf09.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1657381502-301267 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: GFP_ATOMIC doesn't cooperate well with memcg pressure so far, especially if we allocate too much GFP_ATOMIC memory. For example, when we set the memcg limit to limit a non-preallocated bpf memory, the GFP_ATOMIC can easily break the memcg limit by force charge. So it is very dangerous to use GFP_ATOMIC in non-preallocated case. One way to make it safe is to remove __GFP_HIGH from GFP_ATOMIC, IOW, use (__GFP_ATOMIC | __GFP_KSWAPD_RECLAIM) instead, then it will be limited if we allocate too much memory. There's a plan to completely remove __GFP_ATOMIC in the mm side[1], so let's use GFP_NOWAIT instead. We introduced BPF_F_NO_PREALLOC is because full map pre-allocation is too memory expensive for some cases. That means removing __GFP_HIGH doesn't break the rule of BPF_F_NO_PREALLOC, but has the same goal with it-avoiding issues caused by too much memory. So let's remove it. This fix can also apply to other run-time allocations, for example, the allocation in lpm trie, local storage and devmap. So let fix it consistently over the bpf code It also fixes a typo in the comment. [1]. https://lore.kernel.org/linux-mm/163712397076.13692.4727608274002939094@noble.neil.brown.name/ Cc: Roman Gushchin Cc: Shakeel Butt Cc: NeilBrown Signed-off-by: Yafang Shao Reviewed-by: Shakeel Butt --- kernel/bpf/devmap.c | 2 +- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index c2867068e5bd..1400561efb15 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -845,7 +845,7 @@ static struct bpf_dtab_netdev *__dev_map_alloc_node(struct net *net, struct bpf_dtab_netdev *dev; dev = bpf_map_kmalloc_node(&dtab->map, sizeof(*dev), - GFP_ATOMIC | __GFP_NOWARN, + GFP_NOWAIT | __GFP_NOWARN, dtab->map.numa_node); if (!dev) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 17fb69c0e0dc..da7578426a46 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -61,7 +61,7 @@ * * As regular device interrupt handlers and soft interrupts are forced into * thread context, the existing code which does - * spin_lock*(); alloc(GPF_ATOMIC); spin_unlock*(); + * spin_lock*(); alloc(GFP_ATOMIC); spin_unlock*(); * just works. * * In theory the BPF locks could be converted to regular spinlocks as well, @@ -978,7 +978,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, goto dec_count; } l_new = bpf_map_kmalloc_node(&htab->map, htab->elem_size, - GFP_ATOMIC | __GFP_NOWARN, + GFP_NOWAIT | __GFP_NOWARN, htab->map.numa_node); if (!l_new) { l_new = ERR_PTR(-ENOMEM); @@ -996,7 +996,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, } else { /* alloc_percpu zero-fills */ pptr = bpf_map_alloc_percpu(&htab->map, size, 8, - GFP_ATOMIC | __GFP_NOWARN); + GFP_NOWAIT | __GFP_NOWARN); if (!pptr) { kfree(l_new); l_new = ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 8654fc97f5fe..49ef0ce040c7 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -165,7 +165,7 @@ static int cgroup_storage_update_elem(struct bpf_map *map, void *key, } new = bpf_map_kmalloc_node(map, struct_size(new, data, map->value_size), - __GFP_ZERO | GFP_ATOMIC | __GFP_NOWARN, + __GFP_ZERO | GFP_NOWAIT | __GFP_NOWARN, map->numa_node); if (!new) return -ENOMEM; diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index f0d05a3cc4b9..d789e3b831ad 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -285,7 +285,7 @@ static struct lpm_trie_node *lpm_trie_node_alloc(const struct lpm_trie *trie, if (value) size += trie->map.value_size; - node = bpf_map_kmalloc_node(&trie->map, size, GFP_ATOMIC | __GFP_NOWARN, + node = bpf_map_kmalloc_node(&trie->map, size, GFP_NOWAIT | __GFP_NOWARN, trie->map.numa_node); if (!node) return NULL;