From patchwork Fri Feb 10 15:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13135957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38AAFC636D4 for ; Fri, 10 Feb 2023 15:47:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE9C16B0151; Fri, 10 Feb 2023 10:47:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B72DE6B0153; Fri, 10 Feb 2023 10:47:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EC5C6B0154; Fri, 10 Feb 2023 10:47:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 892446B0151 for ; Fri, 10 Feb 2023 10:47:55 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 54B77121077 for ; Fri, 10 Feb 2023 15:47:55 +0000 (UTC) X-FDA: 80451812910.28.C8BCA33 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf15.hostedemail.com (Postfix) with ESMTP id 5FAF9A0018 for ; Fri, 10 Feb 2023 15:47:52 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="BQ/iZFen"; spf=pass (imf15.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676044072; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VfmMbVSSF03x8/EU0op7vkHjUSmnGjVyyVeSw2o/bMM=; b=RKJwYhtO/rFs7PIxSAKog7aSlWK8+G741v+BS+AjjuqvyGtItp1jEH7Yf8xXQGOtOJEomg NVMMdMlIr7Cw4gW22b/czv7vSVnoglcBTCb1OC127PUA9Is8yvZnYKnAmcjoRnIh8oL0HK /PLqDRaDlAudz1Avtv/UcGTHF/GHBD8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="BQ/iZFen"; spf=pass (imf15.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676044072; a=rsa-sha256; cv=none; b=yUGLR5RcZatSWa7Z11F7fUoPVGy7pXO9H+CQUR8a6A5lZ2D5BY8N1fFdlLDug1tSptH0wb 4eVK6rp3ewFEVPZ0hRynJxun8F/SwyU99HYkHKuHGLr7oNyAnEiHGWqq7tNBaRN6Q+G5KO gmu6/zX6A65Hjp5CtdPKjbs3sKkUDL4= Received: by mail-pg1-f181.google.com with SMTP id 24so3973140pgt.7 for ; Fri, 10 Feb 2023 07:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VfmMbVSSF03x8/EU0op7vkHjUSmnGjVyyVeSw2o/bMM=; b=BQ/iZFengvaZKJ3dJwASQiyVaev+KFw4y51P7WZw4NwSnsRQG2uZNOGB3zBFW1QK0V mQx+3kvpRR5HnwelfEksbJwXc2u9EgEayhBNqtt+GwltUw9XeQz8hIUcZfpmzKK9sCdf kl/LTV3UiLSa3CvLJyeV5nTs30q6wPG1V2oyvHiZgKgYJjZl4Gf5skDU/Ey3PEt0xNYN cwaGROZ0gNhbe5wJ6q1scjleFZojcU35EPTGZHAF5Dwfb0knbSHm0dBe58k63hBXTLbY 5MsFuOdqnsx4YIOCLtjhZIicCUH2vtVHUl94uuX3LAX6uFXzti3HsqZA3o0YJuE1FX7o R01g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VfmMbVSSF03x8/EU0op7vkHjUSmnGjVyyVeSw2o/bMM=; b=H3xu9ODpgcjHAMj1FVBT5yXma8/LU4NLxWeVoKMpMJxd1jnLJwuNuDm7LjugylfScF U/9J+NAJzwwndcYAsKMW2ndYZdpuy0Isp5haEkPvnGEUakRRzHgI6uBbUBYe7xgUNJxp vVllAvRs9WdCdLfgAzPnNjYpgQ7L4LX52D5vbER3Cj+78hZB6lzgV89PO/C0G52q9P16 fJsos87o0hhWLP5ppU6dkpMCziHyEwDpJEh/MNnMVh2rMGZkaldADzvcEQaytbi7zAVH zeOiihe6aBtOmZlmc+N2WhwHt8CMltL/p6d3+WHfVKuopONXrTJC6KaYt24p6YWEVGQA CO1A== X-Gm-Message-State: AO0yUKXxoADQgHoxws9EP4I/IvwGLEYCxo1R4Jq/bpuJx/CBZPVSRj8b w7sYRD35t+hHWxQPB3PTFYE= X-Google-Smtp-Source: AK7set8zE9YvB+rlMwZ5YZqUvOjqk39b6oN+vCogBTsbbmo4FDAGcstJKaNToWrU7wwqvzFDSnO41w== X-Received: by 2002:a62:17cc:0:b0:5a7:9e27:faa2 with SMTP id 195-20020a6217cc000000b005a79e27faa2mr5314359pfx.4.1676044071214; Fri, 10 Feb 2023 07:47:51 -0800 (PST) Received: from vultr.guest ([2001:19f0:7001:2f6a:5400:4ff:fe4c:e050]) by smtp.gmail.com with ESMTPSA id t20-20020aa79394000000b005921c46cbadsm3520069pfe.99.2023.02.10.07.47.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 07:47:50 -0800 (PST) From: Yafang Shao To: tj@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 1/4] mm: memcontrol: add new kernel parameter cgroup.memory=nobpf Date: Fri, 10 Feb 2023 15:47:31 +0000 Message-Id: <20230210154734.4416-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230210154734.4416-1-laoar.shao@gmail.com> References: <20230210154734.4416-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5FAF9A0018 X-Stat-Signature: sstm6xnt6mdibiu651pedz56urhb8htg X-HE-Tag: 1676044072-599289 X-HE-Meta: U2FsdGVkX1+scT4Qm/45ZQOBfsCi+f5B2XbQB9xQ6pntDbvuKgfZ0heXjEcKsP43xizjtLq2BeHK5K5bc9CCjaLeG5xElPyF84B0u+TT6rcnEYhbtW1ZmHlB0PU6N5OeIPtu4VuIac2tX2GdWYx/OES9yjzs+t28BzWUl6oipUQ5R7GvGJca83ob2b60eCSIgexT1BJ/BQZky2ro4M2SduLgHVryqQQwNp5xllYNoh1vFFsJcgxD1DYnLPO9/xIONRmf30K9jQmGYIHTniRbn2bnsW6DIPyzE9XlltzAuJxBKu8ko5NxsuFfllqIws/iJg8Gr+yReQYTjll6hUZ5wmyVu8sslXFJyiyBxnNn5AxqiSrwHUjaOUAubt5usIgByOt5yyx9WZzksrw5zgbiFV6BcGRZmsBfErODeHF8CkVvZ7UkGqJ5up6Sjvql3rnoKQ5y3Eebun0eEE6hl4z/kaOQ82XO/6IMAkWDAzbGd4WaxGigpiEtkG7h2oZ1Id5R9XNd5FXXFKgZYsQJCbGXWRdUoD+Y4haGa58a7UAi0ygSsLOclXOR7N50Ie0EqKklRq9RmYi6KHtDP1bp3SNQPZ8XZJwmMqzZti+qITmeCfE8zFoMACVahPiMXOjwNwtlXzgVAh4nQFigVnil3wbuo1hIQHJSyEgw5yNiKLC2BjsRDTs5FEnQkyG+YubjcDa6zZ0mB7XMDSZbDRizhMDLkXLxXFqyJVgJeA7qJyU3luI5dkvHpjJvpvc6jZHu+I92yspqBdQZU2nNvVeR6virmbYpMc7tcyQzKhGpq9Fr/wDP9+Gd+D75qMMatnqUFzsisiXZwRaMFrX84QgT6wUMnZzbb5CiRmhWLdQHsDS/QXaCNhonJPKxY6NnwejYIJ6alHB+V9NDRiexT94fC2WOCIyQwISoiNogB+zmj9BofEsE6jaKBrojI/bH5dozQfxKBnHlgVDmyUMDg/YaztP xnH2O/D/ zsZWk/I6ofXb1d1+fdmc6dFEtDOlMOAIKmzuVzWxhw4kvXvqDdjKElDI72GQjfKo1z4CkPdEcGS6d7WUa1lnGmZ4PSgIe9Sj6n6D5qzvxFVThrTz3uCqLEO5oeJ+wceeWy7GcduhF7RiYjo8PXbEMX8FTxqocpky7g5wW/LWxkRKQIn6nGSwbQPY8ljLcNe6q/Xu5I/jVtmkN7G0I/P/zghjItlJNJUE1csxb4uV4Ll9NjZaOKKH+8M7KwF5sakwK7gjxrW6FRDVXiLXWP0j4fdmdQRwBhrDz27LRd2u7wS5VjvFP9KaXV/aR4hjO3T/+IxBXXcSEp3YMUFtRAlkxBkp+mAKTliGBitUmDvYmyUqxcw6IA6Bam02ODoH6oJznfL00dG52ikd0gszERU7hTyI4oOZdFLQunwjcVH1jrXE4eXmh79u3OzREEx0K0xXUjBirPeb0SoLBz9kwKOEB8tkyPUrAgIdpaJwKf4lmatH7MMxby4N+D9QLtuIJEqIrHeaiR6iIpOrsLWi6ivGmsT/o66f+JkrsgC+LjHDl3K2xkhw8Fip2KB58GY79JNXxrE9xr8NnG/kR29/QEbnvhp0Epe5/7ZbkECL7Sp0GhZpd+gA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add new kernel parameter cgroup.memory=nobpf to allow user disable bpf memory accounting. This is a preparation for the followup patch. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- Documentation/admin-guide/kernel-parameters.txt | 1 + include/linux/memcontrol.h | 11 +++++++++++ mm/memcontrol.c | 18 ++++++++++++++++++ 3 files changed, 30 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 6cfa6e3..29fb41e 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -557,6 +557,7 @@ Format: nosocket -- Disable socket memory accounting. nokmem -- Disable kernel memory accounting. + nobpf -- Disable BPF memory accounting. checkreqprot= [SELINUX] Set initial checkreqprot flag value. Format: { "0" | "1" } diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d3c8203..26d4bf2 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1751,6 +1751,12 @@ static inline void set_shrinker_bit(struct mem_cgroup *memcg, int obj_cgroup_charge(struct obj_cgroup *objcg, gfp_t gfp, size_t size); void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size); +extern struct static_key_false memcg_bpf_enabled_key; +static inline bool memcg_bpf_enabled(void) +{ + return static_branch_likely(&memcg_bpf_enabled_key); +} + extern struct static_key_false memcg_kmem_enabled_key; static inline bool memcg_kmem_enabled(void) @@ -1829,6 +1835,11 @@ static inline struct obj_cgroup *get_obj_cgroup_from_page(struct page *page) return NULL; } +static inline bool memcg_bpf_enabled(void) +{ + return false; +} + static inline bool memcg_kmem_enabled(void) { return false; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ab457f0..590526f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -89,6 +89,9 @@ /* Kernel memory accounting disabled? */ static bool cgroup_memory_nokmem __ro_after_init; +/* BPF memory accounting disabled? */ +static bool cgroup_memory_nobpf __ro_after_init; + #ifdef CONFIG_CGROUP_WRITEBACK static DECLARE_WAIT_QUEUE_HEAD(memcg_cgwb_frn_waitq); #endif @@ -348,6 +351,9 @@ static void memcg_reparent_objcgs(struct mem_cgroup *memcg, */ DEFINE_STATIC_KEY_FALSE(memcg_kmem_enabled_key); EXPORT_SYMBOL(memcg_kmem_enabled_key); + +DEFINE_STATIC_KEY_FALSE(memcg_bpf_enabled_key); +EXPORT_SYMBOL(memcg_bpf_enabled_key); #endif /** @@ -5362,6 +5368,11 @@ static struct mem_cgroup *mem_cgroup_alloc(void) if (cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_nosocket) static_branch_inc(&memcg_sockets_enabled_key); +#if defined(CONFIG_MEMCG_KMEM) + if (!cgroup_memory_nobpf) + static_branch_inc(&memcg_bpf_enabled_key); +#endif + return &memcg->css; } @@ -5446,6 +5457,11 @@ static void mem_cgroup_css_free(struct cgroup_subsys_state *css) if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->tcpmem_active) static_branch_dec(&memcg_sockets_enabled_key); +#if defined(CONFIG_MEMCG_KMEM) + if (!cgroup_memory_nobpf) + static_branch_dec(&memcg_bpf_enabled_key); +#endif + vmpressure_cleanup(&memcg->vmpressure); cancel_work_sync(&memcg->high_work); mem_cgroup_remove_from_trees(memcg); @@ -7310,6 +7326,8 @@ static int __init cgroup_memory(char *s) cgroup_memory_nosocket = true; if (!strcmp(token, "nokmem")) cgroup_memory_nokmem = true; + if (!strcmp(token, "nobpf")) + cgroup_memory_nobpf = true; } return 1; } From patchwork Fri Feb 10 15:47:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13135958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13404C05027 for ; Fri, 10 Feb 2023 15:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CAB36B0153; Fri, 10 Feb 2023 10:47:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 753F46B0155; Fri, 10 Feb 2023 10:47:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CD516B0156; Fri, 10 Feb 2023 10:47:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 44FCE6B0153 for ; Fri, 10 Feb 2023 10:47:59 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C2384403CE for ; Fri, 10 Feb 2023 15:47:58 +0000 (UTC) X-FDA: 80451813036.27.4EE647A Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf23.hostedemail.com (Postfix) with ESMTP id DC51B140022 for ; Fri, 10 Feb 2023 15:47:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oZ0tIoXB; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676044076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=cCI2Zvjq1ZcLtvZk1jdKQxiixcpxn3E1JdgY2H8peaVelP3xd+sEfpjVC+JjNXHCbb2xaj fK/lX+G5iTOH7C/oQO/qg1Im+wQvuzjnbwVW7RouqB2CudOXkp1lgB54QhmI98N1ohuTWE tddGLihQx7QE/cJSD6GjIxJI5rB29a8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oZ0tIoXB; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676044076; a=rsa-sha256; cv=none; b=BQZh/mWg3bnFg3kQTzcq9mMkDllkjQ5cjUxcFYA0yivsjiR8BS4MLyTcEH50knyz0Qj9en petOQU5M6go2reHIosHkmOMUIBqunXBtnbXgMIRCOKYtEgjvLO8NsG0ByVVlj8BzKElHGT ITi+0ArZyKbjGow99xg2FqOUliuBzoY= Received: by mail-pf1-f181.google.com with SMTP id ay1so3719386pfb.7 for ; Fri, 10 Feb 2023 07:47:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=oZ0tIoXBZYNSfCI21Hayv+qmA7VSMOoMMOB6Lb5LFDSNKWa5ZA1OvywGXFbCWh8rAn fveeyllUwSACZz2OlzAqiEo59qwmLf7GZhgB7W3iSPCLWMmsW6NA4ZnSXreyGxulD3/y 9BMmX3XJwetCzbqJ0iYfMiIavh+R1uAh+K3hErbzqpOKmoIYaCalasRZ8UQ2h7L6Ne2i nkGZnjmlwKEJDDtCSqQOZJFTk3tODlmIZL5QpGwW6jhKIrJzTnsshBKamVpjVcGittNQ 6qyTM5pH4ToQwYnaUukshYiUudfoiKYDUGugo9i8aciEhz/+nkNDHKspgmXIQmp4xwow ncQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i3DzMoNfpXAhki9P1E058MwckxQ0M07ZDtVShvGycKA=; b=zXMkRLYm9IAai8auZfzVOamBMOMav9OQ+CzHvahSvoAvsU+TSt4T1AbPn3Z/406G91 HFre0Y0KSEFWKnr7682kSR1xsR/+xQYXL/awM7fBLTou9XAaaM+RkFFZusSrDFyR1dZ7 yyl14XGfvBS0F1m7kqQuz4Q51SE+cCINljWsF7M2z/Cvd8y8zJ4af/rhh9XMpnv8rCaQ egGyExW/2BxaDPIiTDIsdtIamd43MbMstQU7Nctbqm06LpWZ1VGVonFSrHlrX6pOSkbd ck/pDXvRde/mI3VQqwfuiTjCIrNhC1uZizFSZJ0qFMjoDogGip9tizbV1fM3f+byLQSz J/QA== X-Gm-Message-State: AO0yUKWWV/29i97PMIqW9zhdyvn3bGPyxyw8DHQMN4vk6ITNcKAA/7WX pMAfDjSHpSNXsqx+LwnpKJg= X-Google-Smtp-Source: AK7set/wDZlxvee4YGZQEWH6VZZuQhWxaDntwzaB44hF8rNeick+jUa8cfDxbgwrQy6oPT1YaThwYg== X-Received: by 2002:a62:380b:0:b0:5a8:58b5:bfa9 with SMTP id f11-20020a62380b000000b005a858b5bfa9mr5165943pfa.12.1676044075752; Fri, 10 Feb 2023 07:47:55 -0800 (PST) Received: from vultr.guest ([2001:19f0:7001:2f6a:5400:4ff:fe4c:e050]) by smtp.gmail.com with ESMTPSA id t20-20020aa79394000000b005921c46cbadsm3520069pfe.99.2023.02.10.07.47.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 07:47:55 -0800 (PST) From: Yafang Shao To: tj@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 2/4] bpf: use bpf_map_kvcalloc in bpf_local_storage Date: Fri, 10 Feb 2023 15:47:32 +0000 Message-Id: <20230210154734.4416-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230210154734.4416-1-laoar.shao@gmail.com> References: <20230210154734.4416-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: gxq6bxee7njif1oa8ffu1pjyjkncnn13 X-Rspam-User: X-Rspamd-Queue-Id: DC51B140022 X-Rspamd-Server: rspam06 X-HE-Tag: 1676044076-319532 X-HE-Meta: U2FsdGVkX182fOQuTZTZhrn53m4nzYcb0nfHLJBeXXfLBBBo7EwlBROh9bLZhZCCFL4Q/PuyxQ2bNdV2L1/UKkXTb5c0Q/R6xyWT4oTleqEk/vNNJFQDIsHzHBn3mRzj4h7K20+l0vFXrvCxC2ptXyv9rCJIXK9oxHBzUtkGqh5sfeTYCE07FkVUd+TiQkAUxH6hxdmctaGOAkt9cU0dWeyj/cfP9/ddhWlIJ2i+xJDKobQZhsXQF4XdkBLJ7wcjZeAMsVBtC9wwFND/pwKBKLthR0f0J8hMrS7huuiUA9inqneKqHswQA96P4vR05PVZE1cKr8U1DA2wquaShF0uoXnJqR1Juhs7MeFnxAvxxFsuKol9OpMKKjkc/jwq3C3Cehz1YygLMcpMI2r6w0tDAWAlcwEuMNuAdJhALdTlQguXNyvrejGeL2SI4rAR8rKRiFjiNuIQXhxqyYgExC0vBxQWJQV1mW4tCmK8KfnX7p/REQA6Q7bJj9sdDNFtOBpiS4w8sDyiXsJM5lOdyNFGa8fsQhSZrUU+lhpOBdAbZLkBHbNdkmbiiNTgBMeVn9oA6KEeWnnw9DwBbOL/3Ab0CcCJADDEZCrgQyl+gQamUCmIhytKKbKwYGEIlhuUEWYpQV4Kt/Ad14cjXm4BGkLZ/Dt5xJTpPqpujENb2hBbYpLaS+oViVQOxhN5toyFxGWV+AZvPbdNcSX3n6e2NYuFdbYP0q2AgFMXzw1nNOzD9l6oYnoCk8IOQjvBC/gNIBATzUgytYDY5YH9nh+NQyEqj/yBoukYmnjOgjD+Tbqd6Kc6feHwAb+oZWhVQl7pzwb0XYnIDxG+7S8RhmYHwHdLRBzTlKjft5MdYzNG8YiRBZ8BrfjkU2Xagp7LLd+2E61zJevlC/wbPnCBq1mirjt2hMkO5oZhP3hx4TLBQBTia+Tf92t+EY21iZ/RDcgNi/eq3O9IsuEpFaaoGrKYqk GwRgq66q bMqa9dO0WWVve7tOGNq1IKHHDnBXaoBmOBIM5JoiSAJS9/r5+E788krV6XrF+BPOIRzGFBLpEeerE+NxNDJE8sCEJVlnlx+/jKtBG8603m6vA+lbRWUY4EaGqeiJnJPRRkimdrlrJ804TUYX5N5o79gZ+kPoJSFTD+mFVu6tk2yriFqTUk//lwHZa9exj8kb5U5jXoHh9e1a/gRfkV6TS7q8vpaUzx6i/MfVRrklkCSNi19YnOTEIxGdQDYHxjxGaiI6kzOS3/i68iv7NAEaytetMFsrRCkET7vedQTwnea+yO2SpxyjqvE7FXUmI4pjHlevI2+b+CuHwADnd2iKsTAI1gM+eatHbysUJ0L330yKBothxx23LxUOUAzLRAWKol4ZxisVl4lBD/rK5TZxHA4XCKS+2QwZsqf+Q8y4CTwFcQVR3lsTd0qH918nchj0gdIKDZiqTG3r33x1TheYfqHOsuwsdfgEfAk17PSvT46J3tR9b4nhQnhdwfbW3LMDQYbHowwdKERPK1aLv182bwre6f87VCVQl8eZA/5M8VbNNu9JJKT+NLgwAJTZdhQn+yuVLpbQthR6R7nvpI7PNHcB6ft/dAYsCiPJifYb0Bshg0H1Z+fbWLHVHiyklquPxwGyI8IJEWUW6N+CgNigifBvZgA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper bpf_map_kvcalloc() for the memory allocation in bpf_local_storage(). Then the allocation will charge the memory from the map instead of from current, though currently they are the same thing as it is only used in map creation path now. By charging map's memory into the memcg from the map, it will be more clear. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- include/linux/bpf.h | 8 ++++++++ kernel/bpf/bpf_local_storage.c | 4 ++-- kernel/bpf/syscall.c | 15 +++++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 35c18a9..fe0bf48 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1886,6 +1886,8 @@ int generic_map_delete_batch(struct bpf_map *map, void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node); void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags); +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags); void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags); #else @@ -1902,6 +1904,12 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return kzalloc(size, flags); } +static inline void * +bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, gfp_t flags) +{ + return kvcalloc(n, size, flags); +} + static inline void __percpu * bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 373c3c2..35f4138 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -568,8 +568,8 @@ static struct bpf_local_storage_map *__bpf_local_storage_map_alloc(union bpf_att nbuckets = max_t(u32, 2, nbuckets); smap->bucket_log = ilog2(nbuckets); - smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap->buckets = bpf_map_kvcalloc(&smap->map, sizeof(*smap->buckets), + nbuckets, GFP_USER | __GFP_NOWARN); if (!smap->buckets) { bpf_map_area_free(smap); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index bcc9761..9d94a35 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -464,6 +464,21 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) return ptr; } +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags) +{ + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = kvcalloc(n, size, flags | __GFP_ACCOUNT); + set_active_memcg(old_memcg); + mem_cgroup_put(memcg); + + return ptr; +} + void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) { From patchwork Fri Feb 10 15:47:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13135959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3D8BC636D7 for ; Fri, 10 Feb 2023 15:48:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 902EE6B0155; Fri, 10 Feb 2023 10:48:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 88C6C6B0157; Fri, 10 Feb 2023 10:48:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7062B6B0158; Fri, 10 Feb 2023 10:48:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5DB086B0155 for ; Fri, 10 Feb 2023 10:48:03 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 379F2C0247 for ; Fri, 10 Feb 2023 15:48:03 +0000 (UTC) X-FDA: 80451813246.25.375CBAD Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf04.hostedemail.com (Postfix) with ESMTP id 609EB4000C for ; Fri, 10 Feb 2023 15:48:01 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Jn96c5OS; spf=pass (imf04.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676044081; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KSCoSnxCH+oujuHlV2ueRvXQv9wOk6fb9r3EfXFy+Xw=; b=QcyDcIHWHFqoxC/pMc258Xl1iOEeMCA5+N5t0olIhV0HQoIlpCweNzKhVQ38pgpK5WnelS tdAl10kPdJGrDRGUKG2hJeAlKaolvR3RavFHk+xfC3uG2jqEo5cLO+C17hy2DgMG1umC5d a5PxYl8IIEMSoQXd0hAJo8xx0HKFeTA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Jn96c5OS; spf=pass (imf04.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676044081; a=rsa-sha256; cv=none; b=Np6Gaijw2hWcz5LLH7WatwGKJ+NkuSq3uffkQz/yzuIqLew4TOhAGw9eKbgC9fc+IRZAjZ Tzq9TIW7RJuleweSOqcfOT1VyuzRKBiD0IDaNW/j4A3+5Pb9DJTuhCbJXdMx0QqOxDr7qf qvn9QqNX+QwXnH/Sb5E3r/t20LL6O5U= Received: by mail-pf1-f171.google.com with SMTP id g9so3721738pfo.5 for ; Fri, 10 Feb 2023 07:48:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KSCoSnxCH+oujuHlV2ueRvXQv9wOk6fb9r3EfXFy+Xw=; b=Jn96c5OSkEZ79jtvvUAgxCfdeK/vT5MXjAismyeWwqGOzgWUC7tOQxF5/wxGDKBd1+ NQDqmadFBtS46XsKBvPQanuRJ4CwVD32afSccI431oQcKgoPI4yakO5ePhv3luG5q4Kq G1OmhSBaxjhnTpnCLISjIYzOyY4hc8swEO+5XOnMEsBhV8Ya3KN0EuX4XEAqdxIK1Qux u/rEUAhudYR05Nnyx77XVHNXBp+2OJQuOwOWJb7EYssyfW/YRSbb/a/Nj8C/Epr16s8S l5ToX2SA9BGj3aEmQ8g2629uZDbs5iDhJl7GvAh40gqex9vmEEaTyV1bl6WUkifgR+Ui sbUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KSCoSnxCH+oujuHlV2ueRvXQv9wOk6fb9r3EfXFy+Xw=; b=I6OdMfByeblFWPj4OEH6cFTfkflxBfY6FvcZBxvVu563J46TXVCOcPDokH0WM9re+4 hCMdDVS0gpCOFR2E/S/N9ruh9plbxjJ9p2h0UFHhoWoyDW7eDLzF52YnK0Jce7Hv1nfr YNlkeHvw6YtxaC3bmIeFPPBS3KiaITbGcStRNoH5WIRaKpDRvmznLgqV3cUZ3YBdOgb6 nktub8Ea+1QMgWe1i4FPJaCJ0Fl67EHb9eFguNwQWPpOWk+pbO2PVm2SEDkFzbpnKb6c CnkDQi1OOibom+qsPNi1Bbb1VLVX0kE5c86ZDtsdZ/LFbq+9M3hJpn+02Kv09sDdxFFb InsQ== X-Gm-Message-State: AO0yUKUN/BP7LZk+hdo/lczme3JX+SBNxk9fnvfip8SJlFRx+n3fWPDr DDx/CDGcETvnOOt/fYtqoRk= X-Google-Smtp-Source: AK7set+0d0W5Q0AkiHpDwGMIBAsDbfXJdRCxZFFceps6/hPlTFfDvxf+o5NgKqtSmcIekhrFGRYC2A== X-Received: by 2002:a62:8490:0:b0:5a8:4b27:5db1 with SMTP id k138-20020a628490000000b005a84b275db1mr8495229pfd.3.1676044080310; Fri, 10 Feb 2023 07:48:00 -0800 (PST) Received: from vultr.guest ([2001:19f0:7001:2f6a:5400:4ff:fe4c:e050]) by smtp.gmail.com with ESMTPSA id t20-20020aa79394000000b005921c46cbadsm3520069pfe.99.2023.02.10.07.47.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 07:47:59 -0800 (PST) From: Yafang Shao To: tj@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 3/4] bpf: allow to disable bpf map memory accounting Date: Fri, 10 Feb 2023 15:47:33 +0000 Message-Id: <20230210154734.4416-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230210154734.4416-1-laoar.shao@gmail.com> References: <20230210154734.4416-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 609EB4000C X-Stat-Signature: 8nbhmsnp31oi4q88i4jw5fonqzmts7mc X-HE-Tag: 1676044081-358307 X-HE-Meta: U2FsdGVkX1+rvhasTJRpZ7B5O0rJq8eJQ0FAO8K4dVG/eVx8B2E795BUF42bgn5BYZWi6vi1029sZ+dgHEu9+cBKf3zxktsXR7g/6yO5HNOyceXpj5Usan0tWvKWMBnDAR8XfAytBIUi9CUTRHM5Kh3XxeGoYLwad+RW5EgUf6rnvKQfU9J35C1vpUmuumIagMPuWPz2pnxDh4W0uRC7r3lchRLAZO8waIAf+RLWloT0MlKjgm6tilDCEJYLjghrPn6F0ovUp3/I7PZpk5UXCK7Jacml/6/fmY4YfyTD2ZIZWFOD7f/ozlYAZdTPVEdZmo2+dhlpKYqNMl+5y2Qs8SPmoRBP59NEkjE3LkGtRzCbvfY+mOq0KmEgvSAtRO3FhKhY1O6CF2/f3D8RhukLcR62PuFSNAubVE23TM3zCLmd6C4JWg2eLjrViKU0PtwwWoZXOv8vAYacxTHKLbeWgP3kRzzOxpfBzRKpp1U8HeqoV1ghRH4h4c9/yzO26PgMRXbGtALlvdHE/hdu136bM9G5qJyLmC2MtcoWO0ARWsTbe02rfDqTWMrJfqVBLJq+xMG7jKFfv0c0NFRw8mid9SesQxyxJSH25ysEvjG1mSK7Xkye9SPoK0XWLN+LwaADsA6cBRJdtY98aMC3yBHTBTLQvoacjJMKmygZF848cnKg0358f1HmBpVlopDLGb22ST/1l2YCiwid3JPqv8aPn4RjX7OOoiI9TifWidt7dZuPnSO7HKETsmdQiEK9GT08jygQifI5P6CDNsxS2dHgrQERhMvEHO2Tx/0JgRChx4aARjHbT6c6PVPQP0ov79buymVu90VXncV4tqIBUa/bWohGHEIEiPD6DLK4SkFl+Slp4mpaQpYOpGjRl+1O369pt84rsPrKdkUkeGl5hXcgOXXNeoy9io/7OibF2iRwhEQFDsQsYh9so6nCTR857kNAuD9TXku0PeU1Etm+ZXN WoSDR0cs qO00gYFZS8Lg7zlviXiHGfUet7OYKfmjc6RKyaKpqqxC+4jMXCI0TmJt2+eyeIQnz45vfbLGXZa1XvPDmyN7remO0MavP5efxd9om7uUxNyFStnqq/77dMWIi7zPSJe6s5q/z/mW8KDmC9rwOFSRziYCGuQPeseBnw2UhvyZBJQzZP+aUqIwmT9kZwUM1rlKWo4oWjIHHtU8cdPuqT251od4kwa8W6wvizEyy135TwX8EIcyJSiTh08oFMhR9KGk1z+r+NogQ0EMLOakfVImZbnEUxYD62hqiJ+9C6usACANRGCbIWeayOB9S7EPaAzpW7zfaDiBgTNNmWK9dyBZECHBQ4eqRNCvsqhCWS5psEyP7LlDeRJEngPhuzg31WquuKxdKcwvzGrL5hgmiFyeTKNzCkXne2JYcHnqqQoDVHTpjFmcKEQLUQ8xSZsdavS8WHUaqxXWaX2F4zfWN1xdnemlbprMraGaEphOkfoDj3WLSrmoau9JwH63aTWti40DKvbG0AoD/clR/Yu6UH9K+iZxqsRPEv26leQYUoZ35bUApEOV0VLA9aVutcNhSZFrD0568no/pCSD0sPGeMwkmCwSBbxXmnDSffHhuT7xUmCOccu0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can simply set root memcg as the map's memcg to disable bpf memory accounting. bpf_map_area_alloc is a little special as it gets the memcg from current rather than from the map, so we need to disable GFP_ACCOUNT specifically for it. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- include/linux/bpf.h | 8 ++++++++ kernel/bpf/memalloc.c | 3 ++- kernel/bpf/syscall.c | 5 +++-- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index fe0bf48..4385418 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -28,6 +28,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -2933,4 +2934,11 @@ static inline bool type_is_alloc(u32 type) return type & MEM_ALLOC; } +static inline gfp_t bpf_memcg_flags(gfp_t flags) +{ + if (memcg_bpf_enabled()) + return flags | __GFP_ACCOUNT; + return flags; +} + #endif /* _LINUX_BPF_H */ diff --git a/kernel/bpf/memalloc.c b/kernel/bpf/memalloc.c index ebcc3dd..6da9051 100644 --- a/kernel/bpf/memalloc.c +++ b/kernel/bpf/memalloc.c @@ -395,7 +395,8 @@ int bpf_mem_alloc_init(struct bpf_mem_alloc *ma, int size, bool percpu) unit_size = size; #ifdef CONFIG_MEMCG_KMEM - objcg = get_obj_cgroup_from_current(); + if (memcg_bpf_enabled()) + objcg = get_obj_cgroup_from_current(); #endif for_each_possible_cpu(cpu) { c = per_cpu_ptr(pc, cpu); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 9d94a35..cda8d00 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -309,7 +309,7 @@ static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) * __GFP_RETRY_MAYFAIL to avoid such situations. */ - const gfp_t gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_ACCOUNT; + gfp_t gfp = bpf_memcg_flags(__GFP_NOWARN | __GFP_ZERO); unsigned int flags = 0; unsigned long align = 1; void *area; @@ -418,7 +418,8 @@ static void bpf_map_save_memcg(struct bpf_map *map) * So we have to check map->objcg for being NULL each time it's * being used. */ - map->objcg = get_obj_cgroup_from_current(); + if (memcg_bpf_enabled()) + map->objcg = get_obj_cgroup_from_current(); } static void bpf_map_release_memcg(struct bpf_map *map) From patchwork Fri Feb 10 15:47:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13135960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5377C636CD for ; Fri, 10 Feb 2023 15:48:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5327B6B0157; Fri, 10 Feb 2023 10:48:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BBBC6B0159; Fri, 10 Feb 2023 10:48:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 335916B015A; Fri, 10 Feb 2023 10:48:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 15F286B0157 for ; Fri, 10 Feb 2023 10:48:08 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C5F8016012C for ; Fri, 10 Feb 2023 15:48:07 +0000 (UTC) X-FDA: 80451813414.08.0D759C0 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by imf07.hostedemail.com (Postfix) with ESMTP id DE9D24000F for ; Fri, 10 Feb 2023 15:48:05 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=j+gxHQfB; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676044085; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3BzE4dHWY04pl7eRXElI/6nAwQrbbF33iW2gxKhD+d4=; b=7JIG2yKaPzdCpg2JkwLAkDQdm6BtAyV9BFg7jt9w31lnc2WlYBbkikyDoi+d1aYDffAIxO Zq2QXyB9fnMkdC7tgdCNPs28stwgaVoDLqLimMc5DCzgvqsAGxZV2MtwllGxERhlu2MqwF ZCIbZIh2neYkf0A9vg12Aken008UD7A= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=j+gxHQfB; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676044085; a=rsa-sha256; cv=none; b=NaTpI9kv4TSgRlmJ8mfqvUG4URmEMyBCpU5lDGxETB5IYaJ79hIwGMPAgxR6q9aEbYIcTt iDI5EhRjfpoGW70I//zfGmrebL17PC8HUWbThTf/b0Po1uOON8uZpEeoZgLk1M9/lhDiaR EaJxEb2n/+3/K2YBciLSxqbx5gJIf7I= Received: by mail-pg1-f172.google.com with SMTP id s8so3952884pgg.11 for ; Fri, 10 Feb 2023 07:48:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3BzE4dHWY04pl7eRXElI/6nAwQrbbF33iW2gxKhD+d4=; b=j+gxHQfBd7YR9DKjd+QCY3jhunM0qE502k1Q1kOFjTYec6K5h0eypt2BdeB7fPrask zyOtXXHaIaHjLTKHKppAk+Iy+jmgbTPFqltPNUJ8CKtzG4mVYDU3DV9d/VNVRXP8ypOf lK6HEtis6SdYcjcUu0PABvv/+/8NvYPV7hnLtRbdKElboXcv350wk00qsAXJKMK3Fy+g XXIjW6cnwBzFFcNKYrzsNYgo5j/TBfrHuKQOy5NrkBMgMceLhjn1k0+oWJKmb0Qoq8Z4 qJX68FnAXTZOjPtdUc3wYrlty43ptBoVJ6kLPOUWuI1ZILmy8c8lp1GSGuApFVrT5irw ISmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3BzE4dHWY04pl7eRXElI/6nAwQrbbF33iW2gxKhD+d4=; b=DfUa0ifjIDI00g+cAVwGbqtxbcWFYqabSrtd1hcxEo/a2rYxSTdjs51zzXK5pTyOHx 1i9yokd7Ok/rhs3RQlrOPKxnzo79DjG8QV4CaAOFTzjeX+9mbNqNRbV3dFOUHiQT5k05 ONYFRyh4yod5k5erc88xMx/YWFROKI8vJrX8un0fNUzYrSydZQ+ll2Kl8A6/tGijD2fW dpKpDnKYcdFzKS077axHUIegVqkyhVHKEus1KRyzRYPLbRFQOgkARIXfvDB2ZflZSgiu I7baymGJkOwLmdQ9lHeCHp9nLMLIKj45aT6tyOwU+5uJceLG+RZWWcnwtnfIyY2TggTg 1fHQ== X-Gm-Message-State: AO0yUKWAYi5JMKMQ5ODvpKTorVu/uNUX2ZMrwx1Bmi4Adiv1IIVua4ez qeS3EOOZaGp5NwMmRYUSal0= X-Google-Smtp-Source: AK7set8W5dANe0k+W3DZarFPjaiIci6Hk8I9WJxnZToYOnQzh2s0R3baoCshnGT3Pp6WNhUtjdz6zw== X-Received: by 2002:a62:6d87:0:b0:5a8:58b5:bfa7 with SMTP id i129-20020a626d87000000b005a858b5bfa7mr4558913pfc.5.1676044084887; Fri, 10 Feb 2023 07:48:04 -0800 (PST) Received: from vultr.guest ([2001:19f0:7001:2f6a:5400:4ff:fe4c:e050]) by smtp.gmail.com with ESMTPSA id t20-20020aa79394000000b005921c46cbadsm3520069pfe.99.2023.02.10.07.48.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Feb 2023 07:48:04 -0800 (PST) From: Yafang Shao To: tj@kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org Cc: bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 4/4] bpf: allow to disable bpf prog memory accounting Date: Fri, 10 Feb 2023 15:47:34 +0000 Message-Id: <20230210154734.4416-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230210154734.4416-1-laoar.shao@gmail.com> References: <20230210154734.4416-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DE9D24000F X-Rspam-User: X-Stat-Signature: mn1dbonmfry3kbsykf61iuo1i44h7hje X-HE-Tag: 1676044085-353437 X-HE-Meta: U2FsdGVkX18Y16Yp2H9eLCUkSlbC+s5PSiObyd3Chlb551RVAi2wkwXwErjHxVSA2njtLzykv0x5LqdQQULRUrNg25WMYmqgNW4qFzj98loFesGa8aG550Mmbji1HzjP0o8jn40Tee2m+7agARppwN3TjRmc0SDUrqHz+Q86QjXvwOOTWXVEzT7twmqM6Up0z20OnkgnZnz7XGcxJirCoAhnjPrzdJwgWxN/3dbpiHJntNwsAr7+SZG6+WYNpFax4YxIKGfOUbyHmPpLSZb7LLe3NjCXrUbwE8+WkG7oqfMZjqS9/wl3CbEuJNf7s7QoUQ5fsGmyFvpBZ7Sdwu1WxfrBgH64K17ke3OHE7O1x82JIFobiCKuWykrnlGV/o8p9NdzKSBNzTa77MjrlXRtNOgOvgOHGnQGfFzuhpTyd4akqPMODPQHIABWwxWQcjhxltXFN/TI8SbxwHmGiylsQ0Rz5Jxo13JJZbauGPkxUi4njSdg30Lo2+WrE2GJbDFnjdx4rZpxRosF9NAVEeocTFApCjXWN5AYBMMNh3mSNKlzd0NezYHBegmV27SMPlwyFNoD8qXAiNCWW7R1TqNIbBc0hcHfssMvFAYRXppiLINqH76agiSHPKsc3vt8maE/HLYSlbSfRN/ZoVu7dpQ79BtDUfFd1X6wkwrWq/8DyP23Tn6RL8uTGxW9C0w9PtlexHpF1zDRaB05415u36H/nixOMsHOG+C7WdKlA25u1p+bcDpUEACrfjpteFrRe+MGmQkT0qn3yyHGhavHuMr3j+rIZyOsxbPOq7HeBTM/Uu0aIY18nad38MP260oK7rhImtXkUCjEUa7gDOXnnpr6li+nEIVnyFeV44J+BgZQUn2fV8/Y9flW81b/NLUj4W/yvnlF+kK+9LBKZWWylpiLxX/jwm/FNE/QiZSHbEqpggzxcNIxyTVkH9N9ulIdGxGetuyeSgK0K+zAgxk74n/ iCQ7Kn16 H66uO0Wc02IyGX3qX5z7p44PhqDOTvuD0o0Uy+06eXDuo9Ve7TXynVxUxejorKFVABvIAJEcMt0mpTQMDO9n1oS5iB+HdAIFSFNMV25cqE15C77NgRqkBOQbCErc9d3FP3qm9aSeuNGdKBun5Dfor2N3WSZ7wXSDNHGqxpYcFozAXaG7dRf9pdbcr5mDD1zekqQhRPH2Tf2F0o/htsYmRJ/rlUmYgll6Dvmc6DW4QZgRVNCYKCKueaNRuQkD1n6z0MLcZARkIFBlsfMnNrsT1LRjStAAinaBwszNYbkEuzYXHZ7WETmX44eqVVc0b1c6FmJ9RMA+NgLgmzrUj6zlsEBSlQGx0d99C+OhRe19V5kNz1xh3CiLTv/DPvfK9grMOScskQq+qGgIWpRG6eyICZ9qq0KBezrg74+4nz2eSALh0ARujntPqJd0yAIS44qOmu+i0hg2NcdP/ZSGZW+b4ebg8pIu6H/YiE2t8CP0YR0B6ARoz2B6rrH06GYp9wKX3chP0omkSTOFHdCCg7tdzEXcWcslYxrnnhp5fKbvOKj90y9lMYxtyTeGqVDPGclHGBbw+Fu+d9m0DXqNKG2xa51YsPimSo070dCoEwzX9UDdEMFs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can simply disable the bpf prog memory accouting by not setting the GFP_ACCOUNT. Signed-off-by: Yafang Shao Acked-by: Johannes Weiner Acked-by: Roman Gushchin --- kernel/bpf/core.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 16da510..3390961 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -87,7 +88,7 @@ void *bpf_internal_load_pointer_neg_helper(const struct sk_buff *skb, int k, uns struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = bpf_memcg_flags(GFP_KERNEL | __GFP_ZERO | gfp_extra_flags); struct bpf_prog_aux *aux; struct bpf_prog *fp; @@ -96,12 +97,12 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag if (fp == NULL) return NULL; - aux = kzalloc(sizeof(*aux), GFP_KERNEL_ACCOUNT | gfp_extra_flags); + aux = kzalloc(sizeof(*aux), bpf_memcg_flags(GFP_KERNEL | gfp_extra_flags)); if (aux == NULL) { vfree(fp); return NULL; } - fp->active = alloc_percpu_gfp(int, GFP_KERNEL_ACCOUNT | gfp_extra_flags); + fp->active = alloc_percpu_gfp(int, bpf_memcg_flags(GFP_KERNEL | gfp_extra_flags)); if (!fp->active) { vfree(fp); kfree(aux); @@ -126,7 +127,7 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag struct bpf_prog *bpf_prog_alloc(unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = bpf_memcg_flags(GFP_KERNEL | __GFP_ZERO | gfp_extra_flags); struct bpf_prog *prog; int cpu; @@ -159,7 +160,7 @@ int bpf_prog_alloc_jited_linfo(struct bpf_prog *prog) prog->aux->jited_linfo = kvcalloc(prog->aux->nr_linfo, sizeof(*prog->aux->jited_linfo), - GFP_KERNEL_ACCOUNT | __GFP_NOWARN); + bpf_memcg_flags(GFP_KERNEL | __GFP_NOWARN)); if (!prog->aux->jited_linfo) return -ENOMEM; @@ -234,7 +235,7 @@ void bpf_prog_fill_jited_linfo(struct bpf_prog *prog, struct bpf_prog *bpf_prog_realloc(struct bpf_prog *fp_old, unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = bpf_memcg_flags(GFP_KERNEL | __GFP_ZERO | gfp_extra_flags); struct bpf_prog *fp; u32 pages;