From patchwork Thu Aug 18 14:31:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12948195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E495C00140 for ; Thu, 18 Aug 2022 23:55:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E6A18D0002; Thu, 18 Aug 2022 19:55:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 696248D0001; Thu, 18 Aug 2022 19:55:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5366B8D0002; Thu, 18 Aug 2022 19:55:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 44BEC8D0001 for ; Thu, 18 Aug 2022 19:55:24 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0FF201C5D35 for ; Thu, 18 Aug 2022 23:55:24 +0000 (UTC) X-FDA: 79814372568.25.1A2086F Received: from mail-oa1-f52.google.com (mail-oa1-f52.google.com [209.85.160.52]) by imf08.hostedemail.com (Postfix) with ESMTP id 9107D160649 for ; Thu, 18 Aug 2022 23:53:29 +0000 (UTC) Received: by mail-oa1-f52.google.com with SMTP id 586e51a60fabf-11c5ee9bf43so3557725fac.5 for ; Thu, 18 Aug 2022 16:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=64R3bxHMvv5+V0T/PDUTVz1Uk0VIs+H8eHbhqMNvmEQ=; b=EX8ou0RTkSODLqaqCgCGl9TJKOPxlpSZvuMeBQIXCinFHIrXw9t+gJjdaOl1mdSXVT FIAy96Vr2dV3mTuOnnHpKPLUiP9iI2Of1ABEgqlQg49YBMVR4yPV9qLCh5X0Fcn9hc+9 cl543oOVxhHqJyYddxl+ATvDR2FjVNUBymnWUayB/yeDtZtpbqdoWXZWgoZnRR5/ceUg l6ajY8CycQHl70bSisAS55nOgS1iLAsM/DWjfN8EiJj9GAYQvLcJsjEZooxZ0KR84cQc iu8CpiidQdVIJI3fOVPgpk6h2wy/3yjI4rkfhJA0KVBw+Y93nYe/m5zCOQRx7Ahtz1Xs jhVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=64R3bxHMvv5+V0T/PDUTVz1Uk0VIs+H8eHbhqMNvmEQ=; b=CS5wSzU1gJulAh/q9DALxiCRPsUDM74RwLudT5Ey68B2FuuMV1EOJDIVWK/RNHTMgE 1xSrxNEbZcSKJk+hxsQX5rUhSGF5mWZpAk3rZxpL1dg0gWyOgA6/HLANc6+XfLtTPGmr 1CjISslB1pf0s0lj7rPd0if+3TWYNxhNXorR6SKnZ7tZbpN/iYSZJcRRbcWqQkCDXwYP mkhIcZVzYcT+FHwUTlCjzizOd/FDd74a4ry2O+9AeMeM6p3V+8NKYmvDfwy/x4uholW0 krnq3N9ViTuNBJ28mz5tb+mgUe11Nu0N8U9iV5aXd6MNoagJUSXvaYt2GP3qv4twwETW JAow== X-Gm-Message-State: ACgBeo0oLD6ceUDDW4u5f3liUs/SENTgsSakhbx/E/o0rn9FkNkGPdN2 HmizTUjP+LpozPmGQ6CMX+oE2+0eE6ClKYcA X-Google-Smtp-Source: AA6agR6BXiviu/fMhtQYEidUfIBaxKQz8CADJ9PvR9/TZPkp8KxyBxMr5H555oupXKEWVyIiR5O4fg== X-Received: by 2002:a17:90a:7f89:b0:1fa:ad33:7289 with SMTP id m9-20020a17090a7f8900b001faad337289mr8838077pjl.173.1660833136171; Thu, 18 Aug 2022 07:32:16 -0700 (PDT) Received: from vultr.guest ([45.32.72.237]) by smtp.gmail.com with ESMTPSA id h5-20020a63f905000000b003fdc16f5de2sm1379124pgi.15.2022.08.18.07.32.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Aug 2022 07:32:15 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org, tj@kernel.org, lizefan.x@bytedance.com Cc: cgroups@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next v2 12/12] bpf: Introduce selectable memcg for bpf map Date: Thu, 18 Aug 2022 14:31:18 +0000 Message-Id: <20220818143118.17733-13-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220818143118.17733-1-laoar.shao@gmail.com> References: <20220818143118.17733-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660866809; a=rsa-sha256; cv=none; b=bAX+2AscI7The0DeEZeGYFNyg+YCQZJRCEKO5Sp3H6hmLa+zHoC+9QqXH1tSdHpAuR2ssz M1wSE5BcvTwHwI1NmaWPQM3PKaKc3jCf0jJyJpYYVvAxT+EZyTVwP1lewaJpsjqd5E931h AYcLNFWO3IdlqYbLO41JW00rSVlDjns= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=EX8ou0RT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.160.52 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660866809; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=64R3bxHMvv5+V0T/PDUTVz1Uk0VIs+H8eHbhqMNvmEQ=; b=cVVUfBHWXl2NU8pXA8h4pA0WrVpsk8+Bqpy+tsJyRdRI+WyhxPk4Lc9x4UadawgvcxF+BM kpc5wSZ2t3EdQruT8EWJTq0rtxUmZ3DM0ioXdjz28myUWK5SGqSiyBtKSZ6WYz3NpD7moa uR2/amE6gf1OIjHVCL2eozW0N41SB6Q= X-Stat-Signature: d3thy4wracwk6drjew1ucytrocrz7sdo X-Rspamd-Queue-Id: 9107D160649 X-Rspam-User: Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=EX8ou0RT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.160.52 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Server: rspam11 X-HE-Tag: 1660866809-679766 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new bpf attr memcg_fd is introduced for map creation. The map creation path in libbpf is changed consequently to set this attr. A new member memcg_fd is introduced into bpf attr of BPF_MAP_CREATE command, which is the fd of an opened cgroup directory. In this cgroup, the memory subsystem must be enabled. The valid memcg_fd must be a postive number, that means it can't be zero(a valid return value of open(2)). Once the kernel get the memory cgroup from this fd, it will set this memcg into bpf map, then all the subsequent memory allocation of this map will be charged to the memcg. The map creation paths in libbpf are also changed consequently. The usage of this new member as follows, struct bpf_map_create_opts map_opts = { .sz = sizeof(map_opts), }; int memcg_fd, map_fd, old_fd; int key, value; memcg_fd = open("/sys/fs/cgroup/memory/bpf", O_DIRECTORY); if (memcg_fd < 0) { perror("memcg dir open"); return -1; } /* 0 is a invalid fd */ if (memcg_fd == 0) { old_fd = memcg_fd; memcg_fd = fcntl(memcg_fd, F_DUPFD_CLOEXEC, 3); close(old_fd); if (memcg_fd < 0) { perror("fcntl"); return -1; } } map_opts.memcg_fd = memcg_fd; map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, "map_for_memcg", sizeof(key), sizeof(value), 1024, &map_opts); if (map_fd <= 0) { close(memcg_fd); perror("map create"); return -1; } Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 45 ++++++++++++++++++++++++++++++++---------- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 3 ++- tools/lib/bpf/bpf.h | 3 ++- tools/lib/bpf/gen_loader.c | 2 +- tools/lib/bpf/libbpf.c | 2 ++ tools/lib/bpf/skel_internal.h | 2 +- 8 files changed, 45 insertions(+), 14 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 7d1e279..048b692 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1300,6 +1300,7 @@ struct bpf_stack_build_id { * to using 5 hash functions). */ __u64 map_extra; + __u32 memcg_fd; /* selectable memcg */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 5f5cade4..fd1f67c 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -294,14 +294,33 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, u32 memcg_fd) { - /* Currently if a map is created by a process belonging to the root - * memory cgroup, get_obj_cgroup_from_current() will return NULL. - * So we have to check map->objcg for being NULL each time it's - * being used. - */ - map->objcg = get_obj_cgroup_from_current(); + struct obj_cgroup *objcg; + struct cgroup *cgrp; + + if (memcg_fd) { + cgrp = cgroup_get_from_fd(memcg_fd); + if (IS_ERR(cgrp)) + return -EINVAL; + + objcg = get_obj_cgroup_from_cgroup(cgrp); + if (IS_ERR(objcg)) { + cgroup_put(cgrp); + return PTR_ERR(objcg); + } + cgroup_put(cgrp); + } else { + /* Currently if a map is created by a process belonging to the root + * memory cgroup, get_obj_cgroup_from_current() will return NULL. + * So we have to check map->objcg for being NULL each time it's + * being used. + */ + objcg = get_obj_cgroup_from_current(); + } + + map->objcg = objcg; + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -311,8 +330,9 @@ static void bpf_map_release_memcg(struct bpf_map *map) } #else -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, u32 memcg_fd) { + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -405,7 +425,12 @@ static u32 bpf_map_flags_retain_permanent(u32 flags) int bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) { - bpf_map_save_memcg(map); + int err; + + err = bpf_map_save_memcg(map, attr->memcg_fd); + if (err) + return err; + map->map_type = attr->map_type; map->key_size = attr->key_size; map->value_size = attr->value_size; @@ -1091,7 +1116,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, return ret; } -#define BPF_MAP_CREATE_LAST_FIELD map_extra +#define BPF_MAP_CREATE_LAST_FIELD memcg_fd /* called via syscall */ static int map_create(union bpf_attr *attr) { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index e174ad2..1e9efe7 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1300,6 +1300,7 @@ struct bpf_stack_build_id { * to using 5 hash functions). */ __u64 map_extra; + __u32 memcg_fd; /* selectable memcg */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 6a96e66..6517553 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -171,7 +171,7 @@ int bpf_map_create(enum bpf_map_type map_type, __u32 max_entries, const struct bpf_map_create_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, map_extra); + const size_t attr_sz = offsetofend(union bpf_attr, memcg_fd); union bpf_attr attr; int fd; @@ -199,6 +199,7 @@ int bpf_map_create(enum bpf_map_type map_type, attr.map_extra = OPTS_GET(opts, map_extra, 0); attr.numa_node = OPTS_GET(opts, numa_node, 0); attr.map_ifindex = OPTS_GET(opts, map_ifindex, 0); + attr.memcg_fd = OPTS_GET(opts, memcg_fd, 0); fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, attr_sz); return libbpf_err_errno(fd); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index 9c50bea..dd0d929 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -51,8 +51,9 @@ struct bpf_map_create_opts { __u32 numa_node; __u32 map_ifindex; + __u32 memcg_fd; }; -#define bpf_map_create_opts__last_field map_ifindex +#define bpf_map_create_opts__last_field memcg_fd LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 23f5c46..f35b014 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -451,7 +451,7 @@ void bpf_gen__map_create(struct bpf_gen *gen, __u32 key_size, __u32 value_size, __u32 max_entries, struct bpf_map_create_opts *map_attr, int map_idx) { - int attr_size = offsetofend(union bpf_attr, map_extra); + int attr_size = offsetofend(union bpf_attr, memcg_fd); bool close_inner_map_fd = false; int map_create_attr, idx; union bpf_attr attr; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 3f01f5c..ad7aa39 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -511,6 +511,7 @@ struct bpf_map { bool reused; bool autocreate; __u64 map_extra; + __u32 memcg_fd; }; enum extern_type { @@ -4936,6 +4937,7 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.map_flags = def->map_flags; create_attr.numa_node = map->numa_node; create_attr.map_extra = map->map_extra; + create_attr.memcg_fd = map->memcg_fd; if (bpf_map__is_struct_ops(map)) create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index bd6f450..83403cc 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -222,7 +222,7 @@ static inline int skel_map_create(enum bpf_map_type map_type, __u32 value_size, __u32 max_entries) { - const size_t attr_sz = offsetofend(union bpf_attr, map_extra); + const size_t attr_sz = offsetofend(union bpf_attr, memcg_fd); union bpf_attr attr; memset(&attr, 0, attr_sz);