From patchwork Wed Mar 19 22:16:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 14023225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1549C36001 for ; Wed, 19 Mar 2025 22:17:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56B04280005; Wed, 19 Mar 2025 18:17:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CA9C280002; Wed, 19 Mar 2025 18:17:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FA6B280005; Wed, 19 Mar 2025 18:17:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0E136280002 for ; Wed, 19 Mar 2025 18:17:10 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 975481A059F for ; Wed, 19 Mar 2025 22:17:11 +0000 (UTC) X-FDA: 83239712262.19.CBEA74E Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf03.hostedemail.com (Postfix) with ESMTP id C410820004 for ; Wed, 19 Mar 2025 22:17:09 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZxQxeDlC; spf=pass (imf03.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742422629; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G+XAb5qZM9EJun3uRHZFaBgl2slfeKVEfwXubaUvgvQ=; b=1GiVbM5W+NP1J+PDM+/Wo4ScxWEWAPIRcdJnfh/WVx2l0diDUInMZlYzH1gQCag7WdUcRe Jeg43FMe5P+eD5KIz7VfGPAkAanX6zXNmCgOz6vUx+7hDIUY9TUgUwqZIN0zQ3ktXWQltC Cjo80+fM4B6qoX3vteTzqIglrpdiO88= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZxQxeDlC; spf=pass (imf03.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742422629; a=rsa-sha256; cv=none; b=caKGgq+pfGoCKmNLIOfhs2ykRrbdAQlsBsOHNc40wFrs2QjqMcRpSVy+ou73hxpWTNQLlb z6GRXqpd2EBM8Ern/8WeMiBBYuMh37Xju87GeqyKwspQPhC9YeaSJ3OGG+TQoG+woaChye NwMEZOcMZmWhXPRva4bSy2+KmhG1VfU= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-225477548e1so1140015ad.0 for ; Wed, 19 Mar 2025 15:17:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742422629; x=1743027429; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G+XAb5qZM9EJun3uRHZFaBgl2slfeKVEfwXubaUvgvQ=; b=ZxQxeDlCVRDCxufVuUWJNX4vVYEcGQn2IlqMUC4KY69j4YIKiDliPYZ7oK85qCF3SI e3AR2PB0X0P1uAJnE6ZL0nbzrFE2yMDxnkAfG3AqWZ6KnXp+6gzhDveTchUq7WlgDNFe X0W2FBN9MK7SZxctLAX68MTIVs8ZWYAYrbmUd4tkuP5Ny32TlZ5Zymm1z3lZzLnrimRt B0qmmJxUDFAzpqAbHxd9MsF1oxRdEcqyslMpioDlBJV9zWeQJOtmklm45e2SRJZ+Fqev /LqSVQZAzph/3zQJtRA19w4V5SbvN77SKZhJWhnfIN7PbAnIo+RO/lAUsN8fk/gXb0ni 0Jew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742422629; x=1743027429; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G+XAb5qZM9EJun3uRHZFaBgl2slfeKVEfwXubaUvgvQ=; b=G+qm75R+92hL/IPMA9u3/h89ez1VDE9jd5PqRO7B37eg470ffDoiF6Dq4yLxImNciQ nU+0Nvtq++qj3F23toqBhOdin2Rs80Oa4zdU2d5nwLktz3+FGPdTiC9W39CeTS0KWp4S s/AZwzPIF3NvUo5FfurYfTXJ++MREmip7L+Mf5SEYA/qWmL9SLlwbavaqP1BQ1Kbm1Rk FqZaxg63K8YrkC7yT7bc8g+rfijcS2V50fep9lUdT1o+2deXfVaGZcmrClqLL2hojp1U 25iUQAXDw3lzR7urSJTmY1CMCgfCfQ7jDLwUkgkKQZ0I3l9ALnC4SFobocGBpHz9BllC cTww== X-Gm-Message-State: AOJu0YwdZVGksHzfmVzA50VdLto7mWkOaxi/d7DdR84/O539QyCAVT7d 7TvLnCQiXRePI92UKklHhYAhIFxDzif5nOlSO6MYhMRZjojI/v2K X-Gm-Gg: ASbGncsxCNr6JfFW899X2oSqwBaiRKQOclvehflHahfZzW5xEENEcBF7xFFMQd2wlF6 pJwnSZMbuG8QYYr/+2CrdkdR5qSOiiEk6jL6I8zQ4JceJ8GJa+pP9egsk0+sClLSH8uTcgTFrd8 FviLxZ5aap36/0nKrO2DtAxR1D5MvD8v+avCGcz8VUoIn/dyPsMfoX5eKkHAACb0xT3L5vH8+9p ZoMURfa8MDzcaKLA3DCF1ysbqzWZBPsqXtJpH6FXFZXQJG0jMei3dn8ELDJORiKm2ZrxCgog7P4 WpxgrPg8gEcN1deWbDV2Iu0a/xncvBQtCaFIC/s+NxsW2VmxI9qfBTcx6DI3uFJs0HLqeGWb8m0 dn5A5Nb0= X-Google-Smtp-Source: AGHT+IEKKYpDuuecD1uo8ylrhr/00YN1rGh/OPCt+W9SEcE6Jd30rlNrqU8W2oVUc0Hxahi+e0T9+w== X-Received: by 2002:a05:6a00:3c96:b0:732:5611:cbb5 with SMTP id d2e1a72fcca58-7377a866940mr1458881b3a.11.1742422628594; Wed, 19 Mar 2025 15:17:08 -0700 (PDT) Received: from jpkobryn-fedora-PF5CFKNC.thefacebook.com ([2620:10d:c090:500::4:39d5]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-737116af372sm12253977b3a.160.2025.03.19.15.17.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Mar 2025 15:17:08 -0700 (PDT) From: JP Kobryn To: tj@kernel.org, shakeel.butt@linux.dev, yosryahmed@google.com, mkoutny@suse.com, hannes@cmpxchg.org, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 1/4] cgroup: use separate rstat api for bpf programs Date: Wed, 19 Mar 2025 15:16:31 -0700 Message-ID: <20250319221634.71128-2-inwardvessel@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250319221634.71128-1-inwardvessel@gmail.com> References: <20250319221634.71128-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Stat-Signature: he91ay7np7tmw4arwzyb9fbt6to5dxkq X-Rspamd-Queue-Id: C410820004 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1742422629-63594 X-HE-Meta: U2FsdGVkX1+xfDPEAQXhBd/A/0FyZbIO9Z26uusbWYqhWf3JTHdZgHR1mG5ZDWrF0Fd8vHjXIj5znAaNucLVs/YTYlHrYW+YcmxVRP1TgGORiyrmwMbmo/eVIfQ63MrNUpy3rUZs05CuT3GpYYprOkRaCmFD6wbTcO+8VPITuTclBRoVVn8nBpKN5nn/Bd3JkXuGCBAdzJ4mgPaHK4wMr2+bnxTXNAFMxP32+oOaUDRHMAg2N+9qnmUOspMZTEnMvi3ob+DrNMudM584lYG7CLUc+ZM8Dc3wViXtije3II291EpyoOiGGuZeASdMtb29eNfpxkcMon4DVlrhGk0LPjJio6Siu7v8VpHp6Sv94Crrz2OEJtxz6rTAJx8UPQ7X6CmYNmGvHlO21B6o8Hx8VJoMJgqMmvjnPU2HM/I/lYVLLvsegDIoKRBX5+9G1dyvH+T6NFIfVHx9uNCvFotqBKUVpz4He7OJGHZYwUC7OYhHF1okNbRsfH1VWfjmKRvS6Axj03kL9BrJAw7unR4xMk59aWdqYG2SzGmnAFJuFdFOG9dvUFDKgw0B4AgyKS19Tp1f/pZDVzmrW0qA9zhjf8QowjNES5R0EqsZZlHFoUglXkVA8w7MM9xrmc2wHe+FMlZYjLcKdHtWKCi8BYFfPj3pWJq9k2Ar7tIg2qtuJ0VOZNsQ9mtR97cfdlEKNsAIricC9igJSO08nqAMFFjFvZLpwwMDCGhlc+43o6VxWRqPkIYdYi5w1u9UJDJKeFb0/MgWji9WE03NBE/IWe0ZmMuJ76AuUar27IjPzppb28tYdKSHH4VyqlHxEyAU+p3gens/tvI1kjh0bSvZzbBf82/dv7GRXTH8rIfaaFc8WURR5vrTUMvXf6uzdLylEDUQ+6xngVsLbyh7NrOJE7gIAbEb5wWUWWR9liuxg/W9y7BvkaN4czTXaaZo2dJbTvsGPvjL4knwaES0/Njv9a3 uRLX1PY+ oJbZXqYmPwExTdXiYZgo82lH05fwfzcfAuwr8hXzHcNWTkVzTpFb3a7VLZFDFPDaivbeYDQV8hWLnTvT4pL11p0QhuUv+orMFrnEdeTknlAeKbTv+b3euU/TlYvLtRmtsqaX7G/qZrzwsDwY9s4yw0NEyhvDAIVHtxPlhnkiWt/oLvUfhyQdYg0zRSqLFmDI99GWrxwsbIQo35/zmpjsMxlbz5XQKEJXRhFjukZL0a7AZiSTc/mxt1v7z+Dm7N1r7WM6s48owvfkncF1ObXv+25YPIGQoUY92cRpueHcHTIyRdhC5hNX+veTBSjPGsV1s60M06Ja/jiw6uh0yOVqNhN4A5bTuO2fMGGZxO0mnMGwoBB3mVRS+xGTJWo7VdjdTLJAM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The rstat updated/flush API functions are exported as kfuncs so bpf programs can make the same calls that in-kernel code can. Split these API functions into separate in-kernel and bpf versions. Function signatures remain unchanged. The kfuncs are named with the prefix "bpf_". This non-functional change allows for future commits which will modify the signature of the in-kernel API without impacting bpf call sites. The implementations of the kfuncs serve as adapters to the in-kernel API. Signed-off-by: JP Kobryn --- include/linux/cgroup.h | 3 +++ kernel/cgroup/rstat.c | 19 ++++++++++++++----- .../bpf/progs/cgroup_hierarchical_stats.c | 8 ++++---- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index f8ef47f8a634..13fd82a4336d 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -692,6 +692,9 @@ void cgroup_rstat_flush(struct cgroup *cgrp); void cgroup_rstat_flush_hold(struct cgroup *cgrp); void cgroup_rstat_flush_release(struct cgroup *cgrp); +void bpf_cgroup_rstat_updated(struct cgroup *cgrp, int cpu); +void bpf_cgroup_rstat_flush(struct cgroup *cgrp); + /* * Basic resource stats. */ diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index aac91466279f..0d66cfc53061 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -82,7 +82,7 @@ void _cgroup_rstat_cpu_unlock(raw_spinlock_t *cpu_lock, int cpu, * rstat_cpu->updated_children list. See the comment on top of * cgroup_rstat_cpu definition for details. */ -__bpf_kfunc void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) +void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) { raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); unsigned long flags; @@ -129,6 +129,11 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) _cgroup_rstat_cpu_unlock(cpu_lock, cpu, cgrp, flags, true); } +__bpf_kfunc void bpf_cgroup_rstat_updated(struct cgroup *cgrp, int cpu) +{ + cgroup_rstat_updated(cgrp, cpu); +} + /** * cgroup_rstat_push_children - push children cgroups into the given list * @head: current head of the list (= subtree root) @@ -346,7 +351,7 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) * * This function may block. */ -__bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) +void cgroup_rstat_flush(struct cgroup *cgrp) { might_sleep(); @@ -355,6 +360,11 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) __cgroup_rstat_unlock(cgrp, -1); } +__bpf_kfunc void bpf_cgroup_rstat_flush(struct cgroup *cgrp) +{ + cgroup_rstat_flush(cgrp); +} + /** * cgroup_rstat_flush_hold - flush stats in @cgrp's subtree and hold * @cgrp: target cgroup @@ -644,10 +654,9 @@ void cgroup_base_stat_cputime_show(struct seq_file *seq) cgroup_force_idle_show(seq, &cgrp->bstat); } -/* Add bpf kfuncs for cgroup_rstat_updated() and cgroup_rstat_flush() */ BTF_KFUNCS_START(bpf_rstat_kfunc_ids) -BTF_ID_FLAGS(func, cgroup_rstat_updated) -BTF_ID_FLAGS(func, cgroup_rstat_flush, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_cgroup_rstat_updated) +BTF_ID_FLAGS(func, bpf_cgroup_rstat_flush, KF_SLEEPABLE) BTF_KFUNCS_END(bpf_rstat_kfunc_ids) static const struct btf_kfunc_id_set bpf_rstat_kfunc_set = { diff --git a/tools/testing/selftests/bpf/progs/cgroup_hierarchical_stats.c b/tools/testing/selftests/bpf/progs/cgroup_hierarchical_stats.c index c74362854948..24450dd4d3f3 100644 --- a/tools/testing/selftests/bpf/progs/cgroup_hierarchical_stats.c +++ b/tools/testing/selftests/bpf/progs/cgroup_hierarchical_stats.c @@ -37,8 +37,8 @@ struct { __type(value, struct attach_counter); } attach_counters SEC(".maps"); -extern void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) __ksym; -extern void cgroup_rstat_flush(struct cgroup *cgrp) __ksym; +extern void bpf_cgroup_rstat_updated(struct cgroup *cgrp, int cpu) __ksym; +extern void bpf_cgroup_rstat_flush(struct cgroup *cgrp) __ksym; static uint64_t cgroup_id(struct cgroup *cgrp) { @@ -75,7 +75,7 @@ int BPF_PROG(counter, struct cgroup *dst_cgrp, struct task_struct *leader, else if (create_percpu_attach_counter(cg_id, 1)) return 0; - cgroup_rstat_updated(dst_cgrp, bpf_get_smp_processor_id()); + bpf_cgroup_rstat_updated(dst_cgrp, bpf_get_smp_processor_id()); return 0; } @@ -141,7 +141,7 @@ int BPF_PROG(dumper, struct bpf_iter_meta *meta, struct cgroup *cgrp) return 1; /* Flush the stats to make sure we get the most updated numbers */ - cgroup_rstat_flush(cgrp); + bpf_cgroup_rstat_flush(cgrp); total_counter = bpf_map_lookup_elem(&attach_counters, &cg_id); if (!total_counter) {