From patchwork Tue Feb 18 03:14:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13978858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21EA9C021AA for ; Tue, 18 Feb 2025 03:15:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F62F2800C1; Mon, 17 Feb 2025 22:15:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A5532800A4; Mon, 17 Feb 2025 22:15:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FAAC2800C1; Mon, 17 Feb 2025 22:15:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 499CB2800A4 for ; Mon, 17 Feb 2025 22:15:19 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0E34C817A1 for ; Tue, 18 Feb 2025 03:15:19 +0000 (UTC) X-FDA: 83131599558.21.EC38BAB Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf19.hostedemail.com (Postfix) with ESMTP id 34DB51A000F for ; Tue, 18 Feb 2025 03:15:17 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=N7Mym30q; spf=pass (imf19.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739848517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CD1jatsWdmIEiP5eGI+IqcgznEGBHRgmN/6oH6WnB14=; b=sXrJ0m6x3YVigGUixSiN9fVsUIMDqZ7JcVsTh8kryNbYiOLIVeFOT215HJ1WAaRWqAPewa CAEM8es89+5AL0OMscKQCsvppb2TDxp+19v6n/vHS3MuyWeoEPiNlLIJRDyoM2lHWybS2q wAeYCwQZnnY1BSH2uKPclyQUg1/4c7I= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=N7Mym30q; spf=pass (imf19.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739848517; a=rsa-sha256; cv=none; b=iDU072WLpYXsT+p3zLs3mSZW1Yoe8OaQUamH34iE/P2IVOgZGw/q1DTtW5ILA75E7huFwy jpxiKVODGKfiQDtOAd8r1yYOEj9To2RXmJ0aW+uGo1iQN4YNoIxLwOEXYjzEoQRoJGyTjy ur2ybd4yT4EkOWSR+OnznzqM8C9qxcg= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2fbffe0254fso9257164a91.3 for ; Mon, 17 Feb 2025 19:15:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739848516; x=1740453316; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CD1jatsWdmIEiP5eGI+IqcgznEGBHRgmN/6oH6WnB14=; b=N7Mym30q8NpOYKQ/ExU4FHnsLLIPIG2bWgEojls1e0LFSXpTSQII2238GYt5oTyrVz jXttfNbLcdG7Mtnigc0NBdXvtoy4jWI3qORjTiZoJI1A5mcp5+JrGpjQ0bbxJCwH0Cxz brPlz+p9yTCSyJrTHIaI3Dve3NUBaE7TNW9Ti7pVcHuOrdVU0YGZ8OqjdqrBK4X/Jc6R SP0QtdoxakAfzEQL2vZzhAuffjF8gZ6aMYcwlu9wZawJX57bRMr92AQic2MOYG39RCPH jHcA4DHqOqkhmrpqv/d9dpG5PJXPXz1lXgBb1f53LbzCy1NvPbQE5yDs0K4kLuzXQirz FV/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739848516; x=1740453316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CD1jatsWdmIEiP5eGI+IqcgznEGBHRgmN/6oH6WnB14=; b=LZh0/ROW6i5CXqJhsvwT6+xZaDaaKsDe5xNcHbxnnnmyzE1ceReZbcNAF/SIDAFlOX UvkBhNpVTp0c26vcX+dxGBo+1BbkkfPCRjaIxXEWnbGkjKw5zEodxMpdy4xYBDE67Spl eHoYxNnVd8Tqo33Og2bT+ywm50zA/IQtBFQLkCdvyhE8q3k7PFv/uWEvpzLpgceLZszT pUew6F8XMLGiBA2KTPOIJmlkhfbFjBoh2qgEhlUoMSLl1ZguT5DlKhM4VrGagsDymeBa n4Zb8X2Fd6j+R1htRUXITg5RXfwtMYWqvHVnPREFFaqciZKbq12lRdw1w9MUPsSXGoEm BqSw== X-Gm-Message-State: AOJu0YzuXzuSm41JghfxyR6vGELo9VoNDkwGT2HAYWmCe1FhcY6P3acz mjv3g+ekO0Aq2fQeV3kQrjaR/NTvpuMGJGWE7ZyokmvjPdSwFIbKifkiKQ== X-Gm-Gg: ASbGncsvmdmH/8UdKKnboPmNCJygDeauBIcvZiIsj10bPAOOt3SyGLKw7PGfqV4UU2g Qeb8Ii/6SUcIc5D3JzHJWfJFA8u+miUFUYDhINP4tzqa4ZX659XYpSqmqRFl0C15gP1o19G/wWA R3umyTdrC192SHd/sm1kd1cXn7tO/cYS6xKSsSbkQEcrFzSwtgFJOwsygxMqpoM7jjvHqqrjvAY A4ksviHcTLnBB0fzYvIY6FmasTkVbXJGkMrmucrmYbkYgDPpORMrKkGqcUUS5Yg142begCkp8BR o2BP7lOqcKgwDl92VqGUqg8hYA1iGYzrnf4s4BxDVso5AIBJxu1/ X-Google-Smtp-Source: AGHT+IEZ40Ep2/y9fUsKsPY3ph6FBvkIsGW6s0OHsKOFL28M9YkkbAUChBqTJ0u+bam9kXnHkApFwA== X-Received: by 2002:a05:6a00:124b:b0:732:1bad:e245 with SMTP id d2e1a72fcca58-7326179e7d8mr16437947b3a.7.1739848516068; Mon, 17 Feb 2025 19:15:16 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-732425466besm8763451b3a.9.2025.02.17.19.15.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 19:15:14 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 09/11] cgroup: separate rstat locks for bpf cgroups Date: Mon, 17 Feb 2025 19:14:46 -0800 Message-ID: <20250218031448.46951-10-inwardvessel@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250218031448.46951-1-inwardvessel@gmail.com> References: <20250218031448.46951-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 34DB51A000F X-Stat-Signature: zji1intd3ftikzbzydjg5zcg47umrigw X-HE-Tag: 1739848517-743642 X-HE-Meta: U2FsdGVkX1/mG24W8WubGYhyIOD37ShOIqCKjHjG0B2TNRjYhjdJVHBLg/ZHkaPs1KqSKbOXRVMVqwS2/L1nkKfIaZbhzUnVeRNxNe8K20ShPT52PHSzXJGTDBIirEa9pss/SA0R9uu4yEBoa/M4Mwh5uiQk8nDm/M8uiJ/rEGGiEg+hEXcUjMiaea/jXzbGDaN8NkENFlOH3Xvrxk037W0ReJwyzjLLhzIb1vpN1/CnlZF2IRez02Yuk6llx4Gfa1Oy5zbJ+WXuWqnBifXXgLF93U9S78KhffLnGsJpPqtcI0W+T186YBT2riE7cfaSG1ubFUHwB9KhbhfXTqvB4Jy1v6mBF+znQY3+3OYhBAhebpFiOy1Ja3dsOIXv1+6+JtuDxuFpfjbnJkSo+FOiNB9gzz/s0hj6589NyezbwKRe7LOxSg/4pQ6bJwuMu1dbqnQ3VzjSpHq7j6nibtmtvuHSyk42BqfCFnkjnrQl3aYLUbJx+0G9de++8JzXErDnASxOElD2xXbltyfTkbtQ7tILWUxN28pOBFTzjzJm6tVVt0ZGdB9WaQhf139O9GfghD1bicYALaiUImpPE3urBXTWEDjWs130NGfhTo5j2AzzIsNoSjbSY5OzaZ2lZc5YrD26mls/HcBOfKjSv8oZBSvzJj9tck6X8DX8o5bIeP+h/kommKjJ3MOJT0Zi+PWp175WrSrGItcRgfnQghX+RdEYt4A52mCQy6atNUpeOQkzcm8j/kSTImuRetY/MvfQp9DJxBZzYU5fCAWLoKXMXe9qwpIg8vz2CgbGQLQX1xTLsGcZ7Bm+fvHwudxX/xhW5OltUWTbPJQNJmpdZzJiMKGsdzR+yWvHWmsXiCsvpm4RE7Dcg+WacJV/509Mf3MFhOK3bqYUpQdra330zdBcQnv4jQ7lEkljHl0h/c7dU33YZb6zosw028roYD3XZv32SIZcVcq41IKMZ7XXBGV SdIregk0 Tt4dU6FriEqwt5WF3K9rdLfQEtCX3zb889a6xx3tDTplLNEPOMd69UhgA36kNGg0trj0O7szKaJxrmYz06yvup0ymF1e0qd7Lc8UMD7/8+2ooItbTnnQtdvsm4RsOoGQBIP6lsKxP1doE+csY9+6iSq0JFXp6Z0DNFz77Kx34CdAQiJ75wpwiYeCPb8ZWwysU/Mg+PjvFnbM7+KUynz/7IDCIm85T9dHfVZgJBGn+ZmdsvdElTH0n3fR+42mNnYnguYDR1dqu+01Wdvb6KqSv8piiwH7zjpvQtU/0G6ujHb3fgJcL9N7s1g292yjSU9nZ6HuZguEZCf69sJweCc93fIReaSsSgWZdcnbF/Syf2sFB/4DaP2RE9Dsdgg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use new locks with the rstat entities specific to bpf cgroups. Having these locks avoids contention with subsystems such as memory or io while updating/flushing bpf cgroup stats. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 9f6da3ea3c8c..7d9abfd644ca 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -18,6 +18,11 @@ struct cgroup_rstat_ops { static DEFINE_SPINLOCK(cgroup_rstat_lock); static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock); +#ifdef CONFIG_CGROUP_BPF +static DEFINE_SPINLOCK(cgroup_rstat_bpf_lock); +static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_bpf_cpu_lock); +#endif /* CONFIG_CGROUP_BPF */ + static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu); static struct cgroup_rstat_cpu *rstat_cpu(struct cgroup_rstat *rstat, int cpu) @@ -244,7 +249,7 @@ void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) __bpf_kfunc void bpf_cgroup_rstat_updated(struct cgroup *cgroup, int cpu) { __cgroup_rstat_updated(&(cgroup->bpf.rstat), cpu, &rstat_bpf_ops, - &cgroup_rstat_cpu_lock); + &cgroup_rstat_bpf_cpu_lock); } #endif /* CONFIG_CGROUP_BPF */ @@ -490,7 +495,7 @@ void cgroup_rstat_flush(struct cgroup_subsys_state *css) __bpf_kfunc void bpf_cgroup_rstat_flush(struct cgroup *cgroup) { __cgroup_rstat_flush(&(cgroup->bpf.rstat), &rstat_bpf_ops, - &cgroup_rstat_lock, &cgroup_rstat_cpu_lock); + &cgroup_rstat_bpf_lock, &cgroup_rstat_bpf_cpu_lock); } #endif /* CONFIG_CGROUP_BPF */ @@ -617,7 +622,7 @@ int bpf_cgroup_rstat_init(struct cgroup_bpf *bpf) void bpf_cgroup_rstat_exit(struct cgroup_bpf *bpf) { __cgroup_rstat_flush(&bpf->rstat, &rstat_bpf_ops, - &cgroup_rstat_lock, &cgroup_rstat_cpu_lock); + &cgroup_rstat_bpf_lock, &cgroup_rstat_bpf_cpu_lock); __cgroup_rstat_exit(&bpf->rstat); } #endif /* CONFIG_CGROUP_BPF */ @@ -626,8 +631,13 @@ void __init cgroup_rstat_boot(void) { int cpu; - for_each_possible_cpu(cpu) + for_each_possible_cpu(cpu) { raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); + +#ifdef CONFIG_CGROUP_BPF + raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_bpf_cpu_lock, cpu)); +#endif /* CONFIG_CGROUP_BPF */ + } } /*