From patchwork Thu Sep 5 00:30:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13791565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6339DCD4F4A for ; Thu, 5 Sep 2024 00:31:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E34A26B01F5; Wed, 4 Sep 2024 20:31:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBD4E6B01F6; Wed, 4 Sep 2024 20:31:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC1C06B01F8; Wed, 4 Sep 2024 20:31:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 96D816B01F5 for ; Wed, 4 Sep 2024 20:31:52 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 50565120F57 for ; Thu, 5 Sep 2024 00:31:52 +0000 (UTC) X-FDA: 82528806864.21.E9EB4D9 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf13.hostedemail.com (Postfix) with ESMTP id 80DB620004 for ; Thu, 5 Sep 2024 00:31:49 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=joyyH0Rc; spf=pass (imf13.hostedemail.com: domain of 38_vYZggKCIw0y38uEx4w44w1u.s421y3AD-220Bqs0.47w@flex--kinseyho.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=38_vYZggKCIw0y38uEx4w44w1u.s421y3AD-220Bqs0.47w@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725496181; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OCIYE2bxjzqInsqy6M77OBXOLSiNi2ahaVS1P1UY4eQ=; b=oM1YnjYPnjtqCLF3f/GwJwiXFahQ7n+4zdN3uiOUirvCQ5NjHdhNQQlpPK+229kgFQny4D yWFU/kBy2G0MOVHbcER3J2ZA6KrWU7yfH5ZT8ydTGr+SDoOtYn+EboqVewT8uMH0kQWN0z 4j3MK7v1I3UxaSaUxMFzjK4vi6K/6w4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=joyyH0Rc; spf=pass (imf13.hostedemail.com: domain of 38_vYZggKCIw0y38uEx4w44w1u.s421y3AD-220Bqs0.47w@flex--kinseyho.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=38_vYZggKCIw0y38uEx4w44w1u.s421y3AD-220Bqs0.47w@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725496181; a=rsa-sha256; cv=none; b=2HNhXWyrz3KHGAx0knR1sUvlsjfH31FHgIYWpYqH5SkxXtNH3GWmoHZL8UQ+ihtTvsBG9w H3graLI9HVkCcaHV8pqgWb/SMrg5PaU6tJ3JYcS8VdgyCoz9DOBDaEtod22vI/Fl2Llnbt lfEUqmBY9HZbwsEB0HU7TcPv9vOjqks= Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-717453c0677so265456b3a.0 for ; Wed, 04 Sep 2024 17:31:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496308; x=1726101108; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=OCIYE2bxjzqInsqy6M77OBXOLSiNi2ahaVS1P1UY4eQ=; b=joyyH0RcblKxhnnsniUWvgrVNYgCxHF9LFLKBuhQ00Q+2/HXf6QLK4Zyb1k1Fr+ZCT fIR/8hy1KMlPv6Zj93HTesflId5b1ptZoErIQfio7008NbsfUEfi7XqMhOgF9F+AsAXI elWDMJp+P4u/mL4z/Yr9+oJmaEj5+n+tYFunAPms9l6vhDrlsx2aj8qhl+gK+1Wh4TVG fhwVC56bg4fFG6sRFCCNN+51Dmjk/e+nBRhT4cLy5Hec7e770li4K5GIH8Jx5wb3d0RL EcmTGjFU75/U3xDXfS3uNBVQTIUvScHcgmF97sBk9QmzOyamoZOkleuoS3eGMOPRkoWV /9vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496308; x=1726101108; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=OCIYE2bxjzqInsqy6M77OBXOLSiNi2ahaVS1P1UY4eQ=; b=xCQVc42vy9/ZOQHwrfB+C8yMdAtLIBMMTGubT6e4aKDkQQVnlAvVXcdnmGDEZgk2Ye QzS6m1fG7Y+oYSPjNJwp9J2UUFxQCnmL8gYoo6lXJR0zpIURs5+FkIKTul778l5B0g1X qDKK+nTM+SbMpR0AkKoa4N3rQSXsyIIhvR6DhbI3MAvC4AmMRlTdUtKJpy5XEPkc0B+/ drM0tgevNGUninFvtT9wQQInsXei7RBEAqEKufutR6Ux7GohAp4/BxZPakPljNgrfier UKhCSmiK6UxTm9LbtFiBAGSN/ZBhb5V+zB9uL5JP/LNKq4kX5/YruzME66qZ+sZy8evi +kmw== X-Gm-Message-State: AOJu0YzWWlP3QRbIZCag1vlHlCn/WOqXd2AdnwrwOFZcIfv3Pal001k0 fF37FYMK/t5/loUPsIOkg1ngFgjvcBWnG05Lzzd+OnIjJtUy5vVmJd9WJgN6YOhzEbJbyA7HnDJ EnJeyjQDi2Q== X-Google-Smtp-Source: AGHT+IHUPQSbmeHjhZigLeW58QOY18ICrs203NRPh39wz9VYTS5JC1PdHx3HOB6u+ncLsUa1QDiBpyCBj6pKLQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a00:8682:b0:714:1e0f:4477 with SMTP id d2e1a72fcca58-71730791c72mr43937b3a.5.1725496307872; Wed, 04 Sep 2024 17:31:47 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:50 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-2-kinseyho@google.com> Subject: [PATCH mm-unstable v4 1/5] cgroup: clarify css sibling linkage is protected by cgroup_mutex or RCU From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 80DB620004 X-Stat-Signature: gjw3jpegfddimg98qaerfk7bp9xftbqb X-Rspam-User: X-HE-Tag: 1725496309-358706 X-HE-Meta: U2FsdGVkX19iAV+pPD6GV4m/V5yiD6qJQLQSwErgoAlpws6sfExr1h4labV5EPF+b1SLye5iBAYcEgtuYG9AhhyV5uGszk96o7JP/0PhgzF65RrbxoQ7zX5NzV0UE2Rv3lRLr/CFZUpk0kLMiMhM3S8uKMjyiRFwIB3M/baRIH4kUvPbPRXaXXAgUnD790fz0E2GYU1uQeEICXdEZeYEmGif8v8uI8b/8/hWEZcyfzzKEJxW9CHAiShWg4eNG2lvNAKKtQwKOiZKmu21HPHWSwj0kZkg3iPztNC3EAhBk5IYfCKKDlN2eh1dPAeJobwiMODJWmycKCewH4VwW+kj0UG8zXr5hegoPt5qXrtnuHMz5JEkV7nM3pfylg8/TSu+ZHDYTa5K1/CxgnKB89sWDTbFeFlo2hxfsXMIqpgkxsvDNlExmmxcPzVW6RZ+c+TUBQhU7VczU5Qe2twu5tLUcgK4QMRgXVg3roPk5bmoNq7LsxZ+Z/B3IsKOLYg3t7dLem0natSpA8D+3dJu8mfSp2tfJbzJnH6HM1b3vJt5EJmy27b+m34quL2kpjurSjfIFAPXlhbqRdhlKa71Vtoq5lLO9L4PIxX5omXD7JhJyRqnB8IUcvH7U5dSPz/1JlMigjzAMXlytyJ2v6eKaDpcymAbEL1rzOI620j2/LEKuMmMJwg6f653JwmP5Znm/w7wqFmzFHO5PmxNreoPJz0mf3QuXjciNrxM78llKjXgdB/wA6d5tuR5QJgIUWykmPJo8yWnGjoBMve7n2ZA7Ufv0GvfS0rwdPI5UvI/gIqFzKxUEu/SPTkPe5NofN05RdaUxqURrylfiarL0zlLylHVVK4X47eRX78sYghi9IHUzM2xDdow7r7N8hudI0ImIYPEJiXH6eIHw4qaQVk9UIhHifWD0xTZKIzIbV4rYN2ZOFfLLLvUjtcJC6O/C7fZ/61k6P6gxgsStdrfca1tsZw G3UKpEli roWcPy6BHbjk/ptFKH8pUuRHqaTvP1qwnBYpZQKXmo/eE/2JILFECyQXaZUQqYIV9CgdV5ifTKMTHjGZ5geb1Kkt4UC0vHFhM0S3dzDUolRhZBEn8rNJQw08rw2IOp52BIDZgYb8rXCyUqutE/uN0kbXY7WZdLPkzqxVuyzTGvvl44+fTJhsgFb9vCL6I2DLaRsQrumjE1d2ZLcUSBYl44q+brIlkVP5UMZ54sH9B07m58UVrIptkZIWgpWHeXAFR1SSpKR+SddyrRzhinPZfV+LdY1Mu/N30aQLsZWleptR0rn9bjbz70XQbtJeE7gAXOUIS8OKtzsQOkgrkfbzzO22FYk0unT+ZQM2+D3uV/jbPpidCXJQKt2vav5ll7Cho15NQId8Qm1ZZVmRGD6g4X7Q9SdGkOIF6GeznZdM8Emxunb82FD6tVMG6X+SPHvjmoG4jYUDaJyBrzHfjcYUFtV/4iYEuCeb5dIL+VrE//Ach3/0afXZTLDx7CZtxPlHoEHVBcX8kbD+OOSmBkmw1Djn+OUGvaqKnHC9VWcCu8PNuz+aaFT0vYNq7RHDnM2pbKEBzNvfc+WBbMYRHfy6XqvNTGQTVyMzH2fAuncYFPC9bX/2LDMakn2MPDlAEVu2O509w2ZCx2DVOi5OVACoIHS/Y4MZ0o2jWU1hS8Z2kKjLQNkUdxE1KcTrcHch70lMlPXBRB2phuHElm3SKWRRURI6sQ7nfBJLpoKFYuB8M6j/ZoxKQdMZC0lCZa7xob/nF/aZ+HNrmOMhAyHQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly document that css sibling/descendant linkage is protected by cgroup_mutex or RCU. Also, document in css_next_descendant_pre() and similar functions that it isn't necessary to hold a ref on @pos. The following changes in this patchset rely on this clarification for simplification in memcg iteration code. Suggested-by: Yosry Ahmed Reviewed-by: Michal Koutný Signed-off-by: Kinsey Ho Cc: Johannes Weiner Cc: Michal Hocko Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Zefan Li Cc: Hugh Dickins Cc: T.J. Mercier --- include/linux/cgroup-defs.h | 6 +++++- kernel/cgroup/cgroup.c | 16 +++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 7fc2d0195f56..ca7e912b8355 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -172,7 +172,11 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; - /* siblings list anchored at the parent's ->children */ + /* + * siblings list anchored at the parent's ->children + * + * linkage is protected by cgroup_mutex or RCU + */ struct list_head sibling; struct list_head children; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 0a97cb2ef124..ece2316e2bca 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4602,8 +4602,9 @@ struct cgroup_subsys_state *css_next_child(struct cgroup_subsys_state *pos, * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @root are accessible and @pos is a descendant of @root. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @root are accessible and @pos is a descendant of @root. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the @@ -4651,8 +4652,9 @@ EXPORT_SYMBOL_GPL(css_next_descendant_pre); * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct rightmost descendant as - * long as @pos is accessible. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct rightmost descendant as long as @pos + * is accessible. */ struct cgroup_subsys_state * css_rightmost_descendant(struct cgroup_subsys_state *pos) @@ -4696,9 +4698,9 @@ css_leftmost_descendant(struct cgroup_subsys_state *pos) * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @cgroup are accessible and @pos is a descendant of - * @cgroup. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @cgroup are accessible and @pos is a descendant of @cgroup. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the From patchwork Thu Sep 5 00:30:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13791566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3975ECD4F4E for ; Thu, 5 Sep 2024 00:31:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0EF4F6B01FB; Wed, 4 Sep 2024 20:31:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 02A476B01FE; Wed, 4 Sep 2024 20:31:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE60A6B01FF; Wed, 4 Sep 2024 20:31:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BEC926B01FB for ; Wed, 4 Sep 2024 20:31:53 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4859380FEE for ; Thu, 5 Sep 2024 00:31:53 +0000 (UTC) X-FDA: 82528806906.13.15902D3 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf10.hostedemail.com (Postfix) with ESMTP id 73097C000B for ; Thu, 5 Sep 2024 00:31:51 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=B3fqiMfl; spf=pass (imf10.hostedemail.com: domain of 39fvYZggKCI4205AwGz6y66y3w.u64305CF-442Dsu2.69y@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=39fvYZggKCI4205AwGz6y66y3w.u64305CF-442Dsu2.69y@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725496287; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sqTDHTCRiwsvIU2qvmmvg5mPCxzlppgliuAtX0a739I=; b=itANnevBEXl/MkDPD7hz+OWEDukw13UFrVw5D5cDQGi5ja5de4espQYORotTPTMgDeSKMM YpXoWg3D7a7ZSdbWxvHwdzH0bHCmcU5YBLAhQqrLCI+bXdMfMpPsZuftsP2v8CHwx1OGwu 3OfkliKO6G+fVI2Y6CaMqE1dKS33vTA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=B3fqiMfl; spf=pass (imf10.hostedemail.com: domain of 39fvYZggKCI4205AwGz6y66y3w.u64305CF-442Dsu2.69y@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=39fvYZggKCI4205AwGz6y66y3w.u64305CF-442Dsu2.69y@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725496287; a=rsa-sha256; cv=none; b=57QeYotNHj+npcl/desHQOOOd2tVVrnW2bVbsI6PsllOoRSXv1W0gsjeniTj6wTXEfNu2a cHREohKcfslcF2n1+QpLNRwS1JVNAIIalOeeupRvTLHqDnhVanbWyC5IWbjExoCZnlCu6b 4bx3p/9cPsJjrs1LO5BBDhmEbdX3fso= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7cd9ac1fa89so285946a12.1 for ; Wed, 04 Sep 2024 17:31:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496310; x=1726101110; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=sqTDHTCRiwsvIU2qvmmvg5mPCxzlppgliuAtX0a739I=; b=B3fqiMflJFPoIMxv7o0rsmFuj4zjKkdLSC/jas+O40+5P92JBJ5A3LBP/kOV89FS1E zpSUxhHISIz8c7jaL3k1GItynfaC7NvcujiHADXG3JhDQtZg81QJR/WsXXEChusfBvI9 K4lPIy1zCANQLn8lSUj+DLMolm+iVC5XhSFBZXzWbOvV2WSLgr2mba1mi09cQ5XiWhbC Ldrlkkny4gFjG5czpDL9JujmhnvWK5hl1tROD6DfheKQ05TpPf3c3TSxV4+SwpCg/Xz9 TZu7e+L29+6IIgDTKuGxwkAd4ER/0GDwbSdPhAUpaozAX7eN48Eh8U/3j88yab3+6nXJ G2NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496310; x=1726101110; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=sqTDHTCRiwsvIU2qvmmvg5mPCxzlppgliuAtX0a739I=; b=ty02lUyVjR7xEQzaUtZg48DX0/he+jtcJsdadP00f3I4Y+BXuTnhNaRj9qO5JT+IOa RGqtzEJHimEGiqcQYXVPNbalw9gzRdtH/r79B4FUJS53tJyQ4WVkNEQRr0Vx3zoxEvuf 6RHwaGEz8NQvFv5ta0pc8AU3RAFfHxpsS5fpAYge/sGhq2WYggmzZqxtuhWH+VGCfAlu 1YaUCjaIFqpaHZ6Rna1SFzzIJnWf6cj4WKUpnRdcSvEGb/b815oM9WnvluT2RVAjmrrN XxE5zSgq1kiStR3T1NSNmDKmzVum5x8gyY4e68lJ6pRX4E2/qp61o2AyKkll0NA1AzbH mKzA== X-Gm-Message-State: AOJu0Yz/QsH9/aBPxJuBKTHM6/A0hxs6C/byf85wCvnL409myp1XYKn8 msJd323CHDitACAGJpJhjHnhkURVfaiClcm3yYuzhI15e62S7T7bRGN3Mc0kErRon1T3wPXmqqR rXWQTyhPZ2w== X-Google-Smtp-Source: AGHT+IEOy0bHsy4chODc/dIKYmpQwZgXe72gHtWSiqNJNS/vQMmLAYyUUqkUY8wRavMe2EY8pkrILzNoQwJIVw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:e34e:0:b0:6ea:d240:23b0 with SMTP id 41be03b00d2f7-7d4c1019e5emr56861a12.4.1725496309928; Wed, 04 Sep 2024 17:31:49 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:51 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-3-kinseyho@google.com> Subject: [PATCH mm-unstable v4 2/5] mm: don't hold css->refcnt during traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho X-Rspam-User: X-Stat-Signature: 4rikw65zfopo1k9bc7m3tj84oxn435h5 X-Rspamd-Queue-Id: 73097C000B X-Rspamd-Server: rspam11 X-HE-Tag: 1725496311-415485 X-HE-Meta: U2FsdGVkX18z1J4+scrFv+XL86hoEGCADtZpWAnN71f7Yo7pYyhW6jnt+niSeJxiHqzbNDMAFOdbVBJasOYNQoK6NjofZxrnT1iaC4iUocn+AhxoSenLDzQ61x8UvkEVOxoYleGw5DPvoJFIOjB0VoSWqQ4uDgwppy1uFCAUS2LHDvMVCyKHofAOXR2zVxFIsLRLlfE4lc3mEBjSXPEaSAFEB+/2AKyC2IEB2LLlhzlli5qqV4V9fbUa8lodwrxw7qxdr0AX5HimvLw4W9Nd0wGNjNyUr9I8Bch2h7P9gwPTfgo1v+UiP0OjiNHtLLknXRYLoMtrWalQIVnFSJAbUMCZdXmeHlKLY+7wxY8sgz0iyUegzsZGbTwADth+HENI6KfSlRSZka1kDslW91lxNtxraiTWeq5Q6NshSdMhTDud1nlP19e7tvHn829L5ZFh4KV2bXee7dWuSqNuk9TfYLOQTyHfQPPLdkRlVE0gtPXtOOpVq5PfSgjzy6F6hmUoTJ6jj90a0Ark6SOvBynhMtVEL9ai84XRcMDrit9IqtgZY+EAseo+jdgvPLAZCTXVJ15JZmaGKj87m7ItTMOhN0CSw3RBdi6VOnx0vHRxuHdxab1iDtHO6zIpsI7t+jbYBLy42V378HzDmWmmdNrWZ+aHLjanc48x3pHPNCfGR5GYi5X0WDgh8U6lLeaU28OUu+dvpNN3e+IqgjV7tLqMlAb8x/qhy20CusdyWeoEWPOHk6GVv9GfTsWr9TDGX4Qx6aEkjb7XKSKlujup9Tv+jcVsv9agqQgPBE4EzCw9piuDnRELv9UDd2eVx6OdQTNB8Tf3fa2KgcarXnYdfmH0+jly9eEzhhTfWvJEsZUJsNpWLhl99ZbuYO9pL9ajapsYhVmxEoHKR34XgFRJli4C1cw7EYHfGU73qaA5QLCDNTPNReXczZH21qn96p3YXt1HysiCHMFXLcXpRYC7CAN DDeMV90J biOFP0vhGcNhBLkg5/kRG2tWqqYHos3Z07vJ4GMXpyrKgy4Xn1pfhWHIOtLFI2IJW8xxsti4uIiPm0mCPt7vvqAIlkh/So9h/wFCuZ4rI66F4NYGwiUVSDgkk9TpahILfpKrqMABO4dC9sZkNm9lqwx4lNnMh70lSFK0z3LZQMH1zI1BwyWTkvdQh6t8DjX09CO3fMP0AoWgWt/gRI23oMKQFvp2codhqK0ZRlN5tsreRnUF6wv6KXtZnhW+yWWO5hrDrIAsscmabqkqtQlNXuj4nX0GrvZ1toj8uAyXIjW2m9ij5ai3dUsoj7tbJVUGRb5thZpkouZueGiA2owQcD73+J2HZqMniDbg2cEpL/bcytVXpkDMBVCZdwZ9ele4nQenPhFp7ehGOiz7TV1V7r8hGagcG6Rf6waQ4zIyrmNoZToVsnWJsHaQQ6X94i5PT9JIMXFhc6JL70ZKatPi0he1GvpU/k7iWEPSuHiBjK+VW69HHMq3/6GXv6tvR7i+ky04Pm29/DuvXFwojYTpOearok/gIBTqVagI65AMJ67AGNl7RaGbdQySmPR2gzpKXgiALArPGoOZhP9NRaslLLsXKBpShYc65GtlRdtuO6ElgC34T2lJOXJCAH54xWkjuPhUl+j9bIs1CWlGqc5/un1Wq1pm2WEa+D47LK7xtjkZdpDQSUWiJ8UuvVjYzcTDmY468QezQh6IWw2zmaM8GR0wSq4OKnNjfR/0T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To obtain the pointer to the next memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. The use of the RCU read lock with css_next_descendant_pre() guarantees that sibling linkage is safe without holding a ref on the passed-in @css. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutný Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 18 +----------------- 1 file changed, 1 insertion(+), 17 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dbc1d43a5c4c..cdd324812b55 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1015,20 +1015,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, else if (reclaim->generation != iter->generation) goto out_unlock; - while (1) { - pos = READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos = READ_ONCE(iter->position); } else if (prev) { pos = prev; } @@ -1069,9 +1056,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; } From patchwork Thu Sep 5 00:30:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13791567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91798CD4F4A for ; Thu, 5 Sep 2024 00:31:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C89A6B0200; Wed, 4 Sep 2024 20:31:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9794E6B0202; Wed, 4 Sep 2024 20:31:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A4A26B0201; Wed, 4 Sep 2024 20:31:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 55BD76B01FF for ; Wed, 4 Sep 2024 20:31:56 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1DB5CAAE20 for ; Thu, 5 Sep 2024 00:31:56 +0000 (UTC) X-FDA: 82528807032.01.4E8AA0C Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf24.hostedemail.com (Postfix) with ESMTP id 4E761180007 for ; Thu, 5 Sep 2024 00:31:54 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KQJgVTNS; spf=pass (imf24.hostedemail.com: domain of 39_vYZggKCJA427CyI1808805y.w86527EH-664Fuw4.8B0@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=39_vYZggKCJA427CyI1808805y.w86527EH-664Fuw4.8B0@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725496290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ac9f2JvhNnpWVv2ZKYegshHvMbbmkBcnbuU73G1cYt8=; b=JgY3eimbX+XLUlIVy0fI5C5o3rfKrTxE0MB9uj/CiCelXXwY5hcmrdFz4IpOzqLbouBsbc PvPiYzazTrtTk8/EflZOYZuNFaswlqdNg9Ih0d9XVBY2Yk8FgjOarcHfTNoSyLhgxFdpLp Jh8hGU8dQrAHRVdON8s0g1N/fBfmtYM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KQJgVTNS; spf=pass (imf24.hostedemail.com: domain of 39_vYZggKCJA427CyI1808805y.w86527EH-664Fuw4.8B0@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=39_vYZggKCJA427CyI1808805y.w86527EH-664Fuw4.8B0@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725496290; a=rsa-sha256; cv=none; b=BZlVaJCLBG1M9bW9jK0NwqyQ/oqc2XDD0ih/gLP2cpYZDfxk1iaQk9KOKb7+Oj3aflSMbw +aHY59pHDp0SUEtclTGsBLjdYIuYBvdRh2tRFFIdt2+cLhzlQStJ2Lt4fgzVaa+VOnlV5j cTpCwN8h0gIuhW8y5gS7rxfBMhUN6No= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7d4fb78fe49so269305a12.2 for ; Wed, 04 Sep 2024 17:31:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496313; x=1726101113; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Ac9f2JvhNnpWVv2ZKYegshHvMbbmkBcnbuU73G1cYt8=; b=KQJgVTNSP3dXfMyLA9BpXQRCX1gyUttYMBubbPC70gyVO+r/OrfLIsiottaZpQ/XcJ V/SjVhrBY3TCfR9ebtaLezxmA/TVuj2vqILAVLI7AeCzh1+FThkrim3x4IHX+x9GD/s2 d7GRLYrX6Uci08DtBbKHxBx+404qkitpdH9FUrqoXExn7HpzJ3KbjSLGsrMY24RBipgc j/Tx8+26tq1otlJEdqOdnFYx3ek0UAId+RFc/ynV1w5YKpOvFIvf+VWFlcZMa8bn7nj1 bxPKWizwucp6sW9x/KSoBCHqCEx1OvvcOMvv6V3E5Y7asz1iBQdfD6ZjG3E3Q4Z7m6ni 7ZRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496313; x=1726101113; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Ac9f2JvhNnpWVv2ZKYegshHvMbbmkBcnbuU73G1cYt8=; b=V309YqJQjLIA5vL4R1d4ufrI8nOrYYHoCbA1u+Gb7vMdA6h/aTDhvg/ZjOwxkHg5Ic IvfsQoiNVRj1186YAJbpgSEY45xk2gzZasDW6hciL4DxBIpP3ieQhlCy4pvAdBeZAdnz I3kJdGftOjyA1q4P2j91hDhsPVxvDv4AxDL0dGjfx738cAYf+do9v1NBjXcVO2n6U+zv WwjPmMx6/9jwoGVDLvEkKNLmSbq7vMjX5oi3DWa1ty4lqNZLZtgLGWgEWeJZ6J2rFq5r j4zpzti9JiVoQ2fmibEy5I85bxdNex/d8O2bnr8vWt1WEAf7BAqTVkNcRzv2vdQQF0Gq Ckhw== X-Gm-Message-State: AOJu0YwEOCLYkPWmQBB4THnhRLWPcFYx1uXSKY027q+JPRTQKSl8Vvsa LeQQk6RRVLTSKOCE3e9ke7/C4pPzQs3yir2XlV6JuPIFMhjKswZXkUpNiLdGxXz+iX/mxAD7uJq DwQlaLITBmg== X-Google-Smtp-Source: AGHT+IHuJuEjgofhNfeZ87JSGyMalBhwo29t0H9DuDw9GmIw/cnVXOLyLUXVF5+Ftrg6N3vF7BBT4Qhm6qOSmQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:f303:0:b0:7a1:f561:8c99 with SMTP id 41be03b00d2f7-7d4c1087417mr46719a12.6.1725496311779; Wed, 04 Sep 2024 17:31:51 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:52 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-4-kinseyho@google.com> Subject: [PATCH mm-unstable v4 3/5] mm: increment gen # before restarting traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho X-Rspam-User: X-Stat-Signature: z1j1e9m53dpu6afqy3qawxfq8gapfwor X-Rspamd-Queue-Id: 4E761180007 X-Rspamd-Server: rspam11 X-HE-Tag: 1725496314-692738 X-HE-Meta: U2FsdGVkX1/wXbXC2vcgMt8bK195oITni/tWFe330szvsy9QpBCpFrANqVLF9NkisRzabYocGI/e2BTHe8R0wmgNsmdttdAcn5cOUwcy65MoXIafZgS/gDJ58GE4TYAXYz9NsqGNA7bhV6QJJM84YaDpQcEmdeOOx+uq1EmTMdCszdS+OlKrBDPrS1qDhwd+TL2BYBgJyJXE3ZGp9yuY7VtBY0K+HktPyyKSwqiLLoL6eQMt2V7kbb1yODE77TZdwU3V93c7FqdJ5mt+CerERbC9WtJWzOiJO/AckiGTHdCacoQ11hncODITdTvtSwVjzv/mr1tBP0M8hcRBu8V2clwxYpQpiMtyijMqbYW+WRVW7RveUTllYN0Y+w/H7k6+ldbekj7/t8zSfg0AzcD0I/15s9YgNM4X7Fts8uDdY6DIgP01kTPyBSMhqsxH8lSWJxeJb3igtkGX+xRs5e3JZQAP7795E3DY6zES/xgmgKJyIErUFxB/GyaCJYqkACRpe4FoXI8lwOgT7ZXb89+J6A8OtuXqW4xsqBU8cGZd2niS+H6qaV+C08JsOkFaEaolH1iONJlJD5DH/bqfdE6dHWRWRtZBQ6UYcjw7p2Fq0JWYiBjeS3GUvAkrtahqXvDs5cfbUXZ5ThUEfDXBybJ//FnMHhfTT9WJQXcyXq1l3Q0kNh8ARp7FWdJ8LHANDw2akJD+PzlGpBSrUsdJ1WZoxKxkGIBvLq3qefoC3qpYIqrckVkKIU6MvE+fnPZCcTlxdP5j+fjoA2Kzmv4gGF+k7xRqZJMPO9rG1vXq7yJwUYzEiSC0wPERxO1QcmtIs14ZpKW1FdxcGHalHtXmHtLpWMTrA7bDj2+gcll1sj3KIRIa4jms179wfurfFycT7TF+u3Wn8Kv0FU35xNts6QUOnkuFkxmA6aJFvcq3vRNzI3l+OeCKZYBD5OJggim9a6Duyw+07sQLnFIiImgqybP NUMEvmSC a97Iec/EhiqXUYkuyekGDc97ICyv3fdEldzCAHnAs6d9bGjYsecbFjQQlQ+06lmkxYzfmMEG1gWRYaB9Xt8rAqyOnDGRNHwLEu/XnS+ZbPilD6JJHqOeK5xgP4elu0k19VMpPjjWsmQV/t0D4Iz+b4LxsZ5Ub5DXQYLx+rt/vPmyK1U8r4Z4VBALqo994GMOPsEWjjfI799bUmoC0k5invEFkfM8UnMYGOup5l0yNDcMuf3uDRzbsWhcF1GMErb3C73bA9Xpxr1jikzSn9LEZWp8SGWDKmVEeIKzrpz2bc2T0/NKNRCIcE29lK/+HXn3kQdpduT9cHRgU0Tq6QbVnPE6/jFWcZUfYpBLwZoMLyXsoSgP9WjQDbR1B9uu7zHuUPEuA/MMwd4sH8/z0LKBd6/taBR34SOi3wR9IJBQc1hE9JrJd6GDuTpNGrDe5DVURxjR/6jroVa6VwAEmB7d/ClUAhEnPJBLml+Sw+ECWZB8ZWPlgPTyIvPcn/dV4V0lWcaewGVttUbU7DoQFP1Q0LTtvGw/rqhN5tf23F8g4DHnkoPfDiu/jGAnAs2DfJfVNoqBYGKUhmTHxEpieoDb4OVYP8HkIfEXjPKGLQnyMQ3RRWDRfFEu0VDoKLpmPmw+B0zPQBUO7MmGSJoOsYZIflzo0WV1TMslPRiYea+DlX4SSHUPj1xJZNlgQn/JYajbQrRIrpmqy6poV9v/MXY8H8ex8ftWcliJTKnvn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generation number in struct mem_cgroup_reclaim_iter should be incremented on every round-trip. Currently, it is possible for a concurrent reclaimer to jump in at the end of the hierarchy, causing a traversal restart (resetting the iteration position) without incrementing the generation number. By resetting the position without incrementing the generation, it's possible for another ongoing mem_cgroup_iter() thread to walk the tree twice. Move the traversal restart such that the generation number is incremented before the restart. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutný Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index cdd324812b55..c24ef6a106e0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -999,7 +999,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, root = root_mem_cgroup; rcu_read_lock(); - +restart: if (reclaim) { struct mem_cgroup_per_node *mz; @@ -1026,14 +1026,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, for (;;) { css = css_next_descendant_pre(css, &root->css); if (!css) { - /* - * Reclaimers share the hierarchy walk, and a - * new one might jump in right at the end of - * the hierarchy - make sure they see at least - * one group and restart from the beginning. - */ - if (!prev) - continue; break; } @@ -1056,8 +1048,18 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (!memcg) + if (!memcg) { iter->generation++; + + /* + * Reclaimers share the hierarchy walk, and a + * new one might jump in right at the end of + * the hierarchy - make sure they see at least + * one group and restart from the beginning. + */ + if (!prev) + goto restart; + } } out_unlock: From patchwork Thu Sep 5 00:30:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13791568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F965CD4F4C for ; Thu, 5 Sep 2024 00:32:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC54D6B015C; Wed, 4 Sep 2024 20:31:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4D776B015D; Wed, 4 Sep 2024 20:31:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9534D6B0160; Wed, 4 Sep 2024 20:31:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6FD976B015C for ; Wed, 4 Sep 2024 20:31:58 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 32FD312015F for ; Thu, 5 Sep 2024 00:31:58 +0000 (UTC) X-FDA: 82528807116.10.520A1DB Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf03.hostedemail.com (Postfix) with ESMTP id 5623E2000C for ; Thu, 5 Sep 2024 00:31:56 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="3oNAI/Gs"; spf=pass (imf03.hostedemail.com: domain of 3-vvYZggKCJM75AF1L4B3BB381.zB985AHK-997Ixz7.BE3@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3-vvYZggKCJM75AF1L4B3BB381.zB985AHK-997Ixz7.BE3@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725496219; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C+NIQ3hruAHgC9xMj3fhofnWDiiLf7+Cp7wyaHwiSJ4=; b=CQAX2GSUITgLiWjP71AU2XBYVbi7eULUokHjJDgHJ+XlNUJGaMPCwWJc5AVsbtk1dRfHGn g+0a6azpijY/hmuFwI2LIGQXeOU4Qe298ELL+RWjqdHP6qK7iqS11eMHrPv2phRvEC4D7S Ozr8yV+9Yr6ieH4Y9fEYi9DvHGUox0I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725496219; a=rsa-sha256; cv=none; b=MUxqMbEVrBHzwaIYHfer4Kp0zs8DAKnE2GapWpU0TB0WfO4OEDTHhLHjXRZ/7mTjmgzDcw CnGbz+olmJq/a228hHC4mD/i5PYAbdGumSo63O7m2QPiMIkb/5mE3EXnG2ruPnZ4eu42BH ABE+eQh3rCCBScy8J5sHkILilDwa3S0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="3oNAI/Gs"; spf=pass (imf03.hostedemail.com: domain of 3-vvYZggKCJM75AF1L4B3BB381.zB985AHK-997Ixz7.BE3@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3-vvYZggKCJM75AF1L4B3BB381.zB985AHK-997Ixz7.BE3@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7cd98f27becso285676a12.3 for ; Wed, 04 Sep 2024 17:31:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496315; x=1726101115; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=C+NIQ3hruAHgC9xMj3fhofnWDiiLf7+Cp7wyaHwiSJ4=; b=3oNAI/GsiGZkEU7P1lA0+DrQarS7McVWhwn3I2YN4S4QAPmtUL/BMGfaGOlG2ItnaO yscG5FGMI/jfq7vVdyDtCyLQJabLA7FhCv7TtL8n4b9eB8SpDL1zTtgP//Vd4OKL8rfv bjprAbwrxgGyVSle7wbJOclyepdE4qWfERJ7eyoFqFVVsb9UO4Dq2Gibbg5N4bZL5CQw PRzQzXllEnLYIBZmAuEeDfBvhiMlKdKnzdvNIOybvoPxo+SAwt+a4BvSCHm5DztBDwJZ 0PY5E0xHJKA8SzJRvbGm576Ajt1b9IC9RhGYrjvw9cN2Tg+oZzBz3+U1awgsDx7UvRq0 k0+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496315; x=1726101115; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=C+NIQ3hruAHgC9xMj3fhofnWDiiLf7+Cp7wyaHwiSJ4=; b=vHT+M0VxjvFJxhOZloBCjS7vKowoda+a2Vnz2OvIAb5dD+Y3mtz6ik6m2sgmUMZlFt /GUKF4GArLCr4au62i0oro86l1gz5r3EFAoM5ymn1bZdRTJgvlNLig9aG1I530fnchLL imlzVeHVKMLKTn2PrydOMLNkJGCW2oZpVDpL3Hi4P3zz/8T5Z6TxSlpFflwSP88ApfEF G5BTZzbSTPMzhjTcsNtbEKvQMJh7PhksFJ31wa6tHdvrqlboKOhxrCwEn0vUtRyBtOUX SwMjWB1s1332oZ4RtdOGeaPfxQCi8gXiiltuYXZ+B6Is0kVF9QUFGkziRfwDLMh/XQbb PVAg== X-Gm-Message-State: AOJu0Yxaoe3zWyboJFZ6JUQOJCmZ235qEKlh6EglKKITQ0+mTkLGd+5+ lfFsz9Vw3lIGx0jbnmEnXEI6q+wqeUytsvwHf0T5AobzGjx8Ga4u0vj+UB+6avl+sZNfrKJtYUv hmZg5vb1OYQ== X-Google-Smtp-Source: AGHT+IFE9E8HfCvWJ7p3tjF6dU79+RYxA1wZ2PaXCVQ7+B7BuvDX9ZOeiHESUdZETs6+j99tFDtqjKU0xWP7eQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:1c16:0:b0:7cd:8558:9151 with SMTP id 41be03b00d2f7-7d4c1094218mr69531a12.10.1725496314864; Wed, 04 Sep 2024 17:31:54 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:53 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-5-kinseyho@google.com> Subject: [PATCH mm-unstable v4 4/5] mm: restart if multiple traversals raced From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho , syzbot+e099d407346c45275ce9@syzkaller.appspotmail.com X-Stat-Signature: ijc5jn6sejfd31697uk8yz8e9yzt5qe7 X-Rspamd-Queue-Id: 5623E2000C X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1725496316-742609 X-HE-Meta: U2FsdGVkX1+QJE6uTKr9jJMj5Ab/xUPMH0qsU+7jFa3YfqaO5T4yLEd6krrh+rdSoEY4Mukdz/njHeOEOOxvq1mnoGQA6XQYrhoET9rntD/oIVu0V3hVK6UeEo8vg7FbHNe/5hR7G+HHGk1Xz7rpirfwZOjolRtlEL+9d4ydOI4U+Kw4O1khQSqVIG1K+uKBWd3iO2U0RyKiVLHlC5WIMmFetQnpas4jHomeAQUudDlhcUhNVKIkJ7urDaYkcQj4Huz6Aj+J+oA36xxM4kHFNQlunGq7Da+CvAM3RyuHizSdZhEEc93UzOjlJqPayaJ2rIMUXCMh0oR9kxQqisSwDaIHqmK4xDZz9KvzXPuVzMwWOEwzqF3AC75u0LsAcLKslMKYeDoKofp+/jiAV86Xa7rjwkZfVvfQzBk6xBbHKkdNdSTbKhDI7UAdzVygHeleKjLwee77u2nvzHv1w+iz+nxYhNixtake0ivYgeQqRWN1Sc1Qi23CxE4B5lXyOPhyIWuYbYf7McerFxSTNBdiqFEvuy9f6Yl7SuFoCPZIRtiqV0oSmzpymBDoUriiaV7TvROwQe5slQDR/dmpGnSS+C8V9t6WcfzydPBwbUjM3gVWdmnCZE1WlMwvgsBiKoSiBdyhPE6dEWpOZmZMXznHB4nqnhlU09zbBIs345YtLedMV6U8BKCd3dgrjez5oZsDviP2Gv/isbHM0xsEhZGV6YhTA+r0DAV+aV4jF3vpXfSs+ZMqtY3PiwpCEPLCAk7DLaUIPlT0M6bxDwUVrwUE1ATYrEXlDh91+mkzcHPfwBtNmXpnG2avSA6BJagvswMjhTU+XabsMe03FNi6NLFlLrlwbnfih1gnNiCiCK4SXj26bXNtUeXGgEqCyiSE1g/MXMblUsQ/sbOvLh6K2Q394FbaIlR2Xj3lmor2n+RlDLlABipq7Zcksqp2IOO47/Kp8Mr2s68GjBQw1jOSbA6 DmLHWr9e ZIt4llwA93ixfsxClr4H+ZD+MydQSy+5x9APb8MnHv5LyQWrbfdIBJ0AXJq34y69Hw0Uw5pXVLkJmBccJGN2KmZn0EI1T9db9BJBUpncKKzSdH7NAsarTDvogp50fLFAiA/KSnMr8JwyDI+yuTeyKFmKU98br59kp2fvcksMZjIYVkLyHqZEnOWmlsCYOCm68RLkkBxWMLlQ5IZBn/9wsCGFlWMFJudb8NLWNAuG8RXhMWV8EsU4cNKL3tYXJ7xR6NCITa6oy5Vgv1qiALmj2s4Zw1qtuI+rTcRpYtOKQnEW4W/6mrF6UW2yDEge05IdWM9alb61vDDGueqe0Ssd6vOFXZBqXJVSdqQWckhRhd+C3Y75InQUpGyVpcaUiVeY9JbHFy6aXSX2ZEl4e8P9F2oKKU7BaNI0HX7Hh+zC1+IbZbmLwRGrr0irwnIcmY1dth9WDeEeQ3fFdJ8M0eZc8xcGjb/zWaKFxi+ec0/i1VgoMXtKUr3aDOiNn89c6C9T1DdO17TZTBxl54QzWJnS2XUPwRhPG1iiRcq8YyttJFUPIkZCzHwO0jxIMMBBRHzQvk+HzPoq0B07qdBOZzvkGfte7wL4GnOFAjSkX0vgLIHOKLPPH4n3MRD5LzukQqjMYXLtWRXoTgLjt0KNq1IuCteX/xDPpfbUBMtLrqkKO2u50fSA5IWCQlgFEy+7rXKO68lGJcS1+LMpbM5FIqjEtDlISGSMgeU0zkedvYCeZluqPugIkQ9rUWHo3AhuLayHaKbWvMjx/Jmbv2dV/6NbyuF3hyhV2eh/I9bDQ87+ocRyF0NLs4FI2k2kPUiyi0q55mCPZbXH114peast/btCAsFvBA0a1NB44QfqeHdMwKukFpG/4FWLIe3kA5gbpyOBx4Aa6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, if multiple reclaimers raced on the same position, the reclaimers which detect the race will still reclaim from the same memcg. Instead, the reclaimers which detect the race should move on to the next memcg in the hierarchy. So, in the case where multiple traversals race, jump back to the start of the mem_cgroup_iter() function to find the next memcg in the hierarchy to reclaim from. Reported-by: syzbot+e099d407346c45275ce9@syzkaller.appspotmail.com Closes: https://lore.kernel.org/000000000000817cf10620e20d33@google.com/ Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutný Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 26 +++++++++++++++++--------- 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index fe05fdb92779..2ef94c74847d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -57,7 +57,7 @@ enum memcg_memory_event { struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - unsigned int generation; + int generation; }; #ifdef CONFIG_MEMCG @@ -78,7 +78,7 @@ struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; /* scan generation, increased every round-trip */ - unsigned int generation; + atomic_t generation; }; /* diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c24ef6a106e0..ce5d066393c9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -988,8 +988,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup_reclaim_cookie *reclaim) { struct mem_cgroup_reclaim_iter *iter; - struct cgroup_subsys_state *css = NULL; - struct mem_cgroup *memcg = NULL; + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; struct mem_cgroup *pos = NULL; if (mem_cgroup_disabled()) @@ -1000,19 +1000,23 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, rcu_read_lock(); restart: + memcg = NULL; + if (reclaim) { + int gen; struct mem_cgroup_per_node *mz; mz = root->nodeinfo[reclaim->pgdat->node_id]; iter = &mz->iter; + gen = atomic_read(&iter->generation); /* * On start, join the current reclaim iteration cycle. * Exit when a concurrent walker completes it. */ if (!prev) - reclaim->generation = iter->generation; - else if (reclaim->generation != iter->generation) + reclaim->generation = gen; + else if (reclaim->generation != gen) goto out_unlock; pos = READ_ONCE(iter->position); @@ -1020,8 +1024,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, pos = prev; } - if (pos) - css = &pos->css; + css = pos ? &pos->css : NULL; for (;;) { css = css_next_descendant_pre(css, &root->css); @@ -1035,21 +1038,26 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * and kicking, and don't take an extra reference. */ if (css == &root->css || css_tryget(css)) { - memcg = mem_cgroup_from_css(css); break; } } + memcg = mem_cgroup_from_css(css); + if (reclaim) { /* * The position could have already been updated by a competing * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - (void)cmpxchg(&iter->position, pos, memcg); + if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (css && css != &root->css) + css_put(css); + goto restart; + } if (!memcg) { - iter->generation++; + atomic_inc(&iter->generation); /* * Reclaimers share the hierarchy walk, and a From patchwork Thu Sep 5 00:30:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13791569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 110AFCD4F4E for ; Thu, 5 Sep 2024 00:32:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8439F6B020F; Wed, 4 Sep 2024 20:32:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A6A26B0248; Wed, 4 Sep 2024 20:32:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F76D6B0213; Wed, 4 Sep 2024 20:32:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3D0E26B020D for ; Wed, 4 Sep 2024 20:32:00 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E75CEC06EE for ; Thu, 5 Sep 2024 00:31:59 +0000 (UTC) X-FDA: 82528807158.19.AEC9BC3 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf10.hostedemail.com (Postfix) with ESMTP id 2568CC0007 for ; Thu, 5 Sep 2024 00:31:57 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KElEiLIy; spf=pass (imf10.hostedemail.com: domain of 3_PvYZggKCJU97CH3N6D5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3_PvYZggKCJU97CH3N6D5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725496221; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UFlJk+bdMFXrL5bkGb5l4QnCCI5p7k/K3Pj8oKz1txU=; b=zVyft0Qw+3Z8dyXQh3cAOLgAzdRJqxujhq53BX9BpLSagSpeJBLQsnlBgeoHIaYpWLchQO n/7swMRFVNAjxsHFo9us+tkABf2XumvdgF05C/3VPaK/Cmf0hTCiamBnV4GDw+M6Ls6/tW rZNHy+CxPL231kbMapDkUaEO/29q7xk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725496221; a=rsa-sha256; cv=none; b=W1i8wMfdDF84ch1niEH9AO9lKXWpFuMdvPHv0SA3p2AdlcDGE410disQjTK/gGUWWk8yEg aOlVN++5ZeCCBxEKdmg0QhyAr/MRutBCiXWzO7QK76pXHLfYBToSi5M+2ZMMy37HbYMA5C pbDLxNztCOi7JAXw8IPed/rjCSBk5j0= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KElEiLIy; spf=pass (imf10.hostedemail.com: domain of 3_PvYZggKCJU97CH3N6D5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3_PvYZggKCJU97CH3N6D5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7cf546bbcffso289072a12.1 for ; Wed, 04 Sep 2024 17:31:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725496317; x=1726101117; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=UFlJk+bdMFXrL5bkGb5l4QnCCI5p7k/K3Pj8oKz1txU=; b=KElEiLIy2Aec2eJ7zbFfjGaLdwRMT07BKso9vfDDpXsQCvcWD5NbqtAW1q70bC+/uA TA/kwWojLzGyajB8VqyvfeLSQjdBG+MKHCBCBhTH8554YMCv6wFTwGHx3302HNh/6zAv SVIt9SnOZpzhkTaShMU+yL5yDcFQhHbKtR4K/o+8d/FbgXa+XLCycXkmfRZ2DzYmqCOW XP8eLAYQkad/2e9YubJSckwqh0yWRaNo5qPg6OcW5rYSGyhtfLKlpCPQct+ZTrX/QMFD 4V/J0rh4Z8xl6jWNpIWIGY94eIwfnxzdAb6JpmnK7XUGQWSV5p+O0jL4mM0QzWmflSrF YJBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725496317; x=1726101117; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=UFlJk+bdMFXrL5bkGb5l4QnCCI5p7k/K3Pj8oKz1txU=; b=St2wBUnJqUpjLMYMpOpTF4zpGYW3Y5nodxLV8OLJAmJxKBTnK9x6cAtPk5Df0okMwd CRfWj9ZCGU7mf6XCn3oVQW5BTitGkuRYWRenLeb+YXEJcacUOwpzmzpBDIPAv+tbWflF mKrXqDQR9qBiunyZ7o1T4wde/TlsLViKURO/cHQ8tRavlJgzl1RvaJ0fsj+TsE6BI7Q5 dlIx/PYufVWX6H8zIN6ATLUiO9vytFD82gdHnQX5nLruzAyHIGRexPwN59P9+b5Qjr46 br9qD06y9j7vIxyk0xosNNZUWLeyfceEM/zmOiBa6ob/n84RsWcbmsTilyMkC13NfTgT 3Mpg== X-Gm-Message-State: AOJu0YyGEldm2pjO9mi/pjPb6m4I0lCyE3Qp0eFvBVpXxzqJn2mClaRF Z+2JP9gd6IE4RYF+T3lx3PuGEFkZ376WwWsMu5CdSrnrQv9Cpv+hmfQolUfm2Ti3ru/dQcgGzsa OBkP4h2uiQQ== X-Google-Smtp-Source: AGHT+IFCmyZz1oeuwdzh9ujb+lyKCdb7Lv33IMyUDlAfA9IgJETASTtABlliPQg/ISAiLVSGlddqDM0VzXnGuQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a02:2385:b0:6c4:5b35:c986 with SMTP id 41be03b00d2f7-7d50176fca8mr4225a12.8.1725496316657; Wed, 04 Sep 2024 17:31:56 -0700 (PDT) Date: Thu, 5 Sep 2024 00:30:54 +0000 In-Reply-To: <20240905003058.1859929-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240905003058.1859929-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240905003058.1859929-6-kinseyho@google.com> Subject: [PATCH mm-unstable v4 5/5] mm: clean up mem_cgroup_iter() From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, "T . J . Mercier" , Hugh Dickins , Kinsey Ho X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2568CC0007 X-Stat-Signature: qd5664sjknfaouzf9khfuc9zspo7fksh X-HE-Tag: 1725496317-462057 X-HE-Meta: U2FsdGVkX1//IRTDHhOTGopswsRGZOqGZHwlnRqV2dLV0rMdxWPJNJNCaEpcsJ19UZoqKX4Y1h1x8YIqcWCYoOyPqE2O7yzWBmOcLQ5Ca8Oc9Il9q3Z7Nr0KvrSqDyL6bg22EiNwJV/BAfont5uVUhhZja2psMR/QmWcV8mXZHE0i4Y1y1SHcMqTqc/taKxhSiuZSL7H3W2QqQGG4Q758rFfEEXaVPnjKjqFue977NKmnHaoWNGa7s36/7gD1arslWp+a0/n2/aNV5NLiiQ8JCAEzXEUrmkYo2998V1NZiBhnlhc4+E723S0AkAX6eoadFQKAr8zDIcRR/15DZZVgxbkhoTmBNQQ8ob5xx3RH+c4XsSFZIH3CxVMnc6BdPG5mRCo2DgNXw/yH+rBF9MWJP3jLVL/tuOauetxNkuaNY6aY61Qe7imBGPr7yy/Nhipsigm3tILsIjh30cjYa46yjBCy+WEzKgyEDD/pe7y7lmBNeYAN1mk9Q6NHs3oGVpEXWD0NCuWBzD8JEJvCh5KJDT73jKPlLEjRFeFp9iZcF+DOIknanzwNVs4pKWER1p9+PaaUDt15gqjMDLPiv5eYQx63K3g16dFp1RARgMzmzH+YpBL+fXVblB82UjSpNG3IcNRD0b1EtDg+UiUv3ITIm+hHA1i0+GFw7NOEUFUasp+/IfQh9f+fILvj4vDfRMlPU4qnQ/UzIimzoQD5NuF/cIlIjnsMlTtAGkVs+JB/fVEZHBa6o2GVwZwJecW8U+o0Su868xQWqipbaSFeyNjwQ0pXpAckypKChB0j+8/aIvE+Kt7zXwt+gtY9N68f1DPTQg6TF2+CQrxZmTabr+/qC0MGUc+uqKS5lcOHQhnp1BeDTn0+KuF64Ncj07NQRXQcc8YcS6XfLXhFOOWONT78QxRPTeIQxnfjrOjXlM9mGJU0dEmQNNszwEvFE5YmZ/ku3N2o/pHhj7KJgLTQqf ek15fEf+ I8w2kY4zYWal3UwEU++F10HFkuz+x301OFOCqA2kwYVG4nTutgSPjX2I+OTiqwG4Gi25aYncONxDVMOuyjK1I3oE9qLU8l0Sb9NPXOLgZcxl4fQbhQBhfieyuBQJ8xSlX52/YhE7mMAMLRIXCqvBIIPhMGacVSGcP2cgc5f6h5DLna1CziOnXbpW0JcmMOnyx3UnDCtbknIuibkEiI8ErNUadCxY4gQl0Oj2xZkTsZAPpnDWk/G+cl4tzzRKxkNKwOFYAbjGO/fR4VXqDCAJqCiVHDm3PoSiMxKRUdH0D2nALZ8JCfeguc4GdOyI83sOvSPzOiwVea5IgsqyTARAbCRe38tmuE7z9YJ1L/EJNdMnC7pBgpfVIQwX54bu580rAeVJXl0H2X38sRwvdgVgH1DFQTR3+44MsVtWnHDCH7a4XVG+E+xlhzzszfY3bhQY19HcDnHsMQmn/8HxqWYpZjsOlmLErhVroATOezA53iZlhJfCdPvYggTvH1VZc3PJKxIiRnWCkviAviJmBxarm2xoIfPtXvdQws5p3cGFkzJHisoJRDHO9k2b6+c3/uUEp5bpX5odWUQmtKWN+XlLyhY6G566s3zx2tbSMQm7mAB98y96anTRJ+eYLxpiey5LZHxDUftJ7E6j1HR1lt61oHWk8NE509enLq3vkyA8tvaWD0wAL+PxfT1h/rz/SD95WIx4phZJXzBHBxeVmtB3gD2Nt55zzs3ZmUHob874K3TyB2a0J5rOOwPoY1XazybyN3foBTZNssOatyCM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A clean up to make variable names more clear and to improve code readability. No functional change. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Cc: Johannes Weiner Cc: Michal Hocko Cc: Michal Koutný Cc: Muchun Song Cc: Roman Gushchin Cc: Shakeel Butt Cc: Tejun Heo Cc: Yosry Ahmed Cc: Zefan Li Cc: Hugh Dickins --- mm/memcontrol.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ce5d066393c9..31db8467f19d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -989,8 +989,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css; - struct mem_cgroup *memcg; - struct mem_cgroup *pos = NULL; + struct mem_cgroup *pos; + struct mem_cgroup *next; if (mem_cgroup_disabled()) return NULL; @@ -1000,14 +1000,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, rcu_read_lock(); restart: - memcg = NULL; + next = NULL; if (reclaim) { int gen; - struct mem_cgroup_per_node *mz; + int nid = reclaim->pgdat->node_id; - mz = root->nodeinfo[reclaim->pgdat->node_id]; - iter = &mz->iter; + iter = &root->nodeinfo[nid]->iter; gen = atomic_read(&iter->generation); /* @@ -1020,29 +1019,22 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, goto out_unlock; pos = READ_ONCE(iter->position); - } else if (prev) { + } else pos = prev; - } css = pos ? &pos->css : NULL; - for (;;) { - css = css_next_descendant_pre(css, &root->css); - if (!css) { - break; - } - + while ((css = css_next_descendant_pre(css, &root->css))) { /* * Verify the css and acquire a reference. The root * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || css_tryget(css)) break; - } } - memcg = mem_cgroup_from_css(css); + next = mem_cgroup_from_css(css); if (reclaim) { /* @@ -1050,13 +1042,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (cmpxchg(&iter->position, pos, next) != pos) { if (css && css != &root->css) css_put(css); goto restart; } - if (!memcg) { + if (!next) { atomic_inc(&iter->generation); /* @@ -1075,7 +1067,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, if (prev && prev != root) css_put(&prev->css); - return memcg; + return next; } /**