From patchwork Tue Apr 15 02:45:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 14051366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C62DFC369B4 for ; Tue, 15 Apr 2025 02:47:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50C432801BD; Mon, 14 Apr 2025 22:47:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 494E72800C2; Mon, 14 Apr 2025 22:47:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C0F42801BD; Mon, 14 Apr 2025 22:47:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 07FD82800C2 for ; Mon, 14 Apr 2025 22:47:08 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B5F8A140B7B for ; Tue, 15 Apr 2025 02:47:08 +0000 (UTC) X-FDA: 83334741336.19.A7F3545 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf10.hostedemail.com (Postfix) with ESMTP id D8557C0007 for ; Tue, 15 Apr 2025 02:47:06 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=A5XQwmYI; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf10.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744685226; a=rsa-sha256; cv=none; b=XN6Yjy0MZANtFTjIEjar1lN6Q/NMoI7srFSxDkhTF+U+/Pcu+sE1lN6BL/k4dj/UJG92m1 VKqbxJbC/pGLhrn39i4L8y8iTQjOpVjnQvvZpq0xhSXB7IKfFyMOFDYExz7rzKDIBkQ77U bNQKEqg5G1PVAmbTk2XFuqlx7r6M3EA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=A5XQwmYI; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf10.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744685226; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=obht/6Kk0NnhGpMOiil8TzYARUGhrq+7EhlNkpnMaGs=; b=CxqTIW8L+mU9IZao1XFJFJdWHv7rGthrML7JylSbXOzPf+KMrNAiD6HfYiCQ1JGYd/Zcg5 xMR56qvVryadxQwRSkJbSdLWJENG/aaVzKzuPlduE5S7N8cFeZD6P6G8lmhERLMPNu+KFg w7SB6j7U8y0i1r6n1vzLCQvLQi9TaVM= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-227914acd20so48110905ad.1 for ; Mon, 14 Apr 2025 19:47:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1744685226; x=1745290026; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=obht/6Kk0NnhGpMOiil8TzYARUGhrq+7EhlNkpnMaGs=; b=A5XQwmYIVd6L1YQH5Q7TTMiw74CF73O5kNQWGlpZylrZSsxXo/e102Y6iwo2wSPCVA B45av6dYNoDsuGvEPRcE3QUBtdRcAZVjWwcUbjR26Y5iO5001GUc4VVCcqJuje4WWcwL OPrlI23802Jc/Rt8MxLefyNJF3eTCPzREa20XSkOqsAsxleh73CE9AKLUAUJH5SYMXaf V3AGWrHKkIoB8BBtimRPzGSdQXdunvvX9oQEECFwDzpicHbPckdr+yOJqFK+LEeIjJtz pgOOtSXB9ox4FzRO7en33H0YH3cyPcKjpPiXPFGFiUAaXlwoufmKf284li9oapzmVbZM mA8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744685226; x=1745290026; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=obht/6Kk0NnhGpMOiil8TzYARUGhrq+7EhlNkpnMaGs=; b=iKBLDYSiqqN5ONwwvAANE2rJ74wJKnY76AGdMK+nf9Dh3NeyDKiA9AkY5Rke8Ad7u0 NvPwaAJL7TQr/a4+Pi1+/m4PAdMY3R3Y6kAddWwM1+xKJrHxMCVf2gEfmUmrnz5ruICs qxhB5XnwVHDjQwBk0IKIs6qh2j/ZhSOrlOzh9OYPfhUnhQ/VFZ7Fq8pn3vR+9rGLQ5Y+ lwAZP3ncbr2F4Wnpj/bJyJrhelnwmG74MAimsuA0h0fdZhGHy9E9o+lGzzVqoHqwPNqO t00mTvJwGbVfZAWRJmRf0UMPjZW3Yv/1Ydx8q5KXjHZoYy6bBqzyTsxKnP4uV+9HUfDk AIUg== X-Forwarded-Encrypted: i=1; AJvYcCW/JGDJ1LXHmKN7B7TOv2vWvj7n0IB6QwBmeR1BB0Zgqs1Cp8tBKuBvlMuqt64NM2CFDJUv4c3nMw==@kvack.org X-Gm-Message-State: AOJu0YwITwS6Nx/LKJpepLJBT8WIqAPHpRybn562fuRJpulFC+DckxF1 fG3GlZ/Ce6Ce7X+fh4+xVzmDem5IV9h9X4UJSOyTFtpaFpOR5WCO/oA0AmN9sqQ= X-Gm-Gg: ASbGncsxy4CCbsPI1vYsEMivn/Q7o95yoMakDZ2kYTLPagxpAD0/096k+EOwwfoukdY +jaMVi6kznV5qtoiWtbRmY1p71+b8W4HhAm7JmyipOqgcOWE++S/qqBFf/Qt1FcEgF8UykYKDyE VvG26Dfu8CJLgZrHIVwvhHA9PteFnDgHjVkpk+fL1SH1CYF0skBf3Kmsup7FVU8/U1FpCStksfc VYehFEkgqtdyl1LRl+ZQtd5cKWdlzekyk7z4wJu/xDOqPe3180nxSkZjg1ITSszaTHLsPRSzEZV M85EhmBd9MKFcZXgyj54ctad+7W0GbBF6UPX2zraspDG5opf9Ipy2zUIgM+kUKSofIitKDQ1dPa kjb5IJro= X-Google-Smtp-Source: AGHT+IF1G++5FxihLJJNREqKO+5yf6OFv8YN+5YM0gPXaE7ee12PkODsP2jTee4zjVH6VXKddsDutQ== X-Received: by 2002:a17:902:d490:b0:226:3392:3704 with SMTP id d9443c01a7336-22c24984d71mr24086475ad.12.1744685225787; Mon, 14 Apr 2025 19:47:05 -0700 (PDT) Received: from PXLDJ45XCM.bytedance.net ([61.213.176.5]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22ac7ccac49sm106681185ad.217.2025.04.14.19.46.59 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 14 Apr 2025 19:47:05 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, akpm@linux-foundation.org, david@fromorbit.com, zhengqi.arch@bytedance.com, yosry.ahmed@linux.dev, nphamcs@gmail.com, chengming.zhou@linux.dev Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, hamzamahfooz@linux.microsoft.com, apais@linux.microsoft.com, Muchun Song Subject: [PATCH RFC 13/28] writeback: prevent memory cgroup release in writeback module Date: Tue, 15 Apr 2025 10:45:17 +0800 Message-Id: <20250415024532.26632-14-songmuchun@bytedance.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250415024532.26632-1-songmuchun@bytedance.com> References: <20250415024532.26632-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: D8557C0007 X-Rspamd-Server: rspam04 X-Stat-Signature: djf71ed1e94urogazaiucagf9wqknf8o X-HE-Tag: 1744685226-690658 X-HE-Meta: U2FsdGVkX18mKZsf3yHwr7WvjDmvSmcGLr/xGysJix6UXVTxbzaqbINstNWyC4BlrAdf9W7eqQ1wOgravqbauwuWPHNixFMQAxCZ6nm1RqFeFKcmjqA122q/ZqJwxhQtXH9hm5iq6jMFWdA+ok4RPsQQ/4slUTRRSNoLWRyxAtc8Hyshz4YuI7hDX8QvNzZR6OjRyAHUtOlPNLrKVsyS2xJu9UoXPIS0V9Rdr5PmjVwDITPWgHRJtn70bLf0OGORRBB3zzyUb0LZRSeTNnVPWcvNAsQpYx11dxDhY+lFpM3x7TSwz8gdvE+TpAnr5/B4nDZHvxN8x8L/GUj03rEhxBcPvJtzZ6uW5Jkw3slCPFO+9E3Adi8sZ3QJHhrm2l/7IgnHL2/0Pct2EWyBegR1nEbAgdmORKNYEZ3iOgtlzbIEhnGg9CfJbb2g0Zw/vOYbynjXv5Nb22lRehuSwawNOmMDgUfIUqoUftjjaDB+EQUMD6o801yjLT44nPL3D7MJlA5Yqi1hvtKdLXPIyB7GqFQr2PJzmnmyKBagVhg5rLrrFhw3VmrIvu2dtijuxFM27LfzpkDLVNsp5Df8+XMUaESKuX2Y6DzJsGKoNLqOo5aLTAX40VauKUjTTfYorBLGrYpsZXuWjqFgzYf104I6GPgUD1vgxlfxwsaH4hDfx8PtGkTEF+16IYjnQfL+ApQt45R1/wOETZSPbjNI/qwa3kUp9/kuqW2B1ropzpWALd0Pw/+qEiDEbOu7ky5W5oBOgHSrVV7zrz+uFJBUk+pcZSPT+ZxmpXUnVD9R8pZ+iW9lgiffmJTX4ccPvQO5V+l8FpBoL17O5AbKGRhFqlAPwl6P/2XMYzujKgiKUCoODaM8dLlbYuy3T1cJmqL02U/imKbfCLDWfLvs66amZIa/foOq9UFaatF4XF09b6vo9mU19uecXdttuvMiZLJEyV6/QEtFB6uIjG5imjyY8Nt F6UfUW8m k1fxpmO7ox66AhUuFd8evI6/KDmii7aE2ckrsCDlStxump4uE7zxyfqbhxXs339YcHVjwzkE9sd0aeixL+5w8tYPBmW56ftAOiimz93yH5r9auKm4ta+1l7nT4+U+ORT1cEJrpmw/l1aITSaaTwnPQFh1BPYd1dX54TXFbXltCyfU+ct361pr1YS+kThd+um+nF9lrb4WtoRU2eeGqdb6iSWceWCvZwFMzn6EEc2qLZYTBNiMt7JZaIo0cbZHBidcc8q5xxxku6JG8JRiVNYImBYk7jW4ln2zsMNYMTJPWrDOj+zCg3vdboKzvA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the near future, a folio will no longer pin its corresponding memory cgroup. To ensure safety, it will only be appropriate to hold the rcu read lock or acquire a reference to the memory cgroup returned by folio_memcg(), thereby preventing it from being released. In the current patch, the function get_mem_cgroup_css_from_folio() and the rcu read lock are employed to safeguard against the release of the memory cgroup. This serves as a preparatory measure for the reparenting of the LRU pages. Signed-off-by: Muchun Song --- fs/fs-writeback.c | 22 +++++++++++----------- include/linux/memcontrol.h | 9 +++++++-- include/trace/events/writeback.h | 3 +++ mm/memcontrol.c | 14 ++++++++------ 4 files changed, 29 insertions(+), 19 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index cc57367fb641..e3561d486bdb 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -269,15 +269,13 @@ void __inode_attach_wb(struct inode *inode, struct folio *folio) if (inode_cgwb_enabled(inode)) { struct cgroup_subsys_state *memcg_css; - if (folio) { - memcg_css = mem_cgroup_css_from_folio(folio); - wb = wb_get_create(bdi, memcg_css, GFP_ATOMIC); - } else { - /* must pin memcg_css, see wb_get_create() */ + /* must pin memcg_css, see wb_get_create() */ + if (folio) + memcg_css = get_mem_cgroup_css_from_folio(folio); + else memcg_css = task_get_css(current, memory_cgrp_id); - wb = wb_get_create(bdi, memcg_css, GFP_ATOMIC); - css_put(memcg_css); - } + wb = wb_get_create(bdi, memcg_css, GFP_ATOMIC); + css_put(memcg_css); } if (!wb) @@ -929,16 +927,16 @@ void wbc_account_cgroup_owner(struct writeback_control *wbc, struct folio *folio if (!wbc->wb || wbc->no_cgroup_owner) return; - css = mem_cgroup_css_from_folio(folio); + css = get_mem_cgroup_css_from_folio(folio); /* dead cgroups shouldn't contribute to inode ownership arbitration */ if (!(css->flags & CSS_ONLINE)) - return; + goto out; id = css->id; if (id == wbc->wb_id) { wbc->wb_bytes += bytes; - return; + goto out; } if (id == wbc->wb_lcand_id) @@ -951,6 +949,8 @@ void wbc_account_cgroup_owner(struct writeback_control *wbc, struct folio *folio wbc->wb_tcand_bytes += bytes; else wbc->wb_tcand_bytes -= min(bytes, wbc->wb_tcand_bytes); +out: + css_put(css); } EXPORT_SYMBOL_GPL(wbc_account_cgroup_owner); diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e74922d5755d..a9ef2087c735 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -874,7 +874,7 @@ static inline bool mm_match_cgroup(struct mm_struct *mm, return match; } -struct cgroup_subsys_state *mem_cgroup_css_from_folio(struct folio *folio); +struct cgroup_subsys_state *get_mem_cgroup_css_from_folio(struct folio *folio); ino_t page_cgroup_ino(struct page *page); static inline bool mem_cgroup_online(struct mem_cgroup *memcg) @@ -1594,9 +1594,14 @@ static inline void mem_cgroup_track_foreign_dirty(struct folio *folio, if (mem_cgroup_disabled()) return; + if (!folio_memcg_charged(folio)) + return; + + rcu_read_lock(); memcg = folio_memcg(folio); - if (unlikely(memcg && &memcg->css != wb->memcg_css)) + if (unlikely(&memcg->css != wb->memcg_css)) mem_cgroup_track_foreign_dirty_slowpath(folio, wb); + rcu_read_unlock(); } void mem_cgroup_flush_foreign(struct bdi_writeback *wb); diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h index 0ff388131fc9..99665c79856b 100644 --- a/include/trace/events/writeback.h +++ b/include/trace/events/writeback.h @@ -266,7 +266,10 @@ TRACE_EVENT(track_foreign_dirty, __entry->ino = inode ? inode->i_ino : 0; __entry->memcg_id = wb->memcg_css->id; __entry->cgroup_ino = __trace_wb_assign_cgroup(wb); + + rcu_read_lock(); __entry->page_cgroup_ino = cgroup_ino(folio_memcg(folio)->css.cgroup); + rcu_read_unlock(); ), TP_printk("bdi %s[%llu]: ino=%lu memcg_id=%u cgroup_ino=%lu page_cgroup_ino=%lu", diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4802ce1f49a4..09ecb5cb78f2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -229,7 +229,7 @@ DEFINE_STATIC_KEY_FALSE(memcg_bpf_enabled_key); EXPORT_SYMBOL(memcg_bpf_enabled_key); /** - * mem_cgroup_css_from_folio - css of the memcg associated with a folio + * get_mem_cgroup_css_from_folio - acquire a css of the memcg associated with a folio * @folio: folio of interest * * If memcg is bound to the default hierarchy, css of the memcg associated @@ -239,14 +239,16 @@ EXPORT_SYMBOL(memcg_bpf_enabled_key); * If memcg is bound to a traditional hierarchy, the css of root_mem_cgroup * is returned. */ -struct cgroup_subsys_state *mem_cgroup_css_from_folio(struct folio *folio) +struct cgroup_subsys_state *get_mem_cgroup_css_from_folio(struct folio *folio) { - struct mem_cgroup *memcg = folio_memcg(folio); + struct mem_cgroup *memcg; - if (!memcg || !cgroup_subsys_on_dfl(memory_cgrp_subsys)) - memcg = root_mem_cgroup; + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return &root_mem_cgroup->css; - return &memcg->css; + memcg = get_mem_cgroup_from_folio(folio); + + return memcg ? &memcg->css : &root_mem_cgroup->css; } /**