From patchwork Fri Mar 14 13:36:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14016923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AD80C35FF1 for ; Fri, 14 Mar 2025 13:36:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72964280008; Fri, 14 Mar 2025 09:36:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D79F280006; Fri, 14 Mar 2025 09:36:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57914280008; Fri, 14 Mar 2025 09:36:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 13EAB280006 for ; Fri, 14 Mar 2025 09:36:31 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4C755564EB for ; Fri, 14 Mar 2025 13:36:32 +0000 (UTC) X-FDA: 83220256224.15.771792D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 0879A180007 for ; Fri, 14 Mar 2025 13:36:29 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=rYbFLIF9; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741959390; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8CF6B/ZWkNgJvmh6tO93Ml9BwZtYmnZEFRYcylNCRL0=; b=H7zIlL+dfaNadr1STefahnwxaNKcivkT785TLg9A59KJCH3vFeZjt0/Z83M5w2N5fkcYlE 0iDKu0t7mVoEVGOdhvXUhnkVcZgaDRgc6gTkjfjtcH4nEpfD5DvnJ4Le9kaIZlyylYdGpk wVHXziscyQaG16gcEkY5pr9ZNeYeJy0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=rYbFLIF9; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741959390; a=rsa-sha256; cv=none; b=0hoGch3mCXvR/+nETKstssp9bxevL+XNHh4PxSG0wmjFVQC8AOlPEkVp7VYBiRLtcSujA/ jxCmppPAEK5UEG0mRCUAw2jlxWzWw+tFNPjnJLCHh382g/9/1IrpebCRdbIIqk8KjSgDTn HIXN6WDvdyYlUvvVCfjxwLNUb5RiihQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=8CF6B/ZWkNgJvmh6tO93Ml9BwZtYmnZEFRYcylNCRL0=; b=rYbFLIF9gocOedOBWgEnvFJo75 oBqukbfwld0TuH/zzoIJXwCkQmUJ+gIzoSndpEWeGTT/ilfhlWianlya3Fb+pxeNJmur8iRauZP9p Hu/9GvB0IUtbPgrBddNkB+YcOWBhjXVlpgpuTrhcqhFM/+DQzYqTYW95+93/HSfjwT6Htm8d4qfCK MlB3wsXKfHdqGp6/GYPLj3xlYvXX8wIjjWhJB9PiUEWBQ6avrP7NNLL3R36iP69x6iWgcLm0xX0h5 XeuokmYVKSl+vow439lG9YNA5+4gbTrsJlPe6CuL0EQefwUHFNbmCjJ675fyyLF3pMsizfkeEV+UK FehsJTKA==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tt5D4-00000000ZxS-3QiI; Fri, 14 Mar 2025 13:36:23 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Zi Yan , David Hildenbrand Subject: [PATCH v2 1/5] mm: Separate folio_split_memcg_refs() from split_page_memcg() Date: Fri, 14 Mar 2025 13:36:11 +0000 Message-ID: <20250314133617.138071-2-willy@infradead.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250314133617.138071-1-willy@infradead.org> References: <20250314133617.138071-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: rrj3dzybnsff7u8k7hegcxczstk6oct7 X-Rspamd-Queue-Id: 0879A180007 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1741959389-648711 X-HE-Meta: U2FsdGVkX1+Q6XOoxuHn2JpSohZE84GXcGW9l68D0yVqaxKtC5JnpsDVBUyZhz9ASs48VtvRDI1msuv13e82qdfOlwi+0URlqIMbZViKmZeuHo7VZqdqlsY4woBdq7nsbkXICxwPIS5xsRPOeBS0isMo4gQeZI/zxrWQEUGn2NyXilwa/DC+05F8VRpGQRs1LIy/mjSzSGTLbbnuXrucK91x/7/RYaLgGZvci945rVaG36YVh65LjARtj07PorIot2Ix5tI7+oiuZRALD/sIOyCGiJzdQtnvcQt2wMFsQF4nvFA9xqOTqUsxAeHwZIOm7or9mldxCBGpwteSoTaNzlDekZ4F6lhdCbEPVO1Ek3UTpMPJs3OBzXrpaSAud2Wa5/ruzFxTJeateM0RhcF4FZxIEZpKb2Zh6L44kACRo4yvsIM4Gd2OEdP3q+CZrtvxHhuGNVHUVZthyO//hN+Tv2E4FAtdkk3wgnDag7bP2dPVGXJPpUTEJR9zC54NCmTjf5qnMUIa9uNjjfHiDdDSCRP+ab29ZR2A6v2mOYcbn4nUcsrNH7xgW6livNJr5ChW1ODGk/0UY5ORoImn6Zk+Qagx6KQmo7wlkIue9j/npxdyaETG74BsR9u0CfDS4mipYFG2YfY6vLi73imd+2swqVu/LY1GBShmfpkPcf4N/frx/EUPHFKl560m21WZZXirQ/253/MCAMpAKC26b6Mu3RZTudXIWRYPCh9nxiEPMI8zI2NtzGixx3McTb+RZcHIoO1bx07JYC38jGwFBah4Xk64Vx9zoTGXi4nXEurjS+6JIGsZ/GWEZhva7q7hQoP4F479PHIEEKxydTqurx1YcXrFlFaZp6K20/qH9AxuctikqL/iRoxH+5Djw4AnCX44k14mhhl6mm/nxPUoEcAzQQBhWZIBtLQxJ8f+lMInZowCPu5araIaqqv8dkdxJ//ulWjiCF07Dm7OfOuCejU nj/N5TUj ynm/GiVL1DZ04bypCPIRI9o1N7tfQcwV3uTK4MOkSKsJiDKrYWBKkS8IhtNnQeKZtE2DQSHr64e1ZpDOZiPynMIxopGUrlM6un3DmkmPiu+DWdlbRhM6Qq/EmR+MghJZomA3iakQ+Aqb165mDmUERbWjvAHFh53SiFFKbtA9F9luhZfe7S+q4yvYSsZlJBWgtJpT531MSbx1AIAvt+GQ1NagcMyG94HSOQfJBBDp6oZ+gVUi5UuBAIJCBHDu5D9NJLRzL63cRrQm/TX84zhOt75kXimrVB78MCFSOUm1IfqFVpIkATvWp6x9QJR9zGUfBuC6o9lNEj9BwZFY8f8fUB+e0cW85BpfSCz2eO7J508EP5wAVDknAAVOZsQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Folios always use memcg_data to refer to the mem_cgroup while pages allocated with GFP_ACCOUNT have a pointer to the obj_cgroup. Since the caller already knows what it has, split the function into two and then we don't need to check. Move the assignment of split folio memcg_data to the point where we set up the other parts of the new folio. That leaves folio_split_memcg_refs() just handling the memcg accounting. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Zi Yan Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 7 +++++++ mm/huge_memory.c | 16 ++++------------ mm/memcontrol.c | 17 +++++++++++++---- 3 files changed, 24 insertions(+), 16 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 57664e2a8fb7..d090089c5497 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1039,6 +1039,8 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, } void split_page_memcg(struct page *head, int old_order, int new_order); +void folio_split_memcg_refs(struct folio *folio, unsigned old_order, + unsigned new_order); static inline u64 cgroup_id_from_mm(struct mm_struct *mm) { @@ -1463,6 +1465,11 @@ static inline void split_page_memcg(struct page *head, int old_order, int new_or { } +static inline void folio_split_memcg_refs(struct folio *folio, + unsigned old_order, unsigned new_order) +{ +} + static inline u64 cgroup_id_from_mm(struct mm_struct *mm) { return 0; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 14b1963898a7..3e5ecc8f3d13 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3394,6 +3394,9 @@ static void __split_folio_to_order(struct folio *folio, int old_order, folio_set_young(new_folio); if (folio_test_idle(folio)) folio_set_idle(new_folio); +#ifdef CONFIG_MEMCG + new_folio->memcg_data = folio->memcg_data; +#endif folio_xchg_last_cpupid(new_folio, folio_last_cpupid(folio)); } @@ -3525,18 +3528,7 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, } } - /* - * Reset any memcg data overlay in the tail pages. - * folio_nr_pages() is unreliable until prep_compound_page() - * was called again. - */ -#ifdef NR_PAGES_IN_LARGE_FOLIO - folio->_nr_pages = 0; -#endif - - - /* complete memcg works before add pages to LRU */ - split_page_memcg(&folio->page, old_order, split_order); + folio_split_memcg_refs(folio, old_order, split_order); split_page_owner(&folio->page, old_order, split_order); pgalloc_tag_split(folio, old_order, split_order); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 87544df4c3b8..4674d9815a50 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3101,10 +3101,19 @@ void split_page_memcg(struct page *head, int old_order, int new_order) for (i = new_nr; i < old_nr; i += new_nr) folio_page(folio, i)->memcg_data = folio->memcg_data; - if (folio_memcg_kmem(folio)) - obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1); - else - css_get_many(&folio_memcg(folio)->css, old_nr / new_nr - 1); + obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1); +} + +void folio_split_memcg_refs(struct folio *folio, unsigned old_order, + unsigned new_order) +{ + unsigned new_refs; + + if (mem_cgroup_disabled() || !folio_memcg_charged(folio)) + return; + + new_refs = (1 << (old_order - new_order)) - 1; + css_get_many(&__folio_memcg(folio)->css, new_refs); } unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) From patchwork Fri Mar 14 13:36:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14016922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7B20C28B30 for ; Fri, 14 Mar 2025 13:36:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41C37280001; Fri, 14 Mar 2025 09:36:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 385F1280008; Fri, 14 Mar 2025 09:36:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 076BE280001; Fri, 14 Mar 2025 09:36:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D72F8280006 for ; Fri, 14 Mar 2025 09:36:29 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 576231CB546 for ; Fri, 14 Mar 2025 13:36:31 +0000 (UTC) X-FDA: 83220256182.11.8406DB5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 6A3281A0009 for ; Fri, 14 Mar 2025 13:36:27 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NerymVeb; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741959388; a=rsa-sha256; cv=none; b=7PShfI2IlwICNaPLeeKVwawtEYZZNKAQVvcRBxxjN/kfYrtmydYGCSUoMTFwe57c2+hidc mWUsgHZ26vjniEhwB/e39rRWYbJhJ7K2g0dYsyfKIA0Yw/k7YBdP/LROLZVS6oV4UwzwXh PwpiTrS/5vV0431HwRSBM0Z7i7WYgsA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NerymVeb; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741959388; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RoGScLnKmzY91Vb9KOdHkE6QsUyf1AloqQq4Cbq7VXo=; b=kodePDPbytaHnyv8nQVXxZQD4fHHykndFoApCXamFMVO6IFFSSwo1VdpXg+FiOQ0MvGZhx JN2mRLO/xFWjDFwmP8Theh4K/cP9Iewl2lnswp1OeNbA7fgK1aL9AHt7gR4IxIKiaSJYZo /hJNNZKW+MtYZOF0390L87ds2lHGz58= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RoGScLnKmzY91Vb9KOdHkE6QsUyf1AloqQq4Cbq7VXo=; b=NerymVeb1O+wO8MaaZvFSdeOT+ AluiXyWEZqVFRy/pjk/oGPu2uDyXF07Lf6y0mNttEAEZxf4/GHkaVyXALRPa9WsDh87Ex+uBueiPA 80wnDArYVzFdgG+/tMPRMJFs8zuuZwmjfPJWBx4Cgb770Jkws78b2PiPffD4lMcsLsRHJ4nF1KBVi Ba99l3c3YBeY87uGN/ylG/rwwhhzSFCSov/c9+laLBfkJxQzzXNY2ObTuhM1rWS6isiyf+5i4bhxu g/wOgNmdC4kzPBNEDytHj18izY1KoZ7Ra+qjum26UhjzAJLnDUWNsiInUGY6AnDNsf60tOt+w6Z/4 Qrsx67cw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tt5D6-00000000Zxr-0Obc; Fri, 14 Mar 2025 13:36:24 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Zi Yan , David Hildenbrand Subject: [PATCH v2 2/5] mm: Simplify split_page_memcg() Date: Fri, 14 Mar 2025 13:36:12 +0000 Message-ID: <20250314133617.138071-3-willy@infradead.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250314133617.138071-1-willy@infradead.org> References: <20250314133617.138071-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: jubh469tko7fzgo4hw9ga5wy5644ycf5 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6A3281A0009 X-Rspam-User: X-HE-Tag: 1741959387-987389 X-HE-Meta: U2FsdGVkX1/QYU0njY5hoL24nOwnrGMnUO8BAVk73EvJ4U7h4MqNPVuxfKwTFgvVKR1rbOeEburfDjeTLwtiaOStRDtJPtTThSv4u83poN0muzdzN8AwImXon9xm+AEa3keuUgrJp3/3FCPFU/wMROI7AOYdnoJLsXtFbd2VpMoMJJ4PcNvokEHmB1WMrHtjdIOUhGw+tAS8G14GS0yLoYZZzzv+pFecuLXJN3fro3Pn31lONTQOxJpgb4pMQnCUIR5T+2KBYQ9oT5siz5M3IANxAelRkV02R/JDwdwQwEZjRHMgcqpKl6naj+kYgYivZP11aIfPba0PI12svUgm2OkEtPMdiEfEK7vW3AlX34y7rnoVDPmdIH2NYSE4sLnX9nHNv49fD5EGq0GXn4l91lYogzhvu7wcXexwz4g6Tu9YhAqbgSScirWzU4A9BmAL/C7tGfez5iJoRQI0NFxn3i5e2lnLTCknA7P8mLzm5TWTZkbcjP0XPVBYNrSbq8nYCjGhmrHqcOgVhV2eZts43Xn5xEORdorqmZUtcNtYG6/sqGIKn0RwjDVGuGbLyvDBASgEkTdXr10vTyu/AHZfQpvrEjSEoezhJJK/SBWvDVd1Tkq/qtlptIA3vgkjiSN8ZwmwlTnYTmPLY9st9k3zbds/8KD57ZsIkOUiV3+OP5KVDZhC7+84kGdO1rVxhxJ7eCfKTxuhm/iVA+WgHwpmpCbx8rD+B569um7Gb+XwHjhq4cCSE6s2N+7PxIlVdULao1jbMS1ahKIRXr/wv6adcw+eOJbrUCgek3+WU2xvoJgYYju4NsrmM8zzhi6CijvLtp9jh04tdc0C/qbI0e9g4V+t46pWqkwnkRM6KseUWigoO3SyXBumgfK8uun3JPWdSa78/08cMonBoxC5NxielceBV8uLpsDwOAh+pOMaHppHSFcNy6y3/eD/PC6CZ/ntg91HvgLDmS1VkmSsLNK jr8WcqeM EWZbL/1izxpC8tOOrqIEundYGyw6oZWaZoGIoQBuM4a9g2coRcOBOkeKTuBAovWmODzbkjeZS5EvdHggfsz6ruo9xDr7jztPW3xeSVEnpjQiffGx6ISnsO1kaqVNycStCvOtYYOqobX+vRU7syCWFgWM839/BA0R6ve1eC5Kj6YCmzDMksr6EHa53ZXLobdaygtt2NcQ8sWLED22uKzl9xDyDneQBh9Ca13ruKHG5qGpMdh8O6I4J2qpIKTdIwA6Y4s1kzFhpwIO/+2vP0iOngeUJ5MmxmEguBUZDs2UvqwdgDw88+oakPlMoqgqAhI6Uno9GBSMLu0OWMs5BaObdo1ryNtvFsYsVPBXxFKpciERzarf1Ie3ZCx3SDA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The last argument to split_page_memcg() is now always 0, so remove it, effectively reverting commit b8791381d7ed. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Zi Yan Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 15 +++++++-------- mm/page_alloc.c | 4 ++-- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d090089c5497..ea28cacfb0d2 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1038,7 +1038,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, rcu_read_unlock(); } -void split_page_memcg(struct page *head, int old_order, int new_order); +void split_page_memcg(struct page *first, unsigned order); void folio_split_memcg_refs(struct folio *folio, unsigned old_order, unsigned new_order); @@ -1461,7 +1461,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx) { } -static inline void split_page_memcg(struct page *head, int old_order, int new_order) +static inline void split_page_memcg(struct page *first, unsigned order) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4674d9815a50..862bb0d5c0f2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3086,22 +3086,21 @@ void __memcg_slab_free_hook(struct kmem_cache *s, struct slab *slab, } /* - * Because folio_memcg(head) is not set on tails, set it now. + * The objcg is only set on the first page, so transfer it to all the + * other pages. */ -void split_page_memcg(struct page *head, int old_order, int new_order) +void split_page_memcg(struct page *first, unsigned order) { - struct folio *folio = page_folio(head); - int i; - unsigned int old_nr = 1 << old_order; - unsigned int new_nr = 1 << new_order; + struct folio *folio = page_folio(first); + unsigned int i, nr = 1 << order; if (mem_cgroup_disabled() || !folio_memcg_charged(folio)) return; - for (i = new_nr; i < old_nr; i += new_nr) + for (i = 1; i < nr; i++) folio_page(folio, i)->memcg_data = folio->memcg_data; - obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1); + obj_cgroup_get_many(__folio_objcg(folio), nr - 1); } void folio_split_memcg_refs(struct folio *folio, unsigned old_order, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 004c4856e71a..987e14d87932 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2813,7 +2813,7 @@ void split_page(struct page *page, unsigned int order) set_page_refcounted(page + i); split_page_owner(page, order, 0); pgalloc_tag_split(page_folio(page), order, 0); - split_page_memcg(page, order, 0); + split_page_memcg(page, order); } EXPORT_SYMBOL_GPL(split_page); @@ -5033,7 +5033,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, split_page_owner(page, order, 0); pgalloc_tag_split(page_folio(page), order, 0); - split_page_memcg(page, order, 0); + split_page_memcg(page, order); while (page < --last) set_page_refcounted(last); From patchwork Fri Mar 14 13:36:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14016921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD5CC282EC for ; Fri, 14 Mar 2025 13:36:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B322280007; Fri, 14 Mar 2025 09:36:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 13B37280006; Fri, 14 Mar 2025 09:36:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFA08280007; Fri, 14 Mar 2025 09:36:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CF79D280001 for ; Fri, 14 Mar 2025 09:36:29 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E5A0C1CB53E for ; Fri, 14 Mar 2025 13:36:30 +0000 (UTC) X-FDA: 83220256140.05.E5942D3 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id 06BE680011 for ; Fri, 14 Mar 2025 13:36:28 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="El7l/uRH"; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741959389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7XRLfLaf7AZTLwc7u9jX6hFHrAGcQJ5uMAqzQ/rm20U=; b=p2PMoR8SLN6b6F6GCbY9sOzNWeACDUheWeORO982vVarZPb2UQSmcA0LFWN++erLNWcz1u GXa36efvBuyf0fHFVFxLzdmzl2szdFqybF+LuEmOwSgC1PGps+gzLDuVa6GMwICJh1GyWh SVPcCsU+M3F3juxEtU66QjGPGGL9aDA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741959389; a=rsa-sha256; cv=none; b=X76b52+EAPSEb/pEMT3iCRqqF0wUVsQmu3lBCnXTsS5gibY46YWVbqXktLptCWGb9g/sqi mt8MMN+iPG7HIuVHHHAm5djpsmkmG2rOT9w1AvBXsOBLhXr7zIXStRXi5RIT6kx9qY638V 04UaFc6UARmrFZ6erUlGSO9P87Lw5QM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="El7l/uRH"; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=7XRLfLaf7AZTLwc7u9jX6hFHrAGcQJ5uMAqzQ/rm20U=; b=El7l/uRHe8zPK2wBhNXt9f/eLC eVRCKYvP4ngIWcHnbt19T/gicLL7N6toE5O8XtjBfGGxQPB8yZwLrvYdP28AKlZ2mkclGFirrOcWR Hwlv3E6tbn3QMHeZIY1JKjZhR4BVqioOVAXaQZXT8gQjdMYtExSAQISlgX2NTYFkSn2/wRhVOe5Xb joY+8amX/cW60tPnpsrRNYGrtps0gOvHnNdHmFJvZZFaVRGgYJPzAj7/gM/CeA1Q+NyIPxieoqUHz KNWNxT5k03vmnIHiN7To09OIKeO6ktutU1mxYDXfgPEdNDMuqUKQU95PVvoOd73nIHpX72P5MuGnh TLL08kqg==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tt5D6-00000000ZyC-3ANV; Fri, 14 Mar 2025 13:36:24 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Zi Yan , David Hildenbrand Subject: [PATCH v2 3/5] mm: Remove references to folio in split_page_memcg() Date: Fri, 14 Mar 2025 13:36:13 +0000 Message-ID: <20250314133617.138071-4-willy@infradead.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250314133617.138071-1-willy@infradead.org> References: <20250314133617.138071-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 06BE680011 X-Stat-Signature: yqp8ifyjpyntsuaxyckdtcoduq854an8 X-HE-Tag: 1741959388-682117 X-HE-Meta: U2FsdGVkX18IKGbA5Yq1q+wAxSoAA1yoTF8qQcs8iBt9zD8oSuCe/cKfFmmLDUc8lLt49GCURZF22Lde6fuWcAHlCIVq+C4jGVaAEs3RFQYeK7uUtc3/3GjjwNLdKMJAbq51orFzlEvaGYFvjJFv2GYBVHQ83NFI/CvnKbvGTHvkkL2qXETkLayi1JpII6AyHZ8JrTHi0g3lV178qUJx+XJQrjupRjIOJH5/J5p+XBzLbRthCee99MZOzOo74GQOJJ0QFepxdSxfj9veTgaL/u+A2yMivViJzd/zOqSbXga9NKBFYCpkdt5AqYPfHWABo/07ApuhQIdQdivTds95s7vwluJLeiYypqh7c/CKlnpNnkyYYN+Ejr/WCSlEsqdVaPaLiDHGjFq7+5OllqyNVx0ObFNLGpmqLUXXpIP7Dr0xSVlLVbSkTlJilrP9H1AIEnQ8OuPfjQMSFhudVQgLXfee+pLBF+ijls5MW+x5U9Z9omuxcBvbwGKtrkJ88IIZZcGVBRuT/ciXiII0P5PN5JXX+3yJbhhv/Sug02i+ocHmI0o9F2qPQLkRnbKv4wAPlNf37gx9uayySf8mnGD5YCvNlMwQ3fHciKhS+7lfBuI+WYzR1nzG9RO1eRWLi7vKcPfOaLVmMINenAXXGsce8LNLZTAVXukCBCm95asmAKOSRtO6DSlamBPD4FJYvdLTPqJkR3vsPnnFaPlvkc3f321jZnJR2QidgzJbV3ZItMRpUNvUoTBtK6MWXnXM4FDRsRkbVL4VYDzj6X3Z684Jxi9pgE/jKK7N2uSEG/5wrFFOCk3TnC7W9JtXRotyKFSRh85sUZsFNui2rcgSVj4Pge4TTM+hpuMbwf2PNECn0R2lSon4f/waKGCF3cwztvG1Vv4eVSorvSj+bo0IOxFUqFNib+aEA1cMbrWkp2nfFbRBoRpRkfObN/l/JObD8KWFFPfxwQeZhqOHhgEl+oN unCXirF1 /1DGnfBSH8JUY5daXvaM5JGK9vOBj3ZUllzGD8h1+cgqjsz6C0otUSkXBOqRo51AJ+QyPcgvRPG1Srz7zkYNOGGKD7hqYTSMnNG2kc8U9lWTcolvIQGHh30iq3WY/HFpfM15Uz6eo0YM4r8Ki8RPCvw9BWI1cvB9vwj3S430V7Mf/Pj0vBhC6otpY2Ys3eplB8y3bdazs2/gWb5aex+q13RNPkoexjYX5qFvxT1EwWg0/5MRA6VbPoVDGQpdDsQnZYHkLtJlC5rD01mc+maoI7sE6VXd+/TzIbYIkRTVmYcJqwwX8XLpgPTS4zJS5zf6IL15r4QuX5WQ+uF6AUBcAeAUXJlCfdhZA1FXm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We know that the passed in page is not part of a folio (it's a plain page allocated with GFP_ACCOUNT), so we should get rid of the misleading references to folios. Introduce page_objcg() and page_set_objcg() helpers to make things more clear. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Roman Gushchin --- mm/memcontrol.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 862bb0d5c0f2..9e9027dda78c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2697,6 +2697,23 @@ static int obj_cgroup_charge_pages(struct obj_cgroup *objcg, gfp_t gfp, return ret; } +static struct obj_cgroup *page_objcg(const struct page *page) +{ + unsigned long memcg_data = page->memcg_data; + + if (mem_cgroup_disabled() || !memcg_data) + return NULL; + + VM_BUG_ON_PAGE((memcg_data & OBJEXTS_FLAGS_MASK) != MEMCG_DATA_KMEM, + page); + return (struct obj_cgroup *)(memcg_data - MEMCG_DATA_KMEM); +} + +static void page_set_objcg(struct page *page, const struct obj_cgroup *objcg) +{ + page->memcg_data = (unsigned long)objcg | MEMCG_DATA_KMEM; +} + /** * __memcg_kmem_charge_page: charge a kmem page to the current memory cgroup * @page: page to charge @@ -2715,8 +2732,7 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order) ret = obj_cgroup_charge_pages(objcg, gfp, 1 << order); if (!ret) { obj_cgroup_get(objcg); - page->memcg_data = (unsigned long)objcg | - MEMCG_DATA_KMEM; + page_set_objcg(page, objcg); return 0; } } @@ -3089,18 +3105,18 @@ void __memcg_slab_free_hook(struct kmem_cache *s, struct slab *slab, * The objcg is only set on the first page, so transfer it to all the * other pages. */ -void split_page_memcg(struct page *first, unsigned order) +void split_page_memcg(struct page *page, unsigned order) { - struct folio *folio = page_folio(first); + struct obj_cgroup *objcg = page_objcg(page); unsigned int i, nr = 1 << order; - if (mem_cgroup_disabled() || !folio_memcg_charged(folio)) + if (!objcg) return; for (i = 1; i < nr; i++) - folio_page(folio, i)->memcg_data = folio->memcg_data; + page_set_objcg(&page[i], objcg); - obj_cgroup_get_many(__folio_objcg(folio), nr - 1); + obj_cgroup_get_many(objcg, nr - 1); } void folio_split_memcg_refs(struct folio *folio, unsigned old_order, From patchwork Fri Mar 14 13:36:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14016919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D1CC35FF1 for ; Fri, 14 Mar 2025 13:36:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20CA6280004; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1961C280001; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05D65280004; Fri, 14 Mar 2025 09:36:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DE286280001 for ; Fri, 14 Mar 2025 09:36:27 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6C3CC1CB465 for ; Fri, 14 Mar 2025 13:36:29 +0000 (UTC) X-FDA: 83220256098.28.1EE5493 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf09.hostedemail.com (Postfix) with ESMTP id C3723140007 for ; Fri, 14 Mar 2025 13:36:27 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=p0mAc+lF; spf=none (imf09.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741959387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6tl9PTBObfMUzEWfIB674sO/Z5/alQwKMnmfDcqOHoQ=; b=5l+f2rz7Qkqeo37lB5JLAF8ufMrwTz5Vz0aKAfCd6e57BgVr93f+aFkOpLyAozCinO/1n7 au85R220r6dV4n1E0viqDYuKNuinvXYUzJG/l5J39KcJfFi2VUOXtb5vdllFjZNIcyepPH w7QyRcmVYJKNU8z8m55WEFNbzGSvzHU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741959387; a=rsa-sha256; cv=none; b=aXn0k//GnuPslpyaGqvyu9nzR9iiEBejG5mgOo3ztBge8LobWIEr4Olecf79vJlwZQdKF7 w7gm1mbSXACYzUz1YkYhBkIr5HzOu9yidZ1bpG403GEYVVrvDy8LGPm22kQtTmvh4tWlg5 3I8O3xxyCVmxcf/9xOdvChc6kjz8nFo= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=p0mAc+lF; spf=none (imf09.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6tl9PTBObfMUzEWfIB674sO/Z5/alQwKMnmfDcqOHoQ=; b=p0mAc+lF/I4jK+DByBJ9PldLmP x7no93K5JiPgY/XliqPW/H882mFG6sNCsj8IHf9AHU8tiyHBolPZ1ZDqyXGzMhVWIYY51ddjiii+P JTjd/wvoW8PnHe0zUZg/MUZGktqt3SygIunqst8HLFkq8pS8b1E5XBQPuHIJ/pzf78LEf1SwYsrUJ qUdHQoChV78/V3wiB3cvAM7hx+F89FNR1ie3TFhLSbWyxuMgHCZPpuXBl842LL7C1E6ETSV1MlzU0 uDR+FjZJsyZ3Z34ipsnpvq5B82pXut4wuMDYDTKCKQqPkWoT6sSFxXyObeHx8uMRO9R6UrkSLcdXr kp/9gQaw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tt5D7-00000000ZyW-0Gr0; Fri, 14 Mar 2025 13:36:25 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Zi Yan , David Hildenbrand Subject: [PATCH v2 4/5] mm: Simplify folio_memcg_charged() Date: Fri, 14 Mar 2025 13:36:14 +0000 Message-ID: <20250314133617.138071-5-willy@infradead.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250314133617.138071-1-willy@infradead.org> References: <20250314133617.138071-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: C3723140007 X-Stat-Signature: 8z8zo4tsh7zf4khdfi7i49haa3hkt8tg X-HE-Tag: 1741959387-141091 X-HE-Meta: U2FsdGVkX19uIyOUCbiMAG0ETxfTOoMTJCIzDBDvqTFfKjAJpduDl+9eYVw/aRxr2MHoTPC2SWHWcuDpGGwzcHCj2NF5Vro9W+wLRVgShg25kruEVrpWFovXv/qcGrQXrAqU+aHSE1/dr8UItsgm7993TUK/lhodAe/ALeMa7IVOizgbgqgokOgEX+QhqfBdrpxDtm7XJpknuGUnOfCQC0zS8VU7RJbg3X4BjNYs2xQv2sSnudw4sj8WAiVaN6dx4yHYe0/RJWebK299gCbxUqblRirb+irCkbqmpL9saFN2KfLtYwLVLcBxfldaw3QpZtDvxGsr0iayuPt6owiUyDq/3lvsVZ7ulZpvkfjHcFhZxaMKP4jZnAnV3tQ6uPkjHxTJ52QyT2C9QChCVqJL5OBk+fV3BRadQwhVBBUvOkGq7SqM65wmXX0jIPwBWYePbEWUgwKn/Fv+kfA6E971fN84zFmekw3LZDLLRtD3Dl6fkNyhelRjckubA+OfP0zTtMAEWKmHIc3zE3jlwGRoDP5SOOdtKdJ776bhxJ7WOcfSrYEPwece61UaGjDe6qmLuwKbz6HAKB6nC4pcFRvVcx0hA8G9uwK7Q5PvdwHisAOWRWEMufXeMF7rxpsDvAL/6FxVNnKzFY/eVe8+1DKGyMLningR+O8IXhhFsiUFk7WA71zMHPnLMqnr9aEl5kU3Ab7vB48IK4LJRkR0ocrkNRkg2QF54sy9NRPqRpf4+t2mQoa2WePHhnp+KccmwBqMN0DK4Dy9pyxyvdxvjcnY/Elj3aTzorDXqOrkJuFwsH2vgx3Fm3YXqXt9G2iVqeNSy+rC7fHYiXoX1kQ0EpjjU599PQwrTl1rPu67n5PBMMZU2AdSo53irR3wMZjbcf87Z2MAIQ5MOpCLNCEozreo+DOxr0Y4BINk/eNlh9BkaqpAfZ+IbXFdUQjUCWXleEwJDPhZImQtXyScjXnkn1g uxMFxHAB qBOxYJOIU+v/iTF0b1GiPz9a+Izrj8AJVA7ltB/SeFnnH1ph+QlKD8tZRKrSeKQdIYbNaXLqzrf8yrut1r0SeLqLLhyP9k2Xl2HyxzOikkmqeRdO8NZeqb2JUs8hme7VshXLNXBmVPuPMqrioYGpFFNJILyHNYbXUC7tq8kJY60KKfwYRE7hCAVCDn/CKiP18qZid6RmF7J/6I9y8jhiIwHyDz7f3q4uFLOZ+1yXZiVjtOPiP4MaVMuZstG5un9ilndTTZFcaAEPxvBRu9jR0KXAkWdzVhP31P99lKuCtcc+eJtn2VN8NCFGRKY868+F/D3BbO2q7rnEjUu3SC7u/udZZXL2F1O7sFZwh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There's no need to check which kind of pointer is in the memcg_data field, all we actually care about is whether it's zero or not. Saves 70 bytes in workingset_activation() with the Debian config. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ea28cacfb0d2..53364526d877 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -438,9 +438,7 @@ static inline struct mem_cgroup *folio_memcg(struct folio *folio) */ static inline bool folio_memcg_charged(struct folio *folio) { - if (folio_memcg_kmem(folio)) - return __folio_objcg(folio) != NULL; - return __folio_memcg(folio) != NULL; + return folio->memcg_data != 0; } /* From patchwork Fri Mar 14 13:36:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 14016920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E028FC282EC for ; Fri, 14 Mar 2025 13:36:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF43B280005; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BCBF0280001; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A223F280005; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 60E90280001 for ; Fri, 14 Mar 2025 09:36:28 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 80793B8951 for ; Fri, 14 Mar 2025 13:36:29 +0000 (UTC) X-FDA: 83220256098.23.2F5F7D1 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id D5F0F140011 for ; Fri, 14 Mar 2025 13:36:27 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jXR9C9Ca; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741959387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fC5DoxNuYoKV+R7qEc+/iHREUpcwj9ubSKHO4mNm/rQ=; b=cwBPaUlZPsuzNvIFFYtppGoeCtCPMjwG3qv2EETo6Kf5CtDesOhupWRbosuYGDKPC1Ir6M 51GEYwRgEpx70BT6xDkKOfOrxi4Gd5jwNFJbOiOn736N4lgp/2pWCO1I6uomO4A9Fv6fXr Jf7/zAt7QoIG+e2q8edI7/OzDR4HQQ0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741959387; a=rsa-sha256; cv=none; b=KlelrP/ADnAF2CBw4rHJnwYIjZXQ4+wU7GBHQz0MPhV0qAQlPcjUjeXxSA9U4jNg/h4mTx 3KEBH5ofEnE03znGs724YZ46v9qV20MRhoHvEJlz1TmMqnZBYZ5UoVJlEn4kjNF+yTPTSI Czk/xTRToOdFVsbwvJtsnGuDy5oBI7w= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jXR9C9Ca; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=fC5DoxNuYoKV+R7qEc+/iHREUpcwj9ubSKHO4mNm/rQ=; b=jXR9C9Ca7DyodmKtKnqhkF/FWg kGjHvyL+KNTMD/r6MpS0l8yTqrXPVsFEukBXlBu6IgLRoJBqc6VrCNAZJPglvilxUANhkWzkF1ZpU E/lwK11Sv/0lM0l8MqbmkxOlfSsroyWcH28PIE4AqhkyPyquPmLMz47YpWobwIMGF3Ndg3FBrJMkf bCcsZIyb3Q7eWwi21zrNlJhUkrQlBu2CsUzx+lXI/FdRgTLluuI4jE0I0/OonkT67UFL1rQR5Ngnm bsTToHL9DkW66VVf2dAxaedtvInKCUOvfneEsrGABXPH8HUjsRatJegTFW1KA4PdrU05Ic2ZzSay2 cMX1YIWQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tt5D7-00000000Zyo-30bu; Fri, 14 Mar 2025 13:36:25 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Zi Yan , David Hildenbrand Subject: [PATCH v2 5/5] mm: Remove references to folio in __memcg_kmem_uncharge_page() Date: Fri, 14 Mar 2025 13:36:15 +0000 Message-ID: <20250314133617.138071-6-willy@infradead.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250314133617.138071-1-willy@infradead.org> References: <20250314133617.138071-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D5F0F140011 X-Stat-Signature: eh9gq1tmw8ufyuyofcmfcxqw3n8qrhux X-HE-Tag: 1741959387-365891 X-HE-Meta: U2FsdGVkX19MH0jt7JYi7ss2hDzG3kb3mdVnk/3EUHN/Y/LdZCmUHLeskFXLr69bCYzuL1wFVP3QvSSf3mOt3pCBpNYpmvSe/sD5uii+xFakZSwgvWA4MZSUZkB4SXoqBwE1I1BjWpo6+Hf0DLuUGEKcdN7kv3wy+vzumvBuiiEfzJywQZh0XFkqzlAA5051Ma+MuiU86vb7Z54diWpTDR1I4LAKhM2DFoIMWr4ZnMLEQqTuDWsGEJDaDO2gJ8OzOoo6FraprH+YU+/IfYk9y0IFuesY4lF3bInKN0jtu6/2u5hdLVdEiaauzHPu5R+4rP3qS0Jz2cM4WNK6w3myVdlBMl7IuaGP7Vo2qlE44+bPdTcdxmwSp3ChSB61skHVHIfJrAOWqBeLdlNtVT8WIum0ufUZHowWCPU5Xrtq51ujBkN6+frpBZxmDa3GxUJ1kgil3qhimiif8QcHG85E6AYQz0IoEmSBjsHVGD+xW9ezI16ul3aSa2j2QxkK1G5kKO5LGDxX3z5xqVuhO0lkRToslqIFm3DKjFZYYkKw6xWDOr8jta1Zk4kLx4HBZ14x9TN9qW0jzbGa8PTJ9TKVw9Ge6x1vJJw6wJ+VxvALKAAIRrfv43MQCPTz4TtWQ9dMwry67KWfnUasrJuH4QKGY2knXKvi0gcwIbNUoIuDLkoCXavyrHa1ibVS0S6cG1iG+m1gpCDPlCkOs4Ns0n+KXx0tEt75/If6EbZ1yZ4/gWxMzgNmKypQwoi3Uko0ohFtmEDX3vjy7scpAhkGUeIk+Erm5BwRrmbLlx2JSozxKmeoxAQo2KkJ3+1+8NgSpgiTDs+ssh1lz8TGkrAZSSYD1WaBPm2bIwdfkWrvo5zBWlAq2gMXHcvtkviGIk6y0dpDuDC8Zxj4QCHSMlwp+nq+3ep49t7yu6HcrAgdt/ApyMQ8gG92nz/GQKun8KwINy1IbQUaeFJvGYJQBiLemMv X27OGHbf yFG3YP+9ir1XrkjZxTwWoFJN+BvyDUKvI9FHf/Css8LtX4SY6hkbf6etfiARUwZ5FtcSixtLrNvBVi6t3lMIq0hkSpf7TPFQjMlRCBlXO8nGwwNf1Af1hc1KfMY9cxs8LrJrOa6rGB5vMIm0HoFBdsOnflZP+uchcBXhH3d5lhuWSrixyNG6DQbjYwaRHeCNXPucZd2Qrl6R/1/1o0C+OIi3GTdO7Ng4rxVgHfJah1kLFPh9T70lF7Vpgc7JVxqczBMcO0MJ9SLrCCUQBlINYmB7zWwtbu5LzJWCFG1nbPG1Uqz/G3wYX3nvCo1xKnWLWqtNqOsE+ibNyTzpzEhxz70fxNlGv0r+k939z X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This use of folios is misleading because these pages are not part of a folio. Remove an unnecessary call to page_folio(), saving 58 bytes of text in a Debian kernel build. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Roman Gushchin --- mm/memcontrol.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9e9027dda78c..76ba1953173a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2746,16 +2746,14 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order) */ void __memcg_kmem_uncharge_page(struct page *page, int order) { - struct folio *folio = page_folio(page); - struct obj_cgroup *objcg; + struct obj_cgroup *objcg = page_objcg(page); unsigned int nr_pages = 1 << order; - if (!folio_memcg_kmem(folio)) + if (!objcg) return; - objcg = __folio_objcg(folio); obj_cgroup_uncharge_pages(objcg, nr_pages); - folio->memcg_data = 0; + page->memcg_data = 0; obj_cgroup_put(objcg); }