From patchwork Mon Nov 4 21:05:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13862012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54D85D1BDF3 for ; Mon, 4 Nov 2024 21:06:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7DE56B00A0; Mon, 4 Nov 2024 16:06:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D078A6B00A4; Mon, 4 Nov 2024 16:06:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B80FB6B00A5; Mon, 4 Nov 2024 16:06:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9987F6B00A0 for ; Mon, 4 Nov 2024 16:06:17 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 59F71140ACE for ; Mon, 4 Nov 2024 21:06:17 +0000 (UTC) X-FDA: 82749643368.20.57BB9E8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 4C28C20007 for ; Mon, 4 Nov 2024 21:05:42 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LBGDMQwO; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730754292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n4HJqHIFL6KoDyu6o3HDh3SUN9SIibGS4LEHqczrBb0=; b=bxAD9Tmz/kBKrCZiXzDVECEpZ0p9Eu2jxd6h+oz05fid/ijDkj1qL4ru1e+hQ/zBTRS+MH M//8a03Fv4kU9FwH236o1vkRHf4IGfLEFtfjto444td8SleSn41ZnPhzYZRmbLJIF4Dgpj +TpwS9wLqYwUwtfpwYLWPwLG9D2FjvU= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LBGDMQwO; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730754292; a=rsa-sha256; cv=none; b=kZqzQAd9lD0NQMFaD9oWtbrcsVuw94HK6m4CcO3voHErz6IpMpu/k5nRlLfkXCYESKKc1w NDzpz1B7jbsCSnlz8WlXxdsfEfNLgfEOnLVQtksKeVNiJTKiwchPZmaOfi3kOmqVcOG/Nl OiWjEef91wJHzwG+FB5lwRXPo+OnxZc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=n4HJqHIFL6KoDyu6o3HDh3SUN9SIibGS4LEHqczrBb0=; b=LBGDMQwOUt8tta8VHNMF+zE9ED SVBWmfKJzxz+yptFcLR0vrOQCmdJ9MPFhP9i/N/OTwaUsy7KfnOY2V/QeD9deQuKfWrxakLdBbpRM ddImJROE0Ug4pFwod2ux+4/ao6ATz71tF7RTeSGcQhoUz142XbT4FH5V750HdeGxhk4DBEqA5Y2KR rGDqgBflCT1dpbTbiXe0X9lrMIgEMrouVA7GYw66o7sbQ7H94EFQsIuBho3ivKvCkGntqyzBPKnE9 nNS4KpMWfWmiHR16S9pzfqeBz4s3zNtRbkVKpE3rdm0PcQYUUSpKGhDdp0dMVkxzzezuuQ1jm+SdP AIkfZaIQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1t84Gy-00000001ZYN-15RY; Mon, 04 Nov 2024 21:06:04 +0000 From: "Matthew Wilcox (Oracle)" To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt Cc: "Matthew Wilcox (Oracle)" , Muchun Song , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 2/3] mm: Simplify split_page_memcg() Date: Mon, 4 Nov 2024 21:05:59 +0000 Message-ID: <20241104210602.374975-3-willy@infradead.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241104210602.374975-1-willy@infradead.org> References: <20241104210602.374975-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 4C28C20007 X-Stat-Signature: 4m45wy7i5mdiqx5boawczpzirg89nb7r X-HE-Tag: 1730754342-439810 X-HE-Meta: U2FsdGVkX18089v/AMaI4faAXdRDGBIG2TKdaj5aCmbAFTA4VQ5mBzImcgwKxLsGJx+v/8jo5DdtJ09Qne9053v6Go5G/C6KQeki8zHztVUfhgAv/6PIakbTBduiBRN5FzIzdfr7fTwhcJARNtcasXs/BURqvLEUPnc+/g0S1IfMbFrbki/Sr0tHSf+evkP0z9ypdNQWiS+0vXPWWEvCvVhrl6TfobX13PUiQdcf8fbIfqUlUU5v7fyT1PYXE2WQFTTU/zj5/uriMXBDJv/t171yaRuXCU+Wpw45dZv0L6FtzXrarng47x72+V4MT5NsQ5c7gdYb2Lm8hqqiF8tPTzY7KhttspsDVn2LCdeq84vELcxe2XHxqJF8dZmxdK3GaNLctAoO0hFwmN6IQ+pTH4LgGb0nhZ6H4zk32z7RJMtirjhbbZeatrJ+SYzC9iMj9e4XrcTA0YYEa5NG4wXTRzRClNhKHGkwhPEEiP80dai3x3PtQ/qam3qign7C8ur5Li/S2KTesGyhp9xeysKaSEztqv6JjaRZMrz1OywO5i72gKvq0eY2SGv963khmo/QrJdBmwd5ehfw62hUoPUzcjCVXfNWTS/iDSKjDI4CtXZe6BOK7JheDPud2Hi7Av/BAKWNKT3A2rruWYfwhCVoPJeG+rRQhjlZgA6UCx8Z2VYdHQhu4/BOchZDhG079xwX7rz+5ni9cxRjWfBjRw8Hgaxmu5p9NzDIWf87eVjiyun2MfVSv8xO6ktQcXfW9y34I5bDccidXyl9OjFSnNKZ4O79eMThqwJnhl0IFWxRN1QBfh7MBR1rzMyDXLfypGalqoblykeah0guLcUVVeHg0p5hEVtW8E7d7r1aRz4gxmUoU7ICPu+GO9Iuz8pDkob1QhD8XbjCpK4iF7kceE7vK+bR9PhDGkQfKpqBb2N3HLkQSUdH4tabX6yq4HYWxYWqEEwpWtUswfyygeYsFsO WNTQjj8o NPlyRCO6MvPrUDsHUoLh+rpSPCb8XUJgYYR6VvFzkAoL4T/X1tPlT6K6+BkaD9Tx2bmQjeEtmSiH3U60jcSCGXdU6m50MOuciiqu5JuwYVX+LDpDTjxxaoimGPOaiwxnLpBjXPagDpdmjAQBGVB+O8vYbuIpSV5si2aZDIx8HQhVRdUV6ic/TLPr8Eqe+z+hx1pJSk7hqRfit4T2Yp4qdeyZvKbwEwt13RjHZ4OqThihIHLlUS21swLlzmmNfSD44eHMj93vl3guZr7CAfkempbYXfkQutLYEeBDv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The last argument to split_page_memcg() is now always 0, so remove it, effectively reverting commit b8791381d7ed. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 26 ++++++++++++++------------ mm/page_alloc.c | 4 ++-- 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 5502aa8e138e..a787080f814f 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1044,7 +1044,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm, rcu_read_unlock(); } -void split_page_memcg(struct page *head, int old_order, int new_order); +void split_page_memcg(struct page *first, int order); #else /* CONFIG_MEMCG */ @@ -1463,7 +1463,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx) { } -static inline void split_page_memcg(struct page *head, int old_order, int new_order) +static inline void split_page_memcg(struct page *first, int order) { } #endif /* CONFIG_MEMCG */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 5e44d6e7591e..506439a5dcfe 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3034,25 +3034,27 @@ void __memcg_slab_free_hook(struct kmem_cache *s, struct slab *slab, } /* - * Because folio_memcg(head) is not set on tails, set it now. + * The memcg data is only set on the first page, now transfer it to all the + * other pages. */ -void split_page_memcg(struct page *head, int old_order, int new_order) +void split_page_memcg(struct page *first, int order) { - struct folio *folio = page_folio(head); + unsigned long memcg_data = first->memcg_data; + struct obj_cgroup *objcg; int i; - unsigned int old_nr = 1 << old_order; - unsigned int new_nr = 1 << new_order; + unsigned int nr = 1 << order; - if (mem_cgroup_disabled() || !folio_memcg_charged(folio)) + if (!memcg_data) return; - for (i = new_nr; i < old_nr; i += new_nr) - folio_page(folio, i)->memcg_data = folio->memcg_data; + VM_BUG_ON_PAGE((memcg_data & OBJEXTS_FLAGS_MASK) != MEMCG_DATA_KMEM, + first); + objcg = (void *)(memcg_data & ~OBJEXTS_FLAGS_MASK); - if (folio_memcg_kmem(folio)) - obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1); - else - css_get_many(&folio_memcg(folio)->css, old_nr / new_nr - 1); + for (i = 1; i < nr; i++) + first[i].memcg_data = memcg_data; + + obj_cgroup_get_many(objcg, nr - 1); } unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 47048b39b8ca..5523654c9759 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2823,7 +2823,7 @@ void split_page(struct page *page, unsigned int order) set_page_refcounted(page + i); split_page_owner(page, order, 0); pgalloc_tag_split(page_folio(page), order, 0); - split_page_memcg(page, order, 0); + split_page_memcg(page, order); } EXPORT_SYMBOL_GPL(split_page); @@ -5020,7 +5020,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, split_page_owner(page, order, 0); pgalloc_tag_split(page_folio(page), order, 0); - split_page_memcg(page, order, 0); + split_page_memcg(page, order); while (page < --last) set_page_refcounted(last);