From patchwork Fri Apr 5 15:32:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13619217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC492CD1299 for ; Fri, 5 Apr 2024 15:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB1896B0092; Fri, 5 Apr 2024 11:32:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E13BE6B009A; Fri, 5 Apr 2024 11:32:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3ED76B009C; Fri, 5 Apr 2024 11:32:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A4ECA6B0092 for ; Fri, 5 Apr 2024 11:32:50 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6FF49C05E8 for ; Fri, 5 Apr 2024 15:32:50 +0000 (UTC) X-FDA: 81975870900.03.925E1B4 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id C42EF40017 for ; Fri, 5 Apr 2024 15:32:48 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LGDaBvzJ; dmarc=none; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712331168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BsqG5grOKnH4ugNINXZJnED8tFK/8c8jG45dU6qaWfw=; b=7NyM9nzYR26m5kyueER+lYrtnBNh+7GmhpynUIMlzbiudW4Jc8pbxGUgcB6iDnKmL/GzE9 +YPuE46HsP3Nt3yi377kiR3A760rGFvlioAB+TAlhJ/QxxVDMwijlVLOah42p+Yrey24Dc lWtcZ9bz4x9gDYJSMiWIq9ygRkQnAqk= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LGDaBvzJ; dmarc=none; spf=none (imf17.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712331168; a=rsa-sha256; cv=none; b=cCI8IBh0goarUKkBynNzYyx4oFw0Hl9Rp0GhjrVP0GtlC5u6+D4xfQdHwmjga0c8ef+wQb 11J7Z2FYVJmj9vXLlLNXozcL0/UtgS1kN/HeX5YLnGotdUY1sI7QqKcJ+J/LPZByfTpDlo HStb03bsEmSZFZrpAr0a+OpuLB4yskc= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=BsqG5grOKnH4ugNINXZJnED8tFK/8c8jG45dU6qaWfw=; b=LGDaBvzJv8uWb0su7LvfhLMgMy elpO9+AF2Cfg64qXs531LpmgwjA4njlHr6cCFG6tGb9LblSH6gsD4hnPAHRJbHW8cx3+cRucWDm1o VmEYnmUcVMBBMpHEb4mGOJV544wfJ+GtVEVf9zZVUeBPzcf1cWyeb58+ieaNtPnpM4Df/+znu1FrK MAUKJ+44fenOhfMISljMRZY74ZVV/p9E8FBhpVB24vKiLChPYetkhATuuT/g8a0CYiW8eSmAnw8eJ YK8dI3ECBM+qlLvqeMiZqZPCz2TD0rlhPphGQtmtsG8fwfaQW20p73/bsWhwmTCtaF2jNc52QPrh3 KUKi/AWw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rslYM-0000000AkxD-0STN; Fri, 05 Apr 2024 15:32:30 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 2/5] mm: Combine free_the_page() and free_unref_page() Date: Fri, 5 Apr 2024 16:32:24 +0100 Message-ID: <20240405153228.2563754-3-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240405153228.2563754-1-willy@infradead.org> References: <20240405153228.2563754-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: z76sbpury1p6str5g3tcd9dpnkbxkp5j X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C42EF40017 X-HE-Tag: 1712331168-306591 X-HE-Meta: U2FsdGVkX1+0dnOoPy44o7Nfx5onDqNsqsv1kfbJDZU7IHqGrkpK/BTq1t4uCGbC/HxUTWc7ktO7xrjTsAcWTYiDJ2EfFA6fFGWzd1f2L9Bx6Wowxxbs8dQA2maQ35TCDE8wSheQZCe2pp/IMH5zxfBz+l8gmpEukDcziSb/F0NaRgY5OLMCCGnBUU0R+uuy4LpRj0sQ5OelexD4WIJkTnHCF4ozMuBHoW5Mu8Vlj0OdKWNh1Q4IEGlJI9Q2Gv0y/M6VnhhW5ZoOiLyp0P2gFN7b05bvKjhyq0jedBhdMRvg2CeCNSCACxh3YGxD3TcXqKMJgmnMZNdsL1PBuZDzFUbPN+P8vRZzHlj3BOEbbc9H4mmBkojRDSYUfvh2+fIb3z5MqexHHv4FJKMdP+jwNexrQ+sm5VQ3Hg/sBT8IH6rIU7B00NBUuK23Fs1dH2hkYnn6Lb5cLva4ljeprvWn+1FS4SmCqJjwSRcXRyYp6rq/XoVCAJXTlO+N3e6DH8NqG0KldFUsHIHKsRLQZFV5X08tn3f7TinZSEjjNtHdnZt96t2UCoXTLN5a7+YrupwQbn8KQ75PUjXbXqqZnXfS7QKE4AwdNqYHxLzX8z+r4jKmZByU1Kcojt/6rB1b68nTXuVPx8zS72cSGCXhDgOZy10ktUdOaxNhRORQdWG+hYuosqvJUhbCOAwvPOmC1AOqTBjkWwutQfpFvZcrN0aCyfF+hh5yQVi/QugCOEge4f37mj1KXJKDwX3fHrT67hnM975HHxoYyM1lz1o7ClS5kGfbIzG9K8VHtaPGbdN4VZ6YXXOPmQGDty9W848hpbYizGd1vyc8JEjLStzGwbghcLY4qkw8Aid+IXRpwWpCVxx96S4rZ0bU1Gs5excaBPNusJ5u+YWlU1k/kEnHnpNU2xkoykv8AIbbVgGntOJip4ecnb423DH6OSvEWy0as+LSb251HeuVd9O3ktiHHAO 8k4MR52l Urie6SkNnrezqGGbIij+mZjhFgTy8f3YjpJWXOwDBmpCB6txrvlzEgYbh65xiQvGIUCPUO/mKgajLdZTo7noHujB48nNAeccZQTblEkBsT+/VIwv3IReJbOvg5v9hqe+9SCPGnGb3xKphQLTLQ3ltUopmS9StGvr/DrEQTePC/7n7/lEeBn0IOhpqmc/66KjFIxpDNMnRh1RRhzMOsE4nJrtobuhHyhad1AGZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The pcp_allowed_order() check in free_the_page() was only being skipped by __folio_put_small() which is about to be rearranged. Signed-off-by: Matthew Wilcox (Oracle) --- mm/page_alloc.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 808b4b720834..8d255f18f6db 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -541,14 +541,6 @@ static inline bool pcp_allowed_order(unsigned int order) return false; } -static inline void free_the_page(struct page *page, unsigned int order) -{ - if (pcp_allowed_order(order)) /* Via pcp? */ - free_unref_page(page, order); - else - __free_pages_ok(page, order, FPI_NONE); -} - /* * Higher-order pages are called "compound pages". They are structured thusly: * @@ -584,7 +576,7 @@ void destroy_large_folio(struct folio *folio) folio_undo_large_rmappable(folio); mem_cgroup_uncharge(folio); - free_the_page(&folio->page, folio_order(folio)); + free_unref_page(&folio->page, folio_order(folio)); } static inline void set_buddy_order(struct page *page, unsigned int order) @@ -2614,6 +2606,11 @@ void free_unref_page(struct page *page, unsigned int order) unsigned long pfn = page_to_pfn(page); int migratetype; + if (!pcp_allowed_order(order)) { + __free_pages_ok(page, order, FPI_NONE); + return; + } + if (!free_pages_prepare(page, order)) return; @@ -4800,11 +4797,11 @@ void __free_pages(struct page *page, unsigned int order) struct alloc_tag *tag = pgalloc_tag_get(page); if (put_page_testzero(page)) - free_the_page(page, order); + free_unref_page(page, order); else if (!head) { pgalloc_tag_sub_pages(tag, (1 << order) - 1); while (order-- > 0) - free_the_page(page + (1 << order), order); + free_unref_page(page + (1 << order), order); } } EXPORT_SYMBOL(__free_pages); @@ -4866,7 +4863,7 @@ void __page_frag_cache_drain(struct page *page, unsigned int count) VM_BUG_ON_PAGE(page_ref_count(page) == 0, page); if (page_ref_sub_and_test(page, count)) - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); } EXPORT_SYMBOL(__page_frag_cache_drain); @@ -4907,7 +4904,7 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, goto refill; if (unlikely(nc->pfmemalloc)) { - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); goto refill; } @@ -4951,7 +4948,7 @@ void page_frag_free(void *addr) struct page *page = virt_to_head_page(addr); if (unlikely(put_page_testzero(page))) - free_the_page(page, compound_order(page)); + free_unref_page(page, compound_order(page)); } EXPORT_SYMBOL(page_frag_free);