From patchwork Tue Aug 15 03:26:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13353486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D1CDEB64DD for ; Tue, 15 Aug 2023 03:27:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF31C900017; Mon, 14 Aug 2023 23:27:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA37F90000B; Mon, 14 Aug 2023 23:27:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96CF4900017; Mon, 14 Aug 2023 23:27:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8A69990000B for ; Mon, 14 Aug 2023 23:27:22 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4F57A14025F for ; Tue, 15 Aug 2023 03:27:22 +0000 (UTC) X-FDA: 81124903524.20.1FA521E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 912071C0006 for ; Tue, 15 Aug 2023 03:27:20 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jaNFDNXx; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692070040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9AFjGPtcNZvRdattiMeM+NhO363S0DBsMAr8j6pNUgk=; b=nDd6dvFDXAhEpthW+A0qQJoN/edrVXOf4i3izv6t02pvb6E0SDJS9kKqPazXt7tKbcRcQM hpnWP34z/IqoVsyOgkk3SGdbB7lGNZeC/zW94l+72XRS478ICNwHIK6VWeT04Uy4vRv4oa BSMe0+v9ldRngY7pfdZPt+wBD+wtiCA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jaNFDNXx; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692070040; a=rsa-sha256; cv=none; b=1beh3vhiiI/aEOm2y9ikputLe1ded4Se3XGBAxK+myZeq/rwQXJRwT6V6jePEnQT6y+P8O yJy5sgUct+Q+ylgxHDv8yk8+nZWAYI+VZpJfGNUSRUwo6WqOGv1yICjjbxXM+FDO05rZw+ QrUG2m94nA0M7Nxox48pEEaN+ZDvG3w= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9AFjGPtcNZvRdattiMeM+NhO363S0DBsMAr8j6pNUgk=; b=jaNFDNXx63oA6YktKdyny5R+FA RVh5LmTlc1op/Sst8fvW70x9QNRDwuwJOHdIIjJZWUf6guT7CgC+x3iC4VTc0Z5C7ZeV8vywxhRn2 iQv7KMpfL46Nk/RHw5syw8gNRhvNS9JKOkvCC4s91EKuD+3TD2PM7Lck/sH50znrz4zhJ66K/6xqW frQp6MmSz5oZtlGIHcjgVJoS3tN28ZkHgwNnW2HzRzSYdibignJU8xbNbAUpTqqzvPL/2uF4lwhl+ HhW8C6yHRX3CyVf75d3s/k2AEl+VW5wqASKAdQSwMdsKUrccMFJq47rXEAyQWAHYMWzz3V5pr2xvE 93mcRMSg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qVkhm-005qad-Or; Tue, 15 Aug 2023 03:26:50 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , Jens Axboe , io-uring@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 7/9] mm: Add deferred_list page flag Date: Tue, 15 Aug 2023 04:26:43 +0100 Message-Id: <20230815032645.1393700-8-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230815032645.1393700-1-willy@infradead.org> References: <20230815032645.1393700-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 912071C0006 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 413sqoxhddeot68gq1sbewooys4ndcz5 X-HE-Tag: 1692070040-363195 X-HE-Meta: U2FsdGVkX1+3pjPb654G1ZfNLZUkHxnSN2qJ+Zi9KlbOwcbWLi9/Y6IdoVlmSUlpxUAPoKre5ALQcwI+q02c5yyvZJr6QFCUABcOD5yqeuEtBfLfLkcxxi1nvxvb77SIBbK7sj5L3ybnACUp4sc0neNPazOBn3Yn9lFmG0UQSsJ1UVh7Jazauu6k75xm7lLUNf3FUp1MvIt2SQ4RMhjf8fiPc2/1QVyE4jELSfCBekAgfEhnoICnSTUGdo7hPs94OGXNgeAeaEniYqOzDVaNqohFYLsVXfVKKLI8vgrGMi22RcnrW6MoXgVM29cq4ioiRDKNvAmbIDAYDHO3q2D2u6yDsRbhmHrLn3rqLmHN2DtkRby/8/5ZtnVC0BI+/AYCAV+GbrOKwz1x3GOd1nbOX285MzT2LuzjUiwt9u0Oa3lvSiECiBjuK+zRgc9sei1ve1irvzrHpl6NyxySjmL43MUK5A4U723k8RqlJVPeLzR20OaKdvg7f8RXcX5jrkGsQIYi/HWw3WyPZTclIHhMyYvLOoyB6w6GtKD286Hsz+zEmVqWP01lk3/BabBjrbXsQE+AjZg/oUxkhxjsGe/hxqTsqbnE1VISAnYmGf8WIFp/k0slRp0CksVbQnF6obDYDWgaRKvJ9Q7u3oQMxKnE32qzKSC3R+ZyqJMuHXqfelcLOYPQ6HXY5eSVHIOWutnid6jtrOdwYIaYXpyQp/FFaVZ4R3ndF57YYNtPncFUgx4ZJIi4keD0SvEiZ7Do/oVhxLrVbfyB8tWJxVG1KoIlk9Bsy4T/BquVgpt4tG4RvBUUPSICw0eeqOVaw0yoOzBzYBwnupVwYz/hpwtNeeHeSa/4/XU/tKfFdict2LHELm45p3pl+SqOL0GGCBU8M6fgISvdxXCESekMCm+JUOcMjCvl0l4OgmWtJQ0U6V8uZi7oLe+XOK19c/LwXo6DfnpLREeVShmiWBfa4sr9qdL 1m+AWGaF ZWD5aAwhd2CfWR+Pqot7hZz3XXJlBtM8pdcne3OwTrlZj4Dw94lQ9XSOqOxgjW0/CNls9zTOYiXZNP7c3TM2wLvZuibfK+5HK5kkFkgnLapCKe6mUx3abjLeYnpfxT61P3D2GKBXN5Sdy6IXQFfBjtims/Bowa8fH1F7OOu7R6F5RSaF5Q87V2km9KgY9kSVRnYd5KmLindE3vRt5UPBqiiz6spPKMoofOkb1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Stored in the first tail page's flags, this flag replaces the destructor. That removes the last of the destructors, so remove all references to folio_dtor and compound_dtor. Signed-off-by: Matthew Wilcox (Oracle) --- Documentation/admin-guide/kdump/vmcoreinfo.rst | 4 ++-- include/linux/mm.h | 14 -------------- include/linux/mm_types.h | 2 -- include/linux/page-flags.h | 6 ++++++ kernel/crash_core.c | 1 - mm/huge_memory.c | 4 ++-- mm/internal.h | 1 - mm/page_alloc.c | 7 +------ 8 files changed, 11 insertions(+), 28 deletions(-) diff --git a/Documentation/admin-guide/kdump/vmcoreinfo.rst b/Documentation/admin-guide/kdump/vmcoreinfo.rst index baa1c355741d..3bd38ac0e7de 100644 --- a/Documentation/admin-guide/kdump/vmcoreinfo.rst +++ b/Documentation/admin-guide/kdump/vmcoreinfo.rst @@ -141,8 +141,8 @@ nodemask_t The size of a nodemask_t type. Used to compute the number of online nodes. -(page, flags|_refcount|mapping|lru|_mapcount|private|compound_dtor|compound_order|compound_head) -------------------------------------------------------------------------------------------------- +(page, flags|_refcount|mapping|lru|_mapcount|private|compound_order|compound_head) +---------------------------------------------------------------------------------- User-space tools compute their values based on the offset of these variables. The variables are used when excluding unnecessary pages. diff --git a/include/linux/mm.h b/include/linux/mm.h index c8c8b1fd64d3..cf0ae8c51d7f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1267,20 +1267,6 @@ void folio_copy(struct folio *dst, struct folio *src); unsigned long nr_free_buffer_pages(void); -/* Compound pages may have a special destructor */ -enum compound_dtor_id { - COMPOUND_PAGE_DTOR, - TRANSHUGE_PAGE_DTOR, - NR_COMPOUND_DTORS -}; - -static inline void folio_set_compound_dtor(struct folio *folio, - enum compound_dtor_id compound_dtor) -{ - VM_BUG_ON_FOLIO(compound_dtor >= NR_COMPOUND_DTORS, folio); - folio->_folio_dtor = compound_dtor; -} - void destroy_large_folio(struct folio *folio); /* Returns the number of bytes in this potentially compound page. */ diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index da538ff68953..d45a2b8041e0 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -282,7 +282,6 @@ static inline struct page *encoded_page_ptr(struct encoded_page *page) * @_refcount: Do not access this member directly. Use folio_ref_count() * to find how many references there are to this folio. * @memcg_data: Memory Control Group data. - * @_folio_dtor: Which destructor to use for this folio. * @_folio_order: Do not use directly, call folio_order(). * @_entire_mapcount: Do not use directly, call folio_entire_mapcount(). * @_nr_pages_mapped: Do not use directly, call folio_mapcount(). @@ -336,7 +335,6 @@ struct folio { unsigned long _flags_1; unsigned long _head_1; /* public: */ - unsigned char _folio_dtor; unsigned char _folio_order; atomic_t _entire_mapcount; atomic_t _nr_pages_mapped; diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 01d98695e79f..aabf50dc71a3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -183,6 +183,9 @@ enum pageflags { /* Is a hugetlb page. Stored in first tail page. */ PG_hugetlb = PG_writeback, + /* Has a deferred list (may be empty). First tail page. */ + PG_deferred_list = PG_reclaim, + /* non-lru isolated movable page */ PG_isolated = PG_reclaim, @@ -809,6 +812,9 @@ static inline void ClearPageCompound(struct page *page) BUG_ON(!PageHead(page)); ClearPageHead(page); } +PAGEFLAG(DeferredList, deferred_list, PF_SECOND) +#else +TESTPAGEFLAG_FALSE(DeferredList, deferred_list) #endif #define PG_head_mask ((1UL << PG_head)) diff --git a/kernel/crash_core.c b/kernel/crash_core.c index dd5f87047d06..934dd86e19f5 100644 --- a/kernel/crash_core.c +++ b/kernel/crash_core.c @@ -455,7 +455,6 @@ static int __init crash_save_vmcoreinfo_init(void) VMCOREINFO_OFFSET(page, lru); VMCOREINFO_OFFSET(page, _mapcount); VMCOREINFO_OFFSET(page, private); - VMCOREINFO_OFFSET(folio, _folio_dtor); VMCOREINFO_OFFSET(folio, _folio_order); VMCOREINFO_OFFSET(page, compound_head); VMCOREINFO_OFFSET(pglist_data, node_zones); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 99e36ad540c4..3b5db99eb7ac 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -583,7 +583,7 @@ void prep_transhuge_page(struct page *page) VM_BUG_ON_FOLIO(folio_order(folio) < 2, folio); INIT_LIST_HEAD(&folio->_deferred_list); - folio_set_compound_dtor(folio, TRANSHUGE_PAGE_DTOR); + folio_set_deferred_list(folio); } static inline bool is_transparent_hugepage(struct page *page) @@ -595,7 +595,7 @@ static inline bool is_transparent_hugepage(struct page *page) folio = page_folio(page); return is_huge_zero_page(&folio->page) || - folio->_folio_dtor == TRANSHUGE_PAGE_DTOR; + folio_test_deferred_list(folio); } static unsigned long __thp_get_unmapped_area(struct file *filp, diff --git a/mm/internal.h b/mm/internal.h index 5a03bc4782a2..e3d11119b04e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -417,7 +417,6 @@ static inline void prep_compound_head(struct page *page, unsigned int order) { struct folio *folio = (struct folio *)page; - folio_set_compound_dtor(folio, COMPOUND_PAGE_DTOR); folio_set_order(folio, order); atomic_set(&folio->_entire_mapcount, -1); atomic_set(&folio->_nr_pages_mapped, 0); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 21af71aea6eb..9fe9209605a5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -582,9 +582,6 @@ static inline void free_the_page(struct page *page, unsigned int order) * The remaining PAGE_SIZE pages are called "tail pages". PageTail() is encoded * in bit 0 of page->compound_head. The rest of bits is pointer to head page. * - * The first tail page's ->compound_dtor describes how to destroy the - * compound page. - * * The first tail page's ->compound_order holds the order of allocation. * This usage means that zero-order pages may not be compound. */ @@ -603,14 +600,12 @@ void prep_compound_page(struct page *page, unsigned int order) void destroy_large_folio(struct folio *folio) { - enum compound_dtor_id dtor = folio->_folio_dtor; - if (folio_test_hugetlb(folio)) { free_huge_page(folio); return; } - if (folio_test_transhuge(folio) && dtor == TRANSHUGE_PAGE_DTOR) + if (folio_test_deferred_list(folio)) free_transhuge_folio(folio); mem_cgroup_uncharge(folio); free_the_page(&folio->page, folio_order(folio));