From patchwork Thu Aug 11 23:16:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 12941798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32D9CC25B06 for ; Thu, 11 Aug 2022 23:16:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44BED8E0001; Thu, 11 Aug 2022 19:16:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D4B28E0002; Thu, 11 Aug 2022 19:16:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 164658E0001; Thu, 11 Aug 2022 19:16:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E670D6B007B for ; Thu, 11 Aug 2022 19:16:48 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BF7D8A19BE for ; Thu, 11 Aug 2022 23:16:48 +0000 (UTC) X-FDA: 79788873696.30.A67A1D1 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by imf10.hostedemail.com (Postfix) with ESMTP id 72431C0185 for ; Thu, 11 Aug 2022 23:16:48 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 193645C013A; Thu, 11 Aug 2022 19:16:48 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 11 Aug 2022 19:16:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm1; t=1660259808; x= 1660346208; bh=3J6erN9Meq/qjQOGkR6S/xWYYRm40MXKsAmQN5L+EFk=; b=t 0SW/+nwNBkcP5/lWXHShtewEsmEPAI04/9aw2b9RP2wvNLXAFCo5e2pNZGGQc0/5 enT6UZIKJ63YsQ+DOQOstXJMI2NZ2rweD8x7gQsCgyZ6t9hTPvkEBNcIx27MSmoR NPecHeLTR8ULn03DdCssP70gdFtUJ1q45yeKYawweuOFT4QW5gLntHUti5i3YgoJ VSDuCKoSkcL8DQttPdRd+4kOMfIw7NJfVQ6Ee5Tu7in2tW9GZnj4zcx04vlRTWzn VdyTLID40U64nuwVJ0+kziZaPk9L1ocb7MTGqgq9ehqDjUad68S8lx9E0EUCqYD3 GPp1Ce7fDbNHWqf3O+56g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1660259808; x=1660346208; bh=3 J6erN9Meq/qjQOGkR6S/xWYYRm40MXKsAmQN5L+EFk=; b=NatkXLZ8QrtCktwi7 HEtm6vzzaK0AmMkDeJPoDyP5ESfnZRBL9gJGwZa6Mi/g5zGhUbHGnIZBNpTEPuKI Bd16M1dGGP90ranNjSUBIYyLZqkLMdM3MNN7+Sa89qYoZPcg8z7ZUsDrccXWdXa3 Xa1fYT/p69HpXQ14C1BefJbQgQWXMwimcUUwUuc+MSgsDLPFIDPpJmjVifSE4CIO +6eNGJg8xNm0IslzJN97LIKJ27EhBwycwrlUNR5LNOTChpOza3Z6Kb0BpbRmBz0I ze7HyFDMU6wgVI0aJM3jLsaDBwR/yc0pP3ksyhviAnXktnw4IXS4KWBR7fr40/r2 9il2A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeghedgudelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcu jggrnhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeegge ehudfgudduvdelheehteegledtteeiveeuhfffveekhfevueefieeijeegvdenucevlhhu shhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhessh gvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 11 Aug 2022 19:16:47 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: David Hildenbrand , Matthew Wilcox , Vlastimil Babka , "Kirill A . Shutemov" , Mike Kravetz , John Hubbard , Yang Shi , David Rientjes , James Houghton , Mike Rapoport , linux-kernel@vger.kernel.org Subject: [RFC PATCH v2 04/12] mm: adapt deferred struct page init to new MAX_ORDER. Date: Thu, 11 Aug 2022 19:16:35 -0400 Message-Id: <20220811231643.1012912-5-zi.yan@sent.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220811231643.1012912-1-zi.yan@sent.com> References: <20220811231643.1012912-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660259808; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3J6erN9Meq/qjQOGkR6S/xWYYRm40MXKsAmQN5L+EFk=; b=d/rf7wA28MDbcw8U8L8L6+WTHOEBoIDjt6mkFZNVM7SEV3nb+c5LoNp3obzAla5c79eLt6 aHzI2nkq+CDm1pCfSQiR5H6L62wnPWquxTkNEtKoYMSX/42nHrhkCtBXclIbBhW1UlA2lW uVLxxJRRj4PGeQapBkBmkfuiSJ05tp8= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b="t 0SW/+n"; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=NatkXLZ8; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf10.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.28 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660259808; a=rsa-sha256; cv=none; b=d1AUjSeZcUlWB7C4W1U/3irxmcze24nSTIovc4vdUTYbY45hJz5Hs87lUqkgJe2PcXgaNW sEvR3qH2tI85t9XzzzVRyXSQG7gPbKtL9sDEuVqbai1+nuMvkcO1Tex3Jsuqa6Otae7+V1 mvqRXE96I+vxpxF1+JWhKk8tvZnAVZ0= X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 72431C0185 X-Rspam-User: X-Stat-Signature: yn6ef1wkcbjuzhbijtqxjsja85c5n6na Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b="t 0SW/+n"; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=NatkXLZ8; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf10.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.28 as permitted sender) smtp.mailfrom=zi.yan@sent.com X-HE-Tag: 1660259808-151912 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan deferred_init only initializes first section of a zone and defers the rest and the rest of the zone will be initialized in size of a section. When MAX_ORDER grows beyond a section size, early_page_uninitialised() did not prevent pages beyond first section from initialization, since it only checked the starting pfn and assumes MAX_ORDER is smaller than a section size. In addition, deferred_init_maxorder() uses MAX_ORDER_NR_PAGES as the initialization unit, which can cause the initialized chunk of memory overlapping with other initialization jobs. For the first issue, make early_page_uninitialised() decrease the order for non-deferred memory initialization when it is bigger than first section. For the second issue, when adjust pfn alignment in deferred_init_maxorder(), make sure the alignment is not bigger than a section size. Signed-off-by: Zi Yan --- mm/internal.h | 2 +- mm/memblock.c | 6 ++++-- mm/page_alloc.c | 26 +++++++++++++++++++------- 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 1433e3a6fdd0..cbe745670c6e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -355,7 +355,7 @@ extern int __isolate_free_page(struct page *page, unsigned int order); extern void __putback_isolated_page(struct page *page, unsigned int order, int mt); extern void memblock_free_pages(struct page *page, unsigned long pfn, - unsigned int order); + unsigned int *order); extern void __free_pages_core(struct page *page, unsigned int order); extern void prep_compound_page(struct page *page, unsigned int order); extern void post_alloc_hook(struct page *page, unsigned int order, diff --git a/mm/memblock.c b/mm/memblock.c index d1525463c05e..dc2ce6df8fe3 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1640,7 +1640,9 @@ void __init memblock_free_late(phys_addr_t base, phys_addr_t size) end = PFN_DOWN(base + size); for (; cursor < end; cursor++) { - memblock_free_pages(pfn_to_page(cursor), cursor, 0); + unsigned int order = 0; + + memblock_free_pages(pfn_to_page(cursor), cursor, &order); totalram_pages_inc(); } } @@ -2035,7 +2037,7 @@ static void __init __free_pages_memory(unsigned long start, unsigned long end) while (start + (1UL << order) > end) order--; - memblock_free_pages(pfn_to_page(start), start, order); + memblock_free_pages(pfn_to_page(start), start, &order); start += (1UL << order); } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 07ad8074950f..3f3af7cd5164 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -463,13 +463,19 @@ static inline bool deferred_pages_enabled(void) } /* Returns true if the struct page for the pfn is uninitialised */ -static inline bool __meminit early_page_uninitialised(unsigned long pfn) +static inline bool __meminit early_page_uninitialised(unsigned long pfn, unsigned int *order) { int nid = early_pfn_to_nid(pfn); if (node_online(nid) && pfn >= NODE_DATA(nid)->first_deferred_pfn) return true; + /* clamp down order to not exceed first_deferred_pfn */ + if (order) + *order = min_t(unsigned int, + *order, + ilog2(NODE_DATA(nid)->first_deferred_pfn - pfn)); + return false; } @@ -515,7 +521,7 @@ static inline bool deferred_pages_enabled(void) return false; } -static inline bool early_page_uninitialised(unsigned long pfn) +static inline bool early_page_uninitialised(unsigned long pfn, unsigned int *order) { return false; } @@ -1644,7 +1650,7 @@ static void __meminit init_reserved_page(unsigned long pfn) pg_data_t *pgdat; int nid, zid; - if (!early_page_uninitialised(pfn)) + if (!early_page_uninitialised(pfn, NULL)) return; nid = early_pfn_to_nid(pfn); @@ -1800,11 +1806,11 @@ int __meminit early_pfn_to_nid(unsigned long pfn) #endif /* CONFIG_NUMA */ void __init memblock_free_pages(struct page *page, unsigned long pfn, - unsigned int order) + unsigned int *order) { - if (early_page_uninitialised(pfn)) + if (early_page_uninitialised(pfn, order)) return; - __free_pages_core(page, order); + __free_pages_core(page, *order); } /* @@ -2030,7 +2036,13 @@ static unsigned long __init deferred_init_maxorder(u64 *i, struct zone *zone, unsigned long *start_pfn, unsigned long *end_pfn) { - unsigned long mo_pfn = ALIGN(*start_pfn + 1, MAX_ORDER_NR_PAGES); + /* + * deferred_init_memmap_chunk gives out jobs with max size to + * PAGES_PER_SECTION. Do not align mo_pfn beyond that. + */ + unsigned long align = min_t(unsigned long, + MAX_ORDER_NR_PAGES, PAGES_PER_SECTION); + unsigned long mo_pfn = ALIGN(*start_pfn + 1, align); unsigned long spfn = *start_pfn, epfn = *end_pfn; unsigned long nr_pages = 0; u64 j = *i;