From patchwork Thu Sep 22 01:12:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 12984359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E11C6ECAAD8 for ; Thu, 22 Sep 2022 01:13:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E84198000E; Wed, 21 Sep 2022 21:13:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D47A080007; Wed, 21 Sep 2022 21:13:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B04A18000E; Wed, 21 Sep 2022 21:13:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9872E80007 for ; Wed, 21 Sep 2022 21:13:02 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 65657AB4CF for ; Thu, 22 Sep 2022 01:13:02 +0000 (UTC) X-FDA: 79937947404.01.DDB26FD Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by imf26.hostedemail.com (Postfix) with ESMTP id 1B1B114001F for ; Thu, 22 Sep 2022 01:13:01 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id B15CF5C0170; Wed, 21 Sep 2022 21:13:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 21 Sep 2022 21:13:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; t=1663809181; x= 1663895581; bh=drAyzX7q0CQRps8Kpo+CSOrvAuWQIQAnWi+G2OzIrVU=; b=i GuLuO3jU2GQuuLSP5rquoJ9Q66R9kNIQ06+wnMo83snRE++DdvM+3D/Qr3/PS7dL ZyOqx8UdYyIIUcZyAKFwdWzoDRpqtLgdHyj9dPY4Gg20oEc3sAvwHRiECv9sJJQe UPnHxjeMV+J4wwT2/4O75pzgRIqW+AcGKtSb6rAmMV0FvXg1D8s5GiURoFusv3OA E9SnpY4j7t3CdnUGeGnWHgWpZn1HM265Wtrvpmd9mXSAFt1nYo5Q7hYYmc3jYrII N8YXijBBCwM3rTvotVGVdFPFt0XGP4Q/0080WOwndFipQogqj/DNjYBQklbhYNya OVl/lw8ZAl9jtyzvZbtQw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1663809181; x=1663895581; bh=d rAyzX7q0CQRps8Kpo+CSOrvAuWQIQAnWi+G2OzIrVU=; b=zvW5wzKvkvNzVbc6S GlLj8OKCD6LGmwYPQNZJPP18bJ5OLK50VTs/Kz1A5qZXr8jN8CQyJPE8n1NCxa7/ shDr0aEQfZR9Ub800iz4TXEvRm2CRTRlabieZQQKOkz2iCE3BVe+SgZ8x69Ltk2O nvFPy3tWJ4Kz5Pdo8iKevp84USnXo7BVbd1Ux5emLK7m0+qzO9F3jnmymg2lUocc z1RWcLtktLIV5Sm3wvpKmiy5is7Y2yVbDpcTSPUUO2L8LNkoUMKWvcn45t3P/RCp 9o2GVYxmJN+GBw1csGAFLZHEXLzTdir9qCT0sUDqU6EZPeujEPlevqsKspnpfGXk XdjqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeefvddggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcu jggrnhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeegge ehudfgudduvdelheehteegledtteeiveeuhfffveekhfevueefieeijeegvdenucevlhhu shhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhessh gvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 21 Sep 2022 21:13:00 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: Zi Yan , David Hildenbrand , Matthew Wilcox , Vlastimil Babka , "Kirill A . Shutemov" , Mike Kravetz , John Hubbard , Yang Shi , David Rientjes , James Houghton , Mike Rapoport , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org Subject: [PATCH v1 08/12] mm: replace MAX_ORDER when it is used to indicate max physical contiguity. Date: Wed, 21 Sep 2022 21:12:48 -0400 Message-Id: <20220922011252.2266780-9-zi.yan@sent.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220922011252.2266780-1-zi.yan@sent.com> References: <20220922011252.2266780-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663809182; a=rsa-sha256; cv=none; b=aX1Gx2nL5qCSHVVqm64Mw+x8XBC49H8j9St1DW7LtjIxCpJ0VFmGXR4ZUtuDR2zJYVnWVk jxYXo7jMvZ0de8tNIId4bgfrqw26Y2ZqTNIEWk4U29CKtBMaf+PmV7pCpeJNuKNBVsWres Fp9/elOHZlFsIXmplmH2jBbV0QpRVKI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="i GuLuO3"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=zvW5wzKv; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf26.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663809182; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=drAyzX7q0CQRps8Kpo+CSOrvAuWQIQAnWi+G2OzIrVU=; b=lgmYpSnq84RCSFvDZi94b+LVbWkMoybizZ5RpSpxnwQaPS7jq6qmmPmmiNgwRgB+3ha8Si xCgQeRj2fIdfDit/KxrERXZdmSgDFVFEdbTd4bEUY+riKX7AHvKgsJ0fZdkMZjloNa64i+ z2SKukq1k2DzZ3T1hbaCEIImvNIlRRE= X-Rspam-User: X-Stat-Signature: 96wpa396ynsqm3scawuca8e8wd4s9891 X-Rspamd-Queue-Id: 1B1B114001F Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="i GuLuO3"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=zvW5wzKv; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf26.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com X-Rspamd-Server: rspam07 X-HE-Tag: 1663809181-829903 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan MAX_ORDER is limited at a memory section size, thus widely used as a variable to indicate maximum physically contiguous page size. But this limitation is no longer necessary as kernel only supports sparse memory model. Add a new variable MAX_PHYS_CONTIG_ORDER to replace such uses of MAX_ORDER. Signed-off-by: Zi Yan --- Documentation/admin-guide/kernel-parameters.txt | 2 +- arch/sparc/mm/tsb.c | 4 ++-- arch/um/kernel/um_arch.c | 4 ++-- include/linux/pageblock-flags.h | 12 ++++++++++++ kernel/dma/pool.c | 8 ++++---- mm/hugetlb.c | 2 +- mm/internal.h | 2 +- mm/memory.c | 4 ++-- mm/memory_hotplug.c | 6 +++--- mm/page_isolation.c | 2 +- mm/page_reporting.c | 2 +- 11 files changed, 30 insertions(+), 18 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index e1094851c328..5f633844daac 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3907,7 +3907,7 @@ [KNL] Minimal page reporting order Format: Adjust the minimal page reporting order. The page - reporting is disabled when it exceeds MAX_ORDER. + reporting is disabled when it exceeds MAX_PHYS_CONTIG_ORDER. panic= [KNL] Kernel behaviour on panic: delay timeout > 0: seconds before rebooting diff --git a/arch/sparc/mm/tsb.c b/arch/sparc/mm/tsb.c index 912205787161..15c31d050dab 100644 --- a/arch/sparc/mm/tsb.c +++ b/arch/sparc/mm/tsb.c @@ -402,8 +402,8 @@ void tsb_grow(struct mm_struct *mm, unsigned long tsb_index, unsigned long rss) unsigned long new_rss_limit; gfp_t gfp_flags; - if (max_tsb_size > (PAGE_SIZE << MAX_ORDER)) - max_tsb_size = (PAGE_SIZE << MAX_ORDER); + if (max_tsb_size > (PAGE_SIZE << MAX_PHYS_CONTIG_ORDER)) + max_tsb_size = (PAGE_SIZE << MAX_PHYS_CONTIG_ORDER); new_cache_index = 0; for (new_size = 8192; new_size < max_tsb_size; new_size <<= 1UL) { diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c index e0de60e503b9..52a474f4f1c7 100644 --- a/arch/um/kernel/um_arch.c +++ b/arch/um/kernel/um_arch.c @@ -368,10 +368,10 @@ int __init linux_main(int argc, char **argv) max_physmem = TASK_SIZE - uml_physmem - iomem_size - MIN_VMALLOC; /* - * Zones have to begin on a 1 << MAX_ORDER page boundary, + * Zones have to begin on a 1 << MAX_PHYS_CONTIG_ORDER page boundary, * so this makes sure that's true for highmem */ - max_physmem &= ~((1 << (PAGE_SHIFT + MAX_ORDER)) - 1); + max_physmem &= ~((1 << (PAGE_SHIFT + MAX_PHYS_CONTIG_ORDER)) - 1); if (physmem_size + iomem_size > max_physmem) { highmem = physmem_size + iomem_size - max_physmem; physmem_size -= highmem; diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flags.h index 95589b24fff9..5469ba6f97c1 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -61,6 +61,18 @@ extern unsigned int pageblock_order; #define pageblock_start_pfn(pfn) ALIGN_DOWN((pfn), pageblock_nr_pages) #define pageblock_end_pfn(pfn) ALIGN((pfn) + 1, pageblock_nr_pages) +/* + * memory section is only defined in sparsemem and in flatmem, pages are always + * physically contiguous, but we use MAX_ORDER since all users assume so. + */ +#ifdef CONFIG_FLATMEM +#define MAX_PHYS_CONTIG_ORDER MAX_ORDER +#else /* SPARSEMEM */ +#define MAX_PHYS_CONTIG_ORDER (min(PFN_SECTION_SHIFT, MAX_ORDER)) +#endif /* CONFIG_FLATMEM */ + +#define MAX_PHYS_CONTIG_NR_PAGES (1UL << MAX_PHYS_CONTIG_ORDER) + /* Forward declaration */ struct page; diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 1acec2e22827..11af1063eb86 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -84,8 +84,8 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size, void *addr; int ret = -ENOMEM; - /* Cannot allocate larger than MAX_ORDER */ - order = min(get_order(pool_size), MAX_ORDER); + /* Cannot allocate larger than MAX_PHYS_CONTIG_ORDER */ + order = min(get_order(pool_size), MAX_PHYS_CONTIG_ORDER); do { pool_size = 1 << (PAGE_SHIFT + order); @@ -190,11 +190,11 @@ static int __init dma_atomic_pool_init(void) /* * If coherent_pool was not used on the command line, default the pool - * sizes to 128KB per 1GB of memory, min 128KB, max MAX_ORDER. + * sizes to 128KB per 1GB of memory, min 128KB, max MAX_PHYS_CONTIG_ORDER. */ if (!atomic_pool_size) { unsigned long pages = totalram_pages() / (SZ_1G / SZ_128K); - pages = min_t(unsigned long, pages, MAX_ORDER_NR_PAGES); + pages = min_t(unsigned long, pages, MAX_PHYS_CONTIG_NR_PAGES); atomic_pool_size = max_t(size_t, pages << PAGE_SHIFT, SZ_128K); } INIT_WORK(&atomic_pool_work, atomic_pool_work_fn); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 84a4ea87f394..e6c829a581d6 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1922,7 +1922,7 @@ pgoff_t hugetlb_basepage_index(struct page *page) pgoff_t index = page_index(page_head); unsigned long compound_idx; - if (compound_order(page_head) > MAX_ORDER) + if (compound_order(page_head) > MAX_PHYS_CONTIG_ORDER) compound_idx = page_to_pfn(page) - page_to_pfn(page_head); else compound_idx = page - page_head; diff --git a/mm/internal.h b/mm/internal.h index d688c0320cda..1b1abfc2196e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -304,7 +304,7 @@ static inline bool page_is_buddy(struct page *page, struct page *buddy, * satisfies the following equation: * P = B & ~(1 << O) * - * Assumption: *_mem_map is contiguous at least up to MAX_ORDER + * Assumption: *_mem_map is contiguous at least up to MAX_PHYS_CONTIG_ORDER */ static inline unsigned long __find_buddy_pfn(unsigned long page_pfn, unsigned int order) diff --git a/mm/memory.c b/mm/memory.c index 118e5f023597..06f5f2de3a2c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5717,7 +5717,7 @@ void clear_huge_page(struct page *page, unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); - if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { + if (unlikely(pages_per_huge_page > MAX_PHYS_CONTIG_NR_PAGES)) { clear_gigantic_page(page, addr, pages_per_huge_page); return; } @@ -5769,7 +5769,7 @@ void copy_user_huge_page(struct page *dst, struct page *src, .vma = vma, }; - if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { + if (unlikely(pages_per_huge_page > MAX_PHYS_CONTIG_NR_PAGES)) { copy_user_gigantic_page(dst, src, addr, vma, pages_per_huge_page); return; diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 2985d5302c57..994a648c0f5c 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -596,16 +596,16 @@ static void online_pages_range(unsigned long start_pfn, unsigned long nr_pages) unsigned long pfn; /* - * Online the pages in MAX_ORDER aligned chunks. The callback might + * Online the pages in MAX_PHYS_CONTIG_ORDER aligned chunks. The callback might * decide to not expose all pages to the buddy (e.g., expose them * later). We account all pages as being online and belonging to this * zone ("present"). * When using memmap_on_memory, the range might not be aligned to - * MAX_ORDER_NR_PAGES - 1, but pageblock aligned. __ffs() will detect + * MAX_PHYS_CONTIG_NR_PAGES - 1, but pageblock aligned. __ffs() will detect * this and the first chunk to online will be pageblock_nr_pages. */ for (pfn = start_pfn; pfn < end_pfn;) { - int order = min_t(unsigned long, MAX_ORDER, __ffs(pfn)); + int order = min_t(unsigned long, MAX_PHYS_CONTIG_ORDER, __ffs(pfn)); (*online_page_callback)(pfn_to_page(pfn), order); pfn += (1UL << order); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 0c2b7d4f7054..2c183827d365 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -226,7 +226,7 @@ static void unset_migratetype_isolate(struct page *page, int migratetype) */ if (PageBuddy(page)) { order = buddy_order(page); - if (order >= pageblock_order && order <= MAX_ORDER) { + if (order >= pageblock_order && order <= MAX_PHYS_CONTIG_ORDER) { buddy = find_buddy_page_pfn(page, page_to_pfn(page), order, NULL); if (buddy && !is_migrate_isolate_page(buddy)) { diff --git a/mm/page_reporting.c b/mm/page_reporting.c index e3d6f4af34e0..001438f3dbeb 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -248,7 +248,7 @@ page_reporting_process_zone(struct page_reporting_dev_info *prdev, return err; /* Process each free list starting from lowest order/mt */ - for (order = page_reporting_order; order <= MAX_ORDER; order++) { + for (order = page_reporting_order; order <= MAX_PHYS_CONTIG_ORDER; order++) { for (mt = 0; mt < MIGRATE_TYPES; mt++) { /* We do not pull pages from the isolate free list */ if (is_migrate_isolate(mt))