From patchwork Thu Sep 22 01:12:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 12984361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80E1DC32771 for ; Thu, 22 Sep 2022 01:13:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A72F8000F; Wed, 21 Sep 2022 21:13:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DFF080007; Wed, 21 Sep 2022 21:13:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 497B38000F; Wed, 21 Sep 2022 21:13:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 35B2B80007 for ; Wed, 21 Sep 2022 21:13:03 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 09A2F161338 for ; Thu, 22 Sep 2022 01:13:03 +0000 (UTC) X-FDA: 79937947446.13.4B942A6 Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by imf07.hostedemail.com (Postfix) with ESMTP id B5CDE40019 for ; Thu, 22 Sep 2022 01:13:02 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 61DEA5C0148; Wed, 21 Sep 2022 21:13:02 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 21 Sep 2022 21:13:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to :reply-to:sender:subject:subject:to:to; s=fm2; t=1663809182; x= 1663895582; bh=Nv99GiHsEcawIlId0x0YqzolK3s7pNZObtF5Lg/ft28=; b=L JfyrA0jVaYrlzfxQwvyHHOLXlUY2oScbbstQsljB4o5boFM/FGWiI1igWh+NHzAb zKYGe0JtTtOJpUrmK0GJp1oYXQ5lCf9eqvwof9eEvIwU/WD23revC2t4Z87Jl2KH dUsc9AbMv06dNVj2r74mGvCTJU42xi2u8YTClW//ZtGFMHEYhasxI/sBnBaE610R wPlixukSML+R9yupUt8tgpqxOz59Jbv/ZSPn7R8/n2TabPLirhpP7B5jHUGBU8IB Dopd+CW8gUw0XcF78bujVC+1zNS4/x80N66Emyjt/nW4TAr9CzxDFUXEFYWyhwnf W39rcuwNQWYpEjpX1oZEg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1663809182; x=1663895582; bh=N v99GiHsEcawIlId0x0YqzolK3s7pNZObtF5Lg/ft28=; b=shhDofdWwykdlWWls 3aki8pc90kkgeL0GzQ5nWWo9d5u97v04wT2Y2KIW8FUAHkIfFWHa6qbiTd6CmKjj zY2ioMXPY1xNj3mKfRsBzk/nsSGrfL2O6pmH3kd34xoR+HYWKLIsGIdQddpQHnYU vtHy5Jzdh+NHRfTuEvBUHb58yUWPWc7PPSULpg4I2REaFNlLjUxUBGDcJMA46Hgg kC6luK8PgaaXn6/ncb6/P6UojGO0S4PFDdxvBgpiY+/3Cg8JcKdEP8lDISJGC7nS gi12gV9+DLHcjFfSBavYMbpgLxkrbbLinarI2I6t2lr+fVw/IBKcudBH8x3TSIGE LAO7g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeefvddggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhhrggfgsedtqhertdertddtnecuhfhrohhmpegkihcu jggrnhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeegge ehudfgudduvdelheehteegledtteeiveeuhfffveekhfevueefieeijeegvdenucevlhhu shhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhessh gvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 21 Sep 2022 21:13:01 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: Zi Yan , David Hildenbrand , Matthew Wilcox , Vlastimil Babka , "Kirill A . Shutemov" , Mike Kravetz , John Hubbard , Yang Shi , David Rientjes , James Houghton , Mike Rapoport , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org Subject: [PATCH v1 09/12] mm: Make MAX_ORDER of buddy allocator configurable via Kconfig SET_MAX_ORDER. Date: Wed, 21 Sep 2022 21:12:49 -0400 Message-Id: <20220922011252.2266780-10-zi.yan@sent.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220922011252.2266780-1-zi.yan@sent.com> References: <20220922011252.2266780-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663809182; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Nv99GiHsEcawIlId0x0YqzolK3s7pNZObtF5Lg/ft28=; b=2jSvUfEQTCJJyLKzVjNCPTBhHF+aXF+nEbVWWcXtH+RoRoOCb5YQoexxm00MEhcWesBY0H Vd1JUZQCOzolxRYT1z0Nez9xZsftEPUvDxNG4UCGgQoJFbTGYnYfTZVZkKCHahuJM2DboJ xmFy0bDLpOKnHP83AQt/fF1HfoO55oc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="L JfyrA0"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=shhDofdW; spf=pass (imf07.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663809182; a=rsa-sha256; cv=none; b=x55Yhvi8hbN3+Kdy4MyxTP+pQezjrUGpbUv/8Y3d16Y8gGrNqToBS0V9gLmrli22SwAmp7 210VNer77NX9B2Tft2VmAAFFeFwoPIb7oVLMLqke4eroEb66o4jNdcFsNbWXKXK9FWVc1k I7MEGObi0nmi2jvYdQOTlZ3efIg2RDQ= X-Stat-Signature: t6p96pqx9s3yumburquiodsbhsaatyt1 X-Rspamd-Queue-Id: B5CDE40019 X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="L JfyrA0"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=shhDofdW; spf=pass (imf07.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com X-Rspamd-Server: rspam09 X-HE-Tag: 1663809182-225266 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan With SPARSEMEM_VMEMMAP, all struct page are virtually contigous, thus kernel can manipulate arbitrarily large pages. By checking PFN validity during buddy page merging process, all free pages in buddy allocator's free area have their PFNs contiguous even if the system has several not physically contiguous memory sections. With these two conditions, it is OK to remove the restriction of MAX_ORDER + PAGE_SHIFT < SECTION_SIZE_BITS and change MAX_ORDER freely. Add SET_MAX_ORDER to allow MAX_ORDER adjustment when arch does not set its own MAX_ORDER via ARCH_FORCE_MAX_ORDER. Make it depend on SPARSEMEM_VMEMMAP, when MAX_ORDER is not limited by SECTION_SIZE_BITS. Signed-off-by: Zi Yan Cc: Kees Cook Cc: Peter Zijlstra Cc: Nicholas Piggin Cc: Thomas Gleixner Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org --- arch/Kconfig | 4 ++++ include/linux/mmzone.h | 17 ++++++++++++++--- mm/Kconfig | 14 ++++++++++++++ mm/internal.h | 2 -- 4 files changed, 32 insertions(+), 5 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 1c2599618eeb..e51c759a82ad 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -11,6 +11,10 @@ source "arch/$(SRCARCH)/Kconfig" menu "General architecture-dependent options" +config ARCH_FORCE_MAX_ORDER + int + default "0" + config CRASH_CORE bool diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index de1548f4fc07..da5745fa15c3 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -24,11 +24,14 @@ #include /* Free memory management - zoned buddy allocator. */ -#ifndef CONFIG_ARCH_FORCE_MAX_ORDER -#define MAX_ORDER 10 -#else +#ifdef CONFIG_SET_MAX_ORDER +#define MAX_ORDER CONFIG_SET_MAX_ORDER +#elif CONFIG_ARCH_FORCE_MAX_ORDER != 0 #define MAX_ORDER CONFIG_ARCH_FORCE_MAX_ORDER +#else +#define MAX_ORDER 10 #endif + #define MAX_ORDER_NR_PAGES (1 << MAX_ORDER) /* @@ -1588,9 +1591,17 @@ static inline bool movable_only_nodes(nodemask_t *nodes) #define SECTION_BLOCKFLAGS_BITS \ ((1UL << (PFN_SECTION_SHIFT - pageblock_order)) * NR_PAGEBLOCK_BITS) +/* + * The MAX_ORDER check is not necessary when CONFIG_SET_MAX_ORDER is set, since + * it depends on CONFIG_SPARSEMEM_VMEMMAP, where all struct page are virtually + * contiguous, thus > section size pages can be allocated and manipulated + * without worrying about non-contiguous struct page. + */ +#ifndef CONFIG_SET_MAX_ORDER #if (MAX_ORDER + PAGE_SHIFT) > SECTION_SIZE_BITS #error Allocator MAX_ORDER exceeds SECTION_SIZE #endif +#endif /* CONFIG_SET_MAX_ORDER*/ static inline unsigned long pfn_to_section_nr(unsigned long pfn) { diff --git a/mm/Kconfig b/mm/Kconfig index ae6711d24e4a..9c7280acd528 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -435,6 +435,20 @@ config SPARSEMEM_VMEMMAP pfn_to_page and page_to_pfn operations. This is the most efficient option when sufficient kernel resources are available. +config SET_MAX_ORDER + int "Set maximum order of buddy allocator" + depends on SPARSEMEM_VMEMMAP && (ARCH_FORCE_MAX_ORDER = 0) + range 10 255 + default "10" + help + The kernel memory allocator divides physically contiguous memory + blocks into "zones", where each zone is a power of two number of + pages. This option selects the largest power of two that the kernel + keeps in the memory allocator. If you need to allocate very large + blocks of physically contiguous memory, then you may need to + increase this value. A value of 10 means that the largest free memory + block is 2^10 pages. + config HAVE_MEMBLOCK_PHYS_MAP bool diff --git a/mm/internal.h b/mm/internal.h index 1b1abfc2196e..1c3f260930d8 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -303,8 +303,6 @@ static inline bool page_is_buddy(struct page *page, struct page *buddy, * 2) Any buddy B will have an order O+1 parent P which * satisfies the following equation: * P = B & ~(1 << O) - * - * Assumption: *_mem_map is contiguous at least up to MAX_PHYS_CONTIG_ORDER */ static inline unsigned long __find_buddy_pfn(unsigned long page_pfn, unsigned int order)