From patchwork Fri Jul 10 12:09:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Song Bao Hua (Barry Song)" X-Patchwork-Id: 11656445 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E89CB13B4 for ; Fri, 10 Jul 2020 12:13:45 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BED0B20748 for ; Fri, 10 Jul 2020 12:13:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="PB63jl5Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BED0B20748 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=OWJjZhXXjkWirUAeWnkS7CXVv2bLllnP9mObVILCOKk=; b=PB63jl5Yw1Na5JyklxzETVJrLq AWlVzZXsF9S3w8vGv6US7+AcP4Iy0UC1WOK+ewdjNfecCuictg4TWvGGrCVgLNI33TnhL3q0hpGO5 Ae9sqYyl9s3VTGZvbBqfXbVD0nG12H38lpVg3prad0BGZMMZaz6fPG2A6xBIpdwhubWPrB3lkjoAc t2ms8G86f01T95bxZ+FQ61PfrBzUfuiwwZX/Ct9DRWmeNEvquFqssRNGLSxWTZHvQBNaYPm96zNW4 aHeqMW+MSPi05rxD/QK779Tq3PIVXRhVFlrh5RjVbNlpGGv80Zn6R96WqScNkEiIqtSacy4hKNM/Y 9Seak0Dw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtrso-0007x3-NO; Fri, 10 Jul 2020 12:12:02 +0000 Received: from szxga07-in.huawei.com ([45.249.212.35] helo=huawei.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jtrsm-0007wB-Nm for linux-arm-kernel@lists.infradead.org; Fri, 10 Jul 2020 12:12:01 +0000 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 9A88CBA74490FCAB9C3A; Fri, 10 Jul 2020 20:11:55 +0800 (CST) Received: from SWX921481.china.huawei.com (10.126.202.234) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.487.0; Fri, 10 Jul 2020 20:11:49 +0800 From: Barry Song To: Subject: [PATCH v3] mm/hugetlb: split hugetlb_cma in nodes with memory Date: Sat, 11 Jul 2020 00:09:50 +1200 Message-ID: <20200710120950.37716-1-song.bao.hua@hisilicon.com> X-Mailer: git-send-email 2.21.0.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.126.202.234] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200710_081201_044799_A991DA34 X-CRM114-Status: GOOD ( 18.15 ) X-Spam-Score: 0.7 (/) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [45.249.212.35 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [45.249.212.35 listed in wl.mailspike.net] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 3.0 AC_FROM_MANY_DOTS Multiple periods in From user name X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Barry Song , Anshuman Khandual , Catalin Marinas , x86@kernel.org, linux-kernel@vger.kernel.org, linuxarm@huawei.com, linux-mm@kvack.org, Ingo Molnar , Borislav Petkov , Jonathan Cameron , "H . Peter Anvin" , Thomas Gleixner , Mike Rapoport , Will Deacon , Roman Gushchin , linux-arm-kernel@lists.infradead.org, Mike Kravetz Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Online nodes are not necessarily memory containing nodes. Splitting huge_cma in online nodes can lead to inconsistent hugetlb_cma size with user setting. For example, for one system with 4 numa nodes and only one of them has memory, if users set hugetlb_cma to 4GB, it will split into four 1GB. So only the node with memory will get 1GB CMA. All other three nodes get nothing. That means the whole system gets only 1GB CMA while users ask for 4GB. Thus, it is more sensible to split hugetlb_cma in nodes with memory. For the above case, the only node with memory will reserve 4GB cma which is same with user setting in bootargs. In order to split cma in nodes with memory, hugetlb_cma_reserve() should scan over those nodes with N_MEMORY state rather than N_ONLINE state. That means the function should be called only after arch code has finished setting the N_MEMORY state of nodes. The problem is always there if N_ONLINE != N_MEMORY. It is a general problem to all platforms. But there is some trivial difference among different architectures. For example, for ARM64, before hugetlb_cma_reserve() is called, all nodes have got N_ONLINE state. So hugetlb will get inconsistent cma size when some online nodes have no memory. For x86 case, the problem is hidden because X86 happens to just set N_ONLINE on the nodes with memory when hugetlb_cma_reserve() is called. Anyway, this patch moves to scan N_MEMORY in hugetlb_cma_reserve() and lets both x86 and ARM64 call the function after N_MEMORY state is ready. It also documents the requirement in the definition of hugetlb_cma_reserve(). Cc: Roman Gushchin Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: H. Peter Anvin Cc: Mike Kravetz Cc: Mike Rapoport Cc: Andrew Morton Cc: Anshuman Khandual Cc: Jonathan Cameron Signed-off-by: Barry Song Signed-off-by: Mike Kravetz --- v3: another try to refine changelog with respect to Anshuman Khandual's comments arch/arm64/mm/init.c | 19 ++++++++++--------- arch/x86/kernel/setup.c | 12 +++++++++--- mm/hugetlb.c | 11 +++++++++-- 3 files changed, 28 insertions(+), 14 deletions(-) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 1e93cfc7c47a..420f5e55615c 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -420,15 +420,6 @@ void __init bootmem_init(void) arm64_numa_init(); - /* - * must be done after arm64_numa_init() which calls numa_init() to - * initialize node_online_map that gets used in hugetlb_cma_reserve() - * while allocating required CMA size across online nodes. - */ -#ifdef CONFIG_ARM64_4K_PAGES - hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); -#endif - /* * Sparsemem tries to allocate bootmem in memory_present(), so must be * done after the fixed reservations. @@ -438,6 +429,16 @@ void __init bootmem_init(void) sparse_init(); zone_sizes_init(min, max); + /* + * must be done after zone_sizes_init() which calls free_area_init() + * that calls node_set_state() to initialize node_states[N_MEMORY] + * because hugetlb_cma_reserve() will scan over nodes with N_MEMORY + * state + */ +#ifdef CONFIG_ARM64_4K_PAGES + hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); +#endif + memblock_dump_all(); } diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index a3767e74c758..a1a9712090ae 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1164,9 +1164,6 @@ void __init setup_arch(char **cmdline_p) initmem_init(); dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT); - if (boot_cpu_has(X86_FEATURE_GBPAGES)) - hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); - /* * Reserve memory for crash kernel after SRAT is parsed so that it * won't consume hotpluggable memory. @@ -1180,6 +1177,15 @@ void __init setup_arch(char **cmdline_p) x86_init.paging.pagetable_init(); + /* + * must be done after zone_sizes_init() which calls free_area_init() + * that calls node_set_state() to initialize node_states[N_MEMORY] + * because hugetlb_cma_reserve() will scan over nodes with N_MEMORY + * state + */ + if (boot_cpu_has(X86_FEATURE_GBPAGES)) + hugetlb_cma_reserve(PUD_SHIFT - PAGE_SHIFT); + kasan_init(); /* diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bc3304af40d0..32b5035ffec1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5665,6 +5665,13 @@ static int __init cmdline_parse_hugetlb_cma(char *p) early_param("hugetlb_cma", cmdline_parse_hugetlb_cma); +/* + * hugetlb_cma_reserve() - reserve CMA for gigantic pages on nodes with memory + * + * must be called after free_area_init() that updates N_MEMORY via node_set_state(). + * hugetlb_cma_reserve() scans over N_MEMORY nodemask and hence expects the platforms + * to have initialized N_MEMORY state. + */ void __init hugetlb_cma_reserve(int order) { unsigned long size, reserved, per_node; @@ -5685,12 +5692,12 @@ void __init hugetlb_cma_reserve(int order) * If 3 GB area is requested on a machine with 4 numa nodes, * let's allocate 1 GB on first three nodes and ignore the last one. */ - per_node = DIV_ROUND_UP(hugetlb_cma_size, nr_online_nodes); + per_node = DIV_ROUND_UP(hugetlb_cma_size, num_node_state(N_MEMORY)); pr_info("hugetlb_cma: reserve %lu MiB, up to %lu MiB per node\n", hugetlb_cma_size / SZ_1M, per_node / SZ_1M); reserved = 0; - for_each_node_state(nid, N_ONLINE) { + for_each_node_state(nid, N_MEMORY) { int res; size = min(per_node, hugetlb_cma_size - reserved);