From patchwork Tue Feb 18 18:16:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank van der Linden X-Patchwork-Id: 13980406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC3CCC021AA for ; Tue, 18 Feb 2025 18:17:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58A0028017D; Tue, 18 Feb 2025 13:17:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 51225280176; Tue, 18 Feb 2025 13:17:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38CBF28017D; Tue, 18 Feb 2025 13:17:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0601A280176 for ; Tue, 18 Feb 2025 13:17:21 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BBF52B0444 for ; Tue, 18 Feb 2025 18:17:20 +0000 (UTC) X-FDA: 83133872640.16.E7D28C3 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf18.hostedemail.com (Postfix) with ESMTP id D1C6E1C0004 for ; Tue, 18 Feb 2025 18:17:18 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=PzRApZi8; spf=pass (imf18.hostedemail.com: domain of 3rc60ZwQKCGIFVDLGOOGLE.COMLINUX-MMKVACK.ORG@flex--fvdl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3rc60ZwQKCGIFVDLGOOGLE.COMLINUX-MMKVACK.ORG@flex--fvdl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739902638; a=rsa-sha256; cv=none; b=OWTmjTsK/J1wEcJQt/SmH4tVhi4Wl1FJ6OAd13/dZk+OHFji8n7JmujtO9m8AH9RO8THbw Xp9t9u/6Orfo2A+jcMKYnjAhISJOYYTIjcJ4pYrwVdkdloaqkd02RL6Nd9htjzBd91Biaq eTCAzvD+cnN94zuZDNKDIfiyGo5p3/Y= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=PzRApZi8; spf=pass (imf18.hostedemail.com: domain of 3rc60ZwQKCGIFVDLGOOGLE.COMLINUX-MMKVACK.ORG@flex--fvdl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3rc60ZwQKCGIFVDLGOOGLE.COMLINUX-MMKVACK.ORG@flex--fvdl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739902638; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UzfWv9BqKfCaTLcusLA/6ZlD1dELHDMIUuMG+2mtrpE=; b=HLXm2VSXB1nV5gBrNryjvHNlV+iZh5nLnROeJU5sP4gnqmCRwUcZoY8xHBQZTEyvHy35Bg fvlvxGnJS1YmmggIMS74bdz6LdR2dw76FXhPuLTx9vAG4iv87H/DwwzWo0r5H5shPGzehq Fh9D/ZEDfFrnh/DSPb1i7MZrw7SHKtQ= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2fc5888c192so5628731a91.0 for ; Tue, 18 Feb 2025 10:17:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1739902638; x=1740507438; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UzfWv9BqKfCaTLcusLA/6ZlD1dELHDMIUuMG+2mtrpE=; b=PzRApZi8fi6lL2IQoRguyApY/U86E+VDJGGkvmXvWCT9+yyZG06IUfGt60l65QxQBi Z6+ecTocB2k9Q/7GZS8UgW52bfIgcRgzBff491e3da9imiOV136neathf3R95iqmxNjx n8w1k3P/MbZWtriMjMElS/hoQLr/xOOtR9b5/bHOfO7gXjZj1EHHEQFL0/tZfdWI5Khp 6sZggDlKpe1aiuMU5tPRnrQZlnyiy4YEIcDAi3kViaU4XhAUjkpM5kppTdLz1zy8K1jm J3sQRvsUMmSsgYAnrpbNlUT9q4sqRVZ1VbVsTzrQAiYis5DH+YJfN2PkWIWKFy6qRhcj FvOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739902638; x=1740507438; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UzfWv9BqKfCaTLcusLA/6ZlD1dELHDMIUuMG+2mtrpE=; b=FQBOtKHIOTm8dJTrUN6T+vsEENaubQ9elQGEUJkHu7H87GXuVU5ZETAnhd6GjY4tgM Y6Yd6f9I0OiA2/jIbSW4sttH4bKApk9YHVHXRVEQ3Hy+5iJ9Ts1CV0aT5HE+D9Mxrvmf R+e9jvG/q914Q7RUXXwvmqQXa+6sPuGgVDkL5wNuPJrGjyAO/NXqM8Y3dUtlHql2Cdaa OLVwf4Cj+v/hK5EYdT3D8Z7ZiLc7j8QzPMXZLySNdtzT8KplU4A9R4WWxfPwvvS6ycLh CXf6qAa2hdhQt3qSeMeAC+jDb0VE9dP+0UKXakMB6y7Bvy1LKh6GcLJGw3kjz20ZVkS/ L4pA== X-Forwarded-Encrypted: i=1; AJvYcCWeM+lOBkYNDMKzuwiGnw8Kc7xZkNxd4VtQU3qehz2LlAUkhMFAyvvs+JY+4IBTP9TkK920wls/Xg==@kvack.org X-Gm-Message-State: AOJu0Yw5X5WsU3cxQ/3SmWcG1whBoJV2wX+FBfpV4mofJQ+T0VjpxkH9 k13AJun/KgyAFGQKkK9TQLZV7cFSaEKriq4u7l85e0eFmRVC7JpjfjDmLlzReP8s6FOoPA== X-Google-Smtp-Source: AGHT+IHa8Y+sJQ8IwQWCMO2RNbH8G/VryDlpbdJULvNEszvFgqi1s498qDm7c8r07sVjg2DWswKH4xZf X-Received: from pfbga22.prod.google.com ([2002:a05:6a00:6216:b0:730:8ca0:1f9b]) (user=fvdl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:2e97:b0:730:87cf:a7cd with SMTP id d2e1a72fcca58-7329de6ecbdmr585107b3a.8.1739902637790; Tue, 18 Feb 2025 10:17:17 -0800 (PST) Date: Tue, 18 Feb 2025 18:16:35 +0000 In-Reply-To: <20250218181656.207178-1-fvdl@google.com> Mime-Version: 1.0 References: <20250218181656.207178-1-fvdl@google.com> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog Message-ID: <20250218181656.207178-8-fvdl@google.com> Subject: [PATCH v4 07/27] mm/hugetlb: convert cmdline parameters from setup to early From: Frank van der Linden To: akpm@linux-foundation.org, muchun.song@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: yuzhao@google.com, usamaarif642@gmail.com, joao.m.martins@oracle.com, roman.gushchin@linux.dev, Frank van der Linden X-Stat-Signature: c33c73sy37m4og73bur3j3j9srrasmq6 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D1C6E1C0004 X-Rspam-User: X-HE-Tag: 1739902638-89410 X-HE-Meta: U2FsdGVkX1945vCbBxI6dQXOwuKv3LN00HgYaeTIOpMq0gMVkg/OVeSuLKFNvJ30vOlYneK3B35v8VFUgZsQAiqJYLnI/B1KaprK14XfTIL/cT3pLjRICoKTXzaaKh7e/ixUtXrU8yWbDZOKfEIepVmRa71kP2HUJXkUKz3IP+dUoF6PApXbr67Q4swBRNO4pOXfGpnEeQpCoH93DO6m7J+T7Bgfs8mtxUKnCoRsgjDaGWUVL0maVd8T1ZyygPp8maHfsytYCuhkbO1NZB2vf6R3/pW8EXfa+QwEvndnDaSVt2/kGza8Rl6tBwX7BaSWoKwacfE5gV2yFL13w2nB4XeA+QUO7P/z0UrFXxYZ2yYBZDRTIHN7666ksSFtIBew9bWM4yDpFYWaliikx1BNVU6pCtruD9iGhygas5g/Y0EvnHCNzPjdf7iKjeHkz4W9metp0ZsKPAXVsiW2sBBRLYDw8KBVSYLyaOAmHJsMfOTkBiXe8DAyLy16vGdonCLt4apuFTGh5RPKkr1k0RNas8e0MoDIiP5/BEtJ3ZeQFsIrn2pR/tt6zfQFgoa250dV23xHp9kBTRBsNkhg50dKNF2hqRw04V6DzQQx4dHmQoGqNiFIbtVAVH0sgoA435GO6L34+wD/xNxKi0dVKJ5wYxb6Q9bvFINO6C+CefD1k+7nnrH294WTpoHuj2WQ5c9dUe8pkbJL0W/9M+GpEhKIQfOdSBW3zIrrV7xOgDbypJy+8Jp6CxJI1IgzQGVW5YQTOv9zbleh7XA0Ni+WhGpIs+CDuJbY/9JT9URzD4b9k+u7skft8ik54ojUeLgtcFmEhTmtS1iUlzxkSGkt7Xoq2xLDYgOcGnWOv28/Et4dsa6vpz/L9XK/Z9QLKP75YR1Pwmi/Mn4TTxu/lrAQToQZFDMTAbsOESY0ZPJb0L2EXD+BOmSHiNr2/9YFNGLO5AxKtXaEmICAWxomuYv13B8 BS9jVYN4 6HKTzV02zxSYZIY7G5mscF03VL7Q6QV9ic5C6L2udOasfZSxMVpvRnoiCeJJw67SKGoB3L9nCFYAmKtqLDoB8d/xVWsDftqlUNV9Sj5z7olsi82b6GIyZZE5rJH6aJVsMzN3KqVV+A3IyUW+l1mCNXU/0zdekY05MsQ1+837lKw71sNx6Gg2f7FVO8Gwwe97HT5bji/lCWVMlso2rqck8UllX9W9/iEYCzRs1ZqZsRyI0Npea5Y/3ttJ0P2J17adoaTnwcAjxg8R8wj1kijGcFeUFEBVbgqcIt+GGjMjxPPFoAeJV9HSBvWybhnd/p2WJc82f/kotAfn7v7CD1EvDzkrwj/zjpohvVblvxfS9hIxNQz+k5m85+lRhXdpaqMaN6cGExLW+3EEkpWYnmXbdkRkxLL3kvo7rWxkDATn6v7u1Jpn4w5GlNwPRv6bFKHxwzvcwa+XK22pXZOHkJgb9YoCk05c8SpVkPVeijXEpNkEZulctnfxdiyEUR77un/7WkyTPygAZ95iZglW+zeMYYf42Zj6/g6iKtt0tLX9ItMsCefms5ITKTdmwj13m6/07oG+ei6jXqWPDWY4QHcuR+72iRg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Convert the cmdline parameters (hugepagesz, hugepages, default_hugepagesz and hugetlb_free_vmemmap) to early parameters. Since parse_early_param might run before MMU setups on some platforms (powerpc), validation of huge page sizes as specified in command line parameters would fail. So instead, for the hstate-related values, just record the them and parse them on demand, from hugetlb_bootmem_alloc. The allocation of hugetlb bootmem pages is now done in hugetlb_bootmem_alloc, which is called explicitly at the start of mm_core_init(). core_initcall would be too late, as that happens with memblock already torn down. This change will allow earlier allocation and initialization of bootmem hugetlb pages later on. No functional change intended. Signed-off-by: Frank van der Linden --- .../admin-guide/kernel-parameters.txt | 14 +- include/linux/hugetlb.h | 6 + mm/hugetlb.c | 133 ++++++++++++++---- mm/hugetlb_vmemmap.c | 6 +- mm/mm_init.c | 3 + 5 files changed, 126 insertions(+), 36 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index fb8752b42ec8..ae21d911d1c7 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -1861,7 +1861,7 @@ hpet_mmap= [X86, HPET_MMAP] Allow userspace to mmap HPET registers. Default set by CONFIG_HPET_MMAP_DEFAULT. - hugepages= [HW] Number of HugeTLB pages to allocate at boot. + hugepages= [HW,EARLY] Number of HugeTLB pages to allocate at boot. If this follows hugepagesz (below), it specifies the number of pages of hugepagesz to be allocated. If this is the first HugeTLB parameter on the command @@ -1873,12 +1873,12 @@ :[,:] hugepagesz= - [HW] The size of the HugeTLB pages. This is used in - conjunction with hugepages (above) to allocate huge - pages of a specific size at boot. The pair - hugepagesz=X hugepages=Y can be specified once for - each supported huge page size. Huge page sizes are - architecture dependent. See also + [HW,EARLY] The size of the HugeTLB pages. This is + used in conjunction with hugepages (above) to + allocate huge pages of a specific size at boot. The + pair hugepagesz=X hugepages=Y can be specified once + for each supported huge page size. Huge page sizes + are architecture dependent. See also Documentation/admin-guide/mm/hugetlbpage.rst. Format: size[KMG] diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ec8c0ccc8f95..9cd7c9dacb88 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -174,6 +174,8 @@ struct address_space *hugetlb_folio_mapping_lock_write(struct folio *folio); extern int sysctl_hugetlb_shm_group; extern struct list_head huge_boot_pages[MAX_NUMNODES]; +void hugetlb_bootmem_alloc(void); + /* arch callbacks */ #ifndef CONFIG_HIGHPTE @@ -1250,6 +1252,10 @@ static inline bool hugetlbfs_pagecache_present( { return false; } + +static inline void hugetlb_bootmem_alloc(void) +{ +} #endif /* CONFIG_HUGETLB_PAGE */ static inline spinlock_t *huge_pte_lock(struct hstate *h, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 0592c076cd36..1a200f89e21a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include @@ -62,6 +63,24 @@ static unsigned long hugetlb_cma_size __initdata; __initdata struct list_head huge_boot_pages[MAX_NUMNODES]; +/* + * Due to ordering constraints across the init code for various + * architectures, hugetlb hstate cmdline parameters can't simply + * be early_param. early_param might call the setup function + * before valid hugetlb page sizes are determined, leading to + * incorrect rejection of valid hugepagesz= options. + * + * So, record the parameters early and consume them whenever the + * init code is ready for them, by calling hugetlb_parse_params(). + */ + +/* one (hugepagesz=,hugepages=) pair per hstate, one default_hugepagesz */ +#define HUGE_MAX_CMDLINE_ARGS (2 * HUGE_MAX_HSTATE + 1) +struct hugetlb_cmdline { + char *val; + int (*setup)(char *val); +}; + /* for command line parsing */ static struct hstate * __initdata parsed_hstate; static unsigned long __initdata default_hstate_max_huge_pages; @@ -69,6 +88,20 @@ static bool __initdata parsed_valid_hugepagesz = true; static bool __initdata parsed_default_hugepagesz; static unsigned int default_hugepages_in_node[MAX_NUMNODES] __initdata; +static char hstate_cmdline_buf[COMMAND_LINE_SIZE] __initdata; +static int hstate_cmdline_index __initdata; +static struct hugetlb_cmdline hugetlb_params[HUGE_MAX_CMDLINE_ARGS] __initdata; +static int hugetlb_param_index __initdata; +static __init int hugetlb_add_param(char *s, int (*setup)(char *val)); +static __init void hugetlb_parse_params(void); + +#define hugetlb_early_param(str, func) \ +static __init int func##args(char *s) \ +{ \ + return hugetlb_add_param(s, func); \ +} \ +early_param(str, func##args) + /* * Protects updates to hugepage_freelists, hugepage_activelist, nr_huge_pages, * free_huge_pages, and surplus_huge_pages. @@ -3484,6 +3517,8 @@ static void __init hugetlb_hstate_alloc_pages(struct hstate *h) for (i = 0; i < MAX_NUMNODES; i++) INIT_LIST_HEAD(&huge_boot_pages[i]); + h->next_nid_to_alloc = first_online_node; + h->next_nid_to_free = first_online_node; initialized = true; } @@ -4546,8 +4581,6 @@ void __init hugetlb_add_hstate(unsigned int order) for (i = 0; i < MAX_NUMNODES; ++i) INIT_LIST_HEAD(&h->hugepage_freelists[i]); INIT_LIST_HEAD(&h->hugepage_activelist); - h->next_nid_to_alloc = first_online_node; - h->next_nid_to_free = first_online_node; snprintf(h->name, HSTATE_NAME_LEN, "hugepages-%lukB", huge_page_size(h)/SZ_1K); @@ -4572,6 +4605,42 @@ static void __init hugepages_clear_pages_in_node(void) } } +static __init int hugetlb_add_param(char *s, int (*setup)(char *)) +{ + size_t len; + char *p; + + if (hugetlb_param_index >= HUGE_MAX_CMDLINE_ARGS) + return -EINVAL; + + len = strlen(s) + 1; + if (len + hstate_cmdline_index > sizeof(hstate_cmdline_buf)) + return -EINVAL; + + p = &hstate_cmdline_buf[hstate_cmdline_index]; + memcpy(p, s, len); + hstate_cmdline_index += len; + + hugetlb_params[hugetlb_param_index].val = p; + hugetlb_params[hugetlb_param_index].setup = setup; + + hugetlb_param_index++; + + return 0; +} + +static __init void hugetlb_parse_params(void) +{ + int i; + struct hugetlb_cmdline *hcp; + + for (i = 0; i < hugetlb_param_index; i++) { + hcp = &hugetlb_params[i]; + + hcp->setup(hcp->val); + } +} + /* * hugepages command line processing * hugepages normally follows a valid hugepagsz or default_hugepagsz @@ -4591,7 +4660,7 @@ static int __init hugepages_setup(char *s) if (!parsed_valid_hugepagesz) { pr_warn("HugeTLB: hugepages=%s does not follow a valid hugepagesz, ignoring\n", s); parsed_valid_hugepagesz = true; - return 1; + return -EINVAL; } /* @@ -4645,24 +4714,16 @@ static int __init hugepages_setup(char *s) } } - /* - * Global state is always initialized later in hugetlb_init. - * But we need to allocate gigantic hstates here early to still - * use the bootmem allocator. - */ - if (hugetlb_max_hstate && hstate_is_gigantic(parsed_hstate)) - hugetlb_hstate_alloc_pages(parsed_hstate); - last_mhp = mhp; - return 1; + return 0; invalid: pr_warn("HugeTLB: Invalid hugepages parameter %s\n", p); hugepages_clear_pages_in_node(); - return 1; + return -EINVAL; } -__setup("hugepages=", hugepages_setup); +hugetlb_early_param("hugepages", hugepages_setup); /* * hugepagesz command line processing @@ -4681,7 +4742,7 @@ static int __init hugepagesz_setup(char *s) if (!arch_hugetlb_valid_size(size)) { pr_err("HugeTLB: unsupported hugepagesz=%s\n", s); - return 1; + return -EINVAL; } h = size_to_hstate(size); @@ -4696,7 +4757,7 @@ static int __init hugepagesz_setup(char *s) if (!parsed_default_hugepagesz || h != &default_hstate || default_hstate.max_huge_pages) { pr_warn("HugeTLB: hugepagesz=%s specified twice, ignoring\n", s); - return 1; + return -EINVAL; } /* @@ -4706,14 +4767,14 @@ static int __init hugepagesz_setup(char *s) */ parsed_hstate = h; parsed_valid_hugepagesz = true; - return 1; + return 0; } hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT); parsed_valid_hugepagesz = true; - return 1; + return 0; } -__setup("hugepagesz=", hugepagesz_setup); +hugetlb_early_param("hugepagesz", hugepagesz_setup); /* * default_hugepagesz command line input @@ -4727,14 +4788,14 @@ static int __init default_hugepagesz_setup(char *s) parsed_valid_hugepagesz = false; if (parsed_default_hugepagesz) { pr_err("HugeTLB: default_hugepagesz previously specified, ignoring %s\n", s); - return 1; + return -EINVAL; } size = (unsigned long)memparse(s, NULL); if (!arch_hugetlb_valid_size(size)) { pr_err("HugeTLB: unsupported default_hugepagesz=%s\n", s); - return 1; + return -EINVAL; } hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT); @@ -4751,17 +4812,33 @@ static int __init default_hugepagesz_setup(char *s) */ if (default_hstate_max_huge_pages) { default_hstate.max_huge_pages = default_hstate_max_huge_pages; - for_each_online_node(i) - default_hstate.max_huge_pages_node[i] = - default_hugepages_in_node[i]; - if (hstate_is_gigantic(&default_hstate)) - hugetlb_hstate_alloc_pages(&default_hstate); + /* + * Since this is an early parameter, we can't check + * NUMA node state yet, so loop through MAX_NUMNODES. + */ + for (i = 0; i < MAX_NUMNODES; i++) { + if (default_hugepages_in_node[i] != 0) + default_hstate.max_huge_pages_node[i] = + default_hugepages_in_node[i]; + } default_hstate_max_huge_pages = 0; } - return 1; + return 0; +} +hugetlb_early_param("default_hugepagesz", default_hugepagesz_setup); + +void __init hugetlb_bootmem_alloc(void) +{ + struct hstate *h; + + hugetlb_parse_params(); + + for_each_hstate(h) { + if (hstate_is_gigantic(h)) + hugetlb_hstate_alloc_pages(h); + } } -__setup("default_hugepagesz=", default_hugepagesz_setup); static unsigned int allowed_mems_nr(struct hstate *h) { diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 7735972add01..5b484758f813 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -444,7 +444,11 @@ DEFINE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); EXPORT_SYMBOL(hugetlb_optimize_vmemmap_key); static bool vmemmap_optimize_enabled = IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON); -core_param(hugetlb_free_vmemmap, vmemmap_optimize_enabled, bool, 0); +static int __init hugetlb_vmemmap_optimize_param(char *buf) +{ + return kstrtobool(buf, &vmemmap_optimize_enabled); +} +early_param("hugetlb_free_vmemmap", hugetlb_vmemmap_optimize_param); static int __hugetlb_vmemmap_restore_folio(const struct hstate *h, struct folio *folio, unsigned long flags) diff --git a/mm/mm_init.c b/mm/mm_init.c index 2630cc30147e..d2dee53e95dd 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -2641,6 +2642,8 @@ static void __init mem_init_print_info(void) */ void __init mm_core_init(void) { + hugetlb_bootmem_alloc(); + /* Initializations relying on SMP setup */ BUILD_BUG_ON(MAX_ZONELISTS > 2); build_all_zonelists(NULL);