From patchwork Wed Mar 30 15:37:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12796032 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4296DC433F5 for ; Wed, 30 Mar 2022 15:38:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF5F86B0073; Wed, 30 Mar 2022 11:38:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA64B8D0002; Wed, 30 Mar 2022 11:38:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9467F8D0001; Wed, 30 Mar 2022 11:38:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 830396B0073 for ; Wed, 30 Mar 2022 11:38:28 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 52EF421C43 for ; Wed, 30 Mar 2022 15:38:28 +0000 (UTC) X-FDA: 79301459496.13.51B6181 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf11.hostedemail.com (Postfix) with ESMTP id BCD4140014 for ; Wed, 30 Mar 2022 15:38:27 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id k14so17924518pga.0 for ; Wed, 30 Mar 2022 08:38:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=h81o0zrOo3MmiqLhXNQs++uBdbtx9emPgi7EUEIWoeg=; b=JXogY18B9Li/QyeeQ4h7enOzye+7Wr8M0oHfdOsyIxsicMR+wphDN2S28Ok7Vv8mdR eL333WRR6cAOAplCN9S83Sr/2pct5HkiQuApm024TGDeF3zdy8gDMgcbfc1C4EWTGo+T CTLxjMbvCmzR069ibJLzijcoTpRYGVpzxPsRdVmJoP5NfbwjQNyJuFizCykL4QX3yARm 0RFDiHzSa8dfgySYTeGQ56ULbR4LK+K0YRzlfqicbWDG0uk832apkSH7eRbg3+x2l9mB I9TRDPNd6KyLjydqjWKxWy+paxKkoOpVdsmEEcnwesmyFej03cLc66wlWa+RVW1qDRdE Fzkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h81o0zrOo3MmiqLhXNQs++uBdbtx9emPgi7EUEIWoeg=; b=XaC2iGckSzCs0PkZqmpHh5Hf9Shc1OlCKRiNirvdUdUtoaYEoALehtJMDvBAOBHL/y dbHXAaiVI9mX+SjKQKDVTWT/mu3bdyan5iw58+j4z+9FJqPUSQymOjoI4vhHpNXbwVD2 I6DNucB6Y3ZDX4cOshB2NhslRNZgHTMme22+J8DtOtHLFemCwKUpE2CAFzQt24s+CjcK KTzZ6UzRG189tPbydNEJ9KmyVE7A5kgSugZl/3DGPm1PGLZIysrJw4IPjuE01k3SnaOH aXuWGXigVVvTub9XRfUy7vJ5nCRixGdkadO5z3U3IhOmLVflw8zC9GfEK6AH7MH236FQ YVlQ== X-Gm-Message-State: AOAM531VxRHdjWsX1CSvv1hGBpPbDBrLB+VFqNzXkLUo2frAqPmekGwR 2TUghvQUSk2VMVa+NgFBPMljUg== X-Google-Smtp-Source: ABdhPJw76LGvWldqRycLoo/x5V2Cwand9sJPfGmDuBBYpLXujydBQNXxINbnTmeUG6iu8MQAG80cQg== X-Received: by 2002:a63:520c:0:b0:382:2953:a338 with SMTP id g12-20020a63520c000000b003822953a338mr6618861pgb.610.1648654706651; Wed, 30 Mar 2022 08:38:26 -0700 (PDT) Received: from FVFYT0MHHV2J.bytedance.net ([139.177.225.239]) by smtp.gmail.com with ESMTPSA id y8-20020a17090aa40800b001c6ccb2c395sm6686039pjp.9.2022.03.30.08.38.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Mar 2022 08:38:26 -0700 (PDT) From: Muchun Song To: corbet@lwn.net, mike.kravetz@oracle.com, akpm@linux-foundation.org, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com, osalvador@suse.de, david@redhat.com, masahiroy@kernel.org Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com, Muchun Song Subject: [PATCH v6 1/4] mm: hugetlb_vmemmap: introduce STRUCT_PAGE_SIZE_IS_POWER_OF_2 Date: Wed, 30 Mar 2022 23:37:42 +0800 Message-Id: <20220330153745.20465-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220330153745.20465-1-songmuchun@bytedance.com> References: <20220330153745.20465-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BCD4140014 X-Stat-Signature: ciwkfp1nxm1gwpe1kwimjpf1ub4cnxeg Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=JXogY18B; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf11.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspam-User: X-HE-Tag: 1648654707-774523 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If the size of "struct page" is not the power of two and this feature is enabled, then the vmemmap pages of HugeTLB will be corrupted after remapping (panic is about to happen in theory). But this only exists when !CONFIG_MEMCG && !CONFIG_SLUB on x86_64. However, it is not a conventional configuration nowadays. So it is not a real word issue, just the result of a code review. But we have to prevent anyone from configuring that combined configuration. In order to avoid many checks like "is_power_of_2 (sizeof(struct page))" through mm/hugetlb_vmemmap.c. Introduce STRUCT_PAGE_SIZE_IS_POWER_OF_2 to detect if the size of struct page is power of 2 and make this feature depends on this new macro. Then we could prevent anyone do any unexpected configuration. Signed-off-by: Muchun Song Suggested-by: Luis Chamberlain Reported-by: kernel test robot --- Kbuild | 15 ++++++++++++++- include/linux/mm_types.h | 2 ++ include/linux/page-flags.h | 3 ++- mm/hugetlb_vmemmap.c | 8 ++------ mm/hugetlb_vmemmap.h | 4 ++-- mm/struct_page_size.c | 20 ++++++++++++++++++++ 6 files changed, 42 insertions(+), 10 deletions(-) create mode 100644 mm/struct_page_size.c diff --git a/Kbuild b/Kbuild index fa441b98c9f6..7f90ba21dd51 100644 --- a/Kbuild +++ b/Kbuild @@ -24,6 +24,19 @@ $(timeconst-file): kernel/time/timeconst.bc FORCE $(call filechk,gentimeconst) ##### +# Generate struct_page_size.h. + +struct_page_size-file := include/generated/struct_page_size.h + +always-y += $(struct_page_size-file) +targets += mm/struct_page_size.s + +mm/struct_page_size.s: $(timeconst-file) $(bounds-file) + +$(struct_page_size-file): mm/struct_page_size.s FORCE + $(call filechk,offsets,__LINUX_STRUCT_PAGE_SIZE_H__) + +##### # Generate asm-offsets.h offsets-file := include/generated/asm-offsets.h @@ -31,7 +44,7 @@ offsets-file := include/generated/asm-offsets.h always-y += $(offsets-file) targets += arch/$(SRCARCH)/kernel/asm-offsets.s -arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file) +arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file) $(struct_page_size-file) $(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE $(call filechk,offsets,__ASM_OFFSETS_H__) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 8834e38c06a4..5fbff44a4310 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -223,6 +223,7 @@ struct page { #endif } _struct_page_alignment; +#ifndef __GENERATING_STRUCT_PAGE_SIZE_IS_POWER_OF_2_H /** * struct folio - Represents a contiguous set of bytes. * @flags: Identical to the page flags. @@ -844,5 +845,6 @@ enum fault_flag { FAULT_FLAG_INSTRUCTION = 1 << 8, FAULT_FLAG_INTERRUPTIBLE = 1 << 9, }; +#endif /* !__GENERATING_STRUCT_PAGE_SIZE_IS_POWER_OF_2_H */ #endif /* _LINUX_MM_TYPES_H */ diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index fc4f294cc8d7..15fcdff0e7ee 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -12,6 +12,7 @@ #ifndef __GENERATING_BOUNDS_H #include #include +#include #endif /* !__GENERATING_BOUNDS_H */ /* @@ -190,7 +191,7 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H -#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +#if defined(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP) && defined(STRUCT_PAGE_SIZE_IS_POWER_OF_2) DECLARE_STATIC_KEY_MAYBE(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON, hugetlb_free_vmemmap_enabled_key); diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 791626983c2e..951cf83010c7 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -178,6 +178,7 @@ #include "hugetlb_vmemmap.h" +#ifdef STRUCT_PAGE_SIZE_IS_POWER_OF_2 /* * There are a lot of struct page structures associated with each HugeTLB page. * For tail pages, the value of compound_head is the same. So we can reuse first @@ -194,12 +195,6 @@ EXPORT_SYMBOL(hugetlb_free_vmemmap_enabled_key); static int __init early_hugetlb_free_vmemmap_param(char *buf) { - /* We cannot optimize if a "struct page" crosses page boundaries. */ - if (!is_power_of_2(sizeof(struct page))) { - pr_warn("cannot free vmemmap pages because \"struct page\" crosses page boundaries\n"); - return 0; - } - if (!buf) return -EINVAL; @@ -302,3 +297,4 @@ void __init hugetlb_vmemmap_init(struct hstate *h) pr_info("can free %d vmemmap pages for %s\n", h->nr_free_vmemmap_pages, h->name); } +#endif /* STRUCT_PAGE_SIZE_IS_POWER_OF_2 */ diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h index cb2bef8f9e73..b137fd8b6ba4 100644 --- a/mm/hugetlb_vmemmap.h +++ b/mm/hugetlb_vmemmap.h @@ -10,7 +10,7 @@ #define _LINUX_HUGETLB_VMEMMAP_H #include -#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +#if defined(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP) && defined(STRUCT_PAGE_SIZE_IS_POWER_OF_2) int alloc_huge_page_vmemmap(struct hstate *h, struct page *head); void free_huge_page_vmemmap(struct hstate *h, struct page *head); void hugetlb_vmemmap_init(struct hstate *h); @@ -41,5 +41,5 @@ static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) { return 0; } -#endif /* CONFIG_HUGETLB_PAGE_FREE_VMEMMAP */ +#endif /* CONFIG_HUGETLB_PAGE_FREE_VMEMMAP && STRUCT_PAGE_SIZE_IS_POWER_OF_2 */ #endif /* _LINUX_HUGETLB_VMEMMAP_H */ diff --git a/mm/struct_page_size.c b/mm/struct_page_size.c new file mode 100644 index 000000000000..6fc29c1227a0 --- /dev/null +++ b/mm/struct_page_size.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Generate definitions needed by the preprocessor. + * This code generates raw asm output which is post-processed + * to extract and format the required data. + */ + +#define __GENERATING_STRUCT_PAGE_SIZE_IS_POWER_OF_2_H +/* Include headers that define the enum constants of interest */ +#include +#include +#include + +int main(void) +{ + if (is_power_of_2(sizeof(struct page))) + DEFINE(STRUCT_PAGE_SIZE_IS_POWER_OF_2, is_power_of_2(sizeof(struct page))); + + return 0; +}