From patchwork Mon Oct 14 10:58:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96E25D1A45A for ; Mon, 14 Oct 2024 11:00:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C0316B009E; Mon, 14 Oct 2024 07:00:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24ABD6B009F; Mon, 14 Oct 2024 07:00:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7E8E6B00A0; Mon, 14 Oct 2024 07:00:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C28876B009E for ; Mon, 14 Oct 2024 07:00:16 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 63680AAB2C for ; Mon, 14 Oct 2024 11:00:01 +0000 (UTC) X-FDA: 82671913380.01.E22C62D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP id BDC3640017 for ; Mon, 14 Oct 2024 11:00:08 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903584; a=rsa-sha256; cv=none; b=o/EuZAdiuwrKjfBOV2PdUP7qBgnmv6NJzqWBSmROnhBcGgG9qIzXyO+BLaFKD8w8ods64U B0qu0hQ57SoJvvin2f0ktjJhnb381Ru1yyNbMugs02G48Jgi68cvmsAc97U/ygvbhy5jV6 ocCizJrw4n/ON1pZOWS4IQgvSKL8oe0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903584; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mddGb9Xpz4LiyvZj0cyiEir/DtiqGjM7+BRPU5mZ/qY=; b=eQO+MDv/pih/zVauC5ASlGlB5sXPpYtc2Rc7gr1WTZkaltjVnihOkB++WRvU/ph2iO0PNJ 9dRuG63NaMi2RDc/J17uqO75AXM45sUe1DydPdzXPEBQ0rHotaamIt6Zyw2Ek7ZO4Fk3K0 xlspIMiZrnDqFrJPszzw5M8IUJrCO30= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BA5741691; Mon, 14 Oct 2024 04:00:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB0D53F51B; Mon, 14 Oct 2024 04:00:11 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 15/57] stackdepot: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:22 +0100 Message-ID: <20241014105912.3207374-15-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hetky5cwf3uqambp4k8jztd86uifmzjs X-Rspamd-Queue-Id: BDC3640017 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903608-267546 X-HE-Meta: U2FsdGVkX1/KBIYTHImmkbJO6vMhYhooP40Ol1JF/RxarUZP0ztc8VwQZekr4oR8ewRSUtjNTJJXhvrjB2ZFNyOQIE8WI63BFQvU/CPIQQAJLVwnYuqM/zvUfe/zySOyQzeHYfrKTmzQ6UeCUoiCCx0QF5r7RACIgnNbWhGsoJ8ztcJ8KDtSKuXzVDVL2AQar3zV9YUqH2xjIhPO1PWxMaN7x4Ij2kEPWlUFUo049OPABNSJ1mIaMJoc0SahP28qyv1k/c6q3peEwpXuDNFy1kAFQJnjvuLoRiG5f10l6ZBvyzaFJSbS312StbWK4DmZSpSMaKr8knUUZHYHefeuEDQetiDbVVIaNtis2s6rabZuP26mFyolYDUv7LTnFgc46qqwsPqNdfTFQxZoWPXVvEHjZuzaV9alI17JMhEyPYd0AwO1sMjLyNAJ4SKIu3AKdYsWz3MjB1nYd+HAqqWjZX5T+oZZUDHpKRia1AuBO4AmzDYF3R/8k6G/DBHSOf4qWsOSVK8i06wqWhkW76kUgT8N13CJQfhpJqEzb27hxbZTo5+FYPN6hltWWR16izwHfFgsa3j1ria4QmkDUF1tj/7ZymaIHUFxKZlVsu5S4crYRnysNa/yc03kv7elsw1nMoReCDM1SSde0o6GSj+Ibe2nF+WwcjJ6BAk4xCelm3ep6jfISS4NUp8tlztvV1OGKNs/hK6GB5hcWi8skErBobs/yisjvIXtLrBxzABKdJefa2jDYO0Ye1QoHdRPYjfR1IO7GKPzOqcki0Onh2uRPiJabVooPv+yqdblGbqMZd0L7qB5PwlbjcbM5lZYVQlHZuoB0dzexjqrmhcB2JQEtmqswf0CU2tGYR2lVlxRj+h4Ei3fs68m35ikwyixj2pcTzoJzagimcbNyCSMwhJVwXi0G86JEWHmNGpwbiwpD65SduI2j+cTTAqc57xcBsGKwKEGLGzYRca3AWcWxa4 wT2FpTzY C+eavLbzJgPRw0voPLw4a7g5lOhxYMQpgm4gVkluCmN3kMU8v/A84/Sdqnf80tvNx1In114pElTfd/RoZcmrdObFMjoM798IZIIUKJazw9AlnW4ioQkL2sJNUhk5YYkUEiCnU1vBhYmOVw/RTKYryqw//ZZ46Sr/rxVQiBP7dTW0YDaruQQbYnmOWfFEvFHXbJ3WA1SGoG8xyFk+m5RcN93YwxJra9CInDI7/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. "union handle_parts" previously calculated the number of bits required for its pool index and offset members based on PAGE_SHIFT. This is problematic for boot-time page size builds because the actual page size isn't known until boot-time. We could use PAGE_SHIFT_MAX in calculating the worst case offset bits, but bits would be wasted that could be used for pool index when PAGE_SIZE is set smaller than MAX, the end result being that stack depot can address less memory than it should. To avoid needing to dynamically define the offset and index bit widths, let's instead fix the pool size and derive the order at runtime based on the PAGE_SIZE. This means that the fields' widths can remain static, with the down side being slightly increased risk of failing to allocate the large folio. This only affects boot-time page size builds. compile-time page size builds will still always allocate order-2 folios. Additionally, wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts Acked-by: Vlastimil Babka --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/stackdepot.h | 6 +++--- lib/stackdepot.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e9ec32fb97d4a..ac877a4e90406 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -32,10 +32,10 @@ typedef u32 depot_stack_handle_t; #define DEPOT_HANDLE_BITS (sizeof(depot_stack_handle_t) * 8) -#define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages */ -#define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT + DEPOT_POOL_ORDER)) +#define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages of PAGE_SIZE_MAX */ +#define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT_MAX + DEPOT_POOL_ORDER)) #define DEPOT_STACK_ALIGN 4 -#define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT - DEPOT_STACK_ALIGN) +#define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT_MAX - DEPOT_STACK_ALIGN) #define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_OFFSET_BITS - \ STACK_DEPOT_EXTRA_BITS) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 5ed34cc963fc3..974351f0e9e3c 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -68,7 +68,7 @@ static void *new_pool; /* Number of pools in stack_pools. */ static int pools_num; /* Offset to the unused space in the currently used pool. */ -static size_t pool_offset = DEPOT_POOL_SIZE; +static DEFINE_GLOBAL_PAGE_SIZE_VAR(size_t, pool_offset, DEPOT_POOL_SIZE); /* Freelist of stack records within stack_pools. */ static LIST_HEAD(free_stacks); /* The lock must be held when performing pool or freelist modifications. */ @@ -625,7 +625,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, */ if (unlikely(can_alloc && !READ_ONCE(new_pool))) { page = alloc_pages(gfp_nested_mask(alloc_flags), - DEPOT_POOL_ORDER); + get_order(DEPOT_POOL_SIZE)); if (page) prealloc = page_address(page); } @@ -663,7 +663,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, exit: if (prealloc) { /* Stack depot didn't use this memory, free it. */ - free_pages((unsigned long)prealloc, DEPOT_POOL_ORDER); + free_pages((unsigned long)prealloc, get_order(DEPOT_POOL_SIZE)); } if (found) handle = found->handle.handle;