From patchwork Mon Sep 25 21:28:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Kinsburskii X-Patchwork-Id: 13398424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B2ABCE79A1 for ; Mon, 25 Sep 2023 21:28:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E02568D0040; Mon, 25 Sep 2023 17:28:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8AD36B0171; Mon, 25 Sep 2023 17:28:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDD9C8D0040; Mon, 25 Sep 2023 17:28:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A4EF36B016F for ; Mon, 25 Sep 2023 17:28:13 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7CAB0160654 for ; Mon, 25 Sep 2023 21:28:13 +0000 (UTC) X-FDA: 81276408066.19.EF5A37A Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf12.hostedemail.com (Postfix) with ESMTP id B987D40004 for ; Mon, 25 Sep 2023 21:28:11 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=e8sIvgnj; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf12.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695677291; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+rq2deN+a4hM/V5imypqZgNWtOmdjsP2ToFxAhOZYp4=; b=kUiYDio79oNh0DK0KmI3wDC93c68gp77kpZ+mRG9TkDPNhiEJkpVCYf1qRrKGulPetPuF9 PoLpprv0ZxkC7LEr3AdIWzkYEzsug6IgY1W4NQ7FTWlvPalqhqHjpBkro+2DOJeKE+9jC6 qf3pp2TXKGAp+rT0DIUwBJgxADWEoFU= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=e8sIvgnj; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf12.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695677291; a=rsa-sha256; cv=none; b=S0uq+mMz8FegoWj2EvX307ryQNk5LFnsVpgYNJFZDtODk4a1Z6FSuIszOTnJytCJdUVIdh OjvchBgF2/4L3OXPtmM4kOEbQQU6PhGwfWtPYcqXAdsrOn/SIbW6p93/dmccV9in7V0aE5 L7dOlTWaa+r0EhXG8c2+rAYkNBxJD5I= Received: from skinsburskii. (c-67-170-100-148.hsd1.wa.comcast.net [67.170.100.148]) by linux.microsoft.com (Postfix) with ESMTPSA id 86DD1212C81C; Mon, 25 Sep 2023 14:28:09 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 86DD1212C81C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1695677291; bh=+rq2deN+a4hM/V5imypqZgNWtOmdjsP2ToFxAhOZYp4=; h=Subject:From:To:Date:In-Reply-To:References:From; b=e8sIvgnjvKkRGvg4avEhFTndLU4vLLOa15fS9Us3dnO+2i0QYibFYvZ8esTSOV86L TlWVjndBFC+3YMx5igZ88kv2Y4AoIVdvfbaxSI6ifkALnqIfGeLfZmAifRBQTjd3s9 w2+h3Ly5Y5vtIQsq2MQ/yQhA23hlVdsrIyi3HnBY= Subject: [RFC PATCH v2 4/7] pmpool: Introduce persistent memory pool From: Stanislav Kinsburskii To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, ebiederm@xmission.com, akpm@linux-foundation.org, stanislav.kinsburskii@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-mm@kvack.org, kys@microsoft.com, jgowans@amazon.com, wei.liu@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, graf@amazon.de, pbonzini@redhat.com Date: Mon, 25 Sep 2023 14:28:09 -0700 Message-ID: <169567728905.19708.12668538787634109172.stgit@skinsburskii.> In-Reply-To: <169567722094.19708.3583735425859054859.stgit@skinsburskii.> References: <169567722094.19708.3583735425859054859.stgit@skinsburskii.> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B987D40004 X-Stat-Signature: xxxmmca8gsypx5smnuxw1b6s8ib1mbwf X-HE-Tag: 1695677291-27668 X-HE-Meta: U2FsdGVkX19p8rVoOdxmtS9fQBa78j05dIo8gjjfL5Ypt2u/bUUaK8KcYWJBAmEUniDUpDglqRGeRRMz9Senbbji+E3G+55dMMw4fJQ6cD312BA7KrzRCxIre1uTDjVFy+Z8Ll2q0Y+4AbsQvTG/NaJHn+gnkkMdIpttv0OlGlDzxWWCpfmLlo7Bpcu7iA/LzVop5BHNt+75b4TAH80YL0Z+O0O+coKQz/WaBiQ33yo47BvY7CH6U8IZvbUwoceEJOOeJvYrbC3Ph/6Mq++FXTjGmELrwj8O2y2aBOhKMRRS/kgMTm2wpaOKWtAkQWssuXZ29E+P+xHWoV284wxqKImLSLxhTb5IBUXpEU2Hf6e6Zz9Z07PEKwIDC1rc0uFEqlScGiok8eoAPST2i28NmNRnHP4cCACySbw6Ldk4yDgmIUU+3MLWIoossPUDcVq/Uk95ha4gwKtM6xi44rcaSYN1HzsQucZumucvMXNmhjmP5fa9CQaAlNhh2YbnfKgxT/zCNPfuzwB55SpRDdXgqzxknNZwk2HiP3HnQQBkArsyO60SpZQQRRxGYm8Y9BuljVbGSj+QoKgaNS54NiqYf0j1KBcBXgaRbxrFDFZrMo9GqfOmiTS+6EHZD51+Wgmm6IGpHOyxt4+oEvOCFX2giDkSn8DZ9hd58kzHbOxcbc8FYJnafUZk8Dx7BjuYzNqX2GTvRnuqqEKPVXtoHsBzKiD0ustE3sc57hP6a4xt3lR3acnQMaPplnKqJNztGJ3OiiB2zFqyMZucR3oRmLzcCGuVzAAJxo/TaOyDZDWaMMEJo5OEp23B+mM2WmA0vjogaj5GY1NZG8a7NivWZoSOEzKgnqAZl/l115zndJkHrGNK1hn4cWfxVBE3KNMh1KWLSTQ+v29rcy7vYKXM+4Xvt0CA73P0FY95pGc8EONlgM1hXCFIE18IttRfLypONxk8DDISbzA2LzlqLRkY2I/ 3b78ODwF WtXNGLEoQy6Aaff4//JwTHl8C/mANHAt6TQCNVYIE7O69SsPeZUXhI3whhiMsTL5oavmZm98teG6SfHUeeHZW9+0oQJQpImzq3hCkvpcljwXRUpif56RXImksPk0RBvCouvk7/v+iQYnU6GZZI5jongGWqzcQpST6S2xP+9RwrEHS7Exc4cUzm6pmfR6/RhOjhnxGwD0KmInG21zdPck0toPxTImcTrfONzFOB+I/mP6KtH0PJQAZAIBYVC3Z34Il2H8KGm+sZORC6cuToS4tdC62T1GRmpe/yJ97MRluxoFQAYk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Stanislav Kinsburskii This patch introduces a memory allocator specifically tailored for persistent memory within the kernel. The allocator maintains kernel-specific states like DMA passthrough device states, IOMMU state, and more across kexec. The current implementation provides a foundation for custom solutions that may be developed in the future. Although the design is kept concise and straightforward to encourage discussion and feedback, it remains fully functional. The persistent memory pool builds upon the continuous memory allocator (CMA) and ensures CMA state persistency across kexec by incorporating the CMA bitmap into the memory region. Potential applications include: 1. Enabling various in-kernel entities to allocate persistent pages from a unified memory pool, obviating the need for reserving multiple regions. 2. For in-kernel components that need the allocation address to be retained on kernel kexec, this address can be exposed to user space and subsequently passed through the command line. 3. Distinct subsystems or drivers can set aside their region, allocating a segment for their persistent memory pool, suitable for uses such as file systems, key-value stores, and other applications. Signed-off-by: Stanislav Kinsburskii --- include/linux/pmpool.h | 22 +++++++++++ mm/Kconfig | 8 ++++ mm/Makefile | 1 mm/pmpool.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+) create mode 100644 include/linux/pmpool.h create mode 100644 mm/pmpool.c diff --git a/include/linux/pmpool.h b/include/linux/pmpool.h new file mode 100644 index 000000000000..b41f16fa9660 --- /dev/null +++ b/include/linux/pmpool.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _PMPOOL_H +#define _PMPOOL_H + +struct page; + +#if defined(CONFIG_PMPOOL) +struct page *pmpool_alloc(unsigned long count); +bool pmpool_release(struct page *pages, unsigned long count); +#else +static inline struct page *pmpool_alloc(unsigned long count) +{ + return NULL; +} +static inline bool pmpool_release(struct page *pages, unsigned long count) +{ + return false; +} +#endif + +#endif /* _PMPOOL_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 09130434e30d..e7c10094fb10 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -922,6 +922,14 @@ config CMA_AREAS If unsure, leave the default value "7" in UMA and "19" in NUMA. +config PMPOOL + bool "Persistent memory pool support" + select CMA + help + This option adds support for CMA-based persistent memory pool + feature, which provides pages allocation and freeing from a set of + persistent memory ranges, deposited to the memory pool. + config MEM_SOFT_DIRTY bool "Track memory changes" depends on CHECKPOINT_RESTORE && HAVE_ARCH_SOFT_DIRTY && PROC_FS diff --git a/mm/Makefile b/mm/Makefile index 678530a07326..8d3579e58c2c 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -139,3 +139,4 @@ obj-$(CONFIG_IO_MAPPING) += io-mapping.o obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o +obj-$(CONFIG_PMPOOL) += pmpool.o diff --git a/mm/pmpool.c b/mm/pmpool.c new file mode 100644 index 000000000000..12a8cac75558 --- /dev/null +++ b/mm/pmpool.c @@ -0,0 +1,100 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define pr_fmt(fmt) "pmpool: " fmt + +#include +#include +#include +#include +#include +#include +#include + +#include "cma.h" + +struct pmpool { + struct cma *cma; +}; + +static struct pmpool *default_pmpool; + +bool pmpool_release(struct page *pages, unsigned long count) +{ + if (!default_pmpool) + return false; + + return cma_release(default_pmpool->cma, pages, count); +} + +struct page *pmpool_alloc(unsigned long count) +{ + if (!default_pmpool) + return NULL; + + return cma_alloc(default_pmpool->cma, count, 0, true); +} + +static void pmpool_fixup_cma(struct cma *cma) +{ + unsigned long bitmap_size; + + bitmap_free(cma->bitmap); + cma->bitmap = phys_to_virt(PFN_PHYS(cma->base_pfn)); + + bitmap_size = BITS_TO_LONGS(cma_bitmap_maxno(cma)); + memset(cma->bitmap, 0, bitmap_size); + bitmap_set(cma->bitmap, 0, PAGE_ALIGN(bitmap_size) >> PAGE_SHIFT); + + pr_info("CMA bitmap moved to %#llx\n", virt_to_phys(cma->bitmap)); +} + +static int __init default_pmpool_fixup_cma(void) +{ + if (!default_pmpool) + return 0; + + pmpool_fixup_cma(default_pmpool->cma); + return 0; +} +postcore_initcall(default_pmpool_fixup_cma); + +static int __init parse_pmpool_opt(char *str) +{ + static struct pmpool pmpool; + phys_addr_t base, size; + int err; + + /* Format is pmpool=, */ + base = memparse(str, &str); + size = memparse(str + 1, NULL); + + err = memblock_is_region_reserved(base, size); + if (err) { + pr_err("memory block overlaps with another one: %d\n", err); + return 0; + } + + err = memblock_reserve(base, size); + if (err) { + pr_err("failed to reerve memory block: %d\n", err); + return 0; + } + + err = cma_init_reserved_mem(base, size, 0, "pmpool", &pmpool.cma); + if (err) { + pr_err("failed to initialize CMA: %d\n", err); + goto free_memblock; + } + + pr_info("default memory pool is created: %#llx-%#llx\n", + base, base + size); + + default_pmpool = &pmpool; + + return 0; + +free_memblock: + memblock_phys_free(base, size); + return 0; +} +early_param("pmpool", parse_pmpool_opt);