From patchwork Wed Oct 4 22:23:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Kinsburskii X-Patchwork-Id: 13409628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 917EBE936EC for ; Wed, 4 Oct 2023 22:23:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E38A6B024C; Wed, 4 Oct 2023 18:23:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 094596B024D; Wed, 4 Oct 2023 18:23:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4F136B0254; Wed, 4 Oct 2023 18:23:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D51356B024C for ; Wed, 4 Oct 2023 18:23:25 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A465F12012D for ; Wed, 4 Oct 2023 22:23:25 +0000 (UTC) X-FDA: 81309206370.22.1D2DB26 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf04.hostedemail.com (Postfix) with ESMTP id DBEEC40003 for ; Wed, 4 Oct 2023 22:23:23 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=DlMtgdVr; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf04.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696458204; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/kL+2ixzH9Y0/6AQP5r2LCMUwHJ2IksXHQmOSZeRk70=; b=Y0AoutMP9e/TurpYurcCRpUbx/HLJal4KwcVBeA/qyoKaE+R48YjHCD74HRFWDPvJI2dQl pMmaZIY7dBKH4nIoEBKWIwqLD9RvHL63uvR/PSUppAXACQkLrJ+7kOmTxB9jQjdssVgtST yxsqboanghK7GIPj8iYT7zuFOL2BRpc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=DlMtgdVr; dmarc=pass (policy=none) header.from=linux.microsoft.com; spf=pass (imf04.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696458204; a=rsa-sha256; cv=none; b=jT+NIYE1sjZF/AUBZJBerpC5ayWiTvp1M//6j+sVe4ugu1AHP7NOHpbIPgEZrcIjl8obfC TvpgKimViJZL3YF5vVRXzQPt8P0+LA7rTyClhGJIGMrPgim4fyUd7/PpWESmQwlg4xFbJx yvgcsglVsbjxToAHF9Ns1nx99B8JFRY= Received: from skinsburskii. (c-67-170-100-148.hsd1.wa.comcast.net [67.170.100.148]) by linux.microsoft.com (Postfix) with ESMTPSA id 7808420B74C3; Wed, 4 Oct 2023 15:23:22 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 7808420B74C3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1696458202; bh=/kL+2ixzH9Y0/6AQP5r2LCMUwHJ2IksXHQmOSZeRk70=; h=Subject:From:To:Date:In-Reply-To:References:From; b=DlMtgdVrA20cf0WTlXXW5RXqRzynEKABAzw/J5WMThiJUmO+Lc0rmP1Rc1OruUeLB R7bcSK9jKhK4ic0bOrJKyjziPapZ1jp5S6vb0OXKD9z7fYUH0i54ceUbwrJ/9GWA2H 2JA3TqJMbqo7iWRfG6KPnIaG2vLSTnd02Tf9D/w0= Subject: [RFC PATCH v3 2/3] pmpool: Introduce persistent memory pool From: Stanislav Kinsburskii To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, ebiederm@xmission.com, akpm@linux-foundation.org, stanislav.kinsburskii@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-mm@kvack.org, kys@microsoft.com, jgowans@amazon.com, wei.liu@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, graf@amazon.de, pbonzini@redhat.com, bhe@redhat.com, dave.hansen@intel.com, kirill.shutemov@intel.com Date: Wed, 04 Oct 2023 15:23:21 -0700 Message-ID: <169645820185.11424.12461972351107674989.stgit@skinsburskii.> In-Reply-To: <169645773092.11424.7258549771090599226.stgit@skinsburskii.> References: <169645773092.11424.7258549771090599226.stgit@skinsburskii.> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Rspamd-Queue-Id: DBEEC40003 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: np4ohzgk341hc6e8fatuxbq97pofnrbi X-HE-Tag: 1696458203-13074 X-HE-Meta: U2FsdGVkX1/ELNke1182SVgMsn+9EtoBn02fiLetjhpHqAy4vgYMFAqB5FoiMfMt+l/4krItzeUuX39HCgOOXfcpibVI5MXAfeO9GsbAenhZp0Siuqzu0OiVCSDvrMNlJ4QIObE5LjATQhI7KYZVPKKPPl2ngbmLP6QqNOLS6gV+gedfwNeQjvcP2oKhi7op/DBlH6b7zydkMxUvVdn5Ueo9yNkJmMpQ/kleOIg3pa40XCrNcw9hp5Wvc/eG6ndtClE8ObupzZi9EBi0zxb74RoeiH++u2cNl2DMWIKOpjkdG3h9lTUpf0VlKbt4vMPhfWNeAs8L2Ea/vg1nZmsQvVCugdYkte3a3xqGXCuyMnDg8MImGH4owNh5yh89201lL1YS/eHbgwEC7M0q5/hbrOIWw+Ckoeem04lFQBZ1F8Yo7s2WjdCEi0SF7SbIDfc+mDT4S/OkVPGiGBQMAC6A+8pAQK+Xmc0Cb25jzHm4cZEaQhOSEcJKrMRMKsYefqAoM2lBSAhDyNd4vKRr0pv5aI2myLSFvqdaf5IuoOIgqSN6IQOyV1EqZWOpkUWEyrYtbx9yoX/8hI4aOLhSAAGAcU4ps2s6I2rNVt6wKiqDSVYlVEtrIqHGidaqoANJA11/2U2febcD6UBJ1R2YsEHcc59eo5jSyWqXN5hpV/GWOAsqfQCMzsgtVe1QCMnthbxn/Xfj4qlQqxBgXgY7gUMLqOndxdx5MvHG/GTU9JdNwHMdV7RZzr/CA9LBMSz3+3JI5jpjbShGYIebKSXyhLiIx5KTSLx9q6uxvFEDtfiq3LGpJiBuGhpF59Y1K2Spc7odZh11ZdYuFrBx3E0S1PdyL8uf0EcaAaQCGe8Q7bgJ7ZTsYLhXwuVjZ/k3U30By+U1vXWN4UqG/JatR5APqLX27jSLX9dJtvdLGNGhqWUazvRjdNDI2Y4jg4p2AEM3JM/Tv3cv/XMs+8MEogUDZ1a 2yw6Zlzk 3LBpnHj1T01gktp8rfarahtfGeogSv8ylEeK9hkaZNyMzcB8FVt6zO3lzKwa7D+Ifw7mRm/PiGR/S0I3XIQM4Z4LnIQjcKOdczqUtCrDSY7c9R/UDbeWzOh2131FVdXbINXCAVTvgcY4sTTj121lOyHkGhmS99Lcj9vN/SfLbal5D5y2CwH/IuT75c/WL9ZprSb/oj3x+GXuv3qpVnXtKJ4jCNTGM7HcECfhWP4j1s9sOuHwlRLVdxgGQabxqYhho9OgnMRVH4kA9iGSQ2VC1JkFUuYbAM8x0GXtnl1TCmxJpXxRl0oMWuUF7bg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch introduces a memory allocator specifically tailored for persistent memory within the kernel. The allocator maintains kernel-specific states like DMA passthrough device states, IOMMU state, and more across kexec. The current implementation provides a foundation for custom solutions that may be developed in the future. Although the design is kept concise and straightforward to encourage discussion and feedback, it remains fully functional. The persistent memory pool builds upon the continuous memory allocator (CMA) and ensures CMA state persistency across kexec by incorporating the CMA bitmap into the memory region. Potential applications include: 1. Enabling various in-kernel entities to allocate persistent pages from a unified memory pool, obviating the need for reserving multiple regions. 2. For in-kernel components that need the allocation address to be retained on kernel kexec, this address can be exposed to user space and subsequently passed through the command line. 3. Distinct subsystems or drivers can set aside their region, allocating a segment for their persistent memory pool, suitable for uses such as file systems, key-value stores, and other applications. Signed-off-by: Stanislav Kinsburskii --- include/linux/pmpool.h | 22 +++++++++ mm/Kconfig | 8 +++ mm/Makefile | 1 mm/pmpool.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 146 insertions(+) create mode 100644 include/linux/pmpool.h create mode 100644 mm/pmpool.c diff --git a/include/linux/pmpool.h b/include/linux/pmpool.h new file mode 100644 index 000000000000..b41f16fa9660 --- /dev/null +++ b/include/linux/pmpool.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _PMPOOL_H +#define _PMPOOL_H + +struct page; + +#if defined(CONFIG_PMPOOL) +struct page *pmpool_alloc(unsigned long count); +bool pmpool_release(struct page *pages, unsigned long count); +#else +static inline struct page *pmpool_alloc(unsigned long count) +{ + return NULL; +} +static inline bool pmpool_release(struct page *pages, unsigned long count) +{ + return false; +} +#endif + +#endif /* _PMPOOL_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 09130434e30d..e7c10094fb10 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -922,6 +922,14 @@ config CMA_AREAS If unsure, leave the default value "7" in UMA and "19" in NUMA. +config PMPOOL + bool "Persistent memory pool support" + select CMA + help + This option adds support for CMA-based persistent memory pool + feature, which provides pages allocation and freeing from a set of + persistent memory ranges, deposited to the memory pool. + config MEM_SOFT_DIRTY bool "Track memory changes" depends on CHECKPOINT_RESTORE && HAVE_ARCH_SOFT_DIRTY && PROC_FS diff --git a/mm/Makefile b/mm/Makefile index 678530a07326..8d3579e58c2c 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -139,3 +139,4 @@ obj-$(CONFIG_IO_MAPPING) += io-mapping.o obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o +obj-$(CONFIG_PMPOOL) += pmpool.o diff --git a/mm/pmpool.c b/mm/pmpool.c new file mode 100644 index 000000000000..c74f09b99283 --- /dev/null +++ b/mm/pmpool.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define pr_fmt(fmt) "pmpool: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "cma.h" + +struct pmpool { + struct resource resource; + struct cma *cma; +}; + +static struct pmpool *default_pmpool; + +bool pmpool_release(struct page *pages, unsigned long count) +{ + if (!default_pmpool) + return false; + + return cma_release(default_pmpool->cma, pages, count); +} + +struct page *pmpool_alloc(unsigned long count) +{ + if (!default_pmpool) + return NULL; + + return cma_alloc(default_pmpool->cma, count, 0, true); +} + +static void pmpool_cma_accomodate_bitmap(struct cma *cma) +{ + unsigned long bitmap_size; + + bitmap_free(cma->bitmap); + cma->bitmap = phys_to_virt(PFN_PHYS(cma->base_pfn)); + + bitmap_size = BITS_TO_LONGS(cma_bitmap_maxno(cma)); + memset(cma->bitmap, 0, bitmap_size); + bitmap_set(cma->bitmap, 0, PAGE_ALIGN(bitmap_size) >> PAGE_SHIFT); + + pr_info("CMA bitmap moved to %#llx\n", virt_to_phys(cma->bitmap)); +} + +static int __init default_pmpool_fixup(void) +{ + if (!default_pmpool) + return 0; + + if (insert_resource(&iomem_resource, &default_pmpool->resource)) + pr_err("failed to insert resource\n"); + + pmpool_cma_accomodate_bitmap(default_pmpool->cma); + return 0; +} +postcore_initcall(default_pmpool_fixup); + +static int __init parse_pmpool_opt(char *str) +{ + static struct pmpool pmpool = { + .resource = { + .name = "Persistent Memory Pool", + .flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM, + .desc = IORES_DESC_CXL + } + }; + phys_addr_t base, size, end; + int err; + + /* Format is pmpool=, */ + base = memparse(str, &str); + size = memparse(str + 1, NULL); + end = base + size - 1; + + err = memblock_is_region_reserved(base, size); + if (err) { + pr_err("memory block overlaps with another one: %d\n", err); + return 0; + } + + err = memblock_reserve(base, size); + if (err) { + pr_err("failed to reserve memory block: %d\n", err); + return 0; + } + + err = cma_init_reserved_mem(base, size, 0, "pmpool", &pmpool.cma); + if (err) { + pr_err("failed to initialize CMA: %d\n", err); + goto free_memblock; + } + + pmpool.resource.start = base; + pmpool.resource.end = end; + + pr_info("default memory pool is created: %#llx-%#llx\n", + base, end); + + default_pmpool = &pmpool; + + return 0; + +free_memblock: + memblock_phys_free(base, size); + return 0; +} +early_param("pmpool", parse_pmpool_opt);