From patchwork Mon Sep 25 21:27:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanislav Kinsburskii X-Patchwork-Id: 13398420 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCE96CE79A4 for ; Mon, 25 Sep 2023 21:27:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A17E6B0164; Mon, 25 Sep 2023 17:27:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32A196B0165; Mon, 25 Sep 2023 17:27:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F1706B0166; Mon, 25 Sep 2023 17:27:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 090536B0164 for ; Mon, 25 Sep 2023 17:27:46 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C75EA1A0D17 for ; Mon, 25 Sep 2023 21:27:45 +0000 (UTC) X-FDA: 81276406890.05.8C5ED8F Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by imf04.hostedemail.com (Postfix) with ESMTP id EEA9040016 for ; Mon, 25 Sep 2023 21:27:43 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=Y6F8iX0P; spf=pass (imf04.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com; dmarc=pass (policy=none) header.from=linux.microsoft.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695677264; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=hmiXPEKTxjWEpYohVVXoJZw5lHawLeSYKZ1nAgM5rOU=; b=fKIiyhA7q0qsRCqUJZ07jF3o4akNplH2DynoZSsrXSstEtFq+993Kl5Mk3UGaqTMWroZhT 38P/gW/YyeAh81kxtGcTH/+GvSsIyeXLLdmU6sJ0giEhMqN+Wh+YTihPz4epWOom6W2MoH 3FBbhwBIKWUToQROQS7Rt70pQvMicug= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695677264; a=rsa-sha256; cv=none; b=aWVbfmtiyBi3/Fyz88Y0i//4Mn/I6lXyzc1yemkjQlsNgBSxN3eeNarw0O9lnQQfcAlkOb 3s6cw1f642ZULCxyFCFtX6HauWOOucg7+TiF5FJbZmBVzWlbTgv8DyLR1QGBU0xNonf6B0 A3VukP8Wn2UFq0pBeYKkYq3CTdNicD4= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.microsoft.com header.s=default header.b=Y6F8iX0P; spf=pass (imf04.hostedemail.com: domain of skinsburskii@linux.microsoft.com designates 13.77.154.182 as permitted sender) smtp.mailfrom=skinsburskii@linux.microsoft.com; dmarc=pass (policy=none) header.from=linux.microsoft.com Received: from skinsburskii. (c-67-170-100-148.hsd1.wa.comcast.net [67.170.100.148]) by linux.microsoft.com (Postfix) with ESMTPSA id DDCDC212C81A; Mon, 25 Sep 2023 14:27:41 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com DDCDC212C81A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1695677262; bh=hmiXPEKTxjWEpYohVVXoJZw5lHawLeSYKZ1nAgM5rOU=; h=Subject:From:To:Date:From; b=Y6F8iX0PEngloXvHldjwJs4qLPA+37QFBLJJLHx8XTvU6BW+MxihCBXHsuCn78M+4 thY7oibUo0B5VVOpodRaMdMC/z/HLKbdB751/pSqGPNHnV7xnrPod7kRANVbANVs1l fKhksFV2MsV3hZfAJmP2wZVchvGg15vCgoUg2qX8= Subject: [RFC PATCH v2 0/7] Introduce persistent memory pool From: Stanislav Kinsburskii To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, ebiederm@xmission.com, akpm@linux-foundation.org, stanislav.kinsburskii@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, kexec@lists.infradead.org, linux-mm@kvack.org, kys@microsoft.com, jgowans@amazon.com, wei.liu@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, graf@amazon.de, pbonzini@redhat.com Date: Mon, 25 Sep 2023 14:27:40 -0700 Message-ID: <169567722094.19708.3583735425859054859.stgit@skinsburskii.> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Rspamd-Queue-Id: EEA9040016 X-Rspam-User: X-Stat-Signature: epfpr5b8z4gfcm75zsyew6c5mg7ewapo X-Rspamd-Server: rspam03 X-HE-Tag: 1695677263-118134 X-HE-Meta: U2FsdGVkX199Wa3xccljVkw0qj3eXfx9GbONl1D1E79JGO8/GxrLyvBZv1WFfVjpcidhoFa1DKpRg6Ji9jlkBXbx8IzFj0OwsrlpLEly+KJJa+bB/bq83Hwdtk1sahcQ+cp0pYf+QisxPVk7PfzjTvQifIQeoI/1JmLIg4faxMY/2I7SOU7MC00+gKpOkcK7VptEz0vqzgrbkWs8iJec5z0qGl/+PuoVaPNT9M+cvJ7XXJ0u8D3oFjPj84VNbBKHDmsC43rsGtc7KmalTiZkogoV9todbACsndk09NhVGGTkkuEwW1eNSUvWE4AzO+9O37xM96SZNKocJ2mUuKz03cvchiADaxStlAH3XXZTH7FvwP0LbDw2qozh9643L9pPpo/l2z5fwEwPhkUTquWzO3rncW23YyoCAx1VVSeXDu82tHf9FaYDyQBwcBqIfsXLs/GO254T5yJ5DlYHEtVRTzkfyZWRBojkAQ90Iqkm+Nj3+3S2tBhc6PDgq5dGNbXyFWROCrkoKp6nzfgQYuTwwEohiAbzuF7AI/66dNcLkW+Zz+jqCiPGPnVFy8w3NGScpaWPbazbsJbz7d6SuO68ZtNT2imCZ72Lxgqss3tT7Hin/5jvOhUEteCrSiUT+0QurnMw1Z/ay9o47gr+telgyFjowl79+kruWNgkiQRWdHRAvpDJ6ydV3U2w1VsWAUm2V038my9TSaWMGYc+/Ek7vUFjyptu8QXpUm268nj5waYGLle2J96f9b3nNEZJ/TvmMGdYHOuL3ZWls+B0rE4fO8ObqpPlDZZAwTeLYKb46pPE9kNw6Lq+AOqPPyh95CEJAGVtJKHKZv/TNjJabY0/EiIuxQvAJ1X692kEOBOuF/Hre6oX6W6CeweiviILwQkFW5EhdrVrOMS3e1jlyCbPqURgmDEUzW5lEWJ3rYsFyuXorczdeHkzLbRy+wzqahYykL0iKIZ9WRms515xg0H OHIwfhjP f13O+1hmgTFgUfytsL5Q29KY4jgICfZn4e0Xv3zEcVdr9DDcF7lSyFaNqNrx1svEz9Oss0xclumNc1kBAn0Q6mVMlbaP2plY7mzpy+ZWSYd8+ZPf8JJf9DrGBV1rY2v+YSq+Eud/vM3+RrYkB2lRKSbzP14QH/QZyXYGgHRQGyk7NInIoUo4Rfk9c5imaq5GNaXCgyFvpuCtI1QBQd0r8hSoTpjyM4It5PrQajeH0RDS3GrPvR0/OvJZKmEZo8F4n+vLqSZIFuiqzIl+L/HOvitMbIlU6QFmIbiioSFdi9rofBzR9rOs/py4lrg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch introduces a memory allocator specifically tailored for persistent memory within the kernel. The allocator maintains kernel-specific states like DMA passthrough device states, IOMMU state, and more across kexec. The current implementation provides a foundation for custom solutions that may be developed in the future. Although the design is kept concise and straightforward to encourage discussion and feedback, it remains fully functional. The persistent memory pool builds upon the continuous memory allocator (CMA) and ensures CMA state persistency across kexec by incorporating the CMA bitmap into the memory region instead of allocation it from kernel memory. Persistent memory pool metadata is passed across kexec by using Flattened Device Tree, which is added as another kexec segment for x86 architecture. Potential applications include: 1. Enabling various in-kernel entities to allocate persistent pages from a unified memory pool, obviating the need for reserving multiple regions. 2. For in-kernel components that need the allocation address to be retained on kernel kexec, this address can be exposed to user space and subsequently passed through the command line. 3. Distinct subsystems or drivers can set aside their region, allocating a segment for their persistent memory pool, suitable for uses such as file systems, key-value stores, and other applications. Notes: 1. The last patch of the series represents a use case for the feature. However, the patch won't compile and is for illustrative purposes only as the code being patched hasn't been merged yet. 2. The code being patched is currently under review by the community. The series is named "Introduce /dev/mshv drivers": https://lkml.org/lkml/2023/9/22/1117 Changes since v1: 1. Persistent memory pool is now a wrapper on top of CMA instead of being a new allocator. 2. Persistent memory pool metadata doesn't belong to the pool anymore and is now passed via Flattened Device Tree instead over kexec to the new kernel. The following series implements... --- Stanislav Kinsburskii (7): kexec_file: Add fdt modification callback support x86: kexec: Transfer existing fdt to the new kernel x86: kexec: Enable fdt modification in callbacks pmpool: Introduce persistent memory pool pmpool: Update device tree on kexec pmpool: Restore state from device tree post-kexec Drivers: hv: Allocate persistent pages for root partition arch/x86/Kconfig | 16 +++ arch/x86/kernel/kexec-bzimage64.c | 97 +++++++++++++++++ drivers/hv/hv_common.c | 13 ++ include/linux/kexec.h | 7 + include/linux/pmpool.h | 22 ++++ kernel/kexec_file.c | 24 ++++ mm/Kconfig | 9 ++ mm/Makefile | 1 mm/pmpool.c | 208 +++++++++++++++++++++++++++++++++++++ 9 files changed, 394 insertions(+), 3 deletions(-) create mode 100644 include/linux/pmpool.h create mode 100644 mm/pmpool.c