From patchwork Thu Mar 20 01:55:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80F6AC36002 for ; Thu, 20 Mar 2025 01:56:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B656D280003; Wed, 19 Mar 2025 21:56:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEDDE280001; Wed, 19 Mar 2025 21:56:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91AB6280003; Wed, 19 Mar 2025 21:56:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 76288280001 for ; Wed, 19 Mar 2025 21:56:04 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E2169120587 for ; Thu, 20 Mar 2025 01:56:04 +0000 (UTC) X-FDA: 83240263848.20.7E1B196 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf18.hostedemail.com (Postfix) with ESMTP id 1BBFB1C000C for ; Thu, 20 Mar 2025 01:56:02 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=A7Hj7w4i; spf=pass (imf18.hostedemail.com: domain of 3sXXbZwoKCFAuzs5yGCs53y66y3w.u64305CF-442Dsu2.69y@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3sXXbZwoKCFAuzs5yGCs53y66y3w.u64305CF-442Dsu2.69y@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435763; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MyKsFNp8HnVmgjRlr9CxzqXpTdvZkVjeQqS6GTtSoj0=; b=vRD5AlWFa+qoiyuwLyYrHWBB//n6sLAYOntLjgTZFPnrhxiEUbBGkqh1LGA0awlqV7aklp ZtN0qSk5Z2sudyfm9426CtHAGMbUFuZizHNLmkbgT6KB8R76X6f0zRXtpIZl8az55kfd+L BYlBhzn+xGTOa3KhN+DifNdhuTpiMxE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=A7Hj7w4i; spf=pass (imf18.hostedemail.com: domain of 3sXXbZwoKCFAuzs5yGCs53y66y3w.u64305CF-442Dsu2.69y@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3sXXbZwoKCFAuzs5yGCs53y66y3w.u64305CF-442Dsu2.69y@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435763; a=rsa-sha256; cv=none; b=O6bZkOMvJ/njn40QMvfIKvMbFyn1YVRzPfhUoDkqrEtL8GDAWYDmGFNDFR16XBciiz0MJj pSAaaV/41KZ8teDJQrlhEYBbn1nNap+YoQhqsZsDpxYcLIdRYSQu6dcP8FgOW7xNM0SJu+ u7iQAIM0cRErOojFgf8V0ZcDYcEGHcA= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff7f9a0b9bso500896a91.0 for ; Wed, 19 Mar 2025 18:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435762; x=1743040562; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MyKsFNp8HnVmgjRlr9CxzqXpTdvZkVjeQqS6GTtSoj0=; b=A7Hj7w4ib2nELWJJwssc2/k2FNvcnO8UaywvJqdxjTPHveui+GLEClmgJO7BM3R4DX z/bUl1kCGt3UfW8BgVq9Fj6tOtGJFVLrQhtY7arYBoWj8zMRql8W8L+sDDl3hZjHUh5R saKzMKuqUVARFboU+ikIIIpRgFojrNdT2KmpRS/7MV5pP9QYGMgj6U1gNqVMdN3spERS S7P5fRRW6pTOvRzan+w097vILcA3ZYwQvk1J/1Iulx1HJxYsNgd8yPBmOxyqq6rlYNR9 P2ye/19PkgxqgeztvF5b0wVVO4E14pjpf53FFtdwahoxYX/eDubHl+jGk82qVhR3ZIAo /fgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435762; x=1743040562; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MyKsFNp8HnVmgjRlr9CxzqXpTdvZkVjeQqS6GTtSoj0=; b=AtYBgDRiBIiWHFhdNEYbVZur56aKoatnkqDITaVNGllGhiujlx2OdO/OluVRV7hYzR XbbVYVr/o/mRB6rX+piz0XPmpaJ96DREXK7vo9tIbseNgpVdtspt6tJFuMn825fEcHWk r5LVZufPEer2JjUfnf03A1gCpJc505uyIOVW8aRjw/0OzmPl0TBXoaX7fgSJAYzdk7d6 yCdHq+cWuuh+CshmC5+OmQS3x4UN97aYxNzXJQtDd8PyZGYcfhi029xKcEr7pcbSSkF6 3B1uYIUHc3tU0LCyoR011zQeZaNlzqAZrbVNfdel52qrZw9yEKq+OntrBHNBLFXzgr0C PEKA== X-Forwarded-Encrypted: i=1; AJvYcCWwtFQheb6anR6LQ7x5qewM4YG3FDmaP4CYxlRmFhUBxTPoZvD3H3s2KQZRnNwTuZ/qhrWVzj5R5w==@kvack.org X-Gm-Message-State: AOJu0Yy0uUNDxzuopOxCw2hTUxfA8jNub3XR7lSjh9WcE6EDSt4I2kvD OHdg8yYMpXdq8YnASVp0lb1NUKQ9vi2GZCgN8ptfEMxpPluJjbr59/HtW+PCqrQ+cjJFmyFxgY/ lB9Cti2P2QLIgJ/O9nQ== X-Google-Smtp-Source: AGHT+IH4fNuT0QWlfSI42heq+dLpm/GFEXTXaL6bQRWMvudZOwxIJGtvzZFRNXPgaqxisIVnXeLUFEDTHfoI9HMH X-Received: from pjur6.prod.google.com ([2002:a17:90a:d406:b0:2ee:4a90:3d06]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4fc5:b0:2f6:d266:f462 with SMTP id 98e67ed59e1d1-301be20750dmr7815826a91.35.1742435761976; Wed, 19 Mar 2025 18:56:01 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:36 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-2-changyuanl@google.com> Subject: [PATCH v5 01/16] kexec: define functions to map and unmap segments From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, steven chen , Tushar Sugandhi , Changyuan Lyu X-Stat-Signature: u976gj8hfd8k63ztgtn1c6tpunhzoh68 X-Rspamd-Queue-Id: 1BBFB1C000C X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1742435762-907315 X-HE-Meta: U2FsdGVkX196fO3R11jVRo6NygiSWODsU+1zIDTs4tSTCY31L7jojswt/QP43/PCJOc7qIL9BCIBGjVS1QwUGmlhmjAuijlOCEeC/UysQj7NTw1ER2d0vAEwXfTmDV69k08RQOofZtsgwjR3HuKhXA85AOz0BNwjxaey2eibPszc4xsKEyo5Uw+9aJMgYBiibHDpjIxjaaArV9QpVefKf2NDzTFPiu//5/lvbz3cZAhSJJMs29opeX/wXIuLbcUFR7fiZnzH9uvfDwxpHT2gkwvJttkdf9l3mnIbnctoU8W+oXM6UdnlmzCnyWxa+zKYDqLCIRZm+rMeAOJcX4CqK8WXIwwLGkyYVwXBq8hYoGMqDka6mgXmK0+SfcpjRenetqmt5NgjoRZS6/0vGTZ/DBAo4n8m2BWRyH5NDn854pUK61c8f3eb5v0zdghagjzl82sxRenp3QX88Xow+VVze64Evf5MyxBPw9+ts7ydwrKo7MAuoI3NRhWsS/fKVz1EOCpZSVUPSZ4LvHLo2Qp5N5tMV2zOWtoWgf5xwvHjAm1vmjJLAnY5gAcVwsTvHFiH/VDZ8sCirbyZRoHWhPvWM8u1J7eeKnpfXXXQLtrXHVoOG4qfxHLn1LsE94uV9GFe2vYAxG4n2SelWiHlPu3IKIvTVsBo6cxjZdRU2rwvp8S3l51QlGVwquB/Wdds2RarrtWkIDd1ZwAvMzvHgSsrnAxVnNiJMfzbKV8DCs0c5JOEznZ0IXlUDlpS6Kk9n0dejetumvjOc0fw9IOmIKqztVTUX2QrGs/WsGGpjBGJ+HiUtw/b/32sSxssvO6ASsa12eeS1e/nozy8EdKQtIXJWPFxPtehRVxq8InZQWtNjFlu1Y5QrG9EJ2ZYHQgPToxYYYGhNrgTvvhDAGrrIofh/pg+s72HgstFXPfUpvx+KUdzJgNREG0N1/cxMyi5OF+VxrZLY8U1t8qMizvnb8d aSUUCmfH vjpkD7GVQ5x/O/X/PoUFdvFxYd28tYGdPGIZ/DiAP9vYfzHZLiVlZVzJnFQjC6kW9ueLfMhB/PBc8y3BLH2IRunY4cYKLoFoADkA1VMXJ93gKSrkos7YlZIS+Hd2Ta3NWg1tWerzbe4opaWowZf2enqcEMZN11mEL5FD00iJ3ghHC+6Gx1VwHOZ7AjWkdtEAsSwkEiE7aAbg00Q2xrgoJxo0MacV4oQpyvjE2/xId3D7dfmAuJYpcdM7sWESRcp+XofrXYp97QJz2snUHOUN1WLQZQ+0LQHSdEWt1kPVuIlV1qTz2puNzGJdIW1hb/PZgYe9wV/dNR12UAt/7Z5rWJhfW4p4RfKfNL0KCL3/+trnVJbH+cNi2MEk+hX/YmlpmwB/1ScxFQp481Rnmv870tlkZms3S04sDdIkkzznMGDxNTIqbCs1LLULcH/FtOLYK9LB+Juqk0bMae3qo/qC57hpVAJLDCUuWHZU/1kbQYcbS4lYYEPHKvKltoLY34bRJU4S7tCpeJ+qf1Wcjd0a7Oc7fo6wke+Y1/7dAg0g21197dnO0uoXXHOekrdJM3SXqMu2xb/IFqS1PYW3rP9HJg0JIx2ZnbgDRChoY42yEbYZ/0PtORRH6XC0GJ4BizNE7Z/0IQQGxPi3vmrAHichKSv/vJWnPRX3q6ON1MJOMPARoqys= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: steven chen Currently, the mechanism to map and unmap segments to the kimage structure is not available to the subsystems outside of kexec. This functionality is needed when IMA is allocating the memory segments during kexec 'load' operation. Implement functions to map and unmap segments to kimage. Implement kimage_map_segment() to enable mapping of IMA buffer source pages to the kimage structure post kexec 'load'. This function, accepting a kimage pointer, an address, and a size, will gather the source pages within the specified address range, create an array of page pointers, and map these to a contiguous virtual address range. The function returns the start of this range if successful, or NULL if unsuccessful. Implement kimage_unmap_segment() for unmapping segments using vunmap(). Signed-off-by: Tushar Sugandhi Signed-off-by: steven chen Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec.h | 5 ++++ kernel/kexec_core.c | 54 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/include/linux/kexec.h b/include/linux/kexec.h index f0e9f8eda7a3..fad04f3bcf1d 100644 --- a/include/linux/kexec.h +++ b/include/linux/kexec.h @@ -467,6 +467,8 @@ extern bool kexec_file_dbg_print; #define kexec_dprintk(fmt, arg...) \ do { if (kexec_file_dbg_print) pr_info(fmt, ##arg); } while (0) +void *kimage_map_segment(struct kimage *image, unsigned long addr, unsigned long size); +void kimage_unmap_segment(void *buffer); #else /* !CONFIG_KEXEC_CORE */ struct pt_regs; struct task_struct; @@ -474,6 +476,9 @@ static inline void __crash_kexec(struct pt_regs *regs) { } static inline void crash_kexec(struct pt_regs *regs) { } static inline int kexec_should_crash(struct task_struct *p) { return 0; } static inline int kexec_crash_loaded(void) { return 0; } +static inline void *kimage_map_segment(struct kimage *image, unsigned long addr, unsigned long size) +{ return NULL; } +static inline void kimage_unmap_segment(void *buffer) { } #define kexec_in_progress false #endif /* CONFIG_KEXEC_CORE */ diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index c0bdc1686154..640d252306ea 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -867,6 +867,60 @@ int kimage_load_segment(struct kimage *image, return result; } +void *kimage_map_segment(struct kimage *image, + unsigned long addr, unsigned long size) +{ + unsigned long eaddr = addr + size; + unsigned long src_page_addr, dest_page_addr; + unsigned int npages; + struct page **src_pages; + int i; + kimage_entry_t *ptr, entry; + void *vaddr = NULL; + + /* + * Collect the source pages and map them in a contiguous VA range. + */ + npages = PFN_UP(eaddr) - PFN_DOWN(addr); + src_pages = kvmalloc_array(npages, sizeof(*src_pages), GFP_KERNEL); + if (!src_pages) { + pr_err("Could not allocate source pages array for destination %lx.\n", addr); + return NULL; + } + + i = 0; + for_each_kimage_entry(image, ptr, entry) { + if (entry & IND_DESTINATION) { + dest_page_addr = entry & PAGE_MASK; + } else if (entry & IND_SOURCE) { + if (dest_page_addr >= addr && dest_page_addr < eaddr) { + src_page_addr = entry & PAGE_MASK; + src_pages[i++] = + virt_to_page(__va(src_page_addr)); + if (i == npages) + break; + dest_page_addr += PAGE_SIZE; + } + } + } + + /* Sanity check. */ + WARN_ON(i < npages); + + vaddr = vmap(src_pages, npages, VM_MAP, PAGE_KERNEL); + kvfree(src_pages); + + if (!vaddr) + pr_err("Could not map segment source pages for destination %lx.\n", addr); + + return vaddr; +} + +void kimage_unmap_segment(void *segment_buffer) +{ + vunmap(segment_buffer); +} + struct kexec_load_limit { /* Mutex protects the limit count. */ struct mutex mutex; From patchwork Thu Mar 20 01:55:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CEF3C36003 for ; Thu, 20 Mar 2025 01:56:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14466280004; Wed, 19 Mar 2025 21:56:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0AAFC280001; Wed, 19 Mar 2025 21:56:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2E0A280004; Wed, 19 Mar 2025 21:56:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AF45D280001 for ; Wed, 19 Mar 2025 21:56:06 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 860EF808FF for ; Thu, 20 Mar 2025 01:56:06 +0000 (UTC) X-FDA: 83240263932.08.07EC9AA Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf21.hostedemail.com (Postfix) with ESMTP id BD0331C0003 for ; Thu, 20 Mar 2025 01:56:04 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=I3VbfoX6; spf=pass (imf21.hostedemail.com: domain of 3s3XbZwoKCFIw1u70IEu7508805y.w86527EH-664Fuw4.8B0@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3s3XbZwoKCFIw1u70IEu7508805y.w86527EH-664Fuw4.8B0@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435764; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ilsQeNiRJnAcOliWxyBDLHi9DId7Z2YXJm9lAmFy6DI=; b=Yx2MydW3+Eo403rO8J34p8vxolLxdosTkuHCLw5BapBdx800c0qB+Jl7dnCO2oEXqIsjY+ x8FYY3aD7MqHkabMF6m7VrEUkvDcdeoo/d83IBDcfj2ugZTZWrVRkpg6NRqv0tNkNCnQZS GooXGweY1rGcF3NNUltjQXidDsrLUgQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435764; a=rsa-sha256; cv=none; b=VE8ip1JkRcWUVP4MuUK48FCH1fQcPguY5N1lmBIgZmD8NQPQ85qJYSzBwjpqqWeSON2OVm 1oHFBBx6KIWzMsQI65Mkn3IMkANFUV6AYKRBqavA87+qBVs7uRRV3vjfyNmdU2ONqyWH3T Gw4U0xNvd7lHaThRJH3apVhBUhm/jOc= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=I3VbfoX6; spf=pass (imf21.hostedemail.com: domain of 3s3XbZwoKCFIw1u70IEu7508805y.w86527EH-664Fuw4.8B0@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3s3XbZwoKCFIw1u70IEu7508805y.w86527EH-664Fuw4.8B0@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff7cf599beso661853a91.0 for ; Wed, 19 Mar 2025 18:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435764; x=1743040564; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ilsQeNiRJnAcOliWxyBDLHi9DId7Z2YXJm9lAmFy6DI=; b=I3VbfoX6YGC5EY43jAhbi0SawbUFe/AVfhGuMdb3NB7HjcAWnWDOksbPTzGRcexpBu QNGeQCzOMLbBnWBEPy69bG/qPB4xpBZS1Y2dVKxWIMo2Dh56CLLFK/Bdf8oU72cvFfqj 0osWfJYQHGes6SVK4v+CQu7U1NdiNvpOvODwyrmHgD8b9a6i/IHKSujJ47VUqsK5VAE5 VFCaiejh+/qRmbRinWKgyne+GiAqRT2ZVmkTjLWr7Eko3wRq1qt3GAOiPbGafOJqqzRa hRRtq3rl1R4WoXA0p+BOkVmdn/7L3Wd04aYUrNtU7VKGqhzv/e/LXSfoZEvTy2M5TtMq OGuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435764; x=1743040564; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ilsQeNiRJnAcOliWxyBDLHi9DId7Z2YXJm9lAmFy6DI=; b=I0h5L5voKrnEitDyNad4cpz0CRIxkoS8BYRvcqvXgnquliQrCRnLqNcFpdubwrk3nD +nSAwtAoEWjU9p7GJmkCgj674rXsSFr+cTp4hzBDo9fT/JFxprAN+k5/FseTQa4+0gY/ jiU5bCtAqWojYJH3jHHWcwxkSbD8pRP3KFeKFBFNFVeGcpte04DLgUXmI19/C2cSvEdV LhF3Tzvazn2x2SRpOZFoXO5Ah9JHkjtqkP4TN1GYHfVj0SFC6xIZCUEF7RbzTPw3RG1r g46SfCigHYdTEcBOZUKSh6jM+QdeweU736FSceUo/ShYQQJR9uKJL3TJmc9F1OzTLNUt DSeA== X-Forwarded-Encrypted: i=1; AJvYcCXPXPCr193v4CA9IZHSgmncQwMNQNnBy5dSVxzB0KF6fAo8aw+PlXMihUDZ3zH7LyFvkIIpHWu1aw==@kvack.org X-Gm-Message-State: AOJu0Yy/GfMym5H3Hr2pu10Z4/MWZZZyQDWKiD8LlmIPH6DZVpoC+PqV YV7++qsH37/mMvyA/+AZnHaz/LzA65WC5Db2pUUy7i2AYhxZ6bj2x69HRKhG6ZE7G/duV3ws+4i Za0Kjlm6v0cM91gNqog== X-Google-Smtp-Source: AGHT+IHLp0TM25605mfB4iWKOjzImtCB+u2sC+62Tcd+2pSs/X0minmUzplavXz6x1NXqtyXk3N9lUuGrg30J6Xe X-Received: from pgbdo13.prod.google.com ([2002:a05:6a02:e8d:b0:af2:8474:f67e]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:594:b0:1f5:7b6f:f8e8 with SMTP id adf61e73a8af0-1fbeae9141bmr8613381637.6.1742435763548; Wed, 19 Mar 2025 18:56:03 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:37 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-3-changyuanl@google.com> Subject: [PATCH v5 02/16] mm/mm_init: rename init_reserved_page to init_deferred_page From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: BD0331C0003 X-Stat-Signature: n3quo3jm6trgqd6feqaxommg94dy5e76 X-HE-Tag: 1742435764-518147 X-HE-Meta: U2FsdGVkX1/4uZ5ZJ5VWHC2dsqgHWeGYAwoEKkiZ0TlSiBrHk2qbY0mCb1yJPqbBsw+NIlXnx5aTK9HKk4yXHwF3e90QdG4IiDZ6NG+dAkhHzlMoZ6ihZPtnZCwKGsDs1p3UgREJlakdH8GkO5YG9PhpOHDrSroRYuA5YzLjpNeiyy3VLymznINJ7VLzMZ2DZmS3KYG0z7ozfeB0QQStebUTcRDmqP0XSW1fw98uAA8XgnoMA9CiD5YaNeqetKDrRw5qGYGTsEGBznE6yXmTcB+fYS0tnMaykQY6wowYVS4NimWfQ/mkIz+S6MgMmMIWyBokAqjVZpPF6+KjUs1NlZofCgiKS4sd2LlvLTQnZOX0tOuSa2gRYd5fLS1dpz2TiROCBZHHLq5o/x8ETVu6hr7KwfQhDKr/W09HXw6CJGaOslSssymQs3iht0a+f6IY3MxMIwQsnKrxJWCmoxnvdoYDMWe4s5WWtaNgLUA1Xv/WWJJA9RqdkWOf0wZE+2FbHn1vUlha/SwUogsil+nh7AaXkkpH79jAaqHKb/0SgqNcpatDYY4KXxiS/mHbD/Yv9ggyCDBoA7Lnf1JYez21RU8Fw7KKspsS1idsnhO01NQLLwMT2rEpNoCCu5J6BG/Wbcb3VuZuAJ3vRG59y+YmakrJvVJf1bzrOeXYXV/FdL4HRV5Vqhajdw3VdEhnyyWUviAZge9vwxT5iiWHNF+dKhYyamYIbev1dnyjTCaUhOpsijBBt9eUBrR5/xy7AsPzCoF+jftkG1wSPX0gHsDG5T4rCJwuZ9dLRlg0hsRp2XjTbifvqArzz1pppnFFDcrf13SJViYHslj6WDiYNjV2t6y7GpyKfCbkWAowxhDi3pnElIQ2WMBPQ1w8mMEYlzd0qKw6zXmJj18MwAIHUCWSM16X4LK0YIog2UZfx5mqe9UhmQn3IePXOI6/pEvmT9CBNC8dSHV5KL7IzYjbuMR eoVaQiH0 k/yl6rANHVFvUHytKAFOCrjVOZEBfVKgqQ/PmpAKXRHsQIRnqx4EaXWU+P7Rvw0LE6jkwlfisL/R+c+ehkCPBsLN0+DkdPtbVxxsBB5MnUpQYrdsYlf4JEjUSBB92ellWrKVLr2dt/BlUEGu+nUhG6tUC5KC4h9Yp2Y7f8E96dRsS7SDiyhTMrG3N2NQkdWusNLxm4xwoLfuXGmP/fjx4Y5gApa+iMu/i2B4THBAUsew964rr2dP1XV/VllyjGHEjR994NBMDxEJSLXkIrC4kPCR+NIwldzwnp0If4/pO0FptgEWGVKcjHADi6/YJ3g8GLAE71UEvIRixvSNs4ph44NAn/eA7bTMykD9Nne6++P/tcBJ9bz0fcur7G6JShV/lTjC/LjW54S1EBJOVGeZzIuTxBP9pAGvds90kek2CkIB0Mw3SFHg40IW8K8OFmscIDA9KQJcrtObXTn/6EqD7Twp1f5b1fREpPPGx4nIqoyeXmgoSybtoLR/0rr0rOK1FDSC2K9TBczq6Ne0AASAl8w2GNxu/UJEcrfa1bj03KIE5KOAVZbKwPPVyZXgDNzqS5bY59VoYK9hw0K3X+PfAy9bB6H9Ed2hveO9xYCzkkiH9ntm+LHqUDavWhWFzgx47t+7nC8vccJuELBx4Ie4cAYA3RyImZAtLvNjMPEKqv70I2a7pNm72WWcSxThHgN46s+0lDgOD6prFNnRXUJK3NlJ/om27oa8mauOH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" When CONFIG_DEFERRED_STRUCT_PAGE_INIT is enabled, init_reserved_page() function performs initialization of a struct page that would have been deferred normally. Rename it to init_deferred_page() to better reflect what the function does. Signed-off-by: Mike Rapoport (Microsoft) Signed-off-by: Changyuan Lyu --- mm/mm_init.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 2630cc30147e..c4b425125bad 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -705,7 +705,7 @@ defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } -static void __meminit init_reserved_page(unsigned long pfn, int nid) +static void __meminit init_deferred_page(unsigned long pfn, int nid) { pg_data_t *pgdat; int zid; @@ -739,7 +739,7 @@ static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } -static inline void init_reserved_page(unsigned long pfn, int nid) +static inline void init_deferred_page(unsigned long pfn, int nid) { } #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */ @@ -760,7 +760,7 @@ void __meminit reserve_bootmem_region(phys_addr_t start, if (pfn_valid(start_pfn)) { struct page *page = pfn_to_page(start_pfn); - init_reserved_page(start_pfn, nid); + init_deferred_page(start_pfn, nid); /* * no need for atomic set_bit because the struct From patchwork Thu Mar 20 01:55:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9291FC35FFC for ; Thu, 20 Mar 2025 01:56:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 166FE280005; Wed, 19 Mar 2025 21:56:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 11E29280001; Wed, 19 Mar 2025 21:56:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAAF3280005; Wed, 19 Mar 2025 21:56:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B567A280001 for ; Wed, 19 Mar 2025 21:56:07 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1C4361A07F3 for ; Thu, 20 Mar 2025 01:56:08 +0000 (UTC) X-FDA: 83240264016.29.261A229 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf23.hostedemail.com (Postfix) with ESMTP id 3F708140009 for ; Thu, 20 Mar 2025 01:56:06 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=R2xd7aWr; spf=pass (imf23.hostedemail.com: domain of 3tXXbZwoKCFQy3w92KGw972AA270.yA8749GJ-886Hwy6.AD2@flex--changyuanl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3tXXbZwoKCFQy3w92KGw972AA270.yA8749GJ-886Hwy6.AD2@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435766; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yN4bzpkkVfQ/uH0w0E99LDF9bPjq6B3vGfE1Jc/6gP4=; b=i9VvvlFNaKEgjo9x9XEL0XsAbOab/CNZJtEJFOb882cCqLYxJRCBUiuCHToQ1gDPv9Aw0C 0dPzib1UyhMDgHFOUfwMU6cgYtLR10fSBQuY0CyHWCwHdlepqiqh+L39MICMDiJGxjSB8e qm9lb0Z/CoQJSG+TOL7iaqVcLZPR9a0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435766; a=rsa-sha256; cv=none; b=rgplwyx+xuC6Bs2igNS8jmu+VLEzUVqcKjdwaHKtuXWYWcZmFmgi6zsPFs47TsvsFrwk84 8cEX3FEwmlr73bC1fkp6RLPHPm7xqUMXcvunk3cQsQiwRsWkxrP6w2ljFFY1lb9QX6dtQD MyPdAakJRvED/Qtpj/wDdZwYdfBAboI= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=R2xd7aWr; spf=pass (imf23.hostedemail.com: domain of 3tXXbZwoKCFQy3w92KGw972AA270.yA8749GJ-886Hwy6.AD2@flex--changyuanl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3tXXbZwoKCFQy3w92KGw972AA270.yA8749GJ-886Hwy6.AD2@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-301bbe9e084so851825a91.0 for ; Wed, 19 Mar 2025 18:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435765; x=1743040565; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=yN4bzpkkVfQ/uH0w0E99LDF9bPjq6B3vGfE1Jc/6gP4=; b=R2xd7aWrI2KkGwEphq784ENnfyVvb7JHOdIBdfOuH/cxliT+mk7kOaLFw2QG7a+rJL kcdMezU2c/l0Nmj6BaCzNos62YmzruWb9TD47qFSeG0Lm2Esid1sdUX65q3kDd0OMX2E k69vLfPXS5QBCheiIIJgOMZSd6C1ZvzcSDeSPVPCPU9H5o8Ytu3q+3LbtISpPNtxS64v m/XwZHrEmUCorFlusbGAKLuWxg0YPbmuTXpmOkf81Jy6n/zMGziAahbvu7u9P6NVpgMR 9reLvfUDGljgumSqTTVMXAzEAt8UqI7JGDNQxWw/+q6otcaBezve0NseQj/c9uz77RgI wMvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435765; x=1743040565; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yN4bzpkkVfQ/uH0w0E99LDF9bPjq6B3vGfE1Jc/6gP4=; b=HIIFXQnjT923dVRnQI7ynCH1bcJu1+1TU5PtQrvlPN8xRXZh/hPtPgblUCubnA8AGl SnnXZDPA8bdtBJncV7tXVInnAFec1EwMs5/S4lhxrHRAMecIvrwaBgPT7LVcAJqsuJP/ oUrkr2EnC+48lUh0irielRJI467tfcdyZP0ax6x4SWuiXzMmBNydvES1uUymacycAcu/ aFcMjAtvZKlRZ+qkxBcP1LNZqbAPgexlukw/XRC3gxKqBjnz8sA1ZwS96B3PPFL7Ini5 c9iCU92DqZu/EOzzK/nxYKIdbtLyYbt2qfqqL6L+hTHMf945/Sqh8z0kJWC3NOVZ2TdF ASSg== X-Forwarded-Encrypted: i=1; AJvYcCX0CZOg2d1HFFXULHGQ/9b0tSk75gZVL79ORjQLdLT/tsAtiw9z8NSfsaQDOJZ5qk9b84JGljgQyA==@kvack.org X-Gm-Message-State: AOJu0Yxk1LEakaD/DztTKNgiAhPmxMy6/GCduD60l9GauArjlr1JvKJD NjlmZxvW67OIv7Dj9W8EORtfGNDNP4ezXT6p1+Z6/pL9fYCCYSqBq8UKj8fqz3gOy4B9rNOCR+n EY4fnNBWQUk6FBiG2tA== X-Google-Smtp-Source: AGHT+IEsdt/zCxXN1IHqiLuLR1iz8KpffEW0rcwgNYsBwKcCWy7uT/QglIx/zEe+p2m6KToDSLlcFSwk6G8J4CvU X-Received: from pgkk67.prod.google.com ([2002:a63:2446:0:b0:af5:6108:71e4]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:9188:b0:1f5:60ce:6cc6 with SMTP id adf61e73a8af0-1fbebd7cea6mr8202773637.21.1742435765048; Wed, 19 Mar 2025 18:56:05 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:38 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-4-changyuanl@google.com> Subject: [PATCH v5 03/16] memblock: add MEMBLOCK_RSRV_KERN flag From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3F708140009 X-Stat-Signature: qt49afxedexa3ne4hnf9z6jrfztu613x X-HE-Tag: 1742435766-482362 X-HE-Meta: U2FsdGVkX1950mXR+ijMsiTynZCRjRa68uHzwyMYN2b5U0J63acQ82wEZAZWKhJItSiWATVo4aBPwmxyOODpT3/io+LkhAwBHfm27UdWcuvWQDgCzRbzuLKL0lwRIf8ao1LLDWHzm9z2AshWtwkFd1BqJjXnoGcTJWM5VobsQOn6UzmxXLrhKmrht3pu9FK5QnpUTjdC1JeikrNh15TiPFil6dXLYb1TsYOn9gWQ3bqLSYfboeRgQCjA6f6DJi37X5oA5x1nbdYGbgxIZXBIFWKLnsaFnZMSEEE4mXLY6BpeJhDUw9+CVp0c2Y4pYm4190rvgXsSn3fFPwRWLHciVP+52j6gW2AHTWMRIoW4Xyuz0z/TaanFLqQHtUVQiCAMH7K3uj2A1ilkhuu3ZLc0WcVwmDYVMh11wTXvxB3UNde/YwtnNG4Ql+FJGjXo3h14c02mdeqsXJUZeVyQzTEP+i0gd+B0l5n/XLQ2YmmUkfbB6gTzzV8gJoskawb/D2M7J06xZ9wQ4+oFaX+O6LR03MP+ZE2rBJYoLy338wK9qa6xVmHTZpjroIE5CObj3MNMV/NIHcrQ1bvfBN1Ii5lwiyoKy+ZcFNIkSk2TlXHo2ejBnIiTFUAxiNnmBLH+tJgnZksaxfz20uQFquQnRxT8X9E9CVEHZsX9ApMcWteCwEs77+nmNHJ8jfuWlbUeyHtiKKalIWIeoz6nIsEL4IifZ+LoCSGZ/6mfEJbnlHmOQvtSMExkertTcECnjGIjm99sgL16aMily94nwswysdJT80i/8grfx9ezw4F8bbBUiIjeRbdaxtFVtM4nBPE0AjojTEIUsI7MgmSzHOFkgV14qlDJtnilJCvd/a6AMkgJUqqRUBvps6O4RmlyLBhmPUPXQW+Vwh00usj5jP+IDW15JR+X22SP5h3/KTBhXSoMCmYQF85jSc33z5ZipbgqDt8UCSGIYyVnmcvvmX4+Wy6 iqGgBUoV DF5NVOG0LezBZBmZHe2ib1yvgLuJtgc5gxkvpivmc44dzPpeixbIbZAECCPt+HbB9JYJ18LCKtrfPs6KNs7u5CfqvnkHbuLJxeByKLcmWjOLgGQK40CUb3QE48pNTDoIriV9huBwHKPa6FVlVbK3Q1vjYDw11jmscM4nQIhCDggWxZpQRaBYhEdKEmVZ7/F0MiVr7pSN3Gn/5B/aJlN9ZGE0yvqT7jth9lbk6uR+KuYYBMorL0K/j1uUx9ba5EmlNuz48ivbssOzVyAsbMrcOtm2gNHA6RgRvH/e41mJ1BX45bi5z1szdBcrFDkeFm2dw+lu9/SHCQKM/X7zqgSzcGeVqGb9nL/yD301zSVIJFm7baXKI6TyInRnQV7BHCcNU9rS9mbNaMHm/oMu86rVGvj3sK6awRo9/Vxs8UitbkFc8QyXmZ5UMWhmjMXB25y+x0Q2k0eIrIIILc7hPcUeZ5/TKvsG75TQlUWxaqB8IDUHG4mAYaNtuvWYcPYirRyZqDO7Vvsdv1iFF+Yc1Uxhn2fcjS1AMKXsWdphZQ9S/mXH4s6scsUvEOAaZMlur5NRes/E144GFyoIA3uUclS/AZj/C5/ep9A/8kVyqIS85yZ+pyNcR201WVzdyQeUQb2qbU2fBjug6+KE8/moY6YcYXeTf7E2q+N1ENOROs8QUGtATTAa6mgZuw8VazjBJBqE4jrtiV+ty+CrpCqHc2UQIFcUd8A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" to denote areas that were reserved for kernel use either directly with memblock_reserve_kern() or via memblock allocations. Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/memblock.h | 19 ++++++++- mm/memblock.c | 40 +++++++++++++++---- tools/testing/memblock/tests/alloc_api.c | 22 +++++----- .../memblock/tests/alloc_helpers_api.c | 4 +- tools/testing/memblock/tests/alloc_nid_api.c | 20 +++++----- 5 files changed, 73 insertions(+), 32 deletions(-) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index e79eb6ac516f..1037fd7aabf4 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -42,6 +42,9 @@ extern unsigned long long max_possible_pfn; * kernel resource tree. * @MEMBLOCK_RSRV_NOINIT: memory region for which struct pages are * not initialized (only for reserved regions). + * @MEMBLOCK_RSRV_KERN: memory region that is reserved for kernel use, + * either explictitly with memblock_reserve_kern() or via memblock + * allocation APIs. All memblock allocations set this flag. */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -50,6 +53,7 @@ enum memblock_flags { MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ MEMBLOCK_RSRV_NOINIT = 0x10, /* don't initialize struct pages */ + MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ }; /** @@ -116,7 +120,19 @@ int memblock_add_node(phys_addr_t base, phys_addr_t size, int nid, int memblock_add(phys_addr_t base, phys_addr_t size); int memblock_remove(phys_addr_t base, phys_addr_t size); int memblock_phys_free(phys_addr_t base, phys_addr_t size); -int memblock_reserve(phys_addr_t base, phys_addr_t size); +int __memblock_reserve(phys_addr_t base, phys_addr_t size, int nid, + enum memblock_flags flags); + +static __always_inline int memblock_reserve(phys_addr_t base, phys_addr_t size) +{ + return __memblock_reserve(base, size, NUMA_NO_NODE, 0); +} + +static __always_inline int memblock_reserve_kern(phys_addr_t base, phys_addr_t size) +{ + return __memblock_reserve(base, size, NUMA_NO_NODE, MEMBLOCK_RSRV_KERN); +} + #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP int memblock_physmem_add(phys_addr_t base, phys_addr_t size); #endif @@ -477,6 +493,7 @@ static inline __init_memblock bool memblock_bottom_up(void) phys_addr_t memblock_phys_mem_size(void); phys_addr_t memblock_reserved_size(void); +phys_addr_t memblock_reserved_kern_size(phys_addr_t limit, int nid); unsigned long memblock_estimated_nr_free_pages(void); phys_addr_t memblock_start_of_DRAM(void); phys_addr_t memblock_end_of_DRAM(void); diff --git a/mm/memblock.c b/mm/memblock.c index 95af35fd1389..e704e3270b32 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -491,7 +491,7 @@ static int __init_memblock memblock_double_array(struct memblock_type *type, * needn't do it */ if (!use_slab) - BUG_ON(memblock_reserve(addr, new_alloc_size)); + BUG_ON(memblock_reserve_kern(addr, new_alloc_size)); /* Update slab flag */ *in_slab = use_slab; @@ -641,7 +641,7 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, #ifdef CONFIG_NUMA WARN_ON(nid != memblock_get_region_node(rgn)); #endif - WARN_ON(flags != rgn->flags); + WARN_ON(flags != MEMBLOCK_NONE && flags != rgn->flags); nr_new++; if (insert) { if (start_rgn == -1) @@ -901,14 +901,15 @@ int __init_memblock memblock_phys_free(phys_addr_t base, phys_addr_t size) return memblock_remove_range(&memblock.reserved, base, size); } -int __init_memblock memblock_reserve(phys_addr_t base, phys_addr_t size) +int __init_memblock __memblock_reserve(phys_addr_t base, phys_addr_t size, + int nid, enum memblock_flags flags) { phys_addr_t end = base + size - 1; - memblock_dbg("%s: [%pa-%pa] %pS\n", __func__, - &base, &end, (void *)_RET_IP_); + memblock_dbg("%s: [%pa-%pa] nid=%d flags=%x %pS\n", __func__, + &base, &end, nid, flags, (void *)_RET_IP_); - return memblock_add_range(&memblock.reserved, base, size, MAX_NUMNODES, 0); + return memblock_add_range(&memblock.reserved, base, size, nid, flags); } #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP @@ -1459,14 +1460,14 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size, again: found = memblock_find_in_range_node(size, align, start, end, nid, flags); - if (found && !memblock_reserve(found, size)) + if (found && !__memblock_reserve(found, size, nid, MEMBLOCK_RSRV_KERN)) goto done; if (numa_valid_node(nid) && !exact_nid) { found = memblock_find_in_range_node(size, align, start, end, NUMA_NO_NODE, flags); - if (found && !memblock_reserve(found, size)) + if (found && !memblock_reserve_kern(found, size)) goto done; } @@ -1751,6 +1752,28 @@ phys_addr_t __init_memblock memblock_reserved_size(void) return memblock.reserved.total_size; } +phys_addr_t __init_memblock memblock_reserved_kern_size(phys_addr_t limit, int nid) +{ + struct memblock_region *r; + phys_addr_t total = 0; + + for_each_reserved_mem_region(r) { + phys_addr_t size = r->size; + + if (r->base > limit) + break; + + if (r->base + r->size > limit) + size = limit - r->base; + + if (nid == memblock_get_region_node(r) || !numa_valid_node(nid)) + if (r->flags & MEMBLOCK_RSRV_KERN) + total += size; + } + + return total; +} + /** * memblock_estimated_nr_free_pages - return estimated number of free pages * from memblock point of view @@ -2397,6 +2420,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_NOMAP)] = "NOMAP", [ilog2(MEMBLOCK_DRIVER_MANAGED)] = "DRV_MNG", [ilog2(MEMBLOCK_RSRV_NOINIT)] = "RSV_NIT", + [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", }; static int memblock_debug_show(struct seq_file *m, void *private) diff --git a/tools/testing/memblock/tests/alloc_api.c b/tools/testing/memblock/tests/alloc_api.c index 68f1a75cd72c..c55f67dd367d 100644 --- a/tools/testing/memblock/tests/alloc_api.c +++ b/tools/testing/memblock/tests/alloc_api.c @@ -134,7 +134,7 @@ static int alloc_top_down_before_check(void) PREFIX_PUSH(); setup_memblock(); - memblock_reserve(memblock_end_of_DRAM() - total_size, r1_size); + memblock_reserve_kern(memblock_end_of_DRAM() - total_size, r1_size); allocated_ptr = run_memblock_alloc(r2_size, SMP_CACHE_BYTES); @@ -182,7 +182,7 @@ static int alloc_top_down_after_check(void) total_size = r1.size + r2_size; - memblock_reserve(r1.base, r1.size); + memblock_reserve_kern(r1.base, r1.size); allocated_ptr = run_memblock_alloc(r2_size, SMP_CACHE_BYTES); @@ -231,8 +231,8 @@ static int alloc_top_down_second_fit_check(void) total_size = r1.size + r2.size + r3_size; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc(r3_size, SMP_CACHE_BYTES); @@ -285,8 +285,8 @@ static int alloc_in_between_generic_check(void) total_size = r1.size + r2.size + r3_size; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc(r3_size, SMP_CACHE_BYTES); @@ -422,7 +422,7 @@ static int alloc_limited_space_generic_check(void) setup_memblock(); /* Simulate almost-full memory */ - memblock_reserve(memblock_start_of_DRAM(), reserved_size); + memblock_reserve_kern(memblock_start_of_DRAM(), reserved_size); allocated_ptr = run_memblock_alloc(available_size, SMP_CACHE_BYTES); @@ -608,7 +608,7 @@ static int alloc_bottom_up_before_check(void) PREFIX_PUSH(); setup_memblock(); - memblock_reserve(memblock_start_of_DRAM() + r1_size, r2_size); + memblock_reserve_kern(memblock_start_of_DRAM() + r1_size, r2_size); allocated_ptr = run_memblock_alloc(r1_size, SMP_CACHE_BYTES); @@ -655,7 +655,7 @@ static int alloc_bottom_up_after_check(void) total_size = r1.size + r2_size; - memblock_reserve(r1.base, r1.size); + memblock_reserve_kern(r1.base, r1.size); allocated_ptr = run_memblock_alloc(r2_size, SMP_CACHE_BYTES); @@ -705,8 +705,8 @@ static int alloc_bottom_up_second_fit_check(void) total_size = r1.size + r2.size + r3_size; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc(r3_size, SMP_CACHE_BYTES); diff --git a/tools/testing/memblock/tests/alloc_helpers_api.c b/tools/testing/memblock/tests/alloc_helpers_api.c index 3ef9486da8a0..e5362cfd2ff3 100644 --- a/tools/testing/memblock/tests/alloc_helpers_api.c +++ b/tools/testing/memblock/tests/alloc_helpers_api.c @@ -163,7 +163,7 @@ static int alloc_from_top_down_no_space_above_check(void) min_addr = memblock_end_of_DRAM() - SMP_CACHE_BYTES * 2; /* No space above this address */ - memblock_reserve(min_addr, r2_size); + memblock_reserve_kern(min_addr, r2_size); allocated_ptr = memblock_alloc_from(r1_size, SMP_CACHE_BYTES, min_addr); @@ -199,7 +199,7 @@ static int alloc_from_top_down_min_addr_cap_check(void) start_addr = (phys_addr_t)memblock_start_of_DRAM(); min_addr = start_addr - SMP_CACHE_BYTES * 3; - memblock_reserve(start_addr + r1_size, MEM_SIZE - r1_size); + memblock_reserve_kern(start_addr + r1_size, MEM_SIZE - r1_size); allocated_ptr = memblock_alloc_from(r1_size, SMP_CACHE_BYTES, min_addr); diff --git a/tools/testing/memblock/tests/alloc_nid_api.c b/tools/testing/memblock/tests/alloc_nid_api.c index 49bb416d34ff..562e4701b0e0 100644 --- a/tools/testing/memblock/tests/alloc_nid_api.c +++ b/tools/testing/memblock/tests/alloc_nid_api.c @@ -324,7 +324,7 @@ static int alloc_nid_min_reserved_generic_check(void) min_addr = max_addr - r2_size; reserved_base = min_addr - r1_size; - memblock_reserve(reserved_base, r1_size); + memblock_reserve_kern(reserved_base, r1_size); allocated_ptr = run_memblock_alloc_nid(r2_size, SMP_CACHE_BYTES, min_addr, max_addr, @@ -374,7 +374,7 @@ static int alloc_nid_max_reserved_generic_check(void) max_addr = memblock_end_of_DRAM() - r1_size; min_addr = max_addr - r2_size; - memblock_reserve(max_addr, r1_size); + memblock_reserve_kern(max_addr, r1_size); allocated_ptr = run_memblock_alloc_nid(r2_size, SMP_CACHE_BYTES, min_addr, max_addr, @@ -436,8 +436,8 @@ static int alloc_nid_top_down_reserved_with_space_check(void) min_addr = r2.base + r2.size; max_addr = r1.base; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc_nid(r3_size, SMP_CACHE_BYTES, min_addr, max_addr, @@ -499,8 +499,8 @@ static int alloc_nid_reserved_full_merge_generic_check(void) min_addr = r2.base + r2.size; max_addr = r1.base; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc_nid(r3_size, SMP_CACHE_BYTES, min_addr, max_addr, @@ -563,8 +563,8 @@ static int alloc_nid_top_down_reserved_no_space_check(void) min_addr = r2.base + r2.size; max_addr = r1.base; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc_nid(r3_size, SMP_CACHE_BYTES, min_addr, max_addr, @@ -909,8 +909,8 @@ static int alloc_nid_bottom_up_reserved_with_space_check(void) min_addr = r2.base + r2.size; max_addr = r1.base; - memblock_reserve(r1.base, r1.size); - memblock_reserve(r2.base, r2.size); + memblock_reserve_kern(r1.base, r1.size); + memblock_reserve_kern(r2.base, r2.size); allocated_ptr = run_memblock_alloc_nid(r3_size, SMP_CACHE_BYTES, min_addr, max_addr, From patchwork Thu Mar 20 01:55:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D51FC36000 for ; Thu, 20 Mar 2025 01:56:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5EFA6280006; Wed, 19 Mar 2025 21:56:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52915280001; Wed, 19 Mar 2025 21:56:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3571D280006; Wed, 19 Mar 2025 21:56:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0AD53280001 for ; Wed, 19 Mar 2025 21:56:09 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 73CD055F85 for ; Thu, 20 Mar 2025 01:56:09 +0000 (UTC) X-FDA: 83240264058.18.098FC95 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf12.hostedemail.com (Postfix) with ESMTP id 9D25640004 for ; Thu, 20 Mar 2025 01:56:07 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HgmtgbKG; spf=pass (imf12.hostedemail.com: domain of 3tnXbZwoKCFUz4xA3LHxA83BB381.zB985AHK-997Ixz7.BE3@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3tnXbZwoKCFUz4xA3LHxA83BB381.zB985AHK-997Ixz7.BE3@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435767; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o8JvgS9AJeolBSiA/4pI8Lzjs3pyle8TVUuTg3xWzw4=; b=4NW31wbBRQ7mihtMJabbw1rK0NxdDLHEgI79bpNHFLQ7C/amntKWykq0kEpaFqPWfBX6P9 wrXLYbWvM0YX0KKPIOBxSFw1No5+TzwVkix8oJQM0Isl7kiQM4WmzylolVzV/HrFCoIvlp oZzIRvnD6xcwwdqspgk6atj7eq+3PrY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435767; a=rsa-sha256; cv=none; b=GEth6o1iKy7GXZGaYQO/7WIV7rDhLoLsIGXJnASQ9yaQaU6hLS4tyHzaazKeXoPLnaVPDM 2kW2yG0kZL+b8yH9fCW0a/YUnDFrBTYf/ZzrhZ3kMYPHBavroTwRaI6hOGQyOTF4MqwBZn 3glt4H9nuzaBKtQtWIZfEWUerI4PtW0= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HgmtgbKG; spf=pass (imf12.hostedemail.com: domain of 3tnXbZwoKCFUz4xA3LHxA83BB381.zB985AHK-997Ixz7.BE3@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3tnXbZwoKCFUz4xA3LHxA83BB381.zB985AHK-997Ixz7.BE3@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff5296726fso820997a91.0 for ; Wed, 19 Mar 2025 18:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435766; x=1743040566; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=o8JvgS9AJeolBSiA/4pI8Lzjs3pyle8TVUuTg3xWzw4=; b=HgmtgbKGpv9YkW9Fl6ZqsfMwMSDu+Vp23e6tr4kSMxAIgfuladUqCgk4yhXMdPcgkM xOD/gLd4zuIjEXoEAXavwWovT9etsdTRuVNha+PBiDO5FtZsijs5ta5NLZYGHASmkvno q6l+DPBn5VQ7cAPGtCJS8rs44tPTmaTL1HCMdslV8VH4kp6OTGnJeysETdKDeIH4mkZG 8rclsmCipaMvcsoaxFwKKz4YyZTLNGWDePotnzMhbBAoneXIi2jF96Db4FhxJf8Qp/7E 8ICAQhW8OQB4VMR6dSZVrw4/2aULi0/oG22201sm4EFx9Z/z+fu/f8MsdUOaWLvLMQH/ W4pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435766; x=1743040566; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o8JvgS9AJeolBSiA/4pI8Lzjs3pyle8TVUuTg3xWzw4=; b=G8233zVZv2cUC1nLD4KFy8o8buueuZvoddogYhE9Ht+HBNWpD7WXXPdemPnLJDGFjP iX+E6pbZN5JmAafoHpYpqEi/DdBVk26KMK9w04b4YcejN4bpsvgLWQYQWIZqfNCeuBvo qVhcDOFAnmZYEEehknWdgBWwhlY7LDot1prFdD7QMDQheayCDaa1YUrw4DNLmflPV2y/ 77wzeAm0MVMTmVrddCYTZ4oex6OS+fn/2uD1myxGiHm6EtkasIrBK/ivNQD5vYM1dhzV mDTO83DMPTsDek0Uo1T8HVrIpph8oUSEhIzeob7xQeTNhT6XbEQfQS4In0M5tTNb09Ht 4VOw== X-Forwarded-Encrypted: i=1; AJvYcCXmWVvBY5k3rNpaAIWuab9x2l7kdlUhh2TZ6q+pROcbThyFCnGyhZ5x9LOHDwSGdW0EYR2hYnzuTg==@kvack.org X-Gm-Message-State: AOJu0Yyn7owDugPCKoQrzANDF+ODkynSOOrkdxeZKwor7ygtp/Fmby+w nXd2OJn13zcUtVYSsQDcKDyTLUChDg5tRLy5wqLSinSGNz9j84heDyQ/d/Ui/f9+2v0cQrCob90 v1FaRFbzYNKGBd0Tn+Q== X-Google-Smtp-Source: AGHT+IHJ+6KP5vNZbXdYS1/CKUc+TX5VPg88WvJbDadlOPM+I9Fd2xGdM5rk7YaUv/4ZvrxELn+iFnZzlrBohdZi X-Received: from pjk8.prod.google.com ([2002:a17:90b:5588:b0:2fc:ccfe:368]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2651:b0:2ff:6ac2:c5a5 with SMTP id 98e67ed59e1d1-301be1f90d5mr6693304a91.26.1742435766515; Wed, 19 Mar 2025 18:56:06 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:39 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-5-changyuanl@google.com> Subject: [PATCH v5 04/16] memblock: Add support for scratch memory From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9D25640004 X-Stat-Signature: p6ncmnjj9z9zm98o59ngon4c99nbk5ww X-HE-Tag: 1742435767-951240 X-HE-Meta: U2FsdGVkX1+IMrjRM7HRUmPmGDwODrVtIyke7B20YX5fcuIkexFg5Nvmq6u2KOb8cqjF9maL7WEvSzBptRVrP8Qb2n/E9W+5TUtJSJZAACXV9u8NgXQw0/zBnly5r9+8x4DuyqnhAkc4EoPpVwBekS1zGsNgKpY/Ogo2qNhxEl2pCCEiQMuEWNMPi+9vUd/wpFzSHa3BZvVlGMGS1dB94a9rY2Earyli4byvGoro9kCPA309eJJU870Q2YyYIHg40yfwoqsLw4HUx3cKF8CthD6l37fFER575lZk7EwlAxrMmDQzHaYNMSBtlUNl41TMKDxxy2D0I/grMuh3cmjOnDhmpkczpkIjVBRZTHEHBstS4AMoUw7rWYCyrdImLir+Vj+Elbb9+NRsecW0JsSsZryqjzvoj57fwb2Zjwhh0KZVLemPqpGIFTnGSCMKdGSXkKO9sGqvatWyjSZfsh/C7JqqdVIIQZHk7kpcXfwnzwYgeo4s1WLX0ci5ydXQVh57CTXbRJ3UYvtICR1BIVWNC56v0XyWYWBX9BpEIQY/GZUekpvUz3ulrFTXBfYnqOSrjVqUiz0FOE1LDt7HLwqVV1aUWtFfiw/BO34I3Apmk4gCTdc1pFkAAEApVuiaX6fdeI3/s46xJcl0zBTSbDz+wAE8xOeaJ/mjw1IrXerkO4ycfP7iPZ3lKVuvyzgsZB3wu57wdDseXv0EgOpisa/pzWHOxm+f7y4QqGoQm6E+0ssNhery3RxfH04gEF5dpJ6uLBG2mR9G0nNztR2kvH4M5nuezJG34danuK1LT7reb1pgMbeKNIHfUEyAbWpJVIHtCanKvI8YsfCNvocCu+WCGDF6+chdRh6yJnezLy9oWYiebQcKRi1Xgm38Hszu10oRNgfyX70L5tdKEXAuygHkMu8C0w6rSFlA3u4HMKVpK7J7RFQFrdELbwvxBbutxH0pkegXtdwdjVQl/I1Eo3x IicF4bgQ 31k9WTdXVrTTqOlTNQq6VV8X3ajJ8Cjaer8lTsEJ51WUEV/L1SoWOrvaJRUnP9afXXe+/24//hM0phtHkTXCUOPoT+nOHuFWzOt6ZSiJZ+0K8RcyAQwrtsE7ZIiNL+nY7MdPMfAruULl56h6TqLM7lQ/hoDnogQzKBhUr9WjjTEPAkyuUdiDoBgPv3meWIZbsGNMHaeVXoQ8bESzaQcqQJYx3s4O7DHAYGLd1ClIOFsbtBetWcvKeADwGzs9GEX1oxv21iU593eOiDyta5lFbRTStkHxlahdquSWWvoHYAzE6geiYNDEn3UuhisAJO7qOZlVH4ZC6F0yk41Qkz9HW09gOb3DXPf890govMLKJi6X4Rf7xb8DCogw+NWSC086f7te6G8++l7a7zT2SblhCChtFLoYaRKsPGmOqYAVsCzG5p+FtvhE/seN2iV25Wv+BkhRxGC6FTrbZ3vQMuWpfgDV6XX5xcVwfgd8r4Oz6mwbJP5XpP/jL/V8LeLfs+FEi0YsETQ1I3lvR9Cn2f4HLNIytqvJf3VRNJFxpyEu76Yz5h4MJulC1MlHL2lgiepoZgpCHBpSglHT4dw8P6f+6n5Wwxv+GVB2eMKdcISh2Fzdq2250cVJf+GSH4MiCAtHfpl9+Y7nrpPlrycLMpU5TsweyvYnbdP8Rfw8lj5ME48LqkX64jZT6frFn6Br++c2TE++BfFlKRM/uo4Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf With KHO (Kexec HandOver), we need a way to ensure that the new kernel does not allocate memory on top of any memory regions that the previous kernel was handing over. But to know where those are, we need to include them in the memblock.reserved array which may not be big enough to hold all ranges that need to be persisted across kexec. To resize the array, we need to allocate memory. That brings us into a catch 22 situation. The solution to that is limit memblock allocations to the scratch regions: safe regions to operate in the case when there is memory that should remain intact across kexec. KHO provides several "scratch regions" as part of its metadata. These scratch regions are contiguous memory blocks that known not to contain any memory that should be persisted across kexec. These regions should be large enough to accommodate all memblock allocations done by the kexeced kernel. We introduce a new memblock_set_scratch_only() function that allows KHO to indicate that any memblock allocation must happen from the scratch regions. Later, we may want to perform another KHO kexec. For that, we reuse the same scratch regions. To ensure that no eventually handed over data gets allocated inside a scratch region, we flip the semantics of the scratch region with memblock_clear_scratch_only(): After that call, no allocations may happen from scratch memblock regions. We will lift that restriction in the next patch. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) --- include/linux/memblock.h | 20 +++++++++++++ mm/Kconfig | 4 +++ mm/memblock.c | 61 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 85 insertions(+) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index 1037fd7aabf4..a83738b7218b 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -45,6 +45,11 @@ extern unsigned long long max_possible_pfn; * @MEMBLOCK_RSRV_KERN: memory region that is reserved for kernel use, * either explictitly with memblock_reserve_kern() or via memblock * allocation APIs. All memblock allocations set this flag. + * @MEMBLOCK_KHO_SCRATCH: memory region that kexec can pass to the next + * kernel in handover mode. During early boot, we do not know about all + * memory reservations yet, so we get scratch memory from the previous + * kernel that we know is good to use. It is the only memory that + * allocations may happen from in this phase. */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -54,6 +59,7 @@ enum memblock_flags { MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ MEMBLOCK_RSRV_NOINIT = 0x10, /* don't initialize struct pages */ MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ + MEMBLOCK_KHO_SCRATCH = 0x40, /* scratch memory for kexec handover */ }; /** @@ -148,6 +154,8 @@ int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); +int memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size); +int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size); void memblock_free_all(void); void memblock_free(void *ptr, size_t size); @@ -292,6 +300,11 @@ static inline bool memblock_is_driver_managed(struct memblock_region *m) return m->flags & MEMBLOCK_DRIVER_MANAGED; } +static inline bool memblock_is_kho_scratch(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_KHO_SCRATCH; +} + int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, @@ -620,5 +633,12 @@ static inline void early_memtest(phys_addr_t start, phys_addr_t end) { } static inline void memtest_report_meminfo(struct seq_file *m) { } #endif +#ifdef CONFIG_MEMBLOCK_KHO_SCRATCH +void memblock_set_kho_scratch_only(void); +void memblock_clear_kho_scratch_only(void); +#else +static inline void memblock_set_kho_scratch_only(void) { } +static inline void memblock_clear_kho_scratch_only(void) { } +#endif #endif /* _LINUX_MEMBLOCK_H */ diff --git a/mm/Kconfig b/mm/Kconfig index 1b501db06417..550bbafe5c0b 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -506,6 +506,10 @@ config HAVE_GUP_FAST depends on MMU bool +# Enable memblock support for scratch memory which is needed for kexec handover +config MEMBLOCK_KHO_SCRATCH + bool + # Don't discard allocated memory used to track "memory" and "reserved" memblocks # after early boot, so it can still be used to test for validity of memory. # Also, memblocks are updated with memory hot(un)plug. diff --git a/mm/memblock.c b/mm/memblock.c index e704e3270b32..c0f7da7dff47 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -106,6 +106,13 @@ unsigned long min_low_pfn; unsigned long max_pfn; unsigned long long max_possible_pfn; +#ifdef CONFIG_MEMBLOCK_KHO_SCRATCH +/* When set to true, only allocate from MEMBLOCK_KHO_SCRATCH ranges */ +static bool kho_scratch_only; +#else +#define kho_scratch_only false +#endif + static struct memblock_region memblock_memory_init_regions[INIT_MEMBLOCK_MEMORY_REGIONS] __initdata_memblock; static struct memblock_region memblock_reserved_init_regions[INIT_MEMBLOCK_RESERVED_REGIONS] __initdata_memblock; #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP @@ -165,6 +172,10 @@ bool __init_memblock memblock_has_mirror(void) static enum memblock_flags __init_memblock choose_memblock_flags(void) { + /* skip non-scratch memory for kho early boot allocations */ + if (kho_scratch_only) + return MEMBLOCK_KHO_SCRATCH; + return system_has_some_mirror ? MEMBLOCK_MIRROR : MEMBLOCK_NONE; } @@ -924,6 +935,18 @@ int __init_memblock memblock_physmem_add(phys_addr_t base, phys_addr_t size) } #endif +#ifdef CONFIG_MEMBLOCK_KHO_SCRATCH +__init_memblock void memblock_set_kho_scratch_only(void) +{ + kho_scratch_only = true; +} + +__init_memblock void memblock_clear_kho_scratch_only(void) +{ + kho_scratch_only = false; +} +#endif + /** * memblock_setclr_flag - set or clear flag for a memory region * @type: memblock type to set/clear flag for @@ -1049,6 +1072,36 @@ int __init_memblock memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t MEMBLOCK_RSRV_NOINIT); } +/** + * memblock_mark_kho_scratch - Mark a memory region as MEMBLOCK_KHO_SCRATCH. + * @base: the base phys addr of the region + * @size: the size of the region + * + * Only memory regions marked with %MEMBLOCK_KHO_SCRATCH will be considered + * for allocations during early boot with kexec handover. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.memory, base, size, 1, + MEMBLOCK_KHO_SCRATCH); +} + +/** + * memblock_clear_kho_scratch - Clear MEMBLOCK_KHO_SCRATCH flag for a + * specified region. + * @base: the base phys addr of the region + * @size: the size of the region + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.memory, base, size, 0, + MEMBLOCK_KHO_SCRATCH); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -1080,6 +1133,13 @@ static bool should_skip_region(struct memblock_type *type, if (!(flags & MEMBLOCK_DRIVER_MANAGED) && memblock_is_driver_managed(m)) return true; + /* + * In early alloc during kexec handover, we can only consider + * MEMBLOCK_KHO_SCRATCH regions for the allocations + */ + if ((flags & MEMBLOCK_KHO_SCRATCH) && !memblock_is_kho_scratch(m)) + return true; + return false; } @@ -2421,6 +2481,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_DRIVER_MANAGED)] = "DRV_MNG", [ilog2(MEMBLOCK_RSRV_NOINIT)] = "RSV_NIT", [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", + [ilog2(MEMBLOCK_KHO_SCRATCH)] = "KHO_SCRATCH", }; static int memblock_debug_show(struct seq_file *m, void *private) From patchwork Thu Mar 20 01:55:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99D65C36002 for ; Thu, 20 Mar 2025 01:56:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D12F0280007; Wed, 19 Mar 2025 21:56:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C995E280001; Wed, 19 Mar 2025 21:56:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9E08280007; Wed, 19 Mar 2025 21:56:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 80DF6280001 for ; Wed, 19 Mar 2025 21:56:10 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BFA411604C7 for ; Thu, 20 Mar 2025 01:56:10 +0000 (UTC) X-FDA: 83240264100.05.9793AE6 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf15.hostedemail.com (Postfix) with ESMTP id 3577FA000D for ; Thu, 20 Mar 2025 01:56:09 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dpcPSI4+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf15.hostedemail.com: domain of 3uHXbZwoKCFc16zC5NJzCA5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--changyuanl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3uHXbZwoKCFc16zC5NJzCA5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--changyuanl.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435769; a=rsa-sha256; cv=none; b=WCbhv4efoPStpa0CHYKiOpc4WkYPesX2u4OTdQqDetjXRBf6GkNLwdkV0yOZ/sSHi/Fz6q u8k+F29bSjafmpXE6z8NTQ5MkBovor9wJH1FcNKMul7SAn4qHSXdqJVRfzP3OtuGRPDyOH JC3F4i8JV8g1mPsjkdAMz0sOZwXVQgQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dpcPSI4+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf15.hostedemail.com: domain of 3uHXbZwoKCFc16zC5NJzCA5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--changyuanl.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3uHXbZwoKCFc16zC5NJzCA5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--changyuanl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435769; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PiwjRVxMAh+aRSJQzROvUDuvUP7bfHphETD8QOGYAo8=; b=vtqZlOcnZGMSXoGIJaljNNGXKMlLJTPbxfm0aBEta3L78gLGdDpeSHkvgOzxra7vLsZgn0 +sFfyJ9sLYTu1sqOd8vLY0P6kJjCQs7vSuUXDrTCle8NEc6Qoczzrf4uOkjXOe9lhLCQTz QQZWJWEqrTwOjyxgSc2aZJ4KSOJImpE= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-300fefb8e05so325507a91.3 for ; Wed, 19 Mar 2025 18:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435768; x=1743040568; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PiwjRVxMAh+aRSJQzROvUDuvUP7bfHphETD8QOGYAo8=; b=dpcPSI4+6PdTm2w3o+QRghLpILxS6Ids5Hw+A8pZxMEY+Iy2TCwXucR5BTMqcvkBJq P9rI/l7NzZWFHuHkufeOyGHFS/LYY/GukoCZbXS3azD8+gfBrvHxB5wB9x7x/v71njqI b14XjIC+2wahxi6KhCFBnQfWR46umZTLel2BCJIZ3sJvOQxEVSjpZ/vU0bT9aGHcfWy6 mIo9hhZCB174gNYmVxdyZJ/AR2Q44gIr0oz3CJ7bSDFXcgLXNwAzJvbvzHkYQ15aphKS uC/NNmadkviLB2W96NAQG5+uvyFu9yyx4naJrU5WQAknyhmNbHTfj0bT5BTyGeDfPqy8 XyTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435768; x=1743040568; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PiwjRVxMAh+aRSJQzROvUDuvUP7bfHphETD8QOGYAo8=; b=Dz/+mFelHEcqYIUxjUdtrvy9S2tZmNZKctUMyEnjiS9Cek5Fm1D8aBva282kPgmOSs lxQWbchQTjpWzKIg/UypeDZ3eDWipqiMRe5V1RYe23uahQGKH8h5KuCs9HyJy+gSFfuR 49cJ0DOchmsICEC7OwEbStL9Csp8RvPPzy/f8/aOcrFdq2OKF16afCuF5fWioXXmax6h VbndkBS8QvqzcLDroWYyQ6MpBsZLsqZcgGYItM2DNAWeNYCo8zDCCeRgoKpAKnOj9/36 YFNRZ3lcu9Q79HL2Ths/P5Eb9QoHnqu6v1g17++0ZElKLOUEZc/H949EV/ST276iGp2C wXjA== X-Forwarded-Encrypted: i=1; AJvYcCXhGlfxeMABd3YLaSxE8r3DmtYfHygDfk+HS3vbNnlyBU8uk2txG8ajZZA6IVQbkLgNLj7sQkdKKQ==@kvack.org X-Gm-Message-State: AOJu0YxTY1fUfTt76VBgPgBppQSEg78qQkdl5mCsLKg4fCI381+wpWz+ /AUQVcLAbk7HQ/1EtnJwf7ORUak92alXsJWtivcxupFHWOanj1Auipt4asWIN07U0T8r3FLFNcB 4d1a8JSjtYfeN2V2EDw== X-Google-Smtp-Source: AGHT+IEhVbWDRhW3hPJYHJMT+HzW9sk9EOurakKCEQdD5tgMb9uQ2NYyCFbTRsSUUBpQ3kTf4vM/PmcUJOjFZ+gE X-Received: from pglu36.prod.google.com ([2002:a63:1424:0:b0:af2:22fe:cfb3]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:4cc7:b0:1ee:e96a:d9ed with SMTP id adf61e73a8af0-1fbeaf88adfmr8896191637.7.1742435768113; Wed, 19 Mar 2025 18:56:08 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:40 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-6-changyuanl@google.com> Subject: [PATCH v5 05/16] memblock: introduce memmap_init_kho_scratch() From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Stat-Signature: znrswdgaxmddapdzy6w1w61amx4d39eb X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3577FA000D X-Rspam-User: X-HE-Tag: 1742435769-535909 X-HE-Meta: U2FsdGVkX18of8q/vT1qdW+w6c+/Fo/3zDZFN8/83O0fHjdD2pxVO9BGBJxqwuZnROxzJ21cbNmyboBwje2H4JHvaTf+w5Gw+ILeqizPbf4cZCeSSH3tgHmbY4DZpEjHjA7H1wKvY5jg7WiZxFV4w7/FDh/HC8GbdPy/IOfTZW/iySWsfmm3gvHNOfIFxrDO2GzzIUQ2mikprNr2Js2M5FNDlSwXPKy1m3cNO0VQnbBinKVygmVyD9GzOdC8CjjY1/Q3bc5sAzY3e50ZxhrHqjTAvI8rOrdXw+U5pE8X+ZprqFrohWKw5GaFV+pq4tyVWhU1x6RZW6AE55e9q6CQWFNjNbkc4Fttgc5m4qloQ7uNgRfgL3XZoPZfQIDT3JN9rsMrl+dIOK6FUpAaTZ2myi6/mh1D/4WR04lX6Mqt5aD73K2V9frmUOeoOHiPqzzbdL5xNVrerTqzEKWVvjCsfxp90XeKAvu/bvO6c2lktAnmpZDZH5jUszS2LuoD7r2/EIayRtbQJ26yjnPdS6Vi2P9F1bzeO0+N+FlzeHIYRdnefye6nnEodPEchiNyn6y++zcTsiCtAQs/GAfgl+VIEssceYX+2YmWqlv2MU6utTJ2vlwlLd+y8JopUD3HnRC1X9QjNH77ilmh5p6v4bBSveXMCjEh8i4kAeHPwxLQ+C0FMHgOp62Mkxou1lqsqn99zm0Zsp5IHm780vTkXiZIMOdrjJyzjJkUcTjmthdRDTznUfBWiFLSBlEXcSQnfkP+nR6ajYkeD0SCtxbeuUKKzJqG/NJQNXZ6/qC7FqtaBthPJFvoWe8TTLxldEbhKSw4KP5lKj5QY4iwoZ1GFGYAq9Uk0dCs6zPUEeK89ps0Timd5rS7tObSwB8hVkUbj3X50Vd37Ax2iX94ybm8TSfjRGmOPC5IqHxGdLwW8B0HDBxBtjqdYDGxLVx43REK8NN0+RluGwU1wh9RwhlHTPV KADbyyV8 XyoS9D0XvaA74y9x45Tuw835K354z0bskSqe1dp1wJYczoal5kPuf9xDg4vpw+AAROjU7+azamx25W8HfEt5GaQkyjg4I3zkuV50qkolLcKn9cQvt/bE5+ONV65ULuvzgohxpbpbTIqHriH588uKGYVUKr0Z2NqcKmdDs7wmawGiXnVJ6RxsG14dXM7m2ihssHr+qzJ/HiOepLhBjlNlKDnBdN3NRX5ZCyIdFk6L96Duh8Az7ZDYo7csFXqpxVKohPt+z1rpjLUfs2xsfvKGNtJBUCG6Xmh9U5Kw2SH7284foAFxBtLXJIAYd50Z88fdvrwDzf1tYMq0xQ7AnXJ5wbBt5S1M0kpwfgRu2gXcG6YxIZOqBNbvZbIby7iwXzSXcOTu3R7tvwCUTyGXz7/J70WJ17BuKZiy5aTLt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" With deferred initialization of struct page it will be necessary to initialize memory map for KHO scratch regions early. Add memmap_init_kho_scratch() method that will allow such initialization in upcoming patches. Signed-off-by: Mike Rapoport (Microsoft) --- include/linux/memblock.h | 2 ++ mm/internal.h | 2 ++ mm/memblock.c | 22 ++++++++++++++++++++++ mm/mm_init.c | 11 ++++++++--- 4 files changed, 34 insertions(+), 3 deletions(-) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index a83738b7218b..497e2c1364a6 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -636,9 +636,11 @@ static inline void memtest_report_meminfo(struct seq_file *m) { } #ifdef CONFIG_MEMBLOCK_KHO_SCRATCH void memblock_set_kho_scratch_only(void); void memblock_clear_kho_scratch_only(void); +void memmap_init_kho_scratch_pages(void); #else static inline void memblock_set_kho_scratch_only(void) { } static inline void memblock_clear_kho_scratch_only(void) { } +static inline void memmap_init_kho_scratch_pages(void) {} #endif #endif /* _LINUX_MEMBLOCK_H */ diff --git a/mm/internal.h b/mm/internal.h index 20b3535935a3..8e45a2ae961a 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1053,6 +1053,8 @@ DECLARE_STATIC_KEY_TRUE(deferred_pages); bool __init deferred_grow_zone(struct zone *zone, unsigned int order); #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */ +void init_deferred_page(unsigned long pfn, int nid); + enum mminit_level { MMINIT_WARNING, MMINIT_VERIFY, diff --git a/mm/memblock.c b/mm/memblock.c index c0f7da7dff47..d5d406a5160a 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -945,6 +945,28 @@ __init_memblock void memblock_clear_kho_scratch_only(void) { kho_scratch_only = false; } + +void __init_memblock memmap_init_kho_scratch_pages(void) +{ + phys_addr_t start, end; + unsigned long pfn; + int nid; + u64 i; + + if (!IS_ENABLED(CONFIG_DEFERRED_STRUCT_PAGE_INIT)) + return; + + /* + * Initialize struct pages for free scratch memory. + * The struct pages for reserved scratch memory will be set up in + * reserve_bootmem_region() + */ + __for_each_mem_range(i, &memblock.memory, NULL, NUMA_NO_NODE, + MEMBLOCK_KHO_SCRATCH, &start, &end, &nid) { + for (pfn = PFN_UP(start); pfn < PFN_DOWN(end); pfn++) + init_deferred_page(pfn, nid); + } +} #endif /** diff --git a/mm/mm_init.c b/mm/mm_init.c index c4b425125bad..04441c258b05 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -705,7 +705,7 @@ defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } -static void __meminit init_deferred_page(unsigned long pfn, int nid) +static void __meminit __init_deferred_page(unsigned long pfn, int nid) { pg_data_t *pgdat; int zid; @@ -739,11 +739,16 @@ static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } -static inline void init_deferred_page(unsigned long pfn, int nid) +static inline void __init_deferred_page(unsigned long pfn, int nid) { } #endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */ +void __meminit init_deferred_page(unsigned long pfn, int nid) +{ + __init_deferred_page(pfn, nid); +} + /* * Initialised pages do not have PageReserved set. This function is * called for each range allocated by the bootmem allocator and @@ -760,7 +765,7 @@ void __meminit reserve_bootmem_region(phys_addr_t start, if (pfn_valid(start_pfn)) { struct page *page = pfn_to_page(start_pfn); - init_deferred_page(start_pfn, nid); + __init_deferred_page(start_pfn, nid); /* * no need for atomic set_bit because the struct From patchwork Thu Mar 20 01:55:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023306 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D87FAC36000 for ; Thu, 20 Mar 2025 01:56:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2957280008; Wed, 19 Mar 2025 21:56:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD77B280001; Wed, 19 Mar 2025 21:56:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B532C280008; Wed, 19 Mar 2025 21:56:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 91FDD280001 for ; Wed, 19 Mar 2025 21:56:12 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D8387C08FC for ; Thu, 20 Mar 2025 01:56:12 +0000 (UTC) X-FDA: 83240264184.17.66F966F Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf19.hostedemail.com (Postfix) with ESMTP id 16BD61A0007 for ; Thu, 20 Mar 2025 01:56:10 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4H4Qysn2; spf=pass (imf19.hostedemail.com: domain of 3uXXbZwoKCFg270D6OK0DB6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3uXXbZwoKCFg270D6OK0DB6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435771; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8yL5b2WTU5WHGZETUIu4ND+tiGH8mntIZf0oxmIMUMA=; b=Pt+BM0Q5867zVd4QsV9OOOckTiFcoDgGUQ8K0kH7S8X9UyCsPhUUDD4a5yuqod7lrjOCfw Zbpc7qWKGv8ALcMl0mYcoc9xVEqDMQfxs23Yc0gX3B6UDo5DL2L9lKtVPQlBaLUldez8oO UQuYPN0UfcukTbaDbFrbxSqdBA67M78= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435771; a=rsa-sha256; cv=none; b=bHA1pwQmX7m3KvRd71EZBEOwqy8Gpp4HWTSdyMw5UjhZ70VXOYNDUQu1qS1hSbhXbeCzSX zlw+NAUxeP/BST9X6I5RRyr7CWHALZL2X5o/pS9NQoaEQ+k8LOc/re4/3aCUgEYJCoeq9H ISnDr8XACKG64fhIvOFR9vAdcChnV9Q= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4H4Qysn2; spf=pass (imf19.hostedemail.com: domain of 3uXXbZwoKCFg270D6OK0DB6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3uXXbZwoKCFg270D6OK0DB6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2254bdd4982so4814445ad.1 for ; Wed, 19 Mar 2025 18:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435770; x=1743040570; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8yL5b2WTU5WHGZETUIu4ND+tiGH8mntIZf0oxmIMUMA=; b=4H4Qysn28D4YMj8mnkVawjkJgs4Bm8sB2aFr1FZ3w6gYeMz9D97s+xCmHXxyARHB1c OMo1ee9+OgAx6PIjbW9WXA7yM1L4EZLfS4od7Bm+0wlJ5fNQafVVXgkRMHWxTX7tRD9X NWXcNCmCuStuuRt45k8mYNsHqas7hojUT0KELGZ7B8k61Ud0VOvNXw1jQQoYND+X6IlK Nii42532lz/Qt9+OtIqM4IbEHOOKqmtGRyKpeFh9UXQYgWJbo2tB0+GRD07a0U9eBE1H XaUMOLVXMJXTGExcYZMPf9vjELBdp2rtBIoJmySmaO7Bu/pF/upoU3JpEVurW5snldR4 gXYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435770; x=1743040570; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8yL5b2WTU5WHGZETUIu4ND+tiGH8mntIZf0oxmIMUMA=; b=mxN7ieFxEDrZJLPpYDM17KmDEkIYgQvaiY6nKNZGWCA5P+nurdx7oxJG+DRe5/JCmE DBCIx+j5l3EKBm07gCqyZM98l6hEh1AJhrMirV2LwslHC09lBvBoKMdNdX5MlpVI6Tum 6Qq/c57aMBeCPoYY2T2L8lSFRib8NXlyW29yDiDD+/eGpeD8BY10fakbKJdLlzbgFkl9 HlyNRIBtkZfFDBA+i819lz3mHDyZSN3TmP3qOAdRYl8HnFVUTe3MFiBh9UA6fpMLZLV3 yIkpRNBBnsuunYO8/qzocPTgmuWr+/+O/umMNhGNml6SkN8JkCQh8Q5mD3DBDrmXYQrB IbuA== X-Forwarded-Encrypted: i=1; AJvYcCXocmqw3FqL1fI2xCw6+xmOt4uUtLi0fXsXPmS2MhcO1LGPNUSfdaSeZ5UC8z3rn9td7iYQUJt9Ig==@kvack.org X-Gm-Message-State: AOJu0Yw/0MZqLkEqjPf25pEuYtTEHzA1B3Myxq7JPDY2qojJnjP3fZvV ahG/A+X7Wae4aU9vLYE+OjKMuC/bPf4D7jyV/FmVmPx8DZg5phAPcJcYPAvClakGb6mpKwkN9fI d1hdTLCZjTJ9koAQrwQ== X-Google-Smtp-Source: AGHT+IHW15FvPpMLWn1HKaL85qfGSHpQDCxTWGFYK6N/65irCZHXzpE1wn1H8XnTV124IW6pqODmzO2IIQUyBB4r X-Received: from pfbna11.prod.google.com ([2002:a05:6a00:3e0b:b0:730:4672:64ac]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4187:b0:736:fff2:99b with SMTP id d2e1a72fcca58-7376d6ff535mr8193650b3a.23.1742435769805; Wed, 19 Mar 2025 18:56:09 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:41 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-7-changyuanl@google.com> Subject: [PATCH v5 06/16] hashtable: add macro HASHTABLE_INIT From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 16BD61A0007 X-Stat-Signature: q637xacf8g56pgp7xa1cmbzyrrpy4djd X-HE-Tag: 1742435770-916396 X-HE-Meta: U2FsdGVkX19iVFvksaOdd782jwstX7BE+Em3yBL11dZz3HhHpPK6a2viduvZ5Xh+ksQ4iMV7U11hXmZai/IFv8nvfDsNEopn3ukGeYnaxax47kXeaVy1TPaGvk+mz1F7CfhiBcOlEbmNr7zFDeAP4TRFiGG7XYefzg48lmBB6XEh38sfXCkk7Wptv8jDNGZMSBXQcU+YKl2xHST9r5TqYhEumVV8m5fJYHuc6IT3ynlh6CexlFKqmWl5AhI2NM2j4+ZZQfc26lVppOi089t1ZHn0VWhd7HOZuCJLYKpsk1EqBK7JrlS6MDnKlEJu2Hdw780aGlhjHuBB12klkCWgNnNqFna5YsjYrEfLrr48Xs4eHk+FEntZiRSUfm7Pru+YlfcDmJP8tzO0TB8oCT9q00lZqEnwI1ASf43ROZVa3esqRZDyDFjktYPyiJURY0cK1/E4uxsOfrSISCQubo2x5jkL2p+72+ce76qBu9pbUCX//qqJpdLdYTgBFJeYb4Ax3c0/3hbD9mswRPTpD5vnIPq4tsPV/7LqpsKri0aFN5VQovp7zjV5WXYdN1u4ZL34IY7RFzMXcoPL0PnVn3CgEL4BubNNzni6S1x+UuNoOMY8MbXSBs/Jjv2nD9A9ytz5gLHj9UF1xTizni8tgqNDLgEeOYQSTo5q2CBwUme9Jk7zV4dqSEz1XzJY5bDhbP/Y+t04N+uf5hY2945pm4SuX1YgmByKBZnLuNf7vcn4nNzjh3P24knyydcjUCeA84y/q3MRvlazOZYJx/ZMYfHx/6qiSrmAQ7H8Tjn4QnIvkvkvylxqiITHUZ2FKR6k+n/acRh5Xl0Iie3c4YSyqFE17tCkeh8d0PgcFu5P6dRtBMVQTQUHaDHGUf20K7oPjFVMbmSZYpBuugil+z0hYjh0rSO2Y4ItkkG9u/YLmvnqU3ppOTjSBRJBzSxLf6JGXlEUUTXDyo4h52C7srHtoG8 1hnsYdj7 fW39yshxzK/BLc5nEXedoIDaRX0NTadzwtxe7wTo2lySe3tuGnfJVP475m98sEHHWrHo89aVgDkQc9/0px7lFQ90cwuIXId0JqVVELeSUoWeCGpOyjd8DpWJS3u7kagUHCQ+uGzAGChPHm+Cy7hZa72a4DjBgsuVdwpLnJManc8L+WLtSk9H1O6qlICA89PBSziu9kbaWfc2/BjANFOycvPCmZKR5mZY0TV/3CNrwwzcSZJB4ljwMUdcaBevnb+8+51oL4EpTEUuVMdyd+w7dyyZCw6++bHzQbPfGOlRDegnNAFDgt8/M2LgrloHaeJJ0A502m33gbu4IkRln2znzBPpEGsR3moCe6ZkswdDvyy62Xd2X6nV4zv5XjtL2P0UzXl7U9ERfvw38dLMrXpFD40mdK1n+jlaZ2KH6vx7BrfKXULLfRxYe4uOWLgS2Usfv2Se8ZeY3MCx6OKyJjG3wLl2tu4tTqPBFaFgTvUZl8ebFjw+BZnRFT/Pf6eqpwlVmp/vrLbdGkopWEONcmAyzpJaCYXUewKwrADWe6v7D2lxbQPTm+uV7jWd/gPyvoYumRk9FZHHHw3AchOy0rw7ze98yPIsl20N1g3rQfRowzSyusYjzkr0Z9gpG0tp1BhYmyPufzSjWANbq1xgFjUpaFFuaqVIDIAb+DO03CLR3fXLiiH9Yu2X5GJD5+jgHTVfndWWh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Similar to HLIST_HEAD_INIT, HASHTABLE_INIT allows a hashtable embedded in another structure to be initialized at compile time. Example, struct tree_node { DECLARE_HASHTABLE(properties, 4); DECLARE_HASHTABLE(sub_nodes, 4); }; static struct tree_node root_node = { .properties = HASHTABLE_INIT(4), .sub_nodes = HASHTABLE_INIT(4), }; Signed-off-by: Changyuan Lyu --- include/linux/hashtable.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/include/linux/hashtable.h b/include/linux/hashtable.h index f6c666730b8c..27e07a436e2a 100644 --- a/include/linux/hashtable.h +++ b/include/linux/hashtable.h @@ -13,13 +13,14 @@ #include #include +#define HASHTABLE_INIT(bits) { [0 ... ((1 << (bits)) - 1)] = HLIST_HEAD_INIT } + #define DEFINE_HASHTABLE(name, bits) \ - struct hlist_head name[1 << (bits)] = \ - { [0 ... ((1 << (bits)) - 1)] = HLIST_HEAD_INIT } + struct hlist_head name[1 << (bits)] = HASHTABLE_INIT(bits) \ #define DEFINE_READ_MOSTLY_HASHTABLE(name, bits) \ struct hlist_head name[1 << (bits)] __read_mostly = \ - { [0 ... ((1 << (bits)) - 1)] = HLIST_HEAD_INIT } + HASHTABLE_INIT(bits) #define DECLARE_HASHTABLE(name, bits) \ struct hlist_head name[1 << (bits)] From patchwork Thu Mar 20 01:55:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC801C36002 for ; Thu, 20 Mar 2025 01:56:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8148C280009; Wed, 19 Mar 2025 21:56:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 79E21280001; Wed, 19 Mar 2025 21:56:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A69D280009; Wed, 19 Mar 2025 21:56:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 28F61280001 for ; Wed, 19 Mar 2025 21:56:14 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8A5081C8106 for ; Thu, 20 Mar 2025 01:56:14 +0000 (UTC) X-FDA: 83240264268.14.01B27A5 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf17.hostedemail.com (Postfix) with ESMTP id 9CBD440004 for ; Thu, 20 Mar 2025 01:56:12 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=gqI5mf4I; spf=pass (imf17.hostedemail.com: domain of 3u3XbZwoKCFo492F8QM2FD8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3u3XbZwoKCFo492F8QM2FD8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435772; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UfibToy0nGoCGguqNLW86QgSIt/epNH6HRSzzhgRgfw=; b=Rn5Ku5/xa47s2nHI+m4wQ/4XlcFfUWGtxXa50gM4tdhaF6XMezGoss57HB2WVsMI/NNEDT 2SVuj4F9lh8uxZoPr/7FxbUwLP5ccSA1fjS6Qw3th1ZmoBH4ZGQYImqofMDoBYeUjdW91J nD3veowpXx9l+pV+ioRMS1Dnje9r9uA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435772; a=rsa-sha256; cv=none; b=tT2Bodm4v1G+CNwtIiLPTcVQHjTXKtN48hX+E1yDojttEHV4ZUPkIvY9ekH+iytQlj+axT RT7iQEyjpOS8G4ZU3PS2As3NeFapfexsQPCat4s15THL90jbqwg6hqJhSIWuM9aSX8YZaf 1Jlrsl1Wy3YOrzZlYf23IhjQPuB+tV8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=gqI5mf4I; spf=pass (imf17.hostedemail.com: domain of 3u3XbZwoKCFo492F8QM2FD8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3u3XbZwoKCFo492F8QM2FD8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-225107fbdc7so3366495ad.0 for ; Wed, 19 Mar 2025 18:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435771; x=1743040571; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UfibToy0nGoCGguqNLW86QgSIt/epNH6HRSzzhgRgfw=; b=gqI5mf4IDqnUxcHvExFpQIHNV4LIkeWwq+uyWGBMqVE2EPPxUqWTPYIlRIkjomwODG xjwpL3HP4+4qJK7vq5rsn0ldNwnAfNFiM4Nzrsr37/B6aQbqcC14u4SKptlUTOv+iiDw k+Xv87/1rK/7jMQMY0QSYjI09jaJzCRh6jZHTncYNX6DCGruzeQPsnxAcAmmvonWbBVV 2iL7iDiUw/ceadeUrQOncF14D1PBw849QCnsM2Tvaw7Kjnd0Bv9FC1x/yACoFs/u/eYb zrD+2rmpOJ2UKlb1orRCBM7GLqs/61rQsxmLFfxalOFavjXhNQ4L0JPmTSaDmChhG/Yz MBSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435771; x=1743040571; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UfibToy0nGoCGguqNLW86QgSIt/epNH6HRSzzhgRgfw=; b=lvId09GsQTMwvCTyumDainR1FxSk4YBMKQ8l3RTkAI0uS0kGPbSCCJcq0f09SzSq6x LvlINtwlJsfZk5UI1Y4Kcnfl/j7U7EWG02qNIMcovNhN9BByEsQKVrszge/RpkxyrRL2 AFz+EiR4T5np8nspT9MawagXFo2+aYpeOtAWlUM7BPtrtGjclgDUZLj32D9eIs9MPvkE FD3rCI1Gi1556dtu4p32moT/ro6nvKzIBjYITP7jfRqk1ZLG3KPbfkxS1yB1KRtZDMPf Toz9+oFtyRZkKio3VRpBQ/h+bS6l5TVt8XWR1ud6oUAD13FtVysaqaZarwT5VWl62Z7R uiSQ== X-Forwarded-Encrypted: i=1; AJvYcCUrdArFMpQnxCHZgG+dN+jyoANDCckq6OycZDJGgcnYhSREQE/2JkibDmcd/+izZLxF9+c3OWjnKQ==@kvack.org X-Gm-Message-State: AOJu0Yz1AhBJkyyVi3wyVH6A/cTCu9qA7/YKtDTZyV2wXP1sXaxXMwwH tWHQlwVM1bolXObGIYYdY3ZP2CZJWG33wAkKrw4VmL1s+RW5Y0tKWLGFbRjx29QEo6/aOxFPHq5 Z7QYPd6PnkmGIszejXw== X-Google-Smtp-Source: AGHT+IHonsyQBMv9RBxtQtceR/ApPbVZwOG1MadGL1Le7a4EopOnz4DX+w8JLu4FBcso+eobsS8t+IumlBcRGPW6 X-Received: from plbi21.prod.google.com ([2002:a17:903:20d5:b0:223:5c58:1855]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:8c8:b0:224:1294:1d26 with SMTP id d9443c01a7336-2265edbf303mr23035225ad.13.1742435771495; Wed, 19 Mar 2025 18:56:11 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:42 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-8-changyuanl@google.com> Subject: [PATCH v5 07/16] kexec: add Kexec HandOver (KHO) generation helpers From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 9CBD440004 X-Stat-Signature: 9xoweptebf43akkwsx8rx97ek5gyxuha X-HE-Tag: 1742435772-156674 X-HE-Meta: U2FsdGVkX197K7MZpqpOU0qQuNICbgNkTKzIA5gCOgOcyDwkpV4OsxPyjQsStyCK2kvrIWDmm4yATZepPlVK/btRfx/McSeIkodz9PD/7revys2reZ11xX8Ik6LWuZu6GoSrl2fa2PvM6A8CJ83jWrMUFXZa6xflV7Vv5C2ef8I6sgs2K6Zkrvd306BaLUopgCtW4ykWLi7TS+7GliYhbxrZOIA06qiCnoFrw3cai9zzT1+UoPuVedhUtBdVJNE2+XXTUcoI51/h+AUfFqTf3sdgbay86/N2As2v7nb96C3LxoAp3Da345FLpMP90fLHkuGzgMATm9COgQAZP8Ve05vqDHK7GrZQDKjr3N/CWCzTZJOZj8wvkraZ/GruLGA3WBshOcWZic5LQVEFaSfbXmGet8yrtmKco+1OrEExqXXdRc9Kx/bkhmuad0UOfdCtPiDUAncD20Yy+Jf9DP7VBmxfVL7xSIW7/sVSA1xSLDX9igvG8p9dNlW7r235ukvhlBYcrA0H3Oo3CcSKmLJqkxvt6p3880/HOC9TidC6vGUOJ6cgVJWFNTj7W1UMtRYC16HRwTQvtepHI4K0DdiC2aNhYrT7rjqSIELyzseS96tGCbDPTjA8o8nKaee8ohi0aoCb9QQXw5DXvIJIdHY8mX46Qd7ox9OwNWB4ZhwkJuV7oEzzv60SOJkRYkkhExqi0JfLDWNlCB2rtD2VOJJnol1aTC4g0yb02SYiqsjctoVuIUAns1dpkejIymBvAquG9P6d6DL0qO1gdmEnFRgaoLfuN6it5y8bAM8ZkNPARfr4ZYTxpEDBh78gUppw+Q7FzsIL3YfcT4Pf/GSI20/LhNLQ/SXJpXccMG69NCSkyzIVDJN+69Gkdf6fiYPuEOXDtTfxBstsn0fK5aSeuhVHYfFog5Gj8u7Yx3Sz/NtRKX7lGYWlhdGSbCgHb7wBLezsHW58d+Zcv70aXu9D47N TmkrXFWC /0WyfKemvfq7weKBxz/cd7BKw4PoQMZmi8j2qzMhIS5G95mnaemrJbnpzmWngMHMsBdseO/tMx72e1wWZRCLX+bezdgF0Cq/DOTYYHSd9R6Z+HJqaP9nTrb5vU98aDbfom2w2orpydYyPVw9pdhR/jxQCEnQ49bFWQ+E2zGb0ktN25A5b5cOIvP/ZUMoEtdt6M2T/yQ22rvFL5t2SPwoH4gERbmLWOciq6Ygd9tvzL3D9Ec4bsHnNNMu4AQYk3oBq6xUuQRoM49/hdDmI4aE6eUoME4Yquh1RgsN6SvyzZ2CAAbgvuo59OIYpyEGtMIU+tDl4RbfbOTNy2F6z2MJSv7Sbo1+3vsVbUn12HVMd8SSoO6aC5PtRqfST2cQvLfZwohUw9brE8B+B9Brn7PlVAAywVVw7qba35haXNjm/Og6YCFi8rOWv7dkpYSZjZ3zZHwJgPSxBjB6TP++ihzl70e0OT7Tj46t3OixWyxafAqkOFOuOEYHHiYsGEd6z8yDBoatoUUPnzVueH+NHjt5cuzVOTAtOu8OPwmankR/OMnbeZeCUY7iYduMZG1M6LSRVDjwDbawE5MkaDh4XP5EjDi+GbNZ6MFvxTslcTea15Sv/R1n6hnGdHmgAwm5+bLzvT0NiRPCl0dUrSbI3MtLZCyr/J64nky0/W598iSXtDlmkk3fDmJC8P2onpcHi3jK2St30OVUNLJep8GLaDlvGa6JVXxFuJdA177tQTLEBg86TAqcWSYcWynQOZokzu+BMf41o X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf Add the core infrastructure to generate Kexec HandOver metadata. Kexec HandOver is a mechanism that allows Linux to preserve state - arbitrary properties as well as memory locations - across kexec. It does so using 2 concepts: 1) State Tree - Every KHO kexec carries a state tree that describes the state of the system. The state tree is represented as hash-tables. Device drivers can add/remove their data into/from the state tree at system runtime. On kexec, the tree is converted to FDT (flattened device tree). 2) Scratch Regions - CMA regions that we allocate in the first kernel. CMA gives us the guarantee that no handover pages land in those regions, because handover pages must be at a static physical memory location. We use these regions as the place to load future kexec images so that they won't collide with any handover data. Signed-off-by: Alexander Graf Co-developed-by: Pratyush Yadav Signed-off-by: Pratyush Yadav Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- MAINTAINERS | 2 +- include/linux/kexec_handover.h | 109 +++++ kernel/Makefile | 1 + kernel/kexec_handover.c | 865 +++++++++++++++++++++++++++++++++ mm/mm_init.c | 8 + 5 files changed, 984 insertions(+), 1 deletion(-) create mode 100644 include/linux/kexec_handover.h create mode 100644 kernel/kexec_handover.c diff --git a/MAINTAINERS b/MAINTAINERS index 12852355bd66..a000a277ccf7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12828,7 +12828,7 @@ F: include/linux/kernfs.h KEXEC L: kexec@lists.infradead.org W: http://kernel.org/pub/linux/utils/kernel/kexec/ -F: include/linux/kexec.h +F: include/linux/kexec*.h F: include/uapi/linux/kexec.h F: kernel/kexec* diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h new file mode 100644 index 000000000000..9cd9ad31e2d1 --- /dev/null +++ b/include/linux/kexec_handover.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef LINUX_KEXEC_HANDOVER_H +#define LINUX_KEXEC_HANDOVER_H + +#include +#include +#include + +struct kho_scratch { + phys_addr_t addr; + phys_addr_t size; +}; + +/* KHO Notifier index */ +enum kho_event { + KEXEC_KHO_FINALIZE = 0, + KEXEC_KHO_UNFREEZE = 1, +}; + +#define KHO_HASHTABLE_BITS 3 +#define KHO_NODE_INIT \ + { \ + .props = HASHTABLE_INIT(KHO_HASHTABLE_BITS), \ + .nodes = HASHTABLE_INIT(KHO_HASHTABLE_BITS), \ + } + +struct kho_node { + struct hlist_node hlist; + + const char *name; + DECLARE_HASHTABLE(props, KHO_HASHTABLE_BITS); + DECLARE_HASHTABLE(nodes, KHO_HASHTABLE_BITS); + + struct list_head list; + bool visited; +}; + +#ifdef CONFIG_KEXEC_HANDOVER +bool kho_is_enabled(void); +void kho_init_node(struct kho_node *node); +int kho_add_node(struct kho_node *parent, const char *name, + struct kho_node *child); +struct kho_node *kho_remove_node(struct kho_node *parent, const char *name); +int kho_add_prop(struct kho_node *node, const char *key, const void *val, + u32 size); +void *kho_remove_prop(struct kho_node *node, const char *key, u32 *size); +int kho_add_string_prop(struct kho_node *node, const char *key, + const char *val); + +int register_kho_notifier(struct notifier_block *nb); +int unregister_kho_notifier(struct notifier_block *nb); + +void kho_memory_init(void); +#else +static inline bool kho_is_enabled(void) +{ + return false; +} + +static inline void kho_init_node(struct kho_node *node) +{ +} + +static inline int kho_add_node(struct kho_node *parent, const char *name, + struct kho_node *child) +{ + return -EOPNOTSUPP; +} + +static inline struct kho_node *kho_remove_node(struct kho_node *parent, + const char *name) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline int kho_add_prop(struct kho_node *node, const char *key, + const void *val, u32 size) +{ + return -EOPNOTSUPP; +} + +static inline void *kho_remove_prop(struct kho_node *node, const char *key, + u32 *size) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline int kho_add_string_prop(struct kho_node *node, const char *key, + const char *val) +{ + return -EOPNOTSUPP; +} + +static inline int register_kho_notifier(struct notifier_block *nb) +{ + return -EOPNOTSUPP; +} + +static inline int unregister_kho_notifier(struct notifier_block *nb) +{ + return -EOPNOTSUPP; +} + +static inline void kho_memory_init(void) +{ +} +#endif /* CONFIG_KEXEC_HANDOVER */ + +#endif /* LINUX_KEXEC_HANDOVER_H */ diff --git a/kernel/Makefile b/kernel/Makefile index 87866b037fbe..cef5377c25cd 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -75,6 +75,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_core.o obj-$(CONFIG_KEXEC) += kexec.o obj-$(CONFIG_KEXEC_FILE) += kexec_file.o obj-$(CONFIG_KEXEC_ELF) += kexec_elf.o +obj-$(CONFIG_KEXEC_HANDOVER) += kexec_handover.o obj-$(CONFIG_BACKTRACE_SELF_TEST) += backtracetest.o obj-$(CONFIG_COMPAT) += compat.o obj-$(CONFIG_CGROUPS) += cgroup/ diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c new file mode 100644 index 000000000000..df0d9debbb64 --- /dev/null +++ b/kernel/kexec_handover.c @@ -0,0 +1,865 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * kexec_handover.c - kexec handover metadata processing + * Copyright (C) 2023 Alexander Graf + * Copyright (C) 2025 Microsoft Corporation, Mike Rapoport + * Copyright (C) 2024 Google LLC + */ + +#define pr_fmt(fmt) "KHO: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +/* + * KHO is tightly coupled with mm init and needs access to some of mm + * internal APIs. + */ +#include "../mm/internal.h" +#include "kexec_internal.h" + +static bool kho_enable __ro_after_init; + +bool kho_is_enabled(void) +{ + return kho_enable; +} +EXPORT_SYMBOL_GPL(kho_is_enabled); + +static int __init kho_parse_enable(char *p) +{ + return kstrtobool(p, &kho_enable); +} +early_param("kho", kho_parse_enable); + +/* + * With KHO enabled, memory can become fragmented because KHO regions may + * be anywhere in physical address space. The scratch regions give us a + * safe zones that we will never see KHO allocations from. This is where we + * can later safely load our new kexec images into and then use the scratch + * area for early allocations that happen before page allocator is + * initialized. + */ +static struct kho_scratch *kho_scratch; +static unsigned int kho_scratch_cnt; + +static struct dentry *debugfs_root; + +struct kho_out { + struct blocking_notifier_head chain_head; + + struct debugfs_blob_wrapper fdt_wrapper; + struct dentry *fdt_file; + struct dentry *dir; + + struct rw_semaphore tree_lock; + struct kho_node root; + + void *fdt; + u64 fdt_max; +}; + +static struct kho_out kho_out = { + .chain_head = BLOCKING_NOTIFIER_INIT(kho_out.chain_head), + .tree_lock = __RWSEM_INITIALIZER(kho_out.tree_lock), + .root = KHO_NODE_INIT, + .fdt_max = 10 * SZ_1M, +}; + +int register_kho_notifier(struct notifier_block *nb) +{ + return blocking_notifier_chain_register(&kho_out.chain_head, nb); +} +EXPORT_SYMBOL_GPL(register_kho_notifier); + +int unregister_kho_notifier(struct notifier_block *nb) +{ + return blocking_notifier_chain_unregister(&kho_out.chain_head, nb); +} +EXPORT_SYMBOL_GPL(unregister_kho_notifier); + +/* Helper functions for KHO state tree */ + +struct kho_prop { + struct hlist_node hlist; + + const char *key; + const void *val; + u32 size; +}; + +static unsigned long strhash(const char *s) +{ + return xxhash(s, strlen(s), 1120); +} + +void kho_init_node(struct kho_node *node) +{ + hash_init(node->props); + hash_init(node->nodes); +} +EXPORT_SYMBOL_GPL(kho_init_node); + +/** + * kho_add_node - add a child node to a parent node. + * @parent: parent node to add to. + * @name: name of the child node. + * @child: child node to be added to @parent with @name. + * + * If @parent is NULL, @child is added to KHO state tree root node. + * + * @child must be a valid pointer through KHO FDT finalization. + * @name is duplicated and thus can have a short lifetime. + * + * Callers must use their own locking if there are concurrent accesses to + * @parent or @child. + * + * Return: 0 on success, 1 if @child is already in @parent with @name, or + * - -EOPNOTSUPP: KHO is not enabled in the kernel command line, + * - -ENOMEM: failed to duplicate @name, + * - -EBUSY: KHO state tree has been converted to FDT, + * - -EEXIST: another node of the same name has been added to the parent. + */ +int kho_add_node(struct kho_node *parent, const char *name, + struct kho_node *child) +{ + unsigned long name_hash; + int ret = 0; + struct kho_node *node; + char *child_name; + + if (!kho_enable) + return -EOPNOTSUPP; + + if (!parent) + parent = &kho_out.root; + + child_name = kstrdup(name, GFP_KERNEL); + if (!child_name) + return -ENOMEM; + + name_hash = strhash(child_name); + + if (parent == &kho_out.root) + down_write(&kho_out.tree_lock); + else + down_read(&kho_out.tree_lock); + + if (kho_out.fdt) { + ret = -EBUSY; + goto out; + } + + hash_for_each_possible(parent->nodes, node, hlist, name_hash) { + if (!strcmp(node->name, child_name)) { + ret = node == child ? 1 : -EEXIST; + break; + } + } + + if (ret == 0) { + child->name = child_name; + hash_add(parent->nodes, &child->hlist, name_hash); + } + +out: + if (parent == &kho_out.root) + up_write(&kho_out.tree_lock); + else + up_read(&kho_out.tree_lock); + + if (ret) + kfree(child_name); + + return ret; +} +EXPORT_SYMBOL_GPL(kho_add_node); + +/** + * kho_remove_node - remove a child node from a parent node. + * @parent: parent node to look up for. + * @name: name of the child node. + * + * If @parent is NULL, KHO state tree root node is looked up. + * + * Callers must use their own locking if there are concurrent accesses to + * @parent or @child. + * + * Return: the pointer to the child node on success, or an error pointer, + * - -EOPNOTSUPP: KHO is not enabled in the kernel command line, + * - -ENOENT: no node named @name is found. + * - -EBUSY: KHO state tree has been converted to FDT. + */ +struct kho_node *kho_remove_node(struct kho_node *parent, const char *name) +{ + struct kho_node *child, *ret = ERR_PTR(-ENOENT); + unsigned long name_hash; + + if (!kho_enable) + return ERR_PTR(-EOPNOTSUPP); + + if (!parent) + parent = &kho_out.root; + + name_hash = strhash(name); + + if (parent == &kho_out.root) + down_write(&kho_out.tree_lock); + else + down_read(&kho_out.tree_lock); + + if (kho_out.fdt) { + ret = ERR_PTR(-EBUSY); + goto out; + } + + hash_for_each_possible(parent->nodes, child, hlist, name_hash) { + if (!strcmp(child->name, name)) { + ret = child; + break; + } + } + + if (!IS_ERR(ret)) { + hash_del(&ret->hlist); + kfree(ret->name); + ret->name = NULL; + } + +out: + if (parent == &kho_out.root) + up_write(&kho_out.tree_lock); + else + up_read(&kho_out.tree_lock); + + return ret; +} +EXPORT_SYMBOL_GPL(kho_remove_node); + +/** + * kho_add_prop - add a property to a node. + * @node: KHO node to add the property to. + * @key: key of the property. + * @val: pointer to the property value. + * @size: size of the property value in bytes. + * + * @val and @key must be valid pointers through KHO FDT finalization. + * Generally @key is a string literal with static lifetime. + * + * Callers must use their own locking if there are concurrent accesses to @node. + * + * Return: 0 on success, 1 if the value is already added with @key, or + * - -ENOMEM: failed to allocate memory, + * - -EBUSY: KHO state tree has been converted to FDT, + * - -EEXIST: another property of the same key exists. + */ +int kho_add_prop(struct kho_node *node, const char *key, const void *val, + u32 size) +{ + unsigned long key_hash; + int ret = 0; + struct kho_prop *prop, *p; + + key_hash = strhash(key); + prop = kmalloc(sizeof(*prop), GFP_KERNEL); + if (!prop) + return -ENOMEM; + + prop->key = key; + prop->val = val; + prop->size = size; + + down_read(&kho_out.tree_lock); + if (kho_out.fdt) { + ret = -EBUSY; + goto out; + } + + hash_for_each_possible(node->props, p, hlist, key_hash) { + if (!strcmp(p->key, key)) { + ret = (p->val == val && p->size == size) ? 1 : -EEXIST; + break; + } + } + + if (!ret) + hash_add(node->props, &prop->hlist, key_hash); + +out: + up_read(&kho_out.tree_lock); + + if (ret) + kfree(prop); + + return ret; +} +EXPORT_SYMBOL_GPL(kho_add_prop); + +/** + * kho_add_string_prop - add a string property to a node. + * + * See kho_add_prop() for details. + */ +int kho_add_string_prop(struct kho_node *node, const char *key, const char *val) +{ + return kho_add_prop(node, key, val, strlen(val) + 1); +} +EXPORT_SYMBOL_GPL(kho_add_string_prop); + +/** + * kho_remove_prop - remove a property from a node. + * @node: KHO node to remove the property from. + * @key: key of the property. + * @size: if non-NULL, the property size is stored in it on success. + * + * Callers must use their own locking if there are concurrent accesses to @node. + * + * Return: the pointer to the property value, or + * - -EBUSY: KHO state tree has been converted to FDT, + * - -ENOENT: no property with @key is found. + */ +void *kho_remove_prop(struct kho_node *node, const char *key, u32 *size) +{ + struct kho_prop *p, *prop = NULL; + unsigned long key_hash; + void *ret = ERR_PTR(-ENOENT); + + key_hash = strhash(key); + + down_read(&kho_out.tree_lock); + + if (kho_out.fdt) { + ret = ERR_PTR(-EBUSY); + goto out; + } + + hash_for_each_possible(node->props, p, hlist, key_hash) { + if (!strcmp(p->key, key)) { + prop = p; + break; + } + } + + if (prop) { + ret = (void *)prop->val; + if (size) + *size = prop->size; + hash_del(&prop->hlist); + kfree(prop); + } + +out: + up_read(&kho_out.tree_lock); + + return ret; +} +EXPORT_SYMBOL_GPL(kho_remove_prop); + +static int kho_out_update_debugfs_fdt(void) +{ + int err = 0; + + if (kho_out.fdt) { + kho_out.fdt_wrapper.data = kho_out.fdt; + kho_out.fdt_wrapper.size = fdt_totalsize(kho_out.fdt); + kho_out.fdt_file = debugfs_create_blob("fdt", 0400, kho_out.dir, + &kho_out.fdt_wrapper); + if (IS_ERR(kho_out.fdt_file)) + err = -ENOENT; + } else { + debugfs_remove(kho_out.fdt_file); + } + + return err; +} + +static int kho_unfreeze(void) +{ + int err; + void *fdt; + + down_write(&kho_out.tree_lock); + fdt = kho_out.fdt; + kho_out.fdt = NULL; + up_write(&kho_out.tree_lock); + + if (fdt) + kvfree(fdt); + + err = blocking_notifier_call_chain(&kho_out.chain_head, + KEXEC_KHO_UNFREEZE, NULL); + err = notifier_to_errno(err); + + return notifier_to_errno(err); +} + +static int kho_flatten_tree(void *fdt) +{ + int iter, err = 0; + struct kho_node *node, *sub_node; + struct list_head *ele; + struct kho_prop *prop; + LIST_HEAD(stack); + + kho_out.root.visited = false; + list_add(&kho_out.root.list, &stack); + + for (ele = stack.next; !list_is_head(ele, &stack); ele = stack.next) { + node = list_entry(ele, struct kho_node, list); + + if (node->visited) { + err = fdt_end_node(fdt); + if (err) + return err; + list_del_init(ele); + continue; + } + + err = fdt_begin_node(fdt, node->name); + if (err) + return err; + + hash_for_each(node->props, iter, prop, hlist) { + err = fdt_property(fdt, prop->key, prop->val, + prop->size); + if (err) + return err; + } + + hash_for_each(node->nodes, iter, sub_node, hlist) { + sub_node->visited = false; + list_add(&sub_node->list, &stack); + } + + node->visited = true; + } + + return 0; +} + +static int kho_convert_tree(void *buffer, int size) +{ + void *fdt = buffer; + int err = 0; + + err = fdt_create(fdt, size); + if (err) + goto out; + + err = fdt_finish_reservemap(fdt); + if (err) + goto out; + + err = kho_flatten_tree(fdt); + if (err) + goto out; + + err = fdt_finish(fdt); + if (err) + goto out; + + err = fdt_check_header(fdt); + if (err) + goto out; + +out: + if (err) { + pr_err("failed to flatten state tree: %d\n", err); + return -EINVAL; + } + return 0; +} + +static int kho_finalize(void) +{ + int err = 0; + void *fdt; + + fdt = kvmalloc(kho_out.fdt_max, GFP_KERNEL); + if (!fdt) + return -ENOMEM; + + err = blocking_notifier_call_chain(&kho_out.chain_head, + KEXEC_KHO_FINALIZE, NULL); + err = notifier_to_errno(err); + if (err) + goto unfreeze; + + down_write(&kho_out.tree_lock); + kho_out.fdt = fdt; + up_write(&kho_out.tree_lock); + + err = kho_convert_tree(fdt, kho_out.fdt_max); + +unfreeze: + if (err) { + int abort_err; + + pr_err("Failed to convert KHO state tree: %d\n", err); + + abort_err = kho_unfreeze(); + if (abort_err) + pr_err("Failed to abort KHO state tree: %d\n", + abort_err); + } + + return err; +} + +/* Handling for debug/kho/out */ +static int kho_out_finalize_get(void *data, u64 *val) +{ + *val = !!kho_out.fdt; + + return 0; +} + +static int kho_out_finalize_set(void *data, u64 _val) +{ + int ret = 0; + bool val = !!_val; + + if (!kexec_trylock()) + return -EBUSY; + + if (val == !!kho_out.fdt) { + if (kho_out.fdt) + ret = -EEXIST; + else + ret = -ENOENT; + goto unlock; + } + + if (val) + ret = kho_finalize(); + else + ret = kho_unfreeze(); + + if (ret) + goto unlock; + + ret = kho_out_update_debugfs_fdt(); + +unlock: + kexec_unlock(); + return ret; +} + +DEFINE_DEBUGFS_ATTRIBUTE(fops_kho_out_finalize, kho_out_finalize_get, + kho_out_finalize_set, "%llu\n"); + +static int kho_out_fdt_max_get(void *data, u64 *val) +{ + *val = kho_out.fdt_max; + + return 0; +} + +static int kho_out_fdt_max_set(void *data, u64 val) +{ + int ret = 0; + + if (!kexec_trylock()) { + ret = -EBUSY; + goto unlock; + } + + /* FDT already exists, it's too late to change fdt_max */ + if (kho_out.fdt) { + ret = -EBUSY; + goto unlock; + } + + kho_out.fdt_max = val; + +unlock: + kexec_unlock(); + return ret; +} + +DEFINE_DEBUGFS_ATTRIBUTE(fops_kho_out_fdt_max, kho_out_fdt_max_get, + kho_out_fdt_max_set, "%llu\n"); + +static int scratch_phys_show(struct seq_file *m, void *v) +{ + for (int i = 0; i < kho_scratch_cnt; i++) + seq_printf(m, "0x%llx\n", kho_scratch[i].addr); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(scratch_phys); + +static int scratch_len_show(struct seq_file *m, void *v) +{ + for (int i = 0; i < kho_scratch_cnt; i++) + seq_printf(m, "0x%llx\n", kho_scratch[i].size); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(scratch_len); + +static __init int kho_out_debugfs_init(void) +{ + struct dentry *dir, *f; + + dir = debugfs_create_dir("out", debugfs_root); + if (IS_ERR(dir)) + return -ENOMEM; + + f = debugfs_create_file("scratch_phys", 0400, dir, NULL, + &scratch_phys_fops); + if (IS_ERR(f)) + goto err_rmdir; + + f = debugfs_create_file("scratch_len", 0400, dir, NULL, + &scratch_len_fops); + if (IS_ERR(f)) + goto err_rmdir; + + f = debugfs_create_file("fdt_max", 0600, dir, NULL, + &fops_kho_out_fdt_max); + if (IS_ERR(f)) + goto err_rmdir; + + f = debugfs_create_file("finalize", 0600, dir, NULL, + &fops_kho_out_finalize); + if (IS_ERR(f)) + goto err_rmdir; + + kho_out.dir = dir; + return 0; + +err_rmdir: + debugfs_remove_recursive(dir); + return -ENOENT; +} + +static __init int kho_init(void) +{ + int err; + + if (!kho_enable) + return 0; + + kho_out.root.name = ""; + err = kho_add_string_prop(&kho_out.root, "compatible", "kho-v1"); + if (err) + goto err_free_scratch; + + debugfs_root = debugfs_create_dir("kho", NULL); + if (IS_ERR(debugfs_root)) { + err = -ENOENT; + goto err_free_scratch; + } + + err = kho_out_debugfs_init(); + if (err) + goto err_free_scratch; + + for (int i = 0; i < kho_scratch_cnt; i++) { + unsigned long base_pfn = PHYS_PFN(kho_scratch[i].addr); + unsigned long count = kho_scratch[i].size >> PAGE_SHIFT; + unsigned long pfn; + + for (pfn = base_pfn; pfn < base_pfn + count; + pfn += pageblock_nr_pages) + init_cma_reserved_pageblock(pfn_to_page(pfn)); + } + + return 0; + +err_free_scratch: + for (int i = 0; i < kho_scratch_cnt; i++) { + void *start = __va(kho_scratch[i].addr); + void *end = start + kho_scratch[i].size; + + free_reserved_area(start, end, -1, ""); + } + kho_enable = false; + return err; +} +late_initcall(kho_init); + +/* + * The scratch areas are scaled by default as percent of memory allocated from + * memblock. A user can override the scale with command line parameter: + * + * kho_scratch=N% + * + * It is also possible to explicitly define size for a lowmem, a global and + * per-node scratch areas: + * + * kho_scratch=l[KMG],n[KMG],m[KMG] + * + * The explicit size definition takes precedence over scale definition. + */ +static unsigned int scratch_scale __initdata = 200; +static phys_addr_t scratch_size_global __initdata; +static phys_addr_t scratch_size_pernode __initdata; +static phys_addr_t scratch_size_lowmem __initdata; + +static int __init kho_parse_scratch_size(char *p) +{ + unsigned long size, size_pernode, size_global; + char *endptr, *oldp = p; + + if (!p) + return -EINVAL; + + size = simple_strtoul(p, &endptr, 0); + if (*endptr == '%') { + scratch_scale = size; + pr_notice("scratch scale is %d percent\n", scratch_scale); + } else { + size = memparse(p, &p); + if (!size || p == oldp) + return -EINVAL; + + if (*p != ',') + return -EINVAL; + + oldp = p; + size_global = memparse(p + 1, &p); + if (!size_global || p == oldp) + return -EINVAL; + + if (*p != ',') + return -EINVAL; + + size_pernode = memparse(p + 1, &p); + if (!size_pernode) + return -EINVAL; + + scratch_size_lowmem = size; + scratch_size_global = size_global; + scratch_size_pernode = size_pernode; + scratch_scale = 0; + + pr_notice("scratch areas: lowmem: %lluMB global: %lluMB pernode: %lldMB\n", + (u64)(scratch_size_lowmem >> 20), + (u64)(scratch_size_global >> 20), + (u64)(scratch_size_pernode >> 20)); + } + + return 0; +} +early_param("kho_scratch", kho_parse_scratch_size); + +static void __init scratch_size_update(void) +{ + phys_addr_t size; + + if (!scratch_scale) + return; + + size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT, + NUMA_NO_NODE); + size = size * scratch_scale / 100; + scratch_size_lowmem = round_up(size, CMA_MIN_ALIGNMENT_BYTES); + + size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE, + NUMA_NO_NODE); + size = size * scratch_scale / 100 - scratch_size_lowmem; + scratch_size_global = round_up(size, CMA_MIN_ALIGNMENT_BYTES); +} + +static phys_addr_t __init scratch_size_node(int nid) +{ + phys_addr_t size; + + if (scratch_scale) { + size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE, + nid); + size = size * scratch_scale / 100; + } else { + size = scratch_size_pernode; + } + + return round_up(size, CMA_MIN_ALIGNMENT_BYTES); +} + +/** + * kho_reserve_scratch - Reserve a contiguous chunk of memory for kexec + * + * With KHO we can preserve arbitrary pages in the system. To ensure we still + * have a large contiguous region of memory when we search the physical address + * space for target memory, let's make sure we always have a large CMA region + * active. This CMA region will only be used for movable pages which are not a + * problem for us during KHO because we can just move them somewhere else. + */ +static void __init kho_reserve_scratch(void) +{ + phys_addr_t addr, size; + int nid, i = 0; + + if (!kho_enable) + return; + + scratch_size_update(); + + /* FIXME: deal with node hot-plug/remove */ + kho_scratch_cnt = num_online_nodes() + 2; + size = kho_scratch_cnt * sizeof(*kho_scratch); + kho_scratch = memblock_alloc(size, PAGE_SIZE); + if (!kho_scratch) + goto err_disable_kho; + + /* + * reserve scratch area in low memory for lowmem allocations in the + * next kernel + */ + size = scratch_size_lowmem; + addr = memblock_phys_alloc_range(size, CMA_MIN_ALIGNMENT_BYTES, 0, + ARCH_LOW_ADDRESS_LIMIT); + if (!addr) + goto err_free_scratch_desc; + + kho_scratch[i].addr = addr; + kho_scratch[i].size = size; + i++; + + /* reserve large contiguous area for allocations without nid */ + size = scratch_size_global; + addr = memblock_phys_alloc(size, CMA_MIN_ALIGNMENT_BYTES); + if (!addr) + goto err_free_scratch_areas; + + kho_scratch[i].addr = addr; + kho_scratch[i].size = size; + i++; + + for_each_online_node(nid) { + size = scratch_size_node(nid); + addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, + 0, MEMBLOCK_ALLOC_ACCESSIBLE, + nid, true); + if (!addr) + goto err_free_scratch_areas; + + kho_scratch[i].addr = addr; + kho_scratch[i].size = size; + i++; + } + + return; + +err_free_scratch_areas: + for (i--; i >= 0; i--) + memblock_phys_free(kho_scratch[i].addr, kho_scratch[i].size); +err_free_scratch_desc: + memblock_free(kho_scratch, kho_scratch_cnt * sizeof(*kho_scratch)); +err_disable_kho: + kho_enable = false; +} + +void __init kho_memory_init(void) +{ + kho_reserve_scratch(); +} diff --git a/mm/mm_init.c b/mm/mm_init.c index 04441c258b05..757659b7a26b 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -2661,6 +2662,13 @@ void __init mm_core_init(void) report_meminit(); kmsan_init_shadow(); stack_depot_early_init(); + + /* + * KHO memory setup must happen while memblock is still active, but + * as close as possible to buddy initialization + */ + kho_memory_init(); + mem_init(); kmem_cache_init(); /* From patchwork Thu Mar 20 01:55:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C51AC35FFC for ; Thu, 20 Mar 2025 01:56:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CBCB28000A; Wed, 19 Mar 2025 21:56:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17B4A280001; Wed, 19 Mar 2025 21:56:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E99CE28000A; Wed, 19 Mar 2025 21:56:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BF96E280001 for ; Wed, 19 Mar 2025 21:56:15 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2979F8069E for ; Thu, 20 Mar 2025 01:56:16 +0000 (UTC) X-FDA: 83240264352.29.0AF0064 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf07.hostedemail.com (Postfix) with ESMTP id 4860D40006 for ; Thu, 20 Mar 2025 01:56:14 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=V0EgG2Qw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3vXXbZwoKCFw6B4HASO4HFAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3vXXbZwoKCFw6B4HASO4HFAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--changyuanl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uzt28YGX83ujtBiUFsJhvG1AtvyLncXc8B9iuS5ri6U=; b=dGwMX9ZrwAnPxYRlempPWEHZJfjxr33sKuFLQVKDMqYeuVZ30lwOEtPRUp4lVJO3ynjwG6 obIHQ4CKRnCoAVoDfTJkvvqgl7sskWkeo5ZLzA9TvPgufqLcGOt1yH80FuwBr7cTFkujtb 9dx+oVkZAU5vGgO3MzFKDkudgvaUDjE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435774; a=rsa-sha256; cv=none; b=RRRzDtB3UIUuEoinQPdyn6W58z9jHuQbZqBY70kYA0bscrer3VABqGz5fTrp4drJ4ld9pT w1XFhA2u3CEvIFvS6xqIa2HB4N+AAe0eRFTK6An3fIAC8f0kAce6pw0kclnFAO7278bTVk XA2rH7RfcWtAGynOjcdL8+2ivVPIVz8= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=V0EgG2Qw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3vXXbZwoKCFw6B4HASO4HFAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3vXXbZwoKCFw6B4HASO4HFAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--changyuanl.bounces.google.com Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff6167e9ccso701579a91.1 for ; Wed, 19 Mar 2025 18:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435773; x=1743040573; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uzt28YGX83ujtBiUFsJhvG1AtvyLncXc8B9iuS5ri6U=; b=V0EgG2Qw812+goG9xGSOD8Kk0877s2Krl0MFH5WfBSW7DHTJlhwHKr+zfht8K9bgop gTdj10Y4+oIx0eexH1madYiwnRe1zRTFJaOjdXE33j++JSFiMPS1aC/u2HIP6hZukei3 kYLAcQSmUtgFXMgePdCQ8Od9bmL9pRGzRSCpnRZ1PmVecloDH6cWPB1NJpiy4toRB3bw tEKRQOCEEq6rpX79R37t4nzXwhwpj/um5iUnQoglejvPuxv1IapEttumjQ09XJ1Xa+mi iKDuMy0qVVd2JqMAc++Ggeb0iQDBTdAcmQIv+71hI7Peh/ePQyhclxfbxpph0KKFzJP0 tuvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435773; x=1743040573; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uzt28YGX83ujtBiUFsJhvG1AtvyLncXc8B9iuS5ri6U=; b=rL9txkWSEU477zbyKf84VpF3tKqpXAb2u8mayRqiAf/OtrYmjndY8rtddR2cxUSm6C yRpnD3XhDbWAWD973OKD2M2B3EzaIjX617FmhFSwrZ94IPnPQ4OdtTJqEjZb84sdOUrr UobsF1pfaigx/9KwRWV2xdyfKasXKy6XiUJzf04xPcp3GirU98eNgaMQU8nnYO5SK3fm Ba4zxtQnEZQ3i+QIeVuu2x+d+xWGzVpzjROuuk3ZLSxdteFm2SZkcOG14EA9pwBPjOR0 8qDC0SxnZiOgAMfg7uL6KujWJzaGFV/Ig4gT1REB9tZj0/XBXgPGcAB4Fhxmbcj0dhSD WAyg== X-Forwarded-Encrypted: i=1; AJvYcCWY62PGsdt6FMsrHu+djaj4hZ/oDCQ17Oex/i3jSbgd2g3Paab7AXBGNLdb5b7qeZy2ZoGcTRggNQ==@kvack.org X-Gm-Message-State: AOJu0YwNaup4GDkthaREPVC37OrGEb27vg7nZ+Y0vxK1OsPE7JQiUm9N 1LhhlWFf1CQ3U7u5flJoxBAGl3NwZlpN0+J62nG1cQCsc0mALZs1mYTl5OhvNW4/nIeZuKsHh3M l55iJJ/88npbd6Vb4pQ== X-Google-Smtp-Source: AGHT+IETZEnQ1fnebG+EPXz8EIs4aUlnfPIGCh4puq8YWTGEpiNjSNK1+MOm9oAvMrmsXQRIpafhM24FzfM8r9f8 X-Received: from pgct9.prod.google.com ([2002:a05:6a02:5289:b0:aef:faff:6861]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:c901:b0:1f5:7f45:7f8a with SMTP id adf61e73a8af0-1fd13ed2b9fmr2759978637.38.1742435773077; Wed, 19 Mar 2025 18:56:13 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:43 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-9-changyuanl@google.com> Subject: [PATCH v5 08/16] kexec: add KHO parsing support From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4860D40006 X-Stat-Signature: 4fbta839cm66pfbedqu18jgqnakzrsbk X-HE-Tag: 1742435774-706931 X-HE-Meta: U2FsdGVkX18FQhjvy1Y47HF1+q38fq61fM1Aaf/i5lUMYjx3qFdt9C/fNem/82HT+pHnEO9URlnmjCQVTZ/anHQfa6vtozBPInmk54e6s41A6clZLbXGSkrwXruJKIywpSinLq6VXVjhV2M8gprFiA+ineRC9F8vVNnoTAjnOsH0C+rSYBxDtADHBtAXaG3WVLUOG0ZF53x4BhfKdTOICC0R6nDxLQD47IuUFVwfLY9WCtQwxQt9sUXxmF6+BFM57GsIo4DC5uV6kKxFTd/IhN3qiGqK0yA2hixGU58+62TL56oUN8cFXDpaJvFGKgOA2Kh0XIsHrcPOXh95X53zAX9c/yXDFXadjnSQ+WKzlCR8MRdyOpNF2vwTbi+k6C8CF56Tpo3tJ4E5Jic14IsVy/FszW+pFsu7ZJbDGwiiVaSmkeN1dQYbapoZoGr27RP7/HwfsGViSRExh7nN+jhSpdRCsae3Mb74fN5xFHTy1IotQS/2bhua6BRnECo0ow+BwR7K50h669uxOjcj2st8X+dCL1icwkE13oX+t9E69FRoURHfP3Q3kMrwIu2nkv7iOoj6Lej+koqyEJONqpFvHnLljxn8p5HyPYxweSIQoSR14NqU+4tOLaje0DVI53m249KS/jFKHLE+ASYQEZsa+rl9uvmo2Wv9cnYz/JCw24kT75s/tgH5lX9VMso8VMj2wcW6d6n/oC85es0bgZPCpU4ZCcwDVelkKYA5nUgqy5/gCNaJC5Umzg1Gzc8vGtkuV3PU23KhXHXRg0ryxj9JO6n97gF6HqkWq53tptiqPAzEy237PkyusX+uWdXbS/YuaG9PsDQYYQR2GmV7MhQNYCN5//ygMig2/UPp8BuYgLhZAajZHUxGZikX0mUxiNiAMoQcSot1z3cHUQnXFKvNlYbWskrvINDfJqH2XZ9v0UihGj9TKFX8YzJmyxXozizFhZHfLhfckOIW2gBj6fB UWJGR6JS AXwPx8CCR4rbyP/qvi6Cmbw/Ipm6T/+DFmTqfs/NSG8eDwQ2fKacyJLYGo4aYWwlRQkIjOBBmFDpRPGjfrWQRxzz03ZISpUOpFXFW2jI4rLVaziRbbCv7DfXkabyezDwwTvzixV8G1gp7RpXkNl0RDGQSm7abSB/l2B9L+/Z7GmIUY535NOOscZZ7/G0bVvFVuJWMXXLrMqAs7slkdr/CYoQBMouolNq3pZXI5Qfgm/yxQKA4adYqKz3MtyavwTmepP4LVC1ZdLN0ebJcCz9LANeonDN7wfIjotVN7S7d261HWhfe/SsL6O5zesy8imy8BqH7hIOH2jV9SuMh8nipDPf0FDSyqanJtwI3H6UsJkU5znpESPXTxD3HhIy3IM05P7szrt31IG4BKy+eTaHte3E8vAjaGuaHGznOvdRcnrn3m6fEfnAy2FnBf3hCJD/YPPeJjj7cUwsLjG2gb7Y/DED2s5CFZOLaHbngqurdQnnzBeT6ZKtZG59mOrBhzWRLM/2Q7QjdLK+vYL33+9BYpjn6lO2rVqapWP3BewuN0+qsVIWEid43d4+VrEfBDvZb+qfoz+IM9SISdYV17yWUU5tQRZOhm5XZjeAP0DSOJjU0aPJHegh3/FmKBp+lzkDNGqS/kNhUQDSKOHSNQHQt0Y9vvdAuGz9WG2rmep1lw0LfaKhxcNetCS1Po91b9iCQUf0R X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf When we have a KHO kexec, we get an FDT blob and scratch region to populate the state of the system. Provide helper functions that allow architecture code to easily handle memory reservations based on them and give device drivers visibility into the KHO FDT and memory reservations so they can recover their own state. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec_handover.h | 48 ++++++ kernel/kexec_handover.c | 302 ++++++++++++++++++++++++++++++++- mm/memblock.c | 1 + 3 files changed, 350 insertions(+), 1 deletion(-) diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h index 9cd9ad31e2d1..c665ff6cd728 100644 --- a/include/linux/kexec_handover.h +++ b/include/linux/kexec_handover.h @@ -35,6 +35,10 @@ struct kho_node { bool visited; }; +struct kho_in_node { + int offset; +}; + #ifdef CONFIG_KEXEC_HANDOVER bool kho_is_enabled(void); void kho_init_node(struct kho_node *node); @@ -51,6 +55,19 @@ int register_kho_notifier(struct notifier_block *nb); int unregister_kho_notifier(struct notifier_block *nb); void kho_memory_init(void); + +void kho_populate(phys_addr_t handover_fdt_phys, phys_addr_t scratch_phys, + u64 scratch_len); + +int kho_get_node(const struct kho_in_node *parent, const char *name, + struct kho_in_node *child); +int kho_get_nodes(const struct kho_in_node *parent, + int (*func)(const char *, const struct kho_in_node *, void *), + void *data); +const void *kho_get_prop(const struct kho_in_node *node, const char *key, + u32 *size); +int kho_node_check_compatible(const struct kho_in_node *node, + const char *compatible); #else static inline bool kho_is_enabled(void) { @@ -104,6 +121,37 @@ static inline int unregister_kho_notifier(struct notifier_block *nb) static inline void kho_memory_init(void) { } + +static inline void kho_populate(phys_addr_t handover_fdt_phys, + phys_addr_t scratch_phys, u64 scratch_len) +{ +} + +static inline int kho_get_node(const struct kho_in_node *parent, + const char *name, struct kho_in_node *child) +{ + return -EOPNOTSUPP; +} + +static inline int kho_get_nodes(const struct kho_in_node *parent, + int (*func)(const char *, + const struct kho_in_node *, void *), + void *data) +{ + return -EOPNOTSUPP; +} + +static inline const void *kho_get_prop(const struct kho_in_node *node, + const char *key, u32 *size) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline int kho_node_check_compatible(const struct kho_in_node *node, + const char *compatible) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_KEXEC_HANDOVER */ #endif /* LINUX_KEXEC_HANDOVER_H */ diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index df0d9debbb64..6ebad2f023f9 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -73,6 +73,20 @@ static struct kho_out kho_out = { .fdt_max = 10 * SZ_1M, }; +struct kho_in { + struct debugfs_blob_wrapper fdt_wrapper; + struct dentry *dir; + phys_addr_t kho_scratch_phys; + phys_addr_t fdt_phys; +}; + +static struct kho_in kho_in; + +static const void *kho_get_fdt(void) +{ + return kho_in.fdt_phys ? phys_to_virt(kho_in.fdt_phys) : NULL; +} + int register_kho_notifier(struct notifier_block *nb) { return blocking_notifier_chain_register(&kho_out.chain_head, nb); @@ -85,6 +99,144 @@ int unregister_kho_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(unregister_kho_notifier); +/** + * kho_get_node - retrieve a node saved in KHO FDT. + * @parent: the parent node to look up for. + * @name: the name of the node to look for. + * @child: if a node named @name is found under @parent, it is stored in @child. + * + * If @parent is NULL, this function looks up for @name under KHO root node. + * + * Return: 0 on success, and @child is populated, error code on failure. + */ +int kho_get_node(const struct kho_in_node *parent, const char *name, + struct kho_in_node *child) +{ + int parent_offset = 0; + int offset = 0; + const void *fdt = kho_get_fdt(); + + if (!fdt) + return -ENOENT; + + if (!child) + return -EINVAL; + + if (parent) + parent_offset = parent->offset; + + offset = fdt_subnode_offset(fdt, parent_offset, name); + if (offset < 0) + return -ENOENT; + + child->offset = offset; + return 0; +} +EXPORT_SYMBOL_GPL(kho_get_node); + +/** + * kho_get_nodes - iterate over all direct child nodes. + * @parent: the parent node to look for child nodes. + * @func: a function pointer to be called on each child node. + * @data: auxiliary data to be passed to @func. + * + * For every direct child node of @parent, @func is called with the child node + * name, the child node (a struct kho_in_node *), and @data. + * + * If @parent is NULL, this function iterates over the child nodes of the KHO + * root node. + * + * Return: 0 on success, error code on failure. + */ +int kho_get_nodes(const struct kho_in_node *parent, + int (*func)(const char *, const struct kho_in_node *, void *), + void *data) +{ + int parent_offset = 0; + struct kho_in_node child; + const char *name; + int ret = 0; + const void *fdt = kho_get_fdt(); + + if (!fdt) + return -ENOENT; + + if (parent) + parent_offset = parent->offset; + + fdt_for_each_subnode(child.offset, fdt, parent_offset) { + if (child.offset < 0) + return -EINVAL; + + name = fdt_get_name(fdt, child.offset, NULL); + + if (!name) + return -EINVAL; + + ret = func(name, &child, data); + + if (ret < 0) + break; + } + + return ret; +} +EXPORT_SYMBOL_GPL(kho_get_nodes); + +/** + * kho_get_prop - retrieve the property data stored in the KHO tree. + * @node: the node to look up for. + * @key: the key of the property. + * @size: a pointer to store the size of the data in bytes. + * + * Return: pointer to the data, and data size is stored in @size, or NULL on + * failure. + */ +const void *kho_get_prop(const struct kho_in_node *node, const char *key, + u32 *size) +{ + int offset = 0; + u32 s; + const void *fdt = kho_get_fdt(); + + if (!fdt) + return NULL; + + if (node) + offset = node->offset; + + if (!size) + size = &s; + + return fdt_getprop(fdt, offset, key, size); +} +EXPORT_SYMBOL_GPL(kho_get_prop); + +/** + * kho_node_check_compatible - check a node's compatible property. + * @node: the node to check. + * @compatible: the compatible stirng. + * + * Wrapper of fdt_node_check_compatible(). + * + * Return: 0 if @compatible is in the node's "compatible" list, or + * error code on failure. + */ +int kho_node_check_compatible(const struct kho_in_node *node, + const char *compatible) +{ + int result = 0; + const void *fdt = kho_get_fdt(); + + if (!fdt) + return -ENOENT; + + result = fdt_node_check_compatible(fdt, node->offset, compatible); + + return result ? -EINVAL : 0; +} +EXPORT_SYMBOL_GPL(kho_node_check_compatible); + /* Helper functions for KHO state tree */ struct kho_prop { @@ -605,6 +757,32 @@ static int scratch_len_show(struct seq_file *m, void *v) } DEFINE_SHOW_ATTRIBUTE(scratch_len); +/* Handling for debugfs/kho/in */ +static __init int kho_in_debugfs_init(const void *fdt) +{ + struct dentry *file; + int err; + + kho_in.dir = debugfs_create_dir("in", debugfs_root); + if (IS_ERR(kho_in.dir)) + return PTR_ERR(kho_in.dir); + + kho_in.fdt_wrapper.size = fdt_totalsize(fdt); + kho_in.fdt_wrapper.data = (void *)fdt; + file = debugfs_create_blob("fdt", 0400, kho_in.dir, + &kho_in.fdt_wrapper); + if (IS_ERR(file)) { + err = PTR_ERR(file); + goto err_rmdir; + } + + return 0; + +err_rmdir: + debugfs_remove(kho_in.dir); + return err; +} + static __init int kho_out_debugfs_init(void) { struct dentry *dir, *f; @@ -644,6 +822,7 @@ static __init int kho_out_debugfs_init(void) static __init int kho_init(void) { int err; + const void *fdt = kho_get_fdt(); if (!kho_enable) return 0; @@ -663,6 +842,21 @@ static __init int kho_init(void) if (err) goto err_free_scratch; + if (fdt) { + err = kho_in_debugfs_init(fdt); + /* + * Failure to create /sys/kernel/debug/kho/in does not prevent + * reviving state from KHO and setting up KHO for the next + * kexec. + */ + if (err) + pr_err("failed exposing handover FDT in debugfs\n"); + + kho_scratch = __va(kho_in.kho_scratch_phys); + + return 0; + } + for (int i = 0; i < kho_scratch_cnt; i++) { unsigned long base_pfn = PHYS_PFN(kho_scratch[i].addr); unsigned long count = kho_scratch[i].size >> PAGE_SHIFT; @@ -859,7 +1053,113 @@ static void __init kho_reserve_scratch(void) kho_enable = false; } +static void __init kho_release_scratch(void) +{ + phys_addr_t start, end; + u64 i; + + memmap_init_kho_scratch_pages(); + + /* + * Mark scratch mem as CMA before we return it. That way we + * ensure that no kernel allocations happen on it. That means + * we can reuse it as scratch memory again later. + */ + __for_each_mem_range(i, &memblock.memory, NULL, NUMA_NO_NODE, + MEMBLOCK_KHO_SCRATCH, &start, &end, NULL) { + ulong start_pfn = pageblock_start_pfn(PFN_DOWN(start)); + ulong end_pfn = pageblock_align(PFN_UP(end)); + ulong pfn; + + for (pfn = start_pfn; pfn < end_pfn; pfn += pageblock_nr_pages) + set_pageblock_migratetype(pfn_to_page(pfn), + MIGRATE_CMA); + } +} + void __init kho_memory_init(void) { - kho_reserve_scratch(); + if (!kho_get_fdt()) + kho_reserve_scratch(); + else + kho_release_scratch(); +} + +void __init kho_populate(phys_addr_t handover_fdt_phys, + phys_addr_t scratch_phys, u64 scratch_len) +{ + void *handover_fdt; + struct kho_scratch *scratch; + u32 fdt_size = 0; + + /* Determine the real size of the FDT */ + handover_fdt = + early_memremap(handover_fdt_phys, sizeof(struct fdt_header)); + if (!handover_fdt) { + pr_warn("setup: failed to memremap kexec FDT (0x%llx)\n", + handover_fdt_phys); + return; + } + + if (fdt_check_header(handover_fdt)) { + pr_warn("setup: kexec handover FDT is invalid (0x%llx)\n", + handover_fdt_phys); + early_memunmap(handover_fdt, sizeof(struct fdt_header)); + return; + } + + fdt_size = fdt_totalsize(handover_fdt); + kho_in.fdt_phys = handover_fdt_phys; + + early_memunmap(handover_fdt, sizeof(struct fdt_header)); + + /* Reserve the DT so we can still access it in late boot */ + memblock_reserve(handover_fdt_phys, fdt_size); + + kho_in.kho_scratch_phys = scratch_phys; + kho_scratch_cnt = scratch_len / sizeof(*kho_scratch); + scratch = early_memremap(scratch_phys, scratch_len); + if (!scratch) { + pr_warn("setup: failed to memremap kexec scratch (0x%llx)\n", + scratch_phys); + return; + } + + /* + * We pass a safe contiguous blocks of memory to use for early boot + * purporses from the previous kernel so that we can resize the + * memblock array as needed. + */ + for (int i = 0; i < kho_scratch_cnt; i++) { + struct kho_scratch *area = &scratch[i]; + u64 size = area->size; + + memblock_add(area->addr, size); + + if (WARN_ON(memblock_mark_kho_scratch(area->addr, size))) { + pr_err("Kexec failed to mark the scratch region. Disabling KHO revival."); + kho_in.fdt_phys = 0; + scratch = NULL; + break; + } + pr_debug("Marked 0x%pa+0x%pa as scratch", &area->addr, &size); + } + + early_memunmap(scratch, scratch_len); + + if (!scratch) + return; + + memblock_reserve(scratch_phys, scratch_len); + + /* + * Now that we have a viable region of scratch memory, let's tell + * the memblocks allocator to only use that for any allocations. + * That way we ensure that nothing scribbles over in use data while + * we initialize the page tables which we will need to ingest all + * memory reservations from the previous kernel. + */ + memblock_set_kho_scratch_only(); + + pr_info("setup: Found kexec handover data. Will skip init for some devices\n"); } diff --git a/mm/memblock.c b/mm/memblock.c index d5d406a5160a..d28abf3def1c 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -2374,6 +2374,7 @@ void __init memblock_free_all(void) free_unused_memmap(); reset_all_zones_managed_pages(); + memblock_clear_kho_scratch_only(); pages = free_low_memory_core_early(); totalram_pages_add(pages); } From patchwork Thu Mar 20 01:55:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80B3AC35FFC for ; Thu, 20 Mar 2025 01:56:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 938B928000B; Wed, 19 Mar 2025 21:56:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8BFEA280001; Wed, 19 Mar 2025 21:56:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 715B028000B; Wed, 19 Mar 2025 21:56:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 483D8280001 for ; Wed, 19 Mar 2025 21:56:17 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8F824805CE for ; Thu, 20 Mar 2025 01:56:17 +0000 (UTC) X-FDA: 83240264394.30.D75EB28 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf15.hostedemail.com (Postfix) with ESMTP id B0708A0005 for ; Thu, 20 Mar 2025 01:56:15 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Luv8jv5s; spf=pass (imf15.hostedemail.com: domain of 3vnXbZwoKCF07C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3vnXbZwoKCF07C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435775; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uSm1C3hzri2Wj9pRnB1afKKuALL7+tUjT9kdNp30IT0=; b=QBJYw2LCx4rHnQkvrYK4PTqfkw1hiCNzyvUQ9asoJbKOaaoUE+u77vYq4tOKK8zEGRQSsm nAlX+LhHi4PdhO6z7EoGg5wZPIFQ5aHVfXOnWv5Gxzc4Laxjm3sLwTDp1M201hR4sXv0ES MAFfPIQikbF6KzGvFZ182uyIJq7vbwc= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Luv8jv5s; spf=pass (imf15.hostedemail.com: domain of 3vnXbZwoKCF07C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3vnXbZwoKCF07C5IBTP5IGBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435775; a=rsa-sha256; cv=none; b=ogLQfhmFgyUTOi0gtFetrMCbDIw3OqfSfRz6mwcO1mH9IzivInlbsk9MsvbA0qoOvzq73K ap4N/ThPR1otB+uZzvBvmWnWc57NUzPUgMucQR+J8C7Z7edNmPu8RRExdAux59hghMyubH o0uDAqwOmHNge6iYCP84iG/oYN2FqKg= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff78dd28ecso644046a91.1 for ; Wed, 19 Mar 2025 18:56:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435774; x=1743040574; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uSm1C3hzri2Wj9pRnB1afKKuALL7+tUjT9kdNp30IT0=; b=Luv8jv5sCfGKtkuHE9Hni/BO68qoMNdVj+8GRvGcxhAnyCiDoiv5Sl0d7dFM6KPeKy Uo3LIjQ8fJt3F1czHpLA2+buhQ57EOk5iOomRoq8uiUzMwcG0npCmMCoaDG9E+2GHOWG G5ha0D3kxFURFl66j66dd26W2b/us/HGHiagESBnAC3WaaI6VNS+psZMPvTpKx/XkyWa F8yLXs9whXYODQcMiac7oSzalDqKCRMRHtjk5YiKs+8rW0snuHwKoPRSfM7Iiaj2WAuw UpQjvxTDCB8LHngW27wi0rjAY1AIHYUy5tfIrkaoaY++Xz0cIRMuQ52GNzo2RB78iY62 q3Cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435774; x=1743040574; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uSm1C3hzri2Wj9pRnB1afKKuALL7+tUjT9kdNp30IT0=; b=jC9H/rQmoOg70KmQ8jTieX87Cw8Hc6dP6mvSm8ZuHTYBKZuXNEXod6HbXaO67QdgF/ mV9u9I3Mz+Q7Nvhpyz0B6RfI2UhV6YFkJNfw5jrl7AewHPAbpO5BJsjbTq2dj9pEQOi9 gtJToISI4G+hc59IMIXg0SEZcxH8rRcPPuR+KwYTMUhDxA9VX4XvOgoJRlmyebOC+wZ4 HNpieFX3oIc8O9VZusweGcF0qI/wko/5OdNuRWqWI61sKCDY08oxqAdMWwSDnBqHoPcS iv8qVnNqxrXTHIAluNBdpUv7qjLiq225Nb5wu34EyMDrkusx3+ry/WdHr4KLgoqS92BS Awag== X-Forwarded-Encrypted: i=1; AJvYcCUNVHKblwKDOtMlqJ2HeZDNI7XO0rIhQGPWQctFAocF7MCC62ocvcsC12oCmpI6inVe0auFHaUFEw==@kvack.org X-Gm-Message-State: AOJu0YxCxhg61iAAxnANDnvRJSY9XhVUdl/NQbz+Uk7+I5cxc9OIeIsV 3Xwa/sQX9cP8zr3B5byoR3fDT+V+J2JmjRa+ftopL6ouY0OpRAZyulcZIRMhyXOzDM9DQvryvmW bWUo9T/PENd2MNtygug== X-Google-Smtp-Source: AGHT+IGDFxZ224ZfpKuOQMpb0L5eyzWuzuuNb6JbsQOka9qAXNPHwzXyZCt1rphKnSoqunSh0fOsVTw3awCrNzrI X-Received: from pfbay13.prod.google.com ([2002:a05:6a00:300d:b0:730:7e2d:df69]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:12d4:b0:1f5:837b:1868 with SMTP id adf61e73a8af0-1fbecd48a69mr9047217637.29.1742435774464; Wed, 19 Mar 2025 18:56:14 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:44 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-10-changyuanl@google.com> Subject: [PATCH v5 09/16] kexec: enable KHO support for memory preservation From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Jason Gunthorpe , Changyuan Lyu X-Stat-Signature: asfxb3u5n3ewcdu8qea8cwuh4oqzywtm X-Rspamd-Queue-Id: B0708A0005 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1742435775-340894 X-HE-Meta: U2FsdGVkX18r1/u1R8fhYTXuFsHW4kZePopHt0L2SWWDsVwylx5ParEajRVx1nh2opBrn9OIqd3xMq59BdsRNQ837nhlkp2WAQKBHikX1soo7dFbzCVvbs8+3CkkuRhPppbYxz9/j3SE0RLXvB5InMy6U/CRQYOT9w4sNkKYT2znUxw6T7g/IoJGwvP/dVSJyEvzTCTUfo6zlbXJQNVCjqKiWo5VpBQ4z66d95jYiU1TPcUrgN0R9HVdfULRJv9sJD88ArY+Txukycko56ynh8mk285DzAGtdo8kzun2BnkcHHpE7xpNs2sfOh7jhaVHM5cjk3VM1TJUVL/bzWYWCrQLv6RZ2yhgrCWVRlk3cF7MBGzTGdwNLRE0epxwiDG8l7FWhrjY2Yp4KVNY8FltsQC2dnQyN9bETuyGlZavwrvvOcTNgpuauyqR9fOgH2hS5o8uYVoGjVTbLY7zakK3ZUnteSYCfXyKAxtDyc6A9Po8ASh2/DEpVKdj2ffZeHmR3fWv8yH/Blru4/QPTp54bTApojyJf5gmTCx5Xhp0rjOp9FlC2/3DKktGr404t5kbzdKCSeKP2qP4mMTAbZGXfkVZEh7JMV3mcptT8b4Wd9KQV/8UygRj831AQJCic8ZJeu56YGpJ1Cl359cR40qZTvZxcfkNaPixtGCmd7K3Fyi2pi0z4IMfNocU3Pb6OfqklVacaQ2vASqQeV9ZwZIuQb7GKFMU0YVQSkSSHcVLyx4lWPafcwX9hOgQXg7ejUtmu+7yKmyHB0klgWEVLRNxeZGcfrgCOlyA2ZyiAYd4s1ezDaE49OGhXp+6VnPTCbpBh4IHWkEzXdqbeyt4M6xjQQj7hkw1TF+cfN1oP2wlrhnYnrWFFg+CWy/jYg/ZTUIhiZXtChnudcVPP3aTcxPiV7QHxz/fEkOtOjU+yjaK5YG0JVDXhAndnyhr9jeOcstCwFKHBIGeZ3iCkd0ruPc OKY1vKJq 7WC+0uU0Cm2ZwlEYgDSCr534HBeGLnhoEYb456OUpMYAuacrXIMvHfIkDze7icnGMzSiXOxjVPi9jvsPiQMDnecxGbtlqgf/CZm6sXP6X2uXn+v1mvJYxdkkb0EV3dI3zUpT0Yvb/GMa4hrwtSu3EvMpkKAK+84AuB5sA4ouYUN7r2PKsAWAMPF/eRaAv/Ut7HUbB46kv5OWc0Kv4bWwlXe+p2PhXR4VaT93EAcRNclJ0sfe/emMn2UyoPfI69lVOCFSoEFJC27qYDkXVa4PWCjskGvba3n3pc8w/y8BPCUBUbFnClZ2HrP9qoLPXjzG8KcwDVzaE6LEBdxB7fBnM9x4RWC79vkg4USE+e9ErJXW8j5HKRugSBdZ+7xD7Wni0aR5jl884kD8QDI0pZFiwp5QUkEExlOLtHT+cJWjdDsFcjK4xQaefx4K0rOUQBZ1AlGdMIdR3vf22j9tzDXW8RNJEKtUAc/G3Pt5F5dvkfmp+5dJNtjEVK3WyjS2TM876vdnwfgU/IdebmP5RKfShlQQDRrXUMy37QmEnfv3KojRieUPzHIXvJNc/i5uybvkux7760NUIo11fwLf5XGgMDvakEokEcLGb8h5bh2vWDJF4ryZi93kzgF0mE740NsDOpq+qFb/Rx+R0hJ+hh3abfZ/0jwstlIWGiVWTOGhhmmS51m/+iGfRqsINsrGgtl6/Ns9X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" Introduce APIs allowing KHO users to preserve memory across kexec and get access to that memory after boot of the kexeced kernel kho_preserve_folio() - record a folio to be preserved over kexec kho_restore_folio() - recreates the folio from the preserved memory kho_preserve_phys() - record physically contiguous range to be preserved over kexec. kho_restore_phys() - recreates order-0 pages corresponding to the preserved physical range The memory preservations are tracked by two levels of xarrays to manage chunks of per-order 512 byte bitmaps. For instance the entire 1G order of a 1TB x86 system would fit inside a single 512 byte bitmap. For order 0 allocations each bitmap will cover 16M of address space. Thus, for 16G of memory at most 512K of bitmap memory will be needed for order 0. At serialization time all bitmaps are recorded in a linked list of pages for the next kernel to process and the physical address of the list is recorded in KHO FDT. The next kernel then processes that list, reserves the memory ranges and later, when a user requests a folio or a physical range, KHO restores corresponding memory map entries. Suggested-by: Jason Gunthorpe Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec_handover.h | 38 +++ kernel/kexec_handover.c | 486 ++++++++++++++++++++++++++++++++- 2 files changed, 522 insertions(+), 2 deletions(-) diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h index c665ff6cd728..d52a7b500f4c 100644 --- a/include/linux/kexec_handover.h +++ b/include/linux/kexec_handover.h @@ -5,6 +5,7 @@ #include #include #include +#include struct kho_scratch { phys_addr_t addr; @@ -54,6 +55,13 @@ int kho_add_string_prop(struct kho_node *node, const char *key, int register_kho_notifier(struct notifier_block *nb); int unregister_kho_notifier(struct notifier_block *nb); +int kho_preserve_folio(struct folio *folio); +int kho_unpreserve_folio(struct folio *folio); +int kho_preserve_phys(phys_addr_t phys, size_t size); +int kho_unpreserve_phys(phys_addr_t phys, size_t size); +struct folio *kho_restore_folio(phys_addr_t phys); +void *kho_restore_phys(phys_addr_t phys, size_t size); + void kho_memory_init(void); void kho_populate(phys_addr_t handover_fdt_phys, phys_addr_t scratch_phys, @@ -118,6 +126,36 @@ static inline int unregister_kho_notifier(struct notifier_block *nb) return -EOPNOTSUPP; } +static inline int kho_preserve_folio(struct folio *folio) +{ + return -EOPNOTSUPP; +} + +static inline int kho_unpreserve_folio(struct folio *folio) +{ + return -EOPNOTSUPP; +} + +static inline int kho_preserve_phys(phys_addr_t phys, size_t size) +{ + return -EOPNOTSUPP; +} + +static inline int kho_unpreserve_phys(phys_addr_t phys, size_t size) +{ + return -EOPNOTSUPP; +} + +static inline struct folio *kho_restore_folio(phys_addr_t phys) +{ + return NULL; +} + +static inline void *kho_restore_phys(phys_addr_t phys, size_t size) +{ + return NULL; +} + static inline void kho_memory_init(void) { } diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 6ebad2f023f9..592563c21369 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -62,6 +62,13 @@ struct kho_out { struct rw_semaphore tree_lock; struct kho_node root; + /** + * Physical address of the first struct khoser_mem_chunk containing + * serialized data from struct kho_mem_track. + */ + phys_addr_t first_chunk_phys; + struct kho_node preserved_memory; + void *fdt; u64 fdt_max; }; @@ -70,6 +77,7 @@ static struct kho_out kho_out = { .chain_head = BLOCKING_NOTIFIER_INIT(kho_out.chain_head), .tree_lock = __RWSEM_INITIALIZER(kho_out.tree_lock), .root = KHO_NODE_INIT, + .preserved_memory = KHO_NODE_INIT, .fdt_max = 10 * SZ_1M, }; @@ -237,6 +245,461 @@ int kho_node_check_compatible(const struct kho_in_node *node, } EXPORT_SYMBOL_GPL(kho_node_check_compatible); +/* + * Keep track of memory that is to be preserved across KHO. + * + * The serializing side uses two levels of xarrays to manage chunks of per-order + * 512 byte bitmaps. For instance the entire 1G order of a 1TB system would fit + * inside a single 512 byte bitmap. For order 0 allocations each bitmap will + * cover 16M of address space. Thus, for 16G of memory at most 512K + * of bitmap memory will be needed for order 0. + * + * This approach is fully incremental, as the serialization progresses folios + * can continue be aggregated to the tracker. The final step, immediately prior + * to kexec would serialize the xarray information into a linked list for the + * successor kernel to parse. + */ + +#define PRESERVE_BITS (512 * 8) + +struct kho_mem_phys_bits { + DECLARE_BITMAP(preserve, PRESERVE_BITS); +}; + +struct kho_mem_phys { + /* + * Points to kho_mem_phys_bits, a sparse bitmap array. Each bit is sized + * to order. + */ + struct xarray phys_bits; +}; + +struct kho_mem_track { + /* Points to kho_mem_phys, each order gets its own bitmap tree */ + struct xarray orders; +}; + +static struct kho_mem_track kho_mem_track; + +static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size_t sz) +{ + void *elm, *res; + + elm = xa_load(xa, index); + if (elm) + return elm; + + elm = kzalloc(sz, GFP_KERNEL); + if (!elm) + return ERR_PTR(-ENOMEM); + + res = xa_cmpxchg(xa, index, NULL, elm, GFP_KERNEL); + if (xa_is_err(res)) + res = ERR_PTR(xa_err(res)); + + if (res) { + kfree(elm); + return res; + } + + return elm; +} + +static void __kho_unpreserve(struct kho_mem_track *tracker, unsigned long pfn, + unsigned int order) +{ + struct kho_mem_phys_bits *bits; + struct kho_mem_phys *physxa; + unsigned long pfn_hi = pfn >> order; + + physxa = xa_load(&tracker->orders, order); + if (!physxa) + return; + + bits = xa_load(&physxa->phys_bits, pfn_hi / PRESERVE_BITS); + if (!bits) + return; + + clear_bit(pfn_hi % PRESERVE_BITS, bits->preserve); +} + +static int __kho_preserve(struct kho_mem_track *tracker, unsigned long pfn, + unsigned int order) +{ + struct kho_mem_phys_bits *bits; + struct kho_mem_phys *physxa; + unsigned long pfn_hi = pfn >> order; + + might_sleep(); + + physxa = xa_load_or_alloc(&tracker->orders, order, sizeof(*physxa)); + if (IS_ERR(physxa)) + return PTR_ERR(physxa); + + bits = xa_load_or_alloc(&physxa->phys_bits, pfn_hi / PRESERVE_BITS, + sizeof(*bits)); + if (IS_ERR(bits)) + return PTR_ERR(bits); + + set_bit(pfn_hi % PRESERVE_BITS, bits->preserve); + + return 0; +} + +/** + * kho_preserve_folio - preserve a folio across KHO. + * @folio: folio to preserve + * + * Records that the entire folio is preserved across KHO. The order + * will be preserved as well. + * + * Return: 0 on success, error code on failure + */ +int kho_preserve_folio(struct folio *folio) +{ + unsigned long pfn = folio_pfn(folio); + unsigned int order = folio_order(folio); + int err; + + if (!kho_enable) + return -EOPNOTSUPP; + + down_read(&kho_out.tree_lock); + if (kho_out.fdt) { + err = -EBUSY; + goto unlock; + } + + err = __kho_preserve(&kho_mem_track, pfn, order); + +unlock: + up_read(&kho_out.tree_lock); + + return err; +} +EXPORT_SYMBOL_GPL(kho_preserve_folio); + +/** + * kho_unpreserve_folio - unpreserve a folio + * @folio: folio to unpreserve + * + * Remove the record of a folio previously preserved by kho_preserve_folio(). + * + * Return: 0 on success, error code on failure + */ +int kho_unpreserve_folio(struct folio *folio) +{ + unsigned long pfn = folio_pfn(folio); + unsigned int order = folio_order(folio); + int err = 0; + + down_read(&kho_out.tree_lock); + if (kho_out.fdt) { + err = -EBUSY; + goto unlock; + } + + __kho_unpreserve(&kho_mem_track, pfn, order); + +unlock: + up_read(&kho_out.tree_lock); + + return err; +} +EXPORT_SYMBOL_GPL(kho_unpreserve_folio); + +/** + * kho_preserve_phys - preserve a physically contiguous range across KHO. + * @phys: physical address of the range + * @size: size of the range + * + * Records that the entire range from @phys to @phys + @size is preserved + * across KHO. + * + * Return: 0 on success, error code on failure + */ +int kho_preserve_phys(phys_addr_t phys, size_t size) +{ + unsigned long pfn = PHYS_PFN(phys), end_pfn = PHYS_PFN(phys + size); + unsigned int order = ilog2(end_pfn - pfn); + unsigned long failed_pfn; + int err = 0; + + if (!kho_enable) + return -EOPNOTSUPP; + + down_read(&kho_out.tree_lock); + if (kho_out.fdt) { + err = -EBUSY; + goto unlock; + } + + for (; pfn < end_pfn; + pfn += (1 << order), order = ilog2(end_pfn - pfn)) { + err = __kho_preserve(&kho_mem_track, pfn, order); + if (err) { + failed_pfn = pfn; + break; + } + } + + if (err) + for (pfn = PHYS_PFN(phys); pfn < failed_pfn; + pfn += (1 << order), order = ilog2(end_pfn - pfn)) + __kho_unpreserve(&kho_mem_track, pfn, order); + +unlock: + up_read(&kho_out.tree_lock); + + return err; +} +EXPORT_SYMBOL_GPL(kho_preserve_phys); + +/** + * kho_unpreserve_phys - unpreserve a physically contiguous range + * @phys: physical address of the range + * @size: size of the range + * + * Remove the record of a range previously preserved by kho_preserve_phys(). + * + * Return: 0 on success, error code on failure + */ +int kho_unpreserve_phys(phys_addr_t phys, size_t size) +{ + unsigned long pfn = PHYS_PFN(phys), end_pfn = PHYS_PFN(phys + size); + unsigned int order = ilog2(end_pfn - pfn); + int err = 0; + + down_read(&kho_out.tree_lock); + if (kho_out.fdt) { + err = -EBUSY; + goto unlock; + } + + for (; pfn < end_pfn; pfn += (1 << order), order = ilog2(end_pfn - pfn)) + __kho_unpreserve(&kho_mem_track, pfn, order); + +unlock: + up_read(&kho_out.tree_lock); + + return err; +} +EXPORT_SYMBOL_GPL(kho_unpreserve_phys); + +/* almost as free_reserved_page(), just don't free the page */ +static void kho_restore_page(struct page *page) +{ + ClearPageReserved(page); + init_page_count(page); + adjust_managed_page_count(page, 1); +} + +struct folio *kho_restore_folio(phys_addr_t phys) +{ + struct page *page = pfn_to_online_page(PHYS_PFN(phys)); + unsigned long order = page->private; + + if (!page) + return NULL; + + order = page->private; + if (order) + prep_compound_page(page, order); + else + kho_restore_page(page); + + return page_folio(page); +} +EXPORT_SYMBOL_GPL(kho_restore_folio); + +void *kho_restore_phys(phys_addr_t phys, size_t size) +{ + unsigned long start_pfn, end_pfn, pfn; + void *va = __va(phys); + + start_pfn = PFN_DOWN(phys); + end_pfn = PFN_UP(phys + size); + + for (pfn = start_pfn; pfn < end_pfn; pfn++) { + struct page *page = pfn_to_online_page(pfn); + + if (!page) + return NULL; + kho_restore_page(page); + } + + return va; +} +EXPORT_SYMBOL_GPL(kho_restore_phys); + +#define KHOSER_PTR(type) \ + union { \ + phys_addr_t phys; \ + type ptr; \ + } +#define KHOSER_STORE_PTR(dest, val) \ + ({ \ + (dest).phys = virt_to_phys(val); \ + typecheck(typeof((dest).ptr), val); \ + }) +#define KHOSER_LOAD_PTR(src) \ + ((src).phys ? (typeof((src).ptr))(phys_to_virt((src).phys)) : NULL) + +struct khoser_mem_bitmap_ptr { + phys_addr_t phys_start; + KHOSER_PTR(struct kho_mem_phys_bits *) bitmap; +}; + +struct khoser_mem_chunk; + +struct khoser_mem_chunk_hdr { + KHOSER_PTR(struct khoser_mem_chunk *) next; + unsigned int order; + unsigned int num_elms; +}; + +#define KHOSER_BITMAP_SIZE \ + ((PAGE_SIZE - sizeof(struct khoser_mem_chunk_hdr)) / \ + sizeof(struct khoser_mem_bitmap_ptr)) + +struct khoser_mem_chunk { + struct khoser_mem_chunk_hdr hdr; + struct khoser_mem_bitmap_ptr bitmaps[KHOSER_BITMAP_SIZE]; +}; +static_assert(sizeof(struct khoser_mem_chunk) == PAGE_SIZE); + +static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chunk, + unsigned long order) +{ + struct khoser_mem_chunk *chunk; + + chunk = (struct khoser_mem_chunk *)get_zeroed_page(GFP_KERNEL); + if (!chunk) + return NULL; + chunk->hdr.order = order; + if (cur_chunk) + KHOSER_STORE_PTR(cur_chunk->hdr.next, chunk); + return chunk; +} + +static void kho_mem_ser_free(struct khoser_mem_chunk *first_chunk) +{ + struct khoser_mem_chunk *chunk = first_chunk; + + while (chunk) { + unsigned long chunk_page = (unsigned long)chunk; + + chunk = KHOSER_LOAD_PTR(chunk->hdr.next); + free_page(chunk_page); + } +} + +/* + * Record all the bitmaps in a linked list of pages for the next kernel to + * process. Each chunk holds bitmaps of the same order and each block of bitmaps + * starts at a given physical address. This allows the bitmaps to be sparse. The + * xarray is used to store them in a tree while building up the data structure, + * but the KHO successor kernel only needs to process them once in order. + * + * All of this memory is normal kmalloc() memory and is not marked for + * preservation. The successor kernel will remain isolated to the scratch space + * until it completes processing this list. Once processed all the memory + * storing these ranges will be marked as free. + */ +static struct khoser_mem_chunk *kho_mem_serialize(void) +{ + struct kho_mem_track *tracker = &kho_mem_track; + struct khoser_mem_chunk *first_chunk = NULL; + struct khoser_mem_chunk *chunk = NULL; + struct kho_mem_phys *physxa; + unsigned long order; + + xa_for_each(&tracker->orders, order, physxa) { + struct kho_mem_phys_bits *bits; + unsigned long phys; + + chunk = new_chunk(chunk, order); + if (!chunk) + goto err_free; + + if (!first_chunk) + first_chunk = chunk; + + xa_for_each(&physxa->phys_bits, phys, bits) { + struct khoser_mem_bitmap_ptr *elm; + + if (chunk->hdr.num_elms == ARRAY_SIZE(chunk->bitmaps)) { + chunk = new_chunk(chunk, order); + if (!chunk) + goto err_free; + } + + elm = &chunk->bitmaps[chunk->hdr.num_elms]; + chunk->hdr.num_elms++; + elm->phys_start = (phys * PRESERVE_BITS) + << (order + PAGE_SHIFT); + KHOSER_STORE_PTR(elm->bitmap, bits); + } + } + + return first_chunk; + +err_free: + kho_mem_ser_free(first_chunk); + return ERR_PTR(-ENOMEM); +} + +static void deserialize_bitmap(unsigned int order, + struct khoser_mem_bitmap_ptr *elm) +{ + struct kho_mem_phys_bits *bitmap = KHOSER_LOAD_PTR(elm->bitmap); + unsigned long bit; + + for_each_set_bit(bit, bitmap->preserve, PRESERVE_BITS) { + int sz = 1 << (order + PAGE_SHIFT); + phys_addr_t phys = + elm->phys_start + (bit << (order + PAGE_SHIFT)); + struct page *page = phys_to_page(phys); + + memblock_reserve(phys, sz); + memblock_reserved_mark_noinit(phys, sz); + page->private = order; + } +} + +static void __init kho_mem_deserialize(void) +{ + struct khoser_mem_chunk *chunk; + struct kho_in_node preserved_mem; + const phys_addr_t *mem; + int err; + u32 len; + + err = kho_get_node(NULL, "preserved-memory", &preserved_mem); + if (err) { + pr_err("no preserved-memory node: %d\n", err); + return; + } + + mem = kho_get_prop(&preserved_mem, "metadata", &len); + if (!mem || len != sizeof(*mem)) { + pr_err("failed to get preserved memory bitmaps\n"); + return; + } + + chunk = *mem ? phys_to_virt(*mem) : NULL; + while (chunk) { + unsigned int i; + + memblock_reserve(virt_to_phys(chunk), sizeof(*chunk)); + + for (i = 0; i != chunk->hdr.num_elms; i++) + deserialize_bitmap(chunk->hdr.order, + &chunk->bitmaps[i]); + chunk = KHOSER_LOAD_PTR(chunk->hdr.next); + } +} + /* Helper functions for KHO state tree */ struct kho_prop { @@ -545,6 +1008,11 @@ static int kho_unfreeze(void) if (fdt) kvfree(fdt); + if (kho_out.first_chunk_phys) { + kho_mem_ser_free(phys_to_virt(kho_out.first_chunk_phys)); + kho_out.first_chunk_phys = 0; + } + err = blocking_notifier_call_chain(&kho_out.chain_head, KEXEC_KHO_UNFREEZE, NULL); err = notifier_to_errno(err); @@ -633,6 +1101,7 @@ static int kho_finalize(void) { int err = 0; void *fdt; + struct khoser_mem_chunk *first_chunk; fdt = kvmalloc(kho_out.fdt_max, GFP_KERNEL); if (!fdt) @@ -648,6 +1117,13 @@ static int kho_finalize(void) kho_out.fdt = fdt; up_write(&kho_out.tree_lock); + first_chunk = kho_mem_serialize(); + if (IS_ERR(first_chunk)) { + err = PTR_ERR(first_chunk); + goto unfreeze; + } + kho_out.first_chunk_phys = first_chunk ? virt_to_phys(first_chunk) : 0; + err = kho_convert_tree(fdt, kho_out.fdt_max); unfreeze: @@ -829,6 +1305,10 @@ static __init int kho_init(void) kho_out.root.name = ""; err = kho_add_string_prop(&kho_out.root, "compatible", "kho-v1"); + err |= kho_add_prop(&kho_out.preserved_memory, "metadata", + &kho_out.first_chunk_phys, sizeof(phys_addr_t)); + err |= kho_add_node(&kho_out.root, "preserved-memory", + &kho_out.preserved_memory); if (err) goto err_free_scratch; @@ -1079,10 +1559,12 @@ static void __init kho_release_scratch(void) void __init kho_memory_init(void) { - if (!kho_get_fdt()) + if (!kho_get_fdt()) { kho_reserve_scratch(); - else + } else { + kho_mem_deserialize(); kho_release_scratch(); + } } void __init kho_populate(phys_addr_t handover_fdt_phys, From patchwork Thu Mar 20 01:55:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 059E8C35FFC for ; Thu, 20 Mar 2025 01:56:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E3CA928000C; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE617280001; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEDB328000C; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9C79B280001 for ; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1DF7F805CE for ; Thu, 20 Mar 2025 01:56:19 +0000 (UTC) X-FDA: 83240264478.23.BC925C5 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf28.hostedemail.com (Postfix) with ESMTP id 4C5AAC0004 for ; Thu, 20 Mar 2025 01:56:17 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wJl0mFuG; spf=pass (imf28.hostedemail.com: domain of 3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435777; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=lXPfUZnHXCu/089HKRzxD1tqElN9KTEC+S1CBa6UIoL13f6b/EPoy2bGX0m5OsZdgTh6dr tL54SBue9+hAdRvxf+VrW6aZL0KMQBHrUCzTMtIAdbpxHRskAitHMkU2VAQ9Ee7cGDT33J dVJ1H08bYRHCqMFmEJhgkZOijxj1cek= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435777; a=rsa-sha256; cv=none; b=KULWqOPzVzUSbcjKvdWq0EokShiqscXFPyxCjlgwFtvl8Rb9jr/9k8Ssdw5ToLR+putUji V2rUKMWp0ziurf2WsAoYNpLpH/ojS7+8GsoRycyszyHfTZUag7X+cdPs/io3DUDGmblv2b ca2NJqQDgLWNyKV1VujXsjV9S5+hQ90= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wJl0mFuG; spf=pass (imf28.hostedemail.com: domain of 3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2242ade807fso6782495ad.2 for ; Wed, 19 Mar 2025 18:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435776; x=1743040576; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=wJl0mFuGaffdLfdIIjSfKWymlQzC0VbPLFU/bq6N2JLfwRm0TC12oICJGF7kAs/Tdu HJbKSZATaSZp+GmAUi/trhSvTgNHK0tSol3DE2Zx+pblD/CL/In8wliecosOMLBn1SVn yenGN0vWSB1rzsRZHVZo/SyPcusiFS3ra+HvRQ8RzhqrLoPKA/JWxegna+yIboyklkS/ ncL/MyEUmm0l/hWQm4TR9LuD93W8BoWIESGj2Xz2PQsjaaUyO8Pb2kUdplX7venA+LKI 54sNTQ8suD76cI+T3c/wKQpCnUW++qpuJBm85YqRtm2JIPNhk6S593ifxEwvrWGU+5rO oY0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435776; x=1743040576; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=ryZOMaZxco1GjyYAr061EFoJhpwVbJhVfOGO059lIDUeVkrJflih3HfTRrM/SLsGcK 3uC7YBDuaDjWn+xRJ9kuGwJT49/jtgxI5KuKQ52zA+5kBjl1i0uavgwRogBu5mJrWD10 IRzkdE+H+YtIO5/5EDDWIdne35GGPWtp4ZKpglBBjJ+hWHaK5VPvsfdOIV1AS9tiRrSt 4980DkJ38OvnG/WmeMCA22sloTBexv7mQtSXhXy38gSKGAkCqeI8Vx1XR0bAAUhS7qhq iSNmUps7348lQA83jgnOwZQRIezAXVCCK8c4DGmBlrhJPJSSUah4No4ZLBnAaX2E+2mk 5IFQ== X-Forwarded-Encrypted: i=1; AJvYcCWLNskcL5CjiaJI/yajzP70DSXBp+UtDn6GxrCGatIFNAtZIoclvFEUCqHFf4I9QH13ix/yhsz+mg==@kvack.org X-Gm-Message-State: AOJu0YzZBibCrD9JKOfVYhrkX7mb3L8CkFCz65y9MgTnEEkdm8BSnWZ+ LQ5fkj4L7SjcxjBRw7A53tc+Jh4GE2ry76Qgwx/P8Wbi6Wc/2KkI7WbyrkUssT2p77O19Ye6+XA yo3h8VO0rYvcqa3vt8g== X-Google-Smtp-Source: AGHT+IEM1zhUI4aB8FBWWC+mYMClc6ahuDGj6ZxX8EDQ9DE5DKRFOEqA+5eaTn1Hw9R605eYXv4+w1iX7+c1hN7N X-Received: from pltk7.prod.google.com ([2002:a17:902:6947:b0:223:6930:304e]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:db0a:b0:223:f408:c3f7 with SMTP id d9443c01a7336-2265edc7b36mr19232405ad.16.1742435776171; Wed, 19 Mar 2025 18:56:16 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:45 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-11-changyuanl@google.com> Subject: [PATCH v5 10/16] kexec: add KHO support to kexec file loads From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4C5AAC0004 X-Stat-Signature: dyc3xc8f3xk44ja88r5w5fjnqco6dhmr X-HE-Tag: 1742435777-559283 X-HE-Meta: U2FsdGVkX18oGaVBZ6aBFdk931UM90mA2pzylG/RsD+EQXKGnoXxcmKzP1ym+gPC+cVP4p6y/tmxSEGcZ5h4fP+kcE2i9kfxINWQ9Wd9w9azp39HqvM4CDVCSXNMM93x/P5Yy05vShLIwBTqzczfaB9cV0Pyxmsk8uq0Uxnbo8LY161zl1+lu/zZtKOfHQo9yjbau5jeuFslSVEPNT/GBC/tn3CDvTyVsPi8yx5pOeZ17UWuxUhYrbxXONx+EHRp1c1/TNWGqfe2lazFrrU+5TB+G3pfn+DkY2hVp3+OIuUmjue36z7aRLkFfGVcVwJD4H9LaFUtJoB0mRBAdu0Kln2bJiJ2hzcBHFLh/QV3nlzLM04UJvvIu6CGKcf1u2UaMl5FpmgctAwKyu/N8weoFBgH8ary1qcK9DFTweQ/jVkU8Fu8olVrPf2OUrUBaEDg2D2kSkh54odDqhDBQUMcUOViSM6+8MMP0xue4ZJ/VCHxADC9wDWo4Niuf47P0DPG1n3VFMiaX2cUQA6OWqVIpbvm1rrlFxI05AXemmFnVM4TQFWy5oPPrR3djiCgPBEeaGpFZSlYpZG6gqYyssJfg1VawppH+A0rdIRRW8Y7vvmDhU/+A0YXNv4vMnd4DfI0tlDtiSzieBTR55I3SzyfihjZKMq0EVlPHxeHqNwPAzx6ww484ABAx2UPFzEC75Go8xEngUeUnG7ReEjtIhPIT85ziPhEZLsqz+lhOSHyLmAtjk+nWFMHcmJrZ7THKHIyY1RrFQZXCBJ//xcMu/xcd7bgf7JqpQx8XltuLx8a2bRDrMJyhse7szYC6qI5MynR+ikwfwc2zRTavCtPWCEVKW98c5ogcNzJUJrAijhHk4V5y6JGZrL7EU00WhZ45yJvfvdM7nfZLM0iG3eYR/d9M+HIBTALqmiFvSNIkQWV4QTaJk912PGkPnTlR6iu6jijhskFxcHolfetnjQ0hnl Bj69fAJ8 fC8yVLu7UfwC5c/0396P2uAVByT9Dtl8hxeHj0F36ZPKEtVI5hSnZTeaWLIkY8H0yBZq8Fef2RS9+1+OxVN2p8S1rUO25pFA7SLxwp/9rifdK/o9jVpPFjnsW+xsRGzLkz8TCxpp6y8iwSoJvdWKWweXrIbc8VzpqohvHwQnX0BsRXonxenOxvKPLMmV3aBO/K2rTBIVuZ4gumGU29T+yBOydAMfXFGrbKyGvUrqYPenIH/TuZ6mGJQGuesavYcU/5jod5sLjarHcslqSw0DadOKxT6+hQBbWSc+Nmo1xjXOd93V41Mp5B5j22osH10rJXhxkEloI3pLTJm0jFomXgsdm+v8X5LIJ19hyQHH8MsnivIGahJP+HGC6Gze4Z3/+3GQDttpsag9jiTHnEkh5Nowyn7LbZWJ9kIzT66jsLmbbd1yTKzFjofHhq+0Y92z3vgsv85V8lviuefVo2VoRJcI72OqtGZuYa9y4KtsWJNvhGi4tLFWL+76pOZ49gM3hEUPO54SpqMn939ppR1vePDthtkCxDwImI2zx5TfefzmobDhqioxPjFMGCZHvy3eHeesOT8rb1vcEeqahWoCpBThpPLAi8v5dSus2m37GVFP345T8bZrjiX3hxZa7hBFWrzpI3/KfEgivkgzA2ZuOJGnsw41HXJfO5w0j7w9I6KtxXVoiHDzDFTWi2KRIz0s3DoL0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf Kexec has 2 modes: A user space driven mode and a kernel driven mode. For the kernel driven mode, kernel code determines the physical addresses of all target buffers that the payload gets copied into. With KHO, we can only safely copy payloads into the "scratch area". Teach the kexec file loader about it, so it only allocates for that area. In addition, enlighten it with support to ask the KHO subsystem for its respective payloads to copy into target memory. Also teach the KHO subsystem how to fill the images for file loads. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec.h | 7 +++ kernel/kexec_core.c | 4 ++ kernel/kexec_file.c | 19 +++++++ kernel/kexec_handover.c | 108 ++++++++++++++++++++++++++++++++++++++++ kernel/kexec_internal.h | 18 +++++++ 5 files changed, 156 insertions(+) diff --git a/include/linux/kexec.h b/include/linux/kexec.h index fad04f3bcf1d..d59eee60e36e 100644 --- a/include/linux/kexec.h +++ b/include/linux/kexec.h @@ -364,6 +364,13 @@ struct kimage { size_t ima_buffer_size; #endif +#ifdef CONFIG_KEXEC_HANDOVER + struct { + struct kexec_segment *scratch; + struct kexec_segment *fdt; + } kho; +#endif + /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_sz; diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index 640d252306ea..67fb9c0b3714 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -1053,6 +1053,10 @@ int kernel_kexec(void) goto Unlock; } + error = kho_copy_fdt(kexec_image); + if (error) + goto Unlock; + #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { /* diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 3eedb8c226ad..070ef206f573 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -253,6 +253,11 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd, /* IMA needs to pass the measurement list to the next kernel. */ ima_add_kexec_buffer(image); + /* If KHO is active, add its images to the list */ + ret = kho_fill_kimage(image); + if (ret) + goto out; + /* Call image load handler */ ldata = kexec_image_load_default(image); @@ -636,6 +641,14 @@ int kexec_locate_mem_hole(struct kexec_buf *kbuf) if (kbuf->mem != KEXEC_BUF_MEM_UNKNOWN) return 0; + /* + * If KHO is active, only use KHO scratch memory. All other memory + * could potentially be handed over. + */ + ret = kho_locate_mem_hole(kbuf, locate_mem_hole_callback); + if (ret <= 0) + return ret; + if (!IS_ENABLED(CONFIG_ARCH_KEEP_MEMBLOCK)) ret = kexec_walk_resources(kbuf, locate_mem_hole_callback); else @@ -764,6 +777,12 @@ static int kexec_calculate_store_digests(struct kimage *image) if (ksegment->kbuf == pi->purgatory_buf) continue; +#ifdef CONFIG_KEXEC_HANDOVER + /* Skip KHO FDT as its contects are copied in kernel_kexec(). */ + if (ksegment == image->kho.fdt) + continue; +#endif + ret = crypto_shash_update(desc, ksegment->kbuf, ksegment->bufsz); if (ret) diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 592563c21369..5108e2cc1a22 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -245,6 +245,85 @@ int kho_node_check_compatible(const struct kho_in_node *node, } EXPORT_SYMBOL_GPL(kho_node_check_compatible); +int kho_fill_kimage(struct kimage *image) +{ + ssize_t scratch_size; + int err = 0; + + if (!kho_enable) + return 0; + + /* Allocate target memory for KHO FDT */ + struct kexec_buf fdt = { + .image = image, + .buffer = NULL, + .bufsz = 0, + .mem = KEXEC_BUF_MEM_UNKNOWN, + .memsz = kho_out.fdt_max, + .buf_align = SZ_64K, /* Makes it easier to map */ + .buf_max = ULONG_MAX, + .top_down = true, + }; + err = kexec_add_buffer(&fdt); + if (err) { + pr_err("failed to reserved a segment for KHO FDT: %d\n", err); + return err; + } + image->kho.fdt = &image->segment[image->nr_segments - 1]; + + scratch_size = sizeof(*kho_scratch) * kho_scratch_cnt; + struct kexec_buf scratch = { + .image = image, + .buffer = kho_scratch, + .bufsz = scratch_size, + .mem = KEXEC_BUF_MEM_UNKNOWN, + .memsz = scratch_size, + .buf_align = SZ_64K, /* Makes it easier to map */ + .buf_max = ULONG_MAX, + .top_down = true, + }; + err = kexec_add_buffer(&scratch); + if (err) + return err; + image->kho.scratch = &image->segment[image->nr_segments - 1]; + + return 0; +} + +static int kho_walk_scratch(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret = 0; + int i; + + for (i = 0; i < kho_scratch_cnt; i++) { + struct resource res = { + .start = kho_scratch[i].addr, + .end = kho_scratch[i].addr + kho_scratch[i].size - 1, + }; + + /* Try to fit the kimage into our KHO scratch region */ + ret = func(&res, kbuf); + if (ret) + break; + } + + return ret; +} + +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret; + + if (!kho_enable || kbuf->image->type == KEXEC_TYPE_CRASH) + return 1; + + ret = kho_walk_scratch(kbuf, func); + + return ret == 1 ? 0 : -EADDRNOTAVAIL; +} + /* * Keep track of memory that is to be preserved across KHO. * @@ -1141,6 +1220,35 @@ static int kho_finalize(void) return err; } +int kho_copy_fdt(struct kimage *image) +{ + int err = 0; + void *fdt; + + if (!kho_enable || !image->file_mode) + return 0; + + if (!kho_out.fdt) { + err = kho_finalize(); + kho_out_update_debugfs_fdt(); + if (err) + return err; + } + + fdt = kimage_map_segment(image, image->kho.fdt->mem, + PAGE_ALIGN(kho_out.fdt_max)); + if (!fdt) { + pr_err("failed to vmap fdt ksegment in kimage\n"); + return -ENOMEM; + } + + memcpy(fdt, kho_out.fdt, fdt_totalsize(kho_out.fdt)); + + kimage_unmap_segment(fdt); + + return 0; +} + /* Handling for debug/kho/out */ static int kho_out_finalize_get(void *data, u64 *val) { diff --git a/kernel/kexec_internal.h b/kernel/kexec_internal.h index d35d9792402d..ec9555a4751d 100644 --- a/kernel/kexec_internal.h +++ b/kernel/kexec_internal.h @@ -39,4 +39,22 @@ extern size_t kexec_purgatory_size; #else /* CONFIG_KEXEC_FILE */ static inline void kimage_file_post_load_cleanup(struct kimage *image) { } #endif /* CONFIG_KEXEC_FILE */ + +struct kexec_buf; + +#ifdef CONFIG_KEXEC_HANDOVER +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)); +int kho_fill_kimage(struct kimage *image); +int kho_copy_fdt(struct kimage *image); +#else +static inline int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + return 1; +} + +static inline int kho_fill_kimage(struct kimage *image) { return 0; } +static inline int kho_copy_fdt(struct kimage *image) { return 0; } +#endif /* CONFIG_KEXEC_HANDOVER */ #endif /* LINUX_KEXEC_INTERNAL_H */ From patchwork Thu Mar 20 01:55:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B84D2C35FFC for ; Thu, 20 Mar 2025 01:56:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63D1928000D; Wed, 19 Mar 2025 21:56:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5EE8A280001; Wed, 19 Mar 2025 21:56:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41AEA28000D; Wed, 19 Mar 2025 21:56:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1C9B3280001 for ; Wed, 19 Mar 2025 21:56:20 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 82346C08E1 for ; Thu, 20 Mar 2025 01:56:20 +0000 (UTC) X-FDA: 83240264520.28.DBB106A Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf29.hostedemail.com (Postfix) with ESMTP id B7EDB120008 for ; Thu, 20 Mar 2025 01:56:18 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TU6ORYOt; spf=pass (imf29.hostedemail.com: domain of 3wXXbZwoKCGAAF8LEWS8LJEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wXXbZwoKCGAAF8LEWS8LJEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435778; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UkcR0SU+CU01cj/UMVFVKIzr3SF+GMypT0ANn4wKrhE=; b=iflzZ01SwFPp1qX1fQw7xGz0RTPlb2XdVEBNH6FV36Cju2rkruou6iyLjSkkxssTViWN1u /kAHiWwSRnOfvDp41BOYQV39HV63spaMXGUsw/SkvEkKr+/f2m3hM/uKAnx+Ak6YWlWqMy 25n6Sqgv/cD2st6xWaY3ZmT2lazN6Ww= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TU6ORYOt; spf=pass (imf29.hostedemail.com: domain of 3wXXbZwoKCGAAF8LEWS8LJEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wXXbZwoKCGAAF8LEWS8LJEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435778; a=rsa-sha256; cv=none; b=u3g77DHeORzKbTXjpGe5L+sbekQ1p6LYDjV3WndiigZrfKdRKnec/uItdiKKP0Fpeh1Emj hqiq5A/S3tAVLDpDabKa+ym9JeeiIcQ0Y+XD3+DaqbDV/+nx1qA7TxgRWxEZwnqoioD0R0 p6MnhzmhO5pV0EzMJZPkPhWYZN0HD1M= Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2241ae15dcbso4041185ad.0 for ; Wed, 19 Mar 2025 18:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435777; x=1743040577; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UkcR0SU+CU01cj/UMVFVKIzr3SF+GMypT0ANn4wKrhE=; b=TU6ORYOt8kN5zzsm+Oe0cZ5LJJInlTogngbFejaqvJQyBnPrYoaWhJ4rzsOVXCpLeb nm16l7A/gCp4jaSPrBDtac/mhFlb2n4vuEVxrva56laxPZMJuAo4DEdnKSK7jkTPq47k R/XmlDBB92KzP9QFI8zTpim3owTSMZ1+9/SdGT9+ULC9xJMdXXQyM/WG1n5pcsYb5rzz eo2+UANb9Fo/ZQ4FGoOdqRAmcgW5LLVWRRoI3uRn+UGhpe4ygoMC7TcmjENXztSw1NFb KJYiqt7nh51ZUPx2ErILA71XhjNFgWjCHHh+FNCnvSOUHb9mK4HAY9gW9sAE8J0d0aep yoSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435777; x=1743040577; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UkcR0SU+CU01cj/UMVFVKIzr3SF+GMypT0ANn4wKrhE=; b=A8W4NkA1Gm5kvCh7+fdItKOFhT8f1HwFqqKSzbtmAwFQb/dP/9eu0lVLhjGlrOAo5w tjinMttTw/v1QCg7mlKx8Z/X3agqiRZyTHsylgxvs3cpeZRzwNjKgBRceAcnyJ+ywnYc VjnJsQnBar5SH2hVY+18tjz02suhj63vTsgDkc35uzXxT0TWd+WT3z4MmD2FP62lcHq3 m848CQTm+Hjp0s/Izbfxa7TKGWzyGi8tHqCabmP9jVSdRVz7SwYMUnaOUXH0ZIBKI7VM MWjNuz9Yq5OV4jTgC32ab5E+vmsYl4nXAYG5QdqUeh3VTyJeXWO6JAtYiiWWsI87QjFn cvcQ== X-Forwarded-Encrypted: i=1; AJvYcCXyk8ylOtjczwksFYFWjMqx+8fb2wJuQf4HYMZMqiFA64ax5B5B+NSUY7YLxuBlnhR9PvpjM0gMyw==@kvack.org X-Gm-Message-State: AOJu0YwVXbRK0cH6jL5Qr2A9PLtUDgk7x0oYDSndV7aOh/3/QAcIwxmU 7A4B4ct72fMPK5n7tOLaEHKG8WiWKGlVDY2FCsidNSLqgNiU3bO5sbeW9RpWIr9SpD+yORGVUJU IQ74ZqIkR6Mu2vMtOoA== X-Google-Smtp-Source: AGHT+IEq833KvKiO9EApYrAEuf1UTsJySLJu5CjTMNPxyZpCp1rooyCfDuUQ3lxBHQoctugFxssTdHmFh/kEHnmc X-Received: from plsq4.prod.google.com ([2002:a17:902:bd84:b0:223:3ab:e4a0]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:e801:b0:21f:c67:a68a with SMTP id d9443c01a7336-22649a2ee71mr75511855ad.31.1742435777590; Wed, 19 Mar 2025 18:56:17 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:46 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-12-changyuanl@google.com> Subject: [PATCH v5 11/16] kexec: add config option for KHO From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Queue-Id: B7EDB120008 X-Rspamd-Server: rspam03 X-Stat-Signature: 99znnczotuy6tw1ed8e956n9pszt56t3 X-HE-Tag: 1742435778-308071 X-HE-Meta: U2FsdGVkX19IwbiZYouEP31Hwy0FGsvCDY98E3HG6DO0TKc7oWrrBn+27g4uQ1nrRJmvRtiF8V/D5AEN0hxfRHhQMOBQEMZM5Q/ZC/yTVW/mQTnTXx+XhZ9pvLrCOcBPI0lX92/couP+zki8Xo0h3bOqeu/7WLeuIj8bCL8YBuUBkixl2ho2S4Zgyo4hVgPTOVp0sJsIV1ihVu9nwePfnOIc76kSekXv1EffDhW4cYlUJtIcVm3QAaWQgOUCX6WJ70/K80Q6rwDxWjSpTm9+dSnQMNtNSc0RAKDWQLaUxoSMwKu8UKiYh/jZxe5Ybkc9O1k7Vh3CX35uekHNZdLcnn8IDS0j6vx7gI7FwMxW6RTTvQkJB+OltyP1eLH6PKanLU0ooL4Kl9GfU66T6rz1BESa5MuG1nQnoTqzaUP2WB9FeGoPPRNkFmxTiOUa44LoUpPhC6/9N2fKP+8tCc7GkjK/p1KASACrmxf78Wrw1IiE7XknBJ6BL2ZZrV0fBxO8cowMlotTqWEpyVhMPY9T8dB+JUsg/NxzJYhGR27utN/z4qEoBTyROmMT04zuFLFNuwNtSJpDkWqQR8zpcxbPcPKWuVtAJnXnkyihWM6lfzAiOhizMWJpiY0TwkrNQ02GNvGRicg9tJ4XSapsRNA1F2z2dazR/13AA+5WSCh4LQRNNhjHQioZIBPe+H2wzMjEZzT2R7/RZUyJ7TPW5BHF0RDskekWMHA0VJkyxAjUX9uX97TjG9ZQhj+NU8hKQl2KiYsRR+Cgvyp4r5w6GJF5Ql68iJ/G+EdcY9uyI8TvzWPL0Bf6V+1F3eGpUFm6msb6rVreua1BGsYiWSGopWOggwwkTSXbWsWFljt98oMM6p/xPp75xyaj7BFUdnmW+pbT+13KTwDjrEgJHxSJ33uOrr5XLN+T94stcINw5oPJZLO5N2Br8uThezmSQn2yOT01PdEymxSgIZEJcyf++Z9 qJAjj1SB 9OL5bDnXGuod/f/uI7+VPYTJ8Uazm24A2dYPJTNI4hc8Zu9xPKEj6mdv+T+iY9S3bIAcIQDk6xdqP8jLm+7i8Izcrt+/Yd96KEbTBEvlQcCZS4EwNqnnMKgeRPBbnr6SdN0XDyMpHWcEPY8APweNTqAvaeUC5HGAmXSVpTqZZ9oZ1m40rTlcX5mGJFTmB9VYN9nsjFK7EUBdey2AJXAyrth270GBeY7Tz95nDVi/rbAhhFXwMsM3U4+RbWfW2IYLw67q794dh9uyv1ggM1nFzFQftzg89xTebzRYQMeS1OUGPmZjBc7phVYWuCcqWebqQNgr+CKdotjYx4QEoybVlwsBMTf3tCDnxQv4YQJxAZNmJCCJ1axMDPoFhVYC8RKhnWMjgcFYUZigUcbHHBLo/rc3pEt1V7eRXxfW+FekwQgoXnwdCJk4n4pDyn3yncPorPuMqGK9UnquYkveFgzXDr/SDFQhklyY0sEGQ+bxxYJkpUwF6CMFVuX6ishKg8QRwhZswfDhaw95hF6gV6FIRgPDmssmG0IMUgnSUeayHR3PEA82t0M+4LbmKgcjr1U51MmUOG8wbhQkPySMUf74b3Bw7FCOo8G0kw/zfmD2oeIyFCLEF8q54YDU/dqlnE4qkcm4FYveIAEQE+TKojdwmUfQTbyfi3lQIkzwiMi1Vq9fsYOxAduI2g6GS9FdhS36h/dAv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf We have all generic code in place now to support Kexec with KHO. This patch adds a config option that depends on architecture support to enable KHO support. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- kernel/Kconfig.kexec | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec index 4d111f871951..57db99e758a8 100644 --- a/kernel/Kconfig.kexec +++ b/kernel/Kconfig.kexec @@ -95,6 +95,21 @@ config KEXEC_JUMP Jump between original kernel and kexeced kernel and invoke code in physical address mode via KEXEC +config KEXEC_HANDOVER + bool "kexec handover" + depends on ARCH_SUPPORTS_KEXEC_HANDOVER && ARCH_SUPPORTS_KEXEC_FILE + select MEMBLOCK_KHO_SCRATCH + select KEXEC_FILE + select DEBUG_FS + select LIBFDT + select CMA + select XXHASH + help + Allow kexec to hand over state across kernels by generating and + passing additional metadata to the target kernel. This is useful + to keep data or state alive across the kexec. For this to work, + both source and target kernels need to have this option enabled. + config CRASH_DUMP bool "kernel crash dumps" default ARCH_DEFAULT_CRASH_DUMP From patchwork Thu Mar 20 01:55:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023312 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02128C35FFC for ; Thu, 20 Mar 2025 01:56:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 32B1028000E; Wed, 19 Mar 2025 21:56:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AE12280001; Wed, 19 Mar 2025 21:56:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B18F28000E; Wed, 19 Mar 2025 21:56:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D814C280001 for ; Wed, 19 Mar 2025 21:56:21 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 50557B78EF for ; Thu, 20 Mar 2025 01:56:22 +0000 (UTC) X-FDA: 83240264604.10.06C1E10 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf19.hostedemail.com (Postfix) with ESMTP id 786441A0005 for ; Thu, 20 Mar 2025 01:56:20 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kb+joydF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3w3XbZwoKCGICHANGYUANLGOOGLE.COMLINUX-MMKVACK.ORG@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3w3XbZwoKCGICHANGYUANLGOOGLE.COMLINUX-MMKVACK.ORG@flex--changyuanl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435780; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rd3wJc2IZkbxeklI0oyF9EA5atShYqSy3T5piwpZMms=; b=5eOs4mKW+HAL1JYbZKo3H0G/cdg9BsYLr97qE9WrfHvD+K5pWAv5ea5Qu0/jT1+FyG/bQ4 MOgSIUcfuXM1O3lUtjJI/i6skGcW0puhvGUjuDEJ8AnFjoshDmtBszN57I6X/ymUaNgICq tdNLEg1BgDxOmcmPBLXodqDkEriOkpQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435780; a=rsa-sha256; cv=none; b=iU/SDztaTUB8+sg/FMEoTD7R92jGCLQgeXc1k8dE+uAg8m1eqCHmPLBT/f9K8lwTi7IpE3 pNK6v9LlIcs8Ktkoco5WDnRavg3ldiTFaQHJbUuj6uPneaRQOqBszE2OeWHn9fv4By9qQ5 7W6u8+dEKNmRGn6J6vLYaFhFas61I2g= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kb+joydF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3w3XbZwoKCGICHANGYUANLGOOGLE.COMLINUX-MMKVACK.ORG@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3w3XbZwoKCGICHANGYUANLGOOGLE.COMLINUX-MMKVACK.ORG@flex--changyuanl.bounces.google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2240a960f9cso2984805ad.0 for ; Wed, 19 Mar 2025 18:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435779; x=1743040579; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rd3wJc2IZkbxeklI0oyF9EA5atShYqSy3T5piwpZMms=; b=kb+joydFT3j/GE4h0VsP/8f4fDFza6t3l7l3csjnbroOduWaAqLruxfAjVhNvDoVTU u+JgsrOwEDeoaAjH6lJfe2ZM5MRYzmOi3ApYi4Tia3D4w1H4U5qNdP36t3BzlqNCj5gK eV+ysn8H0razPFCiCOBpgPNMV/5XJ3P9UIq2Rg31uAiZUtIm0KOqGb3mvYkpWUZpCYBi Ggak8JG4abd9R524w61iLKNX+2+kk4EUftHajTNNshfvwo6gc23GQNM4TCgRCZHzULrZ 64BQlMSBC65xqPGbHUe9JMLSGrarLxQol+c3Rpp2iiHNNjBF/1BAnLyv6pVCMkkoVYBn ZPPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435779; x=1743040579; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rd3wJc2IZkbxeklI0oyF9EA5atShYqSy3T5piwpZMms=; b=SDzINnf4lb6Mzl2f8uaXAigiMk/av40nlfdIoBgA3szLwDxyE0nTqmecAuzDcpXzzZ sSQ0xfOw24vnTrmrCok9QZa5hPZI0En3KjVRKSZvw97LvyQd0F+XV/NQx8vzR0zESUiH S3LAlTzZm4vyNLrCRXxOQF4XOsoBYhvFlLnqbJRh0X8Zp+HsOIESW+TXpvALvOZ0aJ9J Ymc72P5EWWG0tRiGiFcbhAvK6ludm4JUs0Z5pwZMfMltuOXE5S5XWm8pKssm+hgpWCRg klpm8ZRjdAIk6oMlEIdjW2RcIhIZZE74t6DLnQ/hnDGPT+JoKN8e4wy0opK8Sw+GSeY2 aWuA== X-Forwarded-Encrypted: i=1; AJvYcCWtTTfALGfyvKQ0pRd+0b+F/CSQ8OmyWvWNkV5c7okzYhPfwFvR1NXpujtMBuWtXlRwWenTQzc/Qw==@kvack.org X-Gm-Message-State: AOJu0YynNfiGtQfzZPtfzapb964cikuvQ57PvMKzIs4lkYHvc6Aw4e2n taOB2GJKHzsZitoigFd+lYx4ZJCzZk49LJqhgY6oSmbYhP4sHZJq9WMynL6CUjfrfFBognHCzHO p7Who1htQPDXZBbQpGw== X-Google-Smtp-Source: AGHT+IGJMwZEQr7XX54NJ2Wc09gQFNrebDpGN2LmiuqxXXpWV4PLS6zLIf0YQX/7suAC0aQ3d5AJ2BOMHO4N4Oqa X-Received: from pfbig24.prod.google.com ([2002:a05:6a00:8b98:b0:730:597f:f227]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:3a08:b0:736:57cb:f2b6 with SMTP id d2e1a72fcca58-7376d62aac5mr7233223b3a.12.1742435779245; Wed, 19 Mar 2025 18:56:19 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:47 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-13-changyuanl@google.com> Subject: [PATCH v5 12/16] arm64: add KHO support From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 786441A0005 X-Stat-Signature: anscemsm4i5afboxe5cmceyi7xxy1758 X-HE-Tag: 1742435780-955774 X-HE-Meta: U2FsdGVkX18qV1Nm82Hp9fKu+ExXQhHvC9nNVVNGkScxP073znHWPDr+GVybxLbScmhZs8A6MU3NX4c82Xddc+Tdp9AkhazQ1JXok/4WO1+0YV7JCsyy9z4WXZoDQ++fqIWno3CEXKk84Btx6Tb653rNOpxneONdM0uF8oop57zbOxNOwd6iBWHZtk+utvwmqjskLbre0tTDtySXxvnOHH9VWRQcUpW2zOnj6yzdCHkzb9oIAuf4FzWxSEuVhnSJ/dGZlGqUCPMvc13wNTpmegQTV0ntw1ThM24Z7RCX48hFO+UfQhrTW8MUhAg6SnBY1CmbvuI3aILl9x7oivqaT6d14Kdcr6TySd9gT0iUaO8S9ISjSl1X5aojnR//Jb9tiQYrf7nTVAnVLW3FkW7o6RRFJaXsZV4LhDkvvoR02IVTZTHhIAZfMRUv9LD+4CiP5036Q2CHaOlasAXMdxf1GdRWvibhYv8t7yCK73qDjcM0ms/a5RVn3/GxVLpkGJVeIOoNxQWGydOCi+UhsTkq17BBxcjHKxHEcS0IfUzGnMqS33OcWZseLLarbG5QMqNybZ5LzGdpTJXvcvmVAjDMX3jvASi8eKFnKunGD2p/skWXyiHxEYVMIiRdh2VJfYLt/kfJy75R+BdiJtAn8M/68WH4WYRWT7eAUj0QkKih0l1DyZP3NTd5qVfSN51nS7+mh4U3Z7yyQ3Tc0241kxrFKIw29UcBXCkqwSoF7QHM2zJEvc9yAen7Y8d8xgfNNaj53m4EnNqAvum9RM/qds7kfGyGoXdNz/1p25Um9RV+KBgCJ/sCAZh2mRJON6S+1PMyIQld6ad52XCeel01W4cD9dvJDq9csqb8elekBw1+iDr3hs3LJMmK6PYepOVWcK1VWdl35pOuJinAs5f7fJrJfhH3OLIrnGfWvGYA5Rsi+mI9gr3Hp8ClRhgx0oYVkMS9+s+2B8iVaaOw9bYjgge 7jwc0PaZ P1K9hgShcED6T9fwPIYFr+zmL/L2/TN4z3PEcnIj4lThB2onGmsTjVT8B0bEDWu6yiMG0VmS2+hYmO4LuDTWoVXyB/KQA6ptGR1RNrfTM3aOtjsqxcQvXU6ks925225Yic4MfV3XCE5ehlhRcRF3b0bmHnY+55LunuWzjdzNLZpm7Zxc+T6TzYg+wIkjl647V+GZQHeo3ZTC++h4IJFMV0uGEb1ojC0BR22ETRlmmchZviRzFhDxStNfmIj97mGMH8scDXeXQEWmMgoCC/q279CTduY8RfMVrrz+IG9wIRwq7+4WauZf3/6mVC8ccYPTYCuxoR3TWCg3E32cQQSfb7AHLI4qmQ7P7fjBFIqhG9U0agzbxEIYndsNb8kLiTjI+scAYYiXPfFoNeAETLJktJuCF+a40D0XYe2Vt+Uu9zBToOXwMKdJlBkrCK00gX9X9n/IlxJ6zvRl83uk4FID93Ad1BwA2sxrqQFDsi2p9dtEFnX0upzxUuN0jiIrZp0ORj5LSFTsw7cd0oEFHvawMDoKCiKp89O7Q7H8vnNUYg2n63/54b58eNHsEluZtW40WvwqIWG2aF5b84/GE1O87eGU3CHox1Kk4VYUNojpwORP1rtR50oipvRxIEiTQU0fDKgEnwYR5aIPa52hY00aSEEbvHv0FhNtrHAW5pa7eWmkC/ObXbZ1u5p76IRhIr+VGvkzI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf We now have all bits in place to support KHO kexecs. Add awareness of KHO in the kexec file as well as boot path for arm64 and adds the respective kconfig option to the architecture so that it can use KHO successfully. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- arch/arm64/Kconfig | 3 +++ drivers/of/fdt.c | 33 +++++++++++++++++++++++++++++++++ drivers/of/kexec.c | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..c997b27b7da1 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1589,6 +1589,9 @@ config ARCH_SUPPORTS_KEXEC_IMAGE_VERIFY_SIG config ARCH_DEFAULT_KEXEC_IMAGE_VERIFY_SIG def_bool y +config ARCH_SUPPORTS_KEXEC_HANDOVER + def_bool y + config ARCH_SUPPORTS_CRASH_DUMP def_bool y diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c index aedd0e2dcd89..73f80e3f7188 100644 --- a/drivers/of/fdt.c +++ b/drivers/of/fdt.c @@ -25,6 +25,7 @@ #include #include #include +#include #include /* for COMMAND_LINE_SIZE */ #include @@ -875,6 +876,35 @@ void __init early_init_dt_check_for_usable_mem_range(void) memblock_add(rgn[i].base, rgn[i].size); } +/** + * early_init_dt_check_kho - Decode info required for kexec handover from DT + */ +static void __init early_init_dt_check_kho(void) +{ + unsigned long node = chosen_node_offset; + u64 kho_start, scratch_start, scratch_size; + const __be32 *p; + int l; + + if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER) || (long)node < 0) + return; + + p = of_get_flat_dt_prop(node, "linux,kho-fdt", &l); + if (l != (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32)) + return; + + kho_start = dt_mem_next_cell(dt_root_addr_cells, &p); + + p = of_get_flat_dt_prop(node, "linux,kho-scratch", &l); + if (l != (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32)) + return; + + scratch_start = dt_mem_next_cell(dt_root_addr_cells, &p); + scratch_size = dt_mem_next_cell(dt_root_addr_cells, &p); + + kho_populate(kho_start, scratch_start, scratch_size); +} + #ifdef CONFIG_SERIAL_EARLYCON int __init early_init_dt_scan_chosen_stdout(void) @@ -1169,6 +1199,9 @@ void __init early_init_dt_scan_nodes(void) /* Handle linux,usable-memory-range property */ early_init_dt_check_for_usable_mem_range(); + + /* Handle kexec handover */ + early_init_dt_check_kho(); } bool __init early_init_dt_scan(void *dt_virt, phys_addr_t dt_phys) diff --git a/drivers/of/kexec.c b/drivers/of/kexec.c index 5b924597a4de..db7d7014d8b4 100644 --- a/drivers/of/kexec.c +++ b/drivers/of/kexec.c @@ -264,6 +264,38 @@ static inline int setup_ima_buffer(const struct kimage *image, void *fdt, } #endif /* CONFIG_IMA_KEXEC */ +static int kho_add_chosen(const struct kimage *image, void *fdt, int chosen_node) +{ + int ret = 0; +#ifdef CONFIG_KEXEC_HANDOVER + phys_addr_t dt_mem = 0; + phys_addr_t dt_len = 0; + phys_addr_t scratch_mem = 0; + phys_addr_t scratch_len = 0; + + if (!image->kho.fdt || !image->kho.scratch) + return 0; + + dt_mem = image->kho.fdt->mem; + dt_len = image->kho.fdt->memsz; + + scratch_mem = image->kho.scratch->mem; + scratch_len = image->kho.scratch->bufsz; + + pr_debug("Adding kho metadata to DT"); + + ret = fdt_appendprop_addrrange(fdt, 0, chosen_node, "linux,kho-fdt", + dt_mem, dt_len); + if (ret) + return ret; + + ret = fdt_appendprop_addrrange(fdt, 0, chosen_node, "linux,kho-scratch", + scratch_mem, scratch_len); + +#endif /* CONFIG_KEXEC_HANDOVER */ + return ret; +} + /* * of_kexec_alloc_and_setup_fdt - Alloc and setup a new Flattened Device Tree * @@ -414,6 +446,11 @@ void *of_kexec_alloc_and_setup_fdt(const struct kimage *image, #endif } + /* Add kho metadata if this is a KHO image */ + ret = kho_add_chosen(image, fdt, chosen_node); + if (ret) + goto out; + /* add bootargs */ if (cmdline) { ret = fdt_setprop_string(fdt, chosen_node, "bootargs", cmdline); From patchwork Thu Mar 20 01:55:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D836C36000 for ; Thu, 20 Mar 2025 01:56:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E65C428000F; Wed, 19 Mar 2025 21:56:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DA32B280001; Wed, 19 Mar 2025 21:56:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF12328000F; Wed, 19 Mar 2025 21:56:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9DCCF280001 for ; Wed, 19 Mar 2025 21:56:23 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1BD401C6DD3 for ; Thu, 20 Mar 2025 01:56:24 +0000 (UTC) X-FDA: 83240264688.12.9E67A43 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf07.hostedemail.com (Postfix) with ESMTP id 5AB454000A for ; Thu, 20 Mar 2025 01:56:22 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MxkSBlT7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3xXXbZwoKCGQEJCPIaWCPNIQQING.EQONKPWZ-OOMXCEM.QTI@flex--changyuanl.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3xXXbZwoKCGQEJCPIaWCPNIQQING.EQONKPWZ-OOMXCEM.QTI@flex--changyuanl.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435782; a=rsa-sha256; cv=none; b=u6LXqNAsQHxawEKQdv7ry4CrVQco03VnNKXuWzJoS+wrgG1dZc0OP3CkR90NkXXHV/yfzD mbXf47+6YKcQyKWTcXhJKeq7AAKuD6zb4vQv6tJWaA7tJ6Uj6Yc9BR9L9sYwJiAROblz9O 6x+F8Kj89ZnDCZe59WtP3nG0bOAqVuw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MxkSBlT7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3xXXbZwoKCGQEJCPIaWCPNIQQING.EQONKPWZ-OOMXCEM.QTI@flex--changyuanl.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3xXXbZwoKCGQEJCPIaWCPNIQQING.EQONKPWZ-OOMXCEM.QTI@flex--changyuanl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435782; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q4HakJ1xIrca+h2lsnqHQuPoA99HtUJXt7bntXeIsss=; b=Zm4kwTxU+Eys+cT7ZwIvMSi/1KDid5JZs9gSpng1flX0/HIEh0c1QpxIkBS5zBTb7a9SLo F5hCV79Ub13S/7V+HeTTn5trZt/X+2Dv+X+EWvione3jP1wmSDg65p5b0sF2OGLDjXAEte vmuYJfgK79MclDN4xO2pvWuBZiynBac= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-224347aef79so4644155ad.2 for ; Wed, 19 Mar 2025 18:56:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435781; x=1743040581; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Q4HakJ1xIrca+h2lsnqHQuPoA99HtUJXt7bntXeIsss=; b=MxkSBlT7WA0m71S6BHprCyYRiwOExiM4DwrTWhgjDoCGPvE3g2h0zRof/Sr4BOHbQ+ SSPW5uZzXc3o5Z4XhDXNlH9cJthrndGlJPY3uw9bFxZuaTkentGvFFrNdKVXVKni168d t+qyT0dI5DOqpi44R4wRvB3YYaMj2qLnh4zmNa/xozm8KRUfQaxPvpF/uhrPizi6+bE8 yLEXuK7KLO4jAwPEOd1BgGbzFFOrZTKa376nyryfTSMxCXrWFPQHTefdRan1FJpzCtbj uqe4nA2ky9usJ3GYKIHMCYwNEJGmycP14in+dKJRWUPTCkRRc74C1LNPpgkryGxiSj2R UC9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435781; x=1743040581; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Q4HakJ1xIrca+h2lsnqHQuPoA99HtUJXt7bntXeIsss=; b=C4CfRNlUXVAgGFc9jd6OvDhu3un0IbvqtedVxDSwKHlKHjOIFJozVBNMd9rYLqWmkD bgZuZcFflzZPtTk+bVSc4ONloJa30GgPcBDcj6Ms0zAkYkWGx/ho65JhbXFdCR3mHCeO vWBCe5jO8Ew6NuNK4QWRbTqw408ESoUaNMt5szMfCsC6snpEY4GS7J8nxx9PC2V27DHw UHxuM+bsjkpKSmIiQ6rXY9PyvClsWDAxaAZcO00TNtSDNdYNHQJeG1yAu99pCPUY2DSu lAr+3V/VGM1isgG+LEqB2mP0RS4acUrnuJQiMBvGF8YnGdqZiS8A+M4kOwAA1757S3fW rehw== X-Forwarded-Encrypted: i=1; AJvYcCXWSX8zJDb5zcwznllHb1s783gU7+i4ZrFr69HNf0IuCDiOzYn4w13aZN8T31OCS2yfzZm4VL9a+g==@kvack.org X-Gm-Message-State: AOJu0Yxkrx1qqOMhzgYFrDZwFvnXI13ckmB4mJw10hvLDgh0iwEE6Bnl RZ/tdUB/clHTuWzDAVT8G6iswFGBeuCjSTrcU+kqtK2UHq88Xjpu9pHvHIz6zKPPQ5psNeU3gA/ DMvR7N7rBUXA1bV/QGQ== X-Google-Smtp-Source: AGHT+IGeUFjJTWpK4QjhVVj4506qhZmgM6+vP3bTkcGXoNreK6tGzf61M1Gbm1yeqL/C/F63sl0IwvGvOY21BMKy X-Received: from plof5.prod.google.com ([2002:a17:902:8605:b0:21f:56e1:c515]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:f548:b0:21f:b483:2ad5 with SMTP id d9443c01a7336-2264993273bmr61032515ad.20.1742435781232; Wed, 19 Mar 2025 18:56:21 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:48 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-14-changyuanl@google.com> Subject: [PATCH v5 13/16] x86/setup: use memblock_reserve_kern for memory used by kernel From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org X-Stat-Signature: hztkww7x1ojbc4c4erskwa7pqck3ywng X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5AB454000A X-Rspam-User: X-HE-Tag: 1742435782-523393 X-HE-Meta: U2FsdGVkX19voyeFrhfw8FjZsUplqADNR4Ga63TjwM33AE3ZUd43jm153VBeuLCZ181jWVUEUdg1/V9VrZsU1eaCZZ+mlMip6EkwjdNhAMCIbwMKYReVlEKOR4oEWiHB+slyGizJqqoW67HznYZcE0S0sy2H9YCuCVawk/Ck4c5w8I7wXHhPPtBG5xv/O5GzVlUjtKa2NwU/6C8ML/r84aX7OaFioiyqgr0cLNUyTXYjKq3BBhpsgFo/hEzrpGoczdVQINpIwYABY31WKvAbdnG/aijSbn8EdKLti1bB8Ni/x13ylu/HO6Q1TZMPru887CE8sjZI2SLzqxDtzLDPwEgVNKUFQV0lUO7KROgYPNTewrwWnSvoMsboYU0HH8JLU9ARm4e1876ZXc5yGXCrk4pCRmwIOCEHu1dgCRC5v5+usqO4upZ3PwfKYzd59Zp+XwoUMOba92YTIOTR5oxa0D3KQ30wY5ShT+4WgH/NLtLX2CC/89nc248AGk6MSvMR+sn71btlMq3k6IfI7liFQvIVt+MBeN79xSuXo6YHtqQo1KgLS7AkzIuBot5nZD04++7YJ8pfP/E78dTerFq7PnpXg8AOWBNtIfPjXz37d4aPiXWhVw4Lmi0TM4EFvTPkp5RVnGQHEoeYlvfKRIoH3c01cAnWHGfHQMyhw5iU5kJJThq/+4T33w5mkIBcznVhhpbu5bRCqnAfwa0TI6rUKw9bjP5pmyAssbdl6FOqpoulaLbr2E5B6vVJVtTEEf5N7ba5qoV3WQSjrgGrzryPZvA2dBFBmWU98Qipz1rODrhrXrQezVdItLfeISABMIeLk4vPAU3ji7ITVgx3re6Sso8+jHjDhTw88hkqYPnBCfv8vsCV1Oh9C7C1Qv7jflyT7SjWSViF6XFUjXJGs4UU2bHpgqwuoTFHTUsLq/bZQm49EhjJYZWNAedx2B4yFjBE0nvGmUML6i/9TVls63E x99ycqan MnMAnC9RzIlurJWuIgHnlXXLSpqb5lmI1r/pz5SqX/Wet71IjfzRGJqJq4lAvDm6fuPTTafpEFceikc6U1+q/Y74gxZWf1Lilf4vR968wCTE+ytwqqJQQpE03eIpoKqogDMGcp2dJj8aVugsghPfoyER1Y/et5LreA+vPr/ZKTt5Tl4PivVWcbTd6nauC+0FyusMiE6CbG5+XnQ/el8TjKo1Imz13zAqfv0qm4p/aYN8g1PhrUOsMnmgMBpTyGoVLA1Bbl/hIEwStphq0Yvtmdzh1LNdR3NS17zJ4AsEYqNWXUcHG35G2Gdk6d4pIaIHt83/nU1Ycb7myvMgwPXYDDeoJAxGlGuitBs6s5Ys+s6j/G8QDvECc5ajh4lY8Z2uX65sAWhuXigFvWSOByV8WeWkHecB0JI/CfHM1hiDiA4+NdMKR1u15IBjJvpOtVdgMG115KxBDqekCpjaJnPOSxmVdkzsXxNwKKICnSRL23xFurXJ5svpDjk59hriDV+FduxOKszFGEy82Mfrsq7Jk5Teg4q9t5lMe2Pjxv0NP3eG0UoQ30MzJqrOIZEfveTnEU0yfGMTVdkG7sjcGi/b5ZBIdYnbTn76MKu3pmq2UNmwhtud/WnlwpIfHBdkQL+9v7XxDW7XikSaEYArJwCObxnypyBTKar83Hsk4yEugEcVWLCG0Vi2PLNhnCtATxOmSzqf4RAEhh2Tcdphc7rQPHkN5OSaKewZSWX49 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" memblock_reserve() does not distinguish memory used by firmware from memory used by kernel. The distinction is nice to have for accounting of early memory allocations and reservations, but it is essential for kexec handover (kho) to know how much memory kernel consumes during boot. Use memblock_reserve_kern() to reserve kernel memory, such as kernel image, initrd and setup data. Signed-off-by: Mike Rapoport (Microsoft) --- arch/x86/kernel/setup.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index cebee310e200..ead370570eb2 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -220,8 +220,8 @@ static void __init cleanup_highmap(void) static void __init reserve_brk(void) { if (_brk_end > _brk_start) - memblock_reserve(__pa_symbol(_brk_start), - _brk_end - _brk_start); + memblock_reserve_kern(__pa_symbol(_brk_start), + _brk_end - _brk_start); /* Mark brk area as locked down and no longer taking any new allocations */ @@ -294,7 +294,7 @@ static void __init early_reserve_initrd(void) !ramdisk_image || !ramdisk_size) return; /* No initrd provided by bootloader */ - memblock_reserve(ramdisk_image, ramdisk_end - ramdisk_image); + memblock_reserve_kern(ramdisk_image, ramdisk_end - ramdisk_image); } static void __init reserve_initrd(void) @@ -347,7 +347,7 @@ static void __init add_early_ima_buffer(u64 phys_addr) } if (data->size) { - memblock_reserve(data->addr, data->size); + memblock_reserve_kern(data->addr, data->size); ima_kexec_buffer_phys = data->addr; ima_kexec_buffer_size = data->size; } @@ -447,7 +447,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) len = sizeof(*data); pa_next = data->next; - memblock_reserve(pa_data, sizeof(*data) + data->len); + memblock_reserve_kern(pa_data, sizeof(*data) + data->len); if (data->type == SETUP_INDIRECT) { len += data->len; @@ -461,7 +461,7 @@ static void __init memblock_x86_reserve_range_setup_data(void) indirect = (struct setup_indirect *)data->data; if (indirect->type != SETUP_INDIRECT) - memblock_reserve(indirect->addr, indirect->len); + memblock_reserve_kern(indirect->addr, indirect->len); } pa_data = pa_next; @@ -649,8 +649,8 @@ static void __init early_reserve_memory(void) * __end_of_kernel_reserve symbol must be explicitly reserved with a * separate memblock_reserve() or they will be discarded. */ - memblock_reserve(__pa_symbol(_text), - (unsigned long)__end_of_kernel_reserve - (unsigned long)_text); + memblock_reserve_kern(__pa_symbol(_text), + (unsigned long)__end_of_kernel_reserve - (unsigned long)_text); /* * The first 4Kb of memory is a BIOS owned area, but generally it is From patchwork Thu Mar 20 01:55:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023314 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3374C35FFC for ; Thu, 20 Mar 2025 01:56:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5B7F280010; Wed, 19 Mar 2025 21:56:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE35C280001; Wed, 19 Mar 2025 21:56:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A38C5280010; Wed, 19 Mar 2025 21:56:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 82D80280001 for ; Wed, 19 Mar 2025 21:56:25 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EE044C08FC for ; Thu, 20 Mar 2025 01:56:25 +0000 (UTC) X-FDA: 83240264730.20.5965A82 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf20.hostedemail.com (Postfix) with ESMTP id E84CB1C0002 for ; Thu, 20 Mar 2025 01:56:23 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0YFOOCSo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3xnXbZwoKCGUFKDQJbXDQOJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3xnXbZwoKCGUFKDQJbXDQOJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--changyuanl.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435784; a=rsa-sha256; cv=none; b=bmvgMnnlcBEZcA3DHDe0ervZF1kJzeB4IlbGT1jgN/s0201qKVcJDlfNZJwzgMvHZb7lNs HJtl7SPn2oPUT0B+YCsTpX7HJOP3yw+RVxeU2NNg225K0aOsanNswoXq1U1tWFhOqxO5nW PTtrKY6mcm6gypniJceBG60xExPxqP0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0YFOOCSo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3xnXbZwoKCGUFKDQJbXDQOJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--changyuanl.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3xnXbZwoKCGUFKDQJbXDQOJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--changyuanl.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QmV5BWqKnlxPhrRMqlARz/TZT0WTjCpLvcGNlHyxpKs=; b=QRfALLq2NMfg7XCsxX3ix6bi+dW1c7bQnMBx+W1Ks2MmdDJPa8HdVp9tW729qmBElVuWhw OvQRAVNIciN0VNj17Q9BqOpHrL/+EhP6deZNoEr+KJyZLqxHWURa2EPTeIVvxXhkm8VDE7 bqwwHt7uDZlgNcUIw0H8Ks8KhI1oxW4= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ff6167e9ccso701835a91.1 for ; Wed, 19 Mar 2025 18:56:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435783; x=1743040583; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QmV5BWqKnlxPhrRMqlARz/TZT0WTjCpLvcGNlHyxpKs=; b=0YFOOCSoQzwu8JX5ESUc95n2BSGex+V5TD7apBGv1y/G+TrPQV39h6Qzk4Txtte4CX WpCu78eHZlSe/EM7b1knSA1ZEFwpbWWx8alDmE8+KcIIWhPtfBZmSkwygszupQ9V4niO RUKA9mJ9ZOtprWdLCvDO6FteOR5pBTWTwrKhNjIpyuO92m7W0g5DJBP9r0UN0dSnIu0r 0UcVvuRts7Hbw4jk2LxhBs2ulLePtMVEHwnIrhWmMl6yUVAdp5FJHnH3jG9Mads2bep7 GbEOLGAprf4kHU+juYsA7O/A9Kj1jggbDl+/6HTvP3843Yn90jRTtiH00nXZI5VqOqGW wPRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435783; x=1743040583; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QmV5BWqKnlxPhrRMqlARz/TZT0WTjCpLvcGNlHyxpKs=; b=daw9ey1l8JvLEE8gBPSrOwJLVpdsJsegpB4Uz1EBQqmYkP+AadTMlFngT5CyursxMN a5QomkZnlUf6HT5Lamu2uLFZmki1mQ4Kg+vzMgU5sJlQJetygKNqxCG/Bdn82vQXTZoH C3Y2uHCqUzA3/lI+FscGl155z30hl8z1tl8tipppbPagplyAdPfO4WtIcC1CKGYi2jSx TiUTge4eSweRtw8NrcHdQpoxPOtq7iSUM98ED09ISOEF6AFMAmr6wQ8A0TKv6/F9gKKL JYX475n+2ouVIQ/QfIAAe2ZsQ+/GupG8I6AlIrOzpqWtScqYHoG8t2bkrcnFawUkxjLS sCaA== X-Forwarded-Encrypted: i=1; AJvYcCVkid4q1ZWiRdBiVNvfW4gJO2eI4az46yHJhLQZzoS3VKo1pKuRmBm/118G0SGCHToSzBsG5d8D7Q==@kvack.org X-Gm-Message-State: AOJu0Yx9S02jEB+4c4HNHAqdJ0n4nr1NRrWSAHDcKttslL6Rk3BHBEnW ysS9G0XhWBS1TaiwjUFU6Je7yQfmvoGjjvR/i/n/RGo9tKH5jcrlmhofpvqJETjYHHR0L6y04uT EUyfLvuamoBgvJtf4aQ== X-Google-Smtp-Source: AGHT+IFcz6xgCpsdaGMV/lA4NVDUFar/eMRSs8b1l17t60oWu+KTL3OhVFDVjETsYQaFDiZdylma2/Q1A+VwCJiL X-Received: from pjbrr14.prod.google.com ([2002:a17:90b:2b4e:b0:2ef:82c0:cb8d]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1f86:b0:301:6343:1626 with SMTP id 98e67ed59e1d1-301d507fab7mr2148203a91.1.1742435782874; Wed, 19 Mar 2025 18:56:22 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:49 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-15-changyuanl@google.com> Subject: [PATCH v5 14/16] x86: add KHO support From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Stat-Signature: cmqiumxffbpq84zxsxetkjmpn7hzhytg X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E84CB1C0002 X-Rspam-User: X-HE-Tag: 1742435783-267189 X-HE-Meta: U2FsdGVkX1/k8OagJV0NL4wY3N516z2p014zydONWQcoDbdrlyMQ4kc/GHX7HjXir8IQb5BqFx0plpSiQRf84MGR8SjR1cevwV4+QQszujnvN50frqHeCx10Iz7mnoPdn3VQejxByVg4p/9liV2TdsYiZvBahN5jfasYFaZb29N+ej8HgcUuz7JKony5T8B7zzEZKqGtUj0wieaTliLs2TrXVGxxL2aDTH1ZgTLxTp0GphqNioUkIr/Aq0w7hSYPBMtfaKSqVpnwQFz4qoYJrSriC/CyWQasJLFzIwgoButtAF4AnW5HqRhsMaIG+DStPKi4nr2cJnqWa2YIWFfiWlFpkO+nGE/PoOz0S1UP/53+9cs1y0YyKj88IfctV02dtWRIlxmLtvbRAlmmD+aOMelCKDIx8YixkQfyYZ1FkFdvz4Kup2pMihdUQm06dMSMcWfGHxcy1ga3vU49AgW9DfhNS9YfkyT9eQftDoXLlsL6Bm5C3DS2kRfuKlJxQzWOt338E9FRw0XX4Ys6/8O0cqXtXnAPelVWiXubI6dOsnO3wcaSgnqyiTWI2bbDBxv2/zHrnGkaiw6Ri6/SB7wH+7kyid1bS3ciVkTk7Kx/fidgSn23C4u2FxhE8mTyLsgaZmTGonY0xLjo7UukNPBxNAzSsfo3zTkIxcLdgeDYTK0njzwMjFwHlhybIyofj97xJepnWI2vtAg8Zoh2Gbf7pr4uEkF0p75MQPT4cEnZ906pz5TNQdPRo1CKjh51mahkENgkEZwvr2y7SOy/M1MyYV+tgFxHG7ZKTAMeIM7QRMBj3KR/H/zbMI+ZgYOzMtP5AaJYHO+I9UMg9tU9JkJJja7oqB3Ey1FLsn4FFNEMShrQQA9XKKI2YRKH+2pv6Und2TsLVyBZWxEUiIp2+02bK9ClK9qBt/7UjwO4xG/SZctQgSoWDmsUeoCByVjj4ea4loXfloCztKhiv8/lYIi rePbJku9 eLpcZ34ZTV2d6RrbECYTFJm5HowIkjdfO8kXAlQfkeEW9VsgDhaqjaFkC9YU8UKZwk9CsvNDg7ZkxzI+Pw2zXFMW/Qv7j25shOFasy/ftX0qQSaISlxAssOu4RB5AUAvfdBAJC4EVUTo4gNUGeni6n9C/V4s1125xarZA26EAhe+J+k2VZdyLLVZBnRZdZAtQEeXakWD+SbeFeOSvDVCXctZHJMPjehAIvSGBIHJRLaSjHXvm7o/FK+WT5sUD9nLX+gHFc/7XlBFDfW4R3DcJecLV9wLPWAB7b7Qm9ExteAlC1eUoNLw0nG4xI06l6ixccyEZC/XlDx45WN5XmK399vNRrcVuZ55pPbFUGMTcpE3QOjkwj7vl+nuODbyB61Pe1ziVMC2wgssnUEAWr0QxROc/a4Gg7EvOIYLZXrVTb1DdQlpf6YAEvWSvwnjbHf6cxR+zO6IqAoSwtkNps5he1olng1NIXPBN6/b8k+lteUn6l83sXSdYS2hoXfHKkojqeDmz+vkeJxSGpjcBs34KVzEZHXH2TFOBdZ6XZ3/soZ406jbuJA3hUtJ4Zuv35hFALSd1AozDYxSpKVby6nss2EUuazvI/wlbsilDbcaaLk1zVyhZ+JYZS65P+GeHjdO9dCSVZvSTLdwh9ipxEDRs5rtyID1oUe6lOuaoAg/4+7JJNYNssZv3MVakm1IDZP6px9Am X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf We now have all bits in place to support KHO kexecs. This patch adds awareness of KHO in the kexec file as well as boot path for x86 and adds the respective kconfig option to the architecture so that it can use KHO successfully. In addition, it enlightens it decompression code with KHO so that its KASLR location finder only considers memory regions that are not already occupied by KHO memory. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- arch/x86/Kconfig | 3 ++ arch/x86/boot/compressed/kaslr.c | 52 +++++++++++++++++++++++++- arch/x86/include/asm/setup.h | 4 ++ arch/x86/include/uapi/asm/setup_data.h | 13 ++++++- arch/x86/kernel/e820.c | 18 +++++++++ arch/x86/kernel/kexec-bzimage64.c | 36 ++++++++++++++++++ arch/x86/kernel/setup.c | 25 +++++++++++++ arch/x86/realmode/init.c | 2 + include/linux/kexec_handover.h | 13 +++++-- 9 files changed, 161 insertions(+), 5 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 0e27ebd7e36a..acd180e3002f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2091,6 +2091,9 @@ config ARCH_SUPPORTS_KEXEC_BZIMAGE_VERIFY_SIG config ARCH_SUPPORTS_KEXEC_JUMP def_bool y +config ARCH_SUPPORTS_KEXEC_HANDOVER + def_bool y + config ARCH_SUPPORTS_CRASH_DUMP def_bool X86_64 || (X86_32 && HIGHMEM) diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index f03d59ea6e40..ff1168881016 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -760,6 +760,55 @@ static void process_e820_entries(unsigned long minimum, } } +/* + * If KHO is active, only process its scratch areas to ensure we are not + * stepping onto preserved memory. + */ +#ifdef CONFIG_KEXEC_HANDOVER +static bool process_kho_entries(unsigned long minimum, unsigned long image_size) +{ + struct kho_scratch *kho_scratch; + struct setup_data *ptr; + int i, nr_areas = 0; + + ptr = (struct setup_data *)(unsigned long)boot_params_ptr->hdr.setup_data; + while (ptr) { + if (ptr->type == SETUP_KEXEC_KHO) { + struct kho_data *kho = (struct kho_data *)ptr->data; + + kho_scratch = (void *)kho->scratch_addr; + nr_areas = kho->scratch_size / sizeof(*kho_scratch); + + break; + } + + ptr = (struct setup_data *)(unsigned long)ptr->next; + } + + if (!nr_areas) + return false; + + for (i = 0; i < nr_areas; i++) { + struct kho_scratch *area = &kho_scratch[i]; + struct mem_vector region = { + .start = area->addr, + .size = area->size, + }; + + if (process_mem_region(®ion, minimum, image_size)) + break; + } + + return true; +} +#else +static inline bool process_kho_entries(unsigned long minimum, + unsigned long image_size) +{ + return false; +} +#endif + static unsigned long find_random_phys_addr(unsigned long minimum, unsigned long image_size) { @@ -775,7 +824,8 @@ static unsigned long find_random_phys_addr(unsigned long minimum, return 0; } - if (!process_efi_entries(minimum, image_size)) + if (!process_kho_entries(minimum, image_size) && + !process_efi_entries(minimum, image_size)) process_e820_entries(minimum, image_size); phys_addr = slots_fetch_random(); diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 85f4fde3515c..70e045321d4b 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -66,6 +66,10 @@ extern void x86_ce4100_early_setup(void); static inline void x86_ce4100_early_setup(void) { } #endif +#ifdef CONFIG_KEXEC_HANDOVER +#include +#endif + #ifndef _SETUP #include diff --git a/arch/x86/include/uapi/asm/setup_data.h b/arch/x86/include/uapi/asm/setup_data.h index b111b0c18544..c258c37768ee 100644 --- a/arch/x86/include/uapi/asm/setup_data.h +++ b/arch/x86/include/uapi/asm/setup_data.h @@ -13,7 +13,8 @@ #define SETUP_CC_BLOB 7 #define SETUP_IMA 8 #define SETUP_RNG_SEED 9 -#define SETUP_ENUM_MAX SETUP_RNG_SEED +#define SETUP_KEXEC_KHO 10 +#define SETUP_ENUM_MAX SETUP_KEXEC_KHO #define SETUP_INDIRECT (1<<31) #define SETUP_TYPE_MAX (SETUP_ENUM_MAX | SETUP_INDIRECT) @@ -78,6 +79,16 @@ struct ima_setup_data { __u64 size; } __attribute__((packed)); +/* + * Locations of kexec handover metadata + */ +struct kho_data { + __u64 dt_addr; + __u64 dt_size; + __u64 scratch_addr; + __u64 scratch_size; +} __attribute__((packed)); + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_X86_SETUP_DATA_H */ diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c index 82b96ed9890a..0b81cd70b02a 100644 --- a/arch/x86/kernel/e820.c +++ b/arch/x86/kernel/e820.c @@ -1329,6 +1329,24 @@ void __init e820__memblock_setup(void) memblock_add(entry->addr, entry->size); } + /* + * At this point with KHO we only allocate from scratch memory. + * At the same time, we configure memblock to only allow + * allocations from memory below ISA_END_ADDRESS which is not + * a natural scratch region, because Linux ignores memory below + * ISA_END_ADDRESS at runtime. Beside very few (if any) early + * allocations, we must allocate real-mode trapoline below + * ISA_END_ADDRESS. + * + * To make sure that we can actually perform allocations during + * this phase, let's mark memory below ISA_END_ADDRESS as scratch + * so we can allocate from there in a scratch-only world. + * + * After real mode trampoline is allocated, we clear scratch + * marking from the memory below ISA_END_ADDRESS + */ + memblock_mark_kho_scratch(0, ISA_END_ADDRESS); + /* Throw away partial pages: */ memblock_trim_memory(PAGE_SIZE); diff --git a/arch/x86/kernel/kexec-bzimage64.c b/arch/x86/kernel/kexec-bzimage64.c index 68530fad05f7..09d6a068b14c 100644 --- a/arch/x86/kernel/kexec-bzimage64.c +++ b/arch/x86/kernel/kexec-bzimage64.c @@ -233,6 +233,31 @@ setup_ima_state(const struct kimage *image, struct boot_params *params, #endif /* CONFIG_IMA_KEXEC */ } +static void setup_kho(const struct kimage *image, struct boot_params *params, + unsigned long params_load_addr, + unsigned int setup_data_offset) +{ +#ifdef CONFIG_KEXEC_HANDOVER + struct setup_data *sd = (void *)params + setup_data_offset; + struct kho_data *kho = (void *)sd + sizeof(*sd); + + sd->type = SETUP_KEXEC_KHO; + sd->len = sizeof(struct kho_data); + + /* Only add if we have all KHO images in place */ + if (!image->kho.fdt || !image->kho.scratch) + return; + + /* Add setup data */ + kho->dt_addr = image->kho.fdt->mem; + kho->dt_size = image->kho.fdt->memsz; + kho->scratch_addr = image->kho.scratch->mem; + kho->scratch_size = image->kho.scratch->bufsz; + sd->next = params->hdr.setup_data; + params->hdr.setup_data = params_load_addr + setup_data_offset; +#endif /* CONFIG_KEXEC_HANDOVER */ +} + static int setup_boot_parameters(struct kimage *image, struct boot_params *params, unsigned long params_load_addr, @@ -312,6 +337,13 @@ setup_boot_parameters(struct kimage *image, struct boot_params *params, sizeof(struct ima_setup_data); } + if (IS_ENABLED(CONFIG_KEXEC_HANDOVER)) { + /* Setup space to store preservation metadata */ + setup_kho(image, params, params_load_addr, setup_data_offset); + setup_data_offset += sizeof(struct setup_data) + + sizeof(struct kho_data); + } + /* Setup RNG seed */ setup_rng_seed(params, params_load_addr, setup_data_offset); @@ -479,6 +511,10 @@ static void *bzImage64_load(struct kimage *image, char *kernel, kbuf.bufsz += sizeof(struct setup_data) + sizeof(struct ima_setup_data); + if (IS_ENABLED(CONFIG_KEXEC_HANDOVER)) + kbuf.bufsz += sizeof(struct setup_data) + + sizeof(struct kho_data); + params = kzalloc(kbuf.bufsz, GFP_KERNEL); if (!params) return ERR_PTR(-ENOMEM); diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index ead370570eb2..e2c54181405b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -385,6 +385,28 @@ int __init ima_get_kexec_buffer(void **addr, size_t *size) } #endif +static void __init add_kho(u64 phys_addr, u32 data_len) +{ +#ifdef CONFIG_KEXEC_HANDOVER + struct kho_data *kho; + u64 addr = phys_addr + sizeof(struct setup_data); + u64 size = data_len - sizeof(struct setup_data); + + kho = early_memremap(addr, size); + if (!kho) { + pr_warn("setup: failed to memremap kho data (0x%llx, 0x%llx)\n", + addr, size); + return; + } + + kho_populate(kho->dt_addr, kho->scratch_addr, kho->scratch_size); + + early_memunmap(kho, size); +#else + pr_warn("Passed KHO data, but CONFIG_KEXEC_HANDOVER not set. Ignoring.\n"); +#endif +} + static void __init parse_setup_data(void) { struct setup_data *data; @@ -413,6 +435,9 @@ static void __init parse_setup_data(void) case SETUP_IMA: add_early_ima_buffer(pa_data); break; + case SETUP_KEXEC_KHO: + add_kho(pa_data, data_len); + break; case SETUP_RNG_SEED: data = early_memremap(pa_data, data_len); add_bootloader_randomness(data->data, data->len); diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c index f9bc444a3064..9b9f4534086d 100644 --- a/arch/x86/realmode/init.c +++ b/arch/x86/realmode/init.c @@ -65,6 +65,8 @@ void __init reserve_real_mode(void) * setup_arch(). */ memblock_reserve(0, SZ_1M); + + memblock_clear_kho_scratch(0, SZ_1M); } static void __init sme_sev_setup_real_mode(struct trampoline_header *th) diff --git a/include/linux/kexec_handover.h b/include/linux/kexec_handover.h index d52a7b500f4c..2dd51a77d56c 100644 --- a/include/linux/kexec_handover.h +++ b/include/linux/kexec_handover.h @@ -3,9 +3,6 @@ #define LINUX_KEXEC_HANDOVER_H #include -#include -#include -#include struct kho_scratch { phys_addr_t addr; @@ -18,6 +15,15 @@ enum kho_event { KEXEC_KHO_UNFREEZE = 1, }; +#ifdef _SETUP +struct notifier_block; +struct kho_node; +struct folio; +#else +#include +#include +#include + #define KHO_HASHTABLE_BITS 3 #define KHO_NODE_INIT \ { \ @@ -35,6 +41,7 @@ struct kho_node { struct list_head list; bool visited; }; +#endif /* _SETUP */ struct kho_in_node { int offset; From patchwork Thu Mar 20 01:55:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4645C36002 for ; Thu, 20 Mar 2025 01:56:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04712280011; Wed, 19 Mar 2025 21:56:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F116B280001; Wed, 19 Mar 2025 21:56:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D64DF280011; Wed, 19 Mar 2025 21:56:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B4E2B280001 for ; Wed, 19 Mar 2025 21:56:26 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3410AC08E1 for ; Thu, 20 Mar 2025 01:56:27 +0000 (UTC) X-FDA: 83240264814.03.C6BB0D4 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 978F580002 for ; Thu, 20 Mar 2025 01:56:25 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KICoNkro; spf=pass (imf30.hostedemail.com: domain of 3yHXbZwoKCGcHMFSLdZFSQLTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3yHXbZwoKCGcHMFSLdZFSQLTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435785; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+uiHbtbR2+LTCG9tPghyUo9+PEq69RRB7eOXIX8kmmQ=; b=cFo5P5X3bUAjV32D8+mlAlNWy2YNfrGMioCsoVNeD+HH77RQNZLS469hWMB10LHVaTdsrf iHrAB3Z/8mW/ypA8mgRFovNtmAWuO3JejBx2+U+Ec+5YbbkcMR4uCfDC5Zt1SfCJkm9qAE LKN9vP/m2Ue41gEmNhhAh0NGS0iLvMk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KICoNkro; spf=pass (imf30.hostedemail.com: domain of 3yHXbZwoKCGcHMFSLdZFSQLTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3yHXbZwoKCGcHMFSLdZFSQLTTLQJ.HTRQNSZc-RRPaFHP.TWL@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435785; a=rsa-sha256; cv=none; b=aChAUFmxyxYchEyzUk9epCgpKJurSttdToHEya9dWAvYfKEjx8FIe5A2IAkQUEMMiKbU56 BEEAmavqLzSlWVZh5QINsh359+2MvilYEmoFl89GFJPdPK9CKaszr/TfRUAeJ/sqtzJHnQ 89u3LRPwZlDawv9YXoJnOEa02EszX0Q= Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2255ae39f8fso26184295ad.0 for ; Wed, 19 Mar 2025 18:56:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435784; x=1743040584; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+uiHbtbR2+LTCG9tPghyUo9+PEq69RRB7eOXIX8kmmQ=; b=KICoNkroLqPmO/4SDmOgwynUy6CiOLsC6pk2ezM7zF/B3g4O1cx6seFT5LfLhh299m et/DYQee+zYFX+IxWN4VDSoiBCJ/aSNMsQoOQI4kP5j6+npCV4i3miNTA301t9fjiyOE rdk60P5U7WNio+Fhk24Xt6XEPWovs4/3w8A9ZHO212GYpNNzpKcyWuno/MbH/RCnzk9p QG+Fedr/LxfD32NeQF9riXDRNI2r3s5gW3B6KNe7iJYk2ZVD92R9bSh2QI2FnsrSzf+p +a5KduR/yxHzAqN9/NKcgZP6YXorXZmwn8vEj9HNPV4+Zg3ffR8n4BU1oIlv91bNhBbz Q1Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435784; x=1743040584; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+uiHbtbR2+LTCG9tPghyUo9+PEq69RRB7eOXIX8kmmQ=; b=FKXkdq5oM+HMsOVzxEPv/adJEsyq+C3kx6pFtCkRE6ZFoNqiyeVj7zBZPSuE6RsdpF wfNV7finzCkSnYuVkR0ZQLRVSkjTMo0esSIO7Ywa/MU22X4EFdZcJCNiG92UiRv4anlY /Y10gbAwJ/fNrbEtE4XB1KPVd6E5WWZ91dyZFl7TQblyhgio+UdKIVNmmea6Ostoyw8X 5Xutn1Mj451QLQqTHWsfGbNuZbxoe5khZSvhX5QFwRyu5JQOCjS15mErx064VXgcAyjJ uTC560Iv+fHS5lhBwkQ8w+yutO1PZgeN5DmIbMjko070pVBt1DvaKaOr+mGdY0kxDaEU wUcQ== X-Forwarded-Encrypted: i=1; AJvYcCWe1jV6mIyAuju6zqFIjaHOG2/HgMUo7UTUCZJyF4YcM+WUkAahxRf3vLQ8g0ytGZmLWfbiIQdgLA==@kvack.org X-Gm-Message-State: AOJu0Yzd51RzYgwiOYw1AHAG3dO3T9o5iflAlf9L0cB+c3gm0vKE9ZOL 9QksAYfKz9b8CwSX8l4JUI8sc0Ujc6o1TSxuJkGNQCyvNTpkayc2ea536nGgnKSwMfTC+ZjHscW 4Q75B08iPunv24/Rhkg== X-Google-Smtp-Source: AGHT+IGco0TbFvrpWaz7RU09LVDLIbrwQOB0hzzeh7i66UCW7yrWSDQtXXKFJgzbGrQWcK/LzqGUCqggvCd+K/bd X-Received: from pgvr15.prod.google.com ([2002:a65:60cf:0:b0:ad8:bdc2:8a33]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:69c:b0:1f5:8bf4:fde0 with SMTP id adf61e73a8af0-1fd0904c18cmr2493103637.9.1742435784427; Wed, 19 Mar 2025 18:56:24 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:50 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-16-changyuanl@google.com> Subject: [PATCH v5 15/16] memblock: add KHO support for reserve_mem From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Queue-Id: 978F580002 X-Rspamd-Server: rspam08 X-Stat-Signature: mi8umushbpj4ygf44487qrfsxc6r45jj X-HE-Tag: 1742435785-450515 X-HE-Meta: U2FsdGVkX1/zxSosrJFEMya2LjWIP7hwjpafGCjjuQ4h2uEF0JpAAL8O7JXAO0QT9UgeO8wC3O17uyoJImHzLcyO+KAdrVRprcsfyIaMq/Dk3yhGNcKiUQSBin5HVhLdYC/vQ6Nrzj2CJsVdNn5pPh+nrwcvPjvKeKGUxIra0ET449l5oa2jKK3eZv5LCjsRdwhl7ZvfoBk5CdtBhOuCsfU8jHVbwI494RIgoSI/kwap1bIaSq8nmn4c6pkEXj2ryHpay62f04Ac/YzfPd1qFXdTb9GbkUPxx3dFlCJ9U+iEdkup28tNX7w4HzeAoHDRYpjqLPcfL7nyR0BUtCuwc9QrzhDYh7hjx9QFWQRONASrJvxhAgx7T219VmHzvw+Hau7+4HsmbwY/TgRvPxCwgo+WQNDI3l/1XwZWuFj3gHqLSn8O9bhrGKbHkAxpCyvctHt36xog4Frd2OmMd1c6QtbRRFN1QvGbvVjtjRDzmP2jWCXYxului4Yp7Kzjvi1FbOhQrxYvCJscjqAPDsGRRXaKitWXM7aYVDUgucLZIWQSjSlZDeVrkHg+aDtiRVCDuZDx7s5UdLxXMrXfmG7SvXlCwSxziFqGZZoh15ipdcyqDqNiFzsVIqrYP0yk+J+S3WCjiReVknZ9mB2aEJKzvRHN2R1bNp3m+Up9T/wUG0dJIzJMiNQOq27DhzMX42zqkg2L81fNBQ4hAPhuqo//1Yiu0HSMb+j9WKuPORdUnSrvMVUrwvLS3ncZKaeFiHmRE75xzmK01MF0gZU98ahoScxh7YYdANmUTf+zAA2P1kpjq23XGU8Ne45aYCCVnjxVc1HlYgCwxqmlsLC36Xc4NLw7KgKSd+RajoL3KxS1czMWZKuGVtf7yyWvJXaOXC9RqxMw+HMglGR0B4Rqas5bcgsMQYDs5E/3cH3DB3Yh2Noyo0YYUtxeWCH9XXQxVURqsx4f+qEM7aHLjh3WazG nDMymkey SV8G4gxMsy8vjC6H23Yt0q3RA43wH9jhIjoF3iIKNyayO3N9mPT/TLIrAg0nYerf+5kq9cotXA+QnElPtNMSRmNmTabMMnY/qEB7TegDbqS2j7x4EqT0QG8NH3LP7PZWIaZuFFM99/mmcFZDa2Ex0gazaap71Z6aAoU1JW9JdBacf0FqtMg1yEp6VrSnUuN7O1/tkskjY1USrxVSAAvqlm2HldvMt9G2u1szBcs8i4Tlhyj+qnRkygGm8OGVOiFAp5Nu8n4uygYtOkN14qh7TNVGtHKTYxmdMcslhAIpiaKWNakHEGdnZ9yn50AfRFiyAAk7Hki/gj6YaVobk4h7LKb9jNY+6V/wPCs5hBA8BQmeVi1H8/yJHpIqUzJC+zzUasNYieR2RR+OyQK93DGILtj2VwlzXoUqXwnKk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf Linux has recently gained support for "reserve_mem": A mechanism to allocate a region of memory early enough in boot that we can cross our fingers and hope it stays at the same location during most boots, so we can store for example ftrace buffers into it. Thanks to KASLR, we can never be really sure that "reserve_mem" allocations are static across kexec. Let's teach it KHO awareness so that it serializes its reservations on kexec exit and deserializes them again on boot, preserving the exact same mapping across kexec. This is an example user for KHO in the KHO patch set to ensure we have at least one (not very controversial) user in the tree before extending KHO's use to more subsystems. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- mm/memblock.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/mm/memblock.c b/mm/memblock.c index d28abf3def1c..dd698c55b87e 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -17,6 +17,10 @@ #include #include +#ifdef CONFIG_KEXEC_HANDOVER +#include +#endif /* CONFIG_KEXEC_HANDOVER */ + #include #include @@ -2431,6 +2435,176 @@ int reserve_mem_find_by_name(const char *name, phys_addr_t *start, phys_addr_t * } EXPORT_SYMBOL_GPL(reserve_mem_find_by_name); +#ifdef CONFIG_KEXEC_HANDOVER +#define MEMBLOCK_KHO_NODE "memblock" +#define MEMBLOCK_KHO_NODE_COMPATIBLE "memblock-v1" +#define RESERVE_MEM_KHO_NODE_COMPATIBLE "reserve-mem-v1" + +static struct kho_node memblock_kho_node = KHO_NODE_INIT; + +static void reserve_mem_kho_reset(void) +{ + int i; + struct kho_node *node; + + kho_remove_node(NULL, MEMBLOCK_KHO_NODE); + kho_remove_prop(&memblock_kho_node, "compatible", NULL); + + for (i = 0; i < reserved_mem_count; i++) { + struct reserve_mem_table *map = &reserved_mem_table[i]; + + node = kho_remove_node(&memblock_kho_node, map->name); + if (IS_ERR(node)) + continue; + + kho_unpreserve_phys(map->start, map->size); + + kho_remove_prop(node, "compatible", NULL); + kho_remove_prop(node, "start", NULL); + kho_remove_prop(node, "size", NULL); + + kfree(node); + } +} + +static int reserve_mem_kho_finalize(void) +{ + int i, err = 0; + struct kho_node *node; + + if (!reserved_mem_count) + return NOTIFY_DONE; + + err = kho_add_node(NULL, MEMBLOCK_KHO_NODE, &memblock_kho_node); + if (err == 1) + return NOTIFY_DONE; + + err |= kho_add_string_prop(&memblock_kho_node, "compatible", + MEMBLOCK_KHO_NODE_COMPATIBLE); + + for (i = 0; i < reserved_mem_count; i++) { + struct reserve_mem_table *map = &reserved_mem_table[i]; + + node = kmalloc(sizeof(*node), GFP_KERNEL); + if (!node) { + err = -ENOMEM; + break; + } + + err |= kho_preserve_phys(map->start, map->size); + + kho_init_node(node); + err |= kho_add_string_prop(node, "compatible", + RESERVE_MEM_KHO_NODE_COMPATIBLE); + err |= kho_add_prop(node, "start", &map->start, + sizeof(map->start)); + err |= kho_add_prop(node, "size", &map->size, + sizeof(map->size)); + err |= kho_add_node(&memblock_kho_node, map->name, node); + + if (err) + break; + } + + if (err) { + pr_err("failed to save reserve_mem to KHO: %d\n", err); + reserve_mem_kho_reset(); + return NOTIFY_STOP; + } + + return NOTIFY_DONE; +} + +static int reserve_mem_kho_notifier(struct notifier_block *self, + unsigned long cmd, void *v) +{ + switch (cmd) { + case KEXEC_KHO_FINALIZE: + return reserve_mem_kho_finalize(); + case KEXEC_KHO_UNFREEZE: + return NOTIFY_DONE; + default: + return NOTIFY_BAD; + } +} + +static struct notifier_block reserve_mem_kho_nb = { + .notifier_call = reserve_mem_kho_notifier, +}; + +static int __init reserve_mem_init(void) +{ + if (!kho_is_enabled()) + return 0; + + return register_kho_notifier(&reserve_mem_kho_nb); +} +core_initcall(reserve_mem_init); + +static bool __init reserve_mem_kho_revive(const char *name, phys_addr_t size, + phys_addr_t align) +{ + int err, len_start, len_size; + struct kho_in_node node, child; + const phys_addr_t *p_start, *p_size; + + err = kho_get_node(NULL, MEMBLOCK_KHO_NODE, &node); + if (err) + return false; + + err = kho_node_check_compatible(&node, MEMBLOCK_KHO_NODE_COMPATIBLE); + if (err) { + pr_warn("Node '%s' is incompatible with %s: %d\n", + MEMBLOCK_KHO_NODE, MEMBLOCK_KHO_NODE_COMPATIBLE, err); + return false; + } + + err = kho_get_node(&node, name, &child); + if (err) { + pr_warn("Node '%s' has no child '%s': %d\n", + MEMBLOCK_KHO_NODE, name, err); + return false; + } + err = kho_node_check_compatible(&child, RESERVE_MEM_KHO_NODE_COMPATIBLE); + if (err) { + pr_warn("Node '%s/%s' is incompatible with %s: %d\n", + MEMBLOCK_KHO_NODE, name, + RESERVE_MEM_KHO_NODE_COMPATIBLE, err); + return false; + } + + p_start = kho_get_prop(&child, "start", &len_start); + p_size = kho_get_prop(&child, "size", &len_size); + if (!p_start || len_start != sizeof(*p_start) || !p_size || + len_size != sizeof(*p_size)) { + return false; + } + + if (*p_start & (align - 1)) { + pr_warn("KHO reserve-mem '%s' has wrong alignment (0x%lx, 0x%lx)\n", + name, (long)align, (long)*p_start); + return false; + } + + if (*p_size != size) { + pr_warn("KHO reserve-mem '%s' has wrong size (0x%lx != 0x%lx)\n", + name, (long)*p_size, (long)size); + return false; + } + + reserved_mem_add(*p_start, size, name); + pr_info("Revived memory reservation '%s' from KHO\n", name); + + return true; +} +#else +static bool __init reserve_mem_kho_revive(const char *name, phys_addr_t size, + phys_addr_t align) +{ + return false; +} +#endif /* CONFIG_KEXEC_HANDOVER */ + /* * Parse reserve_mem=nn:align:name */ @@ -2486,6 +2660,11 @@ static int __init reserve_mem(char *p) if (reserve_mem_find_by_name(name, &start, &tmp)) return -EBUSY; + /* Pick previous allocations up from KHO if available */ + if (reserve_mem_kho_revive(name, size, align)) + return 1; + + /* TODO: Allocation must be outside of scratch region */ start = memblock_phys_alloc(size, align); if (!start) return -ENOMEM; From patchwork Thu Mar 20 01:55:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023316 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0B78C35FFC for ; Thu, 20 Mar 2025 01:56:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D614280012; Wed, 19 Mar 2025 21:56:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93876280001; Wed, 19 Mar 2025 21:56:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DC0E280012; Wed, 19 Mar 2025 21:56:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 56C00280001 for ; Wed, 19 Mar 2025 21:56:28 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C97181407FE for ; Thu, 20 Mar 2025 01:56:28 +0000 (UTC) X-FDA: 83240264856.04.9DAB773 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf11.hostedemail.com (Postfix) with ESMTP id 3187040002 for ; Thu, 20 Mar 2025 01:56:26 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=l9eBXVkM; spf=pass (imf11.hostedemail.com: domain of 3ynXbZwoKCGkJOHUNfbHUSNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--changyuanl.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3ynXbZwoKCGkJOHUNfbHUSNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435787; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iApjQxSPhTF/tl2rkU2LC3L+Sofo1JAgj9zbjrL2yA0=; b=HvYXpe8J3L+XE9xCSnNpU9OudPTUmp0f5uSSMCKvhvrakAidZ65AhadcNjX7qrztPTljEp iouFt1ZGmwe0yL/g6uWeU3MDYfAIuBNxGcwvCEnUUXrEa/EBme7FZmZN1UmvZd6EEOfdfU Xk/OGmnDuzR+dAD2b6KliCD4t6ZO+bY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435787; a=rsa-sha256; cv=none; b=Lb1blhTGjxJxHNHCLEwS6wPGMiaucS8Bu4ZRmyKPyW3WYYClcbwL/vUbC7qILzAQA5btmx nWJVviYDiG9BvkDvTdHLmLGMQkrWjUgSDxJza+dzYbMqlV74HPmNFWjrJeqHxW0s5eLJhj 9DmTi4foGy5YyJGSCIgSnTbZVPl1AH4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=l9eBXVkM; spf=pass (imf11.hostedemail.com: domain of 3ynXbZwoKCGkJOHUNfbHUSNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--changyuanl.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3ynXbZwoKCGkJOHUNfbHUSNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2240c997059so4594845ad.0 for ; Wed, 19 Mar 2025 18:56:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435786; x=1743040586; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iApjQxSPhTF/tl2rkU2LC3L+Sofo1JAgj9zbjrL2yA0=; b=l9eBXVkMlqenI+MaYI9/dU3ubF/iiFs+MpEHlCiNOmtMBuaafLxRA1c9L4uaIs1uHT kHn2eCJlt1SORhwQmMMeiyeVkKyPQl+Oz311Suys4XS6Pt/BEJvcquoxera/uVNWmwOP ge+R+ruVcOQXwpXuuuio5wv98lIua80D7lavcNzXh2/rNRDrzz3PO4jDofPygHVWNkzr M3P/6d4dsknjXqHS8xKkYxkOjGafasSrfk+pSAQAgrCoI5kMxNNrJzFGwrneuS0bL5VZ 8tUrnQZ3xHaukeSpyVNRB94YnRHJUzqoQpS2c4U604F0A0Q7t5Dfx+2WeuVZYgUsd37n 5GyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435786; x=1743040586; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iApjQxSPhTF/tl2rkU2LC3L+Sofo1JAgj9zbjrL2yA0=; b=sWs+V/P2Wh+SxHLcIJdi32KL+nxwc5AHRHK/LlPcz3IXE2CB2xG8RRr8h3avObqDwS 8gu8xA17oCOjr6r3WS3YN9zEQXrpuK47onWcaZLDeiKnS78/koBVqT6plpWPxZERiiMu /YekDl/v2OoCHxu5pUxqSaWrafN/nYJzCjFndID/R7+gcS5chC58gbLoq31tgT6wWedF ldFhwc23+0P1AT88m4BLslVM/rBG3q4rEtGpmwrepmGCZ+ueC4AdqdieNZwFTOVdtr2V fPFmta4FwMiA6dTKrXA/WRgVR7tLgUH+hZ+gRQkBjmJHMe/MdtP7y7SvLQWCa0bcQeZH /TaA== X-Forwarded-Encrypted: i=1; AJvYcCUl3WUP2hMCt7Z9eCcoF7ivglbrsxnxf5MBkqKh0MweJxWt+IOMG+jFV1s7Uuyg0XhL64A6G1bMtQ==@kvack.org X-Gm-Message-State: AOJu0YxqtuRf7p0lRFJ4LZBTG9EvWK+LcyhzRrdXwya3fpk3Ki41w0sW NGjpLDKUMNwXKhjGteN8RoJvLRsfT9GIbm5d8LDN3KOvK8s+ABiNqKog+XAgSZiz9KBrKzuaYOL UtvgbeUxrbMdVqnBxrQ== X-Google-Smtp-Source: AGHT+IGFSfyI7p/Qh0SdAoE1uQ2Phsg//Ps3doqJu8c++ecy3AeyWqH3lvhpNMjdI0PgAE047ECxcGLwH834CeR5 X-Received: from pjbpw8.prod.google.com ([2002:a17:90b:2788:b0:2fa:15aa:4d2b]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:22c1:b0:223:fd7f:2752 with SMTP id d9443c01a7336-22649a34325mr73764035ad.29.1742435786002; Wed, 19 Mar 2025 18:56:26 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:51 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-17-changyuanl@google.com> Subject: [PATCH v5 16/16] Documentation: add documentation for KHO From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3187040002 X-Stat-Signature: gnut3y7g53a7zpsxntcc6z9r1o1rsp6n X-HE-Tag: 1742435786-4988 X-HE-Meta: U2FsdGVkX18tt7v1sv65RuKy6hh0E88+wPX+cISgpjvpQeR1wBGmhZvte+aAGBz+UTfEX24jpDoioqn2vcUtU/T4YkLbEY+hJeTExObmE2jOEtSo/ynPOZGOq9we69a8RtNgr3mU6KbQ+XfnJTClAPB3/Ve+Dk1Cm8rXXewlLcbOBg3nbH9rFrBNMFrfk+XJpS2PotEF6yYqjCpi603dp7CxuGNlVxgPpSfXEFi90j/ArSpC8tx4lApxqILaeIpZvdyIljQHLl6uqUqCWY5vXX/fc/nItREaAMsoZLIF0j2b3uJjmsW2M5WWS+Sr0zlPxHI3FALgmvoLZ20im+HviCi55oPC2YArxm+te79dux6ZZs+79+qLsJ65HROpMax8jlIeXpYUXWGa4ZCikd4RFUrdkujiIZRPX0VLHwqUmbCc4o1T/t8dsEAynyrWZ8DKkzSeNcSNL+XunGFJhtHOcW+TWf+66yNN/XE0NWWkR8UixTv9aJqqq62NIryQJ+3CtRh6ealEAB6Joij9kxzaDY/RWNTPS4OuCvhZ6bq/lNnLrwpr5WIubDhSbcfPAzu/60kTWWD0OW2jz8WcQqZs6/6Vx+d9S9Vne9CGOj7BakEaO7UtCHfeptaxqjKp8Jzbc86wK+p/jqUNuwhvJhC6OWfk7tu0RdZk2x+Pug+X4JIA3W3JPWeTZAhryI+rUK00xHPnspF+F2oHxSEtEsSOfrna/ZY41yI8oRXHSmILYpTDcWlUm13adH0qpUZytWtyQcL1Gw/4UAsGIjRPy21DCIfVfEZOKSPn3TIKxT/N9bxfCjtot7ahXKH5TmR/ajxF55pcyTk1jE1pS6xUC0aS95DqprC8dpxNYOgb0omIaHkeCEvAciK6xhaH7NGMagU3eMdU12ga28NJpIMI6HVrIB1YGrW79ACSrtFtjVj4uKuCCgronokPMXo1IgVX6RebHvtANnmFGNMj3/34mfc E4x2oRJH nUFmS67y/DLQhfD+hLc18gBUxvczdiwNTStrslmlPav7ObsgzTtX4bE2Kaeo05N2NrcruDlY9FvvUFJeR4BF8jZEoYBiDxzsDwBEj4xRnbyAWObdpxOADIEN6oUdHTeX1h/5FRxzunyTr7kl1t0koziGdpoiMbuvbalTAYFekf6BCWqsKG0zQHN7UYQER0a7xJdNfXrBCI3FIsAh4AsQF3866X4rWC3vNZPdfDNpkN+83eCEFpl8aoEgNHniCpaiHVflJqKD6KE9TMKvpxjPD6lmJebesyWMlwnGhOinFZ4e/6OfV+QWFS/qEAanX/m4j/LglGM+tM/UxHmluOR/hmF2oCbUBMNXNkM9If7z5d3tXD3U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf With KHO in place, let's add documentation that describes what it is and how to use it. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- .../admin-guide/kernel-parameters.txt | 25 ++++ Documentation/kho/concepts.rst | 70 +++++++++++ Documentation/kho/fdt.rst | 62 +++++++++ Documentation/kho/index.rst | 14 +++ Documentation/kho/usage.rst | 118 ++++++++++++++++++ Documentation/subsystem-apis.rst | 1 + MAINTAINERS | 1 + 7 files changed, 291 insertions(+) create mode 100644 Documentation/kho/concepts.rst create mode 100644 Documentation/kho/fdt.rst create mode 100644 Documentation/kho/index.rst create mode 100644 Documentation/kho/usage.rst diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index fb8752b42ec8..d715c6d9dbb3 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2698,6 +2698,31 @@ kgdbwait [KGDB,EARLY] Stop kernel execution and enter the kernel debugger at the earliest opportunity. + kho= [KEXEC,EARLY] + Format: { "0" | "1" | "off" | "on" | "y" | "n" } + Enables or disables Kexec HandOver. + "0" | "off" | "n" - kexec handover is disabled + "1" | "on" | "y" - kexec handover is enabled + + kho_scratch= [KEXEC,EARLY] + Format: ll[KMG],mm[KMG],nn[KMG] | nn% + Defines the size of the KHO scratch region. The KHO + scratch regions are physically contiguous memory + ranges that can only be used for non-kernel + allocations. That way, even when memory is heavily + fragmented with handed over memory, the kexeced + kernel will always have enough contiguous ranges to + bootstrap itself. + + It is possible to specify the exact amount of + memory in the form of "ll[KMG],mm[KMG],nn[KMG]" + where the first parameter defines the size of a low + memory scratch area, the second parameter defines + the size of a global scratch area and the third + parameter defines the size of additional per-node + scratch areas. The form "nn%" defines scale factor + (in percents) of memory that was used during boot. + kmac= [MIPS] Korina ethernet MAC address. Configure the RouterBoard 532 series on-chip Ethernet adapter MAC address. diff --git a/Documentation/kho/concepts.rst b/Documentation/kho/concepts.rst new file mode 100644 index 000000000000..174e23404ebc --- /dev/null +++ b/Documentation/kho/concepts.rst @@ -0,0 +1,70 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later +.. _concepts: + +======================= +Kexec Handover Concepts +======================= + +Kexec HandOver (KHO) is a mechanism that allows Linux to preserve state - +arbitrary properties as well as memory locations - across kexec. + +It introduces multiple concepts: + +KHO State tree +============== + +Every KHO kexec carries a state tree, in the format of flattened device tree +(FDT), that describes the state of the system. Device drivers can register to +KHO to serialize their state before kexec. After KHO, device drivers can read +the FDT and extract previous state. + +KHO only uses the FDT container format and libfdt library, but does not +adhere to the same property semantics that normal device trees do: Properties +are passed in native endianness and standardized properties like ``regs`` and +``ranges`` do not exist, hence there are no ``#...-cells`` properties. + +Scratch Regions +=============== + +To boot into kexec, we need to have a physically contiguous memory range that +contains no handed over memory. Kexec then places the target kernel and initrd +into that region. The new kernel exclusively uses this region for memory +allocations before during boot up to the initialization of the page allocator. + +We guarantee that we always have such regions through the scratch regions: On +first boot KHO allocates several physically contiguous memory regions. Since +after kexec these regions will be used by early memory allocations, there is a +scratch region per NUMA node plus a scratch region to satisfy allocations +requests that do not require particular NUMA node assignment. +By default, size of the scratch region is calculated based on amount of memory +allocated during boot. The ``kho_scratch`` kernel command line option may be +used to explicitly define size of the scratch regions. +The scratch regions are declared as CMA when page allocator is initialized so +that their memory can be used during system lifetime. CMA gives us the +guarantee that no handover pages land in that region, because handover pages +must be at a static physical memory location and CMA enforces that only +movable pages can be located inside. + +After KHO kexec, we ignore the ``kho_scratch`` kernel command line option and +instead reuse the exact same region that was originally allocated. This allows +us to recursively execute any amount of KHO kexecs. Because we used this region +for boot memory allocations and as target memory for kexec blobs, some parts +of that memory region may be reserved. These reservations are irrelevant for +the next KHO, because kexec can overwrite even the original kernel. + +.. _finalization_phase: + +KHO finalization phase +====================== + +To enable user space based kexec file loader, the kernel needs to be able to +provide the FDT that describes the previous kernel's state before +performing the actual kexec. The process of generating that FDT is +called serialization. When the FDT is generated, some properties +of the system may become immutable because they are already written down +in the FDT. That state is called the KHO finalization phase. + +With the in-kernel kexec file loader, i.e., using the syscall +``kexec_file_load``, KHO FDT is not created until the actual kexec. Thus the +finalization phase is much shorter. User space can optionally choose to generate +the FDT early using the debugfs interface. diff --git a/Documentation/kho/fdt.rst b/Documentation/kho/fdt.rst new file mode 100644 index 000000000000..70b508533b77 --- /dev/null +++ b/Documentation/kho/fdt.rst @@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +======= +KHO FDT +======= + +KHO uses the flattened device tree (FDT) container format and libfdt +library to create and parse the data that is passed between the +kernels. The properties in KHO FDT are stored in native format and can +include any data KHO users need to preserve. Parsing of FDT subnodes is +responsibility of KHO users, except for nodes and properties defined by +KHO itself. + +KHO nodes and properties +======================== + +Node ``preserved-memory`` +------------------------- + +KHO saves a special node named ``preserved-memory`` under the root node. +This node contains the metadata for KHO to preserve pages across kexec. + +Property ``compatible`` +----------------------- + +The ``compatible`` property determines compatibility between the kernel +that created the KHO FDT and the kernel that attempts to load it. +If the kernel that loads the KHO FDT is not compatible with it, the entire +KHO process will be bypassed. + +Examples +======== + +The following example demonstrates KHO FDT that preserves two memory +regions create with ``reserve_mem`` kernel command line parameter:: + + /dts-v1/; + + / { + compatible = "kho-v1"; + + memblock { + compatible = "memblock-v1"; + + region1 { + compatible = "reserve-mem-v1"; + start = <0xc07a 0x4000000>; + size = <0x01 0x00>; + }; + + region2 { + compatible = "reserve-mem-v1"; + start = <0xc07b 0x4000000>; + size = <0x8000 0x00>; + }; + + }; + + preserved-memory { + metadata = <0x00 0x00>; + }; + }; diff --git a/Documentation/kho/index.rst b/Documentation/kho/index.rst new file mode 100644 index 000000000000..d108c3f8d15c --- /dev/null +++ b/Documentation/kho/index.rst @@ -0,0 +1,14 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +======================== +Kexec Handover Subsystem +======================== + +.. toctree:: + :maxdepth: 1 + + concepts + usage + fdt + +.. only:: subproject and html diff --git a/Documentation/kho/usage.rst b/Documentation/kho/usage.rst new file mode 100644 index 000000000000..b45dc58e8d3f --- /dev/null +++ b/Documentation/kho/usage.rst @@ -0,0 +1,118 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +==================== +Kexec Handover Usage +==================== + +Kexec HandOver (KHO) is a mechanism that allows Linux to preserve state - +arbitrary properties as well as memory locations - across kexec. + +This document expects that you are familiar with the base KHO +:ref:`concepts `. If you have not read +them yet, please do so now. + +Prerequisites +============= + +KHO is available when the ``CONFIG_KEXEC_HANDOVER`` config option is set to y +at compile time. Every KHO producer may have its own config option that you +need to enable if you would like to preserve their respective state across +kexec. + +To use KHO, please boot the kernel with the ``kho=on`` command line +parameter. You may use ``kho_scratch`` parameter to define size of the +scratch regions. For example ``kho_scratch=16M,512M,256M`` will reserve a +16 MiB low memory scratch area, a 512 MiB global scratch region, and 256 MiB +per NUMA node scratch regions on boot. + +Perform a KHO kexec +=================== + +First, before you perform a KHO kexec, you can optionally move the system into +the :ref:`KHO finalization phase ` :: + + $ echo 1 > /sys/kernel/debug/kho/out/finalize + +After this command, the KHO FDT is available in +``/sys/kernel/debug/kho/out/fdt``. + +Next, load the target payload and kexec into it. It is important that you +use the ``-s`` parameter to use the in-kernel kexec file loader, as user +space kexec tooling currently has no support for KHO with the user space +based file loader :: + + # kexec -l Image --initrd=initrd -s + # kexec -e + +If you skipped finalization in the first step, ``kexec -e`` triggers +FDT finalization automatically. The new kernel will boot up and contain +some of the previous kernel's state. + +For example, if you used ``reserve_mem`` command line parameter to create +an early memory reservation, the new kernel will have that memory at the +same physical address as the old kernel. + +Unfreeze KHO FDT data +===================== + +You can move the system out of KHO finalization phase by calling :: + + $ echo 0 > /sys/kernel/debug/kho/out/finalize + +After this command, the KHO FDT is no longer available in +``/sys/kernel/debug/kho/out/fdt``, and the states kept in KHO can be +modified by other kernel subsystems again. + +debugfs Interfaces +================== + +Currently KHO creates the following debugfs interfaces. Notice that these +interfaces may change in the future. They will be moved to sysfs once KHO is +stabilized. + +``/sys/kernel/debug/kho/out/finalize`` + Kexec HandOver (KHO) allows Linux to transition the state of + compatible drivers into the next kexec'ed kernel. To do so, + device drivers will serialize their current state into an FDT. + While the state is serialized, they are unable to perform + any modifications to state that was serialized, such as + handed over memory allocations. + + When this file contains "1", the system is in the transition + state. When contains "0", it is not. To switch between the + two states, echo the respective number into this file. + +``/sys/kernel/debug/kho/out/fdt_max`` + KHO needs to allocate a buffer for the FDT that gets + generated before it knows the final size. By default, it + will allocate 10 MiB for it. You can write to this file + to modify the size of that allocation. + +``/sys/kernel/debug/kho/out/fdt`` + When KHO state tree is finalized, the kernel exposes the + flattened device tree blob that carries its current KHO + state in this file. Kexec user space tooling can use this + as input file for the KHO payload image. + +``/sys/kernel/debug/kho/out/scratch_len`` + To support continuous KHO kexecs, we need to reserve + physically contiguous memory regions that will always stay + available for future kexec allocations. This file describes + the length of these memory regions. Kexec user space tooling + can use this to determine where it should place its payload + images. + +``/sys/kernel/debug/kho/out/scratch_phys`` + To support continuous KHO kexecs, we need to reserve + physically contiguous memory regions that will always stay + available for future kexec allocations. This file describes + the physical location of these memory regions. Kexec user space + tooling can use this to determine where it should place its + payload images. + +``/sys/kernel/debug/kho/in/fdt`` + When the kernel was booted with Kexec HandOver (KHO), + the state tree that carries metadata about the previous + kernel's state is in this file in the format of flattened + device tree. This file may disappear when all consumers of + it finished to interpret their metadata. diff --git a/Documentation/subsystem-apis.rst b/Documentation/subsystem-apis.rst index b52ad5b969d4..5fc69d6ff9f0 100644 --- a/Documentation/subsystem-apis.rst +++ b/Documentation/subsystem-apis.rst @@ -90,3 +90,4 @@ Other subsystems peci/index wmi/index tee/index + kho/index diff --git a/MAINTAINERS b/MAINTAINERS index a000a277ccf7..d0df0b380e34 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12828,6 +12828,7 @@ F: include/linux/kernfs.h KEXEC L: kexec@lists.infradead.org W: http://kernel.org/pub/linux/utils/kernel/kexec/ +F: Documentation/kho/ F: include/linux/kexec*.h F: include/uapi/linux/kexec.h F: kernel/kexec*