From patchwork Thu Mar 20 01:55:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changyuan Lyu X-Patchwork-Id: 14023310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 059E8C35FFC for ; Thu, 20 Mar 2025 01:56:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E3CA928000C; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE617280001; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEDB328000C; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9C79B280001 for ; Wed, 19 Mar 2025 21:56:18 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1DF7F805CE for ; Thu, 20 Mar 2025 01:56:19 +0000 (UTC) X-FDA: 83240264478.23.BC925C5 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf28.hostedemail.com (Postfix) with ESMTP id 4C5AAC0004 for ; Thu, 20 Mar 2025 01:56:17 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wJl0mFuG; spf=pass (imf28.hostedemail.com: domain of 3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742435777; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=lXPfUZnHXCu/089HKRzxD1tqElN9KTEC+S1CBa6UIoL13f6b/EPoy2bGX0m5OsZdgTh6dr tL54SBue9+hAdRvxf+VrW6aZL0KMQBHrUCzTMtIAdbpxHRskAitHMkU2VAQ9Ee7cGDT33J dVJ1H08bYRHCqMFmEJhgkZOijxj1cek= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742435777; a=rsa-sha256; cv=none; b=KULWqOPzVzUSbcjKvdWq0EokShiqscXFPyxCjlgwFtvl8Rb9jr/9k8Ssdw5ToLR+putUji V2rUKMWp0ziurf2WsAoYNpLpH/ojS7+8GsoRycyszyHfTZUag7X+cdPs/io3DUDGmblv2b ca2NJqQDgLWNyKV1VujXsjV9S5+hQ90= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wJl0mFuG; spf=pass (imf28.hostedemail.com: domain of 3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3wHXbZwoKCF89E7KDVR7KIDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--changyuanl.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2242ade807fso6782495ad.2 for ; Wed, 19 Mar 2025 18:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742435776; x=1743040576; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=wJl0mFuGaffdLfdIIjSfKWymlQzC0VbPLFU/bq6N2JLfwRm0TC12oICJGF7kAs/Tdu HJbKSZATaSZp+GmAUi/trhSvTgNHK0tSol3DE2Zx+pblD/CL/In8wliecosOMLBn1SVn yenGN0vWSB1rzsRZHVZo/SyPcusiFS3ra+HvRQ8RzhqrLoPKA/JWxegna+yIboyklkS/ ncL/MyEUmm0l/hWQm4TR9LuD93W8BoWIESGj2Xz2PQsjaaUyO8Pb2kUdplX7venA+LKI 54sNTQ8suD76cI+T3c/wKQpCnUW++qpuJBm85YqRtm2JIPNhk6S593ifxEwvrWGU+5rO oY0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742435776; x=1743040576; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8V+X6vapuQnm7T0BErSGY0fSGWOIaeixziHNtlYjz6I=; b=ryZOMaZxco1GjyYAr061EFoJhpwVbJhVfOGO059lIDUeVkrJflih3HfTRrM/SLsGcK 3uC7YBDuaDjWn+xRJ9kuGwJT49/jtgxI5KuKQ52zA+5kBjl1i0uavgwRogBu5mJrWD10 IRzkdE+H+YtIO5/5EDDWIdne35GGPWtp4ZKpglBBjJ+hWHaK5VPvsfdOIV1AS9tiRrSt 4980DkJ38OvnG/WmeMCA22sloTBexv7mQtSXhXy38gSKGAkCqeI8Vx1XR0bAAUhS7qhq iSNmUps7348lQA83jgnOwZQRIezAXVCCK8c4DGmBlrhJPJSSUah4No4ZLBnAaX2E+2mk 5IFQ== X-Forwarded-Encrypted: i=1; AJvYcCWLNskcL5CjiaJI/yajzP70DSXBp+UtDn6GxrCGatIFNAtZIoclvFEUCqHFf4I9QH13ix/yhsz+mg==@kvack.org X-Gm-Message-State: AOJu0YzZBibCrD9JKOfVYhrkX7mb3L8CkFCz65y9MgTnEEkdm8BSnWZ+ LQ5fkj4L7SjcxjBRw7A53tc+Jh4GE2ry76Qgwx/P8Wbi6Wc/2KkI7WbyrkUssT2p77O19Ye6+XA yo3h8VO0rYvcqa3vt8g== X-Google-Smtp-Source: AGHT+IEM1zhUI4aB8FBWWC+mYMClc6ahuDGj6ZxX8EDQ9DE5DKRFOEqA+5eaTn1Hw9R605eYXv4+w1iX7+c1hN7N X-Received: from pltk7.prod.google.com ([2002:a17:902:6947:b0:223:6930:304e]) (user=changyuanl job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:db0a:b0:223:f408:c3f7 with SMTP id d9443c01a7336-2265edc7b36mr19232405ad.16.1742435776171; Wed, 19 Mar 2025 18:56:16 -0700 (PDT) Date: Wed, 19 Mar 2025 18:55:45 -0700 In-Reply-To: <20250320015551.2157511-1-changyuanl@google.com> Mime-Version: 1.0 References: <20250320015551.2157511-1-changyuanl@google.com> X-Mailer: git-send-email 2.49.0.rc1.451.g8f38331e32-goog Message-ID: <20250320015551.2157511-11-changyuanl@google.com> Subject: [PATCH v5 10/16] kexec: add KHO support to kexec file loads From: Changyuan Lyu To: linux-kernel@vger.kernel.org Cc: graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, ptyadav@amazon.de, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, Changyuan Lyu X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4C5AAC0004 X-Stat-Signature: dyc3xc8f3xk44ja88r5w5fjnqco6dhmr X-HE-Tag: 1742435777-559283 X-HE-Meta: U2FsdGVkX18oGaVBZ6aBFdk931UM90mA2pzylG/RsD+EQXKGnoXxcmKzP1ym+gPC+cVP4p6y/tmxSEGcZ5h4fP+kcE2i9kfxINWQ9Wd9w9azp39HqvM4CDVCSXNMM93x/P5Yy05vShLIwBTqzczfaB9cV0Pyxmsk8uq0Uxnbo8LY161zl1+lu/zZtKOfHQo9yjbau5jeuFslSVEPNT/GBC/tn3CDvTyVsPi8yx5pOeZ17UWuxUhYrbxXONx+EHRp1c1/TNWGqfe2lazFrrU+5TB+G3pfn+DkY2hVp3+OIuUmjue36z7aRLkFfGVcVwJD4H9LaFUtJoB0mRBAdu0Kln2bJiJ2hzcBHFLh/QV3nlzLM04UJvvIu6CGKcf1u2UaMl5FpmgctAwKyu/N8weoFBgH8ary1qcK9DFTweQ/jVkU8Fu8olVrPf2OUrUBaEDg2D2kSkh54odDqhDBQUMcUOViSM6+8MMP0xue4ZJ/VCHxADC9wDWo4Niuf47P0DPG1n3VFMiaX2cUQA6OWqVIpbvm1rrlFxI05AXemmFnVM4TQFWy5oPPrR3djiCgPBEeaGpFZSlYpZG6gqYyssJfg1VawppH+A0rdIRRW8Y7vvmDhU/+A0YXNv4vMnd4DfI0tlDtiSzieBTR55I3SzyfihjZKMq0EVlPHxeHqNwPAzx6ww484ABAx2UPFzEC75Go8xEngUeUnG7ReEjtIhPIT85ziPhEZLsqz+lhOSHyLmAtjk+nWFMHcmJrZ7THKHIyY1RrFQZXCBJ//xcMu/xcd7bgf7JqpQx8XltuLx8a2bRDrMJyhse7szYC6qI5MynR+ikwfwc2zRTavCtPWCEVKW98c5ogcNzJUJrAijhHk4V5y6JGZrL7EU00WhZ45yJvfvdM7nfZLM0iG3eYR/d9M+HIBTALqmiFvSNIkQWV4QTaJk912PGkPnTlR6iu6jijhskFxcHolfetnjQ0hnl Bj69fAJ8 fC8yVLu7UfwC5c/0396P2uAVByT9Dtl8hxeHj0F36ZPKEtVI5hSnZTeaWLIkY8H0yBZq8Fef2RS9+1+OxVN2p8S1rUO25pFA7SLxwp/9rifdK/o9jVpPFjnsW+xsRGzLkz8TCxpp6y8iwSoJvdWKWweXrIbc8VzpqohvHwQnX0BsRXonxenOxvKPLMmV3aBO/K2rTBIVuZ4gumGU29T+yBOydAMfXFGrbKyGvUrqYPenIH/TuZ6mGJQGuesavYcU/5jod5sLjarHcslqSw0DadOKxT6+hQBbWSc+Nmo1xjXOd93V41Mp5B5j22osH10rJXhxkEloI3pLTJm0jFomXgsdm+v8X5LIJ19hyQHH8MsnivIGahJP+HGC6Gze4Z3/+3GQDttpsag9jiTHnEkh5Nowyn7LbZWJ9kIzT66jsLmbbd1yTKzFjofHhq+0Y92z3vgsv85V8lviuefVo2VoRJcI72OqtGZuYa9y4KtsWJNvhGi4tLFWL+76pOZ49gM3hEUPO54SpqMn939ppR1vePDthtkCxDwImI2zx5TfefzmobDhqioxPjFMGCZHvy3eHeesOT8rb1vcEeqahWoCpBThpPLAi8v5dSus2m37GVFP345T8bZrjiX3hxZa7hBFWrzpI3/KfEgivkgzA2ZuOJGnsw41HXJfO5w0j7w9I6KtxXVoiHDzDFTWi2KRIz0s3DoL0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexander Graf Kexec has 2 modes: A user space driven mode and a kernel driven mode. For the kernel driven mode, kernel code determines the physical addresses of all target buffers that the payload gets copied into. With KHO, we can only safely copy payloads into the "scratch area". Teach the kexec file loader about it, so it only allocates for that area. In addition, enlighten it with support to ask the KHO subsystem for its respective payloads to copy into target memory. Also teach the KHO subsystem how to fill the images for file loads. Signed-off-by: Alexander Graf Co-developed-by: Mike Rapoport (Microsoft) Signed-off-by: Mike Rapoport (Microsoft) Co-developed-by: Changyuan Lyu Signed-off-by: Changyuan Lyu --- include/linux/kexec.h | 7 +++ kernel/kexec_core.c | 4 ++ kernel/kexec_file.c | 19 +++++++ kernel/kexec_handover.c | 108 ++++++++++++++++++++++++++++++++++++++++ kernel/kexec_internal.h | 18 +++++++ 5 files changed, 156 insertions(+) diff --git a/include/linux/kexec.h b/include/linux/kexec.h index fad04f3bcf1d..d59eee60e36e 100644 --- a/include/linux/kexec.h +++ b/include/linux/kexec.h @@ -364,6 +364,13 @@ struct kimage { size_t ima_buffer_size; #endif +#ifdef CONFIG_KEXEC_HANDOVER + struct { + struct kexec_segment *scratch; + struct kexec_segment *fdt; + } kho; +#endif + /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_sz; diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index 640d252306ea..67fb9c0b3714 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -1053,6 +1053,10 @@ int kernel_kexec(void) goto Unlock; } + error = kho_copy_fdt(kexec_image); + if (error) + goto Unlock; + #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { /* diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 3eedb8c226ad..070ef206f573 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -253,6 +253,11 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd, /* IMA needs to pass the measurement list to the next kernel. */ ima_add_kexec_buffer(image); + /* If KHO is active, add its images to the list */ + ret = kho_fill_kimage(image); + if (ret) + goto out; + /* Call image load handler */ ldata = kexec_image_load_default(image); @@ -636,6 +641,14 @@ int kexec_locate_mem_hole(struct kexec_buf *kbuf) if (kbuf->mem != KEXEC_BUF_MEM_UNKNOWN) return 0; + /* + * If KHO is active, only use KHO scratch memory. All other memory + * could potentially be handed over. + */ + ret = kho_locate_mem_hole(kbuf, locate_mem_hole_callback); + if (ret <= 0) + return ret; + if (!IS_ENABLED(CONFIG_ARCH_KEEP_MEMBLOCK)) ret = kexec_walk_resources(kbuf, locate_mem_hole_callback); else @@ -764,6 +777,12 @@ static int kexec_calculate_store_digests(struct kimage *image) if (ksegment->kbuf == pi->purgatory_buf) continue; +#ifdef CONFIG_KEXEC_HANDOVER + /* Skip KHO FDT as its contects are copied in kernel_kexec(). */ + if (ksegment == image->kho.fdt) + continue; +#endif + ret = crypto_shash_update(desc, ksegment->kbuf, ksegment->bufsz); if (ret) diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c index 592563c21369..5108e2cc1a22 100644 --- a/kernel/kexec_handover.c +++ b/kernel/kexec_handover.c @@ -245,6 +245,85 @@ int kho_node_check_compatible(const struct kho_in_node *node, } EXPORT_SYMBOL_GPL(kho_node_check_compatible); +int kho_fill_kimage(struct kimage *image) +{ + ssize_t scratch_size; + int err = 0; + + if (!kho_enable) + return 0; + + /* Allocate target memory for KHO FDT */ + struct kexec_buf fdt = { + .image = image, + .buffer = NULL, + .bufsz = 0, + .mem = KEXEC_BUF_MEM_UNKNOWN, + .memsz = kho_out.fdt_max, + .buf_align = SZ_64K, /* Makes it easier to map */ + .buf_max = ULONG_MAX, + .top_down = true, + }; + err = kexec_add_buffer(&fdt); + if (err) { + pr_err("failed to reserved a segment for KHO FDT: %d\n", err); + return err; + } + image->kho.fdt = &image->segment[image->nr_segments - 1]; + + scratch_size = sizeof(*kho_scratch) * kho_scratch_cnt; + struct kexec_buf scratch = { + .image = image, + .buffer = kho_scratch, + .bufsz = scratch_size, + .mem = KEXEC_BUF_MEM_UNKNOWN, + .memsz = scratch_size, + .buf_align = SZ_64K, /* Makes it easier to map */ + .buf_max = ULONG_MAX, + .top_down = true, + }; + err = kexec_add_buffer(&scratch); + if (err) + return err; + image->kho.scratch = &image->segment[image->nr_segments - 1]; + + return 0; +} + +static int kho_walk_scratch(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret = 0; + int i; + + for (i = 0; i < kho_scratch_cnt; i++) { + struct resource res = { + .start = kho_scratch[i].addr, + .end = kho_scratch[i].addr + kho_scratch[i].size - 1, + }; + + /* Try to fit the kimage into our KHO scratch region */ + ret = func(&res, kbuf); + if (ret) + break; + } + + return ret; +} + +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + int ret; + + if (!kho_enable || kbuf->image->type == KEXEC_TYPE_CRASH) + return 1; + + ret = kho_walk_scratch(kbuf, func); + + return ret == 1 ? 0 : -EADDRNOTAVAIL; +} + /* * Keep track of memory that is to be preserved across KHO. * @@ -1141,6 +1220,35 @@ static int kho_finalize(void) return err; } +int kho_copy_fdt(struct kimage *image) +{ + int err = 0; + void *fdt; + + if (!kho_enable || !image->file_mode) + return 0; + + if (!kho_out.fdt) { + err = kho_finalize(); + kho_out_update_debugfs_fdt(); + if (err) + return err; + } + + fdt = kimage_map_segment(image, image->kho.fdt->mem, + PAGE_ALIGN(kho_out.fdt_max)); + if (!fdt) { + pr_err("failed to vmap fdt ksegment in kimage\n"); + return -ENOMEM; + } + + memcpy(fdt, kho_out.fdt, fdt_totalsize(kho_out.fdt)); + + kimage_unmap_segment(fdt); + + return 0; +} + /* Handling for debug/kho/out */ static int kho_out_finalize_get(void *data, u64 *val) { diff --git a/kernel/kexec_internal.h b/kernel/kexec_internal.h index d35d9792402d..ec9555a4751d 100644 --- a/kernel/kexec_internal.h +++ b/kernel/kexec_internal.h @@ -39,4 +39,22 @@ extern size_t kexec_purgatory_size; #else /* CONFIG_KEXEC_FILE */ static inline void kimage_file_post_load_cleanup(struct kimage *image) { } #endif /* CONFIG_KEXEC_FILE */ + +struct kexec_buf; + +#ifdef CONFIG_KEXEC_HANDOVER +int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)); +int kho_fill_kimage(struct kimage *image); +int kho_copy_fdt(struct kimage *image); +#else +static inline int kho_locate_mem_hole(struct kexec_buf *kbuf, + int (*func)(struct resource *, void *)) +{ + return 1; +} + +static inline int kho_fill_kimage(struct kimage *image) { return 0; } +static inline int kho_copy_fdt(struct kimage *image) { return 0; } +#endif /* CONFIG_KEXEC_HANDOVER */ #endif /* LINUX_KEXEC_INTERNAL_H */