From patchwork Tue Jan 16 14:37:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13520891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 733B0C47077 for ; Tue, 16 Jan 2024 14:37:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.667805.1039435 (Exim 4.92) (envelope-from ) id 1rPkZ1-0007Av-Hp; Tue, 16 Jan 2024 14:37:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 667805.1039435; Tue, 16 Jan 2024 14:37:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ1-0007A6-BW; Tue, 16 Jan 2024 14:37:15 +0000 Received: by outflank-mailman (input) for mailman id 667805; Tue, 16 Jan 2024 14:37:14 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ0-00077K-HD for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 14:37:14 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ0-0002vu-0l; Tue, 16 Jan 2024 14:37:14 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPkYz-0000Te-Oi; Tue, 16 Jan 2024 14:37:13 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=thlQxTTR/2J0TcN6DCYxzWx3fz1JwxED7pOaGuRoX08=; b=SgvlFM/MpSkQPUAJkDG4HFWQUY sghrtjogXA8iZevCApX76/SAWy21G9vPCX8E2NXXV4vve/PhlyHU0KsaPcw72EoouACxYTOk+bSx7 oJlvbFRnnALqf/n+23KmE2/McMoiZV8BGbiupJhrT4O98tAt4YOLsZOfll5Qe/Ro397U=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v1 repost 1/4] arm/mmu: Move init_ttbr to a new section .data.idmap Date: Tue, 16 Jan 2024 14:37:06 +0000 Message-Id: <20240116143709.86584-2-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116143709.86584-1-julien@xen.org> References: <20240116143709.86584-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall With the upcoming work to color Xen, the binary will not be anymore physically contiguous. This will be a problem during boot as the assembly code will need to work out where each piece of Xen reside. An easy way to solve the issue is to have all code/data accessed by the secondary CPUs while the MMU is off within a single page. Right now, init_ttbr is used by secondary CPUs to find there page-tables before the MMU is on. Yet it is currently in .data which is unlikely to be within the same page as the rest of the idmap. Create a new section .data.idmap that will be used for variables accessed by the early boot code. The first one is init_ttbr. The idmap is currently part of the text section and therefore will be mapped read-only executable. This means that we need to temporarily remap init_ttbr in order to update it. Introduce a new function set_init_ttbr() for this purpose so the code is not duplicated between arm64 and arm32. Signed-off-by: Julien Grall Reviewed-by: Michal Orzel --- xen/arch/arm/mmu/smpboot.c | 34 +++++++++++++++++++++++++++++----- xen/arch/arm/xen.lds.S | 1 + 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index b6fc0aae07f1..f1cf9252710c 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -9,6 +9,10 @@ #include +/* Override macros from asm/page.h to make them work with mfn_t */ +#undef virt_to_mfn +#define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) + /* * Static start-of-day pagetables that we use before the allocators * are up. These are used by all CPUs during bringup before switching @@ -44,7 +48,7 @@ DEFINE_BOOT_PAGE_TABLE(boot_second); DEFINE_BOOT_PAGE_TABLES(boot_third, XEN_NR_ENTRIES(2)); /* Non-boot CPUs use this to find the correct pagetables. */ -uint64_t init_ttbr; +uint64_t __section(".data.idmap") init_ttbr; /* Clear a translation table and clean & invalidate the cache */ static void clear_table(void *table) @@ -68,6 +72,27 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } +static void set_init_ttbr(lpae_t *root) +{ + /* + * init_ttbr is part of the identity mapping which is read-only. So + * We need to re-map the region so it can be updated + */ + void *ptr = map_domain_page(virt_to_mfn(&init_ttbr)); + + ptr += PAGE_OFFSET(&init_ttbr); + + *(uint64_t *)ptr = virt_to_maddr(root); + + /* + * init_ttbr will be accessed with the MMU off, so ensure the update + * is visible by cleaning the cache. + */ + clean_dcache(ptr); + + unmap_domain_page(ptr); +} + #ifdef CONFIG_ARM_64 int prepare_secondary_mm(int cpu) { @@ -77,8 +102,8 @@ int prepare_secondary_mm(int cpu) * Set init_ttbr for this CPU coming up. All CPUs share a single setof * pagetables, but rewrite it each time for consistency with 32 bit. */ - init_ttbr = virt_to_maddr(xen_pgtable); - clean_dcache(init_ttbr); + set_init_ttbr(xen_pgtable); + return 0; } #else @@ -109,8 +134,7 @@ int prepare_secondary_mm(int cpu) clear_boot_pagetables(); /* Set init_ttbr for this CPU coming up */ - init_ttbr = __pa(first); - clean_dcache(init_ttbr); + set_init_ttbr(first); return 0; } diff --git a/xen/arch/arm/xen.lds.S b/xen/arch/arm/xen.lds.S index 20598c6963ce..470c8f22084f 100644 --- a/xen/arch/arm/xen.lds.S +++ b/xen/arch/arm/xen.lds.S @@ -36,6 +36,7 @@ SECTIONS *(.text.header) *(.text.idmap) *(.rodata.idmap) + *(.data.idmap) _idmap_end = .; *(.text.cold) From patchwork Tue Jan 16 14:37:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13520890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 812FDC47DA9 for ; Tue, 16 Jan 2024 14:37:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.667806.1039448 (Exim 4.92) (envelope-from ) id 1rPkZ2-0007ar-Nr; Tue, 16 Jan 2024 14:37:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 667806.1039448; Tue, 16 Jan 2024 14:37:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ2-0007ak-Jv; Tue, 16 Jan 2024 14:37:16 +0000 Received: by outflank-mailman (input) for mailman id 667806; Tue, 16 Jan 2024 14:37:15 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ1-0007Cm-FO for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 14:37:15 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ1-0002wE-6Q; Tue, 16 Jan 2024 14:37:15 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPkZ0-0000Te-V0; Tue, 16 Jan 2024 14:37:15 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Vg9wt2C8p9IZLdVpC0c8HQykip9na6+1kKIeLfSU34o=; b=C7ixQ6ITvzTEhmC5fzz/v8yROa 9W2V8HrFpkFQww9LlI0Jt8fsTgRlKMNhN9GpZzd+o8noZ0eGSNcbYav/Z7MGEix7MTDXWnwLh21JO PtJuNBLxOv+W9XWbLq/laLuo23mATbhDKnkC6cVmxnF15MSn/wpAkWLmFco9EtRAQXH0=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v1 repost 2/4] arm/smpboot: Move smp_up_cpu to a new section .data.idmap Date: Tue, 16 Jan 2024 14:37:07 +0000 Message-Id: <20240116143709.86584-3-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116143709.86584-1-julien@xen.org> References: <20240116143709.86584-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall With the upcoming work to color Xen, the binary will not be anymore physically contiguous. This will be a problem during boot as the assembly code will need to work out where each piece of Xen reside. An easy way to solve the issue is to have all code/data accessed by the secondary CPUs while the MMU is off within a single page. Right now, smp_up_cpu is used by secondary CPUs to wait there turn for booting before the MMU is on. Yet it is currently in .data which is unlikely to be within the same page as the rest of the idmap. Move smp_up_cpu to the recently create section .data.idmap. The idmap is currently part of the text section and therefore will be mapped read-onl executable. This means that we need to temporarily remap smp_up_cpu in order to update it. Introduce a new function set_smp_up_cpu() for this purpose so the code is not duplicated between when opening and closing the gate. Signed-off-by: Julien Grall Reviewed-by: Michal Orzel --- xen/arch/arm/smpboot.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 7110bc11fc05..8d508a1bb258 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -29,6 +29,10 @@ #include #include +/* Override macros from asm/page.h to make them work with mfn_t */ +#undef virt_to_mfn +#define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) + cpumask_t cpu_online_map; cpumask_t cpu_present_map; cpumask_t cpu_possible_map; @@ -56,7 +60,7 @@ struct init_info init_data = }; /* Shared state for coordinating CPU bringup */ -unsigned long smp_up_cpu = MPIDR_INVALID; +unsigned long __section(".data.idmap") smp_up_cpu = MPIDR_INVALID; /* Shared state for coordinating CPU teardown */ static bool cpu_is_dead; @@ -429,6 +433,28 @@ void stop_cpu(void) wfi(); } +static void set_smp_up_cpu(unsigned long mpidr) +{ + /* + * smp_up_cpu is part of the identity mapping which is read-only. So + * We need to re-map the region so it can be updated. + */ + void *ptr = map_domain_page(virt_to_mfn(&smp_up_cpu)); + + ptr += PAGE_OFFSET(&smp_up_cpu); + + *(unsigned long *)ptr = mpidr; + + /* + * init_ttbr will be accessed with the MMU off, so ensure the update + * is visible by cleaning the cache. + */ + clean_dcache(ptr); + + unmap_domain_page(ptr); + +} + int __init cpu_up_send_sgi(int cpu) { /* We don't know the GIC ID of the CPU until it has woken up, so just @@ -460,8 +486,7 @@ int __cpu_up(unsigned int cpu) init_data.cpuid = cpu; /* Open the gate for this CPU */ - smp_up_cpu = cpu_logical_map(cpu); - clean_dcache(smp_up_cpu); + set_smp_up_cpu(cpu_logical_map(cpu)); rc = arch_cpu_up(cpu); @@ -497,8 +522,9 @@ int __cpu_up(unsigned int cpu) */ init_data.stack = NULL; init_data.cpuid = ~0; - smp_up_cpu = MPIDR_INVALID; - clean_dcache(smp_up_cpu); + + set_smp_up_cpu(MPIDR_INVALID); + arch_cpu_up_finish(); if ( !cpu_online(cpu) ) From patchwork Tue Jan 16 14:37:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13520892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DABE4C47DAF for ; Tue, 16 Jan 2024 14:37:26 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.667807.1039457 (Exim 4.92) (envelope-from ) id 1rPkZ3-0007qj-V6; Tue, 16 Jan 2024 14:37:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 667807.1039457; Tue, 16 Jan 2024 14:37:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ3-0007qc-SJ; Tue, 16 Jan 2024 14:37:17 +0000 Received: by outflank-mailman (input) for mailman id 667807; Tue, 16 Jan 2024 14:37:16 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ2-0007d6-R8 for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 14:37:16 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ2-0002wi-Cj; Tue, 16 Jan 2024 14:37:16 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPkZ2-0000Te-52; Tue, 16 Jan 2024 14:37:16 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=QB4GZQCd//Kzo6y8zaEONm7rNVnOunv9HV6HbpL0NS0=; b=A10stZBEien/4deDkPtbAED6FI z8KyK9NuT23J+Tfi8gKy5JLzHdPt0933+ubBHtxhUvqYQZfgaJFp/SPeERvR3iaGbqTjumRISYRKV uand0Fk37UNxaOzAeV9lVt5+5vid6AHgf/WoRxHKtwZEiyQUT88SCsM4hORVi6MBDuhU=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v1 repost 3/4] xen/arm64: head: Use PRINT_ID() for secondary CPU MMU-off boot code Date: Tue, 16 Jan 2024 14:37:08 +0000 Message-Id: <20240116143709.86584-4-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116143709.86584-1-julien@xen.org> References: <20240116143709.86584-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall With the upcoming work to color Xen, the binary will not be anymore physically contiguous. This will be a problem during boot as the assembly code will need to work out where each piece of Xen reside. An easy way to solve the issue is to have all code/data accessed by the secondary CPUs while the MMU is off within a single page. Right now, most of the early printk messages are using PRINT() which will add the message in .rodata. This is unlikely to be within the same page as the rest of the idmap. So replace all the PRINT() that can be reachable by the secondary CPU with MMU-off with PRINT_ID(). Signed-off-by: Julien Grall Reviewed-by: Michal Orzel --- xen/arch/arm/arm64/head.S | 14 +++++++------- xen/arch/arm/arm64/mmu/head.S | 2 +- xen/arch/arm/include/asm/arm64/macros.h | 9 ++++++--- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index cfc04c755400..fa8b00b6f1db 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -289,9 +289,9 @@ GLOBAL(init_secondary) #ifdef CONFIG_EARLY_PRINTK ldr x23, =CONFIG_EARLY_UART_BASE_ADDRESS /* x23 := UART base address */ - PRINT("- CPU ") + PRINT_ID("- CPU ") print_reg x24 - PRINT(" booting -\r\n") + PRINT_ID(" booting -\r\n") #endif bl check_cpu_mode bl cpu_init @@ -314,10 +314,10 @@ ENDPROC(init_secondary) * Clobbers x0 - x5 */ check_cpu_mode: - PRINT("- Current EL ") + PRINT_ID("- Current EL ") mrs x5, CurrentEL print_reg x5 - PRINT(" -\r\n") + PRINT_ID(" -\r\n") /* Are we in EL2 */ cmp x5, #PSR_MODE_EL2t @@ -326,8 +326,8 @@ check_cpu_mode: ret 1: /* OK, we're boned. */ - PRINT("- Xen must be entered in NS EL2 mode -\r\n") - PRINT("- Please update the bootloader -\r\n") + PRINT_ID("- Xen must be entered in NS EL2 mode -\r\n") + PRINT_ID("- Please update the bootloader -\r\n") b fail ENDPROC(check_cpu_mode) @@ -361,7 +361,7 @@ ENDPROC(zero_bss) * Clobbers x0 - x3 */ cpu_init: - PRINT("- Initialize CPU -\r\n") + PRINT_ID("- Initialize CPU -\r\n") /* Set up memory attribute type tables */ ldr x0, =MAIRVAL diff --git a/xen/arch/arm/arm64/mmu/head.S b/xen/arch/arm/arm64/mmu/head.S index 92b62ae94ce5..fa40b696ddc8 100644 --- a/xen/arch/arm/arm64/mmu/head.S +++ b/xen/arch/arm/arm64/mmu/head.S @@ -276,7 +276,7 @@ ENDPROC(create_page_tables) enable_mmu: mov x4, x0 mov x5, x1 - PRINT("- Turning on paging -\r\n") + PRINT_ID("- Turning on paging -\r\n") /* * The state of the TLBs is unknown before turning on the MMU. diff --git a/xen/arch/arm/include/asm/arm64/macros.h b/xen/arch/arm/include/asm/arm64/macros.h index 10e652041f57..6a0108f778a2 100644 --- a/xen/arch/arm/include/asm/arm64/macros.h +++ b/xen/arch/arm/include/asm/arm64/macros.h @@ -39,9 +39,12 @@ * There are multiple flavors: * - PRINT_SECT(section, string): The @string will be located in @section * - PRINT(): The string will be located in .rodata.str. - * - PRINT_ID(): When Xen is running on the Identity Mapping, it is - * only possible to have a limited amount of Xen. This will create - * the string in .rodata.idmap which will always be mapped. + * - PRINT_ID(): This will create the string in .rodata.idmap which + * will always be accessible. This is used when: + * - Xen is running on the identity mapping because not all of Xen is mapped + * - Running with the MMU-off on secondary boots as Xen may not be + * physically contiguous in memory (e.g. in the case of cache + * coloring). * * Clobbers x0 - x3 */ From patchwork Tue Jan 16 14:37:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 13520893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FB32C47077 for ; Tue, 16 Jan 2024 14:37:29 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.667808.1039468 (Exim 4.92) (envelope-from ) id 1rPkZ6-0008A5-78; Tue, 16 Jan 2024 14:37:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 667808.1039468; Tue, 16 Jan 2024 14:37:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ6-00089w-3F; Tue, 16 Jan 2024 14:37:20 +0000 Received: by outflank-mailman (input) for mailman id 667808; Tue, 16 Jan 2024 14:37:18 +0000 Received: from mail.xenproject.org ([104.130.215.37]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ4-0007rR-0H for xen-devel@lists.xenproject.org; Tue, 16 Jan 2024 14:37:18 +0000 Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rPkZ3-0002x4-Iy; Tue, 16 Jan 2024 14:37:17 +0000 Received: from 54-240-197-232.amazon.com ([54.240.197.232] helo=dev-dsk-jgrall-1b-035652ec.eu-west-1.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1rPkZ3-0000Te-BJ; Tue, 16 Jan 2024 14:37:17 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=nSJb8H4AUwBfcQwQar1U/80V0XEGtzBfpVJPiZPhupE=; b=H/1jQx0a60F1WvaASINcygpe+i 3DoLBQ9H1cGwWwHYkFWv9dkUr7+mzfSNUCck1NSj4+QqHRkqzYoaJAlPQk2bMH36rrKL7f9quC+Wk aO3jz/844pY4Kly63gIbysACvKzsYleORF1vlwL+QVYQU3SYGvxcJI25sLyoKJUq7OSA=; From: Julien Grall To: xen-devel@lists.xenproject.org Cc: carlo.nonato@minervasys.tech, Julien Grall , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: [PATCH v1 repost 4/4] [DO NOT COMMIT] xen/arm: Create a trampoline for secondary boot CPUs Date: Tue, 16 Jan 2024 14:37:09 +0000 Message-Id: <20240116143709.86584-5-julien@xen.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240116143709.86584-1-julien@xen.org> References: <20240116143709.86584-1-julien@xen.org> MIME-Version: 1.0 From: Julien Grall In order to confirm the early boot code is self-contained, allocate a separate trampoline region for secondary to boot from it. Signed-off-by: Julien Grall --- xen/arch/arm/arm64/mmu/mm.c | 7 +++++++ xen/arch/arm/mmu/smpboot.c | 4 +++- xen/arch/arm/psci.c | 5 ++++- xen/arch/arm/smpboot.c | 15 ++++++++++++++- 4 files changed, 28 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/arm64/mmu/mm.c b/xen/arch/arm/arm64/mmu/mm.c index d2651c948698..3c4988dc75d1 100644 --- a/xen/arch/arm/arm64/mmu/mm.c +++ b/xen/arch/arm/arm64/mmu/mm.c @@ -110,11 +110,18 @@ void __init arch_setup_page_tables(void) prepare_runtime_identity_mapping(); } +extern mfn_t trampoline_start; + void update_identity_mapping(bool enable) { paddr_t id_addr = virt_to_maddr(_start); int rc; + if ( !mfn_eq(trampoline_start, INVALID_MFN) ) + { + id_addr = mfn_to_maddr(trampoline_start); + } + if ( enable ) rc = map_pages_to_xen(id_addr, maddr_to_mfn(id_addr), 1, PAGE_HYPERVISOR_RX); diff --git a/xen/arch/arm/mmu/smpboot.c b/xen/arch/arm/mmu/smpboot.c index f1cf9252710c..d768dfe065a5 100644 --- a/xen/arch/arm/mmu/smpboot.c +++ b/xen/arch/arm/mmu/smpboot.c @@ -72,13 +72,15 @@ static void clear_boot_pagetables(void) clear_table(boot_third); } +extern mfn_t trampoline_start; + static void set_init_ttbr(lpae_t *root) { /* * init_ttbr is part of the identity mapping which is read-only. So * We need to re-map the region so it can be updated */ - void *ptr = map_domain_page(virt_to_mfn(&init_ttbr)); + void *ptr = map_domain_page(trampoline_start); ptr += PAGE_OFFSET(&init_ttbr); diff --git a/xen/arch/arm/psci.c b/xen/arch/arm/psci.c index 695d2fa1f1b5..a00574d559d6 100644 --- a/xen/arch/arm/psci.c +++ b/xen/arch/arm/psci.c @@ -36,11 +36,14 @@ static uint32_t psci_cpu_on_nr; #define PSCI_RET(res) ((int32_t)(res).a0) +extern mfn_t trampoline_start; + int call_psci_cpu_on(int cpu) { struct arm_smccc_res res; - arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), __pa(init_secondary), + arm_smccc_smc(psci_cpu_on_nr, cpu_logical_map(cpu), + mfn_to_maddr(trampoline_start) + PAGE_OFFSET(init_secondary), &res); return PSCI_RET(res); diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 8d508a1bb258..ef84b73ebd46 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -293,10 +293,13 @@ unsigned int __init smp_get_max_cpus(void) return cpus; } +mfn_t trampoline_start = INVALID_MFN_INITIALIZER; + void __init smp_prepare_cpus(void) { int rc; + void *trampoline; cpumask_copy(&cpu_present_map, &cpu_possible_map); @@ -304,6 +307,16 @@ smp_prepare_cpus(void) if ( rc ) panic("Unable to allocate CPU sibling/core maps\n"); + /* Create a trampoline to confirm early boot code is self-contained */ + trampoline = alloc_xenheap_page(); + BUG_ON(!trampoline); + + memcpy(trampoline, _start, PAGE_SIZE); + clean_dcache_va_range(trampoline, PAGE_SIZE); + invalidate_icache(); + + printk("Trampoline 0x%lx\n", virt_to_maddr(trampoline)); + trampoline_start = virt_to_mfn(trampoline); } /* Boot the current CPU */ @@ -439,7 +452,7 @@ static void set_smp_up_cpu(unsigned long mpidr) * smp_up_cpu is part of the identity mapping which is read-only. So * We need to re-map the region so it can be updated. */ - void *ptr = map_domain_page(virt_to_mfn(&smp_up_cpu)); + void *ptr = map_domain_page(trampoline_start); ptr += PAGE_OFFSET(&smp_up_cpu);