From patchwork Wed Jan 8 17:00:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 11324111 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 67F05138C for ; Wed, 8 Jan 2020 17:02:12 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3AD0420692 for ; Wed, 8 Jan 2020 17:02:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="XGD4jha2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3AD0420692 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ipEhS-0007jC-KS; Wed, 08 Jan 2020 17:00:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ipEhR-0007j7-K8 for xen-devel@lists.xenproject.org; Wed, 08 Jan 2020 17:00:53 +0000 X-Inumbo-ID: 684f4d04-3238-11ea-b1f0-bc764e2007e4 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 684f4d04-3238-11ea-b1f0-bc764e2007e4; Wed, 08 Jan 2020 17:00:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1578502844; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=jV/HQ45EaL2pnMlfwyFunb2FcePKMKqVE1cSDu3XnZE=; b=XGD4jha2NW4nRN1fSVg/47dpj9DFOv6CCywESvAEzMdiM8zuc9sD/y89 iujUEQA7W74oYmhqZcHF8oaErsL1a1Sjr7YMIlRQvNTXea8GfKjVCxct/ u3t8Bu2Uta6rJ9OfwqKDsKMGEZ29XHViRIQdZyELIdyIfO/jEcQ4VbMp0 o=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=andrew.cooper3@citrix.com; spf=Pass smtp.mailfrom=Andrew.Cooper3@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of andrew.cooper3@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="andrew.cooper3@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of Andrew.Cooper3@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="Andrew.Cooper3@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="Andrew.Cooper3@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: WAaJUkaTBakYE+mqm7rOBuvG/UGrAVSGzn0bqPa/ojUeLQzdHIKEU7XgoiFT6rggPXMqjcr2Yf u5GzYm4yMrn+ukL4rgUCfqZql/e8Y4ylTSTJeWIoWIRLDizs744dGFGQCvzt/+Trb8pQheFzLc Wfbjk+PKkE8Omxgtlu3g06tEMwBsqDUpB/0WG/wPmlWTc4zGwKlc9R1TbYSnh2DGgvacLcmKOy MZZcgsoYZAkWkic+w5kDOKdC5fBFcrX/V2XNRUasOHzbdKKQvGrJfbuZwTlOL7Z31TnwG9BDSM 61I= X-SBRS: 2.7 X-MesageID: 10653259 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,410,1571716800"; d="scan'208";a="10653259" From: Andrew Cooper To: Xen-devel Date: Wed, 8 Jan 2020 17:00:40 +0000 Message-ID: <20200108170040.7728-1-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] x86/boot: Rationalise stack handling during early boot X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The top (numerically higher addresses) of cpu0_stack[] contains the BSP's cpu_info block. Logic in Xen expects this to be initialised to 0, but this area of stack is also used during early boot. Update the head.S code to avoid using the cpu_info block. Additionally, update the stack_start variable to match, which avoids __high_start() and efi_arch_post_exit_boot() needing to make the adjustment manually. Finally, leave a big warning by the BIOS BSS initialisation, because it is by no means obvious that the stack doesn't survive the REP STOS. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Wei Liu CC: Roger Pau Monné --- xen/arch/x86/boot/head.S | 10 +++++++--- xen/arch/x86/boot/x86_64.S | 3 +-- xen/arch/x86/efi/efi-boot.h | 13 +++++++------ xen/arch/x86/smpboot.c | 2 +- 4 files changed, 16 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S index 8d0ffbd1b0..2382b61dd4 100644 --- a/xen/arch/x86/boot/head.S +++ b/xen/arch/x86/boot/head.S @@ -400,7 +400,7 @@ __pvh_start: sub $sym_offs(1b), %esi /* Set up stack. */ - lea STACK_SIZE + sym_esi(cpu0_stack), %esp + lea STACK_SIZE - CPUINFO_sizeof + sym_esi(cpu0_stack), %esp mov %ebx, sym_esi(pvh_start_info_pa) @@ -447,7 +447,7 @@ __start: sub $sym_offs(1b), %esi /* Set up stack. */ - lea STACK_SIZE + sym_esi(cpu0_stack), %esp + lea STACK_SIZE - CPUINFO_sizeof + sym_esi(cpu0_stack), %esp /* Bootloaders may set multiboot{1,2}.mem_lower to a nonzero value. */ xor %edx,%edx @@ -616,7 +616,11 @@ trampoline_setup: cmpb $0,sym_fs(efi_platform) jnz 1f - /* Initialize BSS (no nasty surprises!). */ + /* + * Initialise the BSS. + * + * !!! WARNING - also zeroes the current stack !!! + */ mov $sym_offs(__bss_start),%edi mov $sym_offs(__bss_end),%ecx push %fs diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S index b54d3aceea..0acf5e860c 100644 --- a/xen/arch/x86/boot/x86_64.S +++ b/xen/arch/x86/boot/x86_64.S @@ -16,7 +16,6 @@ ENTRY(__high_start) mov %rcx,%cr4 mov stack_start(%rip),%rsp - or $(STACK_SIZE-CPUINFO_sizeof),%rsp /* Reset EFLAGS (subsumes CLI and CLD). */ pushq $0 @@ -42,7 +41,7 @@ multiboot_ptr: .long 0 GLOBAL(stack_start) - .quad cpu0_stack + .quad cpu0_stack + STACK_SIZE - CPUINFO_sizeof .section .data.page_aligned, "aw", @progbits .align PAGE_SIZE, 0 diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h index 676d616ff8..8debdc7ca8 100644 --- a/xen/arch/x86/efi/efi-boot.h +++ b/xen/arch/x86/efi/efi-boot.h @@ -249,23 +249,24 @@ static void __init noreturn efi_arch_post_exit_boot(void) "or $"__stringify(X86_CR4_PGE)", %[cr4]\n\t" "mov %[cr4], %%cr4\n\t" #endif - "movabs $__start_xen, %[rip]\n\t" "lgdt boot_gdtr(%%rip)\n\t" - "mov stack_start(%%rip), %%rsp\n\t" "mov %[ds], %%ss\n\t" "mov %[ds], %%ds\n\t" "mov %[ds], %%es\n\t" "mov %[ds], %%fs\n\t" "mov %[ds], %%gs\n\t" - "movl %[cs], 8(%%rsp)\n\t" - "mov %[rip], (%%rsp)\n\t" - "lretq %[stkoff]-16" + + /* Jump to higher mappings. */ + "mov stack_start(%%rip), %%rsp\n\t" + "movabs $__start_xen, %[rip]\n\t" + "push %[cs]\n\t" + "push %[rip]\n\t" + "lretq" : [rip] "=&r" (efer/* any dead 64-bit variable */), [cr4] "+&r" (cr4) : [cr3] "r" (idle_pg_table), [cs] "ir" (__HYPERVISOR_CS), [ds] "r" (__HYPERVISOR_DS), - [stkoff] "i" (STACK_SIZE - sizeof(struct cpu_info)), "D" (&mbi) : "memory" ); unreachable(); diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 7e29704080..0d0526e2b2 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -554,7 +554,7 @@ static int do_boot_cpu(int apicid, int cpu) printk("Booting processor %d/%d eip %lx\n", cpu, apicid, start_eip); - stack_start = stack_base[cpu]; + stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info); /* This grunge runs the startup process for the targeted processor. */