From patchwork Tue Apr 12 17:32:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BF8DC433EF for ; Tue, 12 Apr 2022 17:32:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237589AbiDLRfN (ORCPT ); Tue, 12 Apr 2022 13:35:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358263AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97B3B62126 for ; Tue, 12 Apr 2022 10:32:25 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2A5C6212B7; Tue, 12 Apr 2022 17:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784744; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Wce6EA/3d83jsmt+Eiw/mjQUgCKuA+ubMAnpiIG4Vfo=; b=h6dAeijWpjzrbsHNe69B3Oj/gvf4WNlq4w7AUIMdztpSmD4P96qY8Vj3jv0QLdmoy3X3g/ ZwgMrjVqX6AZ+xxPPtXtjiVrbr5BfrhQn9PmUGb8exB+79yUjL4p8IloNjCZiuyRrigl74 1RpY1U5n3PnwcoB7q6hBaszOeUkt47o= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9F8DB13780; Tue, 12 Apr 2022 17:32:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id BQNMI6e3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:23 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 01/10] x86: Move ap_init() to smp.c Date: Tue, 12 Apr 2022 19:32:12 +0200 Message-Id: <20220412173221.13315-1-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org ap_init() copies the SIPI vector to lowmem, sends INIT/SIPI to APs and waits on the APs to come up. Port this routine to C from asm and move it to smp.c to allow sharing this functionality between the EFI (-fPIC) and non-EFI builds. Call ap_init() from the EFI setup path to reset the APs to a known location. Signed-off-by: Varad Gautam --- lib/x86/setup.c | 1 + lib/x86/smp.c | 28 ++++++++++++++++++++++++++-- lib/x86/smp.h | 1 + x86/cstart64.S | 20 ++------------------ x86/efi/efistart64.S | 9 +++++++++ 5 files changed, 39 insertions(+), 20 deletions(-) diff --git a/lib/x86/setup.c b/lib/x86/setup.c index 2d63a44..86ba6de 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -323,6 +323,7 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) load_idt(); mask_pic_interrupts(); enable_apic(); + ap_init(); enable_x2apic(); smp_init(); setup_page_table(); diff --git a/lib/x86/smp.c b/lib/x86/smp.c index 683b25d..d7f5aba 100644 --- a/lib/x86/smp.c +++ b/lib/x86/smp.c @@ -18,6 +18,9 @@ static volatile int ipi_done; static volatile bool ipi_wait; static int _cpu_count; static atomic_t active_cpus; +extern u8 sipi_entry; +extern u8 sipi_end; +volatile unsigned cpu_online_count = 1; static __attribute__((used)) void ipi(void) { @@ -114,8 +117,6 @@ void smp_init(void) int i; void ipi_entry(void); - _cpu_count = fwcfg_get_nb_cpus(); - setup_idt(); init_apic_map(); set_idt_entry(IPI_VECTOR, ipi_entry, 0); @@ -142,3 +143,26 @@ void smp_reset_apic(void) atomic_inc(&active_cpus); } + +void ap_init(void) +{ + u8 *dst_addr = 0; + size_t sipi_sz = (&sipi_end - &sipi_entry) + 1; + + asm volatile("cld"); + + /* Relocate SIPI vector to dst_addr so it can run in 16-bit mode. */ + memcpy(dst_addr, &sipi_entry, sipi_sz); + + /* INIT */ + apic_icr_write(APIC_DEST_ALLBUT | APIC_DEST_PHYSICAL | APIC_DM_INIT | APIC_INT_ASSERT, 0); + + /* SIPI */ + apic_icr_write(APIC_DEST_ALLBUT | APIC_DEST_PHYSICAL | APIC_DM_STARTUP, 0); + + _cpu_count = fwcfg_get_nb_cpus(); + + while (_cpu_count != cpu_online_count) { + ; + } +} diff --git a/lib/x86/smp.h b/lib/x86/smp.h index bd303c2..9c92853 100644 --- a/lib/x86/smp.h +++ b/lib/x86/smp.h @@ -78,5 +78,6 @@ void on_cpu(int cpu, void (*function)(void *data), void *data); void on_cpu_async(int cpu, void (*function)(void *data), void *data); void on_cpus(void (*function)(void *data), void *data); void smp_reset_apic(void); +void ap_init(void); #endif diff --git a/x86/cstart64.S b/x86/cstart64.S index 7272452..f371d06 100644 --- a/x86/cstart64.S +++ b/x86/cstart64.S @@ -157,6 +157,7 @@ gdt32: gdt32_end: .code16 +.globl sipi_entry sipi_entry: mov %cr0, %eax or $1, %eax @@ -168,6 +169,7 @@ gdt32_descr: .word gdt32_end - gdt32 - 1 .long gdt32 +.globl sipi_end sipi_end: .code32 @@ -240,21 +242,3 @@ lvl5: online_cpus: .fill (max_cpus + 7) / 8, 1, 0 - -ap_init: - cld - lea sipi_entry, %rsi - xor %rdi, %rdi - mov $(sipi_end - sipi_entry), %rcx - rep movsb - mov $APIC_DEFAULT_PHYS_BASE, %eax - movl $(APIC_DEST_ALLBUT | APIC_DEST_PHYSICAL | APIC_DM_INIT | APIC_INT_ASSERT), APIC_ICR(%rax) - movl $(APIC_DEST_ALLBUT | APIC_DEST_PHYSICAL | APIC_DM_STARTUP), APIC_ICR(%rax) - call fwcfg_get_nb_cpus -1: pause - cmpw %ax, cpu_online_count - jne 1b - ret - -.align 2 -cpu_online_count: .word 1 diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 017abba..0425153 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -57,3 +57,12 @@ load_gdt_tss: pushq $0x08 /* 2nd entry in gdt64: 64-bit code segment */ pushq %rdi lretq + +.code16 + +.globl sipi_entry +sipi_entry: + jmp sipi_entry + +.globl sipi_end +sipi_end: From patchwork Tue Apr 12 17:32:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18BEAC433FE for ; Tue, 12 Apr 2022 17:32:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239478AbiDLRfM (ORCPT ); Tue, 12 Apr 2022 13:35:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358268AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 107FB5BE4A for ; Tue, 12 Apr 2022 10:32:26 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C0F111F38D; Tue, 12 Apr 2022 17:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784744; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XbejgVpF5ZgYr/RTBSxRo0TSVOQLZCYLw9P+IhZ+VAg=; b=NY/l9y8VjIuuDGV9lU3c5QRRCa3ZKnrhYtvQyVJ/Uf4e62fmwkO3mEsr2ClSr5b4zh0S5U ArIJ8nQ2zJIvzB3+UreZ3bM5lDgiqu+EpotI2N3W3VFz5zgVnm9xBy11Mp+xOLYqTd3DbP HTychjkpOtG0CiLZlPqwMmOVvS53P2Y= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 366A413780; Tue, 12 Apr 2022 17:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IHfMCqi3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:24 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 02/10] x86: Move load_idt() to desc.c Date: Tue, 12 Apr 2022 19:32:13 +0200 Message-Id: <20220412173221.13315-2-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This allows sharing IDT setup code between EFI (-fPIC) and non-EFI builds. Signed-off-by: Varad Gautam --- lib/x86/desc.c | 5 +++++ lib/x86/desc.h | 1 + lib/x86/setup.c | 1 - x86/cstart64.S | 3 ++- x86/efi/efistart64.S | 5 ----- 5 files changed, 8 insertions(+), 7 deletions(-) diff --git a/lib/x86/desc.c b/lib/x86/desc.c index 0677fcd..087e85c 100644 --- a/lib/x86/desc.c +++ b/lib/x86/desc.c @@ -294,6 +294,11 @@ void setup_idt(void) handle_exception(13, check_exception_table); } +void load_idt(void) +{ + lidt(&idt_descr); +} + unsigned exception_vector(void) { return this_cpu_read_exception_vector(); diff --git a/lib/x86/desc.h b/lib/x86/desc.h index 5224b58..3044409 100644 --- a/lib/x86/desc.h +++ b/lib/x86/desc.h @@ -4,6 +4,7 @@ #include void setup_idt(void); +void load_idt(void); void setup_alt_stack(void); struct ex_regs { diff --git a/lib/x86/setup.c b/lib/x86/setup.c index 86ba6de..94e9f86 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -170,7 +170,6 @@ void setup_multiboot(struct mbi_bootinfo *bi) #ifdef CONFIG_EFI /* From x86/efi/efistart64.S */ -extern void load_idt(void); extern void load_gdt_tss(size_t tss_offset); static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo) diff --git a/x86/cstart64.S b/x86/cstart64.S index f371d06..30012ca 100644 --- a/x86/cstart64.S +++ b/x86/cstart64.S @@ -66,7 +66,6 @@ MSR_GS_BASE = 0xc0000101 .endm .macro load_tss - lidtq idt_descr movq %rsp, %rdi call setup_tss ltr %ax @@ -191,6 +190,7 @@ save_id: ap_start64: call reset_apic + call load_idt load_tss call enable_apic call save_id @@ -204,6 +204,7 @@ ap_start64: start64: call reset_apic + call load_idt load_tss call mask_pic_interrupts call enable_apic diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 0425153..ea3d1c0 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -26,11 +26,6 @@ ptl4: .code64 .text -.globl load_idt -load_idt: - lidtq idt_descr(%rip) - retq - .globl load_gdt_tss load_gdt_tss: /* Load GDT */ From patchwork Tue Apr 12 17:32:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811044 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F000C433EF for ; Tue, 12 Apr 2022 17:32:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234605AbiDLRfL (ORCPT ); Tue, 12 Apr 2022 13:35:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358212AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB48C62103 for ; Tue, 12 Apr 2022 10:32:26 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 64CAF1F856; Tue, 12 Apr 2022 17:32:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784745; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/pFXfUXu/vz1zLE9GWCvC1fGrTciKuC4HIFFLsdpaco=; b=pwYLAm05u5+/SDhv9B26QJAZtl1ggzn0HECef4eS8isWev7BGS2SNXDvH0Xdrk1CwUx9rQ OViEk126Wjik12wU4tHAZyloSfaQtpAWbq7yFQvL2xh/26ddj5p5oFFxvTvyqvwKnMqQKY 7RMz1rZ0VjbxhtzGujFPzcL3FrHvSHY= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CD85013780; Tue, 12 Apr 2022 17:32:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id uNbNL6i3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:24 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 03/10] x86: desc: Split IDT entry setup into a generic helper Date: Tue, 12 Apr 2022 19:32:14 +0200 Message-Id: <20220412173221.13315-3-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org EFI bootstrapping code configures a call gate in a later commit to jump from 16-bit to 32-bit code. Introduce a set_idt_entry_t() routine which can be used to fill both an interrupt descriptor and a call gate descriptor on x86. Signed-off-by: Varad Gautam --- lib/x86/desc.c | 28 ++++++++++++++++++++++------ lib/x86/desc.h | 1 + 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/lib/x86/desc.c b/lib/x86/desc.c index 087e85c..049adeb 100644 --- a/lib/x86/desc.c +++ b/lib/x86/desc.c @@ -57,22 +57,38 @@ __attribute__((regparm(1))) #endif void do_handle_exception(struct ex_regs *regs); -void set_idt_entry(int vec, void *addr, int dpl) +/* + * Fill an idt_entry_t, clearing e_sz bytes first. + * + * This can also be used to set up x86 call gates, since the gate + * descriptor layout is identical to idt_entry_t, except for the + * absence of .offset2 and .reserved fields. To do so, pass in e_sz + * according to the gate descriptor size. + */ +void set_idt_entry_t(idt_entry_t *e, size_t e_sz, void *addr, + u16 sel, u16 type, u16 dpl) { - idt_entry_t *e = &boot_idt[vec]; - memset(e, 0, sizeof *e); + memset(e, 0, e_sz); e->offset0 = (unsigned long)addr; - e->selector = read_cs(); + e->selector = sel; e->ist = 0; - e->type = 14; + e->type = type; e->dpl = dpl; e->p = 1; e->offset1 = (unsigned long)addr >> 16; #ifdef __x86_64__ - e->offset2 = (unsigned long)addr >> 32; + if (e_sz == sizeof(*e)) { + e->offset2 = (unsigned long)addr >> 32; + } #endif } +void set_idt_entry(int vec, void *addr, int dpl) +{ + idt_entry_t *e = &boot_idt[vec]; + set_idt_entry_t(e, sizeof *e, addr, read_cs(), 14, dpl); +} + void set_idt_dpl(int vec, u16 dpl) { idt_entry_t *e = &boot_idt[vec]; diff --git a/lib/x86/desc.h b/lib/x86/desc.h index 3044409..ae0928f 100644 --- a/lib/x86/desc.h +++ b/lib/x86/desc.h @@ -217,6 +217,7 @@ unsigned exception_vector(void); int write_cr4_checking(unsigned long val); unsigned exception_error_code(void); bool exception_rflags_rf(void); +void set_idt_entry_t(idt_entry_t *e, size_t e_sz, void *addr, u16 sel, u16 type, u16 dpl); void set_idt_entry(int vec, void *addr, int dpl); void set_idt_sel(int vec, u16 sel); void set_idt_dpl(int vec, u16 dpl); From patchwork Tue Apr 12 17:32:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7536DC433F5 for ; Tue, 12 Apr 2022 17:33:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344705AbiDLRfP (ORCPT ); Tue, 12 Apr 2022 13:35:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358277AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 566086212B for ; Tue, 12 Apr 2022 10:32:27 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 0E4BA212C2; Tue, 12 Apr 2022 17:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784746; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+HSlUzAnDrnkt5ZIEqctxfIdxXPjKoe0ncBFZr0OO60=; b=ros5k46d9rq4XrQnEYWVfhaTrXGo1ezn0rVJJxGmjiLkFkXurfUVEWF356XtNZwy0jdoqQ T1gmnZ9i8DQIB7QK3/rYTtFCiSwqEeqaJG9BtK2nfoSFyNhVwAmnhx8V34oG21aXJdjJDQ lrEyemNSbEsp1fGKZ8khr0G+zip4fHU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 72E1113780; Tue, 12 Apr 2022 17:32:25 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CBOtGam3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:25 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 04/10] x86: Move load_gdt_tss() to desc.c Date: Tue, 12 Apr 2022 19:32:15 +0200 Message-Id: <20220412173221.13315-4-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Split load_gdt_tss() functionality into: 1. Load gdt/tss 2. Setup segments in 64-bit mode and update %cs via far-return and move load_gdt_tss() to desc.c to share this code between EFI and non-EFI tests. Signed-off-by: Varad Gautam --- lib/x86/desc.c | 6 ++++++ lib/x86/desc.h | 1 + lib/x86/setup.c | 4 +++- x86/efi/efistart64.S | 11 ++--------- 4 files changed, 12 insertions(+), 10 deletions(-) diff --git a/lib/x86/desc.c b/lib/x86/desc.c index 049adeb..3b1208b 100644 --- a/lib/x86/desc.c +++ b/lib/x86/desc.c @@ -362,6 +362,12 @@ void set_gdt_entry(int sel, unsigned long base, u32 limit, u8 type, u8 flags) #endif } +void load_gdt_tss(size_t tss_offset) +{ + lgdt(&gdt_descr); + ltr(tss_offset); +} + #ifndef __x86_64__ void set_gdt_task_gate(u16 sel, u16 tss_sel) { diff --git a/lib/x86/desc.h b/lib/x86/desc.h index ae0928f..83a16dd 100644 --- a/lib/x86/desc.h +++ b/lib/x86/desc.h @@ -222,6 +222,7 @@ void set_idt_entry(int vec, void *addr, int dpl); void set_idt_sel(int vec, u16 sel); void set_idt_dpl(int vec, u16 dpl); void set_gdt_entry(int sel, unsigned long base, u32 limit, u8 access, u8 gran); +void load_gdt_tss(size_t tss_offset); void set_intr_alt_stack(int e, void *fn); void print_current_tss_info(void); handler handle_exception(u8 v, handler fn); diff --git a/lib/x86/setup.c b/lib/x86/setup.c index 94e9f86..7dd6677 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -170,7 +170,7 @@ void setup_multiboot(struct mbi_bootinfo *bi) #ifdef CONFIG_EFI /* From x86/efi/efistart64.S */ -extern void load_gdt_tss(size_t tss_offset); +extern void setup_segments64(void); static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo) { @@ -275,6 +275,8 @@ static void setup_gdt_tss(void) /* 64-bit setup_tss does not use the stacktop argument. */ tss_offset = setup_tss(NULL); load_gdt_tss(tss_offset); + + setup_segments64(); } efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index ea3d1c0..8eadca5 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -26,15 +26,8 @@ ptl4: .code64 .text -.globl load_gdt_tss -load_gdt_tss: - /* Load GDT */ - lgdt gdt_descr(%rip) - - /* Load TSS */ - mov %rdi, %rax - ltr %ax - +.globl setup_segments64 +setup_segments64: /* Update data segments */ mov $0x10, %ax /* 3rd entry in gdt64: 32/64-bit data segment */ mov %ax, %ds From patchwork Tue Apr 12 17:32:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B8BBC433EF for ; Tue, 12 Apr 2022 17:33:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346023AbiDLRfT (ORCPT ); Tue, 12 Apr 2022 13:35:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358281AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1884580C5 for ; Tue, 12 Apr 2022 10:32:27 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A5B381F859; Tue, 12 Apr 2022 17:32:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784746; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iZzCdYtvFcZR50a4J5LCpvyULpVwo2Z9H1VXfFAXa4E=; b=ayGfVnNhAmrTN2yX9qW1ZMuFvEVWyOVrEYUPEDh4CXlqz5s19MJwBbChB32wSTPUlMtJ5h SfhU/OOCqStBa6Ee2PEq+4o2H0zhxYC1xb83bv5k51MMwVa18yQ58jCD0/7rAMs10I5xLh BRHFaNIZH0+ARfYeWl/qLGz/64YLMIM= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1BFF713780; Tue, 12 Apr 2022 17:32:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id QASoBKq3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:26 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 05/10] x86: efi: Stop using UEFI-provided stack Date: Tue, 12 Apr 2022 19:32:16 +0200 Message-Id: <20220412173221.13315-5-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org UEFI test builds currently use the stack pointer configured by the UEFI implementation. Reserve stack space in .data for EFI testcases and switch %rsp to use this memory on early boot. This provides one 4K page per CPU to store its stack / percpu data. Signed-off-by: Varad Gautam --- x86/efi/crt0-efi-x86_64.S | 3 +++ x86/efi/efistart64.S | 8 ++++++++ 2 files changed, 11 insertions(+) diff --git a/x86/efi/crt0-efi-x86_64.S b/x86/efi/crt0-efi-x86_64.S index eaf1656..1708ed5 100644 --- a/x86/efi/crt0-efi-x86_64.S +++ b/x86/efi/crt0-efi-x86_64.S @@ -58,6 +58,9 @@ _start: popq %rdi popq %rsi + /* Switch away from EFI stack. */ + lea stacktop(%rip), %rsp + call efi_main addq $8, %rsp diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 8eadca5..cb08230 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -6,6 +6,14 @@ .data +max_cpus = MAX_TEST_CPUS + +/* Reserve stack in .data */ + . = . + 4096 * max_cpus + .align 16 +.globl stacktop +stacktop: + .align PAGE_SIZE .globl ptl2 ptl2: From patchwork Tue Apr 12 17:32:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8342EC433EF for ; Tue, 12 Apr 2022 17:33:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352207AbiDLRfV (ORCPT ); Tue, 12 Apr 2022 13:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358290AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FF3F62133 for ; Tue, 12 Apr 2022 10:32:28 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4BFD61F85B; Tue, 12 Apr 2022 17:32:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784747; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zMgbODgQd4DT4BH60EetBQO7SAUNkwNAdYwqbBoEWQI=; b=rMP3sb2U0+p4uoDxDixhSXqpgHRO1eDHHhya8psoClubruRCgBIUt9K8pGpWjhvQyd6ctk errfBEzaSqE94X+VBv94F3OfklUchGCVMgw/g5P7QC8odow6ndk57CyMbwKLN/G8PJRTh9 tG73U4vbjxyWYLZyITtEP9ENEns23tA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id B34DE13780; Tue, 12 Apr 2022 17:32:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id AIl/Kaq3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:26 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 06/10] x86: efi: Stop using UEFI-provided %gs for percpu storage Date: Tue, 12 Apr 2022 19:32:17 +0200 Message-Id: <20220412173221.13315-6-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org UEFI tests do not update MSR_GS_BASE during bringup, and continue using the GS_BASE set up by the UEFI implementation for percpu storage. Update this MSR during setup_segments64() to allow storing percpu data at a sane location reserved by the testcase, and ensure that this happens before any operation that ends up storing to the percpu space. Signed-off-by: Varad Gautam --- lib/x86/setup.c | 9 ++++++--- x86/efi/efistart64.S | 7 +++++++ 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/lib/x86/setup.c b/lib/x86/setup.c index 7dd6677..5d32d3f 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -170,7 +170,8 @@ void setup_multiboot(struct mbi_bootinfo *bi) #ifdef CONFIG_EFI /* From x86/efi/efistart64.S */ -extern void setup_segments64(void); +extern void setup_segments64(u64 gs_base); +extern u8 stacktop; static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo) { @@ -271,12 +272,14 @@ static void setup_page_table(void) static void setup_gdt_tss(void) { size_t tss_offset; + u64 gs_base; /* 64-bit setup_tss does not use the stacktop argument. */ tss_offset = setup_tss(NULL); load_gdt_tss(tss_offset); - setup_segments64(); + gs_base = (u64)(&stacktop) - (PAGE_SIZE * (pre_boot_apic_id() + 1)); + setup_segments64(gs_base); } efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) @@ -318,8 +321,8 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) return status; } - reset_apic(); setup_gdt_tss(); + reset_apic(); setup_idt(); load_idt(); mask_pic_interrupts(); diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index cb08230..1c38355 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -44,6 +44,13 @@ setup_segments64: mov %ax, %gs mov %ax, %ss + /* Setup percpu base */ + MSR_GS_BASE = 0xc0000101 + mov %rdi, %rax + mov $0, %edx + mov $MSR_GS_BASE, %ecx + wrmsr + /* * Update the code segment by putting it on the stack before the return * address, then doing a far return: this will use the new code segment From patchwork Tue Apr 12 17:32:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07AB2C433FE for ; Tue, 12 Apr 2022 17:33:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352417AbiDLRfW (ORCPT ); Tue, 12 Apr 2022 13:35:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358296AbiDLRfE (ORCPT ); Tue, 12 Apr 2022 13:35:04 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3479859390 for ; Tue, 12 Apr 2022 10:32:29 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E6CBC210EE; Tue, 12 Apr 2022 17:32:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784747; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Io2XlkanWMGUITfmzhn32guih3M+Uf4bisZWPwHYYuA=; b=odePKD478uu4p+utKJJHJuVs+yiE52N7H8wcG8VKo16XHLkvK7hb9o3E5kC0/dEIHSXDtN mmBeB1OmiWM74xVcXn61AuHBDbpIQHg5Q3yq+Za9Feief9cUTl3olPGyhfuYpkXPn20YCk QLgWBBR7ShbnvQ9PGRAIe+geyNlyISw= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 57FE313780; Tue, 12 Apr 2022 17:32:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id cL6BE6u3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:27 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 07/10] x86: efi, smp: Transition APs from 16-bit to 32-bit mode Date: Tue, 12 Apr 2022 19:32:18 +0200 Message-Id: <20220412173221.13315-7-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Sending INIT/SIPI to APs from ap_init() resets them into 16-bit mode to loop into sipi_entry(). To drive the APs into 32-bit mode, the SIPI vector needs: 1. A GDT descriptor reachable from 16-bit code (gdt32_descr). 2. A 32-bit entrypoint reachable from 16-bit code (ap_start32). 3. The locations of GDT and the 32-bit entrypoint. Setting these up at compile time (like on non-EFI builds) is not possible since EFI builds with -shared -fPIC and efistart64.S cannot reference any absolute addresses. Relative addressing is unavailable on 16-bit mode. Moreover, EFI may not load the 32-bit entrypoint to be reachable from 16-bit mode. To overcome these problems, 1. Fill the GDT descriptor at runtime after relocating [sipi_entry-sipi_end] to lowmem. Since sipi_entry does not know the address of this descriptor, use the last two bytes of SIPI page to communicate it. 2. Place a call gate in the GDT to point to ap_start32. 3. Popluate sipi_entry() to lcall to ap_start32. With this, the APs can transition to 32-bit mode and loop at a known location. Signed-off-by: Varad Gautam --- lib/x86/smp.c | 56 ++++++++++++++++++++++++++++++++++++++++++++ x86/efi/efistart64.S | 29 ++++++++++++++++++++++- 2 files changed, 84 insertions(+), 1 deletion(-) diff --git a/lib/x86/smp.c b/lib/x86/smp.c index d7f5aba..5cc1648 100644 --- a/lib/x86/smp.c +++ b/lib/x86/smp.c @@ -6,6 +6,7 @@ #include "apic.h" #include "fwcfg.h" #include "desc.h" +#include "asm/page.h" #define IPI_VECTOR 0x20 @@ -144,16 +145,71 @@ void smp_reset_apic(void) atomic_inc(&active_cpus); } +#ifdef CONFIG_EFI +extern u8 gdt32_descr, gdt32, gdt32_end; +extern u8 ap_start32; +#endif + void ap_init(void) { u8 *dst_addr = 0; size_t sipi_sz = (&sipi_end - &sipi_entry) + 1; + assert(sipi_sz < PAGE_SIZE); + asm volatile("cld"); /* Relocate SIPI vector to dst_addr so it can run in 16-bit mode. */ + memset(dst_addr, 0, PAGE_SIZE); memcpy(dst_addr, &sipi_entry, sipi_sz); +#ifdef CONFIG_EFI + volatile struct descriptor_table_ptr *gdt32_descr_rel; + idt_entry_t *gate_descr; + u16 *gdt32_descr_reladdr = (u16 *) (PAGE_SIZE - sizeof(u16)); + + /* + * gdt32_descr for CONFIG_EFI needs to be filled here dynamically + * since compile time calculation of offsets is not allowed when + * building with -shared, and rip-relative addressing is not supported + * in 16-bit mode. + * + * Use the last two bytes of SIPI page to store relocated gdt32_descr + * addr. + */ + *gdt32_descr_reladdr = (&gdt32_descr - &sipi_entry); + + gdt32_descr_rel = (struct descriptor_table_ptr *) ((u64) *gdt32_descr_reladdr); + gdt32_descr_rel->limit = (u16) (&gdt32_end - &gdt32 - 1); + gdt32_descr_rel->base = (ulong) ((u32) (&gdt32 - &sipi_entry)); + + /* + * EFI may not load the 32-bit AP entrypoint (ap_start32) low enough + * to be reachable from the SIPI vector. Since we build with -shared, this + * location needs to be fetched at runtime, and rip-relative addressing is + * not supported in 16-bit mode. + * To perform 16-bit -> 32-bit far jump, our options are: + * - ljmpl $cs, $label : unusable since $label is not known at build time. + * - push $cs; push $label; lret : requires an intermediate trampoline since + * $label must still be within 0 - 0xFFFF for 16-bit far return to work. + * - lcall into a call-gate : best suited. + * + * Set up call gate to ap_start32 within GDT. + * + * gdt32 layout: + * + * Entry | Segment + * 0 | NULL descr + * 1 | Code segment descr + * 2 | Data segment descr + * 3 | Call gate descr + */ + gate_descr = (idt_entry_t *) ((u8 *)(&gdt32 - &sipi_entry) + + 3 * sizeof(gdt_entry_t)); + set_idt_entry_t(gate_descr, sizeof(gdt_entry_t), (void *) &ap_start32, + 0x8 /* sel */, 0xc /* type */, 0 /* dpl */); +#endif + /* INIT */ apic_icr_write(APIC_DEST_ALLBUT | APIC_DEST_PHYSICAL | APIC_DM_INIT | APIC_INT_ASSERT, 0); diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 1c38355..00279b8 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -65,7 +65,34 @@ setup_segments64: .globl sipi_entry sipi_entry: - jmp sipi_entry + mov %cr0, %eax + or $1, %eax + mov %eax, %cr0 + + /* Retrieve relocated gdt32_descr address at (PAGE_SIZE - 2). */ + mov (PAGE_SIZE - 2), %ebx + lgdtl (%ebx) + + lcall $0x18, $0x0 + +.globl gdt32 +gdt32: + .quad 0 + .quad 0x00cf9b000000ffff // flat 32-bit code segment + .quad 0x00cf93000000ffff // flat 32-bit data segment + .quad 0 // call gate to 32-bit AP entrypoint +.globl gdt32_end +gdt32_end: + +.globl gdt32_descr +gdt32_descr: + .word 0 + .long 0 .globl sipi_end sipi_end: + +.code32 +.globl ap_start32 +ap_start32: + jmp ap_start32 From patchwork Tue Apr 12 17:32:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47948C433F5 for ; Tue, 12 Apr 2022 17:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352784AbiDLRfZ (ORCPT ); Tue, 12 Apr 2022 13:35:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358303AbiDLRfF (ORCPT ); Tue, 12 Apr 2022 13:35:05 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D78ED5BE49 for ; Tue, 12 Apr 2022 10:32:29 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 89715215FE; Tue, 12 Apr 2022 17:32:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784748; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JOznw2/rVQNwAfti2sXZsErpfAdmx9iEccd5lEv0c4g=; b=GS130OoDhZ5eEZMJgbziGbn3dZSJjWKOwNwSYmzJ4+Fl635zbvcdmku2R4AKvuk0VdocQM +APXc+CY3eoL/RVvyWNcp0BjxCWJFBxUKO1GcYFyTsF+aSauBswh3cxY+QGcY5RhvfkLEb XX4y/wc0D99ag/dP/sk61z5cziKNqMw= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 009B113780; Tue, 12 Apr 2022 17:32:27 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2DPOOau3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:27 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 08/10] x86: Move 32-bit bringup routines to start32.S Date: Tue, 12 Apr 2022 19:32:19 +0200 Message-Id: <20220412173221.13315-8-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org These can be shared across EFI and non-EFI builds. Signed-off-by: Varad Gautam --- x86/cstart64.S | 60 +----------------------------------------------- x86/start32.S | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 59 deletions(-) create mode 100644 x86/start32.S diff --git a/x86/cstart64.S b/x86/cstart64.S index 30012ca..6eb109d 100644 --- a/x86/cstart64.S +++ b/x86/cstart64.S @@ -56,35 +56,13 @@ mb_flags = 0x0 .long mb_magic, mb_flags, 0 - (mb_magic + mb_flags) mb_cmdline = 16 -MSR_GS_BASE = 0xc0000101 - -.macro setup_percpu_area - lea -4096(%esp), %eax - mov $0, %edx - mov $MSR_GS_BASE, %ecx - wrmsr -.endm - .macro load_tss movq %rsp, %rdi call setup_tss ltr %ax .endm -.macro setup_segments - mov $MSR_GS_BASE, %ecx - rdmsr - - mov $0x10, %bx - mov %bx, %ds - mov %bx, %es - mov %bx, %fs - mov %bx, %gs - mov %bx, %ss - - /* restore MSR_GS_BASE */ - wrmsr -.endm +#include "start32.S" .globl start start: @@ -118,33 +96,6 @@ switch_to_5level: call enter_long_mode jmpl $8, $lvl5 -prepare_64: - lgdt gdt_descr - setup_segments - - xor %eax, %eax - mov %eax, %cr4 - -enter_long_mode: - mov %cr4, %eax - bts $5, %eax // pae - mov %eax, %cr4 - - mov pt_root, %eax - mov %eax, %cr3 - -efer = 0xc0000080 - mov $efer, %ecx - rdmsr - bts $8, %eax - wrmsr - - mov %cr0, %eax - bts $0, %eax - bts $31, %eax - mov %eax, %cr0 - ret - smp_stacktop: .long stacktop - 4096 .align 16 @@ -171,15 +122,6 @@ gdt32_descr: .globl sipi_end sipi_end: -.code32 -ap_start32: - setup_segments - mov $-4096, %esp - lock xaddl %esp, smp_stacktop - setup_percpu_area - call prepare_64 - ljmpl $8, $ap_start64 - .code64 save_id: movl $(APIC_DEFAULT_PHYS_BASE + APIC_ID), %eax diff --git a/x86/start32.S b/x86/start32.S new file mode 100644 index 0000000..9e00474 --- /dev/null +++ b/x86/start32.S @@ -0,0 +1,62 @@ +/* Common 32-bit code between EFI and non-EFI bootstrapping. */ + +.code32 + +MSR_GS_BASE = 0xc0000101 + +.macro setup_percpu_area + lea -4096(%esp), %eax + mov $0, %edx + mov $MSR_GS_BASE, %ecx + wrmsr +.endm + +.macro setup_segments + mov $MSR_GS_BASE, %ecx + rdmsr + + mov $0x10, %bx + mov %bx, %ds + mov %bx, %es + mov %bx, %fs + mov %bx, %gs + mov %bx, %ss + + /* restore MSR_GS_BASE */ + wrmsr +.endm + +prepare_64: + lgdt gdt_descr + setup_segments + + xor %eax, %eax + mov %eax, %cr4 + +enter_long_mode: + mov %cr4, %eax + bts $5, %eax // pae + mov %eax, %cr4 + + mov pt_root, %eax + mov %eax, %cr3 + +efer = 0xc0000080 + mov $efer, %ecx + rdmsr + bts $8, %eax + wrmsr + + mov %cr0, %eax + bts $0, %eax + bts $31, %eax + mov %eax, %cr0 + ret + +ap_start32: + setup_segments + mov $-4096, %esp + lock xaddl %esp, smp_stacktop + setup_percpu_area + call prepare_64 + ljmpl $8, $ap_start64 From patchwork Tue Apr 12 17:32:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EC4FC433FE for ; Tue, 12 Apr 2022 17:33:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352685AbiDLRf0 (ORCPT ); Tue, 12 Apr 2022 13:35:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358308AbiDLRfF (ORCPT ); Tue, 12 Apr 2022 13:35:05 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8790F57B0F for ; Tue, 12 Apr 2022 10:32:30 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 34D45212C2; Tue, 12 Apr 2022 17:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784749; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CBjx8U+Z04M3A/pBNknkTYb+aqc/22NGqocCiDbQczY=; b=TcWSFnRHaw4GzPeGf8tH3dzreFO54OIRuE3P2AJwRhkYOlaLObVkpy8hxuwq3J3aCUi5MG fHAUHSJs/ZK1UcBv36gWtW+dsqV7MsO+1ukR+pjQgJhlTcMV6QQAy6bG1W1Jd2rzniH7D0 i16bHW9QsdghuxVjpnFnAwvHyThels0= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 98FA513780; Tue, 12 Apr 2022 17:32:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GB83I6y3VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:28 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 09/10] x86: efi, smp: Transition APs from 32-bit to 64-bit mode Date: Tue, 12 Apr 2022 19:32:20 +0200 Message-Id: <20220412173221.13315-9-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Reaching 64-bit mode requires setting up a valid stack and percpu regions for each CPU and configuring a page table before far-jumping to the 64-bit entrypoint. This functionality is already present as prepare_64() and ap_start32() routines in start32.S for non-EFI test builds. However since EFI builds (-fPIC) cannot use absolute addressing, and 32-bit mode does not allow RIP-relative addressing, these routines need some changes. Modify prepare_64() and ap_start32() asm routines to calculate label addresses during runtime on CONFIG_EFI. To ease the common case, replace the far-jump to ap_start64() with a far-return. This brings the APs into 64-bit mode to loop at a known location. Signed-off-by: Varad Gautam --- lib/x86/setup.c | 2 +- lib/x86/smp.c | 3 +++ x86/efi/efistart64.S | 13 ++++++++++--- x86/start32.S | 46 +++++++++++++++++++++++++++++++++++++++++--- 4 files changed, 57 insertions(+), 7 deletions(-) diff --git a/lib/x86/setup.c b/lib/x86/setup.c index 5d32d3f..e2f7967 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -326,11 +326,11 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) setup_idt(); load_idt(); mask_pic_interrupts(); + setup_page_table(); enable_apic(); ap_init(); enable_x2apic(); smp_init(); - setup_page_table(); return EFI_SUCCESS; } diff --git a/lib/x86/smp.c b/lib/x86/smp.c index 5cc1648..cee82ac 100644 --- a/lib/x86/smp.c +++ b/lib/x86/smp.c @@ -148,6 +148,8 @@ void smp_reset_apic(void) #ifdef CONFIG_EFI extern u8 gdt32_descr, gdt32, gdt32_end; extern u8 ap_start32; +extern u32 smp_stacktop; +extern u8 stacktop; #endif void ap_init(void) @@ -168,6 +170,7 @@ void ap_init(void) idt_entry_t *gate_descr; u16 *gdt32_descr_reladdr = (u16 *) (PAGE_SIZE - sizeof(u16)); + smp_stacktop = ((u64) (&stacktop)) - 4096; /* * gdt32_descr for CONFIG_EFI needs to be filled here dynamically * since compile time calculation of offsets is not allowed when diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 00279b8..149e3f7 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -14,6 +14,9 @@ max_cpus = MAX_TEST_CPUS .globl stacktop stacktop: +.globl smp_stacktop +smp_stacktop: .long 0 + .align PAGE_SIZE .globl ptl2 ptl2: @@ -93,6 +96,10 @@ gdt32_descr: sipi_end: .code32 -.globl ap_start32 -ap_start32: - jmp ap_start32 + +#include "../start32.S" + +.code64: + +ap_start64: + jmp ap_start64 diff --git a/x86/start32.S b/x86/start32.S index 9e00474..2089be7 100644 --- a/x86/start32.S +++ b/x86/start32.S @@ -27,7 +27,16 @@ MSR_GS_BASE = 0xc0000101 .endm prepare_64: - lgdt gdt_descr +#ifdef CONFIG_EFI + call prepare_64_1 +prepare_64_1: + pop %edx + add $gdt_descr - prepare_64_1, %edx +#else + mov $gdt_descr, %edx +#endif + lgdtl (%edx) + setup_segments xor %eax, %eax @@ -38,7 +47,14 @@ enter_long_mode: bts $5, %eax // pae mov %eax, %cr4 +#ifdef CONFIG_EFI + call prepare_64_2 +prepare_64_2: + pop %eax + add $ptl4 - prepare_64_2, %eax +#else mov pt_root, %eax +#endif mov %eax, %cr3 efer = 0xc0000080 @@ -53,10 +69,34 @@ efer = 0xc0000080 mov %eax, %cr0 ret +.globl ap_start32 ap_start32: setup_segments + +#ifdef CONFIG_EFI + call ap_start32_1 +ap_start32_1: + pop %edx + add $smp_stacktop - ap_start32_1, %edx +#else + mov $smp_stacktop, %edx +#endif mov $-4096, %esp - lock xaddl %esp, smp_stacktop + lock xaddl %esp, (%edx) + setup_percpu_area call prepare_64 - ljmpl $8, $ap_start64 + +#ifdef CONFIG_EFI + call ap_start32_2 +ap_start32_2: + pop %edx + add $ap_start64 - ap_start32_2, %edx +#else + mov $ap_start64, %edx +#endif + + pushl $0x08 + pushl %edx + + lretl From patchwork Tue Apr 12 17:32:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Varad Gautam X-Patchwork-Id: 12811053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D0E5C433F5 for ; Tue, 12 Apr 2022 17:33:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358134AbiDLRf2 (ORCPT ); Tue, 12 Apr 2022 13:35:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358309AbiDLRfF (ORCPT ); Tue, 12 Apr 2022 13:35:05 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 283985BE54 for ; Tue, 12 Apr 2022 10:32:31 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C8E8A215FF; Tue, 12 Apr 2022 17:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649784749; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OinhSGityeOeLhFUHCEBqO4+4VkBx2Lm7bYMHLdQlrM=; b=FFC9cdyPj5tbNN+1EPyTqs5QZ55en2XPlmZQ8E5Kwxl+NsTH0iI1nHGGZlIytETTFoS1RM MFmCiJa6o4LsLHiVoL8oU8dFC6eJYNM3niBOwUlhhz9WNSJvI/376ulw8+4pO87/hASkyT GhHuVNi3hSXV8VNX9bDzwMiFViUEX0E= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 421CB13780; Tue, 12 Apr 2022 17:32:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WG8bDq23VWJJewAAMHmgww (envelope-from ); Tue, 12 Apr 2022 17:32:29 +0000 From: Varad Gautam To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, drjones@redhat.com, marcorr@google.com, zxwang42@gmail.com, erdemaktas@google.com, rientjes@google.com, seanjc@google.com, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jroedel@suse.de, bp@suse.de, varad.gautam@suse.com Subject: [PATCH 10/10] x86: Move ap_start64 and save_id to setup.c Date: Tue, 12 Apr 2022 19:32:21 +0200 Message-Id: <20220412173221.13315-10-varad.gautam@suse.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220412173221.13315-1-varad.gautam@suse.com> References: <20220412173221.13315-1-varad.gautam@suse.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Writing ap_start64() and save_id() in C allows sharing these routines between EFI and non-EFI tests. Move these to setup.c and use ap_start64 as the 64-bit entrypoint in the EFI boot flow. Signed-off-by: Varad Gautam --- lib/x86/asm/setup.h | 3 +++ lib/x86/setup.c | 54 +++++++++++++++++++++++++++++++++----------- lib/x86/smp.c | 1 + x86/cstart64.S | 24 -------------------- x86/efi/efistart64.S | 5 ---- 5 files changed, 45 insertions(+), 42 deletions(-) diff --git a/lib/x86/asm/setup.h b/lib/x86/asm/setup.h index 24d4fa9..8502e7d 100644 --- a/lib/x86/asm/setup.h +++ b/lib/x86/asm/setup.h @@ -16,4 +16,7 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo); void setup_5level_page_table(void); #endif /* CONFIG_EFI */ +void save_id(void); +void ap_start64(void); + #endif /* _X86_ASM_SETUP_H_ */ diff --git a/lib/x86/setup.c b/lib/x86/setup.c index e2f7967..a0e0b0c 100644 --- a/lib/x86/setup.c +++ b/lib/x86/setup.c @@ -14,8 +14,12 @@ #include "apic.h" #include "apic-defs.h" #include "asm/setup.h" +#include "processor.h" +#include "atomic.h" extern char edata; +extern unsigned char online_cpus[(MAX_TEST_CPUS + 7) / 8]; +extern unsigned cpu_online_count; struct mbi_bootinfo { u32 flags; @@ -172,7 +176,22 @@ void setup_multiboot(struct mbi_bootinfo *bi) /* From x86/efi/efistart64.S */ extern void setup_segments64(u64 gs_base); extern u8 stacktop; +#endif + +static void setup_gdt_tss(void) +{ + size_t tss_offset; + /* 64-bit setup_tss does not use the stacktop argument. */ + tss_offset = setup_tss(NULL); + load_gdt_tss(tss_offset); +#ifdef CONFIG_EFI + u64 gs_base = (u64)(&stacktop) - (PAGE_SIZE * (pre_boot_apic_id() + 1)); + setup_segments64(gs_base); +#endif +} + +#ifdef CONFIG_EFI static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo) { int i; @@ -269,19 +288,6 @@ static void setup_page_table(void) write_cr3((ulong)&ptl4); } -static void setup_gdt_tss(void) -{ - size_t tss_offset; - u64 gs_base; - - /* 64-bit setup_tss does not use the stacktop argument. */ - tss_offset = setup_tss(NULL); - load_gdt_tss(tss_offset); - - gs_base = (u64)(&stacktop) - (PAGE_SIZE * (pre_boot_apic_id() + 1)); - setup_segments64(gs_base); -} - efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) { efi_status_t status; @@ -328,6 +334,7 @@ efi_status_t setup_efi(efi_bootinfo_t *efi_bootinfo) mask_pic_interrupts(); setup_page_table(); enable_apic(); + save_id(); ap_init(); enable_x2apic(); smp_init(); @@ -350,3 +357,24 @@ void setup_libcflat(void) add_setup_arg("bootloader"); } } + +void save_id(void) +{ + u32 id = apic_id(); + + /* atomic_fetch_or() emits `lock or %dl, (%eax)` */ + atomic_fetch_or(&online_cpus[id / 8], (1 << (id % 8))); +} + +void ap_start64(void) +{ + setup_gdt_tss(); + reset_apic(); + load_idt(); + save_id(); + enable_apic(); + enable_x2apic(); + sti(); + atomic_fetch_inc(&cpu_online_count); + asm volatile("1: hlt; jmp 1b"); +} diff --git a/lib/x86/smp.c b/lib/x86/smp.c index cee82ac..779d346 100644 --- a/lib/x86/smp.c +++ b/lib/x86/smp.c @@ -22,6 +22,7 @@ static atomic_t active_cpus; extern u8 sipi_entry; extern u8 sipi_end; volatile unsigned cpu_online_count = 1; +unsigned char online_cpus[(MAX_TEST_CPUS + 7) / 8]; static __attribute__((used)) void ipi(void) { diff --git a/x86/cstart64.S b/x86/cstart64.S index 6eb109d..6363293 100644 --- a/x86/cstart64.S +++ b/x86/cstart64.S @@ -123,27 +123,6 @@ gdt32_descr: sipi_end: .code64 -save_id: - movl $(APIC_DEFAULT_PHYS_BASE + APIC_ID), %eax - movl (%rax), %eax - shrl $24, %eax - lock btsl %eax, online_cpus - retq - -ap_start64: - call reset_apic - call load_idt - load_tss - call enable_apic - call save_id - call enable_x2apic - sti - nop - lock incw cpu_online_count - -1: hlt - jmp 1b - start64: call reset_apic call load_idt @@ -182,6 +161,3 @@ setup_5level_page_table: lretq lvl5: retq - -online_cpus: - .fill (max_cpus + 7) / 8, 1, 0 diff --git a/x86/efi/efistart64.S b/x86/efi/efistart64.S index 149e3f7..a5d7219 100644 --- a/x86/efi/efistart64.S +++ b/x86/efi/efistart64.S @@ -98,8 +98,3 @@ sipi_end: .code32 #include "../start32.S" - -.code64: - -ap_start64: - jmp ap_start64