diff mbox

[v3,15/16,-,RFC] x86: make Xen early boot code relocatable

Message ID 1460723596-13261-16-git-send-email-daniel.kiper@oracle.com (mailing list archive)
State New, archived
Headers show

Commit Message

Daniel Kiper April 15, 2016, 12:33 p.m. UTC
Every multiboot protocol (regardless of version) compatible image must
specify its load address (in ELF or multiboot header). Multiboot protocol
compatible loader have to load image at specified address. However, there
is no guarantee that the requested memory region (in case of Xen it starts
at 1 MiB and ends at 17 MiB) where image should be loaded initially is a RAM
and it is free (legacy BIOS platforms are merciful for Xen but I found at
least one EFI platform on which Xen load address conflicts with EFI boot
services; it is Dell PowerEdge R820 with latest firmware). To cope with
that problem we must make Xen early boot code relocatable. This patch does
that. However, it does not add multiboot2 protocol interface which is done
in "x86: add multiboot2 protocol support for relocatable images" patch.

This patch changes following things:
  - default load address is changed from 1 MiB to 2 MiB; I did that because
    initial page tables are using 2 MiB huge pages and this way required
    updates for them are quite easy; it means that e.g. we avoid spacial
    cases for start and end of required memory region if it live at address
    not aligned to 2 MiB,
  - %esi and %r15d registers are used as a storage for Xen image load base
    address (%r15d shortly because %rsi is used for EFI SystemTable address
    in 64-bit code); both registers are (%esi is mostly) unused in early
    boot code and preserved during C functions calls,
  - %esi is used as base for Xen data relative addressing in 32-bit code.

Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
---
v3 - suggestions/fixes:
   - improve segment registers initialization
     (suggested by Jan Beulich),
   - simplify Xen image load base address calculation
     (suggested by Jan Beulich),
   - use %esi and %r15d instead of %ebp to store
     Xen image load base address,
   - use %esi instead of %fs for relative addressing;
     this way we get shorter and simpler code,
   - rename some variables and constants
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Konrad Rzeszutek Wilk),
   - improve commit message
     (suggested by Jan Beulich).

v3 - not fixed yet:
   - small issue with remapping code in xen/arch/x86/setup.c,
   - <final-exec-addr> mkelf32 argument should
     be calculated dynamically; this issue has
     minimal impact on other parts of this patch.
---
 xen/arch/x86/Makefile          |    6 +-
 xen/arch/x86/Rules.mk          |    4 +
 xen/arch/x86/boot/head.S       |  162 ++++++++++++++++++++++++++++++----------
 xen/arch/x86/boot/trampoline.S |    6 +-
 xen/arch/x86/boot/wakeup.S     |    6 +-
 xen/arch/x86/boot/x86_64.S     |   44 +++++------
 xen/arch/x86/setup.c           |   44 ++++++-----
 xen/arch/x86/xen.lds.S         |    2 +-
 xen/include/asm-x86/config.h   |    1 +
 xen/include/asm-x86/page.h     |    2 +-
 10 files changed, 186 insertions(+), 91 deletions(-)

Comments

Jan Beulich May 25, 2016, 10:48 a.m. UTC | #1
>>> On 15.04.16 at 14:33, <daniel.kiper@oracle.com> wrote:
> Every multiboot protocol (regardless of version) compatible image must
> specify its load address (in ELF or multiboot header). Multiboot protocol
> compatible loader have to load image at specified address. However, there
> is no guarantee that the requested memory region (in case of Xen it starts
> at 1 MiB and ends at 17 MiB) where image should be loaded initially is a RAM
> and it is free (legacy BIOS platforms are merciful for Xen but I found at
> least one EFI platform on which Xen load address conflicts with EFI boot
> services; it is Dell PowerEdge R820 with latest firmware). To cope with
> that problem we must make Xen early boot code relocatable. This patch does
> that.

I don't follow: If we have to specify a load address, and if the
loader is required to put us there or fail, how does the code being
made relocatable help? And how does moving ourselves from 1Mb
to 2Mb make it any less likely that the designated address range is
actually free? Perhaps it's just the description that's misleading
here...

Even with that resolved I expect - as a result of the discussion on
the hackathon - quite a bit of change to this patch, so I don't view
actually reviewing the code as usefully spent time.

Jan
diff mbox

Patch

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index 32d2407..0cc6f5f 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -71,8 +71,10 @@  efi-y := $(shell if [ ! -r $(BASEDIR)/include/xen/compile.h -o \
                          echo '$(TARGET).efi'; fi)
 
 $(TARGET): $(TARGET)-syms $(efi-y) boot/mkelf32
-	./boot/mkelf32 $(TARGET)-syms $(TARGET) 0x100000 \
-	`$(NM) -nr $(TARGET)-syms | head -n 1 | sed -e 's/^\([^ ]*\).*/0x\1/'`
+#	THIS IS UGLY HACK! PLEASE DO NOT COMPLAIN. I WILL FIX IT IN NEXT RELEASE.
+	./boot/mkelf32 $(TARGET)-syms $(TARGET) $(XEN_IMG_OFFSET) 0xffff82d081000000
+#	./boot/mkelf32 $(TARGET)-syms $(TARGET) 0x100000 \
+#	`$(NM) -nr $(TARGET)-syms | head -n 1 | sed -e 's/^\([^ ]*\).*/0x\1/'`
 
 
 ALL_OBJS := $(BASEDIR)/arch/x86/boot/built_in.o $(BASEDIR)/arch/x86/efi/built_in.o $(ALL_OBJS)
diff --git a/xen/arch/x86/Rules.mk b/xen/arch/x86/Rules.mk
index 3139886..7c76f80 100644
--- a/xen/arch/x86/Rules.mk
+++ b/xen/arch/x86/Rules.mk
@@ -1,6 +1,10 @@ 
 ########################################
 # x86-specific definitions
 
+XEN_IMG_OFFSET = 0x200000
+
+CFLAGS += -DXEN_IMG_OFFSET=$(XEN_IMG_OFFSET)
+
 CFLAGS += -I$(BASEDIR)/include
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-generic
 CFLAGS += -I$(BASEDIR)/include/asm-x86/mach-default
diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 964851b..e322270 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -12,7 +12,7 @@ 
         .text
         .code32
 
-#define sym_phys(sym)     ((sym) - __XEN_VIRT_START)
+#define sym_offset(sym)   ((sym) - __XEN_VIRT_START)
 
 #define BOOT_CS32        0x0008
 #define BOOT_CS64        0x0010
@@ -94,7 +94,7 @@  multiboot2_header_start:
 
         /* EFI64 entry point. */
         mb2ht_init MB2_HT(ENTRY_ADDRESS_EFI64), MB2_HT(OPTIONAL), \
-                   sym_phys(__efi64_start)
+                   sym_offset(__efi64_start)
 
         /* Multiboot2 header end tag. */
         mb2ht_init MB2_HT(END), MB2_HT(REQUIRED)
@@ -106,11 +106,12 @@  multiboot2_header_end:
         .word   0
 gdt_boot_descr:
         .word   6*8-1
-        .long   sym_phys(trampoline_gdt)
+gdt_boot_base:
+        .long   sym_offset(trampoline_gdt)
         .long   0 /* Needed for 64-bit lgdt */
 
 cs32_switch_addr:
-        .long   sym_phys(cs32_switch)
+        .long   sym_offset(cs32_switch)
         .word   BOOT_CS32
 
 .Lbad_cpu_msg: .asciz "ERR: Not a 64-bit CPU!"
@@ -120,16 +121,16 @@  cs32_switch_addr:
         .section .init.text, "ax", @progbits
 
 bad_cpu:
-        mov     $(sym_phys(.Lbad_cpu_msg)),%esi # Error message
-        mov     $0xB8000,%edi                   # VGA framebuffer
+        lea     sym_offset(.Lbad_cpu_msg)(%esi),%esi # Error message
+        mov     $0xB8000,%edi                        # VGA framebuffer
         jmp     1f
 not_multiboot:
-        mov     $(sym_phys(.Lbad_ldr_msg)),%esi # Error message
-        mov     $0xB8000,%edi                   # VGA framebuffer
+        lea     sym_offset(.Lbad_ldr_msg)(%esi),%esi # Error message
+        mov     $0xB8000,%edi                        # VGA framebuffer
         jmp     1f
 mb2_too_old:
-        mov     $(sym_phys(.Lbad_ldr_mb2)),%esi # Error message
-        xor     %edi,%edi                       # No VGA framebuffer
+        lea     sym_offset(.Lbad_ldr_mb2)(%esi),%esi # Error message
+        xor     %edi,%edi                            # No VGA framebuffer
 1:      mov     (%esi),%bl
         test    %bl,%bl        # Terminate on '\0' sentinel
         je      .Lhalt
@@ -154,6 +155,9 @@  mb2_too_old:
 __efi64_start:
         cld
 
+        /* Load default Xen image load base address. */
+        mov     $sym_offset(__image_base__),%r15d
+
         /* Check for Multiboot2 bootloader. */
         cmp     $MULTIBOOT2_BOOTLOADER_MAGIC,%eax
         je      efi_multiboot2_proto
@@ -238,6 +242,9 @@  run_bs:
 
         pop     %rax
 
+        /* Store Xen image load base address in place accessible for 32-bit code. */
+        mov     %r15d,%esi
+
         /* Jump to trampoline_setup after switching CPU to x86_32 mode. */
         lea     trampoline_setup(%rip),%rdi
 
@@ -245,9 +252,11 @@  x86_32_switch:
         cli
 
         /* Initialise GDT. */
+        add     %esi,gdt_boot_base(%rip)
         lgdt    gdt_boot_descr(%rip)
 
         /* Reload code selector. */
+        add     %esi,cs32_switch_addr(%rip)
         ljmpl   *cs32_switch_addr(%rip)
 
         .code32
@@ -277,12 +286,8 @@  __start:
         cld
         cli
 
-        /* Initialise GDT and basic data segments. */
-        lgdt    %cs:sym_phys(gdt_boot_descr)
-        mov     $BOOT_DS,%ecx
-        mov     %ecx,%ds
-        mov     %ecx,%es
-        mov     %ecx,%ss
+        /* Load default Xen image load base address. */
+        mov     $sym_offset(__image_base__),%esi
 
         /* Bootloaders may set multiboot{1,2}.mem_lower to a nonzero value. */
         xor     %edx,%edx
@@ -334,6 +339,25 @@  multiboot2_proto:
         jmp     0b
 
 trampoline_bios_setup:
+        /*
+         * Called on legacy BIOS platforms only.
+         *
+         * Initialise GDT and basic data segments.
+         */
+        add     %esi,sym_offset(gdt_boot_base)(%esi)
+        lgdt    sym_offset(gdt_boot_descr)(%esi)
+
+        mov     $BOOT_DS,%ecx
+        mov     %ecx,%ds
+        mov     %ecx,%es
+        mov     %ecx,%ss
+        /* %esp is initialised later. */
+
+        /* Load null descriptor to unused segment registers. */
+        xor     %ecx,%ecx
+        mov     %ecx,%fs
+        mov     %ecx,%gs
+
         /* Set up trampoline segment 64k below EBDA */
         movzwl  0x40e,%ecx          /* EBDA segment */
         cmp     $0xa000,%ecx        /* sanity check (high) */
@@ -355,33 +379,42 @@  trampoline_bios_setup:
         cmovb   %edx,%ecx           /* and use the smaller */
 
 trampoline_setup:
-        /* Reserve 64kb for the trampoline. */
+        /*
+         * Called on legacy BIOS and EFI platforms.
+         *
+         * Reserve 64kb for the trampoline.
+         */
         sub     $0x1000,%ecx
 
         /* From arch/x86/smpboot.c: start_eip had better be page-aligned! */
         xor     %cl, %cl
         shl     $4, %ecx
-        mov     %ecx,sym_phys(trampoline_phys)
+        mov     %ecx,sym_offset(trampoline_phys)(%esi)
+
+        /* Save Xen image load base address for later use. */
+        mov     %esi,sym_offset(xen_img_load_base_addr)(%esi)
+
+        /* Setup stack. %ss was initialized earlier. */
+        lea     (sym_offset(cpu0_stack)+1024)(%esi),%esp
 
         /* Save the Multiboot info struct (after relocation) for later use. */
-        mov     $sym_phys(cpu0_stack)+1024,%esp
         push    %ecx                /* Boot trampoline address. */
         push    %ebx                /* Multiboot information address. */
         push    %eax                /* Multiboot magic. */
         call    reloc
         add     $12,%esp            /* Remove reloc() args from stack. */
-        mov     %eax,sym_phys(multiboot_ptr)
+        mov     %eax,sym_offset(multiboot_ptr)(%esi)
 
         /*
          * Do not zero BSS on EFI platform here.
          * It was initialized earlier.
          */
-        cmpb    $1,sym_phys(skip_realmode)
+        cmpb    $1,sym_offset(skip_realmode)(%esi)
         je      1f
 
         /* Initialize BSS (no nasty surprises!). */
-        mov     $sym_phys(__bss_start),%edi
-        mov     $sym_phys(__bss_end),%ecx
+        lea     sym_offset(__bss_start)(%esi),%edi
+        lea     sym_offset(__bss_end)(%esi),%ecx
         sub     %edi,%ecx
         shr     $2,%ecx
         xor     %eax,%eax
@@ -396,8 +429,8 @@  trampoline_setup:
         jbe     1f
         mov     $0x80000001,%eax
         cpuid
-1:      mov     %edx,sym_phys(cpuid_ext_features)
-        mov     %edx,sym_phys(boot_cpu_data)+CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM)
+1:      mov     %edx,sym_offset(cpuid_ext_features)(%esi)
+        mov     %edx,sym_offset(boot_cpu_data)+CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM)(%esi)
 
         /* Check for availability of long mode. */
         bt      $cpufeat_bit(X86_FEATURE_LM),%edx
@@ -405,64 +438,115 @@  trampoline_setup:
 
         /* Stash TSC to calculate a good approximation of time-since-boot */
         rdtsc
-        mov     %eax,sym_phys(boot_tsc_stamp)
-        mov     %edx,sym_phys(boot_tsc_stamp+4)
+        mov     %eax,sym_offset(boot_tsc_stamp)(%esi)
+        mov     %edx,sym_offset(boot_tsc_stamp+4)(%esi)
+
+        /* Update frame addreses in page tables. */
+        lea     sym_offset(__page_tables_start)(%esi),%edx
+        mov     $((__page_tables_end-__page_tables_start)/8),%ecx
+1:      testl   $_PAGE_PRESENT,(%edx)
+        jz      2f
+        add     %esi,(%edx)
+2:      add     $8,%edx
+        loop    1b
+
+        /* Initialise L2 boot-map page table entries (14MB). */
+        lea     sym_offset(l2_bootmap)(%esi),%edx
+        lea     sym_offset(start)(%esi),%eax
+        and     $~((1<<L2_PAGETABLE_SHIFT)-1),%eax
+        mov     %eax,%ebx
+        shr     $(L2_PAGETABLE_SHIFT-3),%ebx
+        and     $(L2_PAGETABLE_ENTRIES*4*8-1),%ebx
+        add     %ebx,%edx
+        add     $(PAGE_HYPERVISOR|_PAGE_PSE),%eax
+        mov     $7,%ecx
+1:      mov     %eax,(%edx)
+        add     $8,%edx
+        add     $(1<<L2_PAGETABLE_SHIFT),%eax
+        loop    1b
+
+        /* Initialise L3 boot-map page directory entry. */
+        lea     sym_offset(l2_bootmap)+__PAGE_HYPERVISOR(%esi),%eax
+        lea     sym_offset(l3_bootmap)(%esi),%ebx
+        mov     $4,%ecx
+1:      mov     %eax,(%ebx)
+        add     $8,%ebx
+        add     $(L2_PAGETABLE_ENTRIES*8),%eax
+        loop    1b
+
+        /* Initialise L2 direct map page table entries (14MB). */
+        lea     sym_offset(l2_identmap)(%esi),%edx
+        lea     sym_offset(start)(%esi),%eax
+        and     $~((1<<L2_PAGETABLE_SHIFT)-1),%eax
+        mov     %eax,%ebx
+        shr     $(L2_PAGETABLE_SHIFT-3),%ebx
+        and     $(L2_PAGETABLE_ENTRIES*4*8-1),%ebx
+        add     %ebx,%edx
+        add     $(PAGE_HYPERVISOR|_PAGE_PSE),%eax
+        mov     $7,%ecx
+1:      mov     %eax,(%edx)
+        add     $8,%edx
+        add     $(1<<L2_PAGETABLE_SHIFT),%eax
+        loop    1b
 
         /*
          * During boot, hook 4kB mappings of first 2MB of memory into L2.
          * This avoids mixing cachability for the legacy VGA region, and is
          * corrected when Xen relocates itself.
          */
-        mov     $sym_phys(l1_identmap)+__PAGE_HYPERVISOR,%edi
-        mov     %edi,sym_phys(l2_xenmap)
+        lea     (sym_offset(l1_identmap)+__PAGE_HYPERVISOR)(%esi),%edi
+        mov     %edi,sym_offset(l2_bootmap)(%esi)
 
         /* Apply relocations to bootstrap trampoline. */
-        mov     sym_phys(trampoline_phys),%edx
-        mov     $sym_phys(__trampoline_rel_start),%edi
+        mov     sym_offset(trampoline_phys)(%esi),%edx
+        lea     sym_offset(__trampoline_rel_start)(%esi),%edi
+        lea     sym_offset(__trampoline_rel_stop)(%esi),%ebx
 1:
         mov     (%edi),%eax
         add     %edx,(%edi,%eax)
         add     $4,%edi
-        cmp     $sym_phys(__trampoline_rel_stop),%edi
+        cmp     %ebx,%edi
         jb      1b
 
         /* Patch in the trampoline segment. */
         shr     $4,%edx
-        mov     $sym_phys(__trampoline_seg_start),%edi
+        lea     sym_offset(__trampoline_seg_start)(%esi),%edi
+        lea     sym_offset(__trampoline_seg_stop)(%esi),%ebx
 1:
         mov     (%edi),%eax
         mov     %dx,(%edi,%eax)
         add     $4,%edi
-        cmp     $sym_phys(__trampoline_seg_stop),%edi
+        cmp     %ebx,%edi
         jb      1b
 
         /* Do not parse command line on EFI platform here. */
-        cmpb    $1,sym_phys(skip_realmode)
+        cmpb    $1,sym_offset(skip_realmode)(%esi)
         je      1f
 
         /* Bail if there is no command line to parse. */
-        mov     sym_phys(multiboot_ptr),%ebx
+        mov     sym_offset(multiboot_ptr)(%esi),%ebx
         testl   $MBI_CMDLINE,MB_flags(%ebx)
         jz      1f
 
         cmpl    $0,MB_cmdline(%ebx)
         jz      1f
 
-        pushl   $sym_phys(early_boot_opts)
+        lea     sym_offset(early_boot_opts)(%esi),%eax
+        push    %eax
         pushl   MB_cmdline(%ebx)
         call    cmdline_parse_early
         add     $8,%esp             /* Remove cmdline_parse_early() args from stack. */
 
 1:
         /* Switch to low-memory stack.  */
-        mov     sym_phys(trampoline_phys),%edi
+        mov     sym_offset(trampoline_phys)(%esi),%edi
         lea     0x10000(%edi),%esp
         lea     trampoline_boot_cpu_entry-trampoline_start(%edi),%eax
         pushl   $BOOT_CS32
         push    %eax
 
         /* Copy bootstrap trampoline to low memory, below 1MB. */
-        mov     $sym_phys(trampoline_start),%esi
+        lea     sym_offset(trampoline_start)(%esi),%esi
         mov     $trampoline_end - trampoline_start,%ecx
         rep     movsb
 
diff --git a/xen/arch/x86/boot/trampoline.S b/xen/arch/x86/boot/trampoline.S
index 8a32728..47dd65a 100644
--- a/xen/arch/x86/boot/trampoline.S
+++ b/xen/arch/x86/boot/trampoline.S
@@ -60,6 +60,9 @@  trampoline_gdt:
         .long   trampoline_gdt + BOOT_PSEUDORM_DS + 2 - .
         .popsection
 
+GLOBAL(xen_img_load_base_addr)
+        .long   0
+
 GLOBAL(trampoline_misc_enable_off)
         .quad   0
 
@@ -87,7 +90,8 @@  trampoline_protmode_entry:
         mov     %ecx,%cr4
 
         /* Load pagetable base register. */
-        mov     $sym_phys(idle_pg_table),%eax
+        mov     bootsym_rel(xen_img_load_base_addr,4,%eax)
+        lea     sym_offset(idle_pg_table)(%eax),%eax
         add     bootsym_rel(trampoline_xen_phys_start,4,%eax)
         mov     %eax,%cr3
 
diff --git a/xen/arch/x86/boot/wakeup.S b/xen/arch/x86/boot/wakeup.S
index 08ea9b2..61c1261 100644
--- a/xen/arch/x86/boot/wakeup.S
+++ b/xen/arch/x86/boot/wakeup.S
@@ -119,8 +119,10 @@  wakeup_32:
         mov     %eax, %ss
         mov     $bootsym_rel(wakeup_stack, 4, %esp)
 
+        mov     bootsym_rel(xen_img_load_base_addr, 4, %ebx)
+
         # check saved magic again
-        mov     $sym_phys(saved_magic), %eax
+        lea     sym_offset(saved_magic)(%ebx), %eax
         add     bootsym_rel(trampoline_xen_phys_start, 4, %eax)
         mov     (%eax), %eax
         cmp     $0x9abcdef0, %eax
@@ -133,7 +135,7 @@  wakeup_32:
         mov     %ecx, %cr4
 
         /* Load pagetable base register */
-        mov     $sym_phys(idle_pg_table),%eax
+        lea     sym_offset(idle_pg_table)(%ebx),%eax
         add     bootsym_rel(trampoline_xen_phys_start,4,%eax)
         mov     %eax,%cr3
 
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 9ab9231..30d2eba 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -81,7 +81,6 @@  GLOBAL(boot_cpu_compat_gdt_table)
         .quad 0x0000910000000000     /* per-CPU entry (limit == cpu)      */
         .align PAGE_SIZE, 0
 
-GLOBAL(__page_tables_start)
 /*
  * Mapping of first 2 megabytes of memory. This is mapped with 4kB mappings
  * to avoid type conflicts with fixed-range MTRRs covering the lowest megabyte
@@ -101,18 +100,15 @@  GLOBAL(l1_identmap)
         .endr
         .size l1_identmap, . - l1_identmap
 
+GLOBAL(__page_tables_start)
+
 /*
  * Space for mapping the first 4GB of memory, with the first 16 megabytes
  * actualy mapped (mostly using superpages).  Uses 4x 4k pages.
  */
 GLOBAL(l2_identmap)
-        .quad sym_phys(l1_identmap) + __PAGE_HYPERVISOR
-        idx = 1
-        .rept 7
-        .quad (idx << L2_PAGETABLE_SHIFT) | PAGE_HYPERVISOR | _PAGE_PSE
-        idx = idx + 1
-        .endr
-        .fill 4 * L2_PAGETABLE_ENTRIES - 8, 8, 0
+        .quad sym_offset(l1_identmap) + __PAGE_HYPERVISOR
+        .fill 4 * L2_PAGETABLE_ENTRIES - 1, 8, 0
         .size l2_identmap, . - l2_identmap
 
 /*
@@ -121,9 +117,10 @@  GLOBAL(l2_identmap)
  * page.
  */
 GLOBAL(l2_xenmap)
-        idx = 0
-        .rept 8
-        .quad sym_phys(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + (PAGE_HYPERVISOR | _PAGE_PSE)
+        .quad 0
+        idx = 1
+        .rept 7
+        .quad sym_offset(__image_base__) + (idx << L2_PAGETABLE_SHIFT) + (PAGE_HYPERVISOR | _PAGE_PSE)
         idx = idx + 1
         .endr
         .fill L2_PAGETABLE_ENTRIES - 8, 8, 0
@@ -134,7 +131,7 @@  l2_fixmap:
         idx = 0
         .rept L2_PAGETABLE_ENTRIES
         .if idx == l2_table_offset(FIXADDR_TOP - 1)
-        .quad sym_phys(l1_fixmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l1_fixmap) + __PAGE_HYPERVISOR
         .else
         .quad 0
         .endif
@@ -146,7 +143,7 @@  l2_fixmap:
 GLOBAL(l3_identmap)
         idx = 0
         .rept 4
-        .quad sym_phys(l2_identmap) + (idx << PAGE_SHIFT) + __PAGE_HYPERVISOR
+        .quad sym_offset(l2_identmap) + (idx << PAGE_SHIFT) + __PAGE_HYPERVISOR
         idx = idx + 1
         .endr
         .fill L3_PAGETABLE_ENTRIES - 4, 8, 0
@@ -157,9 +154,9 @@  l3_xenmap:
         idx = 0
         .rept L3_PAGETABLE_ENTRIES
         .if idx == l3_table_offset(XEN_VIRT_START)
-        .quad sym_phys(l2_xenmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l2_xenmap) + __PAGE_HYPERVISOR
         .elseif idx == l3_table_offset(FIXADDR_TOP - 1)
-        .quad sym_phys(l2_fixmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l2_fixmap) + __PAGE_HYPERVISOR
         .else
         .quad 0
         .endif
@@ -169,13 +166,13 @@  l3_xenmap:
 
 /* Top-level master (and idle-domain) page directory. */
 GLOBAL(idle_pg_table)
-        .quad sym_phys(l3_bootmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l3_bootmap) + __PAGE_HYPERVISOR
         idx = 1
         .rept L4_PAGETABLE_ENTRIES - 1
         .if idx == l4_table_offset(DIRECTMAP_VIRT_START)
-        .quad sym_phys(l3_identmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l3_identmap) + __PAGE_HYPERVISOR
         .elseif idx == l4_table_offset(XEN_VIRT_START)
-        .quad sym_phys(l3_xenmap) + __PAGE_HYPERVISOR
+        .quad sym_offset(l3_xenmap) + __PAGE_HYPERVISOR
         .else
         .quad 0
         .endif
@@ -190,16 +187,9 @@  GLOBAL(__page_tables_end)
         .align PAGE_SIZE, 0
 
 GLOBAL(l2_bootmap)
-        .quad sym_phys(l1_identmap) + __PAGE_HYPERVISOR
-        idx = 1
-        .rept 7
-        .quad (idx << L2_PAGETABLE_SHIFT) | __PAGE_HYPERVISOR | _PAGE_PSE
-        idx = idx + 1
-        .endr
-        .fill L2_PAGETABLE_ENTRIES - 8, 8, 0
+        .fill 4 * L2_PAGETABLE_ENTRIES, 8, 0
         .size l2_bootmap, . - l2_bootmap
 
 GLOBAL(l3_bootmap)
-        .quad sym_phys(l2_bootmap) + __PAGE_HYPERVISOR
-        .fill L3_PAGETABLE_ENTRIES - 1, 8, 0
+        .fill L3_PAGETABLE_ENTRIES, 8, 0
         .size l3_bootmap, . - l3_bootmap
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 21bbe6a..f6d2fa0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -281,9 +281,6 @@  static void *__init bootstrap_map(const module_t *mod)
     if ( start >= end )
         return NULL;
 
-    if ( end <= BOOTSTRAP_MAP_BASE )
-        return (void *)(unsigned long)start;
-
     ret = (void *)(map_cur + (unsigned long)(start & mask));
     start &= ~mask;
     end = (end + mask) & ~mask;
@@ -661,6 +658,9 @@  void __init noreturn __start_xen(unsigned long mbi_p)
 
     printk("Command line: %s\n", cmdline);
 
+    printk("Xen image base address: 0x%08lx\n",
+           xen_phys_start ? xen_phys_start : (unsigned long)xen_img_load_base_addr);
+
     printk("Video information:\n");
 
     /* Print VGA display mode information. */
@@ -855,10 +855,8 @@  void __init noreturn __start_xen(unsigned long mbi_p)
         uint64_t s, e, mask = (1UL << L2_PAGETABLE_SHIFT) - 1;
         uint64_t end, limit = ARRAY_SIZE(l2_identmap) << L2_PAGETABLE_SHIFT;
 
-        /* Superpage-aligned chunks from BOOTSTRAP_MAP_BASE. */
         s = (boot_e820.map[i].addr + mask) & ~mask;
         e = (boot_e820.map[i].addr + boot_e820.map[i].size) & ~mask;
-        s = max_t(uint64_t, s, BOOTSTRAP_MAP_BASE);
         if ( (boot_e820.map[i].type != E820_RAM) || (s >= e) )
             continue;
 
@@ -896,7 +894,7 @@  void __init noreturn __start_xen(unsigned long mbi_p)
             /* Select relocation address. */
             e = end - reloc_size;
             xen_phys_start = e;
-            bootsym(trampoline_xen_phys_start) = e;
+            bootsym(trampoline_xen_phys_start) = e - xen_img_load_base_addr;
 
             /*
              * Perform relocation to new physical address.
@@ -906,7 +904,7 @@  void __init noreturn __start_xen(unsigned long mbi_p)
              */
             load_start = (unsigned long)_start - XEN_VIRT_START;
             barrier();
-            move_memory(e + load_start, load_start, _end - _start, 1);
+            move_memory(e + load_start, load_start + xen_img_load_base_addr, _end - _start, 1);
 
             /* Walk initial pagetables, relocating page directory entries. */
             pl4e = __va(__pa(idle_pg_table));
@@ -915,27 +913,27 @@  void __init noreturn __start_xen(unsigned long mbi_p)
                 if ( !(l4e_get_flags(*pl4e) & _PAGE_PRESENT) )
                     continue;
                 *pl4e = l4e_from_intpte(l4e_get_intpte(*pl4e) +
-                                        xen_phys_start);
+                                        xen_phys_start - xen_img_load_base_addr);
                 pl3e = l4e_to_l3e(*pl4e);
                 for ( j = 0; j < L3_PAGETABLE_ENTRIES; j++, pl3e++ )
                 {
                     /* Not present, 1GB mapping, or already relocated? */
                     if ( !(l3e_get_flags(*pl3e) & _PAGE_PRESENT) ||
                          (l3e_get_flags(*pl3e) & _PAGE_PSE) ||
-                         (l3e_get_pfn(*pl3e) > 0x1000) )
+                         (l3e_get_pfn(*pl3e) > PFN_DOWN(xen_phys_start)) )
                         continue;
                     *pl3e = l3e_from_intpte(l3e_get_intpte(*pl3e) +
-                                            xen_phys_start);
+                                            xen_phys_start - xen_img_load_base_addr);
                     pl2e = l3e_to_l2e(*pl3e);
                     for ( k = 0; k < L2_PAGETABLE_ENTRIES; k++, pl2e++ )
                     {
                         /* Not present, PSE, or already relocated? */
                         if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ||
                              (l2e_get_flags(*pl2e) & _PAGE_PSE) ||
-                             (l2e_get_pfn(*pl2e) > 0x1000) )
+                             (l2e_get_pfn(*pl2e) > PFN_DOWN(xen_phys_start)) )
                             continue;
                         *pl2e = l2e_from_intpte(l2e_get_intpte(*pl2e) +
-                                                xen_phys_start);
+                                                xen_phys_start - xen_img_load_base_addr);
                     }
                 }
             }
@@ -946,21 +944,30 @@  void __init noreturn __start_xen(unsigned long mbi_p)
              * Undo the temporary-hooking of the l1_identmap.  __2M_text_start
              * is contained in this PTE.
              */
+#if 0
+            /*
+             * Should BUG_ON() be run only if Xen is
+             * loaded with EFI loadedr (xen.efi)?
+             */
             BUG_ON(l2_table_offset((unsigned long)_erodata) ==
                    l2_table_offset((unsigned long)_stext));
             *pl2e++ = l2e_from_pfn(xen_phys_start >> PAGE_SHIFT,
                                    PAGE_HYPERVISOR_RX | _PAGE_PSE);
+#else
+            pl2e++;
+#endif
             for ( i = 1; i < L2_PAGETABLE_ENTRIES; i++, pl2e++ )
             {
                 unsigned int flags;
 
-                if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) )
+                if ( !(l2e_get_flags(*pl2e) & _PAGE_PRESENT) ||
+                     (l2e_get_pfn(*pl2e) > PFN_DOWN(xen_phys_start)) )
                     continue;
 
                 if ( !using_2M_mapping() )
                 {
                     *pl2e = l2e_from_intpte(l2e_get_intpte(*pl2e) +
-                                            xen_phys_start);
+                                            xen_phys_start - xen_img_load_base_addr);
                     continue;
                 }
 
@@ -990,7 +997,7 @@  void __init noreturn __start_xen(unsigned long mbi_p)
                 }
 
                 *pl2e = l2e_from_paddr(
-                    l2e_get_paddr(*pl2e) + xen_phys_start, flags);
+                    l2e_get_paddr(*pl2e) + xen_phys_start - xen_img_load_base_addr, flags);
             }
 
             /* Re-sync the stack and then switch to relocated pagetables. */
@@ -1061,6 +1068,9 @@  void __init noreturn __start_xen(unsigned long mbi_p)
 
     if ( !xen_phys_start )
         panic("Not enough memory to relocate Xen.");
+
+    printk("New Xen image base address: 0x%08lx\n", xen_phys_start);
+
     reserve_e820_ram(&boot_e820, __pa(&_start), __pa(&_end));
 
     /* Late kexec reservation (dynamic start address). */
@@ -1133,14 +1143,12 @@  void __init noreturn __start_xen(unsigned long mbi_p)
 
         set_pdx_range(s >> PAGE_SHIFT, e >> PAGE_SHIFT);
 
-        /* Need to create mappings above BOOTSTRAP_MAP_BASE. */
-        map_s = max_t(uint64_t, s, BOOTSTRAP_MAP_BASE);
+        map_s = s;
         map_e = min_t(uint64_t, e,
                       ARRAY_SIZE(l2_identmap) << L2_PAGETABLE_SHIFT);
 
         /* Pass mapped memory to allocator /before/ creating new mappings. */
         init_boot_pages(s, min(map_s, e));
-        s = map_s;
         if ( s < map_e )
         {
             uint64_t mask = (1UL << L2_PAGETABLE_SHIFT) - 1;
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index fa1da37..8d2ad9f 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -41,7 +41,7 @@  SECTIONS
 
   __2M_text_start = .;         /* Start of 2M superpages, mapped RX. */
 
-  . = __XEN_VIRT_START + MB(1);
+  . = __XEN_VIRT_START + XEN_IMG_OFFSET;
   _start = .;
   .text : {
         _stext = .;            /* Text and read-only data */
diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h
index 4527ce3..bb8c9db 100644
--- a/xen/include/asm-x86/config.h
+++ b/xen/include/asm-x86/config.h
@@ -97,6 +97,7 @@  extern unsigned long trampoline_phys;
                  trampoline_phys-__pa(trampoline_start)))
 extern char trampoline_start[], trampoline_end[];
 extern char trampoline_realmode_entry[];
+extern unsigned int xen_img_load_base_addr;
 extern unsigned int trampoline_xen_phys_start;
 extern unsigned char trampoline_cpu_started;
 extern char wakeup_start[];
diff --git a/xen/include/asm-x86/page.h b/xen/include/asm-x86/page.h
index 224852a..4abd2ea 100644
--- a/xen/include/asm-x86/page.h
+++ b/xen/include/asm-x86/page.h
@@ -288,7 +288,7 @@  extern root_pgentry_t idle_pg_table[ROOT_PAGETABLE_ENTRIES];
 extern l2_pgentry_t  *compat_idle_pg_table_l2;
 extern unsigned int   m2p_compat_vstart;
 extern l2_pgentry_t l2_xenmap[L2_PAGETABLE_ENTRIES],
-    l2_bootmap[L2_PAGETABLE_ENTRIES];
+    l2_bootmap[4*L2_PAGETABLE_ENTRIES];
 extern l3_pgentry_t l3_bootmap[L3_PAGETABLE_ENTRIES];
 extern l2_pgentry_t l2_identmap[4*L2_PAGETABLE_ENTRIES];
 extern l1_pgentry_t l1_identmap[L1_PAGETABLE_ENTRIES],