xen arm64 dom0 question

Commit Message

Peng Fan Sept. 6, 2016, 7:36 a.m. UTC

Hi Julien,

On Fri, Sep 02, 2016 at 02:13:07PM +0100, Julien Grall wrote:
>
>
>On 02/09/16 12:27, Peng Fan wrote:
>>Hi Julien, Stefano
>
>Hi Peng,
>
>>
>>On My ARM64 platform, there is 6GB memory.
>>0x80000000 - 0xfffffff: 2GB
>>0x880000000 - 0x9ffffffff: 4GB
>>
>>xen will alloc 1:1 mapping for Dom0 memory, so if I assign dom0_mem with a bigger
>>value, saying 2048MB or bigger. xen will alloc continus memory from higher address
>>space in the higer 4GB.
>>
>>But the SD controller in my ARM64 platform has a limitation that it can only
>>access 32bit address space. So if Dom0 memory is at higher 4GB, SD controller
>>can not work as expected, because it only works when the dma address is 32bit
>>address.
>>
>>So, Can we assign a hole in lower 32bit address space for Dom0 guest physical
>>memory, just as DomU? Dynamically find out a hole and size 128MB or bigger?
>>Or do you have any ideas?
>
>Looking at the allocation code (allocate_memory in arch/arm/domain_build.c),
>Xen is trying to allocate as much memory as possible below 4GB for 32-bit
>domain to accommodate non-LPAE kernel.
>
>We haven't though about devices that can only handle 32-bit address at that
>time. I think replacing "lowmen = is_32bit_domain(d)" by "lowmem = true"
>should do the job.
>
>Note that, a proper upstream patch would require to modify the description of
>the function and potentially kill lowmen (or gate it with a command line
>parameter?).

Thanks for reply. I pasted two patches here. Both patch is to solve this problem.

In patch 1, allocate_memory will try to allocate memory in lowmem as much
as possible.
My test log for patch 1:
(XEN) Allocated 0x000000a0000000-0x000000c0000000 (512MB/2048MB, order 17)
(XEN) Allocated 0x000000c0000000-0x000000e0000000 (512MB/1536MB, order 17)
(XEN) Allocated 0x00000090000000-0x000000a0000000 (256MB/1024MB, order 16)
(XEN) Allocated 0x000000e0000000-0x000000f0000000 (256MB/768MB, order 16)
(XEN) Allocated 0x00000088000000-0x00000090000000 (128MB/512MB, order 15)
(XEN) Allocated 0x000000f0000000-0x000000f8000000 (128MB/384MB, order 15)
(XEN) Failed at min_low_order, allow high allocations
(XEN) Allocated 0x000009e0000000-0x000009f0000000 (256MB/256MB, order 16)
(XEN) BANK[0] 0x00000088000000-0x000000f8000000 (1792MB)
(XEN) BANK[1] 0x000009e0000000-0x000009f0000000 (256MB)

1792MB allocated in lowmem space.

patch 1:


I prefer patch 2, because I only need some lowmem for DMA. The method of patch 1
consumes too much lowmem.

What do you think?


Thanks,
Peng.

>
>Regards,
>
>-- 
>Julien Grall

Patch

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..cc71e6f 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@ 
 
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static unsigned int __initdata opt_dom0_use_lowmem;
+integer_param("dom0_use_lowmem", opt_dom0_use_lowmem);
 
 int dom0_11_mapping = 1;
 
@@ -244,7 +246,7 @@  static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
     unsigned int order = get_11_allocation_size(kinfo->unassigned_mem);
     int i;
 
-    bool_t lowmem = is_32bit_domain(d);
+    bool_t lowmem = is_32bit_domain(d) || !!opt_dom0_use_lowmem;
     unsigned int bits;
 
     /*

In Patch 2, only alloacte lowmem in the first try and allocate memory
for bank0.
My test log:
(XEN) Allocated 0x000000a0000000-0x000000c0000000 (512MB/2048MB, order 17)
(XEN) Allocated 0x00000980000000-0x000009c0000000 (1024MB/1536MB, order 18)
(XEN) Allocated 0x000009c0000000-0x000009e0000000 (512MB/512MB, order 17)
(XEN) BANK[0] 0x000000a0000000-0x000000c0000000 (512MB)
(XEN) BANK[1] 0x00000980000000-0x000009e0000000 (1536MB)

512MB is allocated in lowmem.

patch 2:
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 35ab08d..ad5926a 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -28,6 +28,8 @@ 
 
 static unsigned int __initdata opt_dom0_max_vcpus;
 integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
+static unsigned int __initdata opt_dom0_use_lowmem;
+integer_param("dom0_use_lowmem", opt_dom0_use_lowmem);
 
 int dom0_11_mapping = 1;
 
@@ -265,7 +267,9 @@  static void allocate_memory(struct domain *d, struct kernel_info *kinfo)
      */
     while ( order >= min_low_order )
     {
-        for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ )
+        for ( bits = order ;
+              bits <= ((lowmem || !!opt_dom0_use_lowmem) ? 32 : PADDR_BITS);
+              bits++ )
         {
             pg = alloc_domheap_pages(d, order, MEMF_bits(bits));
             if ( pg != NULL )