From patchwork Tue Sep 6 07:36:04 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Fan X-Patchwork-Id: 9315835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4BB1760760 for ; Tue, 6 Sep 2016 07:38:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5785028BC2 for ; Tue, 6 Sep 2016 07:38:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C02928BC4; Tue, 6 Sep 2016 07:38:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=no version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BD4A428BC2 for ; Tue, 6 Sep 2016 07:38:58 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bhAvc-0000ET-7q; Tue, 06 Sep 2016 07:36:20 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bhAvb-0000EM-CI for xen-devel@lists.xen.org; Tue, 06 Sep 2016 07:36:19 +0000 Received: from [85.158.137.68] by server-12.bemta-3.messagelabs.com id 44/89-09160-2F17EC75; Tue, 06 Sep 2016 07:36:18 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrOIsWRWlGSWpSXmKPExsVyMfSOi+7HwnP hBu92m1ss+biYxYHR4+ju30wBjFGsmXlJ+RUJrBmtDfOYCvapVMybsZulgfGMXBcjF4eQQD+j xJW5PSwgDovAfFaJb3vnMoE4EgK7WSV2Tupm62LkAHJiJPY9Teli5AQyqyXm3b3CBmILCahIz N3UwwgxaTqTxIOen4wgCRagRM/KS8wgNpuAmsSR9zPB4iIC6hJ/LkwAs5kFjCXe7bgBZgsD1V +4/xLM5gWKb5j/ggViQYpE68VNUHFBiZMzn7BA9GpJ3Pj3kgnkNmYBaYnl/zhAwpwC1hKTfm9 iBQmLAo18dbB+AqPwLCTNs5A0z0JoXsDIvIpRozi1qCy1SNfQQC+pKDM9oyQ3MTMHyDPWy00t Lk5MT81JTCrWS87P3cQIDPJ6BgbGHYzbupwPMUpyMCmJ8qoFngsX4kvKT6nMSCzOiC8qzUktP sSowcEhsHnt6guMUix5+XmpShK8LwqA6gSLUtNTK9Iyc4BxCFMqwcGjJMIrAIxFId7igsTc4s x0iNQpRmOOLb+vrWXi2Db13lomIbBJUuK8l0EmCYCUZpTmwQ2CpYdLjLJSwryMDAwMQjwFqUW 5mSWo8q8YxTkYlYR5FUAW8mTmlcDtewV0ChPQKet2nwY5pSQRISXVwNh3dfvLrbbFibUfAjpe Nsy9J3+//NeU369PvbcPPTRxwzopqQnXuA+KP9pf6yuz8cTsg47CvKEbFpVzz89WmPfySce/M wuWthayByf/PTehw/QEc+OpIhE3w671V4qCPO2kns9xe/Dy8F5Z9ztLv+24rh9pEzQ7yuEga+ BPN6U1zyYWB7maKD9TYinOSDTUYi4qTgQAb4i2egoDAAA= X-Env-Sender: van.freenix@gmail.com X-Msg-Ref: server-10.tower-31.messagelabs.com!1473147376!58452223!1 X-Originating-IP: [209.85.220.68] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 8.84; banners=-,-,- X-VirusChecked: Checked Received: (qmail 57948 invoked from network); 6 Sep 2016 07:36:17 -0000 Received: from mail-pa0-f68.google.com (HELO mail-pa0-f68.google.com) (209.85.220.68) by server-10.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 6 Sep 2016 07:36:17 -0000 Received: by mail-pa0-f68.google.com with SMTP id gi6so784002pac.1 for ; Tue, 06 Sep 2016 00:36:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=4np5wQ+11ebJlOP5covg9H9xs47vKkiiDLAKkqYyLG8=; b=NYjyZVK8eNSWnLlGc1FJaZoJknlNO70mKeUf1+98CpFJjHGoLZVNZz2SmSA7ufSVWq w1SN8nMUc56K/oM80xsU0RBdHYmSy33tSJ+S7rXUoPAm6/Jj88VaE72DQgjFpP82riBY uSBS49taenPXt1K1kARxWSmc5J79cfzJhjMEa0BSCcXynxuivCavzmVfUzr5MFEKe0+0 5OFJxhGDTWilHwUpMy+HwLXFZei+rVVP1p1zFZZULvL4TGhqqKB7syF0qktvUGcH4MAP 10gfxnFWFj4gNG0fm23/pqglarqtwjyu6EkSOI1q7JGwgk8wJlsQt3oGb1J9pJfh4Dai 2zlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=4np5wQ+11ebJlOP5covg9H9xs47vKkiiDLAKkqYyLG8=; b=ENsyaOXjimtOPkK/cBD0yVIc3aIbiAbv8DtfC7LM5Pi26ejSle/TJSiCF4sZRdHPUN 4AFKvkPTy09Souo8Zq8Z7TEHmDB7wiYButNH9CAjfuHDYy/RTNSqhy9l38lSippCFvjv FWbKXOZuZm/n9P6W0IkxQsSy3PqsF4cN7gRHBkdpaAVFXpcXzfMuaPHQuInx1yQyGKhP 7t7LbCZqqYd6PEMseB4BqEUFDYojZhh1v+87bReKEb/0GOFLKnV5kGP88ajD0PFqjBy0 gaK1RiSe7pXWJXwuBABx/kl+sgnZturmrE6K6+tTV013HxTLaCiDAGdBvKKyMzH9ea/E u7iw== X-Gm-Message-State: AE9vXwNR3h+hFkJ4jRigkCZqUVDzn4IoEpZlcAfoF/67f/tymlC/5DYQzKGaM2uyjfIzgw== X-Received: by 10.66.222.202 with SMTP id qo10mr69733816pac.76.1473147375870; Tue, 06 Sep 2016 00:36:15 -0700 (PDT) Received: from linux-7smt.suse (gate-zmy3.freescale.com. [192.88.167.1]) by smtp.googlemail.com with ESMTPSA id tm1sm38421067pac.23.2016.09.06.00.36.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Sep 2016 00:36:15 -0700 (PDT) Date: Tue, 6 Sep 2016 15:36:04 +0800 From: Peng Fan To: Julien Grall Message-ID: <20160906073602.GA5493@linux-7smt.suse> References: <20160902112734.GB31373@linux-7smt.suse> <118835c0-d85d-7eec-e470-03c3ef70072f@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <118835c0-d85d-7eec-e470-03c3ef70072f@arm.com> User-Agent: Mutt/1.5.24 (2015-08-30) Cc: sstabellini@kernel.org, xen-devel@lists.xen.org Subject: Re: [Xen-devel] xen arm64 dom0 question X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi Julien, On Fri, Sep 02, 2016 at 02:13:07PM +0100, Julien Grall wrote: > > >On 02/09/16 12:27, Peng Fan wrote: >>Hi Julien, Stefano > >Hi Peng, > >> >>On My ARM64 platform, there is 6GB memory. >>0x80000000 - 0xfffffff: 2GB >>0x880000000 - 0x9ffffffff: 4GB >> >>xen will alloc 1:1 mapping for Dom0 memory, so if I assign dom0_mem with a bigger >>value, saying 2048MB or bigger. xen will alloc continus memory from higher address >>space in the higer 4GB. >> >>But the SD controller in my ARM64 platform has a limitation that it can only >>access 32bit address space. So if Dom0 memory is at higher 4GB, SD controller >>can not work as expected, because it only works when the dma address is 32bit >>address. >> >>So, Can we assign a hole in lower 32bit address space for Dom0 guest physical >>memory, just as DomU? Dynamically find out a hole and size 128MB or bigger? >>Or do you have any ideas? > >Looking at the allocation code (allocate_memory in arch/arm/domain_build.c), >Xen is trying to allocate as much memory as possible below 4GB for 32-bit >domain to accommodate non-LPAE kernel. > >We haven't though about devices that can only handle 32-bit address at that >time. I think replacing "lowmen = is_32bit_domain(d)" by "lowmem = true" >should do the job. > >Note that, a proper upstream patch would require to modify the description of >the function and potentially kill lowmen (or gate it with a command line >parameter?). Thanks for reply. I pasted two patches here. Both patch is to solve this problem. In patch 1, allocate_memory will try to allocate memory in lowmem as much as possible. My test log for patch 1: (XEN) Allocated 0x000000a0000000-0x000000c0000000 (512MB/2048MB, order 17) (XEN) Allocated 0x000000c0000000-0x000000e0000000 (512MB/1536MB, order 17) (XEN) Allocated 0x00000090000000-0x000000a0000000 (256MB/1024MB, order 16) (XEN) Allocated 0x000000e0000000-0x000000f0000000 (256MB/768MB, order 16) (XEN) Allocated 0x00000088000000-0x00000090000000 (128MB/512MB, order 15) (XEN) Allocated 0x000000f0000000-0x000000f8000000 (128MB/384MB, order 15) (XEN) Failed at min_low_order, allow high allocations (XEN) Allocated 0x000009e0000000-0x000009f0000000 (256MB/256MB, order 16) (XEN) BANK[0] 0x00000088000000-0x000000f8000000 (1792MB) (XEN) BANK[1] 0x000009e0000000-0x000009f0000000 (256MB) 1792MB allocated in lowmem space. patch 1: I prefer patch 2, because I only need some lowmem for DMA. The method of patch 1 consumes too much lowmem. What do you think? Thanks, Peng. > >Regards, > >-- >Julien Grall diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 35ab08d..cc71e6f 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -28,6 +28,8 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static unsigned int __initdata opt_dom0_use_lowmem; +integer_param("dom0_use_lowmem", opt_dom0_use_lowmem); int dom0_11_mapping = 1; @@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo) unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); int i; - bool_t lowmem = is_32bit_domain(d); + bool_t lowmem = is_32bit_domain(d) || !!opt_dom0_use_lowmem; unsigned int bits; /* In Patch 2, only alloacte lowmem in the first try and allocate memory for bank0. My test log: (XEN) Allocated 0x000000a0000000-0x000000c0000000 (512MB/2048MB, order 17) (XEN) Allocated 0x00000980000000-0x000009c0000000 (1024MB/1536MB, order 18) (XEN) Allocated 0x000009c0000000-0x000009e0000000 (512MB/512MB, order 17) (XEN) BANK[0] 0x000000a0000000-0x000000c0000000 (512MB) (XEN) BANK[1] 0x00000980000000-0x000009e0000000 (1536MB) 512MB is allocated in lowmem. patch 2: diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 35ab08d..ad5926a 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -28,6 +28,8 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static unsigned int __initdata opt_dom0_use_lowmem; +integer_param("dom0_use_lowmem", opt_dom0_use_lowmem); int dom0_11_mapping = 1; @@ -265,7 +267,9 @@ static void allocate_memory(struct domain *d, struct kernel_info *kinfo) */ while ( order >= min_low_order ) { - for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ ) + for ( bits = order ; + bits <= ((lowmem || !!opt_dom0_use_lowmem) ? 32 : PADDR_BITS); + bits++ ) { pg = alloc_domheap_pages(d, order, MEMF_bits(bits)); if ( pg != NULL )