From patchwork Tue Mar 31 05:09:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 11466733 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA3E392C for ; Tue, 31 Mar 2020 05:10:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 93F3E20776 for ; Tue, 31 Mar 2020 05:10:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 93F3E20776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CA6AD6B006C; Tue, 31 Mar 2020 01:10:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C2F596B006E; Tue, 31 Mar 2020 01:10:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF87F6B0070; Tue, 31 Mar 2020 01:10:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0223.hostedemail.com [216.40.44.223]) by kanga.kvack.org (Postfix) with ESMTP id 95A366B006C for ; Tue, 31 Mar 2020 01:10:51 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6373A2C79 for ; Tue, 31 Mar 2020 05:10:51 +0000 (UTC) X-FDA: 76654482702.04.box14_41a0e5e8f8463 X-Spam-Summary: 2,0,0,c277371f0c1168ff,d41d8cd98f00b204,anshuman.khandual@arm.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1261:1345:1359:1431:1437:1535:1605:1730:1747:1777:1792:1963:2393:2553:2559:2562:2693:2903:3138:3139:3140:3141:3142:3167:3865:3867:3868:3870:3871:3872:3874:4049:4119:4321:4605:5007:6119:6261:6742:6743:8603:8634:8660:9036:10004:11026:11233:11658:11914:12043:12114:12291:12296:12297:12438:12555:12683:12895:12986:13148:13153:13161:13228:13229:13230:13255:13846:14394:21080:21451:21627:21990:30054:30090,0,RBL:217.140.110.172:@arm.com:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: box14_41a0e5e8f8463 X-Filterd-Recvd-Size: 8120 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 31 Mar 2020 05:10:50 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F354B1FB; Mon, 30 Mar 2020 22:10:49 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.1.70]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CA48B3F71E; Mon, 30 Mar 2020 22:10:38 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org Cc: robin.murphy@arm.com, dan.j.williams@intel.com, jglisse@redhat.com, jgg@mellanox.com, rcampbell@nvidia.com, aneesh.kumar@linux.ibm.com, Anshuman Khandual , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Andrew Morton , x86@kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 2/3] mm/sparsemem: Enable vmem_altmap support in vmemmap_alloc_block_buf() Date: Tue, 31 Mar 2020 10:39:46 +0530 Message-Id: <1585631387-18819-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1585631387-18819-1-git-send-email-anshuman.khandual@arm.com> References: <1585631387-18819-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are many instances where vmemap allocation is often switched between regular memory and device memory just based on whether altmap is available or not. vmemmap_alloc_block_buf() is used in various platforms to allocate vmemmap mappings. Lets also enable it to handle altmap based device memory allocation along with existing regular memory allocations. This will help in avoiding the altmap based allocation switch in many places. While here also implement a regular memory allocation fallback mechanism when the first preferred device memory allocation fails. This will ensure preserving the existing semantics on powerpc platform. To summarize there are three different methods to call vmemmap_alloc_block_buf(). (., NULL, false) /* Allocate from system RAM */ (., altmap, false) /* Allocate from altmap without any fallback */ (., altmap, true) /* Allocate from altmap with fallback (system RAM) */ Cc: Catalin Marinas Cc: Will Deacon Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Andrew Morton Cc: x86@kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Suggested-by: Robin Murphy Signed-off-by: Anshuman Khandual --- arch/arm64/mm/mmu.c | 3 ++- arch/powerpc/mm/init_64.c | 10 +++++----- arch/x86/mm/init_64.c | 6 ++---- include/linux/mm.h | 3 ++- mm/sparse-vmemmap.c | 30 ++++++++++++++++++++++++------ 5 files changed, 35 insertions(+), 17 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 2feed38106d6..81f88c88484f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1063,7 +1063,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, if (pmd_none(READ_ONCE(*pmdp))) { void *p = NULL; - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + p = vmemmap_alloc_block_buf(PMD_SIZE, node, + NULL, false); if (!p) return -ENOMEM; diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index 4002ced3596f..f67f2b909fe5 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -226,12 +226,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, * fall back to system memory if the altmap allocation fail. */ if (altmap && !altmap_cross_boundary(altmap, start, page_size)) { - p = altmap_alloc_block_buf(page_size, altmap); - if (!p) - pr_debug("altmap block allocation failed, falling back to system memory"); + p = vmemmap_alloc_block_buf(page_size, node, + altmap, true); + } else { + p = vmemmap_alloc_block_buf(page_size, node, + NULL, false); } - if (!p) - p = vmemmap_alloc_block_buf(page_size, node); if (!p) return -ENOMEM; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index c22677571619..35cc0c9d9578 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1444,10 +1444,8 @@ static int __meminit vmemmap_populate_hugepages(unsigned long start, if (pmd_none(*pmd)) { void *p; - if (altmap) - p = altmap_alloc_block_buf(PMD_SIZE, altmap); - else - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + p = vmemmap_alloc_block_buf(PMD_SIZE, node, + altmap, false); if (p) { pte_t entry; diff --git a/include/linux/mm.h b/include/linux/mm.h index 955be0331833..b8d3d90c9c47 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2991,7 +2991,8 @@ pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, struct vmem_altmap *altmap); void *vmemmap_alloc_block(unsigned long size, int node); struct vmem_altmap; -void *vmemmap_alloc_block_buf(unsigned long size, int node); +void *vmemmap_alloc_block_buf(unsigned long size, int node, + struct vmem_altmap *altmap, bool sysram_fallback); void *altmap_alloc_block_buf(unsigned long size, struct vmem_altmap *altmap); void vmemmap_verify(pte_t *, int, unsigned long, unsigned long); int vmemmap_populate_basepages(unsigned long start, unsigned long end, diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index a407abc9b46c..ff5adc233e38 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -71,10 +71,31 @@ void * __meminit vmemmap_alloc_block(unsigned long size, int node) } /* need to make sure size is all the same during early stage */ -void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node) +void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node, + struct vmem_altmap *altmap, + bool sysram_fallback) { - void *ptr = sparse_buffer_alloc(size); + void *ptr; + /* + * There is no point in asking for sysram fallback + * without an altmap request to begin with. So just + * warn here to catch potential call sites that may + * be violating this. + */ + WARN_ON(!altmap && sysram_fallback); + + if (altmap) { + ptr = altmap_alloc_block_buf(size, altmap); + if (ptr) + return ptr; + pr_debug("altmap block allocation failed\n"); + if (!sysram_fallback) + return NULL; + pr_debug("falling back to system memory\n"); + } + + ptr = sparse_buffer_alloc(size); if (!ptr) ptr = vmemmap_alloc_block(size, node); return ptr; @@ -148,10 +169,7 @@ pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, pte_t entry; void *p; - if (altmap) - p = altmap_alloc_block_buf(PAGE_SIZE, altmap); - else - p = vmemmap_alloc_block_buf(PAGE_SIZE, node); + p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap, false); if (!p) return NULL; entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL);