From patchwork Thu Oct 19 23:59:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 10018527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C792D60224 for ; Fri, 20 Oct 2017 00:00:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8C4628E71 for ; Fri, 20 Oct 2017 00:00:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD49F28E74; Fri, 20 Oct 2017 00:00:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1D91628E71 for ; Fri, 20 Oct 2017 00:00:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Mime-Version:References:In-Reply-To: Message-Id:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5vz7uiomvdypbWXUlUNXYAbWe/CGHd5ocGt4pJPfx1A=; b=iRY0ByfrNyJYc8 T7Coq6m4AgzSloorQo/IAQeW+HdKUzvAzJ+XbXl6V6NWtsX/RyVBJKDDrKs2NoEkIiSQ0uH8pIi62 LlDe+I946g54YQ2LZfuE41SKp/qjlmm5vhyBYpB3Rm9z8m3GyqiE8Pvy8u7WngF9ErY0giOfYjNVS 8F/AMeczUZvc4UePz/ujYCLp49+fwkTc6LvGY7MTXbxugkhMCSyNbomLwPIZq93KySIhKkBfesxm3 DDgBcCdt1YUnadf+mIbuTlbbH3KjNPrtAAuM+eBvZfjyh/QdKAgD3ZsS6ak0fGyzkL+Ub1IGQfaMl q+0G6u57WK49c6/W3yFg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1e5Kj8-0003fc-PN; Thu, 19 Oct 2017 23:59:50 +0000 Received: from mail.linuxfoundation.org ([140.211.169.12]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1e5Kj4-0003dl-Mb for linux-arm-kernel@lists.infradead.org; Thu, 19 Oct 2017 23:59:48 +0000 Received: from akpm3.svl.corp.google.com (unknown [104.133.9.92]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 8F1A12C; Thu, 19 Oct 2017 23:59:22 +0000 (UTC) Date: Thu, 19 Oct 2017 16:59:21 -0700 From: Andrew Morton To: Pavel Tatashin Subject: Re: [PATCH v12 09/11] mm: stop zeroing memory during allocation in vmemmap Message-Id: <20171019165921.de4224c8e627b1477cfb50de@linux-foundation.org> In-Reply-To: <20171013173214.27300-10-pasha.tatashin@oracle.com> References: <20171013173214.27300-1-pasha.tatashin@oracle.com> <20171013173214.27300-10-pasha.tatashin@oracle.com> X-Mailer: Sylpheed 3.4.1 (GTK+ 2.24.23; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171019_165946_804532_D6755E49 X-CRM114-Status: GOOD ( 13.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, mhocko@kernel.org, linux-mm@kvack.org, steven.sistare@oracle.com, willy@infradead.org, sparclinux@vger.kernel.org, sam@ravnborg.org, linux-s390@vger.kernel.org, x86@kernel.org, kasan-dev@googlegroups.com, daniel.m.jordan@oracle.com, borntraeger@de.ibm.com, heiko.carstens@de.ibm.com, linux-arm-kernel@lists.infradead.org, ard.biesheuvel@linaro.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, bob.picco@oracle.com, mgorman@techsingularity.net, davem@davemloft.net Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, 13 Oct 2017 13:32:12 -0400 Pavel Tatashin wrote: > vmemmap_alloc_block() will no longer zero the block, so zero memory > at its call sites for everything except struct pages. Struct page memory > is zero'd by struct page initialization. > > Replace allocators in sprase-vmemmap to use the non-zeroing version. So, > we will get the performance improvement by zeroing the memory in parallel > when struct pages are zeroed. > > Add struct page zeroing as a part of initialization of other fields in > __init_single_page(). > > This single thread performance collected on: Intel(R) Xeon(R) CPU E7-8895 > v3 @ 2.60GHz with 1T of memory (268400646 pages in 8 nodes): > > BASE FIX > sparse_init 11.244671836s 0.007199623s > zone_sizes_init 4.879775891s 8.355182299s > -------------------------- > Total 16.124447727s 8.362381922s > > sparse_init is where memory for struct pages is zeroed, and the zeroing > part is moved later in this patch into __init_single_page(), which is > called from zone_sizes_init(). x86_64 allmodconfig: WARNING: vmlinux.o(.text+0x29d099): Section mismatch in reference from the function T.1331() to the function .meminit.text:vmemmap_alloc_block() The function T.1331() references the function __meminit vmemmap_alloc_block(). This is often because T.1331 lacks a __meminit annotation or the annotation of vmemmap_alloc_block is wrong. From a quick scan it's unclear to me why this is happening. Maybe gcc-4.4.4 decided to create an out-of-line version of vmemmap_alloc_block_zero() for some reason. Anyway. I see no reason to publish vmemmap_alloc_block_zero() to the whole world when it's only used in sparse-vmemmap.c. The below fixes the section mismatch: --- a/include/linux/mm.h~mm-stop-zeroing-memory-during-allocation-in-vmemmap-fix +++ a/include/linux/mm.h @@ -2529,17 +2529,6 @@ static inline void *vmemmap_alloc_block_ return __vmemmap_alloc_block_buf(size, node, NULL); } -static inline void *vmemmap_alloc_block_zero(unsigned long size, int node) -{ - void *p = vmemmap_alloc_block(size, node); - - if (!p) - return NULL; - memset(p, 0, size); - - return p; -} - void vmemmap_verify(pte_t *, int, unsigned long, unsigned long); int vmemmap_populate_basepages(unsigned long start, unsigned long end, int node); --- a/mm/sparse-vmemmap.c~mm-stop-zeroing-memory-during-allocation-in-vmemmap-fix +++ a/mm/sparse-vmemmap.c @@ -178,6 +178,17 @@ pte_t * __meminit vmemmap_pte_populate(p return pte; } +static void * __meminit vmemmap_alloc_block_zero(unsigned long size, int node) +{ + void *p = vmemmap_alloc_block(size, node); + + if (!p) + return NULL; + memset(p, 0, size); + + return p; +} + pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node) { pmd_t *pmd = pmd_offset(pud, addr);