From patchwork Tue Jul 24 23:55:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 10543345 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B5BC17FD for ; Wed, 25 Jul 2018 00:26:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28F2729634 for ; Wed, 25 Jul 2018 00:26:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B43B29686; Wed, 25 Jul 2018 00:26:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D5F429634 for ; Wed, 25 Jul 2018 00:26:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A66BF6B000E; Tue, 24 Jul 2018 20:26:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9ECA46B0010; Tue, 24 Jul 2018 20:26:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 866E26B0266; Tue, 24 Jul 2018 20:26:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk0-f200.google.com (mail-qk0-f200.google.com [209.85.220.200]) by kanga.kvack.org (Postfix) with ESMTP id 56E8E6B000E for ; Tue, 24 Jul 2018 20:26:00 -0400 (EDT) Received: by mail-qk0-f200.google.com with SMTP id v65-v6so5064113qka.23 for ; Tue, 24 Jul 2018 17:26:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references; bh=GiLkvI8Rp3OQo1+Ryx+tNZlttGtHfZulAZ/cM6I2/6Y=; b=Ml7UvJ72nPAN7Zo1jrGnZs5dYGmgSBw9IEewrp6vgOwnX6sLaNQya8DrpZ8OQbPr8x FzklFrP96R7Fsdh9xWQLbuqt/STr1tkJ0YGivdyYevORQ2Yz+rbj/zpXfBpz4n2sTIWH 9wOAZzlosNcdBvEeQFH0DCwfuFp9OlcuztCHTIhOVbWg4yV06ObBIH+4Hh/QEk43p4gi deK4xX1HM4q3a75bTWU9NCFc6+Kt/nQMCXPX+x2mZg/OGdpsFiGa6IIHi5SFBUPaxqGj v9pqWlztAOpPiPFd+Mc7iuKFUeoj7ErtZf9+m2JCQb7oarpdNU7NVU3t3bWu59DUwUpO yOjg== X-Gm-Message-State: AOUpUlGPfjz6pt/QxqRePeCFHTfglDiO+JbyQk80F7/LGIT0WIe0JXiE g8DbFoZfkvxTZiHylCktDjNr29dFZS4DqKrL4HUjF35jqWCyDDFYjZaabWzh3kdPztlKcHya/iU yh6WwhEhn3kUKrE7W+kA+gjRgm58SgrT7mJQ2gtmN6uBe7AqOcn6Y/lR8C9Y9cqxeLw== X-Received: by 2002:ae9:e007:: with SMTP id m7-v6mr12878107qkk.268.1532478360093; Tue, 24 Jul 2018 17:26:00 -0700 (PDT) X-Google-Smtp-Source: AAOMgpc7jP5qCrmnpmasHmIC0Ld+nEFROkk2i5uvoK3XXd0MmKzDaAPSGqNpyWnX/ISAfP9tB9ix X-Received: by 2002:ae9:e007:: with SMTP id m7-v6mr12878082qkk.268.1532478359537; Tue, 24 Jul 2018 17:25:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532478359; cv=none; d=google.com; s=arc-20160816; b=Y+i4ShGKFXlOwDLOz/Zp2tuqQ9qV0QJ/sQQGlIki5jlLWK9Me5PP8SvTkT+aGaEkIn hq/yNfvyg+vk6CCrF1JEJwE9CagYNTeaSA3kL3haOHh/UJnT369xfvOp8WRz5SeUENJH xaqp/RMo4yqLY2U0jm14q/i0z15zioaBVAU+rK+trVYLWSIzu9lqgqZs1eB7vIonkNsY s4yZCgu8zrgqWMTefzKlwwQz1b/Uu0m3ZAtqNypCxL2i6F/rKJbTK3+PRAEbKahg1PrU Gi0JPilFTguWBhiMtps1/yvT7s4jCXEcVLQZYIPEJtQS10H/H/eSx+NwaapHrs8VcNCe cvIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from :dkim-signature:arc-authentication-results; bh=GiLkvI8Rp3OQo1+Ryx+tNZlttGtHfZulAZ/cM6I2/6Y=; b=Vgbf4hJtONZt4yqDlVScYxSfagy+QRKQn/9ADSBq1HFeUk/cYov+QBr1JJPv9jcXN0 QF6i69FIXQiZeI+wKScbBoZXcZTwtz271+DnOrHhN7+piYumMHEp6PV7uO+qT3pdNA9y tZpOccCXdB19CoNOzCRP2C7mY4d3FuBFr48pRtpjw9JqY6lvgXGnlgIlX0PMLnmUn9/S QOyoG3gAI7ZCvMcrbjU6PYFrJiYqqoGkNL6CIDVxuIOl6fNiwRFxxd0/v5juvT9VOgEI JgFXZRaxUIyTPf4kFKGFB1WUloTUHEqb61oVx+SQpZZ1/izFoboVLf+tv2fU7BqBb4W8 cotw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=VPFq6wjj; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id g187-v6si3872003qkb.320.2018.07.24.17.25.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jul 2018 17:25:59 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=VPFq6wjj; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w6P0O5Uq119841; Wed, 25 Jul 2018 00:25:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=GiLkvI8Rp3OQo1+Ryx+tNZlttGtHfZulAZ/cM6I2/6Y=; b=VPFq6wjj6HC3TyWIxOtTQlEht/S1fAm1umaLHyOrqa1ryEWJcg42S3BwFYeySC8bp1jY 9VJVoqrmIDFQg97jFTzoB5T2Ze/JzYWhqNeRZCM2z6bPdJ1jDKa2pFT19fUIHS4YxMLN duWdpjcm4ysQEmNjRH72gInuIca2wYJ2AbNKHSaH5yrO+EMwnEUI8R6OA+gQXeWqL/pR JzmdlegPtfeXZpesVfdEL9ejDin98DNJn4sMg0jhKZqIO5bwdRlFU2yiAf9RHzPG7gZ9 GBs0himAaB0CFj4gb+v4Cpwn8apN3TBShBdBVy0CIXPgOrOGRhv+IjbifMyUEdYYmMkX Cw== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2kbwfpu969-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Jul 2018 00:25:50 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w6P0PmwF004846 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Jul 2018 00:25:49 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w6P0PmCk003387; Wed, 25 Jul 2018 00:25:48 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 24 Jul 2018 16:55:29 -0700 From: Pavel Tatashin To: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, mhocko@suse.com, linux-mm@kvack.org, dan.j.williams@intel.com, jack@suse.cz, jglisse@redhat.com, jrdr.linux@gmail.com, bhe@redhat.com, gregkh@linuxfoundation.org, vbabka@suse.cz, richard.weiyang@gmail.com, dave.hansen@intel.com, rientjes@google.com, mingo@kernel.org, osalvador@techadventures.net, pasha.tatashin@oracle.com, abdhalee@linux.vnet.ibm.com, mpe@ellerman.id.au Subject: [PATCH 2/3] mm: calculate deferred pages after skipping mirrored memory Date: Tue, 24 Jul 2018 19:55:19 -0400 Message-Id: <20180724235520.10200-3-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180724235520.10200-1-pasha.tatashin@oracle.com> References: <20180724235520.10200-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8964 signatures=668706 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807250002 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP update_defer_init() should be called only when struct page is about to be initialized. Because it counts number of initialized struct pages, but there we may skip struct pages if there is some mirrored memory. So move, update_defer_init() after checking for mirrored memory. Also, rename update_defer_init() to defer_init() and reverse the return boolean to emphasize that this is a boolean function, that tells that the reset of memmap initialization should be deferred. Make this function self-contained: do not pass number of already initialized pages in this zone by using static counters. Signed-off-by: Pavel Tatashin --- mm/page_alloc.c | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cea749b26394..86c678cec6bd 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -306,24 +306,28 @@ static inline bool __meminit early_page_uninitialised(unsigned long pfn) } /* - * Returns false when the remaining initialisation should be deferred until + * Returns true when the remaining initialisation should be deferred until * later in the boot cycle when it can be parallelised. */ -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { + static unsigned long prev_end_pfn, nr_initialised; + + if (prev_end_pfn != end_pfn) { + prev_end_pfn = end_pfn; + nr_initialised = 0; + } + /* Always populate low zones for address-constrained allocations */ - if (zone_end < pgdat_end_pfn(pgdat)) - return true; - (*nr_initialised)++; - if ((*nr_initialised > pgdat->static_init_pgcnt) && - (pfn & (PAGES_PER_SECTION - 1)) == 0) { - pgdat->first_deferred_pfn = pfn; + if (end_pfn < pgdat_end_pfn(NODE_DATA(nid))) return false; + nr_initialised++; + if ((nr_initialised > NODE_DATA(nid)->static_init_pgcnt) && + (pfn & (PAGES_PER_SECTION - 1)) == 0) { + NODE_DATA(nid)->first_deferred_pfn = pfn; + return true; } - - return true; + return false; } #else static inline bool early_page_uninitialised(unsigned long pfn) @@ -331,11 +335,9 @@ static inline bool early_page_uninitialised(unsigned long pfn) return false; } -static inline bool update_defer_init(pg_data_t *pgdat, - unsigned long pfn, unsigned long zone_end, - unsigned long *nr_initialised) +static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) { - return true; + return false; } #endif @@ -5462,9 +5464,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, struct vmem_altmap *altmap) { unsigned long end_pfn = start_pfn + size; - pg_data_t *pgdat = NODE_DATA(nid); unsigned long pfn; - unsigned long nr_initialised = 0; struct page *page; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP struct memblock_region *r = NULL, *tmp; @@ -5492,8 +5492,6 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, continue; if (!early_pfn_in_nid(pfn, nid)) continue; - if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised)) - break; #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP /* @@ -5516,6 +5514,8 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, } } #endif + if (defer_init(nid, pfn, end_pfn)) + break; not_early: page = pfn_to_page(pfn);