From patchwork Mon Nov 5 21:19:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10669175 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A72261751 for ; Mon, 5 Nov 2018 21:19:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D16229C30 for ; Mon, 5 Nov 2018 21:19:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 90EB529C5E; Mon, 5 Nov 2018 21:19:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2FE5429C30 for ; Mon, 5 Nov 2018 21:19:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E38326B0010; Mon, 5 Nov 2018 16:19:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE9F86B0266; Mon, 5 Nov 2018 16:19:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD9076B0269; Mon, 5 Nov 2018 16:19:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 8EC8F6B0010 for ; Mon, 5 Nov 2018 16:19:27 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id d7-v6so10655619pfj.6 for ; Mon, 05 Nov 2018 13:19:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:user-agent:mime-version :content-transfer-encoding; bh=eIuGEuTqQI6Ljbp3utvnSbwmx0ZjZtDqA8kSYT/ldys=; b=JASxyyjZCcHvL2+qjAl7Jqb6ifPia5JOGmyZv2o3w+Jyq6TAOd/MYLQiSJoaWSLeW/ cXhgoSvNSrfZnIyXnwgYCxex6o8K5lhbascyaEWOHU5Khsz+2udNvZrxJopi3OdSECrD K1xZcwDmt2AQWRlkQA/wryzQWXo6Y6qPBZTNX41OTkSA9ZkSJba3CjNg4BN0zzCal6tq NXhhaY4uheahmXvOyKQJVhyjXR+enfn7R42pmX3MtmjQ2vIAZ2MyN3clq0gUtmF5OnPg Qv6/be1bTJhLh4+r6HiQhRDfLVAVGu6NiFF/L8y9tMi71oP0FIo3BwjpqJvqXPFfrong losw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AGRZ1gIQt4821wNQbiGw2lry9jH27eY8t8WlTXk7N5wK56pzqMI38nkr sOiMfHVWT9buj9WVmp+dFj6BUySJ32SgooSyH+D2bzxdRDblZnycyHXBU3jFOFv4IvNU5dI9ig/ 2D+LB0R43ADmiyAkGMAkXENpKv3UahqC6k7P+une/3LNZry1zJXfSojtj/ifMQ3awKQ== X-Received: by 2002:a17:902:925:: with SMTP id 34-v6mr1383046plm.243.1541452767235; Mon, 05 Nov 2018 13:19:27 -0800 (PST) X-Google-Smtp-Source: AJdET5c5jtH1B+00jgbIPg2Z/rvnOftelLj1qRldQ/qKMiaN334B+4aWMxmFLgmvKB1c5MWp9U4E X-Received: by 2002:a17:902:925:: with SMTP id 34-v6mr1382993plm.243.1541452766419; Mon, 05 Nov 2018 13:19:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541452766; cv=none; d=google.com; s=arc-20160816; b=FXqLnsgW7Ik9uuMGJWeMiwGWW9UZbgsJv8UZITS0v0xMr3MF6KMfRd48MGYdQAtgge 5Tpah/qiSR0RCm9PUuRiatHmrgxnUZ1RU+Jo12aur/CahfZ/lvr+Ov43+BoCAOdcSLz/ U/HATwCTN3QcClBYqnd1g0tYF4eBEnWJTT55N4+Ou4E2vQD7cIw+P5nxVA+A+qKXmJFc 04isDAcA//eUMWzYennUc5cVnK7n6GB+RCT7rwXnnEYlyw3BbeSV8qeVdsOdLYqa7fv0 T/P0RtuNpp3xlkz4EuSY5Ub6dBSTcHr5DIgvsGb9DO2gIUsHYAvwNpV3QEKsKwWP0Fme dqsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:message-id:date :cc:to:from:subject; bh=eIuGEuTqQI6Ljbp3utvnSbwmx0ZjZtDqA8kSYT/ldys=; b=gykavhiXS667tp9FaX1I/YtZx/Zk1yfSiOEbRRjhqrm94tAfJMqE1NX73lilt/+p8S sAgZ0TXjmwkFCWiK6ePgAQn1+j6TH5h07uRA46oIeaYL5fv2RLeOIrRO04osC8sF51Jg 0jYBcTIIj754GHMZPEPLcLSglvnZaJ3h0q3tC0TYXD9iJWQ1DfnW4z7AnXU9bJXxBTjl K3yBvwfe8peL0X6qJ9KK4MXDJkTFGChmuhwGbQXFhlO4PFuAFtfxhXXuAUL8Wa8EHPzJ fzSjJ/Uk3LRI4FwzuB7LQRrlnT2qUvwucz9RGx40ZMxqvj1SQI3ZKP3e0ZjAWxn0O1IC dapQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga12.intel.com (mga12.intel.com. [192.55.52.136]) by mx.google.com with ESMTPS id k190-v6si34117926pgk.261.2018.11.05.13.19.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Nov 2018 13:19:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) client-ip=192.55.52.136; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Nov 2018 13:19:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,469,1534834800"; d="scan'208";a="271595983" Received: from ahduyck-desk1.jf.intel.com ([10.7.198.76]) by orsmga005.jf.intel.com with ESMTP; 05 Nov 2018 13:19:25 -0800 Subject: [mm PATCH v5 0/7] Deferred page init improvements From: Alexander Duyck To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: sparclinux@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, davem@davemloft.net, pavel.tatashin@microsoft.com, mhocko@suse.com, mingo@kernel.org, kirill.shutemov@linux.intel.com, dan.j.williams@intel.com, dave.jiang@intel.com, alexander.h.duyck@linux.intel.com, rppt@linux.vnet.ibm.com, willy@infradead.org, vbabka@suse.cz, khalid.aziz@oracle.com, ldufour@linux.vnet.ibm.com, mgorman@techsingularity.net, yi.z.zhang@linux.intel.com, alexander.h.duyck@linux.intel.com Date: Mon, 05 Nov 2018 13:19:25 -0800 Message-ID: <154145268025.30046.11742652345962594283.stgit@ahduyck-desk1.jf.intel.com> User-Agent: StGit/unknown-version MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patchset is essentially a refactor of the page initialization logic that is meant to provide for better code reuse while providing a significant improvement in deferred page initialization performance. In my testing on an x86_64 system with 384GB of RAM and 3TB of persistent memory per node I have seen the following. In the case of regular memory initialization the deferred init time was decreased from 3.75s to 1.06s on average. For the persistent memory the initialization time dropped from 24.17s to 19.12s on average. This amounts to a 253% improvement for the deferred memory initialization performance, and a 26% improvement in the persistent memory initialization performance. I have called out the improvement observed with each patch. v1->v2: Fixed build issue on PowerPC due to page struct size being 56 Added new patch that removed __SetPageReserved call for hotplug v2->v3: Rebased on latest linux-next Removed patch that had removed __SetPageReserved call from init Added patch that folded __SetPageReserved into set_page_links Tweaked __init_pageblock to use start_pfn to get section_nr instead of pfn v3->v4: Updated patch description and comments for mm_zero_struct_page patch Replaced "default" with "case 64" Removed #ifndef mm_zero_struct_page Fixed typo in comment that ommited "_from" in kerneldoc for iterator Added Reviewed-by for patches reviewed by Pavel Added Acked-by from Michal Hocko Added deferred init times for patches that affect init performance Swapped patches 5 & 6, pulled some code/comments from 4 into 5 v4->v5: Updated Acks/Reviewed-by Rebased on latest linux-next Split core bits of zone iterator patch from MAX_ORDER_NR_PAGES init Tested-by: Pavel Tatashin --- Alexander Duyck (7): mm: Use mm_zero_struct_page from SPARC on all 64b architectures mm: Drop meminit_pfn_in_nid as it is redundant mm: Implement new zone specific memblock iterator mm: Initialize MAX_ORDER_NR_PAGES at a time instead of doing larger sections mm: Move hot-plug specific memory init into separate functions and optimize mm: Add reserved flag setting to set_page_links mm: Use common iterator for deferred_init_pages and deferred_free_pages arch/sparc/include/asm/pgtable_64.h | 30 -- include/linux/memblock.h | 38 ++ include/linux/mm.h | 50 +++ mm/memblock.c | 63 ++++ mm/page_alloc.c | 567 +++++++++++++++++++++-------------- 5 files changed, 492 insertions(+), 256 deletions(-) --