From patchwork Thu Sep 20 22:27:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10608805 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 006E315A6 for ; Thu, 20 Sep 2018 22:28:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0C9728816 for ; Thu, 20 Sep 2018 22:28:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CEE3F286EC; Thu, 20 Sep 2018 22:28:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00CF92DF06 for ; Thu, 20 Sep 2018 22:27:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F5C28E0004; Thu, 20 Sep 2018 18:27:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1A5A58E0001; Thu, 20 Sep 2018 18:27:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 094A98E0004; Thu, 20 Sep 2018 18:27:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id BFD768E0001 for ; Thu, 20 Sep 2018 18:27:57 -0400 (EDT) Received: by mail-pg1-f199.google.com with SMTP id v186-v6so3668312pgb.14 for ; Thu, 20 Sep 2018 15:27:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=rtBDAMbGmBlNQ1vdQfjvBiLqtdFJw4/gYxGBxmLdmPM=; b=TVj7MZ1+97UTXre3PFKgXeNzDCFBunisteWtka0L5qcCTg47z++9wM6P3cEAB94Yuo hbxghD8y0uDQSaf/jAYo+P41MucGvVbo9BxxQKIGrKrDCtaYo4/Qxl/rur9wwqN2YCuc qLREXz6JnyUv9oiU8CT12kCs6qhfHKd+65F+2QvvLjGK/RlVMWtFrJWsthCmnzwiHi3u 3ctE5vqZSCxltulAeabv28rRHr/4tfs0+1jeWAO05NlKsb1XjfSj6YOIkxyh5mVDsoPh I/YmI4UXS9JVOkuHH2vjRIx/T1/zlj3A82b+Gi02P1+UCE3Exr90WQuAFUdNSJMKEyPP tVYg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51D/AuN6SvRXo+k7SczpcC/BuG5qdttSfKa1zxVhqK5amxJ9BPHt SLU/1NoAFGioFZ0oXy3CM3s5z+O5iB4pwmxpE6GcH5XkIBjsMzvtZRa7k1qi3RRx9jsPRbpulK8 RGRoW6h37Et7yLW8xhJ+R31XKAtRz4A1zBCT0NULfANMgQXRvuQ5h/BgCVhvn6nr1yA== X-Received: by 2002:a17:902:bd4a:: with SMTP id b10-v6mr41545728plx.209.1537482477431; Thu, 20 Sep 2018 15:27:57 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaQNGF4sIS0soN+oJ4ksnYP6gOcf1eDsXtsNgz35spDTXJ2WEJX2kNhtxZH29EmMDnumj4p X-Received: by 2002:a17:902:bd4a:: with SMTP id b10-v6mr41545689plx.209.1537482476529; Thu, 20 Sep 2018 15:27:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537482476; cv=none; d=google.com; s=arc-20160816; b=dzQ938xBsJFC6I37xxXMBXguX59Sg4fR36S1mubd6av0Ql47ETyolPEyZUEXGRTwxC UFLIvc8rAVpWHuIyqqSaQioA1fVCDmocgRNU8iBBuebJ05u0U1RmFby1edLDolPNVHse wF3e3D17TwOjtRgnr1go7kGC0pdfU5UW8eOm9w4amy5XOCuOhakV/8swc2ujYHycBJ5j WEPzJBd/YFt0hVovD0rN5WG3INCtUPHPmuXxDJPFECn40XEOo96EvTDE7lYxXYSifUEm StSoQ3xnWr35LCyC9DRL9GMLa65lI2Pr2ZFFDST13LbfuwBEGVUPs/hQ34AGrHkf+A+m EOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=rtBDAMbGmBlNQ1vdQfjvBiLqtdFJw4/gYxGBxmLdmPM=; b=KXdjAEUskAslhkMvnjeXtKG6VsVopk1kb/Ceyta6d4X8xHyWGFChNC0xGa97TObXL0 rpSN/977WOFUc38U+49MLLeDhINtOk9cGyEBftwz+lFlIxIEyYErySStiNtrEnzSKX/s O6TuPgf+2XGMpORpyMYIVDOxTI0jPzIXIcVftv/ifuMg8T2oPGu1KHa6XHsF4L5U1bay 5aTjcmR48q6AdSobtVuqt+PI55B+c3syUw4lahm1EEFTsSOeliU1cBi9tXyT+RlY+HJm PY6CYrL/bGyyth/UnXeEDsoNVKzz9zlwh3clzAQPNuGAAL9DOqnpNpvTjlmGrRbP57lH NPLw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id p7-v6si24646254plo.159.2018.09.20.15.27.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 15:27:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Sep 2018 15:27:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,282,1534834800"; d="scan'208";a="85285823" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga003.jf.intel.com with ESMTP; 20 Sep 2018 15:27:53 -0700 Subject: [PATCH v4 2/5] mm: Create non-atomic version of SetPageReserved for init use From: Alexander Duyck To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, mingo@kernel.org, dave.hansen@intel.com, jglisse@redhat.com, akpm@linux-foundation.org, logang@deltatee.com, dan.j.williams@intel.com, kirill.shutemov@linux.intel.com Date: Thu, 20 Sep 2018 15:27:53 -0700 Message-ID: <20180920222641.19464.75787.stgit@localhost.localdomain> In-Reply-To: <20180920215824.19464.8884.stgit@localhost.localdomain> References: <20180920215824.19464.8884.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I believe the main advantage of this is that it allows for more compiler optimization as the __set_bit operation can be reordered whereas the atomic version cannot. I tried adding a bit of documentation based on commit ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko Signed-off-by: Alexander Duyck Reviewed-by: Pavel Tatashin --- v4: Added comment about __set_bit vs set_bit to the patch description include/linux/page-flags.h | 1 + mm/page_alloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 934f91ef3f54..50ce1bddaf56 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -303,6 +303,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 712cab17f86f..29bd662fffd7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1239,7 +1239,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5513,7 +5518,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for