From patchwork Tue Sep 25 20:19:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10614781 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15F59913 for ; Tue, 25 Sep 2018 20:19:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 087182AD55 for ; Tue, 25 Sep 2018 20:19:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F09132AD5B; Tue, 25 Sep 2018 20:19:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 936462AD55 for ; Tue, 25 Sep 2018 20:19:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AACDC8E00BB; Tue, 25 Sep 2018 16:19:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A31FC8E00A4; Tue, 25 Sep 2018 16:19:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D5C58E00BB; Tue, 25 Sep 2018 16:19:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 4582E8E00A4 for ; Tue, 25 Sep 2018 16:19:18 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id j15-v6so13108847pff.12 for ; Tue, 25 Sep 2018 13:19:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=Tb8udKRBxLygqYHpLg+w+EDOcH2OiGwPljwi6ffZtBA=; b=C3Yps+0ffY6W6MQ5A7jJvVCL0DgJdpylf24CNJy03IiPPFMlAWMORjBcUV/r23yiYo U9SfrXEBlnqnl+j0wFBB5q/JfLZfWIfp0vh+bdHNqInRl7SDLpp0bnwg68iKwRiDrHiT tI6Ge4sj9pEY7uegDtRxnByTz4YeBCzr52E2ZVuPj7tiGzXjVlN0gA2bXHxybSSnKSsL 8DJVyjHQYFsWgWBq+erx59oDJJgGNNjPJYPk292X/ZuKqW70Os7Gkz/dV9fHYUcs04VI x+xr/5vNgi3PenA92711ici0bmMrFXHUOsAyW9AuPq7eYIDwUHs1zVIWv1vdkWCX1L5e gx8w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfohjPlhUSJLSsOzZNaOg+LS3vlQ4790/gMsZ+F/N4n9QFIU1vnUN 4pJjDcIhZuQtiAZqaKbK0O+vduOUYdfL7ODxYkvvEvaMPH9OW9sMB1wYA61Mc23QdWfmXeIWHAa 4JVXn3DF9jQZinCy3WyihpWpKxNXzOvOV3TcstqaB1a9JZIFD5EaKNt1URreafshNGw== X-Received: by 2002:a63:65c7:: with SMTP id z190-v6mr2535248pgb.146.1537906757938; Tue, 25 Sep 2018 13:19:17 -0700 (PDT) X-Google-Smtp-Source: ACcGV60g4VG9qQGImW+2HLwFeUJlXYwsW6o0WsiuEH+ofcV1HXI3mafAqgCDGBaokvxzgd2mUMZH X-Received: by 2002:a63:65c7:: with SMTP id z190-v6mr2535211pgb.146.1537906757140; Tue, 25 Sep 2018 13:19:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537906757; cv=none; d=google.com; s=arc-20160816; b=ywWFUupMuD2PTPLKSDVk5Rb004QzXIfgIwgeIJBlqDLK9g+4wRXpDBNiBgF898/FVD hoARRu7r1x2FIQ0sRRnb7L9uEHqEv5hD1VOTYZlKYmpIyGYZ4Mh3YdGFfBIY+s3tZNMX ZgHs+b5vmiQbZNfINcoAMnWE0sVnADkQie/9DWm+U8XSTezQVeHS5eGgv12miYJv1Lct fz4Vo6O35qtP35iGJq53/ERw4KqifEAzk3BewwHkgrFBFNL+LOiPYDUTDFZuzh27GPJY ksnLdIcB3H7yVFcdgh284Vq7L0TkA1ovRQGpfQr0NiuWxLN6NbXDbu8hvRzrcZd1btEl HGkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=Tb8udKRBxLygqYHpLg+w+EDOcH2OiGwPljwi6ffZtBA=; b=f/9X3C1Npf077kozwnxtLboGe4DHxa/HqnOZZZZuWQh93Z9mDg8siLJGvyAycd+0+0 Q3uAJvmuJH18HuLksOh/jjwTvWaUpAU8ilQw0fdaCEX2W9i02qiSjkPsQUSke37DzH+b fV1wCXUu5G4H+XRtSDZSXyjgsfuWVtDt9Eoxtmtw1yNSi0qYPEWbDKe3N2dd8H/nkGSJ M/kNAGMygKb2qHU8gvFh+rFOqSYnS0QBb2yNjf3aOU/5nsSlC7rjOAcyvrWP6DeTgEB8 Cir5HHCfvRMj0owXEh0gWi0SrlA16cmk+uEOjbAiUPo/TQkuwiqL7BujZnZyCvP0ahF1 5a9g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga14.intel.com (mga14.intel.com. [192.55.52.115]) by mx.google.com with ESMTPS id s21-v6si3300561pgm.651.2018.09.25.13.19.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Sep 2018 13:19:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.115 as permitted sender) client-ip=192.55.52.115; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Sep 2018 13:19:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,303,1534834800"; d="scan'208";a="77230044" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga006.jf.intel.com with ESMTP; 25 Sep 2018 13:19:15 -0700 Subject: [PATCH v5 1/4] mm: Remove now defunct NO_BOOTMEM from depends list for deferred init From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Tue, 25 Sep 2018 13:19:15 -0700 Message-ID: <20180925201814.3576.15105.stgit@localhost.localdomain> In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> References: <20180925200551.3576.18755.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The CONFIG_NO_BOOTMEM config option was recently removed by the patch "mm: remove CONFIG_NO_BOOTMEM" (https://patchwork.kernel.org/patch/10600647/). However it looks like it missed a few spots. The biggest one being the dependency for deferred init. This patch goes through and removes the remaining spots that appear to have been missed in the patch so that I am able to build again with deferred memory initialization. Signed-off-by: Alexander Duyck Acked-by: Mike Rapoport --- v5: New patch, added to fix regression found in latest linux-next arch/csky/Kconfig | 1 - mm/Kconfig | 1 - 2 files changed, 2 deletions(-) diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index fe2c94b94fe3..fb2a0ae84dd5 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -38,7 +38,6 @@ config CSKY select HAVE_MEMBLOCK select MAY_HAVE_SPARSE_IRQ select MODULES_USE_ELF_RELA if MODULES - select NO_BOOTMEM select OF select OF_EARLY_FLATTREE select OF_RESERVED_MEM diff --git a/mm/Kconfig b/mm/Kconfig index c6a0d82af45f..b4421aa608c4 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -631,7 +631,6 @@ config MAX_STACK_SIZE_MB config DEFERRED_STRUCT_PAGE_INIT bool "Defer initialisation of struct pages to kthreads" default n - depends on NO_BOOTMEM depends on SPARSEMEM depends on !NEED_PER_CPU_KM depends on 64BIT From patchwork Tue Sep 25 20:20:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10614785 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37C2D174A for ; Tue, 25 Sep 2018 20:20:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C7672AD74 for ; Tue, 25 Sep 2018 20:20:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E0B32AD65; Tue, 25 Sep 2018 20:20:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 410CE2AD6A for ; Tue, 25 Sep 2018 20:20:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 418268E00BC; Tue, 25 Sep 2018 16:20:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3C8268E00A4; Tue, 25 Sep 2018 16:20:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DE738E00BC; Tue, 25 Sep 2018 16:20:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id E09848E00A4 for ; Tue, 25 Sep 2018 16:20:31 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id j15-v6so13226702pfi.10 for ; Tue, 25 Sep 2018 13:20:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=6yCc4dIGwjjqYKqZsk0S704AskE7eH68lGQlfn8PRns=; b=Fm2IYVI765rWbYXikjvJJUS4DsFwsqoxKdXU54OZ6/0ksAcEMyk1ITvRsuxEDpoFMC 6vca77hKSNyFgWugu3VpxlY8WOkbnQDJK/2lSzHpwiNNNj9tGqgZNauxrADmrCiRLzfw jIvChJrz+BuN7XWDYKcDiDXiKBFWCk1qEqHyoQZgijTF7gU+bwAhWaeaT9CEgg20DuNW 7KcOyXfZnUXlBcR0TGLpF1CAY3nJMF8yp1i9GuwnZcW3AaER2uB/BBa0Zy/OI03Pwrxi Lp/UW2UMqZ4ccV302pzr1Flr/oHm8th6UOd0CbWzlDkiOsMczMnuueKyT5GRKcbOW8eQ 2ylA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfohSlp7nsCQuO9QUgbi2FGm8U71OZe4x9yWlyDNV0sBc0yrts1zE Fp0y/+zxZDTdcvuh4rmkORaKZwiD0LQRGF0P3cwaU98wSHvMstddf6w1EasQS3mtWnUkm0ok3FB DqYSv7Em9pTf3OFrtPNSE21AaQPMH80auHRdZMB5v4mp2/lmHodr8mVdI7d0jTfONVw== X-Received: by 2002:a63:6c89:: with SMTP id h131-v6mr2457996pgc.237.1537906831484; Tue, 25 Sep 2018 13:20:31 -0700 (PDT) X-Google-Smtp-Source: ACcGV60n66/bVK1INgCsSvwAq5TKKYCdtcn+bQ0jh9/HuJvQMjQH2yrBiM7cFkWW0pFCqVdeubbe X-Received: by 2002:a63:6c89:: with SMTP id h131-v6mr2457945pgc.237.1537906830401; Tue, 25 Sep 2018 13:20:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537906830; cv=none; d=google.com; s=arc-20160816; b=IjfIhYGDEuerNGET/itkRWoERpiRdgraMxEmiHh1zy4XQw6ICgw30t8CQahPZlLVys kCXdikrGS8ukRz6aUsHAhOMSof/PPEC3p6cvkAROA2t/rfoCJW3fJEN3RUbQ4XRY6mna zbo0mpj71mIdFGPCbSmjqHn9NBxrzg+LkgpERnrMfLM/0qgwA2F1wi6vEQDWrDBGnLQl G+w750XA7Fe0ZuTidqCCyjpODN2SazezmNrtwMYpd48syMirml/BJG11PWxtBy85GjJ3 ekim3VIUBLSbWzHprljSsxqhoQm1MmehbquxiNgx/tbCV6f6562oHNM7M0A5lT9uMrEG 55FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=6yCc4dIGwjjqYKqZsk0S704AskE7eH68lGQlfn8PRns=; b=cTRYQn28K6kTP7WfSH0tU2bggiuFK5N/exo39K7JL+n/bo02afQgm8yJc+o9ArnQ/S xtGMTOWofxw2rlwyuNPdmjekqgh8i0swzDNHjmD9vmspjB3e36lv2RWOx7TFyQZAI+30 7M7bhUvW5Sd3sdAWrnOXtbnyIOIcFKx0xGpgyYkmuNUIOHOsdHrtuLNb2+Im4E0ZDNug uZhJMZTX4ETwPplLgp6XS4xBimSoVyx2iWxx5bYWJHawxLJ20j2imQG4fWsU/K0qefRE xwoY/GkCuX26mPUSO49wycy4R+sJJvMAc2pDtSr0bzYRhhopidX5QmbjfyjPau5FPZq/ AQXw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga12.intel.com (mga12.intel.com. [192.55.52.136]) by mx.google.com with ESMTPS id l10-v6si2298288pfe.310.2018.09.25.13.20.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Sep 2018 13:20:30 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) client-ip=192.55.52.136; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Sep 2018 13:20:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,303,1534834800"; d="scan'208";a="77230187" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga006.jf.intel.com with ESMTP; 25 Sep 2018 13:20:12 -0700 Subject: [PATCH v5 2/4] mm: Provide kernel parameter to allow disabling page init poisoning From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Tue, 25 Sep 2018 13:20:12 -0700 Message-ID: <20180925201921.3576.84239.stgit@localhost.localdomain> In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> References: <20180925200551.3576.18755.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On systems with a large amount of memory it can take a significant amount of time to initialize all of the page structs with the PAGE_POISON_PATTERN value. I have seen it take over 2 minutes to initialize a system with over 12TB of RAM. In order to work around the issue I had to disable CONFIG_DEBUG_VM and then the boot time returned to something much more reasonable as the arch_add_memory call completed in milliseconds versus seconds. However in doing that I had to disable all of the other VM debugging on the system. In order to work around a kernel that might have CONFIG_DEBUG_VM enabled on a system that has a large amount of memory I have added a new kernel parameter named "vm_debug" that can be set to "-" in order to disable it. Reviewed-by: Pavel Tatashin Signed-off-by: Alexander Duyck --- v3: Switched from kernel config option to parameter v4: Added comment to parameter handler to record when option is disabled Updated parameter description based on feedback from Michal Hocko Fixed GB vs TB typo in patch description. Switch to vm_debug option similar to slub_debug v5: Rebased on latest linux-next Documentation/admin-guide/kernel-parameters.txt | 12 ++++++ include/linux/page-flags.h | 8 ++++ mm/debug.c | 46 +++++++++++++++++++++++ mm/memblock.c | 5 +-- mm/sparse.c | 4 +- 5 files changed, 69 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 42d9150047f2..d9ad70ccbdc2 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4811,6 +4811,18 @@ This is actually a boot loader parameter; the value is passed to the kernel using a special protocol. + vm_debug[=options] [KNL] Available with CONFIG_DEBUG_VM=y. + May slow down system boot speed, especially when + enabled on systems with a large amount of memory. + All options are enabled by default, and this + interface is meant to allow for selectively + enabling or disabling specific virtual memory + debugging features. + + Available options are: + P Enable page structure init time poisoning + - Disable all of the above options + vmalloc=nn[KMG] [KNL,BOOT] Forces the vmalloc area to have an exact size of . This can be used to increase the minimum size (128MB on x86). It can also be used to diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 4d99504f6496..934f91ef3f54 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -163,6 +163,14 @@ static inline int PagePoisoned(const struct page *page) return page->flags == PAGE_POISON_PATTERN; } +#ifdef CONFIG_DEBUG_VM +void page_init_poison(struct page *page, size_t size); +#else +static inline void page_init_poison(struct page *page, size_t size) +{ +} +#endif + /* * Page flags policies wrt compound pages * diff --git a/mm/debug.c b/mm/debug.c index bd10aad8539a..cdacba12e09a 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "internal.h" @@ -175,4 +176,49 @@ void dump_mm(const struct mm_struct *mm) ); } +static bool page_init_poisoning __read_mostly = true; + +static int __init setup_vm_debug(char *str) +{ + bool __page_init_poisoning = true; + + /* + * Calling vm_debug with no arguments is equivalent to requesting + * to enable all debugging options we can control. + */ + if (*str++ != '=' || !*str) + goto out; + + __page_init_poisoning = false; + if (*str == '-') + goto out; + + while (*str) { + switch (tolower(*str)) { + case'p': + __page_init_poisoning = true; + break; + default: + pr_err("vm_debug option '%c' unknown. skipped\n", + *str); + } + + str++; + } +out: + if (page_init_poisoning && !__page_init_poisoning) + pr_warn("Page struct poisoning disabled by kernel command line option 'vm_debug'\n"); + + page_init_poisoning = __page_init_poisoning; + + return 1; +} +__setup("vm_debug", setup_vm_debug); + +void page_init_poison(struct page *page, size_t size) +{ + if (page_init_poisoning) + memset(page, PAGE_POISON_PATTERN, size); +} +EXPORT_SYMBOL_GPL(page_init_poison); #endif /* CONFIG_DEBUG_VM */ diff --git a/mm/memblock.c b/mm/memblock.c index 32e5c62ee142..b0ebca546ba1 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1503,10 +1503,9 @@ void * __init memblock_alloc_try_nid_raw( ptr = memblock_alloc_internal(size, align, min_addr, max_addr, nid); -#ifdef CONFIG_DEBUG_VM if (ptr && size > 0) - memset(ptr, PAGE_POISON_PATTERN, size); -#endif + page_init_poison(ptr, size); + return ptr; } diff --git a/mm/sparse.c b/mm/sparse.c index c0788e3d8513..ab2ac45e0440 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -696,13 +696,11 @@ int __meminit sparse_add_one_section(struct pglist_data *pgdat, goto out; } -#ifdef CONFIG_DEBUG_VM /* * Poison uninitialized struct pages in order to catch invalid flags * combinations. */ - memset(memmap, PAGE_POISON_PATTERN, sizeof(struct page) * PAGES_PER_SECTION); -#endif + page_init_poison(memmap, sizeof(struct page) * PAGES_PER_SECTION); section_mark_present(ms); sparse_init_one_section(ms, section_nr, memmap, usemap); From patchwork Tue Sep 25 20:20:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10614789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FD85913 for ; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 613682AD55 for ; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 552922AD65; Tue, 25 Sep 2018 20:21:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7AC72AD55 for ; Tue, 25 Sep 2018 20:21:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0978F8E00BD; Tue, 25 Sep 2018 16:21:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 047D88E00A4; Tue, 25 Sep 2018 16:21:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E78378E00BD; Tue, 25 Sep 2018 16:21:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id A43B18E00A4 for ; Tue, 25 Sep 2018 16:21:14 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id x85-v6so13241993pfe.13 for ; Tue, 25 Sep 2018 13:21:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=E9r7KhEKHjUQ1Pm7kAQ0/qTduElaq7EW1gzJiqC69fo=; b=fSoF/PFiLSjnDoLFNpcOjVkG8fQnWurWF7HiicRSHzU2LeU36xzet368lbAPf0C9XJ Bdxfi2MNmVxg37tJeg42p6AsM8gT38/imfrAmkh7h7znFlBjIDPTxVouZBJrlm53aJhM aql3mJZD86Dy6TWRmyhN5aFUM98k5cU4zpjDo+QoPk+OYDtbpseLTVaD/Y+7NeYn74M6 LUG3J80+xqEGY01Q+eWg4nlKXUmnebzYREm9qisKweVEg9/5U7Hw+isOZCehmOGATjJd OpFu8s4Kjt4DmF3+BXTI4WnPkMhBVcQRDFUHaBA+IaLgrOUPORA4FOtubyeefawmEi4h Zkqw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfogzRPZ1dcnZmwH8kX8SlRd9aOeiWRsCA4ei2tj0O/XPsNxbhFHQ F3wBGAS+oW8yAfZi8qNeJ40F6MkzCc4Lp58iK3HRaSRwSivUFOBDsIgF3fe6eZ+lB5uEM/trT2t pLYKEl0+Rm//wTLOYigPAfokRn1Y8pRm5e0ZV3NUcVFvJooUMRH2MIct8VLWePBjpvQ== X-Received: by 2002:a17:902:8697:: with SMTP id g23-v6mr2722647plo.292.1537906874338; Tue, 25 Sep 2018 13:21:14 -0700 (PDT) X-Google-Smtp-Source: ACcGV63e2B/NLjOecJNneWbd5wcxxugyb8IAGrrP/8zRVQO0jdm/ywvimBoDORRGOyUGabodG1sC X-Received: by 2002:a17:902:8697:: with SMTP id g23-v6mr2722583plo.292.1537906873046; Tue, 25 Sep 2018 13:21:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537906873; cv=none; d=google.com; s=arc-20160816; b=BBrI9i80gCE1fqkaPWjW+IPxk4d3lkLWMkOepjrSNLzI72lXoFDU8vKA3dP1ujqrQP oVxfWp5UPPjDI1ulMewruhRD/mfh/a04JKSut40rsjZra704lxfBMtHDF+1rzNWfMbII 5CqVYImJhQ7q66HKZsIK+8USAjj+odRltjL9IFFvZIXpHaJJCuTMvsOvxmiQpyueqa7e gWx+Uwxo4I3sytGIhpj0vU6/3ixmlv/pa8ZpYkkEUprniIW7Fyy8mFc08Ci4RaDE4tRq nUlSVudwiinHCDvt8tjkMB68pDIyl2m9SKUan2nJAPIzIlg2YfCB7IDTH3+r3cZtMS4U UvAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=E9r7KhEKHjUQ1Pm7kAQ0/qTduElaq7EW1gzJiqC69fo=; b=IXNEMHhlUwU/XVerfr/kil8AVWkAezDqtyktm0rbddayezS4EwM2dovQQklOGGh3mF BB3ewjzP8RLpZLCmPjEghFypgbCu7xtAvt8GAdlM6FloSuZ2AHMPWb+uk0zSSdWOOMg6 uyY7wfnBaRXV3gg7J9rihVb2JFha6ZpwqSSUx4y4Pjj4PL46JOY0IiP0LfLsvyv9VJvH UB7oJlsrSjDMEf+GKH1nAC5+VjZTEL3/40u+n93qRAhB5Y45hh9U8wMbHPYvJLnt7oow YmN45UTENvpQ10fwwEbTou+BrCXJhfl2VhoABnFS/gJNU7ySrVgScibqOm1uTV6103Mc Ntqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id c63-v6si2892404pfb.178.2018.09.25.13.21.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Sep 2018 13:21:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Sep 2018 13:21:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,303,1534834800"; d="scan'208";a="77230394" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga006.jf.intel.com with ESMTP; 25 Sep 2018 13:20:47 -0700 Subject: [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Tue, 25 Sep 2018 13:20:47 -0700 Message-ID: <20180925202018.3576.11607.stgit@localhost.localdomain> In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> References: <20180925200551.3576.18755.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It doesn't make much sense to use the atomic SetPageReserved at init time when we are using memset to clear the memory and manipulating the page flags via simple "&=" and "|=" operations in __init_single_page. This patch adds a non-atomic version __SetPageReserved that can be used during page init and shows about a 10% improvement in initialization times on the systems I have available for testing. On those systems I saw initialization times drop from around 35 seconds to around 32 seconds to initialize a 3TB block of persistent memory. I believe the main advantage of this is that it allows for more compiler optimization as the __set_bit operation can be reordered whereas the atomic version cannot. I tried adding a bit of documentation based on commit ("mm, memory_hotplug: do not associate hotadded memory to zones until online"). Ideally the reserved flag should be set earlier since there is a brief window where the page is initialization via __init_single_page and we have not set the PG_Reserved flag. I'm leaving that for a future patch set as that will require a more significant refactor. Acked-by: Michal Hocko Reviewed-by: Pavel Tatashin Signed-off-by: Alexander Duyck --- v4: Added comment about __set_bit vs set_bit to the patch description v5: No change include/linux/page-flags.h | 1 + mm/page_alloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 934f91ef3f54..50ce1bddaf56 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -303,6 +303,7 @@ static inline void page_init_poison(struct page *page, size_t size) PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) __CLEARPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) + __SETPAGEFLAG(Reserved, reserved, PF_NO_COMPOUND) PAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __CLEARPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) __SETPAGEFLAG(SwapBacked, swapbacked, PF_NO_TAIL) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 511447ac02cf..926ad3083b28 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1238,7 +1238,12 @@ void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end) /* Avoid false-positive PageTail() */ INIT_LIST_HEAD(&page->lru); - SetPageReserved(page); + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } } @@ -5512,7 +5517,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, page = pfn_to_page(pfn); __init_single_page(page, pfn, zone, nid); if (context == MEMMAP_HOTPLUG) - SetPageReserved(page); + __SetPageReserved(page); /* * Mark the block movable so that blocks are reserved for From patchwork Tue Sep 25 20:21:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10614793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78D11913 for ; Tue, 25 Sep 2018 20:21:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BB112AD55 for ; Tue, 25 Sep 2018 20:21:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F77E2AD65; Tue, 25 Sep 2018 20:21:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1B3A2AD55 for ; Tue, 25 Sep 2018 20:21:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D68B38E00BE; Tue, 25 Sep 2018 16:21:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D3E538E00A4; Tue, 25 Sep 2018 16:21:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C54CB8E00BE; Tue, 25 Sep 2018 16:21:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 858338E00A4 for ; Tue, 25 Sep 2018 16:21:39 -0400 (EDT) Received: by mail-pf1-f199.google.com with SMTP id o27-v6so13241877pfj.6 for ; Tue, 25 Sep 2018 13:21:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=HTTzsKvBji4ObeeCB5X0ItbEtI4cDH5BeFgG169zYto=; b=hSKrOl0swwIgn3lbfMIJDwjF/5fsPpLHW6etuISbGy/LLtHMErYMC2qthxwT8E/6xQ zzXkOHFTDkpu4osevNe3QXaYrUFVZddV76HmA7hmyhqt4/hbVWDDuT9zEe293PeJFu46 IB3ts1B6RxKq8DuWORZisvpKVmronsHwfoeaduvszxUSeAo2Ii3iTcWKh8krlqtCMQN0 +6nUoP/EvRNP0NMIe0Jm+4HifkZwosFbsk668eyJcB6FOWTTzjAEunReljNbfvkffZk8 K29Nnk7jdiONuPur9ASs7nGe7WbQ8w2s43mbrENJICn3/ZzmZFL3e4F54PLVQdiOYOj1 LltQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfoiBsyBjQ6nHSWGe2X3WpVoTzCG7HovDSTCH6FSGk+HU+JlxM6iK sHLdTRAio1zfsB7ITZkM50jDxjjYWpmDynb2nGUNipVJVtYQ2uk49aOqrwDkJm4U6OgRCf5eWLi EVNLoKV3HQiADNhAgIatEa3QHn74he7R1jc8eTwuQgPNhkgfYd2MBlDhP5pdgy60JDg== X-Received: by 2002:a62:c8c3:: with SMTP id i64-v6mr2726918pfk.183.1537906899173; Tue, 25 Sep 2018 13:21:39 -0700 (PDT) X-Google-Smtp-Source: ACcGV61G5buKEKr4QDHoH8duchCQa4gqNN6RiKXRVjBbt4trc2m4n6VXz9a/mmgXa7IrPtMM0yfj X-Received: by 2002:a62:c8c3:: with SMTP id i64-v6mr2726846pfk.183.1537906897917; Tue, 25 Sep 2018 13:21:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537906897; cv=none; d=google.com; s=arc-20160816; b=XVztvwo9fYzexIFe0rUzvFX9eJp/o4twTkQZXlBikyC6ZlRQU74157I4Uzt/djjHy4 YyaTPkHg7poXYxnvj8goR2A0Lr1SZQFpGhJUratBnHkgP1+2ZaIwDUQpj69wVYDoc/aT j5WC4Gk/WSI70xHXoxKj+C3sinp1TPYwzW+g+2+KAwCVDBbWmBNjYqpTuUJNRL/ykHae sXfu0WO8TMn5LBTsHoO7+IGlTOgcrkv0SkICNFzvFM4vmYZLhNHegVlpLSMKyrmlXL/6 2kAZfjb/bJFtwMGU76Bbn7NO3wr3mRs6mdqQkC0T9y0cYfmyDKU2Tp2cluXK/z7KT3JY om7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=HTTzsKvBji4ObeeCB5X0ItbEtI4cDH5BeFgG169zYto=; b=GZiBKrUlNthdGIgOKTtIh9wLJIdmN+l2kPAk8sP+fTKzN57v8c4gqUw3zeaMCfnial runM8dMcv2wCbDeSQZYL03Uhz0mVK3bxbHQjZ0+N+83PywEWcorzuY1z8TwZfDmv+9uB B5YCvBrAHH8JNCiUWF+kyNaYlYPtujlBR4QHuEnJfLmGkhvI8je/aq9tKeUJ1za0bvvm Sq5G8DPSnnXXXMWQv/biSvlZQNoTMYKwAZW1ANnGKuXl5atRNIC/Loc6oFtUm4QVsVX0 bsa1HXuAdfQF9sD1DSy0u8wm/9FgzpkS4WAeuRvZryJTe/WIO1iO8nVCSw4jmiJ7D/ll 7d/Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id s7-v6si2864832pfm.217.2018.09.25.13.21.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Sep 2018 13:21:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of alexander.h.duyck@linux.intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=alexander.h.duyck@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Sep 2018 13:21:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,303,1534834800"; d="scan'208";a="77230417" Received: from ahduyck-mobl.amr.corp.intel.com (HELO localhost.localdomain) ([10.7.198.152]) by orsmga006.jf.intel.com with ESMTP; 25 Sep 2018 13:21:25 -0700 Subject: [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap From: Alexander Duyck To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.jiang@intel.com, dave.hansen@intel.com, jglisse@redhat.com, rppt@linux.vnet.ibm.com, dan.j.williams@intel.com, logang@deltatee.com, mingo@kernel.org, kirill.shutemov@linux.intel.com Date: Tue, 25 Sep 2018 13:21:24 -0700 Message-ID: <20180925202053.3576.66039.stgit@localhost.localdomain> In-Reply-To: <20180925200551.3576.18755.stgit@localhost.localdomain> References: <20180925200551.3576.18755.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The ZONE_DEVICE pages were being initialized in two locations. One was with the memory_hotplug lock held and another was outside of that lock. The problem with this is that it was nearly doubling the memory initialization time. Instead of doing this twice, once while holding a global lock and once without, I am opting to defer the initialization to the one outside of the lock. This allows us to avoid serializing the overhead for memory init and we can instead focus on per-node init times. One issue I encountered is that devm_memremap_pages and hmm_devmmem_pages_create were initializing only the pgmap field the same way. One wasn't initializing hmm_data, and the other was initializing it to a poison value. Since this is something that is exposed to the driver in the case of hmm I am opting for a third option and just initializing hmm_data to 0 since this is going to be exposed to unknown third party drivers. Reviewed-by: Pavel Tatashin Signed-off-by: Alexander Duyck --- v4: Moved moved memmap_init_zone_device to below memmmap_init_zone to avoid merge conflicts with other changes in the kernel. v5: No change include/linux/mm.h | 2 + kernel/memremap.c | 24 +++++--------- mm/hmm.c | 12 ++++--- mm/page_alloc.c | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 107 insertions(+), 23 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 06d7d7576f8d..7312fb78ef31 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -848,6 +848,8 @@ static inline bool is_zone_device_page(const struct page *page) { return page_zonenum(page) == ZONE_DEVICE; } +extern void memmap_init_zone_device(struct zone *, unsigned long, + unsigned long, struct dev_pagemap *); #else static inline bool is_zone_device_page(const struct page *page) { diff --git a/kernel/memremap.c b/kernel/memremap.c index 5b8600d39931..d0c32e473f82 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -175,10 +175,10 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) struct vmem_altmap *altmap = pgmap->altmap_valid ? &pgmap->altmap : NULL; struct resource *res = &pgmap->res; - unsigned long pfn, pgoff, order; + struct dev_pagemap *conflict_pgmap; pgprot_t pgprot = PAGE_KERNEL; + unsigned long pgoff, order; int error, nid, is_ram; - struct dev_pagemap *conflict_pgmap; align_start = res->start & ~(SECTION_SIZE - 1); align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE) @@ -256,19 +256,13 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) if (error) goto err_add_memory; - for_each_device_pfn(pfn, pgmap) { - struct page *page = pfn_to_page(pfn); - - /* - * ZONE_DEVICE pages union ->lru with a ->pgmap back - * pointer. It is a bug if a ZONE_DEVICE page is ever - * freed or placed on a driver-private list. Seed the - * storage with LIST_POISON* values. - */ - list_del(&page->lru); - page->pgmap = pgmap; - percpu_ref_get(pgmap->ref); - } + /* + * Initialization of the pages has been deferred until now in order + * to allow us to do the work while not holding the hotplug lock. + */ + memmap_init_zone_device(&NODE_DATA(nid)->node_zones[ZONE_DEVICE], + align_start >> PAGE_SHIFT, + align_size >> PAGE_SHIFT, pgmap); devm_add_action(dev, devm_memremap_pages_release, pgmap); diff --git a/mm/hmm.c b/mm/hmm.c index c968e49f7a0c..774d684fa2b4 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -1024,7 +1024,6 @@ static int hmm_devmem_pages_create(struct hmm_devmem *devmem) resource_size_t key, align_start, align_size, align_end; struct device *device = devmem->device; int ret, nid, is_ram; - unsigned long pfn; align_start = devmem->resource->start & ~(PA_SECTION_SIZE - 1); align_size = ALIGN(devmem->resource->start + @@ -1109,11 +1108,14 @@ static int hmm_devmem_pages_create(struct hmm_devmem *devmem) align_size >> PAGE_SHIFT, NULL); mem_hotplug_done(); - for (pfn = devmem->pfn_first; pfn < devmem->pfn_last; pfn++) { - struct page *page = pfn_to_page(pfn); + /* + * Initialization of the pages has been deferred until now in order + * to allow us to do the work while not holding the hotplug lock. + */ + memmap_init_zone_device(&NODE_DATA(nid)->node_zones[ZONE_DEVICE], + align_start >> PAGE_SHIFT, + align_size >> PAGE_SHIFT, &devmem->pagemap); - page->pgmap = &devmem->pagemap; - } return 0; error_add_memory: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 926ad3083b28..7ec0997ded39 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5489,12 +5489,23 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, if (highest_memmap_pfn < end_pfn - 1) highest_memmap_pfn = end_pfn - 1; +#ifdef CONFIG_ZONE_DEVICE /* * Honor reservation requested by the driver for this ZONE_DEVICE - * memory + * memory. We limit the total number of pages to initialize to just + * those that might contain the memory mapping. We will defer the + * ZONE_DEVICE page initialization until after we have released + * the hotplug lock. */ - if (altmap && start_pfn == altmap->base_pfn) - start_pfn += altmap->reserve; + if (zone == ZONE_DEVICE) { + if (!altmap) + return; + + if (start_pfn == altmap->base_pfn) + start_pfn += altmap->reserve; + end_pfn = altmap->base_pfn + vmem_altmap_offset(altmap); + } +#endif for (pfn = start_pfn; pfn < end_pfn; pfn++) { /* @@ -5538,6 +5549,81 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, } } +#ifdef CONFIG_ZONE_DEVICE +void __ref memmap_init_zone_device(struct zone *zone, + unsigned long start_pfn, + unsigned long size, + struct dev_pagemap *pgmap) +{ + unsigned long pfn, end_pfn = start_pfn + size; + struct pglist_data *pgdat = zone->zone_pgdat; + unsigned long zone_idx = zone_idx(zone); + unsigned long start = jiffies; + int nid = pgdat->node_id; + + if (WARN_ON_ONCE(!pgmap || !is_dev_zone(zone))) + return; + + /* + * The call to memmap_init_zone should have already taken care + * of the pages reserved for the memmap, so we can just jump to + * the end of that region and start processing the device pages. + */ + if (pgmap->altmap_valid) { + struct vmem_altmap *altmap = &pgmap->altmap; + + start_pfn = altmap->base_pfn + vmem_altmap_offset(altmap); + size = end_pfn - start_pfn; + } + + for (pfn = start_pfn; pfn < end_pfn; pfn++) { + struct page *page = pfn_to_page(pfn); + + __init_single_page(page, pfn, zone_idx, nid); + + /* + * Mark page reserved as it will need to wait for onlining + * phase for it to be fully associated with a zone. + * + * We can use the non-atomic __set_bit operation for setting + * the flag as we are still initializing the pages. + */ + __SetPageReserved(page); + + /* + * ZONE_DEVICE pages union ->lru with a ->pgmap back + * pointer and hmm_data. It is a bug if a ZONE_DEVICE + * page is ever freed or placed on a driver-private list. + */ + page->pgmap = pgmap; + page->hmm_data = 0; + + /* + * Mark the block movable so that blocks are reserved for + * movable at startup. This will force kernel allocations + * to reserve their blocks rather than leaking throughout + * the address space during boot when many long-lived + * kernel allocations are made. + * + * bitmap is created for zone's valid pfn range. but memmap + * can be created for invalid pages (for alignment) + * check here not to call set_pageblock_migratetype() against + * pfn out of zone. + * + * Please note that MEMMAP_HOTPLUG path doesn't clear memmap + * because this is done early in sparse_add_one_section + */ + if (!(pfn & (pageblock_nr_pages - 1))) { + set_pageblock_migratetype(page, MIGRATE_MOVABLE); + cond_resched(); + } + } + + pr_info("%s initialised, %lu pages in %ums\n", dev_name(pgmap->dev), + size, jiffies_to_msecs(jiffies - start)); +} + +#endif static void __meminit zone_init_free_lists(struct zone *zone) { unsigned int order, t;