From patchwork Mon Sep 10 23:44:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 10594919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CF6414E0 for ; Mon, 10 Sep 2018 23:44:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B4F9291BB for ; Mon, 10 Sep 2018 23:44:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0F8BE291F4; Mon, 10 Sep 2018 23:44:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AB0AB291F2 for ; Mon, 10 Sep 2018 23:44:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9F14321127CC4; Mon, 10 Sep 2018 16:44:03 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2607:f8b0:4864:20::643; helo=mail-pl1-x643.google.com; envelope-from=alexander.duyck@gmail.com; receiver=linux-nvdimm@lists.01.org Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 477302112500D for ; Mon, 10 Sep 2018 16:44:02 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id b30-v6so8163125pla.0 for ; Mon, 10 Sep 2018 16:44:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=MJv9XTQcitO3gy5LNlz5Iv9gvth5EHKXzc6g36JI3Fc=; b=PO+/yHzKVe6BrtyAYDCrPoH1MS/MgG2SdH2MWE4Pu16baBk2jwxUqndfwe7rrtFeej oNi0xoLwWDoh/gd29YEKhez5Fy1B0bGTUGlQS4awi43cRSQ8iOBoIPq5oivMRFj72UY5 f/VQa99Utf8IKzhwwIc7PqOHfuuG7xBmEB/exPT+8K0id8EJdHbVG4JJatsFCVZDMBi5 2gMw6sr3IyU1oH20TXT94TN7u7jJ3We/C7qEycP6m4tSiuK2GvyPZIZu3sCMOBGm/r0U e8ToHPOnpYI6rp3t/sHjxch8Nt0PHUEgdYOTrCXwZ1s33o4RIGLboXtgZK6qo7kLre/9 vMFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=MJv9XTQcitO3gy5LNlz5Iv9gvth5EHKXzc6g36JI3Fc=; b=YIMC2ljlbXF6SUiSGGU/DTo7Qo957cazPDg1uU3HtO9MhVEXKHalW6fCxB12DIEggs UQ9rBUbfXuwkUznihIhBsN52PSElvM9ps4byOiekGA/dhtPEuXs1JzInYQXfyLhzZDFW hSONZWWRbN569oQuphVa7ee1GDloKlpT4erhiI45aW5VrCl+N06MpfzlU2pQOFWktnuK 6l237eSZe1MtpbTFEQ1ZcFCtuBNvWGkAzNWBu/Kj8FIgFH8wKqFTj6m+qZQa3u3aTAT/ FV5tbReC9glzyhzjs41EShpyj2meFukOJHC4htZFVjgAzJ2du7tuj1KDAIoYKGPmdYy1 UgUw== X-Gm-Message-State: APzg51DHsEhQpFoMoV3Q2GoVI4d0ZMyElz0cZk8B7iVrjyKQW8KpGJkv jVANcHjKm0rvwKQPW3ZPozE= X-Google-Smtp-Source: ANB0Vdae3zS05W6i/7y646GBZXtkkHI+KeQ/8novIzql8yYtiMRq7VA37016jMUlgSw8jVTKp/3lWw== X-Received: by 2002:a17:902:7402:: with SMTP id g2-v6mr24285976pll.321.1536623041876; Mon, 10 Sep 2018 16:44:01 -0700 (PDT) Received: from localhost.localdomain (static-50-53-21-37.bvtn.or.frontiernet.net. [50.53.21.37]) by smtp.gmail.com with ESMTPSA id u184-v6sm21581039pgd.46.2018.09.10.16.44.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Sep 2018 16:44:01 -0700 (PDT) Subject: [PATCH 4/4] nvdimm: Trigger the device probe on a cpu local to the device From: Alexander Duyck To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org Date: Mon, 10 Sep 2018 16:44:00 -0700 Message-ID: <20180910234400.4068.15541.stgit@localhost.localdomain> In-Reply-To: <20180910232615.4068.29155.stgit@localhost.localdomain> References: <20180910232615.4068.29155.stgit@localhost.localdomain> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pavel.tatashin@microsoft.com, mhocko@suse.com, dave.hansen@intel.com, jglisse@redhat.com, akpm@linux-foundation.org, mingo@kernel.org, kirill.shutemov@linux.intel.com Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP From: Alexander Duyck This patch is based off of the pci_call_probe function used to initialize PCI devices. The general idea here is to move the probe call to a location that is local to the memory being initialized. By doing this we can shave significant time off of the total time needed for initialization. With this patch applied I see a significant reduction in overall init time as without it the init varied between 23 and 37 seconds to initialize a 3GB node. With this patch applied the variance is only between 23 and 26 seconds to initialize each node. I hope to refine this further in the future by combining this logic into the async_schedule_domain code that is already in use. By doing that it would likely make this functionality redundant. Signed-off-by: Alexander Duyck Reviewed-by: Pavel Tatashin --- drivers/nvdimm/bus.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c index 8aae6dcc839f..5b73953176b1 100644 --- a/drivers/nvdimm/bus.c +++ b/drivers/nvdimm/bus.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "nd-core.h" #include "nd.h" #include "pfn.h" @@ -90,6 +91,48 @@ static void nvdimm_bus_probe_end(struct nvdimm_bus *nvdimm_bus) nvdimm_bus_unlock(&nvdimm_bus->dev); } +struct nvdimm_drv_dev { + struct nd_device_driver *nd_drv; + struct device *dev; +}; + +static long __nvdimm_call_probe(void *_nddd) +{ + struct nvdimm_drv_dev *nddd = _nddd; + struct nd_device_driver *nd_drv = nddd->nd_drv; + + return nd_drv->probe(nddd->dev); +} + +static int nvdimm_call_probe(struct nd_device_driver *nd_drv, + struct device *dev) +{ + struct nvdimm_drv_dev nddd = { nd_drv, dev }; + int rc, node, cpu; + + /* + * Execute driver initialization on node where the device is + * attached. This way the driver will be able to access local + * memory instead of having to initialize memory across nodes. + */ + node = dev_to_node(dev); + + cpu_hotplug_disable(); + + if (node < 0 || node >= MAX_NUMNODES || !node_online(node)) + cpu = nr_cpu_ids; + else + cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask); + + if (cpu < nr_cpu_ids) + rc = work_on_cpu(cpu, __nvdimm_call_probe, &nddd); + else + rc = __nvdimm_call_probe(&nddd); + + cpu_hotplug_enable(); + return rc; +} + static int nvdimm_bus_probe(struct device *dev) { struct nd_device_driver *nd_drv = to_nd_device_driver(dev->driver); @@ -104,7 +147,7 @@ static int nvdimm_bus_probe(struct device *dev) dev->driver->name, dev_name(dev)); nvdimm_bus_probe_start(nvdimm_bus); - rc = nd_drv->probe(dev); + rc = nvdimm_call_probe(nd_drv, dev); if (rc == 0) nd_region_probe_success(nvdimm_bus, dev); else