From patchwork Fri May 17 21:54:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 10948693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E1E876 for ; Fri, 17 May 2019 21:54:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EBEB2848F for ; Fri, 17 May 2019 21:54:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 320DC28495; Fri, 17 May 2019 21:54:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC1602848F for ; Fri, 17 May 2019 21:54:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 739D16B0005; Fri, 17 May 2019 17:54:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6C4E56B0006; Fri, 17 May 2019 17:54:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53D6C6B0008; Fri, 17 May 2019 17:54:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 2FEDA6B0005 for ; Fri, 17 May 2019 17:54:44 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id w184so6958059qka.15 for ; Fri, 17 May 2019 14:54:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=154pK5P8ksWbhOO3IYrUyqpSsi0GfqRRa2gpHRslqUk=; b=qCg59FDZEI7vFMLxdjNevVM1V4NtcKPGHCKQp1V+IWHx7i+O/1FDLvzHtdrDRK1iNJ jE0Y9CLFCPwAzklxdet4Jn8RFf3tsMtlSP5QfuPhwWvz3+qCNvaLyKHEkYsHNqJk3OcM 3Y+PgK45zsIGrrrXYMM/oZSsD9dAbNj43kbhkj+KXqWXS69itpGRz1GeQuDjxMw5ZVzj JjjtNu8cw4RCqAP3KqBTHjj81lnwHq/aQXbtcB3ym2F+PkiJIMew3wV/NXMG4bZqBhen 7xQc0lQgxXZFMV5x9HEszHN7EM2cZMhnsOz5JtlGyPm190Ky48JWnuKlb0DCCow1VRcb ro3g== X-Gm-Message-State: APjAAAXzzRugsLJKHTofkZTQju3KOt20ajnYTRTKMgOXEMa0Owzeld9e VRJSAPsy+9UrtYj77qpN4gCIuIuRSy89Hx470h10Jqw5MLMYJQRAXgDcdamLk+2rDK11B65RGqu 8bPYHmmvnvSaw0lWy5dfoEtASD0cLdN9K2GEJWTobJOAjqlnZm3RGEgtXegJFdxEV7g== X-Received: by 2002:a37:f50f:: with SMTP id l15mr19292490qkk.343.1558130083790; Fri, 17 May 2019 14:54:43 -0700 (PDT) X-Received: by 2002:a37:f50f:: with SMTP id l15mr19292460qkk.343.1558130083216; Fri, 17 May 2019 14:54:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558130083; cv=none; d=google.com; s=arc-20160816; b=Nal4RdTHuBpsWQ8/ZQM+rKVORnKWojNDHDYjTweaSo3LZEjMQ3YVw44QzSf33k2cq2 pMx6oVjBbyQB6eX1JCs0MLrJ49+/35T0+JPN9hMZ75hO3993t08mK+BkxsnGurgb3DRm Ww318Pyw1XTe+wuE464zhdZeKShL8JBvfzesorFeB70ydihg/giH5Ui8VbvXxT1NjtRu FdPRwSljSYrJ7FQTWc/t/o5lGMFjGwSzSLUcBYfN73E0jFoqSNHWIFMoUAL+VaCryZAu hP8VdXrGn7Z7WM6af6zS+8c4qx4FJEhhf7LLItK6V6KVJquUlDXyxPBtw2t8m/BjnwnF FU1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=154pK5P8ksWbhOO3IYrUyqpSsi0GfqRRa2gpHRslqUk=; b=sZik8ol8fs3TSWSCSEu1wyYNsukkj3XsAOHuR7dVClq+yZ4GPRtjaCNB5TrNzmxAht qlSF9fPL6rNWWoLWxn9M5LU6XI9w36oyjCmJNGHnRfZCm4LB6DNQDEy51zoYnbMiX7s7 2e3y3nGQ8IQRhZz8sryYcQzN0/CCda2+plCBedzhSLBF/XDmBllXiBk9NXvgW0ZP8yrX J3SWPK5XgfARS8xqBYBrK3Coa9MlThTfaoGMtZjAjC6BaR0wEnVbIAaJ9A3P8TKn2FH2 1saKO88eHYk1yJpfFGmxFRdGeuqfWPyrFER7fN+U5S9q9o/mJpA5nzId30pDf5i1eEax G05w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=MAG7Kwt5; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n13sor13230680qtk.35.2019.05.17.14.54.43 for (Google Transport Security); Fri, 17 May 2019 14:54:43 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=MAG7Kwt5; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=154pK5P8ksWbhOO3IYrUyqpSsi0GfqRRa2gpHRslqUk=; b=MAG7Kwt56YLXgGpHlSwF6peeLCFWoe9w9yw1/4MNfZedRugiM5L9tJph4Z0qIqbysQ MQa00cOfj0ilwkOa8A4nZtTqACZDP70GVXMoFcppyNW6lFn1r8367ZG5q+tQtiGSR5mv QW2I+pdoS9IGt4WkV3+HceuXhZGG1MyYkpAD57R+twSBkhOqG+AEgv14CPl/X5joRCH/ 4Fu0gWVewSN9wTffe2BpgeBjoHryPvlUQAiajnV/yBfZ8fLU4J6TKSjKSAJu+0zFO3CM DYCy9YBFL8fXrbOMGGU+IeIiyzKMEGD2okiZG3r35R7oCY0l5vm2B9D5QUI1zTTygD27 nSLg== X-Google-Smtp-Source: APXvYqzgOTJEZbvtrTHUCsU4W9MY9QwY+/vt23cg4Q1By0cu8+zJHQ1zc5/kIujRbiw57m5j08mTsQ== X-Received: by 2002:aed:35c4:: with SMTP id d4mr50747244qte.311.1558130082975; Fri, 17 May 2019 14:54:42 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n36sm6599813qtk.9.2019.05.17.14.54.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 May 2019 14:54:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v6 1/3] device-dax: fix memory and resource leak if hotplug fails Date: Fri, 17 May 2019 17:54:36 -0400 Message-Id: <20190517215438.6487-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190517215438.6487-1-pasha.tatashin@soleen.com> References: <20190517215438.6487-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When add_memory() function fails, the resource and the memory should be freed. Fixes: c221c0b0308f ("device-dax: "Hotplug" persistent memory for use like normal RAM") Signed-off-by: Pavel Tatashin Reviewed-by: Dave Hansen --- drivers/dax/kmem.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index a02318c6d28a..4c0131857133 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -66,8 +66,11 @@ int dev_dax_kmem_probe(struct device *dev) new_res->name = dev_name(dev); rc = add_memory(numa_node, new_res->start, resource_size(new_res)); - if (rc) + if (rc) { + release_resource(new_res); + kfree(new_res); return rc; + } return 0; } From patchwork Fri May 17 21:54:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 10948697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 495EE14C0 for ; Fri, 17 May 2019 21:54:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 347FB2849B for ; Fri, 17 May 2019 21:54:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2795C2849D; Fri, 17 May 2019 21:54:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 406E028492 for ; Fri, 17 May 2019 21:54:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0B2A6B0006; Fri, 17 May 2019 17:54:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A70206B0008; Fri, 17 May 2019 17:54:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 898C26B000A; Fri, 17 May 2019 17:54:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by kanga.kvack.org (Postfix) with ESMTP id 651026B0006 for ; Fri, 17 May 2019 17:54:46 -0400 (EDT) Received: by mail-qt1-f199.google.com with SMTP id l20so7766849qtq.21 for ; Fri, 17 May 2019 14:54:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=oZtfX1EsFJjeahXRqy4pGct5cturWJyiPbutyS49o0o=; b=W4OEKcobouoao6QpDqZCz26ukJ7pnXKgKJj4xLHXfQR3n7YLGpdx42xBpZAu5oXat8 9EuthqsJjun/1b5jDdTLL49Ksw3HXR0gd92mydWpNsGM8f4Adj2lKsvaaFJ6wa296bNO XAxT3bFhvvHQ4WKaU3cMt1JuMw0W8lPmFuIRcU/dXC5gZ7v48JSsLaMF8avFO+gbt6em Nkjm6lP3/TcmRAxmm7Ko2KxgULObnj70cwXWrDRel5Rrt7v3V7acUoYBPC4fjT76QhRa WG9F+AYerH8bCU+STnggJUqEM7YKC9BscWuHktadU+1Q4ebpmiS/XAAHmPNN9pGBU0fC rFMg== X-Gm-Message-State: APjAAAUzN2+NNsP0KPLIWTS28WOlN86xOE6yCSrr6kvX/1PhTlzIgh2o 5Id7BjInrcJv5zp5vfrXrIGqIyuVFhUInv3ITsAS+tjMYUntwLhD8P5w0WBCXx8thLeDavN5doW 5a8P8qjjJ4YLQBk3CY8iv5FSV50pm5GTaihTaPgiwvmfKJKaU3eSk2Po6rCycLLo/cg== X-Received: by 2002:ac8:33aa:: with SMTP id c39mr49832502qtb.258.1558130085705; Fri, 17 May 2019 14:54:45 -0700 (PDT) X-Received: by 2002:ac8:33aa:: with SMTP id c39mr49832467qtb.258.1558130084915; Fri, 17 May 2019 14:54:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558130084; cv=none; d=google.com; s=arc-20160816; b=lhj5Y36MT7s082cSjxfQ+FL4SiQ8fD+uG2ymbw/xLTJtCL7yY6DBLLYyIZA8fQ0C6D lByHlUeLsn3APRB+Nvksap8xu/heePBCDb30iBrjvW9WeCsTqDJqZUlxHPYi7ZUIDX9w SJYDYmhQPWgOEGUL2o2kOeL4sVYJW4APG+bvQntthpl6BTDfa+4nWe11kOOagrbN/Hq5 y18IQwUXXyLjnAEsNM146Vz/7hT6TW8OLY0fepHW5VPOB248baTZFAbExyM/jCOeaAiW 30ZgKv1iGQhUQkp1rh+Sueh+MoM1ITlDrZELOqNDnvRNRIYeIYEuUM9r1YwsWdkGBxqW Sh7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=oZtfX1EsFJjeahXRqy4pGct5cturWJyiPbutyS49o0o=; b=QkcQVaxIv1hyJ/b/c3js0w3uB0e0WLIlMxswBSrYgjD2bA4aHtD1Z9QALbGcFZraXL wuF++JFGpK6o37UF1BlbeVpB0/HZAtjYUKfMqzKfrusvH4ebwJKS5o2OQ2HhmRhN5vRk uA9hFpRcGDAtK4WkjlB4TQ5eKUULb1/41lM9muKa+v5bCLDSEvHuMthR/tXZHfATYTJw 8r37x7c+ooCIdtFAqk+AzzYaB9mcD8JWfQYtoQ5vZJIEf3W35pRwcx96Dw4VQ1oQjoL7 7Ft1LCibcjIvoZUX/kWwn1BYtXPMX3oDeBl0dqggILKycuA0fT4yWcyHG63qzOsqB1M9 UxUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b="OCk6/zZy"; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i59sor7872101qva.50.2019.05.17.14.54.44 for (Google Transport Security); Fri, 17 May 2019 14:54:44 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b="OCk6/zZy"; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=oZtfX1EsFJjeahXRqy4pGct5cturWJyiPbutyS49o0o=; b=OCk6/zZyF14XcPyyMATB4Ym+LlfbjeEqQItqmdENfhxAIL7cHoM94ugihG5R1YlUwP jyOovQljik1bZ9u1Q17gwNSzs//6T0sHHsn2nZn6oduYrc1RwPHPU82m9/ULK5W2vkrF 4+ZOr0KMFsimF91CAv01CSFgbFsxosShzFArqF02FMtG9sDp0Orljcvp+xse0RqkWqKq K3kiIonSoJQg0yVwJ2g6tZMJKiovC3dzhCAAP42XaahkEEEEZ8Qb3VLkPAin2xpWGCFO 1HB+ik713/SslbvcIJeCr1EhRo4dQwxqJXMBiS6l96Ce9QQeD79P47HlY+Bi+I+kUnbX KdxQ== X-Google-Smtp-Source: APXvYqw1HhlmDQ1zjvj92mE60iFZ0AD+1AWuI0vYk2b6i/mKmMR7uORFYBtfk6du8rP1a68lCkbwRw== X-Received: by 2002:a0c:9ac1:: with SMTP id k1mr47616206qvf.36.1558130084638; Fri, 17 May 2019 14:54:44 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n36sm6599813qtk.9.2019.05.17.14.54.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 May 2019 14:54:43 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v6 2/3] mm/hotplug: make remove_memory() interface useable Date: Fri, 17 May 2019 17:54:37 -0400 Message-Id: <20190517215438.6487-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190517215438.6487-1-pasha.tatashin@soleen.com> References: <20190517215438.6487-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP As of right now remove_memory() interface is inherently broken. It tries to remove memory but panics if some memory is not offline. The problem is that it is impossible to ensure that all memory blocks are offline as this function also takes lock_device_hotplug that is required to change memory state via sysfs. So, between calling this function and offlining all memory blocks there is always a window when lock_device_hotplug is released, and therefore, there is always a chance for a panic during this window. Make this interface to return an error if memory removal fails. This way it is safe to call this function without panicking machine, and also makes it symmetric to add_memory() which already returns an error. Signed-off-by: Pavel Tatashin Reviewed-by: David Hildenbrand Acked-by: Michal Hocko --- include/linux/memory_hotplug.h | 8 +++-- mm/memory_hotplug.c | 64 +++++++++++++++++++++++----------- 2 files changed, 49 insertions(+), 23 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index ae892eef8b82..988fde33cd7f 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -324,7 +324,7 @@ static inline void pgdat_resize_init(struct pglist_data *pgdat) {} extern bool is_mem_section_removable(unsigned long pfn, unsigned long nr_pages); extern void try_offline_node(int nid); extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages); -extern void remove_memory(int nid, u64 start, u64 size); +extern int remove_memory(int nid, u64 start, u64 size); extern void __remove_memory(int nid, u64 start, u64 size); #else @@ -341,7 +341,11 @@ static inline int offline_pages(unsigned long start_pfn, unsigned long nr_pages) return -EINVAL; } -static inline void remove_memory(int nid, u64 start, u64 size) {} +static inline int remove_memory(int nid, u64 start, u64 size) +{ + return -EBUSY; +} + static inline void __remove_memory(int nid, u64 start, u64 size) {} #endif /* CONFIG_MEMORY_HOTREMOVE */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 328878b6799d..ace2cc614da4 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1735,9 +1735,10 @@ static int check_memblock_offlined_cb(struct memory_block *mem, void *arg) endpa = PFN_PHYS(section_nr_to_pfn(mem->end_section_nr + 1))-1; pr_warn("removing memory fails, because memory [%pa-%pa] is onlined\n", &beginpa, &endpa); - } - return ret; + return -EBUSY; + } + return 0; } static int check_cpu_on_node(pg_data_t *pgdat) @@ -1820,19 +1821,9 @@ static void __release_memory_resource(resource_size_t start, } } -/** - * remove_memory - * @nid: the node ID - * @start: physical address of the region to remove - * @size: size of the region to remove - * - * NOTE: The caller must call lock_device_hotplug() to serialize hotplug - * and online/offline operations before this call, as required by - * try_offline_node(). - */ -void __ref __remove_memory(int nid, u64 start, u64 size) +static int __ref try_remove_memory(int nid, u64 start, u64 size) { - int ret; + int rc = 0; BUG_ON(check_hotplug_memory_range(start, size)); @@ -1840,13 +1831,13 @@ void __ref __remove_memory(int nid, u64 start, u64 size) /* * All memory blocks must be offlined before removing memory. Check - * whether all memory blocks in question are offline and trigger a BUG() + * whether all memory blocks in question are offline and return error * if this is not the case. */ - ret = walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1), NULL, - check_memblock_offlined_cb); - if (ret) - BUG(); + rc = walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1), NULL, + check_memblock_offlined_cb); + if (rc) + goto done; /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); @@ -1858,14 +1849,45 @@ void __ref __remove_memory(int nid, u64 start, u64 size) try_offline_node(nid); +done: mem_hotplug_done(); + return rc; } -void remove_memory(int nid, u64 start, u64 size) +/** + * remove_memory + * @nid: the node ID + * @start: physical address of the region to remove + * @size: size of the region to remove + * + * NOTE: The caller must call lock_device_hotplug() to serialize hotplug + * and online/offline operations before this call, as required by + * try_offline_node(). + */ +void __remove_memory(int nid, u64 start, u64 size) +{ + + /* + * trigger BUG() is some memory is not offlined prior to calling this + * function + */ + if (try_remove_memory(nid, start, size)) + BUG(); +} + +/* + * Remove memory if every memory block is offline, otherwise return -EBUSY is + * some memory is not offline + */ +int remove_memory(int nid, u64 start, u64 size) { + int rc; + lock_device_hotplug(); - __remove_memory(nid, start, size); + rc = try_remove_memory(nid, start, size); unlock_device_hotplug(); + + return rc; } EXPORT_SYMBOL_GPL(remove_memory); #endif /* CONFIG_MEMORY_HOTREMOVE */ From patchwork Fri May 17 21:54:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 10948701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38A4B76 for ; Fri, 17 May 2019 21:54:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2887F28497 for ; Fri, 17 May 2019 21:54:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C44D2849D; Fri, 17 May 2019 21:54:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D6AD2849B for ; Fri, 17 May 2019 21:54:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB7266B0008; Fri, 17 May 2019 17:54:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C41F16B000A; Fri, 17 May 2019 17:54:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABB456B000C; Fri, 17 May 2019 17:54:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 84B1C6B0008 for ; Fri, 17 May 2019 17:54:47 -0400 (EDT) Received: by mail-qt1-f197.google.com with SMTP id w34so7775869qtc.16 for ; Fri, 17 May 2019 14:54:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=8p3wsS4SMTLuQgzuXCRhTKHP61mCg8wFpFeHB9x4QIQ=; b=succA8z45ZOsiesf04L+hXIrQdIQDKYpeRjeM5YnMrqWUZOvLN8mriKOa1kjEDaeGl gPrXpKd0nx9HM7Nm7eb87Zm0qks6Q/BdGOKDwdtG5lJriA8rEILO7vDNjYsxQ4MbkXDL civl7TBzbHZXwfvqaTJpc4rpBZssR2QeCz5v81fymbx9FfEnyWu67burKk483B9I7Q7K d3y0MtRk5OjXfPZSCg4qpqLQO0aD5Ff5BYTZ15HwXYMF6/7I6VdwoEKudXqUDOW34ts/ wAbcWC/AOR0NYlJuQwhhk8nik9VuklscX16pHznvo7RIw9HUyE0wQtEL+LM2JKbseCB4 2uUA== X-Gm-Message-State: APjAAAUvd76O3k4h2G+X3CLxMzSceieCMiQwY72v/oiGpw7Lwltib3Rw RyAb5GpaNF/LiN8wcmKnWUO3jvBiB2rriMoZ/faGhfDwHIIXOywb//VTQrf1QNF2rtDq3DZH2aA ZrJyia6GqVhpqLOiseT13kuXbyF+xJzl2tUZK5p6G6oiyXeI/pyFi2yDcTlS0Av9P3A== X-Received: by 2002:ac8:3894:: with SMTP id f20mr49876074qtc.84.1558130087304; Fri, 17 May 2019 14:54:47 -0700 (PDT) X-Received: by 2002:ac8:3894:: with SMTP id f20mr49876037qtc.84.1558130086533; Fri, 17 May 2019 14:54:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558130086; cv=none; d=google.com; s=arc-20160816; b=XGtVsA1sEf3RvGdSEwh1wjUZPBYiw6l3plzrgj9oqrXZi2ZdYMZP3SN42x1PxA1MxL 4CbVnQVrgPmBCcIoQw6FtL/HbwmsK7JEA+RI8W2w9w80KYpnDMbF9Fc2MbrCfsTIyQvN QY6sg2vxyXTcs9T/eu+zTT6EHB4T4C/gLNJjpEfgoYxRz5DkCAtXB5QO5ez9VEggNJdC 48l/0XumfKnfRJT/W/Ew6bkiLOyLMELQMoPuos1Qnz8/uPR1y+bNahe1NWlYC6PK+fIg PJZIsEdElzpvyiFfAYqFk2bBl/IbqTEzCJutoyXjmokpH38cdbdV698uqxnvk4oYH6qi yncg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:dkim-signature; bh=8p3wsS4SMTLuQgzuXCRhTKHP61mCg8wFpFeHB9x4QIQ=; b=DrOgQCcStcD62chUVodbDkWd24zdBGwjVV8KPRD1+x+oDMya9jx+0w2yf+JUyFbO6N q/cy3LGv4C/Ct/fhdhaDCCHTvHg4EpK/YvGJdKRXKoERN7Xt13A2gg3w4rrZ7+P3QBJQ degP+xa3iiT2PT8ksCm5IdaMmnALi03TtGA1uUApCoxmbTB0SnCbaTnbI5HMmeJm3u69 CYAx4+9BSK84fGFHipJjHOJeAjdemYv+R53/Wu9RbV1RRxora6s2wewrqtDUTzFkIfnF SsnA1DMp01d/le9OicryIxSq1SRrWBllH3AQ8WIBooe0v05RGr6HtaoJMf/DlQi05l0T Oeog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=dxODNsdP; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 31sor12278284qta.70.2019.05.17.14.54.46 for (Google Transport Security); Fri, 17 May 2019 14:54:46 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=dxODNsdP; spf=pass (google.com: domain of pasha.tatashin@soleen.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8p3wsS4SMTLuQgzuXCRhTKHP61mCg8wFpFeHB9x4QIQ=; b=dxODNsdP4xdmGLZgD3wGpBVlOvBLAVbSmuov53M2mG0sFDyfGFIGchYc5sW1e4tdjh E4c/jto60Apec/FyJ74ps0C/lZTwCHSVCGEu/P9hjQyCnKCnrzUieZ4VeCLSkNoimqU6 vTLoXPIvlC8WqNcEBLsvkQh9YYqP+L4H5r2GbwuKQ+kEujoYI57NpN8mOTCWgmfsr7Aw Spg565JQb5vi1JEv/tGDOgJGc//CmfQNLmu7NjApk78ZskurVXGDnNCuawRCwtqnE6X6 JljgK2qNEAxVFEPGagOdIZlQ2j6+4SDQF/UVFwdv64lQUFlavVTkaDTXSW2ZYOAZXFsD rTLg== X-Google-Smtp-Source: APXvYqwFk9kJSaOoaMhXOv97FFwjvOjt9oKxB+WBIkvvLLcYVDlAeyLAX4m1Cd2gwueE6vntUoBbxw== X-Received: by 2002:ac8:3696:: with SMTP id a22mr50806933qtc.296.1558130086264; Fri, 17 May 2019 14:54:46 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n36sm6599813qtk.9.2019.05.17.14.54.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 May 2019 14:54:45 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, akpm@linux-foundation.org, mhocko@suse.com, dave.hansen@linux.intel.com, dan.j.williams@intel.com, keith.busch@intel.com, vishal.l.verma@intel.com, dave.jiang@intel.com, zwisler@kernel.org, thomas.lendacky@amd.com, ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de, bhelgaas@google.com, baiyaowei@cmss.chinamobile.com, tiwai@suse.de, jglisse@redhat.com, david@redhat.com Subject: [v6 3/3] device-dax: "Hotremove" persistent memory that is used like normal RAM Date: Fri, 17 May 2019 17:54:38 -0400 Message-Id: <20190517215438.6487-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190517215438.6487-1-pasha.tatashin@soleen.com> References: <20190517215438.6487-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It is now allowed to use persistent memory like a regular RAM, but currently there is no way to remove this memory until machine is rebooted. This work expands the functionality to also allows hotremoving previously hotplugged persistent memory, and recover the device for use for other purposes. To hotremove persistent memory, the management software must first offline all memory blocks of dax region, and than unbind it from device-dax/kmem driver. So, operations should look like this: echo offline > /sys/devices/system/memory/memoryN/state ... echo dax0.0 > /sys/bus/dax/drivers/kmem/unbind Note: if unbind is done without offlining memory beforehand, it won't be possible to do dax0.0 hotremove, and dax's memory is going to be part of System RAM until reboot. Signed-off-by: Pavel Tatashin Reviewed-by: David Hildenbrand --- drivers/dax/dax-private.h | 2 ++ drivers/dax/kmem.c | 41 +++++++++++++++++++++++++++++++++++---- 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/drivers/dax/dax-private.h b/drivers/dax/dax-private.h index a45612148ca0..999aaf3a29b3 100644 --- a/drivers/dax/dax-private.h +++ b/drivers/dax/dax-private.h @@ -53,6 +53,7 @@ struct dax_region { * @pgmap - pgmap for memmap setup / lifetime (driver owned) * @ref: pgmap reference count (driver owned) * @cmp: @ref final put completion (driver owned) + * @dax_mem_res: physical address range of hotadded DAX memory */ struct dev_dax { struct dax_region *region; @@ -62,6 +63,7 @@ struct dev_dax { struct dev_pagemap pgmap; struct percpu_ref ref; struct completion cmp; + struct resource *dax_kmem_res; }; static inline struct dev_dax *to_dev_dax(struct device *dev) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 4c0131857133..3d0a7e702c94 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -71,21 +71,54 @@ int dev_dax_kmem_probe(struct device *dev) kfree(new_res); return rc; } + dev_dax->dax_kmem_res = new_res; return 0; } +#ifdef CONFIG_MEMORY_HOTREMOVE +static int dev_dax_kmem_remove(struct device *dev) +{ + struct dev_dax *dev_dax = to_dev_dax(dev); + struct resource *res = dev_dax->dax_kmem_res; + resource_size_t kmem_start = res->start; + resource_size_t kmem_size = resource_size(res); + int rc; + + /* + * We have one shot for removing memory, if some memory blocks were not + * offline prior to calling this function remove_memory() will fail, and + * there is no way to hotremove this memory until reboot because device + * unbind will succeed even if we return failure. + */ + rc = remove_memory(dev_dax->target_node, kmem_start, kmem_size); + if (rc) { + dev_err(dev, + "DAX region %pR cannot be hotremoved until the next reboot\n", + res); + return rc; + } + + /* Release and free dax resources */ + release_resource(res); + kfree(res); + dev_dax->dax_kmem_res = NULL; + + return 0; +} +#else static int dev_dax_kmem_remove(struct device *dev) { /* - * Purposely leak the request_mem_region() for the device-dax - * range and return '0' to ->remove() attempts. The removal of - * the device from the driver always succeeds, but the region - * is permanently pinned as reserved by the unreleased + * Without hotremove purposely leak the request_mem_region() for the + * device-dax range and return '0' to ->remove() attempts. The removal + * of the device from the driver always succeeds, but the region is + * permanently pinned as reserved by the unreleased * request_mem_region(). */ return 0; } +#endif /* CONFIG_MEMORY_HOTREMOVE */ static struct dax_device_driver device_dax_kmem_driver = { .drv = {