From patchwork Wed Sep 25 02:57:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 11159939 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D200F14E5 for ; Wed, 25 Sep 2019 02:58:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 98A532146E for ; Wed, 25 Sep 2019 02:58:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98A532146E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A20426B000A; Tue, 24 Sep 2019 22:58:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9CF0A6B000C; Tue, 24 Sep 2019 22:58:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BDB76B000D; Tue, 24 Sep 2019 22:58:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0067.hostedemail.com [216.40.44.67]) by kanga.kvack.org (Postfix) with ESMTP id 6AE026B000A for ; Tue, 24 Sep 2019 22:58:01 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 05F6F181AC9B6 for ; Wed, 25 Sep 2019 02:58:01 +0000 (UTC) X-FDA: 75971933562.04.food03_28c7c4b83f704 X-Spam-Summary: 50,0,0,dbf32635ea5926ea,d41d8cd98f00b204,anshuman.khandual@arm.com,::linux-kernel@vger.kernel.org:akpm@linux-foundation.org:anshuman.khandual@arm.com:osalvador@suse.de:mhocko@suse.com:david@redhat.com:pasha.tatashin@soleen.com:dan.j.williams@intel.com,RULES_HIT:41:355:379:541:560:800:960:965:966:967:968:973:988:989:1260:1261:1345:1431:1437:1534:1543:1711:1730:1747:1777:1792:1963:2196:2198:2199:2200:2393:2525:2561:2566:2682:2685:2693:2731:2859:2899:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4390:4395:4470:5007:6261:7875:8603:8634:8985:9025:9389:10004:10128:11026:11232:11658:11914:12043:12297:12438:12555:12895:12986:13095:13215:13229:13255:13972:14096:14181:14394:14721:21080:21212:21433:21451:21627:21740:21788:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Cu stom_rul X-HE-Tag: food03_28c7c4b83f704 X-Filterd-Recvd-Size: 4249 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Sep 2019 02:58:00 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E9CBA337; Tue, 24 Sep 2019 19:57:58 -0700 (PDT) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.41.120]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9FA8D3F694; Tue, 24 Sep 2019 19:57:55 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org Cc: Anshuman Khandual , Oscar Salvador , Michal Hocko , David Hildenbrand , Pavel Tatashin , Dan Williams Subject: [PATCH V2] mm/hotplug: Reorder memblock_[free|remove]() calls in try_remove_memory() Date: Wed, 25 Sep 2019 08:27:53 +0530 Message-Id: <1569380273-7708-1-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently during memory hot add procedure, memory gets into memblock before calling arch_add_memory() which creates it's linear mapping. add_memory_resource() { .................. memblock_add_node() .................. arch_add_memory() .................. } But during memory hot remove procedure, removal from memblock happens first before it's linear mapping gets teared down with arch_remove_memory() which is not consistent. Resource removal should happen in reverse order as they were added. However this does not pose any problem for now, unless there is an assumption regarding linear mapping. One example was a subtle failure on arm64 platform [1]. Though this has now found a different solution. try_remove_memory() { .................. memblock_free() memblock_remove() .................. arch_remove_memory() .................. } This changes the sequence of resource removal including memblock and linear mapping tear down during memory hot remove which will now be the reverse order in which they were added during memory hot add. The changed removal order looks like the following. try_remove_memory() { .................. arch_remove_memory() .................. memblock_free() memblock_remove() .................. } [1] https://patchwork.kernel.org/patch/11127623/ Cc: Andrew Morton Cc: Oscar Salvador Cc: Michal Hocko Cc: David Hildenbrand Cc: Pavel Tatashin Cc: Dan Williams Signed-off-by: Anshuman Khandual Acked-by: Michal Hocko Reviewed-by: David Hildenbrand --- Changes in V2: - Changed the commit message as per Michal and David Changed in V1: https://patchwork.kernel.org/patch/11146361/ Original patch https://lkml.org/lkml/2019/9/3/327 Memory hot remove now works on arm64 without this because a recent commit 60bb462fc7ad ("drivers/base/node.c: simplify unregister_memory_block_under_nodes()"). David mentioned that re-ordering should still make sense for consistency purpose (removing stuff in the reverse order they were added). This patch is now detached from arm64 hot-remove series. https://lkml.org/lkml/2019/9/3/326 mm/memory_hotplug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 49f7bf91c25a..4f7d426a84d0 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1763,13 +1763,13 @@ static int __ref try_remove_memory(int nid, u64 start, u64 size) /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); - memblock_free(start, size); - memblock_remove(start, size); /* remove memory block devices before removing memory */ remove_memory_block_devices(start, size); arch_remove_memory(nid, start, size, NULL); + memblock_free(start, size); + memblock_remove(start, size); __release_memory_resource(start, size); try_offline_node(nid);