From patchwork Tue Oct 1 09:45:31 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Zhang X-Patchwork-Id: 2969051 Return-Path: X-Original-To: patchwork-linux-acpi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 7D359BFF0B for ; Tue, 1 Oct 2013 09:46:35 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 32F5D203DC for ; Tue, 1 Oct 2013 09:46:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DCFA4203DA for ; Tue, 1 Oct 2013 09:46:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752394Ab3JAJqb (ORCPT ); Tue, 1 Oct 2013 05:46:31 -0400 Received: from cn.fujitsu.com ([222.73.24.84]:56299 "EHLO song.cn.fujitsu.com" rhost-flags-OK-FAIL-OK-OK) by vger.kernel.org with ESMTP id S1751477Ab3JAJqa (ORCPT ); Tue, 1 Oct 2013 05:46:30 -0400 X-IronPort-AV: E=Sophos;i="4.90,1013,1371052800"; d="scan'208";a="8662825" Received: from unknown (HELO tang.cn.fujitsu.com) ([10.167.250.3]) by song.cn.fujitsu.com with ESMTP; 01 Oct 2013 17:43:12 +0800 Received: from fnstmail02.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id r919kOsB013702; Tue, 1 Oct 2013 17:46:24 +0800 Received: from [10.167.226.121] ([10.167.226.121]) by fnstmail02.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.3) with ESMTP id 2013100117443165-1917092 ; Tue, 1 Oct 2013 17:44:31 +0800 Message-ID: <524A99BB.5040000@cn.fujitsu.com> Date: Tue, 01 Oct 2013 17:45:31 +0800 From: Zhang Yanfei User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130514 Thunderbird/17.0.6 MIME-Version: 1.0 To: robert.moore@intel.com, lv.zheng@intel.com, "Rafael J . Wysocki" , Len Brown , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Andrew Morton , Tejun Heo , Thomas Renninger , Yinghai Lu , Jiang Liu , Wen Congyang , Lai Jiangshan , Yasuaki Ishimatsu , Taku Izumi , Mel Gorman , Minchan Kim , "mina86@mina86.com" , gong.chen@linux.intel.com, vasilis.liaskovitis@profitbricks.com, Rik van Riel , prarit@redhat.com, Toshi Kani CC: Zhang Yanfei , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , Linux MM , linux-acpi@vger.kernel.org, Tang Chen , imtangchen@gmail.com, Zhang Yanfei Subject: [PATCH -mm 1/8] x86: get pg_data_t's memory from other node References: <524A991D.3050005@cn.fujitsu.com> In-Reply-To: <524A991D.3050005@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.3|September 15, 2011) at 2013/10/01 17:44:31, Serialize by Router on mailserver/fnst(Release 8.5.3|September 15, 2011) at 2013/10/01 17:44:34, Serialize complete at 2013/10/01 17:44:34 Sender: linux-acpi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,KHOP_BIG_TO_CC, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Yasuaki Ishimatsu If system can create movable node which all memory of the node is allocated as ZONE_MOVABLE, setup_node_data() cannot allocate memory for the node's pg_data_t. So, invoke memblock_alloc_nid(...MAX_NUMNODES) again to retry when the first allocation fails. Otherwise, the system could failed to boot. (We don't use memblock_alloc_try_nid() to retry because in this function, if the allocation fails, it will panic the system.) The node_data could be on hotpluggable node. And so could pagetable and vmemmap. But for now, doing so will break memory hot-remove path. A node could have several memory devices. And the device who holds node data should be hot-removed in the last place. But in NUMA level, we don't know which memory_block (/sys/devices/system/node/nodeX/memoryXXX) belongs to which memory device. We only have node. So we can only do node hotplug. But in virtualization, developers are now developing memory hotplug in qemu, which support a single memory device hotplug. So a whole node hotplug will not satisfy virtualization users. So at last, we concluded that we'd better do memory hotplug and local node things (local node node data, pagetable, vmemmap, ...) in two steps. Please refer to https://lkml.org/lkml/2013/6/19/73 For now, we put node_data of movable node to another node, and then improve it in the future. Signed-off-by: Yasuaki Ishimatsu Signed-off-by: Lai Jiangshan Signed-off-by: Tang Chen Signed-off-by: Jiang Liu Signed-off-by: Tang Chen Signed-off-by: Zhang Yanfei Reviewed-by: Wanpeng Li Acked-by: Toshi Kani --- arch/x86/mm/numa.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 8bf93ba..1f68d0e 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -211,9 +211,14 @@ static void __init setup_node_data(int nid, u64 start, u64 end) */ nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, nid); if (!nd_pa) { - pr_err("Cannot find %zu bytes in node %d\n", - nd_size, nid); - return; + pr_warn("Cannot find %zu bytes in node %d, so try other nodes", + nd_size, nid); + nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, + MAX_NUMNODES); + if (!nd_pa) { + pr_err("Cannot find %zu bytes in any node\n", nd_size); + return; + } } nd = __va(nd_pa);