From patchwork Mon Jul 16 17:44:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 10527375 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C0C8F600D0 for ; Mon, 16 Jul 2018 17:45:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAE9828421 for ; Mon, 16 Jul 2018 17:45:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F27F28DEF; Mon, 16 Jul 2018 17:45:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0521428421 for ; Mon, 16 Jul 2018 17:45:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 703376B0266; Mon, 16 Jul 2018 13:45:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6B6BC6B0269; Mon, 16 Jul 2018 13:45:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 557056B026A; Mon, 16 Jul 2018 13:45:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-vk0-f72.google.com (mail-vk0-f72.google.com [209.85.213.72]) by kanga.kvack.org (Postfix) with ESMTP id 286376B0266 for ; Mon, 16 Jul 2018 13:45:14 -0400 (EDT) Received: by mail-vk0-f72.google.com with SMTP id 11-v6so14246346vko.21 for ; Mon, 16 Jul 2018 10:45:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:subject:date:message-id :in-reply-to:references; bh=HshFKhNMdbDY+R2VFmveccDHbY+ei3+gykokCKNUd+k=; b=GWvq7SGrNXyjSyfnW8L43OEmtbUWg/Z5cOy0ovlG7wMZxav2EqmB954O19Rqb6riaV pVg4dX+bppsD/ekTozloYMd2JcyaLoFEl0iVXKU7RLlsVvKZUekEjc0POP/Nkml70UXQ xeItzlXuPL1pEtmX+2+SGPJhYcdvnvFL5T9/VKi3+3elbwbnNAaykALshrtbx2nWO1s8 ZnrHWQFD4ysdU74A4gDQprHcvptmP7bZFrSxi+7J5tLLzj+IYjvoiqVgC63S3atO7tJT wsO00AZkZYwpPCy+sbPxo5VeZxgJcbtUx6Om5GY9TyCbwBPmJeVFdSxXoI7q3P7fWiiP It2w== X-Gm-Message-State: AOUpUlH4Q5MSapwRd4FjN5F/pZfeDB6H9NiAZRpGsF0lygMyVTQUJXPR gH20aLkHtjJdIDFon6VyDigZ08y2y5+U2dSURNd1Uv/hVAIwjodiwYRT3MR4pw53tZBkoxqgnGC y8NSW2lzNbMYRdN7OwlNzz1HwGxBvnc/tj5z3QBMUdP3B3JhadXWqp0JDpd4e6930vA== X-Received: by 2002:a1f:f44f:: with SMTP id s76-v6mr9855708vkh.125.1531763113094; Mon, 16 Jul 2018 10:45:13 -0700 (PDT) X-Google-Smtp-Source: AAOMgpduRTecnPsgAeVGJ/FFByfqlk35GIIQ1B6PSMyFjjO4YcKRXFrnkjPlSToilE8CUeCc3t8+ X-Received: by 2002:a1f:f44f:: with SMTP id s76-v6mr9855673vkh.125.1531763112339; Mon, 16 Jul 2018 10:45:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531763112; cv=none; d=google.com; s=arc-20160816; b=xoU4cQhyMDjYOZcNqxcxrUyxcBuVrtUMe9IDtK8QtrJ3Ee008QM9tv7huQB6W0iCeX 6RbA//zfwx7Gb2lYNHEzPzCAEYKeR/TnooI46fwiYGwaOhNyC6K87A3WJZt+/i+ZJQH1 jCHrBnm2KXfS7x/x+VLZvMkDz77wCb2cQHQBeo/FcFDsfpLtW14i9oiILYoDfb6BYB4D MpGMwrgUXu6WVOAlBuRinWzMOyKsx2MfIkf5dBNTWp9judYGEE3te164XKjzhcBQR5ip svsab4mEIvBkCpXObm2uT5Z5O0KHZIEkCx7JybbnWnIYgBz2Rb/mf94Bb7/qToZmZ1N6 ik5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:to:from :dkim-signature:arc-authentication-results; bh=HshFKhNMdbDY+R2VFmveccDHbY+ei3+gykokCKNUd+k=; b=U8AJi+Yoj6kCzi4kp0gGv1y8ab3HJrSUSk4upugtBitX+LKiLZVd49sqn49GFnhGXZ P7/zhL+74cy8hG9Z0aczdnbawEMOtv9qJBgi6Occi8CVt/IsBtJUzd/Be+CgpVfSS9s9 Tw5wmczfXQRe62G0ChC6dceNbgdqaL9KdVwXYKQ7lCEP3NWkdWz+YtUg260LFiEjhX3f vEFRYp75cnbQ4ueqW6YQI5TPGPjZM29pWvBmA1sulUQUD0fPMklX57b6F3Ojmjq3ubK1 stdwJdHvBlOcBngrZ4jtFSUZkEtlOgqWTGWCoyzTgdb3ij1rW6Ko8MzZRJGpM9sIl4Ke 70Tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="tHMS/GdX"; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id l64-v6si466063ual.60.2018.07.16.10.45.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Jul 2018 10:45:12 -0700 (PDT) Received-SPF: pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="tHMS/GdX"; spf=pass (google.com: domain of pasha.tatashin@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=pasha.tatashin@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w6GHi06b163561; Mon, 16 Jul 2018 17:45:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=HshFKhNMdbDY+R2VFmveccDHbY+ei3+gykokCKNUd+k=; b=tHMS/GdXbvuKdXJaxMIZB3+lLFQIO0Rg7Ba+C5l1kOvSYVCJ9W9CBFW63mdbpZXhG6li Z5ZWRQX0CNLGmBNABFeiDAaEPabF2gsUZRD7FiidB7slIuyLBxMfGWRr5Sv/CVYL9hvA diwZuDvhufOMdP271lNOWFc4GUf4HG5QCOxumJI+zP3XAK/b/5d1ZxO1EK6oaXhHyW81 lEmWEWUSqYKU+z4sDFdkn0dkakQNmATpA55NtubaNP+4HCKTTke3xXKu2+2unqIBWUz5 JzwpyZflWULCuCkLeDwb2HAWzOMncD9t+j5+XKCoNU3jMafepj/904hyQeCZ+CLLT7ov og== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2120.oracle.com with ESMTP id 2k7a3jnh18-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Jul 2018 17:45:03 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w6GHj2XT017269 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Jul 2018 17:45:03 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w6GHj29f005977; Mon, 16 Jul 2018 17:45:02 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 16 Jul 2018 10:45:02 -0700 From: Pavel Tatashin To: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, mhocko@suse.com, linux-mm@kvack.org, dan.j.williams@intel.com, jack@suse.cz, jglisse@redhat.com, jrdr.linux@gmail.com, bhe@redhat.com, gregkh@linuxfoundation.org, vbabka@suse.cz, richard.weiyang@gmail.com, dave.hansen@intel.com, rientjes@google.com, mingo@kernel.org, osalvador@techadventures.net, pasha.tatashin@oracle.com, abdhalee@linux.vnet.ibm.com, mpe@ellerman.id.au Subject: [PATCH v6 4/5] mm/sparse: add new sparse_init_nid() and sparse_init() Date: Mon, 16 Jul 2018 13:44:46 -0400 Message-Id: <20180716174447.14529-5-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180716174447.14529-1-pasha.tatashin@oracle.com> References: <20180716174447.14529-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8956 signatures=668706 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807160202 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP sparse_init() requires to temporary allocate two large buffers: usemap_map and map_map. Baoquan He has identified that these buffers are so large that Linux is not bootable on small memory machines, such as a kdump boot. The buffers are especially large when CONFIG_X86_5LEVEL is set, as they are scaled to the maximum physical memory size. Baoquan provided a fix, which reduces these sizes of these buffers, but it is much better to get rid of them entirely. Add a new way to initialize sparse memory: sparse_init_nid(), which only operates within one memory node, and thus allocates memory either in large contiguous block or allocates section by section. This eliminates the need for use of temporary buffers. For simplified bisecting and review temporarly call sparse_init() new_sparse_init(), the new interface is going to be enabled as well as old code removed in the next patch. Signed-off-by: Pavel Tatashin Reviewed-by: Oscar Salvador --- mm/sparse.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/mm/sparse.c b/mm/sparse.c index 20ca292d8f11..248d5d7bbf55 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -200,6 +200,11 @@ static inline int next_present_section_nr(int section_nr) (section_nr <= __highest_present_section_nr)); \ section_nr = next_present_section_nr(section_nr)) +static inline unsigned long first_present_section_nr(void) +{ + return next_present_section_nr(-1); +} + /* * Record how many memory sections are marked as present * during system bootup. @@ -668,6 +673,86 @@ void __init sparse_init(void) memblock_free_early(__pa(usemap_map), size); } +/* + * Initialize sparse on a specific node. The node spans [pnum_begin, pnum_end) + * And number of present sections in this node is map_count. + */ +static void __init sparse_init_nid(int nid, unsigned long pnum_begin, + unsigned long pnum_end, + unsigned long map_count) +{ + unsigned long pnum, usemap_longs, *usemap; + struct page *map; + + usemap_longs = BITS_TO_LONGS(SECTION_BLOCKFLAGS_BITS); + usemap = sparse_early_usemaps_alloc_pgdat_section(NODE_DATA(nid), + usemap_size() * + map_count); + if (!usemap) { + pr_err("%s: node[%d] usemap allocation failed", __func__, nid); + goto failed; + } + sparse_buffer_init(map_count * section_map_size(), nid); + for_each_present_section_nr(pnum_begin, pnum) { + if (pnum >= pnum_end) + break; + + map = sparse_mem_map_populate(pnum, nid, NULL); + if (!map) { + pr_err("%s: node[%d] memory map backing failed. Some memory will not be available.", + __func__, nid); + pnum_begin = pnum; + goto failed; + } + check_usemap_section_nr(nid, usemap); + sparse_init_one_section(__nr_to_section(pnum), pnum, map, usemap); + usemap += usemap_longs; + } + sparse_buffer_fini(); + return; +failed: + /* We failed to allocate, mark all the following pnums as not present */ + for_each_present_section_nr(pnum_begin, pnum) { + struct mem_section *ms; + + if (pnum >= pnum_end) + break; + ms = __nr_to_section(pnum); + ms->section_mem_map = 0; + } +} + +/* + * Allocate the accumulated non-linear sections, allocate a mem_map + * for each and record the physical to section mapping. + */ +void __init new_sparse_init(void) +{ + unsigned long pnum_begin = first_present_section_nr(); + int nid_begin = sparse_early_nid(__nr_to_section(pnum_begin)); + unsigned long pnum_end, map_count = 1; + + /* Setup pageblock_order for HUGETLB_PAGE_SIZE_VARIABLE */ + set_pageblock_order(); + + for_each_present_section_nr(pnum_begin + 1, pnum_end) { + int nid = sparse_early_nid(__nr_to_section(pnum_end)); + + if (nid == nid_begin) { + map_count++; + continue; + } + /* Init node with sections in range [pnum_begin, pnum_end) */ + sparse_init_nid(nid_begin, pnum_begin, pnum_end, map_count); + nid_begin = nid; + pnum_begin = pnum_end; + map_count = 1; + } + /* cover the last node */ + sparse_init_nid(nid_begin, pnum_begin, pnum_end, map_count); + vmemmap_populate_print_last(); +} + #ifdef CONFIG_MEMORY_HOTPLUG /* Mark all memory sections within the pfn range as online */