From patchwork Wed May 29 09:16:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10966573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98B891398 for ; Wed, 29 May 2019 09:16:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89B2528847 for ; Wed, 29 May 2019 09:16:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7DCF728894; Wed, 29 May 2019 09:16:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7BD6288B7 for ; Wed, 29 May 2019 09:16:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5E866B000C; Wed, 29 May 2019 05:16:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E12126B0010; Wed, 29 May 2019 05:16:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD7A16B0266; Wed, 29 May 2019 05:16:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 7FBAD6B000C for ; Wed, 29 May 2019 05:16:31 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id z5so2484007edz.3 for ; Wed, 29 May 2019 02:16:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=JZND1Y4YyPcCZ9xJ+jjp+6LxXZaNcWkZP+CX//HNdqM=; b=QmI66DEGdY46MGEjopgZOoIfJ6PQo+EpoAScsLu1yf761LCdCL1dTM8wqnKX7rvTCa JZcOHeaPScFGDmxwCDFHhPa6chnJfUP7ROE/NAuyg/7jM09a9+jkaTVpUd2YRzFumUTR IL0JUVOLumjxjAXr+T3f2JyfvuUSUP85ZYr9yu/m6iFVxx6lBkyqVIwZZNM4rEbzHtSz DlyQfZwDR8/gnRK66BV1ESHcgmTJj4jQ0PARNPQljRXX5FSDTcXx6KYHueY/SoPemNbx 6+5E/33XHGryfaSSwDhfPm4UKUsk9hfrp8hY/O+z2phBDdCB6ppdA9WXzNsP27H8RNqf pejg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAW6haBna82ePq9gmpKAeG6EHPX1F3cPSYKZx/lbVb8dxsQ2LcxF G6k0Ah35XhHt6XUKto1j0dm3rqJk21KghQvDmP7YwzR/nI/HaBgHrWuRvMqU1lIYMu+NGB8EiBf F2LsdnG4r9JcEGfLYhLiLLNTxCL3/jMQEvNu3PAJ6v1PXmHFgF+BWzJ1vQPsA+VuL7A== X-Received: by 2002:a17:906:1288:: with SMTP id k8mr23941676ejb.303.1559121390971; Wed, 29 May 2019 02:16:30 -0700 (PDT) X-Google-Smtp-Source: APXvYqyAZIYYZrXGkvB2urvZu066i/7lVWq6xbHie7nXQBk9FDtF5mC8KejeifTGzXJUy6qq8qH8 X-Received: by 2002:a17:906:1288:: with SMTP id k8mr23941602ejb.303.1559121389808; Wed, 29 May 2019 02:16:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559121389; cv=none; d=google.com; s=arc-20160816; b=FJ+xmR7akRsNRlo2LEqwFYpplmmX6x9E2zE9itliP8zHnIkBo05gRe6YyrC3zkoo9r 3JtP/IkvXMDKWY0HcWqOMlP3kT4rTbsGsqdwOGXbkpklnqB5CEiN4BqJESb6bZ9/qp9g nr7Empird20Xvd7NkHFthVbTbbs+/WTYVzmkwIjNgqrdHSlPbzKKj6VBv1aNJcJri4HU BTi80kvwiyaTVvsvPzrsKX3voRZCMI2sIx1Lq+nTG/z8//Mwix1jn6n+ztHZe0XjHCyT 6St1MoXQJQM7Wdrr9mj45E4ZmPARjTkdSjMgenndmAH/OBMpJcQ7vxyJ1OETULEaxtez 0PAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=JZND1Y4YyPcCZ9xJ+jjp+6LxXZaNcWkZP+CX//HNdqM=; b=Q7UDBRWwP0npPcDdw72pQkgnncG6rfDRg8x3PLb08MyfxkrSnn712uTpt3leUSfAgX c4cReyyjJnEytbFoWx6YAm7ikiJG1HIrMKR0QnTD6EjtU+ViBCK7XmArLwwtHvfVGYKg gpRKivv2LHKQF0NwYv/ZnS5cNSb9FtoHKk9H5UhU3ts3GjM2x3QUQcsvQv7b+imFPB2T 1kcnTPM6UDeib7s270adSDQHVpGKEY7IKQzn5wOmma5lHO2/nyKJlHKDXyY52D9IpoI7 6IES5HpcBXNeuFCZeRoRU7H6OAxTJEqfaSjgnIacem9QpEKpGFuLYox6EXrfbrZn89Yf yGHQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id h18si4738282ejq.269.2019.05.29.02.16.29 for ; Wed, 29 May 2019 02:16:29 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AFFB7A78; Wed, 29 May 2019 02:16:28 -0700 (PDT) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.41.181]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6FDE23F5AF; Wed, 29 May 2019 02:16:23 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, akpm@linux-foundation.org, catalin.marinas@arm.com, will.deacon@arm.com Cc: mark.rutland@arm.com, mhocko@suse.com, ira.weiny@intel.com, david@redhat.com, cai@lca.pw, logang@deltatee.com, james.morse@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, mgorman@techsingularity.net, osalvador@suse.de, ard.biesheuvel@arm.com Subject: [PATCH V5 1/3] mm/hotplug: Reorder arch_remove_memory() call in __remove_memory() Date: Wed, 29 May 2019 14:46:25 +0530 Message-Id: <1559121387-674-2-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> References: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Memory hot remove uses get_nid_for_pfn() while tearing down linked sysfs entries between memory block and node. It first checks pfn validity with pfn_valid_within() before fetching nid. With CONFIG_HOLES_IN_ZONE config (arm64 has this enabled) pfn_valid_within() calls pfn_valid(). pfn_valid() is an arch implementation on arm64 (CONFIG_HAVE_ARCH_PFN_VALID) which scans all mapped memblock regions with memblock_is_map_memory(). This creates a problem in memory hot remove path which has already removed given memory range from memory block with memblock_[remove|free] before arriving at unregister_mem_sect_under_nodes(). Hence get_nid_for_pfn() returns -1 skipping subsequent sysfs_remove_link() calls leaving node <-> memory block sysfs entries as is. Subsequent memory add operation hits BUG_ON() because of existing sysfs entries. [ 62.007176] NUMA: Unknown node for memory at 0x680000000, assuming node 0 [ 62.052517] ------------[ cut here ]------------ [ 62.053211] kernel BUG at mm/memory_hotplug.c:1143! [ 62.053868] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 62.054589] Modules linked in: [ 62.054999] CPU: 19 PID: 3275 Comm: bash Not tainted 5.1.0-rc2-00004-g28cea40b2683 #41 [ 62.056274] Hardware name: linux,dummy-virt (DT) [ 62.057166] pstate: 40400005 (nZcv daif +PAN -UAO) [ 62.058083] pc : add_memory_resource+0x1cc/0x1d8 [ 62.058961] lr : add_memory_resource+0x10c/0x1d8 [ 62.059842] sp : ffff0000168b3ce0 [ 62.060477] x29: ffff0000168b3ce0 x28: ffff8005db546c00 [ 62.061501] x27: 0000000000000000 x26: 0000000000000000 [ 62.062509] x25: ffff0000111ef000 x24: ffff0000111ef5d0 [ 62.063520] x23: 0000000000000000 x22: 00000006bfffffff [ 62.064540] x21: 00000000ffffffef x20: 00000000006c0000 [ 62.065558] x19: 0000000000680000 x18: 0000000000000024 [ 62.066566] x17: 0000000000000000 x16: 0000000000000000 [ 62.067579] x15: ffffffffffffffff x14: ffff8005e412e890 [ 62.068588] x13: ffff8005d6b105d8 x12: 0000000000000000 [ 62.069610] x11: ffff8005d6b10490 x10: 0000000000000040 [ 62.070615] x9 : ffff8005e412e898 x8 : ffff8005e412e890 [ 62.071631] x7 : ffff8005d6b105d8 x6 : ffff8005db546c00 [ 62.072640] x5 : 0000000000000001 x4 : 0000000000000002 [ 62.073654] x3 : ffff8005d7049480 x2 : 0000000000000002 [ 62.074666] x1 : 0000000000000003 x0 : 00000000ffffffef [ 62.075685] Process bash (pid: 3275, stack limit = 0x00000000d754280f) [ 62.076930] Call trace: [ 62.077411] add_memory_resource+0x1cc/0x1d8 [ 62.078227] __add_memory+0x70/0xa8 [ 62.078901] probe_store+0xa4/0xc8 [ 62.079561] dev_attr_store+0x18/0x28 [ 62.080270] sysfs_kf_write+0x40/0x58 [ 62.080992] kernfs_fop_write+0xcc/0x1d8 [ 62.081744] __vfs_write+0x18/0x40 [ 62.082400] vfs_write+0xa4/0x1b0 [ 62.083037] ksys_write+0x5c/0xc0 [ 62.083681] __arm64_sys_write+0x18/0x20 [ 62.084432] el0_svc_handler+0x88/0x100 [ 62.085177] el0_svc+0x8/0xc Re-ordering arch_remove_memory() with memblock_[free|remove] solves the problem on arm64 as pfn_valid() behaves correctly and returns positive as memblock for the address range still exists. arch_remove_memory() removes applicable memory sections from zone with __remove_pages() and tears down kernel linear mapping. Removing memblock regions afterwards is safe because there is no other memblock (bootmem) allocator user that late. So nobody is going to allocate from the removed range just to blow up later. Also nobody should be using the bootmem allocated range else we wouldn't allow to remove it. So reordering is indeed safe. Acked-by: Michal Hocko Reviewed-by: David Hildenbrand Reviewed-by: Oscar Salvador Signed-off-by: Anshuman Khandual Acked-by: Mark Rutland --- mm/memory_hotplug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index e096c98..67dfdb8 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1851,10 +1851,10 @@ void __ref __remove_memory(int nid, u64 start, u64 size) /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); + arch_remove_memory(nid, start, size, NULL); memblock_free(start, size); memblock_remove(start, size); - arch_remove_memory(nid, start, size, NULL); __release_memory_resource(start, size); try_offline_node(nid); From patchwork Wed May 29 09:16:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10966577 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCC641398 for ; Wed, 29 May 2019 09:16:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE32528894 for ; Wed, 29 May 2019 09:16:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1EEF288E0; Wed, 29 May 2019 09:16:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FAFE28894 for ; Wed, 29 May 2019 09:16:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E89E6B0010; Wed, 29 May 2019 05:16:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 498676B0266; Wed, 29 May 2019 05:16:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 361776B026A; Wed, 29 May 2019 05:16:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id E03B16B0010 for ; Wed, 29 May 2019 05:16:36 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id z5so2484319edz.3 for ; Wed, 29 May 2019 02:16:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Uhvy+OSflg2PNCXiYNPvnH6BRNLk2GvqTjsi+E6Ajyo=; b=Zbj4CgDz0M7llSc8/FMvNbwPPoZ34CIbeZoKeuMgVK1jIGcwqjW2PYNSNZUGFdQyUo WqVflovgPVSn0WXNtrBDobZfDyd02/Ffw5El/euwwAZYNIIxi/dgo0tU7xkNZ/G6kfFE KPyp5A22dhrYXICcG9v2r2ItrxUMxxDg1Jf3+n+S3IBCcVkxDH/m6Yi5EbchlQtSJOiO HK/3jMKOAXARzA0xtpEEBZlUefo/+7M5YlJqBV64jK9B6r5R9A1btefCCIcDQi237UNn gjm72vSODGE+OlUQRSJ7LuWZI0DrT7dlC7/M9mUWRLiV3VovI9OFHxl2rKOhRvPTXWtu WoDw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAUnM7R9PQeZimj4S5VATjhYdZ6fGJv0+HcaEfIE0xzkk+LLiWw6 Df+hUCBD+yV/9NgG87WYjDtMz5BhwLxVYOTKklAHf0G8ikOuD8p2UWzitabTgthyAJbvJHCy7pS 5QfPAAxWBOPrMD2wdtby1YwwzpYkIiym36z/UgYQBpKER1qRpfokZzAguxIsafCf4jg== X-Received: by 2002:a17:906:c82e:: with SMTP id dd14mr45411305ejb.133.1559121396342; Wed, 29 May 2019 02:16:36 -0700 (PDT) X-Google-Smtp-Source: APXvYqxBneP1pFXEFq0ar7Wb4lM8EGjaYQcx6ZwNfkvvOmfpbRF2ehQjq8IYxinUErfmYCTc+Mct X-Received: by 2002:a17:906:c82e:: with SMTP id dd14mr45411253ejb.133.1559121395503; Wed, 29 May 2019 02:16:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559121395; cv=none; d=google.com; s=arc-20160816; b=FRmA7UJLVgwAapFWtEa/VE2xjXqBZXVLmyxqSaAFMSnSjLqtIuMT8nVP1+WSallvQa EZ3vQrn8x0EWOp+z6qqYYUjXvFtkoJmcAAw7Ol6f/IJk7MSb5oZ6jr11Qv39VJb78lr3 TaVvHKbVGyD24OSLEiB0C8rs8yzJtF2VrluLTkAdb/HwQorGIpxbPok8g119IuMRT/MP di0e5xGzbnbt67+ET/Bd+5y90RiWnq40mdtYGMkx0UHRGrYHz/wng6eCYUTxqb60IB52 LS/BbNEWu5HuPL23AacX5FHS7eG8bn1Sw9vaG5+n6vUzORt8RbgXvcSIilVbTCiDJYob /HXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Uhvy+OSflg2PNCXiYNPvnH6BRNLk2GvqTjsi+E6Ajyo=; b=Od08jQhYDAi/zCgjxYGE6HX3ob51mIusdvGsXV+CcekXXWk84x20YjRXZctqqsqCMo nRkBm/+088WsUpisDVuja0x5MqUOH/hT/x9Jc7YMdwB4fy2lV6sUiEx+6iXFXehvK1kH VEak9nsuVcmo3nvWKWSW9Tlwspbe64QQkstcHDoSkrx3Ykw9UuVYAzFVltGIf6hU2hPs FY+vXQ2WZGfLrtrFGaPbgGzNy+ikD9ACwD8JqObeDPmtZFP9d4dobt1E7XoF2cSWrgQ4 hk0aI/24rABQ38KuX+15yq3H6BM/xW5R1AlRrdEqLwMUqVgNvmLrQbolVGoo41wU801t XTnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id w1si5880693eji.251.2019.05.29.02.16.35 for ; Wed, 29 May 2019 02:16:35 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E45D15AD; Wed, 29 May 2019 02:16:34 -0700 (PDT) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.41.181]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2E2683F5AF; Wed, 29 May 2019 02:16:28 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, akpm@linux-foundation.org, catalin.marinas@arm.com, will.deacon@arm.com Cc: mark.rutland@arm.com, mhocko@suse.com, ira.weiny@intel.com, david@redhat.com, cai@lca.pw, logang@deltatee.com, james.morse@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, mgorman@techsingularity.net, osalvador@suse.de, ard.biesheuvel@arm.com Subject: [PATCH V5 2/3] arm64/mm: Hold memory hotplug lock while walking for kernel page table dump Date: Wed, 29 May 2019 14:46:26 +0530 Message-Id: <1559121387-674-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> References: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The arm64 page table dump code can race with concurrent modification of the kernel page tables. When a leaf entries are modified concurrently, the dump code may log stale or inconsistent information for a VA range, but this is otherwise not harmful. When intermediate levels of table are freed, the dump code will continue to use memory which has been freed and potentially reallocated for another purpose. In such cases, the dump code may dereference bogus addresses, leading to a number of potential problems. Intermediate levels of table may by freed during memory hot-remove, which will be enabled by a subsequent patch. To avoid racing with this, take the memory hotplug lock when walking the kernel page table. Acked-by: David Hildenbrand Signed-off-by: Anshuman Khandual Acked-by: Mark Rutland --- arch/arm64/mm/ptdump_debugfs.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/mm/ptdump_debugfs.c b/arch/arm64/mm/ptdump_debugfs.c index 064163f..80171d1 100644 --- a/arch/arm64/mm/ptdump_debugfs.c +++ b/arch/arm64/mm/ptdump_debugfs.c @@ -7,7 +7,10 @@ static int ptdump_show(struct seq_file *m, void *v) { struct ptdump_info *info = m->private; + + get_online_mems(); ptdump_walk_pgd(m, info); + put_online_mems(); return 0; } DEFINE_SHOW_ATTRIBUTE(ptdump); From patchwork Wed May 29 09:16:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 10966579 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 458441398 for ; Wed, 29 May 2019 09:16:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3538128894 for ; Wed, 29 May 2019 09:16:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2994828847; Wed, 29 May 2019 09:16:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D79828894 for ; Wed, 29 May 2019 09:16:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 537376B0266; Wed, 29 May 2019 05:16:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E6C46B026A; Wed, 29 May 2019 05:16:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D6CB6B026B; Wed, 29 May 2019 05:16:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id E2DFC6B0266 for ; Wed, 29 May 2019 05:16:42 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id r5so2416525edd.21 for ; Wed, 29 May 2019 02:16:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=bLiBWyQLr/9YLn0a0hHBHpRRb/bQ//w0i5F3N/eBaHM=; b=t8OYoFrRW93k2rIJauUKutnyoAG+o7d/Ojq6iKOKjMtgz9geIVMLY/lDXpGfj7H0IS 4vi1Hz4rL5NvIPPBZtq1HZBHfAEVrhrPvqoBS21mSiR3FiuZOlCtIEhYbBNc83E/Twig Cforxz7GABGEznQfICwnFhrcYUxYB/n0K5+tMpv6W70rlYQviyMp2SifYnKuPkfoSUbX we9H0f1a5YIBqTRM+99z7YUOTw57MbRcPqKMqSQ79v5Mi6rdJCcke1Uv7rxIyFML+oc6 b5HFn7H90iTElD/k0DfvnUl8WWaDjQRt2aGC5hzP44Qs/1IU6wD/il1jtY+B2VkB7nfL gGUw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Gm-Message-State: APjAAAUBLmzy2NuBkRcy7rzHG6KMUKvqL1/pw9NabNPV3VEKzRrpYswO K2n3Cty3YzDndMW33QSAB3py5yxNvS7/NSV7o1/++8TF3WG03FEdlznBPTDEE9m0btGiTSePVMF +h2cz90jYPIo/ZsCFdsWy9VcqKAeY5Q1l9f0cfsrofRBKvzdfrgkGr3hsA93HTjbMYg== X-Received: by 2002:a17:906:f84a:: with SMTP id ks10mr88981703ejb.65.1559121402484; Wed, 29 May 2019 02:16:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqz3wraVFtnrWaQdy4J4EMTM98mDciwUoZjcCfu119ttmOIX3NKzJTubYNG0xd5hCw6LkOd+ X-Received: by 2002:a17:906:f84a:: with SMTP id ks10mr88981634ejb.65.1559121401387; Wed, 29 May 2019 02:16:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559121401; cv=none; d=google.com; s=arc-20160816; b=HnSN/O+xVHcMtU6+yQlm5lpIar9yB8TfNyyvnKHYzNJLM9cRMibmBOLtfGwkpm8fuc Y+BUQEyRf7c7FdpbMjhS7YBtANQbElfCsnWXWcUN4HAvTh7yVA4BL45LwNkRaiVLTizw /yZMl0CmUW7tjG9Dp3GhWtCCo2BmAJq+ppb82qNXnPrhPm16Yn9ztPJHNNghAoCvdZq5 dPTNjQMvKjvAQaqg/ytqtbqe8i3fE56hHO5QCsaRZivn4EqH+fk3M6QYZFKTE4xR9+Ha EYkGs1XxPuhDus/5K9ZBN8zKq0dC5fWZf6IiyU83dkqpRz3PdH+IN7DJuJ/EXaovZOzD cNlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=bLiBWyQLr/9YLn0a0hHBHpRRb/bQ//w0i5F3N/eBaHM=; b=aIJUxr06feq4T5nZ6OuL3vB+sOtqa3e7rEzEqdl1KESaFksxK94Sjp4R2j/5mEPJye 6Qbw/Rw/F01o/IPC3F/IYMyOPiVKJ+KJJ26j12agxStknE4auFST2bxe7oYMgaRzR0/w FMvqd91X6s4U2R6yBVK13RFtCXBTiZ9wiZtVOz/oc8LTX0c2rDtitrYVZsRP9PXQfq8E fsWRytRvPH5rzHtw16j6t8I4FifFRDQCbRH98LJ1GNVZp71dkme22Q75wvllpXR7Da0m MGQULZHDCUKAmONuGAe1sGWywiOeRLI629FVtjG7Gji8DDu2PmcR4xpgIpWwBqQXQuYV kxgw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id h23si5332175edb.300.2019.05.29.02.16.40 for ; Wed, 29 May 2019 02:16:41 -0700 (PDT) Received-SPF: pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of anshuman.khandual@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D180165C; Wed, 29 May 2019 02:16:40 -0700 (PDT) Received: from p8cg001049571a15.blr.arm.com (p8cg001049571a15.blr.arm.com [10.162.41.181]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E0E943F5AF; Wed, 29 May 2019 02:16:34 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, akpm@linux-foundation.org, catalin.marinas@arm.com, will.deacon@arm.com Cc: mark.rutland@arm.com, mhocko@suse.com, ira.weiny@intel.com, david@redhat.com, cai@lca.pw, logang@deltatee.com, james.morse@arm.com, cpandya@codeaurora.org, arunks@codeaurora.org, dan.j.williams@intel.com, mgorman@techsingularity.net, osalvador@suse.de, ard.biesheuvel@arm.com Subject: [PATCH V5 3/3] arm64/mm: Enable memory hot remove Date: Wed, 29 May 2019 14:46:27 +0530 Message-Id: <1559121387-674-4-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> References: <1559121387-674-1-git-send-email-anshuman.khandual@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The arch code for hot-remove must tear down portions of the linear map and vmemmap corresponding to memory being removed. In both cases the page tables mapping these regions must be freed, and when sparse vmemmap is in use the memory backing the vmemmap must also be freed. This patch adds a new remove_pagetable() helper which can be used to tear down either region, and calls it from vmemmap_free() and ___remove_pgd_mapping(). The sparse_vmap argument determines whether the backing memory will be freed. While freeing intermediate level page table pages bail out if any of it's entries are still valid. This can happen for partially filled kernel page table either from a previously attempted failed memory hot add or while removing an address range which does not span the entire page table page range. The vmemmap region may share levels of table with the vmalloc region. Take the kernel ptl so that we can safely free potentially-shared tables. While here update arch_add_memory() to handle __add_pages() failures by just unmapping recently added kernel linear mapping. Now enable memory hot remove on arm64 platforms by default with ARCH_ENABLE_MEMORY_HOTREMOVE. This implementation is overall inspired from kernel page table tear down procedure on X86 architecture. Signed-off-by: Anshuman Khandual Acked-by: David Hildenbrand --- arch/arm64/Kconfig | 3 + arch/arm64/mm/mmu.c | 211 +++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 212 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 697ea05..7f917fe 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -268,6 +268,9 @@ config HAVE_GENERIC_GUP config ARCH_ENABLE_MEMORY_HOTPLUG def_bool y +config ARCH_ENABLE_MEMORY_HOTREMOVE + def_bool y + config SMP def_bool y diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a1bfc44..4803624 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -733,6 +733,187 @@ int kern_addr_valid(unsigned long addr) return pfn_valid(pte_pfn(pte)); } + +#ifdef CONFIG_MEMORY_HOTPLUG +static void free_hotplug_page_range(struct page *page, ssize_t size) +{ + WARN_ON(PageReserved(page)); + free_pages((unsigned long)page_address(page), get_order(size)); +} + +static void free_hotplug_pgtable_page(struct page *page) +{ + free_hotplug_page_range(page, PAGE_SIZE); +} + +static void free_pte_table(pte_t *ptep, pmd_t *pmdp, unsigned long addr) +{ + struct page *page; + int i; + + for (i = 0; i < PTRS_PER_PTE; i++) { + if (!pte_none(ptep[i])) + return; + } + + page = pmd_page(READ_ONCE(*pmdp)); + pmd_clear(pmdp); + __flush_tlb_kernel_pgtable(addr); + free_hotplug_pgtable_page(page); +} + +static void free_pmd_table(pmd_t *pmdp, pud_t *pudp, unsigned long addr) +{ + struct page *page; + int i; + + if (CONFIG_PGTABLE_LEVELS <= 2) + return; + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (!pmd_none(pmdp[i])) + return; + } + + page = pud_page(READ_ONCE(*pudp)); + pud_clear(pudp); + __flush_tlb_kernel_pgtable(addr); + free_hotplug_pgtable_page(page); +} + +static void free_pud_table(pud_t *pudp, pgd_t *pgdp, unsigned long addr) +{ + struct page *page; + int i; + + if (CONFIG_PGTABLE_LEVELS <= 3) + return; + + for (i = 0; i < PTRS_PER_PUD; i++) { + if (!pud_none(pudp[i])) + return; + } + + page = pgd_page(READ_ONCE(*pgdp)); + pgd_clear(pgdp); + __flush_tlb_kernel_pgtable(addr); + free_hotplug_pgtable_page(page); +} + +static void +remove_pte_table(pmd_t *pmdp, unsigned long addr, + unsigned long end, bool sparse_vmap) +{ + struct page *page; + pte_t *ptep, pte; + unsigned long start = addr; + + for (; addr < end; addr += PAGE_SIZE) { + ptep = pte_offset_kernel(pmdp, addr); + pte = READ_ONCE(*ptep); + + if (pte_none(pte)) + continue; + + WARN_ON(!pte_present(pte)); + if (sparse_vmap) { + page = pte_page(pte); + free_hotplug_page_range(page, PAGE_SIZE); + } + pte_clear(&init_mm, addr, ptep); + } + flush_tlb_kernel_range(start, end); +} + +static void +remove_pmd_table(pud_t *pudp, unsigned long addr, + unsigned long end, bool sparse_vmap) +{ + unsigned long next; + struct page *page; + pte_t *ptep_base; + pmd_t *pmdp, pmd; + + for (; addr < end; addr = next) { + next = pmd_addr_end(addr, end); + pmdp = pmd_offset(pudp, addr); + pmd = READ_ONCE(*pmdp); + + if (pmd_none(pmd)) + continue; + + WARN_ON(!pmd_present(pmd)); + if (pmd_sect(pmd)) { + if (sparse_vmap) { + page = pmd_page(pmd); + free_hotplug_page_range(page, PMD_SIZE); + } + pmd_clear(pmdp); + continue; + } + ptep_base = pte_offset_kernel(pmdp, 0UL); + remove_pte_table(pmdp, addr, next, sparse_vmap); + free_pte_table(ptep_base, pmdp, addr); + } +} + +static void +remove_pud_table(pgd_t *pgdp, unsigned long addr, + unsigned long end, bool sparse_vmap) +{ + unsigned long next; + struct page *page; + pmd_t *pmdp_base; + pud_t *pudp, pud; + + for (; addr < end; addr = next) { + next = pud_addr_end(addr, end); + pudp = pud_offset(pgdp, addr); + pud = READ_ONCE(*pudp); + + if (pud_none(pud)) + continue; + + WARN_ON(!pud_present(pud)); + if (pud_sect(pud)) { + if (sparse_vmap) { + page = pud_page(pud); + free_hotplug_page_range(page, PUD_SIZE); + } + pud_clear(pudp); + continue; + } + pmdp_base = pmd_offset(pudp, 0UL); + remove_pmd_table(pudp, addr, next, sparse_vmap); + free_pmd_table(pmdp_base, pudp, addr); + } +} + +static void +remove_pagetable(unsigned long start, unsigned long end, bool sparse_vmap) +{ + unsigned long addr, next; + pud_t *pudp_base; + pgd_t *pgdp, pgd; + + spin_lock(&init_mm.page_table_lock); + for (addr = start; addr < end; addr = next) { + next = pgd_addr_end(addr, end); + pgdp = pgd_offset_k(addr); + pgd = READ_ONCE(*pgdp); + + if (pgd_none(pgd)) + continue; + + WARN_ON(!pgd_present(pgd)); + pudp_base = pud_offset(pgdp, 0UL); + remove_pud_table(pgdp, addr, next, sparse_vmap); + free_pud_table(pudp_base, pgdp, addr); + } + spin_unlock(&init_mm.page_table_lock); +} +#endif + #ifdef CONFIG_SPARSEMEM_VMEMMAP #if !ARM64_SWAPPER_USES_SECTION_MAPS int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, @@ -780,6 +961,9 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, void vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) { +#ifdef CONFIG_MEMORY_HOTPLUG + remove_pagetable(start, end, true); +#endif } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ @@ -1070,10 +1254,16 @@ int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) } #ifdef CONFIG_MEMORY_HOTPLUG +static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size) +{ + WARN_ON(pgdir != init_mm.pgd); + remove_pagetable(start, start + size, false); +} + int arch_add_memory(int nid, u64 start, u64 size, struct mhp_restrictions *restrictions) { - int flags = 0; + int ret, flags = 0; if (rodata_full || debug_pagealloc_enabled()) flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; @@ -1081,7 +1271,24 @@ int arch_add_memory(int nid, u64 start, u64 size, __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), size, PAGE_KERNEL, __pgd_pgtable_alloc, flags); - return __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT, + ret = __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT, restrictions); + if (ret) + __remove_pgd_mapping(swapper_pg_dir, + __phys_to_virt(start), size); + return ret; +} + +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory(int nid, u64 start, u64 size, + struct vmem_altmap *altmap) +{ + unsigned long start_pfn = start >> PAGE_SHIFT; + unsigned long nr_pages = size >> PAGE_SHIFT; + struct zone *zone = page_zone(pfn_to_page(start_pfn)); + + __remove_pages(zone, start_pfn, nr_pages, altmap); + __remove_pgd_mapping(swapper_pg_dir, __phys_to_virt(start), size); } #endif +#endif