From patchwork Tue Oct 2 15:00:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10623819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1ED4C175A for ; Tue, 2 Oct 2018 15:00:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0EBB8212DA for ; Tue, 2 Oct 2018 15:00:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 02F40212D6; Tue, 2 Oct 2018 15:00:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54C3B200DF for ; Tue, 2 Oct 2018 15:00:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A9036B000A; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 62E9A6B000C; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40C906B0010; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id C70886B000A for ; Tue, 2 Oct 2018 11:00:52 -0400 (EDT) Received: by mail-wm1-f69.google.com with SMTP id 189-v6so1992814wme.0 for ; Tue, 02 Oct 2018 08:00:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=9SeupK+KhOJ9JTSOt0AEzf3o4SV6rGMu7uHKPkHuPW8=; b=hVI1R2yTaQQAE4U2KENlBpcVuuxrnsaUw499j08o8TrhQ97aZ9iZ6PFfXnFAjIXN5a B0tamrg20bjYl1xuqWQTXqxs+fNPcPfRYRF9iet8QrEYqO8rNyIiXlEDvTOo7ef7C+hP vIGn39chTnglkxuRKS3PMHUkQXe9pBo2unS046aKWMMx74A/AOaZYUPMjC1UOZdtGKS1 BFThKZ6ojJUgdMi5AdO0VzCR/shPm1+qyn50YMpZAJ4f7kQSmwqWvS1RtCpp+lQFIQmN f5z8IbKakl+jcRzDzL5Z6m13v9M7pVWHOxKuFqFNXfATizKx6V3eqK3nO7AzEPMTG08W S2rQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: ABuFfohhiDpeivXWw0x69B/9WZhP1VqGD7/KeIgD6t6iSpkjm0huARSD 4SmYyVqZX/IYtWiq4c8cAtyf+CB4vXNy7WwzACTyKlgEHgeywSZlbwdTjK2MP+bNhUovHE3MxI+ t3b0UEywMG/9jNgPYlHG4ri7dmWd3658+bxHJoUStG8z6xQTGUIEIM5RxxTkzxvwxdglMGbGDSK Wx8E3yPmi7esipSXRtWsUXMm0UhjTLFEAEC/pcGXuRJxFzBqe8BFwTgjfxNuGLkFGzFLQlEcmds Ube0rLXHncJOlN8ToTtoNjRfOrZejFR7UhOcPrW2BmEec0OZpGgHyWP46pENa+jTmyGr67HOw0N 8qLcxizObWzv4z7UMtto8jH4hOXn1QmEm2+LmfIp0r/7AJYq++PG0D4RHVDbMhOIWLj36AVUDQ= = X-Received: by 2002:a1c:14d1:: with SMTP id 200-v6mr2222476wmu.106.1538492451585; Tue, 02 Oct 2018 08:00:51 -0700 (PDT) X-Received: by 2002:a1c:14d1:: with SMTP id 200-v6mr2222325wmu.106.1538492449036; Tue, 02 Oct 2018 08:00:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538492449; cv=none; d=google.com; s=arc-20160816; b=pG09to3Pu6cNesI5SntomTl7mWAVU3CMxIkK6nB+2HksQsDwYtNlwuxsN6wVD2EjaR 0ghJi7Yu/f5l/3VDWbzumSyxaIs3oXyMX1OZ/x6H44XG2Fpjw2C0dmgbs0a67gkxhCRx vRrb0ih1tB5WnLFyj9q/bprX1HxrfwIyr/kpx5dcaGvA9jpkLxGORtImEbnw9+b8NPMJ jv+rCwk3gV5QQXoNn3QGsn1JfvncvVevrYXT5oK1c81ayvAG8g/imtOuEYROS1cUUlKa ol+oPgBJr4brgt6FyOnLiKtQulAyrxrlOzYM/iMdfx2LGRBDfvGJetzURUpfhF/ulkbt xtRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=9SeupK+KhOJ9JTSOt0AEzf3o4SV6rGMu7uHKPkHuPW8=; b=IPlYUjfbc1cZLA++SB+7JjTrezav8EpE9NV6IRqM8FMIKMYNb3gBLd7FJFR1fXML/N spGWpbb7drcrBTR7zCvmMdJ4oDU/+C7z7yL+wPTvQsnH1ONhftDDXZX/qlpNmiox2kUF ONWMwpj3aIAMfaU9nWbClPAU2tOyxIi21Zi3tfkRsKzqMDeNGunhKjIK2WK0l9jBc+Bu YIQNXpnvmj8PYXmKRF0WR4iW0WamGhA9L1qNjY9fmWbf6HVMZyG6btdKoeBfVWGNZsvv ow/U2m1nETY5M+hvGDy5cxpPSoZn+N7v8PkuYdkKxHWz2OJw5aOP6a7B0pSXMnvfSJms mToA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b15-v6sor11433017wrf.36.2018.10.02.08.00.48 for (Google Transport Security); Tue, 02 Oct 2018 08:00:49 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ACcGV60zkt5o1uNzAHLv6hE7KNYU2LEZm9OdUWFqaRjiEcyYI1/pgrPf+MRFrnk/nVw0U1B/W1zfAA== X-Received: by 2002:a5d:4706:: with SMTP id y6-v6mr10462685wrq.256.1538492448577; Tue, 02 Oct 2018 08:00:48 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id j9-v6sm14427601wrt.32.2018.10.02.08.00.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 08:00:47 -0700 (PDT) Received: from d104.suse.de (charybdis-ext.suse.de [195.135.221.2]) by techadventures.net (Postfix) with ESMTPA id 1D7CC125729; Tue, 2 Oct 2018 17:00:47 +0200 (CEST) From: Oscar Salvador To: linux-mm@kvack.org Cc: mhocko@suse.com, dan.j.williams@intel.com, yasu.isimatu@gmail.com, rppt@linux.vnet.ibm.com, malat@debian.org, linux-kernel@vger.kernel.org, pavel.tatashin@microsoft.com, jglisse@redhat.com, Jonathan.Cameron@huawei.com, rafael@kernel.org, david@redhat.com, dave.jiang@intel.com, Oscar Salvador Subject: [RFC PATCH v3 1/5] mm/memory_hotplug: Add nid parameter to arch_remove_memory Date: Tue, 2 Oct 2018 17:00:25 +0200 Message-Id: <20181002150029.23461-2-osalvador@techadventures.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181002150029.23461-1-osalvador@techadventures.net> References: <20181002150029.23461-1-osalvador@techadventures.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Oscar Salvador This patch is only a preparation for the following-up patches. The idea of passing the nid is that will allow us to get rid of the zone parameter in the patches that follow Signed-off-by: Oscar Salvador Reviewed-by: David Hildenbrand --- arch/ia64/mm/init.c | 2 +- arch/powerpc/mm/mem.c | 2 +- arch/s390/mm/init.c | 2 +- arch/sh/mm/init.c | 2 +- arch/x86/mm/init_32.c | 2 +- arch/x86/mm/init_64.c | 2 +- include/linux/memory_hotplug.h | 2 +- kernel/memremap.c | 4 +++- mm/memory_hotplug.c | 2 +- 9 files changed, 11 insertions(+), 9 deletions(-) diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c index d5e12ff1d73c..904fe55e10fc 100644 --- a/arch/ia64/mm/init.c +++ b/arch/ia64/mm/init.c @@ -661,7 +661,7 @@ int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, } #ifdef CONFIG_MEMORY_HOTREMOVE -int arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 578cbb262c01..445fce705f91 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -138,7 +138,7 @@ int __meminit arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap * } #ifdef CONFIG_MEMORY_HOTREMOVE -int __meminit arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int __meminit arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index e472cd763eb3..f705da1a085f 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -239,7 +239,7 @@ int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, } #ifdef CONFIG_MEMORY_HOTREMOVE -int arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { /* * There is no hardware or firmware interface which could trigger a diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c index c8c13c777162..a8e5c0e00fca 100644 --- a/arch/sh/mm/init.c +++ b/arch/sh/mm/init.c @@ -443,7 +443,7 @@ EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid); #endif #ifdef CONFIG_MEMORY_HOTREMOVE -int arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = PFN_DOWN(start); unsigned long nr_pages = size >> PAGE_SHIFT; diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index f2837e4c40b3..b2a698d87a0e 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -860,7 +860,7 @@ int arch_add_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap, } #ifdef CONFIG_MEMORY_HOTREMOVE -int arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 5fab264948c2..c754d9543ae1 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1147,7 +1147,7 @@ kernel_physical_mapping_remove(unsigned long start, unsigned long end) remove_pagetable(start, end, true, NULL); } -int __ref arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +int __ref arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index ffd9cd10fcf3..f9fc35819e65 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -107,7 +107,7 @@ static inline bool movable_node_is_enabled(void) } #ifdef CONFIG_MEMORY_HOTREMOVE -extern int arch_remove_memory(u64 start, u64 size, +extern int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap); extern int __remove_pages(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, struct vmem_altmap *altmap); diff --git a/kernel/memremap.c b/kernel/memremap.c index e3036433ce4e..fe54bba2d7e2 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -121,6 +121,7 @@ static void devm_memremap_pages_release(void *data) struct resource *res = &pgmap->res; resource_size_t align_start, align_size; unsigned long pfn; + int nid; pgmap->kill(pgmap->ref); for_each_device_pfn(pfn, pgmap) @@ -130,6 +131,7 @@ static void devm_memremap_pages_release(void *data) align_start = res->start & ~(SECTION_SIZE - 1); align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE) - align_start; + nid = dev_to_node(dev); mem_hotplug_begin(); if (pgmap->type == MEMORY_DEVICE_PRIVATE) { @@ -137,7 +139,7 @@ static void devm_memremap_pages_release(void *data) __remove_pages(page_zone(pfn_to_page(pfn)), pfn, align_size >> PAGE_SHIFT, NULL); } else { - arch_remove_memory(align_start, align_size, + arch_remove_memory(nid, align_start, align_size, pgmap->altmap_valid ? &pgmap->altmap : NULL); kasan_remove_zero_shadow(__va(align_start), align_size); } diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index d4c7e42e46f3..11b7dcf83323 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1890,7 +1890,7 @@ void __ref __remove_memory(int nid, u64 start, u64 size) memblock_free(start, size); memblock_remove(start, size); - arch_remove_memory(start, size, NULL); + arch_remove_memory(nid, start, size, NULL); try_offline_node(nid); From patchwork Tue Oct 2 15:00:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10623821 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 875A215A7 for ; Tue, 2 Oct 2018 15:01:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A40B204FB for ; Tue, 2 Oct 2018 15:01:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E98B2237D; Tue, 2 Oct 2018 15:01:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99319204FB for ; Tue, 2 Oct 2018 15:01:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B11616B000C; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AC65B6B000E; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 960256B000D; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 36AC86B000E for ; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id k44-v6so1771610wre.18 for ; Tue, 02 Oct 2018 08:00:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=rffykfS8GnnC5FY6HrPy1m+B8yF9Iyvro/mIhUV67Fk=; b=OeroRTS9R3gB/2UF3J/vCjm86yCRqXazd8ghyj/2tjFQ+FLS91Fxfqw8zoKZN0cZny HA4cFJCkWFDQ2E7M5JMExItSRthBswGjbAyIFe5q09Fln5+MI7AL5AJG1ozGV9jw1xGY 6dis17+yVweqdnkFqBPLBemjxV60xKOLLK8p5+rQjjt/gxZrLYmGbObEzPgMzKQTVsZi jKRSW5jbQt0fc6QOSyHK1RjE0tB5Yt62CsibOoki4bjKYIsNXT523RpUzsRXpC2VwQlu f1BTg1o++ZF6qzUdRyOuqCiKVVMnj6o8gu02WTqBMpTnk5DTGuWw3eAx0Ez/1otkaWK8 6+WQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: ABuFfoi3pToNuwXGJpedtFocZQqkrjiVih8JobAY8bQeF8A7lBC8bSzu mGgTfd1f4LGcDSq055qqg3M7VrPl/CDLoExhQYJRGIm4lK//5rtiWNO9vpAd1htbKPqGuOZLsbT EdAr21bCbREN+zeMtT2+B560o2WfeXrwKdTbiYkyyaLony9BREUTN1V3YzR9o129WcpHOjTf17S TSueaw5b6gQyUsX4S4ocIWnS4cWTGy7K2fxebMb9j68EeLwTHtjlYfFxAakVvnHqGBwj7+eJUh6 jiRsfC/mp3bPdr9rHDUdOIoJwvhShmRQw6xpT9hbjpEeGcfFNvDK+bkANdZ3ZOPutO9lWsdwn0N 9tcWxkOliQkOmrODmE5FufP2Tz6hJy18JJtoQbgv7ujOR/PDEtkGVKcQbzraucRtY540xeZkKw= = X-Received: by 2002:a5d:4d4b:: with SMTP id a11-v6mr11481368wru.40.1538492452212; Tue, 02 Oct 2018 08:00:52 -0700 (PDT) X-Received: by 2002:a5d:4d4b:: with SMTP id a11-v6mr11481202wru.40.1538492449948; Tue, 02 Oct 2018 08:00:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538492449; cv=none; d=google.com; s=arc-20160816; b=NPo7KyS1y3fw3xJZB8D+6xHznxX7gfcYDPMpTDS5FJu9X03wHXu5fJeKTpKjJMyruS g7QALDL7wIyBB/gzJrGBmy62wl/vtVkxJnvTDONBNSrUkF6fxSuWMkuKaDVBPqUI+tQ9 H5xXUzpmecrGp14C/6UTCErT+VKXgdgYjqxqB3tbNv+ctf+Hr71wGU+Qr/4Oj3ajoGHr YdkK2k1d1WT7qaJHPmSyFXJ0rjD/kp6XQkuBNzCvzkkWMnrr+8MV2xFbrvxOKrOQL9qR IMP19k9H8yv1CPdoJ1tUMJQXjwkFVXPcFIMmBNqjR3Kfw536CMEKIgDfD8GhacWL1MWC oBBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=rffykfS8GnnC5FY6HrPy1m+B8yF9Iyvro/mIhUV67Fk=; b=FSK2JOyA51dnya1FGCNy5cpm4XpR2rVxK/BY4hMGrYXMnLKYHWUK/hBkTAcwwdy9h9 QnDfaHfNKp2YMFEk2SjtFLbPFa7qJMNAUKsmvoGIz0sb6I7eFWfP9K1ryGKpS1frWonb mOr4yHFU1b/JMZ2ZL3ijxrFh8+WkCOec5mFRZHNrGfEAx6xpuGeQlqjYI2xZvLouTyGC DXW4zWmh+kCjU/3nyP/t7skSh/aK59QXHJfi5drq9/xulSt8zT2hlo4IdT3KP6/O3Ky7 c1Z9M/AtGERbEjkaLtwQKz8uz+AKVE7RCtvOMpUd3Zu4c9BAxhyHxVUv2RA7rQM6nQ72 2vDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n198-v6sor8940953wmd.1.2018.10.02.08.00.49 for (Google Transport Security); Tue, 02 Oct 2018 08:00:49 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ACcGV63lWXoZ8OOrBe9MPcA/4IoJOYcPZxofXSp0Ou4pVNE+mt6ihwP9AZLU96bk43bnXO9Dfp3Lzw== X-Received: by 2002:a1c:2052:: with SMTP id g79-v6mr2246463wmg.42.1538492449309; Tue, 02 Oct 2018 08:00:49 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id r16-v6sm4286132wrv.21.2018.10.02.08.00.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 08:00:48 -0700 (PDT) Received: from d104.suse.de (charybdis-ext.suse.de [195.135.221.2]) by techadventures.net (Postfix) with ESMTPA id 61D6912561E; Tue, 2 Oct 2018 17:00:47 +0200 (CEST) From: Oscar Salvador To: linux-mm@kvack.org Cc: mhocko@suse.com, dan.j.williams@intel.com, yasu.isimatu@gmail.com, rppt@linux.vnet.ibm.com, malat@debian.org, linux-kernel@vger.kernel.org, pavel.tatashin@microsoft.com, jglisse@redhat.com, Jonathan.Cameron@huawei.com, rafael@kernel.org, david@redhat.com, dave.jiang@intel.com, Oscar Salvador Subject: [RFC PATCH v3 2/5] mm/memory_hotplug: Create add/del_device_memory functions Date: Tue, 2 Oct 2018 17:00:26 +0200 Message-Id: <20181002150029.23461-3-osalvador@techadventures.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181002150029.23461-1-osalvador@techadventures.net> References: <20181002150029.23461-1-osalvador@techadventures.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Oscar Salvador HMM/devm have a particular handling of memory-hotplug. They do not go through the common path, and so, they do not call either offline_pages() or online_pages(). The operations they perform are the following ones: 1) Create the linear mapping in case the memory is not private 2) Initialize the pages and add the sections 3) Move the pages to ZONE_DEVICE in case the memory is not private. Due to this particular handling of hot-add/remove memory from HMM/devm, I think it would be nice to provide a helper function in order to make this cleaner, and not populate other regions with code that should belong to memory-hotplug. The helpers are named: del_device_memory add_device_memory The idea is that add_device_memory will be in charge of: a) call either arch_add_memory() or add_pages(), depending on whether we want a linear mapping b) online the memory sections that correspond to the pfn range c) call move_pfn_range_to_zone() being zone ZONE_DEVICE to expand zone/pgdat spanned pages and initialize its pages del_device_memory, on the other hand, will be in charge of: a) offline the memory sections that correspond to the pfn range b) call shrink_zone_pgdat_pages(), which shrinks node/zone spanned pages. c) call either arch_remove_memory() or __remove_pages(), depending on whether we need to tear down the linear mapping or not In order to split up better the patches and ease the review, this patch will only make a) case work for add_device_memory(), and case c) for del_device_memory. The other cases will be added in the next patch. Since [1], hmm is using devm_memremap_pages and devm_memremap_pages_release instead of its own functions, so these two functions have to only be called from devm code. add_device_memory: - devm_memremap_pages() del_device_memory: - devm_memremap_pages_release() Another thing that this patch does is to move init_currently_empty_zone to be protected by the span_lock lock. Zone locking rules states the following: * Locking rules: * * zone_start_pfn and spanned_pages are protected by span_seqlock. * It is a seqlock because it has to be read outside of zone->lock, * and it is done in the main allocator path. But, it is written * quite infrequently. * Since init_currently_empty_zone changes zone_start_pfn, it makes sense to have it envolved by its lock. [1] https://patchwork.kernel.org/patch/10598657/ Signed-off-by: Oscar Salvador --- include/linux/memory_hotplug.h | 7 ++++++ kernel/memremap.c | 48 ++++++++++++++------------------------ mm/memory_hotplug.c | 53 +++++++++++++++++++++++++++++++++++++++--- 3 files changed, 74 insertions(+), 34 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index f9fc35819e65..2f7b8eb4cddb 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -117,6 +117,13 @@ extern int __remove_pages(struct zone *zone, unsigned long start_pfn, extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages, struct vmem_altmap *altmap, bool want_memblock); +#ifdef CONFIG_ZONE_DEVICE +extern int del_device_memory(int nid, unsigned long start, unsigned long size, + struct vmem_altmap *altmap, bool private_mem); +extern int add_device_memory(int nid, unsigned long start, unsigned long size, + struct vmem_altmap *altmap, bool private_mem); +#endif + #ifndef CONFIG_ARCH_HAS_ADD_PAGES static inline int add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages, struct vmem_altmap *altmap, diff --git a/kernel/memremap.c b/kernel/memremap.c index fe54bba2d7e2..0f168a75c5b0 100644 --- a/kernel/memremap.c +++ b/kernel/memremap.c @@ -120,8 +120,11 @@ static void devm_memremap_pages_release(void *data) struct device *dev = pgmap->dev; struct resource *res = &pgmap->res; resource_size_t align_start, align_size; + struct vmem_altmap *altmap = pgmap->altmap_valid ? + &pgmap->altmap : NULL; unsigned long pfn; int nid; + bool private_mem; pgmap->kill(pgmap->ref); for_each_device_pfn(pfn, pgmap) @@ -133,17 +136,14 @@ static void devm_memremap_pages_release(void *data) - align_start; nid = dev_to_node(dev); - mem_hotplug_begin(); - if (pgmap->type == MEMORY_DEVICE_PRIVATE) { - pfn = align_start >> PAGE_SHIFT; - __remove_pages(page_zone(pfn_to_page(pfn)), pfn, - align_size >> PAGE_SHIFT, NULL); - } else { - arch_remove_memory(nid, align_start, align_size, - pgmap->altmap_valid ? &pgmap->altmap : NULL); + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + private_mem = true; + else + private_mem = false; + + del_device_memory(nid, align_start, align_size, altmap, private_mem); + if (!private_mem) kasan_remove_zero_shadow(__va(align_start), align_size); - } - mem_hotplug_done(); untrack_pfn(NULL, PHYS_PFN(align_start), align_size); pgmap_radix_release(res, -1); @@ -180,6 +180,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) pgprot_t pgprot = PAGE_KERNEL; int error, nid, is_ram; struct dev_pagemap *conflict_pgmap; + bool private_mem; if (!pgmap->ref || !pgmap->kill) return ERR_PTR(-EINVAL); @@ -239,8 +240,6 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) if (error) goto err_pfn_remap; - mem_hotplug_begin(); - /* * For device private memory we call add_pages() as we only need to * allocate and initialize struct page for the device memory. More- @@ -252,29 +251,16 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) * the CPU, we do want the linear mapping and thus use * arch_add_memory(). */ - if (pgmap->type == MEMORY_DEVICE_PRIVATE) { - error = add_pages(nid, align_start >> PAGE_SHIFT, - align_size >> PAGE_SHIFT, NULL, false); - } else { + if (pgmap->type == MEMORY_DEVICE_PRIVATE) + private_mem = true; + else { error = kasan_add_zero_shadow(__va(align_start), align_size); - if (error) { - mem_hotplug_done(); + if (error) goto err_kasan; - } - - error = arch_add_memory(nid, align_start, align_size, altmap, - false); - } - - if (!error) { - struct zone *zone; - - zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; - move_pfn_range_to_zone(zone, align_start >> PAGE_SHIFT, - align_size >> PAGE_SHIFT, altmap); + private_mem = false; } - mem_hotplug_done(); + error = add_device_memory(nid, align_start, align_size, altmap, private_mem); if (error) goto err_add_memory; diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 11b7dcf83323..72928808c5e9 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -764,14 +764,13 @@ void __ref move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn, int nid = pgdat->node_id; unsigned long flags; - if (zone_is_empty(zone)) - init_currently_empty_zone(zone, start_pfn, nr_pages); - clear_zone_contiguous(zone); /* TODO Huh pgdat is irqsave while zone is not. It used to be like that before */ pgdat_resize_lock(pgdat, &flags); zone_span_writelock(zone); + if (zone_is_empty(zone)) + init_currently_empty_zone(zone, start_pfn, nr_pages); resize_zone_range(zone, start_pfn, nr_pages); zone_span_writeunlock(zone); resize_pgdat_range(pgdat, start_pfn, nr_pages); @@ -1904,4 +1903,52 @@ void remove_memory(int nid, u64 start, u64 size) unlock_device_hotplug(); } EXPORT_SYMBOL_GPL(remove_memory); + +#ifdef CONFIG_ZONE_DEVICE +int del_device_memory(int nid, unsigned long start, unsigned long size, + struct vmem_altmap *altmap, bool private_mem) +{ + int ret; + unsigned long start_pfn = PHYS_PFN(start); + unsigned long nr_pages = size >> PAGE_SHIFT; + struct zone *zone = page_zone(pfn_to_page(pfn)); + + mem_hotplug_begin(); + + if (private_mem) + ret = __remove_pages(zone, start_pfn, nr_pages, NULL); + else + ret = arch_remove_memory(nid, start, size, altmap); + + mem_hotplug_done(); + + return ret; +} +#endif #endif /* CONFIG_MEMORY_HOTREMOVE */ + +#ifdef CONFIG_ZONE_DEVICE +int add_device_memory(int nid, unsigned long start, unsigned long size, + struct vmem_altmap *altmap, bool private_mem) +{ + int ret; + unsigned long start_pfn = PHYS_PFN(start); + unsigned long nr_pages = size >> PAGE_SHIFT; + + mem_hotplug_begin(); + + if (private_mem) + ret = add_pages(nid, start_pfn, nr_pages, NULL, false); + else + ret = arch_add_memory(nid, start, size, altmap, false); + + mem_hotplug_done(); + + if (!ret) { + struct zone *zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; + move_pfn_range_to_zone(zone, start_pfn, nr_pages, altmap); + } + + return ret; +} +#endif From patchwork Tue Oct 2 15:00:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10623817 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A659415A7 for ; Tue, 2 Oct 2018 15:00:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96DBA20683 for ; Tue, 2 Oct 2018 15:00:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9400621BED; Tue, 2 Oct 2018 15:00:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C0C3212BE for ; Tue, 2 Oct 2018 15:00:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34A646B0008; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2F8E96B000D; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1BF6C6B000E; Tue, 2 Oct 2018 11:00:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id B1D816B0008 for ; Tue, 2 Oct 2018 11:00:52 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id v33-v6so1873700wrc.13 for ; Tue, 02 Oct 2018 08:00:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Lwbd+UcWSdJD5I10DIEnUzZN/CNn2JSB+Sr2UG85mb8=; b=OYnLmYxBQgaXRVs+UUby5z8GMu7MFpQXXpNfRZVr1NJW3MUfQrm2r9V7wGj7/0bYxl U8Gm8TGZ2eChik5a8kvEFMUDuaYN5zWrVYf7NRqJqu+QiCxLuBnbZTfBIWiSFoTL3QAZ Eaj+ycWY3875r2/4GczeU1p0DnHuStm105lxkdNpwVCsoIgJAFenUBh8l4rLU0UDAwmr 3WM82NYekPWBL75EIgysJBwigL6inxCRWbVtyt8m3dXtLFTJyI7D5kWV5DDbZbVoEAZ2 E1RYc/WKPpORu27hVvfm8n/NmSL48onbjLSkLcJQC+AOIhWQPOltrmcVzNq5po+ucl2s oMbg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: ABuFfoicru9mYxJynYeMYLnkNSMurxtdpBqMCHhA8JgyHalW99XMHQHo b8p+FhRC3sPWyTvR4r/s3NLJVaYLFI++MVnVGjHvteq8csFh+JkOZKNbf/OiCQaV6d68IJv8y63 4fZS8ykAuRYihaisOQ1qH1AZozqXSc3kkCgwBQ6jZk1TzjNYVRNJciw6QDhpjCBKB+VRLcNRyz3 2lcPTRx6n/1HPzhhLWZnHQrgzQtSPuVk6qWVHPofwZHLHrty31TNDxmDxqP3nUX97G47xodYJpt wJe4M6S2F1d6k3xe+1MxFZEaw9oXnuFYamsaBb9gc5intBOZYRfL36SdDrKCpk+5tI5A5ysYYxv eFtDorXeWQtY0cX5NyRdEmMAmupk/cWsrujeT+skW6ljoOxi+InIay6gA07yb0up6pBX2eYZrA= = X-Received: by 2002:a1c:2052:: with SMTP id g79-v6mr2246628wmg.42.1538492451960; Tue, 02 Oct 2018 08:00:51 -0700 (PDT) X-Received: by 2002:a1c:2052:: with SMTP id g79-v6mr2246510wmg.42.1538492450082; Tue, 02 Oct 2018 08:00:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538492450; cv=none; d=google.com; s=arc-20160816; b=CMDB0B9WMoakokxbFK6fgLoEhDNUtWJGbZzS8D92xYhycoHqc2uLTBIVEQq9vLYpeX 5JxnT3naW5BXpWmXgf2+iB5Eqird6qNXT50kTW8aCPyn5V42MxQ3N7Z4yYTTOJToot5u lJn1yoDx113ndgfEVwEkDxDK1/Qmb4c81SHJBiyO/u+dt7qlJlBYJLGhDvo72u59oyiR s7iXzw04KJKeM/ZQmnAYODrH39LaC/nuACxPXvYbTbpCMoNnKTSyGdR0wN44X3oleWJ7 +PpLmnT5sE6pb4wc8C5LhZKjRsli5lF9z6aKhjgQlkwkS2GUQiYeO+oEDy1ccMtGc/XT vKqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Lwbd+UcWSdJD5I10DIEnUzZN/CNn2JSB+Sr2UG85mb8=; b=MrWloqFpGpA4WNJE2ti2IDc3zRJQrjhrDd9u0eI0G3E66Ahf24s3BDP5IzSradoG2V pBUnrhEqAlgZ63PLEwNknb/YMzzXIxSfjisEhTrc2FCFbiSBxvccOzL0mmyK+k1hmLic iGmwId56OD/ZoVGWKwcQahOy1INvp1Rd1QEKiVoR/A9Jb1bJtnd2pNEwieU15y80gZ8b Vz9koYWhAgCEHQSYI2gHjEejwBp9z914UT4v8XRCyZkN4uvEfJeiNhdmg4KAOEB+j23g K5qi1cttlM2yCoH1vCpw1OMzjBQl1DLKysoIMCcv1GDOqF3k37VzSeNCrTZF7QfQXGnt TKMg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a12-v6sor8251259wrc.42.2018.10.02.08.00.49 for (Google Transport Security); Tue, 02 Oct 2018 08:00:50 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ACcGV612eFxvdBhOaQBF5tFOJo32uiL1RlMkLE+CfelcEbtS8cxLLU0cgvxMMiBOfvjwD0RCQXjugg== X-Received: by 2002:adf:eec4:: with SMTP id a4-v6mr11645875wrp.242.1538492449727; Tue, 02 Oct 2018 08:00:49 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id z89-v6sm37733989wrb.3.2018.10.02.08.00.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 08:00:48 -0700 (PDT) Received: from d104.suse.de (charybdis-ext.suse.de [195.135.221.2]) by techadventures.net (Postfix) with ESMTPA id A677612572A; Tue, 2 Oct 2018 17:00:47 +0200 (CEST) From: Oscar Salvador To: linux-mm@kvack.org Cc: mhocko@suse.com, dan.j.williams@intel.com, yasu.isimatu@gmail.com, rppt@linux.vnet.ibm.com, malat@debian.org, linux-kernel@vger.kernel.org, pavel.tatashin@microsoft.com, jglisse@redhat.com, Jonathan.Cameron@huawei.com, rafael@kernel.org, david@redhat.com, dave.jiang@intel.com, Oscar Salvador Subject: [RFC PATCH v3 3/5] mm/memory_hotplug: Check for IORESOURCE_SYSRAM in release_mem_region_adjustable Date: Tue, 2 Oct 2018 17:00:27 +0200 Message-Id: <20181002150029.23461-4-osalvador@techadventures.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181002150029.23461-1-osalvador@techadventures.net> References: <20181002150029.23461-1-osalvador@techadventures.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Oscar Salvador This is a preparation for the next patch. Currently, we only call release_mem_region_adjustable() in __remove_pages if the zone is not ZONE_DEVICE, because resources that belong to HMM/devm are being released by themselves with devm_release_mem_region. Since we do not want to touch any zone/page stuff during the removing of the memory (but during the offlining), we do not want to check for the zone here. So we need another way to tell release_mem_region_adjustable() to not realease the resource in case it belongs to HMM/devm. HMM/devm acquires/releases a resource through devm_request_mem_region/devm_release_mem_region. These resources have the flag IORESOURCE_MEM, while resources acquired by hot-add memory path (register_memory_resource()) contain IORESOURCE_SYSTEM_RAM. So, we can check for this flag in release_mem_region_adjustable, and if the resource does not contain such flag, we know that we are dealing with a HMM/devm resource, so we can back off. Signed-off-by: Oscar Salvador --- kernel/resource.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/kernel/resource.c b/kernel/resource.c index 81937830a42f..6956ce3a4730 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1272,6 +1272,21 @@ int release_mem_region_adjustable(struct resource *parent, continue; } + /* + * All memory regions added from memory-hotplug path + * have the flag IORESOURCE_SYSTEM_RAM. + * If the resource does not have this flag, we know that + * we are dealing with a resource coming from HMM/devm. + * HMM/devm use another mechanism to add/release a resource. + * This goes via devm_request_mem_region/devm_release_mem_region. + * HMM/devm take care to release their resources when they want, so + * if we are dealing with them, let us just back off here. + */ + if (!(res->flags & IORESOURCE_SYSRAM)) { + ret = 0; + break; + } + if (!(res->flags & IORESOURCE_MEM)) break; From patchwork Tue Oct 2 15:00:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10623823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2755A175A for ; Tue, 2 Oct 2018 15:01:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18395204BA for ; Tue, 2 Oct 2018 15:01:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0C550212D5; Tue, 2 Oct 2018 15:01:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF0DF204BA for ; Tue, 2 Oct 2018 15:01:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CB746B000D; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 67E6C6B0266; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51D056B0269; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id D00C56B000D for ; Tue, 2 Oct 2018 11:00:54 -0400 (EDT) Received: by mail-wm1-f71.google.com with SMTP id y203-v6so1825096wmg.9 for ; Tue, 02 Oct 2018 08:00:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=tT7E5iZo3Eil7nwkQZ6b6AbbDmpw+EG+ggUz5kXtCSo=; b=f68F3eAetD8QnBa561huD49TESbwN8M7BVEC2xPbkdkCBUlNX8CixV0gedbWSfQI2b DtkwE9aRKyI5UKJWS0Mwnp9G6sngDhSi2w57mZD34wkLS0CCtRNMRklRCiRuvjJ9M/IK 0R3b6aZnY7UH/v2NgIQk+7QwDOEXA33IZYnpxPNDT4qo+oHdaKR+/xaoD2jqiXGNK66J TCRpdD2gYR0cBigeFSmsWK3x/e2qxuXK1PMouW/VNupJq0XrqIdQtPXu/bIWz2bcFFq7 cc5IgrACIbKsXLcwruJWcLeuJXcN3tTCm4xrJWZGmIcU3KYaPG1WmpNsxtJ2/FKu8qJk jUzQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: ABuFfogcWX5jENBWtoE5JQPx0y7WjkEabxD5NtBbMpZCyvM/YZltk1hT 9cUw86Q5PHIipZQ7dq78TlckNobG3nRQqBDdyu++70L3QOYGffbpjKbm+fsKk37V+6wAtsz7XWs DUfvQZw+ldmvQk5V6HGijQhfSn8MB8DlDB2UWXecGDYK7e6TDEiIQ74z//Nw6eFiVvXv67F0Hpk 4qnMGn1K1nT5MDPvQGzz+A1WAWYFaTBUUYrsYQDEDN8MBR+dxCP8DHhgp0aAW8epIleVqpH0c1Q rwU+8u7GmlxL05Wnlw722XzSGOaBHjjn7cw36MiJHjEkNwz9PS5Ak7TnuLmTuGDmzkwR9dBL08l pYl8Viw3CC9BJ+1eQD+FC5/eVVWESWTf/JkI2n0rUQUrbRuw2qvVkK6nquV1sNCD9w9yZfhUvA= = X-Received: by 2002:a1c:5f54:: with SMTP id t81-v6mr2253520wmb.67.1538492453854; Tue, 02 Oct 2018 08:00:53 -0700 (PDT) X-Received: by 2002:a1c:5f54:: with SMTP id t81-v6mr2253319wmb.67.1538492451243; Tue, 02 Oct 2018 08:00:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538492451; cv=none; d=google.com; s=arc-20160816; b=Yyzwc9Kj705AwSEGwF88a6sAa9HXnKFyZud3iVPLb3rPr8SrbBUb5INgas0KFusGbv PPKI+bIy8cz/XxQ0jtHlFxqDNLsarTtnNtse3AVqD8MPwzaQVC3SgyyYoDjJClaoUthE aQ75p5KqdaGE8HClpzR/2oifQX3P0igqGI9ZP7DmFQTvvz+AWXKYbQwZR37pCNcZIB3R HtESe/YhkKW1R844M8s5bjUGVWi8HvhwixYawXLfqrkhD2tAQRWYTzzupPI61zxciKtt /92MkfeEv3Fc6tXuFAYJObRqZjbs1zSMwSSgOQQYNBuE0kuLatDppIkN1JOdb6WRPDHW NNMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=tT7E5iZo3Eil7nwkQZ6b6AbbDmpw+EG+ggUz5kXtCSo=; b=hnUgar9pb2ljoddPBAYTOHG92dwmUWgBQLsz3k9znGTlDDYYmSUjwGWxukrxNa8g3k 65WC0gbow3fkBY0OMpKYIQwsnh/oy+FWMUXUbHeT2TvbPkJVIxrWs8A688BNUzMKj+/I IeqmgIKbCss0qKNM3mW6WdI+vKyuWzuRSA/KwOiYsA0mk5HPbGU/YB86LegcrQ04tQxk 3N4E3Dt0nqIafBVsO2kVOFU+jZfW0O0GAK4q07Cz3b3Gk2wE0gyjT4yK+iTAL302XiEg G4bJ0BaGXKCWIAzkyixhjpuGPhPrjF0Ooc4E3pODTHXRPXbUDgtuqiH4oQuJJ2UpWk14 UwYQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 192-v6sor1464146wmw.0.2018.10.02.08.00.51 for (Google Transport Security); Tue, 02 Oct 2018 08:00:51 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ACcGV63pR3fSobiWsfoxVLDTCGJwc2/dnnzAYI5YzK6aKDoTttSCqUEVYeY0xTSOGxQxRcBY7erQPg== X-Received: by 2002:a1c:d10d:: with SMTP id i13-v6mr2107453wmg.68.1538492450285; Tue, 02 Oct 2018 08:00:50 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id g186-v6sm7623885wmf.19.2018.10.02.08.00.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 08:00:48 -0700 (PDT) Received: from d104.suse.de (charybdis-ext.suse.de [195.135.221.2]) by techadventures.net (Postfix) with ESMTPA id F054C12572B; Tue, 2 Oct 2018 17:00:47 +0200 (CEST) From: Oscar Salvador To: linux-mm@kvack.org Cc: mhocko@suse.com, dan.j.williams@intel.com, yasu.isimatu@gmail.com, rppt@linux.vnet.ibm.com, malat@debian.org, linux-kernel@vger.kernel.org, pavel.tatashin@microsoft.com, jglisse@redhat.com, Jonathan.Cameron@huawei.com, rafael@kernel.org, david@redhat.com, dave.jiang@intel.com, Oscar Salvador Subject: [RFC PATCH v3 4/5] mm/memory_hotplug: Move zone/pages handling to offline stage Date: Tue, 2 Oct 2018 17:00:28 +0200 Message-Id: <20181002150029.23461-5-osalvador@techadventures.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181002150029.23461-1-osalvador@techadventures.net> References: <20181002150029.23461-1-osalvador@techadventures.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Oscar Salvador Currently, we decrement zone/node spanned_pages during the hot-remove path. The picture we have now is: - hot-add memory: a) Allocate a new resouce based on the hot-added memory b) Add memory sections for the hot-added memory - online memory: c) Re-adjust zone/pgdat nr of pages (managed, spanned, present) d) Initialize the pages from the new memory-range e) Online memory sections - offline memory: f) Offline memory sections g) Re-adjust zone/pgdat nr of managed/present pages - hot-remove memory: i) Re-adjust zone/pgdat nr of spanned pages j) Remove memory sections k) Release resource This, besides of not being consistent with the current code, implies that we can access steal pages if we never get to online that memory. So we should move i) to the offline stage. Hot-remove path should only care about memory sections and memory blocks. There is a particularity and that is HMM/devm. When the memory is being handled by HMM/devm, this memory is moved to ZONE_DEVICE by means of move_pfn_range_to_zone, but since this memory does not get "online", the sections do not get online either. This is a problem because shrink_zone_pgdat_pages will now look for online sections, so we need to explicitly offline the sections before calling in. add_device_memory takes care of online them, and del_device_memory offlines them. Finally, shrink_zone_pgdat_pages() is moved to offline_pages(), so now, all pages/zone handling is being taken care in online/offline_pages stage. So now we will have: - hot-add memory: a) Allocate a new resource based on the hot-added memory b) Add memory sections for the hot-added memory - online memory: c) Re-adjust zone/pgdat nr of pages (managed, spanned, present) d) Initialize the pages from the new memory-range e) Online memory sections - offline memory: f) Offline memory sections g) Re-adjust zone/pgdat nr of managed/present/spanned pages - hot-remove memory: i) Remove memory sections j) Release resource Signed-off-by: Oscar Salvador --- arch/ia64/mm/init.c | 4 +- arch/powerpc/mm/mem.c | 11 +---- arch/sh/mm/init.c | 4 +- arch/x86/mm/init_32.c | 4 +- arch/x86/mm/init_64.c | 8 +--- include/linux/memory_hotplug.h | 8 ++-- mm/memory_hotplug.c | 100 +++++++++++++++++++---------------------- mm/sparse.c | 4 +- 8 files changed, 57 insertions(+), 86 deletions(-) diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c index 904fe55e10fc..a6b5f351620c 100644 --- a/arch/ia64/mm/init.c +++ b/arch/ia64/mm/init.c @@ -665,11 +665,9 @@ int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; - struct zone *zone; int ret; - zone = page_zone(pfn_to_page(start_pfn)); - ret = __remove_pages(zone, start_pfn, nr_pages, altmap); + ret = __remove_pages(nid, start_pfn, nr_pages, altmap); if (ret) pr_warn("%s: Problem encountered in __remove_pages() as" " ret=%d\n", __func__, ret); diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 445fce705f91..6d02171b2d0f 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -142,18 +142,9 @@ int __meminit arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altma { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; - struct page *page; int ret; - /* - * If we have an altmap then we need to skip over any reserved PFNs - * when querying the zone. - */ - page = pfn_to_page(start_pfn); - if (altmap) - page += vmem_altmap_offset(altmap); - - ret = __remove_pages(page_zone(page), start_pfn, nr_pages, altmap); + ret = __remove_pages(nid, start_pfn, nr_pages, altmap); if (ret) return ret; diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c index a8e5c0e00fca..6e80a7a50f8b 100644 --- a/arch/sh/mm/init.c +++ b/arch/sh/mm/init.c @@ -447,11 +447,9 @@ int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = PFN_DOWN(start); unsigned long nr_pages = size >> PAGE_SHIFT; - struct zone *zone; int ret; - zone = page_zone(pfn_to_page(start_pfn)); - ret = __remove_pages(zone, start_pfn, nr_pages, altmap); + ret = __remove_pages(nid, start_pfn, nr_pages, altmap); if (unlikely(ret)) pr_warn("%s: Failed, __remove_pages() == %d\n", __func__, ret); diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index b2a698d87a0e..72f403816053 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -864,10 +864,8 @@ int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap) { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; - struct zone *zone; - zone = page_zone(pfn_to_page(start_pfn)); - return __remove_pages(zone, start_pfn, nr_pages, altmap); + return __remove_pages(nid, start_pfn, nr_pages, altmap); } #endif #endif diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index c754d9543ae1..9872307d9c88 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1151,15 +1151,9 @@ int __ref arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *a { unsigned long start_pfn = start >> PAGE_SHIFT; unsigned long nr_pages = size >> PAGE_SHIFT; - struct page *page = pfn_to_page(start_pfn); - struct zone *zone; int ret; - /* With altmap the first mapped page is offset from @start */ - if (altmap) - page += vmem_altmap_offset(altmap); - zone = page_zone(page); - ret = __remove_pages(zone, start_pfn, nr_pages, altmap); + ret = __remove_pages(nid, start_pfn, nr_pages, altmap); WARN_ON_ONCE(ret); kernel_physical_mapping_remove(start, start + size); diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index 2f7b8eb4cddb..90c97a843094 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -109,8 +109,8 @@ static inline bool movable_node_is_enabled(void) #ifdef CONFIG_MEMORY_HOTREMOVE extern int arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap); -extern int __remove_pages(struct zone *zone, unsigned long start_pfn, - unsigned long nr_pages, struct vmem_altmap *altmap); +extern int __remove_pages(int nid, unsigned long start_pfn, + unsigned long nr_pages, struct vmem_altmap *altmap); #endif /* CONFIG_MEMORY_HOTREMOVE */ /* reasonably generic interface to expand the physical pages */ @@ -342,8 +342,8 @@ extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages); extern bool is_memblock_offlined(struct memory_block *mem); extern int sparse_add_one_section(struct pglist_data *pgdat, unsigned long start_pfn, struct vmem_altmap *altmap); -extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms, - unsigned long map_offset, struct vmem_altmap *altmap); +extern void sparse_remove_one_section(int nid, struct mem_section *ms, + unsigned long map_offset, struct vmem_altmap *altmap); extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map, unsigned long pnum); extern bool allow_online_pfn_range(int nid, unsigned long pfn, unsigned long nr_pages, diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 72928808c5e9..1f71aebd598b 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -319,12 +319,10 @@ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone, unsigned long start_pfn, unsigned long end_pfn) { - struct mem_section *ms; - for (; start_pfn < end_pfn; start_pfn += PAGES_PER_SECTION) { - ms = __pfn_to_section(start_pfn); + struct mem_section *ms = __pfn_to_section(start_pfn); - if (unlikely(!valid_section(ms))) + if (unlikely(!online_section(ms))) continue; if (unlikely(pfn_to_nid(start_pfn) != nid)) @@ -344,15 +342,14 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone, unsigned long start_pfn, unsigned long end_pfn) { - struct mem_section *ms; unsigned long pfn; /* pfn is the end pfn of a memory section. */ pfn = end_pfn - 1; for (; pfn >= start_pfn; pfn -= PAGES_PER_SECTION) { - ms = __pfn_to_section(pfn); + struct mem_section *ms = __pfn_to_section(pfn); - if (unlikely(!valid_section(ms))) + if (unlikely(!online_section(ms))) continue; if (unlikely(pfn_to_nid(pfn) != nid)) @@ -414,7 +411,7 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, for (; pfn < zone_end_pfn; pfn += PAGES_PER_SECTION) { ms = __pfn_to_section(pfn); - if (unlikely(!valid_section(ms))) + if (unlikely(!online_section(ms))) continue; if (page_zone(pfn_to_page(pfn)) != zone) @@ -482,7 +479,7 @@ static void shrink_pgdat_span(struct pglist_data *pgdat, for (; pfn < pgdat_end_pfn; pfn += PAGES_PER_SECTION) { ms = __pfn_to_section(pfn); - if (unlikely(!valid_section(ms))) + if (unlikely(!online_section(ms))) continue; if (pfn_to_nid(pfn) != nid) @@ -501,23 +498,31 @@ static void shrink_pgdat_span(struct pglist_data *pgdat, pgdat->node_spanned_pages = 0; } -static void __remove_zone(struct zone *zone, unsigned long start_pfn) +static void shrink_zone_pgdat_pages(struct zone *zone, unsigned long start_pfn, + unsigned long end_pfn, unsigned long offlined_pages) { struct pglist_data *pgdat = zone->zone_pgdat; int nr_pages = PAGES_PER_SECTION; unsigned long flags; + unsigned long pfn; - pgdat_resize_lock(zone->zone_pgdat, &flags); - shrink_zone_span(zone, start_pfn, start_pfn + nr_pages); - shrink_pgdat_span(pgdat, start_pfn, start_pfn + nr_pages); - pgdat_resize_unlock(zone->zone_pgdat, &flags); + zone->present_pages -= offlined_pages; + clear_zone_contiguous(zone); + + pgdat_resize_lock(pgdat, &flags); + pgdat->node_present_pages -= offlined_pages; + for (pfn = start_pfn; pfn < end_pfn; pfn += nr_pages) { + shrink_zone_span(zone, pfn, pfn + nr_pages); + shrink_pgdat_span(pgdat, pfn, pfn + nr_pages); + } + pgdat_resize_unlock(pgdat, &flags); + + set_zone_contiguous(zone); } -static int __remove_section(struct zone *zone, struct mem_section *ms, +static int __remove_section(int nid, struct mem_section *ms, unsigned long map_offset, struct vmem_altmap *altmap) { - unsigned long start_pfn; - int scn_nr; int ret = -EINVAL; if (!valid_section(ms)) @@ -527,17 +532,13 @@ static int __remove_section(struct zone *zone, struct mem_section *ms, if (ret) return ret; - scn_nr = __section_nr(ms); - start_pfn = section_nr_to_pfn((unsigned long)scn_nr); - __remove_zone(zone, start_pfn); - - sparse_remove_one_section(zone, ms, map_offset, altmap); + sparse_remove_one_section(nid, ms, map_offset, altmap); return 0; } /** * __remove_pages() - remove sections of pages from a zone - * @zone: zone from which pages need to be removed + * @nid: nid from which pages belong to * @phys_start_pfn: starting pageframe (must be aligned to start of a section) * @nr_pages: number of pages to remove (must be multiple of section size) * @altmap: alternative device page map or %NULL if default memmap is used @@ -547,35 +548,27 @@ static int __remove_section(struct zone *zone, struct mem_section *ms, * sure that pages are marked reserved and zones are adjust properly by * calling offline_pages(). */ -int __remove_pages(struct zone *zone, unsigned long phys_start_pfn, - unsigned long nr_pages, struct vmem_altmap *altmap) +int __remove_pages(int nid, unsigned long phys_start_pfn, + unsigned long nr_pages, struct vmem_altmap *altmap) { unsigned long i; unsigned long map_offset = 0; int sections_to_remove, ret = 0; + resource_size_t start, size; - /* In the ZONE_DEVICE case device driver owns the memory region */ - if (is_dev_zone(zone)) { - if (altmap) - map_offset = vmem_altmap_offset(altmap); - } else { - resource_size_t start, size; - - start = phys_start_pfn << PAGE_SHIFT; - size = nr_pages * PAGE_SIZE; + start = phys_start_pfn << PAGE_SHIFT; + size = nr_pages * PAGE_SIZE; - ret = release_mem_region_adjustable(&iomem_resource, start, - size); - if (ret) { - resource_size_t endres = start + size - 1; + if (altmap) + map_offset = vmem_altmap_offset(altmap); - pr_warn("Unable to release resource <%pa-%pa> (%d)\n", - &start, &endres, ret); - } + ret = release_mem_region_adjustable(&iomem_resource, start, size); + if (ret) { + resource_size_t endres = start + size - 1; + pr_warn("Unable to release resource <%pa-%pa> (%d)\n", &start, + &endres, ret); } - clear_zone_contiguous(zone); - /* * We can only remove entire sections */ @@ -586,15 +579,13 @@ int __remove_pages(struct zone *zone, unsigned long phys_start_pfn, for (i = 0; i < sections_to_remove; i++) { unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION; - ret = __remove_section(zone, __pfn_to_section(pfn), map_offset, - altmap); + ret = __remove_section(nid, __pfn_to_section(pfn), map_offset, + altmap); map_offset = 0; if (ret) break; } - set_zone_contiguous(zone); - return ret; } #endif /* CONFIG_MEMORY_HOTREMOVE */ @@ -1580,7 +1571,6 @@ static int __ref __offline_pages(unsigned long start_pfn, unsigned long pfn, nr_pages; long offlined_pages; int ret, node; - unsigned long flags; unsigned long valid_start, valid_end; struct zone *zone; struct memory_notify arg; @@ -1658,11 +1648,9 @@ static int __ref __offline_pages(unsigned long start_pfn, undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); /* removal success */ adjust_managed_page_count(pfn_to_page(start_pfn), -offlined_pages); - zone->present_pages -= offlined_pages; - pgdat_resize_lock(zone->zone_pgdat, &flags); - zone->zone_pgdat->node_present_pages -= offlined_pages; - pgdat_resize_unlock(zone->zone_pgdat, &flags); + /* Here we will shrink zone/node's spanned/present_pages */ + shrink_zone_pgdat_pages(zone, valid_start, valid_end, offlined_pages); init_per_zone_wmark_min(); @@ -1911,12 +1899,15 @@ int del_device_memory(int nid, unsigned long start, unsigned long size, int ret; unsigned long start_pfn = PHYS_PFN(start); unsigned long nr_pages = size >> PAGE_SHIFT; - struct zone *zone = page_zone(pfn_to_page(pfn)); + struct zone *zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; mem_hotplug_begin(); + offline_mem_sections(start_pfn, start_pfn + nr_pages); + shrink_zone_pgdat_pages(zone, start_pfn, start_pfn + nr_pages, 0); + if (private_mem) - ret = __remove_pages(zone, start_pfn, nr_pages, NULL); + ret = __remove_pages(nid, start_pfn, nr_pages, NULL); else ret = arch_remove_memory(nid, start, size, altmap); @@ -1946,6 +1937,7 @@ int add_device_memory(int nid, unsigned long start, unsigned long size, if (!ret) { struct zone *zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; + online_mem_sections(start_pfn, start_pfn + nr_pages); move_pfn_range_to_zone(zone, start_pfn, nr_pages, altmap); } diff --git a/mm/sparse.c b/mm/sparse.c index c0788e3d8513..21d5f6ad0d14 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -766,12 +766,12 @@ static void free_section_usemap(struct page *memmap, unsigned long *usemap, free_map_bootmem(memmap); } -void sparse_remove_one_section(struct zone *zone, struct mem_section *ms, +void sparse_remove_one_section(int nid, struct mem_section *ms, unsigned long map_offset, struct vmem_altmap *altmap) { struct page *memmap = NULL; unsigned long *usemap = NULL, flags; - struct pglist_data *pgdat = zone->zone_pgdat; + struct pglist_data *pgdat = NODE_DATA(nid); pgdat_resize_lock(pgdat, &flags); if (ms->section_mem_map) { From patchwork Tue Oct 2 15:00:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 10623825 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60C9B175A for ; Tue, 2 Oct 2018 15:01:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 516D7205E9 for ; Tue, 2 Oct 2018 15:01:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45BAE20881; Tue, 2 Oct 2018 15:01:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0EF4B21E5A for ; Tue, 2 Oct 2018 15:01:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BABB16B0010; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B5EEA6B0266; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A277F6B0269; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 44F7C6B0010 for ; Tue, 2 Oct 2018 11:00:55 -0400 (EDT) Received: by mail-wm1-f70.google.com with SMTP id y199-v6so1836788wmc.6 for ; Tue, 02 Oct 2018 08:00:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=PhCT3TBAUk3AvVLhiqqr/i+tBqrnuE7S7sAJRx6RsvU=; b=jCOGz7cMulCfCYCZPdW3cNuDKW93ITKkTpogSbkr7v20mjTNcPhXTVgIga2fS7JHh6 m8sTpe1G1pgOntNP9vcNBpQh7OfIaeh4O0WafybLD2E66+hv3juk1S9DX8EdTHjPNC+d SfBA7rk0XyeMeZtiizFFZgxT0OE+Obck5td1IL1gnRIW7hPOavYaonGnH11wuCKg1vd7 Gc8iGZxJRfZRLYsymZ7cyfebnDsXjZQioSW1XaWwLDgnYJCzzpw/tlbjWORKT5WtJ4Yi 2tmQjHbYZ71NZ5XkKkR7RsRy0yD8A+4KK1SUrLonUCQdNw71rgt1+rc09+CWaYj/ZX4u 7s2A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Gm-Message-State: ABuFfoho4WusukSxn9pA9cRhnOsE3ftY/6DugidKNIup/x27ExxqHyQg dcGuIhpZrR1v0MXT5uRRiKw7l16q52VsS/IY3Zor0X8agrsTgo1njCJmqFLOD+C90QuI+AdO6z3 7EKAsDyBybQcJO09jxS6cRGDlKMxBXFZETUa+c3NqXH2mRF5wlIyN0Sr6ZYAVwZ8t5kzmxmjj+N mu6v5Mwl2+rM/ar8MGJpoI/O24AreZcMU5IEP3+znAgY0w+zsqCLpszqjQFj8//5xLuy169U1qw v5EhfBMbkiiMnCojXmnXYgMiwhrlgf7mas0J5m03gCCMauJUKC0ehyY9RInQT4xUw2KK75/+HN4 fjsZqIuiIFq69fB12tOpHrJlsseSnTemOI0MFvqhvUA2bXKaREICyPjiSbNpuxmlbHvHwHOf5g= = X-Received: by 2002:adf:8382:: with SMTP id 2-v6mr2680783wre.13.1538492454434; Tue, 02 Oct 2018 08:00:54 -0700 (PDT) X-Received: by 2002:adf:8382:: with SMTP id 2-v6mr2680565wre.13.1538492451680; Tue, 02 Oct 2018 08:00:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538492451; cv=none; d=google.com; s=arc-20160816; b=SC/XDCtXwIEjSQXu1tUKJhJnRLiU5JdYnlvgyoNE/65L8EZTeUcSXxtRxM/jDMHk9G qn+KrA4KdpnPk7Z9DjzJWiHHw/2OQCWzvxo8XhQ2Y3nM6Sfp54sUHQFM/nTLVQlJzqex CF+3teC0eA0JvzN2st/7fHxcCpNbHqDRrqYTHXtXUkv4uZxK2yviQ3AUxBJq6kyEXXk5 ctPqEh4M4iSRDxAIsmTOiYR5k94PRP1wKYHbEnrjeYkNdAhDuM9x3VBtJp37lM2yLOZi BycidIk+wyzpNbpYgeigBy3ZcIe6hBH9qHRh0OosRX0NIfq0WK8lFJrptlRSNzu90Y70 qMEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=PhCT3TBAUk3AvVLhiqqr/i+tBqrnuE7S7sAJRx6RsvU=; b=vdDPMT9zPv3Gla1+QbSkxRVzqPJ7uZS1r2FmrNxJu7/NUnlgE3RIaUX3QVQfG1xvVE yr4jCvHXxcd6RHMcJ7iyKN/w2bgPoGuQKWERkjJ9oKelbh8ogtcOhdfWd7R0nKfsdiXr SY3JRPc6bA1uMA6OaOB7UCxA9cJtIp9yB/BBz1VajWThJijIdqlZK9qDBHOmq7DdnsIx wIQ2MI2tFEN+hObf5DN/c+eR9z4EdR1sOe8xnnl6CXBEWiK81zxpWkZVYHJXiTNsqWgf MYRmnrmQRuWm7ROnruomggg5miQC6/wdpL1HI7fl5/55b+ov+rW2Yv+iAslD31fjhw1M nGAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v3-v6sor3241442wrw.43.2018.10.02.08.00.51 for (Google Transport Security); Tue, 02 Oct 2018 08:00:51 -0700 (PDT) Received-SPF: pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of osalvador.vilardaga@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=osalvador.vilardaga@gmail.com X-Google-Smtp-Source: ACcGV62GC8tXsIt/wBzYRBj5U9Ujpuc1DEAM6x8ZaIfds4Yu9bzQsN3D+oR0SbAJPePtnFPKBG1bkw== X-Received: by 2002:adf:9c12:: with SMTP id f18-v6mr11165017wrc.93.1538492451294; Tue, 02 Oct 2018 08:00:51 -0700 (PDT) Received: from techadventures.net (techadventures.net. [62.201.165.239]) by smtp.gmail.com with ESMTPSA id q200-v6sm14232186wmd.2.2018.10.02.08.00.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Oct 2018 08:00:50 -0700 (PDT) Received: from d104.suse.de (charybdis-ext.suse.de [195.135.221.2]) by techadventures.net (Postfix) with ESMTPA id 444CE12572C; Tue, 2 Oct 2018 17:00:48 +0200 (CEST) From: Oscar Salvador To: linux-mm@kvack.org Cc: mhocko@suse.com, dan.j.williams@intel.com, yasu.isimatu@gmail.com, rppt@linux.vnet.ibm.com, malat@debian.org, linux-kernel@vger.kernel.org, pavel.tatashin@microsoft.com, jglisse@redhat.com, Jonathan.Cameron@huawei.com, rafael@kernel.org, david@redhat.com, dave.jiang@intel.com, Oscar Salvador Subject: [RFC PATCH v3 5/5] mm/memory-hotplug: Rework unregister_mem_sect_under_nodes Date: Tue, 2 Oct 2018 17:00:29 +0200 Message-Id: <20181002150029.23461-6-osalvador@techadventures.net> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181002150029.23461-1-osalvador@techadventures.net> References: <20181002150029.23461-1-osalvador@techadventures.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Oscar Salvador This tries to address another issue about accessing unitiliazed pages. Jonathan reported a problem [1] where we can access steal pages in case we hot-remove memory without onlining it first. This time is in unregister_mem_sect_under_nodes. This function tries to get the nid from the pfn and then tries to remove the symlink between mem_blk <-> nid and vice versa. Since we already know the nid in remove_memory(), we can pass it down the chain to unregister_mem_sect_under_nodes. There we can just remove the symlinks without the need to look into the pages. [1] https://www.spinics.net/lists/linux-mm/msg161316.html Signed-off-by: Oscar Salvador --- drivers/base/memory.c | 9 ++++----- drivers/base/node.c | 38 +++++++------------------------------- include/linux/memory.h | 2 +- include/linux/node.h | 7 ++----- mm/memory_hotplug.c | 2 +- 5 files changed, 15 insertions(+), 43 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 0e5985682642..3d8c65d84bea 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -744,8 +744,7 @@ unregister_memory(struct memory_block *memory) device_unregister(&memory->dev); } -static int remove_memory_section(unsigned long node_id, - struct mem_section *section, int phys_device) +static int remove_memory_section(unsigned long nid, struct mem_section *section) { struct memory_block *mem; @@ -759,7 +758,7 @@ static int remove_memory_section(unsigned long node_id, if (!mem) goto out_unlock; - unregister_mem_sect_under_nodes(mem, __section_nr(section)); + unregister_mem_sect_under_nodes(nid, mem); mem->section_count--; if (mem->section_count == 0) @@ -772,12 +771,12 @@ static int remove_memory_section(unsigned long node_id, return 0; } -int unregister_memory_section(struct mem_section *section) +int unregister_memory_section(int nid, struct mem_section *section) { if (!present_section(section)) return -EINVAL; - return remove_memory_section(0, section, 0); + return remove_memory_section(nid, section); } #endif /* CONFIG_MEMORY_HOTREMOVE */ diff --git a/drivers/base/node.c b/drivers/base/node.c index 86d6cd92ce3d..65bc5920bd3d 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -453,40 +453,16 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg) return 0; } -/* unregister memory section under all nodes that it spans */ -int unregister_mem_sect_under_nodes(struct memory_block *mem_blk, - unsigned long phys_index) +/* + * This mem_blk is going to be removed, so let us remove the link + * to the node and vice versa + */ +void unregister_mem_sect_under_nodes(int nid, struct memory_block *mem_blk) { - NODEMASK_ALLOC(nodemask_t, unlinked_nodes, GFP_KERNEL); - unsigned long pfn, sect_start_pfn, sect_end_pfn; - - if (!mem_blk) { - NODEMASK_FREE(unlinked_nodes); - return -EFAULT; - } - if (!unlinked_nodes) - return -ENOMEM; - nodes_clear(*unlinked_nodes); - - sect_start_pfn = section_nr_to_pfn(phys_index); - sect_end_pfn = sect_start_pfn + PAGES_PER_SECTION - 1; - for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) { - int nid; - - nid = get_nid_for_pfn(pfn); - if (nid < 0) - continue; - if (!node_online(nid)) - continue; - if (node_test_and_set(nid, *unlinked_nodes)) - continue; - sysfs_remove_link(&node_devices[nid]->dev.kobj, + sysfs_remove_link(&node_devices[nid]->dev.kobj, kobject_name(&mem_blk->dev.kobj)); - sysfs_remove_link(&mem_blk->dev.kobj, + sysfs_remove_link(&mem_blk->dev.kobj, kobject_name(&node_devices[nid]->dev.kobj)); - } - NODEMASK_FREE(unlinked_nodes); - return 0; } int link_mem_sections(int nid, unsigned long start_pfn, unsigned long end_pfn) diff --git a/include/linux/memory.h b/include/linux/memory.h index a6ddefc60517..d75ec88ca09d 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -113,7 +113,7 @@ extern int register_memory_isolate_notifier(struct notifier_block *nb); extern void unregister_memory_isolate_notifier(struct notifier_block *nb); int hotplug_memory_register(int nid, struct mem_section *section); #ifdef CONFIG_MEMORY_HOTREMOVE -extern int unregister_memory_section(struct mem_section *); +extern int unregister_memory_section(int nid, struct mem_section *); #endif extern int memory_dev_init(void); extern int memory_notify(unsigned long val, void *v); diff --git a/include/linux/node.h b/include/linux/node.h index 257bb3d6d014..e8aa9e6d95f9 100644 --- a/include/linux/node.h +++ b/include/linux/node.h @@ -72,8 +72,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid); extern int unregister_cpu_under_node(unsigned int cpu, unsigned int nid); extern int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg); -extern int unregister_mem_sect_under_nodes(struct memory_block *mem_blk, - unsigned long phys_index); +extern void unregister_mem_sect_under_nodes(int nid, struct memory_block *mem_blk); #ifdef CONFIG_HUGETLBFS extern void register_hugetlbfs_with_node(node_registration_func_t doregister, @@ -105,10 +104,8 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, { return 0; } -static inline int unregister_mem_sect_under_nodes(struct memory_block *mem_blk, - unsigned long phys_index) +static inline void unregister_mem_sect_under_nodes(int nid, struct memory_block *mem_blk) { - return 0; } static inline void register_hugetlbfs_with_node(node_registration_func_t reg, diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 1f71aebd598b..e7a38471fdc2 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -528,7 +528,7 @@ static int __remove_section(int nid, struct mem_section *ms, if (!valid_section(ms)) return ret; - ret = unregister_memory_section(ms); + ret = unregister_memory_section(nid, ms); if (ret) return ret;