From patchwork Fri Jul 31 09:18:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11694527 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C96EF913 for ; Fri, 31 Jul 2020 09:19:59 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A55122245C for ; Fri, 31 Jul 2020 09:19:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UJSVVoXW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A55122245C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBt-0000W5-TC; Fri, 31 Jul 2020 09:19:01 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBs-0000Vm-4g for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 09:19:00 +0000 X-Inumbo-ID: dab94e05-d30e-11ea-8e1f-bc764e2007e4 Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.81]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id dab94e05-d30e-11ea-8e1f-bc764e2007e4; Fri, 31 Jul 2020 09:18:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1596187134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nm9hdEOPm77EQVaUlUcV47hpK3bGj715U/FFPYWuGso=; b=UJSVVoXWWXWnPd07mj93j8NHefizfRRtku8wBHOJuMlX142lwynJHx32O4stSDZynh4sxs DxHxlGwM0vgm0x9fHuUCI+PF56uv27CrzoO/iXQi4KslGuZ4WqbupwPp5lYmZkZlro7zBW gW0s6HIIZScVS22Aj+wC2Fwnubi5Y40= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-423-eybjEKR1P9C14MolFOWKEA-1; Fri, 31 Jul 2020 05:18:51 -0400 X-MC-Unique: eybjEKR1P9C14MolFOWKEA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0D2E6101C8A7; Fri, 31 Jul 2020 09:18:50 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-22.ams2.redhat.com [10.36.113.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 564781A835; Fri, 31 Jul 2020 09:18:47 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH RFCv1 1/5] kernel/resource: make release_mem_region_adjustable() never fail Date: Fri, 31 Jul 2020 11:18:34 +0200 Message-Id: <20200731091838.7490-2-david@redhat.com> In-Reply-To: <20200731091838.7490-1-david@redhat.com> References: <20200731091838.7490-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Jason Gunthorpe , linux-hyperv@vger.kernel.org, Michal Hocko , Kees Cook , David Hildenbrand , Ard Biesheuvel , virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Wei Yang , xen-devel@lists.xenproject.org, Andrew Morton , Dan Williams Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's make sure splitting a resource on memory hotunplug will never fail. This will become more relevant once we merge selected system ram resources - then, we'll trigger that case more often un memory unplug. In general, this function is already unlikely to fail. When we remove memory, we free up quite a lot of metadata (memmap, page tables, memory block device, etc.). All other error cases inside release_mem_region_adjustable() seem to be sanity checks if the function would be abused in different context - let's add WARN_ON_ONCE(). Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 4 ++-- kernel/resource.c | 49 ++++++++++++++++++++++++------------------ mm/memory_hotplug.c | 22 +------------------ 3 files changed, 31 insertions(+), 44 deletions(-) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 6c2b06fe8beb7..52a91f5fa1a36 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -248,8 +248,8 @@ extern struct resource * __request_region(struct resource *, extern void __release_region(struct resource *, resource_size_t, resource_size_t); #ifdef CONFIG_MEMORY_HOTREMOVE -extern int release_mem_region_adjustable(struct resource *, resource_size_t, - resource_size_t); +extern void release_mem_region_adjustable(struct resource *, resource_size_t, + resource_size_t); #endif /* Wrappers for managed devices */ diff --git a/kernel/resource.c b/kernel/resource.c index 841737bbda9e5..249c6b54014de 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1255,21 +1255,28 @@ EXPORT_SYMBOL(__release_region); * assumes that all children remain in the lower address entry for * simplicity. Enhance this logic when necessary. */ -int release_mem_region_adjustable(struct resource *parent, - resource_size_t start, resource_size_t size) +void release_mem_region_adjustable(struct resource *parent, + resource_size_t start, resource_size_t size) { + struct resource *new_res = NULL; + bool alloc_nofail = false; struct resource **p; struct resource *res; - struct resource *new_res; resource_size_t end; - int ret = -EINVAL; end = start + size - 1; - if ((start < parent->start) || (end > parent->end)) - return ret; + if (WARN_ON_ONCE((start < parent->start) || (end > parent->end))) + return; - /* The alloc_resource() result gets checked later */ - new_res = alloc_resource(GFP_KERNEL); + /* + * We free up quite a lot of memory on memory hotunplug (esp. memap), + * just before releasing the region. This is highly unlikely to + * fail - let's play save and make it never fail as the caller cannot + * perform any error handling (e.g., trying to re-add memory will fail + * similarly). + */ +retry: + new_res = alloc_resource(GFP_KERNEL | alloc_nofail ? __GFP_NOFAIL : 0); p = &parent->child; write_lock(&resource_lock); @@ -1295,7 +1302,6 @@ int release_mem_region_adjustable(struct resource *parent, * so if we are dealing with them, let us just back off here. */ if (!(res->flags & IORESOURCE_SYSRAM)) { - ret = 0; break; } @@ -1312,20 +1318,23 @@ int release_mem_region_adjustable(struct resource *parent, /* free the whole entry */ *p = res->sibling; free_resource(res); - ret = 0; } else if (res->start == start && res->end != end) { /* adjust the start */ - ret = __adjust_resource(res, end + 1, - res->end - end); + WARN_ON_ONCE(__adjust_resource(res, end + 1, + res->end - end)); } else if (res->start != start && res->end == end) { /* adjust the end */ - ret = __adjust_resource(res, res->start, - start - res->start); + WARN_ON_ONCE(__adjust_resource(res, res->start, + start - res->start)); } else { - /* split into two entries */ + /* split into two entries - we need a new resource */ if (!new_res) { - ret = -ENOMEM; - break; + new_res = alloc_resource(GFP_ATOMIC); + if (!new_res) { + alloc_nofail = true; + write_unlock(&resource_lock); + goto retry; + } } new_res->name = res->name; new_res->start = end + 1; @@ -1336,9 +1345,8 @@ int release_mem_region_adjustable(struct resource *parent, new_res->sibling = res->sibling; new_res->child = NULL; - ret = __adjust_resource(res, res->start, - start - res->start); - if (ret) + if (WARN_ON_ONCE(__adjust_resource(res, res->start, + start - res->start))) break; res->sibling = new_res; new_res = NULL; @@ -1349,7 +1357,6 @@ int release_mem_region_adjustable(struct resource *parent, write_unlock(&resource_lock); free_resource(new_res); - return ret; } #endif /* CONFIG_MEMORY_HOTREMOVE */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index da374cd3d45b3..258656b819dbe 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1709,26 +1709,6 @@ void try_offline_node(int nid) } EXPORT_SYMBOL(try_offline_node); -static void __release_memory_resource(resource_size_t start, - resource_size_t size) -{ - int ret; - - /* - * When removing memory in the same granularity as it was added, - * this function never fails. It might only fail if resources - * have to be adjusted or split. We'll ignore the error, as - * removing of memory cannot fail. - */ - ret = release_mem_region_adjustable(&iomem_resource, start, size); - if (ret) { - resource_size_t endres = start + size - 1; - - pr_warn("Unable to release resource <%pa-%pa> (%d)\n", - &start, &endres, ret); - } -} - static int __ref try_remove_memory(int nid, u64 start, u64 size) { int rc = 0; @@ -1762,7 +1742,7 @@ static int __ref try_remove_memory(int nid, u64 start, u64 size) memblock_remove(start, size); } - __release_memory_resource(start, size); + release_mem_region_adjustable(&iomem_resource, start, size); try_offline_node(nid); From patchwork Fri Jul 31 09:18:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11694533 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 241B6912 for ; Fri, 31 Jul 2020 09:20:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 008A72074B for ; Fri, 31 Jul 2020 09:20:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gwyJn3jU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 008A72074B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBz-0000X0-Hx; Fri, 31 Jul 2020 09:19:07 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBx-0000Wj-Tl for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 09:19:05 +0000 X-Inumbo-ID: dff4855a-d30e-11ea-ab95-12813bfff9fa Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.120]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id dff4855a-d30e-11ea-ab95-12813bfff9fa; Fri, 31 Jul 2020 09:19:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1596187142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GX38+opkWxBZfD2far/R2rRkc2X2qlfoUgt4pP5SXDg=; b=gwyJn3jU7N/REiKzRAhgTbpkjsF3/nUITttfsKZRuHB1p23+rMMRAwX0ywfGW4cRJyE5x9 AobB+bvssCqavcJHHZaS6yRmHlBo7MzkztYsCI4ZKmtE8AddPcP19Y7FpfEnil8oK66xtL Wmn+xyRPyWX32GAtaulDphrYkhHmomY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-301-Z7A2oVdiNTiak_-Gh5Vfug-1; Fri, 31 Jul 2020 05:18:58 -0400 X-MC-Unique: Z7A2oVdiNTiak_-Gh5Vfug-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4358480382D; Fri, 31 Jul 2020 09:18:55 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-22.ams2.redhat.com [10.36.113.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D0681A835; Fri, 31 Jul 2020 09:18:50 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH RFCv1 2/5] kernel/resource: merge_child_mem_resources() to merge memory resources after adding succeeded Date: Fri, 31 Jul 2020 11:18:35 +0200 Message-Id: <20200731091838.7490-3-david@redhat.com> In-Reply-To: <20200731091838.7490-1-david@redhat.com> References: <20200731091838.7490-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Boris Ostrovsky , Jason Gunthorpe , linux-hyperv@vger.kernel.org, Michal Hocko , Stephen Hemminger , Kees Cook , David Hildenbrand , Ard Biesheuvel , Haiyang Zhang , Wei Liu , virtualization@lists.linux-foundation.org, Juergen Gross , linux-mm@kvack.org, Stefano Stabellini , Thomas Gleixner , Julien Grall , xen-devel@lists.xenproject.org, Andrew Morton , "K. Y. Srinivasan" , Dan Williams , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Some add_memory*() users add memory in small, contiguous memory blocks. Examples include virtio-mem, hyper-v balloon, and the XEN balloon. This can quickly result in a lot of memory resources, whereby the actual resource boundaries are not of interest (e.g., it might be relevant for DIMMs, exposed via /proc/iomem to user space). We really want to merge added resources in this scenario where possible. Let's provide an interface to trigger merging of applicable child resources. It will be, for example, used by virtio-mem to trigger merging of memory resources it added (via add_memory_driver() managed) to its resource container. Note: We really want to merge after the whole operation succeeded, not directly when adding a resource to the resource tree (it would break add_memory_resource() and require splitting resources again when the operation failed - e.g., due to -ENOMEM). Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Thomas Gleixner Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Julien Grall Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 3 +++ kernel/resource.c | 56 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 52a91f5fa1a36..743b87fe2205b 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -251,6 +251,9 @@ extern void __release_region(struct resource *, resource_size_t, extern void release_mem_region_adjustable(struct resource *, resource_size_t, resource_size_t); #endif +#ifdef CONFIG_MEMORY_HOTPLUG +extern void merge_child_mem_resources(struct resource *res, const char *name); +#endif /* Wrappers for managed devices */ struct device; diff --git a/kernel/resource.c b/kernel/resource.c index 249c6b54014de..01ecc5b7956f5 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1360,6 +1360,62 @@ void release_mem_region_adjustable(struct resource *parent, } #endif /* CONFIG_MEMORY_HOTREMOVE */ +#ifdef CONFIG_MEMORY_HOTPLUG +static bool mem_resources_mergeable(struct resource *r1, struct resource *r2) +{ + return r1->end + 1 == r2->start && + r1->name == r2->name && + r1->flags == r2->flags && + (r1->flags & IORESOURCE_MEM) && + r1->desc == r2->desc && + !r1->child && !r2->child; +} + +/* + * merge_child_mem_resources - try to merge contiguous child IORESOURCE_MEM + * resources with the given name that match all + * other properties + * @parent: parent resource descriptor + * @name: name of the child resources to consider for merging + * + * This interface is intended for memory hotplug, whereby lots of consecutive + * memory resources are added (e.g., via add_memory*()) by a driver, and the + * actual resource boundaries are not of interest (e.g., it might be + * relevant for DIMMs). Only immediate child resources are considered. All + * applicable child resources must be immutable during the request. + * + * Note: + * - The caller has to make sure that no pointers to resources that might + * get merged are held anymore. Callers should only trigger merging of child + * resources when they are the only one adding such resources to the parent. + * E.g., if two mechanisms could add "System RAM" immediately below the + * same parent, this function is not safe to use. + * - release_mem_region_adjustable() will split on demand on memory hotunplug + */ +void merge_child_mem_resources(struct resource *parent, const char *name) +{ + struct resource *cur, *next; + + write_lock(&resource_lock); + + cur = parent->child; + while (cur && cur->sibling) { + next = cur->sibling; + if (!strcmp(cur->name, name) && + mem_resources_mergeable(cur, next)) { + cur->end = next->end; + cur->sibling = next->sibling; + free_resource(next); + next = cur->sibling; + } + cur = next; + } + + write_unlock(&resource_lock); +} +EXPORT_SYMBOL(merge_child_mem_resources); +#endif /* CONFIG_MEMORY_HOTPLUG */ + /* * Managed region resource */ From patchwork Fri Jul 31 09:18:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11694529 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B367A138C for ; Fri, 31 Jul 2020 09:20:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8E5212245C for ; Fri, 31 Jul 2020 09:20:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bJTt0i+i" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E5212245C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBy-0000Wp-9O; Fri, 31 Jul 2020 09:19:06 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RBx-0000Vm-4p for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 09:19:05 +0000 X-Inumbo-ID: de6814e0-d30e-11ea-8e1f-bc764e2007e4 Received: from us-smtp-1.mimecast.com (unknown [205.139.110.120]) by us1-rack-iad1.inumbo.com (Halon) with ESMTP id de6814e0-d30e-11ea-8e1f-bc764e2007e4; Fri, 31 Jul 2020 09:19:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1596187140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nOON8+ajaLpXIcBxCYJ6eMATKXtAWnIlcMGpRLJHnBo=; b=bJTt0i+iPUoecHL6hE6yF2u6x9u/VnocEw77lYLaQeSQ8bQ+VJLfFs84wLWoqDbvlHAHCj xjILaBJeVmtLrchGYQpBeEA4rhve19vwAU4KmfiCJXixltInJfy184Vg9QIkmWuV5suksV c7wfg/ODoXhy2UfPQVf3lkGWXgDfmpA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-14-a6wjUmLxO62-OZabls5-1w-1; Fri, 31 Jul 2020 05:18:58 -0400 X-MC-Unique: a6wjUmLxO62-OZabls5-1w-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DFFD618C63C0; Fri, 31 Jul 2020 09:18:56 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-22.ams2.redhat.com [10.36.113.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 919091C92D; Fri, 31 Jul 2020 09:18:55 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH RFCv1 3/5] virtio-mem: try to merge "System RAM (virtio_mem)" resources Date: Fri, 31 Jul 2020 11:18:36 +0200 Message-Id: <20200731091838.7490-4-david@redhat.com> In-Reply-To: <20200731091838.7490-1-david@redhat.com> References: <20200731091838.7490-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: linux-mm@kvack.org, linux-hyperv@vger.kernel.org, David Hildenbrand , xen-devel@lists.xenproject.org, virtualization@lists.linux-foundation.org Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of resources not only wastes memory, it also makes traversing that tree more expensive, and makes /proc/iomem explode in size (e.g., requiring kexec-tools to manually merge resources later when e.g., trying to create a kdump header). Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem on x86-64: [...] 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-147ffffff : System RAM (virtio_mem) 148000000-14fffffff : System RAM (virtio_mem) 150000000-157ffffff : System RAM (virtio_mem) 158000000-15fffffff : System RAM (virtio_mem) 160000000-167ffffff : System RAM (virtio_mem) 168000000-16fffffff : System RAM (virtio_mem) 170000000-177ffffff : System RAM (virtio_mem) 178000000-17fffffff : System RAM (virtio_mem) 180000000-187ffffff : System RAM (virtio_mem) 188000000-18fffffff : System RAM (virtio_mem) 190000000-197ffffff : System RAM (virtio_mem) 198000000-19fffffff : System RAM (virtio_mem) 1a0000000-1a7ffffff : System RAM (virtio_mem) 1a8000000-1afffffff : System RAM (virtio_mem) 1b0000000-1b7ffffff : System RAM (virtio_mem) 1b8000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 With this patch, we get (/proc/iomem): [...] fffc0000-ffffffff : Reserved 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 Of course, with more hotplugged memory, it gets worse. When unplugging memory blocks again, try_remove_memory() (via offline_and_remove_memory()) will properly split the resource up again. Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index f26f5f64ae822..2396a8d67875e 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -415,6 +415,7 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) { const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); int nid = vm->nid; + int rc; if (nid == NUMA_NO_NODE) nid = memory_add_physaddr_to_nid(addr); @@ -431,8 +432,17 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) } dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); - return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), - vm->resource_name); + rc = add_memory_driver_managed(nid, addr, memory_block_size_bytes(), + vm->resource_name); + if (!rc) { + /* + * Try to reduce the number of resources by merging them. The + * memory removal path will properly split them up again. + */ + merge_child_mem_resources(vm->parent_resource, + vm->resource_name); + } + return rc; } /* From patchwork Fri Jul 31 09:18:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11694531 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 39EA8138C for ; Fri, 31 Jul 2020 09:20:33 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 15D722074B for ; Fri, 31 Jul 2020 09:20:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eKz2GmRz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15D722074B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RC3-0000Xo-QH; Fri, 31 Jul 2020 09:19:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RC2-0000Wj-PZ for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 09:19:10 +0000 X-Inumbo-ID: e1ba0b76-d30e-11ea-ab95-12813bfff9fa Received: from us-smtp-delivery-1.mimecast.com (unknown [207.211.31.81]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id e1ba0b76-d30e-11ea-ab95-12813bfff9fa; Fri, 31 Jul 2020 09:19:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1596187145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WXo92L6xzVp7lbkKaGhQOH5PjMjS2UMyqhJq+SqZmjU=; b=eKz2GmRzt71ZPjgS2zsWodplxYWV8Uzf3VNCS/zSN/Ixo1Sf06HXasBEzxNvRSR6NI5iBj tYpCPWqZft+a7ljcRaO0FmaQrcFBIvLYbJPkPOnFrHjoTJSgq6YOthrjZn2m2ZFGiSD1FH TBP6Q25KFwaqg7pQ276qpbNqxVmDjLo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-403-Vj_nV24LO0GNaNZsmopdDQ-1; Fri, 31 Jul 2020 05:19:01 -0400 X-MC-Unique: Vj_nV24LO0GNaNZsmopdDQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A93C718C63C0; Fri, 31 Jul 2020 09:18:59 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-22.ams2.redhat.com [10.36.113.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 39C481A835; Fri, 31 Jul 2020 09:18:57 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH RFCv1 4/5] xen/balloon: try to merge "System RAM" resources Date: Fri, 31 Jul 2020 11:18:37 +0200 Message-Id: <20200731091838.7490-5-david@redhat.com> In-Reply-To: <20200731091838.7490-1-david@redhat.com> References: <20200731091838.7490-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , linux-hyperv@vger.kernel.org, Michal Hocko , Julien Grall , David Hildenbrand , virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Stefano Stabellini , xen-devel@lists.xenproject.org, Andrew Morton , Boris Ostrovsky , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's reuse the new mechanism to merge "System RAM" resources below the root. We are the only one hotplugging "System RAM" and DIMMs don't apply, so this is safe to use. Cc: Andrew Morton Cc: Michal Hocko Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Julien Grall Signed-off-by: David Hildenbrand --- drivers/xen/balloon.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index 77c57568e5d7f..644ae2e3798e2 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -353,6 +353,10 @@ static enum bp_state reserve_additional_memory(void) if (rc) { pr_warn("Cannot add additional memory (%i)\n", rc); goto err; + } else { + resource = NULL; + /* Try to reduce the number of "System RAM" resources. */ + merge_child_mem_resources(&iomem_resource, "System RAM"); } balloon_stats.total_pages += balloon_hotplug; From patchwork Fri Jul 31 09:18:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11694535 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A7FC138C for ; Fri, 31 Jul 2020 09:20:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 43DF1208E4 for ; Fri, 31 Jul 2020 09:20:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="D/HxuniB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43DF1208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RC9-0000al-2e; Fri, 31 Jul 2020 09:19:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1k1RC7-0000Wj-Pr for xen-devel@lists.xenproject.org; Fri, 31 Jul 2020 09:19:15 +0000 X-Inumbo-ID: e5293f84-d30e-11ea-ab95-12813bfff9fa Received: from us-smtp-1.mimecast.com (unknown [205.139.110.61]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id e5293f84-d30e-11ea-ab95-12813bfff9fa; Fri, 31 Jul 2020 09:19:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1596187151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ITjza8QZs334kZDawRKP4E/FGkviB+rOAxL++W/FFU=; b=D/HxuniBO2RN0kp9Zkm3YpHn+w9DEuPfw9K2pbvYWH1mKV4tntKnmq2yQC+y3q4a5W4M2A wyz2S6sSz0B4nIqvPCzpOee0ijPcp/TUvX96JrJtuytnVzA7uf6OuDlnZ2S+Sr0PE0ugtT jcPgO0U+4y+UWEBRrmiVTyTYQ/z92co= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-352-w-XOsNLgPyeml39HbUP41A-1; Fri, 31 Jul 2020 05:19:06 -0400 X-MC-Unique: w-XOsNLgPyeml39HbUP41A-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2F36F801504; Fri, 31 Jul 2020 09:19:05 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-22.ams2.redhat.com [10.36.113.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 035EC1A835; Fri, 31 Jul 2020 09:18:59 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH RFCv1 5/5] hv_balloon:: try to merge "System RAM" resources Date: Fri, 31 Jul 2020 11:18:38 +0200 Message-Id: <20200731091838.7490-6-david@redhat.com> In-Reply-To: <20200731091838.7490-1-david@redhat.com> References: <20200731091838.7490-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: linux-hyperv@vger.kernel.org, Michal Hocko , Stephen Hemminger , David Hildenbrand , Haiyang Zhang , Wei Liu , virtualization@lists.linux-foundation.org, linux-mm@kvack.org, xen-devel@lists.xenproject.org, Andrew Morton , "K. Y. Srinivasan" Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Let's reuse the new mechanism to merge "System RAM" resources below the root. We are the only one hotplugging "System RAM" and DIMMs don't apply, so this is safe to use. Cc: Andrew Morton Cc: Michal Hocko Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 32e3bc0aa665a..0745f7cc1727b 100644 --- a/drivers/hv/hv_balloon.c +++ b/drivers/hv/hv_balloon.c @@ -745,6 +745,9 @@ static void hv_mem_hot_add(unsigned long start, unsigned long size, has->covered_end_pfn -= processed_pfn; spin_unlock_irqrestore(&dm_device.ha_lock, flags); break; + } else { + /* Try to reduce the number of "System RAM" resources. */ + merge_child_mem_resources(&iomem_resource, "System RAM"); } /*