| Message ID | 20200731091838.7490-4-david@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | mm/memory_hotplug: selective merging of memory resources | expand |
On 31.07.20 11:18, David Hildenbrand wrote: Grml, forgot to add cc: list for this patch, ccing the right people. > virtio-mem adds memory in memory block granularity, to be able to > remove it in the same granularity again later, and to grow slowly on > demand. This, however, results in quite a lot of resources when > adding a lot of memory. Resources are effectively stored in a list-based > tree. Having a lot of resources not only wastes memory, it also makes > traversing that tree more expensive, and makes /proc/iomem explode in > size (e.g., requiring kexec-tools to manually merge resources later > when e.g., trying to create a kdump header). > > Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem > on x86-64: > [...] > 100000000-13fffffff : System RAM > 140000000-33fffffff : virtio0 > 140000000-147ffffff : System RAM (virtio_mem) > 148000000-14fffffff : System RAM (virtio_mem) > 150000000-157ffffff : System RAM (virtio_mem) > 158000000-15fffffff : System RAM (virtio_mem) > 160000000-167ffffff : System RAM (virtio_mem) > 168000000-16fffffff : System RAM (virtio_mem) > 170000000-177ffffff : System RAM (virtio_mem) > 178000000-17fffffff : System RAM (virtio_mem) > 180000000-187ffffff : System RAM (virtio_mem) > 188000000-18fffffff : System RAM (virtio_mem) > 190000000-197ffffff : System RAM (virtio_mem) > 198000000-19fffffff : System RAM (virtio_mem) > 1a0000000-1a7ffffff : System RAM (virtio_mem) > 1a8000000-1afffffff : System RAM (virtio_mem) > 1b0000000-1b7ffffff : System RAM (virtio_mem) > 1b8000000-1bfffffff : System RAM (virtio_mem) > 3280000000-32ffffffff : PCI Bus 0000:00 > > With this patch, we get (/proc/iomem): > [...] > fffc0000-ffffffff : Reserved > 100000000-13fffffff : System RAM > 140000000-33fffffff : virtio0 > 140000000-1bfffffff : System RAM (virtio_mem) > 3280000000-32ffffffff : PCI Bus 0000:00 > > Of course, with more hotplugged memory, it gets worse. When unplugging > memory blocks again, try_remove_memory() (via > offline_and_remove_memory()) will properly split the resource up again. > > Signed-off-by: David Hildenbrand <david@redhat.com> > --- > drivers/virtio/virtio_mem.c | 14 ++++++++++++-- > 1 file changed, 12 insertions(+), 2 deletions(-) > > diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c > index f26f5f64ae822..2396a8d67875e 100644 > --- a/drivers/virtio/virtio_mem.c > +++ b/drivers/virtio/virtio_mem.c > @@ -415,6 +415,7 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) > { > const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); > int nid = vm->nid; > + int rc; > > if (nid == NUMA_NO_NODE) > nid = memory_add_physaddr_to_nid(addr); > @@ -431,8 +432,17 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) > } > > dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); > - return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), > - vm->resource_name); > + rc = add_memory_driver_managed(nid, addr, memory_block_size_bytes(), > + vm->resource_name); > + if (!rc) { > + /* > + * Try to reduce the number of resources by merging them. The > + * memory removal path will properly split them up again. > + */ > + merge_child_mem_resources(vm->parent_resource, > + vm->resource_name); > + } > + return rc; > } > > /* >
diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index f26f5f64ae822..2396a8d67875e 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -415,6 +415,7 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) { const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); int nid = vm->nid; + int rc; if (nid == NUMA_NO_NODE) nid = memory_add_physaddr_to_nid(addr); @@ -431,8 +432,17 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) } dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); - return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), - vm->resource_name); + rc = add_memory_driver_managed(nid, addr, memory_block_size_bytes(), + vm->resource_name); + if (!rc) { + /* + * Try to reduce the number of resources by merging them. The + * memory removal path will properly split them up again. + */ + merge_child_mem_resources(vm->parent_resource, + vm->resource_name); + } + return rc; } /*
virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of resources not only wastes memory, it also makes traversing that tree more expensive, and makes /proc/iomem explode in size (e.g., requiring kexec-tools to manually merge resources later when e.g., trying to create a kdump header). Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem on x86-64: [...] 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-147ffffff : System RAM (virtio_mem) 148000000-14fffffff : System RAM (virtio_mem) 150000000-157ffffff : System RAM (virtio_mem) 158000000-15fffffff : System RAM (virtio_mem) 160000000-167ffffff : System RAM (virtio_mem) 168000000-16fffffff : System RAM (virtio_mem) 170000000-177ffffff : System RAM (virtio_mem) 178000000-17fffffff : System RAM (virtio_mem) 180000000-187ffffff : System RAM (virtio_mem) 188000000-18fffffff : System RAM (virtio_mem) 190000000-197ffffff : System RAM (virtio_mem) 198000000-19fffffff : System RAM (virtio_mem) 1a0000000-1a7ffffff : System RAM (virtio_mem) 1a8000000-1afffffff : System RAM (virtio_mem) 1b0000000-1b7ffffff : System RAM (virtio_mem) 1b8000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 With this patch, we get (/proc/iomem): [...] fffc0000-ffffffff : Reserved 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 Of course, with more hotplugged memory, it gets worse. When unplugging memory blocks again, try_remove_memory() (via offline_and_remove_memory()) will properly split the resource up again. Signed-off-by: David Hildenbrand <david@redhat.com> --- drivers/virtio/virtio_mem.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)