| Message ID | 20200911103459.10306-7-david@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | selective merging of system ram resources | expand |
On Fri, Sep 11, 2020 at 12:34:57PM +0200, David Hildenbrand wrote: >virtio-mem adds memory in memory block granularity, to be able to >remove it in the same granularity again later, and to grow slowly on >demand. This, however, results in quite a lot of resources when >adding a lot of memory. Resources are effectively stored in a list-based >tree. Having a lot of resources not only wastes memory, it also makes >traversing that tree more expensive, and makes /proc/iomem explode in >size (e.g., requiring kexec-tools to manually merge resources later >when e.g., trying to create a kdump header). > >Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem >on x86-64: > [...] > 100000000-13fffffff : System RAM > 140000000-33fffffff : virtio0 > 140000000-147ffffff : System RAM (virtio_mem) > 148000000-14fffffff : System RAM (virtio_mem) > 150000000-157ffffff : System RAM (virtio_mem) > 158000000-15fffffff : System RAM (virtio_mem) > 160000000-167ffffff : System RAM (virtio_mem) > 168000000-16fffffff : System RAM (virtio_mem) > 170000000-177ffffff : System RAM (virtio_mem) > 178000000-17fffffff : System RAM (virtio_mem) > 180000000-187ffffff : System RAM (virtio_mem) > 188000000-18fffffff : System RAM (virtio_mem) > 190000000-197ffffff : System RAM (virtio_mem) > 198000000-19fffffff : System RAM (virtio_mem) > 1a0000000-1a7ffffff : System RAM (virtio_mem) > 1a8000000-1afffffff : System RAM (virtio_mem) > 1b0000000-1b7ffffff : System RAM (virtio_mem) > 1b8000000-1bfffffff : System RAM (virtio_mem) > 3280000000-32ffffffff : PCI Bus 0000:00 > >With this patch, we get (/proc/iomem): > [...] > fffc0000-ffffffff : Reserved > 100000000-13fffffff : System RAM > 140000000-33fffffff : virtio0 > 140000000-1bfffffff : System RAM (virtio_mem) > 3280000000-32ffffffff : PCI Bus 0000:00 > >Of course, with more hotplugged memory, it gets worse. When unplugging >memory blocks again, try_remove_memory() (via >offline_and_remove_memory()) will properly split the resource up again. > >Reviewed-by: Pankaj Gupta <pankaj.gupta.linux@gmail.com> >Cc: Andrew Morton <akpm@linux-foundation.org> >Cc: Michal Hocko <mhocko@suse.com> >Cc: Dan Williams <dan.j.williams@intel.com> >Cc: Michael S. Tsirkin <mst@redhat.com> >Cc: Jason Wang <jasowang@redhat.com> >Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com> >Cc: Baoquan He <bhe@redhat.com> >Signed-off-by: David Hildenbrand <david@redhat.com> Reviewed-by: Wei Yang <richard.weiyang@linux.alibaba.com> >--- > drivers/virtio/virtio_mem.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > >diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c >index ed99e43354010..ba4de598f6636 100644 >--- a/drivers/virtio/virtio_mem.c >+++ b/drivers/virtio/virtio_mem.c >@@ -424,7 +424,8 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) > > dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); > return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), >- vm->resource_name, MHP_NONE); >+ vm->resource_name, >+ MEMHP_MERGE_RESOURCE); > } > > /* >-- >2.26.2
diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index ed99e43354010..ba4de598f6636 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -424,7 +424,8 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) dev_dbg(&vm->vdev->dev, "adding memory block: %lu\n", mb_id); return add_memory_driver_managed(nid, addr, memory_block_size_bytes(), - vm->resource_name, MHP_NONE); + vm->resource_name, + MEMHP_MERGE_RESOURCE); } /*
virtio-mem adds memory in memory block granularity, to be able to remove it in the same granularity again later, and to grow slowly on demand. This, however, results in quite a lot of resources when adding a lot of memory. Resources are effectively stored in a list-based tree. Having a lot of resources not only wastes memory, it also makes traversing that tree more expensive, and makes /proc/iomem explode in size (e.g., requiring kexec-tools to manually merge resources later when e.g., trying to create a kdump header). Before this patch, we get (/proc/iomem) when hotplugging 2G via virtio-mem on x86-64: [...] 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-147ffffff : System RAM (virtio_mem) 148000000-14fffffff : System RAM (virtio_mem) 150000000-157ffffff : System RAM (virtio_mem) 158000000-15fffffff : System RAM (virtio_mem) 160000000-167ffffff : System RAM (virtio_mem) 168000000-16fffffff : System RAM (virtio_mem) 170000000-177ffffff : System RAM (virtio_mem) 178000000-17fffffff : System RAM (virtio_mem) 180000000-187ffffff : System RAM (virtio_mem) 188000000-18fffffff : System RAM (virtio_mem) 190000000-197ffffff : System RAM (virtio_mem) 198000000-19fffffff : System RAM (virtio_mem) 1a0000000-1a7ffffff : System RAM (virtio_mem) 1a8000000-1afffffff : System RAM (virtio_mem) 1b0000000-1b7ffffff : System RAM (virtio_mem) 1b8000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 With this patch, we get (/proc/iomem): [...] fffc0000-ffffffff : Reserved 100000000-13fffffff : System RAM 140000000-33fffffff : virtio0 140000000-1bfffffff : System RAM (virtio_mem) 3280000000-32ffffffff : PCI Bus 0000:00 Of course, with more hotplugged memory, it gets worse. When unplugging memory blocks again, try_remove_memory() (via offline_and_remove_memory()) will properly split the resource up again. Reviewed-by: Pankaj Gupta <pankaj.gupta.linux@gmail.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Jason Wang <jasowang@redhat.com> Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com> Cc: Baoquan He <bhe@redhat.com> Cc: Wei Yang <richardw.yang@linux.intel.com> Signed-off-by: David Hildenbrand <david@redhat.com> --- drivers/virtio/virtio_mem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)