| Message ID | 0cfb3c69ba6ca9ff55e1fc2528d18d108416ba57.1660897864.git.remckee0@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2,1/4] memblock tests: add simulation of physical memory with multiple NUMA nodes | expand |
On 19.08.22 11:05, Rebecca Mckeever wrote: > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > for setting up a memory layout with multiple NUMA nodes in a previously > allocated dummy physical memory. These functions can be used in place of > setup_memblock() in tests that need to simulate a NUMA system. > > setup_numa_memblock_generic(): > - allows for setting up a custom memory layout by specifying the amount > of memory in each node, the number of nodes, and a factor that will be > used to scale the memory in each node > > setup_numa_memblock(): > - allows for setting up a default memory layout > > Introduce constant MEM_FACTOR, which is used to scale the default memory > layout based on MEM_SIZE. > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > 16 NUMA nodes. > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > --- > .../testing/memblock/scripts/Makefile.include | 2 +- > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > tools/testing/memblock/tests/common.h | 9 ++++- > 3 files changed, 47 insertions(+), 2 deletions(-) > > diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include > index aa6d82d56a23..998281723590 100644 > --- a/tools/testing/memblock/scripts/Makefile.include > +++ b/tools/testing/memblock/scripts/Makefile.include > @@ -3,7 +3,7 @@ > > # Simulate CONFIG_NUMA=y > ifeq ($(NUMA), 1) > - CFLAGS += -D CONFIG_NUMA > + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 > endif > > # Use 32 bit physical addresses. > diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c > index eec6901081af..15d8767dc70c 100644 > --- a/tools/testing/memblock/tests/common.c > +++ b/tools/testing/memblock/tests/common.c > @@ -34,6 +34,10 @@ static const char * const help_opts[] = { > > static int verbose; > > +static const phys_addr_t node_sizes[] = { > + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K > +}; > + > /* sets global variable returned by movable_node_is_enabled() stub */ > bool movable_node_enabled; > > @@ -72,6 +76,40 @@ void setup_memblock(void) > fill_memblock(); > } > > +/** > + * setup_numa_memblock_generic: > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > + * dummy physical memory. > + * @nodes: an array containing the amount of memory in each node > + * @node_cnt: the size of @nodes > + * @factor: a factor that will be used to scale the memory in each node > + * > + * The nids will be set to 0 through node_cnt - 1. > + */ > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > + int node_cnt, int factor) > +{ > + phys_addr_t base; > + int flags; > + > + reset_memblock_regions(); > + base = (phys_addr_t)memory_block.base; > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > + > + for (int i = 0; i < node_cnt; i++) { > + phys_addr_t size = factor * nodes[i]; I'm a bit lost why we need the factor if we already provide sizes in the array. Can you enlighten me? :) Why can't we just stick to the sizes in the array?
On Tue, Aug 30, 2022 at 01:17:56PM +0200, David Hildenbrand wrote: > On 19.08.22 11:05, Rebecca Mckeever wrote: > > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > > for setting up a memory layout with multiple NUMA nodes in a previously > > allocated dummy physical memory. These functions can be used in place of > > setup_memblock() in tests that need to simulate a NUMA system. > > > > setup_numa_memblock_generic(): > > - allows for setting up a custom memory layout by specifying the amount > > of memory in each node, the number of nodes, and a factor that will be > > used to scale the memory in each node > > > > setup_numa_memblock(): > > - allows for setting up a default memory layout > > > > Introduce constant MEM_FACTOR, which is used to scale the default memory > > layout based on MEM_SIZE. > > > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > > 16 NUMA nodes. > > > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > > --- > > .../testing/memblock/scripts/Makefile.include | 2 +- > > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > > tools/testing/memblock/tests/common.h | 9 ++++- > > 3 files changed, 47 insertions(+), 2 deletions(-) > > > > diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include > > index aa6d82d56a23..998281723590 100644 > > --- a/tools/testing/memblock/scripts/Makefile.include > > +++ b/tools/testing/memblock/scripts/Makefile.include > > @@ -3,7 +3,7 @@ > > > > # Simulate CONFIG_NUMA=y > > ifeq ($(NUMA), 1) > > - CFLAGS += -D CONFIG_NUMA > > + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 > > endif > > > > # Use 32 bit physical addresses. > > diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c > > index eec6901081af..15d8767dc70c 100644 > > --- a/tools/testing/memblock/tests/common.c > > +++ b/tools/testing/memblock/tests/common.c > > @@ -34,6 +34,10 @@ static const char * const help_opts[] = { > > > > static int verbose; > > > > +static const phys_addr_t node_sizes[] = { > > + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K > > +}; > > + > > /* sets global variable returned by movable_node_is_enabled() stub */ > > bool movable_node_enabled; > > > > @@ -72,6 +76,40 @@ void setup_memblock(void) > > fill_memblock(); > > } > > > > +/** > > + * setup_numa_memblock_generic: > > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > > + * dummy physical memory. > > + * @nodes: an array containing the amount of memory in each node > > + * @node_cnt: the size of @nodes > > + * @factor: a factor that will be used to scale the memory in each node > > + * > > + * The nids will be set to 0 through node_cnt - 1. > > + */ > > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > > + int node_cnt, int factor) > > +{ > > + phys_addr_t base; > > + int flags; > > + > > + reset_memblock_regions(); > > + base = (phys_addr_t)memory_block.base; > > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > > + > > + for (int i = 0; i < node_cnt; i++) { > > + phys_addr_t size = factor * nodes[i]; > > I'm a bit lost why we need the factor if we already provide sizes in the > array. > > Can you enlighten me? :) > > Why can't we just stick to the sizes in the array? > Without the factor, some of the tests will break if we increase MEM_SIZE in the future (which we may need to do). I could rewrite them so that the factor is not needed, but I thought the code would be over-complicated if I did. > -- > Thanks, > > David / dhildenb > Thanks, Rebecca
On Tue, Aug 30, 2022 at 10:49:09PM -0500, Rebecca Mckeever wrote: > On Tue, Aug 30, 2022 at 01:17:56PM +0200, David Hildenbrand wrote: > > On 19.08.22 11:05, Rebecca Mckeever wrote: > > > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > > > for setting up a memory layout with multiple NUMA nodes in a previously > > > allocated dummy physical memory. These functions can be used in place of > > > setup_memblock() in tests that need to simulate a NUMA system. > > > > > > setup_numa_memblock_generic(): > > > - allows for setting up a custom memory layout by specifying the amount > > > of memory in each node, the number of nodes, and a factor that will be > > > used to scale the memory in each node > > > > > > setup_numa_memblock(): > > > - allows for setting up a default memory layout > > > > > > Introduce constant MEM_FACTOR, which is used to scale the default memory > > > layout based on MEM_SIZE. > > > > > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > > > 16 NUMA nodes. > > > > > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > > > --- > > > .../testing/memblock/scripts/Makefile.include | 2 +- > > > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > > > tools/testing/memblock/tests/common.h | 9 ++++- > > > 3 files changed, 47 insertions(+), 2 deletions(-) > > > > > > diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include > > > index aa6d82d56a23..998281723590 100644 > > > --- a/tools/testing/memblock/scripts/Makefile.include > > > +++ b/tools/testing/memblock/scripts/Makefile.include > > > @@ -3,7 +3,7 @@ > > > > > > # Simulate CONFIG_NUMA=y > > > ifeq ($(NUMA), 1) > > > - CFLAGS += -D CONFIG_NUMA > > > + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 > > > endif > > > > > > # Use 32 bit physical addresses. > > > diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c > > > index eec6901081af..15d8767dc70c 100644 > > > --- a/tools/testing/memblock/tests/common.c > > > +++ b/tools/testing/memblock/tests/common.c > > > @@ -34,6 +34,10 @@ static const char * const help_opts[] = { > > > > > > static int verbose; > > > > > > +static const phys_addr_t node_sizes[] = { > > > + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K > > > +}; > > > + > > > /* sets global variable returned by movable_node_is_enabled() stub */ > > > bool movable_node_enabled; > > > > > > @@ -72,6 +76,40 @@ void setup_memblock(void) > > > fill_memblock(); > > > } > > > > > > +/** > > > + * setup_numa_memblock_generic: > > > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > > > + * dummy physical memory. > > > + * @nodes: an array containing the amount of memory in each node > > > + * @node_cnt: the size of @nodes > > > + * @factor: a factor that will be used to scale the memory in each node > > > + * > > > + * The nids will be set to 0 through node_cnt - 1. > > > + */ > > > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > > > + int node_cnt, int factor) > > > +{ > > > + phys_addr_t base; > > > + int flags; > > > + > > > + reset_memblock_regions(); > > > + base = (phys_addr_t)memory_block.base; > > > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > > > + > > > + for (int i = 0; i < node_cnt; i++) { > > > + phys_addr_t size = factor * nodes[i]; > > > > I'm a bit lost why we need the factor if we already provide sizes in the > > array. > > > > Can you enlighten me? :) > > > > Why can't we just stick to the sizes in the array? > > > Without the factor, some of the tests will break if we increase MEM_SIZE > in the future (which we may need to do). I could rewrite them so that the > factor is not needed, but I thought the code would be over-complicated if > I did. What if we make nodes[] to represent the fraction of the memory rather than a node size? Then the factor won't be required. > Thanks, > Rebecca
On Fri, Aug 19, 2022 at 02:05:31AM -0700, Rebecca Mckeever wrote: > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > for setting up a memory layout with multiple NUMA nodes in a previously > allocated dummy physical memory. These functions can be used in place of > setup_memblock() in tests that need to simulate a NUMA system. > > setup_numa_memblock_generic(): > - allows for setting up a custom memory layout by specifying the amount > of memory in each node, the number of nodes, and a factor that will be > used to scale the memory in each node > > setup_numa_memblock(): > - allows for setting up a default memory layout > > Introduce constant MEM_FACTOR, which is used to scale the default memory > layout based on MEM_SIZE. > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > 16 NUMA nodes. > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > --- > .../testing/memblock/scripts/Makefile.include | 2 +- > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > tools/testing/memblock/tests/common.h | 9 ++++- > 3 files changed, 47 insertions(+), 2 deletions(-) ... > +/** > + * setup_numa_memblock_generic: > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > + * dummy physical memory. > + * @nodes: an array containing the amount of memory in each node > + * @node_cnt: the size of @nodes > + * @factor: a factor that will be used to scale the memory in each node > + * > + * The nids will be set to 0 through node_cnt - 1. > + */ > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > + int node_cnt, int factor) I only had time for a quick look and it seems this function is never used on its own. Let's fold it into setup_numa_memblock() for now. > +{ > + phys_addr_t base; > + int flags; > + > + reset_memblock_regions(); > + base = (phys_addr_t)memory_block.base; > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > + > + for (int i = 0; i < node_cnt; i++) { > + phys_addr_t size = factor * nodes[i]; > + > + memblock_add_node(base, size, i, flags); > + base += size; > + } > + fill_memblock(); > +} > + > +void setup_numa_memblock(void) > +{ > + setup_numa_memblock_generic(node_sizes, NUMA_NODES, MEM_FACTOR); > +} > +
On 31.08.22 05:49, Rebecca Mckeever wrote: > On Tue, Aug 30, 2022 at 01:17:56PM +0200, David Hildenbrand wrote: >> On 19.08.22 11:05, Rebecca Mckeever wrote: >>> Add functions setup_numa_memblock_generic() and setup_numa_memblock() >>> for setting up a memory layout with multiple NUMA nodes in a previously >>> allocated dummy physical memory. These functions can be used in place of >>> setup_memblock() in tests that need to simulate a NUMA system. >>> >>> setup_numa_memblock_generic(): >>> - allows for setting up a custom memory layout by specifying the amount >>> of memory in each node, the number of nodes, and a factor that will be >>> used to scale the memory in each node >>> >>> setup_numa_memblock(): >>> - allows for setting up a default memory layout >>> >>> Introduce constant MEM_FACTOR, which is used to scale the default memory >>> layout based on MEM_SIZE. >>> >>> Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to >>> 16 NUMA nodes. >>> >>> Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> >>> --- >>> .../testing/memblock/scripts/Makefile.include | 2 +- >>> tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ >>> tools/testing/memblock/tests/common.h | 9 ++++- >>> 3 files changed, 47 insertions(+), 2 deletions(-) >>> >>> diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include >>> index aa6d82d56a23..998281723590 100644 >>> --- a/tools/testing/memblock/scripts/Makefile.include >>> +++ b/tools/testing/memblock/scripts/Makefile.include >>> @@ -3,7 +3,7 @@ >>> >>> # Simulate CONFIG_NUMA=y >>> ifeq ($(NUMA), 1) >>> - CFLAGS += -D CONFIG_NUMA >>> + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 >>> endif >>> >>> # Use 32 bit physical addresses. >>> diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c >>> index eec6901081af..15d8767dc70c 100644 >>> --- a/tools/testing/memblock/tests/common.c >>> +++ b/tools/testing/memblock/tests/common.c >>> @@ -34,6 +34,10 @@ static const char * const help_opts[] = { >>> >>> static int verbose; >>> >>> +static const phys_addr_t node_sizes[] = { >>> + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K >>> +}; >>> + >>> /* sets global variable returned by movable_node_is_enabled() stub */ >>> bool movable_node_enabled; >>> >>> @@ -72,6 +76,40 @@ void setup_memblock(void) >>> fill_memblock(); >>> } >>> >>> +/** >>> + * setup_numa_memblock_generic: >>> + * Set up a memory layout with multiple NUMA nodes in a previously allocated >>> + * dummy physical memory. >>> + * @nodes: an array containing the amount of memory in each node >>> + * @node_cnt: the size of @nodes >>> + * @factor: a factor that will be used to scale the memory in each node >>> + * >>> + * The nids will be set to 0 through node_cnt - 1. >>> + */ >>> +void setup_numa_memblock_generic(const phys_addr_t nodes[], >>> + int node_cnt, int factor) >>> +{ >>> + phys_addr_t base; >>> + int flags; >>> + >>> + reset_memblock_regions(); >>> + base = (phys_addr_t)memory_block.base; >>> + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; >>> + >>> + for (int i = 0; i < node_cnt; i++) { >>> + phys_addr_t size = factor * nodes[i]; >> >> I'm a bit lost why we need the factor if we already provide sizes in the >> array. >> >> Can you enlighten me? :) >> >> Why can't we just stick to the sizes in the array? >> > Without the factor, some of the tests will break if we increase MEM_SIZE > in the future (which we may need to do). I could rewrite them so that the > factor is not needed, but I thought the code would be over-complicated if > I did. Independent of the suggestion from Mike, I wonder if we should really care about (eventual) MEM_SIZE changes for now if not caring simplifies the current code.
On Wed, Aug 31, 2022 at 06:12:10PM +0300, Mike Rapoport wrote: > On Tue, Aug 30, 2022 at 10:49:09PM -0500, Rebecca Mckeever wrote: > > On Tue, Aug 30, 2022 at 01:17:56PM +0200, David Hildenbrand wrote: > > > On 19.08.22 11:05, Rebecca Mckeever wrote: > > > > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > > > > for setting up a memory layout with multiple NUMA nodes in a previously > > > > allocated dummy physical memory. These functions can be used in place of > > > > setup_memblock() in tests that need to simulate a NUMA system. > > > > > > > > setup_numa_memblock_generic(): > > > > - allows for setting up a custom memory layout by specifying the amount > > > > of memory in each node, the number of nodes, and a factor that will be > > > > used to scale the memory in each node > > > > > > > > setup_numa_memblock(): > > > > - allows for setting up a default memory layout > > > > > > > > Introduce constant MEM_FACTOR, which is used to scale the default memory > > > > layout based on MEM_SIZE. > > > > > > > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > > > > 16 NUMA nodes. > > > > > > > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > > > > --- > > > > .../testing/memblock/scripts/Makefile.include | 2 +- > > > > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > > > > tools/testing/memblock/tests/common.h | 9 ++++- > > > > 3 files changed, 47 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include > > > > index aa6d82d56a23..998281723590 100644 > > > > --- a/tools/testing/memblock/scripts/Makefile.include > > > > +++ b/tools/testing/memblock/scripts/Makefile.include > > > > @@ -3,7 +3,7 @@ > > > > > > > > # Simulate CONFIG_NUMA=y > > > > ifeq ($(NUMA), 1) > > > > - CFLAGS += -D CONFIG_NUMA > > > > + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 > > > > endif > > > > > > > > # Use 32 bit physical addresses. > > > > diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c > > > > index eec6901081af..15d8767dc70c 100644 > > > > --- a/tools/testing/memblock/tests/common.c > > > > +++ b/tools/testing/memblock/tests/common.c > > > > @@ -34,6 +34,10 @@ static const char * const help_opts[] = { > > > > > > > > static int verbose; > > > > > > > > +static const phys_addr_t node_sizes[] = { > > > > + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K > > > > +}; > > > > + > > > > /* sets global variable returned by movable_node_is_enabled() stub */ > > > > bool movable_node_enabled; > > > > > > > > @@ -72,6 +76,40 @@ void setup_memblock(void) > > > > fill_memblock(); > > > > } > > > > > > > > +/** > > > > + * setup_numa_memblock_generic: > > > > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > > > > + * dummy physical memory. > > > > + * @nodes: an array containing the amount of memory in each node > > > > + * @node_cnt: the size of @nodes > > > > + * @factor: a factor that will be used to scale the memory in each node > > > > + * > > > > + * The nids will be set to 0 through node_cnt - 1. > > > > + */ > > > > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > > > > + int node_cnt, int factor) > > > > +{ > > > > + phys_addr_t base; > > > > + int flags; > > > > + > > > > + reset_memblock_regions(); > > > > + base = (phys_addr_t)memory_block.base; > > > > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > > > > + > > > > + for (int i = 0; i < node_cnt; i++) { > > > > + phys_addr_t size = factor * nodes[i]; > > > > > > I'm a bit lost why we need the factor if we already provide sizes in the > > > array. > > > > > > Can you enlighten me? :) > > > > > > Why can't we just stick to the sizes in the array? > > > > > Without the factor, some of the tests will break if we increase MEM_SIZE > > in the future (which we may need to do). I could rewrite them so that the > > factor is not needed, but I thought the code would be over-complicated if > > I did. > > What if we make nodes[] to represent the fraction of the memory rather than > a node size? Then the factor won't be required. > I think that will work. I'll try it. > > Thanks, > > Rebecca > > -- > Sincerely yours, > Mike. Thanks, Rebecca
On Thu, Sep 01, 2022 at 10:06:48AM +0200, David Hildenbrand wrote: > On 31.08.22 05:49, Rebecca Mckeever wrote: > > On Tue, Aug 30, 2022 at 01:17:56PM +0200, David Hildenbrand wrote: > >> On 19.08.22 11:05, Rebecca Mckeever wrote: > >>> Add functions setup_numa_memblock_generic() and setup_numa_memblock() > >>> for setting up a memory layout with multiple NUMA nodes in a previously > >>> allocated dummy physical memory. These functions can be used in place of > >>> setup_memblock() in tests that need to simulate a NUMA system. > >>> > >>> setup_numa_memblock_generic(): > >>> - allows for setting up a custom memory layout by specifying the amount > >>> of memory in each node, the number of nodes, and a factor that will be > >>> used to scale the memory in each node > >>> > >>> setup_numa_memblock(): > >>> - allows for setting up a default memory layout > >>> > >>> Introduce constant MEM_FACTOR, which is used to scale the default memory > >>> layout based on MEM_SIZE. > >>> > >>> Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > >>> 16 NUMA nodes. > >>> > >>> Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > >>> --- > >>> .../testing/memblock/scripts/Makefile.include | 2 +- > >>> tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > >>> tools/testing/memblock/tests/common.h | 9 ++++- > >>> 3 files changed, 47 insertions(+), 2 deletions(-) > >>> > >>> diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include > >>> index aa6d82d56a23..998281723590 100644 > >>> --- a/tools/testing/memblock/scripts/Makefile.include > >>> +++ b/tools/testing/memblock/scripts/Makefile.include > >>> @@ -3,7 +3,7 @@ > >>> > >>> # Simulate CONFIG_NUMA=y > >>> ifeq ($(NUMA), 1) > >>> - CFLAGS += -D CONFIG_NUMA > >>> + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 > >>> endif > >>> > >>> # Use 32 bit physical addresses. > >>> diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c > >>> index eec6901081af..15d8767dc70c 100644 > >>> --- a/tools/testing/memblock/tests/common.c > >>> +++ b/tools/testing/memblock/tests/common.c > >>> @@ -34,6 +34,10 @@ static const char * const help_opts[] = { > >>> > >>> static int verbose; > >>> > >>> +static const phys_addr_t node_sizes[] = { > >>> + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K > >>> +}; > >>> + > >>> /* sets global variable returned by movable_node_is_enabled() stub */ > >>> bool movable_node_enabled; > >>> > >>> @@ -72,6 +76,40 @@ void setup_memblock(void) > >>> fill_memblock(); > >>> } > >>> > >>> +/** > >>> + * setup_numa_memblock_generic: > >>> + * Set up a memory layout with multiple NUMA nodes in a previously allocated > >>> + * dummy physical memory. > >>> + * @nodes: an array containing the amount of memory in each node > >>> + * @node_cnt: the size of @nodes > >>> + * @factor: a factor that will be used to scale the memory in each node > >>> + * > >>> + * The nids will be set to 0 through node_cnt - 1. > >>> + */ > >>> +void setup_numa_memblock_generic(const phys_addr_t nodes[], > >>> + int node_cnt, int factor) > >>> +{ > >>> + phys_addr_t base; > >>> + int flags; > >>> + > >>> + reset_memblock_regions(); > >>> + base = (phys_addr_t)memory_block.base; > >>> + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > >>> + > >>> + for (int i = 0; i < node_cnt; i++) { > >>> + phys_addr_t size = factor * nodes[i]; > >> > >> I'm a bit lost why we need the factor if we already provide sizes in the > >> array. > >> > >> Can you enlighten me? :) > >> > >> Why can't we just stick to the sizes in the array? > >> > > Without the factor, some of the tests will break if we increase MEM_SIZE > > in the future (which we may need to do). I could rewrite them so that the > > factor is not needed, but I thought the code would be over-complicated if > > I did. > > Independent of the suggestion from Mike, I wonder if we should really > care about (eventual) MEM_SIZE changes for now if not caring simplifies > the current code. > Maybe not. I'm going to try Mike's suggestion, but I will keep this in mind if the code seems too complicated. > -- > Thanks, > > David / dhildenb > Thanks, Rebecca
On Wed, Aug 31, 2022 at 06:15:41PM +0300, Mike Rapoport wrote: > On Fri, Aug 19, 2022 at 02:05:31AM -0700, Rebecca Mckeever wrote: > > Add functions setup_numa_memblock_generic() and setup_numa_memblock() > > for setting up a memory layout with multiple NUMA nodes in a previously > > allocated dummy physical memory. These functions can be used in place of > > setup_memblock() in tests that need to simulate a NUMA system. > > > > setup_numa_memblock_generic(): > > - allows for setting up a custom memory layout by specifying the amount > > of memory in each node, the number of nodes, and a factor that will be > > used to scale the memory in each node > > > > setup_numa_memblock(): > > - allows for setting up a default memory layout > > > > Introduce constant MEM_FACTOR, which is used to scale the default memory > > layout based on MEM_SIZE. > > > > Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to > > 16 NUMA nodes. > > > > Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> > > --- > > .../testing/memblock/scripts/Makefile.include | 2 +- > > tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ > > tools/testing/memblock/tests/common.h | 9 ++++- > > 3 files changed, 47 insertions(+), 2 deletions(-) > > ... > > > +/** > > + * setup_numa_memblock_generic: > > + * Set up a memory layout with multiple NUMA nodes in a previously allocated > > + * dummy physical memory. > > + * @nodes: an array containing the amount of memory in each node > > + * @node_cnt: the size of @nodes > > + * @factor: a factor that will be used to scale the memory in each node > > + * > > + * The nids will be set to 0 through node_cnt - 1. > > + */ > > +void setup_numa_memblock_generic(const phys_addr_t nodes[], > > + int node_cnt, int factor) > > I only had time for a quick look and it seems this function is never used > on its own. > Let's fold it into setup_numa_memblock() for now. > Okay, will do. > > +{ > > + phys_addr_t base; > > + int flags; > > + > > + reset_memblock_regions(); > > + base = (phys_addr_t)memory_block.base; > > + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; > > + > > + for (int i = 0; i < node_cnt; i++) { > > + phys_addr_t size = factor * nodes[i]; > > + > > + memblock_add_node(base, size, i, flags); > > + base += size; > > + } > > + fill_memblock(); > > +} > > + > > +void setup_numa_memblock(void) > > +{ > > + setup_numa_memblock_generic(node_sizes, NUMA_NODES, MEM_FACTOR); > > +} > > + > > -- > Sincerely yours, > Mike. Thanks, Rebecca
diff --git a/tools/testing/memblock/scripts/Makefile.include b/tools/testing/memblock/scripts/Makefile.include index aa6d82d56a23..998281723590 100644 --- a/tools/testing/memblock/scripts/Makefile.include +++ b/tools/testing/memblock/scripts/Makefile.include @@ -3,7 +3,7 @@ # Simulate CONFIG_NUMA=y ifeq ($(NUMA), 1) - CFLAGS += -D CONFIG_NUMA + CFLAGS += -D CONFIG_NUMA -D CONFIG_NODES_SHIFT=4 endif # Use 32 bit physical addresses. diff --git a/tools/testing/memblock/tests/common.c b/tools/testing/memblock/tests/common.c index eec6901081af..15d8767dc70c 100644 --- a/tools/testing/memblock/tests/common.c +++ b/tools/testing/memblock/tests/common.c @@ -34,6 +34,10 @@ static const char * const help_opts[] = { static int verbose; +static const phys_addr_t node_sizes[] = { + SZ_4K, SZ_1K, SZ_2K, SZ_2K, SZ_1K, SZ_1K, SZ_4K, SZ_1K +}; + /* sets global variable returned by movable_node_is_enabled() stub */ bool movable_node_enabled; @@ -72,6 +76,40 @@ void setup_memblock(void) fill_memblock(); } +/** + * setup_numa_memblock_generic: + * Set up a memory layout with multiple NUMA nodes in a previously allocated + * dummy physical memory. + * @nodes: an array containing the amount of memory in each node + * @node_cnt: the size of @nodes + * @factor: a factor that will be used to scale the memory in each node + * + * The nids will be set to 0 through node_cnt - 1. + */ +void setup_numa_memblock_generic(const phys_addr_t nodes[], + int node_cnt, int factor) +{ + phys_addr_t base; + int flags; + + reset_memblock_regions(); + base = (phys_addr_t)memory_block.base; + flags = (movable_node_is_enabled()) ? MEMBLOCK_NONE : MEMBLOCK_HOTPLUG; + + for (int i = 0; i < node_cnt; i++) { + phys_addr_t size = factor * nodes[i]; + + memblock_add_node(base, size, i, flags); + base += size; + } + fill_memblock(); +} + +void setup_numa_memblock(void) +{ + setup_numa_memblock_generic(node_sizes, NUMA_NODES, MEM_FACTOR); +} + void dummy_physical_memory_init(void) { memory_block.base = malloc(MEM_SIZE); diff --git a/tools/testing/memblock/tests/common.h b/tools/testing/memblock/tests/common.h index 4fd3534ff955..e5117d959d6c 100644 --- a/tools/testing/memblock/tests/common.h +++ b/tools/testing/memblock/tests/common.h @@ -10,7 +10,11 @@ #include <linux/printk.h> #include <../selftests/kselftest.h> -#define MEM_SIZE SZ_16K +#define MEM_SIZE SZ_16K +#define NUMA_NODES 8 + +/* used to resize values that need to scale with MEM_SIZE */ +#define MEM_FACTOR (MEM_SIZE / SZ_16K) enum test_flags { TEST_ZEROED = 0x0, @@ -100,6 +104,9 @@ struct region { void reset_memblock_regions(void); void reset_memblock_attributes(void); void setup_memblock(void); +void setup_numa_memblock_generic(const phys_addr_t nodes[], + int node_cnt, int factor); +void setup_numa_memblock(void); void dummy_physical_memory_init(void); void dummy_physical_memory_cleanup(void); void parse_args(int argc, char **argv);
Add functions setup_numa_memblock_generic() and setup_numa_memblock() for setting up a memory layout with multiple NUMA nodes in a previously allocated dummy physical memory. These functions can be used in place of setup_memblock() in tests that need to simulate a NUMA system. setup_numa_memblock_generic(): - allows for setting up a custom memory layout by specifying the amount of memory in each node, the number of nodes, and a factor that will be used to scale the memory in each node setup_numa_memblock(): - allows for setting up a default memory layout Introduce constant MEM_FACTOR, which is used to scale the default memory layout based on MEM_SIZE. Set CONFIG_NODES_SHIFT to 4 when building with NUMA=1 to allow for up to 16 NUMA nodes. Signed-off-by: Rebecca Mckeever <remckee0@gmail.com> --- .../testing/memblock/scripts/Makefile.include | 2 +- tools/testing/memblock/tests/common.c | 38 +++++++++++++++++++ tools/testing/memblock/tests/common.h | 9 ++++- 3 files changed, 47 insertions(+), 2 deletions(-)