[v3,01/17] xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS

Commit Message

Henry Wang April 20, 2023, 11:25 a.m. UTC

From: Wei Chen <[e-mail]>

As a memory range described in device tree cannot be split across
multiple nodes. And it is very likely than if you have more than
64 nodes, you may need a lot more than 2 regions per node. So the
default NR_NODE_MEMBLKS value (MAX_NUMNODES * 2) makes no sense
on Arm.

So, for Arm, we would just define NR_NODE_MEMBLKS as an alias to
NR_MEM_BANKS. And in the future NR_MEM_BANKS will be user-configurable
via kconfig, but for now leave NR_MEM_BANKS as 128 on Arm. This
avoid to have different way to define the value based NUMA vs non-NUMA.

Further discussions can be found here[1].

[1] https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg02322.html

Signed-off-by: Wei Chen <[e-mail]>
Signed-off-by: Henry Wang <[e-mail]>
---
By checking the discussion in [1] and [2]
[1] https://lists.xenproject.org/archives/html/xen-devel/2023-01/msg00595.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg02322.html
v2 -> v3:
1. No change
v1 -> v2:
1. Add code comments to explain using NR_MEM_BANKS for Arm
2. Refine commit messages.
---
 xen/arch/arm/include/asm/numa.h | 19 ++++++++++++++++++-
 xen/include/xen/numa.h          |  9 +++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

Comments

Jan Beulich April 20, 2023, 12:21 p.m. UTC | #1

On 20.04.2023 13:25, Henry Wang wrote:
> From: Wei Chen <[e-mail]>
> 
> As a memory range described in device tree cannot be split across
> multiple nodes. And it is very likely than if you have more than
> 64 nodes, you may need a lot more than 2 regions per node. So the
> default NR_NODE_MEMBLKS value (MAX_NUMNODES * 2) makes no sense
> on Arm.
> 
> So, for Arm, we would just define NR_NODE_MEMBLKS as an alias to
> NR_MEM_BANKS. And in the future NR_MEM_BANKS will be user-configurable
> via kconfig, but for now leave NR_MEM_BANKS as 128 on Arm. This
> avoid to have different way to define the value based NUMA vs non-NUMA.
> 
> Further discussions can be found here[1].
> 
> [1] https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg02322.html
> 
> Signed-off-by: Wei Chen <[e-mail]>
> Signed-off-by: Henry Wang <[e-mail]>

Acked-by: Jan Beulich <[e-mail]>

Patch

diff --git a/xen/arch/arm/include/asm/numa.h b/xen/arch/arm/include/asm/numa.h
index e2bee2bd82..7d6ae36a19 100644
--- a/xen/arch/arm/include/asm/numa.h
+++ b/xen/arch/arm/include/asm/numa.h
@@ -3,9 +3,26 @@ 
 
 #include <xen/mm.h>
 
+#include <asm/setup.h>
+
 typedef u8 nodeid_t;
 
-#ifndef CONFIG_NUMA
+#ifdef CONFIG_NUMA
+
+/*
+ * It is very likely that if you have more than 64 nodes, you may
+ * need a lot more than 2 regions per node. So, for Arm, we would
+ * just define NR_NODE_MEMBLKS as an alias to NR_MEM_BANKS.
+ * And in the future NR_MEM_BANKS will be bumped for new platforms,
+ * but for now leave NR_MEM_BANKS as it is on Arm. This avoid to
+ * have different way to define the value based NUMA vs non-NUMA.
+ *
+ * Further discussions can be found here:
+ * https://lists.xenproject.org/archives/html/xen-devel/2021-09/msg02322.html
+ */
+#define NR_NODE_MEMBLKS NR_MEM_BANKS
+
+#else
 
 /* Fake one node for now. See also node_online_map. */
 #define cpu_to_node(cpu) 0
diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h
index 29b8c2df89..b86d0851fc 100644
--- a/xen/include/xen/numa.h
+++ b/xen/include/xen/numa.h
@@ -13,7 +13,16 @@ 
 #define MAX_NUMNODES 1
 #endif
 
+/*
+ * Some architectures may have different considerations for
+ * number of node memory blocks. They can define their
+ * NR_NODE_MEMBLKS in asm/numa.h to reflect their architectural
+ * implementation. If the arch does not have specific implementation,
+ * the following default NR_NODE_MEMBLKS will be used.
+ */
+#ifndef NR_NODE_MEMBLKS
 #define NR_NODE_MEMBLKS (MAX_NUMNODES * 2)
+#endif
 
 #define vcpu_to_node(v) (cpu_to_node((v)->processor))