[RFC,v3,0/2] ACPI: NUMA: debug invalid unused PXM value for CFMWs

Message

Yuquan Wang March 21, 2025, 2:36 a.m. UTC

Changes in v3:
- Introduce numa_add_reserved_memblk()
- Use more explicit way to make sure CFMWs fake node >= 1 
- Replace numa_add_memblk() with numa_add_reserved_memblk() in acpi_parse_cfmws()

Problem
-------
The absence of SRAT would cause the fake_pxm to be -1 and increment
to 0, then send to acpi_parse_cfmws(). If there exists CXL memory
ranges that are defined in the CFMWS and not already defined in the
SRAT, the new node (node0) for the CXL memory would be invalid, as
node0 is already in "used", and all CXL memory might be online on
node0.

Control cxl fake node value
---------------------------
This utilizes node_set(0, nodes_found_map) to set pxm&node map. With
this setting, acpi_map_pxm_to_node() could return the expected node
value even if no SRAT.

numa_add_memblk() problem on this issue
---------------------------------------
If SRAT is valid, the numa_memblks_init() would then utilize
numa_move_tail_memblk() to move the numa_memblk from numa_meminfo to
numa_reserved_meminfo in CFMWs fake node situation.

If SRAT is missing or bad, the numa_memblks_init() would fail since
init_func() would fail. And it causes that no numa_memblk in
numa_reserved_meminfo list and the following dax_cxl driver could
find the expected fake node.

numa_add_reserved_memblk()
--------------------------
Use numa_add_reserved_memblk() to replace numa_add_memblk(), since
the cxl numa_memblk added by numa_add_memblk() would finally be moved
to numa_reserved_meminfo, and numa_add_reserved_memblk() here could
add cxl numa_memblk into reserved list directly. Hence, no matter
SRAT is good or not, cxl numa_memblk could be allocated to reserved
list.

dax_kmem problem
----------------
The dax_kmem driver will fail if no SRAT because something wrong in
memory_group_register_static(). The good result is our cxl memory would
not be assigned to node0 anymore!

Discussion
----------
As papering these things looks like not easily, in v2 patch[1] I chose
to aggressively fail the acpi_parse_cfmws() in srat.c since it mainly
works for building cxl fake nodes and also fail the CXL init in
cxl_acpi_probe per the discussion from Jonathan[2]. Actually, I am not
sure if this(v3) is the best way to handle this issue, so hopes more
comments to guide me!

Link:
[1] https://lore.kernel.org/linux-cxl/[e-mail]/T/#t
[2] https://lists.nongnu.org/archive/html/qemu-devel/2025-03/msg03668.html


Yuquan Wang (2):
  mm: numa_memblks: introduce numa_add_reserved_memblk
  ACPI: NUMA: debug invalid unused PXM value for CFMWs

 drivers/acpi/numa/srat.c     | 11 ++++++++---
 include/linux/numa_memblks.h |  1 +
 mm/numa_memblks.c            | 16 ++++++++++++++++
 3 files changed, 25 insertions(+), 3 deletions(-)

Comments

Dan Williams March 26, 2025, 11:19 p.m. UTC | #1

Yuquan Wang wrote:
> Changes in v3:
> - Introduce numa_add_reserved_memblk()
> - Use more explicit way to make sure CFMWs fake node >= 1 
> - Replace numa_add_memblk() with numa_add_reserved_memblk() in acpi_parse_cfmws()
> 
> Problem
> -------
> The absence of SRAT

Can you say more about CXL ACPI platforms that do not have an SRAT? That
seems like the first problem, and this information helps determine
whether this is a fix that needs backporting, or just a cleaner way to
handle this quirk.

> would cause the fake_pxm to be -1 and increment

Not an issue in your patch but it strikes me that "fake_pxm" is
misnamed. There is nothing "fake" about it. It is simply the first
synthesized CXL proximity domain for future CXL region provisioning.

I would not mind a separate patch renaming "fake_pxm" to "cxl_pxm", just
to not confuse future readers with the simulated nodes produced by
mm/numa_emulation.c.