[RFC,0/2] Sbsa-ref CXL Enablement

Message

Yuquan Wang Aug. 30, 2024, 4:15 a.m. UTC

RFC because
- Many contents are ported from Jonathan' patch on qemu virt design

- Bring plenty of PCDs values and modifying the original PCIE values

- Less experience and not particularly confident in ACPI area so this might be
  stupidly broken in a way I've not considered.

Currently the base CXL support for arm platforms is only on Jonathan's patches[1] which
have not yet merged into upstream. SBSA-REF can be more like a real machine, thus the
support of cxl could be meaningful.

Regard to the burden of edk2 firmware, I try to build a static CEDT table and add acpi0016,
acpi0017 objects on DSDT at the initial development phase[2][3]. Hence it doesn't need to
communicate cxl contents via DT to edk2. 

This series leverages Jonathan's patches[1] to design [SBSA_CXL_HOST] and [SBSA_CXL_FIXED_WINDOW]
spaces for sbsa-ref layout. 

For [SBSA_CXL_HOST], new memory layout places 1M space for 16 host bridge register regions
in the sbsa-ref memmap. In addition, this only creates a default pxb-cxl (bus_nr=0xfe) bridge 
with one cxl-rp on sbsa-ref, so only one cxl device could be added by user on this cxl Bus.
With the 'create_pxb_cxl', users don't need to input '-device pxb-cxl' and '-device cxl-rp'
parameters.

For [SBSA_CXL_FIXED_WINDOW], this extends 1TB space from the hole above RAM Memory [SBSA_MEM]
for CXL Fixed Memory Window. 0xA0000000000 is chosen as the base address of this space because
of 3 reasons:

1) It is more suitable to choose a static address instead of that
implementation in virt, since a dynamic address space layout of
sbsa-ref is not appropriate for its original purpose as a reference
platform.

2) The Hotplug Memory address range should in the range of maximum
addressable range of sbsa-ref platform(0x10000000000-0x80ffffffffff).
It is satisfied the requirements of memory hotplug in linux kernel.

3) The start pfn of CFMW should exceed the reserved_pfn_range for
onlined numa node.

Based on 'cxl_fmws_link_targets', this adds a new function
'sbsa_cxl_fmws_link_targets' for binding cfmws.target with the default
pxb-cxl-bus on sbsa-ref.

In addition, this also adds 'create_cxl_fixed_window_region' which
based on 'machine_set_cfmw' to support creating a static cfmw region on
sbsa-ref, so users don't need to input '-M cxl-fmw' parameter.

Thus, to run sbsa-ref with a cxl device could use:
qemu-system-aarch64 \
-machine sbsa-ref \
-cpu cortex-a57 \
-smp 4 \
-m 4G \
-object memory-backend-ram,size=2G,id=mem0 \
-numa node,nodeid=0,cpus=0-1,memdev=mem0 \
-object memory-backend-ram,size=2G,id=mem1 \
-numa node,nodeid=1,cpus=2-3,memdev=mem1 \
-object memory-backend-file,id=mem2,mem-path=/tmp/mem2,size=256M,share=true \
-device cxl-type3,bus=cxl.0,volatile-memdev=mem2,id=cxl-mem1 \
-hda ubuntu.ext4 \
-pflash SBSA_FLASH0.fd \
-pflash SBSA_FLASH1.fd \

This series patches are here to hopefully some comments to guide me!

Link:
[1]: https://lore.kernel.org/linux-cxl/[e-mail]/
[2]: https://edk2.groups.io/g/devel/topic/rfc_patch_0_1/108173029
[3]: https://edk2.groups.io/g/devel/topic/rfc_patch_edk2_platforms/108173682

Yuquan Wang (2):
  hw/arm/sbsa-ref: Enable CXL Host Bridge by pxb-cxl
  hw/arm/sbsa-ref: Support CXL Fixed Memory Window

 hw/arm/sbsa-ref.c                   | 127 +++++++++++++++++++++++++++-
 hw/cxl/cxl-host-stubs.c             |   1 +
 hw/cxl/cxl-host.c                   |   2 +-
 hw/pci-bridge/pci_expander_bridge.c |   1 -
 include/hw/cxl/cxl_host.h           |   1 +
 include/hw/pci/pci_bridge.h         |   1 +
 6 files changed, 128 insertions(+), 5 deletions(-)

Comments

Jonathan Cameron Aug. 30, 2024, 11:18 a.m. UTC | #1

On Fri, 30 Aug 2024 12:15:55 +0800
Yuquan Wang <[e-mail]> wrote:

> RFC because
> - Many contents are ported from Jonathan' patch on qemu virt design
> 
> - Bring plenty of PCDs values and modifying the original PCIE values
> 
> - Less experience and not particularly confident in ACPI area so this might be
>   stupidly broken in a way I've not considered.

Hi Yuquan,

So an opening question for you.  What do you want to use this for?
If the aim is to do full software stack verification, I'd be tempted to
make a slightly more complex setup from the start and have at least
2 CXL host bridges so that you can enable interleaving + probably 2 or
3 CFMWS so that you can test that interleaving.
Even then it won't meet my requirements which is to stress the software
stack but then that's not the aim of sbsa ref so fair enough.

What you have here looks good to me in general, just superficial
suggestions in the various patches.

Thanks,

Jonathan