[v3,0/9] cxl: Add support for Restricted CXL hosts (RCD mode)

Message

Robert Richter Nov. 9, 2022, 10:40 a.m. UTC

In Restricted CXL Device (RCD) mode (formerly referred to as CXL 1.1)
the PCIe enumeration hierarchy is different from CXL VH Enumeration
(formerly referred to as 2.0, for both modes see CXL spec 3.0: 9.11
and 9.12, [1]). This series adds support for RCD mode. It implements
the detection of Restricted CXL Hosts (RCHs) and its corresponding
Restricted CXL Devices (RCDs). It does the necessary enumeration of
ports and connects the endpoints. With all the plumbing an RCH/RCD
pair is registered at the Linux CXL bus and becomes visible in sysfs
in the same way as CXL VH hosts and devices do already. RCDs are
brought up as CXL endpoints and bound to subsequent drivers such as
cxl_mem.

For CXL VH the host driver (cxl_acpi) starts host bridge discovery
once the ACPI0017 CXL root device is detected and then searches for
ACPI0016 host bridges to enable CXL. This implementation requires the
ACPI device enumeration for RCD mode. It also expects the host's
downstream and upstream port RCRBs base address being reported by
firmware using the optional CEDT CHBS entry of the host bridge (see
CXL spec 3.0, 9.17.1.2).

RCD mode does not support hot-plug, so host discovery is at boot time
only.

This version bases on 2b76fc22aefd ("cxl/acpi: Improve debug messages
in cxl_acpi_probe()") containing the already accepted patches.

[1] https://www.computeexpresslink.org/spec-landing

---

v3:

 * Rebased onto 2b76fc22aefd ("cxl/acpi: Improve debug messages in
   cxl_acpi_probe()").
 * Added Co-developed-by tag. (Rafael)
 * Added public function cxl_rcrb_to_component() to regs.c for later
   reuse. Added arg to switch between upstream and downstream RCRB.
   (Dan, Dave)
 * Added patch to register CXL host ports by bridge device. Note the
   alias detection in ndctl (cxl list command) need to check both
   sysfs entries, firmware_node and physical_node. A rework is needed
   here. (Dan, Vishal)
 * Reworked implementation to skip intermediate port enumeration of
   restricted endpoints (RCDs). (Dan, Dave)
 * Added check to only register RCDs with device 0, function 0 as CXL
   memory.
 * Added Terry's patch to set ACPI's CXL _OSC to indicate CXL1.1
   support.

v2:
 * Reworked series to use add_host_bridge_dport() and
   add_host_bridge_uport(). There is a single cxl root device
   (ACPI0017) also for RCHs (must have a ACPI0016 id). (Dan)
 * Rebased onto 6.1-rc1.
 * Added a WARN_ON_ONCE() to CXL_RESOURCE_NONE check. Updated patch
   description with an example case. (Dan, Jonathan)
 * Added wrapper functions to devm_cxl_add_port() and
   devm_cxl_add_dport(). (Dan)
 * Dropped "PCI/ACPI: Link host bridge to its ACPI fw node" patch.
 * Updated spec refs to use 3.0. Added PCIe base spec refs. (Jonathan)
 * Reused UID detect code. (Dan)
 * Dropped "cxl/acpi: Specify module load order dependency for the
   cxl_acpi module" patch. Return -EINVAL if host not yet ready. (Dan)
 * Minor other changes.
 * Note: I haven't included most of the received Reviewed-by tags due
   to the major rework. In any case, thanks to all here.

Robert Richter (8):
  cxl/acpi: Register CXL host ports by bridge device
  cxl/acpi: Extract component registers of restricted hosts from RCRB
  cxl/mem: Adjust cxl_mem_find_port() to find an RCH's port
  cxl/mem: Skip intermediate port enumeration of restricted endpoints
    (RCDs)
  cxl/pci: Only register RCDs with device 0, function 0 as CXL memory
    device
  cxl/pci: Do not ignore PCI config read errors in match_add_dports()
  cxl/pci: Factor out code in match_add_dports() to pci_dev_add_dport()
  cxl/pci: Extend devm_cxl_port_enumerate_dports() to support restricted
    hosts (RCH)

Terry Bowman (1):
  cxl/acpi: Set ACPI's CXL _OSC to indicate CXL1.1 support

 drivers/acpi/pci_root.c |  1 +
 drivers/cxl/acpi.c      | 89 +++++++++++++++++++++++++++++------------
 drivers/cxl/core/pci.c  | 74 ++++++++++++++++++++++++++++------
 drivers/cxl/core/port.c | 56 +++++++++++++++++++++++++-
 drivers/cxl/core/regs.c | 46 +++++++++++++++++++++
 drivers/cxl/cxl.h       |  8 ++++
 drivers/cxl/pci.c       | 25 +++++++++++-
 7 files changed, 257 insertions(+), 42 deletions(-)


base-commit: 2b76fc22aefd39820c0520255875f99b326ede99