Documentation: CXL Maturity Map

Commit Message

Dan Williams May 25, 2024, 12:53 a.m. UTC

Provide a survey of the work-in-progress maturity (implementation
status) of various aspects of the CXL subsystem.

Clarify that in addition to ongoing upkeep relative to specification
updates, there are some long running themes in the driver that respond
to the discovery of new corner cases (bugs) and new use cases (feature
extensions).

The primary audience is distribution maintainers, but it also serves as
a guide for kernel developers to understand what aspects of the CXL
subsystem need more help. It is a landing page to document ongoing
progress, and a guide to discern exposure to work-in-progress features.

Help wanted / welcome to expand on the "Details" section.

Signed-off-by: Dan Williams <[e-mail]>
---
 Documentation/driver-api/cxl/index.rst        |    2 
 Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
 MAINTAINERS                                   |    1 
 3 files changed, 176 insertions(+)
 create mode 100644 Documentation/driver-api/cxl/maturity-map.rst

Comments

Alison Schofield June 6, 2024, 3:23 a.m. UTC | #1

On Fri, May 24, 2024 at 05:53:49PM -0700, Dan Williams wrote:
> Provide a survey of the work-in-progress maturity (implementation
> status) of various aspects of the CXL subsystem.
> 
> Clarify that in addition to ongoing upkeep relative to specification
> updates, there are some long running themes in the driver that respond
> to the discovery of new corner cases (bugs) and new use cases (feature
> extensions).
> 
> The primary audience is distribution maintainers, but it also serves as
> a guide for kernel developers to understand what aspects of the CXL
> subsystem need more help. It is a landing page to document ongoing
> progress, and a guide to discern exposure to work-in-progress features.

Hi Dan,

I'm wondering if Scan Media can be added to the list. I was within 
striking distance of posting that set when higher priority work called.
Since it got pushed aside no one has cared :(  I'd welcome the chance
to contribute it upstream if there is a justification. Can we add it
here as a [0] here to keep it on the radar.

-- Alison

> 
> Help wanted / welcome to expand on the "Details" section.
> 
> Signed-off-by: Dan Williams <[e-mail]>
> ---
>  Documentation/driver-api/cxl/index.rst        |    2 
>  Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
>  MAINTAINERS                                   |    1 
>  3 files changed, 176 insertions(+)
>  create mode 100644 Documentation/driver-api/cxl/maturity-map.rst
> 
> diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
> index 036e49553542..12b82725d322 100644
> --- a/Documentation/driver-api/cxl/index.rst
> +++ b/Documentation/driver-api/cxl/index.rst
> @@ -9,4 +9,6 @@ Compute Express Link
>  
>     memory-devices
>  
> +   maturity-map
> +
>  .. only::  subproject and html
> diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
> new file mode 100644
> index 000000000000..9c5bff6484dd
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/maturity-map.rst
> @@ -0,0 +1,173 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>
> +
> +===========================================
> +Compute Express Link Subsystem Maturity Map
> +===========================================
> +
> +The Linux CXL subsystem tracks the dynamic `CXL specification
> +<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
> +continues to respond to new use cases with new features, capability
> +updates and fixes. At any given point some aspects of the subsystem are
> +more mature than others. While the periodic pull requests summarize the
> +`work being incorporated each merge window
> +<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,

Thanks for the lore link!

Jonathan Cameron June 6, 2024, 2:40 p.m. UTC | #2

On Fri, 24 May 2024 17:53:49 -0700
Dan Williams <[e-mail]> wrote:

> Provide a survey of the work-in-progress maturity (implementation
> status) of various aspects of the CXL subsystem.
> 
> Clarify that in addition to ongoing upkeep relative to specification
> updates, there are some long running themes in the driver that respond
> to the discovery of new corner cases (bugs) and new use cases (feature
> extensions).
> 
> The primary audience is distribution maintainers, but it also serves as
> a guide for kernel developers to understand what aspects of the CXL
> subsystem need more help. It is a landing page to document ongoing
> progress, and a guide to discern exposure to work-in-progress features.
> 
> Help wanted / welcome to expand on the "Details" section.
> 
> Signed-off-by: Dan Williams <[e-mail]>
This feels like a really bit opportunity for bikeshedding..

Ah well, a few comments inline. I've tried restrain my natural tendency
to argue about number assignments.

Adam's email address was garbled in my email client - fixed up.

> ---
>  Documentation/driver-api/cxl/index.rst        |    2 
>  Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
>  MAINTAINERS                                   |    1 
>  3 files changed, 176 insertions(+)
>  create mode 100644 Documentation/driver-api/cxl/maturity-map.rst
> 
> diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
> index 036e49553542..12b82725d322 100644
> --- a/Documentation/driver-api/cxl/index.rst
> +++ b/Documentation/driver-api/cxl/index.rst
> @@ -9,4 +9,6 @@ Compute Express Link
>  
>     memory-devices
>  
> +   maturity-map
> +
>  .. only::  subproject and html
> diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
> new file mode 100644
> index 000000000000..9c5bff6484dd
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/maturity-map.rst
> @@ -0,0 +1,173 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>
> +
> +===========================================
> +Compute Express Link Subsystem Maturity Map
> +===========================================
> +
> +The Linux CXL subsystem tracks the dynamic `CXL specification
> +<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
> +continues to respond to new use cases with new features, capability
> +updates and fixes. At any given point some aspects of the subsystem are
> +more mature than others. While the periodic pull requests summarize the
> +`work being incorporated each merge window
> +<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,
> +those do not always convey progress relative to a starting point and a
> +future end goal.
> +
> +What follows is a coarse breakdown of the subsystem's major
> +responsibilities along with a maturity score. The expectation is that
> +the change-history of this document provides an overview summary of the
> +subsystem maturation over time.
> +
> +The maturity scores are:
> +
> +- [3] Mature: Work in this area is complete and no changes on the horizon.
> +  Note that this score can regress from one kernel release to the next
> +  based on new test results or end user reports.
> +
> +- [2] Stabilizing: Major functionality operational, common cases are
> +  mature, but known corner cases are still a work in progress.
> +
> +- [1] Initial: Capability that has exited the Proof of Concept phase, but
> +  may still have significant gaps to close and fixes to apply as real
> +  world testing occurs.
> +
> +- [0] Known gap: Feature is on a medium to long term horizon to
> +  implement.  If the specification has a feature that does even have a '0'
> +  score in this document, there is a good chance that no one in the
> +  [e-mail] community has started to look at it.
> +
> +- X: Out of scope for kernel enabling, or kernel enabling not required
> +
> +Feature and Capabilities
> +========================
> +
> +Enumeration / Provisioning
> +--------------------------
> +All of the fundamental enumeration an object model of the subsystem is
> +in place, but there are several corner cases that are pending closure.
> +

Feels like we should in passing mention that we support none of the
bits of CXL fabrics that are host visible (so mainly G-FAM I think).

[0] Fabrics / G-FAM (chapter 7)
[0] Global Access Enpoint

> +
> +* [2] CXL Window Enumeration
> +
> +  * [0] :ref:`Extended-linear memory-side cache <extended-linear>`
> +  * [0] Low Memory-hole
> +  * [0] Hetero-interleave
> +
> +* [2] Switch Enumeration
> +
> +  * [0] CXL register enumeration link-up dependency
> +
> +* [2] HDM Decoder Configuration
> +
> +  * [0] Decoder target and granularity constraints

Feels like you added this one just so we can tick it off ;)

> +
> +2 Performance enumeration
> +
> +  * [3] Endpoint CDAT
> +  * [3] Switch CDAT
> +  * [1] CDAT to Core-mm integration

[1] x86
[0] Arm64
[0] All other arch.

I guess could argue that's a core-mm problem, but the CEDT code
just all functions that are only implemented on x86.

> +  * [1] Shared link

[0] Shared link. We've not merged that one yet though getting closer.

> +
> +* [2] Hotplug
> +  (see CXL Window Enumeration)
> +
> +  * [0] Handle Soft Reserved conflicts
> +
> +* [0] :ref:`RCH link status <rch-link-status>`
> +
> +RAS
> +---
> +In many ways CXL can be seen as a standardization of what would normally
> +be handled by custom EDAC drivers. The open development here is
> +mainly caused by the enumeration corner cases above.
> +
> +* [3] Component events (OS)
> +* [2] Component events (FFM)
> +* [1] Endpoint protocol errors (OS)
> +* [1] Endpoint protocol errors (FFM)
> +* [0] Switch protocol errors (OS)
> +* [1] Switch protocol errors (FFM)
> +* [2] DPA->HPA Address translation
> +
> +    * [1] XOR Interleave translation
> +      (see CXL Window Enumeration)
> +
> +* [1] Memory Failure coordination
> +* [0] Scrub control

[0] PPR
[0] Sparing 
[0] Device built in test

> +* [2] ACPI error injection EINJ
> +
> +  * [0] EINJ v2
> +  * [X] Compliance DOE
> +
> +* [2] Native error injection
> +* [3] RCH error handling
> +* [1] VH error handling
> +
> +Mailbox commands
> +----------------
> +
> +* [3] Firmware update
> +* [3] Health / Alerts
> +* [1] :ref:`Background commands <background-commands>`
> +* [3] Santization
> +* [3] Security commands
> +* [3] RAW Command Debug Passthrough
> +* [0] CEL-only-validtion Passthrough
> +* [0] Switch CCI
> +* [3] Timestamp
> +* [1] PMU

Not mailbox, plus breakdown into switch vs type3 - see below.

> +* [1] PMEM labels
> +* [0] PMEM GPF / Dirty Shutdown
> +

PMU
---

[1] Type 3 PMU
[0] Switch USP/ DSP, Root Port

> +Security
> +--------
> +
> +* [X] CXL Trusted Execution Environment Security Protocol (TSP)
> +* [X] CXL IDE (subsumed by TSP)
> +
> +Multi-host memory
> +-----------------
> +
> +* [0] Dynamic Capacity Device Support
> +* [0] Sharing

Break this one up

Memory-pooling
--------------

[1] Hotplug of LDs (via PCI hotplug)
[0] Dynamic Capacity Device (DCD) Support

Multi-host sharing
------------------

Requires DCD support.
[0] Hardware coherent shared memory
[0] Software managed coherency shared memory

> +
> +Accelerator
> +-----------
> +
> +* [0] Accelerator memory enumeration HDM-D (CXL 1.1/2.0 Type-2)
> +* [0] Accelerator memory enumeration HDM-DB (CXL 3.0 Type-2)
> +* [0] CXL.cache 68b (CXL 2.0)
> +* [0] CXL.cache 256b Cache IDs (CXL 3.0)
> +
> +User Flow Support
> +-----------------
> +
> +* [0] HPA->DPA Address translation (need xormaps export solution)

Adam Manzanares June 17, 2024, 4:58 p.m. UTC | #3

On Fri, May 24, 2024 at 05:53:49PM -0700, Dan Williams wrote:
> Provide a survey of the work-in-progress maturity (implementation
> status) of various aspects of the CXL subsystem.
> 
> Clarify that in addition to ongoing upkeep relative to specification
> updates, there are some long running themes in the driver that respond
> to the discovery of new corner cases (bugs) and new use cases (feature
> extensions).
> 
> The primary audience is distribution maintainers, but it also serves as
> a guide for kernel developers to understand what aspects of the CXL
> subsystem need more help. It is a landing page to document ongoing
> progress, and a guide to discern exposure to work-in-progress features.
> 

Thanks for putting this together Dan and thanks Jonathan for putting this
thread in my inbox.

> Help wanted / welcome to expand on the "Details" section.
> 
> Signed-off-by: Dan Williams <[e-mail]>
> ---
>  Documentation/driver-api/cxl/index.rst        |    2 
>  Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
>  MAINTAINERS                                   |    1 
>  3 files changed, 176 insertions(+)
>  create mode 100644 Documentation/driver-api/cxl/maturity-map.rst
> 
> diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
> index 036e49553542..12b82725d322 100644
> --- a/Documentation/driver-api/cxl/index.rst
> +++ b/Documentation/driver-api/cxl/index.rst
> @@ -9,4 +9,6 @@ Compute Express Link
>  
>     memory-devices
>  
> +   maturity-map
> +
>  .. only::  subproject and html
> diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
> new file mode 100644
> index 000000000000..9c5bff6484dd
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/maturity-map.rst
> @@ -0,0 +1,173 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>
> +
> +===========================================
> +Compute Express Link Subsystem Maturity Map
> +===========================================
> +
> +The Linux CXL subsystem tracks the dynamic `CXL specification
> +<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
> +continues to respond to new use cases with new features, capability
> +updates and fixes. At any given point some aspects of the subsystem are
> +more mature than others. While the periodic pull requests summarize the
> +`work being incorporated each merge window
> +<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,
> +those do not always convey progress relative to a starting point and a
> +future end goal.
> +
> +What follows is a coarse breakdown of the subsystem's major
> +responsibilities along with a maturity score. The expectation is that
> +the change-history of this document provides an overview summary of the
> +subsystem maturation over time.
> +
> +The maturity scores are:
> +
> +- [3] Mature: Work in this area is complete and no changes on the horizon.
> +  Note that this score can regress from one kernel release to the next
> +  based on new test results or end user reports.
> +
> +- [2] Stabilizing: Major functionality operational, common cases are
> +  mature, but known corner cases are still a work in progress.
> +
> +- [1] Initial: Capability that has exited the Proof of Concept phase, but
> +  may still have significant gaps to close and fixes to apply as real
> +  world testing occurs.
> +
> +- [0] Known gap: Feature is on a medium to long term horizon to
> +  implement.  If the specification has a feature that does even have a '0'

s/that does even have/that does not even have

> +  score in this document, there is a good chance that no one in the
> +  [e-mail] community has started to look at it.
> +
> +- X: Out of scope for kernel enabling, or kernel enabling not required
> +
> +Feature and Capabilities
> +========================
> +
> +Enumeration / Provisioning
> +--------------------------
> +All of the fundamental enumeration an object model of the subsystem is
> +in place, but there are several corner cases that are pending closure.
> +
> +
> +* [2] CXL Window Enumeration
> +
> +  * [0] :ref:`Extended-linear memory-side cache <extended-linear>`
> +  * [0] Low Memory-hole
> +  * [0] Hetero-interleave
> +
> +* [2] Switch Enumeration
> +
> +  * [0] CXL register enumeration link-up dependency
> +
> +* [2] HDM Decoder Configuration
> +
> +  * [0] Decoder target and granularity constraints
> +
> +2 Performance enumeration
> +
> +  * [3] Endpoint CDAT
> +  * [3] Switch CDAT
> +  * [1] CDAT to Core-mm integration
> +  * [1] Shared link
> +
> +* [2] Hotplug
> +  (see CXL Window Enumeration)
> +
> +  * [0] Handle Soft Reserved conflicts
> +
> +* [0] :ref:`RCH link status <rch-link-status>`
> +
> +RAS
> +---
> +In many ways CXL can be seen as a standardization of what would normally
> +be handled by custom EDAC drivers. The open development here is
> +mainly caused by the enumeration corner cases above.
> +
> +* [3] Component events (OS)
> +* [2] Component events (FFM)
> +* [1] Endpoint protocol errors (OS)
> +* [1] Endpoint protocol errors (FFM)
> +* [0] Switch protocol errors (OS)
> +* [1] Switch protocol errors (FFM)
> +* [2] DPA->HPA Address translation
> +
> +    * [1] XOR Interleave translation
> +      (see CXL Window Enumeration)
> +
> +* [1] Memory Failure coordination
> +* [0] Scrub control
> +* [2] ACPI error injection EINJ
> +
> +  * [0] EINJ v2
> +  * [X] Compliance DOE
> +
> +* [2] Native error injection
> +* [3] RCH error handling
> +* [1] VH error handling
> +
> +Mailbox commands
> +----------------
> +
> +* [3] Firmware update
> +* [3] Health / Alerts
> +* [1] :ref:`Background commands <background-commands>`
> +* [3] Santization

s/Santization/Sanitation

> +* [3] Security commands
> +* [3] RAW Command Debug Passthrough
> +* [0] CEL-only-validtion Passthrough
> +* [0] Switch CCI
> +* [3] Timestamp
> +* [1] PMU
> +* [1] PMEM labels
> +* [0] PMEM GPF / Dirty Shutdown
> +
> +Security
> +--------
> +
> +* [X] CXL Trusted Execution Environment Security Protocol (TSP)
> +* [X] CXL IDE (subsumed by TSP)
> +
> +Multi-host memory
> +-----------------
> +
> +* [0] Dynamic Capacity Device Support
> +* [0] Sharing
> +
> +Accelerator
> +-----------
> +
> +* [0] Accelerator memory enumeration HDM-D (CXL 1.1/2.0 Type-2)
> +* [0] Accelerator memory enumeration HDM-DB (CXL 3.0 Type-2)
> +* [0] CXL.cache 68b (CXL 2.0)
> +* [0] CXL.cache 256b Cache IDs (CXL 3.0)
> +
> +User Flow Support
> +-----------------
> +
> +* [0] HPA->DPA Address translation (need xormaps export solution)
> +
> +Details
> +=======
> +
> +.. _extended-linear:
> +
> +* **Extended-linear memory-side cache**: An HMAT proposal to enumerate the presence of a
> +  memory-side cache where the cache capacity extends the SRAT address
> +  range capacity. `See the ECN
> +  <https://lore.kernel.org/linux-cxl/[e-mail]/>`_
> +  for more details:
> +
> +.. _rch-link-status:
> +
> +* **RCH Link Status**: RCH (Restricted CXL Host) topologies, end up
> +  hiding some standard registers like PCIe Link Status / Capabilities in
> +  the CXL RCRB (Root Complex Register Block).
> +
> +.. _background-commands:
> +
> +* **Background commands**: The CXL background command mechanism is
> +  awkward as the single slot is monopolized potentially indefinitely by
> +  various commands. A `cancel on conflict
> +  <http://lore.kernel.org/r/[e-mail]>`_
> +  facility is needed to make sure the kernel can ensure forward progress
> +  of priority commands.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e533476081ba..d092b2ae5f25 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -5331,6 +5331,7 @@ M:	Ira Weiny <[e-mail]>
>  M:	Dan Williams <[e-mail]>
>  L:	[e-mail]
>  S:	Maintained
> +F:	Documentation/driver-api/cxl
>  F:	drivers/cxl/
>  F:	include/linux/einj-cxl.h
>  F:	include/linux/cxl-event.h
>

Davidlohr Bueso June 17, 2024, 10:25 p.m. UTC | #4

Hi Alison,

On Wed, 05 Jun 2024, Alison Schofield wrote:\n
>I'm wondering if Scan Media can be added to the list. I was within
>striking distance of posting that set when higher priority work called.
>Since it got pushed aside no one has cared :(  I'd welcome the chance
>to contribute it upstream if there is a justification. Can we add it
>here as a [0] here to keep it on the radar.

I'd be interested in seeing these posted, and am curious as to what policy
you chose to use to trigger this. fyi there is support in Jonathan's qemu
newer branches for Scan Media, maybe that might serve you for any testing.
The kernel implementation I used to exercise the qemu changes was based
on poison list overflow, fwiw.

Thanks,
Davidlohr

Dan Williams June 17, 2024, 10:38 p.m. UTC | #5

Alison Schofield wrote:
> On Fri, May 24, 2024 at 05:53:49PM -0700, Dan Williams wrote:
> > Provide a survey of the work-in-progress maturity (implementation
> > status) of various aspects of the CXL subsystem.
> > 
> > Clarify that in addition to ongoing upkeep relative to specification
> > updates, there are some long running themes in the driver that respond
> > to the discovery of new corner cases (bugs) and new use cases (feature
> > extensions).
> > 
> > The primary audience is distribution maintainers, but it also serves as
> > a guide for kernel developers to understand what aspects of the CXL
> > subsystem need more help. It is a landing page to document ongoing
> > progress, and a guide to discern exposure to work-in-progress features.
> 
> Hi Dan,
> 
> I'm wondering if Scan Media can be added to the list. I was within 
> striking distance of posting that set when higher priority work called.
> Since it got pushed aside no one has cared :(  I'd welcome the chance
> to contribute it upstream if there is a justification. Can we add it
> here as a [0] here to keep it on the radar.

Added.

Dan Williams June 17, 2024, 10:50 p.m. UTC | #6

Jonathan Cameron wrote:
> On Fri, 24 May 2024 17:53:49 -0700
> Dan Williams <[e-mail]> wrote:
> 
> > Provide a survey of the work-in-progress maturity (implementation
> > status) of various aspects of the CXL subsystem.
> > 
> > Clarify that in addition to ongoing upkeep relative to specification
> > updates, there are some long running themes in the driver that respond
> > to the discovery of new corner cases (bugs) and new use cases (feature
> > extensions).
> > 
> > The primary audience is distribution maintainers, but it also serves as
> > a guide for kernel developers to understand what aspects of the CXL
> > subsystem need more help. It is a landing page to document ongoing
> > progress, and a guide to discern exposure to work-in-progress features.
> > 
> > Help wanted / welcome to expand on the "Details" section.
> > 
> > Signed-off-by: Dan Williams <[e-mail]>
> This feels like a really bit opportunity for bikeshedding..
> 
> Ah well, a few comments inline. I've tried restrain my natural tendency
> to argue about number assignments.
> 
> Adam's email address was garbled in my email client - fixed up.
> 
> > ---
> >  Documentation/driver-api/cxl/index.rst        |    2 
> >  Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
> >  MAINTAINERS                                   |    1 
> >  3 files changed, 176 insertions(+)
> >  create mode 100644 Documentation/driver-api/cxl/maturity-map.rst
> > 
> > diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
> > index 036e49553542..12b82725d322 100644
> > --- a/Documentation/driver-api/cxl/index.rst
> > +++ b/Documentation/driver-api/cxl/index.rst
> > @@ -9,4 +9,6 @@ Compute Express Link
> >  
> >     memory-devices
> >  
> > +   maturity-map
> > +
> >  .. only::  subproject and html
> > diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
> > new file mode 100644
> > index 000000000000..9c5bff6484dd
> > --- /dev/null
> > +++ b/Documentation/driver-api/cxl/maturity-map.rst
> > @@ -0,0 +1,173 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +.. include:: <isonum.txt>
> > +
> > +===========================================
> > +Compute Express Link Subsystem Maturity Map
> > +===========================================
> > +
> > +The Linux CXL subsystem tracks the dynamic `CXL specification
> > +<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
> > +continues to respond to new use cases with new features, capability
> > +updates and fixes. At any given point some aspects of the subsystem are
> > +more mature than others. While the periodic pull requests summarize the
> > +`work being incorporated each merge window
> > +<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,
> > +those do not always convey progress relative to a starting point and a
> > +future end goal.
> > +
> > +What follows is a coarse breakdown of the subsystem's major
> > +responsibilities along with a maturity score. The expectation is that
> > +the change-history of this document provides an overview summary of the
> > +subsystem maturation over time.
> > +
> > +The maturity scores are:
> > +
> > +- [3] Mature: Work in this area is complete and no changes on the horizon.
> > +  Note that this score can regress from one kernel release to the next
> > +  based on new test results or end user reports.
> > +
> > +- [2] Stabilizing: Major functionality operational, common cases are
> > +  mature, but known corner cases are still a work in progress.
> > +
> > +- [1] Initial: Capability that has exited the Proof of Concept phase, but
> > +  may still have significant gaps to close and fixes to apply as real
> > +  world testing occurs.
> > +
> > +- [0] Known gap: Feature is on a medium to long term horizon to
> > +  implement.  If the specification has a feature that does even have a '0'
> > +  score in this document, there is a good chance that no one in the
> > +  [e-mail] community has started to look at it.
> > +
> > +- X: Out of scope for kernel enabling, or kernel enabling not required
> > +
> > +Feature and Capabilities
> > +========================
> > +
> > +Enumeration / Provisioning
> > +--------------------------
> > +All of the fundamental enumeration an object model of the subsystem is
> > +in place, but there are several corner cases that are pending closure.
> > +
> 
> Feels like we should in passing mention that we support none of the
> bits of CXL fabrics that are host visible (so mainly G-FAM I think).
> 
> [0] Fabrics / G-FAM (chapter 7)
> [0] Global Access Enpoint

Are those [0] or [X], not even sure what the kernel is expected to do with those?

> > +
> > +* [2] CXL Window Enumeration
> > +
> > +  * [0] :ref:`Extended-linear memory-side cache <extended-linear>`
> > +  * [0] Low Memory-hole
> > +  * [0] Hetero-interleave
> > +
> > +* [2] Switch Enumeration
> > +
> > +  * [0] CXL register enumeration link-up dependency
> > +
> > +* [2] HDM Decoder Configuration
> > +
> > +  * [0] Decoder target and granularity constraints
> 
> Feels like you added this one just so we can tick it off ;)

True, the list was a bit of brain dump. I also expect that "git log -p
Documentation/driver-api/cxl/maturity-map.rst" to show small edits like
this. No need to overthink this informal list.

> > +
> > +2 Performance enumeration
> > +
> > +  * [3] Endpoint CDAT
> > +  * [3] Switch CDAT
> > +  * [1] CDAT to Core-mm integration
> 
> [1] x86
> [0] Arm64
> [0] All other arch.
> 
> I guess could argue that's a core-mm problem, but the CEDT code
> just all functions that are only implemented on x86.

Oh, good point, will add.

> 
> > +  * [1] Shared link
> 
> [0] Shared link. We've not merged that one yet though getting closer.

Ok.

> 
> > +
> > +* [2] Hotplug
> > +  (see CXL Window Enumeration)
> > +
> > +  * [0] Handle Soft Reserved conflicts
> > +
> > +* [0] :ref:`RCH link status <rch-link-status>`
> > +
> > +RAS
> > +---
> > +In many ways CXL can be seen as a standardization of what would normally
> > +be handled by custom EDAC drivers. The open development here is
> > +mainly caused by the enumeration corner cases above.
> > +
> > +* [3] Component events (OS)
> > +* [2] Component events (FFM)
> > +* [1] Endpoint protocol errors (OS)
> > +* [1] Endpoint protocol errors (FFM)
> > +* [0] Switch protocol errors (OS)
> > +* [1] Switch protocol errors (FFM)
> > +* [2] DPA->HPA Address translation
> > +
> > +    * [1] XOR Interleave translation
> > +      (see CXL Window Enumeration)
> > +
> > +* [1] Memory Failure coordination
> > +* [0] Scrub control
> 
> [0] PPR
> [0] Sparing 
> [0] Device built in test

Added.

> 
> > +* [2] ACPI error injection EINJ
> > +
> > +  * [0] EINJ v2
> > +  * [X] Compliance DOE
> > +
> > +* [2] Native error injection
> > +* [3] RCH error handling
> > +* [1] VH error handling
> > +
> > +Mailbox commands
> > +----------------
> > +
> > +* [3] Firmware update
> > +* [3] Health / Alerts
> > +* [1] :ref:`Background commands <background-commands>`
> > +* [3] Santization
> > +* [3] Security commands
> > +* [3] RAW Command Debug Passthrough
> > +* [0] CEL-only-validtion Passthrough
> > +* [0] Switch CCI
> > +* [3] Timestamp
> > +* [1] PMU
> 
> Not mailbox, plus breakdown into switch vs type3 - see below.

Done.

> 
> > +* [1] PMEM labels
> > +* [0] PMEM GPF / Dirty Shutdown
> > +
> 
> PMU
> ---
> 
> [1] Type 3 PMU
> [0] Switch USP/ DSP, Root Port

...got it.

> 
> > +Security
> > +--------
> > +
> > +* [X] CXL Trusted Execution Environment Security Protocol (TSP)
> > +* [X] CXL IDE (subsumed by TSP)
> > +
> > +Multi-host memory
> > +-----------------
> > +
> > +* [0] Dynamic Capacity Device Support
> > +* [0] Sharing
> 
> Break this one up
> 
> Memory-pooling
> --------------
> 
> [1] Hotplug of LDs (via PCI hotplug)
> [0] Dynamic Capacity Device (DCD) Support
> 
> Multi-host sharing
> ------------------
> 
> Requires DCD support.
> [0] Hardware coherent shared memory
> [0] Software managed coherency shared memory

Looks good.

Dan Williams June 17, 2024, 10:54 p.m. UTC | #7

Adam Manzanares wrote:
> On Fri, May 24, 2024 at 05:53:49PM -0700, Dan Williams wrote:
> > Provide a survey of the work-in-progress maturity (implementation
> > status) of various aspects of the CXL subsystem.
> > 
> > Clarify that in addition to ongoing upkeep relative to specification
> > updates, there are some long running themes in the driver that respond
> > to the discovery of new corner cases (bugs) and new use cases (feature
> > extensions).
> > 
> > The primary audience is distribution maintainers, but it also serves as
> > a guide for kernel developers to understand what aspects of the CXL
> > subsystem need more help. It is a landing page to document ongoing
> > progress, and a guide to discern exposure to work-in-progress features.
> > 
[..]
> > +- [0] Known gap: Feature is on a medium to long term horizon to
> > +  implement.  If the specification has a feature that does even have a '0'
> 
> s/that does even have/that does not even have

got it.

[..]
> > +Mailbox commands
> > +----------------
> > +
> > +* [3] Firmware update
> > +* [3] Health / Alerts
> > +* [1] :ref:`Background commands <background-commands>`
> > +* [3] Santization
> 
> s/Santization/Sanitation

Hmm, no, "sanitation" appears nowhere in the spec.

Alejandro Lucero Palau June 19, 2024, 4:02 p.m. UTC | #8

Apologies for the late answer.


RAS is covered but what about how CXL/PCI devices are handled after a 
slot reset?

I did expose my concern in that area when discussing the Type2 support 
patchset, and if I'm not wrong this is not properly supported: nothing 
for device detachment which should occur before the slot reset, nothing 
for resume function after the slot reset. This is obviously more 
important for Type2 devices and maybe not required for Type3.


On 5/25/24 01:53, Dan Williams wrote:
> Provide a survey of the work-in-progress maturity (implementation
> status) of various aspects of the CXL subsystem.
>
> Clarify that in addition to ongoing upkeep relative to specification
> updates, there are some long running themes in the driver that respond
> to the discovery of new corner cases (bugs) and new use cases (feature
> extensions).
>
> The primary audience is distribution maintainers, but it also serves as
> a guide for kernel developers to understand what aspects of the CXL
> subsystem need more help. It is a landing page to document ongoing
> progress, and a guide to discern exposure to work-in-progress features.
>
> Help wanted / welcome to expand on the "Details" section.
>
> Signed-off-by: Dan Williams <[e-mail]>
> ---
>   Documentation/driver-api/cxl/index.rst        |    2
>   Documentation/driver-api/cxl/maturity-map.rst |  173 +++++++++++++++++++++++++
>   MAINTAINERS                                   |    1
>   3 files changed, 176 insertions(+)
>   create mode 100644 Documentation/driver-api/cxl/maturity-map.rst
>
> diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
> index 036e49553542..12b82725d322 100644
> --- a/Documentation/driver-api/cxl/index.rst
> +++ b/Documentation/driver-api/cxl/index.rst
> @@ -9,4 +9,6 @@ Compute Express Link
>   
>      memory-devices
>   
> +   maturity-map
> +
>   .. only::  subproject and html
> diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
> new file mode 100644
> index 000000000000..9c5bff6484dd
> --- /dev/null
> +++ b/Documentation/driver-api/cxl/maturity-map.rst
> @@ -0,0 +1,173 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. include:: <isonum.txt>
> +
> +===========================================
> +Compute Express Link Subsystem Maturity Map
> +===========================================
> +
> +The Linux CXL subsystem tracks the dynamic `CXL specification
> +<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
> +continues to respond to new use cases with new features, capability
> +updates and fixes. At any given point some aspects of the subsystem are
> +more mature than others. While the periodic pull requests summarize the
> +`work being incorporated each merge window
> +<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,
> +those do not always convey progress relative to a starting point and a
> +future end goal.
> +
> +What follows is a coarse breakdown of the subsystem's major
> +responsibilities along with a maturity score. The expectation is that
> +the change-history of this document provides an overview summary of the
> +subsystem maturation over time.
> +
> +The maturity scores are:
> +
> +- [3] Mature: Work in this area is complete and no changes on the horizon.
> +  Note that this score can regress from one kernel release to the next
> +  based on new test results or end user reports.
> +
> +- [2] Stabilizing: Major functionality operational, common cases are
> +  mature, but known corner cases are still a work in progress.
> +
> +- [1] Initial: Capability that has exited the Proof of Concept phase, but
> +  may still have significant gaps to close and fixes to apply as real
> +  world testing occurs.
> +
> +- [0] Known gap: Feature is on a medium to long term horizon to
> +  implement.  If the specification has a feature that does even have a '0'
> +  score in this document, there is a good chance that no one in the
> +  [e-mail] community has started to look at it.
> +
> +- X: Out of scope for kernel enabling, or kernel enabling not required
> +
> +Feature and Capabilities
> +========================
> +
> +Enumeration / Provisioning
> +--------------------------
> +All of the fundamental enumeration an object model of the subsystem is
> +in place, but there are several corner cases that are pending closure.
> +
> +
> +* [2] CXL Window Enumeration
> +
> +  * [0] :ref:`Extended-linear memory-side cache <extended-linear>`
> +  * [0] Low Memory-hole
> +  * [0] Hetero-interleave
> +
> +* [2] Switch Enumeration
> +
> +  * [0] CXL register enumeration link-up dependency
> +
> +* [2] HDM Decoder Configuration
> +
> +  * [0] Decoder target and granularity constraints
> +
> +2 Performance enumeration
> +
> +  * [3] Endpoint CDAT
> +  * [3] Switch CDAT
> +  * [1] CDAT to Core-mm integration
> +  * [1] Shared link
> +
> +* [2] Hotplug
> +  (see CXL Window Enumeration)
> +
> +  * [0] Handle Soft Reserved conflicts
> +
> +* [0] :ref:`RCH link status <rch-link-status>`
> +
> +RAS
> +---
> +In many ways CXL can be seen as a standardization of what would normally
> +be handled by custom EDAC drivers. The open development here is
> +mainly caused by the enumeration corner cases above.
> +
> +* [3] Component events (OS)
> +* [2] Component events (FFM)
> +* [1] Endpoint protocol errors (OS)
> +* [1] Endpoint protocol errors (FFM)
> +* [0] Switch protocol errors (OS)
> +* [1] Switch protocol errors (FFM)
> +* [2] DPA->HPA Address translation
> +
> +    * [1] XOR Interleave translation
> +      (see CXL Window Enumeration)
> +
> +* [1] Memory Failure coordination
> +* [0] Scrub control
> +* [2] ACPI error injection EINJ
> +
> +  * [0] EINJ v2
> +  * [X] Compliance DOE
> +
> +* [2] Native error injection
> +* [3] RCH error handling
> +* [1] VH error handling
> +
> +Mailbox commands
> +----------------
> +
> +* [3] Firmware update
> +* [3] Health / Alerts
> +* [1] :ref:`Background commands <background-commands>`
> +* [3] Santization
> +* [3] Security commands
> +* [3] RAW Command Debug Passthrough
> +* [0] CEL-only-validtion Passthrough
> +* [0] Switch CCI
> +* [3] Timestamp
> +* [1] PMU
> +* [1] PMEM labels
> +* [0] PMEM GPF / Dirty Shutdown
> +
> +Security
> +--------
> +
> +* [X] CXL Trusted Execution Environment Security Protocol (TSP)
> +* [X] CXL IDE (subsumed by TSP)
> +
> +Multi-host memory
> +-----------------
> +
> +* [0] Dynamic Capacity Device Support
> +* [0] Sharing
> +
> +Accelerator
> +-----------
> +
> +* [0] Accelerator memory enumeration HDM-D (CXL 1.1/2.0 Type-2)
> +* [0] Accelerator memory enumeration HDM-DB (CXL 3.0 Type-2)
> +* [0] CXL.cache 68b (CXL 2.0)
> +* [0] CXL.cache 256b Cache IDs (CXL 3.0)
> +
> +User Flow Support
> +-----------------
> +
> +* [0] HPA->DPA Address translation (need xormaps export solution)
> +
> +Details
> +=======
> +
> +.. _extended-linear:
> +
> +* **Extended-linear memory-side cache**: An HMAT proposal to enumerate the presence of a
> +  memory-side cache where the cache capacity extends the SRAT address
> +  range capacity. `See the ECN
> +  <https://lore.kernel.org/linux-cxl/[e-mail]/>`_
> +  for more details:
> +
> +.. _rch-link-status:
> +
> +* **RCH Link Status**: RCH (Restricted CXL Host) topologies, end up
> +  hiding some standard registers like PCIe Link Status / Capabilities in
> +  the CXL RCRB (Root Complex Register Block).
> +
> +.. _background-commands:
> +
> +* **Background commands**: The CXL background command mechanism is
> +  awkward as the single slot is monopolized potentially indefinitely by
> +  various commands. A `cancel on conflict
> +  <http://lore.kernel.org/r/[e-mail]>`_
> +  facility is needed to make sure the kernel can ensure forward progress
> +  of priority commands.
> diff --git a/MAINTAINERS b/MAINTAINERS
> index e533476081ba..d092b2ae5f25 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -5331,6 +5331,7 @@ M:	Ira Weiny <[e-mail]>
>   M:	Dan Williams <[e-mail]>
>   L:	[e-mail]
>   S:	Maintained
> +F:	Documentation/driver-api/cxl
>   F:	drivers/cxl/
>   F:	include/linux/einj-cxl.h
>   F:	include/linux/cxl-event.h
>
>

Jonathan Cameron June 20, 2024, 5:45 p.m. UTC | #9

ling, or kernel enabling not required
> > > +
> > > +Feature and Capabilities
> > > +========================
> > > +
> > > +Enumeration / Provisioning
> > > +--------------------------
> > > +All of the fundamental enumeration an object model of the subsystem is
> > > +in place, but there are several corner cases that are pending closure.
> > > +  
> > 
> > Feels like we should in passing mention that we support none of the
> > bits of CXL fabrics that are host visible (so mainly G-FAM I think).
> > 
> > [0] Fabrics / G-FAM (chapter 7)
> > [0] Global Access Enpoint  
> 
> Are those [0] or [X], not even sure what the kernel is expected to do with those?

I'm not 100% sure, but they are host programmable and there
are a bunch of more general structures to program that are
'kind of like' the hdm decoders with stuff like interleave
setup.  So I think [0] but I've really not figure out the expected
software model for that stuff yet.

Jonathan

Adam Manzanares June 27, 2024, 5:52 p.m. UTC | #10

On Mon, Jun 17, 2024 at 03:54:31PM -0700, Dan Williams wrote:
> Adam Manzanares wrote:
> > On Fri, May 24, 2024 at 05:53:49PM -0700, Dan Williams wrote:
> > > Provide a survey of the work-in-progress maturity (implementation
> > > status) of various aspects of the CXL subsystem.
> > > 
> > > Clarify that in addition to ongoing upkeep relative to specification
> > > updates, there are some long running themes in the driver that respond
> > > to the discovery of new corner cases (bugs) and new use cases (feature
> > > extensions).
> > > 
> > > The primary audience is distribution maintainers, but it also serves as
> > > a guide for kernel developers to understand what aspects of the CXL
> > > subsystem need more help. It is a landing page to document ongoing
> > > progress, and a guide to discern exposure to work-in-progress features.
> > > 
> [..]
> > > +- [0] Known gap: Feature is on a medium to long term horizon to
> > > +  implement.  If the specification has a feature that does even have a '0'
> > 
> > s/that does even have/that does not even have
> 
> got it.
> 
> [..]
> > > +Mailbox commands
> > > +----------------
> > > +
> > > +* [3] Firmware update
> > > +* [3] Health / Alerts
> > > +* [1] :ref:`Background commands <background-commands>`
> > > +* [3] Santization
> > 
> > s/Santization/Sanitation
> 
> Hmm, no, "sanitation" appears nowhere in the spec.

s/Santization/Sanitization

11.5.5.7.1


Feel free to add

Reviewed-by: Adam Manzanares <[e-mail]>

Dan Williams July 3, 2024, 11:26 p.m. UTC | #11

Alejandro Lucero Palau wrote:
> Apologies for the late answer.
> 
> 
> RAS is covered but what about how CXL/PCI devices are handled after a 
> slot reset?
> 
> I did expose my concern in that area when discussing the Type2 support 
> patchset, and if I'm not wrong this is not properly supported: nothing 
> for device detachment which should occur before the slot reset, nothing 
> for resume function after the slot reset. This is obviously more 
> important for Type2 devices and maybe not required for Type3.

Circling back to this as I go to post v2...

I think those type-2 RAS aspects are important, but it is not clear that
they are a concern of the CXL core. Endpoint drivers are enabled to get
pre and post reset notification via their registered 'struct
pci_error_handlers' operations. This maturity list is more targeted at
the CXL core infrastructure capabilities and type-2 drivers are going to
have their own localized maturity level.

I do need to go look more closely at your RAS assertions on that set. I
had otherwise been waiting a new posting with the sfc integration.

Patch

diff --git a/Documentation/driver-api/cxl/index.rst b/Documentation/driver-api/cxl/index.rst
index 036e49553542..12b82725d322 100644
--- a/Documentation/driver-api/cxl/index.rst
+++ b/Documentation/driver-api/cxl/index.rst
@@ -9,4 +9,6 @@  Compute Express Link
 
    memory-devices
 
+   maturity-map
+
 .. only::  subproject and html
diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
new file mode 100644
index 000000000000..9c5bff6484dd
--- /dev/null
+++ b/Documentation/driver-api/cxl/maturity-map.rst
@@ -0,0 +1,173 @@ 
+.. SPDX-License-Identifier: GPL-2.0
+.. include:: <isonum.txt>
+
+===========================================
+Compute Express Link Subsystem Maturity Map
+===========================================
+
+The Linux CXL subsystem tracks the dynamic `CXL specification
+<https://computeexpresslink.org/cxl-specification-landing-page>`_ that
+continues to respond to new use cases with new features, capability
+updates and fixes. At any given point some aspects of the subsystem are
+more mature than others. While the periodic pull requests summarize the
+`work being incorporated each merge window
+<https://lore.kernel.org/linux-cxl/?q=s%3APULL+s%3ACXL+tc%3Atorvalds+NOT+s%3ARe>`_,
+those do not always convey progress relative to a starting point and a
+future end goal.
+
+What follows is a coarse breakdown of the subsystem's major
+responsibilities along with a maturity score. The expectation is that
+the change-history of this document provides an overview summary of the
+subsystem maturation over time.
+
+The maturity scores are:
+
+- [3] Mature: Work in this area is complete and no changes on the horizon.
+  Note that this score can regress from one kernel release to the next
+  based on new test results or end user reports.
+
+- [2] Stabilizing: Major functionality operational, common cases are
+  mature, but known corner cases are still a work in progress.
+
+- [1] Initial: Capability that has exited the Proof of Concept phase, but
+  may still have significant gaps to close and fixes to apply as real
+  world testing occurs.
+
+- [0] Known gap: Feature is on a medium to long term horizon to
+  implement.  If the specification has a feature that does even have a '0'
+  score in this document, there is a good chance that no one in the
+  [e-mail] community has started to look at it.
+
+- X: Out of scope for kernel enabling, or kernel enabling not required
+
+Feature and Capabilities
+========================
+
+Enumeration / Provisioning
+--------------------------
+All of the fundamental enumeration an object model of the subsystem is
+in place, but there are several corner cases that are pending closure.
+
+
+* [2] CXL Window Enumeration
+
+  * [0] :ref:`Extended-linear memory-side cache <extended-linear>`
+  * [0] Low Memory-hole
+  * [0] Hetero-interleave
+
+* [2] Switch Enumeration
+
+  * [0] CXL register enumeration link-up dependency
+
+* [2] HDM Decoder Configuration
+
+  * [0] Decoder target and granularity constraints
+
+2 Performance enumeration
+
+  * [3] Endpoint CDAT
+  * [3] Switch CDAT
+  * [1] CDAT to Core-mm integration
+  * [1] Shared link
+
+* [2] Hotplug
+  (see CXL Window Enumeration)
+
+  * [0] Handle Soft Reserved conflicts
+
+* [0] :ref:`RCH link status <rch-link-status>`
+
+RAS
+---
+In many ways CXL can be seen as a standardization of what would normally
+be handled by custom EDAC drivers. The open development here is
+mainly caused by the enumeration corner cases above.
+
+* [3] Component events (OS)
+* [2] Component events (FFM)
+* [1] Endpoint protocol errors (OS)
+* [1] Endpoint protocol errors (FFM)
+* [0] Switch protocol errors (OS)
+* [1] Switch protocol errors (FFM)
+* [2] DPA->HPA Address translation
+
+    * [1] XOR Interleave translation
+      (see CXL Window Enumeration)
+
+* [1] Memory Failure coordination
+* [0] Scrub control
+* [2] ACPI error injection EINJ
+
+  * [0] EINJ v2
+  * [X] Compliance DOE
+
+* [2] Native error injection
+* [3] RCH error handling
+* [1] VH error handling
+
+Mailbox commands
+----------------
+
+* [3] Firmware update
+* [3] Health / Alerts
+* [1] :ref:`Background commands <background-commands>`
+* [3] Santization
+* [3] Security commands
+* [3] RAW Command Debug Passthrough
+* [0] CEL-only-validtion Passthrough
+* [0] Switch CCI
+* [3] Timestamp
+* [1] PMU
+* [1] PMEM labels
+* [0] PMEM GPF / Dirty Shutdown
+
+Security
+--------
+
+* [X] CXL Trusted Execution Environment Security Protocol (TSP)
+* [X] CXL IDE (subsumed by TSP)
+
+Multi-host memory
+-----------------
+
+* [0] Dynamic Capacity Device Support
+* [0] Sharing
+
+Accelerator
+-----------
+
+* [0] Accelerator memory enumeration HDM-D (CXL 1.1/2.0 Type-2)
+* [0] Accelerator memory enumeration HDM-DB (CXL 3.0 Type-2)
+* [0] CXL.cache 68b (CXL 2.0)
+* [0] CXL.cache 256b Cache IDs (CXL 3.0)
+
+User Flow Support
+-----------------
+
+* [0] HPA->DPA Address translation (need xormaps export solution)
+
+Details
+=======
+
+.. _extended-linear:
+
+* **Extended-linear memory-side cache**: An HMAT proposal to enumerate the presence of a
+  memory-side cache where the cache capacity extends the SRAT address
+  range capacity. `See the ECN
+  <https://lore.kernel.org/linux-cxl/[e-mail]/>`_
+  for more details:
+
+.. _rch-link-status:
+
+* **RCH Link Status**: RCH (Restricted CXL Host) topologies, end up
+  hiding some standard registers like PCIe Link Status / Capabilities in
+  the CXL RCRB (Root Complex Register Block).
+
+.. _background-commands:
+
+* **Background commands**: The CXL background command mechanism is
+  awkward as the single slot is monopolized potentially indefinitely by
+  various commands. A `cancel on conflict
+  <http://lore.kernel.org/r/[e-mail]>`_
+  facility is needed to make sure the kernel can ensure forward progress
+  of priority commands.
diff --git a/MAINTAINERS b/MAINTAINERS
index e533476081ba..d092b2ae5f25 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5331,6 +5331,7 @@  M:	Ira Weiny <[e-mail]>
 M:	Dan Williams <[e-mail]>
 L:	[e-mail]
 S:	Maintained
+F:	Documentation/driver-api/cxl
 F:	drivers/cxl/
 F:	include/linux/einj-cxl.h
 F:	include/linux/cxl-event.h