| Message ID | 20241213074143.374-1-smadhavan@nvidia.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC,v1] cxl: add support for cxl reset | expand |
Srirangan Madhavan wrote: > Currently, there is a lack of support for the CXL Reset method for cxl > devices. This change adds the support and implements the common reset > steps as laid out by the CXL Spec v3.1 Sections 9.6 & 9.7. I'm not sure what the use case for this support is. Do you have a use case in mind? What happens if there are current cxl regions mapped to the device being reset? I don't think it is enough to flush the caches. Section 9.7 talks about system software requirements for the HDMs. How are those requirements met with this patch? I think some more detail on the use case and how this functionality is to be used will clear up some of these questions. Ira > > For devices that support CXL Reset, cache lines are disabled and WB+I > is asserted. After waiting for cache invalid status, mem clr bit is > asserted and finally reset is initiated. After the recommended timeout > period, the reset status is checked and returned. > > The patch has not been tested completely as the infrastructure is not > yet in place. As this is my first kernel patch, I am looking to get the > community feedback through the RFC ahead of time. > > Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> > --- > drivers/pci/pci.c | 184 ++++++++++++++++++++++++++++++++++ > include/linux/pci.h | 2 +- > include/uapi/linux/pci_regs.h | 25 +++++ > 3 files changed, 210 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 0b29ec6e8e5e..cf4fc027565e 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -5034,6 +5034,12 @@ static int pci_dev_reset_slot_function(struct pci_dev *dev, bool probe) > return pci_reset_hotplug_slot(dev->slot->hotplug, probe); > } > > +static u16 cxl_device_dvsec(struct pci_dev *dev) > +{ > + return pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, > + PCI_DVSEC_CXL_DEV); > +} > + > static u16 cxl_port_dvsec(struct pci_dev *dev) > { > return pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, > @@ -5124,6 +5130,183 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe) > return rc; > } > > +static int cxl_reset_prepare(struct pci_dev *dev, u16 dvsec) > +{ > + u16 reg, val, cap; > + int rc; > + u32 timeout_us = 100, timeout_tot_us = 10000; > + > + /* > + * Wait for any pending transactions. > + * Assuming this does cxl.io stuff. > + */ > + if (!pci_wait_for_pending_transaction(dev)) > + pci_err(dev, "timed out waiting for pending transaction; performing cxl reset anyway\n"); > + > + /* > + * Disable caching and then write back and invalidate lines. > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, > + &cap); > + if (rc) > + return rc; > + > + if (!(cap & PCI_DVSEC_CXL_DEVCAP_CACHE_CAPABLE)) > + return 0; > + > + /* > + * Disable cache. > + * WB and invalidate cahce if capability is advertised. > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + ®); > + if (rc) > + return rc; > + val = reg | PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING; > + > + if (cap & PCI_DVSEC_CXL_DEVCAP_CACHE_WB_INVALIDATE) > + val = reg | PCI_DVSEC_CXL_DEVCTL2_INIT_CACHE_WB_INVALIDATE; > + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + val); > + > + /* > + * From Section 9.6: "Software may leverage the cache size reported in > + * the DVSEC CXL Capability2 register to compute a suitable timeout > + * value". > + * Given there is no conversion factor for cache size -> timeout, > + * setting timer for default 10ms. > + */ > + do { > + if (timeout_tot_us < 0) > + return -ETIMEDOUT; > + usleep_range(timeout_us, timeout_us+1); > + timeout_tot_us -= timeout_us; > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + ®); > + if (rc) > + return rc; > + } while (!(reg & PCI_DVSEC_CXL_DEVSTATUS2_CACHE_INVALID)); > + > + return 0; > +} > + > +/** > + * cxl_reset_init - initiate a cxl reset > + * @dev: device to reset > + * > + * Initiate a cxl reset. > + */ > +static int cxl_reset_init(struct pci_dev *dev, u16 dvsec) > +{ > + u16 reg, val; > + u32 timeout_ms; > + int rc; > + u32 reset_timeouts_ms[] = {10, 100, 1000, 10000, 100000}; > + > + /* > + * Check if CXL Reset MEM CLR is supported. > + * TODO: Add check to do MEM CLR only if requested. > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, > + ®); > + if (rc) > + return rc; > + > + if (reg & PCI_DVSEC_CXL_DEVCAP_CXL_RST_MEM_CLR) { > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + ®); > + if (rc) > + return rc; > + > + val = reg | PCI_DVSEC_CXL_DEVCTL2_CXL_RST_MEM_CLR_ENABLE; > + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + val); > + } > + > + /* > + * Read timeout value > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, > + ®); > + if (rc) > + return rc; > + timeout_ms = reset_timeouts_ms[FIELD_GET(PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_MASK, reg)]; > + > + /* > + * Write reset config > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + ®); > + if (rc) > + return rc; > + > + val = reg | PCI_DVSEC_CXL_DEVCTL2_CXL_INIT_RST; > + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + val); > + > + /* > + * Wait till timeout and then check reset status is complete. > + */ > + msleep(timeout_ms); > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVSTATUS2, > + ®); > + if (rc) > + return rc; > + if (reg & PCI_DVSEC_CXL_DEVSTATUS2_RST_ERR || > + ~reg & PCI_DVSEC_CXL_DEVSTATUS2_RST_COMPLETE) > + return -ETIMEDOUT; > + > + /* > + * Revert cashing disable. > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + ®); > + if (rc) > + return rc; > + val = (reg & (~PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING)); > + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, > + val); > + > + return 0; > +} > + > +/** > + * cxl_reset - initiate a cxl reset > + * @dev: device to reset > + * @probe: if true, return 0 if device can be reset this way > + * > + * Initiate a cxl reset on @dev. > + */ > +static int cxl_reset(struct pci_dev *dev, bool probe) > +{ > + u16 dvsec, reg; > + int rc; > + > + dvsec = cxl_device_dvsec(dev); > + if (!dvsec) > + return -ENOTTY; > + > + /* > + * Check if CXL Reset is supported. > + */ > + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, > + ®); > + if (rc) > + return -ENOTTY; > + > + if (~(reg & PCI_DVSEC_CXL_DEVCAP_CXL_RST)) > + return -ENOTTY; > + > + if (probe) > + return 0; > + > + rc = cxl_reset_prepare(dev, dvsec); > + if (rc) > + return rc; > + > + return cxl_reset_init(dev, dvsec); > +} > + > void pci_dev_lock(struct pci_dev *dev) > { > /* block PM suspend, driver probe, etc. */ > @@ -5210,6 +5393,7 @@ static const struct pci_reset_fn_method pci_reset_fn_methods[] = { > { pci_dev_acpi_reset, .name = "acpi" }, > { pcie_reset_flr, .name = "flr" }, > { pci_af_flr, .name = "af_flr" }, > + { cxl_reset, .name = "cxl_reset" }, > { pci_pm_reset, .name = "pm" }, > { pci_reset_bus_function, .name = "bus" }, > { cxl_reset_bus_function, .name = "cxl_bus" }, > diff --git a/include/linux/pci.h b/include/linux/pci.h > index db9b47ce3eef..60bacca8e379 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -51,7 +51,7 @@ > PCI_STATUS_PARITY) > > /* Number of reset methods used in pci_reset_fn_methods array in pci.c */ > -#define PCI_NUM_RESET_METHODS 8 > +#define PCI_NUM_RESET_METHODS 9 > > #define PCI_RESET_PROBE true > #define PCI_RESET_DO_RESET false > diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h > index 1601c7ed5fab..ec5b408a0789 100644 > --- a/include/uapi/linux/pci_regs.h > +++ b/include/uapi/linux/pci_regs.h > @@ -1209,6 +1209,31 @@ > #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX 0xff000000 > > /* Compute Express Link (CXL r3.1, sec 8.1.5) */ > +#define PCI_DVSEC_CXL_DEV 0 > +#define PCI_DVSEC_CXL_DEVCAP 0x0a > +#define PCI_DVSEC_CXL_DEVCAP_CACHE_CAPABLE 0x00000001 > +#define PCI_DVSEC_CXL_DEVCAP_CACHE_WB_INVALIDATE 0x00000040 > +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST 0x00000080 > +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_IND 0x8 > +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_MASK 0x00000700 > +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_MEM_CLR 0x00000800 > +#define PCI_DVSEC_CXL_DEVCTL 0x0c > +#define PCI_DVSEC_CXL_DEVCTL2 0x10 > +#define PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING 0x1 > +#define PCI_DVSEC_CXL_DEVCTL2_INIT_CACHE_WB_INVALIDATE 0x2 > +#define PCI_DVSEC_CXL_DEVCTL2_CXL_INIT_RST 0x4 > +#define PCI_DVSEC_CXL_DEVCTL2_CXL_RST_MEM_CLR_ENABLE 0x8 > +#define PCI_DVSEC_CXL_DEVSTATUS2 0x12 > +#define PCI_DVSEC_CXL_DEVSTATUS2_CACHE_INVALID 0x1 > +#define PCI_DVSEC_CXL_DEVSTATUS2_RST_COMPLETE 0x2 > +#define PCI_DVSEC_CXL_DEVSTATUS2_RST_ERR 0x4 > +#define PCI_DVSEC_CXL_DEVCAP2 0x16 > +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT 0x0000000F > +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_0 0x0 > +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_1 0x40 > +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_2 0x400 > +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE(x) (((x) & 0x0000FF00) >> 8) > + > #define PCI_DVSEC_CXL_PORT 3 > #define PCI_DVSEC_CXL_PORT_CTL 0x0c > #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001 > -- > 2.25.1 > >
External email: Use caution opening links or attachments Hello Ira! Thank you so much for your comments. I’ve added my response below. Apologies for the slight delay. > Srirangan Madhavan wrote: >> Currently, there is a lack of support for the CXL Reset method for cxl >> devices. This change adds the support and implements the common reset >> steps as laid out by the CXL Spec v3.1 Sections 9.6 & 9.7. > I'm not sure what the use case for this support is. Do you have a use > case in mind? I think that this change would be broadly useful due to few interrelated reasons. Without the change, even if devices are capable of CXL reset, users will not be able to access the method. Considering the changes to support type 2 devices are being added now, there will be more such devices. Adding the support would enable cases where more fine grained reset is required than bus reset methods/conventional reset. And FLR does not affect the CXL.cache/CXL.mem protocols. Spec also mentions cases like FM uses CXL reset for rebinding, error recovery. So I think that adding this change would be useful for all devices with the capability. > What happens if there are current cxl regions mapped to the device being > reset? I don't think it is enough to flush the caches. Section 9.7 > talks about system software requirements for the HDMs. How are those > requirements met with this patch? Sorry for not covering that. Your & other maintainer input here would be much appreciated. Since “system software” here is not very specifc, my proposal here is not very specific as well. Considering different platforms might have specific operations to effectively clear the regions, some type of optional helper routines in the CXL core would be appropriate. This way if required, a Type 2 device driver for example can choose to use it if required. Thank you. Regards, Srirangan.
Srirangan Madhavan wrote: > External email: Use caution opening links or attachments > > Hello Ira! > > Thank you so much for your comments. I’ve added my response below. > Apologies for the slight delay. > > > Srirangan Madhavan wrote: > >> Currently, there is a lack of support for the CXL Reset method for cxl > >> devices. This change adds the support and implements the common reset > >> steps as laid out by the CXL Spec v3.1 Sections 9.6 & 9.7. > > > I'm not sure what the use case for this support is. Do you have a use > > case in mind? > > I think that this change would be broadly useful due to few interrelated > reasons. Without the change, even if devices are capable of CXL reset, > users will not be able to access the method. Considering the changes > to support type 2 devices are being added now, there will be more such > devices. I see use cases for type 2 for sure. > > Adding the support would enable cases where more fine grained > reset is required than bus reset methods/conventional reset. And FLR does > not affect the CXL.cache/CXL.mem protocols. Spec also mentions cases like > FM uses CXL reset for rebinding, error recovery. So I think that adding this > change would be useful for all devices with the capability. > > > What happens if there are current cxl regions mapped to the device being > > reset? I don't think it is enough to flush the caches. Section 9.7 > > talks about system software requirements for the HDMs. How are those > > requirements met with this patch? > > Sorry for not covering that. Your & other maintainer input here would be > much appreciated. Since “system software” here is not very specifc, > my proposal here is not very specific as well. > > Considering different platforms might have specific operations to > effectively clear the regions, some type of optional helper routines in the > CXL core would be appropriate. This way if required, a Type 2 device driver > for example can choose to use it if required. There are a couple of ways to go here. For memory devices I think this is on the user. But I don't think that is a primary use case as I'm still unclear of why a user would need to do any reset of the memory device. For type 2 devices I think the use case will be device dependant. The specific driver in that case will need to tear down their regions (with or without user space coordination) and then issue the reset. All that said I think this patch belongs with a series which implements some device support. Or is more general for type 2. Which is what Alejandro (cc'ed) is working on. https://lore.kernel.org/linux-cxl/20241216161042.42108-1-alejandro.lucero-palau@amd.com/ Ira
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 0b29ec6e8e5e..cf4fc027565e 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5034,6 +5034,12 @@ static int pci_dev_reset_slot_function(struct pci_dev *dev, bool probe) return pci_reset_hotplug_slot(dev->slot->hotplug, probe); } +static u16 cxl_device_dvsec(struct pci_dev *dev) +{ + return pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, + PCI_DVSEC_CXL_DEV); +} + static u16 cxl_port_dvsec(struct pci_dev *dev) { return pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, @@ -5124,6 +5130,183 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe) return rc; } +static int cxl_reset_prepare(struct pci_dev *dev, u16 dvsec) +{ + u16 reg, val, cap; + int rc; + u32 timeout_us = 100, timeout_tot_us = 10000; + + /* + * Wait for any pending transactions. + * Assuming this does cxl.io stuff. + */ + if (!pci_wait_for_pending_transaction(dev)) + pci_err(dev, "timed out waiting for pending transaction; performing cxl reset anyway\n"); + + /* + * Disable caching and then write back and invalidate lines. + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, + &cap); + if (rc) + return rc; + + if (!(cap & PCI_DVSEC_CXL_DEVCAP_CACHE_CAPABLE)) + return 0; + + /* + * Disable cache. + * WB and invalidate cahce if capability is advertised. + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + ®); + if (rc) + return rc; + val = reg | PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING; + + if (cap & PCI_DVSEC_CXL_DEVCAP_CACHE_WB_INVALIDATE) + val = reg | PCI_DVSEC_CXL_DEVCTL2_INIT_CACHE_WB_INVALIDATE; + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + val); + + /* + * From Section 9.6: "Software may leverage the cache size reported in + * the DVSEC CXL Capability2 register to compute a suitable timeout + * value". + * Given there is no conversion factor for cache size -> timeout, + * setting timer for default 10ms. + */ + do { + if (timeout_tot_us < 0) + return -ETIMEDOUT; + usleep_range(timeout_us, timeout_us+1); + timeout_tot_us -= timeout_us; + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + ®); + if (rc) + return rc; + } while (!(reg & PCI_DVSEC_CXL_DEVSTATUS2_CACHE_INVALID)); + + return 0; +} + +/** + * cxl_reset_init - initiate a cxl reset + * @dev: device to reset + * + * Initiate a cxl reset. + */ +static int cxl_reset_init(struct pci_dev *dev, u16 dvsec) +{ + u16 reg, val; + u32 timeout_ms; + int rc; + u32 reset_timeouts_ms[] = {10, 100, 1000, 10000, 100000}; + + /* + * Check if CXL Reset MEM CLR is supported. + * TODO: Add check to do MEM CLR only if requested. + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, + ®); + if (rc) + return rc; + + if (reg & PCI_DVSEC_CXL_DEVCAP_CXL_RST_MEM_CLR) { + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + ®); + if (rc) + return rc; + + val = reg | PCI_DVSEC_CXL_DEVCTL2_CXL_RST_MEM_CLR_ENABLE; + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + val); + } + + /* + * Read timeout value + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, + ®); + if (rc) + return rc; + timeout_ms = reset_timeouts_ms[FIELD_GET(PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_MASK, reg)]; + + /* + * Write reset config + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + ®); + if (rc) + return rc; + + val = reg | PCI_DVSEC_CXL_DEVCTL2_CXL_INIT_RST; + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + val); + + /* + * Wait till timeout and then check reset status is complete. + */ + msleep(timeout_ms); + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVSTATUS2, + ®); + if (rc) + return rc; + if (reg & PCI_DVSEC_CXL_DEVSTATUS2_RST_ERR || + ~reg & PCI_DVSEC_CXL_DEVSTATUS2_RST_COMPLETE) + return -ETIMEDOUT; + + /* + * Revert cashing disable. + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + ®); + if (rc) + return rc; + val = (reg & (~PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING)); + pci_write_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCTL2, + val); + + return 0; +} + +/** + * cxl_reset - initiate a cxl reset + * @dev: device to reset + * @probe: if true, return 0 if device can be reset this way + * + * Initiate a cxl reset on @dev. + */ +static int cxl_reset(struct pci_dev *dev, bool probe) +{ + u16 dvsec, reg; + int rc; + + dvsec = cxl_device_dvsec(dev); + if (!dvsec) + return -ENOTTY; + + /* + * Check if CXL Reset is supported. + */ + rc = pci_read_config_word(dev, dvsec + PCI_DVSEC_CXL_DEVCAP, + ®); + if (rc) + return -ENOTTY; + + if (~(reg & PCI_DVSEC_CXL_DEVCAP_CXL_RST)) + return -ENOTTY; + + if (probe) + return 0; + + rc = cxl_reset_prepare(dev, dvsec); + if (rc) + return rc; + + return cxl_reset_init(dev, dvsec); +} + void pci_dev_lock(struct pci_dev *dev) { /* block PM suspend, driver probe, etc. */ @@ -5210,6 +5393,7 @@ static const struct pci_reset_fn_method pci_reset_fn_methods[] = { { pci_dev_acpi_reset, .name = "acpi" }, { pcie_reset_flr, .name = "flr" }, { pci_af_flr, .name = "af_flr" }, + { cxl_reset, .name = "cxl_reset" }, { pci_pm_reset, .name = "pm" }, { pci_reset_bus_function, .name = "bus" }, { cxl_reset_bus_function, .name = "cxl_bus" }, diff --git a/include/linux/pci.h b/include/linux/pci.h index db9b47ce3eef..60bacca8e379 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -51,7 +51,7 @@ PCI_STATUS_PARITY) /* Number of reset methods used in pci_reset_fn_methods array in pci.c */ -#define PCI_NUM_RESET_METHODS 8 +#define PCI_NUM_RESET_METHODS 9 #define PCI_RESET_PROBE true #define PCI_RESET_DO_RESET false diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index 1601c7ed5fab..ec5b408a0789 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -1209,6 +1209,31 @@ #define PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX 0xff000000 /* Compute Express Link (CXL r3.1, sec 8.1.5) */ +#define PCI_DVSEC_CXL_DEV 0 +#define PCI_DVSEC_CXL_DEVCAP 0x0a +#define PCI_DVSEC_CXL_DEVCAP_CACHE_CAPABLE 0x00000001 +#define PCI_DVSEC_CXL_DEVCAP_CACHE_WB_INVALIDATE 0x00000040 +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST 0x00000080 +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_IND 0x8 +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_TIMEOUT_MASK 0x00000700 +#define PCI_DVSEC_CXL_DEVCAP_CXL_RST_MEM_CLR 0x00000800 +#define PCI_DVSEC_CXL_DEVCTL 0x0c +#define PCI_DVSEC_CXL_DEVCTL2 0x10 +#define PCI_DVSEC_CXL_DEVCTL2_DISABLE_CACHING 0x1 +#define PCI_DVSEC_CXL_DEVCTL2_INIT_CACHE_WB_INVALIDATE 0x2 +#define PCI_DVSEC_CXL_DEVCTL2_CXL_INIT_RST 0x4 +#define PCI_DVSEC_CXL_DEVCTL2_CXL_RST_MEM_CLR_ENABLE 0x8 +#define PCI_DVSEC_CXL_DEVSTATUS2 0x12 +#define PCI_DVSEC_CXL_DEVSTATUS2_CACHE_INVALID 0x1 +#define PCI_DVSEC_CXL_DEVSTATUS2_RST_COMPLETE 0x2 +#define PCI_DVSEC_CXL_DEVSTATUS2_RST_ERR 0x4 +#define PCI_DVSEC_CXL_DEVCAP2 0x16 +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT 0x0000000F +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_0 0x0 +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_1 0x40 +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE_UNIT_2 0x400 +#define PCI_DVSEC_CXL_DEVCAP2_CACHE_SIZE(x) (((x) & 0x0000FF00) >> 8) + #define PCI_DVSEC_CXL_PORT 3 #define PCI_DVSEC_CXL_PORT_CTL 0x0c #define PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR 0x00000001
Currently, there is a lack of support for the CXL Reset method for cxl devices. This change adds the support and implements the common reset steps as laid out by the CXL Spec v3.1 Sections 9.6 & 9.7. For devices that support CXL Reset, cache lines are disabled and WB+I is asserted. After waiting for cache invalid status, mem clr bit is asserted and finally reset is initiated. After the recommended timeout period, the reset status is checked and returned. The patch has not been tested completely as the infrastructure is not yet in place. As this is my first kernel patch, I am looking to get the community feedback through the RFC ahead of time. Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> --- drivers/pci/pci.c | 184 ++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 +- include/uapi/linux/pci_regs.h | 25 +++++ 3 files changed, 210 insertions(+), 1 deletion(-)