| Message ID | 20250221043906.1593189-3-smadhavan@nvidia.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add CXL Reset Support for CXL Devices | expand |
On Thu, Feb 20, 2025 at 08:39:06PM -0800, Srirangan Madhavan wrote: > Type 2 devices are being introduced and will require finer-grained > reset mechanisms beyond bus-wide reset methods. > > Add support for CXL reset per CXL v3.2 Section 9.6/9.7 > > Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> > --- > drivers/pci/pci.c | 146 ++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci.c is basically a catch-all for anything that doesn't fit in one of the other .c files in drivers/pci. I'm slightly worried that this (otherwise legitimate) patch increases the clutter in pci.c further, rendering it unmaintainable in the long term. At the very least, I'm wondering if this can be #ifdef'ed to CONFIG_CXL_PCI? One idea would be to move this newly added reset method, as well as the existing cxl_reset_bus_function(), to a new drivers/pci/cxl.c file. I guess moving it to drivers/cxl/ isn't an option because cxl can be modular. Another idea would be to move all the reset handling (which makes up a significant portion of pci.c) to a separate drivers/pci/reset.c. This might be beyond the scope of your patch, but in the interim, maybe at least an #ifdef can be added because the PCI core is also used e.g. on memory-constrained wifi routers which don't care about CXL at all. Thanks, Lukas
On 2/21/25 04:39, Srirangan Madhavan wrote: > Type 2 devices are being introduced and will require finer-grained > reset mechanisms beyond bus-wide reset methods. > > Add support for CXL reset per CXL v3.2 Section 9.6/9.7 Hi, This seems too early as there is no plans for supporting CXL.cache yet. It is the second part of the current ongoing type2 support though. I guess starting the discussion about how to proceed is not a problem, so my comments below, but my first comment is if the decisions about what to do should be generic. I think having some helpers for accel drivers will be good, but the invocation should be in the hands of the accel driver. > Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> > --- > drivers/pci/pci.c | 146 ++++++++++++++++++++++++++++++++++++++++++++ > include/cxl/pci.h | 40 ++++++++---- > include/linux/pci.h | 2 +- > 3 files changed, 174 insertions(+), 14 deletions(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 3ab1871ecf8a..9108daae252b 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -5117,6 +5117,151 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe) > return rc; > } > > +static int cxl_reset_prepare(struct pci_dev *dev, u16 dvsec) > +{ > + u32 timeout_us = 100, timeout_tot_us = 10000; > + u16 reg, cap; > + int rc; > + > + if (!pci_wait_for_pending_transaction(dev)) > + pci_err(dev, "timed out waiting for pending transaction; performing cxl reset anyway\n"); > + > + /* Check if the device is cache capable. */ > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, &cap); > + if (rc) > + return rc; > + > + if (!(cap & CXL_DVSEC_CACHE_CAPABLE)) > + return 0; > + > + /* Disable cache. WB and invalidate cache if capability is advertised */ I do not know how safe is this. IMO, this needs to be synchronized by the accel driver which could imply to tell user space first. In our case it would imply to stop rx queues in the netdev for CXL.cache, and tx queues for CXL.mem. Doing it unconditionally could make current CXL transactions to stall ... although it could be argued the reset event implies something is broken, but let's try to do it properly if there is a chance of the system not unreliable. > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); > + if (rc) > + return rc; > + reg |= CXL_DVSEC_DISABLE_CACHING; > + /* > + * DEVCTL2 bits are written only once. So check WB+I capability while > + * keeping disable caching set. > + */ > + if (cap & CXL_DVSEC_CACHE_WBI_CAPABLE) > + reg |= CXL_DVSEC_INIT_CACHE_WBI; > + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); > + > + /* > + * From Section 9.6: "Software may leverage the cache size reported in > + * the DVSEC CXL Capability2 register to compute a suitable timeout > + * value". > + * Given there is no conversion factor for cache size -> timeout, > + * setting timer for default 10ms. > + */ > + do { > + if (timeout_tot_us == 0) > + return -ETIMEDOUT; > + usleep_range(timeout_us, timeout_us + 1); > + timeout_tot_us -= timeout_us; > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, > + ®); > + if (rc) > + return rc; > + } while (!(reg & CXL_DVSEC_CACHE_INVALID)); > + > + return 0; > +} > + > +static int cxl_reset_init(struct pci_dev *dev, u16 dvsec) I think cxl_reset_start after the *_prepare call makes more sense here. > +{ > + /* > + * Timeout values ref CXL Spec v3.2 Ch 8 Control and Status Registers, > + * under section 8.1.3.1 DVSEC CXL Capability. > + */ > + u32 reset_timeouts_ms[] = { 10, 100, 1000, 10000, 100000 }; > + u16 reg; > + u32 timeout_ms; > + int rc, ind; > + > + /* Check if CXL Reset MEM CLR is supported. */ > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); > + if (rc) > + return rc; > + > + if (reg & CXL_DVSEC_CXL_RST_MEM_CLR_CAPABLE) { > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, > + ®); > + if (rc) > + return rc; > + > + reg |= CXL_DVSEC_CXL_RST_MEM_CLR_ENABLE; > + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); > + } > + > + /* Read timeout value. */ > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); > + if (rc) > + return rc; > + ind = FIELD_GET(CXL_DVSEC_CXL_RST_TIMEOUT_MASK, reg); > + timeout_ms = reset_timeouts_ms[ind]; > + > + /* Write reset config. */ > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); > + if (rc) > + return rc; > + > + reg |= CXL_DVSEC_INIT_CXL_RESET; > + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); > + > + /* Wait till timeout and then check reset status is complete. */ > + msleep(timeout_ms); > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_STATUS2_OFFSET, ®); > + if (rc) > + return rc; > + if (reg & CXL_DVSEC_CXL_RESET_ERR || > + ~reg & CXL_DVSEC_CXL_RST_COMPLETE) > + return -ETIMEDOUT; > + > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); > + if (rc) > + return rc; > + reg &= (~CXL_DVSEC_DISABLE_CACHING); > + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); > + > + return 0; > +} > + > +/** > + * cxl_reset - initiate a cxl reset > + * @dev: device to reset > + * @probe: if true, return 0 if device can be reset this way > + * > + * Initiate a cxl reset on @dev. > + */ > +static int cxl_reset(struct pci_dev *dev, bool probe) > +{ > + u16 dvsec, reg; > + int rc; > + > + dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, > + CXL_DVSEC_PCIE_DEVICE); > + if (!dvsec) > + return -ENOTTY; > + > + /* Check if CXL Reset is supported. */ > + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); > + if (rc) > + return -ENOTTY; > + > + if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) > + return -ENOTTY; > + > + if (probe) > + return 0; > + > + rc = cxl_reset_prepare(dev, dvsec); > + if (rc) > + return rc; > + > + return cxl_reset_init(dev, dvsec); One thing this does not cover, and I do not know if it should, is the fact that the device CXL regs will be reset, so the question is if the old values should be restored or the device/driver should go through the same initialization, if a hotplug device, or do it specifically if already present at boot time and the BIOS doing that first initialization. In one case the restoration needs to happen, in the other the old values/objects need to be removed. I think the second case is more problematic because this is likely involving CXL root complex configuration performed by the BIOS ... Not trivial at all IMO. This is the main concern I expressed some time ago when looking at how type2 should support resets. Anyway, thank you for sending this series which will foster further discussions about all this.
On Fri, Feb 21, 2025 at 11:45:56AM +0100, Lukas Wunner wrote: > On Thu, Feb 20, 2025 at 08:39:06PM -0800, Srirangan Madhavan wrote: > > Type 2 devices are being introduced and will require finer-grained > > reset mechanisms beyond bus-wide reset methods. > > > > Add support for CXL reset per CXL v3.2 Section 9.6/9.7 > > > > Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> > > --- > > drivers/pci/pci.c | 146 ++++++++++++++++++++++++++++++++++++++++++++ > > drivers/pci/pci.c is basically a catch-all for anything that doesn't fit > in one of the other .c files in drivers/pci. I'm slightly worried that > this (otherwise legitimate) patch increases the clutter in pci.c further, > rendering it unmaintainable in the long term. +1 The reset-related content in drivers/pci/pci.c has been growing recently. Maybe we should consider moving it all to a reset.c file. > At the very least, I'm wondering if this can be #ifdef'ed to > CONFIG_CXL_PCI? > > One idea would be to move this newly added reset method, as well as the > existing cxl_reset_bus_function(), to a new drivers/pci/cxl.c file. > > I guess moving it to drivers/cxl/ isn't an option because cxl can be > modular. > > Another idea would be to move all the reset handling (which makes up > a significant portion of pci.c) to a separate drivers/pci/reset.c. > This might be beyond the scope of your patch, but in the interim, > maybe at least an #ifdef can be added because the PCI core is also > used e.g. on memory-constrained wifi routers which don't care about > CXL at all. Agree, we'll need some way to make this optional. Bjorn
Hi Srirangan, kernel test robot noticed the following build warnings: [auto build test WARNING on next-20250220] [cannot apply to pci/next pci/for-linus linus/master v6.14-rc3 v6.14-rc2 v6.14-rc1 v6.14-rc3] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Srirangan-Madhavan/cxl-de-duplicate-cxl-DVSEC-reg-defines/20250221-124043 base: next-20250220 patch link: https://lore.kernel.org/r/20250221043906.1593189-3-smadhavan%40nvidia.com patch subject: [PATCH v2 2/2] cxl: add support for cxl reset config: arc-randconfig-001-20250222 (https://download.01.org/0day-ci/archive/20250222/202502221354.Zmyi5Mgf-lkp@intel.com/config) compiler: arc-elf-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250222/202502221354.Zmyi5Mgf-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202502221354.Zmyi5Mgf-lkp@intel.com/ All warnings (new ones prefixed by >>): drivers/pci/pci.c: In function 'cxl_reset': >> drivers/pci/pci.c:5258:17: warning: suggest parentheses around comparison in operand of '&' [-Wparentheses] 5258 | if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) | ^ vim +5258 drivers/pci/pci.c 5235 5236 /** 5237 * cxl_reset - initiate a cxl reset 5238 * @dev: device to reset 5239 * @probe: if true, return 0 if device can be reset this way 5240 * 5241 * Initiate a cxl reset on @dev. 5242 */ 5243 static int cxl_reset(struct pci_dev *dev, bool probe) 5244 { 5245 u16 dvsec, reg; 5246 int rc; 5247 5248 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, 5249 CXL_DVSEC_PCIE_DEVICE); 5250 if (!dvsec) 5251 return -ENOTTY; 5252 5253 /* Check if CXL Reset is supported. */ 5254 rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); 5255 if (rc) 5256 return -ENOTTY; 5257 > 5258 if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) 5259 return -ENOTTY; 5260 5261 if (probe) 5262 return 0; 5263 5264 rc = cxl_reset_prepare(dev, dvsec); 5265 if (rc) 5266 return rc; 5267 5268 return cxl_reset_init(dev, dvsec); 5269 } 5270
Hi Srirangan, kernel test robot noticed the following build warnings: [auto build test WARNING on next-20250220] [cannot apply to pci/next pci/for-linus linus/master v6.14-rc3 v6.14-rc2 v6.14-rc1 v6.14-rc3] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Srirangan-Madhavan/cxl-de-duplicate-cxl-DVSEC-reg-defines/20250221-124043 base: next-20250220 patch link: https://lore.kernel.org/r/20250221043906.1593189-3-smadhavan%40nvidia.com patch subject: [PATCH v2 2/2] cxl: add support for cxl reset config: arm64-randconfig-003-20250222 (https://download.01.org/0day-ci/archive/20250222/202502221438.j0UgOryU-lkp@intel.com/config) compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250222/202502221438.j0UgOryU-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202502221438.j0UgOryU-lkp@intel.com/ All warnings (new ones prefixed by >>): >> drivers/pci/pci.c:5258:10: warning: & has lower precedence than ==; == will be evaluated first [-Wparentheses] 5258 | if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/pci/pci.c:5258:10: note: place parentheses around the '==' expression to silence this warning 5258 | if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) | ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/pci/pci.c:5258:10: note: place parentheses around the & expression to evaluate it first 5258 | if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) | ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 warning generated. vim +5258 drivers/pci/pci.c 5235 5236 /** 5237 * cxl_reset - initiate a cxl reset 5238 * @dev: device to reset 5239 * @probe: if true, return 0 if device can be reset this way 5240 * 5241 * Initiate a cxl reset on @dev. 5242 */ 5243 static int cxl_reset(struct pci_dev *dev, bool probe) 5244 { 5245 u16 dvsec, reg; 5246 int rc; 5247 5248 dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, 5249 CXL_DVSEC_PCIE_DEVICE); 5250 if (!dvsec) 5251 return -ENOTTY; 5252 5253 /* Check if CXL Reset is supported. */ 5254 rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); 5255 if (rc) 5256 return -ENOTTY; 5257 > 5258 if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) 5259 return -ENOTTY; 5260 5261 if (probe) 5262 return 0; 5263 5264 rc = cxl_reset_prepare(dev, dvsec); 5265 if (rc) 5266 return rc; 5267 5268 return cxl_reset_init(dev, dvsec); 5269 } 5270
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 3ab1871ecf8a..9108daae252b 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5117,6 +5117,151 @@ static int cxl_reset_bus_function(struct pci_dev *dev, bool probe) return rc; } +static int cxl_reset_prepare(struct pci_dev *dev, u16 dvsec) +{ + u32 timeout_us = 100, timeout_tot_us = 10000; + u16 reg, cap; + int rc; + + if (!pci_wait_for_pending_transaction(dev)) + pci_err(dev, "timed out waiting for pending transaction; performing cxl reset anyway\n"); + + /* Check if the device is cache capable. */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, &cap); + if (rc) + return rc; + + if (!(cap & CXL_DVSEC_CACHE_CAPABLE)) + return 0; + + /* Disable cache. WB and invalidate cache if capability is advertised */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); + if (rc) + return rc; + reg |= CXL_DVSEC_DISABLE_CACHING; + /* + * DEVCTL2 bits are written only once. So check WB+I capability while + * keeping disable caching set. + */ + if (cap & CXL_DVSEC_CACHE_WBI_CAPABLE) + reg |= CXL_DVSEC_INIT_CACHE_WBI; + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); + + /* + * From Section 9.6: "Software may leverage the cache size reported in + * the DVSEC CXL Capability2 register to compute a suitable timeout + * value". + * Given there is no conversion factor for cache size -> timeout, + * setting timer for default 10ms. + */ + do { + if (timeout_tot_us == 0) + return -ETIMEDOUT; + usleep_range(timeout_us, timeout_us + 1); + timeout_tot_us -= timeout_us; + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, + ®); + if (rc) + return rc; + } while (!(reg & CXL_DVSEC_CACHE_INVALID)); + + return 0; +} + +static int cxl_reset_init(struct pci_dev *dev, u16 dvsec) +{ + /* + * Timeout values ref CXL Spec v3.2 Ch 8 Control and Status Registers, + * under section 8.1.3.1 DVSEC CXL Capability. + */ + u32 reset_timeouts_ms[] = { 10, 100, 1000, 10000, 100000 }; + u16 reg; + u32 timeout_ms; + int rc, ind; + + /* Check if CXL Reset MEM CLR is supported. */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); + if (rc) + return rc; + + if (reg & CXL_DVSEC_CXL_RST_MEM_CLR_CAPABLE) { + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, + ®); + if (rc) + return rc; + + reg |= CXL_DVSEC_CXL_RST_MEM_CLR_ENABLE; + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); + } + + /* Read timeout value. */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); + if (rc) + return rc; + ind = FIELD_GET(CXL_DVSEC_CXL_RST_TIMEOUT_MASK, reg); + timeout_ms = reset_timeouts_ms[ind]; + + /* Write reset config. */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); + if (rc) + return rc; + + reg |= CXL_DVSEC_INIT_CXL_RESET; + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); + + /* Wait till timeout and then check reset status is complete. */ + msleep(timeout_ms); + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_STATUS2_OFFSET, ®); + if (rc) + return rc; + if (reg & CXL_DVSEC_CXL_RESET_ERR || + ~reg & CXL_DVSEC_CXL_RST_COMPLETE) + return -ETIMEDOUT; + + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, ®); + if (rc) + return rc; + reg &= (~CXL_DVSEC_DISABLE_CACHING); + pci_write_config_word(dev, dvsec + CXL_DVSEC_CTRL2_OFFSET, reg); + + return 0; +} + +/** + * cxl_reset - initiate a cxl reset + * @dev: device to reset + * @probe: if true, return 0 if device can be reset this way + * + * Initiate a cxl reset on @dev. + */ +static int cxl_reset(struct pci_dev *dev, bool probe) +{ + u16 dvsec, reg; + int rc; + + dvsec = pci_find_dvsec_capability(dev, PCI_VENDOR_ID_CXL, + CXL_DVSEC_PCIE_DEVICE); + if (!dvsec) + return -ENOTTY; + + /* Check if CXL Reset is supported. */ + rc = pci_read_config_word(dev, dvsec + CXL_DVSEC_CAP_OFFSET, ®); + if (rc) + return -ENOTTY; + + if (reg & CXL_DVSEC_CXL_RST_CAPABLE == 0) + return -ENOTTY; + + if (probe) + return 0; + + rc = cxl_reset_prepare(dev, dvsec); + if (rc) + return rc; + + return cxl_reset_init(dev, dvsec); +} + void pci_dev_lock(struct pci_dev *dev) { /* block PM suspend, driver probe, etc. */ @@ -5203,6 +5348,7 @@ const struct pci_reset_fn_method pci_reset_fn_methods[] = { { pci_dev_acpi_reset, .name = "acpi" }, { pcie_reset_flr, .name = "flr" }, { pci_af_flr, .name = "af_flr" }, + { cxl_reset, .name = "cxl_reset" }, { pci_pm_reset, .name = "pm" }, { pci_reset_bus_function, .name = "bus" }, { cxl_reset_bus_function, .name = "cxl_bus" }, diff --git a/include/cxl/pci.h b/include/cxl/pci.h index 3977425ec477..05d4b1a63cfe 100644 --- a/include/cxl/pci.h +++ b/include/cxl/pci.h @@ -13,19 +13,33 @@ /* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ #define CXL_DVSEC_PCIE_DEVICE 0 -#define CXL_DVSEC_CAP_OFFSET 0xA -#define CXL_DVSEC_MEM_CAPABLE BIT(2) -#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) -#define CXL_DVSEC_CTRL_OFFSET 0xC -#define CXL_DVSEC_MEM_ENABLE BIT(2) -#define CXL_DVSEC_RANGE_SIZE_HIGH(i) (0x18 + (i * 0x10)) -#define CXL_DVSEC_RANGE_SIZE_LOW(i) (0x1C + (i * 0x10)) -#define CXL_DVSEC_MEM_INFO_VALID BIT(0) -#define CXL_DVSEC_MEM_ACTIVE BIT(1) -#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) -#define CXL_DVSEC_RANGE_BASE_HIGH(i) (0x20 + (i * 0x10)) -#define CXL_DVSEC_RANGE_BASE_LOW(i) (0x24 + (i * 0x10)) -#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) +#define CXL_DVSEC_CAP_OFFSET 0xA +#define CXL_DVSEC_CACHE_CAPABLE BIT(0) +#define CXL_DVSEC_MEM_CAPABLE BIT(2) +#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) +#define CXL_DVSEC_CACHE_WBI_CAPABLE BIT(6) +#define CXL_DVSEC_CXL_RST_CAPABLE BIT(7) +#define CXL_DVSEC_CXL_RST_TIMEOUT_MASK GENMASK(10, 8) +#define CXL_DVSEC_CXL_RST_MEM_CLR_CAPABLE BIT(11) +#define CXL_DVSEC_CTRL_OFFSET 0xC +#define CXL_DVSEC_MEM_ENABLE BIT(2) +#define CXL_DVSEC_CTRL2_OFFSET 0x10 +#define CXL_DVSEC_DISABLE_CACHING BIT(0) +#define CXL_DVSEC_INIT_CACHE_WBI BIT(1) +#define CXL_DVSEC_INIT_CXL_RESET BIT(2) +#define CXL_DVSEC_CXL_RST_MEM_CLR_ENABLE BIT(3) +#define CXL_DVSEC_STATUS2_OFFSET 0x12 +#define CXL_DVSEC_CACHE_INVALID BIT(0) +#define CXL_DVSEC_CXL_RST_COMPLETE BIT(1) +#define CXL_DVSEC_CXL_RESET_ERR BIT(2) +#define CXL_DVSEC_RANGE_SIZE_HIGH(i) (0x18 + ((i) * 0x10)) +#define CXL_DVSEC_RANGE_SIZE_LOW(i) (0x1C + ((i) * 0x10)) +#define CXL_DVSEC_MEM_INFO_VALID BIT(0) +#define CXL_DVSEC_MEM_ACTIVE BIT(1) +#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) +#define CXL_DVSEC_RANGE_BASE_HIGH(i) (0x20 + ((i) * 0x10)) +#define CXL_DVSEC_RANGE_BASE_LOW(i) (0x24 + ((i) * 0x10)) +#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) #define CXL_DVSEC_RANGE_MAX 2 diff --git a/include/linux/pci.h b/include/linux/pci.h index 47b31ad724fa..efcb06598f26 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -51,7 +51,7 @@ PCI_STATUS_PARITY) /* Number of reset methods used in pci_reset_fn_methods array in pci.c */ -#define PCI_NUM_RESET_METHODS 8 +#define PCI_NUM_RESET_METHODS 9 #define PCI_RESET_PROBE true #define PCI_RESET_DO_RESET false
Type 2 devices are being introduced and will require finer-grained reset mechanisms beyond bus-wide reset methods. Add support for CXL reset per CXL v3.2 Section 9.6/9.7 Signed-off-by: Srirangan Madhavan <smadhavan@nvidia.com> --- drivers/pci/pci.c | 146 ++++++++++++++++++++++++++++++++++++++++++++ include/cxl/pci.h | 40 ++++++++---- include/linux/pci.h | 2 +- 3 files changed, 174 insertions(+), 14 deletions(-)