| Message ID | 20240702060418.387500-3-alistair.francis@wdc.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
| Series | [v13,1/4] PCI/DOE: Rename DOE protocol to feature | expand |
> @@ -707,6 +773,45 @@ void pci_doe_init(struct pci_dev *pdev) > pci_doe_destroy_mb(doe_mb); > } > } > + > + if (doe_mb) { > + xa_for_each(&doe_mb->feats, i, entry) > + num_features++; > + > + sysfs_attrs = kcalloc(num_features + 1, sizeof(*sysfs_attrs), GFP_KERNEL); > + if (!sysfs_attrs) > + return; > + > + attrs = kcalloc(num_features, sizeof(*attrs), GFP_KERNEL); > + if (!attrs) { > + kfree(sysfs_attrs); > + return; > + } > + > + doe_mb->device_attrs = attrs; > + doe_mb->sysfs_attrs = sysfs_attrs; > + > + xa_for_each(&doe_mb->feats, i, entry) { > + sysfs_attr_init(&attrs[i].attr); > + > + vid = xa_to_value(entry) >> 8; > + type = xa_to_value(entry) & 0xFF; > + > + attrs[i].attr.name = kasprintf(GFP_KERNEL, "%04lx:%02lx", vid, type); > + if (!attrs[i].attr.name) { > + pci_doe_sysfs_feature_remove(pdev, doe_mb); > + return; > + } > + attrs[i].attr.mode = 0444; > + attrs[i].show = pci_doe_sysfs_feature_show; > + > + sysfs_attrs[i] = &attrs[i].attr; > + } > + > + sysfs_attrs[num_features] = NULL; shouldn't sysfs_attrs[num_features] be already NULL since it's allocated using kcalloc() ? > + > + pci_doe_sysfs_group.attrs = sysfs_attrs; > + } -ck
On Tue, 2 Jul 2024 16:04:17 +1000 Alistair Francis <alistair23@gmail.com> wrote: > The PCIe 6 specification added support for the Data Object > Exchange (DOE). > When DOE is supported the DOE Discovery Feature must be implemented per > PCIe r6.1 sec 6.30.1.1. The protocol allows a requester to obtain > information about the other DOE features supported by the device. > > The kernel is already querying the DOE features supported and cacheing > the values. Expose the values in sysfs to allow user space to > determine which DOE features are supported by the PCIe device. > > By exposing the information to userspace tools like lspci can relay the > information to users. By listing all of the supported features we can > allow userspace to parse the list, which might include > vendor specific features as well as yet to be supported features. > > After this patch is supported you can see something like this when > attaching a DOE device > > $ ls /sys/devices/pci0000:00/0000:00:02.0//doe* > 0001:00 0001:01 0001:02 > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com> > --- > v13: > - Drop pci_doe_sysfs_init() and use pci_doe_sysfs_group > - As discussed in https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > we can just modify pci_doe_sysfs_group at the DOE init and let Can't do that as it is global so you expose the same DOE features for all DOEs. Also, I think that this is only processing features on last doe_mb found for a given device. Fix that and the duplicates problem resurfaces. > device_add() handle the sysfs attributes. > diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c > index defc4be81bd4..e7b702afce88 100644 > --- a/drivers/pci/doe.c > +++ b/drivers/pci/doe.c > + > static int pci_doe_wait(struct pci_doe_mb *doe_mb, unsigned long timeout) > { > if (wait_event_timeout(doe_mb->wq, > @@ -687,6 +747,12 @@ void pci_doe_init(struct pci_dev *pdev) > { > struct pci_doe_mb *doe_mb; > u16 offset = 0; > + struct attribute **sysfs_attrs; > + struct device_attribute *attrs; > + unsigned long num_features = 0; > + unsigned long i; > + unsigned long vid, type; > + void *entry; > int rc; > > xa_init(&pdev->doe_mbs); > @@ -707,6 +773,45 @@ void pci_doe_init(struct pci_dev *pdev) > pci_doe_destroy_mb(doe_mb); > } > } The above is looping over multiple DOEs but this just considers last one. That doesn't look right... I think this needs to be in the loop and having done that the duplicate handing may be an issue. I'm not sure what happens in that path with a presupplied set of attributes. > + > + if (doe_mb) { > + xa_for_each(&doe_mb->feats, i, entry) > + num_features++; > + > + sysfs_attrs = kcalloc(num_features + 1, sizeof(*sysfs_attrs), GFP_KERNEL); > + if (!sysfs_attrs) > + return; > + > + attrs = kcalloc(num_features, sizeof(*attrs), GFP_KERNEL); > + if (!attrs) { > + kfree(sysfs_attrs); > + return; > + } > + > + doe_mb->device_attrs = attrs; > + doe_mb->sysfs_attrs = sysfs_attrs; > + > + xa_for_each(&doe_mb->feats, i, entry) { > + sysfs_attr_init(&attrs[i].attr); > + > + vid = xa_to_value(entry) >> 8; > + type = xa_to_value(entry) & 0xFF; > + > + attrs[i].attr.name = kasprintf(GFP_KERNEL, "%04lx:%02lx", vid, type); > + if (!attrs[i].attr.name) { > + pci_doe_sysfs_feature_remove(pdev, doe_mb); > + return; > + } > + attrs[i].attr.mode = 0444; > + attrs[i].show = pci_doe_sysfs_feature_show; > + > + sysfs_attrs[i] = &attrs[i].attr; > + } > + > + sysfs_attrs[num_features] = NULL; > + > + pci_doe_sysfs_group.attrs = sysfs_attrs; Hmm. Isn't this global? What if you have multiple devices. > + } > } >
On Tue, Jul 2, 2024 at 11:58 PM Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote: > > On Tue, 2 Jul 2024 16:04:17 +1000 > Alistair Francis <alistair23@gmail.com> wrote: > > > The PCIe 6 specification added support for the Data Object > > Exchange (DOE). > > When DOE is supported the DOE Discovery Feature must be implemented per > > PCIe r6.1 sec 6.30.1.1. The protocol allows a requester to obtain > > information about the other DOE features supported by the device. > > > > The kernel is already querying the DOE features supported and cacheing > > the values. Expose the values in sysfs to allow user space to > > determine which DOE features are supported by the PCIe device. > > > > By exposing the information to userspace tools like lspci can relay the > > information to users. By listing all of the supported features we can > > allow userspace to parse the list, which might include > > vendor specific features as well as yet to be supported features. > > > > After this patch is supported you can see something like this when > > attaching a DOE device > > > > $ ls /sys/devices/pci0000:00/0000:00:02.0//doe* > > 0001:00 0001:01 0001:02 > > > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com> > > --- > > v13: > > - Drop pci_doe_sysfs_init() and use pci_doe_sysfs_group > > - As discussed in https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > > we can just modify pci_doe_sysfs_group at the DOE init and let > > Can't do that as it is global so you expose the same DOE features for > all DOEs. > > Also, I think that this is only processing features on last doe_mb found > for a given device. Fix that and the duplicates problem resurfaces. > > > > device_add() handle the sysfs attributes. > > > > diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c > > index defc4be81bd4..e7b702afce88 100644 > > --- a/drivers/pci/doe.c > > +++ b/drivers/pci/doe.c > > > + > > static int pci_doe_wait(struct pci_doe_mb *doe_mb, unsigned long timeout) > > { > > if (wait_event_timeout(doe_mb->wq, > > @@ -687,6 +747,12 @@ void pci_doe_init(struct pci_dev *pdev) > > { > > struct pci_doe_mb *doe_mb; > > u16 offset = 0; > > + struct attribute **sysfs_attrs; > > + struct device_attribute *attrs; > > + unsigned long num_features = 0; > > + unsigned long i; > > + unsigned long vid, type; > > + void *entry; > > int rc; > > > > xa_init(&pdev->doe_mbs); > > @@ -707,6 +773,45 @@ void pci_doe_init(struct pci_dev *pdev) > > pci_doe_destroy_mb(doe_mb); > > } > > } > > The above is looping over multiple DOEs but this just considers last one. > That doesn't look right... Yeah... That isn't > > I think this needs to be in the loop and having done that > the duplicate handing may be an issue. I'm not sure what happens > in that path with a presupplied set of attributes. > > > + > > + if (doe_mb) { > > + xa_for_each(&doe_mb->feats, i, entry) > > + num_features++; > > + > > + sysfs_attrs = kcalloc(num_features + 1, sizeof(*sysfs_attrs), GFP_KERNEL); > > + if (!sysfs_attrs) > > + return; > > + > > + attrs = kcalloc(num_features, sizeof(*attrs), GFP_KERNEL); > > + if (!attrs) { > > + kfree(sysfs_attrs); > > + return; > > + } > > + > > + doe_mb->device_attrs = attrs; > > + doe_mb->sysfs_attrs = sysfs_attrs; > > + > > + xa_for_each(&doe_mb->feats, i, entry) { > > + sysfs_attr_init(&attrs[i].attr); > > + > > + vid = xa_to_value(entry) >> 8; > > + type = xa_to_value(entry) & 0xFF; > > + > > + attrs[i].attr.name = kasprintf(GFP_KERNEL, "%04lx:%02lx", vid, type); > > + if (!attrs[i].attr.name) { > > + pci_doe_sysfs_feature_remove(pdev, doe_mb); > > + return; > > + } > > + attrs[i].attr.mode = 0444; > > + attrs[i].show = pci_doe_sysfs_feature_show; > > + > > + sysfs_attrs[i] = &attrs[i].attr; > > + } > > + > > + sysfs_attrs[num_features] = NULL; > > + > > + pci_doe_sysfs_group.attrs = sysfs_attrs; > Hmm. Isn't this global? What if you have multiple devices. Any input from a PCI maintainer here? There are basically two approaches. 1. We can have a pci_doe_sysfs_init() function that is called where we dynamically add the entries, like in v12 2. We can go down the dev->groups and device_add() path, like this patch and discussed at https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ For the second we will have to create a global pci_doe_sysfs_group that contains all possible DOE entries on the system and then have the show functions determine if they should be displayed for that device. Everytime we call pci_doe_init() we can check for any missing entries in pci_doe_sysfs_group.attrs and then realloc pci_doe_sysfs_group.attrs to add them. Untested, but that should work even for hot-plugged devices. pci_doe_sysfs_group.attrs would just grow forever though as I don't think we have an easy way to deallocate anything as we aren't sure if we are the only entry. Thoughts? Alistair
On Fri, 5 Jul 2024 11:24:25 +1000 Alistair Francis <alistair23@gmail.com> wrote: > On Tue, Jul 2, 2024 at 11:58 PM Jonathan Cameron > <Jonathan.Cameron@huawei.com> wrote: > > > > On Tue, 2 Jul 2024 16:04:17 +1000 > > Alistair Francis <alistair23@gmail.com> wrote: > > > > > The PCIe 6 specification added support for the Data Object > > > Exchange (DOE). > > > When DOE is supported the DOE Discovery Feature must be implemented per > > > PCIe r6.1 sec 6.30.1.1. The protocol allows a requester to obtain > > > information about the other DOE features supported by the device. > > > > > > The kernel is already querying the DOE features supported and cacheing > > > the values. Expose the values in sysfs to allow user space to > > > determine which DOE features are supported by the PCIe device. > > > > > > By exposing the information to userspace tools like lspci can relay the > > > information to users. By listing all of the supported features we can > > > allow userspace to parse the list, which might include > > > vendor specific features as well as yet to be supported features. > > > > > > After this patch is supported you can see something like this when > > > attaching a DOE device > > > > > > $ ls /sys/devices/pci0000:00/0000:00:02.0//doe* > > > 0001:00 0001:01 0001:02 > > > > > > Signed-off-by: Alistair Francis <alistair.francis@wdc.com> > > > --- > > > v13: > > > - Drop pci_doe_sysfs_init() and use pci_doe_sysfs_group > > > - As discussed in https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > > > we can just modify pci_doe_sysfs_group at the DOE init and let > > > > Can't do that as it is global so you expose the same DOE features for > > all DOEs. > > > > Also, I think that this is only processing features on last doe_mb found > > for a given device. Fix that and the duplicates problem resurfaces. > > > > > > > device_add() handle the sysfs attributes. > > > > > > > diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c > > > index defc4be81bd4..e7b702afce88 100644 > > > --- a/drivers/pci/doe.c > > > +++ b/drivers/pci/doe.c > > > > > + > > > static int pci_doe_wait(struct pci_doe_mb *doe_mb, unsigned long timeout) > > > { > > > if (wait_event_timeout(doe_mb->wq, > > > @@ -687,6 +747,12 @@ void pci_doe_init(struct pci_dev *pdev) > > > { > > > struct pci_doe_mb *doe_mb; > > > u16 offset = 0; > > > + struct attribute **sysfs_attrs; > > > + struct device_attribute *attrs; > > > + unsigned long num_features = 0; > > > + unsigned long i; > > > + unsigned long vid, type; > > > + void *entry; > > > int rc; > > > > > > xa_init(&pdev->doe_mbs); > > > @@ -707,6 +773,45 @@ void pci_doe_init(struct pci_dev *pdev) > > > pci_doe_destroy_mb(doe_mb); > > > } > > > } > > > > The above is looping over multiple DOEs but this just considers last one. > > That doesn't look right... > > Yeah... That isn't > > > > > I think this needs to be in the loop and having done that > > the duplicate handing may be an issue. I'm not sure what happens > > in that path with a presupplied set of attributes. > > > > > + > > > + if (doe_mb) { > > > + xa_for_each(&doe_mb->feats, i, entry) > > > + num_features++; > > > + > > > + sysfs_attrs = kcalloc(num_features + 1, sizeof(*sysfs_attrs), GFP_KERNEL); > > > + if (!sysfs_attrs) > > > + return; > > > + > > > + attrs = kcalloc(num_features, sizeof(*attrs), GFP_KERNEL); > > > + if (!attrs) { > > > + kfree(sysfs_attrs); > > > + return; > > > + } > > > + > > > + doe_mb->device_attrs = attrs; > > > + doe_mb->sysfs_attrs = sysfs_attrs; > > > + > > > + xa_for_each(&doe_mb->feats, i, entry) { > > > + sysfs_attr_init(&attrs[i].attr); > > > + > > > + vid = xa_to_value(entry) >> 8; > > > + type = xa_to_value(entry) & 0xFF; > > > + > > > + attrs[i].attr.name = kasprintf(GFP_KERNEL, "%04lx:%02lx", vid, type); > > > + if (!attrs[i].attr.name) { > > > + pci_doe_sysfs_feature_remove(pdev, doe_mb); > > > + return; > > > + } > > > + attrs[i].attr.mode = 0444; > > > + attrs[i].show = pci_doe_sysfs_feature_show; > > > + > > > + sysfs_attrs[i] = &attrs[i].attr; > > > + } > > > + > > > + sysfs_attrs[num_features] = NULL; > > > + > > > + pci_doe_sysfs_group.attrs = sysfs_attrs; > > Hmm. Isn't this global? What if you have multiple devices. > > Any input from a PCI maintainer here? > > There are basically two approaches. > > 1. We can have a pci_doe_sysfs_init() function that is called where > we dynamically add the entries, like in v12 > 2. We can go down the dev->groups and device_add() path, like this > patch and discussed at > https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > > For the second we will have to create a global pci_doe_sysfs_group > that contains all possible DOE entries on the system and then have the > show functions determine if they should be displayed for that device. > > Everytime we call pci_doe_init() we can check for any missing entries > in pci_doe_sysfs_group.attrs and then realloc > pci_doe_sysfs_group.attrs to add them. > Untested, but that should work > even for hot-plugged devices. pci_doe_sysfs_group.attrs would just > grow forever though as I don't think we have an easy way to deallocate > anything as we aren't sure if we are the only entry. I think this needs to be per device, not global and you'll have to manually do the group visibility magic rather than using the macros. > > Thoughts? > > Alistair
Hello, > > Any input from a PCI maintainer here? Something that I am curious about: can we make this a single file with a bitmask inside that denotes what DOE features are enabled? Would this be approach be even feasible here? Thoughts? Or is it too late to think about this now? > > There are basically two approaches. > > > > 1. We can have a pci_doe_sysfs_init() function that is called where > > we dynamically add the entries, like in v12 > > 2. We can go down the dev->groups and device_add() path, like this > > patch and discussed at > > https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > > > > For the second we will have to create a global pci_doe_sysfs_group > > that contains all possible DOE entries on the system and then have the > > show functions determine if they should be displayed for that device. > > > > Everytime we call pci_doe_init() we can check for any missing entries > > in pci_doe_sysfs_group.attrs and then realloc > > pci_doe_sysfs_group.attrs to add them. > > Untested, but that should work > > even for hot-plugged devices. pci_doe_sysfs_group.attrs would just > > grow forever though as I don't think we have an easy way to deallocate > > anything as we aren't sure if we are the only entry. > > I think this needs to be per device, not global and you'll have to manually > do the group visibility magic rather than using the macros. Lukas proposes a very interesting feature of kernfs recently per: https://lore.kernel.org/linux-pci/16490618cbde91b5aac04873c39c8fb7666ff686.1719771133.git.lukas@wunner.de Would this help with DOE features? Krzysztof
On Mon, Jul 8, 2024 at 10:55 AM Krzysztof Wilczyński <kw@linux.com> wrote: > > Hello, > > > > Any input from a PCI maintainer here? > > Something that I am curious about: can we make this a single file with a > bitmask inside that denotes what DOE features are enabled? Would this be > approach be even feasible here? In theory there can be any vendor ID (16-bits but not 0xFFFF) and any feature (8-bits). So there is a huge possibility of values here. > > Thoughts? Or is it too late to think about this now? It's just too many possible options to use a bitmask. I guess we could use a feature bit mask per vendor if people feel strongly > > > > There are basically two approaches. > > > > > > 1. We can have a pci_doe_sysfs_init() function that is called where > > > we dynamically add the entries, like in v12 > > > 2. We can go down the dev->groups and device_add() path, like this > > > patch and discussed at > > > https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ > > > > > > For the second we will have to create a global pci_doe_sysfs_group > > > that contains all possible DOE entries on the system and then have the > > > show functions determine if they should be displayed for that device. > > > > > > Everytime we call pci_doe_init() we can check for any missing entries > > > in pci_doe_sysfs_group.attrs and then realloc > > > pci_doe_sysfs_group.attrs to add them. > > > Untested, but that should work > > > even for hot-plugged devices. pci_doe_sysfs_group.attrs would just > > > grow forever though as I don't think we have an easy way to deallocate > > > anything as we aren't sure if we are the only entry. > > > > I think this needs to be per device, not global and you'll have to manually > > do the group visibility magic rather than using the macros. > > Lukas proposes a very interesting feature of kernfs recently per: > > https://lore.kernel.org/linux-pci/16490618cbde91b5aac04873c39c8fb7666ff686.1719771133.git.lukas@wunner.de > > Would this help with DOE features? That was the previous approach used here: https://lore.kernel.org/linux-pci/20240626045926.680380-3-alistair.francis@wdc.com/ Bjorn wanted to try and avoid using a function pci_doe_sysfs_init() [1], which is what I tried here. It sounds like the v12 approach is the way to go then. I'll send a v14 based on v12 with the comments addressed 1: https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ Alistair > > Krzysztof
diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci index ecf47559f495..e1b8f15e4a3a 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -500,3 +500,31 @@ Description: console drivers from the device. Raw users of pci-sysfs resourceN attributes must be terminated prior to resizing. Success of the resizing operation is not guaranteed. + +What: /sys/bus/pci/devices/.../doe_features +Date: July 2024 +Contact: Linux PCI developers <linux-pci@vger.kernel.org> +Description: + This directory contains a list of the supported + Data Object Exchange (DOE) features. The feature values are + the file name. The contents of each file is the raw vendor id and + data object feature values, the same as the name. + + The value comes from the device and specifies the vendor and + data object type supported. The lower (RHS of the colon) is + the data object type in hex. The upper (LHS of the colon) + is the vendor ID. + + As all DOE devices must support the DOE discovery protocol, if + DOE is supported you will at least see this file, with + this contents + + # cat doe_features/0001:00 + 0001:00 + + If the device supports other protocols you will see other files + as well. For example is CMA/SPDM and secure CMA/SPDM are supported + the doe_features directory will look like this + + # ls doe_features + 0001:00 0001:01 0001:02 diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c index defc4be81bd4..e7b702afce88 100644 --- a/drivers/pci/doe.c +++ b/drivers/pci/doe.c @@ -14,6 +14,7 @@ #include <linux/bitfield.h> #include <linux/delay.h> +#include <linux/device.h> #include <linux/jiffies.h> #include <linux/mutex.h> #include <linux/pci.h> @@ -47,6 +48,8 @@ * @wq: Wait queue for work item * @work_queue: Queue of pci_doe_work items * @flags: Bit array of PCI_DOE_FLAG_* flags + * @device_attrs: Array of device attributes, used in cleanup + * @sysfs_attrs: Array of sysfs attributes, used in cleanup */ struct pci_doe_mb { struct pci_dev *pdev; @@ -56,6 +59,11 @@ struct pci_doe_mb { wait_queue_head_t wq; struct workqueue_struct *work_queue; unsigned long flags; + +#ifdef CONFIG_SYSFS + struct device_attribute *device_attrs; + struct attribute **sysfs_attrs; +#endif }; struct pci_doe_feature { @@ -92,6 +100,58 @@ struct pci_doe_task { struct pci_doe_mb *doe_mb; }; +#ifdef CONFIG_SYSFS +static bool pci_doe_features_sysfs_group_visible(struct kobject *kobj) +{ + struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj)); + struct pci_doe_mb *doe_mb; + unsigned long index; + + xa_for_each(&pdev->doe_mbs, index, doe_mb) { + if (!xa_empty(&doe_mb->feats)) + return true; + } + + return false; +} +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_doe_features_sysfs) + +struct attribute_group pci_doe_sysfs_group = { + .name = "doe_features", + .is_visible = SYSFS_GROUP_VISIBLE(pci_doe_features_sysfs), +}; + +static ssize_t pci_doe_sysfs_feature_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + return sysfs_emit(buf, "%s\n", attr->attr.name); +} + +static void pci_doe_sysfs_feature_remove(struct pci_dev *pdev, + struct pci_doe_mb *doe_mb) +{ + struct device_attribute *attrs = doe_mb->device_attrs; + struct attribute **sysfs_attrs = doe_mb->sysfs_attrs; + unsigned long i; + void *entry; + + if (!attrs) + return; + + doe_mb->device_attrs = NULL; + doe_mb->sysfs_attrs = NULL; + + xa_for_each(&doe_mb->feats, i, entry) { + if (attrs[i].attr.name) + kfree(attrs[i].attr.name); + } + + kfree(attrs); + kfree(sysfs_attrs); +} +#endif + static int pci_doe_wait(struct pci_doe_mb *doe_mb, unsigned long timeout) { if (wait_event_timeout(doe_mb->wq, @@ -687,6 +747,12 @@ void pci_doe_init(struct pci_dev *pdev) { struct pci_doe_mb *doe_mb; u16 offset = 0; + struct attribute **sysfs_attrs; + struct device_attribute *attrs; + unsigned long num_features = 0; + unsigned long i; + unsigned long vid, type; + void *entry; int rc; xa_init(&pdev->doe_mbs); @@ -707,6 +773,45 @@ void pci_doe_init(struct pci_dev *pdev) pci_doe_destroy_mb(doe_mb); } } + + if (doe_mb) { + xa_for_each(&doe_mb->feats, i, entry) + num_features++; + + sysfs_attrs = kcalloc(num_features + 1, sizeof(*sysfs_attrs), GFP_KERNEL); + if (!sysfs_attrs) + return; + + attrs = kcalloc(num_features, sizeof(*attrs), GFP_KERNEL); + if (!attrs) { + kfree(sysfs_attrs); + return; + } + + doe_mb->device_attrs = attrs; + doe_mb->sysfs_attrs = sysfs_attrs; + + xa_for_each(&doe_mb->feats, i, entry) { + sysfs_attr_init(&attrs[i].attr); + + vid = xa_to_value(entry) >> 8; + type = xa_to_value(entry) & 0xFF; + + attrs[i].attr.name = kasprintf(GFP_KERNEL, "%04lx:%02lx", vid, type); + if (!attrs[i].attr.name) { + pci_doe_sysfs_feature_remove(pdev, doe_mb); + return; + } + attrs[i].attr.mode = 0444; + attrs[i].show = pci_doe_sysfs_feature_show; + + sysfs_attrs[i] = &attrs[i].attr; + } + + sysfs_attrs[num_features] = NULL; + + pci_doe_sysfs_group.attrs = sysfs_attrs; + } } void pci_doe_destroy(struct pci_dev *pdev) @@ -714,8 +819,10 @@ void pci_doe_destroy(struct pci_dev *pdev) struct pci_doe_mb *doe_mb; unsigned long index; - xa_for_each(&pdev->doe_mbs, index, doe_mb) + xa_for_each(&pdev->doe_mbs, index, doe_mb) { + pci_doe_sysfs_feature_remove(pdev, doe_mb); pci_doe_destroy_mb(doe_mb); + } xa_destroy(&pdev->doe_mbs); } diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index 40cfa716392f..cd838b85d6ab 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -16,6 +16,7 @@ #include <linux/kernel.h> #include <linux/sched.h> #include <linux/pci.h> +#include <linux/pci-doe.h> #include <linux/stat.h> #include <linux/export.h> #include <linux/topology.h> @@ -1661,6 +1662,9 @@ const struct attribute_group *pci_dev_attr_groups[] = { #endif #ifdef CONFIG_PCIEASPM &aspm_ctrl_attr_group, +#endif +#ifdef CONFIG_PCI_DOE + &pci_doe_sysfs_group, #endif NULL, }; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index fd44565c4756..a26c586cdeb4 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -189,6 +189,7 @@ extern const struct attribute_group *pci_dev_groups[]; extern const struct attribute_group *pci_dev_attr_groups[]; extern const struct attribute_group *pcibus_groups[]; extern const struct attribute_group *pci_bus_groups[]; +extern struct attribute_group pci_doe_sysfs_group; #else static inline int pci_create_sysfs_dev_files(struct pci_dev *pdev) { return 0; } static inline void pci_remove_sysfs_dev_files(struct pci_dev *pdev) { } @@ -196,6 +197,7 @@ static inline void pci_remove_sysfs_dev_files(struct pci_dev *pdev) { } #define pci_dev_attr_groups NULL #define pcibus_groups NULL #define pci_bus_groups NULL +#define pci_doe_sysfs_group NULL #endif extern unsigned long pci_hotplug_io_size;
The PCIe 6 specification added support for the Data Object Exchange (DOE). When DOE is supported the DOE Discovery Feature must be implemented per PCIe r6.1 sec 6.30.1.1. The protocol allows a requester to obtain information about the other DOE features supported by the device. The kernel is already querying the DOE features supported and cacheing the values. Expose the values in sysfs to allow user space to determine which DOE features are supported by the PCIe device. By exposing the information to userspace tools like lspci can relay the information to users. By listing all of the supported features we can allow userspace to parse the list, which might include vendor specific features as well as yet to be supported features. After this patch is supported you can see something like this when attaching a DOE device $ ls /sys/devices/pci0000:00/0000:00:02.0//doe* 0001:00 0001:01 0001:02 Signed-off-by: Alistair Francis <alistair.francis@wdc.com> --- v13: - Drop pci_doe_sysfs_init() and use pci_doe_sysfs_group - As discussed in https://lore.kernel.org/all/20231019165829.GA1381099@bhelgaas/ we can just modify pci_doe_sysfs_group at the DOE init and let device_add() handle the sysfs attributes. v12: - Drop pci_doe_features_sysfs_attr_visible() v11: - Gracefully handle multiple entried of same feature - Minor fixes and code cleanups v10: - Rebase to use DEFINE_SYSFS_GROUP_VISIBLE and remove special setup function v9: - Add a teardown function - Rename functions to be clearer - Tidy up the commit message - Remove #ifdef from header v8: - Inlucde an example in the docs - Fixup removing a file that wasn't added - Remove a blank line v7: - Fixup the #ifdefs to keep the test robot happy v6: - Use "feature" instead of protocol - Don't use any devm_* functions - Add two more patches to the series v5: - Return the file name as the file contents - Code cleanups and simplifications v4: - Fixup typos in the documentation - Make it clear that the file names contain the information - Small code cleanups - Remove most #ifdefs - Remove extra NULL assignment v3: - Expose each DOE feature as a separate file v2: - Add documentation - Code cleanups Documentation/ABI/testing/sysfs-bus-pci | 28 ++++++ drivers/pci/doe.c | 109 +++++++++++++++++++++++- drivers/pci/pci-sysfs.c | 4 + drivers/pci/pci.h | 2 + 4 files changed, 142 insertions(+), 1 deletion(-)