| Message ID | 171291193308.3532867.129739584130889725.stgit@dwillia2-xfh.jf.intel.com (mailing list archive) |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | Towards a shared TSM sysfs-ABI for Confidential Computing | expand |
On Fri, Apr 12, 2024 at 01:52:13AM -0700, Dan Williams wrote: > The PCIe 6.1 specification, section 11, introduces the Trusted Execution > Environment (TEE) Device Interface Security Protocol (TDISP). This > interface definition builds upon Component Measurement and > Authentication (CMA), and link Integrity and Data Encryption (IDE). It > adds support for assigning devices (PCI physical or virtual function) to > a confidential VM such that the assigned device is enabled to access > guest private memory protected by technologies like Intel TDX, AMD > SEV-SNP, RISCV COVE, or ARM CCA. > > The "TSM" (TEE Security Manager) is a concept in the TDISP specification > of an agent that mediates between a "DSM" (Device Security Manager) and > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs > to setup link security and assign devices. A confidential VM uses TSM > ABIs to transition an assigned device into the TDISP "RUN" state and > validate its configuration. From a Linux perspective the TSM abstracts > many of the details of TDISP, IDE, and CMA. Some of those details leak > through at times, but for the most part TDISP is an internal > implementation detail of the TSM. > > Similar to the PCI core extensions to support CONFIG_PCI_CMA, > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs > attribute, and add more properties + controls in a tsm/ subdirectory of > the PCI device sysfs interface. Unlike CMA that can depend on a local to > the PCI core implementation, PCI_TSM needs to be prepared for late > loading of the platform TSM driver. Consider that the TSM driver may > itself be a PCI driver. Userspace can depend on the common TSM device > uevent to know when the PCI core has TSM services enabled. The PCI > device tsm/ subdirectory is supplemented by the TSM device pci/ > directory for platform global TSM properties + controls. > > The common verbs that the low-level TSM drivers implement are defined by > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for > establishing a trust relationship between the host and the device, > securing the interconnect (optionally establishing IDE), and tearing > that down. > > The locking allows for multiple devices to be executing commands > simultaneously, one outstanding command per-device and an rwsem flushes > all inflight commands when a TSM low-level driver/device is removed. > > In addition to commands submitted through an 'exec' operation the > low-level TSM driver is notified of device arrival and departure events > via 'add' and 'del' operations. With those it can setup per-device > context, or filter devices that the TSM is not prepared to support. > > Cc: Wu Hao <hao.wu@intel.com> > Cc: Yilun Xu <yilun.xu@intel.com> > Cc: Lukas Wunner <lukas@wunner.de> > Cc: Samuel Ortiz <sameo@rivosinc.com> > Cc: Alexey Kardashevskiy <aik@amd.com> > Cc: Bjorn Helgaas <bhelgaas@google.com> > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com> > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com> > Signed-off-by: Dan Williams <dan.j.williams@intel.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> # PCI parts > +What: /sys/bus/pci/devices/.../tsm/ > +Date: March 2024 > +Contact: linux-coco@lists.linux.dev > +Description: > + This directory only appears if a device supports CMA and IDE, > + and only after a TSM driver has loaded and evaluated this > + PCI device. All present devices shall be dispositioned > + after the 'add' event for /sys/class/tsm/tsm0 triggers. What does "dispositioned" mean? What devices does "all present devices" cover? Is "tsm0" a special global thing? Is there doc for /sys/class/tsm/...? > +++ b/drivers/pci/Makefile > @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB) += vgaarb.o > obj-$(CONFIG_PCI_DOE) += doe.o > obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o > > +obj-$(CONFIG_PCI_TSM) += tsm.o Maybe put it next to CONFIG_PCI_DOE or at least not off in a special separate list? Bjorn
On 12/4/24 18:52, Dan Williams wrote: > The PCIe 6.1 specification, section 11, introduces the Trusted Execution > Environment (TEE) Device Interface Security Protocol (TDISP). This > interface definition builds upon Component Measurement and > Authentication (CMA), and link Integrity and Data Encryption (IDE). It > adds support for assigning devices (PCI physical or virtual function) to > a confidential VM such that the assigned device is enabled to access > guest private memory protected by technologies like Intel TDX, AMD > SEV-SNP, RISCV COVE, or ARM CCA. > > The "TSM" (TEE Security Manager) is a concept in the TDISP specification > of an agent that mediates between a "DSM" (Device Security Manager) and > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs > to setup link security and assign devices. A confidential VM uses TSM > ABIs to transition an assigned device into the TDISP "RUN" state and > validate its configuration. From a Linux perspective the TSM abstracts > many of the details of TDISP, IDE, and CMA. Some of those details leak > through at times, but for the most part TDISP is an internal > implementation detail of the TSM. > > Similar to the PCI core extensions to support CONFIG_PCI_CMA, > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs > attribute, and add more properties + controls in a tsm/ subdirectory of > the PCI device sysfs interface. Unlike CMA that can depend on a local to > the PCI core implementation, PCI_TSM needs to be prepared for late > loading of the platform TSM driver. Consider that the TSM driver may > itself be a PCI driver. Userspace can depend on the common TSM device > uevent to know when the PCI core has TSM services enabled. The PCI > device tsm/ subdirectory is supplemented by the TSM device pci/ > directory for platform global TSM properties + controls. > > The common verbs that the low-level TSM drivers implement are defined by > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for > establishing a trust relationship between the host and the device, > securing the interconnect (optionally establishing IDE), and tearing > that down. > > The locking allows for multiple devices to be executing commands > simultaneously, one outstanding command per-device and an rwsem flushes > all inflight commands when a TSM low-level driver/device is removed. > > In addition to commands submitted through an 'exec' operation the > low-level TSM driver is notified of device arrival and departure events > via 'add' and 'del' operations. With those it can setup per-device > context, or filter devices that the TSM is not prepared to support. > > Cc: Wu Hao <hao.wu@intel.com> > Cc: Yilun Xu <yilun.xu@intel.com> > Cc: Lukas Wunner <lukas@wunner.de> > Cc: Samuel Ortiz <sameo@rivosinc.com> > Cc: Alexey Kardashevskiy <aik@amd.com> > Cc: Bjorn Helgaas <bhelgaas@google.com> > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com> > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com> > Signed-off-by: Dan Williams <dan.j.williams@intel.com> > --- > Documentation/ABI/testing/sysfs-bus-pci | 46 +++++ > MAINTAINERS | 2 > drivers/pci/Kconfig | 13 + > drivers/pci/Makefile | 2 > drivers/pci/pci-sysfs.c | 4 > drivers/pci/pci.h | 10 + > drivers/pci/probe.c | 1 > drivers/pci/remove.c | 1 > drivers/pci/tsm.c | 270 +++++++++++++++++++++++++++++++ > drivers/virt/coco/host/tsm-core.c | 22 ++- > include/linux/pci-tsm.h | 80 +++++++++ > include/linux/pci.h | 11 + > include/linux/tsm.h | 4 > include/uapi/linux/pci_regs.h | 4 > 14 files changed, 466 insertions(+), 4 deletions(-) > create mode 100644 drivers/pci/tsm.c > create mode 100644 include/linux/pci-tsm.h > > diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci > index ecf47559f495..4ae50621e65b 100644 > --- a/Documentation/ABI/testing/sysfs-bus-pci > +++ b/Documentation/ABI/testing/sysfs-bus-pci > @@ -500,3 +500,49 @@ Description: > console drivers from the device. Raw users of pci-sysfs > resourceN attributes must be terminated prior to resizing. > Success of the resizing operation is not guaranteed. > + > +What: /sys/bus/pci/devices/.../authenticated > +Date: March 2024 > +Contact: linux-pci@vger.kernel.org > +Description: > + This file contains 1 if the device authenticated successfully. > + It contains 0 if the device failed authentication (and may thus > + be malicious). There are 2 potential authentication methods: > + native PCI CMA (Component Measurement and Authentication) and > + PCI TSM (TEE Security Manager). In the PCI TSM case the device's > + PCI CMA interface is subsumed by the TSM driver. A TSM > + implementation uses its own private certificate store + keys to > + authenticate the device. Without a TSM the kernel can > + authenticate using its own certificate chain. > + > + In the TSM case, "authenticated" is read-only (0444) and the > + "tsm/connect" attribute reflects whether the device is TSM > + "connected" which includes not only CMA authentication, but > + optionally IDE (link Integrity and Data encryption) if the TSM > + deems that is necessary. When the device is disconnected from > + the TSM the kernel may attempt authentication with its own > + certificate chain. See > + Documentation/ABI/testing/sysfs-devices-spdm. > + > + The file is not visible if authentication is unsupported by the > + device, or if PCI CMA support is disabled and the TSM > + driver has no available authentication methods for the device. > + > +What: /sys/bus/pci/devices/.../tsm/ > +Date: March 2024 > +Contact: linux-coco@lists.linux.dev > +Description: > + This directory only appears if a device supports CMA and IDE, > + and only after a TSM driver has loaded and evaluated this > + PCI device. All present devices shall be dispositioned > + after the 'add' event for /sys/class/tsm/tsm0 triggers. > + > +What: /sys/bus/pci/devices/.../tsm/connect > +Date: March 2024 > +Contact: linux-coco@lists.linux.dev > +Description: > + (RW) Writing "1" to this file triggers the TSM to establish a > + connection with the device. This typically includes an SPDM > + (DMTF Security Protocols and Data Models) session over PCIe DOE > + (Data Object Exchange) and may also include PCIe IDE (Integrity > + and Data Encryption) establishment. > diff --git a/MAINTAINERS b/MAINTAINERS > index 8d5bcd9d43ac..0e1d995e7a16 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -22466,8 +22466,10 @@ M: Dan Williams <dan.j.williams@intel.com> > L: linux-coco@lists.linux.dev > S: Maintained > F: Documentation/ABI/testing/configfs-tsm > +F: drivers/pci/tsm.c > F: drivers/virt/coco/guest/tsm_report.c > F: drivers/virt/coco/host/ > +F: include/linux/pci-tsm.h > F: include/linux/tsm.h > > TTY LAYER AND SERIAL DRIVERS > diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig > index d35001589d88..cd863c5e49ca 100644 > --- a/drivers/pci/Kconfig > +++ b/drivers/pci/Kconfig > @@ -121,6 +121,19 @@ config XEN_PCIDEV_FRONTEND > config PCI_ATS > bool > > +config PCI_TSM > + bool "TEE Security Manager for PCI Device Security" > + help > + The TEE (Trusted Execution Environment) Device Interface > + Security Protocol (TDISP) defines a "TSM" as a platform agent > + that manages device authentication, link encryption, link > + integrity protection, and assignment of PCI device functions > + (virtual or physical) to confidential computing VMs that can > + access (DMA) guest private memory. > + > + Enable a platform TSM driver to use this capability. > + > + > config PCI_DOE > bool > > diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile > index 175302036890..b9884a669c5f 100644 > --- a/drivers/pci/Makefile > +++ b/drivers/pci/Makefile > @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB) += vgaarb.o > obj-$(CONFIG_PCI_DOE) += doe.o > obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o > > +obj-$(CONFIG_PCI_TSM) += tsm.o > + > # Endpoint library must be initialized before its users > obj-$(CONFIG_PCI_ENDPOINT) += endpoint/ > > diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c > index 40cfa716392f..c6ea624dd76c 100644 > --- a/drivers/pci/pci-sysfs.c > +++ b/drivers/pci/pci-sysfs.c > @@ -1661,6 +1661,10 @@ const struct attribute_group *pci_dev_attr_groups[] = { > #endif > #ifdef CONFIG_PCIEASPM > &aspm_ctrl_attr_group, > +#endif > +#ifdef CONFIG_PCI_TSM > + &pci_tsm_auth_attr_group, > + &pci_tsm_attr_group, > #endif > NULL, > }; > diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h > index 17fed1846847..9b864cbf8682 100644 > --- a/drivers/pci/pci.h > +++ b/drivers/pci/pci.h > @@ -335,6 +335,16 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { } > static inline void pci_doe_disconnected(struct pci_dev *pdev) { } > #endif > > +#ifdef CONFIG_PCI_TSM > +void pci_tsm_init(struct pci_dev *pdev); > +void pci_tsm_destroy(struct pci_dev *pdev); > +extern const struct attribute_group pci_tsm_attr_group; > +extern const struct attribute_group pci_tsm_auth_attr_group; > +#else > +static inline void pci_tsm_init(struct pci_dev *pdev) { } > +static inline void pci_tsm_destroy(struct pci_dev *pdev) { } > +#endif > + > /** > * pci_dev_set_io_state - Set the new error state if possible. > * > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > index 1325fbae2f28..d89b67541d02 100644 > --- a/drivers/pci/probe.c > +++ b/drivers/pci/probe.c > @@ -2481,6 +2481,7 @@ static void pci_init_capabilities(struct pci_dev *dev) > pci_dpc_init(dev); /* Downstream Port Containment */ > pci_rcec_init(dev); /* Root Complex Event Collector */ > pci_doe_init(dev); /* Data Object Exchange */ > + pci_tsm_init(dev); /* TEE Security Manager connection */ > > pcie_report_downtraining(dev); > pci_init_reset_methods(dev); > diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c > index d749ea8250d6..d94b2458934a 100644 > --- a/drivers/pci/remove.c > +++ b/drivers/pci/remove.c > @@ -39,6 +39,7 @@ static void pci_destroy_dev(struct pci_dev *dev) > list_del(&dev->bus_list); > up_write(&pci_bus_sem); > > + pci_tsm_destroy(dev); > pci_doe_destroy(dev); > pcie_aspm_exit_link_state(dev); > pci_bridge_d3_update(dev); > diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c > new file mode 100644 > index 000000000000..9c5fb2c46662 > --- /dev/null > +++ b/drivers/pci/tsm.c > @@ -0,0 +1,270 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * TEE Security Manager for the TEE Device Interface Security Protocol > + * (TDISP, PCIe r6.1 sec 11) > + * > + * Copyright(c) 2024 Intel Corporation. All rights reserved. > + */ > + > +#define dev_fmt(fmt) "TSM: " fmt > + > +#include <linux/pci.h> > +#include <linux/pci-doe.h> > +#include <linux/sysfs.h> > +#include <linux/xarray.h> > +#include <linux/pci-tsm.h> > +#include <linux/bitfield.h> > +#include "pci.h" > + > +/* collect TSM capable devices to rendezvous with the tsm driver */ > +static DEFINE_XARRAY(pci_tsm_devs); imho either this or pci_dev::tsm is enough but not necessarily both. > + > +/* > + * Provide a read/write lock against the init / exit of pdev tsm > + * capabilities and arrival/departure of a tsm instance > + */ > +static DECLARE_RWSEM(pci_tsm_rwsem); > +static const struct pci_tsm_ops *tsm_ops; > + > +static int pci_tsm_disconnect(struct pci_dev *pdev) > +{ > + struct pci_tsm *pci_tsm = pdev->tsm; > + > + lockdep_assert_held_read(&pci_tsm_rwsem); > + scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) { > + int rc; > + > + if (pci_tsm->state < PCI_TSM_CONNECT) > + return 0; > + if (pci_tsm->state < PCI_TSM_INIT) > + return -ENXIO; > + > + rc = tsm_ops->exec(pdev, TSM_EXEC_DISCONNECT); > + if (rc) > + return rc; > + pci_tsm->state = PCI_TSM_INIT; > + } > + return 0; > +} > + > +static int pci_tsm_connect(struct pci_dev *pdev) > +{ > + struct pci_tsm *pci_tsm = pdev->tsm; > + > + lockdep_assert_held_read(&pci_tsm_rwsem); > + scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) { > + int rc; > + > + if (pci_tsm->state >= PCI_TSM_CONNECT) > + return 0; > + if (pci_tsm->state < PCI_TSM_INIT) > + return -ENXIO; > + > + rc = tsm_ops->exec(pdev, TSM_EXEC_CONNECT); > + if (rc) > + return rc; > + pci_tsm->state = PCI_TSM_CONNECT; > + } > + return 0; > +} > + > +static ssize_t connect_store(struct device *dev, struct device_attribute *attr, > + const char *buf, size_t len) > +{ > + int rc; > + bool connect; > + struct pci_dev *pdev = to_pci_dev(dev); > + > + rc = kstrtobool(buf, &connect); > + if (rc) > + return rc; > + > + if (connect) > + rc = pci_tsm_connect(pdev); > + else > + rc = pci_tsm_disconnect(pdev); > + if (rc) > + return rc; > + return len; > +} > + > +static ssize_t connect_show(struct device *dev, struct device_attribute *attr, > + char *buf) > +{ > + struct pci_dev *pdev = to_pci_dev(dev); > + > + return sysfs_emit(buf, "%d\n", pdev->tsm->state >= PCI_TSM_CONNECT); > +} > +static DEVICE_ATTR_RW(connect); > + > +static bool pci_tsm_group_visible(struct kobject *kobj) > +{ > + struct device *dev = kobj_to_dev(kobj); > + struct pci_dev *pdev = to_pci_dev(dev); > + > + if (pdev->tsm && pdev->tsm->state > PCI_TSM_IDLE) > + return true; > + return false; > +} > +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_tsm); > + > +static struct attribute *pci_tsm_attrs[] = { > + &dev_attr_connect.attr, > + NULL, > +}; > + > +const struct attribute_group pci_tsm_attr_group = { > + .name = "tsm", > + .attrs = pci_tsm_attrs, > + .is_visible = SYSFS_GROUP_VISIBLE(pci_tsm), > +}; > + > +static ssize_t authenticated_show(struct device *dev, > + struct device_attribute *attr, char *buf) > +{ > + /* > + * When device authentication is TSM owned, 'authenticated' is > + * identical to the connect state. > + */ > + return connect_show(dev, attr, buf); > +} > +static DEVICE_ATTR_RO(authenticated); > + > +static struct attribute *pci_tsm_auth_attrs[] = { > + &dev_attr_authenticated.attr, > + NULL, > +}; > + > +const struct attribute_group pci_tsm_auth_attr_group = { > + .attrs = pci_tsm_auth_attrs, > + .is_visible = SYSFS_GROUP_VISIBLE(pci_tsm), > +}; > + > +static int pci_tsm_add(struct pci_dev *pdev) > +{ > + struct pci_tsm *pci_tsm = pdev->tsm; > + > + lockdep_assert_held(&pci_tsm_rwsem); > + if (!tsm_ops) > + return 0; > + if (pci_tsm->state < PCI_TSM_INIT) { > + int rc = tsm_ops->add(pdev); > + > + if (rc) > + return rc; > + } > + pci_tsm->state = PCI_TSM_INIT; > + return sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group); > +} > + > +static void pci_tsm_del(struct pci_dev *pdev) > +{ > + struct pci_tsm *pci_tsm = pdev->tsm; > + > + lockdep_assert_held(&pci_tsm_rwsem); > + /* shutdown sysfs operations before tsm delete */ > + scoped_guard(mutex, &pdev->tsm->exec_lock) > + pci_tsm->state = PCI_TSM_IDLE; > + sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group); > + tsm_ops->del(pdev); > +} > + > +int pci_tsm_register(const struct pci_tsm_ops *ops) > +{ > + struct pci_dev *pdev; > + unsigned long index; > + > + if (!ops) > + return 0; > + guard(rwsem_write)(&pci_tsm_rwsem); > + if (tsm_ops) > + return -EBUSY; > + tsm_ops = ops; > + xa_for_each(&pci_tsm_devs, index, pdev) > + pci_tsm_add(pdev); > + return 0; > +} > +EXPORT_SYMBOL_GPL(pci_tsm_register); > + > +void pci_tsm_unregister(const struct pci_tsm_ops *ops) > +{ > + struct pci_dev *pdev; > + unsigned long index; > + > + if (!ops) > + return; > + guard(rwsem_write)(&pci_tsm_rwsem); > + if (ops != tsm_ops) > + return; > + xa_for_each(&pci_tsm_devs, index, pdev) > + pci_tsm_del(pdev); > + tsm_ops = NULL; > +} > +EXPORT_SYMBOL_GPL(pci_tsm_unregister); > + > +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type, > + const void *req, size_t req_sz, void *resp, > + size_t resp_sz) > +{ > + if (!pdev->tsm || !pdev->tsm->doe_mb) > + return -ENXIO; > + > + return pci_doe(pdev->tsm->doe_mb, PCI_VENDOR_ID_PCI_SIG, type, req, > + req_sz, resp, resp_sz); > +} > +EXPORT_SYMBOL_GPL(pci_tsm_doe_transfer); > + > +static unsigned long pci_tsm_devid(struct pci_dev *pdev) > +{ > + return FIELD_PREP(GENMASK(15, 0), > + PCI_DEVID(pdev->bus->number, pdev->devfn)) | > + FIELD_PREP(GENMASK(31, 16), pci_domain_nr(pdev->bus)); > +} > + > +void pci_tsm_init(struct pci_dev *pdev) > +{ > + bool tee_cap; > + u16 ide_cap; > + int rc; > + > + ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE); > + tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE; > + if (ide_cap || tee_cap) I'd swap if with else. > + pci_dbg(pdev, > + "Device security capabailities detected (%s%s ), init TSM\n", capabailities > + ide_cap ? " ide" : "", tee_cap ? " tee" : ""); > + else > + return; If (!ide_cap && tee_cap), we get here but doing the below does not make sense for TEE (which are likely to be VFs). > + struct pci_tsm *pci_tsm __free(kfree) = kzalloc(sizeof(*pci_tsm), GFP_KERNEL); > + if (!pci_tsm) > + return; > + > + pci_tsm->ide_cap = ide_cap; > + mutex_init(&pci_tsm->exec_lock); > + > + pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, > + PCI_DOE_PROTO_CMA); > + if (!pci_tsm->doe_mb) > + return; > + > + rc = xa_insert(&pci_tsm_devs, pci_tsm_devid(pdev), pdev, GFP_KERNEL); > + if (rc) { > + pci_dbg(pdev, "failed to register TSM capable device\n"); > + return; > + } > + > + guard(rwsem_write)(&pci_tsm_rwsem); > + pdev->tsm = no_free_ptr(pci_tsm); > + pci_tsm_add(pdev); > +} > + > +void pci_tsm_destroy(struct pci_dev *pdev) > +{ > + guard(rwsem_write)(&pci_tsm_rwsem); > + pci_tsm_del(pdev); > + xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev)); > + kfree(pdev->tsm); > + pdev->tsm = NULL; > +} > diff --git a/drivers/virt/coco/host/tsm-core.c b/drivers/virt/coco/host/tsm-core.c > index 0ee738fc40ed..994a7f77c5c9 100644 > --- a/drivers/virt/coco/host/tsm-core.c > +++ b/drivers/virt/coco/host/tsm-core.c > @@ -8,11 +8,13 @@ > #include <linux/device.h> > #include <linux/module.h> > #include <linux/cleanup.h> > +#include <linux/pci-tsm.h> > > static DECLARE_RWSEM(tsm_core_rwsem); > static struct class *tsm_class; > static struct tsm_subsys { > struct device dev; > + const struct pci_tsm_ops *pci_ops; > } *tsm_subsys; > > static struct tsm_subsys * > @@ -40,7 +42,8 @@ static void put_tsm_subsys(struct tsm_subsys *subsys) > DEFINE_FREE(put_tsm_subsys, struct tsm_subsys *, > if (!IS_ERR_OR_NULL(_T)) put_tsm_subsys(_T)) > struct tsm_subsys *tsm_register(struct device *parent, > - const struct attribute_group **groups) > + const struct attribute_group **groups, > + const struct pci_tsm_ops *pci_ops) > { > struct device *dev; > int rc; > @@ -62,10 +65,20 @@ struct tsm_subsys *tsm_register(struct device *parent, > if (rc) > return ERR_PTR(rc); > > + rc = pci_tsm_register(pci_ops); > + if (rc) { > + dev_err(parent, "PCI initialization failure: %pe\n", > + ERR_PTR(rc)); > + return ERR_PTR(rc); > + } > + > rc = device_add(dev); > - if (rc) > + if (rc) { > + pci_tsm_unregister(pci_ops); > return ERR_PTR(rc); > + } > > + subsys->pci_ops = pci_ops; > tsm_subsys = no_free_ptr(subsys); > > return tsm_subsys; > @@ -74,13 +87,18 @@ EXPORT_SYMBOL_GPL(tsm_register); > > void tsm_unregister(struct tsm_subsys *subsys) > { > + const struct pci_tsm_ops *pci_ops; > + > guard(rwsem_write)(&tsm_core_rwsem); > if (!tsm_subsys || subsys != tsm_subsys) { > pr_warn("failed to unregister, not currently registered\n"); > return; > } > > + pci_ops = subsys->pci_ops; > device_unregister(&subsys->dev); > + > + pci_tsm_unregister(pci_ops); > tsm_subsys = NULL; > } > EXPORT_SYMBOL_GPL(tsm_unregister); > diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h > new file mode 100644 > index 000000000000..d17f5e0574c4 > --- /dev/null > +++ b/include/linux/pci-tsm.h > @@ -0,0 +1,80 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef __PCI_TSM_H > +#define __PCI_TSM_H > +#include <linux/mutex.h> > + > +enum pci_tsm_cmd { > + TSM_EXEC_CONNECT, > + TSM_EXEC_DISCONNECT, > +}; > + > +struct pci_dev; > +/** > + * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core > + * @add: accept device for tsm operation What does "accept" means here? "Accept" sounds like some action needed from something but this is what exec() for. So far I have not noticed that allocating platform-specific structures prior calling a verb is any useful, i.e. having a separate state - PCI_TSM_INIT - is just an extra noise in the "painfully simple first TSM implementation". > + * @del: teardown tsm context for @pdev > + * @exec: synchronously execute @cmd > + * > + * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to > + * synchronize with TSM driver bind/unbind events and > + * pci_device_add()/pci_destroy_dev(). @exec runs in > + * @pdev->tsm->exec_lock context to synchronize @exec results with > + * @pdev->tsm->state > + */ > +struct pci_tsm_ops { > + int (*add)(struct pci_dev *pdev); > + void (*del)(struct pci_dev *pdev); > + int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd); A nit: the verbs seem working (especially reducing the amount of cut-n-paste of all this spdm forwarding) until I get to things like "get_status" which returns a structure to dump in the sysfs. Doing it like this means adding a structure in pci_tsm and manage its state (valid/partial/empty/...). Or we might want to generalize some input parameters for the verbs, adding u64 params is meh. > +}; > + > +enum pci_tsm_state { > + PCI_TSM_IDLE, > + PCI_TSM_INIT, > + PCI_TSM_CONNECT, > +}; > + > +/** > + * struct pci_tsm - per device TSM context > + * @state: reflect device initialized, connected, or bound > + * @ide_cap: PCIe IDE Extended Capability offset > + * @exec_lock: protect @state vs pci_tsm_ops.exec() results > + * @doe_mb: PCIe Data Object Exchange mailbox > + * @tsm_data: TSM driver private context > + */ > +struct pci_tsm { > + enum pci_tsm_state state; > + u16 ide_cap; > + struct mutex exec_lock; > + struct pci_doe_mb *doe_mb; > + void *tsm_data; > +}; > + > +enum pci_doe_proto { > + PCI_DOE_PROTO_CMA = 1, > + PCI_DOE_PROTO_SSESSION = 2, > +}; > + > +#ifdef CONFIG_PCI_TSM > +int pci_tsm_register(const struct pci_tsm_ops *ops); > +void pci_tsm_unregister(const struct pci_tsm_ops *ops); > +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type, > + const void *req, size_t req_sz, void *resp, > + size_t resp_sz); > +#else > +static inline int pci_tsm_register(const struct pci_tsm_ops *ops) > +{ > + return 0; > +} > +static inline void pci_tsm_unregister(const struct pci_tsm_ops *ops) > +{ > +} > +static inline int pci_tsm_doe_transfer(struct pci_dev *pdev, > + enum pci_doe_proto type, const void *req, > + size_t req_sz, void *resp, > + size_t resp_sz) > +{ > + return -ENOENT; > +} > + > +#endif > +#endif /*__PCI_TSM_H */ > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 16493426a04f..dd4dc8719c5c 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -515,6 +515,9 @@ struct pci_dev { > #endif > #ifdef CONFIG_PCI_DOE > struct xarray doe_mbs; /* Data Object Exchange mailboxes */ > +#endif > +#ifdef CONFIG_PCI_TSM > + struct pci_tsm *tsm; /* TSM operation state */ I am wondering if pdev->dev.platform_data can be used for this. > #endif > u16 acs_cap; /* ACS Capability offset */ > phys_addr_t rom; /* Physical address if not from BAR */ > @@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev) > return (pdev->error_state != pci_channel_io_normal); > } > > +/* id resources that may be shared across host-bridges */ > +struct pci_hb_id_pool { > + int nr_stream_ids; > + int nr_cxl_cache_ids; > +}; > + > /* > * Currently in ACPI spec, for each PCI host bridge, PCI Segment > * Group number is limited to a 16-bit value, therefore (int)-1 is > @@ -568,6 +577,8 @@ struct pci_host_bridge { > void *sysdata; > int busnr; > int domain_nr; > + struct pci_hb_id_pool __pool; > + struct pci_hb_id_pool *pool; /* &self->__pool, unless shared */ What are these for? Something for IDE (which I also have in the works, very basic set of wrappers)? Thanks, > struct list_head windows; /* resource_entry */ > struct list_head dma_ranges; /* dma ranges resource list */ > u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */ > diff --git a/include/linux/tsm.h b/include/linux/tsm.h > index 2867c2ecbd11..6481cc99ea6d 100644 > --- a/include/linux/tsm.h > +++ b/include/linux/tsm.h > @@ -68,7 +68,9 @@ int tsm_report_register(const struct tsm_report_ops *ops, void *priv, > const struct config_item_type *type); > int tsm_report_unregister(const struct tsm_report_ops *ops); > struct tsm_subsys; > +struct pci_tsm_ops; > struct tsm_subsys *tsm_register(struct device *parent, > - const struct attribute_group **groups); > + const struct attribute_group **groups, > + const struct pci_tsm_ops *ops); > void tsm_unregister(struct tsm_subsys *subsys); > #endif /* __TSM_H */ > diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h > index a39193213ff2..9aaffa379cae 100644 > --- a/include/uapi/linux/pci_regs.h > +++ b/include/uapi/linux/pci_regs.h > @@ -498,6 +498,7 @@ > #define PCI_EXP_DEVCAP_PWR_VAL 0x03fc0000 /* Slot Power Limit Value */ > #define PCI_EXP_DEVCAP_PWR_SCL 0x0c000000 /* Slot Power Limit Scale */ > #define PCI_EXP_DEVCAP_FLR 0x10000000 /* Function Level Reset */ > +#define PCI_EXP_DEVCAP_TEE 0x40000000 /* TEE I/O (TDISP) Support */ > #define PCI_EXP_DEVCTL 0x08 /* Device Control */ > #define PCI_EXP_DEVCTL_CERE 0x0001 /* Correctable Error Reporting En. */ > #define PCI_EXP_DEVCTL_NFERE 0x0002 /* Non-Fatal Error Reporting Enable */ > @@ -742,7 +743,8 @@ > #define PCI_EXT_CAP_ID_PL_16GT 0x26 /* Physical Layer 16.0 GT/s */ > #define PCI_EXT_CAP_ID_PL_32GT 0x2A /* Physical Layer 32.0 GT/s */ > #define PCI_EXT_CAP_ID_DOE 0x2E /* Data Object Exchange */ > -#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_DOE > +#define PCI_EXT_CAP_ID_IDE 0x30 /* Integrity and Data Encryption */ > +#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_IDE > > #define PCI_EXT_CAP_DSN_SIZEOF 12 > #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40 >
Bjorn Helgaas wrote: > On Fri, Apr 12, 2024 at 01:52:13AM -0700, Dan Williams wrote: > > The PCIe 6.1 specification, section 11, introduces the Trusted Execution > > Environment (TEE) Device Interface Security Protocol (TDISP). This > > interface definition builds upon Component Measurement and > > Authentication (CMA), and link Integrity and Data Encryption (IDE). It > > adds support for assigning devices (PCI physical or virtual function) to > > a confidential VM such that the assigned device is enabled to access > > guest private memory protected by technologies like Intel TDX, AMD > > SEV-SNP, RISCV COVE, or ARM CCA. > > > > The "TSM" (TEE Security Manager) is a concept in the TDISP specification > > of an agent that mediates between a "DSM" (Device Security Manager) and > > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs > > to setup link security and assign devices. A confidential VM uses TSM > > ABIs to transition an assigned device into the TDISP "RUN" state and > > validate its configuration. From a Linux perspective the TSM abstracts > > many of the details of TDISP, IDE, and CMA. Some of those details leak > > through at times, but for the most part TDISP is an internal > > implementation detail of the TSM. > > > > Similar to the PCI core extensions to support CONFIG_PCI_CMA, > > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs > > attribute, and add more properties + controls in a tsm/ subdirectory of > > the PCI device sysfs interface. Unlike CMA that can depend on a local to > > the PCI core implementation, PCI_TSM needs to be prepared for late > > loading of the platform TSM driver. Consider that the TSM driver may > > itself be a PCI driver. Userspace can depend on the common TSM device > > uevent to know when the PCI core has TSM services enabled. The PCI > > device tsm/ subdirectory is supplemented by the TSM device pci/ > > directory for platform global TSM properties + controls. > > > > The common verbs that the low-level TSM drivers implement are defined by > > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for > > establishing a trust relationship between the host and the device, > > securing the interconnect (optionally establishing IDE), and tearing > > that down. > > > > The locking allows for multiple devices to be executing commands > > simultaneously, one outstanding command per-device and an rwsem flushes > > all inflight commands when a TSM low-level driver/device is removed. > > > > In addition to commands submitted through an 'exec' operation the > > low-level TSM driver is notified of device arrival and departure events > > via 'add' and 'del' operations. With those it can setup per-device > > context, or filter devices that the TSM is not prepared to support. > > > > Cc: Wu Hao <hao.wu@intel.com> > > Cc: Yilun Xu <yilun.xu@intel.com> > > Cc: Lukas Wunner <lukas@wunner.de> > > Cc: Samuel Ortiz <sameo@rivosinc.com> > > Cc: Alexey Kardashevskiy <aik@amd.com> > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com> > > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com> > > Acked-by: Bjorn Helgaas <bhelgaas@google.com> # PCI parts Great, thanks Bjorn! This lets us move forward on the TSM details knowing that it looks like this meets your expectations on the PCI integration aspects. Will continue to include you on revisions as this evolves, but this is good news for the cross vendor collaboration effort in process here. > > +What: /sys/bus/pci/devices/.../tsm/ > > +Date: March 2024 > > +Contact: linux-coco@lists.linux.dev > > +Description: > > + This directory only appears if a device supports CMA and IDE, > > + and only after a TSM driver has loaded and evaluated this > > + PCI device. All present devices shall be dispositioned > > + after the 'add' event for /sys/class/tsm/tsm0 triggers. > > What does "dispositioned" mean? When /sys/class/tsm/tsm0/uevent signals KOBJ_ADD, arrival of the TSM device, a udev script can assume that the kernel has walked all PCI devices and toggled the visibility of the all /sys/bus/pci/devices/$pdev/tsm/ attribute directories. > What devices does "all present devices" cover? A snapshot of the state of /sys/bus/pci/devices at TSM arrival. So a udev script that, for example, wants to perform the TSM "connect" operation as soon as possible should watch PCI KOBJ_ADD uevents and try to connect if the TSM is present at that time, but if not, try again after TSM KOBJ_ADD. Similar for the PCI-hot-add case. TSM may arrive first, so that policy script should recheck when PCI KOBJ_ADD, for the device it wants to connect, fires. > > Is "tsm0" a special global thing? Is there doc for > /sys/class/tsm/...? Yes, it is a special common object that any platform with a TSM will export. I had yet to create a document since there are no common attributes defined in this version of the patch set, but I can go ahead and create it and mention what userspace can expect about the state of /sys/bus/pci/devices when the device shows up. > > > +++ b/drivers/pci/Makefile > > @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB) += vgaarb.o > > obj-$(CONFIG_PCI_DOE) += doe.o > > obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o > > > > +obj-$(CONFIG_PCI_TSM) += tsm.o > > Maybe put it next to CONFIG_PCI_DOE or at least not off in a special > separate list? Sure, will do.
Alexey Kardashevskiy wrote: > On 12/4/24 18:52, Dan Williams wrote: > > The PCIe 6.1 specification, section 11, introduces the Trusted Execution > > Environment (TEE) Device Interface Security Protocol (TDISP). This > > interface definition builds upon Component Measurement and > > Authentication (CMA), and link Integrity and Data Encryption (IDE). It > > adds support for assigning devices (PCI physical or virtual function) to > > a confidential VM such that the assigned device is enabled to access > > guest private memory protected by technologies like Intel TDX, AMD > > SEV-SNP, RISCV COVE, or ARM CCA. > > > > The "TSM" (TEE Security Manager) is a concept in the TDISP specification > > of an agent that mediates between a "DSM" (Device Security Manager) and > > system software in both a VMM and a confidential VM. A VMM uses TSM ABIs > > to setup link security and assign devices. A confidential VM uses TSM > > ABIs to transition an assigned device into the TDISP "RUN" state and > > validate its configuration. From a Linux perspective the TSM abstracts > > many of the details of TDISP, IDE, and CMA. Some of those details leak > > through at times, but for the most part TDISP is an internal > > implementation detail of the TSM. > > > > Similar to the PCI core extensions to support CONFIG_PCI_CMA, > > CONFIG_PCI_TSM builds upon that to reuse the "authenticated" sysfs > > attribute, and add more properties + controls in a tsm/ subdirectory of > > the PCI device sysfs interface. Unlike CMA that can depend on a local to > > the PCI core implementation, PCI_TSM needs to be prepared for late > > loading of the platform TSM driver. Consider that the TSM driver may > > itself be a PCI driver. Userspace can depend on the common TSM device > > uevent to know when the PCI core has TSM services enabled. The PCI > > device tsm/ subdirectory is supplemented by the TSM device pci/ > > directory for platform global TSM properties + controls. > > > > The common verbs that the low-level TSM drivers implement are defined by > > 'enum pci_tsm_cmd'. For now only connect and disconnect are defined for > > establishing a trust relationship between the host and the device, > > securing the interconnect (optionally establishing IDE), and tearing > > that down. > > > > The locking allows for multiple devices to be executing commands > > simultaneously, one outstanding command per-device and an rwsem flushes > > all inflight commands when a TSM low-level driver/device is removed. > > > > In addition to commands submitted through an 'exec' operation the > > low-level TSM driver is notified of device arrival and departure events > > via 'add' and 'del' operations. With those it can setup per-device > > context, or filter devices that the TSM is not prepared to support. > > > > Cc: Wu Hao <hao.wu@intel.com> > > Cc: Yilun Xu <yilun.xu@intel.com> > > Cc: Lukas Wunner <lukas@wunner.de> > > Cc: Samuel Ortiz <sameo@rivosinc.com> > > Cc: Alexey Kardashevskiy <aik@amd.com> > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com> > > Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com> > > Signed-off-by: Dan Williams <dan.j.williams@intel.com> [..] > > diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c > > new file mode 100644 > > index 000000000000..9c5fb2c46662 > > --- /dev/null > > +++ b/drivers/pci/tsm.c > > @@ -0,0 +1,270 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > +/* > > + * TEE Security Manager for the TEE Device Interface Security Protocol > > + * (TDISP, PCIe r6.1 sec 11) > > + * > > + * Copyright(c) 2024 Intel Corporation. All rights reserved. > > + */ > > + > > +#define dev_fmt(fmt) "TSM: " fmt > > + > > +#include <linux/pci.h> > > +#include <linux/pci-doe.h> > > +#include <linux/sysfs.h> > > +#include <linux/xarray.h> > > +#include <linux/pci-tsm.h> > > +#include <linux/bitfield.h> > > +#include "pci.h" > > + > > +/* collect TSM capable devices to rendezvous with the tsm driver */ > > +static DEFINE_XARRAY(pci_tsm_devs); > > imho either this or pci_dev::tsm is enough but not necessarily both. You mean: s/pci_tsm_devs/tsm_devs/ ? [..] > > +void pci_tsm_init(struct pci_dev *pdev) > > +{ > > + bool tee_cap; > > + u16 ide_cap; > > + int rc; > > + > > + ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE); > > + tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE; > > + if (ide_cap || tee_cap) > > I'd swap if with else. Oh, you mean: if (!(ide_cap || tee_cap)) return; ? > > > + pci_dbg(pdev, > > + "Device security capabailities detected (%s%s ), init TSM\n", > > capabailities Incapabail of spelling correctly apparently. checkpatch spell check let me down. Fixed. > > > + ide_cap ? " ide" : "", tee_cap ? " tee" : ""); > > + else > > + return; > > > If (!ide_cap && tee_cap), we get here but doing the below does not make > sense for TEE (which are likely to be VFs). The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE without TDISP flow". [..] > > +struct pci_dev; > > +/** > > + * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core > > + * @add: accept device for tsm operation > > > What does "accept" means here? "Accept" sounds like some action needed > from something but this is what exec() for. At the lowest level an "accepted" device, one that returns successfully from "->add()" and has its tsm/ attribute group in sysfs enabled. > So far I have not noticed that allocating platform-specific structures > prior calling a verb is any useful, i.e. having a separate state - > PCI_TSM_INIT - is just an extra noise in the "painfully simple first > TSM implementation". As far as I know, not all TSM implementations care about the "!ide_cap && tee_cap" case. That said, regarding painfully simple, if the only difference is some TSMs support IDE without TDISP and some do not, that standalone-IDE support can be added later. > > + * @del: teardown tsm context for @pdev > > + * @exec: synchronously execute @cmd > > + * > > + * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to > > + * synchronize with TSM driver bind/unbind events and > > + * pci_device_add()/pci_destroy_dev(). @exec runs in > > + * @pdev->tsm->exec_lock context to synchronize @exec results with > > + * @pdev->tsm->state > > + */ > > +struct pci_tsm_ops { > > + int (*add)(struct pci_dev *pdev); > > + void (*del)(struct pci_dev *pdev); > > + int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd); > > > A nit: the verbs seem working (especially reducing the amount of > cut-n-paste of all this spdm forwarding) until I get to things like > "get_status" which returns a structure to dump in the sysfs. Doing it > like this means adding a structure in pci_tsm and manage its state > (valid/partial/empty/...). Or we might want to generalize some input > parameters for the verbs, adding u64 params is meh. If it is just for sysfs cases then I would just have a facility for low level TSM drivers to publish some attributes directly at pci_tsm_register time. Something like the following, and just require low level TSM implementations to agree on filenames and formats, but otherwise avoid complicating ->exec(). diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c index 44b998a1c824..d34c3477ddc3 100644 --- a/drivers/pci/tsm.c +++ b/drivers/pci/tsm.c @@ -113,9 +113,9 @@ static struct attribute *pci_tsm_attrs[] = { NULL, }; -const struct attribute_group pci_tsm_attr_group = { +struct attribute_group pci_tsm_attr_group = { .name = "tsm", - .attrs = pci_tsm_attrs, + .attrs = pci_tsm_default_attrs, .is_visible = SYSFS_GROUP_VISIBLE(pci_tsm), }; @@ -169,7 +169,7 @@ static void pci_tsm_del(struct pci_dev *pdev) tsm_ops->del(pdev); } -int pci_tsm_register(const struct pci_tsm_ops *ops) +int pci_tsm_register(const struct pci_tsm_ops *ops, struct attribute **attrs) { struct pci_dev *pdev; unsigned long index; @@ -180,6 +180,7 @@ int pci_tsm_register(const struct pci_tsm_ops *ops) if (tsm_ops) return -EBUSY; tsm_ops = ops; + pci_tsm_attr_group.attrs = attrs; xa_for_each(&pci_tsm_devs, index, pdev) pci_tsm_add(pdev); return 0; @@ -198,6 +199,7 @@ void pci_tsm_unregister(const struct pci_tsm_ops *ops) return; xa_for_each(&pci_tsm_devs, index, pdev) pci_tsm_del(pdev); + pci_tsm_attr_group.attrs = pci_tsm_default_attrs; tsm_ops = NULL; } EXPORT_SYMBOL_GPL(pci_tsm_unregister); > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > index 16493426a04f..dd4dc8719c5c 100644 > > --- a/include/linux/pci.h > > +++ b/include/linux/pci.h > > @@ -515,6 +515,9 @@ struct pci_dev { > > #endif > > #ifdef CONFIG_PCI_DOE > > struct xarray doe_mbs; /* Data Object Exchange mailboxes */ > > +#endif > > +#ifdef CONFIG_PCI_TSM > > + struct pci_tsm *tsm; /* TSM operation state */ > > > I am wondering if pdev->dev.platform_data can be used for this. No, platform_data is for ACPI or OF to populate. For example, see sst_acpi_probe(). > > > > #endif > > u16 acs_cap; /* ACS Capability offset */ > > phys_addr_t rom; /* Physical address if not from BAR */ > > @@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev) > > return (pdev->error_state != pci_channel_io_normal); > > } > > > > +/* id resources that may be shared across host-bridges */ > > +struct pci_hb_id_pool { > > + int nr_stream_ids; > > + int nr_cxl_cache_ids; > > +}; > > + > > /* > > * Currently in ACPI spec, for each PCI host bridge, PCI Segment > > * Group number is limited to a 16-bit value, therefore (int)-1 is > > @@ -568,6 +577,8 @@ struct pci_host_bridge { > > void *sysdata; > > int busnr; > > int domain_nr; > > + struct pci_hb_id_pool __pool; > > + struct pci_hb_id_pool *pool; /* &self->__pool, unless shared */ > > > What are these for? Something for IDE (which I also have in the works, > very basic set of wrappers)? Thanks, Yes, this is something I had sitting in my tree as a rough draft, but did not intend to send out, but I guess a useful accident. Hao Wu has taken this concept further. @Hao, lets post what we are thinking here. The concept is that stream-ids are a limited resource. Host bridges, at least Intel ones, might share their stream-id pool with another host-bridge. Do AMD platforms have similar constraints? The end goal is to be able to convey to a system owner which devices are consuming which stream-ids and which host-bridges have available stream-ids to allocate.
> > If (!ide_cap && tee_cap), we get here but doing the below does not make > > sense for TEE (which are likely to be VFs). > > The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE > without TDISP flow". IIUC, should be "TSM wants to setup TDISP without IDE flow"? But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"), VFs should not be rejected by the following: pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, PCI_DOE_PROTO_CMA); if (!pci_tsm->doe_mb) return; VF should check its PF's doe/ide/tee cap and then be added to pci_tsm_devs, is it? Thanks, Yilun
> > > + > > > +/* collect TSM capable devices to rendezvous with the tsm driver */ > > > +static DEFINE_XARRAY(pci_tsm_devs); > > > > imho either this or pci_dev::tsm is enough but not necessarily both. > > You mean: > > s/pci_tsm_devs/tsm_devs/ > > ? I don't think the concern is just a renaming. My understanding is, we already have a struct pci_tsm embedded in struct pci_dev, we could loop and find all TSM capable devices by: for_each_pci_dev(pdev) { if (pdev->tsm) pci_tsm_add/del(pdev); } A dedicated list for TSM capable devices seems not necessary. But my concern is about VFs. VFs are as well TSM capable but not applicable for tsm_ops->exec(TSM_EXEC_CONNECT), maybe not applicable for tsm_ops->add() either. One way to distinguish PF/VFs is we only collect PFs in pci_tsm_devs, but all TSM capable devices have valid pci_dev::tsm pointer. TSM capable devices in Guest should not been collected in pci_tsm_devs either. Thanks, Yilun
Xu Yilun wrote: > > > If (!ide_cap && tee_cap), we get here but doing the below does not make > > > sense for TEE (which are likely to be VFs). > > > > The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE > > without TDISP flow". > > IIUC, should be "TSM wants to setup TDISP without IDE flow"? Both are possible. The TSM may need to be involved in IDE key establishment even if the PF or its VFs are ever assigned as TDIs. Also, as you say, it is possible for a TSM to trust that a device does not need IDE established because it is has knowledge that the device is integrated into the SOC without physical exposure of its links. > But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"), > VFs should not be rejected by the following: > > pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, > PCI_DOE_PROTO_CMA); > if (!pci_tsm->doe_mb) > return; > > VF should check its PF's doe/ide/tee cap and then be added to > pci_tsm_devs, is it? This path should probably skip VFs because the 'connect' operation is a PF-only semantic. I will fix that up. I still think the PF needs to go through an ->add() callback because I do not think we have a cross-vendor unified concept of when IDE without TDISP, or TDISP without IDE is supported.
Xu Yilun wrote: > > > > + > > > > +/* collect TSM capable devices to rendezvous with the tsm driver */ > > > > +static DEFINE_XARRAY(pci_tsm_devs); > > > > > > imho either this or pci_dev::tsm is enough but not necessarily both. > > > > You mean: > > > > s/pci_tsm_devs/tsm_devs/ > > > > ? > > I don't think the concern is just a renaming. My understanding is, we > already have a struct pci_tsm embedded in struct pci_dev, we could loop > and find all TSM capable devices by: > > for_each_pci_dev(pdev) { > if (pdev->tsm) > pci_tsm_add/del(pdev); > } > A dedicated list for TSM capable devices seems not necessary. Not the first time this criticism has been raised against pci_tsm_devs. I think for_each_pci_dev() is potentially wasteful, but it is trivial to add back if for_each_pci_dev() scanning becomes too expensive. > But my concern is about VFs. VFs are as well TSM capable but not > applicable for tsm_ops->exec(TSM_EXEC_CONNECT), maybe not applicable > for tsm_ops->add() either. One way to distinguish PF/VFs is we only > collect PFs in pci_tsm_devs, but all TSM capable devices have > valid pci_dev::tsm pointer. > > TSM capable devices in Guest should not been collected in pci_tsm_devs > either. Yes, for this initial phase of the enabling only PF operations are relevant.
On Tue, May 07, 2024 at 11:21:37AM -0700, Dan Williams wrote: > Xu Yilun wrote: > > > > If (!ide_cap && tee_cap), we get here but doing the below does not make > > > > sense for TEE (which are likely to be VFs). > > > > > > The "!ide_cap && tee_cap" case may also be the "TSM wants to setup IDE > > > without TDISP flow". > > > > IIUC, should be "TSM wants to setup TDISP without IDE flow"? > > Both are possible. The TSM may need to be involved in IDE key > establishment even if the PF or its VFs are ever assigned as TDIs. Also, > as you say, it is possible for a TSM to trust that a device does not > need IDE established because it is has knowledge that the device is > integrated into the SOC without physical exposure of its links. > > > But I think aik is talking about VFs (which fit "!ide_cap && tee_cap"), > > VFs should not be rejected by the following: > > > > pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, > > PCI_DOE_PROTO_CMA); > > if (!pci_tsm->doe_mb) > > return; > > > > VF should check its PF's doe/ide/tee cap and then be added to > > pci_tsm_devs, is it? > > This path should probably skip VFs because the 'connect' operation is a > PF-only semantic. I will fix that up. Agreed. I drafted some simple changes for the idea, that we keep pci_dev::tsm for every TEE capable device (PF & VF) to execute tsm_ops, but only adds PFs to pci_tsm_devs for "connect". diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c index 9c5fb2c46662..31707f0351c6 100644 --- a/drivers/pci/tsm.c +++ b/drivers/pci/tsm.c @@ -241,9 +241,14 @@ void pci_tsm_init(struct pci_dev *pdev) if (!pci_tsm) return; - pci_tsm->ide_cap = ide_cap; mutex_init(&pci_tsm->exec_lock); + if (pdev->is_virtfn) { + pdev->tsm = no_free_ptr(pci_tsm); + return; + } + + pci_tsm->ide_cap = ide_cap; pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, PCI_DOE_PROTO_CMA); if (!pci_tsm->doe_mb) @@ -262,9 +267,14 @@ void pci_tsm_init(struct pci_dev *pdev) void pci_tsm_destroy(struct pci_dev *pdev) { + if (!pdev->tsm) + return; + guard(rwsem_write)(&pci_tsm_rwsem); - pci_tsm_del(pdev); - xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev)); + if (!pdev->is_virtfn) { + pci_tsm_del(pdev); + xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev)); + } kfree(pdev->tsm); pdev->tsm = NULL; } Thanks, Yilun > > I still think the PF needs to go through an ->add() callback because I > do not think we have a cross-vendor unified concept of when IDE without > TDISP, or TDISP without IDE is supported.
On Tue, 7 May 2024 16:46:29 +0800 Xu Yilun <yilun.xu@linux.intel.com> wrote: > > > > + > > > > +/* collect TSM capable devices to rendezvous with the tsm > > > > driver */ +static DEFINE_XARRAY(pci_tsm_devs); > > > > > > imho either this or pci_dev::tsm is enough but not necessarily > > > both. > > > > You mean: > > > > s/pci_tsm_devs/tsm_devs/ > > > > ? > > I don't think the concern is just a renaming. My understanding is, we > already have a struct pci_tsm embedded in struct pci_dev, we could > loop and find all TSM capable devices by: > > for_each_pci_dev(pdev) { > if (pdev->tsm) > pci_tsm_add/del(pdev); > } > > A dedicated list for TSM capable devices seems not necessary. > > But my concern is about VFs. VFs are as well TSM capable but not > applicable for tsm_ops->exec(TSM_EXEC_CONNECT), maybe not applicable > for tsm_ops->add() either. One way to distinguish PF/VFs is we only > collect PFs in pci_tsm_devs, but all TSM capable devices have > valid pci_dev::tsm pointer. > > TSM capable devices in Guest should not been collected in pci_tsm_devs > either. What is the plan for the TSM capable devices in the guest? My current understanding is there would be host TSM driver and guest TSM driver, or a vendor TSM driver will have a host mode and a guest mode due to its nature to understand if it is running in host or a guest. They will be plugged into the same framework here. If that is the case, the TSM driver should step in and decide if a PF/VF can be managed(added) according to its mode. Maybe TSM driver should also indicate what tdi_verbs it supports. E.g. in the guest mode, it tells CONNECT is not available but the device can be managed by the TSM driver. Thanks, Zhi. > > Thanks, > Yilun >
diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci index ecf47559f495..4ae50621e65b 100644 --- a/Documentation/ABI/testing/sysfs-bus-pci +++ b/Documentation/ABI/testing/sysfs-bus-pci @@ -500,3 +500,49 @@ Description: console drivers from the device. Raw users of pci-sysfs resourceN attributes must be terminated prior to resizing. Success of the resizing operation is not guaranteed. + +What: /sys/bus/pci/devices/.../authenticated +Date: March 2024 +Contact: linux-pci@vger.kernel.org +Description: + This file contains 1 if the device authenticated successfully. + It contains 0 if the device failed authentication (and may thus + be malicious). There are 2 potential authentication methods: + native PCI CMA (Component Measurement and Authentication) and + PCI TSM (TEE Security Manager). In the PCI TSM case the device's + PCI CMA interface is subsumed by the TSM driver. A TSM + implementation uses its own private certificate store + keys to + authenticate the device. Without a TSM the kernel can + authenticate using its own certificate chain. + + In the TSM case, "authenticated" is read-only (0444) and the + "tsm/connect" attribute reflects whether the device is TSM + "connected" which includes not only CMA authentication, but + optionally IDE (link Integrity and Data encryption) if the TSM + deems that is necessary. When the device is disconnected from + the TSM the kernel may attempt authentication with its own + certificate chain. See + Documentation/ABI/testing/sysfs-devices-spdm. + + The file is not visible if authentication is unsupported by the + device, or if PCI CMA support is disabled and the TSM + driver has no available authentication methods for the device. + +What: /sys/bus/pci/devices/.../tsm/ +Date: March 2024 +Contact: linux-coco@lists.linux.dev +Description: + This directory only appears if a device supports CMA and IDE, + and only after a TSM driver has loaded and evaluated this + PCI device. All present devices shall be dispositioned + after the 'add' event for /sys/class/tsm/tsm0 triggers. + +What: /sys/bus/pci/devices/.../tsm/connect +Date: March 2024 +Contact: linux-coco@lists.linux.dev +Description: + (RW) Writing "1" to this file triggers the TSM to establish a + connection with the device. This typically includes an SPDM + (DMTF Security Protocols and Data Models) session over PCIe DOE + (Data Object Exchange) and may also include PCIe IDE (Integrity + and Data Encryption) establishment. diff --git a/MAINTAINERS b/MAINTAINERS index 8d5bcd9d43ac..0e1d995e7a16 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -22466,8 +22466,10 @@ M: Dan Williams <dan.j.williams@intel.com> L: linux-coco@lists.linux.dev S: Maintained F: Documentation/ABI/testing/configfs-tsm +F: drivers/pci/tsm.c F: drivers/virt/coco/guest/tsm_report.c F: drivers/virt/coco/host/ +F: include/linux/pci-tsm.h F: include/linux/tsm.h TTY LAYER AND SERIAL DRIVERS diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index d35001589d88..cd863c5e49ca 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -121,6 +121,19 @@ config XEN_PCIDEV_FRONTEND config PCI_ATS bool +config PCI_TSM + bool "TEE Security Manager for PCI Device Security" + help + The TEE (Trusted Execution Environment) Device Interface + Security Protocol (TDISP) defines a "TSM" as a platform agent + that manages device authentication, link encryption, link + integrity protection, and assignment of PCI device functions + (virtual or physical) to confidential computing VMs that can + access (DMA) guest private memory. + + Enable a platform TSM driver to use this capability. + + config PCI_DOE bool diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 175302036890..b9884a669c5f 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -35,6 +35,8 @@ obj-$(CONFIG_VGA_ARB) += vgaarb.o obj-$(CONFIG_PCI_DOE) += doe.o obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o +obj-$(CONFIG_PCI_TSM) += tsm.o + # Endpoint library must be initialized before its users obj-$(CONFIG_PCI_ENDPOINT) += endpoint/ diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index 40cfa716392f..c6ea624dd76c 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -1661,6 +1661,10 @@ const struct attribute_group *pci_dev_attr_groups[] = { #endif #ifdef CONFIG_PCIEASPM &aspm_ctrl_attr_group, +#endif +#ifdef CONFIG_PCI_TSM + &pci_tsm_auth_attr_group, + &pci_tsm_attr_group, #endif NULL, }; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 17fed1846847..9b864cbf8682 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -335,6 +335,16 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { } static inline void pci_doe_disconnected(struct pci_dev *pdev) { } #endif +#ifdef CONFIG_PCI_TSM +void pci_tsm_init(struct pci_dev *pdev); +void pci_tsm_destroy(struct pci_dev *pdev); +extern const struct attribute_group pci_tsm_attr_group; +extern const struct attribute_group pci_tsm_auth_attr_group; +#else +static inline void pci_tsm_init(struct pci_dev *pdev) { } +static inline void pci_tsm_destroy(struct pci_dev *pdev) { } +#endif + /** * pci_dev_set_io_state - Set the new error state if possible. * diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 1325fbae2f28..d89b67541d02 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2481,6 +2481,7 @@ static void pci_init_capabilities(struct pci_dev *dev) pci_dpc_init(dev); /* Downstream Port Containment */ pci_rcec_init(dev); /* Root Complex Event Collector */ pci_doe_init(dev); /* Data Object Exchange */ + pci_tsm_init(dev); /* TEE Security Manager connection */ pcie_report_downtraining(dev); pci_init_reset_methods(dev); diff --git a/drivers/pci/remove.c b/drivers/pci/remove.c index d749ea8250d6..d94b2458934a 100644 --- a/drivers/pci/remove.c +++ b/drivers/pci/remove.c @@ -39,6 +39,7 @@ static void pci_destroy_dev(struct pci_dev *dev) list_del(&dev->bus_list); up_write(&pci_bus_sem); + pci_tsm_destroy(dev); pci_doe_destroy(dev); pcie_aspm_exit_link_state(dev); pci_bridge_d3_update(dev); diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c new file mode 100644 index 000000000000..9c5fb2c46662 --- /dev/null +++ b/drivers/pci/tsm.c @@ -0,0 +1,270 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * TEE Security Manager for the TEE Device Interface Security Protocol + * (TDISP, PCIe r6.1 sec 11) + * + * Copyright(c) 2024 Intel Corporation. All rights reserved. + */ + +#define dev_fmt(fmt) "TSM: " fmt + +#include <linux/pci.h> +#include <linux/pci-doe.h> +#include <linux/sysfs.h> +#include <linux/xarray.h> +#include <linux/pci-tsm.h> +#include <linux/bitfield.h> +#include "pci.h" + +/* collect TSM capable devices to rendezvous with the tsm driver */ +static DEFINE_XARRAY(pci_tsm_devs); + +/* + * Provide a read/write lock against the init / exit of pdev tsm + * capabilities and arrival/departure of a tsm instance + */ +static DECLARE_RWSEM(pci_tsm_rwsem); +static const struct pci_tsm_ops *tsm_ops; + +static int pci_tsm_disconnect(struct pci_dev *pdev) +{ + struct pci_tsm *pci_tsm = pdev->tsm; + + lockdep_assert_held_read(&pci_tsm_rwsem); + scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) { + int rc; + + if (pci_tsm->state < PCI_TSM_CONNECT) + return 0; + if (pci_tsm->state < PCI_TSM_INIT) + return -ENXIO; + + rc = tsm_ops->exec(pdev, TSM_EXEC_DISCONNECT); + if (rc) + return rc; + pci_tsm->state = PCI_TSM_INIT; + } + return 0; +} + +static int pci_tsm_connect(struct pci_dev *pdev) +{ + struct pci_tsm *pci_tsm = pdev->tsm; + + lockdep_assert_held_read(&pci_tsm_rwsem); + scoped_cond_guard(mutex_intr, return -EINTR, &pci_tsm->exec_lock) { + int rc; + + if (pci_tsm->state >= PCI_TSM_CONNECT) + return 0; + if (pci_tsm->state < PCI_TSM_INIT) + return -ENXIO; + + rc = tsm_ops->exec(pdev, TSM_EXEC_CONNECT); + if (rc) + return rc; + pci_tsm->state = PCI_TSM_CONNECT; + } + return 0; +} + +static ssize_t connect_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t len) +{ + int rc; + bool connect; + struct pci_dev *pdev = to_pci_dev(dev); + + rc = kstrtobool(buf, &connect); + if (rc) + return rc; + + if (connect) + rc = pci_tsm_connect(pdev); + else + rc = pci_tsm_disconnect(pdev); + if (rc) + return rc; + return len; +} + +static ssize_t connect_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + + return sysfs_emit(buf, "%d\n", pdev->tsm->state >= PCI_TSM_CONNECT); +} +static DEVICE_ATTR_RW(connect); + +static bool pci_tsm_group_visible(struct kobject *kobj) +{ + struct device *dev = kobj_to_dev(kobj); + struct pci_dev *pdev = to_pci_dev(dev); + + if (pdev->tsm && pdev->tsm->state > PCI_TSM_IDLE) + return true; + return false; +} +DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_tsm); + +static struct attribute *pci_tsm_attrs[] = { + &dev_attr_connect.attr, + NULL, +}; + +const struct attribute_group pci_tsm_attr_group = { + .name = "tsm", + .attrs = pci_tsm_attrs, + .is_visible = SYSFS_GROUP_VISIBLE(pci_tsm), +}; + +static ssize_t authenticated_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + /* + * When device authentication is TSM owned, 'authenticated' is + * identical to the connect state. + */ + return connect_show(dev, attr, buf); +} +static DEVICE_ATTR_RO(authenticated); + +static struct attribute *pci_tsm_auth_attrs[] = { + &dev_attr_authenticated.attr, + NULL, +}; + +const struct attribute_group pci_tsm_auth_attr_group = { + .attrs = pci_tsm_auth_attrs, + .is_visible = SYSFS_GROUP_VISIBLE(pci_tsm), +}; + +static int pci_tsm_add(struct pci_dev *pdev) +{ + struct pci_tsm *pci_tsm = pdev->tsm; + + lockdep_assert_held(&pci_tsm_rwsem); + if (!tsm_ops) + return 0; + if (pci_tsm->state < PCI_TSM_INIT) { + int rc = tsm_ops->add(pdev); + + if (rc) + return rc; + } + pci_tsm->state = PCI_TSM_INIT; + return sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group); +} + +static void pci_tsm_del(struct pci_dev *pdev) +{ + struct pci_tsm *pci_tsm = pdev->tsm; + + lockdep_assert_held(&pci_tsm_rwsem); + /* shutdown sysfs operations before tsm delete */ + scoped_guard(mutex, &pdev->tsm->exec_lock) + pci_tsm->state = PCI_TSM_IDLE; + sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group); + tsm_ops->del(pdev); +} + +int pci_tsm_register(const struct pci_tsm_ops *ops) +{ + struct pci_dev *pdev; + unsigned long index; + + if (!ops) + return 0; + guard(rwsem_write)(&pci_tsm_rwsem); + if (tsm_ops) + return -EBUSY; + tsm_ops = ops; + xa_for_each(&pci_tsm_devs, index, pdev) + pci_tsm_add(pdev); + return 0; +} +EXPORT_SYMBOL_GPL(pci_tsm_register); + +void pci_tsm_unregister(const struct pci_tsm_ops *ops) +{ + struct pci_dev *pdev; + unsigned long index; + + if (!ops) + return; + guard(rwsem_write)(&pci_tsm_rwsem); + if (ops != tsm_ops) + return; + xa_for_each(&pci_tsm_devs, index, pdev) + pci_tsm_del(pdev); + tsm_ops = NULL; +} +EXPORT_SYMBOL_GPL(pci_tsm_unregister); + +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type, + const void *req, size_t req_sz, void *resp, + size_t resp_sz) +{ + if (!pdev->tsm || !pdev->tsm->doe_mb) + return -ENXIO; + + return pci_doe(pdev->tsm->doe_mb, PCI_VENDOR_ID_PCI_SIG, type, req, + req_sz, resp, resp_sz); +} +EXPORT_SYMBOL_GPL(pci_tsm_doe_transfer); + +static unsigned long pci_tsm_devid(struct pci_dev *pdev) +{ + return FIELD_PREP(GENMASK(15, 0), + PCI_DEVID(pdev->bus->number, pdev->devfn)) | + FIELD_PREP(GENMASK(31, 16), pci_domain_nr(pdev->bus)); +} + +void pci_tsm_init(struct pci_dev *pdev) +{ + bool tee_cap; + u16 ide_cap; + int rc; + + ide_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_IDE); + tee_cap = pdev->devcap & PCI_EXP_DEVCAP_TEE; + + if (ide_cap || tee_cap) + pci_dbg(pdev, + "Device security capabailities detected (%s%s ), init TSM\n", + ide_cap ? " ide" : "", tee_cap ? " tee" : ""); + else + return; + + struct pci_tsm *pci_tsm __free(kfree) = kzalloc(sizeof(*pci_tsm), GFP_KERNEL); + if (!pci_tsm) + return; + + pci_tsm->ide_cap = ide_cap; + mutex_init(&pci_tsm->exec_lock); + + pci_tsm->doe_mb = pci_find_doe_mailbox(pdev, PCI_VENDOR_ID_PCI_SIG, + PCI_DOE_PROTO_CMA); + if (!pci_tsm->doe_mb) + return; + + rc = xa_insert(&pci_tsm_devs, pci_tsm_devid(pdev), pdev, GFP_KERNEL); + if (rc) { + pci_dbg(pdev, "failed to register TSM capable device\n"); + return; + } + + guard(rwsem_write)(&pci_tsm_rwsem); + pdev->tsm = no_free_ptr(pci_tsm); + pci_tsm_add(pdev); +} + +void pci_tsm_destroy(struct pci_dev *pdev) +{ + guard(rwsem_write)(&pci_tsm_rwsem); + pci_tsm_del(pdev); + xa_erase(&pci_tsm_devs, pci_tsm_devid(pdev)); + kfree(pdev->tsm); + pdev->tsm = NULL; +} diff --git a/drivers/virt/coco/host/tsm-core.c b/drivers/virt/coco/host/tsm-core.c index 0ee738fc40ed..994a7f77c5c9 100644 --- a/drivers/virt/coco/host/tsm-core.c +++ b/drivers/virt/coco/host/tsm-core.c @@ -8,11 +8,13 @@ #include <linux/device.h> #include <linux/module.h> #include <linux/cleanup.h> +#include <linux/pci-tsm.h> static DECLARE_RWSEM(tsm_core_rwsem); static struct class *tsm_class; static struct tsm_subsys { struct device dev; + const struct pci_tsm_ops *pci_ops; } *tsm_subsys; static struct tsm_subsys * @@ -40,7 +42,8 @@ static void put_tsm_subsys(struct tsm_subsys *subsys) DEFINE_FREE(put_tsm_subsys, struct tsm_subsys *, if (!IS_ERR_OR_NULL(_T)) put_tsm_subsys(_T)) struct tsm_subsys *tsm_register(struct device *parent, - const struct attribute_group **groups) + const struct attribute_group **groups, + const struct pci_tsm_ops *pci_ops) { struct device *dev; int rc; @@ -62,10 +65,20 @@ struct tsm_subsys *tsm_register(struct device *parent, if (rc) return ERR_PTR(rc); + rc = pci_tsm_register(pci_ops); + if (rc) { + dev_err(parent, "PCI initialization failure: %pe\n", + ERR_PTR(rc)); + return ERR_PTR(rc); + } + rc = device_add(dev); - if (rc) + if (rc) { + pci_tsm_unregister(pci_ops); return ERR_PTR(rc); + } + subsys->pci_ops = pci_ops; tsm_subsys = no_free_ptr(subsys); return tsm_subsys; @@ -74,13 +87,18 @@ EXPORT_SYMBOL_GPL(tsm_register); void tsm_unregister(struct tsm_subsys *subsys) { + const struct pci_tsm_ops *pci_ops; + guard(rwsem_write)(&tsm_core_rwsem); if (!tsm_subsys || subsys != tsm_subsys) { pr_warn("failed to unregister, not currently registered\n"); return; } + pci_ops = subsys->pci_ops; device_unregister(&subsys->dev); + + pci_tsm_unregister(pci_ops); tsm_subsys = NULL; } EXPORT_SYMBOL_GPL(tsm_unregister); diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h new file mode 100644 index 000000000000..d17f5e0574c4 --- /dev/null +++ b/include/linux/pci-tsm.h @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __PCI_TSM_H +#define __PCI_TSM_H +#include <linux/mutex.h> + +enum pci_tsm_cmd { + TSM_EXEC_CONNECT, + TSM_EXEC_DISCONNECT, +}; + +struct pci_dev; +/** + * struct pci_tsm_ops - Low-level TSM-exported interface to the PCI core + * @add: accept device for tsm operation + * @del: teardown tsm context for @pdev + * @exec: synchronously execute @cmd + * + * Note that @add, and @del run in down_write(&pci_tsm_rswem) context to + * synchronize with TSM driver bind/unbind events and + * pci_device_add()/pci_destroy_dev(). @exec runs in + * @pdev->tsm->exec_lock context to synchronize @exec results with + * @pdev->tsm->state + */ +struct pci_tsm_ops { + int (*add)(struct pci_dev *pdev); + void (*del)(struct pci_dev *pdev); + int (*exec)(struct pci_dev *pdev, enum pci_tsm_cmd cmd); +}; + +enum pci_tsm_state { + PCI_TSM_IDLE, + PCI_TSM_INIT, + PCI_TSM_CONNECT, +}; + +/** + * struct pci_tsm - per device TSM context + * @state: reflect device initialized, connected, or bound + * @ide_cap: PCIe IDE Extended Capability offset + * @exec_lock: protect @state vs pci_tsm_ops.exec() results + * @doe_mb: PCIe Data Object Exchange mailbox + * @tsm_data: TSM driver private context + */ +struct pci_tsm { + enum pci_tsm_state state; + u16 ide_cap; + struct mutex exec_lock; + struct pci_doe_mb *doe_mb; + void *tsm_data; +}; + +enum pci_doe_proto { + PCI_DOE_PROTO_CMA = 1, + PCI_DOE_PROTO_SSESSION = 2, +}; + +#ifdef CONFIG_PCI_TSM +int pci_tsm_register(const struct pci_tsm_ops *ops); +void pci_tsm_unregister(const struct pci_tsm_ops *ops); +int pci_tsm_doe_transfer(struct pci_dev *pdev, enum pci_doe_proto type, + const void *req, size_t req_sz, void *resp, + size_t resp_sz); +#else +static inline int pci_tsm_register(const struct pci_tsm_ops *ops) +{ + return 0; +} +static inline void pci_tsm_unregister(const struct pci_tsm_ops *ops) +{ +} +static inline int pci_tsm_doe_transfer(struct pci_dev *pdev, + enum pci_doe_proto type, const void *req, + size_t req_sz, void *resp, + size_t resp_sz) +{ + return -ENOENT; +} + +#endif +#endif /*__PCI_TSM_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index 16493426a04f..dd4dc8719c5c 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -515,6 +515,9 @@ struct pci_dev { #endif #ifdef CONFIG_PCI_DOE struct xarray doe_mbs; /* Data Object Exchange mailboxes */ +#endif +#ifdef CONFIG_PCI_TSM + struct pci_tsm *tsm; /* TSM operation state */ #endif u16 acs_cap; /* ACS Capability offset */ phys_addr_t rom; /* Physical address if not from BAR */ @@ -550,6 +553,12 @@ static inline int pci_channel_offline(struct pci_dev *pdev) return (pdev->error_state != pci_channel_io_normal); } +/* id resources that may be shared across host-bridges */ +struct pci_hb_id_pool { + int nr_stream_ids; + int nr_cxl_cache_ids; +}; + /* * Currently in ACPI spec, for each PCI host bridge, PCI Segment * Group number is limited to a 16-bit value, therefore (int)-1 is @@ -568,6 +577,8 @@ struct pci_host_bridge { void *sysdata; int busnr; int domain_nr; + struct pci_hb_id_pool __pool; + struct pci_hb_id_pool *pool; /* &self->__pool, unless shared */ struct list_head windows; /* resource_entry */ struct list_head dma_ranges; /* dma ranges resource list */ u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */ diff --git a/include/linux/tsm.h b/include/linux/tsm.h index 2867c2ecbd11..6481cc99ea6d 100644 --- a/include/linux/tsm.h +++ b/include/linux/tsm.h @@ -68,7 +68,9 @@ int tsm_report_register(const struct tsm_report_ops *ops, void *priv, const struct config_item_type *type); int tsm_report_unregister(const struct tsm_report_ops *ops); struct tsm_subsys; +struct pci_tsm_ops; struct tsm_subsys *tsm_register(struct device *parent, - const struct attribute_group **groups); + const struct attribute_group **groups, + const struct pci_tsm_ops *ops); void tsm_unregister(struct tsm_subsys *subsys); #endif /* __TSM_H */ diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index a39193213ff2..9aaffa379cae 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -498,6 +498,7 @@ #define PCI_EXP_DEVCAP_PWR_VAL 0x03fc0000 /* Slot Power Limit Value */ #define PCI_EXP_DEVCAP_PWR_SCL 0x0c000000 /* Slot Power Limit Scale */ #define PCI_EXP_DEVCAP_FLR 0x10000000 /* Function Level Reset */ +#define PCI_EXP_DEVCAP_TEE 0x40000000 /* TEE I/O (TDISP) Support */ #define PCI_EXP_DEVCTL 0x08 /* Device Control */ #define PCI_EXP_DEVCTL_CERE 0x0001 /* Correctable Error Reporting En. */ #define PCI_EXP_DEVCTL_NFERE 0x0002 /* Non-Fatal Error Reporting Enable */ @@ -742,7 +743,8 @@ #define PCI_EXT_CAP_ID_PL_16GT 0x26 /* Physical Layer 16.0 GT/s */ #define PCI_EXT_CAP_ID_PL_32GT 0x2A /* Physical Layer 32.0 GT/s */ #define PCI_EXT_CAP_ID_DOE 0x2E /* Data Object Exchange */ -#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_DOE +#define PCI_EXT_CAP_ID_IDE 0x30 /* Integrity and Data Encryption */ +#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_IDE #define PCI_EXT_CAP_DSN_SIZEOF 12 #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40