| Message ID | 20231009085123.463179-1-yi.l.liu@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add SIOV virtual device support | expand |
On Mon, Oct 09, 2023 at 01:51:16AM -0700, Yi Liu wrote: > Intel SIOV allows creating virtual devices of which the vRID is > represented by a pasid of a physical device. It is called as SIOV > virtual device in this series. Such devices can be bound to an iommufd > as physical device does and then later be attached to an IOAS/hwpt > using that pasid. Such PASIDs are called as default pasid. I would want to see the idxd implementation too.. Jason
> From: Jason Gunthorpe <jgg@nvidia.com> > Sent: Monday, October 9, 2023 9:21 PM > > On Mon, Oct 09, 2023 at 01:51:16AM -0700, Yi Liu wrote: > > Intel SIOV allows creating virtual devices of which the vRID is > > represented by a pasid of a physical device. It is called as SIOV > > virtual device in this series. Such devices can be bound to an iommufd > > as physical device does and then later be attached to an IOAS/hwpt > > using that pasid. Such PASIDs are called as default pasid. > > I would want to see the idxd implementation too.. > It still needs some time (and unfortunately the guy working on idxd is currently on a long vacation). Instead of waiting we want to seek early comments on the iommufd changes given that part is relatively self-contained. Same as what Reinette is doing for IMS. Certainly this is not for merging w/o having a driver user.
On 10/9/2023 9:21 PM, Jason Gunthorpe wrote: > On Mon, Oct 09, 2023 at 01:51:16AM -0700, Yi Liu wrote: >> Intel SIOV allows creating virtual devices of which the vRID is >> represented by a pasid of a physical device. It is called as SIOV >> virtual device in this series. Such devices can be bound to an iommufd >> as physical device does and then later be attached to an IOAS/hwpt >> using that pasid. Such PASIDs are called as default pasid. > > I would want to see the idxd implementation too.. > > Jason Hey Jason, ice(E810 NIC) driver implementation for SIOV virtual device is also working in progress. We are working closely with Kevin and Yi on the patchset. We'll send out the ice patch for SIOV as lan driver user example once it is available. (There is some format issue in the last email, re-send again and sorry for any inconvenience) Thanks. Yahui.
Intel SIOV allows creating virtual devices of which the vRID is represented by a pasid of a physical device. It is called as SIOV virtual device in this series. Such devices can be bound to an iommufd as physical device does and then later be attached to an IOAS/hwpt using that pasid. Such PASIDs are called as default pasid. iommufd has already supported pasid attach[1]. So a simple way to support SIOV virtual device attachment is to let device driver call the iommufd_device_pasid_attach() and pass in the default pasid for the virtual device. This should work for now, but it may have problem if iommufd core wants to differentiate the default pasids with other kind of pasids (e.g. pasid given by userspace). In the later forwarding page request to userspace, the default pasids are not supposed to send to userspace as default pasids are mainly used by the SIOV device driver. With above reason, this series chooses to have a new API to bind the default pasid to iommufd, and extends the iommufd_device_attach() to convert the attachment to be pasid attach with the default pasid. Device drivers (e.g. VFIO) that support SIOV shall call the below APIs to interact with iommufd: - iommufd_device_bind_pasid(): Bind virtual device (a pasid of a device) to iommufd; - iommufd_device_attach(): Attach a SIOV virtual device to IOAS/HWPT; - iommufd_device_replace(): Replace IOAS/HWPT of a SIOV virtual device; - iommufd_device_detach(): Detach IOAS/HWPT of a SIOV virtual device; - iommufd_device_unbind(): Unbind virtual device from iommufd; For vfio devices, the device drivers that support SIOV should: - use below API to register vdev for SIOV virtual device vfio_register_pasid_iommu_dev() - use below API to bind vdev to iommufd in .bind_iommufd() callback iommufd_device_bind_pasid() - allocate pasid by itself before calling iommufd_device_bind_pasid() Complete code can be found at[2] [1] https://lore.kernel.org/linux-iommu/20230926092651.17041-1-yi.l.liu@intel.com/ [2] https://github.com/yiliu1765/iommufd/tree/iommufd_pasid_siov Regards, Yi Liu Kevin Tian (5): iommufd: Handle unsafe interrupts in a separate function iommufd: Introduce iommufd_alloc_device() iommufd: Add iommufd_device_bind_pasid() iommufd: Support attach/replace for SIOV virtual device {dev, pasid} vfio: Add vfio_register_pasid_iommu_dev() Yi Liu (2): iommufd/selftest: Extend IOMMU_TEST_OP_MOCK_DOMAIN to pass in pasid iommufd/selftest: Add test coverage for SIOV virtual device drivers/iommu/iommufd/device.c | 163 ++++++++++++++---- drivers/iommu/iommufd/iommufd_private.h | 7 + drivers/iommu/iommufd/iommufd_test.h | 2 + drivers/iommu/iommufd/selftest.c | 10 +- drivers/vfio/group.c | 18 ++ drivers/vfio/vfio.h | 8 + drivers/vfio/vfio_main.c | 10 ++ include/linux/iommufd.h | 3 + include/linux/vfio.h | 1 + tools/testing/selftests/iommu/iommufd.c | 75 ++++++-- .../selftests/iommu/iommufd_fail_nth.c | 42 ++++- tools/testing/selftests/iommu/iommufd_utils.h | 21 ++- 12 files changed, 296 insertions(+), 64 deletions(-)