Message ID | 20240813-sriov-v7-0-8515e3774df7@daynix.com (mailing list archive) |
---|---|
Headers | show |
Series | virtio-net: add support for SR-IOV emulation | expand |
On 2024/08/13 15:36, Akihiko Odaki wrote: > Based-on: <20240802-reuse-v11-0-fb83bb8c19fb@daynix.com> > ("[PATCH for-9.2 v11 00/11] hw/pci: SR-IOV related fixes and improvements") I couldn't apply this patch series after applying "[PATCH for-9.2 v11 00/11] hw/pci: SR-IOV related fixes and improvements". Regards, Yui Washizu > > Introduction > ------------ > > This series is based on the RFC series submitted by Yui Washizu[1]. > See also [2] for the context. > > This series enables SR-IOV emulation for virtio-net. It is useful > to test SR-IOV support on the guest, or to expose several vDPA devices > in a VM. vDPA devices can also provide L2 switching feature for > offloading though it is out of scope to allow the guest to configure > such a feature. > > The PF side code resides in virtio-pci. The VF side code resides in > the PCI common infrastructure, but it is restricted to work only for > virtio-net-pci because of lack of validation. > > User Interface > -------------- > > A user can configure a SR-IOV capable virtio-net device by adding > virtio-net-pci functions to a bus. Below is a command line example: > -netdev user,id=n -netdev user,id=o > -netdev user,id=p -netdev user,id=q > -device pcie-root-port,id=b > -device virtio-net-pci,bus=b,addr=0x0.0x3,netdev=q,sriov-pf=f > -device virtio-net-pci,bus=b,addr=0x0.0x2,netdev=p,sriov-pf=f > -device virtio-net-pci,bus=b,addr=0x0.0x1,netdev=o,sriov-pf=f > -device virtio-net-pci,bus=b,addr=0x0.0x0,netdev=n,id=f > > The VFs specify the paired PF with "sriov-pf" property. The PF must be > added after all VFs. It is user's responsibility to ensure that VFs have > function numbers larger than one of the PF, and the function numbers > have a consistent stride. > > Keeping VF instances > -------------------- > > A problem with SR-IOV emulation is that it needs to hotplug the VFs as > the guest requests. Previously, this behavior was implemented by > realizing and unrealizing VFs at runtime. However, this strategy does > not work well for the proposed virtio-net emulation; in this proposal, > device options passed in the command line must be maintained as VFs > are hotplugged, but they are consumed when the machine starts and not > available after that, which makes realizing VFs at runtime impossible. > > As an strategy alternative to runtime realization/unrealization, this > series proposes to reuse the code to power down PCI Express devices. > When a PCI Express device is powered down, it will be hidden from the > guest but will be kept realized. This effectively implements the > behavior we need for the SR-IOV emulation. > > Summary > ------- > > Patch 1 disables ROM BAR, which virtio-net-pci enables by default, for > VFs. > Patch 2 makes zero stride valid for 1 VF configuration. > Patch 3 and 4 adds validations. > Patch 5 adds user-created SR-IOV VF infrastructure. > Patch 6 makes virtio-pci work as SR-IOV PF for user-created VFs. > Patch 7 allows user to create SR-IOV VFs with virtio-net-pci. > > [1] https://patchew.org/QEMU/1689731808-3009-1-git-send-email-yui.washidu@gmail.com/ > [2] https://lore.kernel.org/all/5d46f455-f530-4e5e-9ae7-13a2297d4bc5@daynix.com/ > > Co-developed-by: Yui Washizu <yui.washidu@gmail.com> > Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> > --- > Changes in v7: > - Removed #include <error-report.h>, which is no longer needed. > - Rebased. > - Link to v6: https://lore.kernel.org/r/20240802-sriov-v6-0-0c8ff49c4276@daynix.com > > Changes in v6: > - Added ARI extended capability. > - Rebased. > - Link to v5: https://lore.kernel.org/r/20240715-sriov-v5-0-3f5539093ffc@daynix.com > > Changes in v5: > - Dropped the RFC tag. > - Fixed device unrealization. > - Rebased. > - Link to v4: https://lore.kernel.org/r/20240428-sriov-v4-0-ac8ac6212982@daynix.com > > Changes in v4: > - Added patch "hw/pci: Fix SR-IOV VF number calculation" to fix division > by zero reported by Yui Washizu. > - Rebased. > - Link to v3: https://lore.kernel.org/r/20240305-sriov-v3-0-abdb75770372@daynix.com > > Changes in v3: > - Rebased. > - Link to v2: https://lore.kernel.org/r/20231210-sriov-v2-0-b959e8a6dfaf@daynix.com > > Changes in v2: > - Changed to keep VF instances. > - Link to v1: https://lore.kernel.org/r/20231202-sriov-v1-0-32b3570f7bd6@daynix.com > > --- > Akihiko Odaki (9): > hw/pci: Do not add ROM BAR for SR-IOV VF > hw/pci: Fix SR-IOV VF number calculation > pcie_sriov: Ensure PF and VF are mutually exclusive > pcie_sriov: Check PCI Express for SR-IOV PF > pcie_sriov: Allow user to create SR-IOV device > virtio-pci: Implement SR-IOV PF > virtio-net: Implement SR-IOV VF > docs: Document composable SR-IOV device > pcie_sriov: Make a PCI device with user-created VF ARI-capable > > MAINTAINERS | 1 + > docs/system/index.rst | 1 + > docs/system/sriov.rst | 37 ++++++ > include/hw/pci/pci_device.h | 6 +- > include/hw/pci/pcie_sriov.h | 21 +++ > include/hw/virtio/virtio-pci.h | 1 + > hw/pci/pci.c | 76 +++++++---- > hw/pci/pcie_sriov.c | 295 +++++++++++++++++++++++++++++++++-------- > hw/virtio/virtio-net-pci.c | 1 + > hw/virtio/virtio-pci.c | 24 +++- > 10 files changed, 378 insertions(+), 85 deletions(-) > --- > base-commit: f5cebc77fe020e6ca0c33d8e06cd36edf3ff1d4c > change-id: 20231202-sriov-9402fb262be8 > > Best regards,
On 2024/08/21 19:18, Yui Washizu wrote: > > On 2024/08/13 15:36, Akihiko Odaki wrote: >> Based-on: <20240802-reuse-v11-0-fb83bb8c19fb@daynix.com> >> ("[PATCH for-9.2 v11 00/11] hw/pci: SR-IOV related fixes and >> improvements") > > I couldn't apply this patch series > after applying "[PATCH for-9.2 v11 00/11] hw/pci: SR-IOV related fixes > and improvements". It was a mistake and intended to apply on: "[PATCH for-9.2 v14 00/11] hw/pci: SR-IOV related fixes and improvements" https://patchew.org/QEMU/20240813-reuse-v14-0-4c15bc6ee0e6@daynix.com/ It can be also cleanly applied on the new version I have just sent: "[PATCH for-9.2 v15 00/11] hw/pci: SR-IOV related fixes and improvements" https://patchew.org/QEMU/20240823-reuse-v15-0-eddcb960e289@daynix.com/ Regards, Akihiko Odaki
Based-on: <20240802-reuse-v11-0-fb83bb8c19fb@daynix.com> ("[PATCH for-9.2 v11 00/11] hw/pci: SR-IOV related fixes and improvements") Introduction ------------ This series is based on the RFC series submitted by Yui Washizu[1]. See also [2] for the context. This series enables SR-IOV emulation for virtio-net. It is useful to test SR-IOV support on the guest, or to expose several vDPA devices in a VM. vDPA devices can also provide L2 switching feature for offloading though it is out of scope to allow the guest to configure such a feature. The PF side code resides in virtio-pci. The VF side code resides in the PCI common infrastructure, but it is restricted to work only for virtio-net-pci because of lack of validation. User Interface -------------- A user can configure a SR-IOV capable virtio-net device by adding virtio-net-pci functions to a bus. Below is a command line example: -netdev user,id=n -netdev user,id=o -netdev user,id=p -netdev user,id=q -device pcie-root-port,id=b -device virtio-net-pci,bus=b,addr=0x0.0x3,netdev=q,sriov-pf=f -device virtio-net-pci,bus=b,addr=0x0.0x2,netdev=p,sriov-pf=f -device virtio-net-pci,bus=b,addr=0x0.0x1,netdev=o,sriov-pf=f -device virtio-net-pci,bus=b,addr=0x0.0x0,netdev=n,id=f The VFs specify the paired PF with "sriov-pf" property. The PF must be added after all VFs. It is user's responsibility to ensure that VFs have function numbers larger than one of the PF, and the function numbers have a consistent stride. Keeping VF instances -------------------- A problem with SR-IOV emulation is that it needs to hotplug the VFs as the guest requests. Previously, this behavior was implemented by realizing and unrealizing VFs at runtime. However, this strategy does not work well for the proposed virtio-net emulation; in this proposal, device options passed in the command line must be maintained as VFs are hotplugged, but they are consumed when the machine starts and not available after that, which makes realizing VFs at runtime impossible. As an strategy alternative to runtime realization/unrealization, this series proposes to reuse the code to power down PCI Express devices. When a PCI Express device is powered down, it will be hidden from the guest but will be kept realized. This effectively implements the behavior we need for the SR-IOV emulation. Summary ------- Patch 1 disables ROM BAR, which virtio-net-pci enables by default, for VFs. Patch 2 makes zero stride valid for 1 VF configuration. Patch 3 and 4 adds validations. Patch 5 adds user-created SR-IOV VF infrastructure. Patch 6 makes virtio-pci work as SR-IOV PF for user-created VFs. Patch 7 allows user to create SR-IOV VFs with virtio-net-pci. [1] https://patchew.org/QEMU/1689731808-3009-1-git-send-email-yui.washidu@gmail.com/ [2] https://lore.kernel.org/all/5d46f455-f530-4e5e-9ae7-13a2297d4bc5@daynix.com/ Co-developed-by: Yui Washizu <yui.washidu@gmail.com> Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> --- Changes in v7: - Removed #include <error-report.h>, which is no longer needed. - Rebased. - Link to v6: https://lore.kernel.org/r/20240802-sriov-v6-0-0c8ff49c4276@daynix.com Changes in v6: - Added ARI extended capability. - Rebased. - Link to v5: https://lore.kernel.org/r/20240715-sriov-v5-0-3f5539093ffc@daynix.com Changes in v5: - Dropped the RFC tag. - Fixed device unrealization. - Rebased. - Link to v4: https://lore.kernel.org/r/20240428-sriov-v4-0-ac8ac6212982@daynix.com Changes in v4: - Added patch "hw/pci: Fix SR-IOV VF number calculation" to fix division by zero reported by Yui Washizu. - Rebased. - Link to v3: https://lore.kernel.org/r/20240305-sriov-v3-0-abdb75770372@daynix.com Changes in v3: - Rebased. - Link to v2: https://lore.kernel.org/r/20231210-sriov-v2-0-b959e8a6dfaf@daynix.com Changes in v2: - Changed to keep VF instances. - Link to v1: https://lore.kernel.org/r/20231202-sriov-v1-0-32b3570f7bd6@daynix.com --- Akihiko Odaki (9): hw/pci: Do not add ROM BAR for SR-IOV VF hw/pci: Fix SR-IOV VF number calculation pcie_sriov: Ensure PF and VF are mutually exclusive pcie_sriov: Check PCI Express for SR-IOV PF pcie_sriov: Allow user to create SR-IOV device virtio-pci: Implement SR-IOV PF virtio-net: Implement SR-IOV VF docs: Document composable SR-IOV device pcie_sriov: Make a PCI device with user-created VF ARI-capable MAINTAINERS | 1 + docs/system/index.rst | 1 + docs/system/sriov.rst | 37 ++++++ include/hw/pci/pci_device.h | 6 +- include/hw/pci/pcie_sriov.h | 21 +++ include/hw/virtio/virtio-pci.h | 1 + hw/pci/pci.c | 76 +++++++---- hw/pci/pcie_sriov.c | 295 +++++++++++++++++++++++++++++++++-------- hw/virtio/virtio-net-pci.c | 1 + hw/virtio/virtio-pci.c | 24 +++- 10 files changed, 378 insertions(+), 85 deletions(-) --- base-commit: f5cebc77fe020e6ca0c33d8e06cd36edf3ff1d4c change-id: 20231202-sriov-9402fb262be8 Best regards,